Escape to the Bayberry Inn Bed and Breakfast, enjoy Cape May's ambiance and the serenity of a by-gone era. Unwind at a relaxing seaside bed and breakfast inn, warmed with hospitality and Victorian charm..Only two blocks from the beach and one block from shopping, dining and theater. The Bayberry Inn, formerly the Joseph Q. Williams House, is one of the oldest homes in Cape May! The original house, which is the back portion and includes the dining room, was built circa 1800. It’s believed to have been a whaling captain’s home. The front of the house, a carpenter gothic, circa 1856, was constructed by Joseph Q. Williams, ...