Of the three buildings which comprise The Mainstay Inn, the most interesting history belongs to the Inn. Designed by famous Philadelphia architect, Stephen Decatur Button, and built in 1872 as a private gambling club, the building was recognized as important by a writer for the Cape May Ocean Wave as early as June, 1872. "The New Club House - In the design on this model building which now adds to the attractiveness of Columbia Avenue, Mr. S.D. Button, the architect, has won himself additional honors...It is symmetrical in its proportions, airy and cheerful in its appointments, and finished in that unpretentious elegance so foreign to mansions of the shoddy order."

The Clubhouse operated until the late 1890's when gambling was made illegal and Cape May was losing ground to Atlantic City, Asbury Park and Cony Island. It was then sold to the Scotts who used it as a summer home for many years and again to the O'Briens who had wonderful parties that Cape May's old timers still remember. In 1949, Rev. and Mrs. John Pemberton bought the building and converted it to a guest house, called The Victorian Mansion, which they operated summer only until 1976.

In 1972 when the entire City of Cape May was placed on the National Register of Historic Sites, The Mainstay was one of sixteen buildings in the City selected to be recorded by The Historic American Buildings Survey. Architectural plans of the building and important historic details where carefully prepared and are now stored in the Library of Congress. In 1977, the building and began an extensive restoration project and opened a bed-and-breakfast, which they called The Mainstay Inn. In 1980, a neighboring property, the Cottage, which was also designed by Stephen D. Button.

It had always been a private summer home, but after months of labor, it was converted to a six room B&B, operated in conjunction with The Mainstay Inn. Tom and Sue continued to own and operate The Mainstay Inn until April of 2004, when all three properties where purchased by David and Susan Macrae. Today, The Mainstay Inn is run by long time Mainstay assistant innkeeper, Diane Clark with the same level of commitment to hospitality and historic preservation as before.