In 1861 Governor Hamilton Fish bought a large plot of land in Garrison New York and constructed a large imposing estate at a cost of $30,000 for him and his family. The home, which he named "Glenclyffe", was considerably large for $30,000 and had 5 bedrooms, 6 bathrooms, 3 servant's rooms, a library, drawing room, dining room, billiard room, den, morning room and a boudoir. The property had a large green house, stables, expansive gardens, gate house and spectacular views of the Hudson.
After leaving the Grant Cabinet, Fish settled down at "Glenclyffe" until his death in 1893, afterward it passed to his son Stuyvesant Fish and his wife Marion "Mamie". Mamie felt that she could not live in such a small home (almost 16,000 square ft was what she considered small), so Mamie had a large addition built to the left of the house that doubled the size of the original house making it a total of 32,000 square ft. The new addition included a new kitchen, service pantry, servant's rooms, expanded dining room, sun room and morning room. Mamie also had the back of the house expanded so she could expand the drawing room and stair hall. Where the old kitchen and servants rooms were Mamie had constructed a large ballroom with decorative oak paneling. Every single room except the library was redone and the den was moved upstairs. The new home now had 15 bedrooms, 13 bathrooms, 15 servant's rooms and stables able to accommodate 25 carriages.
The Fish stayed at "Glenclyffe" during the autumn and spring seasons when society would be either in Europe or in the country. It was then when Mamie would give her weekend house parties, normally inviting 20 guests to stay at "Glenclyffe" for the weekend. During the autumn she would give halloween parties in the house where her own guests and neighbors danced in the ballroom and the servants and their guests danced in the stables. Guests marveled at the elegant but very comfortable interior of "Glenclyffe", perhaps the most luxurious room in the house was Mrs. Fish's bedroom.
It was in this room that Mrs. Fish died in 1915. Mr. Fish continued to occupy "Glenclyffe" along with the couples two other homes until his death in 1923. "Glenclyffe" was later bought by the Garrison Institute which altered it greatly. When one looks at the home as it is now it is almost completely unrecognizable except for the small entrance and window above that was on the original property.