Today, The Morgan Library & Museum is a complex of buildings of differing styles and periods covering half a city block. It began as an intimate palazzo-like structure designed by Charles Follen McKim to serve as the private library of financier Pierpont Morgan. "Mr. Morgan's library", as it became known, was built between 1902 and 1906 to the east of his New York residence at Madison Avenue and 36th Street.
In the years since the Morgan's incorporation as a public institution in 1924, there have been several additions to the original library building. As the collections grew, the Annex was added in 1928, on the site of Morgan's home. In 1988, the mid-nineteenth-century brownstone on Madison Avenue and 37th Street, where J. P. Morgan, Jr., lived was also added to the complex. A garden court was built in 1991 to unite all three buildings in the complex. A century after the completion of the McKim building, The Morgan Library & Museum unveiled the largest expansion and renovation in its history. The Renzo Piano design integrates the three landmark buildings with three intimately scaled new pavilions constructed of steel-and-glass panels to create an accessible, inviting setting.