After attending Massachusetts Institute of Technology for one year, Gilbert worked briefly as a draftsman for the noted New York City architectural firm of McKim, Mead, and White. In 1882 he entered a partnership in St. Paul, Minn. Having attracted national attention by his design for the Minnesota state capitol, St. Paul (built 1896–1903), he moved to New York City. In addition to the Woolworth Building, his major works in New York included the U.S. customhouse (1899–1905; in a Renaissance style with Germanic detail) and the federal courthouse (completed 1936). For some years the 60-story (792-foot) Woolworth skyscraper, with its lacy Gothic detail in terra-cotta over a steel frame, was regarded as a model of tall commercial building design.

In Washington, D.C., Gilbert built the U.S. Treasury Annex (1918–19), as well as the Supreme Court Building, the monumentality of which is sometimes felt to be oppressive. He also planned the campuses of the universities of Minnesota (Minneapolis) and Texas (Austin).