Harrison studied at the École des Beaux-Arts, Paris, and in 1921 won a traveling fellowship to Europe and the Middle East. He was one of the architects responsible for Rockefeller Center, New York City (1929–40). The partnership he formed with J. André Fouilhoux in 1935 became Harrison, Fouilhoux and Abramovitz in 1941. Harrison designed the Trylon and Perisphere theme centre at the New York World's Fair (1939).

Harrison's partnership with Max Abramovitz, formed in 1945, became one of the largest architectural firms in the United States specializing in office buildings. Among his office buildings are the Alcoa Building, Pittsburgh, Pa. (1953), notable for its large aluminum panels cut by relatively small panels, and the Socony Mobil Building, New York City (1956). His First Presbyterian Church, Stamford, Conn., is considered an outstanding example of modern church design. Shaped like a fish, the interior is flooded with coloured light from large expanses of stained glass.

Harrison's organizational skills were well utilized in his major projects, such as the United Nations complex and the Lincoln Center for the Performing Arts, New York City (1962), for which he served as overall design coordinator. He also designed the new Metropolitan Opera House (1965) and its office alterations (1978).