After obtaining a bachelor's degree in engineering from the University of Dacca in 1950, Khan worked as assistant engineer for the India Highway Department and taught at the University of Dacca. Qualifying for a scholarship in 1952, he enrolled at the University of Illinois in Champaign-Urbana, where he received master's degrees in both applied mechanics and structural engineering and a Ph.D. in structural engineering. He returned briefly to Pakistan and won an important position as executive engineer of the Karachi Development Authority. Frustrated by administrative demands that kept him from design work, however, he returned to the United States and joined the prestigious architectural firm of Skidmore, Owings & Merrill in Chicago in 1955, eventually becoming a partner (1966). In 1967 he became a naturalized U.S. citizen.

Khan's many skyscraper projects include Chicago's John Hancock Center (1970) and Sears Tower (1973), which are among the world's tallest buildings. The Sears Tower was his first skyscraper to employ the “bundled tube” structural system, which consists of a group of narrow steel cylinders that are clustered together to form a thicker column. This innovative system minimized the amount of steel needed for high towers, eliminated internal wind braces (since the perimeter columns bear the weight of the wind force), and permitted freer organization of the interior space.

Khan's later projects include the strikingly original Haj Terminal of the King Abdul Aziz International Airport (1976–81) in Jiddah, Saudi Arabia, and King Abdul Aziz University (1977–78), also in Jiddah.