Eiermann studied at Berlin Technical University under Hans Poelzig, later working in the building department of the Karstadt department store company. Beginning in 1930 he practiced architecture in Berlin and, from 1947, in Karlsruhe, where he also served on the faculty of the university. Adhering to an aesthetic of making order visible, Eiermann created a number of major achievements in functional design, including the textile mill at Blumberg (1951), the West German pavilion at the Brussels World Exhibition (with Sep Ruf, 1958), the West German embassy in Washington, D.C. (1958–64), and the IBM-Germany Headquarters in Stuttgart (1967).

Perhaps his most popular work is the Kaiser Wilhelm Memorial Church (1956–63), a symbol of postwar Berlin. Originally, a medieval revival building constructed in the late 19th century stood on the site, but after a World War II bombing raid only the bell tower remained. This ruin was incorporated by Eiermann into his strikingly simple design for a modern church.