After graduating from the University of North Carolina at Chapel Hill in 1928, Webb became a marine pilot. He began his government career in 1932 as a congressional aide in Washington, D.C., and from 1934 to 1936 he studied law at George Washington University. He worked for Sperry Gyroscope from 1936 to 1944, when he reentered the Marine Corps for the remainder of World War II. During the administration of Pres. Harry Truman (1945–53), Webb was director of the Bureau of the Budget and undersecretary of state. When Truman left office, he went to work for the Kerr-McGee Oil Company in Oklahoma.

Webb became the administrator of NASA in 1961, just months before Pres. John F. Kennedy announced the U.S. commitment to sending a man to the Moon by 1970. Webb gave overriding priority to the success of Apollo and used his considerable political skills to rally and maintain support for the program, even after three astronauts died in an accident in 1967. Webb was also a keen student of public administration and used NASA as a laboratory for his ideas about how to organize large-scale public undertakings in ways that both increased the chances of a program's success and provided maximum benefits to the country.

Webb retired from NASA in 1968. He remained in Washington, serving on various advisory boards and as a regent of the Smithsonian Institution.