Rabi's parents settled in New York City in 1899. After earning a bachelor's degree in chemistry at Cornell University in 1919, Rabi switched to physics and received his Ph.D. from Columbia University in 1927. He did postgraduate work in Europe and then joined the faculty of Columbia University in 1929, becoming professor of physics in 1937. From 1940 to 1945 Rabi was a leader of the group of scientists at the Massachusetts Institute of Technology, Cambridge, who helped in the development of radar. He was a member of the General Advisory Committee of the Atomic Energy Commission from 1946 to 1956 and succeeded J. Robert Oppenheimer as its chairman from 1952 to 1956. He originated the concept of the CERN international laboratory for high-energy physics in Geneva, Switz., and he was one of the founders of the Brookhaven National Laboratory, Upton, N.Y. He also built up one of the world's finest physics departments at Columbia University, one which was to produce several Nobel Prize–winning physicists.

Rabi's most important scientific work was his development (in the 1930s) of a method for measuring the magnetic properties of atoms, atomic nuclei, and molecules. The method is based on measuring the spin of the protons in the atom's core, a phenomenon known as nuclear magnetic moments. With the application of his magnetic resonance method, several mechanical and magnetic properties, as well as the shape, of an atomic nucleus can be deduced. The precise measurements yielded by this method made possible such subsequent applications as the atomic clock, the maser, and the laser, as well as the nuclear magnetic resonance imaging used in diagnostic medicine. Rabi's method provided the central technique for virtually all molecular and atomic beam experimentation.