Pupin's family was of Serbian origin, and his parents, who were illiterate, encouraged his education. He immigrated to the United States in 1874 and held a series of odd jobs before attending Columbia College (now Columbia University), where he earned a B.A. in 1883; that year he also became a U.S. citizen. Pupin subsequently studied at Cambridge University and the University of Berlin (Ph.D., 1889). In 1889 he began teaching mathematical physics at Columbia, becoming professor emeritus in 1931.

In 1896 Pupin discovered that atoms struck by X rays (X-ray) emit secondary X-ray radiation. He also invented a means for taking short-exposure X-ray photographs. In 1901 the Bell Telephone Company and some German telephone interests acquired the patent for his invention of long-distance telephony. Pupin received the 1924 Pulitzer Prize in biography for his autobiographical work From Immigrant to Inventor (1923).