Murphy studied at the University of Michigan (LL.B., 1914) and, after serving in the war, held several elective posts in the 1920s. As mayor of Detroit (1930–33), he gained national prominence for his efforts to aid the unemployed. Appointed by President Franklin D. Roosevelt, he served as governor-general (1933–35) and U.S. high commissioner (1935–36) in the Philippines, where he supported the independence movement. As governor of Michigan (1937–38), he earned the admiration of organized labour and the hatred of some industrialists (who brought about his defeat for reelection) by refusing to employ troops to break sit-down strikes by automobile workers. While serving as U.S. attorney general (1939–40), he established the Civil Rights Unit (now Division) of the Department of Justice.

Perhaps Murphy's most notable judicial opinion was his dissent in Korematsu v. United States, 323 U.S. 214 (1944), in which he denounced as “legalization of racism” the government's wartime internment of Japanese-American residents of the West Coast. His dissent in Wolf v. Colorado, 338 U.S. 25 (1949), in which the court held that illegally seized criminal evidence was admissible in state (though not in federal) courts, was vindicated when a later court overruled the Wolf decision (Mapp v. Ohio, 1961).