The son of a Virginia planter and a slave mother, Langston was emancipated at the age of five, attended school in Ohio, and graduated from Oberlin College in 1849. He quickly became a leader among free blacks and was elected to local offices in Brownhelm Township, Ohio (1855), and Oberlin (1865–67). In 1864 he helped organize the National Equal Rights League, of which he was the first president.

After the American Civil War Langston moved to Washington, D.C., practiced law, and was professor of law and dean of the law department (1869–77) and vice president (1872–76) of Howard University. He was U.S. minister to Haiti and chargé d'affaires to Santo Domingo (1877–85) and was elected president of the Virginia Normal and Collegiate Institute (1885). In 1888 he was a Republican candidate from Virginia for the U.S. House of Representatives, and, after a challenge of the election returns that took almost two years, he succeeded in unseating his Democratic opponent and served in Congress from Sept. 23, 1890, to March 3, 1891.