Primarily self-educated, Black served his legal apprenticeship in the offices of a prominent attorney, then in 1830 was himself admitted to the bar. Although active in the ranks of the Democratic Party, Black devoted most of the next two decades to developing a lucrative law practice in Somerset, Pa. In 1842 the governor made him a district judge, and Black served until elected to the Pennsylvania Supreme Court in 1851. He was reelected in 1854 but left three years later, when his old friend James Buchanan appointed him attorney general of the United States.

As attorney general, Black launched a major investigation of land titles in California that eventually revealed systematic fraud. He was the leader of the administration's attacks on Senator Stephen Douglas and Douglas' notion of popular sovereignty as a solution to the slavery-expansion controversy. But Black distinguished himself most for his advocacy of sending troops, if necessary, to protect federal property in seceded states.

Black served as secretary of state in the last months of Buchanan's presidency. The president nominated him for a seat on the Supreme Court, but a coalition of Republicans, Douglas Democrats, and Southerners blocked confirmation. Following Abraham Lincoln's inauguration, Black resumed private law practice in York, Pa. He criticized the federal government's abrogation of constitutional rights during the American Civil War, and he played a prominent role in the Ex parte Milligan and Ex parte McCardle cases at the conclusion of the war. He advised President Andrew Johnson on the veto of the first Reconstruction Act in 1867 and—but for a quarrel with Johnson's other lawyers—would have assisted in the president's defense during the impeachment trial. Black lost the use of his right arm in an accident in 1869, but he retained his acute mental faculties to the end of his life.