The son of a justice of the Connecticut Supreme Court, Waite practiced law in Toledo, Ohio. In 1871–72 he became nationally prominent as one of the U.S. counsels to the Alabama arbitration commission at Geneva, dealing with Great Britain's liability to the United States for permitting Confederate warships to be built and serviced in British ports. The favourable impression he made on President Ulysses S. Grant at that time led to his appointment as chief justice by Grant on Jan. 19, 1874.

In several cases concerning the recently freed and supposedly enfranchised blacks, Waite held that the privileges and immunities of U.S. citizens had not been increased by the Fourteenth Amendment and that neither it nor the Fifteenth Amendment (1870) had given Congress extensive power to safeguard civil rights. In United States v. Cruikshank, 92 U.S. 542 (1876), he stated that, despite its apparently plain language, the Fifteenth Amendment had not conferred a federal right of suffrage on blacks, because “the right to vote comes from the states.” In Hall v. De Cuir, 95 U.S. 485 (1878), he struck down, as a “direct burden” on interstate commerce, a Louisiana Reconstruction statute requiring full racial integration of passengers by common carriers. In Reynolds v. United States, 98 U.S. 145 (1878), in upholding the application of antipolygamy laws to Mormons, Waite distinguished between the freedom to hold a religious belief and the freedom to engage in religious practices (polygamy) that had been outlawed by legislative act.

Waite tried to establish a nonpolitical conception of the chief justiceship. In 1876 he might have had the Republican Party's nomination for president, but he rejected it because, in his view, his candidacy would detract from the court's prestige.