After studying in England, Rutledge returned to Charleston to practice law. Reflecting views acceptable to both planters and merchants in his area, he was chosen as a delegate to the Stamp Act Congress (1765) and to the Continental Congress (1774–77, 1782–83). After chairing the committee that framed the South Carolina constitution (1776), he was elected president of the state's General Assembly, but he resigned in 1778 when the constitution was amended to include provisions he considered too democratic. In 1779 he was elected South Carolina's governor, and, after the state was invaded by the British in that year, he held the skeleton colonial government together until the end of the war.

At the Constitutional Convention in 1787, Rutledge spoke for Southern planters by supporting slavery. He argued in favour of dividing society into classes as a basis for representation and also postulated high property qualifications for holding office. As chairman of the Committee on Detail, he recommended the granting of indefinite powers of legislation to the national government for the purpose of promoting the general welfare.

From 1789 to 1791 he served as an associate justice of the U.S. Supreme Court and for the next four years as chief justice of the South Carolina Supreme Court. Nominated Chief Justice of the United States in 1795, he failed to win Senate confirmation because of his outspoken opposition to the Jay Treaty of the previous year.

His brother Edward Rutledge was a signer of the Declaration of Independence (1776), fought against the British in South Carolina during the American Revolution, and served in the South Carolina legislature (1782–98) and as governor (1798–1800) of the state.