Gerry was the son of Thomas Gerry, a merchant, and Elizabeth Greenleaf. He graduated from Harvard in 1762 and entered his father's business. He was a member of the Massachusetts legislature and General Court (1772–73), served on a Committee of Correspondence (Correspondence, Committees of), was a member of the Massachusetts Provincial Congress (1774–75), and was a delegate to the Continental Congress in Philadelphia (1776–81), where he was an early advocate of independence. He was also a member of Congress (1783–85) under the Articles of Confederation (Confederation, Articles of) and a delegate to the Constitutional Convention in Philadelphia (1787). He was an outspoken opponent of ratification of the United States Constitution (Constitution of the United States of America), fearing that it might give way to aristocratic or monarchical rule. However, he gave it his full support after its ratification, helping to draft the Bill of Rights (Rights, Bill of) and serving as a representative in Congress for two terms (1789–93).

After four attempts to win election as governor of Massachusetts, Gerry succeeded in 1810 and was reelected in 1811. His administration was notable for its use of what became known as gerrymandering, the division of electoral districts for partisan political advantage.

In 1812 Gerry, an ardent supporter of war with Great Britain in the War of 1812 (1812, War of), was elected vice president of the United States on the Jeffersonian Republican ticket with Madison. In 1813, while presiding over the Senate, Gerry, who along with Madison was in ill health, refused to yield his chair at the close of the legislative session, thus preventing William Giles, a senator from Virginia and an advocate of peace with Britain, from becoming president pro tempore of the Senate and thereby second in line (after the vice president) to succeed the president under the Presidential Succession Act of 1792. Gerry suffered a hemorrhage of the lungs on his way to the Senate and died in 1814.