The eldest surviving son of John Hervey, 1st earl of Bristol, (1665–1751), he was educated at Westminster School and the University of Cambridge. He then made the customary “grand tour” of Europe and, at Hannover, Ger., became a firm friend of Frederick Louis (later prince of Wales), the eldest son of King George II. His wife, Mary (Molly) Lepell, whom he married in 1720, was a close friend of the then Prince and Princess of Wales (later George II and Queen Caroline, for whom he served as vice chamberlain of the royal household, 1730–40). Hervey was Whig member of Parliament for Bury St. Edmunds, Suffolk (1725–33) and was then created Baron Hervey of Ickworth. He was disappointed in public life, for, though he was made lord privy seal in 1740, he fell with Sir Robert Walpole's government in 1742.

Hervey's renown, however, lay in the world of fashion and the court rather than in politics. He suffered from a feeble constitution, though his health improved after a visit to Italy, when he formed a close friendship with Stephen Fox (later Lord Ilchester). This friendship evoked scandalous allusions from Alexander Pope in his “Epistle to Dr. Arbuthnot,” a satirical poem in which Hervey figures as “Sporus.” His own outspoken Memoirs (published posthumously, 1848; edited by R. Sedgwick, 3 vol., 1931; rev. 1963) were, he said, written for those who prefer to see the great “dressing and undressing rather than when they are playing their part on the public stage.”