The son of a wealthy Spanish coffee plantation owner and a French mother, Heredia was educated at Senlis, near Paris. He claimed France as “the country of my mind and heart”; and, although he went home after finishing his schooling, he quickly returned to Paris and studied at the School of Paleography. He became the close friend of the poet Charles-Marie-René Leconte de Lisle and, like him, was a leading figure in the Parnassians.

Heredia's 118 sonnets and some longer pieces were published as Les Trophées (1893). These poems capture in verse a fugitive moment of history (usually classical or Renaissance) or else some objet d'art (a vase, a coin, an ornate book binding), usually in one startling image. A selection of his poems in English translation was published in The Flute, with Other Translations and Poems (1977). He uses a wide range of stylistic effects (double rhymes, onomatopoeia, exotic-sounding place-names) and caps the beauty of each poem with a final couplet or line of especially haunting effect.

In 1894 Heredia was elected to the French Academy. In 1901 he became librarian of the Bibliothèque de l'Arsenal, Paris. Just before his death he completed an edition of the Bucoliques by the 18th-century poet André de Chénier.