Reared in great poverty, Eckermann served in the German war of liberation against Napoleon and became a clerk in the war department at Hanover, later studying for a year at Göttingen, from 1821 to 1822. At an early age Goethe became his idol. Eckermann published a book of poems in 1821 and in 1823 attracted Goethe's attention by sending him his Beiträge zur Poesie mit besonderer Hinweisung auf Goethe (“Helps Toward Understanding Poetry with Special Instructions on Goethe”), which contained sensitive appreciations of Goethe's work. Goethe invited Eckermann to Weimar, where he became Goethe's unpaid literary assistant. Eckermann also acted as tutor to the son of the grand duke of Saxe-Weimar-Eisenach and, in 1836, was an appointed ducal librarian at the Weimar court.

Eckermann's Gespräche has been translated into every major European language. The first English translation, Conversations with Goethe (1839), was made by the American critic Margaret Fuller. Based on notes taken with Goethe's permission, Eckermann's Conversations are not mere records of interviews but an artistically selective arrangement of information on Goethe's life and thought. Eckermann also acted as Goethe's literary executor and published his posthumous works (1832–33) and, with F.W. Riemer, prepared the first complete edition of Goethe's works.