Eliot graduated from Harvard in 1853 and was appointed assistant professor of mathematics and chemistry there in 1858. In 1867, during his second trip to Europe, he made a study of European educational systems. His published observations (in The Atlantic Monthly, 1869) brought his name to the attention of the directors of Harvard, who were looking for a new president. Eliot was inaugurated in October 1869. By the time he retired in 1909 he had elevated Harvard into an institution of world renown.

Contending that higher learning in the United States needed to be “broadened, deepened, and invigorated,” Eliot demanded a place for the sciences as well as the humanities in any sound program of liberal education. To counter the rigidity of the Harvard curriculum—which, following what was then general practice, was then almost totally prescribed—Eliot eliminated required courses. Under his successor, A. Lawrence Lowell, a balance was struck between required and elective courses.

Eliot's influence reached into secondary education. During his presidency Harvard raised its entrance requirements, and other major colleges did likewise. This, in turn, effected a corresponding rise in secondary-school standards. In the report of the Committee of Ten, a national commission on secondary education (1893), he urged the introduction of foreign languages and mathematics during the student's seventh school year. The idea was embodied later (1910) by the introduction of junior high schools in the United States. Eliot served as president of the National Education Association (1903) and was the first honorary president of the Progressive Education Association (1919).

Eliot's writings include Educational Reform: Essays and Addresses 1869–1897 (1898) and University Administration (1908).