Grove began his career as a civil engineer and became secretary to the Society of Arts in 1850 and to the Crystal Palace in 1852. He collaborated with William Smith in his Dictionary of the Bible and was largely responsible for organizing the Palestine Exploration Fund in 1865. From 1856 to 1896 he wrote analytical notes for the Crystal Palace concerts; marked by enthusiasm, insight, and thoroughness, these established a standard in program commentary. In 1867 he visited Vienna with the composer Sir Arthur Sullivan and discovered the manuscripts for Schubert's Rosamunde. He was editor of Macmillan's Magazine from 1868 to 1883. During the years 1879–89 his Dictionary was published; in addition to supervising its contents, he contributed several articles to it.

In 1882 Grove became first director of the Royal College of Music and was knighted. His book Beethoven and His Nine Symphonies was published in 1896.