Ippolitov-Ivanov studied under Nikolay Rimsky-Korsakov (Rimsky-Korsakov, Nikolay) at the St. Petersburg Conservatory and in 1882 became conductor of the symphony orchestra and director of the music school in Tiflis (now Tbilisi), Georgia. He taught at the Moscow Conservatory from 1893 to 1906, served as its director from 1906 to 1922, and was conductor of the Mamontova Opera from 1899 to 1906. In 1924–25 he reorganized the Georgian State Conservatory, formerly the Tbilisi School. After 1925 he was conductor at the Bolshoi Theatre.

Ippolitov-Ivanov's 11 years in the Caucasus gave him a lifelong interest in Georgian folk music that inspired several of his orchestral compositions: Caucasian Sketches (1895) suite, Armenian Rhapsody (1909), and the symphonic poem after a poem by Mikhail Lermontov (Lermontov, Mikhail), Mtsyri (1922; “The Novice”). These works were seldom performed after the mid-20th century; likewise, his seven operas did not remain in the repertory.