Trained as a pianist at the Cologne Conservatory, he became a conductor at Luzern, Switz., in 1891. He became renowned as an interpreter of Beethoven, Mahler, and Richard Strauss, who dedicated his tone poem Ein Heldenleben (“A Hero's Life”) to Mengelberg. He toured widely with the Concertgebouw, was regular conductor (1911–14) of the London Symphony and the Royal Philharmonic Society, and was guest conductor (1921–30) of the New York Philharmonic (and of its predecessor, the National Symphony, before the merger of the two), serving for several years as coconductor with Arturo Toscanini.

Near the end of World War II, Mengelberg fled The Netherlands for Switzerland. In 1945 a Dutch council convicted him, in absentia, of having been a Nazi collaborator and sentenced him to exile for life (later reduced to exile for six years).