Scherchen was musically self-taught. Early in his career he played the viola, and for a time he toured with the Austrian composer Arnold Schoenberg. Interned in Russia during World War I, he returned to Berlin after the war and there in 1918 founded the Neue Musikgesellschaft (“Society for New Music”). He edited the music journal Melos in 1920–21. In 1933 he fled Germany to Brussels, where he edited the journal Musica viva (1933–36). He conducted also in Spain, France, and elsewhere in Europe during and after World War II; he made his American debut with the Philadelphia Orchestra in 1964.

Scherchen collaborated with avant-garde composers in introducing their works in recordings and live performances. He recorded with the orchestras of Vienna, London, and Paris and devoted particular attention to the works of Baroque masters and of Ludwig van Beethoven. Serving as an important link between Schoenberg and his school and the younger composers, Scherchen influenced the careers of Luigi Nono and other leading mid-century composers, especially in Italy.

Scherchen wrote several books, including the Lehrbuch des Dirigierens (1929; Handbook of Conducting), Vom Wesen der Musik (1946; The Nature of Music), and Musik für Jedermann (1950; “Music for Everyman”).