Fiersohn grew up on the Lower East Side of New York City and then worked as a stenographer until her marriage in 1902 to Bernard Glick. After having studied voice for just three years, in 1909 she made her debut at the Metropolitan Opera in New York City. There, under the name Alma Gluck, she performed the role of Sophie in Jules Massenet (Massenet, Jules)'s Werther. Over the next three years she sang a wide variety of lyric soprano roles, but opera apparently proved less interesting to her than the recital stage. In 1912 she left the opera stage, divorced her husband, and went to Europe to study. In London in 1914 she married the violinist Efrem Zimbalist.

Until 1920 she toured the United States regularly, performing as a soloist or with her husband in joint recitals. Her popularity was matched by few other singers, and her voice, grace, and attractiveness gave her a commanding stage presence. Gluck also became popular as a recording artist, and her recording of "Carry Me Back to Old Virginny" sold nearly two million copies.