A self-taught musician from a musical family, Montgomery dropped out of school and left home at age 11. He traveled in small towns, lumber camps, and cities in Louisiana, Arkansas, and Mississippi, playing in taverns and barrelhouses, usually as a soloist and blues singer, and also with jazz and dance bands or with blues guitarists. In 1928 he moved to Chicago, where he played at house-rent parties (parties thrown to raise rent money) and made his first recordings, including “Vicksburg Blues,” his own version of “The Forty-Fours,” in 1930. The next year he relocated to Jackson, Miss., where he formed a band, the Southland Troubadors (also called the Collegiate Ramblers), that played in ballrooms throughout the South. On one day in 1936 he recorded 18 solos and vocals, including the original versions of his standards “Shreveport Farewell” and “The First Time I Met the Blues.”

After resettling in Chicago in 1942, Montgomery's graceful, New Orleans-style swing and uncommonly wide repertoire—encompassing blues, boogie-woogie, ragtime, popular songs, and jazz standards—made him a popular pianist in traditional jazz groups. In 1948 he played in Kid Ory (Ory, Kid)'s band at Carnegie Hall. He accompanied classic blues singer Edith Wilson and modern Chicago blues artists such as Buddy Guy and Otis Rush on records, but he appeared most often as a solo performer or leader of his own groups. He also became a popular attraction on international tours and at jazz and blues festivals.