Jackson was brought up in a strict religious atmosphere. Her father's family included several entertainers, but she was forced to confine her own musical activities to singing in the church choir and listening—surreptitiously—to recordings of Bessie Smith (Smith, Bessie) and Ida Cox as well as of Enrico Caruso (Caruso, Enrico). When she was 16 she went to Chicago and joined the Greater Salem Baptist Church choir, where her remarkable contralto voice soon led to her selection as a soloist.

Jackson first came to wide public attention in the 1930s, when she participated in a cross-country gospel tour singing such songs as “He's Got the Whole World in His Hands” and “I Can Put My Trust in Jesus.” In 1934 her first recording, “God Gonna Separate the Wheat from the Tares,” was a success, leading to a series of other recordings. Jackson's first great hit (eight of her records were to sell more than a million copies each) was “Move on Up a Little Higher,” which appeared in 1945. All the songs with which she was identified—including “I Believe,” “Just over the Hill,” “When I Wake Up in Glory,” and “Just a Little While to Stay Here”—were gospel songs, with texts drawn from biblical themes and strongly influenced by the harmonies, rhythms, and emotional force of blues. Jackson refused to sing any but religious songs or indeed to sing at all in surroundings that she considered inappropriate. But she sang on the radio and on television and, starting in 1950, performed to overflow audiences in annual concerts at Carnegie Hall in New York City.

Jackson was enormously popular abroad; her version of “Silent Night,” for example, was one of the all-time best-selling records in Denmark. She made a notable appearance at the Newport (Rhode Island) Jazz Festival in 1957—in a program devoted entirely, at her request, to gospel songs—and she sang at the inauguration of President John F. Kennedy in January 1961. From 1955 she was active in the Civil Rights Movement.