Khan's father, Ustad Fateh Ali Khan, and two of his uncles were famous practitioners (qawwals) who sang in the classical form. Nusrat received music lessons from his father but did not devote himself to the qawwali tradition until he sang at his father's funeral in 1964. Two years later he gave his first public performance, singing with his uncles. By the early 1970s he had established himself throughout Pakistan as the outstanding qawwal of his generation. In 1985 he sang at a world music concert in the United Kingdom, and word of his talent began to spread. He was soon performing regularly throughout Europe. He first toured the United States in 1989 and in the 1990s contributed to the soundtracks for a number of popular films.

Qawwali music originated in 12th-century Persia. The lyrics are based on medieval Ṣūfī poems that often use images of romantic love to express deep religious faith. The traditionally male qawwal, who knows these poems by heart, unites phrases and passages from different poems to create a new expression. Qawwali performances are traditionally held in shrines and are marked by passionate shouting and dancing. It is similar in spirit to American gospel music.

Usually accompanied by tabla (small hand drums), harmoniums, and backing vocals, Nusrat Fateh Ali Khan sang in a very high register (a family trademark) and had a powerful and highly expressive voice. His melodic creativity and stamina were legendary.