In his youth Johnson studied classical and ragtime piano techniques, and by his late teens he was performing in saloons, in dance halls, and at parties in a black community on Manhattan's West Side, near Hell's Kitchen. While playing for dancers before 1920 he became noted for his rare ability to create embellishments, variations, and improvisations on popular songs, including the blues, relatively new at the time. He made piano rolls followed by recordings of his own songs. He also composed and orchestrated music for stage revues, including Keep Shufflin', a 1928 collaboration with his leading student, Fats Waller (Waller, Fats).

Johnson's symphonic works, according to composer Gunther Schuller, use “basic Negro musical traditions that emulated roughly Liszt's approach in his Hungarian rhapsodies.” However, these works, which include Yamecraw (1928), Harlem Symphony (1932), and the one-act opera De Organizer (c. 1940), with a libretto by Langston Hughes (Hughes, Langston), have seldom been performed.

As played by Johnson, stride piano, a development of ragtime, used two-beat left-hand rhythms to accompany right-hand melodies that featured uncommon interpretative variety. Representative pieces range from the heartily swinging, up-tempo “Carolina Shout” and “Carolina Balmoral” to the delicate and reflective, slower-paced “Blueberry Rhyme” and “Snowy Morning Blues.” Grace and elegance of musical line characterize his solos, and among his accompaniments, his work in singer Bessie Smith (Smith, Bessie)'s “Backwater Blues” is especially notable. The most popular songs that he wrote include “The Charleston,” “Old Fashioned Love,” and “If I Could Be with You One Hour Tonight.”