Like many notable jazz pianists, Mehldau was originally classically trained. He began studying piano at age six, and he became interested in jazz in his early teens. He played in the noted Hall High School jazz band in Hartford, Conn., and then studied jazz in New York City at the New School, graduating in 1993; there his teachers included jazz pianists Junior Mance, Fred Hersch, and Kenny Werner. After taking high-profile sideman jobs in New York, Mehldau played (1994–95) with the quartet led by tenor saxophone player Joshua Redman. Mehldau also accompanied top musicians in jazz, country, and rock music. He played on four film sound tracks before composing the score for another film, Ma femme est un actrice (2001).

Meanwhile, Mehldau devoted much of his creative energy to working with his trio, including bassist Larry Grenadier and drummer Jorge Rossy; drummer Jeff Ballard replaced Rossy in 2005. The trio was noted for its sensitive interplay and for Mehldau's arrangements, often in uncommon metres (e.g., five or seven beats to the measure); their reputation grew, especially with a series of annual CDs titled The Art of the Trio (1996–2000). Mehldau also excelled as an unaccompanied pianist—like one of his influences, Keith Jarrett (Jarrett, Keith)—in improvisations that ranged from delicately played, brooding ballads to grand, harmonically rich rhapsodies. His repertoire was eclectic: he played tunes by pop performers, including Radiohead and Paul Simon (Simon, Paul), as well as his own original songs and jazz standards.

The jazz idiom could not contain Mehldau's ambitions. Over the years classical composers such as Johannes Brahms (Brahms, Johannes) and Robert Schumann (Schumann, Robert) often influenced his jazz stylings. Mehldau entered the classical realm himself when he composed settings of poems by Rainer Maria Rilke (Rilke, Rainer Maria) and Louise Bogan (Bogan, Louise). Mehldau accompanied the soprano Renée Fleming (Fleming, Renée)'s performances of the songs, which they recorded on the 2006 album Love Sublime. Mehldau, with his trio and the guitarist Pat Metheny, played concerts across North America, Europe, and Asia. As they toured, their albums Metheny Mehldau (2006) and Metheny Mehldau Quartet (2007) rose to the top of the jazz-album charts.