Wheeler entered the U.S. cavalry from West Point in 1859 but soon resigned to enter the Confederate service. He commanded a brigade at the Battle of Shiloh (April 6–7, 1862), but soon afterward he returned to the cavalry arm, in which he won a reputation second only to Gen. Jeb Stuart's. After the action of Perryville he was promoted to brigadier general and, in 1863, to major general. Throughout the campaigns of Chickamauga, Chattanooga, and Atlanta, he commanded the cavalry of the Confederate Army in the west and was given the task of harassing Gen. William Tecumseh Sherman's army during its march to the sea. In the closing operations of the war, with the rank of lieutenant general, he commanded the cavalry of Gen. Joseph Johnston's weak army in North Carolina and was included in its surrender.

In 1898, during the Spanish–American War, Wheeler commanded the calvary in the actions of Guasimas and San Juan. He wrote The Santiago Campaign (1898).