Educated at the military schools of La Flèche and Saint-Cyr, Pélissier was commissioned as an artillery second lieutenant in 1815. After brief service in Algeria in 1830, he returned there in 1839 to take part in the campaign against the patriot emir of Mascara, Abdelkader. It was during this period that he gained notoriety for killing an entire local population by gassing them in caves, a tactic of irregular warfare employed by the French on several occasions during the Algerian campaign. In June 1845 the Ouled Riah tribe, driven from their settlements by Pélissier's forces, found refuge in the caves of the Dahra mountains. Thomas-Robert Bugeaud (Bugeaud, Thomas-Robert, duke d'Isly, marquis de la Piconnerie), another French military leader, had previously advised Pélissier that if the populace hid themselves in caves, they ought to be “smoked,” as their colleague Louis-Eugène Cavaignac (Cavaignac, Louis-Eugène) had done with the Sbeah tribe; Pélissier duly heeded the advice and asphyxiated the men, women, and children of the Ouled Riah cornered there.

Pélissier was promoted to general in 1846 and served as military commander of the coastal province of Oran from 1848 to 1851. During the 1852 coup d'état of Napoleon III, he was interim governor of Algeria. Subsequently, he played a leading role in the subjugation of the tribes of southern Algeria.

In January 1855 Pélissier was given command of an army corps in the Crimea. He defeated a Russian attack on the Traktir Ridge on August 16 and captured the fort of Malakhov and the city of Sevastopol on September 8 (see Siege of Sevastopol (Sevastopol, Siege of)). Four days later he was made marshal of France, and in July 1858 he was created duc de Malakoff. From March 1858 to April 1859 Pélissier was French ambassador to London. He was governor of Algeria from 1860 until his death.