Educated in Sweden and abroad, Fersen entered the Swedish army in 1737. In 1739 he was given leave to join the French army, in which he soon distinguished himself in the War of the Austrian Succession (1740–48). Returning to Sweden, he joined the ruling Hat Party in the Riksdag (parliament). His position in the party was enhanced not only by his father's having been a founder but also by Fersen's marriage into the prominent Hat family of De la Gardie (1752). He was elected speaker of the noble chamber of the Riksdag in 1756, and he used this powerful office to check all efforts by the crown to regain the power it had lost to the Parliament in 1720.

Fersen served with distinction in the Seven Years' War (1756–63). Afterward he unsuccessfully tried to effect an alliance between the Hats and the crown against a rise of a new generation of rivals (the Nightcap, or Cap, Party) to his own party. When King Gustav III began to reassert the governing power of the monarchy in 1772, Fersen at first supported him, but he again led an antiabsolutist faction after the start of the disastrous Russo-Swedish War of 1788–90. Fersen retired from public life in 1789.