Coming from a noble family, Graham began his military career as a soldier of fortune in France and the Netherlands. He returned to Scotland by 1678 and was made captain of the dragoons sent to the southwest to suppress Presbyterian insurgents who opposed the Anglican regime of King Charles II. Although beaten by the rebels at Drumclog Moss, Lanark, on June 1, 1679, Graham helped defeat them at Bothwell Bridge on June 22.

He spent most of the next two years in England, where he won the favour of Charles's brother James, Duke of York. After the Duke of York assumed the throne as James II in 1685, Graham at first took little part in military or governmental affairs. But when the Dutch Protestant stadtholder William of Orange invaded England in November 1688, Graham became second-in-command of the Scottish army sent to aid the king. He was created Viscount Dundee by James on November 12.

James fled from England in December, and Dundee then returned to Scotland to champion the cause of the exiled monarch. Rallying forces in the central Highlands, he ambushed General Hugh Mackay at the Pass of Killiecrankie on July 17, 1689. Dundee's forces were completely victorious, but he was shot and fell from his horse, mortally wounded. In August the Scottish resistance was crushed at the Battle of Dunkeld.