Thompson was called to the bar in Nova Scotia in 1865 and appointed queen's counsellor in 1879. He entered politics in 1877 as Liberal-Conservative member for Antigonish in the provincial legislature, becoming attorney general in 1878; he became premier of Nova Scotia in 1882 but was defeated in the same year. After serving as judge in the provincial Supreme Court, he was appointed minister of justice in 1885 in Sir John Macdonald's federal administration, having been elected to the Dominion Parliament for Antigonish.

Thompson successfully defended the government in various politico-religious controversies, including reactions to the 1885 execution of Louis Riel, to the Jesuits' Estates Act (1888–89), and to the subordination of the French language. In 1888 he helped arrange a fisheries treaty between Great Britain and the United States, and for this work he was knighted (1888). Following the death of Macdonald in 1891 and the resignation of Macdonald's successor, Sir John Abbott, in 1892, Thompson was named to lead the party. He was premier until his death, which occurred minutes after having been sworn into the privy council by Queen Victoria.