Gouin was called to the bar in 1884 and made Queen's Counsel in 1900. Elected as a Liberal to the Quebec legislature in 1897, he served as Quebec's minister of public works (1900–04) and then was premier and attorney general of the province from 1905 to 1920. His administration built roads, founded technical and professional schools, and did much to foster the industrial development of Quebec. After retiring from the premiership, he was appointed to his province's legislative council. He was knighted in 1908.

Gouin served as the Liberal minister of justice from 1921 to 1924 and represented Canada in the fourth League of Nations assembly in 1924. He became lieutenant governor of Quebec in 1929, shortly before his death.