Rowell was called to the bar in 1891 and made king's counselor in 1902. As a member of the Ontario legislative assembly in 1911, he became leader of the Liberal opposition. He entered the Dominion House of Commons in 1917 and served in Sir Robert Laird Borden's Unionist government as president of the council. He was chairman of the Canadian cabinet's War Committee and a member of British prime minister David Lloyd George's Imperial War Cabinet.

Rowell represented Canada at the International Labour Conference in Washington, D.C., in 1919 and the first assembly of the League of Nations in 1920. Back in Canada, he organized the first federal department of health, then retired from the cabinet in 1920 and from Parliament in 1921. He served as president of the Canadian Bar Association in 1932–34 and became chief justice for Ontario in 1936. In 1937 he served as chairman of the Royal Commission of Dominion Provincial Relations, then retired from all posts in 1938 because of illness.