While studying law in the early 1860s Mercier edited the Conservative newspaper Le Courrier de St. Hyacinthe and supported the Conservative administration and the interests of Quebec. He left the Conservative Party in 1866 and vigorously opposed confederation as damaging to the French-Canadian cause. Mercier soon retired from journalism. In 1871, after the creation of the Dominion of Canada (1867), he began to help form the Parti National (National Party). In 1872 he was elected to the House of Commons for the county of Rouville, serving for two years.

Mercier entered the legislative assembly of Quebec in 1879 and was made solicitor general, succeeding to the leadership of the Quebec Liberal Party in 1883. He succeeded in the 1886 provincial election largely because of his agitation on behalf of Louis Riel, French-speaking leader of the Northwest Rebellion, who was executed in 1885. As premier and attorney general of Quebec beginning in 1887, Mercier pursued a nationalist policy; he succeeded in passing the Jesuits' Estates Act to compensate the Society of Jesus for property formerly confiscated by the British crown.

While visiting Rome in 1891, Mercier was created a Roman count by Pope Leo XIII. Also in 1891, after several years of popularity, Mercier was accused of using railway subsidies for political purposes and was dismissed by the lieutenant governor. The Liberal Party was badly defeated in 1892, but the new government acquitted Mercier of the charges.