Ordained in 1861, Gibbons spent four years as pastor and volunteer chaplain to the Civil War troops in the military hospitals of Baltimore. In 1868 he was consecrated bishop and appointed to organize the new Vicariate Apostolic of North Carolina; in this capacity he attended the first Vatican Council in 1869–70. In 1872 he was created bishop of Richmond, Va., and in 1877 he was named coadjutor to the archbishop of Baltimore. His experiences as a missionary bishop made him aware of the need for a simple and concise statement of Roman Catholic doctrines, and while at Richmond he wrote The Faith of Our Fathers (1876), which became one of the most popular volumes of Roman Catholic apologetics published in the United States.

Appointed archbishop of Baltimore in 1877, thereby becoming a leader of the church in the United States, he was made cardinal in 1886 by Pope Leo XIII. He became the first chancellor of the Catholic University of America (Catholic University of America, The), Washington, D.C. (1889). In the 1880s and '90s he sought peace between immigrating Roman Catholic groups, particularly Irish and German. Politically, he emphasized to Rome the separation of church and state in the United States, whose constitution he believed was the finest instrument of government yet created. Gibbons' golden jubilee celebration in Baltimore (1911) was led by President William Howard Taft. His Discourses and Sermons were published in 1908.