After three years in charge of the Bladensburg, Md., circuit of the Methodist Episcopal Church, Cummins began study for the ministry of the Protestant Episcopal Church. Ordained a priest in 1847, he became a distinguished preacher and served several important parishes in the eastern United States and in Chicago. Consecrated an assistant bishop of Kentucky in 1866, he was one of the younger leaders of the evangelicals and soon found himself in the midst of the controversy over ritual, which reached its peak in the Episcopal church in the years following the American Civil War (1861–65). Those who despaired of the future of evangelicalism within the church appealed to him as early as 1869, seeking to persuade him to begin organizing a new body. At first reluctant to make a break, he organized the Reformed Episcopal Church in December 1873 in New York City and became its first presiding bishop.