As a student at Yale University, Poole learned the principles of indexing from John Edmands (1820–1915), afterward librarian of the Philadelphia Mercantile Library, whom Poole succeeded as librarian of a college literary society. While still in college, Poole prepared An Alphabetical Index to Subjects Treated in the Reviews and Other Periodicals, to Which No Indexes Have Been Published (1848), which was revised and enlarged as Poole's Index to Periodical Literature (1887–1908).

After directing successively two libraries in Boston (1852–69), Poole organized and served as chief librarian of the public libraries of Cincinnati (1871–73) and Chicago (1874–87), building the circulation of the Chicago library to the largest in the United States in its time. Subsequently he organized the Newberry Library, Chicago.