The first stone for the line was laid on July 4, 1828, by Charles Carroll, the American Revolutionary leader and last surviving signer of the Declaration of Independence. The first 13 miles (21 km) of line, from Baltimore to Ellicott's Mills, Md. (now Ellicott City), opened in 1830. Peter Cooper's steam locomotive, the Tom Thumb, ran over this line and demonstrated to doubters that steam traction was feasible on the steep, winding grades.

The railroad was extended to Wheeling, Va. (now in West Virginia), a distance of 379 miles (610 km), in 1852. In the 1860s and '70s, the railroad reached Chicago, Ill., and St. Louis, Mo. In 1896 it went bankrupt. After it was reorganized in 1899, it grew further, reaching Cleveland, Ohio, and Lake Erie in 1901. In 1963 the B&O was acquired by the Chesapeake and Ohio Railway Company and in 1980 became part of the newly formed CSX Corporation.

The B&O's long-distance passenger trains were discontinued in 1971 when the National Railroad Passenger Corporation (Amtrak) took over intercity passenger service, although it continued limited commuter service at Washington, D.C., and Pittsburgh, Pa. About one-quarter of the B&O's freight revenues come from its traditional haulage of bituminous coal from mines in the Allegheny Mountains. Other important freight includes motor vehicles and parts, as well as chemicals.