Its founder, John Lyon (d. 1592), was a yeoman of neighbouring Preston who yearly set aside resources for the education of poor children of Harrow. The school's charter was granted by Elizabeth I in 1571, and its statutes were promulgated by Lyon in 1590, but it was not until 1615 that the first building was opened. About 1660 the headmaster began to receive “foreigners”—i.e., boys from other parishes who could pay fees. Control was originally vested in six parishioners of standing, but under the Public Schools Act of 1868 the governing body consisted of six members elected respectively by the lord chancellor, the universities of Oxford, Cambridge, and London, the Royal Society, and the assistant masters of the school.

The school's main buildings were built in the 19th century, but the old Fourth Form Room dates from the early 17th century; on its oak panels are carved the names of eminent alumni, among whom are Sir Robert Peel (Peel, Sir Robert, 2nd Baronet), Henry John Temple (Lord Palmerston) (Palmerston, Henry John Temple, 3rd Viscount, Baron Temple Of Mount Temple), Richard Brinsley Sheridan (Sheridan, Richard Brinsley), Lord Byron (Byron, George Gordon Byron, 6th Baron), and Henry Cardinal Manning (Manning, Henry Edward). Other famous pupils include John Galsworthy (Galsworthy, John), Anthony Trollope (Trollope, Anthony), Lord Shaftesbury (Shaftesbury, Anthony Ashley Cooper, 7th Earl of, Baron Cooper Of Pawlett, Baron Ashley Of Wimborne St. Giles), and Sir Winston Churchill (Churchill, Sir Winston). The Vaughan Library and the Chapel were designed by Sir George Gilbert Scott (Scott, Sir George Gilbert), and the War Memorial Building is the work of Sir Herbert Baker. Harrow songs, the Bill (roll call in the school yard), and Harrow straw hats are well-known features of the school.