Crerar graduated from the Royal Military College (Kingston, Ontario) in 1910 and received a commission as an artillery officer. He soon quit the military for better-paying civilian work but rejoined in 1914 to fight in France, where he rose to the rank of lieutenant colonel, once again in the artillery. He remained in the army after World War I, holding various staff posts of increasing importance. In 1940, after Canada had entered World War II, Crerar was promoted to major general and became chief of the Canadian army's general staff. In this post he worked to train and transport Canadian troops to Britain. He was promoted to lieutenant general in 1941 but then accepted a demotion in order to obtain a field command. He became commander of the I Canadian Corps, which fought in Sicily (July 1943) and Italy (from September 1943).

He was recalled to England in early 1944 to take command of the First Canadian Army, units of which landed on Juno Beach on D-Day (June 6, 1944) during the Normandy Invasion. Operating temporarily under Miles Dempsey (Dempsey, Miles Christopher)'s British Second Army, Canadian units took part in bitter fighting for the city of Caen (June–July) and then helped to close the northern arm of the Falaise-Argentan gap (August), in which large numbers of Germans were encircled and annihilated. By that time Crerar's army was directly under Field Marshal Bernard Montgomery (Montgomery, Bernard Law Montgomery, 1st Viscount) as part of the Twenty-first Army Group. Operating on the extreme left flank of the Allied drive across France, the First Canadian Army took the French Channel ports of Le Havre and Dieppe and then cleared the Schelde River estuary and captured Antwerp in Belgium. From there the army drove into The Netherlands and then breached the northern end of the Siegfried Line (Germany's fortified western frontier). Crerar was promoted to general in November 1944, and he retired in 1946.