Groener entered the army in 1884. By 1912 he had risen to become head of the railroad department of the German general staff; and, when World War I broke out, he was chiefly responsible for the rapid deployment of Germany's armed forces and their subsequent reinforcement and supply by rail in both the east and west. Later he became chief of several key departments of the War Office and was entrusted with tapping all production reserves for the war economy. In October 1918 Groener replaced Erich Ludendorff as first quartermaster general. Together with Field Marshal Paul von Hindenburg, he informed Emperor William II that he had lost the confidence of the army and advised him to abdicate, advice which the emperor soon took. After the Armistice Groener helped bring Germany's troops home from the front and cooperated with Ebert against the threat of a communist dictatorship by authorizing the formation of special army volunteer units to suppress the communist-inspired uprising.

Retiring from the army in 1919, Groener served as minister of communications (1920–23), minister of defense (1928–32), and minister of the interior (1931–32). He was one of the few top military figures who opposed the Nazis' infiltration of the armed services, and in 1932 he took measures to ban the SA, the Nazis' main paramilitary organization, an action that contributed to his fall from power that same year. After his retirement, he wrote several military histories.