Educated at the universities of Göttingen and Edinburgh, Haldane was called to the English bar in 1879 and became a queen's counsel in 1890. He sat in the House of Commons from 1885 until his elevation to the peerage in 1911. As a member of the imperialist wing of the Liberal Party, he supported the British effort in the South African War (1899–1902), thereby differing from the party leader, Sir Henry Campbell-Bannerman (Campbell-Bannerman, Sir Henry). The latter's appointment of Haldane to the War Office (effective Dec. 11, 1905) proved fortunate for Great Britain because of the administrative abilities Haldane demonstrated in his new post. Although the Territorial Force that he created was nominally an army reserve organization for protecting the British Isles, many of its units volunteered to fight in continental Europe in World War I. The speedy mobilization of the British Expeditionary Force in August 1914 was largely the result of his planning. He also took the lead in forming a national general staff (from 1904) and an imperial general staff (from 1909); for this purpose, Emperor William II allowed him to study German general staff operations at first hand in 1906. As Anglo-German relations were deteriorating, Haldane went to Berlin in February 1912 on a well-publicized but ineffectual mission concerning British neutrality and the relative naval strength of the two countries.

On June 10, 1912, Haldane became lord chancellor in H.H. Asquith's Liberal government. He soon increased the number of lords of appeal and otherwise worked to hasten the judicial process. In May 1915, however, when Asquith formed a wartime coalition ministry, he excluded Haldane, who was unjustly accused of being pro-German. By the end of the war his political orientation had shifted to the left. In Ramsay MacDonald's first Labour Party government (January–November 1924), he once more served as lord chancellor.

Long interested in education, Haldane was associated with the Fabian Socialists Sidney and Beatrice Webb in founding the London School of Economics in 1895. As a philosopher he expounded a kind of neo-Hegelianism. In The Reign of Relativity (1921) he dealt with the philosophical consequences of Albert Einstein's theories of physics. His Autobiography was published posthumously in 1929.