Rutherford was raised by two aunts who encouraged her interest in the theatre. After teaching piano for five years and elocution for three, she entered the Old Vic School in London to study theatre (1925). Appearing frequently on the London stage from the early 1930s, she received critical notice for her role in Henrik Ibsen's Master Builder (1934). Her first great success was as the unconventional Miss Bijou Furse in Spring Meeting (1938). In Oscar Wilde's The Importance of Being Earnest (1939), she played Miss Prism, a part she repeated on the screen in 1952; and in Noël Coward's Blithe Spirit (1941), she played the bicycling medium in both the stage and the motion-picture version (1945).

Although she made her film debut in 1936, Rutherford is best known for later pictures, which include Passport to Pimlico (1949); The Mouse on the Moon (1963); and as Miss Marple in Agatha Christie's detective series, Murder She Said (1961), Murder at the Gallop (1963), Murder Ahoy (1964), and Murder Most Foul (1964). For her portrayal of the unconventional Duchess of Brighton in The VIPs (1963), Rutherford won the Academy Award for best supporting actress. In 1967 she was made a Dame of the British Empire.