At age 19 Virén dropped out of school to train under Rolf Haikkola, a follower of the New Zealand coach Arthur Lydiard, who stressed the importance of endurance gained by running very long distances. At the 1972 Games, Virén fell during the 10,000-metre race but scrambled quickly to his feet and, running faster than ever, made up 50 metres lost to the leaders, held off a charge by Belgium's Emiel Puttemans in the last lap to win the gold medal, and set a world record of 27 min 38.4 sec. The win gave Finland—a country that had dominated the distance events in the 1920s and early '30s—its first gold in 36 years. He also set an Olympic record when he won the 5,000-metre race at Munich. He successfully defended both titles at the 1976 Olympics in Montreal and placed fifth in the marathon. His final Olympic performance was a fifth-place finish in the 10,000 metres at the 1980 Games in Moscow.

Because Virén focused his training on Olympic competition, his highest achievement elsewhere was earning a bronze medal in the 5,000-metre event at the 1974 European championships. At one time a policeman, Virén was a member of Finland's parliament from 1999 to 2007.