Papp, a former railway clerk, competed as a middleweight (161 pounds 【73 kg】) at the 1948 Olympic Games in London. A hard-hitting left-hander, he won the first of his gold medals by defeating Britain's John Wright in the final match. In 1952, when the light middleweight (156-pound 【71-kg】) division was introduced at the Olympic Games in Helsinki, Finland, Papp moved down into that division and defeated Theunis van Schalkwyk of South Africa for his second gold medal. At the 1956 Olympics in Melbourne, Australia, Papp won an unprecedented third gold medal by defeating American José Torres, a future light heavyweight world champion.

As an amateur, Papp also won the European middleweight championship in 1949 and the European light middleweight championship in 1951. In 1957, at age 31, he received permission from the Hungarian government to fight professionally, becoming the first boxer from a communist country to do so. Although in 1962 Papp became the professional middleweight champion of Europe, the Hungarian government refused to allow Papp to challenge for the world championship; in 1989 the World Boxing Council made him an honorary world champion. After successfully defending his European title six times, Papp retired, undefeated, in 1965. He later was the boxing coach (1971–92) for the Hungarian Olympic team. Considered one of the world's greatest middleweight fighters, Papp was inducted into the International Boxing Hall of Fame in 2001.