Born in Sicily, Gambino immigrated to the United States in 1921 as a ship stowaway, settling in Brooklyn and becoming a gang member under, successively, Joe “the Boss” Masseria, Salvatore Maranzano, Philip and Vincent Mangano, and finally (in 1951) Albert Anastasia (Anastasia, Albert). When Anastasia was murdered in 1957 (allegedly by Gambino and another gunman and with Vito Genovese's approval), Gambino inherited leadership of the family, whose operations included gambling, loansharking, hijacking, narcotics trafficking, and, notably, labour racketeering, with extensive controls over waterfront unions.

In 50 years of crime Gambino served only 22 months in prison (1937–38). Although he was indicted in 1970 for hijacking and although the U.S. Supreme Court in 1970 also upheld an order for his deportation (because of illegal entry into the country), severe heart attacks deferred trial or deportation. More heart seizures led to his death by natural causes six years later.