Robinson attended Leland College in Baker, La., where he played quarterback and led the team to a combined 18–1 record over the 1939 and 1940 seasons. During his final two years at Leland, he also served as an assistant coach. He earned his bachelor's degree in 1941 and received a master's degree from the University of Iowa in 1954.

In 1941 Grambling (then known as Louisiana Negro Normal and Industrial Institute) hired Robinson to coach football and basketball and teach physical education. In his first season he had no assistants and no budget for replacing equipment. He handled virtually everything himself, from mowing the field to taping players' ankles to writing accounts of the games for the local newspaper. That season his team posted a record of 3–5. The next season, however, he guided the team to a perfect 8–0 record.

Robinson's Grambling Tigers went on to have two more perfect seasons, capture 17 conference titles, and win several National Negro championships. In the 1960s, after several decades when football at historically black colleges went largely unnoticed by most football fans, Robinson's Grambling teams gained fame for sending more players into professional football than any school except Notre Dame. Among the more than 200 of his players who went on to compete in the National Football League were Hall of Fame members Willie Davis, Willie Brown, and Buck Buchanan. The racial integration of college football in the South in the 1970s ended this brief period of football glory for Grambling and other black colleges.

Surpassing Bear Bryant (Bryant, Bear)'s record for wins, Robinson earned his 324th career victory on Oct. 5, 1985, with a 27–7 defeat of Prairie View A&M in Dallas. At the end of the 1997 season, he retired with a lifetime record of 408–165–15. Robinson's record of 408 career victories stood until 2003, when it was broken by John Gagliardi, coach of St. John's of Minnesota. The recipient of numerous awards, Robinson was inducted into the College Football Hall of Fame in 1997.