Gibbs was the fourth child and only son of Josiah Willard Gibbs, Sr., professor of sacred literature at Yale University. There were college presidents among his ancestors and scientific ability in his mother's family. Facially and mentally, Gibbs resembled his mother. He was a friendly youth but was also withdrawn and intellectually absorbed. This circumstance and his delicate health kept him from participating much in student and social life. He was educated at the local Hopkins Grammar School and in 1854 entered Yale, where he won a succession of prizes. After graduating, Gibbs pursued research in engineering. His thesis on the design of gearing was distinguished by the logical rigour with which he employed geometrical methods of analysis. In 1863 Gibbs received the first doctorate of engineering to be conferred in the United States. He was appointed a tutor at Yale in the same year. He devoted some attention to engineering invention.

Gibbs lost his parents rather early, and he and his two older sisters inherited the family home and a modest fortune. In 1866 they went to Europe, remaining there nearly three years while Gibbs attended the lectures of European masters of mathematics and physics, whose intellectual technique he assimilated. He returned more a European than an American scientist in spirit—one of the reasons why general recognition in his native country came so slowly. He applied his increasing command of theory to the improvement of James Watt's steam-engine governor. In analyzing its equilibrium, he began to develop the method by which the equilibriums of chemical processes could be calculated.

He was appointed professor of mathematical physics at Yale in 1871, before he had published his fundamental work. His first major paper was “Graphical Methods in the Thermodynamics of Fluids,” which appeared in 1873. It was followed in the same year by “A Method of Geometrical Representation of the Thermodynamic Properties of Substances by Means of Surfaces” and in 1876 by his most famous paper, “On the Equilibrium of Heterogeneous Substances.” The importance of his work was immediately recognized by the Scottish physicist James Clerk Maxwell (Maxwell, James Clerk) in England, who constructed a model of Gibbs's thermodynamic surface with his own hands and sent it to him.

He remained a bachelor, living in his surviving sister's household. In his later years he was a tall, dignified gentleman, with a healthy stride and ruddy complexion, performing his share of household chores, approachable and kind (if unintelligible) to students.

Gibbs was highly esteemed by his friends, but U.S. science was too preoccupied with practical questions to make much use of his profound theoretical work during his lifetime. He lived out his quiet life at Yale, deeply admired by a few able students but making no immediate impress on U.S. science commensurate with his genius. He never even became a member of the American Physical Society. He seems to have been unaffected by this. He was aware of the significance of what he had done and was content to let posterity appraise him.

The contemporary historian Henry Adams (Adams, Henry) called Gibbs “the greatest of Americans, judged by his rank in science.” His application of thermodynamics to physical processes led him to develop the science of statistical mechanics; (statistical mechanics) his treatment of it was so general that it was later found to apply as well to quantum mechanics as to the classical physics from which it had been derived.