In Sumerian mythology Nippur was the home of Enlil, the storm god and representation of force and the god who carried out the decrees of the assembly of gods that met at Nippur. Enlil, according to one account, created man at Nippur. Although a king's armies might subjugate the country, the transference to that king of Enlil's divine power to rule had to be sought and sanctioned. The necessity of this confirmation made the city and Enlil's sanctuary there especially sacred, regardless of which dynasty ruled Mesopotamia.

The first American archaeological expedition to Mesopotamia excavated at Nippur from 1889 to 1900; the work was resumed in 1948. The eastern section of the city has been called the scribal quarter because of the many thousands of Sumerian tablets found there; in fact, the excavations at Nippur have been the primary source of the literary writing of Sumer. Excavation in 1990 uncovered an Akkadian tomb and a large temple to Bau (Gula), the Mesopotamian goddess of healing.

Little is known about the prehistoric town, but by 2500 BC the city probably reached the extent of the present ruins and was fortified. Later, Ur-Nammu (reigned 2112–2095 BC), first king of the 3rd dynasty of Ur, laid out Enlil's sanctuary, the E-kur, in its present form. A ziggurat and a temple were built in an open courtyard surrounded by walls.

Parthian construction later buried Enlil's sanctuary and its enclosure walls, and in the 3rd century AD the city fell into decay. It was finally abandoned in the 12th or 13th century.