Caroline Scott was the second daughter of five children born to John Scott, a college teacher and Presbyterian minister, and Mary Potts Neal Scott. Educated in the best schools of southern Ohio, she was an excellent student, showing special talent in painting and music. She met Benjamin Harrison while he was her father's student at a nearby college, and they were married on October 20, 1853. As her husband's legal and political career progressed, she cared for their two children, born in 1854 and 1858, and participated in community activities in Indianapolis, where they made their home until his election to the Senate (1881–86) took them to Washington, D.C. Wherever she lived, she continued to paint for recreation, and she became known as a model housewife.

Her husband's election to the presidency in 1888 brought Caroline enormous public attention. By that time, popular women's magazines regularly published lengthy articles on each president's family and home life, and they eagerly sought information about the many relatives the Harrisons brought with them to live in the White House, including their married children, their children's families, Caroline's father, and Caroline's niece, Mary Dimmick. Caroline laughingly concluded that other people knew more about the Harrisons than they knew themselves.

In order to make White House living more comfortable, Caroline Harrison worked with architect Fred D. Owen to draw up plans to expand the mansion. The plan she favoured would have retained the main structure and added wings on both sides, one for an art gallery and the other for offices. However, Congress rejected the plan, and Caroline had to make do with the same limited space her predecessors had, though she did oversee an extensive renovation, including the installation of electric lighting.

Eager to make the White House a showcase for American creativity and workmanship, Caroline designed new state china featuring goldenrod and sweet corn. Because she was dissatisfied with the quality of china produced in the United States, however, she had the dishes made in France. In addition to her organizational efforts for the Daughters of the American Revolution, she helped to raise funds to start a medical school at Johns Hopkins University. Although she was not an active supporter of women's rights, she agreed with many of the benefactors that the new school should admit women.

During her husband's campaign for reelection in 1892, Caroline contracted tuberculosis, and she died in the White House, the second president's wife (after Letitia Tyler) to die there. She was buried at the Crown Hill Cemetery in Indianapolis.

Four years later her husband married Mary Dimmick. Rumours circulated about when the romance had begun, and when Benjamin died in 1901 his children, who were never reconciled to the second marriage, buried him beside Caroline.