After admission to the bar in Pennsylvania (1875), Knox became a successful corporation lawyer in Pittsburgh and as counsel for the Carnegie Steel Company had a prominent role in the organization of the United States Steel Corporation (1900–01). Appointed attorney general in 1901 by Pres. William McKinley, Knox remained in that office under Pres. Theodore Roosevelt, serving until 1904. At Roosevelt's direction, he instituted a number of suits under the Sherman Anti-Trust Act of 1890, most notably that which resulted in the dissolution of the Northern Securities Company (1902–04). Knox then served in the U.S. Senate (1904–09), resigning to become secretary of state in the administration of Pres. William Howard Taft.

The foreign policy of Taft and Knox, called “dollar diplomacy” by critics, sought to expand American political influence overseas by increasing American investments abroad, and to minimize the danger of European intervention in Central America or the Caribbean by persuading the nations in those regions to borrow from U.S. rather than European banks. It was a policy which had only minimum success, but it was widely misunderstood and widely condemned. Knox was returned to the U.S. Senate in 1917, where he was a leading opponent of the League of Nations.