After studying law, Arnim entered the diplomatic service in 1850 and served in Rome (1853–55) and Lisbon (1862). He was appointed Prussian envoy to the Holy See in 1864. Before the first Vatican Council of 1869–70, he made proposals intended to prevent a declaration of papal infallibility, which, he foresaw, would create certain political difficulties in Germany.

Arnim took part in the negotiations to end the Franco-German War and was appointed Prussian envoy to France on Aug. 23, 1871, becoming ambassador on Jan. 9, 1872. In June 1872 he arranged the war reparations settlement with France, but differences soon arose between him and Bismarck. Arnim, who supported the French monarchists, considered that Bismarck's backing of the new republican regime in France would encourage opponents of the monarchy in Germany. Arnim's favour at court and his support of the conservative groups among the German nobility led Bismarck to suspect that Arnim was planning to supplant him.

Then in 1874 a Viennese newspaper published correspondence on the Vatican Council, including some of Arnim's confidential dispatches, with the apparent aim of suggesting that he had exhibited greater foresight than Bismarck. The subsequent inquiry revealed that more important documents from Arnim's Paris embassy were missing. Arnim refused to return some of the missing documents and so was suspected of keeping them in order to prove that his own French policy had been wiser than Bismarck's. Bismarck thereupon had him temporarily superannuated, then arrested (Oct. 4, 1874). Condemned to three months' imprisonment, Arnim appealed, but his sentence was increased to nine months.

Arnim went into exile and anonymously published Pro Nihilo (1875), a pamphlet attributing his disgrace to Bismarck's jealousy. Convicted of treason, of insulting the emperor, and of libeling Bismarck, Arnim was sentenced in absentia to five years' penal servitude. Since the legal grounds for Arnim's prosecution had been doubtful, Bismarck obtained passage of the Arnim Paragraph in 1876.