After graduating from Rensselaer Polytechnic Institute, Troy, N.Y. (1857), he joined his father in the work of building suspension bridges. The Civil War, however, intervened, and he served in the Union Army, rising to the rank of colonel by war's end, after which he returned to the construction business. His father put him in charge of the construction of the enormous masonry towers that supported the cables of the Cincinnati–Covington Bridge (1865–67) and then sent him to Europe to study new methods for the sinking of the foundations on which the granite towers of the Brooklyn Bridge were to stand.

On his father's death in 1869 he was asked to serve as chief engineer on the Brooklyn Bridge and immediately began work on the foundations for the two towers. The use of pneumatic caissons (watertight chambers) was still in a somewhat experimental stage, and what happened to men working in compressed air at the bottom of the caisson was not yet fully understood. Though every precaution was taken there were more than a hundred cases of decompression sickness (the “bends”) when men were brought up too rapidly. There were also the usual difficulties of fires and breakdowns, as well as so-called blow-outs that shot mud and water into the air. Like his father, Colonel Roebling felt he had to inspect every detail of the work. One day he remained 12 consecutive hours in the compressed-air chamber, finally being carried out unconscious. In those days there was little understanding of the amount of time needed for slow decompression. The nitrogen bubbles in the bloodstream could paralyze a man for life. His health was permanently affected, and, though he lived to be almost 89, the Brooklyn Bridge was his last major undertaking. The bridge, opened in 1883, took 13 years to complete.