He was a direct descendant ofRoger Ludlow (1590–1664), who was electeddeputy governor of the Massachusetts Bay Colony andlieutenant governor of Connecticut and was the first in the family to settle inNorth America. His paternal grandfather, Obadiah Ludlow (Ludlam),[8] an officer of the New Jersey troops during theAmerican Revolution,[9] wasGeorge Washington's right hand man throughout the War.[10][11] Dr. Charles Stedman Bull was the son of Ludlow's first cousin, Henry King Bull.[9] Ludlow's 4th great-grandfather, William Ludlam Sr.(1605-1665), arrived in Southampton, Long Island, about 1653. Ludlam's will is the first recorded will in New York City history.
He joined the73rd New York Volunteer Infantry as a 2nd lieutenant, and was soon attached to the staff of Major GeneralJohn Adams Dix and promoted to lieutenant colonel and aide-de-camp to the general.[16]
At the end of the Civil War, he was the agent for exchange of prisoners atFort Monroe, Virginia, and wasbrevetted brigadier general and major general, US Volunteers, on March 13, 1865, for "faithful and meritorious services during the war".
In 1866, he was appointed as part of the State Central Committee for the State of New York to the National Union State Central Committee, representing Suffolk County.[17] Despite his retirement from politics, he continued to attend political events.[18]
After the war, starting in 1868, he was president of the Tontine Life Insurance Company of New York.[19]
On December 9, 1841, he married Frances Louisa Nicoll (1822–1887),[20] daughter of William Nicoll (1798–1823) and Sarah Greenly, heiress of the Nicoll estate aroundIslip onLong Island, New York. They were the parents of six children, including:[19]
Nicoll Ludlow (1842–1915),[21] an admiral with the Navy[22] who was married to Frances Mary Thomas (1842–1873) and Mary McLean Ludlow (1846–1915).[23]
William Ludlow (1843–1901),[6] who married Genevieve Almira Sprigg (1842–1926), daughter of James Rolfe Sprigg, on January 23, 1866.[24][11]
Newton Perkins Ludlow (1846–1858), who died young.[19]