Joseph John Williams was born in North Carolina in about 1832. The son of a wealthy plantation owner, he attended Princeton and moved to Tallahassee, Leon County, Florida, where he became the owner of several large cotton plantations. He married Eliza Thompson in 1854 and served briefly in the Confederacy as a colonel during the civil war. He died in Raleigh, North Carolina while visiting family on 19 November 1876.