The extravagant home overlooks Lafayette Square and remained in the hands of one extended family for more than a century.