In 1787 Nathaniel Gordon founded the Gordon Tavern, an eating and drinking establishment originally located at the Gordonsville circle. The Gordon Tavern was in Thomas Jefferson’s words a “good house” which played host to many including Marquis de Lafayette, James Madison, James Monroe and Robert E. Lee. The location of the tavern adjoined roads well traveled; Mr. Jefferson advised friends traveling to and from Charlottesville to follow the “Fredericksburg Great Road”, right past the Gordon Tavern.
The site of the Toliver House Restaurant was a part of the original Nathaniel Gordon holdings called “Gordonsville” which encompassed 1300 acres. Originally constructed circa 1870, the building has been among other things a private residence, boarding house, general store and a restaurant for the past 25 years. The name Toliver is the anglicized form of “Taliaferro”, an old Virginian surname found on the oldest recorded deed of the property.

The town of Gordonsville was created when the Gordon Family began selling off large parcels of the estate in 1828. The area changed little duing the following 11 years until the railroad line was extended from Louisa through Gordonsville and onto the town of Orange. When the first trains came in on the Louisa Line, women and children would greet the trains with balanced trays on their heads of “succulent fried chicken, pies, caks, biscuits and other good edibles”. This phenomenon which lasted fifty years was witnessed first hand by a train full of newspaper journalists in the 1870’s and the event was published throughout many states. Gordonsville was forevermore referred to as “the chicken leg center of the universe.