During the spring of 1779, Antonio Gil Y'Barbo founded present day Nacogdoches with the settlement of 350 men, women, and children at the location of the abandoned Nacogdoches mission, and he constructed his stone house sometime between the years of 1788 and 1791. The stone house was a landmark in Nacogdoches through the control of eight different governments in East Texas--Spain, the Magee-Gutierrez expedition of 1813, the James Long's Republic of 1819, the Republic of Fredonia of 1826, Mexico, the Texas Republic, the Confederacy, and the United States of America. Local legend tells that Sam Houston took the oath of allegiance to the Republic of Texas in the house, and Davy Crockett spent two nights there on his way from Tennessee to the Alamo. "Under its roof Jim Bowie and Stephen F. Austin also spent many days in trying times" (Post: 'Preserve the Old Fort', Tuesday morning, March 11, 1902).

The variety of owners and uses of the stone house contribute to the rich texture of Nacogdoches life in the late eighteenth, nineteenth, and early twentieth centuries. "During it's history, the original house served variously as a public building, grocery store, candy store, saloon, and as a temporary fortification during the Magee-Gutierrez, Dr. James Long and Fredonia Rebellion filibusters" -(Carolyn Spears, Curator of the Stone Fort Museum).

In 1902, the oldest stone house in the state of Texas was demolished in order to make room for a more modern structure. After five years of fundraising, the Cum Concilio Club succeeded in raising enough money to erect a memorial building on Washington Square, and on July 4th of 1907, the Club laid the cornerstone for the memorial structure. The memorial building incorporated some of the original stones from Y'Barbo's stone house, and was used for a variety of public functions. In 1936 a replica of the original stone house was built on the campus of the Stephen F. Austin State Teachers College. The memorial structure was torn down and some of the original stones from Y'Barbo's stone house were reused in the construction of the Stone Fort Museum, which is still in use today.