When Archibald Acheson began to erect a large castle in 1820, many would have been forgiven for thinking the Gosford estate would continue to be a flourishing concern. In fact, a report by William Greig, published the following year, showed that the Gosford estate was not in a position to pay for the upkeep of a large castle.
The construction of Gosford Castle took almost forty years to complete and was estimated to have cost the estate £100,000. It was far from an asset. Many commentators of the time considered it to be a blight on the countryside, while others questioned if the Earl was merely trying to demonstrate a wealth he did not possess. In any case, the castle served as a home for the Acheson's for little more than a generation. The third Earl of Gosford spent little time at the estate, using it merely as a country retreat or to entertain shooting parties. Though he established a massive collection in the library, it would not survive the tenure of his son.
The fourth Earl of Gosford was a member of royal circles and, as his father before him, spent very little time at the estate. In 1888, he was forced to sell off much of his father's library to settle a racing debt but the reality was that the estate was in decline. Land agitation of the late-nineteenth century and the flourishing linen industry had deprived the estate of many of its tenants and thereby its income. The castle was too much of a drain on resources and in 1921 the contents of the castle were sold. Though the castle itself was not sold through lack of interest, the Estate ceased to function as it had been intended and became another empty building in the countryside, the Acheson family now forever severed from the area.