For a century, the impressive ruins are visited by numerous tourists and artists who give free rein to their romantic musings. During this period, interesting articles appear in the press reporting on the state of the castle and lamenting its decline.
Victor Joly, who visits the castle in 1844, sums up his impressions in his work "La Belgique Monumentale" in the following words: "In Beersel, situated in a charming valley and at the foot of a slope covered with plane trees, one of the most picturesque ruins to be found unfolds itself. Leaning against a wooded slope, the colossal towers, whose walls defy time, silhouette against the green coppice wood ... the castle's moats, currently dry, serve as a village meadow.
Stripped of their roofs and covered with living plants, the towers have not quite lost all of their splendour. The interior of the castle is in a deplorable state. The floors, hanging down at one end, or collapsed, offer a glimpse of the blue firmament as if through the opening of a shaft... curtain walls thirty-five feet high run from one tower to the next, still showing that they once were the strength of the site. Yet it is especially in the silence and peace of beautiful summer nights, when the moon casts its mysterious glow over the ruined castle, that it reveals its full grandeur. When, in the lap and on the summits of these towers battered by time and by slingshots, a green forest of feathers swings back and forth, you would swear that you see the shadows of the Lords van Witthem waving their silver flags with sky blue crosses. The sighing of the breeze gives a voice to every stone, and awakens a thousand strange noises in the gloomy staircases of the towers. At times, the wind forcing its way through the cross-windows makes the castle tremble as if it were about to be ripped from its foundations. The sight of this great ruin of the past leaves one with a feeling of intense melancholy that cannot be described."
Verses written down by Victor Hugo when he discovered the Castle of Beersel during a visit to Belgium in 1877, evoke a similar atmosphere:
"There it lies in the valley, the lonely manor,
The slightest noise is silenced under its gloomy arches,
And every hour of the day sees a stone fall from its dark battlements.
The raven has nested in its ancient halls,
where the wood owl repeats its cry night after night,
And the blade of grass grows between the cold paving stones of its vast corridors. »
The last inhabitants of the castle moved out and in 1818 parts of the towers collapsed. The moat silted up completely over the years. In the period of the First World War, there was already great concern about the castle ruins falling completely into decay, especially as many Beersel villagers greedily snatched building materials from the site. In 1914, the castle was listed as a monument. All access to the castle was blocked to save it from further destruction, but also because of the danger of collapse. A report from the Royal Commission for Monuments and Landscapes drawn up following a visit to the site on 19/07/1916 proves that this decision did not prevent the castle from falling into further decay or discourage people from stealing building materials. "The delegation was able to establish that the findings of Mr Minner Alfred, a Brussels architect, were not exaggerated. Newly found traces suggest that white stone and bricks have been stolen:
a) In the upper chamber of the tower to the left of the entrance tower. The right foot of the fireplace has completely disappeared, as have the jambs of the windows and doors in this area.The outside of this tower's base has also been attacked.
b) In the southern tower, the top lintel of the hall's fireplace, still partially vaulted, has been stolen. Someone has attempted to remove the wall corbel supporting the ribs of the vaulting.
c) In the curtain wall, the jambs of the window closest to the southern tower have been stolen, as well as the stones of one of the buttresses.
The Commission's advice was as follows: "The windows and existing openings in the curtain wall should be blocked with fences or doors to prevent looters and ill-intentioned people from entering the ruins." The guard who guides the delegation during the visit, claims that no one can enter the ruins. This can however easily be refuted; several visitors engraved their names in the castle walls, including Josse Van den Bossche, building contractor, Anderlecht; C. Cassiers 1916; H. Hannut, Anderlecht; Wauters. In the aftermath of the First World War, a number of people come up with the idea of restoring the castle.