In the 7th century, there was a primitive wooden castle on this spot to offer protection against the Vikings. In the 11th century the castle was modernized and rebuilt in stone. Franco, Lord of Herzele left his family and castle behind in 1096 to go on the holy crusade. But not before he fortified his castle again. For a long time all was quiet and peaceful until 1381 when the Count of Flanders battles against the army of Ghent and the castle was destroyed. Also the church and surrounding houses are burned down. Jan Van Roubaix rebuilt the castle in 1400. The year 1579 was catastrophic for the castle. The Spanish army pointed its canons to the castle and shot it to ruins. Only the tower that we still can admire today survived the bombardment. The reason for this siege of the castle was the sympathy Lord Pieter van Melun had for the Protestants. The ruin changed hands a couple of times before the town of Herzele bought it in 1969. The ruin was consolidated and the remaining tower was restored. The castle ruin is in the middle of the town and you can walk over the remains of this ones mighty castle.