Most likely it was Baldwin the Iron Arm, first count of Flanders, who constructed a wooden tower surrounded by battlements close to where the current castle now stands. After the count the castle came into the hands of several families. The families Voormezele, Hubrecht, Aertrijcke, Pinnock, Baents en Poucke were all owners of the castle through out centuries. In 1573 it became property of the Spanish merchant Matanca. He remodeled the castle in the same style as the Counts castle in Malle. The foundations and some other parts of the first castle are still visible. The family Schietere bought the castle in 1664 and through marriage it became property of the family Bailly in 1718. At the end of the 19th century the castle was sold to M. de Penaranda de Franchimont. He restored the castle completely in neo gothic style. Architects Bethune & van Assche preserved the middle-age look of the mighty castle in their restorations. The Counts of Briey and the Barons Verhaegen were the last private owners of the castle. Since 1980, the castle and beautiful park are property of the Province of West Flanders. The castle can not be visited however the park is open. At the moment the castle has no function, hopefully soon it will be opened to the public.