Nicolas de Fontaine was one of the first big lords of Hainaut. He was the one who built this impressive castle in the 13th century. Beside Lord of Hainaut Nicolas was also Bishop of Cambrai. Beside all of that he was also a smart cookie and was admired for his intelligence by the king of the Romans, Richard Of Cornwall who made him Prince of the Holy Roman Empire. Nicolas sister who was the wife of Baudouin de Hennin-Liétard inherited the castle. Later on the castle became property of renowned families like the Hamals, Herzelles and the Rodoans. The castle was completely restored in the 17th and 18th century and was given an interior that was elegant and refined. In the 19th century the family Bivorts de la Saudée renovated the castle and made it considerable more comfortable. Nowadays the castle is one of the most impressive town halls in the region.