This Byzantine Acropolis was built on an ancient fort used to defend the city in the 7th and the 6th centuries BC. During the Byzantine period it had been quoted in many golden bulls of different Byzantine emperors as "Castle". However, as time went by, the word "Castle" ended up including the whole city. That is the reason why, at the beginning of our century the people of Serres were called "Kastrini" by the inhabitants of the villages around Serres. From the Frankish conquest in 1204 and onwards the name "Kasteli" prevailed for the Acropolis and remained until the years of the Turkish domination. The Turks called it "Bas Koule" (top tower) probably because of the large tower of the west side. The present name "Koulas" (tower) is derived from that Turkish name.
The Acropolis was surrounded by a wall which stretched from the West to the East, in the Byzantine period. It surrounded all kinds of different Byzantine buildings, such as the homes of state governors and officers. Also, the provost marshal called "castle guardian", who was appointed by the Byzantine authorities, lived in the Acropolis. A well-known castle guardian was Leon Azanitis, who countersigned in golden bulls in the first half of the 14th century. According to the Turkish traveller Elvia Tselebi, the wall of the Acropolis had two gates on the east and west sides of the castle, very close to the towers. Some remains of the second gate can still be seen today, close to the large tower on the western side.
High and stiff towers were boosting the defensive strength of the wall, of which only the one on the western side, "the tower of the king" was preserved, destroyed only its highest part. This huge tower, reaching no less than 20 meters in height, helped to protect the city as well as served as the last point of defence in case the enemy stormed the rest of the Acropolis.
A sign in the right end of the tower of the king attracts great interest and most likely reads: "ΠΥΡΓΟΣ ΑΝΔΡΟΝΙΚΟΥ ΒΑΣΙΛΕΩΣ ΟΝ ΕΚΤΙΣΕΝ ΟΡΕΣΤΗΣ" (King Adronikos' tower which was built by Orestis).
The first reference to the Byzantine Acropolis is made in the 9th century, according to historical sources, when the emperor Nikiforos Fokas fortified the city of Serres. The impregnable Acropolis had better luck than the wall of the city: it managed to preserve itself from terrible barbaric irruptions and horrifying devastation. Even in 1204, during the Frankish raid, when the city's wall were completely destroyed, the Acropolis escaped untouched. The gradual delepidation of the castle began when Serres were subjugated to the Turks in 1383. The castle was abanbined because the Turks feared that the castles would be a stronghold of resistance in case a revolutionary movement of the subjugated Greeks.