The Duke of Tczew, Grzymisław, located the Knights of St. John in Skarszewy in the 12th Century. The knights built a brick Castle on a steep headland surrounded on three sides by the Wietcisa River. It was composed of a residential part, preceded by a courtyard and utility homesteads. It was destroyed, rebuilt and renovated many times and was finally demolished by the Prussians. Today, the only remains of the Castle are the utility homesteads, which are home to the Community Centre.