In the past it was called Karlsberg (The Karl Hill). The name Pachołek (servant, lackey) is likely to originate from monastery servants – they would pasture pigs in the nearby forest, so it is possible that they also built a shed at the top of the hill to shelter themselves from the rain. It also could have been a fire watchtower, as might be inferred from a reference dating back to 1739 about a forest fire in the vicinity of the hill, which was collectively put out by the Reverend Prior, the Reverend Father Ambroży and the population of Oliwa.
An observation pavilion was built here in 1798 on the initiative of the Oliwa Abbot – Karl von Hohenzollern-Hechingen – to be later, in 1822, replaced by a construction made of brick and metal. The tower was blown up as a result of military action by German soldiers who were anxious that it would serve as an observation point for the battlefield within the city. Since 1975, a 15-metre-high observation platform has been standing here to provide a panorama of Gdańsk Bay and the city.