Hunmanby Castle is a Norman earthwork motte and bailey fortress, founded by Gilbert de Gant. The motte standing on a natural knoll is the highest point on Castle Hill and a wide ditch gives defence from the high ground to the west. Curving round the motte to the north is the steep scarp of the road embankment but to the south and east the encasing motte ditch has been infilled. The southern part of the large bailey has been altered by terracing but the northern portion which is defined by the steep scarp of the road remains untouched. Destroyed in the 12th century, during the civil war in the reign of King Stephen, the castle and a large area of land to the west of the village, was emparked in the 18th century. 9 miles north is Scarborough Castle and 10 miles south-east is Flamborough Castle.