Orford Castle was built between 1165 and 1173 by Henry II of England who wanted to keep a watch over the barons and lords in that area of Suffolk.