Dunstable Priory was founded by King Henry I and home to an order of canons who ruled over the town.