The castle is believed to be the work of Gundulph, Bishop of Rochester. Gundulph was responsible for the design of the White Tower (the central Keep of the Tower of London) which is the only building in Britain with the same ground plan and the castle would once have looked similar to the tower.