The Great Keep, originally the Gate House, was the work of Richard "Strongbow" de Clare in about 1170, the man who also gave Usk Town its first Charter and who founded Usk Priory.

Before that, there may have been a wooden motte and bailey castle, for William 1's standard bearer at Hastings. The keep was briefly in the hands of Hywel, Welsh prince of Caerleon, who was killed by Henry 11's forces during its recapture.

It is much altered but retains a fine pair of Noman windows.