Excalibur was kept nearby with the Stone of Destiny (Scone), which the last time I saw it many years ago was kept at the old St Columba's Parish Church (long since demolished and now a nursing home) in Dundee. The stone they dug up at Arbroath Abbey after it was repatriated was a fake at was apparently returned to Westminster Cathedral before being put on display at Edinburgh Castle.

http://en.wikipedia.org/wiki/Stone_of_Scone