When Prince Edward, Duke of Kent and Strathearn, fourth son of King George III and the father of Queen Victoria was appointed Governor of Gibraltar by the War Office, he took up residence in this property as the Convent was still undergoing extensive repairs after the damage suffered during the Great Siege and was only being used for entertainment purposes. The Duke had taken up his post on the 24th May 1802 with express orders from the government to restore discipline among the drunken troops. The Duke's harsh discipline precipitated a mutiny by soldiers on Christmas Eve 1802. The Duke of York, then Commander-in-Chief of the Forces, recalled him in May 1803 after receiving reports of the mutiny, but despite this direct order he refused to return to England until his successor had arrived. He was refused permission to return to Gibraltar for an inquiry and, although allowed to continue to hold the governorship of Gibraltar until his death, he was forbidden to return.
After the Duke’s departure the building was converted into an Officers’ Mess until 1833 when it was destroyed by a fire.