NOTE: Prior to this, ensure you setup a Firewall Rule for either your IP address, and/or the new port to ensure you don't lose access!
Start Registry Editor from the Run Prompt (windows key + R)
Locate and then click the following registry subkey:
On the Edit menu, click Modify, and then click Decimal.
Type the new port number, and then click OK.
Quit Registry Editor
Open the Services Controller (services.msc), using the run prompt.
Restart "Remote Desktop Services"
Then you can log back into the server, with the new RDP Port!