When upgrading TortoiseSVN, do I have to uninstall the existing version first?

No. You can just install the new version over the old one. The installer will take care of uninstalling the old version first automatically. But you must reboot your computer after the installer finishes! Or at least you have to log off and log on again. If you don't, TortoiseSVN will crash!