There's no spyware/adware/malware in the Windows installer. The provided uninstall.exe uninstalls everything that is installed during the installation. Anyway, since it takes only about 20K on the drive, and doesn't use any memory, you don't really need to bother uninstalling it, just switch back to your previous layout in the control panel.
Windows is the only operating system that requires running executable code just to install a keyboard layout, which is a very poor design from a security point of view.
The Windows installer doesn't do much more than run Colemak.msi and guides through the installation in the Control Panel, but you can run the Colemak.msi directly.
Colemak.msi is a file that is generated by the Microsoft Keyboard Layout Creator, and the only thing it does is install the i386\Colemak.dll file.