Both these are fixable with relative ease, although MS chose to hide it well. :)
In XP: Control Panel -> Regional and Language opt. -> Language -> Details (or if you have the taskbar Language Band, Settings...)
Firstly, choose your Colemak as the Default input language. This will take care of the per-application problem for you. Another alternative is, as mentioned, to delete all non-Colemak layouts in the field below, but if one of them is the current default you'll need to select Colemak as the default anyway (and press Apply) first.
Now, there's a Key Settings... button that'll let you choose which keys switch layouts. It's only active if you have multiple layouts in the middle field. Remove the checks from both Alt- and Ctrl- switching and you should be fine. What's been changing layouts "spontaneously" is that you've pressed Alt and Shift at the same time, most likely.