For me, Colemak was much easier to learn then Dvorak. That's because most of the keys in Dvorak are changed from QWERTY, while in Colemak only the necessary and most used are changed, and best of all, the famous shortcuts are still in their original locations.

When I was trying to learn Dvorak, my speed was around 20 wpm for 2 weeks and and my accuracy was horrible. Thus, I gave up and went back to QWERTY. However, I didn't want to use it anymore, so I started searching for alternatives. Days later, I found Colemak and within 3 days of practicing I was able to get above 30 wpm with accuracy of above 90% since I didn't have to relearn 1/2 the keys, only the ones that I use mostly. I switched cold turkey - so I use QWERTY only when no other choice is available, which is very rare since I carry my flash drive all the time, with Colemak on it.  Basically, it's easier to learn keys that one has more practice in, thus all those not very popular keys like z, x, v, etc I didn't have to relearn. However, since I didn't technically master Dvorak, the switch was easier for me, since Colemak is closer to QWERTY then it is to Dvorak, but in any case, if you do some practicing, you'll get back your speed and more in no time. In the worst case, if you don't decide to switch, at least use the reg file in the Colemak package to remap the caps to backspace, very useful in my opinion :).