Should one learn colemak for prose first, and once the speed and accuracy is good, move to programming (with all the fancy symbols)?

I'm using colemak on a new kinesis contour, and programming is hell, even remapping all arrow keys to the right. I want to be mildly productive while programming... right now I cannot think :). But i'm afraid that if I stop the cold-turkey, I'll never learn the layout properly.

What do you think?