Well until a couple of years ago I was hunting and pecking away with qwerty at more or less terrible speeds, so I decided to learn touch typing and I taught myself no-dvorak (norwegian dvorak) and I was up to a decent speed (75 - 80 WPM Norwegian 60-65 English and 35-40 japanese) but my pinkies didn't love me at all. And I really love the focus on rolls in colemak. After 3 days of practice (one one the homerow) I'm now typing at 19 WPM with 94% accuracy according to keybr.com , So that isn't too bad, I don't use a mouse, so I get quite a lot of practice (puppy linux + ratpoison), but I hop over to QWERTY hunt and peck when I need something to be written faster, thinking that it would be easier than jumping over to no-dvorak again.
The only problem I have now is that my accuracy drastically sinks when I'm not writing in keybr, and my frustration levels shoot up, but I think I just need to give it some time, only some days of practice this far..