This is a bit of a crosspost-ish from Reddit but I think this topic deserves more attention so might as well make a post here. Link to Reddit Post.

To change the keymappings to Colemak while using an IME like a Japanese IME, or a Pinyin IME, mostly involves registry editing, and there's more info on that in this Reddit Post. I have been a little hesitant to go that route, so I found a (whacky) alternative. Unfortunately it only works for Google Japanse IME and you will have to install it on your device to make it work (I have thankfully spared from figuring out something similar for Chinese since I don't use Pinyin anyways):-

1. First Google Japanese IME here Google 日本語入力 – Google.
2. After installing and setting it up, click on the wrench icon in the language bar to edit properties.
3. Click on the customize option for the Romaji Table and import the .txt file from Colemak romaji table mapping for Google Japanese Input - User contributions - Colemak forum.

This method has some drawbacks:-

1. It edits the Romaji Table, not the keymapping, so typing <DL> (QWERTY) will output a <DL> only and not <SI>; only after typing (QWERTY)<DL>+<space> will the output change to し. So far it has been a non issue, other than the fact that you have to train your brain to ignore the output of the keypresses that appear before pressing the spacebar.
2. The Latin layout which you can toggle using <ALT>+<`> still is Qwerty, since this hack doesn't actually remap the keys.

In case someone figures out a better way, please put them down as I am still searching for more permanent solutions and editing the Romaji table in Google Japanese IME is only a tentative fix. I was suggested to perhaps implement AHK to make it possible but my experience with AHK is pretty low (zero) so I am not entirely sure how would one go about it.