You should download a Release version, as it has fewer files and no compiler etc included. If you want the newest commit, just have a look at which files the Release version keeps.

You can safely remove any layout folders that your layout doesn't need. Some layouts point to other layouts' BaseLayout, so make sure you don't nuke anything you actually need. You can for instance rename or zip down the ones you plan to delete first, to ensure that you don't lose them. Since you're using a modified Colemak, you'll probably just need to keep the Colemak BaseLayout(s). Any other variants, locale and otherwise, that you don't use can be removed.

The most compact way of running EPKL should be to keep your layout (with or without help images) in the same folder as EPKL.exe, and set CompactMode to yes. I haven't tested that in a while though, so let me know if you run into trouble with it. You probably have to change the BaseLayout location pointer in Layout.ini, at the least.