The recommended way to press Z-X-C-V-B is with the pinky-ring-middle-index-index, that way the hand is oriented so that hitting B isn't that difficult. Some keyboards such as the Kinesis contoured keyboard and the Datadesk SmartBoard keyboard align the keys in a more natural way, to avoid the long stretches of the 'B' position.

There is some small benefit exchanging the 'B' (1.3%) and 'V' (0.9%) to reduce the frequency of hitting the 'B' position. However, I don't think that 0.4% difference is going to be very notable. Since the change would be rather confusing, and it will break comptability of the shortcut Ctrl+V and Ctrl+B, I really don't think it's worth it. Moreover, the combos AB/BA (stretch for both directions) are slightly less common than AV (in the word 'have').

There is also some very small benefit exchanging 'C' and 'F', if you really don't like hitting the 'C' position, you can exchange 'C' and 'F'. I have left it there for compatibility reasons, but the improvement is very minor.

I highly recommend leaving 'P' and 'G' in their places.  The 'pt'/'tp' combo is quite common (e.g. ftp, http, apt), and 'G' is usually followed by a letter on the other hand (e.g. 'gh'), so it's better to keep it there to avoid the hand stretching in two different directions.

Even though you say you don't really care about compatibility, it's still a QWERTY world, and the similarity to QWERTY does help in the occasions that you are forced to use a QWERTY keyboard. Moreover, if you'll use a non-standard Colemak variant, you'll also have to adjust all the additional material, such as typing lessons, etc.