Following the xorg.conf directions at https://colemak.com/Unix will cause Caps Lock only to Backspace the last character when held down, rather than erasing characters until the key is lifted. This problem does not occur if one does setxkbmap -v colemak && xset r 66.
While I haven't tested, it's possible that this occurs in other Linux distributions as well.