Oh dear. Yes, that mix you got there is a QWERTY layout with the CurlAngleWide mod applied to it. Obviously, the Curl mod doesn't make good sense on QWERTY...

On the other hand, in your applications it seems you've got it the other way around: The Colemak-DH layout is active but not the keyboard model component (pc104aw-zqu).

Your setup looks about right. If you run the setxkb.sh script with the "4caw us us" shortstring, what's the result? And have you put the resulting setxkbmap command somewhere that's sourced at your startup/logon, like at the end of a ~/.bashrc file (I suppose Ubuntu still sources bashrc?)?

My best guess is you may have selected something by setxkbmap command and something by GUI. Those two methods don't always play nicely together. The thing with GUI settings is that Ubuntu stopped letting you set your keyboard model (which is where I put the CAW mod) by GUI; I don't know how that stands today actually.

Unfortunately, I'm not using Ubuntu right now apart from the Windows Sublayer for Linux which doesn't use the X server to my knowledge. So I'm short on testing possibilities as it stands. I'm looking at installing WSL 2 soon, but so far I haven't.

I'm hoping we can figure this one out together!  (b￣◇￣)b

All setxkb.sh really does is produce a setxkbmap command, so you could just source that directly instead of running my script at each startup. Then you wouldn't have to worry about displacing the script or anything.

Running setxkb.sh :

Welcome to fish, the friendly interactive shell
Type `help` for instructions on how to use fish
qwa@dates ~/s/g/BigBagKbdTrixXKB (master)> chmod +x *.sh
qwa@dates ~/s/g/BigBagKbdTrixXKB (master)> 
sudo bash ./setxkb.sh -a "4caw us us"
[sudo] password for qwa: 

@@@ DreymaR's setxkbmap script (by GadOE, 2020-12) @@@
¤ Using model/layout 'pc104aw-zqu'/'us(cmk_ed_us)' from ShortStr
¤ Changed dir to '/usr/share/X11/xkb'
¤ Looking for and removing any old .xkm server files
¤ Running setxkbmap:

Setting verbose level to 9
locale is C
Warning! Multiple definitions of keyboard model
         Using command line, ignoring X server
Warning! Multiple definitions of keyboard layout
         Using command line, ignoring X server
Trying to load rules file ./rules/evdev...
Success.
Applied rules from evdev:
rules:      evdev
model:      pc104aw-zqu
layout:     us(cmk_ed_us)
options:    misc:extend,lv5:caps_switch_lock,grp:shifts_toggle,compose:menu,misc:cmk_curl_dh
Trying to build keymap using the following components:
keycodes:   evdev(pc104aw-zqu)+aliases(qwerty)
types:      complete
compat:     complete
symbols:    pc+us(cmk_ed_us)+inet(evdev)+group(shifts_toggle)+compose(menu)+level5(caps_switch_lock)+extend(basic)+colemak(cmk_ed_dh)
geometry:   pc(pc104aw-zqu)

¤ Adding setxkbmap cmd to /home/qwa/.bashrc

¤ Changed dir to '/home/qwa/src/git-clones/BigBagKbdTrixXKB'

@@@ setxkb.sh finished! @@@

Command written to .bashrc too. Same behaviour.

##-> DreymaR's SetXKB.sh: Activate layout
setxkbmap -model 'pc104aw-zqu' -layout 'us(cmk_ed_us)' -option 'misc:extend,lv5:caps_switch_lock,grp:shifts_toggle,compose:menu,misc:cmk_curl_dh'
##<- DreymaR's SetXKB.sh

Ran this command manually too. No change in behaviour.

xev on caps lock:

KeyPress event, serial 34, synthetic NO, window 0x1600001,
    root 0x29f, subw 0x0, time 806292, (216,761), root:(254,863),
    state 0x10, keycode 66 (keysym 0xfe11, ISO_Level5_Shift), same_screen YES,
    XLookupString gives 0 bytes: 
    XmbLookupString gives 0 bytes: 
    XFilterEvent returns: False

KeyRelease event, serial 37, synthetic NO, window 0x1600001,
    root 0x29f, subw 0x0, time 806740, (216,761), root:(254,863),
    state 0x12, keycode 66 (keysym 0xfe11, ISO_Level5_Shift), same_screen YES,
    XLookupString gives 0 bytes: 
    XFilterEvent returns: False

It does say lvl5 shift and not caps lock. Isn't that what triggers the Extend layer?

Extend layer is still not enabled by caps lock. Thinking there is a good chance this is issue is specific to my laptop model or just mine.

You were right. The problem is mostly fixed and for some odd reason using Wayland seems to be the problem for me.

I tried fixing the keyboard model in file `/etc/default/keyboard` with the following:

XKBMODEL=pc104aw-zqu
XKBLAYOUT=us
XKBVARIANT=cmk_ed_dh
XKBOPTIONS=misc:extend,lv5:caps_switch_lock,grp:shifts_toggle,compose:menu,misc:cmk_curl_dh
BACKSPACE=guess

This kept not working with multiple formats. Finally switched to X server at login and now it works with zero issues. This inspite of many forums saying xkb is compatible with Wayland. Might be this file is not used in latest versions? But this was the only way I found to force a keyboard model though isn't applicable in Wayland.

Like you said, I enabled two options relating to lvl5 extend on caps using Gnome Tweak tool and Extend works without any flaw.

Thank you so much for the help. Mikka Nandri :D

KMonad sounds nice. Can't wait to see how that turns out.
I just tried xmonad where the gnome GUI settings aren't available but still the keyboard setup works perfect. So just running the setxkb.sh must have been enough if not for Wayland.

WSL and your EPKL is what allows me to stay in Win10 for long and gives me a consistent platform when I can't use linux. So thanks for that.

Glad to hear more functionalities ore being added to it. I personally have never used Compose on either system. Its main use I know is typing special characters? But I have stuck with plain English so far.

Still I am excited to see this progress.

Wow.. autocomplete sounds cool.

The other sounds like running commands as in mod+key ? Might help launch frequently used programs if that is how it works, especially on win where I have to either search or have on task icons and use numbers.