• You are not logged in.

Keyboard Layout Files Creator

  • Started by 39aldo39
  • 27 Replies:
  • Reputation: 15
  • Registered: 12-Sep-2016
  • Posts: 45

Since creating and updating keyboard layout files can be quite tedious, especially ones with a lot of features, I made a program to create keyboard layout files for (currently) XKB, PKL and KLC. It is meant as a primary way to create keyboard layout files. Advanced features, like extend mappings, custom dead keys and mods, are supported.

The source code and documentation can be found at the GitHub repo. Binaries for Linux and Windows can be found at the releases page.

Create advanced keyboard layouts in various formats using my Keyboard Layout Files Creator!

Offline
  • 2
  • Reputation: 214
  • From: Viken, Norway
  • Registered: 13-Dec-2006
  • Posts: 5,364

Wow, that is hella impressive! Kudos!

What about TMK layouts (from Hasu at GitHub), can those be at least imported? I know he makes some JSON output but it's probably not like your format?

Maybe you should provide more sources. When you say 'Extend' for instance, what are your mappings based on? Mine? Farkas'? Your own flavour? A few links in comments would be good.

I didn't see any dead keys for PKL in your examples. Can you support the more hairy variants there (Unicode that's not always implemented in fonts, for instance "slash"+⊆ triggering ⊈; the ⊆ being generated by "maths"+<)?

Last edited by DreymaR (13-Sep-2016 11:19:50)

*** Learn Colemak in 2–5 steps with Tarmak! ***
*** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

Offline
  • 0
  • Reputation: 117
  • From: UK
  • Registered: 14-Apr-2014
  • Posts: 978

Yes, this looks really good! 

I've long wished there was a single format where you define all your key mapping preferences in a platform-independent way. Since that isn't possible, having a tool that at least generates the possible output files you need is the next best thing. Nice work!  I'm sure I'll end up having a look at it in more detail at some point.

Using Colemak-DH with Seniply.

Offline
  • 0
  • Reputation: 214
  • From: Viken, Norway
  • Registered: 13-Dec-2006
  • Posts: 5,364

I too, have wished for such a format. I think that we may have to define it ourselves.

For my projects, I'm close to accepting the TMK keymap. It's a simple 16×16 matrix so it holds all possible scan codes, yet there are compiler macros that allow conversion of simpler keymaps to the full matrix. The way I've set mine up, they're fairly intuitive in relation to a real keyboard – but not perfect to look at still. All keys have names that are based on the usual scan code names, optionally abbreviated to 3–4 characters to fit in the map matrix.

Below is a KEYMAP_ALL (that's translated as a macro to AWide-ISO; that's handy!) so all sorts of special and Asian++ keys are there. There are simpler map formats without those, but here's the full monty.

#if ACTIVELAYOUT == 0
    /* Plain QWERTY
     *         ,---------------. ,---------------. ,---------------.
     *         |F13|F14|F15|F16| |F17|F18|F19|F20| |F21|F22|F23|F24|
     * ,---.   |---------------| |---------------| |---------------| ,-----------. ,---------------. ,-------.
     * |Esc|   |F1 |F2 |F3 |F4 | |F5 |F6 |F7 |F8 | |F9 |F10|F11|F12| |PrS|ScL|Pau| |VoD|VoU|Mut|Pwr| | Help  |
     * `---'   `---------------' `---------------' `---------------' `-----------' `---------------' `-------'
     * ,-----------------------------------------------------------. ,-----------. ,---------------. ,-------.
     * |  `|  1|  2|  3|  4|  5|  6|  7|  8|  9|  0|  -|  =|JPY|BSp| |Ins|Hom|PgU| |NLk|  /|  *|  -| |Stp|Agn|
     * |-----------------------------------------------------------| |-----------| |---------------| |-------|
     * | Tab |  Q|  W|  E|  R|  T|  Y|  U|  I|  O|  P|  [|  ]|  \  | |Del|End|PgD| |  7|  8|  9|  +| |Mnu|Und|
     * |-----------------------------------------------------------| `-----------' |---------------| |-------|
     * |*Caps*|  A|  S|  D|  F|  G|  H|  J|  K|  L|  ;|  :|  #|Ent |               |  4|  5|  6|KP,| |Sel|Cpy|
     * |-----------------------------------------------------------|     ,---.     |---------------| |-------|
     * | Sft|  <|  Z|  X|  C|  V|  B|  N|  M|  ,|  ,|  /| RO| Shft |     |Up |     |  1|  2|  3|KP=| |Exe|Pst|
     * |-----------------------------------------------------------| ,-----------. |---------------| |-------|
     * |Ctrl|Gui|Alt|MHEN|HNJ| Space |H/E|HENK|KANA|Alt|Gui|App|Ctl| |Lft|Dwn|Rgh| |  0    |  .|Ent| |Fnd|Cut|
     * `-----------------------------------------------------------' `-----------' `---------------' `-------'
     */
    [0] = KEYMAP_AWIDEISO(
              F13 ,F14 ,F15 ,F16 ,F17 ,F18 ,F19 ,F20 ,F21 ,F22 ,F23 ,F24 ,
    ESC ,      F1 , F2 , F3 , F4 , F5 , F6 , F7 , F8 , F9 ,F10 ,F11 ,F12 ,         PSCR,SLCK,PAUS,    VOLD,VOLU,MUTE,PWR ,    HELP,
    GRV ,  1 ,  2 ,  3 ,  4 ,  5 ,  6 ,  7 ,  8 ,  9 ,  0 ,MINS,EQL ,JYEN,BSPC,    INS ,HOME,PGUP,    NLCK,PSLS,PAST,PMNS,    STOP,AGIN,
    FN0   ,  Q ,  W ,  E ,  R ,  T ,  Y ,  U ,  I ,  O ,  P ,LBRC,RBRC,   BSLS,    DEL ,END ,PGDN,     P7 , P8 , P9 ,PPLS,    MENU,UNDO,
    FN1    ,  A ,  S ,  D ,  F ,  G ,  H ,  J ,  K ,  L ,SCLN,QUOT,NUHS,  ENT ,                        P4 , P5 , P6 ,PCMM,    SLCT,COPY,
    LSFT ,NUBS,  Z ,  X ,  C ,  V ,  B ,  N ,  M ,COMM,DOT ,SLSH, RO ,    RSFT,          UP ,          P1 , P2 , P3 ,PEQL,    EXEC,PSTE,
    LCTL ,LGUI,FN6 ,MHEN,HANJ,    SPC     ,HAEN,HENK,KANA,FN7 ,RGUI,APP , RCTL,    LEFT,DOWN,RGHT,     P0      ,PDOT,PENT,    FIND,CUT 
    ),    /* <-- Layer 0: Default Layer */

*** Learn Colemak in 2–5 steps with Tarmak! ***
*** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

Offline
  • 0
  • Reputation: 15
  • Registered: 12-Sep-2016
  • Posts: 45

Thanks!

To be able to export to all formats, I needed a format to represent a keyboard layout more general than the output formats, because all formats have a different set of features. That is the JSON format documented in doc/layout.md, which I designed myself. So, now there is such platform-independent format :-).

I will look into the TMK layouts and try to make exports for it.

The examples colemak.json and altgr.json are from the Colemak layout defined in XKB and the extend.json is from your extend mappings. But they are just examples and meant to be edited to fit personal taste.

There are predefined dead keys, with predefined character tables. But it is also possible to define your custom dead keys and use those, with all the hairy variants you want. You can import them from PKL to save work.

Create advanced keyboard layouts in various formats using my Keyboard Layout Files Creator!

Offline
  • 0
  • Reputation: 21
  • From: Chicago
  • Registered: 27-Apr-2016
  • Posts: 221

It would be good to have some examples of usage of the program. I'm talking about the Windows executable, I could not make it to work.

Offline
  • 0
  • Reputation: 15
  • Registered: 12-Sep-2016
  • Posts: 45

Thanks for feedback!
I use Linux mainly, so I have tested less on Windows. But it looks like my program assumes UTF-8, although it should work without UTF-8...
I added a note for Windows and a usage example. It should work now.

Create advanced keyboard layouts in various formats using my Keyboard Layout Files Creator!

Offline
  • 0
  • Reputation: 0
  • Registered: 13-Jan-2017
  • Posts: 3

Anyone know of a typing tutor that allows custom keyboard layouts? I am just learning to touch type and I would like to try a couple of the Colemak mods and/ or add my own changes as to which physical keys I want to press with which fingers.

Offline
  • 0
  • Reputation: 0
  • Registered: 13-Jan-2017
  • Posts: 3

Thanks Davkol - KTouch looks like exactly what I need, but any one know of one that works on Windows?

Offline
  • 0
  • Reputation: 214
  • From: Viken, Norway
  • Registered: 13-Dec-2006
  • Posts: 5,364

Do you really need one though? With PKL you can have any help image showing your layout at the top or bottom of your screen, if you don't want to use a printed help image. And using Tarmak, there's so few keys to learn at a time that you may rely a lot less on help images altogether.

I've not used any Colemak-specific tutors showing the layout. Didn't miss it.

*** Learn Colemak in 2–5 steps with Tarmak! ***
*** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

Offline
  • 0
  • Reputation: 0
  • Registered: 11-Oct-2013
  • Posts: 79

I really like this format for specifying keymaps. It's a little more verbose than some, but it's a lot more flexible than the ones I've used. I really like that shiftstates is customizable and can be overriden locally. The fact that entries can be combined is great as well. The permutation syntax is also really convenient. Today I finally switched to using xkb because of klfc; thanks a lot!

Support for QMK and TMK would definitely be awesome too, especially if their more complex features were supported as well.

Offline
  • 0
  • Reputation: 15
  • Registered: 12-Sep-2016
  • Posts: 45

Glad you like it! I indeed designed the format to be flexible without being too verbose. It could be a bit less verbose, but I wanted to use an existing format (JSON). YAML support, which is a superset of JSON, could be added to remove most quotes around strings. But I don't think its really needed.

I am working on support for TMK, basic support should probably be ready in a few weeks.

Create advanced keyboard layouts in various formats using my Keyboard Layout Files Creator!

Offline
  • 0
  • Reputation: 15
  • Registered: 12-Sep-2016
  • Posts: 45

I have added a branch tmk with TMK support. The generated output does compile, but I don't have the hardware to test whether it also works as expected on a real keyboard. Could someone test it? The shiftstates are the trickiest part. Since they are very general, I have implemented a custom function in the output to switch between layers.

Create advanced keyboard layouts in various formats using my Keyboard Layout Files Creator!

Offline
  • 0
  • Reputation: 15
  • Registered: 12-Sep-2016
  • Posts: 45

Since I experienced some quirks in PKL, and PKL has some missing features, I have added support for exporting directly to AutoHotkey scripts. The implementation doesn't have help images, but it has some other features:
* No stuck modifiers
* Multiple extend layers: combinations like Shift+Extend, as well as multiple extend keys
* (Multiple) shifted, latched or locked modifiers on a key
* Chained dead keys, which enable a compose key
* It still supports nested dead keys, so 'slash'+⊆ triggers ⊈, with ⊆ generated by 'math'+<

Create advanced keyboard layouts in various formats using my Keyboard Layout Files Creator!

Offline
  • 0
  • Reputation: 214
  • From: Viken, Norway
  • Registered: 13-Dec-2006
  • Posts: 5,364

What are you saying, have you made your own PKL now in effect?

Could it be possible to port some more of your improvements into PKL? I'd like to build on the nice features of that program such as help images, while it'd be great to be rid of stuck modifiers and the like. And multiple Extend layers has been on my to-do list for ages!

Last edited by DreymaR (08-May-2017 10:46:16)

*** Learn Colemak in 2–5 steps with Tarmak! ***
*** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

Offline
  • 0
  • Reputation: 15
  • Registered: 12-Sep-2016
  • Posts: 45

You could indeed say that it is like PKL. I may port some improvements to PKL, although the source code doesn't look very flexible. Removing stuck modifiers, adding chained dead keys and adding multiple extend layers should be possible though.

As a start, I tried to compile the PKL source code, but it doesn't work :(. If I compile pkl.ahk and execute pkl.exe, I get the error "You must set the layout file in pkl.ini!". And if I directly run the script, I even get an error inside the script...

Create advanced keyboard layouts in various formats using my Keyboard Layout Files Creator!

Offline
  • 0
  • Reputation: 214
  • From: Viken, Norway
  • Registered: 13-Dec-2006
  • Posts: 5,364

Yes, the pkl.exe needs a pkl.ini file and a layout file. So you should put it in a suitable file environment.

I'm trying to sort through Vortex's PKL mod. He did some good work with PKL but unfortunately forgot to track his changes or comment the code he reworked. So it's a bit of a puzzle.

Last edited by DreymaR (08-May-2017 16:28:38)

*** Learn Colemak in 2–5 steps with Tarmak! ***
*** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

Offline
  • 0
  • Reputation: 15
  • Registered: 12-Sep-2016
  • Posts: 45

The problem is that I have put a valid pkl.ini file in the right place... If I remove it, PKL (correctly) says that pkl.ini is missing, which is a different error.

Create advanced keyboard layouts in various formats using my Keyboard Layout Files Creator!

Offline
  • 0
  • Reputation: 214
  • From: Viken, Norway
  • Registered: 13-Dec-2006
  • Posts: 5,364

Valid, as in defining a layout location that actually holds a valid layout?

You may also set the compact mode in pkl.ini, and plop a layout in the same directory as pkl.

Last edited by DreymaR (09-May-2017 10:36:13)

*** Learn Colemak in 2–5 steps with Tarmak! ***
*** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

Offline
  • 0
  • Reputation: 15
  • Registered: 12-Sep-2016
  • Posts: 45

The problems really occur due to the compilation: if I replace my self-compiled pkl.exe by a precompiled one, everything works fine. I should have been more clear about that.

Create advanced keyboard layouts in various formats using my Keyboard Layout Files Creator!

Offline
  • 0
  • Reputation: 214
  • From: Viken, Norway
  • Registered: 13-Dec-2006
  • Posts: 5,364

It's odd though: I've compiled the PKL v0.3.85 code previously and it worked fine then. However, it's been a while since I did it so I don't remember much of the details.


As mentioned, I'm trying to work off vVv's code now, as it has improvements and supports Unicode AHK. But it's a bit slow going trying to figure out what he's done since he didn't document his changes (and he worked off a decompiled PKL v0.4 because Farkas Máté never released his code for that version). :-(

Last edited by DreymaR (09-May-2017 10:38:28)

*** Learn Colemak in 2–5 steps with Tarmak! ***
*** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

Offline
  • 0
  • Reputation: 15
  • Registered: 12-Sep-2016
  • Posts: 45

I would like to get vVv's code, but his link in this post redirects me to the homepage of filedropper :(. Do you still have it, or do we have to ask vVv? Also, there are nice tools to view the difference between files, so it should be doable to figure out what he's done.

Create advanced keyboard layouts in various formats using my Keyboard Layout Files Creator!

Offline
  • 0
  • Reputation: 214
  • From: Viken, Norway
  • Registered: 13-Dec-2006
  • Posts: 5,364

Yes, I have his code. And I'm using WinMerge or Meld to diff the codes, but since I don't have a proper vanilla PKL v0.4 source (vVv having decompiled a v0.4 pkl.exe and not kept track of his changes thereafter!) it's still a bit of a puzzle.

*** Learn Colemak in 2–5 steps with Tarmak! ***
*** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

Offline
  • 0
  • Reputation: 15
  • Registered: 12-Sep-2016
  • Posts: 45

So... Will you also share his code?

Create advanced keyboard layouts in various formats using my Keyboard Layout Files Creator!

Offline
  • 0
  • Reputation: 214
  • From: Viken, Norway
  • Registered: 13-Dec-2006
  • Posts: 5,364

Yes, that's my plan! But it's a mess right now. Sorry.

I'm quite busy at the moment. For one, the unification of the DH-mod has taken much of my spare time.

Last edited by DreymaR (11-May-2017 12:29:52)

*** Learn Colemak in 2–5 steps with Tarmak! ***
*** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

Offline
  • 0