• You are not logged in.

    Measuring difficulty of learning new layouts

    • Started by stevep99
    • 23 Replies:
    • Reputation: 117
    • From: UK
    • Registered: 14-Apr-2014
    • Posts: 978

    I have been thinking a bit lately of how best to quantify how easy or difficult a new layout is to learn. The most commonly cited measure is usually just a count of how many keys are relocated from the base layout (i.e. Qwerty). I think it would be useful to take into account how common the key is, in addition to whether it is moved, and whether it stays on the same finger, etc.

    To this end, I have attempted to create a "difficulty index" for keyboard layouts. If you would like to read about it, I have added a page on my github repo about it:

    https://colemakmods.github.io/mod-dh/learn.html

    Interested in thoughts on the reasonableness of my model.

    Last edited by stevep99 (01-Dec-2014 17:43:29)

    Using Colemak-DH with Seniply.

    Offline
    • 0
    • Reputation: 117
    • From: UK
    • Registered: 14-Apr-2014
    • Posts: 978
    davkol said:

    That's relevant only to experienced typists in QWERTY, that consider *transition* to a different layout.

    Oh - just to clarify, transitioning from Qwerty is exactly what I was talking about here. Trying to assess the difficulty in learning a layout from scratch (as a first layout) would be a completely different (but still interesting) kettle of fish.

    Last edited by stevep99 (01-Dec-2014 10:32:32)

    Using Colemak-DH with Seniply.

    Offline
    • 0
    • Reputation: 214
    • From: Viken, Norway
    • Registered: 13-Dec-2006
    • Posts: 5,360

    Interesting thoughts!

    The rarity of J is what inspired the current Tarmak transition schemes: Since it's so rare it won't be too disruptive that it flies around.

    However, moving a key (permanently) is a two-edged sword: How much it'll disrupt and how well it'll go back. Moving the E will shock you for sure, but since it's so common won't you relearn it quickly too? On the other hand, the symbol keys hardly seem to disrupt much at all (in the Wide mod for instance) since they're so rare and not letters so for the most part it's okay to look at the board while typing them!

    The shock and awe of moving a key seems to me about the same immediately even for rarely used keys, as the layout will feel estranged – we had a fun discussion about "layout gestalt" a way back. Of course you'll stumble across the change more frequently with a common letter, but all in all the keyboard will feel strange and that's the more important thing I believe. Then if it's only an Angle mod or a similar minor adjustment, safety will be reestablished quickly (it took me a couple of days to feel perfectly safe again with the Angle mod, for instance). More dramatic changes would be moving a key to the neighboring finger. Or maybe moving it to the other hand would be worse? Many questions, and I don't have the answers. One should do actual experiments measuring key-to-key flow times (in Amphetype or similar) versus different types of changes, and how quickly the flow is reestablished.

    Maybe it's the "middle ground" keys that represent the biggest problem? i don't know.

    *** Learn Colemak in 2–5 steps with Tarmak! ***
    *** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

    Offline
    • 0
    • Reputation: 117
    • From: UK
    • Registered: 14-Apr-2014
    • Posts: 978

    I had wondered about the relative effort in relearning frequent vs infrequent keys. It may be true to some extent that more frequent keys can be learned quicker due to them being more commonly typed. But during that learning period there would involve many more errors and no doubt frustration too. Also, the adverse impact on typing ability will more severely affected for the transitional period. After all, it's not just the key itself, as muscle memory often seems stored as bigrams/trigrams. Just think of all the relearned bigrams and trigrams that have to relearned for E!

    Taking everything into account, I think more common letters are harder overall to relearn for these reasons. But perhaps the relationship is not linear with frequency as my formula assumes, but rather some other more complicated relationship. Oh well, at least it gives a quick and easy way to get an overall measure of difference/learning difficulty though, which might be of some use.

    Using Colemak-DH with Seniply.

    Offline
    • 0
    • Reputation: 214
    • From: Viken, Norway
    • Registered: 13-Dec-2006
    • Posts: 5,360

    Yes. It really depends on how the brain learns best, doesn't it? Overtraining isn't so efficient, but I'm not sure how exactly that'd apply to learning keyboard layouts, especially on a key-by-key basis!

    Maybe the E key will be learnt less efficiently per time unit compared to a less frequent key, because it'll be encountered more often than what optimal learning would dictate?

    Last edited by DreymaR (02-Dec-2014 09:31:55)

    *** Learn Colemak in 2–5 steps with Tarmak! ***
    *** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

    Offline
    • 0
    • Reputation: 2
    • Registered: 25-Oct-2013
    • Posts: 136

    A nice web app! About the idea of 'learnability of layouts' I do have some hestitations though:

    1 - sometimes small differences can be more confusing than large differences

    2 - IF small differences are easier to learn than large ones, you would be right for present "qwerty touch typists". A small step (minimak, colemak etc.) would be easier to them than a large step (dvorak, carpalx, mtgap, adnw). But....many people (including me) cannot touch type in Qwerty. Many know roughly where the keys are, but it's not proper touch type. This means learning a new layout involves learning touch type *from scratch*. In that case, why not choose an optimal layout while you're at it? Such an optimal layout could be aiming for "rolls", of for "alternation" or "best balance over fingers & hands"  etc. But "looking like qwerty" does not have to be a goal, however.

    Offline
    • 0
    • Reputation: 214
    • From: Viken, Norway
    • Registered: 13-Dec-2006
    • Posts: 5,360

    *No one* can say they don't know QWERTY, unless they're from Outer Mongolia or thereabouts! How *well* you know it varies greatly, of course. But I'm of the strong opinion that even people who claim to have no knowledge of the QWERTY layout whatsoever are still influenced by it. ;)

    To the contrary, most of us have to actually *use* QWERTY every so often. This means that similarity to QWERTY not only is a goal when learning a new layout, but keeps on being important afterwards.

    Some claim that it's easier to remember a new layout when it's completely different and not just a little different. I just don't buy it. At all. I mean, I speak Norwegian and Swedish and even Danish quite decently with ease because they're so similar and I have a long long way to go before my Greek becomes anywhere near that good!

    Last edited by DreymaR (05-Dec-2014 16:05:29)

    *** Learn Colemak in 2–5 steps with Tarmak! ***
    *** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

    Offline
    • 0
    • Reputation: 2
    • Registered: 25-Oct-2013
    • Posts: 136

    @Dreymar "I speak Norwegian and Swedish and even Danish quite decently with ease because they're so similar"  Wasn't Swedish sort of "Norwegian pronounced in a funny way"? Or was is the other way around ?  ;-) ;-)  Just joking...

    I understand the analogy. But yet, on the other hand: suppose you play upright bass and piano. You would never confuse them. But, switch from the upright bass to a cello, and you will get confused: that cello looks like a smaller bass, but it tuned differently (in 5ths rather than 4ths), meaning that the fingering is completely different. This could mean that maintaining Qwerty knowledge is easier for a Dvorak typist that for a Colemak typist.

    I am not sure how this works in practice: is "qwerty-like" more of an advantage (like your Norwegian-Swedish example) or more of a hindrance (like the bass-cello example) ? The best way to find out would be to compare the qwerty-typing of trained Colemak typists to that of a trained Dvorak typists, I guess ?

    Offline
    • 0
    • Reputation: 117
    • From: UK
    • Registered: 14-Apr-2014
    • Posts: 978

    I don't see the issue of maintaining the ability to type in Qwerty as very significant. Maybe it's easier to be proficient in two layouts if they are totally different, or maybe it makes no difference, who knows?  For me though, if there had been a need to continue to type in Qwerty regularly, I never would have learned an alternative layout. In fact, this was the main issue holding me back for many years before I took the plunge. Nowadays I almost exclusively use my own hardware, so it's not an issue. I wouldn't fancy having to simultaneously keep up my Qwerty skills, regardless of the new layout chosen.

    However, the easyness of learning the layout in the first place (from existing Qwerty) is much more significant in my mind. It's going to be an easier sell if the transition is (relatively) shorter and less painful. Pieter's point that the learning difficulty may be very different depending on whether or not the Qwerty user is already a touch-typist is a good one. I think this would change the model I used somewhat, as non-touch-typists are not necessarily using the correct finger to begin with. For those users, I guess just changing the position of a key in any way at all is going to be a bigger deal.

    Last edited by stevep99 (06-Dec-2014 16:15:50)

    Using Colemak-DH with Seniply.

    Offline
    • 0
    • Reputation: 214
    • From: Viken, Norway
    • Registered: 13-Dec-2006
    • Posts: 5,360

    Have you actually tried swithing between contrabass and cello? I never had a problem between my viola and a bass, and it certainly didn't feel any more confusing than going between the flute and the piano. At all. Bad analogy, or is it that I'm just dabbling with one of them anyway? But interesting observations about jiu jitsu schools! I haven't tried any of that.

    Language fluency doesn't quite enter this analogy as I see it? I can no more speak perfect Danish (which is hard!) than I can speak perfect Japanese or Greek – but I'd have a much much easier time and get a hell of a lot closer a hell of a lot faster trying it with Danish (believe me, I've tried!)! What's the point there? And no, absolutely no problem switching between my perfect Norwegian and my decent Swedish and my half-decent Danish on the fly as necessary. No confusion.

    All languages are pronounced in a funny way. Naturally, this includes the Nordic ones. Actually, Dutch is "only weird German", right? ;-)

    People have very different feelings about ease of learning. I've seen people actually trying to hold ease of learning against Colemak, as if if it's easier to learn something must be wrong with it mustn't it? To others, it seems that Tarmak is necessary to make them even bother to switch. Besides warming my heart, it shows that people have widely different opinions on this. I no longer experience the ease of learning personally, but still feel that similarity to QWERTY helps. Particularly the shortcuts, but keys in general too.

    I see that there may be no general answer to how this works for everybody. Maybe it varies a lot between people? Maybe it's hard to imagine how this works. I don't know.

    Last edited by DreymaR (08-Dec-2014 11:31:08)

    *** Learn Colemak in 2–5 steps with Tarmak! ***
    *** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

    Offline
    • 0
    • Reputation: 2
    • Registered: 25-Oct-2013
    • Posts: 136

    @Dreymar:  I play bass, and tried the "cello tuning" on bass, for a while - utterly confusing! Maybe it IS a bad analogy. And yes, Dutch is like a weird German ;-)   Like you, I don't know it either what is easier to learn.

    By the way, I did find Colemak about as easy to learn as my own mtgap-variant. My own adnw-variant is harder too learn.

    I started with this one. Mtgap for Dutch & English.
    . u o p y x c l b v
    a i e n h m d r t s
    : , ? k q f g w j z
    Very un-qwerty-like. But, not hard to learn because:
    - short distances: much used letters are in the best spots; rare letters are further away - this feels logical.
    - it has lost of rolls. And I feel that rolls are easy to remember, easy to learn.

    But after some time I felt that it was too left-handed, and the rolls started to feel like ' clusters'. Finally I decided to make the switch to Aus Der Neo Welt (adnw). I came up with this good Dutch/English version (* to be filled in later).
    y . u , * w c l h q z
    r i e a o g d t n s k
    x * * j * v p m b f
    This one is better - is found I prefer alternation over rolls. But it is harder to learn.  So here is my "theory", I am thinking out loud here.

    A new keyboard layout is easier to learn when:
    - keys are in logical spots: much used keys close by, rare keys on the harder to reach spots
    - one-hand-rolls are easier to remember than patterns over two hands.
    - unidirectional motions are easier to remember than to-and-fro motions (like " strat" in Colemak)
    - resemblance to Qwerty may be helpful 

    Colemak, may herefore be one of the easiest to learn. I found MTGAP not hard to learn either, though.

    Offline
    • 0
    • Reputation: 0
    • Registered: 05-Apr-2011
    • Posts: 6

    This is really great!

    I would propose renaming "The Difficulty Index" to "QWERTY Switching Difficulty" (QSD), as QWERTY should never score "0.0" on a "difficulty" scale :)

    Two things that I'd like to see tested:

    - Difficulty of switching to the Same Finger and Same Row, but Different Hand.
    - Same Finger, Different hand may be easier than Same Hand, Different Finger.

    I'll be tying QSD into two upcoming keyboard layout projects of mine, so more soon...

    Offline
    • 0
    • Reputation: 117
    • From: UK
    • Registered: 14-Apr-2014
    • Posts: 978

    You're right, it really is the difficulty switching from Qwerty. Although, you could apply the same algorithm to calculate the difficulty in switching between any arbitrary layouts - in practice though switching from Qwerty is the main question of interest.

    Mind, there is some controversy about this index as the relationship between letter frequency and actual switching difficulty is far from clear. And as you say there is also the question of what the weightings should be for Same Finger / Different Finger / Different Hand. 

    I consider this to be only the first stab at trying to quantify switching difficulty. I am sure the model could be improved if we had actual test results to base it on.

    Using Colemak-DH with Seniply.

    Offline
    • 0
    • Reputation: 117
    • From: UK
    • Registered: 14-Apr-2014
    • Posts: 978

    Thinking of how to improve the learning difficulty model:

    I would like to also take into account the fact that changes seem easier to learn when the character moves to an easier position on the keyboard. Certainly it seems to be the case from my experience, but it also makes sense to me that a change that makes things more comfortable will bed in more easily that a change that makes a character more difficult.

    To reflect this in the model, I am thinking about including an additional factor which takes into account the difference in scores (as defined here) between the old an new location...

    s = 1 + ( [key effort in new position] - [key effort in old position] ] / [key effort in old position] )

    Thus, in the Mod-DH change for example, the G (which moves to an easier position) gets a factor of 0.85, whereas M (which moves to a harder position) gets a factor of 1.7.

    Of course, there is still the question of letter frequency. I still think it should be involved in the calculation in some way, as taking the extreme case, moving J or Z must be less work overall than moving E or T. I am toying with the idea of making it a square root factor rather than linear. But I can't think of a logical justification for that other than just wanting to reduce its importance in the overall scheme.

    More to come on the difficulty model soon...

    Last edited by stevep99 (22-Jan-2015 15:03:22)

    Using Colemak-DH with Seniply.

    Offline
    • 0
    • Reputation: 214
    • From: Viken, Norway
    • Registered: 13-Dec-2006
    • Posts: 5,360

    The perceived ease of displacing J is the whole basis for Tarmak. But this is in the context of the displacement being temporary. I'm still not convinced that the time-integrated disruption is less for a rarely used key! That could go both ways really... and in fact, I think it does. Makes modeling a hassle, I know.

    A quadratic term should be backed by a hypothesis or observation. If you have none, guesswork won't help.

    Last edited by DreymaR (22-Jan-2015 15:43:04)

    *** Learn Colemak in 2–5 steps with Tarmak! ***
    *** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

    Offline
    • 0
    • Reputation: 117
    • From: UK
    • Registered: 14-Apr-2014
    • Posts: 978
    DreymaR said:

    The perceived ease of displacing J is the whole basis for Tarmak. But this is in the context of the displacement being temporary. I'm still not convinced that the time-integrated disruption is less for a rarely used key! That could go both ways really... and in fact, I think it does. Makes modeling a hassle, I know.

    A quadratic term should be backed by a hypothesis or observation. If you have none, guesswork won't help.

    I suppose I was thinking along the lines of: If you have Key-A which is 4 times more common than Key-B, then rather than weight it 4 times as much, you'd only be weighting it twice as much. In other words, it would be like saying that the fact that you learn Key-A faster by typing it more often would cancel out half the effect. The whole idea of comparing keyboard layouts is guesswork of course, but still, I do agree with you and I am not inclined to introduce an arbitrary power law unless there's a more reasonable basis for it.

    In the meantime I have updated my Switching Difficulty page to also include the index if letter frequency is excluded (the "Keys Moved Index", Σ m), so those who think letter frequency is irrelevent can consider that instead. I also added MTGAP and Qwpr layouts for comparison as well.

    Using Colemak-DH with Seniply.

    Offline
    • 0
    • Reputation: 214
    • From: Viken, Norway
    • Registered: 13-Dec-2006
    • Posts: 5,360

    Keep up the interesting thinking! :-)

    *** Learn Colemak in 2–5 steps with Tarmak! ***
    *** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

    Offline
    • 0
    • Reputation: 117
    • From: UK
    • Registered: 14-Apr-2014
    • Posts: 978

    I have recently been re-evaluating how to measure layout similarity / switching difficulty, and as a consequence, have developed a new model.

    This area is obviously still highly subjective - probably even more so that layout design - so there is not going to be an entirely uncontroversial way to model layout switching difficulty. But I have tried anyway :p

    Previously the frequency of each key being moved was assumed to have a linear relationship with switching difficulty: a key with a frequency of 4% was deemed twice as hard to learn as a 2% one. However in practise, as DreymaR in particular has argued, it is observed that this is mitigated by the fact that more frequent keys are practised more, and hence tend to have a faster learning rate. I then came across the idea of Experience Curves, which seem describe this idea rather well by the use of a power-law relationship.

    So I have modified the model to include a power-law frequency-related cost, which I think is a little more satisfactory. In case anyone is interested in such geeky details, you can read about it on my updated Switching Difficulty page.

    Last edited by stevep99 (14-Oct-2017 16:13:08)

    Using Colemak-DH with Seniply.

    Offline
    • 0
    • Reputation: 214
    • From: Viken, Norway
    • Registered: 13-Dec-2006
    • Posts: 5,360

    This is getting really interesting, don'cha think? :-)

    Do you really think that Colemak is that much harder to learn than Norman? Is your difficulty index to be taken to mean that Colemak is about twice as hard as Minimak-8 (in which case I'd see virtually no point in Minimak-8 as that layout is a lot less than half as good as Colemak in my opinion!), and 50% harder than Asset/Norman?

    Could you do the Tarmak bunch as well? Maybe not all of them if you feel that it makes your table too cluttered (if so, 2 and 4 maybe?), although I think it'd be interesting to see!

    And maybe a juicy graph would show the scores more strikingly? The most interesting graph would be a "bang for buck" one, combining switching difficulty with typing goodness! Of course, that's fairly conjectural... but still good fun!

    On the other hand: Are all those layouts you listed and analyzed actually in use these days? I don't think anyone actually uses Asset anymore... do they?

    Last edited by DreymaR (14-Oct-2017 17:29:58)

    *** Learn Colemak in 2–5 steps with Tarmak! ***
    *** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

    Offline
    • 0
    • Reputation: 117
    • From: UK
    • Registered: 14-Apr-2014
    • Posts: 978

    I can't really judge how accurate the results are, as I have never tried to learn any layouts other than Colemak. I was initially tempted by Norman, mainly on account of it being seemingly easier to learn. Norman does try to keep a lot of keys on the same finger, which would account for its score. Probably hardly anyone uses any of the layouts listed apart from Colemak, Dvorak, (and perhaps Workman?), but who knows!?

    I just generated results for tarmak layouts:

    layout  |   ∑m   |  d
    ------------------------
    tarmak1 |  2.50  |   8.3
    tarmak2 |  3.25  |  11.6
    tarmak3 |  4.75  |  18.5
    tarmak4 |  6.50  |  24.5

    Yeah, the overall score vs learning difficultly might be interesting.

    But then again, once you start going down the keyboard layout rabbit hole, you are always likely to end up unsatisfied if you picked a layout that's only half optimized, even if it did take a little shorter time to learn!

    Last edited by stevep99 (15-Oct-2017 14:00:43)

    Using Colemak-DH with Seniply.

    Offline
    • 0
    • Reputation: 214
    • From: Viken, Norway
    • Registered: 13-Dec-2006
    • Posts: 5,360

    I agree with you there! Half-baked layouts are evil. Of course, that raises the question of when enough is enough – around which there are different opinions it seems. ;-)

    But the main point of such a metric would be to identify whether your model indicates whether some layouts just aren't worth it since they give little bang for much buck so to speak. In this flowering garden of keyboard layouts I'm sure some would measure up more favorably on your bottom line.

    Thanks for the Tarmath. ;-) Looks like a fairly even ladder up, which is encouraging! Also, Tarmak2 is a little easier to learn than Minimak-8 in this model, which means they're comparable since Tarmak2 moves 7 keys.

    *** Learn Colemak in 2–5 steps with Tarmak! ***
    *** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

    Offline
    • 0
    • Reputation: 214
    • From: Viken, Norway
    • Registered: 13-Dec-2006
    • Posts: 5,360

    Okay, a few more requests:

    Could you do the analysis of Tarmak-Curl(DH) too? Just out of curiosity. ;-) And I thought you also had the Effort scores for the Tarmak layouts but I didn't see them. But I can use the Layout Analyzer myself, sure.

    On your colemakmods pages, I've noticed that the layout effort metric with three decimal places is just a little clunky. Those "lower is better" numbers aren't wrong, but they aren't very easy on the eye either? Maybe it's hard to envision exactly what they're telling us anyway, but I thought it'd be nice to have a Typing Effort or Typing Ease Index that relates the layouts to QWERTY like your Switching Difficulty Index does? The former would be a ratio from 100% (QWERTY) to 69% (Colemak-CurlDH), while the latter would go to 145% for our favourite layout. Sure, it's probably totally wrong to just throw out that Colemak-Curl(DH) is "45% better than QWERTY"; but it'd be fun! ^_^

    Last edited by DreymaR (16-Oct-2017 09:44:50)

    *** Learn Colemak in 2–5 steps with Tarmak! ***
    *** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

    Offline
    • 0
    • Reputation: 117
    • From: UK
    • Registered: 14-Apr-2014
    • Posts: 978

    For a while I have had the idea of turning the overall layout scores into percentages. This would be done on the lines of a 100% score would be the "theoretically best keyboard", and 0% would be the "theoretically worst keyboard".

    Using my scoring system, these best/worst layouts would be defined as follows: Suppose you take the most frequent letter and assign it to the best key. Then the 2nd best, and so on. For a given frequency distribution and key scoring system, this would define the best possible keyboard layout based on simple positioning only (but obviously would be impractical as it ignores bigrams, difficult combos, etc). But still, if you gave such a layout no bigram penalties, it would be the theoretically best scoring, although impossible to reach in practice. A similar but reverse approach would be used to get the worst layout. These two layouts would be the "100%" and "0%" used to calibrate the scores for other layouts.

    It would also be interesting to see, on a scale from worst-possible to best-possible, where on that spectrum Qwerty fits.

    When I have some free time again, I might look into it.

    Last edited by stevep99 (16-Oct-2017 10:38:51)

    Using Colemak-DH with Seniply.

    Offline
    • 0
    • Reputation: 214
    • From: Viken, Norway
    • Registered: 13-Dec-2006
    • Posts: 5,360

    Not sure whether that's the most fruitful way to look at it, but I'm also not sure what to do instead! So, hehe.

    Meanwhile, have a look at my Tarmak Analysis & History topic, in which I've now added your above analysis and that of your pages, with some number gymnastics for good measure. Good, cheap fun! Not sure whether I'm on the ball or not with that table, but it does paint a pretty picture doesn't it? ;-)

    Last edited by DreymaR (16-Oct-2017 12:04:09)

    *** Learn Colemak in 2–5 steps with Tarmak! ***
    *** Check out my Big Bag of Keyboard Tricks for Win/Linux/TMK... ***

    Offline
    • 0