How much is 'so much'? What does one need to believe in to succeed where others fail, and what are false beliefs? Tricky.

Then maybe there's something wrong with your "crap scripts" rather than with Colemak, which has been designed very rigourously.

"Similarity to Qwerty" is often over-rated as a limitation for optimisation, but it's really not so bad.  You can keep Q, W, Z, X, C, V and punctuation characters in place almost "for free" because given their low frequency, they are in good positions already: on the bottom row and in the corners of the layout.  Rare letters will not impact aspects like di/trigraph scores or finger travel a lot, so moving them around will only give you marginal gains (almost within the margins of error).  Then why bother and not just leave them in place?  The rare keys are also the hardest to re-learn; everyone will remember where T and N are after two minutes of practicing any layout, but Z?  And in practice you will have to use Qwerty from time to time, and then the similarity is really a convenience you will appreciate.

I don't see the point. Let's say you create a layout that could (theoretically) shave off a few milliseconds - so what? Some people type at 150wpm with Qwerty, Dvorak, Colemak etc., but that doesn't mean that I will. No layout can promise an increase in speed.

I guess we doubt that your typing model can accurately predict the typing speed achievable by a layout. Your input data only tells a sampling of your current performance with your current layout.

Yeah, I don't buy into the sovereignty of Colemak based on the rigorousness alone either. But I do prefer Colemak over grail chasing these days. My intuitive understanding of the situation is that your grail may be too individually based, and not improve enough on the performance of Colemak/Dvorak to be worth it. Plus, I really like not changing more keys than necessary so it's easy to go back and forth between Colemak and the inevitable rounds of QWERTY I have to cope with.

I like the idea of polling together a load of users' amphetypeoid data. The thing is, I believe your stats to be heavily influenced by which layouts you know well. So my data now would be different from back when I knew Dvorak better than Colemak, of course, and different again if I'd gone directly from QWERTY. Eventually, I'll get to where my data reflects mostly the two parameters of me and Colemak (and a little influence from keyboard hardware, seating position, what I'm writing and other confounds naturally).

I think that getting something really useful out of it would take a lot of users though, since I believe the confounds to be many and significant. But if it gets rolling, I'd seriously consider contributing my own data set; I'm not interested in doing any development work but that much I could do.

Are you sure about that? Theoretically, we wouldn't even need to have letters on the keyboards. The point being made is that human fingers, with very common limitations based on the mechanical structure of the joints and what not, will necessarily have speed constraints based on alliteration, rolling, same-finger, same-key, etc.

Nerve impulse firing speed is a known quantity, which is why same-finger and same key gets the penalties it does with the current measuring methods. In another thread, I noticed that I often mistype the common "io" combination, which is over six times more common than it's "oi" couterpart, and because the shape of the hand indicates outward to inward rolls are much faster than inward to outward, that too can be measured independently of layout.

So far as I know, neither of these things were considered in the construction of Colemak, or even other keyboard layouts. Dvorak is deeply flawed simply because of L being placed on the right pinkie, in non home-row positions no less. All of the existing keyboard layouts make mostly rudimentary assumptions about finger travel without taking into account that the index and middle fingers are highly mobile, especially when switching hands. They basically take the keyboard-centric approach instead of a hand-centered approach, which is still an improvement over QWERTY and it's ilk, but it's only really the first step.

I mean, consider if the positions of E and I were switched in Colemak. It would be terrible for IE rolls. Yet this is exactly what happened with. Heck, try this if you have a Linux system handy:

grep io /usr/share/dict/words | perl -p -e 's/.*(io).*/$1/g; chop' | wc -c
grep oi /usr/share/dict/words | perl -p -e 's/.*(oi).*/$1/g; chop' | wc -c

You'll get 10498  for the first one and 1736 for the second. This indicates that with just the basic English dictionary included with most Linux systems, io happens 6 times more often than oi. Now let's try it with IE and EI:

grep ie /usr/share/dict/words | perl -p -e 's/.*(ie).*/$1/g; chop' | wc -c
grep ei /usr/share/dict/words | perl -p -e 's/.*(ei).*/$1/g; chop' | wc -c

That gives us 9240 and 1756! That's actually just 5.3x more often. So, not only is the ratio higher for IO over OI, it's higher than IE versus EI, which are both on stronger fingers due to the letter frequency. But how do we really know how this affects typing speed? Do the rolls overcome the individual usage?

And these are just one aspect of all the things that likely go into a good layout based on finger mechanics completely independent of the actual layout. These would apply just as validly to QWERTY and Dvorak. Look at QWERTY, and try to type "ion", then try "coin". Since I and O are right next to each other, just not using the pinkie, you can see that the roll is much faster, but again, since I appears about 40% more often in English, this may overcome the difference.

This is the kind of research I think the parent poster wants. An actual study on average finger movements over a keyboard based on key position, not the actual lettering of the key. Is an index->middle always slower than a middle->index combo on the same hand. Extended to the entire hand, a whole masters thesis could probably be done on just that single metric. Can stronger fingers, despite possibly longer travel, overcome the effect gained by adjacent letters on weaker fingers? We don't know because nobody has actually run those studies. We're just guessing based on intuition, and that's not scientific.

But an observation of several thousand typists, so long as we know their chosen layout, will give us this valuable data, it will tell us which key combinations are actually fastest in practice with a weighted average and standard deviation and several other metrics we don't have now. What Colemak does is introduce the valuable insight that computer-aided measurements produce a better layout, so why not take it to the next logical conclusion?

I'll stop rambling now. :p

* There have been loads of debates of inwards rolls vs outwards rolls. I've reached the conclusion is that it's very much a personal thing.

* As has been said before, the pinky can easily type things on the home position, it's when it needs to move around compounded by same finger that it gets tiring (e.g. try typing qzqzqz etc.)

* The ring finger is the least agile finger. The yo/oy digraph occurs about 7 times more often than the yi/iy digraph, so same finger and speed will suffer.

* According to your methodology the 'ti' digraph is a lot more common than the 'th':
grep th /usr/share/dict/words | perl -p -e 's/.*(th).*/$1/g; chop' | wc -c
41072
grep ti /usr/share/dict/words | perl -p -e 's/.*(ti).*/$1/g; chop' | wc -c
106330
Word frequencies are useless. You need to build a large high-quality balanced text corpus, and then run statistics off of that.

* While speed might be important, and while having _high quality and statistically significant_ timing information would be valuable information, it's not the only metric that should considered. Bad statistics can be worse than no statistics.

* You can't focus on one small aspect of the keyboard layout without thinking how it will affect things overall.

Fair enough. But when I drum my fingers on a table, it's impossible not to notice that giong out->in is easy, while the reverse is ridiculously slow, arduous, and inconsistent. But I'm old. :p

See, that's a valid point. Someone brought that up elsewhere, and thinking about that, I certainly wouldn't want to type YOU (for example) with the ring finger having both Y and O.

Noted. I actually downloaded Carpalx and noticed the included corpus produced far different results and made me think about the overall picture.

Exactly. People don't type in a vacuum. I proposed the OI vs. IO theory, which you've pointed out is probably short-sighted, because it was considered in isolation of basically everything else. Scientists have analyzed how fast Usain Bolt can possibly run by mapping muscle contractions, bone structure, and so on. I really wish someone out there did the same thing with hand structure considering how many millions of people type, and actually sat down and created a keyboard that was both ergonomic for the human hand, and balanced for trigraphs, finger repetitions, hand balance, weighted against theoretical finger mobility and muscle percentage distribution.

Designing a keyboard is only an art because we don't have the research to do it "right." :p

But hey, so far Colemak is the best known keyboard layout out there, so it's not like I'm saying it's crap. I just like to ask questions, is all. Thanks for answering them, by the way!

That statement there, is where I call big fat BS. You admit that your improved layout is highly individualized for you (and your layout history not the least!), but then you generalize like that. Big no.

You have no proof whatsoever that I could type the speeds you claim with your layout, and when we do the proposed data collection of a large (and diverse!) enough typist population I feel that we'll end up with something that while it won't let you type as fast as easily as your own one does, it probably won't be notably/measurably (reliably!!!) faster for the average user than Colemak - at the cost of some neat design principles. Which wouldn't be worth it.

One sentiment might be that everyone should indeed make their own layout in these days where layouts can be installed on the fly. But I won't go to such steps for myself and prefer hanging with the Colemak crowd. I also don't know how reliable criteria for the individual layout might be made: If I were to make myself a layout now it'd only show what I could type fast on now and now a couple of years from now. By that criterium, I should never have left QWERTY!

A large corpus of typing data might prove me wrong... if we could make it. If you manage to, make sure you get enough information on each typist: Age, RSI episodes, layout history (x years with QWERTY, then y years/months/weeks with WHATHAVEYOU, then...) etc etc. Otherwise you're bound to find out too late that some big confound messes up your analysis and that'd be a pity after a long and arduous data collection phase.

Hasn't history proven that Dvorak is not significantly faster than Qwerty, despite its many improvements? And that comfort/reduced chance of injury are more important factors?
Bearing that in mind, I'm not sure it's a good idea to define the 'best' layout as the one that is theoretically fastest.

Now it may turn out that the fastest layout may be the most comfortable. But being 10% faster won't mean it's 10% more comfortable.

I think Colemak is already close to optimal within the restraints of the stupidly asymmetrical standard keyboards we have.

DreymaR, sorry if I was too ambiguous.  Clearly the only hypothetical user for which my data works is myself.  But if we/I had used data that fit more typists, no doubt the same possibility exists for them as well.  I don't see signs of Colemak doing what I did, but with a larger data set.  Do you?

edit:  So I don't believe there is a solid case that Colemak so close to optimal that we shouldn't try to look further.  Without even looking into it, there are good reasons to expect this not to be the case.  And in my own "case study" of my limited data, it certainly wasn't.  Therefore, I conclude that we should at least keep our minds open about the issue--that it may be possible to significantly improve on Colemak unless we accept all its restrictions and metrics as complete and correct without any doubts.

Clearly in my own case I started out trying to make something with the same metrics as Colemak but with less restriction (the idea being that there is no free lunch, so if we don't keep Qwertiness the same we should see some kind of improvement).  I made about twenty significant versions of my keyboards in this pursuit, installing and typing on about a third of those, before I realized that there were other goals, other metrics, that mattered more to me.  Clearly all this isn't conclusive, since it's only based on my typing behaviors, but I think it should make us suspect that at least more investigation is warranted and that settling for something a little better than dvorak (like Colemak) might not be right without a lot more data on what is fast/slow, which is possible if some scripter makes it so.

When I first found my new "metrics" based on my amphetype typing data, what I found is that dvorak was indeed flawed.  While way better than QWERTY, it actually seemed to minimize both "fast" trigrams and "slow" ones, probably because it thought hand switching was ideal, but perhaps also because it optimized so much for other metrics that it "took away" from even average performance in metrics it didn't know about.  In my own very limited, personal data on myself--very preliminary(!!!), but still interesting for terms of hypothesizing, I found interesting things like this:  14.3% of the trigraphs you type in QWERTY are what I identify as "fast" patterns [everything I'm saying here is only about my limited, preliminary data and this would clearly change if we talked about a bigger sample of typists, etc. but bear with me].  With colemak that goes up to 17.4%, which seems to confirm that it does in fact have "better rolls."  Dvorak lacks this improvement, actually going down from QWERTY to 11.9%.  But Dvorak also has a lower rate of "slow" trigraphs than QWERTY--however in my estimation, Colemak is faster overall, because the "fast" increase outweighs the "slow" decrease.

But what I found was that the "fast" trigraphs weren't really optimized for (very much), so I wanted to make a keyboard that had less "slows" than Dvorak _and_ "more fasts" than Colemak.  I actually found that Capewell's evolved layout had even more fasts than Colemak, but also more slows (Colemak was still better).  But at least it gave me something to shoot for.  And later on I found that carpalx's layouts had much more "fasts" and actually seemed to be improving on them.

None of the above had to turn out the way it did, so it reinforced my idea that my data wasn't 100% personal.  Capewell's evolved program was more aware of and deliberate in not ruining hand switching.  Carpalx even more so.  Dvorak does the opposite, and Colemak does this a little with its "rolls" philosophy.  But in my own layout, I've gotten "fasts" up to 27.5% of all trigraphs typed, and "slows" down to 32.%, so I'm happy with that.  I know that with more data and a better searching program I could do even better, but at the least I'm pretty confident that settling for Colemak is not something any of my experience has led me to do.  I have been consistently led away from that by any data I've worked with.

But I'm not the best scripter out there and so I'm hoping that someone will be willing to make it more possible to generalize typing data from different persons using different layouts and put them together so that we can look for fast/slow patterns that ring true for multiple people.  Coming up with keyboards that offer different mixtures of fast patterns, slow patterns, balance, low distance, qwerti similarity, or other factors could naturally only come after this reasonable step is achieved first.

You misunderstood, DreymaR.  Dvorak decreases the frequency of my fastest motions.  But it also decreases the frequency of my slowest, too, reduces distance, and other possibly good things.  My model still accounts for Dvorak's advantages.

We don't need huge, well-founded studies, just data from a few hundred users, gathered in a a systematic way.  One good program that many people agree to run would do this.