I changed the program a bit to find the best home row, independent of the other parts. I ran it three times using the following criteria.

In order to simplify things, I limited the home row to ETAOINSRHD. (It was going to be L instead of D because L is more common by most counts, but D works better because it goes well with T, plus I think L can do better off the home row.)

Each position has a value:

9 2 0 0 40 || 40 0 0 2 9

-Using the index finger twice adds 80 to the score.
-Using the same hand more than twice adds 1 for each time it's happened so far.
-Going to the center without changing hands before or after costs 6.
-Rolling inward subtracts 5.
-Rolling outward subtracts 3.

The inroll-outroll values may be a bit high, but I did this on purpose because I don't think it will hurt the layout, and rolls are fun.

All three times I ran it, I got the same result:

s o a t d h n e r i

In terms of just home rows, according to my criteria this is the best possible.

The advantages:
-3 of the 4 most common keys, ETA, are on the strongest 4 fingers
-the pinkies have a pretty small load
-there are some good rolling digraphs: at, en, er, re, so, ne

I thought it was interesting how, when there are only 10 factorial (about 3 million) combinations, it very quickly comes up with the best possible result.

I am working on a way to add in the top row, then mutate that until it is the best, then add in the bottom row. This way it will already be very good before it adds another layer. This is proving to be harder than it sounds.

That is true. I'm not saying that the home row it generates is the best when added to a full layout. But it is still better than a layout that is made through completely random generation; I switched just 2 keys on the home row it generated and made myself a layout which ended up being the best one I've made by hand so far. It's the same idea with the genetic algorithm. It starts with something that is already good (if not the best) so it then has to do less work to improve it.

When creating the best layout, the key is really in the home row. Everything is based around that: once you have the home row, you basically have the layout. There are only a few decisions left to make, because there are a lot of givens: Y goes with I, U goes with E, W goes with R, punctuation goes with vowels, try to keep ZXCV in the corner. (On my current best layout I have JXZV in the corner, and C is on the top of the left pinky.) All that's left is the placement of a few letters such as m, g, p, f, and even those can't go anywhere. GR, PR and FR are all too common for them to go with R, SM is too common, etc.

I have done another interesting experiment. I locked ETAOINSR into the home row, and eliminated all costs other than same finger. The objective was to determine the best possible arrangement where the keys were in groups of very low same finger. I got this result: (well not really, I rearranged it to be a bit more logical but kept all the groupings the same)

. w u l p q f d ' x
a r i n h c s t e o
, ; y m k v b g j z

Capewell same finger: 98320.0
Colemak same finger: 97928.0
This layout same finger: 89190.0

I have also determined through this that S and N are better* to have on the index fingers than T, which is good because having T on the index finger might cause over stress of that finger.

*marginally

Gee, whiz, ya think so? Thanks for the laugh.  ;)

I'm wondering: Does this mean primarily that you've developed a Colemak-friendly test algorithm, or that Colemak has really good placements? If only I could think of a good way to test your test, as it were.

Quis custodet ipsos custodes?  --  Juvenal

I decided to use a formula to determine the cost of each key, and after a little tweaking, it turned out really well. I measured the horizontal and vertical distance between each key, and applied the following formula to each location:

V = vertical distance from home
H = horizontal distance from home
P = pinky
R = ring
M = middle
I = index

P = h(v+2)*2
R = h^2(v+2)*1.2
M = h^2(v+2)*1
I = h(v+2)*1

Ring and middle finger have horizontal distance squared because it's harder for them to move sideways. That extra number at the end is the relative strength of the finger. M and I are both strong, R isn't quite as strong, and P is a good deal weaker. Here's my results, with everything multiplied by 10 and rounded to the nearest 1:

66 35 29 33 60  91 33 29 35 66
20 12 10 10 40  40 10 10 12 20
96 72 60 48 109 48 48 60 72 96

This is very close to my idea of an ideal cost system. However, a few things were unexpected that I didn't originally implement. It has a much wider spectrum than my original design, where the lowest (that's not 0) is 20 and the highest is 48. I also noticed that the center of the home row ended up costing more than ring, index, AND pinky on the top row. I can understand how this would work; index finger stretches are a bit uncomfortable, and actually have to move slightly farther than reaching to the top row (2 cm vs 1.8 cm). I am going to implement this new system, and will have to tweak the other values to work along with it.

Okay, this is probably the last time I post three times in a row. http://mtgap.bilfo.com/ Here I talk about the four layouts I've designed. I will be updating it fairly regularly for a while.

I could, but it's in Ruby which runs on Mac. I think it will run on Windows but you have to download Ruby, i think at ruby-lang.org. I could upload my files onto my website probably.

What ideas did you have in mind? Maybe I could implement them.

http://mtgap.bilfo.com/keyboards.zip

Go to the above website and you can download it. It will download as soon as you navigate there. Right here http://mtgap.bilfo.com/keyboard.html near the bottom of the page it tells you how to use it.

I am working on ways to make it more user-friendly right now, so what's up there right now isn't the final version.

Well I put the trigraphs program in the package so you can get trigraph frequency from whatever file you want, but it has to be plain text. But to have more than one key of one type would mess things up. When I tried making 2 Es, one of them completely disappeared. Plus, using the home keys and not pinkies subtracts from the score, so having something more common there actually would reduce the score. It's still exactly the same as if the home keys costed 0, but I made it negative because of how it was working in relation to the rest of the keyboard. So if the home keys are all Es, that would reduce the score. Hmm, that's a bit of a problem. You could just add 10 to every position. The place where it's scored is right here, in scorer6.rb:

          @key_score = {
              [0,0] => 66,
              [0,1] => 35,
              [0,2] => 29,
              [0,3] => 33,
              [0,4] => 60,
                        ....

I found out why it can't have 2 of the same letter. It could if you could somehow distinguish between those two letters. How do I know which E I'm going to press this time? The program doesn't, which is why it doesn't work. I can think of 3 ways, and I could probably implement any of them.

1. it's random
2. 50% of the time hit E1, 50% of the time hit E2
3. If the key is followed or preceded by this letter(s), use this key. Else, use the other key. But how do you know how to divide it up?
     3a. If the key is to the left or right of E1, use E1. Otherwise, use E2
     3b. If the key is on the same hand as E1, use E1. Otherwise, use E2

I think 3a would be better, but it depends on context of course.

Yes, that is interesting. The way you can do that is have it add to the final score.  (It divides the score by the number of characters at the end.) What it does is adds to the score for every key that isn't on the same hand as its QWERTY position, and subtracts for every key in the QWERTY position. Depending on how much you make the penalty, you get different results.

Exactly.  I can use a similar penalty for shortcuts. The shortcuts I have are X, Z, C, V, with Q at half value and W at one quarter value.

Here's my favorite:

q w l d b j f u k p
a s r t g h n e o i
z x c v ; y m , . /

I was a bit surprised to find that it moved B, but I can see why. It does not have an added penalty for less frequent keys moving off their QWERTY positions.

It has better finger travel than Colemak, worse same finger and same hand, and better row jumping. In terms of score, they are pretty close. However:

-It keeps 14 keys in their Qwerty positions, to Colemak's 12
-6 other keys are on the same finger
-4 keys change hands: E L F ;

I am not planning on learning this layout. Maintaining 3 keyboards (mine, Colemak, Qwerty) is hard enough. If you want to learn it, be my guest.

That gave me a great idea: make an evolutionary algorithm to optimize a scoring system! It would use already known layouts, and work backwards. It would try to find the scoring system that gets these layouts as close as possible to this order:

Colemak
Capewell
MTGAP's Layout 2.0
Arensito
Nisa (by me, you can find it in this thread I believe)
Maltron
Seruxie's Layout
Amuseum's layout
OSTNREAI
Peter Klausler's layout
Asset
Dvorak
Plum
XPeRT
QWERTY

I think that is the order of how good each of these layouts are. Are there any layouts I should add, or do you think I should change this order?

So what it will do is create some copies of the Scorer class. (This is when object-oriented programming comes in handy.) Each Scorer class has random values for each aspect. They score all of these layouts. The half that are farthest from this order are killed, and the other half are duplicated. The duplications change one of their aspects slightly. Repeat. It ends when one of the scorers scores the layouts in this order.

I have a great idea! I was having a lot of trouble balancing my scoring system. Then I realized that instead of trying to find the scoring system that would create the perfect ratio between travel distance, same hand, etc., I could use the ratio itself! I am not completely sure how it would work, but the idea is that it would try to find the layout with the closest to the optimal ratio, while each attribute is as low as possible. I calculated the ratios for travel distance, same finger, same hand, rolls, and row jumping, respectively, on my 2 best layouts, Colemak, Capewell, and Dvorak. I used these 5 because a) it's hard to do and I didn't want to do many and b) they cover a good range: mine 2.0 and Capewell's have low same hand, Colemak and mine 1.0 have medium, Dvorak has high; plus I wanted to do both my main layouts.

Note 1: When I say same hand, I actually mean reversing direction on one hand.
Note 2: Travel distance is way higher than everything else. It's in centimeters I think.

MTGAP 2.0-- 12 million: 1: 3.3: 21: 0.18

MTGAP 1.0-- 10 million: 1: 2.1: 8: 0.45

Colemak-- 13 million: 1: 3.5: 11: 0.45

Capewell-- 11.5 million: 1: 4.5: 16: 0.2

Dvorak-- 8.4 million: 1: 0.45: 3: 0.25 (Finger travel is low because same finger is high, so it gets reduced instead of increased.)

My idea of a good ratio-- 11 million: 1: 2: 15: 0.3

This ratio is sort of an average of the 5 above. I can create a scorer that gives points for being close to this ratio, and also gives points for the numbers being smaller.

I tried implementing my new idea, and it didn't really work. I couldn't get the overall score to stay low, and when I created a penalty for one of the aspects, I couldn't balance it.

DreymaR: The best thing would be something that also gets digraphs. It's not just individual letters, but also the one that came before it. I want to write a program to count letter and digraph frequency, but I see no way to count hundredths of a second.

Leszek: You have a lot of rolls that you like.

If you're using a Mac, you can use my program. I have a cost for everything I could think of, so all you have to do is change the numbers. You should check out http://mtgap.bilfo.com and colemak.com/Alternative_layouts. I have a bunch of stuff there.

Tuning your weighing algorithms takes a LONG time. I did it for about three months, and then I somehow screwed it up and now I can't get it to work better. There are some very subtle changes that can change a lot. My biggest problem was balancing same finger and travel distance. I put same finger higher, but then it put vowels on the same finger, overloading the finger and increasing travel distance. The solution I found was to give the keys on the home positions a negative score, which isn't perfect, but works.

What do you think of the layouts I've made? You can see them at the "Official keyboards" page on my website.

Edited broken link.