Iron Sudoku Rankings Analysis

The rankings were finally posted today at Iron Sudoku!! I've decided to analyze the formula they use, because I have, as Fishy tells me, "too much free time."

The individual puzzles at Iron Sudoku are scored this way:

Easy: 1pt
Medium: 2pts
Hard: 3pts
Expert: 5pts

And there is one new puzzle every day.

Iron Sudoku has a ranking which it calls "Cream O' The Crop," which is computed with the following formula:
(total points earned ÷ total puzzles completed x 100) + (total puzzles completed)

Some have pointed out, therefore, that completing easy and medium level puzzles actually hurts your ranking, and so it makes more sense to do only the hard and expert level puzzles. (More interestingly, I don't believe this formula was posted previously, so nobody realized that they were actually hurting their ranking by returning to the site every day and doing the puzzle!)

The following chart shows the value of every puzzle to date. The blue (left) part shows a running score for someone who successfully completes every day's puzzle. The purple (right) part shows the running score for someone who completes only the hard and expert-level puzzles.

		All Puzzles			Hard & Expert Only
Date	Puzzle#	Points Earned	Total Points	Ranking Score	Points Earned	Total Points	Ranking Score
Mar-11	1	1	1	101.0	0	0	0
Mar-12	2	3	4	202.0	3	3	301.0
Mar-13	3	1	5	169.7	0	3	301.0
Mar-14	4	5	10	254.0	5	8	402.0
Mar-15	5	2	12	245.0	0	8	402.0
Mar-16	6	2	14	239.3	0	8	402.0
Mar-17	7	2	16	235.6	0	8	402.0
Mar-18	8	3	19	245.5	3	11	369.7
Mar-19	9	2	21	242.3	0	11	369.7
Mar-20	10	3	24	250.0	3	14	354.0
Mar-21	11	2	26	247.4	0	14	354.0
Mar-22	12	1	27	237.0	0	14	354.0
Mar-23	13	2	29	236.1	0	14	354.0
Mar-24	14	1	30	228.3	0	14	354.0
Mar-25	15	3	33	235.0	3	17	345.0
Mar-26	16	2	35	234.8	0	17	345.0
Mar-27	17	3	38	240.5	3	20	339.3
Mar-28	18	5	43	256.9	5	25	364.1
Mar-29	19	2	45	255.8	0	25	364.1
Mar-30	20	1	46	250.0	0	25	364.1
Mar-31	21	1	47	244.8	0	25	364.1
Apr-01	22	1	48	240.2	0	25	364.1

What should be obvious is that not only should you avoid doing the easy and medium puzzles, but you should also avoid the hard ones! You can see that every time the purple player finished a hard (3-point) puzzle, his score actually went DOWN. If a player had only completed the expert level puzzles above, his score as of today would be 502.

The most ironic part about this is that the site says that the rankings system is there "For the players with a bit of a competitive spirit," but this ranking system actually discourages players from participating! Indeed, a player trying to maximize his/her score would have only played twice in the last 22 days!

Fortunately, Iron Sudoku does have a second ranking, which it calls "Village Elders," which is simply a straight tally of the number of points a player has accumulated so far. In that system, it makes sense to do the puzzle every day, since your score can only go up. The advantage of the "Cream O' The Crop" ranking is that anyone can start at any time and still have a chance at a good ranking, whereas the highest ranking Village Elders are almost guaranteed to be the people who have been playing the longest. (If you look at the chart again, you'll see that the purple player has a Village Elder ranking of only 25 points, whereas the blue player has a ranking of 48.)

It's been suggested that they create a ranking determined by how quickly you solve the puzzle. Personally, I oppose such a ranking. Sometimes I sit down and work on the puzzle straight through til I'm done, but more often I print it out and do it on the bus, entering the solution on the web later in the day. Or I work on it in bits and pieces while I'm at work. Also, IronSudoku.com is my homepage, so that I don't forget to do it every day. That means as soon as I open my browser, the timer would start, even if I'm not ready to start looking at the puzzle itself yet.

So none of these rankings is an accurate gauge of how good a player is. Cream O' The Crop rewards people who DON'T play. Village Elders punishes people who might be better players, but haven't been on the site as long. And a time-based ranking punishes people who don't have time to complete the puzzle all in one sitting. And of course it's all fairly meaningless anyway in a world where sudoku-solving software like Simple Sudoku et al make it easy to cheat. (Simple Sudoku is a fantastic program, btw, which is good for far more than just cheating!)

None of this will stop me from doing the Iron Sudoku puzzle every day; I just like to do Sudoku, and I like their chat room feature, and the interface for actually doing the puzzle is the best I've seen on a website.

So that's my analysis for now.

UPDATE: I've been fooling around a little more with the Excel spreadsheet I set up to examine the "Cream O' The Crop" formula because something hit me: the formula includes a term for "# of puzzles completed," and at some point, that becomes the dominant term and the other part of the ranking becomes negligible. Assuming that the exact same pattern of puzzle difficulty from these first 22 days repeats itself over and over again, and assuming that the purple and blue players continue their same pattern of playing, then on day 205, purple and blue are tied with a score of 423.5. Then, on day 206, blue finally surpasses purple with a score of 424.9 to 423.6. If purple instead pursues a strategy of playing only expert-level puzzles, then blue doesn't surpass purple until day #310, with a score of 528.1 to 528. This means that the strategy of playing only harder puzzles is only a good one for the first 204 (or 309) days, at which point purple should switch to blue's strategy of finishing every day's puzzle.

Update #2: Wait! Stop the presses! I was wrong again! More in a minute after I figure out the details!

Update #3: Of course, it was a silly mistake. Assuming that the blue player plays every day, then if the purple player misses even a single day, there is no strategy that is guaranteed to keep him ahead of blue forever. So the expert-only strategy works fine for those first 309 days, but after that, purple will never have a higher score than blue ever again. (The logic is sound, even though it's possible that my actual numbers might be a little off if I typed something wrong into Excel at some point, or because Excel is rounding to the nearest tenth of a point.) Hopefully there will be no more updates, if only because it's almost my bed time. G'nite!