The Week in Chess

Wednesday, September 4, 2013

Komodo 5.1r2 x64 - Gauntlet Rematch, 100 Rounds

These are the rematch of Komodo 5.1r2 against the chess engines where its default "threads" setting of 1 was not altered. After observing the rating list of other popular sites like CCRL, CEGT and IPON, I noticed that Komodo was always in the number 2 position, whereas it is just number 5 in my rating list. There must be something wrong, and lately when Stockfish threads parameter was set to 1 by default, I was forced to change it to the number of CPU's in a computer because the score was always very low using the default. Then, I read from the forums that Komodo's threads parameter was set to 1 by default also, I knew what caused the low ELO rating.

I never changed the default parameters of the chess engines when I am doing tournaments because changing any one of them have positive or negative effect on the ELO rating. To my great dismay, the Stockfish author changed the default behavior of the "threads" parameter where the users must alter it to have a better performance. The previous default setting was automatically set by the engine and it worked perfectly all the time. When Stockfish had a very low score, the first thing I check is the "threads" parameter and most of the time it is set to 1 because I forgot to change it whenever I install a new development version.

As far as I know, Komodo was the first to implement such bad idea of requiring the users to tamper the default threads setting of a chess engine. Then Stockfish followed where there was a pattern of implementing Don Dailey's ideas into Stockfish which don't work sometimes.

Now, the Komodo team could blame me for the bad rating of their engine, but I think the author has the responsibility of making sure that the the default settings are the best and could work fine without alteration in a tournament.

EMPHASIS:  ENGINE AUTHORS MUST BE RESPONSIBLE FOR THE PROPER DEFAULT SETTINGS.


Here is the performance of Komodo in the rematch. 
Rank Engine ELO Raw Games Score% Points Win Loss Draw Chg TF% Ply
1 Houdini 3 Pro x64 3187 131 100 56.50 56.5 40 27 33 1 2.00 66
2 Komodo 5.1r2 x64 3134 88 2700 64.07 1730.0 1211 451 1038 34 0.59 108
3 Robodini 1.1 x64 3106 63 100 46.00 46.0 27 35 38 0 1.00 32
4 Critter 1.6a x64 3104 45 100 43.00 43.0 23 37 40 0 0.00 0
5 PanChess 00.537 x64 3071 36 100 41.50 41.5 21 38 41 -1 0.00 0
6 Tactico Power 2011 x64 3045 33 100 41.00 41.0 20 38 42 1 2.00 78
7 Fire 2.2 xTreme x64 3070 32 100 41.50 41.5 25 42 33 0 4.00 48
8 Firenzina 2.3.1 xTreme x64 3072 30 100 40.50 40.5 18 37 45 0 4.00 64
9 ComStock 3 x64 3084 29 100 40.00 40.0 17 37 46 1 0.00 0
10 RyanFish 1 x64 3086 24 100 40.00 40.0 22 42 36 1 0.00 0
11 Gull 2.2 x64 3092 22 100 39.00 39.0 17 39 44 -1 0.00 0
12 Strelka 5.5 x64 3053 20 100 39.00 39.0 17 39 44 -1 2.00 14
13 RobboLito 0.21Q x64 3067 18 100 38.50 38.5 15 38 47 1 2.00 47
14 Ivanhoe 46h x64 3053 3 100 36.00 36.0 13 41 46 1 3.00 101
15 Mars 1 x64 3078 1 100 35.50 35.5 10 39 51 -2 2.00 165
16 Bouquet 1.7b x64 3081 -1 100 36.00 36.0 16 44 40 -1 0.00 0
17 Sting SF 3 x64 2987 -10 100 34.50 34.5 15 46 39 5 4.00 118
18 Rybka 4.1 x64 3039 -12 100 35.50 35.5 21 50 29 0 1.00 121
19 Shredder 12 x64 2928 -20 100 34.50 34.5 22 53 25 0 2.00 130
20 Saros 3.3b x64 2995 -21 100 32.50 32.5 12 47 41 0 4.00 93
21 Ippolit 0.080b3 x64 2989 -25 100 32.00 32.0 12 48 40 -1 5.00 204
22 LEOpard 0.7c x64 3044 -27 100 32.00 32.0 12 48 40 0 0.00 0
23 Igorrit 0.086v9 x64 3069 -29 100 31.00 31.0 9 47 44 -3 1.00 18
24 Vitruvius 1.11C x64 3027 -39 100 30.00 30.0 11 51 38 -2 1.00 40
25 Black Mamba 1.4 x64 2934 -74 100 26.50 26.5 12 59 29 1 0.00 0
26 Akkad 0.52b x64 2974 -75 100 25.50 25.5 8 57 35 1 1.00 172
27 Hannibal 1.3 x64 2910 -98 100 24.00 24.0 11 63 26 -1 3.00 112
28 Naum 4.2 x64 2883 -145 100 18.00 18.0 5 69 26 -1 2.00 50
.
Download the computer chess engines tournament games here.

No comments:

Post a Comment

Chessdom News