The Week in Chess

Tuesday, December 31, 2013

Owl GrandMonster Chess Engines Random Tournament 2013

Happy New Year to Al!

As the year 2013 was about to close, I decided to utilize my idle computers to do something different other than rating list production. What comes to mind was a 4 round robin tournament close to the semi-finals format in the recent TCEC LIve Chess Tournament. The participants were qualifiers in the Stage 4 such as Houdini, Stockfish, Komodo, Gull, Bouquet, Naum and selected Stage 3 qualifiers like Critter and Rybka. The versions used were the latest or the same as in TCEC except for Bouquet and Naum where the latest versions were not released by the authors.

The tournament was labeled "Random" because it was done in 3 computers with random components and different CPU cores. I just want to share to the readers that in a chess engine testing environment, there is a small chance of the similarity of results. It explains the phenomena that no Chess Engines Rating List Sites have the same rankings which confuses the casual observer, because no testing environment are the same and chess is naturally a random game. The time control of this tournament was 2 minutes base + 2 seconds increment and were conducted with only just one instance of tournament in every computer which is not regular in the Owl rating list production. This is not official and will not appear in the Owl Rating List.  

As expected, the 3 strongest engines like Houdini, Stockfish and Komodo were at the top, but accomplishing it was not straight forward. Houdini won 2 tournaments and placed #2 once, while Stockfish won 1 tournament, placed 2nd once and placed 3rd once,  Komodo was in #2 once, #3 once and #6 once. The only chess engine with constant place was Naum at #8 and the rest have varying places throughout the tournament.

Houdini is at its best in a tournament where there are not much threads or processes that are running simultaneously and with very short time controls as evidenced by the results. Stockfish is relatively strong in both multi-threads and short time control. Komodo shines in long time control with the best hardware configuration as proven in the recent TCEC Championship, but relatively weak in very short time control. Gull placed higher than Bouquet which is probably due to the older version of Bouquet used in the tournament. Critter, Rybka and Naum are in the usual places just like in the different rating list rankings.

Congratulations to Robert Houdart of Houdini for winning the Owl GrandMonster Chess Engines Random Tournament!

The results below are the aggregate statistics of the 3 separate tournaments.
Rank Engine ELO Raw Games Score% Points Win Loss Draw TF% Ply
1 Houdini 4 Pro x64 148 84 72.62 61.0 46 8 30 0.00 0
2 Stockfish_13123001_x64 90 84 63.69 53.5 36 13 35 0.00 0
3 Komodo TCEC x64 40 84 55.36 46.5 32 23 29 0.00 0
4 Gull R600 x64 29 84 54.76 46.0 27 19 38 2.38 51
5 Critter 1.6a x64 14 84 52.38 44.0 23 19 42 0.00 0
6 Rybka 4.1 x64 -31 84 44.64 37.5 16 25 43 0.00 0
7 Bouquet 1.8 x64 -61 84 39.88 33.5 12 29 43 0.00 0
8 Naum 4.2 x64 -230 84 16.67 14.0 5 61 18 0.00 0

.


Here are the 3 scoreboards of the tournament:

Owl GrandMonster Chess Engines Random Tournament 2013 A
RankEngineScoreStHoGuCrRyKoBoNaS-B
1Stockfish_13123001_x6418.5/28· ·· ·=01==1100==1==1==111101=11=1 234.50 
2Houdini 4 Pro x64 18.0/28=10=· ·· ·0==01=01=1=1=111=1==111= 224.00 
3Gull R600 x64 17.5/28=0011==1· ·· ·==0==0=1===110111111 219.50 
4Critter 1.6a x64 16.5/281==00=10==1=· ·· ·======11=10==111 212.25 
5Rybka 4.1 x64 13.0/28==0==0=0=1=0====· ·· ·1==01===01== 176.00 
6Komodo TCEC x64 11.0/28=000=000===0==000==1· ·· ·1=111010 137.75 
7Bouquet 1.8 x64 10.5/28010==0==0100=01=0===0=00· ·· ·==1= 147.75 
8Naum 4.2 x64 7.0/2800=0000=0000=00010==0101==0=· ·· · 90.25 


112 games played / Tournament finished

Level: Blitz 2/2
Hardware: AMD Phenom(tm) II X4 945 Processor with 2.0 GB Memory
Operating system: Windows 7 Ultimate Professional Service Pack 1 (Build 7601) 64 bit
Table created with: Arena 3.0
.

Owl GrandMonster Chess Engines Random Tournament 2013 B
RankEngineScoreHoStKoGuCrBoRyNaS-B
1Houdini 4 Pro x64 20.5/28· ·· ·==0=11==111=1=11==1=0=111111 256.75 
2Stockfish_13123001_x6417.0/28==1=· ·· ·==0===11======1==0101111 217.75 
3Komodo TCEC x64 16.0/2800====1=· ·· ·0=0==0101=0111111=11 194.50 
4Gull R600 x64 15.0/28000===001=1=· ·· ·=1=1=======1=1=1 187.50 
5Critter 1.6a x64 14.5/280=00=====101=0=0· ·· ·=====11=111= 176.25 
6Bouquet 1.8 x64 12.5/28==0===0=0=10========· ·· ·=====01= 172.25 
7Rybka 4.1 x64 11.5/281=00=1010000===0=00=====· ·· ·0111 150.25 
8Naum 4.2 x64 5.0/28000000000=00=0=0000==10=1000· ·· · 66.75 


112 games played / Tournament finished

Level: Blitz 2/2
Hardware: Intel (R) (TM) 2 CPU Processor with 2.0 GB Memory
Operating system: Windows 7 Ultimate Professional Service Pack 1 (Build 7601) 64 bit
Table created with: Arena 3.0


.
Owl GrandMonster Chess Engines Random Tournament 2013 C
RankEngineScoreHoKoStGuRyCrBoNaS-B
1Houdini 4 Pro x64 22.5/28· ·· ·=11=110=1111=1=11=1=10111111 275.00 
2Komodo TCEC x64 19.5/28=00=· ·· ·1=010=111=111=111=1==111 230.75 
3Stockfish_13123001_x6418.0/28001=0=10· ·· ·==1===01111=11011111 207.75 
4Gull R600 x64 13.5/2800001=00==0=· ·· ·0==1==1==10=1111 143.75 
5Rybka 4.1 x64 13.0/28=0=00=00==101==0· ·· ·===0=1==11=1 148.00 
6Critter 1.6a x64 13.0/280=0=0=00000===0====1· ·· ·=1111=11 137.75 
7Bouquet 1.8 x64 10.5/2801000=0=0010=01==0===000· ·· ·111= 120.00 
8Naum 4.2 x64 2.0/280000=0000000000000=00=00000=· ·· · 28.00 


112 games played / Tournament finished

evel: Blitz 2/2
Hardware: AMD Athlon (tm) II X2 250 Processor with 2.0 GB Memory
Operating system: Windows 7 Ultimate Professional Service Pack 1 (Build 7601) 64 bit
Table created with: Arena 3.0
.
Download the computer chess engines tournament games here.

No comments:

Post a Comment

Chessdom News