The Week in Chess

Saturday, August 31, 2013

Stockfish - The New Chess Engines Champion!

The TCEC Live Chess Engines Tournament started last August 26, 2013. It is an influential tournament that is considered to be the world championship of chess engines.  The participants were mostly from the previous tournament's season led by Houdini, Stockfish, Rybka and Komodo. Many of the chess engines participants were updated versions notably Stockfish and Komodo which were shown in the rating list sites to be gaining strength.

Based on statistics and personal assessment, I reckoned that Houdini, Stockfish and Komodo will figure prominently going into the finals. To forecast who will be the champion is everyone's personal choice, so I made my own unscientific way of predicting who will be the champion.

I decided to make a round robin tournament of 30 games each for the top 3 likely finalists such as Houdini, Stockfish and Komodo. Time control was 10 minutes base + 10 seconds increment which should be sufficient for an engine to think properly.  The tournament were conducted in the two available computers, an AMD Quad core and AMD Dual core with 4 gigabytes of RAM and running Windows 7. The usual tournament rules in my rating list applies such as opening book (Perfect 2012) moves up to 6 ply, draw in 120 move limit, adjudication as win above 900 centipawns, ponder off, no endgame tablebases. Time forfeit will be part of the game and no replays will be made. Arena ChessGui was used with all the chess engines settings reset to default but Stockfish and Komodo threads option were set to 4 in the quad core and 2 in the dual core computers as recommended.

Stockfish was represented with development version 130827, released just after a day the tournament was started. This was an arbitrary choice since Komodo had a development version entered in the tournament which was not publicly available for testing.

After the games were finished in two days, I hesitated to publish it because of too many time forfeits by Houdini against Komodo despite being conducted in a dual core computer with no other tournament running. But after seeing the initial scores of TCEC where Naum 4.2 lost by time forfeit to Exchess, I was encouraged to just do it because it was the unmodified actual score and chess engines somehow will have losses by time forfeit even with very long time controls. Anyway, this was just for fun.

Here are the individual results:

Stockfish 130827 x64 vs. Houdini 3 Pro x64 = 1-0
1Stockfish 130827 x6418.5/30· ·· ·· ·13-6-11 212.75 
2Houdini 3 Pro x64 11.5/306-13-11· ·· ·· · 212.75 

Stockfish 130827 x64 vs. Komodo 5.1r2 x64 = 1-0
1Stockfish 130827 x6416.0/30· ·· ··9-7-14 224.00 
2Komodo 5.1r2 x64 14.0/307-9-14· ·· ·· 224.00 

Houdini 3 Pro x64 vs. Komodo 5.1r2 x64 = 0-1
1Komodo 5.1r2 x64 18.0/30· ·· ·· ·11-5-14 216.00 
2Houdini 3 Pro x6412.0/305-11-14· ·· ·· · 216.00 

Here are the combined results:

Rank Engine Raw Games Score% Points Win Loss Draw TF% Ply
1 Stockfish 130827 x64 31 60 58 35 22 13 25 1.67 190
2 Komodo 5.1r2 x64 13 60 53 32 18 14 28 3.33 40
3 Houdini 3 Pro x64 -44 60 39 24 11 24 25 18.33 88
Note: TF% = Time forfeit percentage, Ply = Average ply (half-moves) duration of forfeited games.


Rank Engine ELO Games Points c1 c2 c3
1 Stockfish 130827 x64 31 60 34.5 *** 16.0 18.5
2 Komodo 5.1r2 x64 13 60 32.0 14.0 *** 18.0
3 Houdini 3 Pro x64 -44 60 23.5 11.5 12.0 ***

In summary, Stockfish won the tournament individually with 2 points, followed by Komodo with 1 and Houdini with 0.

This was just a tournament for statistical forecasting fun and is not directly related with the TCEC Live Tournament.  The final results may vary, but the results in this tournament serves as my forecast for the top finishers in the TCEC tournament.

Download the chess engines tournament games here.

No comments:

Post a Comment

Chessdom News