The Week in Chess

Tuesday, August 21, 2018

Stockfish 18081801 vs. Clones


The last Stockfish vs. Clones tournament was primarily an experiment using the cutechess tournament manager. I was intrigued that cutechess could handle 16 concurrent matches very easily without noticeable degradation of the responsiveness with the computer which is an old AMD Athlon with 4 cores and 16GB RAM. It was ran under the latest PCLinuxOS without a single time forfeit. The previous tournament manager I used was Arena 3.5.1 which could handle about 7 concurrent games and beyond that the computer become sluggish resulting numerous time forfeits. I doubled the concurrency to 32 and still it ran without sluggishness and time forfeits. Perhaps it could run 64 concurrencies...

The other significant change was the use of cross-compiler under Ubuntu 18.04 with mingw version 7.3. This was the system used by abrok.eu to compile Stockfish which was faster than the one I made under Windows/mingw. I discovered that it could accept the -flto option which does not run under Windows. The resulting benchmark is faster using the cross-compiler and made CFish overtake Stockfish in the ranking. Official CFish does not come with an executable so I made one from the Ubuntu cross-compiler. The other clones were using their own executables which were probably optimized by their respective authors.

There were two new entrants in the tournament, namely CorChess and ShashChess. The other change was the use of the later versions of Stockfish, Brainfish and Raubfisch.

The higher concurrency of 32 made me double the number of rounds from 100 to 200 for greater statistical accuracy. The 10 man round-robin tournament with 200 games each was finished in approximately 16 hours using only 1 computer. This amazed me for it will take a very long time to finish with the Arena GUI.

This is the tournament condition:
Time control: 60 moves in 60 seconds base + 1 second increment
Time margin: 1 second
Option.Threads: 1
Resign score: 200 cp
Draw score: 20 cp after 40 moves
Concurrency: 32
Book: 100 variations, 2 moves deep, alternating

Tournament result with 0 based rating:

Rank Chess Engine ELO Games Score%  Win%  Draw% Loss%
1 Asm Fish 180722 x64  26.94 1800 54.25 23.28 61.90 14.82
2 CFish 180819 x64  23.06 1800 53.64 23.28 60.70 16.02
3 Brain Fish 180818 x64  19.88 1800 53.14 22.00 62.30 15.70
4 Stockfish 18081801 x64  11.25 1800 51.78 21.44 60.70 17.86
5 Shark 180818 x64  6.85 1800 51.08 20.33 61.50 18.17
6 Raubfisch X38e x64  -8.09 1800 48.72 18.22 61.00 20.78
7 McBrain 9.6 x64  -14.95 1800 47.64 16.61 62.10 21.29
8 Cor Chess 2.5 x64  -16.18 1800 47.44 15.94 63.00 21.06
9 Shash 180820 x64  -24.12 1800 46.19 14.94 62.50 22.56
10 Sugar XPro 1.5.3 x64  -24.65 1800 46.11 17.11 58.00 24.89

Download the tournament games here.

1 comment:

  1. I also made position suite test which can proove quality of engines, can send.

    ReplyDelete

Chessdom News