The Week in Chess

Sunday, June 30, 2019

Rubi 1.5 vs. Top Chess Engines

RubiChess 1.5 was released last June 27.  Rubi placed 20th on this gauntlet where in the prior gauntlet, it placed 30th which is a big leap forward.

 Rank  Chess Engine   ELO     Games  Score%  Win%  Draw% Author
 1   Stockfish 10 x64  3637   100   93.00 87.00 12.00 Romstad, Costalba, Kiiski, Linscott
 2   Houdini 6.02 x64  3589   100   91.00 83.00 16.00 Robert Houdart
 3   Komodo 11.2.2 x64  3568   100   90.00 81.00 18.00 Dailey, Kaufman, Lefler
 4   Fire 7.1 x64  3541   100   88.50 78.00 21.00 Norman Schmidt
 5   Xiphos 0.5.4 x64  3516   100   87.00 75.00 24.00 Milos Tatarevic
 6   Ethereal 11.46 x64  3415   100   79.00 65.00 28.00 Andrew Grant
 7   Andscacs 0.95 x64  3381   100   75.50 66.00 19.00 Daniel Jose Queralto
 8   Shredder 13 x64  3371   100   74.50 58.00 33.00 Stefan Meyer-Kahlen
 9   Booot 6.3.1 x64  3344   100   71.50 58.00 27.00 Alex Morozov
 10   Laser 1.7 x64  3340   100   71.00 50.00 42.00 Jeffrey An, Michael An
 11   Rofchade 2.1 x64  3311   100   67.50 50.00 35.00 Ronald Friedrich
 12   Schooner 2.0.34 x64  3296   100   65.50 44.00 43.00 Dennis Sceviour
 13   Gull 3 x64  3280   100   63.50 45.00 37.00 Vadim Demichev
 14   Equinox 3.30 x64  3262   100   61.00 38.00 46.00 Colli, Rocchi
 15   Critter 1.6a x64  3262   100   61.00 40.00 42.00 Richard Vida
 16   Fritz 15 x64  3226   100   56.00 35.00 42.00 Chessbase
 17   Nirvana 2.4 x64  3204   100   53.00 37.00 32.00 Thomas Kolarik
 18   Senpai 2.0 x64  3204   100   53.00 33.00 40.00 Fabien Letouzey
 19   Hannibal 1.7 x64  3190   100   51.00 32.00 38.00 Sam Hamilton, Edsel Apostol
 20   Rubi 1.5 x64  3183   3000   38.68 22.43 32.50 Andreas Matthias
 21   Texel 1.07 x64  3176   100   49.00 27.00 44.00 Peter Osterlund
 22   Fizbo 2.0 x64  3173   100   48.50 36.00 25.00 Youri Matiounine
 23   Nemorino 5.00 x64  3169   100   48.00 30.00 36.00 Christian Gunther
 24   Smarthink 1.98 x64  3162   100   47.00 33.00 28.00 Sergei Markoff
 25   Protector 1.9.0 x64  3152   100   45.50 28.00 35.00 Raimund Heid
 26   ICE 3.0.658 x64  3145   100   44.50 30.00 29.00 Thomas Petzke
 27   Wasp 3.60 x64  3141   100   44.00 23.00 42.00 John Stanback
 28   SCTR 1.1f x64  3134   100   43.00 24.00 38.00 Can Catin, Dogac Eidenk
 29   Vajolet 2.2.7.0 x64  3116   100   40.50 21.00 39.00 Marco Belli
 30   Arasan 21.3 x64  3112   100   40.00 24.00 32.00 Jon Dart
 31   Naum 4.6 x64  3090   100   37.00 21.00 32.00 Aleksandar Naumov

Download the gauntlet PGN games here.

Thursday, June 27, 2019

Shark 190622 vs. Stockfish and Clones


 Rank  Chess Engine   ELO     Games  Score%  Win%  Draw% Author
 1   Asm Fish 2019-05-18 x64  3592   200   54.25 22.00 64.50 M. Li, L. Antonov. J. Dehorty
 2   CFish 190621 x64  3573   200   51.50 15.50 72.00 Ronald De Man
 3   SF Polyglot 190621 x64  3564   200   50.25 14.00 72.50 Massimiliano Goi
 4   Shark 190622 x64  3562   2400   52.40 17.46 69.90 Krahs
 5   McCain X4 x64  3555   200   49.00 12.00 74.00 Michael Byrne
 6   Brain Fish 190616 x64  3554   200   48.75 11.50 74.50 Thomas Zipproth
 7   Cor Chess 3.1 x64  3554   200   48.75 13.00 71.50 Ivan Ivec
 8   Stockfish 190622 x64  3552   200   48.50 16.50 64.00 Romstad, Costalba, Kiiski, Linscott
 9   Stockfish 10 x64  3550   200   48.25 12.50 71.50 Romstad, Costalba, Kiiski, Linscott
 10   Shash 8.0 x64  3539   200   46.75 7.50 78.50 Andrea Manzo
 11   Raubfisch X40c x64  3524   200   44.50 11.00 67.00 Frank Karl Warner
 12   Eman 3.72 x64  3506   200   42.00 9.50 65.00 Khalid Omar
 13   Sugar NN 1.0 x64  3482   200   38.75 7.00 63.50 Marco Zerbinati

Download the gauntlet PGN games here.

Wednesday, June 26, 2019

Ethereal 11.46 vs. Top Chess Engines

It seems a very long that Ethereal has not been updated in most engines list rating sites. Despite that Ethereal 11.46 is just a development release, I decided to make a gauntlet just to have an idea of its latest strength. Xiphos 0.5.4 was also a development release but was given a gauntlet tournament the last time, so it is fair  to make the same to Ethereal.

In this gauntlet, Ethereal managed to defeat Fire 7.1 by a very small margin with 50.5-49.5 score. What is significant about it is that the incremental updates of Ethereal though very difficult to produce a big ELO improvement at this stage finally broke the barrier held by Fire. Another point is that Xiphos scored higher than Fire which reinforce its small superiorty over Fire. In the Hybrid Rating List, Fire is still higher in rank against Ethereal.

The ranking sequence of the top 3 Engines in the rating list remains the same.

 Rank  Chess Engine   ELO     Games  Score%  Win%  Draw% Author
 1   Stockfish 10 x64  3620   100   81.00 64.00 34.00 Romstad, Costalba, Kiiski, Linscott
 2   Houdini 6.02 x64  3593   100   78.50 60.00 37.00 Robert Houdart
 3   Komodo 11.2.2 x64  3419   100   57.50 35.00 45.00 Dailey, Kaufman, Lefler
 4   Xiphos 0.5.4 x64  3398   100   54.50 31.00 47.00 Milos Tatarevic
 5   Ethereal 11.46 x64  3366   3100   71.16 55.23 31.90 Andrew Grant
 6   Fire 7.1 x64  3363   100   49.50 18.00 63.00 Norman Schmidt
 7   Laser 1.7 x64  3306   100   41.50 16.00 51.00 Jeffrey An, Michael An
 8   Shredder 13 x64  3295   100   40.00 16.00 48.00 Stefan Meyer-Kahlen
 9   Rofchade 2.1 x64  3254   100   34.50 13.00 43.00 Ronald Friedrich
 10   Andscacs 0.95 x64  3246   100   33.50 10.00 47.00 Daniel Jose Queralto
 11   Critter 1.6a x64  3226   100   31.00 15.00 32.00 Richard Vida
 12   Schooner 2.0.34 x64  3226   100   31.00 9.00 44.00 Dennis Sceviour
 13   Equinox 3.30 x64  3209   100   29.00 10.00 38.00 Colli, Rocchi
 14   Booot 6.3.1 x64  3188   100   26.50 5.00 43.00 Alex Morozov
 15   Fritz 15 x64  3169   100   24.50 10.00 29.00 Chessbase
 16   Hannibal 1.7 x64  3160   100   23.50 6.00 35.00 Sam Hamilton, Edsel Apostol
 17   Nirvana 2.4 x64  3160   100   23.50 6.00 35.00 Thomas Kolarik
 18   Texel 1.07 x64  3155   100   23.00 7.00 32.00 Peter Osterlund
 19   Gull 3 x64  3155   100   23.00 6.00 34.00 Vadim Demichev
 20   Senpai 2.0 x64  3118   100   19.50 6.00 27.00 Fabien Letouzey
 21   Fizbo 2.0 x64  3112   100   19.00 7.00 24.00 Youri Matiounine
 22   ICE 3.0.658 x64  3112   100   19.00 10.00 18.00 Thomas Petzke
 23   Pedone 1.9 x64  3089   100   17.00 5.00 24.00 Fabio Gobbato
 24   Wasp 3.60 x64  3089   100   17.00 5.00 24.00 John Stanback
 25   Nemorino 5.00 x64  3082   100   16.50 6.00 21.00 Christian Gunther
 26   Arasan 21.3 x64  3076   100   16.00 3.00 26.00 Jon Dart
 27   Protector 1.9.0 x64  3033   100   13.00 4.00 18.00 Raimund Heid
 28   Rubi 1.4 x64  3017   100   12.00 2.00 20.00 Andreas Matthias
 29   Naum 4.6 x64  3000   100   11.00 7.00 8.00 Aleksandar Naumov
 30   SCTR 1.1f x64  3000   100   11.00 2.00 18.00 Can Catin, Dogac Eidenk
 31   Smarthink 1.98 x64  2991   100   10.50 4.00 13.00 Sergei Markoff
 32   Vajolet 2.2.7.0 x64  2913   100   7.00 2.00 10.00 Marco Belli

Download the gauntlet PGN games here.

Tuesday, June 25, 2019

Stockfish 190622 vs. Clones

There is no more Stockfish clones that has their latest version updated, so it is time for the Stockfish 190622 development version to be tested.

The result showed that it just managed to draw with the Stockfish 10 release version. This is the pattern that I see all along where the cummulative ELO gains of the Stockfish development versions are almost worth nothing despite the reported ELO rating gains at abrok.eu for each of the numerous releases.

CFish 190621 is on top of the gauntlet tournament followed by BrainFish and AsmFish. Again we see some random placement of the engines and the overall Stockfish and Clones rating list sees little changes in the ranking. CFish and AsmFish are on top while Sugar NN and Eman were on the bottom. Both Stockfish engines on the list is  somewhere in the middle which serves as the reference point for all the clones strength.

 Rank  Chess Engine   ELO     Games  Score%  Win%  Draw% Author
 1   CFish 190621 x64  3580   200   54.25 23.00 62.50 Ronald De Man
 2   Brain Fish 190616 x64  3571   200   53.00 24.50 57.00 Thomas Zipproth
 3   Asm Fish 2019-05-18 x64  3564   200   52.00 23.50 57.00 M. Li, L. Antonov. J. Dehorty
 4   Shark 190531 x64  3555   200   50.75 17.00 67.50 Krahs
 5   McCain X4 x64  3552   200   50.25 16.50 67.50 Michael Byrne
 6   SF Polyglot 190621 x64  3552   200   50.25 20.50 59.50 Massimiliano Goi
 7   Stockfish 10 x64  3550   200   50.00 22.50 55.00 Romstad, Costalba, Kiiski, Linscott
 8   Stockfish 190622 x64  3550   2400   51.88 21.46 60.80 Romstad, Costalba, Kiiski, Linscott
 9   Cor Chess 3.1 x64  3527   200   46.75 15.00 63.50 Ivan Ivec
 10   Shash 8.0 x64  3520   200   45.75 13.50 64.50 Andrea Manzo
 11   Raubfisch X40c x64  3502   200   43.25 12.00 62.50 Frank Karl Warner
 12   Sugar NN 1.0 x64  3486   200   41.00 13.00 56.00 Marco Zerbinati
 13   Eman 3.72 x64  3481   200   40.25 11.50 57.50 Khalid Omar

Download the gauntlet PGN games here.

Monday, June 24, 2019

Eman 3.72 vs. Stockfish and Clones

This is the second test of Eman with release dated 05/22/2019. The first time was a month earlier and was dropped in the subsequent rating list release as it uncertain if the author could provide consistent releases.
Like the previous test, Eman placed last but it has good description of its capabilites here. The description seems to match my observation that the more features are added for the SF clones, it becomes weaker.

 Rank  Chess Engine   ELO     Games  Score%  Win%  Draw% Author
 1   Asm Fish 2019-05-18 x64  3605   200   67.50 42.00 51.00 M. Li, L. Antonov. J. Dehorty
 2   CFish 190621 x64  3595   200   66.25 40.50 51.50 Ronald De Man
 3   Brain Fish 190616 x64  3556   200   61.00 30.50 61.00 Thomas Zipproth
 4   Shash 8.0 x64  3556   200   61.00 31.00 60.00 Andrea Manzo
 5   McCain X4 x64  3552   200   60.50 31.00 59.00 Michael Byrne
 6   Stockfish 10 x64  3550   200   60.25 32.00 56.50 Romstad, Costalba, Kiiski, Linscott
 7   Stockfish 19053114 x64  3545   200   59.50 32.00 55.00 Romstad, Costalba, Kiiski, Linscott
 8   Raubfisch X40c x64  3539   200   58.75 30.00 57.50 Frank Karl Warner
 9   SF Polyglot 190621 x64  3539   200   58.75 30.50 56.50 Massimiliano Goi
 10   Shark 190531 x64  3535   200   58.25 29.50 57.50 Krahs
 11   Cor Chess 3.1 x64  3527   200   57.00 32.00 50.00 Ivan Ivec
 12   Sugar NN 1.0 x64  3507   200   54.25 29.50 49.50 Marco Zerbinati
 13   Eman 3.72 x64  3477   2400   39.75 12.04 55.40 Khalid Omar

Download the gauntlet PGN games here.

Sunday, June 23, 2019

CFish 190621 vs. Stockfish and Clones

This is CFish 190621 gauntlet matches against Stockfish and Clones. As expected it is tight match against Asm Fish where CFish was defeated with a very narrow score. The resulting rankings is also tight. However, In the overall Stockfish and Clones rating list, CFish is in the lead.

 Rank  Chess Engine   ELO     Games  Score%  Win%  Draw% Author
 1   Asm Fish 2019-05-18 x64  3599   200   50.50 20.50 60.00 M. Li, L. Antonov. J. Dehorty
 2   CFish 190621 x64  3596   2200   55.25 24.68 61.10 Ronald De Man
 3   Shark 190531 x64  3580   200   47.75 16.00 63.50 Krahs
 4   Brain Fish 190616 x64  3571   200   46.50 16.50 60.00 Thomas Zipproth
 5   SF Polyglot 190621 x64  3566   200   45.75 18.00 55.50 Massimiliano Goi
 6   McCain X4 x64  3562   200   45.25 13.00 64.50 Michael Byrne
 7   Stockfish 19053114 x64  3557   200   44.50 12.50 64.00 Romstad, Costalba, Kiiski, Linscott
 8   Cor Chess 3.1 x64  3555   200   44.25 10.50 67.50 Ivan Ivec
 9   Stockfish 10 x64  3550   200   43.50 13.00 61.00 Romstad, Costalba, Kiiski, Linscott
 10   Raubfisch X40c x64  3545   200   42.75 14.50 56.50 Frank Karl Warner
 11   Shash 8.0 x64  3541   200   42.25 11.00 62.50 Andrea Manzo
 12   Sugar NN 1.0 x64  3519   200   39.25 10.50 57.50 Marco Zerbinati

Download the gauntlet PGN games here.

Saturday, June 22, 2019

SF Polyglot 190621 vs. Stockfish and Clones

Stockfish Polyglot was released on June 21, 2019. In this gauntlet the name was renamed to SF Polyglot because of Stockfish name contamination in the rating list. I am sure that most of the Stockfish team members will not feel good when other engines dilute the original Stockfish name at a disadvantage.

With some ranking changes in the gauntlet, it essentially does not change the overall Stockfish and Clones rating list. McCain X4 scored high but Shark was down.
 Rank  Chess Engine   ELO     Games  Score%  Win%  Draw% Author
 1   CFish 190603 x64  3590   200   54.00 24.50 59.00 Ronald De Man
 2   Asm Fish 2019-05-18 x64  3583   200   53.00 25.00 56.00 M. Li, L. Antonov. J. Dehorty
 3   McCain X4 x64  3576   200   52.00 18.00 68.00 Michael Byrne
 4   Brain Fish 190616 x64  3562   200   50.00 20.50 59.00 Thomas Zipproth
 5   SF Polyglot 190621 x64  3562   2200   52.20 23.18 58.00 Massimiliano Goi
 6   Stockfish 19053114 x64  3555   200   49.00 22.00 54.00 Romstad, Costalba, Kiiski, Linscott
 7   Stockfish 10 x64  3550   200   48.25 20.00 56.50 Romstad, Costalba, Kiiski, Linscott
 8   Raubfisch X40c x64  3541   200   47.00 19.50 55.00 Frank Karl Warner
 9   Shark 190531 x64  3539   200   46.75 17.00 59.50 Krahs
 10   Cor Chess 3.1 x64  3538   200   46.50 19.50 54.00 Ivan Ivec
 11   Shash 8.0 x64  3522   200   44.25 12.50 63.50 Andrea Manzo
 12   Sugar NN 1.0 x64  3454   200   35.00 8.00 54.00 Marco Zerbinati

Download the gauntlet PGN games here.

Friday, June 21, 2019

Sugar NN 1.0 vs. Stockfish and Clones

Sugar NN 1.0 rearranged the ranking of Stockfish and Clones. It confused the engines where SF Polyglot was on top, AsmFish is in the 4th and Stockfish 10 is on the 3rd.

This is the latest version of Sugar which was released just yesterday. It is expected that each release will be much stronger than the previous release, but this is not the case here where Sugar placed last by losing to all Stockfish and Clones. It is worth noting that most SF clones which are heavily modified are in the lower half of the ranking sequence. 

 Rank  Chess Engine   ELO     Games  Score%  Win%  Draw% Author
 1   Stockfish Polyglot 190604 x64  3576   200   65.25 43.00 44.50 Massimiliano Goi
 2   CFish 190603 x64  3561   200   63.25 37.50 51.50 Ronald De Man
 3   Stockfish 10 x64  3550   200   61.75 43.00 37.50 Romstad, Costalba, Kiiski, Linscott
 4   Asm Fish 2019-05-18 x64  3544   200   61.00 43.00 36.00 M. Li, L. Antonov. J. Dehorty
 5   Shark 190531 x64  3539   200   60.25 37.50 45.50 Krahs
 6   Brain Fish 190616 x64  3528   200   58.75 35.00 47.50 Thomas Zipproth
 7   Stockfish 19053114 x64  3524   200   58.25 35.00 46.50 Romstad, Costalba, Kiiski, Linscott
 8   Cor Chess 3.1 x64  3512   200   56.50 33.00 47.00 Ivan Ivec
 9   McCain X4 x64  3508   200   56.00 30.50 51.00 Michael Byrne
 10   Raubfisch X40c x64  3492   200   53.75 27.50 52.50 Frank Karl Warner
 11   Shash 8.0 x64  3482   200   52.25 25.50 53.50 Andrea Manzo
 12   Sugar NN 1.0 x64  3466   2200   41.18 17.86 46.60 Marco Zerbinati

Download the gauntlet PGN games here

Thursday, June 20, 2019

Brain Fish 190616 vs. Stockfish and Clones

This gauntlet features Brainfish 190616. Once again, the ranking is re-arranged when CFish scored higher than Asmfish; SF Polyglot, Stockfish 10 and Stockfish 19053114 had positive scores.

In this gauntlet, Brainfish is ranked 6th but 3rd in the Stockfish and Clones Rating List. It confused me, but that's how the rating statistics showed, nothing could change it.

 Rank  Chess Engine   ELO     Games  Score%  Win%  Draw% Author
 1   CFish 190603 x64  3562   200   52.50 27.50 50.00 Ronald De Man
 2   Stockfish Polyglot 190604 x64  3559   200   52.00 22.00 60.00 Massimiliano Goi
 3   Asm Fish 2019-05-18 x64  3559   200   52.00 22.50 59.00 M. Li, L. Antonov. J. Dehorty
 4   Stockfish 10 x64  3550   200   50.75 22.50 56.50 Romstad, Costalba, Kiiski, Linscott
 5   Stockfish 19053114 x64  3550   200   50.75 22.50 56.50 Romstad, Costalba, Kiiski, Linscott
 6   Brain Fish 190616 x64  3545   2200   51.48 21.45 60.00 Thomas Zipproth
 7   Shark 190531 x64  3541   200   49.50 18.00 63.00 Krahs
 8   Cor Chess 3.1 x64  3527   200   47.50 15.50 64.00 Ivan Ivec
 9   Sugar MCTSL 220319 x64  3517   200   46.00 17.50 57.00 Marco Zerbinati
 10   Shash 8.0 x64  3515   200   45.75 12.00 67.50 Andrea Manzo
 11   McCain X4 x64  3506   200   44.50 10.50 68.00 Michael Byrne
 12   Raubfisch X40c x64  3492   200   42.50 13.00 59.00 Frank Karl Warner

Download the gauntlet PGN games here.

Wednesday, June 19, 2019

Raubfisch X40c vs. Stockfish and Clones

Raubfisch X40c dropped from several ranks lower instead of going up.  CFish was on top on this gauntlet and managed to equal the ELO rating with ASM Fish in the Stockfish and Clones Rating List. Stockfish 10 is higher than Stockfish 19053114 in this gauntlet and even the overall rating list. I happened to visit the Stockfish Testing forum and read a thread that Stockfish is having a drought in ELO points. That may explain its struggle in the rating list.

 Rank  Chess Engine   ELO     Games  Score%  Win%  Draw% Author
 1   CFish 190603 x64  3591   200   58.25 29.50 57.50 Ronald De Man
 2   Asm Fish 2019-05-18 x64  3577   200   56.25 33.00 46.50 M. Li, L. Antonov. J. Dehorty
 3   Stockfish Polyglot 190604 x64  3568   200   55.00 28.50 53.00 Massimiliano Goi
 4   Brain Fish 190602 x64  3561   200   54.00 23.50 61.00 Thomas Zipproth
 5   Sugar MCTSL 220319 x64  3559   200   53.75 27.50 52.50 Marco Zerbinati
 6   Shark 190531 x64  3550   200   52.50 21.50 62.00 Krahs
 7   Stockfish 10 x64  3550   200   52.50 25.50 54.00 Romstad, Costalba, Kiiski, Linscott
 8   Cor Chess 3.1 x64  3550   200   52.50 23.00 59.00 Ivan Ivec
 9   Stockfish 19053114 x64  3546   200   52.00 22.50 59.00 Romstad, Costalba, Kiiski, Linscott
 10   McCain X4 x64  3541   200   51.25 18.50 65.50 Michael Byrne
 11   Raubfisch X40c x64  3532   2200   47.09 18.45 57.30 Frank Karl Warner
 12   Shash 8.0 x64  3490   200   44.00 14.00 60.00 Andrea Manzo

Download the gauntlet PGN games here.

McCain X4 vs. Stockfish and Clones


McCain X4 released its compiled binaries last June 12, 2019 but got it just yesterday. So, to update the Stockfish and Clones rating list, I proceeded to organize the 200 rounds gauntlet tournament. This time McCain scored high which made it at the 6th place but ranked 9th in the overall Stockfish and Clones rating list. Stockfish 10 placed 12th but still in the 9th rank overall. CFish topped the gauntlet but is in 2nd rank in rating list. SF Polyglot placed 2nd but 4th in the rankings.

The results seem random that I have no complete control over it. So, I just let it appear as it is and see later what are the causes of this random results.

There are new gauntlets awaiting in the pipeline for CFish, Stockfish, BrainFish, Shark and others that may catch up later. I expect that the rankings will be shuffled each time there is a gauntlet tournament.

 Rank  Chess Engine   ELO     Games  Score%  Win%  Draw% Author
 1   CFish 190603 x64  3599   200   56.00 25.00 62.00 Ronald De Man
 2   Stockfish Polyglot 190604 x64  3596   200   55.50 24.00 63.00 Massimiliano Goi
 3   Asm Fish 2019-05-18 x64  3583   200   53.75 24.50 58.50 M. Li, L. Antonov. J. Dehorty
 4   Brain Fish 190602 x64  3564   200   51.00 17.50 67.00 Thomas Zipproth
 5   Stockfish 19053114 x64  3561   200   50.50 17.50 66.00 Romstad, Costalba, Kiiski, Linscott
 6   McCain X4 x64  3557   2200   48.68 16.50 64.40 Michael Byrne
 7   Shash 8.0 x64  3557   200   50.00 15.50 69.00 Andrea Manzo
 8   Shark 190531 x64  3557   200   50.00 15.00 70.00 Krahs
 9   Raubfisch X40b2 x64  3555   200   49.75 15.50 68.50 Frank Karl Warner
 10   Sugar MCTSL 220319 x64  3555   200   49.75 20.50 58.50 Marco Zerbinati
 11   Cor Chess 3.1 x64  3552   200   49.25 16.50 65.50 Ivan Ivec
 12   Stockfish 10 x64  3550   200   49.00 19.00 60.00 Romstad, Costalba, Kiiski, Linscott

Download the gauntlet PGN games here.

Sunday, June 16, 2019

Stockfish 10 vs. Stockfish 19053114 - One on One

This is the follow-up in yesterday's post. I was shocked to see that the difference between the Stockfish 19053114 and Stockfish 10 is less than 3 ELO points in favor of the development version. The cummulative ELO differentials in the Stockfish development versions as posted in abrok.eu/stockfish could easily reach 20 ELO since the last release of Stockfish 10. To make sure that my test is near statistical correctness I decided to make this one on one match between the official version and the development version. I made a compilation of Stockfish 10 with Ubuntu 18.10/Mingw 7.3. to equalize with the compiler used in Stockfish 19053114 which was compiled by abrok.eu with Ubuntu 18.04/Mingw 7.3.

The match was conducted in one computer with AMD 4 cores and 16GB RAM. The tournament condition is similar with all my previous Stockfish vs. Clones matches with the only difference of using the 2moves_LT_1000.pgn that is the exact fit for the 2000 rounds match.

This is the actual settings for the cutechess-cli tournament manager:
./cutechess-cli1.1 \
-event "Stockfish Test" \
-pgnout ./Out/"Stockfish 10 vs. Stockfish 19053114.pgn" \
-engine conf="Stockfish 19053114 x64" \
-engine conf="Stockfish 10 x64" \
-site "Chessowl" \ -each tc=60+1 proto=uci timemargin=1000 option.Threads=1 \
-openings file=./Books/2moves-all.pgn format=pgn order=sequential plies=4 start=1 \
-resign movecount=1 score=200 twosided=true \
-draw movenumber=40 movecount=1 score=20 \
-maxmoves 80 \
-concurrency 20 \
-games 1 \
-rounds 2000 \
-repeat \
-tournament round-robin \
-recover

 Another shock jolted me when the result was a win by Stockfish 10 by a very small margin of 0.79 ELO. Perhaps I need feedback from experienced testers on their Stockfish testing for confirmation whether my tests are accurate and possibly point me to where my tests are wrong. I appreciate comments that will lead to improvement.

 Here is the result of the one on one match:

   # PLAYER                    :  RATING  POINTS  PLAYED   (%)
   1 Stockfish 10 x64          :    0.79  1004.5    2000  50.2
   2 Stockfish 19053114 x64    :   -0.79   995.5    2000  49.8

Download the match games here.

Saturday, June 15, 2019

Stockfish and Clones Rating List

The past posts featured 100 round robin tournament among Stockfish and Clones. It was just to show observations on the effects of using different different tournament parameters such as number of threads, opening book, tournament manager, time control and others. The results were consistently inconsistent. There is no fixed ranking sequence for each engine which is probably caused by the fact that all are almost of the same strength and that there are just so many random things that are happening like system programs running in the background at different intervals.

To make things a little bit better, I decided to make some changes. First, Houdini is excluded since it is not an Stockfish clone and instead replaced it with Stockfish 10 as the base reference for all the clones strength. It is much easier to see the progress of the Stockfish development versions and the clones whether they are becoming stronger or weaker as newer versions are released. Secondly, the number of rounds are doubled from 100 to 200 for each engine which will minimize the randomness of strength.

In the past few days I duplicated the tournament of the last one to double the rounds then made a 200 gauntlet tournament with Stockfish against the clones. The final result confuses me as Stockfish 10 is in the middle of the pack considering that it was made earlier this year 2019. I expected Stockfish 10 to be the last in the group as the clones were updated on May 2019 at the latest. But the one that seems to be illogical is the less than 3 ELO score difference between the latest Stockfish 190531 and Stockfish 10. If the ELO increases are added for each of the Stockfish development releases since January 2019, it would easily total to more than 20 ELO.

This disturbing result makes me want to do a one on one death match between Stockfish and the latest version at greater number of 2000 rounds to see what really is the true ELO difference.

Here is the final base reference of the Stockfish vs. Clones rating list.

Rank  Chess Engine   ELO    Games  Score%  Win%  Draw% Author
 1  Asm Fish 2019-05-18 x64  3582   2200  55.07 26.68 56.80 M. Li, L. Antonov. J. Dehorty
 2  CFish 190603 x64  3576   2200  54.16 25.68 57.00 Ronald De Man
 3  Shark 190531 x64  3561   2200  51.80 20.82 62.00 Krahs
 4  Stockfish Polyglot 190604 x64  3555   2200  50.86 22.50 56.70 Massimiliano Goi
 5  Brain Fish 190602 x64  3555   2200  50.86 21.77 58.20 Thomas Zipproth
 6  Stockfish 19053114 x64  3552   2200  50.43 20.91 59.00 Romstad, Costalba, Kiiski, Linscott
 7  Stockfish 10 x64  3550   2200  50.07 20.23 59.70 Romstad, Costalba, Kiiski, Linscott
 8  Raubfisch X40b2 x64  3547   2200  49.66 20.95 57.40 Frank Karl Warner
 9  Cor Chess 3.1 x64  3536   2200  47.82 19.18 57.30 Ivan Ivec
 10  Shash 8.0 x64  3530   2200  46.98 14.91 64.10 Andrea Manzo
 11  Sugar MCTSL 220319 x64  3529   2200  46.77 18.36 56.80 Marco Zerbinati
 12  McCain X3 x64  3521   2200  45.52 15.73 59.60 Michael Byrne

Download the Stockfish vs. Clones tournament games here.

Wednesday, June 12, 2019

Xiphos 0.5.4 vs. Top Chess Engines

This is the gauntlet match for Xiphos 0.5.4 against the top chess engines.  The previous post with a preliminary match against Fire 7.1 was a victory with a 7 points ELO margin. In this gauntlet Fire 7.1 got a revenge with 54.5-45.5 score.  Surprisingly, Xiphos still managed to dislodged Fire in the Hybrid Rating List where the major rating list producers are combined with around 9 ELO advantage.

This seems to be a random result but we will just see in the days ahead on how far the development of Xiphos can go as Fire is stale for a long time.  So now we have Xiphos sitting at number 5 rank behind Stockfish, Houdini, Komodo and LCZero in the Hybrid Rating List.

Here is the result:

Rank  Chess Engine   ELO    Games  Score%  Win%  Draw% Author
 1  Stockfish 10 x64  3473   100  76.00 56.00 40.00 Romstad, Costalba, Kiiski, Linscott
 2  Houdini 6.02 x64  3464   100  75.00 52.00 46.00 Robert Houdart
 3  Komodo 11.2.2 x64  3354   100  61.50 33.00 57.00 Dailey, Kaufman, Lefler
 4  Fire 7.1 x64  3303   100  54.50 27.00 55.00 Norman Schmidt
 5  Xiphos 0.5.4 x64  3271   3100  72.82 56.23 33.20 Milos Tatarevic
 6  Ethereal 11.25 x64  3204   100  40.50 16.00 49.00 Andrew Grant
 7  Andscacs 0.95 x64  3174   100  36.50 14.00 45.00 Daniel Jose Queralto
 8  Shredder 13 x64  3167   100  35.50 13.00 45.00 Stefan Meyer-Kahlen
 9  Laser 1.7 x64  3163   100  35.00 12.00 46.00 Jeffrey An, Michael An
 10  Critter 1.6a x64  3143   100  32.50 9.00 47.00 Richard Vida
 11  Gull 3 x64  3135   100  31.50 10.00 43.00 Vadim Demichev
 12  Booot 6.3.1 x64  3123   100  30.00 9.00 42.00 Alex Morozov
 13  Rofchade 2.1 x64  3110   100  28.50 8.00 41.00 Ronald Friedrich
 14  Schooner 2.0.34 x64  3110   100  28.50 7.00 43.00 Dennis Sceviour
 15  Equinox 3.30 x64  3065   100  23.50 7.00 33.00 Colli, Rocchi
 16  Fizbo 2.0 x64  3055   100  22.50 8.00 29.00 Youri Matiounine
 17  Pedone 1.9 x64  3023   100  19.50 1.00 37.00 Fabio Gobbato
 18  Nirvana 2.4 x64  3017   100  19.00 4.00 30.00 Thomas Kolarik
 19  Senpai 2.0 x64  2994   100  17.00 5.00 24.00 Fabien Letouzey
 20  Texel 1.07 x64  2987   100  16.50 0.00 33.00 Peter Osterlund
 21  Nemorino 5.00 x64  2987   100  16.50 5.00 23.00 Christian Gunther
 22  Fritz 15 x64  2981   100  16.00 4.00 24.00 Chessbase
 23  Hannibal 1.7 x64  2981   100  16.00 4.00 24.00 Sam Hamilton, Edsel Apostol
 24  ICE 3.0.658 x64  2974   100  15.50 3.00 25.00 Thomas Petzke
 25  Wasp 3.60 x64  2968   100  15.00 3.00 24.00 John Stanback
 26  Smarthink 1.98 x64  2938   100  13.00 4.00 18.00 Sergei Markoff
 27  Naum 4.6 x64  2930   100  12.50 2.00 21.00 Aleksandar Naumov
 28  Protector 1.9.0 x64  2922   100  12.00 1.00 22.00 Raimund Heid
 29  Arasan 21.3 x64  2922   100  12.00 4.00 16.00 Jon Dart
 30  Vajolet 2.2.7.0 x64  2905   100  11.00 5.00 12.00 Marco Belli
 31  SCTR 1.1f x64  2886   100  10.00 1.00 18.00 Can Catin, Dogac Eidenk
 32  Rubi 1.4 x64  2876   100  9.50 1.00 17.00 Andreas Matthias

Download the gauntlet PGN games here.

Monday, June 10, 2019

Xiphos 0.5.4 vs. Fire 7.1

Xiphos was released last June 4 which I missed. I always make a test when there is an unofficial relase to see if it breaks the barrier of an aspiring chess engine to be at the top. That barrier is the one held by Fire 7.1 which is around 3350 ELO.

What a pleasant surprise to see that Xiphos beat Fire which is breaking the barrier. Although its a win for Xiphos here, I will do a gauntlet match for Xiphos against all the the top 20 engines in the next post to confirm. Watch for the sequel.

Here is the result.

Rank Chess Engine ELO Games Score%  Win%  Draw% Loss%
1 Xiphos 0.5.4 x64  7.01 200 52.00 27.00 50.00 23.00
2 Fire 7.1 x64  -7.01 200 48.00 23.00 50.00 27.00

Download the match PGN games here.

Saturday, June 8, 2019

gcc vs. clang: Which is the fastest?

Which compiler is fastest or best for Stockfish? I have tried different compilers for Stockfish before using Visual Studio, Intel Compiler and gcc. The one that won was gcc not only for its execution speed but also for its frequency of updates. I have not tried Clang seriously because of compile/compatibility problems with the linux OS distribution I use.

The day that the latest CFish 190603 was released with compile from Dan Corbitt, it was perfect time to try the gcc vs. clang test. I made a few hundred rounds of CFish gcc vs. clang and gcc won by 52% vs. 48%. Today I decided to formalize the test with a fixed 1000 rounds using a the 500 positions opening book from Noomen. The result was a 53% vs. 47% win by gcc again. With this result, gcc/mingw will continue to be my primary choice of compiler with chess engines.

Here is the result:
   # PLAYER                :  RATING  POINTS  PLAYED   (%)
   1 CFish 190603 gcc      :    10.9   531.0    1000    53
   2 CFish 190603 clang    :   -10.9   469.0    1000    47

Download the CFish download by Dan Corbitt here.
Download the match PGN games here.

Friday, June 7, 2019

Stockfish 19053114 vs. Clones

This is a snapshot test for Stockfish 19053114 and its clones. Up to date clones are CFish 190603, Shark 190531, SF Polyglot 190604, Brain Fish 190602. Compared with previous test, the upper top half ranking sequence are shuffled while the lower half essentially remained the same.

One of the motivation for this Stockfish and clones tournament is to determine how reliable and efficient is the cutechess-cli tournament manager compared to Arena 3.5. The series of results of these tournaments showed that cutechess-cli is awesomely fast, reliable and efficient. It can handle more than 20 concurrencies (or threads) while at the same time being able to use the same computer for system administration and browsing. Arena 3.5 chokes at more than 7 threads in my 4 core computer.

So far, the results are consistent and if there are deviations in rankings, it is because of the many random things such as; the move sequences are never the same even though they have at most 4 ply same start opening positions, engine versions are not synchronized, cpu usage congestion caused by external programs doing miscellaneous tasks.

The top dog at this time is CFish with an ELO difference of +28 against Stockfish, followed closely by Asm Fish. One of the authors of Asm Fish revealed to me that the last compiled versions of Asm Fish is usually many months older than Stockfish but are still much stronger. That may not work this time because CFish is freshly updated. The strength of CFish is primarily it is made in C (which is portable assembler), whereas Stockfish is a C++ based. Some say that C++ is faster than C but in this situation CFish beats Stockfish which meant that C is faster (assuming that the algorithm is the same). Look at the resulting executables, Stockfish is 5 times bigger than CFish. It is logical that the less codes to execute, the faster it is to reach the end of the code, therefore it does more to search and evaluate positions.

Most Stockfish clones are supposedly meant to enhance the performance or functionality of the original Stockfish. But 70% of the clones are weaker in this tournament. The primary reason is that they are not in lock-step with Stockfish in development versions and secondly, clone authors add numerous features that does not necessarily add ELO score but reduces the search effectiveness. For example, adding opening book code which is totally useless in this tournament because that feature is ignored. The best way to beat Stockfish is to make the chess engine in Assembler or C, or remove useless features in Stockfish that reduces ELO.

Here is the result:

Rank  Chess Engine   ELO    Games  Score%  Win%  Draw% Author
 1  CFish 190603 x64  3550   1100  56.05 28.18 55.70 Ronald De Man
 2  Asm Fish 2019-05-18 x64  3547   1100  55.64 27.36 56.50 M. Li, L. Antonov. J. Dehorty
 3  Shark 190531 x64  3525   1100  52.32 21.36 61.90 Krahs
 4  Stockfish 19053114 x64  3522   1100  51.73 21.55 60.40 Romstad, Costalba, Kiiski, Linscott
 5  Stockfish Polyglot 190604 x64  3520   1100  51.50 22.55 57.90 Massimiliano Goi
 6  Brain Fish 190602 x64  3519   1100  51.32 23.09 56.50 Thomas Zipproth
 7  Raubfisch X40b2 x64  3512   1100  50.23 22.82 54.80 Frank Karl Warner
 8  Cor Chess 3.1 x64  3500   1100  48.45 19.45 58.00 Ivan Ivec
 9  Sugar MCTSL 220319 x64  3493   1100  47.32 18.27 58.10 Marco Zerbinati
 10  Shash 8.0 x64  3492   1100  47.14 16.18 61.90 Andrea Manzo
 11  McCain X3 x64  3481   1100  45.45 16.45 58.00 Michael Byrne
 12  Houdini 6.02 x64  3464   1100  42.86 15.91 53.90 Robert Houdart

Download the gauntlet PGN games here.

Chessdom News