Chessdom News

The Week in Chess

Monday, May 13, 2013

Stockfish 3 x64 vs ComStock 3 x64 - End Game Table Base Showdown

The recent excitement in the Stockfish vs. Houdini match at TCEC-chess.net Live Tournament has subsided.

There were some controversial comments regarding the use of endgame table bases which Stockfish don't have and some people are blaming it for its seemingly idiotic endgame performance. There was a situation in which Stockfish was left with 2 knights but evaluated it as nearly won position, and some rook endgames with 2 pawns up but ended in draws.

Is the Endgame Table Bases really helpful?  +Robert Houdart commented that in some situations it is crucial. He also said that had Stockfish have the EGTB in one of the longest games, it should have been ended early. But on one occasion, where some users observed that Houdini seems to be taking longer to respond in the endgame, Robert acknowledged that the Nalimov EGTB is not working properly.

Marco Costalba's response when asked why EGTB support is not included:

"Because it adds thousands of lines of code, a strong dependency on a
third part tool and all this for nothing. We like SF to be self-contained, small and fully portable."

I've read some comments from other users that the EGTB'S does not add ELO points. To check this statement, I conducted a quick match between the recently released Stockfish 3 against Comstock 3  compiled by Velmarin which is just a Stockfish 3 with Robbobase EGTB included.

ComStock 3 x64 VE vs Stockfish 3 x64 - Match 100R 1M1S
RankEngineScoreStCoS-B
1Stockfish 3 x6454.0/100· ·· ·· ··22-14-64 2484.00 
2ComStock 3 x64 VE 46.0/10014-22-64· ·· ·· ·· 2484.00 


100 games played / Tournament finished

Tournament start: 2013.05.11, 22:24:23
Latest update: 2013.05.12, 06:27:22

Level: Blitz 1/1
Hardware: AMD Phenom(tm) II X4 945 Processor 3000 MHz with 4 GB Memory
Operating system: Microsoft Windows XP 64 Bit Professional Service Pack 2 (Build 3790)
Table created with: Arena 3.0

The above result is just one example that having EGTB's does not really add ELO points in the chess engine's strength where Stockfish beat Comstock with EGTB, all having the same code base.

Before the endgame comes with 6 pieces or less left, there is the middle game phase that usually decides the outcome of the game. The EGTB code gets called sometime just before the endgame and repeatedly during the endgame, and these calls however small are overhead that waste clock cycles, therefore reducing the chess engine's strength.

When a middle game transitions into a win or draw endgame, the EGTB's really shine in ending the game quickly by choosing the shortest sequence of moves. It usually does not alter the outcome of the game. In drawn endgames like having bishop with opposite colors or rook vs. bishop, EGTB's are useless.

What spectators saw in some of the controversial endgames between Houdini and Stockfish could be fixed by code!  The chess science is not yet finished.  Developers are continuously refining their codes and one day we will find that chess engines will be capable of producing perfect endgame performance without the need of EGTB's.


Download the chess engines match games here.

5 comments:

  1. Hi
    Very interesting lecture on usefulness of EGTBs ! I agree with all your theories....most of what you said I have found true in my online Engine-Engine games. I usually use Houdini 3 x64 with Nalimov EGTB.
    Could you tell me what are the best settings for Stockfish, as I'm really interested in trying it out ! ( Using an i7 3930k (6cores) with 32 GB RAM ).

    ReplyDelete
    Replies
    1. The default engine settings are usually optimal. There are occasions that a setting is not properly coded and will produce inconsistent results. With Stockfish, I found that "Use Sleeping Threads" option should be enabled when running a multi-core computer with many programs running simultaneously. The default is turned off in the stable release version but turned on in the development versions.

      Other visible options could also be changed, but this will usually be a waste of time because the results are random. You can try at your spare time.

      Delete
    2. Yes, I read your article regarding enabling sleeping threads in Stockfish.... I was already doing that anyway, as I wanted Stockfish to use all my 6 cores.
      I find that using default Contempt 0 gives me more draws...using Contempt 1 gives me more wins but I also lost 1 game.I never lost while using Contempt 0.
      On another note, regarding the minimum hard probe Depth in Houdini 3, do you think that the disadvantage of using Nalimov EGTB ( wasted clock cycles) can be negated by increasing the minimum hard probe Depth from the default 24 to say, 36 or 48 ? Don't you think this is more useful, especially for people using very fast computers, like me ?
      I mean, I attain 24-25 Depth in next to no time at all. My usual Depth is 29-32. Surely Houdini IS accessing the EGTB on my SSD at Depth 24-25, thereby slowing down the Engine !
      What do you say ?

      Delete
  2. Let me answer you indirectly about EGTB's.

    Perhaps, many people think that by having EGTB's it will improve the strength of a chess engine. What if the opponent also had EGTB enabled? No matter what the situation is, whether it is a win/loss or draw, it will always end that way - the situation is fixed. Why? Because, the chess engine simply picks the right move everytime from the endgame database. It is already a finished exact science, not even Houdini with EGTB can change the outcome favorably.

    The reason why an engine without EGTB will be lost in a draw position or draw a winning position in the endgame, is because the engine's code is not yet mature enough in terms of endgame knowledge, or simply because an engine is just too weak.

    Don't be illusioned that tweaking the EGTB parameters can influence the outcome of the game.

    EGTB is just a tool to finish the endgame quickly.

    ReplyDelete
  3. Very well explained ! Thank You !
    Got some bad results today using the latest Stockfish in online chess... even against players I normally beat !
    On switching back to H3, the problems went away.
    Regardless of the good results in the TCES tournament, I feel that Stockfish still has a long way to go before it can catch up with the amazing H3 Pro, in my opinion.

    ReplyDelete

Blog Archive

Chitika