The Week in Chess

Sunday, May 26, 2013

Stockfish 130525 Demolished Houdini 3 Pro Once Again!

Just a few days ago, I published a test that Stockfish 130520 defeated Houdini 3 Pro. There quickly followed a minor update which I thought was worthy of testing.

And so here we go again, Stockfish 130525 against probably the 3 strongest chess engines in the world, clearly outpointing everyone of them in the 100 round gauntlet.  Stockfish won by 11 points against Houdini 3, 16 points against Houdini 2 and 29 points against Critter 1.6a!

Here is the gaunlet result:

Rank Engine Raw Games Score% Points Win Loss Draw
1 Stockfish 130525 x64 45 300 59.33 178.0 127 71 102
2 Houdini 3 Pro x64 10 100 44.50 44.5 26 37 37
3 Houdini 2.0 Pro x64 -8 100 42.00 42.0 28 44 28
4 Critter 1.6a x64 -47 100 35.50 35.5 17 46 37
.
This is just a test produced in my testing environment to satisfy curiosity and may not be agreeable to everybody.

Download the supporting games matches here.

10 comments:

  1. The code of Firenzina 2.3.1 is now available at https://github.com/Censor/Firenzina/tree/Firenzina_2-3-1_xTreme
    I made a couple of 64-bit compiles available at http://firenzina.wikispaces.com/, but I remember that you have good reasons to compile your own executables. The changes made since 2.2.2 add approximately 20 Elo in my internal tests.

    ReplyDelete
  2. Thank you for the information. Yesterday, I got hold of the source code and 64 bit SSE4 which does not run in my computer. So I made a compile and pitted it against Fire 2.2 xTreme (#7 in my top 100) in a 100 round match. When I woke up this morning it was leading by 5 points with 80% of the games completed.

    The Firenzina 2.3.1 is now in queue. I am hurrying to finish the Stockfish 130525 matches against the 64bit strongest chess engines which has the potential to be number 1 in my rating list.

    ReplyDelete
  3. Thank you very much! Have a good Memorial Day!

    ReplyDelete
    Replies
    1. By the way, the 32bit compile is not successful. It emits a message of undeclared identifier "_MM_HINT_T2".
      I compile using VS2010 under Windows 7.

      Delete
    2. Thank you very much for this feedback! I'll look into this. In the meanwhile, I have added a couple of generic 64-bit Windows compiles.

      Delete
    3. Both the 2 generic 64 bit popcnt and nopopcnt compiles does not work in my computers.

      Maybe your environment is just too advanced that anything produced by it will not run in low-powered generic computers. My guess is that VS2012 is the culprit because this caused me annoying inconveniences before.

      I suggest you try to run your executables in a computer without VS2012 and Intel C13 to test for compatibility before releasing it to the public who are mostly using generic computers. That would help in the issues of credibility that may arise.

      Delete
  4. Yes, the new Stockfish versions are no doubt very strong. But I wonder if you have taken into consideration one important factor that may be handicapping Houdini 3.
    Robert Houdart, in an interview after the IPON Tournament in which Stockfish performed so impressively, clearly states that "In the Superfinal it’s possible that the default Contempt settings (which is relatively strong in Houdini) may have been responsible for a loss, but without it we might have seen even more draws."
    In other words, default Contempt 1 for Houdini is NOT the strongest, but rather Contempt 0 is the strongest ! As you know, the evaluation for both sides is absolutely accurate when Contempt 0 is used.
    In FACT, I can confirm that Houdini 3 with Contempt 0 is almost IMPOSSIBLE to beat...draws yes, but almost ZERO losses.
    I myself have drawn almost always using Contempt 0 against even players using 12-16 Cores ( dual-xeons) , while using 6 Cores myself.
    Now, the interesting fact is that Stockfish at default settings uses Contempt 0.
    So, basically,if you are using default settings, you are pitting a SLIGHTLY weakened Houdini 3 against a full-strength, solidly defensive Stockfish.
    What do you say, Mr John Wilson ?

    ReplyDelete
    Replies
    1. I think Robert Houdart is not stupid to set the contempt to 1 in Houdini without good reason.

      All the default settings in any chess engine are the author's choices and they should be responsible for it.

      They cannot blame anybody when it does not perform well in a particular computer configuration or tournament.

      Delete
  5. WOW ! Don't get angry, Mr John Wilson.
    Whatever Robert Houdart is, I doubt if he is stupid !
    But the fact remains, that for whatever reason, he himself has admitted that the default Contempt 1 is NOT the strongest.
    To be fair to him, his reasoning seems to be that default contempt 1 gives more positive results than Contempt 0, which certainly gives a lot of draws, ( which many may consider to be boring !).
    After all, he is selling a product and no doubt he would like to make it look as attractive as possible to attract buyers, not all of whom are professional Testers like you !
    I wonder if you could run your Gauntlet Tournament again, only this time setting Contempt 0 for Houdini 3, to match the default Contempt 0 of Stockfish.
    The results would be enlightening ! :)

    ReplyDelete
  6. I will try your suggestion after I am through with the Stockfish 130525 matches against the strongest engines which will be posted in the rating list maybe tomorrow.

    ReplyDelete

Chessdom News