The Week in Chess

Monday, April 1, 2013

Stockfish 2.3.1 vs. Firenzina 2.2.2 xTreme - 64bit Battle

Who is better, Stockfish or Firenzina?

The answer:  it depends on the terrain.

Last week I posted that Fire 2.2 won by a big margin over Stockfish 2.3.1 when both were compiled with the same compiler and build parameters in 32bit mode. I attempted to have a rematch for them, this time with 64bit compilation.  The first 20 games phase of the match showed Fire leading comfortably, but then my computer showed signs of abnormal performance, which made me remove the 2 other tournaments running under Arena leaving only Fire vs. Stockfish, to diagnose the perceived problem. As I observed, I noticed that Stockfish is steadily winning games and saw that the KN/sec speed of Stockfish is on par with Fire which was lower before. To confirm my suspicion, I restarted the other stopped tournaments and saw that Fire is steadily regaining the lost games and at the same time the KN/sec is higher in Fire's. Then my computer crashed hard...

To verify what I just observed, I ran the tournament of Fire vs. Stockfish in a separate computer running with AMD dual core 2800ghz, clearing all the other programs.  My suspicion was confirmed that Stockfish is better in conditions when there are no other major processes that will hog the CPU processor.

I made another compilation this time with Firenzina 2.2.2 xTreme in 64bit which was intended to be used in my rating tournament because the author has no 64bit version that will run in my computers.  This was an opportunity to test it against the formidable Stockfish 2.3.1 stable version compiled by Jim Ablett. It is an Intel vs Intel C++ Compiler classic battle which would provide some clue on whether Stockfish is supreme in the open-source chess engines.  I ran the tournament in my restored computer that crashed, an AMD Quad Core which also has other 2 tournaments running simultaneously.  The result was a 55-45 win for Firenzina which was hurriedly posted to chess2u.com forum to update my posts there, then went home to rest.  The following day, I noticed that I picked the wrong version of Stockfish which was a 32bit compilation.  With apology, I restarted a rematch to correct the blunder, this time in two separate computers with the one running in the same AMD Quad Core with 3 simultaneous tournaments and the other in an AMD Dual Core with only 1 tournament running.

Here is the scoreboard:

Engine AMD 4 Core AMD 2 Core Total Points Ave%
Stockfish 2.3.1 x64-ja 26 58.5 84.5 42.25
Firenzina 2.2.2 xTreme x64-owl 74 41.5 115.5 57.75


From the results, it could be said that Firenzina is the better engine. But, it depends on what environment the tournament is run and probably the factor of the referee.

This is something for the Stockfish Team to verify and counter-check.  The main issue is that Stockfish seems to starve when there are lots of simultaneous threads running.

And the final answer of who is better?  Oops, the referee is gone...

Download the Firenzina 2.2.2 x64 owl version used in the match here.

Download the tournament results here.

6 comments:

  1. Firenzina is also by me - for the most part Firenzina 2.3 xTreme focuses on Linux compatibility (I have an 8-core machine, and use Linux).

    We also plan to tune the piece values somewhat.

    Either way, don't expect an Elo increase!

    Matthew:out

    ReplyDelete
    Replies
    1. Just a week ago, I experimented with the piece values which was found to perform best with Houdini that I posted previously at chess2u forum. Then I organized a 100 round-robin with the optimized values for Fire xTreme I found in the Internet such as Banzi, Inferno, SK94z and the original Fire.cfg. To my disappointment, my piece values placed last and the top performer was still Fire xTreme. This made me conclude that nothing significant will be gained from just changing the piece values or other parameters exposed by the chess engine. Robert Houdart of Houdini, said that the best way to increase the strength of the chess engine is through superior algorithm, and his proof is the fantastic strength of Houdini 3.

      By the way, why is it that you don't release Firenzina executables, particularly 64-bit compilation that would run in any machine, whether it be Intel or AMD. I was forced to compile a Firenzina 64-bit so that I could use it in my ongoing tournament. I am not sure if it is better than yours because there is no way to compare it, and I am afraid that the results might decrease the ranking of Firenzina in my Chess Engines ELO rating list.

      Delete
    2. I thought there were sufficiently many compiles at http://dgusev.cs.edinboro.edu/GameDev/Chess/firenzina.html to cover the modern Windows platform (XP and up) for both 32-bit and 64-bit "incarnations".

      Delete
    3. Yes, there were so many compiles for Firenzina but only the 32bit generic version worked in my computers. I downloaded the files in your site several times hoping that a new working executable especially the 64 bit could run successfully, but I'm out of luck. My computers are AMD CPU's multi-core all run with Windows XP SP2 64 bit, virtualized under PCLinuxOS. Any chess engine compiled with SSE4 or higher is not compatible with my CPU's. I suspect that you compiled most of your chess engines with Windows 7 and Visual Studio 2012 which probably is the issue. I tried Visual Studio 2012 in my XP's but will not run because it needs Windows 7.

      Delete
    4. John, it looks like you are correct, so you did the right thing. I simply don't have a 64-bit computer running XP, so I was unaware of the problem.

      Delete
    5. I hope that you can provide a 64 bit compilation in your next version release because my tournament conditions may not change at the moment.

      Delete

Chessdom News