The Week in Chess

Wednesday, January 22, 2014

GrandMonsters SuperBlitz Tournament

Immediately after the Stockfish 140119 x64 Gauntlet Matches, I conducted a follow-up test tournament with the twin purpose of determining whether the latest Arena GUI 3.5 software can reliably handle a tournament with sub-second moves and to know whether Houdini is really the King in Blitz chess engine tournaments at present.

The previous Arena GUI 3.0 version was not very good at very fast time controls below 1 second per move with so many losses by time forfeit. This probably prevented other testers from using it seriously in producing rating lists at super blitz.

Houdini is very well known as the number one chess engine in most rating list websites and has the reputation to be the best in very short time controls. However, Houdini's reputation was blemished when it was not a finalist at the recent TCEC Live Chess Engines Tournament, and some testers reported in the forums that Houdini is no longer the number one in blitz tournaments including this tester.

To make the tournament fair, I selected the other 2 top chess engines like Stockfish and Komodo to battle against Houdini in the competition.  Arena GUI 3.5 was chosen for one of the reasons above which feature a common setting for the number of CPU cores and the Hash table memory size.  The computer used was a generic build with Dual Core AMD processor and 4 GB of memory. 128MB of memory was allocated for the hash table. The time control was 25 seconds base + 250 milliseconds increment. The 3 chess engines face each other in a round robin of 100 games each. There were no other major programs running simultaneously and there is only one instance of the tournament so all the computer resources are available for the chess engines competition.

The tournament was started just before I went to sleep in the early morning, and when I woke up I had these results:

SuperBlitz Tournament - 25S+250MS
RankEngineScoreStHoKoS-B
1Stockfish 140119 x64119.5/200· ·· ·· ··39-23-3847-24-29 10728.00 
2Houdini 4 Pro x64 106.5/20023-39-38· ·· ·· ··47-18-35 9792.00 
3Komodo TCEC x64 74.0/20024-47-2918-47-35· ·· ·· ·· 8381.50 


300 games played / Tournament finished

Tournament start: 2014.01.22, 01:15:38
Latest update: 2014.01.22, 10:43:48
Site/ Country: CYBIRD-PC2, United States
Level: Blitz 0:25/0.25
Hardware: AMD Athlon(tm) 64 X2 Dual Core Processor 5000+ with 1.8 GB Memory
Operating system: Windows 7 Ultimate Professional Service Pack 1 (Build 7601) 64 bit
Table created with: Arena 3.5

The result table above mirrors the ranking of the finishers with the latest rating list.  Also, the scores are very close to the score of the last gauntlet matches of 1 minute base + 1 second increment. Stockfish was the clear winner with +39-23=38 score against Houdini and +47-24=29 score against Komodo. Houdini won against Komodo with +47-18=35.

As for the reliability of Arena GUI 3.5, the resulting PGN file showed only 1 loss by time forfeit which was caused by Houdini. It is well known that Houdini has a time management problem as posted in many forums  and even at the TCEC tournament where it crashed. I can say with delight that Arena GUI 3.5 is now very usable in  the ultra fast tournament time controls. What is nice about this is that I was able to watch the tournament comfortably following the moves, unlike Little Blitzer or CuteChess which show only the statistics but not the the actual game in progress.

The choice of fast or slow time controls is irrelevant in the production of rating  list because the ranking of the chess engines are always in the proper place with minor deviation which can be observed by comparing the rating list of sites with different time controls. Fast time controls are nice to produce quick results, while long time controls are good to produce quality games in PGN.

Conclusion:
Arena GUI 3.5 is now very good for fast time controls while Houdini's dominance in the number 1 position in fast blitz time control is over.

Download the test tournament results here.

No comments:

Post a Comment

Chessdom News