Tag Archives: Stockfish

Game Changer?

This review has been printed in the April 2019 issue of Chess Life.  A penultimate (and unedited) version of the review is reproduced here. Minor differences exist between this and the printed version. My thanks to the good folks at Chess Life for allowing me to do so.

—–

Sadler, Matthew, and Natasha Regan. Game Changer: AlphaZero’s Groundbreaking Chess Strategies and the Promise of AI. Alkmaar: New in Chess, 2019. ISBN 978-9056918187. PB 416pp.

Sigmund Freud once described the “three severe blows” suffered by human narcissism in the course of Western history.[1] The cosmological blow, struck by Copernicus, expelled us from our supposed place at the center of the universe. Darwin’s biological blow denied us the comfort of our separation from, and superiority over, the animal kingdom. And Freud himself landed the final, psychological blow, exposing the irrational unconscious forces beneath even the greatest achievements of human rationality.

To these three psychic wounds chess players can add a fourth: Garry Kasparov’s defeat at the hands of Deep Blue in 1997. Deep Blue’s victory was portrayed in the mass media as a referendum on human intelligence, a ‘canary in the coalmine’ moment in which the inevitable overtaking of human creativity by machine intelligence was made manifest.

Curious thing, though. What was imagined as an antagonistic relationship between man and machine has instead proven to be a constructive one. Sure, humans have given up trying to beat Stockfish or Komodo, even at odds, but our ‘metal friends’ (Tukmakov’s delightful turn of phrase) are now our trusted analytical partners and teachers.

Far from killing our game, chess in the e-sport era now depends on the presence of engines, which play the role of the hole-cam in the poker boom. They give the illusion of prescience, allowing amateurs the heady feeling that they know more than the players themselves.

So imagine the shock when a scientific pre-print appeared on the Internet in December 2017. Its title, “Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm,” was anodyne enough, but the paper announced a seismic shift in artificial intelligence and chess. AlphaZero, a program created by a Google subsidiary known as DeepMind, trounced Stockfish in head-to-head play. In doing so it forced us to rethink everything we know about computer chess.

The principles governing Stockfish’s play are not fundamentally different than those guiding Deep Blue, although they have been profoundly refined in the intervening years. Stockfish uses human-tuned criteria to evaluate each position in its search tree, and through “alpha-beta” search methods it is able to focus on promising continuations while pruning away inferior moves. Each move and each decision are the result of precise mathematical calculations, and human users can extract exact numerical evaluations for any given position.

AlphaZero is different, as Matthew Sadler and Natasha Regan lucidly explain in their new book, Game Changer: AlphaZero’s Groundbreaking Chess Strategies and the Promise of AI. Pre-programmed with only the basic rules of chess, and using general (non-specific) self-training algorithm, AlphaZero trained itself to play chess over the course of nine hours and 44 million self-play games. Periodically the program would refine its neural network, promoting tunable weights and network ‘layers’ that led to favorable outcomes, and demoting those that didn’t.

AlphaZero functions by combining these self-taught evaluative values with a Monte Carlo style tree search, where possible future game positions are spun out, evaluated, and ranked probabilistically. We don’t know exactly how AlphaZero decides what to play. The algorithm is a ‘black box’ in the Latourian sense, where inputs and outputs are known but (in contrast to Stockfish) its internal mechanisms remain opaque, even to DeepMind. What we do know is that AlphaZero is immensely, improbably strong, exhibiting an attractive attacking style reminiscent of Kasparov.

Perhaps this is what makes AlphaZero so remarkable – its style. What we see in its victories over Stockfish should, given all we know about computer chess, be impossible. Stockfish is typically seen as a calculative god and defensive wizard, able to soak up pressure, induce errors, and grind down its opponents. AlphaZero defeated it by playing the kinds of attacking, sacrificial ideas that, played by humans, would inevitably be refuted by the machine.

Sadler and Regan spend two chapters of Game Changer describing the technical aspects of AlphaZero’s self-training regiment, the way it “thinks,” and what its evaluations and expected scores mean. Their extensive access to the DeepMind team and the algorithm allow them to craft accessible explanations of difficult subjects, and the mini-interviews with DeepMind team members are helpful.

The meat of the book, however, focuses squarely on AlphaZero’s style. What makes it so good? How can we reverse-engineer the logic of its moves and apply that knowledge to our own games? By studying the roughly 230 publicly available AlphaZero games, along with approxmiately 2100 additional games provided by DeepMind, Sadler and Regan distill a number of tantalizing traits in AlphaZero’s play.

An example is useful. Consider this game, which Sadler describes as “perhaps AlphaZero’s most beautiful game of all.”[2]

NIMZO-ENGLISH (A17)
AlphaZero
Stockfish 8
AlphaZero v. Stockfish Match, 2017

1.Nf3 Nf6 2.c4 e6 3.Nc3 Bb4 4.Qc2 0–0 5.a3 Bxc3 6.Qxc3 a5 7.b4 d6 8.e3 Ne4 9.Qc2 Ng5 10.b5 Nxf3+ 11.gxf3 Qf6 12.d4!?

Sadler and Regan expected 12.Bb2 Qxf3 13.Rg1 but AlphaZero instead plays for long-term compensation.

12. … Qxf3 13.Rg1 Nd7 14.Be2 Qf6 15.Bb2 Qh4 16.Rg4!?

Giving up the h-pawn to open the file. Stockfish sees this position as better for Black, while AlphaZero thought that White had a slight advantage.

16. … Qxh2 17.Rg3 f5 18.0–0–0

Offering pawn number three!

18. … Rf7

After 18. … Qxf2 19.Rdg1 Rf7 20.R1g2 Qe1+ 21.Bd1 White’s compensation is undeniable.

19.Bf3 Qh4 20.Rh1 Qf6

image

What does AlphaZero have for the two pawns? Two half-open files and massively superior mobility. This is a key idea for Sadler and Regan. As they explained in a conference call for chess journalists – the first such promotional call I’ve been on for a chess book! – the concept of mobility is fundamental for understanding how AlphaZero plays. It works to maximize the mobility of its pieces and minimize the mobility of its opponent’s. One of AlphaZero’s most striking tendencies, the pushing of its rook pawns to restrict the opponent’s king, is emblematic in this regard. Here, having opened lines for its rooks, AlphaZero now proceeds to open diagonals and further increase its mobility.

21.Kb1 g6 22.Rgg1!? a4 23.Ka1 Rg7 24.e4 f4 25.c5 Qe7 26.Rc1 Nf6 27.e5 dxe5 28.Rhe1 e4 29.Bxe4 Qf8

This is a key position in both the game and the book. Sadler and Regan use it to illustrate AlphaZero’s “thought processes” in Chapter 4.

30.d5!

AlphaZero sacrifices another pawn to open the a1–h8 diagonal!

30. … exd5 31.Bd3! Bg4 32.f3 Bd7

White’s initative grows after 32. … Bxf3? 33.Rf1 Be4 34.Rxf4.

33.Qc3 Nh5 34.Re5

AlphaZero rates its winning chances at 80.3% here. (It evaluates positions by win percentage in Monte Carlo game rollouts.) Stockfish 8 thinks White is significantly better, but newer versions of the engine more clearly understand the danger.

34. … c6 35.Rce1 Nf6 36.Qd4 cxb5 37.Bb1 Bc6 38.Re6 Rf7

Stockfish hopes to return some of its material advantage and weather the storm. AlphaZero does not oblige.

39.Rg1 Qg7 40.Qxf4 Re8 41.Rd6 Nd7 42.Qc1 Rf6 43.f4! Qe7 44.Rxf6 Nxf6 45.f5 Qe3 46.fxg6 Qxc1 47.gxh7+ Kf7 48.Rxc1 Nxh7 49.Bxh7 Re3 50.Rd1 Ke8 51.Ka2 Bd7 52.Bd4 Rh3 53.Bc2 Be6 54.Re1 Kd7 55.Kb2 Rf3 56.Re5 Rg3 57.Re3 Rg2 58.Kc3 Rg4 59.Rf3 Ke8 60.Rf2 Rg3+ 61.Kb4 Rg4 62.Rd2 Bd7 63.Ka5 Rf4 64.Be5 Rf3 65.Rd3 Rf2 66.Bd1 Bc6 67.Kb6 1–0

One can’t help but feel as if a superior, alien intelligence has taken the White pieces and opened a new vista on to our beloved game.

Part III of Game Changer brilliantly distills some of the key features of AlphaZero’s attacking prowess. We see, through detailed analysis and clear explanation, how AlphaZero values outposts, why it rams ‘Harry the h-pawn’ forward, how it plays on color complexes and sacrifices for what Kasparov called quality. Part IV, devoted to AlphaZero’s opening choices, is less successful. The authors laud AlphaZero’s novel handling of the White side of the Carlsbad structure, for instance, but the game they cite departs from theory on the sixth move, rendering much of the fine preparatory explanation useless.

Game Changer is an excellent book, fully deserving of the critical praise it has received. Sadler and Regan patiently explain the technical minutia for a non-technical audience, and their attempts to divine the essence of AlphaZero’s style are clear and convincing. Until DeepMind succeeds in “recovering back” AlphaZero’s implicit heuristics through some secondary algorithm, this treatment is as good it gets.

What remains less settled, at least in my mind, is the issue of the book’s title. Is AlphaZero really a game changer? Does its advent herald a revolution in chess?

DeepMind’s novel computational solution – AlphaZero’s self-learned strength and style – is as disruptive today as Deep Blue’s brute force approach was in 1997. Both reconfigured our understanding the possibilities of computer chess and, truth be told, of chess itself.

This, unfortunately, does not exhaust the two project’s similarities. AlphaZero seems doomed to a life behind corporate bars much like its august predecessor, hidden away from the public in the interest of protecting trade secrets. And as with Deep Blue, AlphaZero’s influence on chess will be as a consequence be limited.

I suspect that the real game changer will be Leela Chess, an open-source project that mimics AlphaZero’s self-learning algorithm. Because it is open-source, like the now ubiquitous Stockfish, Leela can be used by anyone without cost. Players can train with Leela, use it to analyze their games, and test their ideas against it. The democratization of chess information that began with Robert Hyatt’s Crafty, Mark Crowther’s The Week in Chess, and Stockfish continues with Leela, and the chess world will be much the richer for it.


[1] See Freud’s Compete Psychological Works (Standard edition, ed. Strachey), volume 17, p.139-141.

[2] https://www.youtube.com/watch?v=0RuIHfNcPO0

And Then There Were Two

Komodo 9, written by Don Dailey, Larry Kaufman and Mark Lefler. Available (1) with Fritz GUI from Amazon ($80ish as of 5/28), (2) for download with Fritz GUI from ChessBase.com ($73.50 w/o VAT as of 5/28) and (3) directly from the Komodo website without GUI for $59.98; also available as part of a 1 year subscription package for $99.97.

Stockfish 6, written by the Stockfish Collective. Open-source and available at the Stockfish website.

—–

Now that Houdini seems to have gone gentle into that good night, there are two engines vying for the title of strongest chess engine in the world. Those two engines – Stockfish and Komodo – have each seen new releases in recent months. Stockfish 6 was released at the end of January, while Komodo 9 became available at the end of April from komodochess.com and the end of May from ChessBase.

Last year I wrote a review of Komodo 8 and Stockfish 5 that was republished at ChessBase.com, and much of what I wrote there applies here as well. Fear not, frazzled reader: you don’t need to go back and read that review, as most of the key points will be reiterated here.

First things first: any top engine (Komodo, Stockfish, Houdini, Rybka, Fritz, Hiarcs, Junior, Chiron, Critter, Equinox, Gull, Fire, Crafty, among many others) is plenty strong to beat any human player alive. This is not because each of these engines are equally strong. While they don’t always play the absolute best moves, none of the aforementioned engines ever make big mistakes. Against fallible humans, that’s a recipe for domination. It’s nearly useless – not to mention soul-crushing! – to play full games against the top engines, although I do recommend using weaker engines (Clueless 1.4, Monarch, Piranha) as sparring partners for playing out positions or endgames.

Even if all the major engines can beat us, they’re not all created equal. Three major testing outfits – CCRL, CEGT, and IPON – engage in ongoing and extensive testing of all the best engines, and they do so by having the engines play thousands of games against one another at various time controls. In my previous review I noted that Komodo, Stockfish and Houdini were the top three engines on the lists, and in that order. This remains the case after the release of Komodo 9 and Stockfish 6:

CCRL (TC 40 moves/40 min, 4-cpu computers):
1. Komodo 9, 3325 (Komodo 8 was rated 3301)
2. Stockfish 6, 3310 (Stockfish 5 was rated 3285)
3. Houdini 4, 3269

CEGT
40/4: 1. Komodo 9, 2. Stockfish 6, 3. Houdini 4
G/5’+3”: 1. Komodo 9, 2. Stockfish 6, 3. Houdini 4
40/20: 1. Komodo 9, 2. Stockfish 6, 3. Houdini 4 (NB: list includes multiple versions of each engine)
40/120: 1. Stockfish 6, 2. Komodo 8 (does not yet include version 9), 3. Houdini 4 (NB””: list includes multiple versions of each engine)

IPON
1. Komodo 9, 3190 (Komodo 8 was 3142)
2. Stockfish 6, 3174 (Stockfish 4 was 3142)
3. Houdini 4, 3118

The results are fairly clear. Komodo 9 is ever so slightly stronger than Stockfish 6 when it comes to engine-engine play, and this advantage seems to grow when longer time controls are used.

For my purposes, though, what’s important is an engine’s analytical strength. This strength is indicated by engine-engine matches, in part, but it is also assessed through test suites and – perhaps most importantly – by experience. Some engines might be more trustworthy in specific types of positions than others or exhibit other misunderstandings. Erik Kislik, for instance, reports in his April 2015 Chess Life article on the TCEC Finals – some of which appeared in his earlier Chessdom piece on TCEC Season 6 – that only Komodo properly understood the imbalance of three minor pieces against a queen. There are undoubtedly other quirks known to strong players who use engines on a daily basis.

In my previous review I ran Komodo, Stockfish and Houdini (among others) through two test suites on my old Q8300. Since then I’ve upgraded my hardware, and now I’m using an i7-4790 with 12gb of RAM and an SSD for the important five and six-man Syzygy tablebases included with ChessBase’s Endgame Turbo 4. (Note: if you have an old-fashioned hard drive, only use the five-man tbs in your search; if you use the six-man, it will slow the engine analysis down dramatically.) Because I have faster hardware I thought that a more difficult test suite would be in order, and – lucky me! – just such a suite was recently made available in the TalkChess forums. I gave Komodo 9 and Stockfish 6 one minute per problem to solve the 112 problems in the suite, and the results were as follows:

Komodo 9 solved 37 out 110 problems (33.6%) with an average time/depth of 20.04 seconds and 24.24 ply. Stockfish 6 solved 30/110 (27.2%) with an average time/depth of 20.90 seconds and 29.70 ply. (Note that while there are 112 problems in the suite, two of them were rejected by both engines because they had incomplete data.) The entire test suite along with embedded results can be found at:

http://www.viewchess.com/cbreader/2015/6/6/Game1753083657.html

I have also been using both Komodo 9 and Stockfish 6 in my analytical work and study. So that you might also get a feeling for how each evaluates typical positions, I recorded a video of the two at work.  Each engine ran simultaneously (2 cpus, 2gb of RAM) as I looked at a few games of interest, most of which came from Alexander Baburin’s outstanding e-magazine Chess Today. The video is 14 minutes long. You can replay the games at this link:

http://www.viewchess.com/cbreader/2015/6/6/Game1752975735.html

Komodo 9 and Stockfish 6 in comparative analysis

Even a brief glance at the above video will make clear just how good top engines are becoming in their ability to correctly assess positions, but it also shows (in Gusev-Averbakh) that they are far from perfect. They rarely agree fully in positions that are not clear wins or draws, and this is due to the differences in evaluation and search between the two. Broadly speaking, we can say that evaluation is the criteria or heuristics used by each engine to ‘understand’ a position, while search is the way that the engine ‘prunes’ the tree of analysis. While many engines might carry similar traits in their evaluation or search, none are identical, and this produces the differences in play and analysis between them.

Stockfish 6 is a rather deep searcher. It achieves these depths through aggressive pruning of the tree of analysis. While there are real advantages to this strategy, not the least of which is quick analytical sight and tactical ingenuity, there are some drawbacks. Stockfish can miss some resources hidden very deep in the position. I find it to be a particularly strong endgame analyst, in part because it now reads Syzygy tablebases and refers to them in its search. Stockfish is an open-source program, meaning that it is free to download and that anyone can contribute a patch, but all changes to evaluation or search are tested on a distributed network of computers (“Fishtest”) to determine their value.

Komodo 9 is slightly more aggressive in its pruning than is Komodo 8, and it is slightly faster in its search as well. (Both changes seem to have been made, to some degree, with the goal of more closely matching Stockfish’s speed – an interesting commercial decision.) While Komodo’s evaluation is, in part, automatically tuned through automated testing, it is also hand-tuned (to what degree I cannot say) by GM Larry Kaufman.

The result is an engine that feels – I know this sounds funny, but it’s true – smart. It seems slightly more attuned to positional nuances than its competitors, and as all the top engines are tactical monsters, even a slight positional superiority can be important.  I have noticed that Komodo is particularly good at evaluating positions where material imbalances exist, although I cannot say exactly why this is the case!

As more users possess multi-core systems, the question of scaling – how well an engine is able to make use of those multiple cores – becomes increasingly important. Because it requires some CPU cycles to hand out different tasks to the processors in use, and because some analysis will inevitably be duplicated on multiple CPUs, there is not a linear relation between number of CPUs and analytical speed.

Komodo 8 was reputedly much better than Stockfish 5 in its implementation of parallel search, but recent tests published on the Talkchess forum suggest that the gap is narrowing. While Stockfish 6 sees an effective speedup of 3.6x as it goes from 1 to 8 cores, Komodo 9’s speedup is about 4.5x. And the gap is further narrowed if we consider the developmental versions of Stockfish, where the speedup is now around 4x.

Hardcore engine enthusiasts have, as the above suggests, become accustomed to downloading developmental versions of Stockfish. In an effort to serve some of the same market share, the authors of Komodo have created a subscription service that provides developmental versions of Komodo to users. This subscription, which costs $99.97, entitles users to all official versions of Komodo released in the following year along with developmental versions on a schedule to be determined. Only those who order Komodo directly from the authors are currently able to choose this subscription option.

The inevitable question remains: which engine should you choose? My answer is the same now as it was in my previous review. You should choose both – and perhaps more.

Both Komodo and Stockfish are insanely strong engines. There remain some positions, however, where one engine will get ‘stuck’ or otherwise prove unable to discern realistic (i.e. human) looking moves for both sides. In that case it is useful to query another engine to get a second (or perhaps even third) opinion. I find myself using Komodo 9 more than Stockfish 6 in my day-to-day work, but your mileage may well vary. Serious analysts, no matter their preference, will want to have both Komodo 9 and Stockfish 6 as part of their ‘teams.’

Choosing a Chess Engine

Note: This review has been updated as of 9/24 to reflect my testing and experience with the newly released Komodo 8.

———

Houdini 4, written by Robert Houdart. Standard (up to six cpu cores, $79.95 list) and Pro (up to 32 cpu cores, $99.95 list) versions with Fritz GUIs available. Also available directly from the Houdini website for approximately $52 (Standard) or $78 (Pro) as of 9/11/14.

Komodo 7a, written by Don Dailey, Larry Kaufman and Mark Lefler. Available directly from the Komodo website for $39.95.

Komodo 8, written by Don Dailey, Larry Kaufman and Mark Lefler. Available (1) with Fritz GUI ($97ish as of 9/24) and (2) directly from the Komodo website without GUI for $59.96

Stockfish 5, written by the Stockfish Collective. Open-source and available at the Stockfish website.

Increasingly I’m convinced that a serious chess player must make use of chess technology to fully harness his or her abilities. This, as I have previously discussed, involves three elements: the GUI, the data, and the engine. ChessBase 12 is the gold standard for chess GUIs, and I will be reviewing a new book about proper use of that GUI in the near future. Here, however, I want to take up the thorny issue of choosing a chess engine. Which engine is ‘best’ for the practical player to use in his or her studies?

I put ‘best’ in scare-quotes because there are two ways to look at this question. (1) There is little question at this point that the best chess engines of the past five years can beat 99.9% of human players on modern hardware. So one way that engines are tested now is in a series of engine vs engine battles. While many people process private matches, there are three main public rating lists: IPON, CCRL and CEGT.

Here there is something of a consensus. Houdini, Stockfish and Komodo are the three top engines at the moment, with very little differentiating between them, and with the particular order of the engines varying due to time control and other criteria.

Update: The three lists mentioned above have tested Komodo 8.

  • It is in first place on the IPON list, leading Stockfish 5 by 6 elo points and Houdini 4 by 17.
  • Komodo 8 appears on two of the CCRL lists. In games played at a rate of 40 moves in 4 minutes (40/4), Stockfish 5 leads Komodo 8 by 7 elo points and Houdini 4 by 30 elo points. In games played at the slower rate of 40 moves in 40 minutes (40/40), Komodo 8 has a 22 elo point lead on Stockfish 5 and a 39 point lead on Houdini.
  • Among the many CEGT lists, we find: (a) Stockfish 5 is first on the 40/4 list, followed by Komodo 8 and Houdini 4; (b) Houdini 4 leads the 5’+3″ list, followed by Stockfish 5 and Komodo 8; (c) Komodo 8 leads the 40/20 list followed by Stockfish 5 and Houdini 4; but (d) the 40/120 list has not yet been updated to include Komodo 8.
  • Note: Larry Kaufman compiles the results from these lists and one other in a thread at Talkchess. He argues (a) that Komodo does better at longer time controls, and that (b)  Komodo 8 is roughly equal in strength to the Stockfish development releases, which are slightly stronger than the officially-released Stockfish 5. </update>

From my perspective, however, (2) analytical strength is more important. If all the engines are strong enough to beat me, I think that the quality of their analysis – the ‘humanness’, for lack of a better word – is critical. It used to be the case that humans could trick engines with locked pawn chains, for example, or that engines would fail to understand long-term compensation for exchange sacrifices. Such failings have largely been overcome as the engines and hardware have improved; nevertheless, there remain certain openings and types of positions that are more problematic for our metal friends. Michael Ayton offers one such position in the ChessPub forums; if you want have a laugh, check out the best lines of play on offer by the engines reviewed here:

Screenshot 2014-09-11 12.33.12

FEN: r1b2rk1/pp1nqpbp/3p1np1/2pPp3/2P1P3/2N1BN2/PP2BPPP/R2Q1RK1 w – c6 0 10

Among the multiple engines available, there are three that stand above the fray. These are Houdini by Robert Houdart, Komodo by the late Don Dailey, Larry Kaufman and Mark Lefler, and Stockfish. Houdini and Komodo are commercial engines, while Stockfish is open-source and maintained by dozens of contributors.

How can we understand the differences between the engines? Let’s consider two key components of chess analysis: search and evaluation. Search is the way that the engine ‘prunes’ the tree of analysis; because each ply (move by White or Black) grows the list of possible moves exponentially, modern engines trim that list dramatically to obtain greater search depth. Evaluation is the set of criteria used by the engine to decipher or evaluate each position encountered during the search.

In a very general sense, what differentiates Houdini, Komodo and Stockfish are their search and evaluation functions. How they are different on a technical / programming level, I cannot say: Houdini and Komodo are closed-source and I can’t decipher code in any event. What I can do, however, is cite what some experts in the field have said, and then see if it coheres with my experience of the three engines.

Larry Kaufman, who works on Komodo, said in an interview on the Quality Chess blog that:

Komodo is best at evaluating middlegame positions accurately once the tactics are resolved. Stockfish seems to be best in the endgame and in seeing very deep tactics. Houdini is the best at blitz and at seeing tactics quickly. Rybka is just obsolete; I like to think of Komodo as its spiritual desceendant, since I worked on the evaluation for both, although the rest of the engines are not similar. Fritz is just too far below these top engines to be useful.

…Komodo’s assessment of positions is its strong point relative to the other top two, Houdini best for tactics, Stockfish for endgames and whenever great depth is required. Both Houdini and Stockfish overvalue the queen, Komodo has the best sense for relative piece values I think. Komodo is also best at playing the opening when out of book very early.

Stockfish is, as Kaufman suggests, very aggressive in the way that it prunes the tree of analysis, searching very deeply but narrowing as the ply go forward. It is important to remember that each engine reports search depth and evaluation differently, so that (as Erik Kislik writes in a fascinating article on the recent TCEC superfinal) the way that Stockfish ‘razors’ the search means that its reported depth can’t be directly compared to Houdini or Komodo. Still, it does seem to search more deeply, if narrowly, than do its competitors.  This has advantages in the endgame and in some tactical positions.

Houdini is a tactical juggernaut. It tends to do best on the various tactical test sets that some engine experts have put together, and it is fairly quick to see those tactics, making it useful for a quick analysis of most positions. Its numerical evaluations also differ from other engines in that they are calibrated to specific predicted outcomes.

A +1.00 pawn advantage gives a 80% chance of winning the game against an equal opponent at blitz time control. At +2.00 the engine will win 95% of the time, and at +3.00 about 99% of the time. If the advantage is +0.50, expect to win nearly 50% of the time. (from the Houdini website)

Kaufman argues that his engine, Komodo, is the most positionally accurate of the three, and I don’t disagree. Kaufman is involved in the tuning of Komodo’s evaluation function; as he is a grandmaster, it does not seem outrageous to believe that his engine’s positional play might benefit from his chess expertise. The engine is slightly ‘slower’ (anecdotally, and not judging by NPS, or nodes per second, and ply count) than are Stockfish and Houdini, but Komodo seems to benefit more from longer analysis time than do Houdini or Stockfish.

I’ve been using Komodo 8 in the Fritz GUI from ChessBase for a few days now. The GUI is the same as the Houdini 4 and the Deep Fritz 14 GUIs; in fact, when you install Komodo 8, I think it just adds some configuration files to your ChessProgram14 folder to allow for a Komodo ‘skin’ to appear. The Komodo 8 engine is slightly faster than 7a judging solely by NPS. While coding changes mean that the two can’t be directly compared, Mark Lefler has said that 8 is approximately 9% faster than 7a. The ChessBase package comes with a 1.5 million game database, an opening book, and a six month Premium membership at Playchess.com; all are standard for Fritz GUI releases such as Deep Fritz 14 or Houdini 4.

From my perspective, I tend to use all three engines as I study chess or check analysis for review purposes, but two more than the third. When I look at my games, which aren’t all that complex, I generally use Houdini as my default kibitzer. It seems to be the fastest at seeing basic tactical problems, and its quickness is a plus on some of my antiquated computers. I also tend to bring Komodo into the mix, especially if I want to spend some time trying to figure out one position. Stockfish serves more as a second (or third) option, but I will use it more heavily in endgame positions – unless we get into tablebase territory, as Stockfish does not (generally) use them.

*Note:* for other perspectives on the ‘personalities’ of these three engines, you might consider a couple of threads at the indispensible ChessPub forum.

As I was working on this review, I thought that I might try to ‘objectively’ test the engines on positions that were more positional or prophylactic in nature, or perhaps in some difficult endgame positions. I took 11 positions from books on hand, including a number from Aagaard’s GM Preparation series, and created a small test suite. Each engine (including Deep Fritz 14 for comparison’s sake) had 4 minutes to solve each problem on my old quad-core Q8300, and each engine had 512mb of RAM and access to Syzygy (5-man) or Nalimov (selected 6-man) tablebases as they preferred. You can see the results at the following link:

http://www.viewchess.com/cbreader/2014/9/24/Game31750181.html

or as summarized below:

First test set

Deep Fritz 14, curiously enough, solved more problems than did Houdini 4, Komodo 7a/8 or Stockfish 5. None could solve the famous Shirov …Bh3 ending. None could solve the Polugaevsky endgame, which illustrates a horizon-related weakness still endemic among even the best engines. Only Komodo 7a, Komodo 8 and Deep Fritz 14 solved position #2, which I thought was the most purely positional test among the bunch. This test is only anecdotal, and perhaps the engines would have gotten more answers right on faster hardware; nevertheless, I was a little surprised.

Test #2: Jon Dart (author of Arasan) has created a series of test suites to torture his engine and others. I took the first 50 problems from the Arasan Testsuite 17 and ran Houdini 4, the two Komodos, Stockfish 5, Deep Rybka 4.1 and Deep Fritz 14 through their paces. (I would have added Crafty 23.08, installed with Komodo 8, but it kept crashing the GUI when I tried to include it in the test.) Here the engines only received 60 seconds to solve the problem – the same standard Dart uses in his tests of Arasan, albeit with a much faster computer. You can see the results at the following link:

http://www.viewchess.com/cbreader/2014/9/24/Game31858867.html

or as summarized below:

Arasan test set

Stockfish 5 and Houdini 4 each solved 38/50 problems in the one minute time limit. Komodo 8 solved 30 problems, improving by one over Komodo 7a’s 29 solved problems, and doing so with a faster average solving time. Deep Rybka and Deep Fritz each solved 28 problems correctly. Given the shorter ‘time control’ and the relatively tactical nature (IMHO) of the test set, these results seem representative of the various engines and their characteristics.

So now we have to answer the real question: which engine is best? Which one should you use? Let’s begin by admitting the obvious: for most analytical tasks you throw at an engine, any one of the three would suffice. Most of the other major ‘second-tier’ engines, including Crafty (free to download), Deep Fritz (commercial), Hiarcs (commercial) and Junior (commercial), are also sufficient to analyse the games of amateurs and point out our tactical oversights. If you’re just looking for an engine to blunder-check your games, you have plenty of options.

If, however, you’re using engines for heavy analytical work or on very difficult positions, I think you need to consider buying both Houdini and Komodo and also downloading the open-source Stockfish. Each engine, as discussed above, has relative strengths and weaknesses. The best strategy is to see what each of the engines have to say in their analysis, and then try to draw your own conclusions. Were I forced to decide between Houdini 4 and Komodo 8, I’d probably – at this moment, anyway! – choose Komodo 8, simply because it seems stronger positionally, and its slight comparative tactical disadvantage doesn’t outweigh that positional strength. Both Houdini and Komodo are well worth their purchase price for the serious player and student. Downloading Stockfish should be mandatory!