go, go, go…deeper!

While visiting Warwick, last week, I came across the very issue of Nature with the highly advertised paper of David Silver and co-authors from DeepMind detailing how they designed their Go player algorithm that bested a European Go master five games in a row last September. Which is a rather unexpected and definitely brilliant feat given the state of the art! And compares (in terms of importance, if not of approach) with the victory of IBM Deep Blue over Gary Kasparov 20 years ago… (Another deep algorithm, showing that the attraction of programmers for this label has not died off over the years!)This paper is not the easiest to read (especially over breakfast), with (obviously) missing details, but I gathered interesting titbits from this cursory read. One being the reinforced learning step where the predictor is improved by being applied against earlier versions. While this can lead to overfitting, the authors used randomisation to reduce this feature. This made me wonder if a similar step could be on predictors like random forests. E.g., by weighting the trees or the probability of including a predictor or another.Another feature of major interest is their parallel use of two neural networks in the decision-making, a first one estimating a probability distribution over moves learned from millions of human Go games and a second one returning a utility or value for each possible move. The first network is used for tree exploration with Monte Carlo steps, while the second leads to the final decision.

This is a fairly good commercial argument for machine learning techniques (and for DeepMind as well), but I do not agree with the doom-sayers predicting the rise of the machines and our soon to be annihilation! (Which is the major theme of Superintelligence.) This result shows that, with enough learning data and sufficiently high optimising power and skills, it is possible to produce an excellent predictor of the set of Go moves leading to a victory. Without the brute force strategy of Deep Blue that simply explored the tree of possible games to a much more remote future than a human player could do (along with the  perfect memory of a lot of games). I actually wonder if DeepMind has also designed a chess algorithm on the same principles: there is no reason why it should no work. However, this success does not predict the soon to come emergence of AI’s able to deal with vaguer and broader scopes: in that sense, automated drivers are much more of an advance (unless they start bumping into other cars and pedestrians on a regular basis!).

4 Responses to “go, go, go…deeper!”

  1. “I do not agree with the doomsayers predicting the rise of the machines and our soon to be annihilation! (Which is the major theme of Superintelligence.) This result shows that, with enough learning data and sufficiently high optimising power and skills, it is possible to produce an excellent predictor of the set of Go moves leading to a victory.”

    You’re missing a key implication of this accomplishment:

    “What this indicates is not that deep learning in particular is going to be the Game Over algorithm. Rather, the background variables are looking more like “Human neural intelligence is not that complicated and current algorithms are touching on keystone, foundational aspects of it.” What’s alarming is not this particular breakthrough, but what it implies about the general background settings of the computational universe.

    To try spelling out the details more explicitly, Go is a game that is very computationally difficult for traditional chess-style techniques. Human masters learn to play Go very intuitively, because the human cortical algorithm turns out to generalize well. If deep learning can do something similar, plus (a previous real sign) have a single network architecture learn to play loads of different old computer games, that may indicate we’re starting to get into the range of “neural algorithms that generalize well, the way that the human cortical algorithm generalizes well”.”

    • I haven’t read the paper in question (yet). However, a comment regarding the “generalizes well” and neural networks. More or less all achievements I have seen in “deep learning” seem to rely on supervised learning techniques requiring massive amounts of labeled data. They are also specifically trained, end-to-end, for one singular purpose and generally don’t build on previously learned concepts and knowledge. So in this regard I agree with Christian (Robert).

      Christian: This is definitely a big achievement, but from what I’ve heard the professional player they beat was not considered very strong by international standards as opposed to Garry Kasparov. However, I’ve heard they have a game scheduled against the reigning champion in March which sounds very interesting!

      • Thanks, Christian, I got the same comment emailed by David Draper last night, looking forward this next game. However, from my bystander’s position, I see the difference as incremental in that a wee further learning and a higher computing power should lead to inhuman abilities…

    • This is exactly where I disagree. I do not find this achievement, and an achievement it is!, as reflecting on the whole human brain and human intelligence. This is a game solver for a game with a clear optimum (“win”) and a finite number of possible actions. Large finite number but finite number nonetheless. There is no intelligence in it. I thus see no rationale in projecting the clear success of the Go algorithm into sentient AIs.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: