AlphaGo, 2016: the stone no human would play
Seoul, March 10, 2016. On the commentary livestream for Game 2 of the AlphaGo versus Lee Sedol match, professional analyst Michael Redmond goes unusually quiet. AlphaGo has just placed its 37th stone — a “shoulder hit” five lines from the corner, a part of the board where professionals do not play at this stage of a game. The algorithm that chose it estimated the probability of any human making that same move at one in ten thousand. Redmond calls it “creative.” He calls it “unique.” Across the table, Lee Sedol, an 18-time world Go champion, stood up and walked away from the board. He was gone for fifteen minutes.
The match — five games, March 9 through March 15, in Seoul — had been arranged as a test of how far machine intelligence had come. Go has roughly 10^360 possible board positions, a number that makes chess look like tic-tac-toe, and the tree-search techniques that let Deep Blue defeat Kasparov in chess nineteen years earlier were computationally useless here. Most experts estimated AI mastery of Go was a decade away. The team at DeepMind — led by researcher David Silver and co-founder Demis Hassabis — published a different answer in Nature on January 28, 2016: two deep neural networks and a Monte Carlo tree search. A policy network identified which moves were worth considering; a value network judged whether the resulting position would win. Trained first on 30 million expert games, the system then played against itself for weeks, guided by nothing but the outcome. Move 37 did not come from any human example. It came from a machine that had played itself until it found what humans had missed.
Lee Sedol had arrived in Seoul confident. Before the first game, he told reporters he expected to win. By the end of Game 3, trailing three to zero, he apologized at the post-game press conference and left early.
He returned for Game 4 with a plan. On move 78, Lee placed a white stone in a crowded section of the board — another one-in-ten-thousand move — which expert commentator Gu Li immediately called a “divine move”: completely unforeseen, structurally devastating. AlphaGo’s calculated win probability collapsed over the next ten moves. Lee won the game, the only one humans would take. He said later that every move from 68 to 78 had been a deliberate trap, seeded over a dozen exchanges. AlphaGo, once, had not seen it coming.
The technology didn’t stay still long enough to be studied properly. In October 2017, DeepMind released AlphaGo Zero — trained on no human games at all, only the rules. It defeated the original AlphaGo 100 games to zero in three days. The insight that emerged from the Lee Sedol match — that self-play against a fixed reward could surface strategies humans hadn’t imagined — became a template. AlphaZero applied it to chess and shogi. MuZero applied it to games where the system wasn’t even told the rules in advance.
Lee Sedol retired on November 19, 2019. “With the advent of AI,” he said, “I’ve realized I’m not at the top even if I become the number one. There is an entity that cannot be defeated.” He was 36. The program had never been taught what good play looked like. It had only played itself, millions of times, until it knew.
Sources
- AlphaGo versus Lee Sedol — Wikipedia — game-by-game results, dates, Move 37 board position and Michael Redmond’s commentary, Game 4 Move 78 and its effect on AlphaGo’s win probability.
- AlphaGo — DeepMind — one-in-ten-thousand probability for Move 37, technical architecture overview (policy network, value network, MCTS), AlphaGo Zero results.
- AlphaGo — Wikipedia — team members David Silver and Demis Hassabis, Nature paper date and title (January 28, 2016), AlphaGo Zero’s 100-0 record in three days.
- Lee Sedol — Wikipedia — 18-time world champion record, Gu Li’s “divine move” quote, retirement announcement November 19, 2019.