Thursday, February 17, 2011

Computers Overlords For $2000, Alex

IBM supercomputer Watson easily bested Jeopardy! champions Ken Jennings and Brad Rutter this week.  In a two-match tournament, it won a combined amount of $77,147 (yes, it makes odd bets) while Jennings won $24,000 and Rutter $21,600.  Note they weren't playing for the money, but for position, with the top winner getting a million dollars.

It was a fascinating display, overall, though I'm still not entirely clear on the logistics.  Unfortunately, I missed the first of the three telecasts while the second had technical glitches, so I missed much of the explanation of how Watson worked.  I don't know how Watson "heard" or "read" the clues, and how it knew when to buzz in.  The key to the game, especially when all the players know most of the correct responses, is to buzz in first, except you can't buzz in until the game allows it.  It seemed that Watson could buzz in before the other two players at will (as it were).  This would pretty much hand any good player the match.  The rest of what Watson did was impressive, but then it has a lot of information it can dip into, with a lot of parallel processing.  I can beat anybody in the world in a trivia contest if you give me access to Google.

TV viewers saw on-screen Watson's top three choices for each response, plus its confidence level.  I couldn't help but notice that when Watson had high confidence--say 80% or more--it almost always buzzed in first, but when it was low, especially below 50%, the other players usually beat it.  I have to assume there's a built-in algorithm that makes Watson buzz in fast when it's sure, but hold back when it isn't.  Odd, but effective, I suppose.  This obviously prevents it from giving more wrong responses.

Which means one potential strategy, which the two humans probably couldn't have been aware of, is to do the exact opposite of what they'd normally do.  Normally, you buzz in as fast as you can, hoping to beat the others.  Here, you can't possible beat Watson when it has the right response, so it may be a good idea to lay back and also let it give out some wrong answers, too--not only do you pick up the money after, but Watson has the same amount subtracted.

The final numbers make it appear to be a rout, but I don't think it was hopeless for the humans.  First, note Watson's strategy made it attempt to find the Daily Doubles, and it got almost all of them.  This is partly due to luck, and in a game such as this, where good strategy for humans is to double up when possible, not getting them probably hurt quite a bit.  Also notice two humans against one machine makes humans split the correct responses that are easier for people.  One human against Waston would have been a closer match--and one human against two computers would have made it even easier for the carbon-based life form.

In the first game, the computer had such a lead at Final Jeopardy that there wasn't much strategy involved in the betting.  (There was a little, since it was a tournament and the point was to add up the two days' winnings.)  But the first day's Final Jeopardy was quite telling.  The computer knew almost everything (as did the humans), but when it was wrong, it could still make mistakes no human would.  The final category the first day was U.S. Cities.  The clue was easy to solve (well, I got it, as did the two people on the show), but not only did the computer miss, it guessed "Toronto."  If the humans had even half of what Watson had at this point, as happened in the second match, one or both of them would probably have prevailed in the first game (which would have affected Watson's betting stategy in the second game).

I'm only sorry they didn't play five games this week.  The programmers have taught the computer quite a bit, but I'd like to see if the human master-players adapt as they go.


Blogger QueensGuy said...

Completely agree with your point about the "buzzing in" strategy. To a great degree, it wound up being a contest between human reaction times and computer reaction times, which we know will always favor the machine. But you still have to credit the programmers for that -- they were given a game to beat, and did so quite effectively. Bravo, IBM!

4:05 AM, February 17, 2011  
Anonymous Denver Guy said...

yeah, but I want to see Watson win at Wipeout!

9:39 AM, February 17, 2011  
Anonymous Anonymous said...

Nice post. Shows you at your best.

10:09 AM, February 17, 2011  
Anonymous Anonymous said...

What I'm really waiting for is the day when computers write their own blogs.

11:07 AM, February 17, 2011  
Anonymous New Buffalo MI fishing said...

Watson does buzz in before the answer is formulated which is similar to how humans respond, but is also unfair in this type of competition as this is a game of mental prowess and not reflex speed. So Watson should not be able to buzz until after it formulates the response to be more fair. Mean it would still win, but perhaps not by as much.

9:33 PM, February 17, 2011  
Blogger LAGuy said...

Hey, New Buffalo! Love that Redamak's.

When Jeopardy! was originally on, contestants could buzz in any time, even before the question was seen. I believe the producers felt this led to too many wrong answers and a slower game, so they changed the rules and didn't allow anyone to buzz in until a certain time after the question was out there. I don't know how Watson knew, but for humans, a light surrounding the game board goes off, signifying they may now buzz in. Unfortunately, this does turn the game into a battle of reflex speed as much as mental prowess.

9:49 PM, February 17, 2011  

Post a Comment

<< Home

web page hit counter