AI hallucination on real life game
AI hallucination on real life game
This is a game played on June 19 between Ueno Asami (as black) and Fujisawa Rina (as white). At move 159, Asami cut with K14.
It involves a followup of the life and death of the white group M10, and most human players would be able to judge it, but AI (from Golaxy to Katago) all seem to deem it "alive" and continue to fight without securing the life of that group first. While a human player can read out the sequence like Rina did during the game and make the right choice.
But it takes like 10k+ playouts before AI realize the K14 cut is a very good move, and (with few playouts AI even judges K14 as a blunder). And it takes millions of playouts before AI realizes the M10 white group is in trouble.
What other AI blindspots and hallucinations have you seen in real games?
As a follow up, I looked into this position with the current strongest rated network, and at 25 visits, K14 is actually the favorite move, followed by the reduction at O8*. After 100,000 visits it hates every move, but thinks that K14 is about 6.5 points better than the next best move H14. There are some big territorial swings, and depending on the exact sequence the network will like the situation more or less, so the network thinks this is a very volatile position.
* Note, now that I've had a look at it, I don't believe that it is possible for black to kill white at M10. Best black can do seems to be seki: bO8, O7, N5, P8, N8, N9, M8, P9. But if you have a better result for black please let me know.
Hmm, from I remember, black seems to be able to play bN8 after the bO8 bO7 (if white choose to play wL11 bK11, wK12 to cut off the black (or if white doesn't respond locally, black can just easily connect back, and white is already too far behind)
If white doesn't choose to take the 3 black stones after the bO8, bO7, bN8, and choose to continue to push P8, black can just block with P9. White has to connect M8, and black can play L10, and white cannot connect M9 (or black connect N5 the whole white group dies). And white would have to capture with N5, black can throw in M9, wN9 capture, bO9, white again cannot connect M9, or bO4 kills all. Hence, the only option for white is to play O4, and black would be able to kill all the M10 white group stones. (this is effectively connect and die problem, and the commentator that day, along with Fujisawa I think, all read these out)
Hence, the best option to keep that M10 group and minimize the lost is to play N5 to capture 3 black stones first right after black K14, and let black to cut off the two K15 K16 white stones (Fujisawa had the chance to save those two stones before, and the game might go into yose, but she played H15 and allow Ueno to push and cut)
Blind spots like this occur all of the time. Can you give some additional details and context? And I'll try to get this position as an sgf position for training.
What network were you using to evaluate this position?
Do you have an sgf file of this game that you can share, so that I can attempt to reproduce this issue? (Sometimes, the real blind spot is few plys into the tree, so it's sometimes useful to be able to look into the position.)