Here's my theory... Google wants to artificially fuck up it's search functionality. It wants to offer good performance for a fee. And it's going to be doing that by giving it's own AI the correct filters while at the same time tripping every other AI capable of searching the net such that the other AI results become garbage and only the Google one works correctly. Anyway that's my conspiracy theory, fuck Google with a bunch of sharp forks.
I think it's simply a question of what are Google's interests. Users doesn't pay anything to Google for the service, so that's not where Google's interests are. Advertisers pay Google, so that's where Google's interests are. Google has no interests to make the search better for users, they want to make it better for advertisers.
To do that, you simply have to inform users about other available search engines and remind them again that those exist when one of their searches fails.
And AI is just making it worse. Entire websites of SEO-optimized content can be generated in seconds now. No one will be able to keep up with all that!