It goes without saying that this shit doesn't really understand what's outputting; it's picking words together and parsing a grammatically coherent whole, with barely any regard to semantics (meaning).
It should not be trying to provide you info directly, it should be showing you where to find it. For example, linking this or this*.
To add injury in this case it isn't even providing you info, it's bossing you around. Typical Microsoft "don't inform a user, tell it [yes, "it"] what it should be doing" mindset. Specially bad in this case because cost vs. benefit varies a fair bit depending on where you are, often there's no single "right" answer.
*OP, check those two links, they might be useful for you.
LLMs don't "understand" anything, and it's unfortunate that we've taken to using language related to human thinking to talk about software. It's all data processing and models.
Yup, 100% this. And there's a crowd of muppets arguing "ackshyually wut u're definishun of unrurrstandin/intellijanse?" or "but hyumans do...", but come on - that's bullshit, and more often than not sealioning.
Don't get me wrong - model-based data processing is still useful in quite a few situations. But they're only a fraction of what big tech pretends that LLMs are useful for.
Yeah, I'm far from anti-AI, but we're just not anywhere close to where people think we are with it. And I'm pretty sick of corporate leadership saying "We need to make more use of AI" without knowing the difference between an LLM and a machine learning application, or having any idea *how" their company could make use of one of the technologies.
It really feels like one of those hammer in search of a nail things.