You should be safe if you are the only one with access to that computer.
If you are sharing the computer with another human being; please Unplug your Yubikey and take it with you when you are not using the computer and it is likely that another human being could be using the computer. Just to be safe; Do Not Leave Your Yubikey Plugged In If Another Person Is Using It...unless you're authorizing them to access something.
Your Yubikey can't tell who clicked it's button as it is NOT a Fingerprint Reader.
It should be safe. It only shares the secrets with legit domains. That’s one of the powers of this tech: it won’t share your secrets with something that looks like a legit domain.
No, some of the functionality is definitely accessible without that, e.g. if you use ykman oath accounts code on Linux to read the TOTP codes you don't need to click and I seem to recall some of the functionality has a configurable click requirement.