On tabletop I play a Way of the Astral Self Monk with defensive duelist. I thought I'd try that in the game and switched Astarion to open hand Monk. I'm not sure if I'm doing something wrong but it never seems to activate.
Monks can use weapons, but not Open Hand. Elemental monk works better with weapons than bare-handed in BG3. I don't think any martial weapons are also finesse weapons, though.
I was aware monks can (at the very least I was aware of Kensei), I'd just never considered it as advantageous enough to go Defensive Duelist, I guess.
In an MP save I'm in, I'm playing an open hand monk and am currently using that staff you can buy from Auntie Ethel in the grove because of the bonuses to unarmed damage.
Your fists aren’t finesse weapons so it can’t possibly work. Also fists aren’t considered weapons for feats so second point for it not working for monks.