Yeah, he was a condescending sexist prick to the Kiyoshi warriors (initially), and while it did earn him some beatings, it didn't get him a restraining order and he even got with Suki in the end. That's high CHA alright.
I don't know about you, but to me, the fact that he showed genuine humility and willingness to step beyond his initial chauvinism (after some hands-on lessons about fighting women) had a lot more to do with that than his charm.