Speaking only for myself, I don't think I've ever intended "fuck you" to mean "I am literally going to fuck you violently with my penis." More like, "I strongly disapprove of your actions and perhaps even your character (but not enough to rape you, of course, that should go without saying)."
To add to this I read years ago how the English phase "fuck you" got associated with the gesture. It comes The Hundred Years War between Britian and France and was originally "Pluck You!" but changed over time.