I'm libertarian and I've never heard anyone make that claim outside of these memes I keep seeing. I wonder about the people you all hang around with if this is the kind of things you're hearing from your local "libertarians".
When I was younger we'd make fun of them for hoarding gold and wanting toll roads but then they all switched to shilling crypto Disney dollars and Teslas, and that's a self-own beyond parody. So now, sadly, the memesters have only the one libertarian joke left.
This is correct, it's just classic prejudice based on bullshit. I have issued a challenge to anyone on this thread to prove me wrong, see my comment near the top.