Does anyone's game perform worst when playing with friends?

Sounds like one of your friends has a real potato computer which is causing sync issues when you play with them.

I imagine that "random" could secretly be biased towards people with lower ping vetween them, for a better experience overall.

are you hosting? host always gets best performance.

I keep the little Steam FPS counter on when I play, I never notice it making a difference when I'm with randoms vs with friends. One of my friends plays on a Steam Deck though, so the loading screen of us all diving to the planet feels like it takes ages compared to with randoms.

No problem here