That said, sometimes it takes forever to load content from other servers, and there isn't any caching that I can see. When I go back from a post it all struggles to reload again.
^ This.
It seems to take several minutes then you get a massive flood from multiple servers. Also change how it is sorted will give you different results.
It seems to be broken right now, but https://lemmy.directory/ attempts to be this. The "all" posts feed on your instance shows all posts
In communities that someone on your instance subscribes to (which might be quite a lot of communities on a big instance, or the same as your own subscribed list on a single-user instance). lemmy.directory has some script that tries to automate subscribing to everything, so it can be a true /r/all analog.
I'm using the jerboa app. Just need to change Local to All in the top left. Sometimes it changes back. I also need to be very particular in how it sorts or it's the same top posts for days.