It could just be delay from the overload, but it could also be a side effect of federation. Federation is a voluntary relation. There's a solid chance that set of federated instances is different for lemmy.world than it is for lemmy.ml. It might seem like that doesn't matter so long as they're federated with one another, and you're looking at a post from one or the other. The catch is that, if they aren't federated to all the same instances, lemmy.world may not see all of the votes that lemmy.ml sees—specifically, the votes from servers not federated with lemmy.world. And that would explain the divergent results between the way the two servers rank posts.
This is due largely to the way that ActivityPub passes interactions, and where those interactions are stored, and how. There's analogous behavior on other ActivityPub services, like Mastodon, where like totals differ depending on which instance you view a post from. And there are probably ways to fix it, but they would either requires some degree of centralization, or major changes to the protocol.
That does seem like it would cover some cases. In my case, I'm noticing this with instances I'm fairly confident my base instance federated with, because other posts from the instances are visible to me. For example, I didn't get a notification for your reply, but looking from Beehaw I was able to paste the link and find it that way. For other instances, I have no luck trying the same thing, even if I try pasting the post or the community first, despite having other posts from the same time period and instances accessible to me.