I have never thought about the fact that manhole covers need to hold up to weight. Of course they do and it's perfectly normal and sensible. It's just not a thought I've ever had.
Structures underground experience buoyant forces that act to push them up. Manholes (and any structure bottom, like storm inlets, pump station wet wells, etc) need special consideration since they can be partially or mostly hollow, so they have to be heavy enough to remain in place.
Right, so the light is actually pushed up by these buoyant forces and I guess that then also explains why it's so dark underground. Fascinating how learning some little new details about the world can sometimes make it all just click together!
Technically, the walls of the manholes, in turn, support the weight of the cover and whatever load is on top of it. The entire manhole is a weight-bearing structure.
For formula one races they weld them down to stop the cars incredible downforce from sucking them up into the air. Even then they sometimes get torn up and thrown around.
Manhole covers are also a common exercise for engineering students. Like designing one that can hold x amount of weight with a specific set of limitations and/or requirements.