I think a good part of that is because 'average human' is not a good way to represent who we are individually. I'm probably above average at specific things but in many other respects I'm average or below or wherever I'm supposed to be. Maybe most people are above average even though on average most of us are average.
Fully above average people IMO are like astronauts and stuff. We all live in the shadow of that former navy seal/doctor/astronaut who is like 45 or some crazy shit.
This really depends on the distribution. If some or all of the people in that bottom 20% are very, very stupid it could actually work out that 80% are above average, because the average is being pulled down by the people at the bottom.
This is why we have different averages like mean, median, mode, and RMS because they each give you different interpretations of the raw data. For example the mean electro motive force of the grid is around 0 volts because it spends as much time in the negative as the positive. We use RMS here because negative numbers become positive when squared.
Just the fact that you’re thinking about this in terms of a distribution that’s non-normal indicates that you’re on the right side of that distribution.
To be honest I don't think intelligence can be boiled down to a single number. Like somebody or something can have a slow processing speed but can do a lot of different things, versus something or someone that is incredibly fast but limited in it's usage. To some extent this actually happens inside human minds with things like system 1 vs system 2 in psychology having different roles within the brain/mind of a human and being suited to different things (flexible but slow and single tasking vs dumb but fast and parralel in this case).
Also I am considered by my society to have a mental disability. So regardless of how far right of this distribution I might be there are still things I don't understand that more average people can. You could argue that those things are only domain-specific forms of intelligence but I don't know if that's actually true or not. There are too many variables and anomalies we don't understand.