Part of it is the amount of energy in low frequencies as loud as the corresponding high frequency. Subwoofers use more energy to move larger volumes of air and produce low frequencies. The low frequencies are also harder to absorb and travel better around corners.
There's also the fact that some people make the bass much louder than the rest of their system. It starts out sounding muddy as hell and gets worse.