Same principle as a gyroscope: a turning wheel will tend to stay perpendicular or parallel to the direction of the gravity vector because if it starts tilting away from such orientation there's a force that pushes it back.
Also works better with bigger wheels (if I remember it correctly the effect is related to spinning momentum).
I was pretty surprised when learning Physics and they show us how to derive the formula for that (which I totally forgot since that was over 3 decades ago).
Edit: Actually the gyroscopic effetc is just a part of it. See this article
Gyroscopic effect is not even significant. Lock your steering and you will fall over no matter how fast your wheels are spinning. (Which can happen with a badly pitted headset)