Thanks for the comments. I agree on the general consensus, that once an encryption key enters the VPS, the encryption is compromised.
However, I'm thinking more in practical terms, eg. the service provider doing just casual scanning across all disks of VPS instances. Some examples could be: cloud authentication keys, torrc files, specific installed software, SSH private keys, TLS certificates.
Modern CPUs have some RAM encryption features, but ultimately you're running on hardware outside your control. Personally, I use full disk encryption (except for /boot) and unlock remotely via SSH, but that only helps against automatic scanning of the storage.
Dropbear + manual input, but I guess you could do that as a single command somehow. I rarely restart this machine, so copying the PW from my PW manager is acceptable for me.