How the fuck would you confirm that? Maybe the sysadmin is running a forked version of matrix that just says it's encrypted but actually logs everything in clear text.
I don't think that's how it works? It's the client application that has the key for the end to end encryption, not the server. I don't think you need to trust the matrix server you use? I could be wrong, I don't know matrix particularly well.