This is my go to response when people are trying to claim that English is hard... Well at least I don't have to remember what gender has randomly been assigned to every noun I want to use.
No, instead you have to learn to read and spell in a system that often sounds quite different to what is written. I want to read a book that's never been read. I want to live a life alive at a live show. Anything ending in ~ough which has something like 6 or 8 different sounds. I'm a native speaker trying to work with my wife on English (we speak Japanese at home). It's insane for any reading/spelling.
I think liason is the harder part about trying to transcribe someone's spoken words as a new learner without a good grasp on vocabulary.
At least with French, a lot of those silent letters are a lot more predictable than English. English has French borrowings (from two different time periods), Latin borrowings (some of which were borrowed via Norman or Old French first), Greek, Germanic, etc. and we did various levels of preserving the native spelling. This is neat for etymology and maybe figuring out a word one doesn't know, but kinda sucks for spelling. A lot of words from Normal and Old French are now spelled differently in modern Parisian, but the more recent loans are closer. It's a hot mess.
And then you also have to get the correct stress on the syllables which are also unguessable. Ask for a banana instead of a banaaaaaaaana and people won't understand.
English has no shortage of exceptions to "rules" (sometimes the rule only seeming such because it applies to the subset most frequently used rather than the whole set of whatevers). English's most common verbs are irregular. That's not necessarily too crazy (be, have, do/make are often weird in most languages because they resist change the most since they are most used). We have all kinds of things that aren't "correct" (prescriptive view) that native speakers get wrong all the time. "I have went" rather than "I have gone" is one that grates to me, but I accept that language changes. A lot of verbs are also losing their endings and patterns and gradually going to the dominant ~ed ending where previously they did not (Tom Scott has a good video on this).
If you task your male dogsitter to give your dog a treat while you are away and somebody asks you whether your pet is taken care of during your vacation, you can say: "Don't worry. When I return, the bone will have been given by the man to the dog."
Then by that metric, Chinese must be incredibly easy. Simple genders, no articles, simple grammar, no verb conjugation whatsoever, very simple tenses. Probably the easiest language out there!
Pronunciation is still incredibly difficult unless you are immersed in it. I'd argue that it's legitimately one of the most difficult languages to pick up in a classroom simply because of how completely different it sounds in the real world versus on tapes.
I didn't list all possible difficulties. Chinese have this never ending list of very complex characters and probably more subtility that I don't know of. If it is close to Japanese though, yes the grammar doesn't seem complicated compared to European languages.
Chinese grammar seems to me to be simpler than Japanese, though I studied Japanese for about a year and have lived here speaking the language daily (primary language at home) for the better part of a decade and have only scratched the surface on Mandarin.
Your anglocentric view is common, but also completely wrong - speakers of strongly gendered languages (Latin, German, Portuguese, French, etc) don't have to remember a word's gender either, it just comes naturally as you become fluent.
I'm semi-fluent in German and Spanish, and my strategy is guesstimate. I figure that I've probably read/heard the word before, so I just test out the genders on it and whichever one "feels more natural" or "sounds less weird", it's probably because I've heard it that way before, so I go with that.
In Swedish you can figure out the gender of a new word because the phrase hints at what it is. In french there is no such luxury, and even worse, it's a Bel (sounds like belle which is feminine) avion not a beau(masc.) avion even if avion is masculine...
Lots of french people don't know the gender of the ocean and other voyel starting words because of that.