Then do sudo apt install xfce4 and sudo apt purge cinnamon* muffin* nemo*.
It's been a while since I installed xfce4 on anything, but if things haven't changed I think the metapackage doesn't include xfce4-goodies and some other packages, so if you're missing something it's likely that you just need to 'apt install xfce4-whatever'. Additionally you can keep cinnamon around as long as you like as a kind of a backup, just change lightdm (or whatever login manager LMDE uses) to use xfce4 as default. And then there's even lighter WM's than XFCE, like LXDE, which is also easy to install via apt and try out if that works for you.