Xaeros is much lighter on resources and by default uses a much more modern design both graphical and in its codebase.
When modpacks are extremely large the minimap mod matters a lot in terms of performance too. The modpack doesn't even need to be huge for wanting a lighter minimap, as there are modpacks meant for weaker systems.