I had to do it in waves because there's so many enemies in there. I'd clear a section, go somewhere else for a bit, then come back. Gotta say, the way camp works is great.
Ok, I heard it was happening to other people as well but couldn't figure out if it was a bug. For instance, on occasion companions not in my party will vanish from camp until I load save again.