I'm pretty comfortable with it since there's no god to be afraid of. It's the people who believe in god that have to avoid pissing him off by not having a good enough reason to call him, or something.
If anything, it's a little bit transgressive and that's fun.
I'm an atheist but saying things like oh my god or calling for Christ or his mother is so ingrained in my mother language that it's just a meaningless jerk reflex sound... Like saying ow, or sneezing...