It's kind of like imposter syndrome for taking a break. Are you really bored enough to enjoy relaxing? Have you really relaxed enough to get back to work?
Try what I do and just cry for an hour or so out of the day. That way you get neither work nor play done and you can feel equally bad about both without the inconvenience of flip flopping between them