This happens because Lemmy does not count the actual number of comments that there are under the post, but instead there is a counter per post. This is not necessarily a bad thing, but it does not seem like the counter is every synced with the actual count of comments.
I think I noticed that if a commenter deleted their comment then the total comments would go down by 1. On a post with only one comment that also got deleted, it made it show as -1 comments.