I've taken to calling this kind of media "Mandatory Adult Television." I think the first of its species was Lost; there were predecessors with some similar traits like The X Files or The Sopranos, heavily serialized adult content television that was very popular water cooler talk, but Lost was the first one I remember as segmenting the population into those who follow he show, and those who don't. Game of Thrones was THE big one. You either watched Game of Thrones or you weren't allowed to socialize as an adult. "Hey, did you see Game of Thrones last night?" "No, I don't watch that show." "Oh. Bye." For nearly a decade. No one wants to talk about that show anymore. Same with Lost.
What, in your mind, is "fantasy?" Because for a genre whose name implies imagination and creativeness, it seems pretty pigeonholed into the European medieval folklore/fairy tale aesthetic somewhere between King Arthur and JRR Tolkien.
I'd agree with that, and most people will probably never venture outside of the hole you've described. For my money it's a book that has to do with fantastical elements that can't easily be lumped into science-fiction.