People who don't like fantasy often base their objections on the claim that they prefer to read books or watch films set in the real world. For these people, the dichotomy is obvious. Fantasy is set in an invented secondary world, which obviously makes it trivial and irrelevant, whereas good fiction (that is, whatever they happen to like) is set in the real world, which automatically makes it superior and relevant.
Leaving aside the fact that many of the books, films and TV shows ostensibly set in the "real world" are neither superior nor particularly relevant (the James Bond stories are nominally real-world stories, for heaven's sake), this attitude shows a fundamental lack of understanding about the nature of fiction — not to mention the nature of reality.
My contention is that every story ever written is actually set in an invented secondary world, and fantasy (as well as some SF) is the label given to those that are upfront about it. It doesn't matter how uncompromisingly gritty a slice of social realism a story might be, it's set in a fictional reality, not an objective reality.
Consider two authors both writing stories about a maverick cop who rides roughshod over the rules and procedures. In one, he might be the hero who nails the bad guys that would get away if he played by the book. In the other, he might end up destroying innocent lives the rules were there to protect.
This isn't just a matter of attitude. Depending on their views or agendas (often, but not always, the same thing), each author will create realities in which their take on the story is objectively true. The first will quite genuinely be a world in which bleeding-heart liberals are letting the crooks get away to prey on their victims. The second will just as genuinely be in a world where the rule of law is the only thing separating the good guys from the bad.
Of course, a reader who entirely agrees with one or the other point of view will interpret that fictional reality as objectively true, but another will see the opposite as being true. The point is that the difference isn't between the attitudes of the characters within the story, but lies in the author's primary worldbuilding. This is analogous to the way Tolkien writes about a world in which morality has the force of a law of nature and can affect the outcome of events just as surely gravity or the weather. The differences can be a lot more subtle, though.
Soap operas* are generally presented as ultra-realistic slice-of-life drama, but actually they tend to take place in an odd half-reality. Besides obvious anomalies like location (EastEnders, for instance, is set in a rearranged version of London) there are usually odd social habits that are unlike anything you'd actually find, simply to facilitate the dramatic necessities. There isn't necessarily anything wrong with this, but it's not the real world.
Most of all, perhaps, the fictional reality of a story will be determined by selecting what to put in and what to leave out. The complete reality of our society contains everything from cosy village life to inner-city gang warfare, but the reality in which a story takes place rarely includes all this. The author will select what's relevant to go into the story, and the rest won't exist.
This kind of selection, like the two ways our maverick cop can go, largely reflects the author's views and/or agenda. The fictional reality of a story isn't the world as it objectively is, but the world as the author wants it to be — not necessarily wants as a good thing, but wants in order to make a point. It's set in a custom-made world, just as a fantasy story is, but masquerading as the real world.
Does any of this matter? I think it does. Fantasy is often accused of portraying unreality, but it doesn't pretend otherwise, concentrating instead on using that unreality to shine a light on the world around us.
The more the fictional reality looks like our own world, though, the harder it is to make that distinction. I recall an argument I had once with a work colleague — I can't remember the exact topic, but I think it may have been about the precise effects of particular illegal drugs. What I do remember, though, is that the killer argument presented by this otherwise intelligent person was "Of course it's like that. Didn't you see EastEnders last week?" To which I gently explained that it had been that way in EastEnders because that was how some author had written it, not because it was necessarily true.
Fictional reality isn't restricted to fiction. Each of us sees the world in a slightly different way from anyone else, selecting what we admit and what we don't, explaining events according to our own assumptions and interpretations of reality. Most of the conflicts in the world are due to the fact that we do this unconsciously and assume our own fictional reality, whether individual or broadly shared, is objectively true.
If we could learn to understand how fiction works, critique it not in absolute terms but in terms of its unique fictional reality — its own secondary world — maybe we'd be better at understanding our own and others' unique inner worlds.
And what place better to learn how to do that than fantasy?
* The term soap opera is used with different meanings in different parts of the world. I'm using it in the usual UK sense of a continuous series (ie no breaks or seasons) about some kind of community that takes place in real time, so that, for instance, the characters are preparing for Christmas or anticipating the Cup Final at the same time the viewers are.