To start this off, the fact that there’s such thing as a "standard" fantasy setting in itself is rather sad, because to be able to reduce something that never existed and never will to a "standard" shows a creative bankruptcy on the part of whole industries.
This all boils down to how "influential" Tolkein was, by which I mean "everyone got lazy and has dug out the same 5 miles of the UK that he wrote about." And that 5 miles of the UK is intensely problematic. Not least because it’s overwhelmingly white despite racial diversity being a thing in Europe and even the UK since literally forever. But I’m getting ahead of myself. The "humans in the sun; elves in the forests; dwarves in the ground; orcs in that shadowy place the light doesn’t touch, never go there Simba" world is so prevalent that everything is at this point past the point of being interesting for the ways it tries to subvert the tropes because the subversions themselves have become their own tropes because this specfic template just. Keeps. Happening.
But let’s try to dissect what made this setting in the first place, the problems with it, the way it’s bled into our perceptions of real-world Europe’s history, and our demographic biases even for modern times.
Middle-earth is not the UK
To start this off right, I’m going to elevate a Black voice here and link this video for your viewing pleasure. It’s not required viewing, but if you want a more serious person than me to listen to about this there you go. But for the purpose of my own discussion, Europe is not the UK, and even if it were, the UK was not 100% white. And when we talk about "Europe," I also want to stress that "Europe" is also not just "the UK, Germany, and France" as is commonly represented in media. Even if it were, there’s plenty of diversity. But the idea that The Witcher is "exotic" when it’s literally just Poland or that Norse mythology is "exotic" or even that Greek and Roman mythology is treated like it comes from way farther away than it actually does kind of sets the bar for what people think of as "Europe" and what they think a "European" is. Like, the whole Mediterranean is sitting right at the bottom of it, being all sunny and warm, and places like that tend to be populated with people with darker skin. If Spain were any closer to touching Africa’s back end, Africa would slap it. Never mind that the Greeks themselves were very well-traveled and dealt extensively with Africa and the Middle East. So far, in fact, that they were at least aware of gorillas, which means their cultural influences extended to sub-Saharan Africa. The idea that the Greeks didn’t know what a Black person was is laughable.
On top of that, even putting Greece aside for a minute, it’s not like there weren’t other references to dark-skinned people. The "Moors" were basically a grouping of any number of dark-skinned people and has literally no actual ethnic basis. People are acknowledged as being "tall, dark, and handsome" or having a "swarthy" complexion, i.e. they were at least olive-skinned and often darker. "Swarthy" as a word basically means "black." There’s maybe a little dancing around some of the reality of this because it also applied to people who were simply tanned, often from travel in context, but the idea that everyone was white as the driven snow simply doesn’t hold up to even light scrutiny.
So what IS Middle-earth? Well, it’s a fairy tale from someone who experienced going to one of the bloodiest wars in human history, World War I. That’s right; the orcs were Germans (no, that’s not a typo; the Germans were against the British both times, but they weren’t Nazis the first time, so the orcs are at least not Nazis). In making his grand fantasy, Tolkein basically distilled the idea of a very complicated, messy event into something simple and easily digested where there was an inherently evil enemy and in his later years even he understood how problematic it was to indicate that the orcs both had free will and were inherently evil and therefore okay to kill, but never found a way to dig himself out of that particular hole before his death. Ultimately he was a complex person from a time with drastically different cultural standards and to be frank, humans are not logical or consistent creatures. If you’re really interested in the arguments, you can read Wikipedia as well as I can. Have one about his writing while you’re at it. The thing you should take away from it is that The Hobbit is basically a bedtime story and is not the place to be including teachable lessons on the complexity of war. And in that regard, it should not be used for the template of racial dynamics that it has been for so long.
The Forgotten Realms aren’t the UK, either
That’s not to say that Tolkein is solely responsible for the proliferation of all this, because Gary Gygax has some blame in it for taking it and running with the wildly popular Dungeons & Dragons, though in that case the influences were quite a bit more myriad, including India, Greece, and discount rubber Ultraman monster toys rebranded as "dinosaurs." No, I am not kidding about that; that’s where the Bulette comes from. The problem being, of course, that for all of these inclusions, very little work was done to make it actually inclusive to different colors of human. Some of this is because it could just as easily be offloaded onto the player to specify, but a lot of it probably owed itself to, once again, fishing out of Tolkein’s barrel of pre-caught fish. So much so that there were legal disputes over it and some things had to be renamed because the crew wasn’t shy about it, either. The additional problem, of course, was that all of these foreign influences were treated as exotic and India itself got name-dropped in the Rakshasa entry, because back then that was a flex because Wikipedia didn’t exist yet and that meant you had done enough research to find it in a book. Those things don’t have links or text search or anything! Basically that was as much credit as they were willing to give their players to look it up for themselves if they were so inclined. But as much as a Rakshasa was from India, humans from India were a bit more of an "ehhhhhhh…"
And on one hand, that’s actually kind of progressive in its own way, because that could have gone down much worse like all of the problems The Elder Scrolls offers with the various capabilities of its multiple human races trying to assign stats to your skin tone, but on the other, I’m really not an expert, but given what D&D art is like now, I’m not holding my breath on a whole lot of representation back when it was having its biggest influence on everyone else, because influence it had, yes indeedie, with Square getting into similar legal hot water for its own liberal fishing out of the D&D bucket for the original Final Fantasy and such staples of the series as the Coeurl remaining relevant to its bestiary to this day. Final Fantasy in its own right is notoriously white, or else notoriously white-passing, but that makes it an unfortunate easy sell to both the Japanese and US audience because people don’t expect a Black person in a castle.
Which kind of brings us to the question of why people think that in the first place, and frankly, the answer is surprisingly simple: racism. Oh, don’t pretend you were expecting anything else! The fact of the matter is Western media is dominated by white people because white people have a stranglehold on political power and as such, they are vastly overrepresented in the media they create. The reason people don’t assume a Black person belongs in a castle is that it’s not something the history books put any focus on and so people have no expectation of it. Most American children don’t learn about Black people outside of the slave trade and George Washington Carver for inventing peanut butter because kids love themselves some peanut butter. As far as most American children are concerned, Black people stayed in Africa until Europeans put them on boats. American kids don’t learn a thing about African history outside of maybe ancient Egypt and that’s not because there’s no history to learn; it’s because the parts of it that don’t involve luring Black people onto boats with scraps of cloth involve them getting their butts served to them on a silver platter by an African queen who was successful enough of a tactician that her folks were taking down Europeans armed with guns with comparatively rudimentary weaponry. As in, Black History is in many ways embarrassing to the white power structure. Ultimately, American children learn about Europe, a bit about China, maybe a bit about India if they’re lucky if only because of their interesting religious iconography, but the "Europe" kids learn about is mostly, wait for it…
The UK, Germany, and France.
Sensing a pattern here? I saw someone point out that more or less the radius of "Europe" that anyone ever hears about is more or less whatever is within 500 miles of London and, oh, heck, have another video.
So what did they get right?
Frankly, not a whole lot. Aside from the fact that an "adventurer" wasn’t really a profession as much as it’s convenient to gameplay, weapons had to be checked in at the inn first thing in town if they were longer than one prominently on display, and the ideas that inns were alehouses or had separate beds, the biggest failure of anything is the idea that people lived and died within a mile of where they were born. Which, you know, kind of would put a crimp on the inn industry. In fact, Medieval people all traveled quite a lot. Not just in the sense that modern Americans actually work more days than Medieval serfs (most of their time off was because there was 1-2 holy days of obligation a week not including Sunday, but even that was a pretty sweet deal because you basically spent an hour in church and had the rest of your day to do whatever), but the average person could just up and go on a pilgramage as basically what we consider a vacation with a little Jesus added, travel for a bit to some notable holy place, see the sights, talk shop with all the locals along the way, get a feel for the trends in their profession, hear news from other places in the inns, all that good stuff, and basically get out of the house and contribute to the free exchange of information. For the time, it was a pretty sweet gig. Obviously they had to go back at some point, because the work doesn’t do itself, but in the process they’d come back with some better idea of what was going on in the world at large, maybe make a few friends who might show up on their doorstep in return, and generally have a good time. It’s really no different than how we do tourism today.
Also, and I must stress this, the low life expectancy was due in large part to infant mortality. There’s a reason people waited a bit before having a Baptism or other christening ceremony; you wanted to make sure the kid was maybe going to stick around a bit before you started with naming and getting attached. For the most part, if you lived to adulthood, you were set. Unless you were a knight, but then you could at least say you made it to 21 to even get to that point. Also, being a knight was itself a fairly sweet deal, since knights were petty nobility. Yeah, not just anyone can afford a custom suit of handcrafted armor or a sweet sword, even though literally everyone carried a knife as a general purpose tool including at the dinner table. Also, I hope you like blinding color, because as it turns out, that was everywhere, too. Their world was far from gray and brown; rose madder (reds and pinks), woad (blue), and weld (yellow) were all easily accessible natural dyes that allowed for some vibrant colors, including the Kelly Green famously worn by Robin Hood and his Merry Men, which was just woad overdyed with weld. The idea that everything was colored like dirt is just dead wrong; people have always loved themselves some colors and that doesn’t even cover things like uaine (dark green from the iris leaf), cudbear (purple from lichen) or frankly several other colors I won’t exhaustively enumerate here. People had color; it wasn’t invented in the 1950s. Never mind the local banners on all the cities with whatever coat of arms the people in charge plastered the place with, flags, pennants, you name it. In fact, for the longest time, a woman’s wedding dress was most properly red, not white. And she’d continue to wear it for special occasions for as long as it could be adjusted and mended. Yeah, things weren’t actually quite so terrible unless there was a pandemic.
So why bother with elves?
I think Tolkein’s use of elves and dwarves has far more to do with social status than race, to be brutally honest. Or, to put it this way, while there are definite pre-existing stories that prove to be an adequate source of some of the traits he ascribes to the fantasy races, the idea of an "elf" as a tall, slender, beautiful person depends heavily on your definition of "elf" and what type specifically you’re talking about, because an "elf" up until that point was not any one thing; it was just another word for the vast array of fey. The definition he ascribed to seems to lean mostly specifically to the Summer Court of seeley fey and A Midsummer Night’s Dream probably didn’t hurt any. But, I stress this, his pointers are absolutely from whatever imagery he could glean about the nobility, which is pretty clearly who they stand in for, which is probably why all of them but Legolas spend their screen time sitting on their thumbs.
The legacy of dwarves as smiths has a basis in existing legend, the humans are more or less as one would expect from a narrow view of the Middle Ages, and the hobbits, in a way, exemplify an idyllic version of a simpler rural life derived from a fondness of farmers along with other working class folk he’d found as an officer during the war. The point is none of these races have any particular strife between them and all of them more or less keep to separate industries, so there isn’t much class warfare represented, either.
Tolkein ultimately borrowed some things from here and there to mix into a world pitted against an unambiguously evil enemy that, in many ways, represents some corruption of all of them except for the hobbits themselves, who are shown to be "pure" or at least "less corruptible" in the sense of not having any real desires or ambitions.
So where did the problem start?
Mostly in the first attempts to have everyone anything but united against one common, unambiguously evil enemy, i.e. when it actually started being treated as serious business.
D&D in many ways I think is more at fault for drawing racial divisions both just in the act of applying solid numbers to it all and in taking certain concepts like elves as haughty and looking down on other races to an extreme Tolkein didn’t. Geek spaces being the open, welcoming places they are where gatekeeping isn’t a thing had absolutely no influence whatsoever in spreading and magnifying it . /s (← That’s a Sarca; feel free to use it for your own purposes. It’s my thing and you have my permission.)
So, the problem was more or less white male players magnifying something that was already problematic through their own lens in a way that helped it spread.
This helped create a cycle where whiteness, which was already the focus of society and history books, ended up with another segment in its own oruboros. Fiction depicting history as white slotted straight in with history books depicting history as white and when you never see anything outside of that, you never know anything outside of that exists. You can only experience the world you’re shown. It’s like driving alone at night; your vision only goes as far as your headlights illuminate. Anything on the road beyond them is completely unknown to you. That’s why it’s important to move forward.
And then straight from there, you get the problems where fantasy races started to stand in for different colors of human. Because Heaven help anyone address racial tensions between humans. No, humans, inasmuch as different colors of them are even present, always have things blissfully figured out with other humans. It’s always the elves vs. the dwarves, or some race ending up as the underclass somehow (often humans, especially to elves) so you know who to root for. And the amount of First Nations iconography that got mixed in with orcs over the years could probably be a Rant of its own, only one that would be 90% just me banging my head on the keyboard. People doing this stuff are NOT SUBTLE, and they always handle it poorly.
So what’s the solution?
Finding another pond entirely to fish out of. Tolkein did his fishing and threw it into a barrel everyone else started fishing out of, but there are many, many other pools.
The Standard Fantasy Setting only exists in any general form because people refuse to look outside of those comfortable bounds. World of Warcraft owes its lore to Tolkein, but Allods and Perfect World decidedly don’t. Allods might be a little closer, but Perfect World takes more influence from Asia. Ultimately, both are long-running games that have done just fine without Tolkein as an influence.
The Standard Fantasy Setting is perhaps useful to get the broad strokes of the world down to hopefully focus on other things, but all of it has been done in every way it can. Dragon Age made the elves live in huts and threw the dwarves into near-Klingon levels of family political maneuvering; Warcraft has made orcs and trolls noble and intelligent; The Elder Scrolls has managed to take orcs from being savage monsters TO noble and civilized people. And don’t kid yourself, The Elder Scrolls owes more than a little to D&D, especially Arena where it more or less stole the weapon and armor tables outright. Most of the games of the time and genre did. The licensed D&D games were genre-defining; of course everyone was going to borrow things from the games to beat! Anyway, Kingdoms of Amalur was in its own way notable for more or less playing the whole thing straight. And, like, what do you even do from there? Really, there’s not much you can do other than throw elves in the caves and dwarves in the trees just to be contrary. And I don’t think there’s a way you could make it constructive.
The Standard Fantasy Setting has been played out. Ultimately that’s what happens when you have a default for long enough. World War II has been played out, too, between all the various identical shooters that have taken place in it. When there’s a finite amount of something, eventually you will twist and turn it in every way you can until there’s nothing left to do with it. We have long since reached that point with the Standard Fantasy Setting. Worse yet, it’s a breeding ground for lazy writing by people who are not as clever as they think they are and nine times out of ten are a white man writing about some minority’s experiences without ever having experienced it personally. If that last sentence offends you, let me lay something down: oppressed voices should be uplifted and unless they asked you, you are not the one who should be telling their story. If you’re going to go and tell it anyway, at least consult them for it, repeatedly, throughout the process, and listen to their feedback. If this sounds like too much work because there’s a story you want to tell, then congratulations, you should not be telling theirs.
Basically, the Standard Fantasy Setting cannot properly stand on its own anymore because it’s past the point of being boring. At most it can serve as a backdrop for something much more novel and interesting and that can be a game of chicken. If your additions couldn’t just as easily be propped against the real world as a backdrop, the Standard Fantasy Setting isn’t going to save it. Ultimately, we’ve seen almost as much of the Standard Fantasy Setting as we have of the real world. The only thing it really allows is an easier justification for breaking physics at this point.
If you’re writing something and have made the decision to use some variant of the Standard Fantasy Setting, maybe sleep on it until better ideas come.