There are a couple of things here I think.
-The first one is simple privilege. White people write about white people because they don't really have a reason not to. (Unless they want to write something "exotic" and that has it's own problems)
-Once you "fantasize" a culture I think they tend to float together a bit. Honestly, apart from language there's not *that* much of a difference between iron-age Sweden and iron-age Finland and Russia. Nomadic hunter-gatherers are pretty similar too (the sami live pretty much like some of the northern First Nations tribes, for instance) you need to be *really* familiar with a culture to tell them apart (most people aren't) if you strip way the superficiality.
So when fantazising (which generally involves at least fuzzing the superficial details, unless you write fantasy set in our world) you only tend to have a few cues. If someone is named Erkki or Pekka it's probably a finnic fantasy, if they're named Sven or Olof it's a scandinavian one.
-Another point is... Familiarity? Not quite the right word but... in a culture there are symbols, that basically everyone inside that culture "gets". We all know what a sword in a stone means, we know what dragons and swords and kings and farmboys mean. (and, kind of unfairly, we've exported this so that a large portion of the rest of the world, if far from everyone, also knows what our symbols mean) but the reverse isn't neccessarily true. So in order to write a non-superficial story set in another culture you need to learn their cultural "language", what symbols mean and such. And that's tricky. For us westerners, checking out how christianity is presented in anime is a good excercise... it tends to be wrong on a level that's really beyond the "Rule of Cool" (which is liberally applied!) And this is something the west does ALL the time. (or to take another example, Marve's Thor character: there's something invariably *wrong* about that character, no matter the revamp they never seem to get him right, which doesen't mean it's a bad comic, far from it, it's just vaguely unsettling)
That barrier cuts both ways by the way, westerners might be okay with an "exotic" setting, but once people starts messing with the expected cultural cues casual readers tends to be alienated.
So in other words, trying to write a setting that's more than superficial kitsch is difficult, and even if you succeed people tend to just be alienated by it.
-Then comes the other solution, keep within your own culture but add people of different ethnicities. That's much easier, it's a minor compromise on worldbuilding consistency, but if the effect is that people is more comfortable in their reading because there's a black guy there why not do it?
-SF tends to have less issues with this, they still tend to assume that western culture will be dominant/recognizable in the future, but at least they can throw in a bunch of non-whites without having to explain everything. (Since all future is american, it won't matter, or something)
Take the Dresden Files. I love the books, but when Europe and America are shown to be the centers of magical power, with more magicians, and no explanation for why billion-people countries like China and India don't have more wizards, I'm left scratching my head.
Dresden Files does the entire annoying "Oh yeah, the rest of the world exists and does stuff, they're just not important to the story." thing. (they mention there's a Jade Court, along with the Black, White & Red, for instance, presumably it's made up of hopping corpses...)