The main reason that there's less dunmer in the rest of Skyrim is because the only other hold that had a significant population (Winterhold) straight up collapsed into the sea of ghosts. Both Windhelm and Winterhold were the closest ports for dunmer refugees, since Winterhold is now mostly rubble all that leaves is Windhelm.

You can't really claim that the grey quarter is proof of lack of racism by the Stormcloaks because the refugees that settled there were the initial wave fleeing the red year and thus way before Ulfric's time.

Later waves of refugees were granted a large portion of Solsthiem (specifically the part with all the ebony mines) and there are a couple of villagers in Raven Rock that relocated from Windhelm.

The citizens of the grey quarter aren't freshly arrived refugees, they've been there for a long time (dunmer live for centuries so some of them (but not all) are first generation).

Settling elsewhere in Eastmarch is even more dangerous for the dunmer because the guard straight up refuse to protect their farms and traveling further afield in Skyrim is an expensive and dangerous task.

Vvardenfell is still being coated in ash from an erupting red mountain so there's no going there, and traveling east lands you in black marsh where dunmer aren't welcome due to a history of House Dres enslaving the population.

So that leaves your average dunmer with two choices: resettle in Raven Rock, or remain in the Grey Quarter.

This is the equivalent of claiming the US isn't racist because some large US cities they allow minorities to live in ghettos. It's absurd.