I think one of the big problems with the X-Men and the metaphor kind of being forgotten over thr years is that as time has gone on, they keep forgetting that a lot of mutants don't have powers, they're just weird-looking

This of course means that the majority of mutants people see in Marvel comics and cartoons and shows are the attractive, powerful ones

This leads people to see the situation as less of an oppression narrative and more of a Tall Poppy Syndrome, with themselves in the place of the Mutants being held down by the inferior normal humans

It really doesn't help that the majority of the films refuse to engage in the oppressed group metaphor beyond a very surface level viewing, instead opting for an easier to grasp (and less meaningful) Xavier vs. Magneto situation, with even the seeming exceptions eventually devolving into Magneto showing up and usurping the villain role

The big exception to this is X-Men 2, but even then it strips away much of the commentary its inspiration had made about the intersection of American Evangelicalism and bigotry