Can't say without context. White privilege is a thing that exists regardless of other factors like wealth. It also doesn't mean that you have an easy life or that your perspective is invalid or something. Literally every person benefits from some kind of privilege. The concept of privilege should be used as a reminder that our perspectives are influenced by who we are and what we have and haven't experienced, and that's that. Unfortunately, in practice it can also be used to shut people down just based on who they are, which is dumb, or to play the "oppression olympics," which is even dumber.

If the context was like, "Is it possible that the reason you don't think representation in media matters that much is because you've never known what it's like to not have it, and you need to check your privilege?" then hey, could be he has a point, might be worth considering. If it's more like, "You don't have anything to complain about because you're white," then he's a fucking prick and you should tell him to shove it.