I guess I see selfishness as a trait you can only attribute to people who understand the consequences of what they're doing and choose to do it anyways. For that reason, I don't call people "selfish" who have kids because they lack access to contraception or because they sincerely believe that bringing more kids into the world is legitimately good and that they're doing the right thing, whether that be due to ignorance, religious belief, societal norms, or necessity for their own survival. The "desire to procreate" isn't always literally just the desire to create a new person in your own image and I think it's really reductive to say it is.

I still think it's wrong regardless of who is doing it, but I would only call it selfish when it's the intentional plan of educated people who have their material needs met and understand that whatever happiness they're creating is likely to be equalled if not outweighed by suffering. I realize that's a load of caveats, but that's how I see it.