Same. If I get rejected because I'm a woman with a penis and someone doesn't like dicks, I'm not going to feel attacked because I'm not being rejected for being trans.

But if I get bottom surgery and a sweet new vagina, and someone else thinks I'm a woman with a vagina, they really need to examine themselves if they suddenly change their minds when they find out the vagina is a new feature. At that point, the only reason why I'm being rejected is because I'm trans. OR, maybe it's because I have a lot of body hair! In which case, that's fine by me. I'm not being singled out for being trans, there's plenty of cis women who like their body hair and some people don't find that appealing.

Tl;dr I think it's always possible to reject someone without being transphobic. So when someone says they're not interested because I'm trans, then they're being transphobic, because I guarantee there exists a trans person who they would otherwise find attractive.