My thoughts is that it's a simple situation really. If they're harassing or assaulting people, the women will call the cops or something, simple situation and get the guy arrested. If he's not doing anything, it's nothing harmful. Apparently that's not a solid enough answer. What should I have said?
The argument made by the person asking that question is not based on any realistic fear they have actually thought through. On the big list of gendered violence, a cis dude saying they are trans to accomplish... being present in a bathroom without themselves getting harrassed... doesn't even register.
Instead, this is just a typical attempt to punch down based on an imagined danger, usually on someone else's behalf. There's nothing an oppressor likes more than a piece of illogic that frames their target of oppression as, sneakily, the real oppressor. In this case, it is trans people wanting bathroom access without having to out themselves or to otherwise just not contradict their gender. Gender neutral bathrooms would also be acceptable, or even better, but cis people have insisted on gendered bathrooms. Rather than say, "yeah trans people shouldn't have to put up with that cis bathroom bullshit", reactionaries try to come up with ways to reframe the ask as its own form of oppression. This is also the mainstay of TERFs, basically just cis women that pretend trans women are oppressing them by wanting basic equal consideration.
Depending on who this person is, you may want to try various tactics. If you think you can be flippant and dismissive of them, definitely do that preferentiallu. Make them feel like they are being ridiculous and that if they really care about violence against women they should work on guaranteeing housing for all and hate cops. If you can't be flippant, I recommend still not taking it seriously but just changing the subject to the actual act of oppression and telling them more or less what I said in previous paragraphs. All of this assumes this us a person whose mind you want to change, like that of a friend or family member. If it is a colleague, start documenting their transphobuc comments and see if anyone else has your concerns, get their documentation as well. If it is just some rando, just tell them to shut the fuck up with bad faith transphobia and move on.