'Okay, but how does that make him wrong?'

Or 'There were plenty of poor people who had similar ideas at the time and couldn't afford the education, time, or connections to have their work published.'

Or 'If he was poor you'd say he couldn't know enough about wealthy people'

Or just pull out a glossy 8x10 of PPB and hand it to them.