Yeah, the first several "generations" of US astronauts were mostly Air Force test pilots. It is a very dangerous job with a low life expectancy, and rich people usually aren't career military people to begin with. So, while the test pilots weren't poor due to the job paying at least decently, a lot were not from very wealthy or privileged, beyond being white men, background.