Regarding the latter concern, I think a lot of this type of thinking comes from misconceptions about how evolution works, largely perpetuated by our culture to be fair.

But most people think evolution is an external pressure on the level of the individual. Which, it is, kinda-- that's one scope of evolution. But evolutionary pressure happens on all levels in different ways: one family against others, one tribe against others, one social group against others, one species against others, etc. And networks of cooperation are just as influential as networks of competition, all happening at the same time in a churning mass of energies.

So rather than thinking that individual humans are losing hardiness to evolution, think of it as our species gaining hardiness through specialization and technology, evolution taking place outside of our individual bodies. It's why we have language instead of tusks.