So in the moral system you're proposing, it would be an offense to use those buildings without their owner's permission. The fact that people denied access to those buildings will die as a result, though, is just a part of nature

Well, yes. Do you donate every least penny you have to the homeless, since it would save lives? Or do you instead spend your money on luxuries for yourself? There are right and wrong ways to go about helping people like the homeless, stealing and ruining property isn't one of those ways.

So, why would you expect a perspective which values property more than people to stir anyone's moral feeling? If you don't expect that, then why are you bothering to frame it in terms of right and wrong?

I assume you have some property, right? If you are an adult you are probably at least renting an apartment and have some basic stuff like a TV. Why don't you let a homeless person sleep on your couch every night? If you don't have your own place yet, would you allow some random homeless person access to your couch for free every night?

It's much easier when it isn't your property in question, but when it's your place you might start to care about random people living rent free in your place.