I think there's a theory that this is essentially what happened to the ancestral Pueblo people. There was no big cataclysm, things were just kinda vaguely shitty till most of them just decided to fuck off and join nomadic groups.