Precisely this. If wealth weren't a means of isolating yourself from the common ills of poor society then it wouldn't be worth accumulating. As a natural byproduct of that isolation, privilege manifests itself within those who have means.