If you come in giving signals that you're willing to pay more for something, or if you're planning on "flipping" a house, or if you patronize more expensive businesses in a cheaper neighborhood, then you have something to worry about.

Otherwise, pay no mind, because it's largely a way of guilt-tripping individuals for the actions of large economic entities.