If you get rid of settlers capitalism will just make new settlers out of whoever is left. Imperialism didn't come about just because someone decided that oppression was good. It had to exist because it was the only way to increase profits and grow without hurting domestic spending. They needed cheaper labor and resources so they could grow domestic production and consumption. If you took away all these people and replaced them with non-settlers then the same thing would happen. There would be some different cultural things happening because different cultures are involved, but the fundamental economic conditions and problems still exist.

If you're pressing the button in an attempt to get rid of capitalism, I wouldn't because getting rid of people doesn't get rid of the system. It's a global system now and became such during colonialism.