Even if they didn't, that argument would only hold up until the very first treaty signed between colonizers and indigenous peoples. 99% of the story of colonization isn't people showing up to uncontacted tribes, it's settlers breaking their agreements with contacted ones.