A good term for this, that Roxanne Dunbar-Ortiz, and Sakai uses (i think), is Settler Garrison. Northern Ireland, the US western frontier, Israel, are some examples.

These are the advance troops of colonialism, using terrorist methods to bully, kill, steal, and wipe out indigenous peoples. Occasionally they even have conflicts with the colonial powers on the methods / best ways to do it. The US revolutionary war was essentially a disagreement between Britain's settler garrison, and Britain trying to put a leash on their westward expansion.