Well first you ask why are they here and why should they have to leave?

Many times they're here because the conditions of their home countries have been deteriorating because of the countries they are fleeing to (imperialism, regime change, exploitation, resource extraction, terrorism, etc).

Many times they have to leave because it is politically convenient scapegoating, often with the old "shouldn't we take care of the homeless first" followed by passing antihomeless laws and then blamimg immigrants for it somehow. Other times its to appease racists/xenophobes, or because neolibs think borders are important.