I expect it had something to do with the Highland Clearances. The landlord class was undertaking a gradual process of enclosure and land commoditization, imposing changes in agriculture to maximize rental incomes which involved evictions and eventually deportations of large numbers of highlanders.

Large scale projects of displacement and proletarianization are often accompanied by a narrative of racial inferiority to explain and justify them.

so by the victorian period industrialization has basically destroyed/is destroying rural life across the whole of Britain, and statutes specifically against highlanders were a bit less relevant as they'd already 'worked' to break catholic political & social organization--this is like a century after their defeat in the last jacobite war.

yeah it comes down to 'normal' kinds of racist/xenophobic attitudes we observe basically everywhere during industrialization, where labor is divided and set upon itself through the divides of regionalism. highlanders were targets because they were newcomers to the cities who'd work for less (like any new migrant population under capitalism), and because they might be catholic.

it might be helpful to imagine them as irish who were living in Britain because just like proper irish who moved to British cities they were away from home and tossed into an environment where they'd attract the ire of the rest of the working class for depressing wages, get extra exploited by the bosses, and face state-side pressures to abandon cultural mores & language to 'integrate'.

the victorian period doesn't get nearly enough guff (i know it gets a lot) but this is the period when previous feudal political dominations start to do the actual damage poor people and languages start dying and shit. its not as flashy as heroic rebellions and bloody suppressions but what actually finally killed (or very nearly did) the rural minorities of europe were the movements of capitalist accumulation and the state infrastructures for facilitating proletarian order. kick the petty farmers off the countryside with high rents, bring them to the cities, discriminate to pay less, build schools (your language, not theirs), create police. just a totalizing machine that grinds the people for profit, highlanders were one of the last peoples fed into it in Britain.

here's an askhistorians thread with cool info/discussion

here's another with earlier immigrant experiences closer to the jacobite wars

idk with how much you think about stuff like this i'd really encourage you to deep dive into the literature, you can only get so much from posts. something as specific as 'highlander going low in the victorian period' probably has first hand accounts as well as monographs. first place to look is ashistorians' booklist/the sources for relevant answers, then an actual library (metropolitan or university), then google.

@Dolores - tagging you since user tags in post titles / bodies don’t work