As others have said, being essentially a backwater place where there wasn't much that other people wanted, left lots of time for the dirty hill people to figure their shit out while the older civilizations who had already figured their shit out where busy fighting each other until their civilizations started to go into decline. This gives some space for the dirty hill people who've, in a more recent sense, got their shit together to actually take advantage of the destabilization and start successful empire building.

I'm sure the deep history nerds can cite instances where England's imperial project could have be hindered but due to some fluke of weather, or a miss written message, or a military officer has a case of diarrhea at an importune moment that turned the tide of a battle or business deal or a political gambit to England's favor.

A growing number of elites, go out and carve out something for themselves as fast as possible in any way possible. So long as there was a frontier to be exploited, those trying to lay claim to it would be less likely to fight their own and take their claims, so all efforts could be focused outward on Africa and Asia and South America and North America instead of Sir Reganald Cockswaller, III getting into a pistol duel with Sir Richmond Baldknobb, Jr over who gets to be the sole supplier of sheered sheep scrotums to the Southwest part of TERF island.