The events of Arena have made it so that everyone doubts that the Emperor's heirs are legitimate, and at one point your crackhead handler gets recalled to Cyrodiil to help out with the situation. There's grumblings of secession/rebellion as always, but nothing ever happens. Mostly though it's got the vibe of an empire that's run out of territory to conquer. The colossal failure of the efforts to invade Akavir set back the Empire a ways, and by the time Uriel and his heirs are dead in Oblivion it's just the final nail in the coffin. Later on the Thalmor rises up and absorbs Valenwood and Elsweyr, Hammerfell breaks away, Morrowind just basically stops answering the phone after the Red Year, and the Black Marsh expanded into Morrowind as vengeance for centuries of being enslaved.