They also serve as a passive invasion deterrent. If an enemy army enters your land you retreat to your castles. The enemy now faces a choice: do they invest each castle or pass them by. If they pass them the soldiers in the castle can ride out and harass your foragers. If you lay siege to the castle and a rival army shows up, the enemy is now between the hammer and the anvil.

Not saying you're wrong, I just fuckin' love the history of castles.