It's not even that unusual.
JFK - 1 term. LBJ - 1 term (and would have lost the Dem primary). Nixon - 2 terms but got removed from office. Ford - 1 term. Carter - 1 term. Reagan - 2 terms. Bush - 1 term. Clinton - 2 terms. Bush - 2 terms. Obama - 2 terms (but lost a ton of votes). Trump - 1 term. Biden - 1 term.
So 4 of the last 12 served 2 complete terms, and only 3 increased their vote share on re-election.
He lost the radical left