I’m told book four is the hardest to get through and most polarizing, so I haven’t attempted it.

That's actually my favourite (either that one or book five), though I can see how it could be polarizing. I suggest giving it a try.