Conservation of mass?
Do they understand that producing energy and fertilizer using the bodies of animals is less efficient than producing the same number of calories or mass of nitrates from plants+less-energy-than-is-required-to-raise-the-animal-in-question?
There's land that isn't good enough to grow crops, but is good enough for wild plants to grow. You can, as you said, 'manage' it - give enough fertiliser and water to make it suitable for agriculture. But that is often unaffordable for the people living in such places, so using animals to gather and concentrate the available nutrients is the best option available to them.
That's an orthogonal injustice though, no? Collectively, our species massively overproduce food, so I would think the fact that there is a prior reason to be trying to cultivate land like this, which ought to be managed for native flora/fauna is a separate and solvable large scale land allocation problem, the solution to which frees whatever livestock use the argument excuses.
The economist Amartya Sen, who studied famines in South Asia and Africa said that 'starvation is the characteristic of some people not having enough food to eat. It is not the characteristic of there being not enough food to eat." Producing more food than we need means nothing if it does not reach everyone.
I would say it is more of a food-allocation problem. Land redistribution is a great thing, and has worked in the past. But natural disasters and crop failures can occur in regions, and larger farms can benefit from long-term planning and more scientific management. So the better solution, in an ideal world, would be to focus on guaranteeing food rather than land.
Solving food security will of course greatly reduce this problem, although personally I am worried that food availability will get worse in the coming years and decades due to the various ways in which we are damaging the environment.