Plant a small plot right in front of your house to begin with, nothing more than you can water in an hour or two on the game clock. It's easy to overextend yourself and miss events and exploration because of farming taking all day.

Fertilizer is a good investment because you get multiple uses out of it, it stays in the ground after the crop gets harvested. Just don't leave the tilled/fertilized soil barren or it might revert to a regular soil square.

If you find fishing hard, just leave it for later. It does get much easier with later equipment.