Yeah you've got it backwards. The gift (even a well-thought one) is the unimportant part. Remembering and caring enough to wish him well despite your own hangups about adequacy and money is the part that matters. If he cares about you that will be more than enough, and even more than that if you explain that it doesn't feel great to not have a gift for him. He'll understand.

For what it's worth, just about everybody below a certain income level experiences this form of alienation at one point in their lives or another. Parenting is a lot of work, and even reliably having a gift for them every year can feel like a deeply inadequate gesture. What is never, ever inadequate though, is sitting down and expressing your feelings.