I took three years of Spanish and got an A every semester. Even when it was still fresh in my mind, I was nowhere near able to hold even a very simple conversation. And now just a few years later it's all totally gone from my brain.
My mother's native language is Spanish and she never taught me, which I resent her for. But I still find it incredible how shitty my public school education in Spanish was. We really should be teaching kids a second language from kindergarten up.
Immersion only works well with acquiring a baseline through which to understand the experience. You can listen to a conversation all day long but without context through visual, audio, previous knowledge or other means you won't actually learn anything.
I know this may come off as pedantic but it's important for people who are trying to learn a language to not expect to pick up on things without having to put in a bit of book learning, even if it's just at the beginning. Jumping off into the deep end doesn't really work.
Comprehensible immersion is the key in my opinion, where you slowly build up your input ability.