I feel like I agree with all of this and obviously current AI is relatively hollow. But that's the point, it's not successfully simulating a human mind and conversing, it's just replicating bits of speech in legible ways.

The point of the focus on language is that it betrays an entire complex of the mind - If we can converse meaningfully about novel topics, then we can almost guaranteedly talk about an how an animal behaves, and replicate it, and come up with a physicality for doing so, etc. This is the fundamental hurdle and there's nothing magic about the human brain that allows, or anything magic about the digital form that disallows it.