I think the problem here is, where do you cram the non physical portion in? Is there an interaction point with the physical?

Either the being with qualia never interacts meaningfully with the object, in which case conscious beings are just floating spirits pretending they make decisions and free will is irrelevant.

Or Qualia interacts with the physical body in some way that a direct simulation does not, producing a different result, in which case we can design an experiment to test where that interaction begins and the needed components for that.

I agree that any simplified or limited understanding of sensation wouldn't be the same as subjective experience - even fully simulating a human mind wouldn't be sufficient - but I don't think that means experience can't be physical.

Imagine an impossibly complex computer, purely composed of physical and electrical components. Imagine it's capable of self-observation, that it has a basic form of consciousness. We could document every bit of that computer's innards, understand every input and output. We could scrutinize its self-observation. We still wouldn't know what it feels like to "be" that computer. We couldn't access its qualia, even though it would be purely physical.

And yeah, the obvious rejoinder to that is that it wouldn't be experiencing qualia, and wouldn't be actually conscious. To me that feels like solipsism, but I admit that it's impossible to know. This whole thought experiment might be circular, actually. If a purely physical thing can experience qualia, then qualia must be physical, but if qualia aren't physical then a purely physical thing couldn't experience them. That question might be irresolvable.

This is conjecture more than anything and I could be proven wrong, but I don’t this is another ‘science will figure it out when it gets advanced enough’ issue.

Agreed. I always felt like assuming science is capable of infinite progression to be pure hubris. There's no guarantee we don't hit an intractable problem in every single field, and I'd be willing to wager real cash money we eventually do.

You can theorize of a hypothetical being that knows the exact atomic make-up of a red object down to the smallest molecule and exactly how it triggers the color-cones in our eyes but that being would not phenomenally know the experience of redness unless they had observed it for themselves subjectively.

I disagree with this though, or rather the more general variant that gets passed around. I see absolutely no reason qualia couldn't be, or arise from, purely physical interactions. In the more general example I've seen it's usually phrased as something like "A being that possesses omniscience of the concept of Red, but has not experienced vision of it." Which seems like a either a non-sequetor or misses the point entirely.

In the first interpretation I cant see how you could have a being that has "Knowledge of all possible notions of a concept" without experiencing qualia, because if it had it wouldn't have had "Knowledge of all possible notions." Congrats, you made a god that cant lift stones and gave it a rock.

In the second interpretation im just like "Ok? What now though?" Visual qualia and cognitive qualia aren't the same thing, cool, agreed. Somebody that can see but has no idea of how vision works is also lacking the qualia of seeing within that context, or having thoughts related to it. Locked out of entire possible mind-space. Simply seeing Red is a different qualia than understanding Red, which is yet still a different qualia than doing both or neither. A blind person and an infant are both qualically(?how tf do you adverb qualia?) starved.

Edit: I should probably say I think materialist panpsychism is dope and good. Probably should have lead with that. Sorry.