Okay, I've spent a long time thinking about this and I'm concluding that we're doomed to talk past each other forever because we're starting from different priors. But here's my best effort:

To suppose the existence of a "qualia pixel" without some underlying physical explanation is to presuppose the existence of some form of Cartesian dualism because you're assuming that qualia have an existence independent of the brain and can be manifested by something other than the physical interaction of matter and energy. To not grant that makes answering your question easier - I can give my brain spurious inputs by staring at a lamp too long and getting an afterimage. In this case, I'd argue what I'm "seeing": the room, the afterimage superimposed on it, as my "real vision" because that's the signal my retina is sending my brain. Damage any parts of those systems - the retina, the optic nerve, the visual centers, etc., and the qualia disappear. The conclusion is that qualia have no independent existence outside of the architecture of the brain.

Now let's tackle the argument that qualia aren't necessary for cognition and therefore aren't subject to evolution. This is true insofar as we consider them independent of the brain. The lobster is capable of receiving, internalizing, and then reacting to stimuli but it probably doesn't experience qualia as such, and from that we can conclude that cognition - at least at some levels - is capable of existing without qualia. However, to extend that and say that all cognition is possible without it or that all cognition is possible without producing qualia as a byproduct is overextending the argument. It kind of reminds me of a reverse riff on Behe's irreducible complexity argument. Behe argues that if you take some functional structure, say a bacterial flagellum, and remove any one part, it stops functioning. Thus, Behe concludes, the flagellum must have been designed in situ by some intelligent force. Hopefully the flaws in this argument are obvious: the flagellum could have arisen from parts that developed for other purposes and by happy coincidence ended up working for propulsion (another example here is feathers. The first feathered animals couldn't fly and we can conclude that feathers didn't evolve "for" flight; they ended up suiting the purpose later). Yes, we can argue that that is one hella big coincidence, but the thing about dice is if you toss them enough times they'll all come up sixes. Monkeys and typewriters. Etc.

The argument Kastrup makes is the inverse of this; let's call it the argument from reduceable complexity. He argues that because we can imagine a philosophical zombie, consciousness is unnecessary and therefore cannot have been produced by evolution. But evolution produces weird, "unnecessary" crap all the time. Take peacocks, for example. Males have those large showy tails that make it harder to move around and easier to get spotted by predators. So why did it evolve? Enter the "handicap hypothesis": a form of sexual selection where a feature that reduces survival is an honest signal of fitness because only the most vigorous males can manage to cope with it. Testosterone in humans might be an example; higher levels of testosterone suppress the immune system. The brain itself might be another given the huge energy demand that it imposes. At any rate, we can envision a species of bird that doesn't rely on a survival-reducing feature for sexual selection. Should we conclude from this premise that the peacock's tail cannot have evolved? I don't think so - evolution just seeks a local maximum from whatever was there before, and that can produce some pretty counterintuitive outcomes.

So to deal with Kastrup's central premise (i.e., consciousness/qualia could not have evolved), he would have to demonstrate the following:

• Qualia play no role in cognition, learning, memory, or communication, all of which are behaviors that can happen in the absence of conscious experience. I think that no one's been able to answer this either way because we don't have sufficient understanding of how the brain works. The argument that a materialist worldview must conclude that they don't exist is one I've been chewing on a lot, and I think the error here is one of definition. The pain qualium, for example, is something that can be produced by a certain brain state. Prevent that brain state and you have no physical perception of pain. Thus, a materialist might conclude that what we call pain is a shorthand that refers to those brain states as they present themselves to our conscious awareness. This is entirely consistent with a materialist worldview, which says that all phenomena must arise out of the interaction of matter and energy, not that all phenomena be tangible, quantifiable, and measurable.
• If qualia play no actual role and are therefore not subject to selection, then they must not be a byproduct of lower-level processes that are subject to selection. Going back to pain, or your vision example, it's possible for us to prevent the pain qualium from being manifested by messing with someone's neurons; it's not possible to manifest the pain qualium in the absence of messing with someone's neurons. All our available evidence points to embodied cognition: that the body and mind are not separable and that our lived experience can be traced to events in the physical substrate of the brain. The existence of simpler physical systems in this case is not sufficient evidence, because we've established that complexity can emerge as a consequence of natural selection and processes/phenomena like exaptation and spandrels can produce side effects that still are subject to natural laws. That qualia also correspond to the "correct" physical experience and are therefore too coincidental to be an accident also isn't persuasive because we can conclude that, if qualia arise from some lower-order functioning of the brain, those lower-order functions must bear some correspondence to reality to be useful. The qualia we experience "make sense" to us because it's literally the only perspective we have.

Now, suppose he's correct and qualia play no role in behavior etc. and are not a byproduct of processes that are subject to selection, then how do we explain their existence? I'm still not a huge fan of Kastrup's last line that consciousness is "can only have been there from the beginning as an intrinsic, irreducible fact of nature." Seriously, how does this enrich our understanding and get us closer to understanding the nature of consciousness? Does it indicate that consciousness is somehow a law like gravity and is simply 'baked in' to the boundary conditions of the universe? How is that different from a materialist/determinist point of view wrapped up in a Deepak Chopra-ism? Does it posit the existence of a "consciousness field" that ordinary matter taps into once it's reached a certain level of complexity? Does it propose the existence of some intangible Mind that is capable of manipulating matter in the brain to produce behavior in a way that's completely consistent with our current understanding of chemistry and physics? Does it propose some realm of the hyperreal where causality works differently and all ordinary matter is just a lower-order reflection of hyperreal phenomena? How are we supposed to investigate any of these possibilities?

Karsten's argument is superficially more sophisticated than "flagellum therefore the Abrahamaic god" but it follows the same format: "Science cannot explain thing X, therefore it will not be able to explain thing X, therefore unfalsifiable claim Y." His conclusion doesn't follow from the premise because he doesn't have - and cannot provide - any positive evidence to support it.