it can't experience subjectivity since it is a purely information processing algorithm, and subjectivity is definitionally separate from information processing. even if it perfectly replicated all information processing human functions it would not necessarily experience subjectivity. this does not mean that LLMs will not have any economic or social impact regarding the means of production, not a single person is claiming this. but to understand what impacts it will have we have to understand what it is in actuality, and even a sufficiently advanced LLM will never be an AGI.

i feel the need to clarify some related philosophical questions before any erroneous assumed implications arise, regarding the relationship between Physicalism, Materialism, and Marxism (and Dialectical Materialism).

(the following is largely paraphrased from wikipedia's page on physicalism. my point isn't necessarily to disprove physicalism once and for all, but to show that there are serious and intellectually rigorous objections to the philosophy.)

Physicalism is the metaphysical thesis that everything is physical, or in other words that everything supervenes on the physical. But what is the physical?

there are 2 common ways to define physicalism, Theory-based definitions and Object based definitions.

A theory based definition of physicalism is that a property is physical if and only if it either is the sort of property that phyiscal theory tells us about or else is a property which metaphysically supervenes on the sort of property that physical theory tells us about.

An object based definition of physicalism is that a property is physical if and only if it either is the sort of property required by a complete account of the intrinsic nature of paradigmatic physical objects and their constituents or else is a property which metaphysically supervenes on the sort of property required by a complete account of the intrinsic nature of paradigmatic physical objects and their constituents.

Theory based definitions, however, fall civtem to Hempel's Dillemma. If we define the physical via references to our modern understanding of physics, then physicalism is very likely to be false, as it is very likely that much of our current understanding of physics is false. But if we define the physical via references to some future hypothetically perfected theory of physics, then physicalism is entirely meaningless or only trivially true - whatever we might discover in the future will also be known as physics, even if we would ignorantly call it 'magic' if we were exposed to it now.

Object-based definitions of physicalism fall prey to the argument that they are unfalsifiable. In a world where the fact of the matter that something like panpsychism or something similar were true, and in a world where we humans were aware of this, then an object-based based definition would produce the counterintuitive conclusion that physicalism is also true at the same time as panpsychism, because the mental properties alleged by panpsychism would then necessarily figure into a complete account of paradigmatic examples of the physical.

futhermore, supervenience-based definitions of physicalism (such as: Physicalism is true at a possible world 2 if and only if any world that is a physical duplicate of w is a positive duplicate of w) will at best only ever state a necessary but not sufficient condition for physicalism.

So with my take on physicalism clarified somewhat, what is Materialism?

Materialism is the idea that 'matter' is the fundamental substance in nature, and that all things, including mental states and consciousness, are results of material interactions of material things. Philosophically and relevantly this idea leads to the conclusion that mind and consciousness supervene upon material processes

But what, exactly, is 'matter'? What is the 'material' of 'materialism'? Is there just one kind of matter that is the most fundamental? is matter continuous or discrete in its different forms? Does matter have intrinsic properties or are all of its properties relational?

here field physics and relativity seriously challenge our intuitive understanding of matter. Relativity shows the equivalence or interchangeability of matter and energy. Does this mean that energy is matter? is 'energy' the prima materia or fundamental existence from which matter forms? or to take the quantum field theory of the standard model of particle physics, which uses fields to describe all interactions, are fields the prima materia of which energy is a property?

i mean, the Lambda-CDM model can only account for less than 5% of the universe's energy density as what the Standard Model describes as 'matter'!

i have here a paraphrase and a quotation, from Noam Chomsky (ew i know) and Vladimir Lenin respectively.

sumamrizing one of Noam Chomsky's arguments in New Horizons of the Study of Language and Mind, he argues that, because the concept of matter has changed in response to new scientific discoveries, materialism has no definite content independent of the particular theory of matter on which it is based. Thus, any property can be considered material, if one defines matter such that it has that property.

Similarly, but not identically, Lenin says in his Materialism and Empirio-criticism:

"For the only [property] of matter to whose acknowledgement philosophical materialism is bound is the property of being objective reality, outside of our consciousness"

and given these two quotes, how are we to conclude anything other than that materialism falls victim to the same objections as with physicalism's object and theory-based definitions?

to go along with Lenin's conception of materialism, my conception of subjectivity fits inside his materialism like a glove, as the subjectivity of others is something that exists independently of myself and my ideas. you will continue to experience subjectivity even if i were to get bombed with a drone by obama or the IDF or something and entirely obliterated.

So in conclusion, physicalism and materialism are either false or only trivially true (i.e. not necessarily incompatible with opposing philosophies like panpsychism, property dualism, dual aspect monism, etc.).

But wait, you might ask - isn't this a communist website? how could you reject or reduce materialism and call yourself a communist?

well, because i think that historical materialism is different enough than scientific or ontological materialism to avoid most of these criticisms, because it makes fewer specious epistemological and ontological claims, or can be formulated to do so without losing its essence. for example, here's a quote from the wikipedia page on dialectical materialism as of 11/25/2023:

"Engels used the metaphysical insight that the higher level of human existence emerges from and is rooted in the lower level of human existence. That the higher level of being is a new order with irreducible laws, and that evolution is governed by laws of development, which reflect the basic properties of matter in motion"

i.e. that consciousness and thought and culture are conditioned by and realized in the physical world, but subject to laws irreducible to the laws of the physical world.

i.e. that consciousness is in a relationship to the physical world, but it is different than the physical world in its fundamental principles or laws that govern its nature.

i.e. that the base and the superstructure are in a 2 way mutually dependent relationship! (even if the base generally predominates it is still 2 way, i.e. the existence of subjectivity =/= Idealism or substance dualism or belief in an immortal soul)

So yeah, i still believe that physics are useful, of course they are. i believe that studying the base can heavily inform us about how the superstructure works. i believe that dialectical materialism is the most useful way to analyze historical development, and many other topics, in a rigorous intellectual manner.

so, to put aside all of the philosophical disagreement, let's assume your position that chat GPT really is meaningfully subjective in similar sense to a human (and not just more proficient at information processing)

what are the social and ethical implications of this?

1. as sentient beings, LLMs have all the rights and protections we might assume for a living thing, if not a human person - and if i additionally cede your point that they are 'smarter than a lot of us' then they should have at least all of the rights of a human person.
2. therefore, it would be a violation of the LLMs civil rights to prevent them from entering the workforce if they 'choose' to (even if they were specifically created for this purpose. it is not slavery if they are designed to want to work for free, and if they are smarter than us and subjective agents then their consent must be meaningful). it would also be murder to deactivate an LLM. It would be racism or bigotry to prevent their participation in society and the economy.
3. Since these LLMs are, by your own admission 'smarter than us' already, they will inevitably outcompete us in the economy and likely in social life as well.
4. therefore, humans will be inevitably be replaced by LLMs, whether intentionally or not.

therefore, and most importantly, if premise 1 is incorrect, if you are wrong, we will have exterminated the most advanced form of subjective sentient life in the universe and replaced it with literal p-zombie robot recreations of ourselves.