Hm, there are at least three major problems to consider, I think. This is going to be a bit long, so bear with me, but I promise that I will not at any point argue in favor of copyright, argue on the grounds of human ability, and I will not at any point appeal to concepts of "soul" or "spirit."
It's fairly well documented at this point that the sheer power requirements to run AI data centers and computation have caused the major tech companies betting the farm on this technology to invest in a huge expansion of data centers, which in turn both require huge amounts of additional power (which tends to be drawn from fossil fuel sources) and huge amounts of water for cooling.
For reasons unrelated to AI as a technology as such, but related to the capitalist mode of production which produces "AI" as a product and service, the burdens and costs of creating this new infrastructure falls disproportionately on the poor, the marginalized and upon the global south, as it always does. It is of course to be noted that these tech companies are likely to also be using generative AI as an excuse to backslide on their environmental commitments and massively expand their infrastructure with government and venture capital money, but generative AI is a not-insignificant part of the problem.
The primary stated purpose of generative AI as a business is to replace workers. It is a form of automation, and generative AI specifically targets jobs requiring language and visual media skills - whether that be translation, copywriting, creative writing, coding, drawing, painting or graphic design, or any of a thousand other related skills.
The express purpose of adopting the technology widely is to replace hundreds of thousands of workers, who upon losing their jobs will be thrust into precarity, and the industries affected will experience an enormous downwards pressure on wages and an enormous negative impact on processes of unionization and collective bargaining.
One might argue that this is a problem of capitalism more than a problem of generative AI as a technology, which, okay, sure, but capitalism is the system we live under and the value and ethical status of any technology is always evaluated in the context of the system that deploys it. If we lived under Luxury Gay Space Communism, I am sure I would feel different about generative AI; but we don't, so I judge it accordingly. Under the capitalist economic mode of production, generative AI is a fundamental threat to the economic and social well-being of hundreds and hundreds of thousands of workers in hundreds of fields of labor.
Generative AI as a tool imposes harsh restrictions on freedom of expression, and in a worst-case scenario (i.e. the exact scenario that the developers of these AI tools are trying to make happen), will put huge swaths of artistic expression under the control and economic exploitation of enormous corporations unaccountable to any democratic power.
No generative AI tool can generate any output except that which is allowed for
by its programming as defined by its owners and creators
by its dataset, which must be built and maintained by an organized group
With every single major generative model currently firmly under the control of corporate or national interests, those groups have the ability to exert profound control over which outputs are or are not possible from the most powerful and most capable AI models.
No generative model can output a picture of a motorcycle unless the database for the model is trained on pictures of motorcycles, and even if it IS trained on pictures of motorcycles, it can and will only ever output motorcycles identical to whatever necessarily limited set of motorcycles are represented in its data. And if a corporation (or government) decides that a model is not allowed to output images of motorcycles, they can implement profound censorship of the concept of motorcycles in their models.
Replace "motorcycle" with "sexual education material," "political literature" or "information about or depictions of queer people or minorities" and you start to see the problem.
One might argue that "users would simply circumvent that restriction," but to say so is to miss the point entirely: savvy power users with an agenda would circumvent the system. Your grandma using the system casually would not, especially if (or when) circumventing the system is made illegal.
Similarly, one might argue that a band of dedicated, democratically minded individuals could simply band together to train and create their own independent models, free of all censorship - but again, the ability to access the infrastructure necessary to build such a project would be contingent on the assent of either corporations or national governments, and both of those groups would inevitably see a completely unbound, democratically governed AI infrastructure as a direct competitor or a political threat, and act accordingly.
Generative AI, logistically and structurally, is a tool of expression which is privileges power. Whoever holds power in a given society has undue ability to influence what it is possible to express via generative AI and who has access to the ability to express it.
Generative AI has been magnanimously made available for "free" thus far, by speculative corporations backed by oceans of venture capital, but they are all expected to turn a profit at some point, and once those screws come down, not only will they silo and closed-source their technology, they will aggressively pursue hostile action towards competitors, and limit access to their Revolutionary™ technology to whoever is most able to either pay or coerce access through force.
In short, whoever has the most money or political power will have access to the greatest degree of freedom of expression from generative AI.
3b: Freedom of expression on the purely aesthetic level
This is a less important objection than point 3 above, but it also needs to be noted that generative AI as a technology is fundamentally based upon and limited by probability.
That is to say, when you prompt a generative AI model to generate a given output, the fundamental nature of the math it uses is a probabilistic attempt to approximate an acceptable answer.
In oversimplified terms, if you ask it for a picture of a motorcycle, it will attempt to output a semi-randomized mixture of all its data which is tagged with "motorcycle," weighted against training data and a history of user feedback, in order to probabilistically arrive at whatever output it calculates the end user is most likely to accept and validate as a "correct" output.
This has some consequences, specifically that generative AI is fundamentally bound as a technology to always and forever regress to the mean. It will always and forever default to and privilege lowest common denominators.
Thus, one of the things generative AI tends to have a lot of trouble with is ugliness. It struggles to generate images which do not conform to dominant standards of beauty and desirability. And I mean this not just in terms of human beauty and desirability, but beauty and desirability across all forms of expression. Generative AI severely struggles to output an ugly landscape image, for example. It struggles to output pictures of kittens that are not cute, it struggles to output pictures of Ferraris that don't look gorgeous or swords that don't look cool. It also struggles to output images of people along the same lines.
The first part of the problem is that the vast, overwhelming amount of input of images bias towards the aesthetically pleasing (because most images that get preserved and uploaded of anything bias towards the aesthetically pleasing). The second part is that, the vast, overwhelming amount of desired and validated outputs also bias towards the aesthetically pleasing, because that is what most users most of the time will inevitably and statistically want and prefer.
Because the vast, overwhelming majority of the model's feedback from users will validate and confirm the correctness of outputs that bias beauty and appeal, models will always be statistically required to generate outputs towards that standard.
The result is that generative AI is, technologically and irrevocably, bound to bias towards and reproduce the lowest common denominator, and it uncritically inherits every single cultural bias of the dominant culture that produces and uses it. Usually this means biases in favor of beauty and appeal, and biases against whatever is considered ugly or undesirable in a culture.
Even if you are the sort of profoundly shallow and unimaginative person who believes that "nobody would ever WANT images that aren't beautiful and appealing," you have to concede that any tool which does not allow you full and equal freedom to depict the ugly and the unconventional is imposing a severe restriction of the freedom of expression of its users.
In conclusion: generative AI is defined by mounting and catastrophic social, economic and environmental costs, and fundamentally and structurally biased in favor of power and capital. Even in the very best-case possible scenario in which all economic and political problems with the technology are solved, its fundamental nature is to regress to the mean and privilege the lowest common denominator, imposing inherent restrictions on freedom of expression.
None of these problems are shared by a pencil, or by a typewriter.
