Slay the Princess as a Framework for the Cyclical Reproduction of Colonialist Narratives in Data Science & Technology
An Essay by FireflySummers
All images are captioned.
Content Warnings: Body Horror, Discussion of Racism and Colonialism
Spoilers for Slay the Princess (2023) by @abby-howard and Black Tabby Games.
If you enjoy this article, consider reading my guide to arguing against the use of AI image generators or the academic article it's based on.
Introduction: The Hero and the Princess
You're on a path in the woods, and at the end of that path is a cabin. And in the basement of that cabin is a Princess.
You're here to slay her.
If you don't, it will be the end of the world.
Slay the Princess is a 2023 indie horror game by Abby Howard and published through Black Tabby Games, with voice talent by Jonathan Sims (yes, that one) and Nichole Goodnight.
The game starts with you dropped without context in the middle of the woods. But that’s alright. The Narrator is here to guide you. You are the hero, you have your weapon, and you have a monster to slay.
From there, it's the player's choice exactly how to proceed--whether that be listening to the voice of the narrator, or attempting to subvert him. You can kill her as instructed, or sit and chat, or even free her from her chains.
It doesn't matter.
Regardless of whether you are successful in your goal, you will inevitably (and often quite violently) die.
And then...
You are once again on a path in the woods.
The cycle repeats itself, the narrator seemingly none the wiser. But the woods are different, and so is the cabin. You're different, and worse... so is she.
Based on your actions in the previous loop, the princess has... changed. Distorted.
Had you attempted a daring rescue, she is now a damsel--sweet and submissive and already fallen in love with you.
Had you previously betrayed her, she has warped into something malicious and sinister, ready to repay your kindness in full.
But once again, it doesn't matter.
Because the no matter what you choose, no matter how the world around you contorts under the weight of repeated loops, it will always be you and the princess.
Why? Because that’s how the story goes.
So says the narrator.
So now that we've got that out of the way, let's talk about data.
Chapter I: Echoes and Shattered Mirrors
The problem with "data" is that we don't really think too much about it anymore. Or, at least, we think about it in the same abstract way we think about "a billion people." It's gotten so big, so seemingly impersonal that it's easy to forget that contemporary concept of "data" in the west is a phenomenon only a couple centuries old [1].
This modern conception of the word describes the ways that we translate the world into words and numbers that can then be categorized and analyzed. As such, data has a lot of practical uses, whether that be putting a rover on mars or tracking the outbreak of a viral contagion. However, this functionality makes it all too easy to overlook the fact that data itself is not neutral. It is gathered by people, sorted into categories designed by people, and interpreted by people. At every step, there are people involved, such that contemporary technology is embedded with systemic injustices, and not always by accident.
The reproduction of systems of oppression are most obvious from the margins. In his 2019 article As If, Ramon Amaro describes the Aspire Mirror (2016): a speculative design project by by Joy Buolamwini that contended with the fact that the standard facial recognition algorithm library had been trained almost exclusively on white faces. The simplest solution was to artificially lighten darker skin-tones for the algorithm to recognize, which Amaro uses to illustrate the way that technology is developed with an assumption of whiteness [2].
This observation applies across other intersections as well, such as trans identity [3], which has been colloquially dubbed "The Misgendering Machine" [4] for its insistence on classifying people into a strict gender binary based only on physical appearance.
This has also popped up in my own research, brought to my attention by the artist @b4kuch1n who has spoken at length with me about the connection between their Vietnamese heritage and the clothing they design in their illustrative work [5]. They call out AI image generators for reinforcing colonialism by stripping art with significant personal and cultural meaning of their context and history, using them to produce a poor facsimile to sell to the highest bidder.
All this describes an iterative cycle which defines normalcy through a white, western lens, with a limited range of acceptable diversity. Within this cycle, AI feeds on data gathered under colonialist ideology, then producing an artifact that reinforces existing systemic bias. When this data is, in turn, once again fed to the machine, that bias becomes all the more severe, and the range of acceptability narrower [2, 6].
Luciana Parisi and Denise Ferreira da Silva touch on a similar point in their article Black Feminist Tools, Critique, and Techno-poethics but on a much broader scale. They call up the Greek myth of Prometheus, who was punished by the gods for his hubris for stealing fire to give to humanity. Parisi and Ferreira da Silva point to how this, and other parts of the “Western Cosmology” map to humanity’s relationship with technology [7].
However, while this story seems to celebrate the technological advancement of humanity, there are darker colonialist undertones. It frames the world in terms of the gods and man, the oppressor and the oppressed; but it provides no other way of being. So instead the story repeats itself, with so-called progress an inextricable part of these two classes of being. This doesn’t bode well for visions of the future, then–because surely, eventually, the oppressed will one day be the machines [7, 8].
It’s… depressing. But it’s only really true, if you assume that that’s the only way the story could go.
“Stories don't care who takes part in them. All that matters is that the story gets told, that the story repeats. Or, if you prefer to think of it like this: stories are a parasitical life form, warping lives in the service only of the story itself.”
― Terry Pratchett, Witches Abroad
Chapter II: The Invisible Narrator
So why does the narrator get to call the shots on how a story might go? Who even are they? What do they want? How much power do they actually have?
With the exception of first person writing, a lot of the time the narrator is invisible. This is different from an unreliable narrator. With an unreliable narrator, at some point the audience becomes aware of their presence in order for the story to function as intended. An invisible narrator is never meant to be seen.
In Slay the Princess, the narrator would very much like to be invisible. Instead, he has been dragged out into the light, because you (and the inner voices you pick up along the way), are starting to argue with him. And he doesn’t like it.
Despite his claims that the princess will lie and cheat in order to escape, as the game progresses it’s clear that the narrator is every bit as manipulative–if not moreso, because he actually knows what’s going on. And, if the player tries to diverge from the path that he’s set before them, the correct path, then it rapidly becomes clear that he, at least to start, has the power to force that correct path.
While this is very much a narrative device, the act of calling attention to the narrator is important beyond that context.
The Hero’s Journey is the true monomyth, something to which all stories can be reduced. It doesn’t matter that the author, Joseph Campbell, was a raging misogynist whose framework flattened cultures and stories to fit a western lens [9, 10]. It was used in Star Wars, so clearly it’s a universal framework.
The metaverse will soon replace the real world and crypto is the future of currency! Never mind that the organizations pushing it are suspiciously pyramid shaped. Get on board or be left behind.
Generative AI is pushed as the next big thing. The harms it inflicts on creatives and the harmful stereotypes it perpetuates are just bugs in the system. Never mind that the evangelists for this technology speak over the concerns of marginalized people [5]. That’s a skill issue, you gotta keep up.
Computers will eventually, likely soon, advance so far as to replace humans altogether. The robot uprising is on the horizon [8].
Who perpetuates these stories? What do they have to gain?
Why is the only story for the future replications of unjust systems of power? Why must the hero always slay the monster?
Because so says the narrator. And so long as they are invisible, it is simple to assume that this is simply the way things are.
Chapter III: The End...?
This is the part where Slay the Princess starts feeling like a stretch, but I’ve already killed the horse so I might as well beat it until the end too.
Because what is the end result here?
According to the game… collapse. A recursive story whose biases narrow the scope of each iteration ultimately collapses in on itself. The princess becomes so sharp that she is nothing but blades to eviscerate you. The princess becomes so perfect a damsel that she is a caricature of the trope. The story whittles itself away to nothing. And then the cycle begins anew.
There’s no climactic final battle with the narrator. He created this box, set things in motion, but he is beyond the player’s reach to confront directly. The only way out is to become aware of the box itself, and the agenda of the narrator. It requires acknowledgement of the artificiality of the roles thrust upon you and the Princess, the false dichotomy of hero or villain.
Slay the Princess doesn’t actually provide an answer to what lies outside of the box, merely acknowledges it as a limit that can be overcome.
With regards to the less fanciful narratives that comprise our day-to-day lives, it’s difficult to see the boxes and dichotomies we’ve been forced into, let alone what might be beyond them. But if the limit placed is that there are no stories that can exist outside of capitalism, outside of colonialism, outside of rigid hierarchies and oppressive structures, then that limit can be broken [12].
Denouement: Doomed by the Narrative
Video games are an interesting artistic medium, due to their inherent interactivity. The commonly accepted mechanics of the medium, such as flavor text that provides in-game information and commentary, are an excellent example of an invisible narrator. Branching dialogue trees and multiple endings can help obscure this further, giving the player a sense of genuine agency… which provides an interesting opportunity to drag an invisible narrator into the light.
There are a number of games that have explored the power differential between the narrator and the player (The Stanley Parable, Little Misfortune, Undertale, Buddy.io, OneShot, etc…)
However, Slay the Princess works well here because it not only emphasizes the artificial limitations that the narrator sets on a story, but the way that these stories recursively loop in on themselves, reinforcing the fears and biases of previous iterations.
Critical data theory probably had nothing to do with the game’s development (Abby Howard if you're reading this, lmk). However, it works as a surprisingly cohesive framework for illustrating the ways that we can become ensnared by a narrative, and the importance of knowing who, exactly, is narrating the story. Although it is difficult or impossible to conceptualize what might exist beyond the artificial limits placed by even a well-intentioned narrator, calling attention to them and the box they’ve constructed is the first step in breaking out of this cycle.
“You can't go around building a better world for people. Only people can build a better world for people. Otherwise it's just a cage.”
― Terry Pratchett, Witches Abroad
Epilogue
If you've read this far, thank you for your time! This was an adaptation of my final presentation for a Critical Data Studies course. Truthfully, this course posed quite a challenge--I found the readings of philosophers such as Kant, Adorno, Foucault, etc... difficult to parse. More contemporary scholars were significantly more accessible. My only hope is that I haven't gravely misinterpreted the scholars and researchers whose work inspired this piece.
I honestly feel like this might have worked best as a video essay, but I don't know how to do those, and don't have the time to learn or the money to outsource.
Slay the Princess is available for purchase now on Steam.
Rosenberg, D. (2018). Data as word. Historical Studies in the Natural Sciences, 48(5), 557-567.
Amaro, Ramon. (2019). As If. e-flux Architecture. Becoming Digital. https://www.e-flux.com/architecture/becoming-digital/248073/as-if/
What Ethical AI Really Means by PhilosophyTube
Keyes, O. (2018). The misgendering machines: Trans/HCI implications of automatic gender recognition. Proceedings of the ACM on human-computer interaction, 2(CSCW), 1-22.
Allred, A.M., Aragon, C. (2023). Art in the Machine: Value Misalignment and AI “Art”. In: Luo, Y. (eds) Cooperative Design, Visualization, and Engineering. CDVE 2023. Lecture Notes in Computer Science, vol 14166. Springer, Cham. https://doi.org/10.1007/978-3-031-43815-8_4
Amaro, R. (2019). Artificial Intelligence: warped, colorful forms and their unclear geometries.
Parisisi, L., Ferreira da Silva, D. Black Feminist Tools, Critique, and Techno-poethics. e-flux. Issue #123. https://www.e-flux.com/journal/123/436929/black-feminist-tools-critique-and-techno-poethics/
AI - Our Shiny New Robot King | Sophie from Mars by Sophie From Mars
Joseph Campbell and the Myth of the Monomyth | Part 1 by Maggie Mae Fish
Joseph Campbell and the N@zis | Part 2 by Maggie Mae Fish
Midjourney output from prompt: “We are developing new integrated multi-sensor, power, and communications systems to address these issues to provide autonomous operational capabilities for months to year-long in situ observations --iw 5.0 --hd --ar 4:3 --s 700″
Text was taken from a list of conference abstracts about various aspects of deep ocean mineral development. Parameters at the end specify image dimensions, quality, and “style”, which is a still mysterious parameter that somehow controls “stylization” of an image.
I’m starting to explore the relationship between (mostly jargony scientific) found text and image outputs from Midjourney’s AI model. When a piece of text is taken out of context, how does the meaning change? And how does the resulting image reflect that change in meaning?
The notion of creating an image from words strikes me as somehow deeply fundamental to the construction of reality. There are the obvious biblical/religious connotations of “the word” and God speaking the universe into existence (”let there be light and crunchy almost realistic textures and uncanny humanoid figures and questionable but convincing physical relationships between objects and spaces...”). There are similar ties to the notion of (hu)mankind being created in God’s image. Does this imply an image existed a priori? Or that humans are an image of God, the real deal?
Something something chanting, mantras, ‘Om’, ‘amen’, ‘namaste’ in spiritual traditions... Also something something legal theory with verbal testimony, bearing witness, swearing on the bible to tell the whole truth and nothing but the truth... These are stand-ins for ideas I want to explore further but don’t have the bandwidth currently to begin parsing out similarities and differences and what it all might mean.
My attitude toward generative AI like Midjourney, DALL-E, Google’s Imagen, all the others (generative of images, text, and beyond) is that conversations around the future of design work being uncertain are boring. I certainly care about designers and their livelihoods but my brief experience twiddling with Midjourney and beginning to understand its capabilities has shown me that there are deeper and more important philosophical/ethical questions these technologies raise.
What is an image? What counts as “training data”? Who owns the output? Who is the artist? Who is the creator? What are the ethics of scraping original artwork and other digital imagery from the web at large to feed to an AI? If an AI is trained entirely on data not owned by the engineers or company or who built it, to what extent should they be able to privatize the model? Does “fair use” doctrine in copyright law provide any useful frame of reference or is the world of big data and entirely new beast?
Measures of transparency are at risk of remaining empty signifiers if no critical and unbiased engagement emerges from their instalment. The key argument that this paper has thus developed is that uncoupling the value of transparency from the practical matter of how that transparency takes shape and how it is likely to be engaged with ultimately paints a limited picture. Measures toward algorithmic accountability are most effective if we consider them a property of socio-technical assemblages of people and machines. Within such assemblages, the value of transparency fundamentally depends on enlisting and maintaining critical and informed audiences.
Kemper, J., & Kolkman, D. (2018). Transparent to whom? No algorithmic accountability without a critical audience.
In fact, by looking at Big Data as agglomerations of local data, we can learn about the heterogeneity of data in general and the importance of data’s origins.
Loukissas, Yanni Alexander. 2016. “Taking Big Data Apart: Local Readings of Composite Media Collections.”
The most important thing to understand about data, no matter how big, is that it is never neutral, objective, or natural but always based in human interest. This is important not just because it is correct and will help those working with big data but also because this understanding is essential for articulating big data’s role in culture. Knowing that big data is not neutral but interested is only the first step of the bigger project: a critical theory of big data that looks at how information and media are used in the processes of power, domination, and social control. We need a robust literature around big data that draws from well-established social theoretical traditions. Just last week I asked for big data thinking that draws from the Frankfurt School. And it should also draw from the sociology of knowledge and philosophy of science.
So I was excited when I saw an academic Call for Papers for the journal Big Data and Society because the central motivating theme is exactly that big data isn’t neutral. Awesome! Well, maybe not. The framework for this issue is “critical data studies”, which is making reference to the history of “critical” theorizing, thinking centered in those issues mentioned above. Authoritarianism, governmentality, social domination, etc are all fundamental, even existential, issues for this tradition of social thought. Meanwhile, this “critical data studies” starts with “data is not disinterested” and never goes on to articulate the stakes of these interests through a frame of social power. Anything like domination and control are written out of this “critical” theory. “Ethics” is nodded to, which is kind-of code for “not power”, as Robin James rightly points out. Here are the aims for this so-called “critical data studies”:
That scope is, eh, normal. Unfortunately, most work, academic or not, is less-than interested in thinking about power and domination, I get that. But to describe a call for papers and a field of study “critical data studies” and purposefully remove such concerns is doubly offensive, first because they are being under-appreciated, but secondly because they are being subverted by co-opting the label of “critical”. In a Sociology of Knowledge or History of Science course you read about things like this: the epistemological and disciplinary moves being made to create a “critical data studies” in the journal “big data and society” that downplay, decenter, and devalue concerns about racism, sexism, class, and other vectors of social domination.
The special issue will probably be useful, but it isn’t “critical data studies”, or at least it shouldn’t be. I can’t stop anyone from calling anything “critical data studies” and admittedly the issue will help do the useful work of showing data is not disinterested so that, hopefully, a better “critical data studies” in the tradition of critical theories can articulate how these social interests are used in the processes of authority, control, and inequality. Or maybe I’m being too charitable; perhaps *all* organized thinking should at some level take these concerns into account. In any case, this call for papers in Big Data and Society doesn’t just fail to address these issues but actively subverts this thinking by taking the space of “critical” theorizing and doing it devoid of power.
This is only the Call, I’ll be interested in seeing what the papers look like, but I also know well that the Call outlines what the editors will be looking for, how peer-reviewers are chosen, and so on (I’ve authored Calls for journals and conferences). In any case, there are others who *are* working on “critical data studies” through the lens of social power, and I’m disappointed in Big Data and Society for not looking to these folks but instead working to subvert such an essential concern.
