As I talked about on the weekend, I am right now trying to figure out what is happening with the auto-moderator here on tumblr and how it tags random images at times as "mature".
What I can say so far is this: whatever software tumblr uses is obviously some sort of AI/neural network that has been trained on image sets to "learn" what images are "mature" and which are not.
Right now my main theory about why the automoderator again and again and again is tagging images of Black people (especially women), fat people (specially women), disabled people (especially women), and some other minorities (like people - trans men and female cancer survivors - with top surgery scars, or folks having a female figure and a beard like some trans women and intersex people) as mature is likely image sampling bias.
As someone who works with AIs, the explanation is something like this: a lot of AI training is done by feeding it a whole lot of images with tags for whatever you want to train the algorithm on. Very often you create a baseline by using any given sort of image set online that have been created for this purpose, and then additionally seed images more specifically for what you want to train for.
I told the story before: both my university projects on neural networks I did for class involved training the AI to recognize fishes/fish behavior. Which was an issue. Because pretty much no animal image set available for training included any or at least a whole lot of fishes. So I spent days looking for and properly curating image collections and videos of fishes.
And this gets us to the typical human image bias: a lot of those image sets of humans sold to train AIs are made up of either exclusively or at least majority young, average to pretty, gender conforming, abled white people. Meaning the AI will learn the "normal and unproblematic" version of a person is a white young ablebodied thin to medium cis person. There is a good chance that in comparison to the "normal" dataset, non-white and especially Black people, fat people, disabled people, and for example trans people were way more represented in the "mature" dataset used to train the AI.
This is not just happening in tumblr, mind you. If you follow AI research you might have also heard those stories of AI-powered self-driving cars not recognizing a Black person as a human it should stop for, or how AI face recognition does not work for Black people, some other non-white people, or people with facial differences. All something that happens because the training data used is biased.
A neural network is dumb. It does not understand what porn is. So if you show the neural network a bunch of images you say are fine and those involve dressed white people doing normal things, and then you show it a set of porn data, which includes the only Black or fat people the AI gets to see during the entire training, the AI will from this come to the understanding that being Black or being fat is indeed part of what makes those images mature. Because every Black or fat person it saw was tagged with "mature" during its training. So, it now looks at a conservatively dressed Black woman and goes: "Yeah, this is clearly porn. Because all I know about Black women existing is that they sure are in porn!" Same with fat people, trans people, and disabled people.
And this is bullshit. This is racism. It is fatphobia. It is ableism. It is transphobia. And tumblr should either retrain their model or retire it completely. Because, frankly, you can quite easily get around the automoderator either way. I am not going into this now, but due to how the automoderator works actors with malicious intend - like those porn bots - can even create an automated pipeline that gets around the automoderator. If you can code - and to make a bot you need to be able to code - you can code something that changes the images just enough for that. That is the reason why we often see very explicit porn bot images that the automoderator did not tag as mature.
All it does right now is pushing further the notion that Black people, fat people, disabled people, and trans people existing is somehow inherently mature and should be hidden from the public.
Stop it, @tumblr, @staff, @support. You are just promoting those things. You are promoting racism, fatphobia, ableism and transphobia. It would be one thing if y'all actually reacted to people reporting falsely labeled posts and reversed it. But given that you have yet to correct the falsely applied tag on my post on the history of fucking santa claus that I reported last November - a post that involves no nudity or even discussion of anything mature (it is a post about the history of santa and krampus, that was either autotagged for an image with a woman with facial differences or for going into WW2 history!), let's be honest here: you are not doing that. So just retire the bloody thing, or retrain it with data that includes those minorities in non-sexual contexts.
(Also note, I am not yet quite sure what triggers the text-post automoderator. I have seen too few that have been autoflagged on that front. I know that certain things - like talking about trans experiences, but also certain historical events - will often set it off. But not always. And I am just not sure what exactly is the factor here. It might be that certain accounts are being treated more harshly on that front. It feels that way at least. But I am not sure.)
I am being so free to tag a couple of people in the hope this gets a bit more visibility. Because I think this entire shit is a big problem, and I know tumblr support will ignore it if nobody interacts.
@malacandrax @unspeakablearchives @in-the-drowning-deep @kawareo @tendermiasma @bobokitty @this-is-antiblackness @this-is-trans-solidarity @this-is-fatphobia @thisis--ableism
