Neither “the magic is going away” nor “the magic is just beginning” but “the magic has been around for fifty years and society has only partly adjusted”
Okay, look. Back in my day, we were warned about Otherrealm safety, right? Don't give your real name, don't make promises, pay attention to exact words, all of that. Yes, maybe people were being excessively paranoid, but also, we understood there are bad things out there. And these days, kids just put their full names on Faerbook? And tell the things with teeth where they live? I think we have gotten so comfortable with this stuff that we're raising an arcanely illiterate generation, that's what we're doing. Oh, and don't get me started on people relying on eyeless chattering constructs rather than actual scrying. I blame Big Thaum, I really do.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
The reason for their newfound popularity is obvious: the rise and rise of algorithmic management tools, in which your boss is an app. That IBM slide is right: turning an app into your boss allows your actual boss to create an "accountability sink" in which there is no obvious way to blame a human or even a company for your maltreatment:
App-based management-by-bossware treats the bug identified by the unknown author of that IBM slide into a feature. When an app is your boss, it can force you to scab:
But tech giveth and tech taketh away. Digital technology is infinitely flexible: the program that spies on you can be defeated by another program that defeats spying. Every time your algorithmic boss hacks you, you can hack your boss back:
Technologists and labor organizers need one another. Even the most precarious and abused workers can team up with hackers to disenshittify their robo-bosses:
One tech-savvy group on the cutting edge of dismantling the Torment Nexus is Algorithms Exposed, a tiny, scrappy group of EU hacker/academics who recruit volunteers to reverse engineer and modify the algorithms that rule our lives as workers and as customers:
Algorithms Exposed have an admirable supply of seemingly boundless energy. Every time I check in with them, I learn that they've spun out yet another special-purpose subgroup. Today, I learned about Reversing Works, a hacking team that reverse engineers gig work apps, revealing corporate wrongdoing that leads to multimillion euro fines for especially sleazy companies.
One such company is Foodinho, an Italian subsidiary of the Spanish food delivery company Glovo. Foodinho/Glovo has been in the crosshairs of Italian labor enforcers since before the pandemic, racking up millions in fines – first for failing to file the proper privacy paperwork disclosing the nature of the data processing in the app that Foodinho riders use to book jobs. Then, after the Italian data commission investigated Foodinho, the company attracted new, much larger fines for its out-of-control surveillance conduct.
As all of this was underway, Reversing Works was conducting its own research into Glovo/Foodinho's app, running it on a simulated Android handset inside a PC so they could peer into app's data collection and processing. They discovered a nightmarish world of pervasive, illegal worker surveillance, and published their findings a year ago in November, 2023:
That report reveals all kinds of extremely illegal behavior. Glovo/Foodinho makes its riders' data accessible across national borders, so Glovo managers outside of Italy can access fine-grained surveillance information and sensitive personal information – a major data protection no-no.
Worse, Glovo's app embeds trackers from a huge number of other tech platforms (for chat, analytics, and more), making it impossible for the company to account for all the ways that its riders' data is collected – again, a requirement under Italian and EU data protection law.
All this data collection continues even when riders have clocked out for the day – its as though your boss followed you home after quitting time and spied on you.
The research also revealed evidence of a secretive worker scoring system that ranked workers based on undisclosed criteria and reserved the best jobs for workers with high scores. This kind of thing is pervasive in algorithmic management, from gig work to Youtube and Tiktok, where performers' videos are routinely suppressed because they crossed some undisclosed line. When an app is your boss, your every paycheck is docked because you violated a policy you're not allowed to know about, because if you knew why your boss was giving you shitty jobs, or refusing to show the video you spent thousands of dollars making to the subscribers who asked to see it, then maybe you could figure out how to keep your boss from detecting your rulebreaking next time.
All this data-collection and processing is bad enough, but what makes it all a thousand times worse is Glovo's data retention policy – they're storing this data on their workers for four years after the worker leaves their employ. That means that mountains of sensitive, potentially ruinous data on gig workers is just lying around, waiting to be stolen by the next hacker that breaks into the company's servers.
Reversing Works's report made quite a splash. A year after its publication, the Italian data protection agency fined Glovo another 5 million euros and ordered them to cut this shit out:
As the report points out, Italy is extremely well set up to defend workers' rights from this kind of bossware abuse. Not only do Italian enforcers have all the privacy tools created by the GDPR, the EU's flagship privacy regulation – they also have the benefit of Italy's 1970 Workers' Statute. The Workers Statute is a visionary piece of legislation that protects workers from automated management practices. Combined with later privacy regulation, it gave Italy's data regulators sweeping powers to defend Italian workers, like Glovo's riders.
Italy is also a leader in recognizing gig workers as de facto employees, despite the tissue-thin pretense that adding an app to your employment means that you aren't entitled to any labor protections. In the case of Glovo, the fine-grained surveillance and reputation scoring were deemed proof that Glovo was employer to its riders.
Reversing Works' report is a fascinating read, especially the sections detailing how the researchers recruited a Glovo rider who allowed them to log in to Glovo's platform on their account.
As Reversing Works points out, this bottom-up approach – where apps are subjected to technical analysis – has real potential for labor organizations seeking to protect workers. Their report established multiple grounds on which a union could seek to hold an abusive employer to account.
But this bottom-up approach also holds out the potential for developing direct-action tools that let workers flex their power, by modifying apps, or coordinating their actions to wring concessions out of their bosses.
After all, the whole reason for the gig economy is to slash wage-bills, by transforming workers into contractors, and by eliminating managers in favor of algorithms. This leaves companies extremely vulnerable, because when workers come together to exercise power, their employer can't rely on middle managers to pressure workers, deal with irate customers, or step in to fill the gap themselves:
Only by seizing the means of computation, workers and organized labor can turn the tables on bossware – both by directly altering the conditions of their employment, and by producing the evidence and tools that regulators can use to force employers to make those alterations permanent.
i'm so glad goncharov happened when it did, right before prolific public use of AI. that was pure honest gaslighting straight from the heart. real human whimsicality and trickery thru blood sweat and tears. we were a family. and we all gonched, together. you cant replicate that with any machine.
I have been on a Willy Wonkified journey today and I need y'all to come with me
It started so innocently. Scrolling Google News I come across this article on Ars Technica:
Dull in-person warehouse for kids doesn't live up to technicolor AI-generated promo images.
At first glance I thought what happened was parents saw AI-generated images of an event their kids were at and became concerned, then realized it was fake. The reality? Oh so much better.
On Saturday, event organizers shut down a Glasgow-based "Willy's Chocolate Experience" after customers complained that the unofficial Wonka-inspired event, which took place in a sparsely decorated venue, did not match the lush AI-generated images listed on its official website.... According to Sky News, police were called to the event, and "advice was given."
Thing is, the people who paid to go were obviously not expecting exactly this:
But I can see how they'd be a bit pissed upon arriving to this:
It gets worse.
"Tempest, how could it possibly--"
source of this video that also includes this charming description:
Made up a villain called The Unknown — 'an evil chocolate maker who lives in the walls'
There is already a meme.
Oh yes, the Wish.com Oompa Loompa:
Who has already done an interview!
As bad (and hilarious) as this all is, I got curious about the company that put on this event. Did they somehow overreach? Did the actors they hired back out at the last minute? (Or after they saw the script...) Oddly enough, it doesn't seem so!
Given what I found when poking around I'm legit surprised there was an event at all. Cuz this outfit seems to be 100% a scam.
The website for this specific event is here and it has many AI generated images on it, as stated. I don't think anyone who bought tickets looked very closely at these images, otherwise they might have been concerned about how much Catgacating their children would be exposed to.
Yes, Catgacating. You know, CATgacating!
I personally don't think anyone should serve exarserdray flavored lollipops in public spaces given how many people are allergic to it. And the sweet teats might not have been age appropriate.
Though the Twilight Tunnel looks pretty cool:
I'm not sure that Dim Tight Twdrding is safe. I've also been warned that Vivue Sounds are in that weird frequency range that makes you poop your pants upon hearing them.
Yes, Virginia, these folks used an AI image generator for everything on the website and used Chat GPT for some of the text! From the FAQ:
Q: I cannot go on the available days. Will you have more dates in the future?
A: Should there be capacity when you arrive, then you will be able to enter without any problems. In the event that this is not the case, we may ask you to wait a bit.
Fear not, for this question is asked again a few lines down and the answer makes more sense.
Curious about the events company behind this disaster, I took myself over to the homepage of House of Illuminati and I was not disappointed.
I would 100% trust these people to plan my wedding.
This abomination of a website is a badly edited WordPress blog filled with AI art and just enough blog posts to make the casual viewer think that it's a legit business for about 0.0004 seconds.
Their attention to detail is stunning, from how they left up the default first post every WP blog gets to how they didn't bother changing the name on several images, thus revealing where they came from. Like this one:
With the lovely and compact filename "DALL·E-2024-01-30-09.50.54-Imagine-a-scene-where-fantasy-and-reality-merge-seamlessly.-In-the-foreground-a-grand-interactive-gala-is-taking-place-filled-with-elegant-guests-i.png"
"Concept.png" came from the same AI generator that gets text almost, but not quiiiiiite right:
There are a suspicious number of .webp images in the uploads, which makes me think they either stole them from other sites where AI "art" was uploaded or they didn't want to pay for the hi-res versions of some and just grabbed the preview image.
The real fun came when I noticed this filename: Before-and-After-Eventologists-Transformation-Edgbaston-Cricket-Ground-1024x1024-1.jpg and decided to do a Google image search. Friends, you will be shocked to hear that the image in question, found on this post touting how they can transform a boring warehouse into a fun event space, was stolen from this actual event planner.
Even better, this weirdly grainy image?
From a post that claims to be about the preparations for a "Willy Wonka" experience (we'll get to this in a minute), is not only NOT an actual image of anyone preparing anything for Illuminati's event, it is stolen from a YouTube thumbnail that's been chopped to remove the name of the company that actually made this. Here's the video.
If you actually read the blog posts they're all copypasta or some AI generated crap. To the point where this seems like not a real business at all. There's very specific business information at the bottom, but nothing else seems real.
As I said, I'm kinda surprised they put on an event at all. This has, "And then they ran off with all our money!" written all over it. I'm perplexed.
And also wondering when the copyright lawyers are gonna start calling, because...
This post explicitly says they're putting together a "Willy Wonka’s Chocolate Factory Experience" complete with golden tickets.
Somewhere along the line someone must have wised up, because the actual event was called "Willys Chocolate Experience" (note the lack of apostrophe) and the script they handed to the actors about 10 minutes before they were supposed to "perform" was about a "Willy McDuff" and his chocolate factory.
As I was going through this madness with friends in a chat, one pointed out that it took very little prompting to get the free Chat GPT to spit out an event description and such very similar to all this while avoiding copyrighted phrases. But he couldn't figure out where the McDuff came from since it wasn't the type of thing GPT would usually spit out...
Until he altered the prompt to include it would be happening in Glasgow, Scotland.
You cannot make this stuff up.
But truly, honestly, I do not even understand why they didn't take the money and run. Clearly this was all set up to be a scam. A lazy, AI generated scam.
Everything from the website to the event images to the copy to the "script" to the names of things was either stolen or AI generated (aka stolen). Hell, I'd be looking for some poor Japanese visitor wandering the streets of Glasgow, confused, after being jacked for his mascot costume.
