#moral hazard

[radiowarp]

att "P.I. suit" turnaround

twitter

[mostlysignssomeportents]

Algorithmic feeds are a twiddler’s playground

Next TUESDAY (May 14), I'm on a livecast about AI AND ENSHITTIFICATION with TIM O'REILLY; on WEDNESDAY (May 15), I'm in NORTH HOLLYWOOD with HARRY SHEARER for a screening of STEPHANIE KELTON'S FINDING THE MONEY; FRIDAY (May 17), I'm at the INTERNET ARCHIVE in SAN FRANCISCO to keynote the 10th anniversary of the AUTHORS ALLIANCE.

Like Oscar Wilde, "I can resist anything except temptation," and my slow and halting journey to adulthood is really just me grappling with this fact, getting temptation out of my way before I can yield to it.

Behavioral economists have a name for the steps we take to guard against temptation: a "Ulysses pact." That's when you take some possibility off the table during a moment of strength in recognition of some coming moment of weakness:

https://archive.org/details/decentralizedwebsummit2016-corydoctorow

Famously, Ulysses did this before he sailed into the Sea of Sirens. Rather than stopping his ears with wax to prevent his hearing the sirens' song, which would lure him to his drowning, Ulysses has his sailors tie him to the mast, leaving his ears unplugged. Ulysses became the first person to hear the sirens' song and live to tell the tale.

Ulysses was strong enough to know that he would someday be weak. He expressed his strength by guarding against his weakness. Our modern lives are filled with less epic versions of the Ulysses pact: the day you go on a diet, it's a good idea to throw away all your Oreos. That way, when your blood sugar sings its siren song at 2AM, it will be drowned out by the rest of your body's unwillingness to get dressed, find your keys and drive half an hour to the all-night grocery store.

Note that this Ulysses pact isn't perfect. You might drive to the grocery store. It's rare that a Ulysses pact is unbreakable – we bind ourselves to the mast, but we don't chain ourselves to it and slap on a pair of handcuffs for good measure.

People who run institutions can – and should – create Ulysses pacts, too. A company that holds the kind of sensitive data that might be subjected to "sneak-and-peek" warrants by cops or spies can set up a "warrant canary":

https://en.wikipedia.org/wiki/Warrant_canary

This isn't perfect. A company that stops publishing regular transparency reports might have been compromised by the NSA, but it's also possible that they've had a change in management and the new boss just doesn't give a shit about his users' privacy:

https://www.fastcompany.com/90853794/twitters-transparency-reporting-has-tanked-under-elon-musk

Likewise, a company making software it wants users to trust can release that code under an irrevocable free/open software license, thus guaranteeing that each release under that license will be free and open forever. This is good, but not perfect: the new boss can take that free/open code down a proprietary fork and try to orphan the free version:

https://news.ycombinator.com/item?id=39772562

A company can structure itself as a public benefit corporation and make a binding promise to elevate its stakeholders' interests over its shareholders' – but the CEO can still take a secret $100m bribe from cryptocurrency creeps and try to lure those stakeholders into a shitcoin Ponzi scheme:

https://fortune.com/crypto/2024/03/11/kickstarter-blockchain-a16z-crypto-secret-investment-chris-dixon/

A key resource can be entrusted to a nonprofit with a board of directors who are charged with stewarding it for the benefit of a broad community, but when a private equity fund dangles billions before that board, they can talk themselves into a belief that selling out is the right thing to do:

https://www.eff.org/deeplinks/2020/12/how-we-saved-org-2020-review

Ulysses pacts aren't perfect, but they are very important. At the very least, creating a Ulysses pact starts with acknowledging that you are fallible. That you can be tempted, and rationalize your way into taking bad action, even when you know better. Becoming an adult is a process of learning that your strength comes from seeing your weaknesses and protecting yourself and the people who trust you from them.

Which brings me to enshittification. Enshittification is the process by which platforms betray their users and their customers by siphoning value away from each until the platform is a pile of shit:

https://en.wikipedia.org/wiki/Enshittification

Enshittification is a spectrum that can be applied to many companies' decay, but in its purest form, enshittification requires:

a) A platform: a two-sided market with business customers and end users who can be played off against each other; b) A digital back-end: a market that can be easily, rapidly and undetectably manipulated by its owners, who can alter search-rankings, prices and costs on a per-user, per-query basis; and c) A lack of constraint: the platform's owners must not fear a consequence for this cheating, be it from competitors, regulators, workforce resignations or rival technologists who use mods, alternative clients, blockers or other "adversarial interoperability" tools to disenshittify your product and sever your relationship with your users.

he founders of tech platforms don't generally set out to enshittify them. Rather, they are constantly seeking some equilibrium between delivering value to their shareholders and turning value over to end users, business customers, and their own workers. Founders are consummate rationalizers; like parenting, founding a company requires continuous, low-grade self-deception about the amount of work involved and the chances of success. A founder, confronted with the likelihood of failure, is absolutely capable of talking themselves into believing that nearly any compromise is superior to shuttering the business: "I'm one of the good guys, so the most important thing is for me to live to fight another day. Thus I can do any number of immoral things to my users, business customers or workers, because I can make it up to them when we survive this crisis. It's for their own good, even if they don't know it. Indeed, I'm doubly moral here, because I'm volunteering to look like the bad guy, just so I can save this business, which will make the world over for the better":

https://locusmag.com/2024/05/cory-doctorow-no-one-is-the-enshittifier-of-their-own-story/

(En)shit(tification) flows downhill, so tech workers grapple with their own version of this dilemma. Faced with constant pressure to increase the value flowing from their division to the company, they have to balance different, conflicting tactics, like "increasing the number of users or business customers, possibly by shifting value from the company to these stakeholders in the hopes of making it up in volume"; or "locking in my existing stakeholders and squeezing them harder, safe in the knowledge that they can't easily leave the service provided the abuse is subtle enough." The bigger a company gets, the harder it is for it to grow, so the biggest companies realize their gains by locking in and squeezing their users, not by improving their service::

https://pluralistic.net/2023/07/28/microincentives-and-enshittification/

That's where "twiddling" comes in. Digital platforms are extremely flexible, which comes with the territory: computers are the most flexible tools we have. This means that companies can automate high-speed, deceptive changes to the "business logic" of their platforms – what end users pay, how much of that goes to business customers, and how offers are presented to both:

https://pluralistic.net/2023/02/19/twiddler/

This kind of fraud isn't particularly sophisticated, but it doesn't have to be – it just has to be fast. In any shell-game, the quickness of the hand deceives the eye:

https://pluralistic.net/2024/03/26/glitchbread/#electronic-shelf-tags

Under normal circumstances, this twiddling would be constrained by counterforces in society. Changing the business rules like this is fraud, so you'd hope that a regulator would step in and extinguish the conduct, fining the company that engaged in it so hard that they saw a net loss from the conduct. But when a sector gets very concentrated, its mega-firms capture their regulators, becoming "too big to jail":

https://pluralistic.net/2022/06/05/regulatory-capture/

Thus the tendency among the giant tech companies to practice the one lesson of the Darth Vader MBA: dismissing your stakeholders' outrage by saying, "I am altering the deal. Pray I don't alter it any further":

https://pluralistic.net/2023/10/26/hit-with-a-brick/#graceful-failure

Where regulators fail, technology can step in. The flexibility of digital platforms cuts both ways: when the company enshittifies its products, you can disenshittify it with your own countertwiddling: third-party ink-cartridges, alternative app stores and clients, scrapers, browser automation and other forms of high-tech guerrilla warfare:

https://www.eff.org/deeplinks/2019/10/adversarial-interoperability

But tech giants' regulatory capture have allowed them to expand "IP rights" to prevent this self-help. By carefully layering overlapping IP rights around their products, they can criminalize the technology that lets you wrestle back the value they've claimed for themselves, creating a new offense of "felony contempt of business model":

https://locusmag.com/2020/09/cory-doctorow-ip/

A world where users must defer to platforms' moment-to-moment decisions about how the service operates, without the protection of rival technology or regulatory oversight is a world where companies face a powerful temptation to enshittify.

That's why we've seen so much enshittification in platforms that algorithmically rank their feeds, from Google and Amazon search to Facebook and Twitter feeds. A search engine is always going to be making a judgment call about what the best result for your search should be. If a search engine is generally good at predicting which results will please you best, you'll return to it, automatically clicking the first result ("I'm feeling lucky").

This means that if a search engine slips in the odd paid result at the top of the results, they can exploit your trusting habits to shift value from you to their investors. The congifurability of a digital service means that they can sprinkle these frauds into their services on a random schedule, making them hard to detect and easy to dismiss as lapses. Gradually, this acquires its own momentum, and the platform becomes addicted to lowering its own quality to raise its profits, and you get modern Google, which cynically lowered search quality to increase search volume:

https://pluralistic.net/2024/04/24/naming-names/#prabhakar-raghavan

And you get Amazon, which makes $38 billion every year, accepting bribes to replace its best search results with paid results for products that cost more and are of lower quality:

https://pluralistic.net/2023/11/06/attention-rents/#consumer-welfare-queens

Social media's enshittification followed a different path. In the beginning, social media presented a deterministic feed: after you told the platform who you wanted to follow, the platform simply gathered up the posts those users made and presented them to you, in reverse-chronological order.

This presented few opportunities for enshittification, but it wasn't perfect. For users who were well-established on a platform, a reverse-chrono feed was an ungovernable torrent, where high-frequency trivialities drowned out the important posts from people whose missives were buried ten screens down in the updates since your last login.

For new users who didn't yet follow many people, this presented the opposite problem: an empty feed, and the sense that you were all alone while everyone else was having a rollicking conversation down the hall, in a room you could never find.

The answer was the algorithmic feed: a feed of recommendations drawn from both the accounts you followed and strangers alike. Theoretically, this could solve both problems, by surfacing the most important materials from your friends while keeping you abreast of the most important and interesting activity beyond your filter bubble. For many of us, this promise was realized, and algorithmic feeds became a source of novelty and relevance.

But these feeds are a profoundly tempting enshittification target. The critique of these algorithms has largely focused on "addictiveness" and the idea that platforms would twiddle the knobs to increase the relevance of material in your feed to "hack your engagement":

https://www.theguardian.com/technology/2018/mar/04/has-dopamine-got-us-hooked-on-tech-facebook-apps-addiction

Less noticed – and more important – was how platforms did the opposite: twiddling the knobs to remove things from your feed that you'd asked to see or that the algorithm predicted you'd enjoy, to make room for "boosted" content and advertisements:

https://www.reddit.com/r/Instagram/comments/z9j7uy/what_happened_to_instagram_only_ads_and_accounts/

Users were helpless before this kind of twiddling. On the one hand, they were locked into the platform – not because their dopamine had been hacked by evil tech-bro wizards – but because they loved the friends they had there more than they hated the way the service was run:

https://locusmag.com/2023/01/commentary-cory-doctorow-social-quitting/

On the other hand, the platforms had such an iron grip on their technology, and had deployed IP so cleverly, that any countertwiddling technology was instantaneously incinerated by legal death-rays:

https://techcrunch.com/2022/10/10/google-removes-the-og-app-from-the-play-store-as-founders-think-about-next-steps/

Newer social media platforms, notably Tiktok, dispensed entirely with deterministic feeds, defaulting every user into a feed that consisted entirely of algorithmic picks; the people you follow on these platforms are treated as mere suggestions by their algorithms. This is a perfect breeding-ground for enshittification: different parts of the business can twiddle the knobs to override the algorithm for their own parochial purposes, shifting the quality:shit ratio by unnoticeable increments, temporarily toggling the quality knob when your engagement drops off:

https://www.forbes.com/sites/emilybaker-white/2023/01/20/tiktoks-secret-heating-button-can-make-anyone-go-viral/

All social platforms want to be Tiktok: nominally, that's because Tiktok's algorithmic feed is so good at hooking new users and keeping established users hooked. But tech bosses also understand that a purely algorithmic feed is the kind of black box that can be plausibly and subtly enshittified without sparking user revolts:

https://pluralistic.net/2023/01/21/potemkin-ai/#hey-guys

Back in 2004, when Mark Zuckerberg was coming to grips with Facebook's success, he boasted to a friend that he was sitting on a trove of emails, pictures and Social Security numbers for his fellow Harvard students, offering this up for his friend's idle snooping. The friend, surprised, asked "What? How'd you manage that one?"

Infamously, Zuck replied, "People just submitted it. I don't know why. They 'trust me.' Dumb fucks."

https://www.esquire.com/uk/latest-news/a19490586/mark-zuckerberg-called-people-who-handed-over-their-data-dumb-f/

This was a remarkable (and uncharacteristic) self-aware moment from the then-nineteen-year-old Zuck. Of course Zuck couldn't be trusted with that data. Whatever Jiminy Cricket voice told him to safeguard that trust was drowned out by his need to boast to pals, or participate in the creepy nonconsensual rating of the fuckability of their female classmates. Over and over again, Zuckerberg would promise to use his power wisely, then break that promise as soon as he could do so without consequence:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3247362

Zuckerberg is a cautionary tale. Aware from the earliest moments that he was amassing power that he couldn't be trusted with, he nevertheless operated with only the weakest of Ulysses pacts, like a nonbinding promise never to spy on his users:

https://web.archive.org/web/20050107221705/http://www.thefacebook.com/policy.php

But the platforms have learned the wrong lesson from Zuckerberg. Rather than treating Facebook's enshittification as a cautionary tale, they've turned it into a roadmap. The Darth Vader MBA rules high-tech boardrooms.

Algorithmic feeds and other forms of "paternalistic" content presentation are necessary and even desirable in an information-rich environment. In many instances, decisions about what you see must be largely controlled by a third party whom you trust. The audience in a comedy club doesn't get to insist on knowing the punchline before the joke is told, just as RPG players don't get to order the Dungeon Master to present their preferred challenges during a campaign.

But this power is balanced against the ease of the players replacing the Dungeon Master or the audience walking out on the comic. When you've got more than a hundred dollars sunk into a video game and an online-only friend-group you raid with, the games company can do a lot of enshittification without losing your business, and they know it:

https://www.theverge.com/2024/5/10/24153809/ea-in-game-ads-redux

Even if they sometimes overreach and have to retreat:

https://www.eurogamer.net/sony-overturns-helldivers-2-psn-requirement-following-backlash

A tech company that seeks your trust for an algorithmic feed needs Ulysses pacts, or it will inevitably yield to the temptation to enshittify. From strongest to weakest, these are:

Not showing you an algorithmic feed at all;

https://joinmastodon.org/

"Composable moderation" that lets multiple parties provide feeds:

https://bsky.social/about/blog/4-13-2023-moderation

Offering an algorithmic "For You" feed alongside of a reverse-chrono "Friends" feed, defaulting to friends;

https://pluralistic.net/2022/12/10/e2e/#the-censors-pen

As above, but defaulting to "For You"

Maturity lies in being strong enough to know your weaknesses. Never trust someone who tells you that they will never yield to temptation! Instead, seek out people – and service providers – with the maturity and honesty to know how tempting temptation is, and who act before temptation strikes to make it easier to resist.

If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:

https://pluralistic.net/2024/05/11/for-you/#the-algorithm-tm

Image: Cryteria (modified) https://commons.wikimedia.org/wiki/File:HAL9000.svg

CC BY 3.0 https://creativecommons.org/licenses/by/3.0/deed.en

--

djhughman https://commons.wikimedia.org/wiki/File:Modular_synthesizer_-_%22Control_Voltage%22_electronic_music_shop_in_Portland_OR_-_School_Photos_PCC_%282015-05-23_12.43.01_by_djhughman%29.jpg

CC BY 2.0 https://creativecommons.org/licenses/by/2.0/deed.en

[keusagi]

yk, I should start posting my gifs here too

( ・∇・)

From Marilyn☆dust school promo video on YouTube!

[hauntednightmaretheorist]

[dreaminginthedeepsouth]

Andrew Marlette, Pensacola News Journal  ::  [Scott Horton]

* * * * *

Moral hazards.  September 19, 2022

Robert B. Hubbell

         “Moral hazard” is an evocative phrase that recognizes the moral dimension of economic and political decisions. An easy example is Trump’s promise to pardon January 6th insurrectionists if he is elected as president in 2024. In that scenario, insurrectionists are encouraged to engage in more insurrection. Moreover, some voters are encouraged to support an aspiring dictator, which in turn encourages the aspiring dictator to take even more lawless positions. In that example of cascading hazards, Trump’s behavior seduces and corrupts people to support a dictatorial regime that is the antithesis of their native beliefs in law, order, and personal liberty.

         Trump is a moral hazard to the Republican Party. In six years, he has convinced the party faithful to support and engage in increasingly corrupt, cruel, and reckless practices that undermine their morality and ethics—while posing a danger to our republic. Ron DeSantis’s callous abuse of refugees to make a political point (and gain a few votes) is an example of the GOP’s willingness to engage in increasingly cruel and reckless behavior to regain political control.

         After the widespread condemnation of DeSantis, one would think that DeSantis would back off the offending behavior. On the contrary, DeSantis has followed Trump’s model by doubling down on the cruel behavior—because many Trump voters supported the heartless trafficking of refugees. See CBS Miami, More migrant flights likely; Gov. DeSantis: “The legislature gave me $12 million.”

         Many commentators say that “The cruelty is the point.” What does that mean? It means that extremist Republicans take perverse glee in seeing the moral indignation of decent people shocked at the sight of gratuitous cruelty. They call it “Owning the libs”—an immature game that comes at a steep cost. By applauding political stunts that treat humans as stage props, extremist Republicans are debasing their humanity, morality, and decency. As Jennifer Rubin wrote over the weekend, “Republicans are wearing cruelty as a badge of honor.” That is the moral hazard of supporting Trump, DeSantis, and their acolytes.

         The cost is particularly steep for those who—like DeSantis—somehow rationalize their cruelty in the name of Christianity. DeSantis has begun spouting the cri de guerre of Christian nationalism, i.e., “Put on the full armor of God,” and Trump has engineered a hostile takeover of large swaths of the evangelical Christian church.

         The PEW Research Center recently released a report modeling the future of religion in America. The report's primary finding is that “[i]f recent trends in religious switching continue, Christians could make up less than half of the US population within a few decades.” Christians currently comprise 64% of the US population, a decrease from 90% in 1970.

         It would be foolish to assert that the substantial decline in Christianity is caused by the increasing political activism in the church. But it would also be foolish to pretend that the increasing political activism of the Christian church is unrelated to its rapid decline in America. When “going to church” feels like “going to a political rally,” the obvious question arises: Why not just go to the political rally?

         The current hatred of immigrants by Christian nationalists hollows the soul of an institution that has welcomed and cared for immigrants since its founding, e.g., the Nativity Narrative and the parable of the Good Samaritan. It is too much to expect that Christian nationalists will recognize their hypocrisy. But it is not too much to expect that other Christians will condemn those who have perverted a founding principle of the Christian church. Until the voices of Christian charity and love drown out the hate-filled rhetoric of Christian nationalism, it is up to the media and concerned Americans to highlight the hypocrisy and cruelty of the Republican Party.  

         And as Christian nationalists follow Trump over a moral precipice, he may be leading them to a freshly minted circle of hell. Trump is now using QAnon imagery and slogans at his rallies. See APNews, Trump openly embraces, amplifies QAnon conspiracy theories. Will Christian evangelicals follow Trump as he promotes a fringe conspiracy theory with religious overtones, e.g., an apocalyptic “Storm” that will end the reign of Satan-worshipping pedophiles? They already are. See CNN, How QAnon uses religion to lure unsuspecting Christians. Such is the moral hazard of supporting and excusing Trump’s politics of hate and cruelty. After a while, nothing seems too outrageous or hateful to believe.

         Finally, DeSantis’s effort to engage in human trafficking poses a legal hazard to DeSantis. He is currently unconcerned, but when Ted Cruz says that a fellow Republican is engaged in illegal behavior, you should worry. See The Independent, Ted Cruz forced to admit trafficking migrants to Martha’s Vineyard is illegal in Sean Hannity interview. An attorney for thirty of the refugees trafficked by DeSantis is demanding a DOJ investigation. See The Independent, Migrants’ lawyers call for criminal investigation into DeSantis’ Martha’s Vineyard stunt.

         And while the political fallout may be the least important consideration, Spanish-language radio stations in Florida were harshly critical of DeSantis’s virtual kidnapping of the Venezuelan refugees. See Politico, ‘Huge mistake’: DeSantis’ migrant transports could undercut support in South Florida. As noted in Politico,

“Florida Republicans in recent years have made it a priority to court Venezuelan Americans, many of whom fled their home country in the past decade amid the political and economic turmoil under Nicolás Maduro and his predecessor Hugo Chávez. It’s a story that has parallels to Cuban Americans — a crucial bloc of support for the GOP — who left their country to escape communism.”

[arcticdementor]

The ability to execute people is a major advantage to the Chinese system, but maybe not in the way you're thinking. In the U.S., the government is forced to bail out failing companies to prevent systemic risk. But the risk takers who caused the crisis never go to jail and even get to keep their wealth most of the time. This causes moral hazard. In China too, the government must bail out failing companies. But, when they do, they confiscate the wealth of the risk takers or even execute them. Thus preventing moral hazard.

bored_at_work_guy

[radiowarp]

twitter

[radiowarp]

Atticus before he goes crazy

The colors were more vibrant before i uploaded it and now they look more bleak and idk how to fix it 😭😭

twitter