This isn’t a metaphor or ~~darkly hinting~~ or anything like that, it’s just the straightforward literal meaning of the terms.
What’s an infohazard? Something that, when you learn it, affects you (usually your thought processes) in ways that are hard or impossible to reverse. Once you’ve succumbed to the infohazard, other things you learn aren’t going to reverse the change. And an infohazard spreads to information in contact with it, so that knowing the infohazard can make you spread it with other information if you’re not careful.
But this is a description that also applies to the truth. “If you once tell a lie, the truth is ever after your enemy”: if you’re dedicated to some falsehood, any truth you let in could be a contaminant which damages that falsehood or some of its support structure. Which, to someone who thinks that the falsehood is good and proper/has the falsehood as a load-bearing pillar of their moral philosophy/something else I haven’t thought of with a similar role, looks just like an infohazard.
This has few practical consequences, but does imply that either infohazards aren’t all bad, or learning the truth is sometimes bad.
Thread by @0x49fa98: Our ruling class has fully embraced the danger of information hazards. In fact, they have decided they are so dangerous that they now advise you to stop thinking all together. The WHO to advise...…
Our ruling class has fully embraced the danger of information hazards. In fact, they have decided they are so dangerous that they now advise you to stop thinking all together. The WHO to advise that you wear a mask on your brain.
…
An information hazard is a piece of true information that causes some harm to the person who learns it. Bostrom identifies six types hazardous information transfer: data, ideas, templates, signals, attention, and evocations.
…
In addition to the infohazard typology by information transfer, we can also classify infohazards by the type of risk they present: adversarial risks, market risk, error risk, psychological risk, information system risk, and development risk
…
When we evolved the power to understand and propagate arbitrary information, we became vulnerable to information hazards. This was a novel danger; only humans can be hurt by true information.
…
We can build AIs today that are capable of thinking, but no one would accuse them of reasoning. This is not because they lack intelligence, but because they lack motivation.
An AI was programmed to seek novelty, and this made it capable of solving a maze, but it encountered the distinctly human problem of procrastinating in front of a TV.
A cynical man might suggest that our motivation mechanisms are also this crude
…
In most cases, biasing hazard is less about things that pull you from the truth, and more about things that pull you from the center of social consensus. This is rule zero of power, which you've heard many times before.
…
Infohazards are so pervasive and dangerous that we have evolved a defense against them in the form of strategic epistemic failure.
It's hard to believe things that go against social consensus, when you know they are signaling hazards. If someone presents you with an ironclad case for a socially dangerous idea, you will tend to doubt it or dismiss it. You might agree one moment, and forget the next.
For example, with Gellman amnesia you forget a disturbing observation the moment it leaves your field of attention. How many other jarring revelations don't stick in your head this way? You'll never know
…
My favorite example is this study, which shows we don't believe scientific studies that come to negative conclusions about women.
If this finding doesn't match your predilections, you will also surely dismiss it
…
And why not? The truth is all persuasion is grounded, not in reason, but in strength. When a logical argument convinces you of something, it's not the mechanics of reason that does it, it's the display of power that reason presents.
A skilled speaker not only projects power, but offers you power. "Think as I do, and you can wield some of my power." All persuasion is seduction. If the truth smells of weakness, most of us follow our nose.
If the news is no longer convincing, that suggests a loss of power.
Machine learning models are vulnerable to adversarial examples: small changes to images can cause computer vision models to make mistakes such as identifying a school bus as an ostrich. However, it is still an open question whether humans are prone to similar mistakes. Here, we address this question by leveraging recent techniques that transfer adversarial examples from computer vision models with known parameters and architecture to other models with unknown parameters and architecture, and by matching the initial processing of the human visual system. We find that adversarial examples that strongly transfer across computer vision models influence the classifications made by time-limited human observers.
Adversarial examples that can fool humans (for a split second, at least.)
(I feel like @itsblehnedict might find this interesting)
[under the cut for non-fourth-wall-breaking infohazards, and also cordyceps spoilers if anyone still cares]
So in my dream this morning I was playing a video game (it might have been a VR game, but the way my dreams work all media is VR media, so I'm not sure if it was *meant* to be VR), and part of the plot was an elephant-induced apocalypse†. I thought it was neat how the game handled that.
(Note: in this game, the elephant is foodborne as well as airborne, and was deliberately developed and put into place by some evil conspiracy. Never reached the part where they explain what the conspiracy was trying to accomplish.)
As you would expect, the game tracks physical infection and memetic infection separately. You can actually survive for quite a while after eating a poisoned cookie, if you play in exactly the right way to keep your character oblivious to the apocalypse going on around them.
But it's really hard to do that and people normally only stumble into it by accident, because the game performs (limited, one-way) fourth-wall breaking.
If this is not your first playthrough to reach the elephant plotline, the game *knows that you know* (because you've played before), and will flag you as memetically contaminated even if your character has no idea.
But it goes farther than that. The plot flag that triggers the apocalypse is finishing your dinner that night. (You then--if you don't have other plans for the night--go to eat poisoned cookies and watch a poisoned movie with your family, and many other people in other places are doing the same. If you do have other plans, your family does it without you.) There is no in-game indication that an apocalypse will start then (in the main branch of the plotline, you actually *die* that night, and are resurrected by plot stuff later). If the game notices you building a bunker, buying gas masks, avoiding finishing your dinner to buy yourself more time to prepare††, the game *realises you must have read a walkthrough* and *flags you as memetically contaminated* (because why would you be doing this stuff if you didn't know what was coming?).
---
†For anyone who has not read Cordyceps but still wants to read this post, the short version is that "the elephant" is a disease that is fatal when symptomatic but can only become symptomatic *if you know the disease exists*. If you're infected without ever learning about the disease, it lies dormant for a few months and then dies out, unless you learn about it during that timeframe. (They call it "the elephant" because it's pink and you mustn't think about it.)
††If you say you aren't hungry and put your dinner in the fridge, the "finished dinner" flag is not set and the apocalypse is postponed. You can eat other stuff later, and as long as it isn't *that* particular meal the flag is not set. Letting the food rot sets the flag, but you can still buy yourself about three days this way.
Roko's Bassilisk is one of my favorite memes because it's like philosophical slight-of-hand. It's like the gif of cutting chocolate and putting it back together as more than you started with.
It hits you with just enough valid conditionals in a small enough block of text that the underlying assumption slips through, and the whole thing gets to pretend to be sound.
