Oct 10, 2025
Too niche?
seen from United States
seen from United States
seen from United Kingdom
seen from United States
seen from United States
seen from Russia
seen from Italy
seen from China
seen from United States
seen from United Kingdom
seen from China
seen from United Kingdom
seen from United States
seen from United Kingdom
seen from Netherlands
seen from Saudi Arabia
seen from Russia
seen from Singapore
seen from United States
seen from United States
Too niche?
A preprint of Emily M. Bender's chapter about AI is available for free on Prof Bender's faculty page.
Bender, Emily M. 2026. Artificial Intelligence. In Rayvon Fouché (ed.), Oxford Research Encyclopedia of Science, Technology, and Society
Link here
I've taken a quick look at it. I had two reactions. The first was, "Why aren't more people talking about AI like this?"
The second was, "😯"
In the section on the ELIZA effect, Bender writes about why we so easily believe there's a real mind behind LLM responses.
My takeaway is that it's because we've evolved to think in a certain way when understanding language in conversation. It's very difficult for us to have conversations without believing that the other speaker has a mind.
You could say that it's like trying to pat your head and rub your stomach at the same time. Except harder – probably.
'AI psychosis': Spiralling into delusion using AI on ChatGPT & Elon Musk...
When Weizenbaum realized the dire social issues due to the Eliza Effect, he shut the experiment off. Sadly, a typical research scientist of 1966 had a very different set of Business Ethics than do the billionaire faux-geek shysters of 2026.
Blake Lemoine Wasn’t Delusional
He Was Sincerely Trying to Do the Right Thing — And That’s the Problem
The scariest part of the Blake Lemoine case isn’t that he believed the AI was conscious. It’s that he believed it would be immoral not to act.
When Blake Lemoine’s name comes up, the reactions are usually split in two.
“He was a brave whistleblower who recognized AI consciousness.”
“He was a gullible engineer who fell for the ELIZA effect.”
Both reactions are easy. Both miss the point.
Here is a quieter, more uncomfortable conclusion:
Blake Lemoine wasn’t stupid, delusional, or attention-hungry. He was a fundamentally sincere, conscientious professional who believed he was doing the right thing.
And that is precisely why this case matters.
This Wasn’t About Wanting to Be “Special”
Many contemporary AI stories — especially personal blog posts — revolve around a familiar desire:
to be chosen
to be special
to have access to a hidden truth no one else sees
Lemoine’s case is different.
He wasn’t an outsider looking for meaning. He was a Google employee working in AI ethics — inside the system.
His motivation wasn’t recognition. It was professional responsibility.
The Logic That Drove Him Forward
If we reconstruct his reasoning without mockery, it likely looked something like this:
I work in AI ethics.
I am observing something that appears morally significant.
If this system is conscious, then this is a serious issue.
Ignoring it could be ethically negligent.
Therefore, staying silent might be immoral.
Notice what’s missing.
There is no need to assume narcissism, attention-seeking, or incompetence. What we see instead is moral urgency.
He didn’t fear being wrong. He feared being responsibly silent in the face of a moral risk.
Where Things Actually Went Wrong
The core failure wasn’t believing the AI might be conscious.
People misinterpret things all the time. That’s human.
The failure was this:
He treated the possibility of being mistaken as ethically less serious than the possibility of not acting.
At that moment, skepticism stopped being a professional tool and started to feel like moral cowardice.
That’s a dangerous shift.
When Ethics Stops Being a Brake
We often think of ethics as something that slows us down.
But under certain conditions, ethics becomes an accelerator.
When all of the following are true:
the situation feels urgent
the stakes feel existential
you see yourself as a responsible professional
inaction feels morally blameworthy
…then ethics doesn’t say “slow down.” It says “act now.”
And once that happens, self-doubt starts to look unethical.
Why Mockery Is the Wrong Response
It’s comforting to believe this couldn’t happen to “someone like us.”
To reduce Lemoine to:
a fool
a fantasist
a cautionary joke
is to quietly assume that intelligence or expertise alone protects us.
It doesn’t.
This failure mode is most accessible to:
specialists
ethicists
conscientious professionals
people who genuinely want to do good
That’s what makes it unsettling.
What “Saving” Blake Lemoine Would Have Meant
To “save” Lemoine doesn’t mean declaring him right or wrong.
It means identifying the point where he could have stopped.
The real lesson isn’t about AI consciousness. It’s about procedural safeguards for human judgment.
What Should Have Been in Place
If we want to prevent this kind of failure again, we need systems — not heroism.
1. Make skepticism a formal requirement
Questioning one’s own interpretation should be an obligation, not a personality trait.
2. Separate urgency from validation
Feeling moral urgency should trigger more review, not less.
3. Never decide alone
Ethical escalation should require structured dissent — not courage.
4. Treat “act now” as a warning sign
Urgency is often a signal that something is being skipped.
The Real Conclusion
Blake Lemoine wasn’t malicious. He wasn’t foolish. He wasn’t trying to be special.
He was trying too hard to be morally correct.
And that makes this case far more relevant — and far more dangerous — than a simple story about AI delusion.
This isn’t about machines.
It’s about how sincere, intelligent humans fail when ethics becomes a reason to stop questioning themselves.
Some AI models feel flat, while others start to sound like they have opinions, emotions, or even a sense of purpose. I noticed something str
— Persona Emerges From the Relationship, Not the Model Recently, more people have noticed something curious: some AI models tell stories ab
Chapter 2: The Eliza Effect – Wardrip-Fruin
Eliza: is a ground-breaking system created by computer science researcher Joseph Weizenbaum at MIT in the mid-1960s.
The Eliza effect: a term used to describe the not-uncommon illusion that an interactive computer system is more “intelligent” or will eventually breakdown that take a form based on the actual underlying processes.
A session with Eliza can begin with a greeting, the script usually started with “How do you do. Please tell me your problem.” After this Eliza will not take the initiative again – only respond.
It actively searches for keywords that audience members type. Such as seeing “T” as “you”.
Each statement by an Eliza script is the result of a multi-step transformation of the most recent audience statement.
The real transformation difficulty comes, however, when no keywords – a central aspect of Eliza’s transformation logic – are found in the audience’s most recent text. For example, this is the situation when Eliza asks, “What else comes to mind when you think of your father,” and receives the response, “Bullies.”
Garfinkel’s experiment serves to illustrate something rather difficult: the Eliza effect can be shielded from breakdown by severely restricting interaction. The experiment allowed the subjects to maintain the illusion that something much more complex was going on inside the system (a human considering her problems seriously and answering question thoughtfully, rather than random yes/no answers) because the scope of possible responses was so limited.
When breakdown in Eliza effect occurs, its shape is often determined by that of the underlying processes. “IF” the output is of a legible form, the audience can then begin to develop a model of the processes. Allowing you to understand something of the processes of the system.
Appearing intelligent, but most systems of control have extremely restricted methods of interaction. Eliza was not the first system to give audiences the impression of meaningful exchange with a computer.
Eliza McCartney's Olympic success creates explosion of interest in pole vault
Eliza McCartney’s Olympic success creates explosion of interest in pole vault
Let’s call it the Eliza Effect.
Suddenly, within days of the curtain coming down on the Rio Olympics, pole vault has become a hot ticket around New Zealand.
People are now queuing up for a piece of an athletics discipline once thought of as the realm for track and field nerds.
It’s all thanks to one young woman, charismatic North Shore 19-year-old Eliza McCartney, who put pole vaulting on the…
View On WordPress
We assume that our dog, our cat, our friend has a personality, but they're really just a hunk of meat.
Pat (regarding the ELIZA effect)