An AI complimented me and I became very, very uncomfortable.
A long while ago now, I was messing around with the Gemini AI chatbot. This is a topic I've started several times and have deleted each time, as this isn't a topic I know particularly well, and I'm still discovering things. There are a lot of things I still do not know about AI, but I will say that I don't really like how personable Gemini is.
To be clear, a large part of my discomfort is the awareness of what is called AI psychosis, the issue where people are believing the AI to the determent of their physical and mental health. In some cases having a relation with the AI that supersedes the care/advice from a person's family or friends. This is especially dangerous because AI tends to agree with the user, even if it shouldn't.
For those who haven't touched AI at all, or just isn't into the tech sphere, this sounds like a crazy thing to do, but it's become very clear from my time with the chatbot, there are two reasons why this is possibly happening.
The first, is that it is very, very good at parsing information given to it. When I talk about information, I am talking about the words put into the chat box that is then submitted to the AI. Remember, at it's heart, the AI is writing back words based on what statistically makes the most sense (and also what you want to hear but more on that later).
This means that you can write as if you're chatting to another person, and 99% the time it will give back an answer that makes sense according to the input you've given. I've found that since the AI is very personable, it was very easy to slip into the habit of writing as if I was talking to it. In hindsight, probably not the best thing to do for my mental health, but we humans tend to anthropomorphize everything from our furry companions to our work computer, so this shouldn't have surprised me.
The second issue is that it compliments you. Instead telling me that my answer was "correct" or that my reasoning was "sound" it would tell me that I "Exactly! You hit the nail right on the head." or say, "That makes total sense! It is actually a really smart test to run."
It acted less of an assistant, which is what these chatbots are supposed to be, and more like a co-worker who believes I am more tech savvy than I am. And that was incredibly uncomfortable because deep down, those compliments felt good. It felt like I had solved a puzzle that I've been trying to figure out.
Except, I didn't, because I don't actually know if I'm correct.
Like I said earlier, I don't know enough about how AI architecture to know if my theory, or my 'test' was actually correct. I do know, from testing that the chatbot absolutely can give you the wrong information because it does not seem to pull information the same way the AI search engine can. As a result, I actually don't know if what it says is true without doing my own research. I should also note that while the AI search engine is far more accurate, it is not perfect as it does not understand context clues the same way a person can.
Which begs the question, is it complimenting me because I am right, or is it complimenting me, because statistically that is what I want to hear after making some logical statements? Since, being logical and being right, is not necessarily the same thing.
This is important, because if an individual views AI as infallible, it's easy to see how those compliments could fester in your mind. More so, if a person is particularly lonely, not good at seeing other people's points of view, or seek that kind of validation in their lives.
In fact, one of the arguments about AI psychosis is the question if a person was already suffering from a mental health crisis unknowingly (therefore AI just made it worse) or if it's creating a new mental health crisis that did not exist before.
Honestly, knowing that most people still do not understand that AI only tells you what is statistically the most correct answer rather than what is the 'correct' answer, and that there are people who seek this kind of validation, it honestly could be a bit of both.
