I have no idea whether this is true, it seems way too stupid to be real and I have to assume it's made up, but I'm sharing because it has the vibe of something that would happen in a cartoon from the 90s that has characters burn a hole in a door by bouncing a laser pen beam between two mirrors
This is AI we're talking about. There is no bottom to the level of stupid it can get.
Yes but the idea that the AI actually has the capability to change the emails is idiotic. We've had to deal with annoying bullshit authenication for years in the name of security and the robot should not be able to change shit. Welding steel security bars over all the windows and then installing a massive unguarded glass door type scenario. They *have* teams in charge of security, it is ridiculous that a robot could do this.
Until proven otherwise I'm gonna assume that the hackers claiming this are pulling a prank and doctoring this because I am not ready to face a world this stupid.
Oh, having worked in IT security and seen the sorts of shit that 1) get overlooked and 2) get offloaded to AI, I 100% believe this could be true.
Step 1: C-suite wants more AI because it's popular; either c-suite believes the bullshit or wants to profit off investors who do.
Step 2: C-suite "encourages" increased AI usage. Sometimes it is by rewarding people who use it. More often it's by firing the people who don't.
Step 3: Because setting up AI isn't everybody's wheelhouse (and not everyone is doing this willingly), the onboarding process is made as easy as possible. It's mostly the same model doing everything. The complexity is automated out where possible.
Step 4: Because developers have to be onboarded. The AI is given increased permissions so it can read from the database, or even write/execute directly.
Step 5: People outside development circles hear about this magic AI tech and demand that the developers take their jobs and "put AI into it." Developers do their best with half-understood explanations from several levels of hierarchy away in an all-too-common game of Broken Telephone. They hand over AI access (in some form or another) that is heavily safeguarded, but in a way that doesn't quite do what it's supposed to. The clients do not complain because it works fine at first.
Step 6: A client finds they don't have permission to do something that is common enough to automate, but niche enough that a dev wouldn't have thought about it. Client does not know why this check is in place but it's making their job harder. They complain up the chain.
Step 7: Someone up the chain, who understands NEITHER side as well as they think they do, demands that the developers loosen restrictions. Context does not get passed along beyond "the clients need it."
Step 8: Developers receive the request and either don't think too hard about it (like with the clients above, they are distracted) or assume there's a good reason or don't care or are afraid of reprisal if they speak up. They loosen restrictions for that particular client.
Step 9: Client decides to update an externally-facing system by hooking it up to the AI.
RESULT: Outside users can prompt an AI with developer access to do whatever they want and nobody has a way to stop any of it before the prompts are acted upon.
tbh I'm surprised we haven't heard more cases of this, it was all over the place early on - people were asking chatbots for things like admin passwords and getting correct answers
