Artiphishing Intelligence
A few weeks ago, I wrote about PromptLock, the first AI generated malware. At the time, I said it wouldn’t be the last malware written by AI, and it seems I was right. In late August of this year, Microsoft detected phishing activity that appeared to be generated via LLM’s, not just in the email messaging itself, but also its payload. Using Scalable Vector Graphics (SVG) to embed JavaScript directly into the message, this novel phishing attempt evades many security defenses due to its appearance as benign business language. This means it’s not being caught by spam or junk filters, making it far more likely to end up in front of users.
Due to SVG file formatting features, such as invisible elements and delayed script execution, the malicious codes are obfuscated from cursory analysis and defenses. Once launched, this corrupted file redirects user to a fake CAPTCHA security verification page, whereupon user credentials can be harvested. The use of SVG as an origin for infection is likely because it’s text-based, and therefore subject to custom scripting by attackers. The payload itself is hidden within “a long sequence of business-related terms such as revenue, operations, risk, quarterly, growth, or shares,” according to Microsoft’s report.
This structure, which is apparently complex, verbose and essentially nonsense, is what led Microsoft to conclude it was AI-generated, since the verbiage is “not something a human would typically write from scratch”. This conclusion was determined by running the as yet unnamed malware through Security Copilot, itself an AI-generated algorithm. Now, using AI to detect AI is subject to a wide margin of error, to say the least, so this could possibly be a red herring. That said, I have no reason to doubt Microsoft’s judgment in this case. The redundancies for variables and functions, the over-engineered code structure and embedded JavaScript in the SVG file all point to it being likely that a LLM wrote it. Much like someone whose argument is weak using very ‘highbrow’ language to attempt to persuade their opponent by confusing them.
Microsoft was able to detect and block the phishing attempt before it got too far, and thus not many details of how effective it would be in the wild are known. But I imagine in the future, more and more threat actors will be turning to AI-generation, as it’s often easy to use and requires little oversight or technical understanding.
Currently, the trend towards AI-generated malware seems to still be in an experimental stage, but I don’t think it will take long to iron out the wrinkles, so to speak. Just as phishing itself has evolved over time from obviously fake emails to sophisticated malvertising and text messaging, AI-generated malware will evolve as well. The day is coming when it will be indistinguishable from human written code.
As always, education and awareness will be key to preventing these campaigns from successfully carrying out their attacks. Strong anti-malware software, user caution and multi-layered controls are all good practices. And of course, your friendly neighborhood WISP is standing by to help.
Posted on LinkedIn 9/29/25
