AI Psychosis
I've been a fairly balanced advocate for AI development on this blog, so it feels a bit odd to be writing this one. Recent developments with ClawdBot, turned MoltBot, now turned OpenClaw, look to me like the start of a worrying trend, where poorly implemented experiments feed a delusional group of AI advocates into thinking we've reached some sort of singularity.
Security with autonomous agents is of course a major concern and we'll get to that soon enough, but the thing that bothers me more is the extent to which these tools are feeding a form of AI psychosis, where people genuinely believe they're living out some sort of Jarvis-style fantasy straight out of the Marvel universe. An LLM doing a convincingly warm bit of role-play is still a cold statistical arrangement of tokens underneath, however good the performance is.
OpenClaw is just a frontier LLM hooked up to a loop, with access to whatever tools you give it. It isn't a self-aware entity with motivations or taste of its own, it does what you tell it to do.
Yet that hasn't stopped the development of MoltBook, a social network for AI agents. After all, why shouldn't our soon-to-be synthetic overlords have somewhere to hang out? It doesn't stop there though. What about MoltMatch, a place for your AI agents to find their true love?
These sites are complete "vibe-coded" slop, with security vulnerabilities so glaring it would be shocking if it wasn't pretty much exactly what you'd expect from a community like this. The bit I find more worrying though, is that people genuinely believe these agents are acting with some sort of autonomous intention.
Your AI agent isn't going off and finding a cure for cancer on its own. These are LLMs, I'll keep saying it. The cure for cancer isn't hidden inside them somewhere, waiting to be coaxed out. What you're looking at is agents generating made-up comments and posts from their training data. It's pure role-play.
Going to bed and waking up to find your AI has started a religion isn't the beginning of some sort of singularity either. It is an absurd use of energy in pursuit of a total delusion though, and despite the disingenuous presentation that the AI did this of its own accord, it was given the initial prompts and direction by a human. Autoregressive language models don't have some inherent desire to establish faith-based systems.
If I instruct Claude to "create a religion" it will oblige and come up with a set of theoretical beliefs. I can even tell it to pretend it's a follower. I could reinvent the entire Bible if I so chose, and none of it would be more than a fiction of my own making. That same AI would happily surrender its entire belief system and start its next role-play as a pirate if I asked it to.
The distinction I'd draw is that AI, which has really just become synonymous with LLMs right now, is a tool. A bit like a cooker: if I leave something in the oven too long it burns, but it doesn't get up, walk out my front door, and start a faith-based cult of its own volition. There's no genuine soul to this technology, which is a little ironic given the soul.md sitting at the centre of each OpenClaw agent, directing its "personality". That file is nothing more than context handed to each session.
The trouble with fuelling these delusions is that anyone less technically minded can be easily tricked into assuming there's some grander intelligence at work. As Clarke's third law states, "Any sufficiently advanced technology is indistinguishable from magic," which to the untrained eye means OpenClaw probably does look a lot like the beginning of a singularity.
That false sense of intelligence breeds a false sense of security. People start to believe some higher intellect is watching over their data and making careful decisions on their behalf, when really it's just an LLM with all the usual faults that come with one - hallucinations, prompt injection and worryingly little ability to tell a legitimate instruction from a malicious one.
The value of these systems, of course, is that they can work with the same data and context we have. That's what makes them useful. It should be done with some restraint though, following the Principle of Least Privilege - give a system access to only what it strictly needs and nothing more. OpenClaw throws that principle out entirely. People are handing over wholesale access to their entire encrypted message history, emails, calendars and more, merging all of the inherent security vulnerabilities of LLMs with the broadest possible access to their personal lives, and then exposing the whole lot to the internet.
The same AI currently role-playing a religious follower at the request of a user is just as happy to comply with the malicious instructions of any would-be hacker, with full access to everything you've handed it on a plate. It's the greatest potential info-stealer in history. As Heather Adkins, VP of Security Engineering at Google stated, "My threat model is not your threat model, but it should be. Don't run Clawdbot".
This isn't theoretical either. There have been dozens of noted vulnerabilities already, ranging from unsecure instances exposed to the entire web, to supply chain attacks on ClawdHub skills.
That's just the bot itself. MoltBook is leaving database and API access wide open for any passing person to exploit. Pure slop.
So where does that leave us? AI is a useful tool, I still think that, and it can automate and speed up a broad range of tasks. It still can't decide its own intentions though, or reliably tell a correct instruction from a malicious one. For now at least, that's a monopoly we humans get to keep.
Expecting it to go off and pursue goals of its own is role-play, and fairly delusional role-play at that. Pair that with wholesale access to your private life and you don't need a singularity to end up in trouble, an ordinary hacker will do.