Human AI
Dr. Ian Malcolm famously said, "Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should." He was talking about dinosaurs. It works just as well for how we're deploying AI at the moment.
Just because you can automate something, doesn't mean you should. I should say I'm broadly optimistic about AI, I've written before about leaning heavily on agents, but the good stuff I've seen comes from working alongside these systems rather than handing things over to them wholesale, not least on anything that leans on human judgement.
Take the rush to swap out customer service agents for LLMs. On paper it looks trivial, everyone's used ChatGPT, so the technical path seems obvious. The hard bit though isn't teaching an LLM to sound like a support agent, it's the cage you then have to build around it to stop it wandering off somewhere it shouldn't.
So how do you put boundaries on one of these things without killing the very thing that made it useful in the first place? ChatGPT is useful precisely because it's general, it'll have a go at almost anything. That same flexibility turns into a liability the moment you need narrow, predictable, well-behaved answers out of a support agent.
Trying to get an LLM to completely replace a person misreads what each side is actually good at. When you hire someone, a whole load of behavioural constraints arrive pre-installed. No manual needs a line saying "don't answer billing queries with a three-page literary essay" or "don't recommend a nice Italian place when someone's software is broken" (though I'd quite like to see the manual that did). People just know where the professional lines sit, it falls out of understanding social norms.
A language model has none of that baked in. It's a pattern-matching machine, just as happy generating the next line of a Python script as the next sentence of a refund reply, and anyone sufficiently bored or motivated can nudge it well past where you wanted it to stop.
What does seem to work is treating the AI as a layer on top of the person. Picture a support call where the human handles the conversation while the AI runs alongside - transcribing, pulling up the right documentation through semantic search, flagging patterns from past cases. The human does what humans are good at, which is talking to another human, while the machine does the grunt work no individual could keep up with.
Of course, plenty of support queries really are routine, and I'd happily let a machine chew through those at scale. Even so, there's an ergonomics problem here as well as a technical one. An LLM with strict guardrails has a habit of becoming the thing that traps you in a circular conversation going nowhere. Deploying something this capable and then frantically boxing it in is a bit like using a nuclear reactor to power your car.
Keeping in mind, when a customer reaches out they've already hit a wall. They've got a problem they couldn't sort themselves, and bolting on another layer of automation, especially one that misreads the context and fires back generic answers, only makes that worse. However clever the stack underneath, it counts for very little if the experience falls apart at the exact moment it matters. [anecdote: a recent run-in of your own with a support bot, if you have one]
Klarna gave us a tidy case study here. After loudly championing AI-driven customer service, they walked it back, admitting customers actually prefer talking to a person. Which is a reminder, if we needed one, that LLMs aren't the answer to every problem, and that customer psychology gets a vote too.
AI is good at retrieving information, spotting patterns and chewing through routine queries at scale. People bring the emotional read, the creative problem solving and the knack for dealing with ambiguity. Treat AI purely as a way to cut headcount and you lose that second half, treat it as something that lets your agents do their job better and faster and you'll probably keep both, empathy included.
Most of this isn't really specific to support bots, of course. Tools are still just tools, however clever, and their worth comes from how thoughtfully you fit them in. I'd say the best solution is rarely the most advanced one, more often it's simply whichever one helps the person on the other end. Dr. Malcolm would probably agree, whether you could was never the hard question anyway.