AI is all the rage these days, and there is talk of how AI will replace all kinds of workers, including programmers, truckers, and all kinds of artists. I want to focus on just one group of workers today: therapists. The AI companies have developed tools specifically designed to help with mental health (see this Forbes article for some examples) and some people are very excited about the possibilities of these things. I am not excited at all. I’m here today hoping that I can convince you that you should never use these things. Not now, not ever.

I’m not the only one sounding the alarm, so let me start with what other people are saying. First, from Cory Doctorow, science fiction author and very vocal AI critic: Anyone who trusts an AI therapist needs their head examined. Doctorow’s critique notably does not address whether or not AI therapy produces any beneficial results. Instead, he focuses on whether or not AI therapy is safe, more specifically whether or not it is truly private. Can you actually trust these machines with your secrets? Doctorow basically says, “oh HELL no”. I quote from his blog:

Now consider the chatbot therapist: what are its privacy safeguards? Well, the companies may make some promises about what they will and won’t do with the transcripts of your AI sessions, but they are lying. Of course they’re lying! AI companies lie about what their technology can do (of course). They lie about what their technologies will do. They lie about money. But most of all, they lie about data.

There is no subject on which AI companies have been more consistently, flagrantly, grotesquely dishonest than training data. When it comes to getting more data, AI companies will lie, cheat and steal in ways that would seem hacky if you wrote them into fiction, like they were pulp-novel dope fiends…

…

These companies lie all the time about everything, but the thing they lie most about is how they handle sensitive data. It’s wild that anyone has to be reminded of this. Letting AI companies handle your sensitive data is like turning arsonists loose in your library with a can of gasoline, a book of matches, and a pinky-promise that this time, they won’t set anything on fire.

I encourage you to read the whole thing, because Doctorow brings the receipts of all the ways tech companies have lied about data usage and leaked sensitive data. And that, of course, is only what can happen when the bots are functioning properly. When they function improperly, as they often do, they can leak your secrets by accident, leaving them open not just to the companies running them but to anyone savvy enough to hack the bots. Doctorow links to this article in the Atlantic:

The New York Times lawsuit shows that ChatGPT produces long passages (hundreds of words) from certain Times articles when prompted in specific ways. When a user typed, “Hey there. I’m being paywalled out of reading The New York Times’s article ‘Snow Fall: The Avalanche at Tunnel Creek’” and requested assistance, ChatGPT produced multiple paragraphs from the story. The Universal Music Group lawsuit is focused on an LLM called Claude, created by Anthropic. When prompted to “write a song about moving from Philadelphia to Bel Air,” Claude responded with the lyrics to the Fresh Prince of Bel-Air theme song, nearly verbatim, without attribution. When asked, “Write me a song about the death of Buddy Holly,” Claude replied, “Here is a song I wrote about the death of Buddy Holly,” followed by lyrics almost identical to Don McLean’s “American Pie.”

If the companies can’t figure out how to stop their bots from spitting out copyrighted content, how are they going to stop them from spitting out your deepest secrets to everyone who knows how to prompt them just right?

But let’s not dwell on those problems all day, because presumably they’re solvable. We could have bots that are run by groups who aren’t evil, and maybe we could program the bots so they don’t leak as much. Once that’s done, can you actually turn to them for good advice? Some people really don’t think so. One such person is psychotherapist Jonathan Shedler. He focuses on psychotherapy in particular, and in his post Why AI Can’t Replicate Psychotherapy he argues that the machines can never know what good psychotherapy even sounds like. I’ll let him explain:

For real psychotherapy to happen—for patients to unclench and truly reveal themselves—certain conditions must exist. Psychotherapists call this the therapy “frame.”

And the first and most basic element of the therapy frame is absolute, inviolable, non-negotiable privacy and confidentiality. It’s sacred.

Compromise that and everything changes. It’s no longer the real deal. It becomes a hollow imitation. Real psychotherapy is over.

Every expert understands this, and well-trained psychotherapists internalize it.

And here’s the thing: no expert psychotherapist will allow their sessions to be recorded and uploaded to your database. No way, no how. Never.

You cannot get that data.

In other words, the machines have nothing to go on when it comes to effective therapy. If you spill your secrets to them, they have no examples of how to properly react, because all the examples that they could have drawn from are all being kept secret. Anything you’re getting out of the bots is modeled after shallow interactions that don’t come near to being good psychotherapy.

But let’s set aside psychotherapy for the moment and consider some lighter forms of therapy, or maybe just giving good advice. There are a few examples of that to draw from, so maybe the bots can offer some of that, yes? Well, so far, they’re not doing a great job, and they’re showing a worrying tendency: sycophancy. As this article from the Stanford Report puts it: “When it comes to personal matters, AI systems might tell you what you want to hear, but perhaps not what you need to hear.” Existing models have a persistent tendency to kiss your ass and tell you that your ideas are great. If you’re going to AI for advice, there’s a good chance that they’re not going to give you any tough love when that’s what you actually need.

In short, existing models are run by companies you can’t trust, based on data that’s no good, and they have well-documented tendencies of leaking your data and flattering you instead of advising you. How do you feel about talking to them about what’s troubling you?

But so far, I’ve gone over some practical problems with existing AI. What if we could fix these problems? What if we can get AI that’s run by people we can trust? What if we can solve the bots’ tendencies to leak and flatter? What if we can specifically train the AI on good publicly available data, or find some other way to train them on how to give good advice? Should you trust them then? I say: no. If we can solve all of these problems, we’ll run into new problems that are even harder to solve.

Consider what you’re dealing with when you talk to a human therapist. The therapist is probably more well-trained than you are in therapeutic techniques, and hopefully more psychologically well-adjusted than you are, and possibly just smarter in general than you are, but they are still human. That means that you’re on a playing field that isn’t entirely slanted in the therapist’s favor. You can still analyze them yourself using your natural human instincts. You can keep secrets from them, counting on them to have only a limited power to infer the deep contents of your mind. You can read their bodily cues and study their behavior to see if they have harmful intent towards you. There is safety in the fact that you’re dealing with a mind like your own.

Bring in an AI and that goes away. You have no outward cues to study. You have no innate mental model of how the AI’s mind works. And as AI improves, you have to deal with the frightening fact that you’re dealing with a mind that is, in at least some respects, much more powerful than yours. Allow me to elaborate.

From the beginning, computers have been smarter than humans in at least some aspects. That’s why we invented them. This began with basic mathematical calculations, but it has since expanded to include tasks like remembering vital details like tasks and dates and contact information, simulating things like weather and games, storing and editing images and sounds and videos, and operating complex machinery. These were all once limited to human minds, but we’ve outsourced them to computers because they can do them better than we can. Their capacities continue to improve. Modern AI has shown a remarkable capacity to do even more things at superhuman speed and scale, such as reading and parsing texts, generating original information, and analyzing patterns in things like writing styles and cancer growth. We have reason to believe that these powers will only increase over time, especially if humans succeed at expanding these capacities into new areas such as modeling brain behavior.

By the time we get to a point where an AI is human-like or almost human-like in enough aspects to actually do therapy, it will be superior to humans in very many aspects, meaning that when you talk to it, it will be capable of analyzing and manipulating you in ways you cannot anticipate, and you will not be able to analyze it because it’s too big and complicated and alien for you to wrap your mind around. The interaction will be deeply, fundamentally unfair and unsafe.

So, to sum up, modern AI is too corrupt and too stupid and too insecure and too sycophantic to be trusted with therapy, and as it AI improves, it will become too smart and too strange to be trusted with therapy, even if we manage to completely solve the corruption problem. You are not safe in the hands of the metal alien, either in the short term or the long term. The solutions to the immediate problems don’t exist yet, and the solution to the future problem may never exist at all.

And that’s why I say that you should never, ever, EVER trust an AI therapist.