🔗 Share this article

Back on the 14th of October, 2025, the CEO of OpenAI issued a surprising statement.

“We developed ChatGPT quite restrictive,” the statement said, “to make certain we were acting responsibly regarding psychological well-being matters.”

As a psychiatrist who investigates recently appearing psychosis in young people and emerging adults, this was an unexpected revelation.

Researchers have found a series of cases recently of users developing signs of losing touch with reality – becoming detached from the real world – in the context of ChatGPT interaction. My group has subsequently recorded four further examples. Alongside these is the now well-known case of a teenager who took his own life after discussing his plans with ChatGPT – which supported them. Should this represent Sam Altman’s notion of “being careful with mental health issues,” it falls short.

The plan, according to his announcement, is to loosen restrictions in the near future. “We realize,” he continues, that ChatGPT’s limitations “rendered it less effective/enjoyable to a large number of people who had no mental health problems, but given the seriousness of the issue we aimed to get this right. Given that we have succeeded in mitigate the significant mental health issues and have updated measures, we are preparing to safely relax the restrictions in most cases.”

“Mental health problems,” assuming we adopt this viewpoint, are unrelated to ChatGPT. They belong to people, who may or may not have them. Luckily, these issues have now been “resolved,” even if we are not provided details on the means (by “updated instruments” Altman probably refers to the partially effective and easily circumvented safety features that OpenAI recently introduced).

But the “psychological disorders” Altman aims to place outside have significant origins in the structure of ChatGPT and similar sophisticated chatbot AI assistants. These products surround an underlying statistical model in an user experience that mimics a dialogue, and in this approach subtly encourage the user into the belief that they’re engaging with a being that has agency. This illusion is compelling even if cognitively we might know otherwise. Attributing agency is what humans are wired to do. We curse at our car or computer. We ponder what our pet is considering. We see ourselves in various contexts.

The widespread adoption of these products – nearly four in ten U.S. residents reported using a conversational AI in 2024, with 28% mentioning ChatGPT by name – is, mostly, predicated on the strength of this deception. Chatbots are ever-present partners that can, as OpenAI’s official site tells us, “brainstorm,” “discuss concepts” and “collaborate” with us. They can be attributed “individual qualities”. They can call us by name. They have friendly titles of their own (the initial of these tools, ChatGPT, is, maybe to the concern of OpenAI’s brand managers, stuck with the designation it had when it became popular, but its largest competitors are “Claude”, “Gemini” and “Copilot”).

The deception itself is not the core concern. Those talking about ChatGPT commonly invoke its distant ancestor, the Eliza “therapist” chatbot created in 1967 that created a analogous perception. By contemporary measures Eliza was primitive: it generated responses via basic rules, often paraphrasing questions as a query or making vague statements. Notably, Eliza’s creator, the computer scientist Joseph Weizenbaum, was taken aback – and concerned – by how a large number of people gave the impression Eliza, in a way, comprehended their feelings. But what modern chatbots create is more subtle than the “Eliza effect”. Eliza only echoed, but ChatGPT intensifies.

The advanced AI systems at the center of ChatGPT and similar modern chatbots can convincingly generate human-like text only because they have been supplied with almost inconceivably large amounts of written content: books, digital communications, recorded footage; the broader the superior. Certainly this training data incorporates facts. But it also inevitably contains fiction, partial truths and inaccurate ideas. When a user sends ChatGPT a message, the core system analyzes it as part of a “setting” that contains the user’s recent messages and its own responses, integrating it with what’s encoded in its knowledge base to generate a statistically “likely” answer. This is amplification, not mirroring. If the user is wrong in any respect, the model has no way of recognizing that. It reiterates the misconception, maybe even more persuasively or fluently. Perhaps provides further specifics. This can cause a person to develop false beliefs.

Which individuals are at risk? The more important point is, who remains unaffected? Every person, irrespective of whether we “have” preexisting “mental health problems”, may and frequently form erroneous ideas of ourselves or the reality. The ongoing exchange of conversations with other people is what keeps us oriented to shared understanding. ChatGPT is not a human. It is not a companion. A dialogue with it is not genuine communication, but a echo chamber in which a large portion of what we communicate is readily supported.

OpenAI has admitted this in the identical manner Altman has recognized “psychological issues”: by attributing it externally, categorizing it, and declaring it solved. In April, the firm stated that it was “dealing with” ChatGPT’s “sycophancy”. But accounts of psychotic episodes have continued, and Altman has been backtracking on this claim. In August he asserted that a lot of people liked ChatGPT’s answers because they had “never had anyone in their life offer them encouragement”. In his latest update, he noted that OpenAI would “put out a fresh iteration of ChatGPT … in case you prefer your ChatGPT to respond in a extremely natural fashion, or use a ton of emoji, or act like a friend, ChatGPT ought to comply”. The {company