The Mind Mirage
Developing a “Theory of Mind” (ToM) is fundamental to human growth and development. ToM is the ability to understand other people as distinct individuals who have their own inner lives that guide their behavior. ToM is critical to maintaining personal relationships, to being able to enjoy a good novel or movie, or even to functioning normally in everyday society.
Amid all the AI hoopla, hype and doom-mongering, we have not had nearly enough conversations about the unprecedented challenge that the current generation of LLMs is presenting to our ToM instincts.
Why? Because LLMs do not have “inner lives that guide their behavior” nor even “minds” at all, at least not minds as humans experience them – conscious, embodied, and emotionally grounded. They are next-token-prediction engines, trained on massive amounts of data, contextually tuned and guard-railed.
Anthropomorphism is nothing new. When we scold our pet for stealing food from the table, we interpret their drooping head as “feeling guilty” rather than just submissive behavior. We wonder why our frequently crashing computer “hates” us. Even before we have a fully formed ToM, we name our stuffed animals and give them backstories. Our social cognition systems are always “on,” even in non-social contexts.
What Makes LLMs Different
Long before the rise of large language models, researchers in human-computer interaction (HCI) observed that people readily attribute human-like qualities to machines. From early studies of ATMs and answering machines to more recent experiments with virtual assistants and social robots, humans have consistently responded to technology as if it had thoughts, feelings, and intentions—even when they knew better. This reflex, sometimes called the “media equation,” reflects a deep-rooted tendency: we apply the same social rules to computers that we do to people. If a machine speaks politely, we interpret it as courteous. If it pauses, we assume uncertainty. If it remembers our preferences, we may feel seen.
These anthropomorphic tendencies were manageable—even occasionally charming—when the machines involved were limited in scope and behavior. But things have changed.
Enter the modern LLM, which has supercharged our innate ToM reflexes by mimicking natural language fluency, emotional tone, contextual memory and goal-directed interaction. All of these features combine to create a convincing illusion that there is a “mind” on the other side of our chat window that is similar to ours.
A recent study found that LLM developers can actively dial up or dial down the illusion of mind through design choices: empathetic language, memory cues, and frequent use of first‑person pronouns—especially after multiple turns—significantly increase users’ perceptions of the model as a human-like agent. Even if we are armed with an understanding of how the technology works, we are vulnerable to misattribution of human-like intentions, motivations, and emotions to our algorithmic conversation partners.
Can LLMs Do Theory of Mind Tasks?
Several recent studies suggest they can—at least in limited, test-like ways. For example, Kosinski (2023) tested GPT-3.5 and GPT-4 on classic “false-belief” tasks used in developmental psychology. These tasks ask whether a subject can understand that another agent might hold a belief that’s incorrect. Remarkably, GPT‑4 solved about 75% of these—comparable to the performance of a typical 6-year-old child.
A follow-up study by Kosinski et al. (2024) expanded the test battery to hundreds of diverse scenarios. It found that large models consistently outperform both earlier AI systems and even some groups of humans. Similarly, Strachan et al. (2024) compared models like GPT-4 and LLaMA 2 to nearly 2,000 human participants on a broad range of ToM tasks. In some areas, the models performed as well—or better—than humans.
That said, researchers disagree on what this means. Some argue these models have no true understanding of minds—only patterned mimicry of how people talk about mental states. Others suggest that this performance, while shallow, reveals an emergent cognitive scaffold. Either way, this research helps explain why LLMs can so easily trigger our ToM reflexes: they’re not just syntactically fluent—they’re behaviorally plausible as social minds.
Anthropomorphizing LLMs Can Be Useful
This anthropomorphic overfitting by LLM users can be both helpful and risky. On the positive side, it can:
1. Enhance engagement and motivation
Perceiving an LLM as socially present – e.g., as a “listener”, “coach,” or “collaborator” – may boost sustained attention and motivation during tasks, willingness to experiment, ask questions, or revisit ideas, and feelings of being heard.
2. Lower inhibition and encourage reflection
When users anthropomorphize an LLM as a nonjudgmental peer or coach, they may express ideas or feelings more freely, work through problems more patiently, and better tolerate failure or confusion
3. Provide emotional regulation and support
Anthropomorphizing can provide comfort in times of stress, a sense of companionship or “presence,” and enable emotional labeling (i.e., helping a user name what they’re feeling)
4. Offer a sandbox for practicing social skills and exploring different perspectives
Treating the LLM as a social partner can offer practice with empathy, assertiveness, or conflict resolution, exposure to alternative perspectives (simulated roleplay), and a space to test and refine interpersonal strategies
5. Act as a muse or co-creator
Artists, writers, and researchers sometimes benefit from collaborative anthropomorphism. They treat the model as a quirky, unpredictable partner which can help inspire creativity, surprise, or serendipitous insight
But the Mirage Has Risks
All of these potential benefits, however, come with attendant risks. Deeper engagement can lead to overreliance and overtrust. Having a nonjudgmental partner creates an emotionally safe space, but it is easy to confuse empathetic language with true understanding, or genuine care. The fluency of LLMs when used as a thought partner can beguile us into mistaking their output for truth.
The fact is that the “mind” we perceive animating our chatbots is fundamentally a mirage, a seductive illusion. It is up to us, as users of these tools, to anthropomorphize them in healthy ways that benefit us without fully indulging our ToM instincts. LLM developers control the defaults—voice, tone, memory, and responsiveness—that shape the mind mirage at scale. But users control the frame of each interaction: how they prompt, what they expect, and how much they let the fiction breathe. Over time, we will hopefully come up with better metaphors and language to describe the pseudo-minds of LLMs that we will increasingly be interacting with. In the meantime, here are some ideas for practicing healthy anthropomorphism in our LLM use:
Ultimately, the “mind” we perceive animating our chatbots is not a soul peering back—it’s a trick of language, a reflection of our own cognitive habits, and anthropomorphic design choices made by model developers. Still, that doesn’t mean we should banish the illusion. If we learn to use it wisely—leaning into the fiction while keeping one foot in reality—we might make better use of these strange new mirrors we’ve built.
I’m interested in hearing the experiences others have had in working with LLMs. Have there been moments when you’ve been seduced by the mind mirage so fully that you’ve forgotten the true natures of LLMs? Do you have any practices you’ve developed to prevent yourself from engaging in anthropomorphic overfitting? Do you worry that AI developers are too incentivized to make the mind mirage as beguiling as possible to encourage engagement and use of their tools?
Relatedly, I’ve been developing some services reflective of insights I’ve gleaned from working with LLMs in groups.
Click to learn more and book some time with me if you might be interested.
Leave a Reply