The Theory of Mind

‍How does the brain determine what it thinks versus what other people think?

We can observe people, but we cannot look directly into someone’s mind. All we can do is to infer another person’s intentions and thoughts based on our experiences with and observations of them. For that, we are using what is called the theory of mind network of our brain. In this section, we will discuss the individual brain parts that make up this network and how it can support our ability to put ourselves into the shoes of others (empathy) and to recognize ourselves in the mirror.

Theory of mind   The concept of theory of mind refers to the ability to imagine what other people (or animals) are thinking and what they know. This provides a significant advantage for hunting (predicting whether the prey can see or otherwise sense you), or social interaction (teaching, trading, and even lying, etc.).

The theory of mind network integrates information that has been processed in the ventral and dorsal streams in the temporal and parietal lobes. The first steps in this network are the left and right temporoparietal junctions (lTPJ and rTPJ). There, both the ventral and dorsal streams are combined to determine what is going on and where it is happening relative to you [Mishkin and Ungerleider, 1982].

Temporoparietal junction   The temporoparietal junction is a brain area between the temporal lobe and the parietal lobe, integrating data from the thalamus, as well as the limbic system, and the visual, auditory, and somatosensory systems. Combining the “what” and the “where” information, it creates a distinction between “self” and “other.”

While the idea of having two separate streams of information is helpful to understand what is going on, research has revealed that their relationship is more complex [Sheth and Young, 2016]. There are interactions between both streams before they reach the TPJ [Schenk and McIntosh, 2010]. For example, if the temporal lobe (ventral stream) recognized a cow in the sense data, the parietal lobe (dorsal stream) could use that information to look for the cow down on the grass first instead of up high in a tree. If the parietal lobe has determined that there is something sitting high up in a tree, it is probably no cow—information the temporal lobe could use to refine its own processing.

As we have seen previously, the function of some brain parts can be explained by what the surrounding brain parts do. Specifically, the area of the TPJ is in the middle of the angular gyrus, supramarginal gyrus, and Wernicke’s area (and in its mirrored homologous area in the right hemisphere). With this in mind, we can predict the functionality of the left and right TPJ.

The left TPJ (lTPJ) contains the Wernicke’s area (mapping meaning to words, creating sentences), the left angular gyrus (written language, positioning of letters, words, and numbers), and the left supramarginal gyrus (phonological memory, extension of limbs, and tool use). The right TPJ (rTPJ) contains an area that is analogous to the lTPJ’s Wernicke’s area (dealing with subordinate meanings of ambiguous words), the right angular gyrus (spatial relationship of self to the external world), and the right supramarginal gyrus (interpreting postures and gestures of other people and inferring people’s emotional states).

Taken together, the TPJs’ responsibilities could be summarized as identifying oneself, differentiating “self” from “other,” and inferring and interpreting one’s own and the other person’s emotions, intentions, and thoughts. While this network also supports the learning of the body schema, it is thought that the left and right TPJ, together with parts of the prefrontal cortex (and connecting structures in between) are especially involved in forming the theory of mind network [Schuwerk et al., 2016].

The Role of the Prefrontal Cortex

For the theory of mind, the prefrontal cortex also plays a significant role. Its basic functionality of counterbalancing other parts of the brain evolved over time to allow for more complex behavior. This is reflected in a number of abilities we can observe in our own thinking:

Object permanence. Just because your senses no longer report the existence of an entity does not mean that the entity has ceased to exist. A tiger hiding behind a tree still exists, yet causes none of your senses to produce a signal. Without your prefrontal cortex, the tiger would literally vanish and you would have to rely on more primitive systems like the hormonal system (fear) to help you to remember that you are in danger. With adrenaline in your blood, you would keep running instead of taking a break once the tiger is out of sight. In this context, the prefrontal cortex can be understood as running a simulation of the world, similar to the way the superior parietal lobe runs a simulation of the internal state of the body, or the supramarginal gyrus continuously updates the body schema. This simulation allows you to mentally track objects outside of your current sense perception (object permanence, Baird et al. [2002]). Human children have to learn this ability, which is often incorporated in games like “peekabo” or “hide and seek.”

Object permanence   The ability of object permanence allows us to track predicted positions or movements of objects even after they have vanished from our field of view. By running a simplified simulation of the world, we are aware that a tiger that has jumped behind a tree is still there.

Reward delay and focus. To focus on one activity, the brain needs to suppress distractions. To accomplish this, the prefrontal cortex generates signals to suppress other signals. For example, reading a book in a public place requires you to tune out what is going on around you. Without the prefrontal cortex, which helps you to suppress acting upon something capturing your attention, you would act out whatever is related to objects in front of you. In patients with frontal lobe damage, this is called “utilization behavior” [Shallice et al., 1989]. Such patients have a reduced ability to resist anything that captures their attention and they have an urge to execute whatever action is connected with that particular item. This could be a pen to write something, or a smartphone demanding attention and stimulating the user to consume its virtual candy (checking on social media). The decision would be made by the rest of the brain without any regard to risks, your goals, or social rules (see below). Similarly, animals with a limited prefrontal cortex will eat whatever snack or play with whatever toy you present them. Training animals to do otherwise requires adding, for example, a social component of obedience to train a dog to resist eating any food that is available.

Memory. Another way of looking at the prefrontal cortex is as a “traditionalist committee” in the context of the neural committees. By making memories, our own past experiences can become advocates for a specific behavior. You might have the urge to pick a beautiful rose, but your memory tells you that it is connected with a negative experience (the pain from touching its thorns). So, you suppress the immediate reaction of grabbing the rose and first examine if and where you can touch its stem.

Strategic thinking. Setting a goal in the future (planning) is a form of suppressing other actions. For example, when visiting downtown, you might want to calculate a sequence of shops you plan to visit and where to eat lunch. Being able to plan for the future also allows you to make predictions about the future by tracking the position and state (e.g., there are two coffee shops downtown but only one is open) of objects, animals, or even the mental state of other people with you (my friend will want coffee).

Long-term goals. If you see a candy bar behind a fence, instead of trying to walk directly to the candy bar (and bumping into the fence like a fly against a window), you take your initial motivation (the candy bar) but suppress the immediate idea of walking directly toward it. Instead, you analyze the situation, develop a strategy, and walk around the fence. A fly bumping against a window does not have this ability.

Risk management. If a tiger is waiting on the other side of the fence, neither memory nor a strategy can help you to get to the candy bar. Instead, the prefrontal cortex analyzes the situational risk and suppresses your desire to get to the candy bar in the first place. Once the perceived danger is gone, the prefrontal cortex sends the signal that the situation is safe. This is put forth as an explanation for obsessive compulsive disorder (OCD). Supposedly, this signal is blocked or too weak in sufferers of OCD who might then be unable to stop, for example, washing their hands [Apergis-Schoute et al., 2017]. Indeed, experiments show that people with OCD have less activation in their prefrontal cortex [Hirosawa et al., 2013]. Some people show OCD-like symptoms regarding their home, taking extra time to visually confirm that the appliances are off or unplugged. This way, they are supporting the prefrontal cortex with a stronger “safe” signal using sense data coming from the parietal lobe.

Processing emotions. In pre-historic times, emotions were crucial to our survival. For example, getting frustrated or angry allowed our ancestors to mobilize additional strength to push a boulder, break a branch, or wrestle an opponent. Likewise, fear and anxiety might have helped our ancestors to have more energy to run away (flight behavior). With the complexity of social interaction, our ancestors were also faced with situations that could not be solved by direct action. Their prefrontal cortex had to learn to reframe the emotional evaluation of a situation in a positive way or redirect frustrations into overcoming the problem. For example, imagine one of our ancestors is very hungry. Instead of stealing food from another member of the tribe and risking a fight, he could suppress his hunger and try to negotiate and trade.

If the underlying frustration remains unresolved, the prefrontal cortex can still find a way to cope with the emotions by, for example, humor in the face of adversity or loss; seeking emotional support from others; denial; escape (including drugs and self-medication); disengagement or dissociation; avoidance; or blame (other people or oneself). Negative maladaptive strategies tend to be more successful in coping in the short-term but not in the long-term as they are not addressing the underlying issue, and thus the actual conflict remains. For positive coping strategies, the prefrontal cortex must be able to construct a different scenario and present that to the amygdala (an imagination). Disorders like PTSD are basically the opposite of positive emotional coping. In people with PTSD, traumatic memories are reactivated and they emotionally relive those moments through their thalamus and amygdala.

Self-generated control. The prefrontal cortex also allows direct self-generated (meaning that there needs to be no external stimulus) control of your skeletal muscles. Life without the prefrontal cortex could be imagined as being in the gym, and the only way you could do any exercise would be if someone scared you in order to cause you to jump or run. Likewise, your eyes would be controlled by reflexes and would move only as a reaction to another moving object within your field of vision—sufficient to duck, but not enough to actually make decisions that are more than just reactions to your immediate environment or your inner state.

Dealing with uncertainty. Anterior to the premotor cortex is a prefrontal cortex area that deals with uncertainty [Volz et al., 2005]. Even with the decision mechanisms like the neural committees using an evolutionary algorithm to reach a decision, internal conflicts can persist. Sometimes, you end up with choices you cannot analyze further and this might lead you to think in circles. If I offer you 100ifyouthrowadieandgeta1,butyouhavetopayme100ifyouthrowadieandgeta1,butyouhavetopayme20 if you do not throw a 1, should you accept the deal? What is more important for you, the small chance of a large win, or the higher chance of a small loss? In this case, this part of the prefrontal cortex can resolve the conflict and at least make a decision (for example, based on probabilities).

Breaking addictions. By developing habits (strategies), you can overcome the addictive behavior promoted by competing brain parts. This is particularly challenging if the addictive behavior itself damages the prefrontal cortex, as is the case with alcoholism [Nakamura-Palacios et al., 2014].

Speech. Broca’s area is part of the left side of the prefrontal cortex and is anterior to the parts of the primary motor and premotor cortex that deal with the face, tongue, and larynx control. Broca’s area is connected to Wernicke’s area, which is responsible for forming sentences, while Broca’s area is responsible for translating those sentences into actual spoken words. Broca’s area is even activated when not actually moving any muscle, which makes speaking or singing “in your head” possible.

Aphasia is an impairment that can be caused by damage to the prefrontal cortex (see Figure 5.24). In such a case, the affected person still knows what he wants to say, but he is significantly hindered in his ability to find the words to express his thoughts. The analogous area on the right side of the prefrontal cortex is involved in non-verbal communication like gesticulation, rate, rhythm, and intonation of speech, as well as facial expression [Johns, 2014].

Broca’s area   Broca’s area is a brain part located in the left side of the frontal lobe and connected to Wernicke’s area. It is responsible for the production of speech. Damage to Broca’s area leads to a person unable to find the words to express what he wants to say. The homologous area in the right hemisphere deals with non-verbal communication.

‍

Not all animals with a prefrontal cortex have all the abilities listed above. For example, besides primates, so far it has only been shown that dogs, cats, and a few species of birds have the ability to track objects once they are no longer visible (object permanence). Surprisingly, many predator animals do not have this ability. When prey suddenly vanishes underwater or behind a tree, they are visibly confused and unsure where it went. Their basic emotional system still works and they are still in hunting mode. But without an idea of the location of their prey, they might give up their hunt.

Mirror Test

Beyond tracking objects in the environment, the prefrontal cortex also tries to predict what other people know. This theory of mind ability can be demonstrated in an animal with the so-called mirror test. For example, we could work with a chimpanzee; let us call her Tinkerbell. A blot of coloring is secretly added to Tinkerbell’s head where she cannot see it without the help of a mirror. We make sure she did not feel us putting a blot of color on her. We then put her in front of a mirror. We know that when faced with a mirror, chimpanzees first make threatening gestures at their own images but after a while learn to use their reflections for self-directed behaviors like making faces at their reflections or grooming previously unobserved parts of their body [Gallup, 1970]. If she starts examining the blot of color, she passes the test, because the only way she could have known that there was a blot of coloring on her head was by recognizing herself in the mirror. Passing the test indicates that she is able to map her internal decisions to her actions, as well as her own movements to the mirror image’s movements.

To pass the mirror test, Tinkerbell needs several sources of information:

• The visual image of the chimpanzee in the mirror (using the visual cortex): “I am seeing a chimpanzee in the mirror.”
• An idea of what chimpanzees usually look like (for example, other members of her tribe) to recognize that something is odd about the chimpanzee in the mirror (the blot of color).
• The information about what initiated the movement (with the help of the premotor cortex): “I know I am moving my arm.”
• The recognition that her own movements match the movements of the chimpanzee in the mirror (with the help of the supramarginal gyrus): “I am seeing a chimpanzee in the mirror who moves like me.”
• Further analysis of the movements with the conclusion that the cause of the mirror’s image actions must be herself . Only she has the information of what she will be doing next. If the ape in the mirror is another ape and not herself, the other ape does not have the same information that Tinkerbell has. This requires the TPJ (labeling whose information belongs to whom) and prefrontal cortex (tracking who knows what): “I am causing the mirror image to move its arm,” “Only I knew that I would move my arm,” and subsequently “That must be me in the mirror.”
• Finally, she notices that her internal body schema does not match the mirror image (“I have something on my head”) and she starts examining the blot of color on her head.

While the mirror test seems to be a special case as mirrors are rare in the natural world, this ability to differentiate self and other helps whenever we are touching someone. For example, imagine holding someone’s hand and then seeing it moving. Who or what caused the hand to move? With additional input from the motor and the somatosensory cortex, the rTPJ figures out how to integrate what you see (the moving hand) with the information from the self (did I move the hand or did the other person move the hand?) to make sense of what is actually happening. If you had to rely only on a single source of information (for example, your eyes), you would not know who is holding whose hands.

Mirror test   The mirror test evaluates the ability of an animal to recognize itself in a mirror after a researcher has secretly added a blot of coloring to the animal’s body and put the animal in front of a mirror. If the animal starts investigating the blot of color on its own body instead of on the mirror image, it passes the test (because that indicates the animal realizes it is the same creature that it sees in the mirror).

Besides primates, also dolphins, Eurasian magpies, African elephants, and possibly giant manta rays pass the mirror test [Ari and D’Agostino, 2016]. Dogs and cats seem to be at the very edge between animals that can recognize themselves and those that cannot. It is important to note, though, that the mirror test relies heavily on an animal’s vision, while the main sensory focus of dogs and cats is smell. Maybe someone will design a “smell mirror” test in the future that a cat or a dog might be able to pass. While dogs can recognize their own smell, it is unclear how to test whether they do anything with that information [Horowitz, 2017]. For example, we do not know if they change their behavior (like their diet) when they smell signs of illness—and even that could be interpreted as an instinct rather than actually recognizing themselves.

Another way to demonstrate the ability for a theory of mind is the “Sally and Ann task” experiment [Wimmer and Perner, 1983]. Here, “Sally puts a marble into a basket. While Sally is away, Ann quickly puts the marble into another box. Then, the subjects are asked at which location Sally will search for the marble. Hence, the participants have to perform a shift in mental states and breach with their former expectation (true location) to understand Sally’s false belief of the location of the marble. This involves inhibiting their intuitive response of naming the true location” [Krall et al., 2014]. In other words, if they answer that Sally will look into the box where Ann put it, they have failed to demonstrate having a theory of mind. Sally could not have known that Ann put the marble into the box.

Yet another situation in which we need the theory of mind is when we want to explain something to someone. For this, we need to have the ability to know what someone else knows. This is something children have to learn while their prefrontal cortex only fully develops throughout adolescence until around the age of 25 years [Arain et al., 2013]. For example, when asked what she is doing, a child might tell her father, “I’m playing with this” and then hold up a toy car, even if this conversation occurs over the phone. This is because the child, at her age, cannot consider that the father does not see the toy car. To solve the misunderstanding, the child needs to understand the other person’s perspective and add additional information. The child would need to use her prefrontal cortex to figure out what the father knows about her situation, and her TPJ to differentiate her own knowledge about the situation from her father’s.

Empathy

The rTPJ enables us to see the world from another person’s point of view. For the calculation of the rTPJ, the point of view (our own or that of other people) is just another input variable. This architecture allows us (with input from the prefrontal cortex) to have sympathy and empathy for other people. Just because we see a person being angry or sad does not make the person unsympathetic or sympathetic—it depends on the context that the rTPJ and prefrontal cortex provide (background information, body posture, facial expression, pitch, tone, etc.).

A theory for lack of empathy and sympathy, especially in criminals, is that the processing ability of their rTPJ is underdeveloped or malfunctioning. Experiments have shown that there is significant activity in the rTPJ when having to decide whether something is moral. In order to make that call, you have to include other people and situations into your judgement.

The connection between the rTPJ and morality was confirmed in a study where the participants’ activity of their rTPJ was disrupted with magnets and then the participants were asked to read several accounts of a situation and judge whether the protagonist (“agent”) acted morally [Young et al., 2010]. The scientists’ conclusion from the study was that we form moral judgments based on a combination of the agent’s intentions and the actual outcome. If our ability to judge an agent’s intentions is disrupted, we put more emphasis on the outcome. To illustrate the differences between intentions and outcome, let us examine one of the stories that was presented to the participants, which was about Grace (“agent”) and a white powder she put into her friend’s coffee (see Figure 5.25):

Neutral belief

- Neutral outcome: Grace thinks the powder is sugar. It is sugar. Her friend is fine.

- Neutral outcome: Grace thinks the powder is sugar. It is toxic. Her friend dies.

Negative belief

- Neutral outcome: Grace thinks the powder is toxic. It is sugar. Her friend is fine.

- Neutral outcome: Grace thinks the powder is toxic. It is toxic. Her friend dies.

Figure 5.25: Four examples with different outcomes and initial beliefs about the situation. Depending whether or not the rTPJ of the participant in the study was disrupted, different moral judgements about Grace were made.

• Unsuccessful attempts to do harm (outcome neutral, belief negative). In the first variation of the story, Grace puts white powder from a container marked “toxic” into the coffee of her friend. The container was mislabeled (it was actually sugar) but Grace had to have assumed that she was putting toxic powder into her friend’s coffee. In the experiment, disrupting participants’ rTPJs while they were thinking about judgment of such intended but failed acts to do harm led to lenient verdicts. They judged Grace depending on the outcome (whether or not her friend died), not her intentions (whether she knew it was toxic powder).
• Well-intended but harmful acts (outcome negative, belief neutral). In the second variation of the story, someone replaced the contents of a sugar container with toxic powder without Grace’s knowledge. When Grace put the toxic powder into her friend’s coffee, she did not know she was poisoning her friend. Again, the particpants’ rTPJs were disrupted while they were thinking about a judgment, and ended up with severe judgements. They judged Grace depending more on the outcome than her intentions.

Possible therapies for real-life violent offenders might include a virtual reality simulation where they see themselves from the eyes of their victims. For example, researchers of the University of Barcelona have developed a virtual reality system so that men who committed a domestic violence crime can experience the act from the victim’s point of view. The study [Seinfeld et al., 2018] compared a group of violent offenders with a control group. The former group showed a significantly lower ability to recognize fear in a woman’s face. After their virtual experience, participants in the study showed improved scores in a test where they had to guess, by looking at a picture of a person, the emotions that person is experiencing. Before they experienced themselves from their victim’s perspectives in the virtual reality, violent offenders “showed a bias towards classifying fearful […] faces as happy.” Using treatment with the virtual reality experience, this bias was reduced. Further questioning also revealed that violent offenders had a significant bias to responding in a manner that would be viewed favorably by others. More research is needed for an actual therapy; it is still unclear whether it is their distorted view of themselves and others that leads to a lower threshold of using violence, or if that view is a result of trying to rationalize past violent behavior.

Such a therapy could train the violent offenders’ rTPJ, putting more emphasis on the thoughts, feelings, and intentions of a person when making a moral judgment. Instead of simply blaming another person for a negative outcome (and then possibly using that as a justification for violence), they could better think about whether or not the other person really had an influence on the situation. For example, someone with a weakened rTPJ might think about his partner: “I had a bad day at work. Whatever you did, the outcome was that I had a bad day. So, you are to blame.” Someone with a trained rTPJ might come to a different conclusion: “I had a bad day at work. You supported me, other factors (maybe even myself) are to blame for my bad day.”

Did you know? The approach of splitting blame into observations, feelings, needs, and requests is the central idea of Marshall Rosenberg’s method of non-violent communication. It provides on way of training your emotional intelligence and creating a psychologically safe space.

This idea is supported by findings that the rTPJ’s volume is strongly associated with an individual’s altruism [Morishima et al., 2012]. Stimulation of the rTPJ seems to enhance a person’s ability to take someone else’s perspective [Santiesteban et al., 2012]. Better connectivity of the rTPJ also seems to result in less bias toward people outside one’s social group [Baumgartner et al., 2015].

Beyond empathizing with other people, the ability to see things from another person’s perspective is also relevant to empathizing with one’s future self . Stimulating the rTPJ seems to have an effect on one’s ability for self-control [Soutschek et al., 2016]. In experiments, participants were asked whether they would prefer to have a pile of money now, or a significantly larger pile of money later. With the rTPJ interrupted, they preferred the short-term win over the long-term gains. This was supported by other studies comparing empathy and impulsivity [Pajevic et al., 2018], and examining the relationship between substance abuse and empathy [McCown, 1989]. It seems that for the brain, it does not matter if the person with whom we are empathizing is another person or our future self.

Similarly, together with the prefrontal cortex, the lTPJ also helps an individual to know her own (and those of others) thoughts, beliefs, intentions, and desires [Gallagher et al., 2000]. For example, detecting a lie requires the ability to recognize the conflict between another person’s inner state (what they know) and what they are expressing. Damage to the lTPJ can impair this ability [Samson et al., 2004]. Such damage also impairs the ability to tell convincing lies if doing so requires knowing what the other person thinks.

Knowing someone else’s internal state enhances our ability to teach and lead other people. If the lTPJ is damaged, our brain might no longer recognize that other people’s minds can differ from our own. In this case, we would lack the motivation to tell other people what we have learned: we would assume that they already know what we know. With the help of a healthy lTPJ and by using context information from the prefrontal cortex, we can track who might know what.

To build a theory of the mind, the prefrontal cortex predicts how people (and objects) in the environment will react and what other people think or believe, while the TPJ connects this information to actual entities in the real world. For example, the lTPJ is involved in the association and memorization of names of individuals and objects [Gorno-Tempini et al., 1998]. If you know that Peter likes Sally, this is just abstract information in your prefrontal cortex. It becomes concrete when the lTPJ provides the information that you are Peter and the person in front of you is Sally.

We are so used to trying to predict what other people are thinking that we even attribute an inner state to non-living things. Think of how people curse at non-functioning computers or cars, despite knowing that this behavior would make sense only if the computers or cars had a human-like mind. A car does not—indeed cannot—care what we say to it. Is consciousness merely the attributing of an internal state to things and people, or does it have a deeper function?