With what we have learned about consciousness, we will re-examine the evolution of attention, and expand it with the attention schema and awareness schema. Instead of focusing on animals of the past, we will focus on a single example of a co-evolving predator-prey system. Imagine a group of tigers and a number of small apes. The tigers are hunting the apes, and the apes are trying to evade the tigers while at the same time trying to find fruit. They are living in a forest where trees provide cover, and where berry bushes provide a (limited) source of food. The forest also contains berry bushes with poisonous berries, and some of the non-poisonous berries are not yet ripe and thus not edible. Over generations, the tiger population evolves better ways to catch prey, while the ape population focuses on improving her abilities to forage, and evade predators. In the first part of this section (unconscious strategies), we will build up the cognitive abilities of the apes (and tigers) step by step using elements we have learned earlier. In the second part, we will enhance the behavior using the attention and awareness schema.
Unconscious Strategies
For the first part of this thought experiment, we will look at strategies using basic parts of the brain like the sense organs, the amygdala, and the hippocampus. Initially, we use a “dumb” tiger as a predator. It will simply run toward an ape when in sight, but will wander around randomly at other times. For the tiger to be successful, all it needs is to detect movement, so a stereoscopic view (and all the mental machinery necessary) is not (yet) necessary for the tiger.
As the tiger could run in any direction, it needs to have something that leads it to prefer one action over the other. To train its own neural network, it needs a reward system (supervised learning) which includes the amygdala. In the beginning, the tiger might wander around aimlessly. But once it has associated meeting (hunting) an ape with food, it will seek out more potential prey (more apes) in the future. It will have associated apes with a positive emotion. To better track the ape, it might even evolve a superior colliculus, which improves eye movements related to moving targets. Now, as the tiger begins to learn to hunt the apes, the apes themselves might evolve better strategies to forage food and evade the tiger. Given how the brain evolved over time, the following list can be seen as one strategy layered over the other, with more brain complexity required for each strategy.
Random. The most basic strategy for an ape would be to just act randomly. This could be walking around, staying close to berry bushes, trying to eat berries, and (lacking sense data) might even try to nibble on trees, tigers, or other apes. That strategy is of course far from optimal because it does not increase the chances of finding berries, getting away from a tiger, or avoiding poisoning from certain berries.
Pain and pleasure. We give the apes a sense of touch, and a sense of taste. If the sense of taste detects toxins, or the sense of touch physical pain, the ape might refrain from continuing to eat something (for example, trees or poisonous berries). Vice versa, if the sense of taste detects nutritious food, she will eat the berries more eagerly.
Tracking inner state. Eating more food than the ape needs unnecessarily increases the chances to get food poisoning, and will deplete food sources. Tracking the current internal state to generate a feeling of hunger or satisfaction can regulate the pain and pleasure mechanism. This already is a primitive model of the body to control food intake: instead of eating everything in sight and then going hungry until a new source of food is found, consumption is distributed over time.
Sense of smell. Adding a sense of smell (the olfactory system) allows an ape to decide in advance whether food is good or not. For this to work, the apes need an amygdala that connects sense data with the pain and pleasure mechanism. Smells associated with a positive experience (nutritious food) are evaluated as positive; smells associated with a negative experience (from eating trees or poisonous berries) as negative. An ape might even learn to associate the smell of the tiger as something dangerous (assuming the ape survives an encounter) and end up not trying to eat the tiger in the future.
Sense of sight. Next, the apes could evolve a basic sense of sight with eyes, their retinas, and a superior colliculus. Connecting the sense data from the eyes to the amygdala, an ape could learn to associate the visual information from berries, tigers, and trees to decide from afar and without needing to smell them to determine whether or not they are dangerous. This is of course only a small advantage as the ape has no sense of direction and all she can do so far is to stay and eat or move around randomly.
Directed movement. Using back propagation with the amygdala as the supervisor for supervised learning, apes can learn more complex movements than chewing (or spiting out) food. The apes will still move around randomly if no entity is in sight. They will either approach or move away from the strongest signal. For an ape as we know it, the challenge is to not only use the visual information, but also the information about which direction the head and the eyes are turned. Our apes do not have this model of their body (yet), so we just assume that neither the eyes nor the head can turn. In that regard, “seeing” always means “seeing in front of us.”
Turn toward movement. The primitive amygdala we gave the apes does not recognize particular entities (like the visual cortex does). Instead, it takes all the incoming sense data and decides whether or not it fits a previous experience. Hence, a berry bush in the open field does not provoke the same degree of response from the amygdala as a berry bush in the forest. While there are some similarities, the ape needs to learn those situations separately. In particular, this means that the ape cannot analyze the visual field to decide in which direction to turn. There is no understanding by the ape that something on the left side of the visual field can be seen better by turning her head to the left. Instead, that movement would have to be initiated by a genetically programmed reflex. Then the ape can turn toward a movement, identify the possible threat, and then use the amygdala to run away from it.
Refined senses. Next, the apes might evolve a sense of hearing. This improves the apes’ ability to detect a nearby tiger. Instead of being able to look only forward, they will have a 360-degree perception of possible dangers nearby. When an ape hears something from a particular direction, she turns toward that direction and her amygdala can react to the new sense data she sees. For the actual comparison, the thalamus is required. It integrates sense data from different sources (in this case, the left and right ear) into abstract information (direction). The thalamus could also create correlations between visual sense data and auditory sense data to strengthen the signal (something is moving and it makes noise, for example, a tiger) or weaken the signal (something is moving and it makes no noise, for example, a cloud).
Filtering sense data. Beyond combining sense data, the thalamus and the connected cortices (auditory cortex, visual cortex, and olfactory cortex) preprocess sense data to arrive in the amygdala with less noise. Noise reduction is important for the amygdala as it learns based on absolute sense data (e.g., the image of a specific apple) and not abstract concepts (e.g., the concept “apple”). For example, a tiger in the shade should be handled similarly to a tiger in the sun. Likewise, just because the ape hears some background noise, the roar of a tiger should elicit a similar response as a roar at night when it is otherwise quiet. For filtering, the thalamus can also combine the information from different senses. For example, it could use visual information like the location of the origin of a sound to improve its auditory filtering. Vice versa, being able to locate the source of a sound with the ears might help to locate the source of a movement with the eyes.
Resolving conflicts. At any single moment, the ape can move only in a single direction. But different thought patterns might try to take control of the muscles at the same time, with one part of the brain wanting to go left while the other wants to go right. For example, in which direction should the ape run when she sees two tigers approaching from two different directions? In the worst case, the ape might end up doing nothing because she cannot decide. To arbitrate such conflicts, the basal ganglia can at least lay out basic rules of how decisions are made and then clearly communicate that decision to the muscles. One such rule of the basal ganglia could be a simple majority rule: execute the action dictated by the strongest signal and suppress the other signal. For the ape, it is better to run into one direction at her maximum speed than to run half-heartedly because she is worrying about the direction she has not taken.
Keep doing things. Next, the apes could evolve a hormonal system (hypothalamus, pituitary gland) to temporarily remember positive and negative experiences. For example, when the ape sees a tiger, the amygdala might induce fear by releasing a hormone through the hypothalamus. As long as that hormone remains in the blood, the ape’s organism could adapt its body chemistry to either run faster (and keep running in one direction, even if the tiger is no longer in sight) or stay frozen in place so that the tiger does not see the ape. Similarly, it could cause an increase in appetite so that the ape stays near a berry bush to eat all the berries before moving on.
Remember locations. Adding the hippocampus to store memories of locations will help the ape to find her way to nearby berries more quickly. A certain formation of trees might make the ape remember that she has found a lot of berries nearby. By associating places with each other, the ape could remember a path toward a field of berries. This works in a similar way as backpropagating for neurons: the ape can associate a place positively if she encountered it before reaching an already positively associated place. For example, a tree-lined path might lead to a group of trees which in turn lead to a group of berry bushes. It can also help the ape to find a path around an obstacle such as a group of trees.
Working memory. Sense data is fleeting, so at least in the short-term, it would be good to keep recently perceived sense data present. For example, if the ape heard something on her left and turned her head, the sound might no longer be audible. Still, knowing her previous location will help the ape to visually identify the location more quickly. With the phonological loop and the visuo-spatial scratchpad, the ape can correlate past with current sense data. Both mechanisms form the working memory and send sense data into a loop between the auditory cortex and the thalamus or the visual cortex and the thalamus. The model the brain uses to access the working memory is the attention schema.
Empathy. If the ape sees another ape throwing up food, it might not be a good idea to eat from the same berry bush as it could be poisonous. As she has no understanding of how berry bushes work, the mechanism would need to evolve slowly over time as a genetically programmed instinct. A mechanism could evolve to recognize basic facial expressions that influence the pain and pleasure mechanism and the tracking of the internal state. As if she had the experience herself, she would be repulsed either by the specific berry bush or (with the help of the hippocampus) the area itself.
Learn the environment. The hippocampus can also store a timer for the ape to return once the berries have grown back. The ape might mentally note to not visit recently harvested berry bushes soon. If there are no known (and ripe) berry bushes nearby, the hippocampus could specifically favor exploring places the ape has not been before, judging already visited places as slightly negative. It could use similar pathways that are used for tracking the internal state to determine hunger. Instead of having just a one-dimensional hunger feeling, the hippocampus generates a feeling of hunger for particular places. Just like we might prefer going to a particular restaurant to order our favorite dish that satisfies our taste (and nutritional needs), the ape would refrain from visiting a place with depleted berry bushes in favor of another place with an abundance of berries.
Flexible eyes and head. Making the head and eyes flexible allows the ape to save energy by turning just the eyes or head instead of the whole body. This requires an extension of the model of the internal state of the ape (that, so far, tracked only hunger) with limb positions by combining sensory information from the skin and internal sensors from the muscles in the somatosensory cortex. By correlating visual sense data with head movements, the position of the eyes and head relative to the environment and body can be calculated. For example, when the ape looks behind herself and sees a tiger, the tiger is in front of her eyes. It is not in front of her body, though. She first has to translate her turned head and eyes to the relative direction of her body. This forms the dorsal stream of sense data between the visual cortex and the somatosensory cortex.
Smooth movements. At this point, the ape still uses instinctive movements to turn toward movement or noise. For example, the ape sees something moving on the right side of her visual field. She could now turn the head and eyes a little to the right, check her visual field again, turn again, and so on, until the interesting part of her visual perception is in the center of her visual field (with maximum definition). However, this going back and forth takes a while and—as with the simple thermostat—might cause under- or overshooting the target. To turn the head faster (and more accurately), the model of the head and eyes can be used to predict how far the head needs to be turned for the target to be in the center of the vision. The ape faces the same problem when using her limbs. To move faster (and with fewer accidents), the ape has to coordinate the movement of her legs and arms. She has to predict what movement leads to which positioning of her limbs. Once the basics of this prediction system are in place, the evolutionary race with the tiger (with the ape getting more agile with each generation) is on. Over time, the motor cortex, somatosensory cortex, and nearby areas of the supramarginal gyrus and angular gyrus evolve.
Motor programs. Enhancing the control of muscular movements, the brain can combine movements into motor programs. These motor programs are like a sheet of music, indicating starts and stops for different instruments (muscles). For this, the ape needs enhanced basal ganglia to not only arbitrate currently conflicting thought patterns, but also to coordinate thought patterns over time. The basal ganglia can achieve this by suppressing or promoting movements in a certain sequence governed by the motor program. A simple example for such a motor program would be turning the head. This could replace the movements the ape previously had to learn separately for each sense data combination. Turning to the right because the ape has heard something coming from the right could use the same motor program as turning to the right because the ape has seen something moving on the right side of her visual field.
Enhanced empathy. Combining the mechanisms for analyzing the face to detect whether or not another ape has eaten poisonous berries, the ape could reuse her supramarginal gyrus and angular gyrus not only to read her own limbs to build a body schema, but also to interpret the posture of other apes. This way, she could indirectly access other apes’ senses. For example, she could approach them if she sees them munching on berries, or run away if she deduces that they must have seen a tiger.
Form groups. When apes form foraging groups, they have much better chances than individual apes to detect an approaching tiger. The preference to form a group could easily evolve by being able to recognize apes and judging them as “positive.” This could evolve genetically or through learned experiences during childhood.
Conscious Strategies
While the strategies discussed in the previous section do not require awareness, they do show relatively complex behavior. The main challenge of this approach is that the ape with this basic setup would be able to use only supervised learning. This means that discovering her environment requires either an instinct, or through slow and possibly dangerous experiences in the real world. While it works well with rather static entities like regrowing plants and with reactive behavior (flight, fight, or fawn), it is problematic in dynamic environments or with situations with a lot of variables. This is no different than the thermostat which also had problems with, for example, changing locations or seasons. Similarly, trying to communicate to other apes where the tiger will probably show up requires higher brain functions. Here we will examine how we can get there step by step, again with strategies that layer on top of each other.
Unsupervised (long-term) learning. Let us add a more diverse flora to our imagined world. The properties of a single type of plant can be learned very quickly, even with supervised learning. But if an ape wants to learn about multiple types of plants, and wants to recognize different sizes and development stages of the plant, she needs unsupervised learning. What the ape needs is a link between the concept of, for example, a berry bush, and the (one-time) experience the ape had consuming berries. For the brain to handle concepts and properties, apes need to evolve a declarative long-term memory, and a process to transfer the sense data to the memory (dreaming!). To connect one memory with other memories, the ape can use her hippocampus’ ability to connect different locations with each other. What the apes now have to evolve is a connection between the declarative memory and the amygdala. Now, instead of the amygdala having to learn each entity separately, sense data are categorized first. For example, different (even yet unknown) types of plants could be categorized as “nutritious” and would all evoke a similar reaction from the amygdala. If she finds a plant that is poisonous after all, she would have to reevaluate her categorization of plants.
Instinctive communication. By adding the ability to make sounds, the ape could communicate danger. Using unsupervised learning to categorize calls and create mappings between observations and calls, this can lead to a basic language. For example, the ape could observe her parents making a certain call when finding berries. The temporal lobe near the ears and could serve both as a way to encode as well as decode individual calls. A loud shriek might indicate that a tiger is seen nearby, while a longer low call might be associated with a food source. This communication would still be unconscious and a direct result of, for example, the amygdala’s emotional response (danger, food, etc.).
Combine information. Information about the what and where (processed by the dorsal and ventral stream in the parietal and temporal lobe) could be combined in a new brain part (the TPJ) to build an abstract description of the world. This enhances the working memory to provide the information to other parts of the brain. In a first step, this could improve signal processing combining the phonological loop and visuo-spatial scratchpad. We see its workings with auditory and optical illusions where both bottom-up processing as well as top-down processing happen. If the ape knows what she is looking for, her brain can use this information to process the image more effectively. For example, berry bushes grow on the ground, not high up in the tree.
Tactics and object permanence. The ape is already using models to track her inner state, to have smoother control over her movements, and to interpret the state of other apes or the tigers. Adding a prefrontal cortex that constantly reads the working memory to create an internal model of the entities in the environment allows smoother tracking of entities. Instead of being surprised when a tiger disappears behind a tree only to be surprised again when it reappears, the prefrontal cortex can mentally track its position. This allows an ape to “keep in mind” that there is a tiger behind the tree. This hidden tiger could be used by the prefrontal cortex to continue to project fear into the amygdala, causing the ape to be more cautious or even leaving the area—even though the tiger is no longer visible.
Episodic buffer. Connecting the hippocampus to the working memory allows the ape to distinguish events in their temporal sequence. This helps with interpreting what is happening. The sound of a crouching tiger means a tiger is nearby, the sound of a crying ape means that an ape is injured. Both in combination might point to a successful hunt by a tiger.
Planning. The ape could be faced with different options to forage. For example, should she take the risk of walking all the way to the large group of berry bushes (higher risk but also higher reward) or take no chances and just forage the berry bush nearby? While choosing the closest berry bush is an effective strategy, it might not be the most efficient. Depending on the risk associated with a particular area, as well as the distance to the area, the ape might prefer one option over another. As the relative distances change all the time when the ape moves, the amygdala’s positive or negative evaluation of a place is not sufficient for the decision. That risk could be calculated by the prefrontal cortex and used to modulate (suppress) the amygdala.
Pathfinding. With access to a model that contains (visible and hidden) entities of the environment, the ape could calculate in advance the shortest path to a location. For example, the ape might know that a bush of berries is behind a hedgerow. Instead of walking toward the hedge and then trying to find a way around it, the ape could calculate in advance the shortest path. To accomplish this task, the prefrontal cortex might need a connection to the basal ganglia and actively suppress decisions to walk outside the optimal path.
Classifying berry bushes. While the hippocampus already uses a temporal component to not visit berry bushes that were only recently harvested, using a model of berries might increase the harvest. There is a specific time window in which harvesting a berry bush is optimal. Too early and the fruits have not sufficiently grown, too late and the fruits are probably already harvested by another ape or have fallen to the ground. This requires classifying the berry bush according to its development phase and suppressing the urge to visit the berry bushes when the model says that they do not carry any ripe berries.
Language. Replacing supervised learning with unsupervised learning and using concepts from her temporal lobe instead of instinctive behavior ruled by the amygdala, the ape could start to communicate more effectively. For this, a connection is needed between her temporal lobe and frontal lobe with the motor cortex (in humans, between the Wernicke’s area and the Broca’s area). For example, she could specify what type of danger she is seeing, rather than having to learn each call separately for each situation.
Advanced tracking of the inner state. The ape’s ability to find the shortest path to a goal can be improved by combining it with the internal state tracked in the primary somatosensory cortex. Tracking also caloric intake, energy, sleeping cycles, and walking speed, the ape might decide not to simply seek the berry bushes with the best reward/risk factor, but instead to strategize using all known factors. For example, the prefrontal cortex could activate the amygdala to increase hunger, which the parietal lobe would translate into eating a lavish meal before the journey, or making a detour to gather berries on the way to a larger field of berries.
Focus and intelligence. Just being able to plan ahead and suppress particular movements might not be enough to actually follow through with a plan. Given that the brain parts compete with each other, a stronger emotion might overrule the influence of the prefrontal cortex. For example, we can plan exercise and diet but ultimately fail when being confronted with dessert. Instead of trying to overpower the amygdala directly with the neurons in the prefrontal cortex, the apes could evolve a way to recruit other parts of the brain to influence the working memory (the awareness schema). Using the new connection between the motor cortex and the temporal lobe, the ape could use it in the opposite direction and activate concepts by thinking about them. The act of thinking would use the same pathway as a physical action. But instead of a motor action at the end, the motor cortex activates the temporal lobe. Hence, “thinking” feels the same way as executing a physical action. Standing in front of an apple the ape cannot immediately reach, she might use her “mental arm” to think about the apple and recruit the rest of her brain to devise a plan. This activation then goes the same route as an activation by the visual cortex, namely into the working memory. From there, it goes back to the thalamus and back into the neural competition and then possibly through the basal ganglia again into the working memory. With this loop, the ape can recruit multiple parts of the brain to, for example, overpower the signal from the amygdala and continue with a more important goal.
Imagining a future or past. With each of these cognitive loops, the ape could also build up a scenario in her mind. Instead of just using the loops to suppress the amygdala, the ape could think one step ahead, have other parts of the brain process this future scenario, and use the result to calculate the next step. For example, the prefrontal cortex has no machinery to imagine three-dimensional structures. But the visual cortex and parietal lobe do, so mentally rotating an object requires the interaction between the prefrontal cortex and the parietal lobe and visual cortex. Helpful in this endeavor is also the episodic memory, which puts thoughts into a sequence. By imagining a future scenario, the ape can better plan ahead. For example, she could imagine the next day and gather more berries than she could eat now and store them for later.
Understand causality and build models. Once the ape can track time and imagine herself in the future, she can also track how properties of other things change. This means that the ape can now conceptualize processes and causality. This allows the ape to observe other apes and understand that the presence of apes diminishes the available berries over time (well, they eat them). This model can be used to enhance the ape’s strategy to evade places with a lot of other apes as they might pick the berries before the ape can reach the berry bush. Similarly, the ape can build models of the tiger’s behavior (not just its current state) and how the tiger hunts apes. She might see the tiger less as a threat if there are other apes nearby, as the tiger can hunt only one ape at a time. To make more informed decisions, the ape could also take note of how fast the tiger can run. For example, the ape could observe that a tiger that just has eaten is not as fast, so the ape could come closer to the tiger. Understanding object permanence allows tracking possible positions of other apes and the tiger. Furthermore, the ape could observe whether or not the tiger has noticed her and sneak away. The ape could also suppress her urge to call out that there is a tiger if the tiger has not noticed her yet. Similarly, the ape could observe whether or not another ape has noticed her, and decide to steal some berries.
Ultimately, being able to look into the future forms the foundation for goals and empathy. Instead of, for example, just assuming tigers are bad because they evoke a negative reaction in her amygdala, she could now imagine what would happen if she got injured or even killed by the tiger. Besides the pain involved, she might be thinking of her family who would starve if she did not return with berries. Her fear can then become rational as she would be acting on her values and on thought instead of just instinct. Similarly, she could refrain from eating berries if relatives are nearby or even stand in the way of the tiger to sacrifice herself to protect her relatives.
Tool use. Besides pure logic and causality, she could also recruit the SMG and AG to think about complex mechanical problems and consider tools as an extension of her arm. It requires three-dimensional thinking about how, for example, her arm, a stick, and the target relate to each other.
Gaze following. Enhancing her abilities to analyze the way eyes of other apes move, she can more quickly detect possible danger. She can also refine her theory of mind of other apes, the model of their inner worlds. By knowing what they pay attention to, she can predict more accurately what they will do. The more the ape interacts with other apes, the more she will know what other apes know. She can deduce what they are seeing, hearing, or thinking. This is a prerequisite for better communication. For example, instead of always calling out when she sees a tiger, the ape could first check whether other apes are seeing the tiger, too. If they do, calling out would just draw attention from the tiger to the apes. Likewise, by observing the tiger, the ape can decide whether or not the tiger has already seen her.
Deception. She could also use her insight into what other apes know to deceive them. For example, by making the “tiger call,” she might scare other apes away so that she has all the berry bushes for herself. Apes could also develop an evaluation for each ape of the tribe. Tracking outcomes of interactions, they might find out who is helping them out and who is deceiving them. Apes could demonstrate their relationships with each other openly to others to demonstrate to all the apes present whether or not an individual ape is trustworthy. The ape could then reflect on how trustworthy other apes judge her. Similarly, the apes could develop a model of a culture that reflects what the tribe allows and what is not allowed, possibly keeping deceptive behavior in check. Having a network of trust, the apes could show more complex behavior following strategies like “I scratch your back, you scratch mine.” Berries could be used as presents to improve one’s standing in society. Especially strong apes might have a better standing as they might scare the tiger away if it is not particularly hungry.
Advanced language. Combining her abilities to think about alternative scenarios and about causality, she can learn to form more complex statements and grammar. This helps with planning a defensive strategy against the tigers. Together with tools, the apes might even start coordinating an attack on a tiger. Better language capabilities also help to form better relationships by allowing the apes to negotiate and form contracts instead of just relying on their feelings toward members of the tribe.
Explaining behavior. Explaining the reasons for her behavior helps other apes to make better predictions about the ape’s future behavior. For example, she might decide not to share her berries with another ape. By verbally explaining that she needs the berries for her starving children at home, the other ape might not value her less. ‘
Writing and the scientific method. At this point, the natural evolution of the apes gets interrupted. With the development of language and tool use, apes will be able to write down their experiences to teach later generations or apes that are far away. This increases the apes’ collective intelligence. The apes will be able to change their environment faster than their biology will be able to keep up. The apes will be able to think about how mental processes work, and will start philosophizing about how they are able to reflect. Maybe the apes will start studying neuroscience to discover how their inner cognitive reflection works. This way, the apes could counteract mental impairments or maladaptations to a changing environment. Knowledge of psychology and neuroscience can help the apes to understand and better process subconscious mental processes and emotions.
And here you are.
