SAN DIEGO— The brain must actively seek out social information from other people rather than just passively receive it, according to Ralph Adolphs, PhD. Speaking at the 17th Annual Meeting of the American Neuropsychiatric Association, Dr. Adolphs described some of the neural underpinnings of processing emotional information from faces. Accurate recognition of emotionally and socially meaningful stimuli requires feedback loops within the brain, in the body, and with the social environment, he said. "We look at one another’s faces all the time. From doing that, we make inferences about what is happening in their bodies and their minds. We end up with a conviction that the person feels happy or afraid—states we can’t observe directly."

Studies also suggest that it may be possible, in some subjects who have autism or brain damage, to improve ability to recognize facial emotion. This can be done by instructing subjects to look at areas that they ordinarily ignore (such as the eyes), noted Dr. Adolphs, of the Department of Psychology and Neuroscience at the California Institute of Technology, Pasadena.

CLASSIFYING AND PROCESSING EMOTION

Emotion researchers classify emotions into two broad categories: basic emotions (happiness, surprise, fear, anger, disgust, sadness) and social/moral emotions (guilt, shame, pride, embarrassment, jealousy). Dr. Adolphs said that the basic emotions can be identified very reliably in facial expressions across different cultures. The social/moral emotions are more difficult to show just on the face. "Typically, we need more information, background, and context to recognize social emotions," he said.

The amygdala is the brain structure most involved in processing social information. "The prediction would be that if you have lesions in some of these regions, you would be impaired in some aspects of your social cognition and social behavior. What is important to keep in mind is that processing information about other people involves not just processes in the brain but also in the body. The brain uses the body to help it process socially relevant information," Dr. Adolphs said.

He noted that the information the brain deals with is much richer than the information presented through the senses. Functional imaging studies have identified different regions of the brain that were activated when subjects looked at faces versus houses. "It’s not the case that there is a single place in the visual cortex that just processes information about faces," he said.

One area of interest is in the fusiform gyrus and is called the fusiform face area. Highly processed visual information from cortical areas is then conveyed to other structures such as the amygdala. Projections from the amygdala also carry information back to the visual processing regions, putting the amygdala into a position to modulate the perceptual processing of the face, based on emotional values.

"If you see somebody with a fear face or an anger face, the amygdala has tagged this as something that is emotionally highly salient and subsequently allocates how facial information is directed toward that face," Dr. Adolphs said.

Animal studies by Heinrich Klüver (1897– 1979) and colleagues showed that damage to the amygdala left animals unable to recognize the emotional and social meaning of what they were seeing. Monkeys with such lesions would calmly handle rubber snakes, which normal monkeys fear and will not handle.

Likewise, a human subject with amygdala damage in Dr. Adolphs’ studies had a normal ability to recognize facial expressions of happiness but could not recognize fear. Dr. Adolphs noted that functional imaging studies have also shown increased regional cerebral blood flow in the amygdala in response to images of faces showing fear. "If there is a lesion in the amygdala, it disproportionately impairs brain recognition of fear in faces," he said.

Studies of facial recognition have shown that brain damage shifts the curve, depending on the area of damage. "If you look at low-scoring versus high-scoring patients with brain damage, you find that certain brain regions are associated with impaired performance in recognizing fear in facial expressions," Dr. Adolphs said.

FEEDBACK PATHWAYS

The processing pathway begins from the early visual cortices and proceeds through the fusiform gyrus, superior temporal gyrus (where a representation of the face is constructed), amygdala, orbitofrontal cortex, and projections to various brain stem and hypothalamus areas that trigger an emotional response involving the autonomic, visceral, and endocrine functions.

"Feedback from the body to the brain allows you to simulate what you think might be going on in the other person. You look at the person’s face, see a facial expression of fear, and think, ‘How would I feel if I had that face?’" Dr. Adolphs said. Feedback from the amygdala back to visual cortices can modulate perceptual presentations, he noted.

THE EYES HAVE IT

Dr. Adolphs described several lines of research supporting the conclusion that the eyes are the most important area of the face for cues in recognizing fear. The researchers also worked with a patient who had amygdala damage and whose ability to recognize fear in faces was impaired. They found that she did not spontaneously look at the eyes in pictures of faces and did not use information about the eyes in interpreting facial expression.

"We know that the eyes are most important for recognizing fear. If we instructed her to look at the eyes, her performance became nearly normal," Dr. Adolphs said. "The simple manipulation of instructing her to look at a feature most people look at but that she failed to [observe] on her own—[showing her] what to allocate visual attention to, in order to sift out information that could be potentially informative—improved her functioning."

The finding has a possible application to autism, and Dr. Adolphs described similar studies showing that high-functioning autistic subjects have significantly fewer fixations on the eyes than do normal control subjects. Instead, they make more use of observation of the mouth in recognizing fear.

EMOTIONAL LOOP-THE-LOOP

"Three kinds of processes come into play when we look at someone’s face to determine how they feel," Dr. Adolphs concluded. "How do we recognize emotionally and socially meaningful stimuli? Loops in the brain reconstruct knowledge about what the person might be feeling, perhaps in part by simulating how you feel if you look like that. Loops in the body can trigger an emotional response in the body and perhaps mimic in your own face the facial expression of the person in the stimulus. Loops with the social environment have not been investigated much, but if you want to know how someone feels, you don’t take a snapshot of their face, you ask them how they feel. This is a dynamic we need to take into account in future studies."

—Janis Kelly

Suggested Reading
Adolphs R, Gosselin F, Buchanan TW, et al. A mechanism for impaired fear recognition after amygdala damage. Nature. 2005;433:68-72.
Bar-Haim Y, Shulman C, Lamy D, Reuveni A. Attention to eyes and mouth in high-functioning children with autism. J Autism Dev Disord. 2006;36:131-137.
Bucy PC, Kluver H. An anatomical investigation of the temporal lobe in the monkey (Macaca mulatta). J Comp Neurol. 1955;103:151-251.
Dalton KM, Nacewicz BM, Johnstone T, et al. Gaze fixation and the neural circuitry of face processing in autism. Nat Neurosci. 2005;8:519-526.

Return to table of contents