Ever felt 'happily disgusted'? Computer maps 21 distinct facial expressions

tt / Ohio State University
By Sarah Bourassa
Recognize these feelings? He's happily disgusted, left, and sadly fearful, right. A computer program recognizes 21 distinct facial expressions.Today

You know that feeling when someone lets you down and you’re angry yet disappointed at the same time? Or what about when you watch a gross-out comedy that leaves you feeling disgusted yet cracking a smile? A new study says computers have found a way to recognize an unexpected 21 distinct facial expressions, including those more complex ones like "sadly angry” or "happily disgusted.”

The connection between internal emotions and facial expressions has captivated people for thousands of years. People can recognize many different emotions, but until now most research was limited to only six basic facial expressions: happy, sad, fearful, angry, surprised and disgusted.

Ohio State University researchers felt surprised when their computer model identified an important group of compound emotion categories.Today

“I was expecting to find more than six but 21 was a huge surprise,” says Aleix Martinez, a cognitive scientist and associate professor of electrical and computer engineering at The Ohio State University. 

Researchers at Ohio State, including Martinez, reported in the current issue of the Proceedings of the National Academy of Sciences that they were able to more than triple the number of documented facial expressions that can be used for cognitive analysis. For example, a happily surprised expression combines muscle movements observed in happiness and surprise.

While some people are able to hide inner emotions in their facial expressions, this progress in identifying facial expression could be significant for helping people with disorders such as PTSD or depression, says Dacher Keltner, associate professor of psychology at the University of California, Berkeley. 

"This could provide clues to how well the person is doing in life; how well they are doing if they just returned from Iraq as a veteran," says Keltner, co-director of the Greater Good Science Center.

These photos represent 20 of the 21 distinct facial expressions perceived in a computational model.Today

The resulting computational model from this study will help map emotion in the brain with greater precision.

“We can only study the brain if we know what we’re looking for,” Martinez says. If you don’t know what these 21 emotional categories are, you’re not going to understand the brain because you won’t know what you’re trying to find, he says.

Understanding the different emotional variables that relate to emotional health could potentially aid with diagnosis and treatment of conditions such as depression or autism. 

The findings could also influence computer science and designing perceptual interfaces (how the user interacts with the computer), allowing computers to interact with you in a more human-like manner, according to Martinez.

For the study, the researchers photographed 230 volunteers (130 female, 100 male, and mostly college students) making faces in response to verbal cues and then tagged prominent landmarks for facial muscles (such as the corners of the mouth). They used Ekman's Facial Action Coding System, or FACS, a system that analyzes which facial muscles that have moved to create an expression. 

They searched the FACS data for similarities and differences in the expressions, and found 21 emotions — the six basic emotions, as well as emotions that exist as combinations of those, including disgustedly surprised, angrily disgusted, or sadly angry.

This study suggests that the 21 emotions are expressed in the same way by almost everyone in American culture. There were no differences in gender facial expressions. The study focused on mostly college students but Martinez suspects that if there are any differences in facial expressions by age, they are small.

“My suspicion is that the basic facial muscles we use to express emotions are the same because they are biological,” Martinez says.

However, previous research on body language has shown that people often misread facial expressions and even trained experts are no better than anyone else detecting when someone is lying, according to a recent story in the New York Times

"The face is not the only way, or even the major way, by which humans communicate emotion," psychology professor Joseph Campos of the University of California, Berkeley said in an email.