MINNEAPOLIS — Athletes from around the world will soon compete for gold in the summer Olympics in Paris, but for those who come up just short, bronze might bring more happiness than silver.
A research team at the University of Minnesota Carlson School of Management used facial expression software to analyze more than 400 photos of Olympic athletes on the medal podium.
"The software uses machine learning or artificial intelligence," said William Hedgcock, an associate professor of marketing at the Carlson School of Management. "It decodes facial expressions and measures the 'emotional intensity' of different expressions, including neutral happy, sad, angry, surprised, scared, disgusted, and contempt.
These are the expressions we can actually encode with a certain amount of accuracy. There are many other emotions but we can't code them by looking at your expressions."
After comparing the expressions of gold, silver and bronze medalists between 2000 and 2016, and ranking the happiness associated with each medal winner, the study revealed an interesting upset.
"It's no surprise that the gold medalist tends to have the happiest expressions," said Hedgcock. "What is surprising is that bronze is more happy than silver. If you're silver (medalist), it's salient to think that I could have been first. If you're bronze, you're sitting here going, I could be that person sitting on, not on the medal stand. So I'm feeling pretty good."
A viral photo of Team USA gymnast McKayla Maroney looking "not impressed" by her silver medal in the vault during the 2016 Olympics stands out as an example of that second place letdown.
"The software shows anger and contempt are the highest expressions from Maroney," he said. "Reactions like this are clear, but the vast majority of comparisons involved smiles."
Still, Hedgcock points out that Maroney was disappointed because she was the clear gold medal favorite that year. He says his team did analyze how expressions corresponded to medal expectations, but he says the results tied to silver and bronze were consistent throughout.
"You know, within any one image, a lot of things could be happening (emotionally), but over 400 images we have a much better picture of these sort of counterfactuals and what likely is driving this behavior," he said.
Why does it matter?
In addition to providing insight into the psychology of elite athletes, the technology used in the study has become a critical tool for marketing research.
Hedgcock: "We use similar facial expression technology to look at how people respond to advertisements and movie trailers."
Kent Erdahl: "What have you learned from photos that maybe you wouldn't have learned from polling or surveys?"
Hedgcock: "I can see how you're reacting without having to ask you. If I ask you, it affects how you process in the future. I like to go to comedy clubs and if you ask me after every joke, how funny was that it affects my subsequent reaction to the next joke."
In addition to decoding photos, the software can also be used to track facial expressions in real time via a webcam.
"That can give us an idea of how you're responding over time and there's a lot of value in that," Hedgcock said. "We often use surveys at the end of an ad and that's great to understand how you process the ad at the end, but we may not accurately remember what we were doing five or 10 seconds into the commercial."
The technology can also serve as a sort of truth test, even for athletes.
"Past work had surveyed people to find out why they had certain expressions, but we know that sometimes people don't want to admit things," Hedgcock said. "Maybe you don't wanna say, well, I was less happy about silver because it doesn't sound good."