As we’re bombarded with countless images on social media — from cat memes to Christmas portraits — some pictures are forgotten almost as quickly as they appear. But have you ever wondered what makes a photograph unforgettable?
Researchers at the Massachusetts Institute of Technology have created an artificial-intelligence algorithm that predicts the memorability of an image at a "near-human" level. The system can also produce a “heat map” that identifies which areas of an image are the most memorable (shown in red) and least memorable (shown in blue).
You can upload your own photos into a demo version of the system here.
This selfie of Dwayne "The Rock" Johnson was deemed highly memorable by the system, which tends to favor images that include faces and exposed body parts. Photo: Instagram/therock
The technology could have future applications in advertising, education, and — more importantly — an Instagram filter that makes posts more memorable with the tap of a finger.
RELATED: Selfie-stick etiquette: 6 tips for taking great shots (without being a nuisance)
In order to “teach” the system what a memorable photo looks like, the researchers fed the algorithm tens of thousands of images that had each been given a “memorability score” based on how well a random group of people remembered them in online experiments. The team then instructed the algorithm — which operates like a simplified brain using multiple layers of processing — to figure out what makes each photo memorable or forgettable.
With enough training, the system was able to predict the memorability of the photos to within a few percentage points of the average human scores. Faces and exposed body parts were among the features the system determined to be highly memorable, though the researchers never told it to look specifically for those features.
The system tends to rate wide shots and landscapes as less memorable than images with prominent subjects. This photo of a girl riding a horse received a low memorability score. Photo: Mark Armao
Initial research has been conducted to figure out what’s going on “under the hood” of the algorithm, but the researchers admit that it’s difficult to tell exactly how the system comes up with its predictions. Aditya Khosla, a graduate student in MIT’s Computer Science and Artificial Intelligence Laboratory, compares the machine-learning system to the mind of a child.
RELATED: How to take better photos of your pet in 9 simple steps
“You teach a baby what a spoon looks like and the baby starts identifying spoons all around the world without you needing to tell him or her, ‘Hey, that’s another spoon,’” Khosla, who is the lead author of a related paper, told TODAY.com. “But you have no idea what’s going on inside the brain of the child.”
In this selfie of Miley Cyrus, the algorithm highlighted the face and tongue as the most memorable regions of the image. Photo: Instagram/mileycyrus
The algorithm seems to favor unusual objects (like a cactus couch) as well as seemingly mundane portraits (such as a woman standing next to an abstract painting). Scenes of open landscapes tend to get low scores even if they are visually striking.
RELATED: Meet the 5-year-old photographer who's landed National Geographic and a new book
While facial expressions were not a focus of the current project, Khosla said that faces showing negative emotions tended to be more memorable than those expressing positive ones.
Khosla foresees several possible applications for the software in fields such as advertising, in which marketing companies often try to hammer a repeated image into the mind of consumers. If companies could optimize the memorability of their images, they could get away with using the stronger photos less often, he said. In the field of education, the system could be used to select more memorable textbook images or diagrams.
Using more than 50,000 images, MIT researchers trained an artificial-intelligence algorithm to predict the memorability of a given image. The system "learned" to key in on faces and exposed body parts, as shown by this photo of Olympic medalist Matt Grevers. Photo: Mark Armao
“We spend so many hours as kids trying to memorize so many things from so many different subjects,” Khosla said. “This system could allow [educators] to pick the right images that would stick in students’ minds.”
The algorithm could also be used to modify photos to make them more memorable. Khosla envisions an app that could blur the forgettable parts of an image, or even add content to an image to make it more memorable — meaning, yes, we may have finally found a way to get those cat memes more strongly embedded into your brain.