From the screech of fingernails down a blackboard to the closing of our eyes and listening to waves hit the shore, sound has a profound impact on how we feel. However when researchers try to understand how viewers feel whilst watching film content, traditional research approaches can overlook the emotional impact that thoughtful music, narration and sound effects can have. 

This is somewhat understandable, as within many research approaches, insights are derived from how consumers describe, in their own words, the creative content they’ve just experienced. 

We should certainly not ignore the value that can be generated from examining such viewer sentiment, but by exclusively focusing on participant responses after they’ve fully reflected on content AND then played this back to researchers in their own words, we’re limiting our understanding. 

Viewers will naturally anchor their descriptions around ‘what they’ve seen’, meaning that the important role of music or audio’s influence can be underestimated, or glossed over in analysis and optimization. In not unlocking the ‘full brain picture’ as to how we process creative stimulus, we can’t see the role that sound plays in influencing viewers unfiltered emotional response.

This challenge drove award winning independent filmmaker Christian Wenger, to partner with Affectiva in order to put sound right at the center of emotional testing for a sci-fi drama. The short film depicts a deeply dystopian setting, where existential themes of health, death, scarcity & what it means to be human are explored. If you’re interested, you can view the film in full here.

An ephemeral viewing experience, punctuated by builds in tension and highly visceral moments, needed a testing framework that could unlock sensorial triggers; to ultimately do justice to the subtitles, nuance and range of emotions the film was trying to communicate.

Generating Audience Empathy 

Central to the film's success was setting up the intended dark atmosphere and emotional tone of the short film. The film featured ‘body scanning’ visual images (see below), so the viewer not only understood the themes of ‘health’ and ‘aging’ in the future, but also so they would truly feel and empathise with the plight of the protagonist. The wider themes being conveyed through the deeply personal experience and reflections of the character in focus. Christian wanted to use Affectiva technology to understand whether the jarring sound effects of the ‘scanner’ could disconnect and distract viewers. 

What does the future of humanity look like? The ‘scanner scenes’ from the award winning ‘Three Pills’ short film 

The Affectiva Approach: How Emotion AI helped select the more emotive edit

Affectiva’s Emotion AI was identified as the critical tool to highlight and diagnose the key emotional moments within the film, in order to select the most emotionally powerful edit. 

An audience of film viewers watched the film from the comfort of their home, on a device of their choosing. The Affectiva technology was deployed unobtrusively throughout the viewing experiences, to understand complex and nuanced emotions and cognitive states by analyzing facial movement throughout the film viewings. 

Specifically, the impact of soundtrack and audio effects was explored by testing two identical edits in terms of visuals, but whilst one edit included a highly visceral sound effect stemming from the ‘scanner’, this was excluded from the second test film.

Action Oriented Outcomes: 

Our ‘moment by moment’ reporting & visualization, showed the breadth and depth of emotional intensity felt at each point during the short film. This helped diagnose important demographic subtleties, such as older men feeling the most “anger” throughout the film. Pointing to the fact that ‘empathy’ played out differently, depending on a viewers background.

However, the key finding from our analysis showed that the uncomfortable and disconcerting sound effects from the scanner actually lead to stronger viewer connections, with higher mental effort (brow furrow), that aligned with the intention of the filmmaker. 

The deliberate discomfort instilled from the audio was not something that viewers would necessarily talk about. How a viewer felt in each and every moment, unbridled from expectation, cultural norms and what they are able or willing to articulate, was only discernible through Affectiva’s Emotion AI technology. 

Ultimately the film went on to win the prestigious best direction award at the Berlin Sci Fi Film festival.

The Bottom Line: Technology That Supports Creativity

Emotion AI technology can add real insight into viewers responses to film & entertainment content. Delivering a deep understanding of the way different audiences respond, unlocking emotional insights that truly guide and support creative development. This allows content creators to research and optimize their narrative among large audiences, without the need to gather people together in a central venue.

Interested in learning more about Affectiva’s technology in the development and testing of entertainment contact? Contact us today for more information, or see a demo now!