At Affectiva, our Emotion AI technology can seamlessly be integrated into any interface, allowing research teams to utilize our facial expression analysis software as one layer of their analysis. By adding other survey research methodologies, researchers can achieve a deeper understanding of how consumers view and digest a piece of creative.
Our Media Analytics team partnered with Oculid to test their advanced eye tracking technology for smartphones with our facial coding. Through our pilot project we wanted to explore how the combination of Emotion AI and Eye Tracking can provide powerful insights to brands interested in using both measures for digital mobile content.
Methodology:
We launched a Friends and Family project of two UK 2022 Christmas ads from TikTok platform: John Lewis “The Beginner” and Aldi “Kevin the Carrot: Home Alone”.
The task was completed through Oculid’s mobile app. Prior to participation, we obtained opt-in consent from our participants for collecting face videos, gaze position and screen recordings for the analysis. Oculid’s app is compatible with both Android and iOS, so everyone was able to take part using their own smartphones, making this a viable solution for digital content evaluation. Emotion AI and Eye Tracking on mobile analysis were both used to generate valuable insights.
John Lewis – The Beginner
John Lewis, a UK department store, released a Christmas 2022 advert focusing on UK children in the foster care system. The ad shows a man attempting to master skateboarding. After a few bumps and mishaps along the way on his quest to skateboard, we learn that he is learning to skate to connect with a child in care when she arrives to the house for Christmas.
When looking at the Smile and Sentimentality Emotion AI traces (below), we can see peaks shining through when the man having mishaps on the skateboard and his family commenting on his “achievements”. Viewers found those moments light and humorous; but could also relate as we have all been there when starting to learn a new skill, especially in adulthood. Further positive reactions (Smile) happen in response to viewers figuring out the mystery of his adventurous behavior, when noticing both the skateboards based on the gaze analysis.
The final messaging and branding moment appearing at the end evoke further emotional reactions. The protagonist faces the camera backwards and leaves the room slowly. This allows participants to read the message without any distractions, indicating that the scene works well with regard to focusing attention on branding and messaging. In the Emotion AI traces, an initial uplift in Brow Furrow is witnessed with people focusing and reading the messages, followed by a final peak in positivity (Smile) as a reaction to the message of John Lewis pledging to continuously support young people in the UK care system (below).
The overall story being linked well to the key communication as well as wisely exposed messaging result in creating emotional resonance with the branding moments benefiting from the Peak-End rule to enhance memorability.
Aldi UK – Kevin the Carrot: Home Alone
Kevin the Carrot has become a well-loved character for Aldi supermarket’s Christmas adverts. This year Aldi created a series of TikTok ads about Kevin’s family Christmas travel leading to the main creative where after missing a plane Kevin is left at home watching football (or soccer, depending on where you are!) and undergoes several evasive maneuvers to escape the clutches of some intruders, referencing the Home Alone Christmas film theme.
As expected, the combination of the cartoon elements with the Home Alone theme generates high emotional engagement with the viewers. The ad does a good job with the respect to visually guiding participants through the story - attention is pretty much centered on the protagonists and in particular on their faces … even when it is carrot faces!
With the whimsical elements, there was some evident Brow Furrow. As the camera pans down through the Christmas dinner table, we can see concentration and high attentional capture as viewers are searching the table to see where Kevin is hiding (below).
The joke about “a snowman’s… nose” becomes the peak of emotional reactions. We can see with the eye tracking that Aldi was effective in setting up this comedic moment as viewers’ attention focused on the snowman’s face before realizing where Kevin actually ended up!
The joke elicited high instantaneous smiles, though there were some mixed reactions provoking smirk (an asymmetrical smile) and Mouth Open (indicating surprise). Despite that, the positivity was the dominant reaction – the joke was elegantly done.
The Aldi ad proved to be an emotionally engaging story. However, the eye tracking data reveals that having five static cues for viewers to focus on doesn’t help to assimilate them all, as viewers naturally need to divide their attention between the different topics while following the carrot family. In comparison, John Lewis’s “The Beginner” had a cleaner approach with one focal point, making the message and partnerships easier to process. Therefore, the social messaging of Aldi donating meals may have limited ability to come through strongly.
CONCLUSION:
By collaborating with Oculid, we were able to successfully integrate our Emotion AI technology with their mobile camera-based eye tracking for this pilot to discover next-level findings. While data collection is powered by the Oculid app, the data outputs can be seamlessly integrated in Affectiva’s interactive dashboard for easy accessibility.
The best analysis is a multimodal approach to capture the different facets of human behavior. By having both Emotion AI and mobile Eye Tracking, we can provide research agencies a more detailed picture of consumers’ nonconscious responses to digital assets seen on mobile devices.
Interested in learning more? Schedule a demo with our team at Affectiva here.
