Authors: Rana el Kaliouby, Evan Kodra, Pankaj Jha, Namita Mediratta

This paper recounts the journey that Millward Brown and Affectiva undertook in 2013 to adopt, scale and operationalize Affdex emotion-sensing technology as an integral part of Link tests globally. Challenges and insights are discussed in depth with a focus on asia.

 Challenges:

• Cultural and market norms

• The influence of online vs offline methodologies

• Subtle expressions

• Technical: scaling the platform

Cultural display rules vary by culture, and their effect has been observed repeatedly in studies of spontaneous expressive behaviors. While some cultures tend to neutralize expressions, and others amplify them, one important finding is that East Asians are more likely to mask or dampen expressions in the presence of others, especially moderators. The result is that proper emotional interpretation requires cultural context, which Affectiva exposes through the use of normative information (norms).

This was an important factor to study as data gathering in Asia, at the time, took place in central locations, and not online. To further evaluate this difference, Affectiva and Millward Brown conducted a joint study in Japan in 2013 to understand these factors. A set of 12 ads were tested online and offline in a central location. The result was that in ads testing online we found increased dynamics but the same overall pattern between online and offline collected data.

Another challenge came in the form of the magnitude of expressions being expressed; specifically in Asia we were observing very subtle expressions. In order to properly capture expressions, Affectiva refined our recognition technology to improve the performance of detecting subtle expressions. While the ability to detect a smile is often demonstrated by posing a large exaggerated smile (read: Smile for the camera!), in practice what was required was for the system to detect very subtle, low intensity smiles as those are the expressions that manifest in the real world.

These innovations were built into an online portal for use by market researchers at Millward Brown. This enabled researchers to provision and launch studies, perform data collection and perform normative comparisons independently.

In additional to the overall findings, the paper goes on to describe three case studies for ads testing in Asia:

• WildAid “Save the Elephants” Campaign  (China)

• Ad optimization for Clear ‘Helmet’ Campaign (Indonesia)

• Sunsilk Radiant Shine – ‘Satin’ (India)

Abstract

Affdex emotion-sensing technology analyzes facial expressions to discern consumers’ emotional reactions. Millward Brown and Affectiva partnered to capture emotions at scale, making facial coding an integral part of Millward Brown’s advertising pretesting worldwide. We discuss the challenges we faced as we rolled this out globally, and present groundbreaking emotion-sensing algorithms and methodological best practices that address cultural influences, especially in Asia  This work led Unilever, the innovative global brand, to incorporate Affdex worldwide to gain novel insights into ad engagement.

Read the paper here.