By: Ashley McManus, Marketing Manager; featuring media artist Ruben van de Ven, creator of Emotion Hero.
Our emotion-sensing and analytics technology is transforming industries, particularly in the marketing and gaming spaces. The premise of its applications: to get an unbiased metric of somebody’s mental state, which allows for statistics and comparisons. With Emotion AI growing in popularity, this tech has gotten the attention of media artists who examine the political implications of technologies and test their propositions.
Emotion Hero, a video-game-artwork that uses the Affectiva SDK, was designed around this concept. The gameplay is similar to Guitar Hero - but for the face. Players are required to show specific emotions, and if they are able to adapt their faces well enough to the requested expression, points are awarded.
We interviewed media artist Ruben van de Ven, creator of Emotion Hero, on how the app works, the process of developing it, and what he thinks is the future of emotion-enabled applications.
What’s the pitch for Emotion Hero? How does it work?
Emotion Hero is a game (for Android) that lets you train your facial expression as guided by emotion analysis software. You’ll compete in a worldwide competition to become the one and only “Emotion Hero”! The game gives you detailed feedback on how your expressions relate to the baseline expression for a given emotion. This allows you to improve your score, and keep on training.
After each level you will get detailed feedback on some of your expressions. Also, you can see your overall ranking compared with other players. If you make specific combinations of expressions you can unlock hidden achievements, which will allow you to progress between levels.
Where did you get the idea to build it?
In my artistic practice I examine the political implications of technologies and test their propositions. Over the last 1.5 years I have been researching emotion recognition software. For one, I find the complex technological procedures fascinating. What intrigues me most however is that the software is being used both as a tool for ‘objective’ measurements as well as a tool for training one’s facial expressions (along the lines of a “Quantified Self” application). In my practice I examine the paradoxical relation between those two applications of the technology. Wondering what it means to feel 23% surprised.
The video-game-artwork Emotion Hero was for me a literal translation of this premise and fascination: it challenges the player to consider how feelings and emotions are addressed by the software if we can train to alter the scores. Which leads one to wonder, what is it we are looking for in those numbers?
It is produced as part of a residency programme called Summer Sessions Network for Talent Development in a co-production of Arquivo 237 (Lisbon, PT) and V2_ Institute for the Unstable Media (Rotterdam, NL).
What was your process in building it?
The basic premise was: a game like Guitar Hero but then for the face. The SDK allowed me to quickly bootstrap a very rudimentary version. As a designer/artist I prefer the dynamics of working iteratively, so from there, I incrementally sketched and implemented various ways of interaction and feedback.
What role does emotion and emotion technology play in the concept of your app?
It is fundamental, as the project critically examines the abilities and premise of emotion recognition technology. It utilizes the software in order to question its use. Because of that, this project is another way for me to get a sense of what the software entails and also to have my work become part of its ecosystem.
How did you arrive at Affectiva’s technology to help achieve your vision?
My research started after seeing Affectiva’s promotional video at a talk at Transmediale (Berlin). Because the video’s atmosphere and design fascinated me I started to further examine the software. As my research started with Affectiva it made sense to try to use Affectiva’s Android SDK for Emotion Hero - which Affectiva kindly let me do. For me, the fact that this sort of technology is available on common day mobile devices was also key.
The SDK made it remarkable easy to ‘plug-in’ Affectiva’s emotion detection algorithms. As with many computer systems, it is an inevitable balance between understanding and ease of use: it allowed me to bootstrap fast.
Which features of Emotion Hero are you most excited about?
How the game, after you have completed a level, provides absurdly detailed feedback on your facial expression, such as: “raise your brow 37.2334% more.” This allows the player to focus on the mechanics of his/her facial expression. There is always another decimal to improve!
Do you have any other advice for those looking to build similar apps of their own?
Try it, but be critical towards what you use the software for. The first steps are relatively simple to do. What makes software development challenging and interesting is the choice of employing tools: as a media designer I am trained to inquiry the tools I use and play with their political implications. That is the harder, but also the most fun part to do. As an introductory read, I would recommend Joseph Weizenbaum’s Computer Power and Human Reason.
How can someone try Emotion Hero? Where can they get it?
It’s available for free, check out emotionhero.com.
About Ruben van de Ven
Combining his backgrounds in filmmaking, programming and media design, artist and researcher Ruben van de Ven (NL) is intrigued by the intersection of highly cognitive procedures and ambiguous experiences. He recently graduated from the Piet Zwart Institute in Rotterdam where he began his critical research into the digitization of emotions. He continues this research as part of the Human Index collective and currently is an artist-in-residence at the MuseumsQuartier in Vienna.
Read more about his Emotion Hero app at emotionhero.com.
