By: Ashley McManus, Global Marketing Manager; featuring Cato Solutions CEO Glenn Longacre and COO David Buder

There has been a fundamental shift in the way we communicate with each other in the last few years. Bland text messages and emails are soon becoming a thing of the past, as colorful communication apps are emerging to add a rich layer of interaction that often gets lost behind words on a screen. What if that next app you were using to talk to your mother, brother, or best friend was emotion-enabled, so that the conversation was able to fully communicate how you were feeling to the other person?

Creative app and media development company Cato Solutions is getting close. Their app - Smirkee - is a video chat app that ID's user emotions in real time with help from our emotion recognition SDK. It records the video call to your phone showing your emotions as you speak, so you can watch the emotions fly in real time on the screen, take selfies of you and others with emotions showing, and offers the ability to post to social networks. It was a 2016 Appsters Awards Finalist, and is available for download on the iTunes store: Android version launching soon.

We interviewed Cato Solution’s COO David Buder, and CEO Glenn Longacre on how the app works, the process of developing it, and what they think is the future of emotion-enabled applications.

1) What’s the pitch for Smirkee?

Smirkee is an easy-to-use, powerful video chat application that solves the mystery of someone's true emotions when talking with them. It is a fun video chat app with real time emotional analysis, where you can find out how emotionally engaged your partner is during your chat, in real time on smartphone or ipad. It’s also recordable on device, and you can go back and review your call for emotion. It adds another level of communication, where in a mobile environment you are able to get instant feedback on an emotional level instead of going by what you just see or hear.

2) What does Smirkee do?

Smirkee is a cross-platform video chat app with “real time” emotion recognition feedback with it's unique user interface. It can also be used to take “emotional selfies” of yourself to test your own facial expressions as they would appear to others. You can grab these micro-expressions to post to Facebook, Twitter, etc. Smirkee maps the 43 facial muscles 24 times per second of the person you are video calling.

These expressions are processed securely on your phone using the Affdex API to recognize both “micro-expressions” and an overall emotional mood during the conversation. Eleven different emotional categories are rated and a detailed graph lets you review the conversation at any time in your log. Examples of what we display include average overall mood and predominant emotion on the call - whether they are depressed or engaged.

As another unique function, you can actually record your video chat for later review with the emotional data layered over the video with the ability to fast forward, pause, etc. to see what you were talking about at any point along the recorded clip. As well, you can see the dominant emotion at that point by tapping the Smirk button.

3) Where did you get the idea to build it? 

Smirkee is social app, but it is part of a greater vision. Our backgrounds are with market research, where we worked with major brands - some in the toy and automotive industries. We found it difficult to test people’s responses to stimuli outside of a controlled environment, as testing platforms lacked mobility. Emotion recognition plays a strong role in market research, which is where we were first exposed to it. We wanted to take people out of the controlled environment of market tests, and get into a normal, comfortable environment via a cellphone situation. So we started a new company, Cato Solutions, with mobility in mind - and from there, we got the idea to build an app.

4) What was your process in building it?

First we had to envision what the best interface for this app would look like: something that was easy to use, attractive, and engaging to look at. The original concept of Smirkee was to display a bar graph of emotion, but we then came up with a (patent-pending) button concept. This is unique right now because it doesn’t involve a lot of space on the screen, it is more reactive, and visually more interesting. Smirkee’s current platform has color-coded buttons that virtually glow and grow bigger or smaller, a graph showing each color-coded emotion, and what’s going on during the call depending on the faces involved. Once we got that idea in place, we hired high level staff of coders to make reality.

It initially was much more difficult than anticipated. In fact, only after Affectiva SDK was released, we were able to achieve the reliability and results that were necessary for our app’s success. It actually worked too well: our CTO said that the speed of emotions are firing too fast, and the app code couldn’t handle it. So while we were very pleased with speed, we actually had to buffer the images to slow it down.

In terms of adoption, we wanted to create a friendly vibe with name and font style, so as to soften the stigma of this new tech as people start to understand it.

5) Which features of Smirkee are you most excited about?

The overall interest rates is exciting, as well as the caption of “you’ve been smirked!” We like the face that you can use the front or back camera, a feature that came about after someone complained about how he had to keep turning camera around to make that work. The video logs that are available after a call allow you to see reactions of big moments during playback, where you can get answers to hypothetical questions, like are you really paying attention? There are some interesting emotional grabs to be had here, and people seem to love it so far.

6) What is the next step for your app?

We are looking at other applications for this technology - a lot of other apps (like video deposition analysis, or integrated into dating apps) can apply the Smirkee engine to their functionality. For Smirkee itself, we are looking into adding new metrics to the interface - like adding positive or negative valence metric to overall mood of the call, which would be huge for research. We also want to add a similar engagement / attention average number, so users can see how attentive the other person was. All these numbers will be added to existing app numbers and can be used as a tool for diagnostics - another direction of interest for Smirkee.

We see quite a few emotional wellness applications for our tech in medical analysis for conditions like autism, asperger's and dementia - as well as early testing for alzheimer's. We are also playing around with another version of the app that will function in the fraud protection and banking space (B2B).

7) Do you have any other advice for those looking to build similar apps of their own?

Do your homework with platforms. Select your integration tools wisely, as it is not easy to integrate all tools easily. These are some of the pitfalls of putting an app together - it usually takes triple the time you think it will. But stick with it: it never goes the way you think it will.

About Smirkee

Smirkee is the first emotions chat app that reads and records facial expressions in real time. Smirkee recognizes 11 different emotions that are active and changing with your facial expressions as you speak.  So when you review the video recording, it displays the dominant mood from the video chat call. Also, take all the selfies you want and practice your own facial expressions. Or snap a picture of your friends at a party and then watch the emotions fly. Users can get Smirkee emotion chat app on the iTunes store, with Android version being launched in near future.