Mental health research takes many forms, but what unites it can be the application of biosensors and psychophysiological methodologies to better understand and ameliorate patient experiences and outcomes.

Smart Eye AB, the global leader in Human Insight AI with its recent acquisition of Emotion AI pioneer Affectiva, and iMotions, the leading biosensor software platform, have joined forces to create a true, integrated powerhouse in delivering unparalleled insights into human behavior.

Recently Smart Eye Deputy CEO Dr. Rana El Kaliouby (formerly Co-Founder and CEO of Affectiva) had a discussion with iMotions Product Specialist Pernille Bülow, PhD. on Emotion AI in mental health applications, the role of biosensors in mental health research and where this technology is heading in the future. 

Here are 5 takeaways from their discussion:

#1 Technology Applications to Improve Mental Health: Suicide Prevention App

From predicting what songs will be the most popular to developing treatments for terrible diseases, technology can be applied to both cool and important use cases. For the last decade, Affectiva and iMotions have had a partnership to bring Emotion AI to iMotions customers. 

One case study involves Professor Steven Vannoy, a mental health services researcher who uses iMotions platform to understand facial biomarkers and identify suicidal intent. Using Affectiva’s Emotion AI, he figured that with the attachment we have to our phones, perhaps our phones could flag an emotional crisis and then provide the necessary resources to prevent suicide. His project routinely analyzes the user’s emotions through periodic “check-ins” in which the user describes what they are doing, who they are with and how they are feeling about the future. Although experimental, the ultimate goal would allow the user’s mental health care provider to easily monitor the patient’s mental health status and be alerted to changes that indicate a move towards crisis.

The gold standard of mental health research is still a survey—for example, at your doctor’s office, they ask you via a survey to rank how depressed you are. We have an opportunity to make this much more sophisticated by bringing data into the equation. 

#2 Tooling and Data Collection: How Data is Collected via Biosensors and the Value it Brings to Understanding Mental Health

In terms of what is being done today with biosensors in mental health research, the vision is that each signal provides one clue to the holistic picture of someone’s mental health: one single sensor is never enough to give a solid prediction of how a person’s mental state will change. 

This is where the case for multi-modal data collection comes into play. In order to build systems that can address and improve mental health, we need data captured from the screens people use, voice analysis, EEG, facial expression analysis, Galvanic Skin Response (GSR), and even accelerometers to indicate movement–just to name a few. There’s also the discussion of collecting this data naturally in someone’s everyday life - aka “in the field,” or more meticulously collected in a lab setting. Less invasive biosensors capture more natural activity, as well as subtle, perhaps unconscious and unfiltered reactions or responses. 

Lastly, there’s the value in longitudinal data–that is, the sensing of a particular individual over time. Establishing this kind of baseline for a person can help technology understand when someone deviates from it, and perhaps flag it for attention. With this type of multi-modal data capture, imagine if your phone knew when your heart rate was too high and prompted you to do some breathing exercises, or prompted you to schedule a session with your therapist? 

Another powerful piece to add into the mix is to pair contextual information with data provided by biosensors. Similar to a survey, what if users could self-identify when they are having a panic attack? What if your phone mic was capturing a level of ambient noise that corresponded to an increase in your stress levels? Categorizing data with physiological differences has great potential in improving individual mental health outcomes. 

#3 - Crossing Over from Academia into Industry: The Power of Mobile for Mental Health Research in the Field

Mental health data can be collected on mobile phones. Researchers that use iMotions that need to be in the field are able to have participants download an app on their phones, export that data to a cloud that a researcher has access to. This provides great flexibility in collecting more naturalistic data when a lab setting is not ideal.

The use of mobile–and ultimately, wearables–really opens up what is possible in terms of mental health research. The suicide detection and prevention app is one example: starting with questions like “what does it mean to look suicidal? To look depressed?” then backtracking to developing predictive analytics to monitor how it changes over time and then identify when the time is to intervene. 

Imagine the potential this technology has for hospitals and consumers; collaborations between academia and commercial organizations are critical for upscaling accessibility to this technology, while also improving each of the many algorithms that are critical for accurate detection and prediction. Pairing this academic knowledge with the creativity and courage of startups can help break the barriers and make an academic achievement become a life changing one.

#4 The Need for Improving Mental Health Outcomes is a Global Issue

All cultures suffer from mental illness in some sense, so this is a relevant issue for everyone. However, people sometimes express themselves very differently depending on the culture they are in/from. A facial expression indicative of depression in one culture, may not be indicative in another. Some sensors are also more resistant to cultural differences than others. To address this, there are a variety of biosensors available to accommodate whatever research makes most sense in a particular country. For example, one culture may respond better to an app, another more interested in wearables. And yes, you can do an EEG to get some valuable data, but is that system realistic for every day screening and monitoring?

This brings us to another important aspect, that is the diversity of data that is collected and how these systems are built. For example, Affectiva spent years on underlying facial analysis models to ensure they were not biased in any way: they account for different ages, ethnicities, facial hair and even hijabs. Where technology gets more sophisticated and deep learning is involved, the diversity of data becomes very important. 

So whether you are on the research or commercial side, you must make sure your algorithm does not have any bias for both the training and validation. This bias can be uncovered by ensuring you collect data from a large, culturally representative group of people as possible. Only then can algorithms take this volume of unique data and identify trends, especially longitudinally. 

#5 - Opt-In and Consent is an Important Consideration, Regardless of Solution

For the everyday consumer whose data is being collected on a daily basis for any number of things, transparency in how their mental health data is being used is of particular interest. 

In the case of research clients, all iMotions customers use subjects that have opted into these studies. They explicitly ask if participants are comfortable doing this, the sensors they are using, how the data will be used. Ultimately, whatever is recorded is stored locally on a computer that is only accessed by the researcher for data analysis and will be deleted after.

The Bottom Line: Get Started!

While great strides have been made in biosensor development, there’s no need to wait for the ultimate “super-sensor” that will give you all the data you need for your mental health research project. Sometimes, it just takes a first step. If we can leverage the optimism and excitement of people looking to improve their mental health as a daily part of their lives, we can put the strategies and technologies in place to get there. Let’s quantify what is hidden or not overtly visible into data that can be of real help to us individually. If this is an area of interest to you, please reach out. 

To learn more about how iMotions clients are leveraging biosensors in their mental health research, register for their upcoming webinar here (or download the recording). For additional content, subscribe to our podcast to listen to more customer interviews doing cool work applying our technology into the mental health space.