By: Ashley McManus, Global Marketing Manager; featuring Erin Smith, Founder of Faceprint

More than 10 million people worldwide are currently living with Parkinson’s Disease - 1 million of those people in the US alone. More than those diagnosed with multiple sclerosis, muscular dystrophy and Lou Gehrig's disease combined, the number of Parkinson’s Disease cases is expected to more than double by the year 2030.

There are currently no definitive means of diagnosis for Parkinson’s disease (PD). Existing methods have severe limitations, such as relying on in-clinic medical examination and the presence of symptoms that do not occur until late stages of the disease. Further, current diagnostic methods are often subjective, and lack of quantified biomarkers and tools hinders the process. So that means even out of the 60,000 Americans diagnosed with Parkinson's disease every year, this number still does not reflect the thousands of cases that go undetected. (Source: Parkinson’s Disease Foundation)

But what if watching a simple computer program at your next physical was able to rule out Parkinson’s Disease, or identify it early enough so you can begin treatment?

Our emotion-sensing and analytics technology is transforming the healthcare industry, giving visionaries the assets they need to create more innovative diagnostic tools with patient emotions. High school student Erin Smith developed FacePrint using Affectiva’s SDK, around this concept. Faceprint is a novel diagnostic tool that seeks to minimize the societal and economic implications of rampant global PD growth and the lack of diagnostic methods. Erin founded the patent-pending telemedicine system for early and accurate diagnosis of Parkinson's disease, and her technology projects have won numerous awards and been featured in Seventeen Magazine, Forbes, Fortune, and TechCrunch. You can read more about Erin and her work on this local Fox News article, and see her full Faceprint pitch here.

We interviewed Erin, on how her project works, the process of developing it, and what she thinks is the future of emotion-enabled healthcare applications.

What’s the name of & pitch for your project?

FacePrint’s process is simple and remote. A user first watches a series of Super Bowl commercials while his/her face is recorded by a computer webcam. Then, the user is asked to replicate three universal emoticons. The collected spontaneous and posed facial response videos are then analyzed using Affectiva AFFDEX, which enables the facial footage to be turned into millisecond-by-millisecond calculations. Finally, FacePrint feeds a user’s responses into the diagnostic algorithms I created that utilizes quantified and validated differences I discovered in spontaneous and posed facial expressions in PD. The result is whether or not a patient has PD.

Where did you get the idea to build it?

About little over a year ago, I was watching a video by the Michael J. Fox Foundation when I noticed that whenever Michael J. Fox or another PD patient would laugh or smile it came off as emotionally distant or “unfelt”. I then wondered if I could quantify these differences and use them to build a diagnostic algorithm. I then talked to local caretakers and clinicians who reported similar observations in their loved ones, even years before diagnosis. Further, as I read through past research, I found that PD patients experience a masked face 5-10 years before the onset of hallmark motor symptoms and subsequent diagnosis, which up until this point has merely been a subjective observation. Further, I found that PD patients experience changes in the amygdala and basal ganglia sections of the brain during early stages of the disease, which are the same sections responsible for spontaneous and posed facial expression formation. These findings acted as the scientific foundation for my idea. Even more, when I initially began data collection, I would spend my free time going around to different PD support groups and retirement villages in my community. These interactions have refined my perspective and it is the human element and continued relationships that have perpetuated my motivation going forward. (Sources: Parkinson’s disease: clinical features and diagnosis, Clinical correlates of selective pathology in the amygdala of patients with Parkinson's disease, Biomarkers of Parkinson's disease: Present and future)

What was your process in building it?

Over the past summer, I was able to get in contact and launch a study with the Michael J. Fox Foundation Trial Finder. I then designed a method to elicit spontaneous and posed facial expressions using Super Bowl commercials and universal emoticons that was consolidated onto a website to enable remote data collection. From the study with the MJF Foundation Trial Finder, I collected a total of 1,060 facial response videos from participants comprised of PD and healthy control subjects. I then broke down the facial footage into millisecond-by-millisecond calculations for different Action Units and specific facial expressions with Affectiva’s AFFDEX facial recognition software. I then wrote software to clean and standardize the raw data. Next, I wrote software to determine if there was in fact a difference in spontaneous and posed facial muscle movements, and then to mine through the data to determine specific moments with significant differences. After I had discovered and validated a series of previously unknown differences, I wrote software to build a series of predictive models and determine the optimal parameters for construction. After several failed attempts and long nights, I finally created a diagnostic algorithm with the specifications that yielded high accuracy. This was such an exciting moment in my research, as it proved that I was not only able to quantify and discover previously unknown PD biomarkers, but I was also able to use the validated differences to accurately predict if a patient the algorithm had never seen before had PD. On a more personal level, I had never written code before this project, so it was really amazing to see that I was able to use the skills I had learned in order to create an impact for the patients I had come to love.

What role does emotion and emotion technology play in the concept of your project?

Affectiva’s emotion technology is the backbone of FacePrint. I am an avid fan of Affectiva, and their technology never ceases to fascinate me. Through FacePrint, I hope to continue to expand Affectiva’s applications to the medical community. FacePrint leverages the idea that emotion acts as external manifestations that provide insights into internal neurological pathology of diseases such as PD.

Shortly after I watched the video by the Michael J. Fox Foundation and came up with the idea for FacePrint, I stumbled across Rana el Kaliouby’s Ted Talk where she explored the application and intersection of emotions and technology. I was instantly fascinated. I then sent her an email with a short synopsis of my idea, not really expecting a reply. Then, within that day, I heard back from her and she was excited and really encouraging about my project. I was subsequently able to use Affectiva’s AFFDEX to break down the facial footage I collected. Affectiva’s AFFDEX seamlessly turns my raw video footage into usable bits of information, which has expanded the applications of FacePrint and has been fundamental in having the data necessary to create the diagnostic algorithm. The Affectiva team have been an incredible help every step of the way. Further, I was able to meet Rana el Kaliouby at the San Francisco Tech Crunch Disrupt conference this past September, which was an amazing experience.

Which features of your project are you most excited about?

I am most excited that FacePrint leverages the idea that emotion acts as external manifestations that provide insights into internal neurological pathology of diseases such as PD.

What is the next step for your project?

I am currently continuing the development of FacePrint. Right now, I am refining my current diagnostic algorithm and building other predictive models. Further, I am comparing accuracy across models in order to best determine whether or not a patient has PD utilizing their spontaneous and posed facial expressions and the specific biomarkers I discovered. I am also launching longitudinal studies which will further the application of this technology as a PD monitoring device. Even more, I am working on expanding data collection to other neurological disorders. Specifically, I am first focusing on parkinsonism disorders that are misdiagnosed as PD and then will move onto other neurological diseases. I hope that FacePrint will ultimately be used as a differencing diagnostic tool. My goal is to have the diagnostic and monitoring tool readily used by the public and clinicians before I graduate high school, which is about a year away.

Are there any plans to build another project like this in the future?

Absolutely! I am currently building a wearable device with a few of my friends that would provide bilateral stimulation to alleviate symptoms associated with PTSD and be connected to an app with additional tools and a social component. Further, I recently got into the Research Summer Institute by the Center for Excellence in Education. As part of this program, I will spend my summer at MIT conducting research on a completely new project that will center around the intersection of neuroscience, psychology, and machine learning. I am really excited for the opportunity to continue building other projects in the future.

Do you have any other advice for those looking to build similar project of their own?

From a pragmatic standpoint, I would heavily advise students, specifically in high school, to not overload their schedule, and instead leverage their free time for creative exploration. This time will ultimately enable you to start projects catered around the intersection of your own unique interests. Cal Newport is one of my favorite researchers, and he writes extensively about this topic. Start by reading all of his books. Trust me. Also, expose yourself to random goodness. Find people you think are awe-inspiring and start Googling them. Also, don’t limit yourself to people in one field; the best innovation comes from the intersection of several interests. Read all you can about their ideas and projects; it will get your creative juices flowing. Follow in their footsteps until you ultimately find your own unique fingerprint and pave your own path. Also, reach out via email (if you need a subject line I suggest “Quick question from a curious high school student”; I don’t know why, but in my own experience, this subject line has increased the probability of getting a response by 90%). Ask them a question. People are amazing and so willing to help if you merely ask.

What do you see as the future of healthcare technology?

I believe better and more creative tools are the future of healthcare technology. For example, we have a deep understanding of emotions from a philosophical standpoint, but from a scientific standpoint we have barely scratched the surface. Similar to how the invention of the telescope helped us better understand the complexities of the universe years ago, the creation of better tools, such as Affectiva’s technology, will help reveal the mysteries of emotions. We simply cannot progress in understanding unless we create and then use better tools and technology. I believe there are two main hurdles that must be addressed before these positive changes can be evident within the healthcare sector. First, we must be willing to adopt a new model of medicine. We must adopt a model that focuses on the intersection of internal and external manifestations of symptoms, and better understands and treats their complementary relationship. Better tools, specifically centered around emotions, will aid in this shift. The second obstacle that I foresee goes hand in hand with the first. We must transform our mindset. As I have talked with several clinicians, I have been struck by the vast divide that occurs in their attitude and outlook of the future of healthcare technologies. Blatantly put, there are far too many clinicians who are stuck in a rut of thinking. However, I have hope. There is a much larger sphere of clinicians that I have interacted with who are ready to shake the dust. These are the innovators. These are the ones creating better, more creative tools. They are the ones helping us better see and understand the complex microcosms of this world. These clinicians and our own support will be the backbone of the new face of healthcare technology.

How can someone try your project? Where can they get it?

I am still refining my diagnostic algorithms. Consequently, the full test and outcome of PD or non-PD is not available to the public. However, the data collection website that mimics the process of the final product is available. Feel free to participate in the study; the data collected is invaluable. I will also be collecting more data within the next few months as I expand the technology to other neurological diseases, so when that occurs, I will also be able to provide that link for more data collection.

About Erin Smith

Erin is a perpetually curious seventeen-year-old. She has been conducting science and engineering research for the past seven years, competing at a national and international level. She is also an avid speaker who has competed at Nationals for the past two years, but her favorite stage is local education and STEM events. Erin is also the founder of KC Steminists, a program for middle and high school students that focuses on greater exposure to opportunities in computer science, with a focus on entrepreneurship. In her free time, Erin loves to read, travel, participate in hackathons, work with local Kansas City startups, and spend time with friends and family.