Affectiva Automotive AI is an In-Cabin Sensing solution that provides a deep understanding of what's happening in a vehicle. Using strategically placed cameras, Affectiva's In-Cabin Sensing software detects the state of the driver, the state of the occupants, and the state of the cabin. This insight helps car manufacturers enhance safety features that meet Euro NCAP requirements and enables them to deliver more comfortable and personalized transportation experiences.
In-Cabin Sensing: Data-Hungry, Deep Learning-Based Algorithms
Affectiva Automotive AI is built with deep learning algorithms that detect complex and nuanced human and cabin states. However, deep learning is data hungry. To train and test our algorithms we need a lot of video data that is representative of what happens in a car. This data includes different types of people doing certain activities, expressing certain states, under certain environmental conditions—such as varying lighting.
To power various In-Cabin Sensing use cases, car manufacturers are looking at placing in-car cameras in various locations. This means that our science team needs to model and train our In-Cabin Sensing algorithms to work with all these different camera angles. And that means that for our data collection we place cameras where our automotive partners need them to capture these different angles.
The type of automotive data sets we need are not readily available. We need to gather this data ourselves, and to do so, we run data collection studies. We recruit a lot of paid participants who come to our labs to either drive in our cars or sit in our simulator.
How We Collect Risky Driver Data
One key capability of our In-Cabin Sensing AI is the detection of dangerous driving behavior, such as drowsiness and distraction. In order for our software to analyze this, we need to collect real world examples of drivers getting drowsy and distracted. While we do find spontaneous occurences of these behaviors in some of our data collection studies, our deep learning algorithms need much more data than that, which poses a challenge.
We would never let people drive in impaired states purposefully: instead, we mimic these driving situations in our lab simulators. For those studies, people stay awake overnight in our lab, or we ask them to sleep-deprive themselves and then come into our lab, so we can collect their drowsy driving data in our simulator. The same goes for capturing distracted driving data: we would never have people text while driving, so we mimic that situation while “driving” a car that is safely parked.
Enter COVID: Evolving our Approach to Data Collection
With this pre-pandemic process of data collection, there were a lot of interactions between our data collection team and the general public. Often, our data collection process involved groups of study participants placed together in a vehicle. And, we had other close interactions such as participants waiting for their appointment in a waiting room, and the in-person signing of waivers.
As a result, we needed to pause dynamic vehicle collection with the onset of COVID for a few months. We used that time to develop new processes centered around safety protocols set forth by state required regulations, which we have always strictly adhered to. We were then able to safely resume our data collection efforts, for both participants and employees.
The Strategy for Safe Data Collection: 6 Steps
Here are the ways Affectiva is collecting data safely for both our team and participants during the pandemic. This protocol also allows us to quickly comply with detailed contact tracing with the local board of health in event of a positive case of COVID-19.
1 - All interactions between employees and participants must follow social distancing guidelines.
This entails keeping to a distance of 6+ feet between data collection specialists and all participants.
2 - Employees AND participants are required to wear masks at all times.
This is the standard unless aspects of the study require participants to remove their masks. They must do so in a way that is safe for both participants and employees.
3 - 100% automated and remote check-in procedures for no-touch interactions.
This starts with participant applications, which also captures their demographic information. Upon arrival to office, participants identify themselves, and our Data Specialists ask questions to verbally confirm the participant is not experiencing any symptoms related to Coronavirus or was exposed to the virus. If the participant answers yes to any of those questions, they will not be able to proceed and we reschedule them for a later date.
Our Data Specialists will instruct participants to use hand sanitizer and wait at designated locations before entering the lab or test vehicle. When a data lab or test vehicle has been set up and verified by a Data Specialist, participants will be instructed to enter. Data capture can be started remotely from a distance of 6 feet or more to comply with social distancing. When the study protocol has been completed, the participants exit the vehicle or lab and are instructed to use hand sanitizer again.
4 - Thorough vehicle and lab sterilization after each participant.
We go through a full sterilization checklist to disinfect each vehicle at the conclusion of each study, and our Specialists wear face masks, face shields, aprons, and disposable gloves at all times. Our Specialists use a CDC recommended solution which is sprayed on all surfaces of the vehicle, then wiped down. This includes the exterior vehicle surfaces like door handles, windows and door frame, as well as interior cabin surfaces like arm rests, steering wheel and controls, all heating and media controls, headrests, and more. Participants are also required to use hand sanitizer prior to entering the vehicle.
5 - Reduce the flow of participants to allow for enhanced sterilization protocols.
After spraying the cleaning solution, we have to let the liquid rest for a minimum of 1 minute before wiping any resting liquid from hard surfaces. The vehicles and lab simulators are then given a minimum of 20 minutes to fully ventilate before the next session will be allowed to begin. Specialists dispose of gloves after sterilization has been completed and immediately wash their hands. This entire sterilization process requires reduced volume of study participants to allow for appropriate cleaning and drying time between participants.
6 - Focus participant recruitment strategy to emphasize individual appointments.
To adhere to social distancing protocols, we focus on having just one individual participate in our data collection whenever possible. If we need to collect data from groups, we work to determine participant groups in advance, selecting groups that are comfortable being in close proximity to each other, such as families.
The Bottom Line
To train and test the deep learning algorithms of our automotive In-Cabin Sensing solution, we need to collect vast amounts of automotive data. With the onset of the global pandemic, the world changed for our data collection team. But we adapted and developed protocols to comply with local state regulations, and to keep our employees and participants safe. With this new, modified data collection process underway, we continue to make strides in developing In-Cabin Sensing technology that will improve road safety and the mobility experience for everyone.
The data collection team is always looking for people to help us in our data collection efforts, and will compensate you for your time. Ask us about booking a group (15 people or more) for the day in our mobile lab: when we book groups, we can either compensate each individual for their time or we can donate the total compensation of the group to a charity of their choice, making an Affectiva study a popular fundraising initiative! If you are based in the Boston area and are interested in participating, please contact us at datacollection[at]affectiva.com to learn more.
