Next week, Affectiva will team up with our partners at Renovo to deliver a webcast around what goes into building the best possible automotive experience based on data collected from sophisticated in-cabin sensing systems. The webcast will focus the vision for the next generation of in cabin sensing, including capabilities today and where the automotive industry is going in building these experiences.
Affectiva has partnered with the automated mobility operating system company, Renovo, who works with a wide range of leading companies who are working to develop and deploy fleets of highly automated vehicles at scale. Their mission is to drive the expansion of Automated Mobility to the greatest scale and highest efficiency possible by deploying AWare - the first operating system built specifically for automated mobility - across highly automated vehicle fleets.
We spoke with Damien Scott, Chief Commercial Officer at Renovo about the work Renovo is doing, how they are working with Affectiva’s Automotive AI technology, and the future of in-cabin sensing.
What does Renovo do, and how does it work?
Our product is AWare, which is an OS for automated mobility. It’s a full vehicle abstraction that allows a range of applications to run on Highly Automated Vehicles (HAVs). It enables everything from automated driving systems through to rider management, fleet management, and really the whole ecosystem of software components that are necessary to bring up a true automated mobility service at scale.
AWare is analogous to AWS, or Microsoft windows and Android. It’s a consistent framework that a third party software developer and service providers can interface with in a consistent, reusable, safe, reliable manner.
Where did you get the idea to build it?
Renovo was founded in 2010, and the original focus of the company was building software platforms for electric vehicles. In 2010, the industry was at a turning point in the shift towards electrification, with growing recognition that this was going to accelerate. So we built a lot of critical pieces of software, lots of software and even full vehicles.
It was our work with Stanford University that prompted a much broader application of the technology we developed for EVs to abstract the full vehicle. The Stanford University program was developing high performance electric and automated systems for extreme environments. Through the work with Stanford we realized that the technology we had built could be extended more broadly. The partnership with Stanford was a key part of the development and extreme testing of the technology that today is the Renovo AWare OS.
What role does in-cabin sensing play in the concept of your product?
Ultimately, mobility is centered around people: making sure that they get to where they want to go, safely, cheaply and comfortably. So when you take an human driver out of the system, it is important to have the capability to understand what the state of the passenger is. That can be the physical state of the occupants - are they sitting down, for example? Have they put their seatbelts? Or at the end of a journey and are unresponsive - have they fallen asleep? What is the appropriate response to that?
These are all things that today in the mobility on demand service, the human driver will perform. If someone is having a medical issue, they can drive them into a hospital or perhaps even perform CPR. These are things that are important for automated mobility services to plan for and integrate relevant features into their final commercial service offering.
For customers that are currently using or looking to use Renovo’s AWare platform, compatible technology like Affectiva that can perform passenger monitoring is important because that is going to shape how the services are developed and operated at scale.
How do you use Affectiva?
Right now, we are looking at 3 different applications of Affectiva’s technology.
#1 - Monitor the state of the human safety driver.
The safety drivers are a very important part of the development of HAVs. They sit in a vehicle behind steering wheel and are there to make sure that if there’s an abnormal situation or something they don’t feel is appropriate to let the HAV continue, can take over control of vehicle.
Unfortunately, humans are distractible, fallible, get tired, suffer from rage and anger - so making sure that the safety driver is always engaged and is in the right emotional state to be performing that situation is important.
#2 - Understand the emotional state of the occupants of HAV.
Not that many people are comfortable in autonomous cars - recent surveys that have been done by AAA still show a high degree of skepticism from people who have had rides in autonomous vehicles. Understanding the level of trust of a person entering the vehicle for the first time, and how that trust evolves as the vehicle navigates its environment, is important to make the customer as comfortable as possible, as quickly as possible.
Imagine someone that is riding an autonomous vehicle for the first time, sensing their concern and heightened sense of fear (using Affectiva’s technology) allows the ADS to perhaps change its operating mode to be less aggressive, increase following distances, be very smooth on accelerations, take a conservative position to build trust with that passenger. Whereas, if you have someone in the vehicle that seems very confident, relaxed, and calm, the vehicle can operate at its optimum because the occupant has a high degree of confidence. Finally, these automated vehicles present very unique opportunities for in-cabin experiences. If someone is perceived by the system as looking stressed or worried, the lighting or music can be changed, or they can be prompted to fire up a meditation app - and many further examples that go beyond just the driving function. The emotional state of the passenger can be used to significantly improve that journey experience.
#3 - Understand emotional state of people OUTSIDE of the vehicle.
HAVs don’t operate in isolation. For many years to come they will be operating in a environment that includes pedestrians, cyclists, and other human driven vehicles on the road. Ensuring that the HAV understands how it is driving and the decisions it makes when a pedestrian steps into street are critical. How does a HAV signal to someone that is on the road that has seen them, that it knows what they are going to do and that is “friendly”, allowing them to cross? All these input signals are all really important things that the vehicle system needs to perform better in the context of its wider environment. Affectiva could help to determine the emotional state of those external human stakeholders allowing HAVs to make more optimal decisions.
What are your metrics for success, or company goals?
We want to support as many different vehicle platforms as possible, and to have the largest and best ecosystem of technologies and services that are compatible with the platform. That is a key initial and ongoing metric of success for Renovo.
What is also important for us is the number of Automated Mobility miles on the road that are delivered using services building on Renovo’s AWare platform. AWare provides the glue to bring together the disparate best-in-class technologies and services required to make automated mobility journeys enjoyable, convenient, and as cost-effective as possible.
What do you see as the future of in-cabin sensing technology?
The benefits of in-cabin sensing, in particular for HAVs, are very strong in terms of increasing trust, and providing tailored experiences to individuals. The focus of Renovo AWare is building platforms for high-utilization automated mobility systems - for ride-hailing vehicles that will be carrying very different people throughout the day. As one journey comes to an end and another one begins, there exists a stream of different passengers getting into the vehicle. Being able to respond to the emotional state and configuring things accordingly is going to be important to the rider experience.
I can see some potential barriers being the valid concerns about data privacy. Companies that have access to in-vehicle cameras, and the derivative data about passenger emotional state must be responsible and transparent as to what they might do with that information outside the original purpose of customizing the experience.
More About Renovo
The Renovo fleet of HAVs in the San Francisco Bay Area is expanding. This fleet is helping AWare ecosystem partners like Affectiva log on road miles and helping engage with a range of companies that are interested in integrating with AWaret. The AWare ecosystem includes a diverse array of companies with technologies and services that are necessary to deliver automated mobility at scale and transform the way people and things move. For more information, please visit renovo.auto
About Damien Scott
Damien Scott is the Chief Commercial Officer at Renovo.auto. He joined the company in 2016 and is responsible for partner relationships, go to market strategy and all other commercial and strategic development initiatives. He has previously worked with a range of firms in Silicon Valley at the forefront of transportation and energy technology including with Google X and grid-edge optimization startup Varentec. Prior to coming to the US, he worked for Williams F1 where he was integral to the establishment and operation of its Advanced Engineering division serving most recently as General Manager. Scott is also the co-founder of the Stanford/Harvard Forum on the Future of Transportation (SHFFT). He graduated with a B.Sc. in physics from the University of Sydney and MS and MBA degrees from Stanford University.
Missed our awesome webcast with Renovo about how we building the best possible automotive experience together? We have the recorded session available for you.
