• Advanced Driver Assistance Systems
  • Automated Driving
  • Detroit Auto Show
  • Driver Monitoring Systems

Vehicle Occupant Monitoring Is Accelerating in Importance

Sam Abuelsamid
Nov 18, 2022

Guidehouse Insights

I’ve been a strong proponent of driver and occupant monitoring systems in vehicles ever since I first drove a Cadillac CT6 with Super Cruise back in 2017. These monitoring systems are among several key safety technologies that could have a substantive impact on safety, and they are becoming more capable all the time. The work being done by companies such as Harman, Magna, Seeing Machines, Toyota, Vayyar, and Volvo could play a critical role in reducing traffic accidents and fatalities.

The Cadillac I drove 5 years ago featured an infrared camera mounted on the steering column that used technology from Seeing Machines. As the first automaker to offer true hands-free driving capabilities with its Super Cruise system, General Motors wanted to make sure that drivers were remaining properly engaged and watching the road. At the time Super Cruise launched in 2017, there had already been multiple crashes, including at least one fatality, among drivers using Tesla’s Autopilot, which lacks an active driver monitoring system (DMS) to determine where the driver is looking.

Super Cruise’s infrared camera allows the system to see the driver’s eyes in the dark or through dark sunglasses. Since GM adopted the system, several other automakers, including BMW, Ford, and Mercedes-Benz, have launched vehicles with similar DMSs.

Production DMSs have mostly been limited to just looking at eye gaze and head pose. However, research is showing that it’s possible to infer much more about the driver’s state from these sensors. At the September 2022 Detroit Auto Show, Harman provided me with an on-road demonstration of its Ready Care system, which utilizes the same basic hardware of an infrared camera looking at the driver but with much more sophisticated software.

In addition to determining whether the driver is watching the road, Ready Care looks for behavior indicative of looking down at a phone, at screens in the car, or at other vehicle occupants. It’s also possible to gauge whether the driver is drowsy or potentially impaired. Driving while too tired is something virtually everyone has done, and it can be just as dangerous as driving drunk. Providing more reliable alerts to stop and rest or even intervening by bringing the vehicle to a safe stop if the driver doesn’t respond could potentially avoid many crashes, injuries, and fatalities. This feature would also work if the driver suffers a medical emergency such as a heart attack or stroke.

Another advantage of using an infrared camera is the ability to detect subtle temperature changes across the driver’s face. Harman has developed algorithms to estimate the driver’s heart rate from these temperature pulses. While this technology won’t be as accurate as a dedicated heart rate monitor, it can be used to detect sudden changes that may indicate an increase in cognitive workload associated with an emergency or sudden change in road conditions. That could in turn be used to adjust thresholds for other features such as stability control or crash avoidance.

Simply ensuring that the driver’s eyes are on the road isn’t sufficient for hands-free driving. When using a hands-free system, a driver can suffer from cognitive distraction: staring straight ahead but not really being aware of what is going on. The upcoming Volvo EX90 will use two cameras to detect both visual and cognitive distraction.

Sensing what’s happening inside the cabin, whether with the driver or other occupants, is becoming as important as monitoring what’s on the road. As long as humans remain part of the mobility ecosystem as drivers or passengers, developing an understanding of their presence and state is likely to be important to ensuring their well-being.