Humans Remain a Threat to Automated Driving

The software that drives automated vehicles (AVs) has made remarkable strides in the decade since the engineers and researchers that participated in the DARPA Grand Challenge turned their attention to commercialization. However, as is often the case, predicting how human beings will interact with technology remains one of the biggest challenges to implementation.

Humans Are Unpredictable

In a recent interview with Axios, Chris Urmson, CEO of Aurora Innovation, said that the hardest part was “Understanding humans and predicting their ‘weird’ behavior.”

Urmson has been working on automated driving longer than almost anyone in the business. He was one of the first people to join the Carnegie Mellon University (CMU) team for the DARPA competition back in 2003, leading the team to the 2007 Urban Challenge victory. After that, he worked with CMU colleague Bryan Salesky on autonomous mining trucks for Caterpillar before leading the Google self-driving car project from 2009-2016.

Observation and Anticipation Are Key

The first stage of any driving task, human or automated, is to recognize everything around the vehicle and then anticipate action of all other road users. At that point, the human or virtual driver can decide what the vehicle should do. Thanks to ever more powerful computers, machine learning systems have become remarkably adept at recognizing what is in each frame of what the sensors see.

Static targets such as signs, traffic signals, and road features aren’t going anywhere. The physics of multi-thousand-pound vehicles mean that their potential pathways are also limited. But the pedestrian approaching the curb or the cyclist in the unprotected lane can change direction almost instantaneously.

What Do We Do to Reduce Pedestrian Fatalities?

In recent years, approximately 6,000 pedestrians annually have died on American roads. The 9% jump in pedestrian fatalities from 2015 to 2016 was the largest percentage increase among any group.

Computer scientist and investor Andrew Ng has suggested that humans must change their behavior in order to speed up the adoption of AVs. However, after more than a century of jaywalking prohibitions, that’s a naive idea. Instead, companies like Argo AI have hired specialists in developing prediction engines. Argo has several engineers that formerly worked on developing algorithms to enable cameras to track athletes in fast moving sports coverage.

Elsewhere, software startups like Perceptive Automata are focused on specific pieces of the overall AV puzzle such as prediction. Perceptive Automata employs behavioral scientists from MIT to develop neural networks that essentially read the body language of pedestrians to predict where they will go next. They are doing this by capturing video of pedestrians and showing short clips to groups of people to observe the body language and annotate those clips with their predictions. These annotated clips are then used to train the neural networks.

Obstacle to AV Adoption

There is no guarantee that any of these approaches will be entirely successful. Pedestrian fatalities may well turn out to be the biggest stumbling block to AV adoption. We may need to resort to physical barriers such as protected lanes for AVs, bikes, scooters, and even pedestrians to keep interaction and risk to a minimum.