Realistic simulated driving environment based on ‘crash-prone’ Michigan intersection: Real-time data collected at the two-lane roundabout will be used to efficiently test the safety of autonomous vehicles

The simulation is a machine-learning model that trained on data collected at a roundabout on the south side of Ann Arbor, recognized as one of the most crash-prone intersections in the state of Michigan and conveniently just a few miles from the offices of the research team.

Known as the Neural Naturalistic Driving Environment or NeuralNDE, it turned that data into a simulation of what drivers experience everyday. Virtual roadways like this are needed to ensure the safety of autonomous vehicle software before other cars, cyclists and pedestrians ever cross its path.

“The NeuralNDE reproduces the driving environment and, more importantly, realistically simulates these safety-critical situations so we can evaluate the safety performance of autonomous vehicles,” said Henry Liu, U-M professor of civil engineering and director of Mcity, a U-M-led public-private mobility research partnership.

Liu is also director of Center for Connected and Automated Transportation and corresponding author of the study in Nature Communications.

Safety critical events, which require a driver to make split-second decisions and take action, don’t happen that often. Drivers can go many hours between events that force them to slam on the brakes or swerve to avoid a collision, and each event has its own unique circumstances.

Together, these represent two bottlenecks in the effort to simulate our roadways, known as the “curse of rarity” and the “curse of dimensionality” respectively. The curse of dimensionality is caused by the complexity of the driving environment, which includes factors like pavement quality, the current weather conditions, and the different types of road users including pedestrians and bicyclists.

To model it all, the team tried to see it all. They installed sensor systems on light poles which continuously collect data at the State Street/Ellsworth Road roundabout.

“The reason that we chose that location is that roundabouts are a very challenging, urban driving scenario for autonomous vehicles. In a roundabout, drivers are required to spontaneously negotiate and cooperate with other drivers moving through the intersection. In addition, this particular roundabout experiences high traffic volume and is two lanes, which adds to its complexity,” said Xintao Yan, a Ph.D. student in civil and environmental engineering and first author of the study, who is advised by Liu.

The NeuralNDE serves as a key component of the CCAT Safe AI Framework for Trustworthy Edge Scenario Tests, or SAFE TEST, a system developed by Liu’s team that uses artificial intelligence to reduce the testing miles required to ensure the safety of autonomous vehicles by 99.99%. It essentially breaks the “curse of rarity,” introducing safety-critical incidents a thousand times more frequently than they occur in real driving. The NeuralNDE is also critical to a project designed to enable the Mcity Test Facility to be used for remote testing of AV software.

But unlike a fully virtual environment, these tests take place in mixed reality on closed test tracks such as the Mcity Test Facility and the American Center for Mobility in Ypsilanti, Michigan. In addition to the real conditions of the track, the autonomous vehicles also experience virtual drivers, cyclists and pedestrians behaving in both safe and dangerous ways. By testing these scenarios in a controlled environment, AV developers can fine-tune their systems to better handle all driving situations.

The NeuralNDE is not only beneficial for AV developers but also for researchers studying human driver behavior. The simulation can interpret data on how drivers respond to different scenarios, which can help develop more functional road infrastructure.

In 2021, the U-M Transportation Research Institute was awarded $9.95 million in funding by the U.S. Department of Transportation to expand the number of intersections equipped with these sensors to 21. This implementation will expand the capabilities of the NeuralNDE and provide real-time alerts to drivers with connected vehicles.

The research was funded by Mcity, CCAT and the U-M Transportation Research Institute. Founded in 1965, UMTRI is a global leader in multidisciplinary research and a partner of choice for industry leaders, foundations and government agencies to advance safe, equitable and efficient transportation and mobility. CCAT is a regional university transportation research center that was recently awarded a $15 million, five-year renewal by the USDOT.