Cornell Self-Driving Car Study Learns From Memories

Researchers at Cornell University have devised a new method for teaching self-driving vehicles how to drive on roads, through the creation of memories that assist in future navigation. 

Three research papers published by researchers at the Cornell Ann S. Bowers College of Computing and Information Science and the College of Engineering all deal with this concept of teaching a vehicle a specific zone or road, so when it approaches it again, it is more aware of what it will face. 

SEE ALSO: New Technology Brings Fully Autonomous Driving Closer

“The fundamental question is, can we learn from repeated traversals?” said senior author Kilian Weinberger, professor of computer science in Cornell Bowers CIS. “For example, a car may mistake a weirdly shaped tree for a pedestrian the first time its laser scanner perceives it from a distance, but once it is close enough, the object category will become clear. So the second time you drive past the very same tree, even in fog or snow, you would hope that the car has now learned to recognize it correctly.”

This is of critical importance to self-driving vehicles, which need to be able to operate in all weathers to reach the fifth level of autonomous driving, as put forward by the Society of Automotive Engineers. 

Almost all self-driving projects aim to reach level five in the future, but most are currently between level two and level three, in which a system can autonomously control the vehicle on defined routes. 

Cornell Researchers have built a dataset of more than 600,000 scenes, which are all taken from a car equipped with Lidar sensors traveling repeatedly in a 15-kilometer loop in Ithaca. The route was completed over 40 times over an 18 month period, giving them diverse weather conditions. 

“It deliberately exposes one of the key challenges in self-driving cars: poor weather conditions,” said Diaz-Ruiz, a co-author of the Ithaca365 paper. “If the street is covered by snow, humans can rely on memories, but without memories a neural network is heavily disadvantaged.”  

Cornell researchers presented one of the three papers at ICLR 2022, the International Conference on Learning Representations. The other two were presented at IEEE Conference 2022 on Computer Vision and Pattern Recognition.