MIT scientists have pioneered an innovative artificial intelligence system that empowers self-driving vehicles to navigate through previously unexplored territories using only basic maps and visual information, mimicking human cognitive reasoning.
Human operators excel at maneuvering through unfamiliar roadways by simply correlating visual observations with basic navigation tools. This intuitive process allows us to identify our location and determine our route with minimal effort. In contrast, autonomous vehicles face significant challenges with this fundamental reasoning capability. Each new environment requires extensive mapping and analysis of all roadways, a time-intensive process. Additionally, these systems depend on intricate maps—typically created through 3-D scanning—that demand substantial computational resources to generate and process in real-time.
Presented at the prestigious International Conference on Robotics and Automation, the MIT research team details an autonomous control mechanism that "learns" from human steering behaviors while navigating limited areas. The system exclusively utilizes data from video cameras and basic GPS-like mapping. Once trained, this system can guide a driverless vehicle along predetermined routes in completely new environments by emulating human driving patterns.
Mirroring human drivers, the technology also identifies discrepancies between its map and actual road features. This capability enables the system to detect potential errors in positioning, sensors, or mapping, allowing for real-time course corrections.
For initial training, researchers equipped a Toyota Prius with multiple cameras and a basic GPS navigation system, then collected data across various suburban roadways featuring diverse structures and obstacles. When deployed autonomously, the system successfully guided the vehicle along a preplanned route through a different forested area specifically designated for autonomous vehicle testing.
"Our technology eliminates the need for pre-training on every individual roadway," explains lead author Alexander Amini, an MIT graduate student. "You can simply upload a new map, and the vehicle can navigate roads it has never previously encountered."
"Our primary goal is developing autonomous navigation that remains robust when driving in novel environments," adds co-author Daniela Rus, director of the Computer Science and Artificial Intelligence Laboratory (CSAIL). "For instance, if we train an autonomous vehicle in an urban setting like Cambridge's streets, the system should seamlessly adapt to driving in forested areas, even without prior exposure to such environments."
The research team includes Rus, Amini, Guy Rosman from the Toyota Research Institute, and Sertac Karaman, an MIT associate professor of aeronautics and astronautics.
Advanced Point-to-Point Navigation
Conventional navigation systems process sensor data through multiple specialized modules designed for specific tasks like localization, mapping, object detection, motion planning, and steering control. For years, Rus's research group has been developing "end-to-end" navigation systems that process sensory input and directly generate steering commands without requiring specialized modules.
However, previous models were primarily designed to follow roadways safely without specific destinations in mind. In this groundbreaking paper, researchers have enhanced their end-to-end system to navigate between specific points in previously unseen environments. To accomplish this, the team trained their system to predict a complete probability distribution across all possible steering commands at any given moment during operation.
The system employs a convolutional neural network (CNN), a machine learning model commonly used for image recognition. During training, the system observes and learns steering patterns from human drivers. The CNN establishes correlations between steering wheel movements and road curvatures detected through cameras and inputted maps. Eventually, it learns the most probable steering commands for various driving scenarios, including straight roads, four-way or T-shaped intersections, forks, and rotaries.
"Initially, when approaching a T-shaped intersection, numerous directional options exist," Rus explains. "The model considers all possible directions, but as it processes more data about human behavior, it learns that some drivers turn left while others turn right, but none proceed straight. The system eliminates straight ahead as a viable option and learns that at T-shaped intersections, only left or right turns are possible."
Map Integration and Verification
During testing, researchers provided the system with a map containing a randomly selected route. While operating, the system extracts visual features from camera feeds, enabling it to predict road structures. For example, it identifies distant stop signs or road line breaks as indicators of approaching intersections. At each moment, it utilizes its predicted probability distribution of steering commands to select the most appropriate option to follow its designated route.
Critically, the researchers emphasize that their system utilizes maps that are significantly easier to store and process. Traditional autonomous control systems typically employ LIDAR scans to create massive, complex maps requiring approximately 4,000 gigabytes (4 terabytes) of storage for just San Francisco. For each new destination, vehicles must generate new maps, resulting in enormous data processing requirements. In contrast, the maps used by the researchers' system can represent the entire world using merely 40 gigabytes of data.
During autonomous operation, the system continuously compares visual data with map information, noting any discrepancies. This process enhances the vehicle's ability to determine its precise position on the roadway. It also ensures the vehicle maintains the safest path when receiving contradictory input information: For instance, if the car is traveling on a straight road with no turns indicated, but the GPS signals a right turn is necessary, the vehicle will recognize the inconsistency and continue straight or stop as appropriate.
"In real-world conditions, sensors can fail," Amini notes. "We've designed our system to remain robust against various sensor failures by creating a technology that can process these imperfect inputs while still maintaining accurate navigation and localization on the roadway."
