Radars for Autonomous Driving: A Review of Deep Learning Methods and Challenges

1Zoox, Inc.   2Brookhaven National Laboratory 
Teaser Image
Autonomous radars can measure agent velocity, detect occluded or distant agents, and resist adverse weather. However, their sparse low-resolution, cluttered, and highly uncertain data makes object detection a significant challenge.

Abstract

Radar is a key component of the suite of perception sensors used for safe and reliable navigation of autonomous vehicles. Its unique capabilities include high-resolution velocity imaging, detection of agents in occlusion and over long ranges, and robust performance in adverse weather conditions. However, the usage of radar data presents some challenges: it is characterized by low resolution, sparsity, clutter, high uncertainty, and lack of good datasets. These challenges have limited radar deep learning research. As a result, current radar models are often influenced by lidar and vision models, which are focused on optical features that are relatively weak in radar data, thus resulting in under-utilization of radar’s capabilities and diminishing its contribution to autonomous perception. This review seeks to encourage further deep learning research on autonomous radar data by 1) identifying key research themes, and 2) offering a comprehensive overview of current opportunities and challenges in the field. Topics covered include early and late fusion, occupancy flow estimation, uncertainty modeling, and multipath detection. The paper also discusses radar fundamentals and data representation, presents a curated list of recent radar datasets, and reviews state-of-the-art lidar and vision models relevant for radar research.


Radar Data Formats

Data Formats

Among several common data formats, the radar point cloud (c) is the most popular format due to its compact information-rich data representation.


Important Radar Datasets

Datasets

We provide a curated list of recent, high-quality radar datasets, focusing particularly on those that contain data from new-generation 4D radars.


Radar's Role in Perception

Perception

Radar models have struggled to make a difference to perception in traditional late fusion detection-based tracking approaches (a). Newer paradigms, such as early sensor fusion (b) and occupancy estimation (c), promise to significantly enhance radar’s contribution to perception.


Radar Deep Learning Models

Radar Deep Learning Models

We provide a modular detector framework for radar deep learning models, where components from state-of-the-art models can be selectively incorporated at relevant stages with only minor modifications.


Early Sensor Fusion with Radar

Early Sensor Fusion with Radar

There are two popular approaches to camera-radar early fusion: (a) fusion in the perspective view, which improves depth estimation of camera detections and adds object velocity to those detections; and (b) fusion the bird’s-eye view (BEV), which infers joint detections from the BEV features of camera and radar.


Handling Uncertainty in Data

Uncertainty in Radars

The high uncertainty in radar data results in less reliable detections by learned radar models. We discuss sources of uncertainty in data and deep learning techniques to improve robustness of detections.

Reducing Noise

Noise in Radar Data

Radar data contains significant amount of clutter, leading to false positive detections. We identify three main sources of clutter and methods to remove such clutter.