Final published version
Research output: Contribution to Journal/Magazine › Journal article › peer-review
Research output: Contribution to Journal/Magazine › Journal article › peer-review
}
TY - JOUR
T1 - Deep learning based object detection from multi-modal sensors
T2 - an overview
AU - Liu, Ye
AU - Meng, Shiyang
AU - Wang, Hongzhang
AU - Liu, Jun
N1 - Publisher Copyright: © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023.
PY - 2024/2/29
Y1 - 2024/2/29
N2 - Object detection is an important problem and has a wide range of applications. In recent years, deep learning based object detection with conventional RGB cameras has made great progress. At the same time, people are more and more aware of the limitations of RGB cameras. The progress of algorithms alone can not fundamentally resolve the challenges of object detection. Unmanned vehicles or mobile robot platforms are often equipped with a variety of sensors in addition to RGB camera, each of which have its own characteristics, and can expand the sensing range of RGB camera from different dimensions. For example, infrared thermal imaging camera and multispectral camera broaden sensing range from spectral dimension, while LiDARs and depth cameras are able to broaden sensing range from the spatial dimension. This paper mainly summarizes the deep learning based object detection methods under the condition of multi-modal sensors, and surveys and categorizes the methods from the perspective of data fusion manner. The datasets of different modality are summarized, and the advantages and disadvantages with different combination of sensors are also discussed in this paper.
AB - Object detection is an important problem and has a wide range of applications. In recent years, deep learning based object detection with conventional RGB cameras has made great progress. At the same time, people are more and more aware of the limitations of RGB cameras. The progress of algorithms alone can not fundamentally resolve the challenges of object detection. Unmanned vehicles or mobile robot platforms are often equipped with a variety of sensors in addition to RGB camera, each of which have its own characteristics, and can expand the sensing range of RGB camera from different dimensions. For example, infrared thermal imaging camera and multispectral camera broaden sensing range from spectral dimension, while LiDARs and depth cameras are able to broaden sensing range from the spatial dimension. This paper mainly summarizes the deep learning based object detection methods under the condition of multi-modal sensors, and surveys and categorizes the methods from the perspective of data fusion manner. The datasets of different modality are summarized, and the advantages and disadvantages with different combination of sensors are also discussed in this paper.
KW - Deep learning
KW - Multi-modal
KW - Object detection
KW - Sensor fusion
U2 - 10.1007/s11042-023-16275-z
DO - 10.1007/s11042-023-16275-z
M3 - Journal article
AN - SCOPUS:85165938559
VL - 83
SP - 19841
EP - 19870
JO - Multimedia Tools and Applications
JF - Multimedia Tools and Applications
SN - 1380-7501
IS - 7
ER -