Radar and video multimodal learning for human activity classification

de Jong, Richard J.; Uysal, Faruk; Heiligers, Matijs J.C.; de Wit, Jacco

doi:10.1109/RADAR41533.2019.171283

Radar and video multimodal learning for human activity classification

Title

Radar and video multimodal learning for human activity classification

Author

de Jong, Richard J. (Student TU Delft)
Uysal, Faruk (TU Delft Microwave Sensing, Signals & Systems)
Heiligers, Matijs J.C. (TNO)
de Wit, Jacco (TNO)

Date

2020

Abstract

Camera systems are widely used for surveillance in the security and defense domains. The main advantages of camera systems are their high resolution, their ease of use, and the fact that optical imagery is easy to interpret for human operators. However, particularly when considering application in the defense domain, cameras have some disadvantages. In poor lighting conditions, dust or smoke the image quality degrades and, additionally, cameras cannot provide range information. These issues may be alleviated by exploiting the strongpoints of radar. Radar performance is largely preserved during nighttime, in varying weather conditions and in dust and smoke. Furthermore, radar provides range information of detected objects. Since their qualities appear to be complementary, can radar and camera systems learn from each other? In the current study, the potential of radar/video multimodal learning is assessed for the classification of human activity.

Subject

human activity classification
micro-Doppler
multimodal learning
radar
video

To reference this document use:

http://resolver.tudelft.nl/uuid:738816cc-5439-4710-99d9-7ae593980e26

DOI

https://doi.org/10.1109/RADAR41533.2019.171283

Publisher

IEEE

ISBN

978-1-7281-3785-8

Source

2019 International Radar Conference (RADAR)