Semi-Supervised Federated Learning for LiDAR-based 3D Object Detection in Autonomous Driving

Abstract

3D object detection is a fundamental perception task for autonomous driving. However, the generalization of 3D object detectors remains constrained by the limited geographical and environmental diversity of available annotated datasets. Although sensor data passively recorded by privately operated vehicles offer a promising source of diverse training data, the absence of object labels, together with the distributed and privacy-sensitive nature of such data, makes it difficult to use for training. Semi-supervised federated learning (SSFL) provides a potential solution by enabling clients to collaboratively train a shared global model on their unlabeled data without requiring raw data to be shared, while leveraging the limited labeled data available at the server to guide training. Nevertheless, SSFL remains unexplored for LiDAR-based 3D object detection. To address this gap, we implement labels-at-server SSFL for LiDAR-based 3D object detection by alternating client-side self-training with server-side supervised fine-tuning. We further investigate the impact of data heterogeneity arising naturally in autonomous-driving settings using controlled partitions of the nuScenes dataset that isolate location- and weather-based differences, alongside a more realistic log-based partition. The results show that the SSFL implementation successfully exploits unlabeled client data to improve global detection performance. Moreover, despite increased variation in pseudo-label quality and greater divergence between client updates under the non-IID partitions, global detection performance remains comparable to that of the corresponding IID configurations. Performance also remains stable under partial client participation.