Abstract
Developing 3D semantic occupancy prediction models often relies on dense 3D annotations for supervised learning, a process that is both labor and resource-intensive, underscoring the need for label-efficient or even label-free approaches. To address this, we introduce MinkOcc, a multi-modal 3D semantic occupancy prediction framework for cameras and LiDARs that proposes a two-step semi-supervised training procedure. Here, a small dataset of explicitly 3D annotations warm-starts the training process; then, the supervision is continued by simpler-to-annotate accumulated LiDAR sweeps and images – semantically labelled through vision foundational models. MinkOcc effectively utilizes these sensor-rich supervisory cues and reduces reliance on manual labeling by 90% while maintaining competitive accuracy. In addition, the proposed model incorporates information from LiDAR and camera data through early fusion and leverages sparse convolution networks for real-time prediction. With its efficiency in both supervision and computation, we aim to extend MinkOcc beyond curated datasets, enabling broader real-world deployment of 3D semantic occupancy prediction in autonomous driving.
Useful links
Paper: https://doi.org/10.1109/iros60139.2025.11247066
Preprint: https://arxiv.org/abs/2504.02270
Bibtex
@inproceedings{minkocc2025,
author={Sze, Samuel and De Martini, Daniele and Kunze, Lars},
title={MinkOcc: Towards real-time label-efficient semantic occupancy prediction},
year={2025},
booktitle={2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
pages={4036--4043},
doi={10.1109/iros60139.2025.11247066},
url={https://doi.org/10.1109/iros60139.2025.11247066},
}