Explainable Action Prediction through Self-Supervision on Scene Graphs

- 1 min read

Abstract

This work explores scene graphs as a distilled representation of high-level information for autonomous driving, applied to future driver-action prediction. Given the scarcity and strong imbalance of data samples, we propose a self-supervision pipeline to infer representative and well-separated embeddings. Key aspects are interpretability and explainability; as such, we embed in our architecture attention mechanisms that can create spatial and temporal heatmaps on the scene graphs. We evaluate our system on the ROAD dataset against a fully-supervised approach, showing the superiority of our training regime.

Paper: https://doi.org/10.1109/icra48891.2023.10161132

Preprint: https://arxiv.org/abs/2302.03477

Bibtex

@inproceedings{explainableactionprediction2023,
  author={Kochakarn, Pawit and De Martini, Daniele and Omeiza, Daniel and Kunze, Lars},
  title={Explainable Action Prediction through Self-Supervision on Scene Graphs},
  year={2023},
  booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)},
  pages={1479--1485},
  doi={10.1109/icra48891.2023.10161132},
  url={https://doi.org/10.1109/icra48891.2023.10161132},
}