cvprtum's video: CVPR 2021 Post-hoc Uncertainty Calibration for Domain Drift Scenarios

CVPR 2021 Oral Presentation Publication: Post-hoc Uncertainty Calibration for Domain Drift Scenarios Paper: https://arxiv.org/abs/2012.10988 Authors: Christian Tomani, Sebastian Gruber, Muhammed Ebrar Erdem, Daniel Cremers and Florian Buettner Abstract: We address the problem of uncertainty calibration. While standard deep neural networks typically yield uncalibrated predictions, calibrated confidence scores that are representative of the true likelihood of a prediction can be achieved using post-hoc calibration methods. However, to date, the focus of these approaches has been on in-domain calibration. Our contribution is two-fold. First, we show that existing post-hoc calibration methods yield highly overconfident predictions under domain shift. Second, we introduce a simple strategy where perturbations are applied to samples in the validation set before performing the post-hoc calibration step. In extensive experiments, we demonstrate that this perturbation step results in substantially better calibration under domain shift on a wide range of architectures and modelling tasks.