Robust Learning via Golden Symmetric Loss of (un)Trusted Labels

Robust Learning via Golden Symmetric Loss of (un)Trusted Labels

Ghiassi, S. (TU Delft Data-Intensive Systems)
Birke, Robert (University of Turin)
Chen, Lydia Y. (TU Delft Data-Intensive Systems)

2023

Learning robust deep models against noisy labels becomes ever critical when today's data is commonly collected from open platforms and subject to adversarial corruption. The information on the label corruption process, i.e., corruption matrix, can greatly enhance the robustness of deep models but still fall behind in combating hard classes. In this paper, we propose to construct a golden symmetric loss (GSL) based on the estimated corruption matrix as to avoid overfitting to noisy labels and learn effectively from hard classes. GSL is the weighted sum of the corrected regular cross entropy and reverse cross entropy. By leveraging a small fraction of trusted clean data, we estimate the corruption matrix and use it to correct the loss as well as to determine the weights of GSL. We theoretically prove the robustness of the proposed loss function in the presence of dirty labels. We provide a heuristics to adaptively tune the loss weights of GSL according to the noise rate and diversity measured from the dataset. We evaluate our proposed golden symmetric loss on both vision and natural language deep models subject to different types of label noise patterns. Empirical results show that GSL can significantly outperform the existing robust training methods on different noise patterns, showing accuracy improvement up to 18% on CIFAR-100 and 1% on real world noisy dataset of Clothing1M.

Deep learning models
Noisy labels
Robust training
Symmetric loss function

http://resolver.tudelft.nl/uuid:753e81de-d1cc-4c05-9c0a-1fe0a796a583

Society for Industrial and Applied Mathematics

2024-01-11

9781611977653

2023 SIAM International Conference on Data Mining, SDM 2023

2023 SIAM International Conference on Data Mining, SDM 2023, 2023-04-27 → 2023-04-29, Minneapolis, United States

2023 SIAM International Conference on Data Mining, SDM 2023

Institutional Repository

conference paper

© 2023 S. Ghiassi, Robert Birke, Lydia Y. Chen