AUC margin loss for limited, imbalanced and noisy medical image diagnosis: a case study on CheXpert5000

The AUC margin loss is a valuable loss function for medical image classification as it addresses the problems of imbalanced and noisy labels. It is used by the current winner of the CheXpert competition. The CheXpert dataset is a large dataset (200k+ images), however datasets in the range of 1k-10k medical datasets are much more common. This raises the question if optimizing AUC margin loss also is effective in scenarios with limited data.We compare AUC margin loss optimization to binary cross-entropy on limited, imbalanced and noisy CheXpert5000, a subset of CheXpert dataset. We show that AUC margin loss is beneficial for limited data and considerably improves accuracy in the presence of label noise. It also improves out-of-box calibration.