王昊
(2022-09-01 14:36):
#paper doi:10.1109/TNNLS.2022.3152527 Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. 2022. Learning From Noisy Labels With Deep Neural Networks: A Survey. IEEE Transactions on Neural Networks and Learning Systems: 1–19. 本文是噪声标签2022年的综述论文,主要介绍结构化数据、图片分类数据集等进行去噪的各种方法。具体类别总结如下:
【Robust Architecture】基于attention注意力机制给干净样本和噪声数据进行打分,文章叫做Attention Feature Mixup,在计算最终loss的时候有两部分,一部分是同一个类的每张图和标签计算的交叉熵损失;另外一个损失是数据mixup得到的新的数据x'和标签y'计算的loss.
【Robust Regularization】
这一部分主要是通过一些添加正则ticks,防止模型过拟合到噪声数据上,常用的正则方法包含:label smooth、l1、l2、MixUp等.
【Sample Selection】Area Under the Margin metric (AUM):在训练过程中一边训练一边筛选数据的方式.
【数据划分】是通过密度聚类的思路,将一个类的数据分成easy dataset、smi-hard dataset 和 hard dataset,一般噪声数据是较为困难训练的数据,对于每张图分配一个权重,文中建议1.0、0.5和0.5;模型的训练借鉴了课程学习的思路.
【Semi-supervised Learning】基于半监督学习的带噪学习算法,首先介绍DivideMix方法,其实还是co-teaching的思路,但是在挑出干净样本和噪音样本后,把噪音样本当做无标签样本,通过 FixMatch 的方法进行训练,目前半监督图像分类的 SOTA 应该还是 FixMatch. (这个性能比较好)
【Label correction】“label correction phase”通过一个pre-trained模型得到随机选择每个类中的几张图采用聚类的方法得到Prototype样本的每个类的聚类中心,对输入图片得到的特征向量和各类聚类中心计算距离,得到图片的伪标签,最后的loss是原始标签计算的交叉熵损失和伪标签计算的伪标签的求和。
IF:10.200Q1
IEEE transactions on neural networks and learning systems,
2023-Nov.
DOI: 10.1109/TNNLS.2022.3152527
PMID: 35254993
Learning From Noisy Labels With Deep Neural Networks: A Survey
翻译
Abstract:
Deep learning has achieved remarkable success in numerous domains with help from large amounts of big data. However, the quality of data labels is a concern because of the lack of high-quality labels in many real-world scenarios. As noisy labels severely degrade the generalization performance of deep neural networks, learning from noisy labels (robust training) is becoming an important task in modern deep learning applications. In this survey, we first describe the problem of learning with label noise from a supervised learning perspective. Next, we provide a comprehensive review of 62 state-of-the-art robust training methods, all of which are categorized into five groups according to their methodological difference, followed by a systematic comparison of six properties used to evaluate their superiority. Subsequently, we perform an in-depth analysis of noise rate estimation and summarize the typically used evaluation methodology, including public noisy datasets and evaluation metrics. Finally, we present several promising research directions that can serve as a guideline for future studies.
翻译