不均衡数据情形的基于聚焦损失的CGAN的集成分类方法

doi:10.3969/j.issn.0253-2778.2020.07.014

中国科学技术大学学报 ›› 2020, Vol. 50 ›› Issue (7): 968-976.DOI: 10.3969/j.issn.0253-2778.2020.07.014

不均衡数据情形的基于聚焦损失的CGAN的集成分类方法

崔文泉，余厚莹，侯晓天

中国科学技术大学管理学院统计与金融系，安徽合肥 230026

收稿日期:2020-05-25 修回日期:2020-06-27 接受日期:2020-06-27 出版日期:2020-07-31 发布日期:2020-06-27
通讯作者: 崔文泉
作者简介:崔文泉（通讯作者），男，1964年生，博士/副教授.研究方向：数理统计.E-mail: wqcui@ustc.edu.cn
基金资助:
国家自然科学基金（71873128）资助.

Focused loss-based for imbalanced data scenarios integrated classification methods for CGAN

CUI Wenquan, YU Houying, HOU Xiaotian

Received:2020-05-25 Revised:2020-06-27 Accepted:2020-06-27 Online:2020-07-31 Published:2020-06-27

摘要/Abstract

摘要： 针对非均衡数据的情形，基于条件生成对抗网络(conditional generative adversarial networks，CGAN)，利用梯度提升树研究了聚焦损失的CGAN的集成分类方法.该方法首先通过CGAN降低不均衡率，通过聚焦损失的权值均衡结合GBDT算法，适当增加对少数类样本的关注度进而进一步提升分类器的分类性能.对方法的性质进行了研究，获得了若干理论成果.证明了：在一定条件下，由CGAN产生的经验条件分布收敛于相应总体的条件分布；聚集损失的CGAN方法其经验风险收敛到期望风险；该方法的估计量会收敛到使得期望风险最小化的函数.实验结果显示了聚焦损失的CGAN方法具有良好的表现.

关键词: 非均衡数据, 条件生成对抗网络, 聚焦损失, 集成学习

Abstract: For the case of imbalanced data, an integrated classification method for CGAN-focal-loss was investigated based on conditional generative adversarial networks (CGAN) using gradient boosting trees. The method first reduces the imbalance rate by CGAN, and further improves the classification performance of the classifier by increasing the focus on a few classes of samples through the weight balancing of the focused loss combined with the GBDT algorithm. The properties of the method were investigated and several theoretical results were obtained. It was proved that the empirical conditional distribution generated by CGAN converges to the conditional distribution of the corresponding aggregate under certain conditions; that the empirical risk of the CGAN method with focused loss converges to the expected risk; and that the estimator of the method converges to the function that minimizes the expected risk. The experimental results show the good performance of the CGAN-focal-loss method.

Key words: imbalanced data, conditional generative adversarial networks(CGAN), focal loss, ensemble learning

中图分类号:

TP181

崔文泉，余厚莹，侯晓天. 不均衡数据情形的基于聚焦损失的CGAN的集成分类方法[J]. 中国科学技术大学学报, 2020, 50(7): 968-976.

参考文献

［1］ AKBANI R, KWEK S, JAPKOWICZ N. Applying support vector machine to imbalanced datasets[C]// Machine Learning: ECML 2004. Berlin: Springer, 2004: 39-50.
[2] MAZUROWSKI M A, HABAS P A, ZURADA J M, et al. Training neural network classifiers for medical decision making: The effects of imbalanced datasets on classification performance[J]. Neural Networks, 2008, 21:427-436.
[3] TAVALLAEE M, STAKHANVA N, GHORBANI A A. Toward credible evaluation of anomaly-based intrusion-detection methods[J]. IEEE Transactions on Systems, Man, and Cybernetics Part C, 2010, 40(5):516-524.
[4] BERMEJO P,GSMEZ J A, PUERTA J M. Improving the performance of Naive Bayes multinomial in e-mail foldering by introducing distribution-based balance of datasets[J]. Expert Systems with Applications, 2011, 38(3): 2072-2080.
[5] WEI W, LI J, CAO L, et al. Effective detection of sophisticated online banking fraud on extremely imbalanced data[J]. World Wide Web, 2013, 16: 449-475.
[6] KERDPRASOP K, KERDPRASOP N. A data mining approach to automate fault detection model development in the semiconductor manufacturing process[J]. International Journal of Mechanics, 2011, 5(4): 336-344.
[7] CHAWLA N V, JAPKOWICZ N, KOTCZ A. Editorial: Special issue on learning from imbalanced data sets[J]. ACM SIGKDD Explorations Newsletter, 2004, 6(1): 1-6.
[8] DOUZAS G, BACAO F. Effective data generation for imbalanced learning using conditional generative adversarial networks[J]. Expert Systems with Applications, 2018, 91: 464-471.
[9] CHAWLA N V. Data mining for imbalanced datasets: An overview[C]// Data Mining and Knowledge Discovery Handbook. Berlin: Springer, 2009: 875-886.
[10] CHAWLA N V, BOWYER K W, HALL L O, et al. SMOTE: Synthetic minority over-sampling technique[J]. Journal of Artificial Intelligence Research, 2002, 16(1): 321-357.
[11] HAN H, WANG W Y, MAO B H. Borderline-SMOTE: A new over-sampling method in imbalanced data sets learning[J]. ICIC 2005: Advances in Intelligent Computing, 2005, 17(12): 878-887.
[12] HE H, BAI Y, GARCIA E A, et al. ADASYN: Adaptive synthetic sampling approach for imbalanced learning[C]// 2008 IEEE International Joint Conference on Neural Networks. IEEE, 2008: 1322-1328.
[13] BATISTA G E, PRATI R C, MONARD M C. A study of the behavior of several methods for balancing machine learning training data[J]. ACM SIGKDD Explorations Newsletter, 2004, 6(1): 20-29.
[14] FAN W, ZHANG J, STOTFO S J, et al. AdaCost: Misclassification cost-sensitive boosting[C]// Proceedings of the 16th International Conference on Learning, Slovenia: Morgan Kaufmann, 1999: 97-105.
[15] WOZNIAK M. Classifiers: Methods of data,knowledge, and classifier combination[M]. Berlin: Springer, 2013.
[16] MARIANI G, SCHEIDEGGER F, ISTRATE R, et al. BAGAN: Data augmentation with balancing GAN[DB/OL]. [2020-05-01]. https://arxiv.org/abs/1803.09655.
[17] RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[DB/OL]. [2020-05-01]. https://arxiv.org/abs/1511.06434v1.
[18] GAO Y , JIAO Y , WANG Y , et al. Deep generative learning via variational gradient flow[DB/OL]. [2020-05-01]. https://arxiv.org/abs/1901.08469.
[19] ZHANG Y. Deep generative model for multi-class imbalanced learning[DB]// Open Access Master’s Theses, 2018: Paper 1277.
[20] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017: 2980-2988.
[21] FAWCETT T. An introduction to ROC analysis[J]. Pattern Recognition Letters, 2006, 27(8): 861-874.
[22] 赵海霞，石洪波，武建，等.基于条件生成对抗网络的不平衡学习研究[J/OL].控制与决策, 2019: https://doi.org/10.13195/j.kzyjc.2019.0522.
[23] 莫赞，盖彦蓉，樊冠龙.基于GAN-AdaBoost-DT不平衡分类算法的信用卡欺诈分类[J].计算机应用, 2019, 39(2):618-622.
[24] 李诒靖,郭海湘,李亚楠,等.一种基于Boosting的集成学习算法在不均衡数据中的分类[J].系统工程理论与实践, 2016, 36(1):189-199.
[25] GOODFELLOW I, POUGET A J, MIRZA M, et al. Generative adversarial nets[J]. Advances in Neural Information Processing Systems Conference, 2014, 27:2672-2680.
[26] SALANT S W, SWITZER S, REYNOLDS R J. Losses from horizontal merger:The effects of an exogenous change in industry structure on Cournot-Nash equilibrium[J]. The Quarterly Journal of Economics, 1983, 98(2):185-199.
[27] MIRZA M, OSINDERO S. Conditional generative adversarial nets[C]// Proceedings of the Neural Information Processing Systems Deep Learning Workshop, 2014.
[28] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout a simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research, 2014, 15(1): 1929-1958.
[29] BERGSTRA J, YAMINS D, DAVID D C. Hyperopt: A Python library for optimizing the hyperparameters of machine learning algorithms[C]// Proceedings of the 12th Python in Science Conference, Austin, TX, 2012.
[30] 李航.统计学习方法[M].北京：清华大学出版社, 2012.
[31] KINGMA D P, BA J. Adam: A method for stochastic optimization[C]// The 3rd International Conference on Learning Representations, San Diego, CA, 2015.
[32] BIAU G, CADRE B, SANGNIER M, et al. Some theoretical properties of GANs[DB/OL]. [2020-05-01]. https://arxiv.org/abs/1803.07819.
[33] TSYBAKOV A B. Introduction to Nonparametric Estimation[M].Berlin: Springer, 2008.
[34] VAN DER VAART A W, WELLNER J. Weak Convergence and Empirical Processes[M]. Berlin: Springer, 2000.
[35] GIN W E, NICKL R. Mathematical Foundations of Infinite Dimensional Statistical Models[M]. Cambridge: Cambridge University Press, 2015.

[36] FRIEDMAN J H. Greedy function approximation: A gradient boosting machine[J]. Annals of Statistics, 2001, 29(5): 1189-1232.

()
()

[1]	崔文泉，陈伟，程浩洋. 非均衡数据情形的一种协同正则化多视图半监督学习分类器[J]. 中国科学技术大学学报, 2020, 50(5): 596-604.
[2]	何捷舟，刘金平，张五霞，肖文辉，唐朝晖，徐鹏飞. 基于在线自适应极限学习机选择性集成的网络入侵检测[J]. 中国科学技术大学学报, 2019, 49(7): 544-554.
[3]	王子一，商琳. 基于电力负荷曲线的设备识别方法[J]. 中国科学技术大学学报, 2019, 49(2): 100-104.
[4]	崔文泉，余德美，程浩洋. 一种处理非均衡数据的非迭代核逻辑回归方法[J]. 中国科学技术大学学报, 2019, 49(12): 965-973.
[5]	崔文泉，黄禹侨. 高维数据情形下的一种基于随机投影的集成分类方法[J]. 中国科学技术大学学报, 2019, 49(12): 974-984.

不均衡数据情形的基于聚焦损失的CGAN的集成分类方法

Focused loss-based for imbalanced data scenarios integrated classification methods for CGAN

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 5

编辑推荐

Metrics

本文评价