基于可视化的卷积神经网络优化方法研究

doi:10.3969/j.issn.0253-2778.2020.07.013

中国科学技术大学学报 ›› 2020, Vol. 50 ›› Issue (7): 959-967.DOI: 10.3969/j.issn.0253-2778.2020.07.013

基于可视化的卷积神经网络优化方法研究

王悦，李京

中国科学技术大学计算机科学与技术学院，安徽合肥 230027

收稿日期:2020-05-24 修回日期:2020-06-24 接受日期:2020-06-24 出版日期:2020-07-31 发布日期:2020-06-24
通讯作者: 李京
作者简介:王悦，男，1994年生，硕士. 研究方向：机器学习. E-mail: cahhbwy@mail.ustc.edu.cn
基金资助:
中国科学院战略性先导科技专项( XDA/B 19020102)资助.

Research on optimization method of convolutional neural network based on visualization

WANG Yue, LI Jing

School of Computer Science and Technology, University of Science and Technology of China,Hefei 230027, China

Received:2020-05-24 Revised:2020-06-24 Accepted:2020-06-24 Online:2020-07-31 Published:2020-06-24

摘要/Abstract

摘要： 随着计算机算力的提升，深度学习的应用范围越来越广，深度学习模型的设计和调优变得困难，对于复杂模型，只对一层网络进行调整可能就导致差异显著的结果.众多研究者往往根据历史经验调参，进行了大量试错，耗费了大量的时间精力.为此根据卷积神经网络模型的数据特征，提出一种基于可视化的辅助调参的方法.通过可视化手段剖析卷积神经网络内部数据，分析其代表的信息，从而快速定位模型故障，实现有针对性地调参，降低了研究者在调参时的工作难度，提升了工作效率.

关键词: 卷积神经网络, 调参, 可视化, 层次聚类法, 核密度估计, 生成对抗网络

Abstract: With the lifting force computer calculation, the application range of the depth of learning more and more widely. However, the design and tuning of deep learning models is very difficult. For complex models, adjusting only one layer of the network may lead to very different results. Many researchers usually adjust their parameters based on past experience, make a lot of trial and error, and wasting a lot of time and energy. Based on the data characteristics of the convolutional neural network model, this paper proposes a method of auxiliary parameter adjustment based on visualization. Analyze the internal data of the convolutional neural network by visualization and analyze the information represented by it, so as to quickly locate the model fault, realize targeted parameter adjustment, reduce the difficulty of researchers in parameter adjustment, and improve work efficiency.

Key words: convolutional neural networks, parameter tuning, visualization, hierarchical clustering, kernel density estimation, generative adversarial networks

中图分类号:

TP183

王悦，李京. 基于可视化的卷积神经网络优化方法研究[J]. 中国科学技术大学学报, 2020, 50(7): 959-967.

WANG Yue, LI Jing. Research on optimization method of convolutional neural network based on visualization[J]. Journal of University of Science and Technology of China, 2020, 50(7): 959-967.

参考文献

［1］
GOODFELLOW I, BENGIO Y, COURVILLE A. Deep Learning[M]. MIT press, 2016.
[2] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. 2012: 1097-1105.
[3] HE T, ZHANG Z, ZHANG H, et al. Bag of tricks for image classification with convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019: 558-567.
[4] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. 2012: 1097-1105.
[5] ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks [C]// Computer Vision - ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8689. Springer, Cham, 2014: 818-833.
[6] YOSINSKI J, CLUNE J, NGUYEN A, et al. Understanding neural networks through deep visualization[EB/OL].(2015-06-22)[2020-04-24]. https://arxiv.org/abs/1506.06579.
[7] LIU M, SHI J, LI Z, et al. Towards better analysis of deep convolutional neural networks[J]. IEEE Transactions on Visualization and Computer Graphics, 2016, 23(1): 91-100.
[8] DUMOULIN V, VISIN F. A guide to convolution arithmetic for deep learning[EB/OL].(2018-01-11)[2020-04-24]. https://arxiv.org/abs/1603.07285.
[9] SIBI P, JONES S A, SIDDARTH P. Analysis of different activation functions using back propagation neural networks[J]. Journal of Theoretical and Applied Information Technology, 2013, 47(3): 1264-1268.
[10] IOFFE S, SZEGEDY C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[C]// Proceedings of the 32nd International Conference on International Conference on Machine Learning ,2015: 448-456; doi: 10.5555/3045118.3045167.
[11] YU D, WANG H, CHEN P, et al. Mixed pooling for convolutional neural networks[C]//International conference on rough sets and knowledge technology.Springer, Cham, 2014: 364-375.
[12] DAHL G E, SAINATH T N, HINTON G E. Improving deep neural networks for LVCSR using rectified linear units and dropout[C]//2013 IEEE international conference on acoustics, speech and signal processing. IEEE, 2013: 8609-8613.
[13] SPENCE R. Information Visualization[M]. New York: Addison-Wesley, 2001.
[14] ABADI M, BARHAM P, CHEN J, et al. Tensorflow: A system for large-scale machine learning[C]//12th USENIX Symposium on Operating Systems Design and Implementation (OSDI’ 16). 2016: 265-283.
[15] LIU Z, LUO P, WANG X, et al. Large-scale celebfaces attributes (celeba) dataset[J]. Retrieved August, 2018, 15: 2018.
[16] JOHNSON S C. Hierarchical clustering schemes[J]. Psychometrika, 1967, 32(3): 241-254.
[17] BOTEV Z I, GROTOWSKI J F, KROESE D P. Kernel density estimation via diffusion[J]. The Annals of Statistics, 2010, 38(5): 2916-2957.
[18] NGUYEN H V, BAI L. Cosine similarity metric learning for face verification[C]//Asian conference on computer vision.Springer, Berlin, Heidelberg, 2010: 709-720.
[19] RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[J]. 2015 arXiv:1511.06434.

()
()

[1]	罗珽, 冷伟. 沉浸式虚拟现实技术在地球科学中的应用[J]. 中国科学技术大学学报, 2021, 51(6): 431-440.
[2]	魏俣童, 鲍秉坤, 张子祺, 朱进. 不稳定传输中受损视频的低延迟修复方法[J]. 中国科学技术大学学报, 2021, 51(10): 717-724.
[3]	白江波，杨阳，张文生. 基于改进CycleGAN的合成孔径雷达图像仿真[J]. 中国科学技术大学学报, 2020, 50(8): 1181-1186.
[4]	李健，阮月，杨亮，刘变化. 构建比色纳米传感器用于检测水中的As(III)[J]. 中国科学技术大学学报, 2020, 50(7): 873-879.
[5]	崔文泉，余厚莹，侯晓天. 不均衡数据情形的基于聚焦损失的CGAN的集成分类方法[J]. 中国科学技术大学学报, 2020, 50(7): 968-976.
[6]	杜淑颖，杜鹏，丁世飞. 基于CNN的假冒域名识别方法研究[J]. 中国科学技术大学学报, 2020, 50(7): 1019-1025.
[7]	孟徐然，毕秀春，张曙光. 基于生成对抗网络的高频算法及其回测研究[J]. 中国科学技术大学学报, 2020, 50(6): 801-810.
[8]	熊军林, 赵铎. 基于RGB图像的二阶段机器人抓取位置检测方法[J]. 中国科学技术大学学报, 2020, 50(1): 1-10.
[9]	曾伟辉，李淼，张健，黄小平，王敬贤，袁媛. 面向农作物病害识别的高阶残差卷积神经网络研究[J]. 中国科学技术大学学报, 2019, 49(10): 781-790.
[10]	杨子文，陈蕾，浦建宇. 基于两层迁移卷积神经网络的抽象图像情感识别[J]. 中国科学技术大学学报, 2019, 49(1): 40-48.
[11]	王渊，孙樱郡，王博，杨朝洲，杨量. 在线社会关系语义化度量的可视化分析[J]. 中国科学技术大学学报, 2018, 48(4): 314-321.
[12]	孙达昌，毕秀春. 基于深度学习算法的高频交易策略及其盈利能力[J]. 中国科学技术大学学报, 2018, 48(11): 923-932.
[13]	赵凡，蒋同海，周喜，马博，程力，. 面向多维稀疏时空数据的可视化研究[J]. 中国科学技术大学学报, 2017, 47(7): 556-568.
[14]	陈东杰，张文生，杨阳. 基于深度学习的高铁接触网定位器检测与识别[J]. 中国科学技术大学学报, 2017, 47(4): 320-327.
[15]	张皓，吴建鑫，. 集成最大汇合: 最大汇合时只有最大值有用吗[J]. 中国科学技术大学学报, 2017, 47(10): 799-807.

基于可视化的卷积神经网络优化方法研究

Research on optimization method of convolutional neural network based on visualization

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价