Research on optimization method of convolutional neural network based on visualization

doi:10.3969/j.issn.0253-2778.2020.07.013

Abstract

Abstract: With the lifting force computer calculation, the application range of the depth of learning more and more widely. However, the design and tuning of deep learning models is very difficult. For complex models, adjusting only one layer of the network may lead to very different results. Many researchers usually adjust their parameters based on past experience, make a lot of trial and error, and wasting a lot of time and energy. Based on the data characteristics of the convolutional neural network model, this paper proposes a method of auxiliary parameter adjustment based on visualization. Analyze the internal data of the convolutional neural network by visualization and analyze the information represented by it, so as to quickly locate the model fault, realize targeted parameter adjustment, reduce the difficulty of researchers in parameter adjustment, and improve work efficiency.

Key words: convolutional neural networks, parameter tuning, visualization, hierarchical clustering, kernel density estimation, generative adversarial networks

CLC Number:

TP183

WANG Yue, LI Jing. Research on optimization method of convolutional neural network based on visualization[J]. Journal of University of Science and Technology of China, 2020, 50(7): 959-967.

References

［1］
GOODFELLOW I, BENGIO Y, COURVILLE A. Deep Learning[M]. MIT press, 2016.
[2] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. 2012: 1097-1105.
[3] HE T, ZHANG Z, ZHANG H, et al. Bag of tricks for image classification with convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019: 558-567.
[4] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. 2012: 1097-1105.
[5] ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks [C]// Computer Vision - ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8689. Springer, Cham, 2014: 818-833.
[6] YOSINSKI J, CLUNE J, NGUYEN A, et al. Understanding neural networks through deep visualization[EB/OL].(2015-06-22)[2020-04-24]. https://arxiv.org/abs/1506.06579.
[7] LIU M, SHI J, LI Z, et al. Towards better analysis of deep convolutional neural networks[J]. IEEE Transactions on Visualization and Computer Graphics, 2016, 23(1): 91-100.
[8] DUMOULIN V, VISIN F. A guide to convolution arithmetic for deep learning[EB/OL].(2018-01-11)[2020-04-24]. https://arxiv.org/abs/1603.07285.
[9] SIBI P, JONES S A, SIDDARTH P. Analysis of different activation functions using back propagation neural networks[J]. Journal of Theoretical and Applied Information Technology, 2013, 47(3): 1264-1268.
[10] IOFFE S, SZEGEDY C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[C]// Proceedings of the 32nd International Conference on International Conference on Machine Learning ,2015: 448-456; doi: 10.5555/3045118.3045167.
[11] YU D, WANG H, CHEN P, et al. Mixed pooling for convolutional neural networks[C]//International conference on rough sets and knowledge technology.Springer, Cham, 2014: 364-375.
[12] DAHL G E, SAINATH T N, HINTON G E. Improving deep neural networks for LVCSR using rectified linear units and dropout[C]//2013 IEEE international conference on acoustics, speech and signal processing. IEEE, 2013: 8609-8613.
[13] SPENCE R. Information Visualization[M]. New York: Addison-Wesley, 2001.
[14] ABADI M, BARHAM P, CHEN J, et al. Tensorflow: A system for large-scale machine learning[C]//12th USENIX Symposium on Operating Systems Design and Implementation (OSDI’ 16). 2016: 265-283.
[15] LIU Z, LUO P, WANG X, et al. Large-scale celebfaces attributes (celeba) dataset[J]. Retrieved August, 2018, 15: 2018.
[16] JOHNSON S C. Hierarchical clustering schemes[J]. Psychometrika, 1967, 32(3): 241-254.
[17] BOTEV Z I, GROTOWSKI J F, KROESE D P. Kernel density estimation via diffusion[J]. The Annals of Statistics, 2010, 38(5): 2916-2957.
[18] NGUYEN H V, BAI L. Cosine similarity metric learning for face verification[C]//Asian conference on computer vision.Springer, Berlin, Heidelberg, 2010: 709-720.
[19] RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[J]. 2015 arXiv:1511.06434.

()
()

[1]	Luo Ting, Leng Wei. The application of immersive virtual reality technology in geoscience [J]. Journal of University of Science and Technology of China, 2021, 51(6): 431-440.
[2]	. Focused loss-based for imbalanced data scenarios integrated classification methods for CGAN [J]. Journal of University of Science and Technology of China, 2020, 50(7): 968-976.
[3]	WANG Yuan, SUN Yingjun, WANG Bo, YANG Chaozhou, YANG Liang. Visual analysis of semantic measuring of online social relationships [J]. Journal of University of Science and Technology of China, 2018, 48(4): 314-321.
[4]	ZHAO Fan, JIANG Tonghai, ZHOU Xi, MA Bo, CHENG Li,. Visualization of multi-dimensional sparse spatial-temporal data [J]. Journal of University of Science and Technology of China, 2017, 47(7): 556-568.
[5]	CHEN Dongjie, ZHANG Wensheng, YANG Yang. Detection and recognition of high-speed railway catenary locator based on Deep Learning [J]. Journal of University of Science and Technology of China, 2017, 47(4): 320-327.
[6]	LI Yilin, ZHU Jiaqi, WU Yunkun, WANG Hongan. A visualization method for analyzing sub-topics of hot events in microblogs [J]. Journal of University of Science and Technology of China, 2017, 47(1): 48-56.