中国科学技术大学学报 ›› 2019, Vol. 49 ›› Issue (1): 40-48.DOI: 10.3969/j.issn.0253-2778.2019.01.006

• 原创论文 • 上一篇    下一篇

基于两层迁移卷积神经网络的抽象图像情感识别

杨子文   

  1. 1.南京邮电大学计算机学院,江苏南京 210023; 2.江苏省无线传感网高技术研究重点实验室,江苏南京 210023; 3.南京航空航天大学计算机科学与技术学院,江苏南京 210016
  • 收稿日期:2018-06-14 修回日期:2018-09-18 出版日期:2019-01-31 发布日期:2019-01-31
  • 通讯作者: 陈蕾
  • 作者简介:杨子文,男,1994年生,硕士生.研究方向:机器学习和数据挖掘.E-mail: yangziwen1994@gmail.com
  • 基金资助:
    国家自然科学基金(61572263);江苏省自然科学基金(BK20161516);中国博士后科学基金(2015M581794);江苏省博士后科研资助计划(1501023C)资助.

Recognizing emotions from abstract paintings using convolutional neural network with two-layer transfer learning scheme

YANG Ziwen   

  1. 1. School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing 210023; 2. Jiangsu High Technology Research Key Laboratory for Wireless Sensor Networks, Nanjing 210023; 3. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 210016
  • Received:2018-06-14 Revised:2018-09-18 Online:2019-01-31 Published:2019-01-31

摘要: 为弥合抽象图像底层视觉特征与高层情感语义间的鸿沟,同时缓解抽象图像情感识别所固有的小样本缺陷,将两层迁移学习策略引入传统的卷积神经网络,提出一种基于两层迁移卷积神经网络的抽象图像情感识别模型.该模型利用深度特征的层次性,首先通过大规模通用图像数据集来学习提取普适的底层图像特征;然后利用抽象图像风格分类数据集来学习提取抽象图像的专有高层语义特征;最后采用抽象图像情感识别数据集来微调整个网络.MART数据集上的实验结果表明,与传统的抽象图像情感识别方法相比,所提出的模型能够有效地提高识别精度.

关键词: 情感识别, 深度学习, 迁移学习, 卷积神经网络, 抽象图像

Abstract: In order to bridge the gap between low-level visual features and high-level emotional semantics, and to alleviate the defects inherent in small sample dataset in abstract paintings emotions recognition datasets, a two-layer transfer learning strategy is introduced into traditional convolutional neural networks and a model for recognizing emotions from abstract paintings is proposed using convolutional neural network with a two-layer transfer learning scheme. According to the hierarchical nature of deep features, a large-scale generalized image dataset is used to learn how extract universal low-level image features. Then the relevant domain dataset is utilized to learn how extract specific high-level semantic features. Finally the abstract painting emotion recognition dataset is used to finetune the network. As shown by our extensive experimental validation on MART datasets, the proposal outperforms current methods when recognizing emotions from abstract paintings.

Key words: emotion recognition, deep learning, transfer learning, convolutional neural network, abstract paintings