基于最优传输模型的人脸颜色转换

doi:10.3969/j.issn.0253-2778.2018.02.009

中国科学技术大学学报 ›› 2018, Vol. 48 ›› Issue (2): 148-153.DOI: 10.3969/j.issn.0253-2778.2018.02.009

基于最优传输模型的人脸颜色转换

李真熙，张举勇

中国科学技术大学数学科学学院，安徽合肥 230026

收稿日期:2017-01-09 修回日期:2017-03-28 出版日期:2018-02-28 发布日期:2018-02-28
通讯作者: 张举勇
作者简介:李真熙，女，1992年生，硕士.研究方向：深度神经网络，人脸图像分割和编辑. E-mail: jancee@mail.ustc.edu.cn
基金资助:
国家重点研发计划(2016YFC0800501)，国家自然科学基金 (61672481) 资助.

Face color transfer based on optimal transport model

LI Zhenxi , ZHANG Juyong

School of Mathematical Sciences, University of Science and Technology of China, Hefei 230026, China）

Received:2017-01-09 Revised:2017-03-28 Online:2018-02-28 Published:2018-02-28

摘要/Abstract

摘要： 现有人脸颜色转换算法仅基于图像的Lαβ颜色空间匹配均值和方差，因而仅限于线性变换且通常适用于自然图像；同时，现有算法利用人脸关键点信息来定位人脸五官的位置，但由此得到的人脸五官区域信息不是非常准确，通常需要进一步优化处理.针对上述问题，在自动获得人脸区域分割的基础上，为得到更自然的人脸颜色转换效果，基于最优传输模型，提出了一种新的人脸颜色转换的方法.首先利用全卷积网络自动得到人脸区域的分割信息，再利用最优传输模型获得对应的人脸区域的颜色转换结果.试验结果表明，所提算法在人脸五官分割的鲁棒性和人脸图像颜色转换的主观视觉上均得到明显的改善.

关键词: 人脸分割, 最优传输, 颜色转换, 全卷积网络

Abstract: Existing face color transfer algorithms are only based on Lαβ color space for matching mean and variance, and such transfer is limited to linear transformation and is usually applied to natural images. At the same time, these algorithms usually use facial landmark information to locate facial feature positions. However, the facial feature information thus obtained lacks accuracy and needs further optimization. To address these problems, a new face color transformation method was proposed based on the optimal transfer model to get the natural face color transfer effect. Firstly, the semantic information of the face was obtained directly by using the fully convolution network, and the corresponding face color transfer result was obtained by using the optimal transfer model. Experimental results show that the proposed algorithm is significantly improved in both the robustness of facial features segmentation and the subjective vision of face image color transfer.

Key words: face segmentation, optimal transport, color transfer, fully convolutional network

中图分类号:

TP391.41

李真熙，张举勇. 基于最优传输模型的人脸颜色转换[J]. 中国科学技术大学学报, 2018, 48(2): 148-153.

LI Zhenxi , ZHANG Juyong. Face color transfer based on optimal transport model[J]. Journal of University of Science and Technology of China, 2018, 48(2): 148-153.

参考文献

［1］
SHIRLEY P, ADHIKHMIN M, GOOCH B, et al. Color transfer between images[J]. IEEE Computer Graphics and Applications, 2001, 21: 34-41.
[2] WELSH T, ASHIKHMIN M, MUELLER K. Transferring color to greyscale images[J]. ACM Transactions on Graphics (TOG), 2002, 21(3): 277-280.
[3] LEVIN A, LISCHINSKI D, WEISSY. Colorization using optimization[J]. ACM Transactions on Graphics (TOG), 2004, 23(3): 689-694.
[4] NEUMANN L, NEUMANNA. Color style transfer techniques using hue, lightness and saturation histogram matching[C]// Proceedings of the First Eurographics conference on Computational Aesthetics in Graphics, Visualization and Imaging. Switzerland: Eurographics Association Aire-la-Ville, 2005: 111-122.

[5] XIAO X, MA L. Gradient-preserving color transfer[C]//Computer Graphics Forum. Hoboken, NJ: Blackwell Publishing Ltd, 2009, 28(7): 1879-1886.
[6] PITIE F, KOKARAM A C, DAHYOT R. N-dimensional probability density function transfer and its application to color transfer[C]// Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1. IEEE, 2005, 2: 1434-1439.
[7] ABADPOUR A, KASAEI S. An efficient PCA-based color transfer method [J]. Journal of Visual Communication and Image Representation, 2007, 18(1): 15-34.
[8] PITI F, KOKARAM A C, DAHYOT R. Automated colour grading using colour distribution transfer[J]. Computer Vision and Image Understanding, 2007, 107(1): 123-137.
[9] TAI Y W, JIA J, TANG C K. Local color transfer via probabilistic segmentation by expectation-maximization[C]// 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). IEEE, 2005, 1: 747-754.
[10] FRIGO O, SABATER N, DEMOULIN V, et al. Optimal transportation for example-guided color transfer[C]// Asian Conference on Computer Vision. Springer International Publishing, 2014: 655-670.
[11] FERRADANS S, PAPADAKIS N, PEYR G, et al. Regularized discrete optimal transport[J]. SIAM Journal on Imaging Sciences, 2014, 7(3): 1853-1882.
[12] YANG Y, ZHAO H, YOU L, ET AL. Semantic portrait color transfer with internet images[J]. Multimedia Tools and Applications, 2017, 76(1): 523-541.
[13] SOLOMON J, DE GOES F, PEYR G, et al. Convolutional Wasserstein distances: Efficient optimal transportation on geometric domains[J]. ACM Transactions on Graphics (TOG), 2015, 34(4): No. 66.
[14] VILLANI C. Topics in Optimal Transportation[M]. Providence, RI: American Mathematical Society, 2003.
[15] KRIZHEVSKY A, SUTSKEVER I, HINTONG E. ImageNet classification with deep convolutional neural networks[C]// Advances in Neural Information Processing Systems 25. Lake Tahoe, Nevada: Neural Information Processing Systems Foundation, 2012: 1097-1105.
[16] SERMANET P, EIGEN D, ZHANG X, et al. Overfeat: Integrated recognition, localization and detection using convolutional networks[DB/OL]. arXiv.org: arXiv:1312.6229, 2013.
[17] LONG J, SHELHAMER E, DARRELLT. Fully convolutional networks for semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2015: 3431-3440.
[18] HUANG G B, RAMESH M, BERG T, et al. Labeled faces in the wild: A database for studying face recognition in unconstrained environments[R]. Amherst, MA: University of Massachusetts, Amherst, 2007: Technical Report 07-49.

()
(

基于最优传输模型的人脸颜色转换

Face color transfer based on optimal transport model

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics

本文评价