Face color transfer based on optimal transport model

doi:10.3969/j.issn.0253-2778.2018.02.009

Abstract

Abstract: Existing face color transfer algorithms are only based on Lαβ color space for matching mean and variance, and such transfer is limited to linear transformation and is usually applied to natural images. At the same time, these algorithms usually use facial landmark information to locate facial feature positions. However, the facial feature information thus obtained lacks accuracy and needs further optimization. To address these problems, a new face color transformation method was proposed based on the optimal transfer model to get the natural face color transfer effect. Firstly, the semantic information of the face was obtained directly by using the fully convolution network, and the corresponding face color transfer result was obtained by using the optimal transfer model. Experimental results show that the proposed algorithm is significantly improved in both the robustness of facial features segmentation and the subjective vision of face image color transfer.

Key words: face segmentation, optimal transport, color transfer, fully convolutional network

CLC Number:

TP391.41

LI Zhenxi , ZHANG Juyong. Face color transfer based on optimal transport model[J]. Journal of University of Science and Technology of China, 2018, 48(2): 148-153.

References

［1］
SHIRLEY P, ADHIKHMIN M, GOOCH B, et al. Color transfer between images[J]. IEEE Computer Graphics and Applications, 2001, 21: 34-41.
[2] WELSH T, ASHIKHMIN M, MUELLER K. Transferring color to greyscale images[J]. ACM Transactions on Graphics (TOG), 2002, 21(3): 277-280.
[3] LEVIN A, LISCHINSKI D, WEISSY. Colorization using optimization[J]. ACM Transactions on Graphics (TOG), 2004, 23(3): 689-694.
[4] NEUMANN L, NEUMANNA. Color style transfer techniques using hue, lightness and saturation histogram matching[C]// Proceedings of the First Eurographics conference on Computational Aesthetics in Graphics, Visualization and Imaging. Switzerland: Eurographics Association Aire-la-Ville, 2005: 111-122.

[5] XIAO X, MA L. Gradient-preserving color transfer[C]//Computer Graphics Forum. Hoboken, NJ: Blackwell Publishing Ltd, 2009, 28(7): 1879-1886.
[6] PITIE F, KOKARAM A C, DAHYOT R. N-dimensional probability density function transfer and its application to color transfer[C]// Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1. IEEE, 2005, 2: 1434-1439.
[7] ABADPOUR A, KASAEI S. An efficient PCA-based color transfer method [J]. Journal of Visual Communication and Image Representation, 2007, 18(1): 15-34.
[8] PITI F, KOKARAM A C, DAHYOT R. Automated colour grading using colour distribution transfer[J]. Computer Vision and Image Understanding, 2007, 107(1): 123-137.
[9] TAI Y W, JIA J, TANG C K. Local color transfer via probabilistic segmentation by expectation-maximization[C]// 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). IEEE, 2005, 1: 747-754.
[10] FRIGO O, SABATER N, DEMOULIN V, et al. Optimal transportation for example-guided color transfer[C]// Asian Conference on Computer Vision. Springer International Publishing, 2014: 655-670.
[11] FERRADANS S, PAPADAKIS N, PEYR G, et al. Regularized discrete optimal transport[J]. SIAM Journal on Imaging Sciences, 2014, 7(3): 1853-1882.
[12] YANG Y, ZHAO H, YOU L, ET AL. Semantic portrait color transfer with internet images[J]. Multimedia Tools and Applications, 2017, 76(1): 523-541.
[13] SOLOMON J, DE GOES F, PEYR G, et al. Convolutional Wasserstein distances: Efficient optimal transportation on geometric domains[J]. ACM Transactions on Graphics (TOG), 2015, 34(4): No. 66.
[14] VILLANI C. Topics in Optimal Transportation[M]. Providence, RI: American Mathematical Society, 2003.
[15] KRIZHEVSKY A, SUTSKEVER I, HINTONG E. ImageNet classification with deep convolutional neural networks[C]// Advances in Neural Information Processing Systems 25. Lake Tahoe, Nevada: Neural Information Processing Systems Foundation, 2012: 1097-1105.
[16] SERMANET P, EIGEN D, ZHANG X, et al. Overfeat: Integrated recognition, localization and detection using convolutional networks[DB/OL]. arXiv.org: arXiv:1312.6229, 2013.
[17] LONG J, SHELHAMER E, DARRELLT. Fully convolutional networks for semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2015: 3431-3440.
[18] HUANG G B, RAMESH M, BERG T, et al. Labeled faces in the wild: A database for studying face recognition in unconstrained environments[R]. Amherst, MA: University of Massachusetts, Amherst, 2007: Technical Report 07-49.

()
(

[1]	WANG Zhihua, KANG Hongmei. Using splines on triangular mesh to solve PDE with nonhomogeneous boundary [J]. Journal of University of Science and Technology of China, 2020, 50(7): 901-905.
[2]	. A modified coherent diffraction algorithm based on the total variation algorithm for insufficient data [J]. Journal of University of Science and Technology of China, 2020, 50(4): 418-427.
[3]	WANG Xuhui, YAN Mingye, WU Meng. Suitable for the construction of degenerate smooth interpolated surfaces patches in isogeometric analysis [J]. Journal of University of Science and Technology of China, 2020, 50(3): 335-342.
[4]	XUE Song, JIANG Xinsheng, DUAN Jimiao, ZHANG Peili. An improved box-counting method for calculating image fractal dimension [J]. Journal of University of Science and Technology of China, 2018, 48(6): 504-511.
[5]	JI Mengyu,LIU Ligang. Perception-aware multi-view rendering optimization for indoor scenes [J]. Journal of University of Science and Technology of China, 2018, 48(2): 140-147.