基于语义相关区域搜索的图像标注

doi:10.3969/j.issn.0253-2778.2014.01.008

中国科学技术大学学报 ›› 2014, Vol. 44 ›› Issue (1): 67-73.DOI: 10.3969/j.issn.0253-2778.2014.01.008

• 论著 • 上一篇

基于语义相关区域搜索的图像标注

戴礼灿，俞能海

中国科学技术大学电子工程与信息科学系，安徽合肥 230027

收稿日期:2013-03-19 修回日期:2013-04-28 接受日期:2013-04-28 出版日期:2013-04-28 发布日期:2013-04-28

Image annotation by searching semantically related regions

DAI Lican, YU Nenghai

Department of Electronic Engineer and Information Science, USTC, Hefei 230027, China

Received:2013-03-19 Revised:2013-04-28 Accepted:2013-04-28 Online:2013-04-28 Published:2013-04-28
Contact: YU Nenghai
About author:DAI Lican, male, born in 1985, PhD student. Research filed: Information retrieval. E-mail: licand@mail.ustc.edu.cn
Supported by:
National Natural Science Foundation (60933013).

摘要/Abstract

摘要： 探讨了如何有效地利用互联网上大规模的图像和文本信息以数据驱动的方式来实现图像的自动标注，并提出了一种基于语义相关区域搜索的图像自动标注框架.该框架首先利用人工建立的视觉和文本知识库Image-Net来训练一组弱分类器；然后将学习好的弱分类器作用于分割后的图像区域块生成Region-level的语义特征表示用以在大规模的图像数据库中进行相关图像区域的搜索，最后从搜索结果的文本描述中通过聚类挖掘的方式产生最终的图像标注结果.对比于image-level的底层特征表示，基于分类学习的区域模块具有更强的语义表达能力和更好的鲁棒性，更容易抓住图像本身包含的多个目标的多重语义；从而使得该框架兼具了大规模数据驱动和传统基于分类算法的优点.大量web图像和公认的测试数据集上进行的实验结果证明了本文提出框架的有效性.

关键词: 图像标注, 区域搜索, 大规模数据驱动, 分类学习

Abstract: Based on abundant partially annotated images on the web, a novel framework for image annotation was proposed. By utilizing both the visual and textual knowledge of public available image database Image-Net, the proposed framework first learnt a set of weakly labeled visual concept classifiers, and then used the outputs of these learnt classifiers on image regions as descriptors to conduct the region-based search in a large scale image database for a query image. After that, search results mining and clustering was introduced to generate annotations to the query image. Compared with image-level representation, the proposed region-based semantic representation performs better at capturing images multi-objects/semantics. The proposed framework takes advantage of both traditional classification-based approaches and large scale data-driven approaches. Experimental results conducted on 24 million web images and challenging image database have demonstrated the effectiveness and efficiency of the proposed approach.

Key words: image annotation, region-based search, large scale data-driven, classeme learning

中图分类号:

TP391

戴礼灿，俞能海. 基于语义相关区域搜索的图像标注[J]. 中国科学技术大学学报, 2014, 44(1): 67-73.

DAI Lican, YU Nenghai. Image annotation by searching semantically related regions[J]. Journal of University of Science and Technology of China, 2014, 44(1): 67-73.

参考文献

［1］ Li J, Wang J Z. Automatic linguistic indexing of pictures by a statistical modeling approach［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(10): 1 075-1 088.
［2］ Jeon J, Lavrenko V, Manmatha R. Automatic image annotation and retrieval using cross-media relevance models［C］// Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Toronto, Canada: ACM Press, 2003: 119-126.
［3］ Fu H, Zhang Q, Qiu G P. Random forest for image annotation［C］// Proceedings of the 12th European Conference on Computer Vision. Florence, Italy: Springer, 2012: 86-99.
［4］ Zhang D S, Islam M M, Lu G J. A review on automatic image annotation techniques［J］. Pattern Recognition, 2012, 45(1): 346-362.
［5］ Wang X J, Zhang L, Li X R, et al. Annotating images by mining image search results［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(11): 1 919-1 932.
［6］ Wang X J, Zhang L, Liu M, et al. ARISTA - image search to annotation on billions of web photos［C］// IEEE Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE Press, 2010: 2 987-2 994.
［7］ Lu Y J, Zhang L, Liu J M, et al. Constructing concept lexica with small semantic gaps［J］. IEEE Transactions on Multimedia, 2010, 12(4): 288-299.
［8］ Li L J, Su H, Xing E P, et al. Object bank: A high-level image representation for scene classification & semantic feature sparsification［C］// Proceedings of the Advances in Neural Information Processing Systems. Vancouver, Canada, 2010: 1 378-1 386.
［9］ Mahajan D K, Sellamanickam S, Nair V. A joint learning framework for attribute models and object descriptions［C］// International Conference on Computer Vision. Barcelona, Spanish: IEEE Press, 2011: 1 227-1 234.
［10］ Yu F X, Ji R R, Tsai, M H, et al. Weak attributes for large-scale image retrieval［C］// Proceedings of IEEE International Conference on Computer Vision and Patten Recognition. Portland, USA: IEEE Press, 2012: 2 949-2 956.
［11］ Deng J, Dong W, Socher R, et al. ImageNet: A large-scale hierarchical image database［C］// IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA: IEEE Press, 2009: 248-255.
［12］ Torresani L, Szummer M, Fitzgibbon A W. Efficient object category recognition using classemes［C］// Proceedings of the 11th European Conference on Computer Vision: Part I. Heraklion, Greece: IEEE Press, 2010: 776-789.
［13］ van de Sande K E A, Gevers T, Snoek C G M. Evaluating color descriptors for object and scene recognition［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(9): 1 582-1 596.
［14］ Shechtman E, Irani M. Matching local self-similarities across images and videos［C］// IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, USA: IEEE Press, 2007: 1-8.
［15］ Bosch A, Zisserman A, Muoz X. Representing shape with a spatial pyramid kernel［C］// Proceedings of the 6th ACM International Conference on Image and Video Retrieval. New York: ACM Press, 2007:401-408.
［16］ Varma M, Babu B R. More generality in efficient multiple kernel learning［C］// Proceedings of the International Conference on Machine Learning. Montreal, Canada: ACM Press, 2009: 1 065-1 072.
［17］ Platt J C. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods［J］. Advances in Large Margin Classifiers, 1999: 61-74.
［18］ Deng Y N, Manjunath B S, Shin H. Color Image segmentation［C］// IEEE Conference on Computer Vision and Pattern Recognition. Fort Collis: IEEE Computer Society, 1999: 1-6.
［19］ Li X R, Chen L, Zhang L, et al. Image annotation by large-scale content-based image retrieval［C］// Proceedings of the 14th Annual ACM International Conference on Multimedia. Santa Barbara, USA: ACM Press, 2006: 607-610.
［20］ Zeng H J, He Q C, Chen Z, et al. Learning to cluster web search results［C］// Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Sheffield, UK: ACM Press, 2004: 210-217.

基于语义相关区域搜索的图像标注

Image annotation by searching semantically related regions

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics

本文评价