面向Tabular库的数据模型及其查询问题

doi:10.3969/j.issn.0253-2778.2016.01.008

中国科学技术大学学报 ›› 2016, Vol. 46 ›› Issue (1): 56-65.DOI: 10.3969/j.issn.0253-2778.2016.01.008

• 论著 • 上一篇

面向Tabular库的数据模型及其查询问题

黄冬梅，孙乐，石少华，苏诚，赵丹枫

收稿日期:2015-08-27 修回日期:2015-09-29 接受日期:2015-09-29 出版日期:2015-09-29 发布日期:2015-09-29
通讯作者: 赵丹枫
作者简介:黄冬梅，女，1964年生，教授，研究方向：数据挖掘.E-mail: dmhuang@shou.edu.cn
基金资助:
国家自然科学基金（61272098）资助.

Tabular-oriented data model and its query issues

HUANG Dongmei, SUN Le, SHI Shaohua, SU Cheng, ZHAO Danfeng

1. School of Information, Shanghai Ocean University, Shanghai 201306, China; 2. East Sea Forecast Center of Oceanic Administration of China, Shanghai 200136, China

Received:2015-08-27 Revised:2015-09-29 Accepted:2015-09-29 Online:2015-09-29 Published:2015-09-29

摘要/Abstract

摘要： 信息化的发展使得数据存储及表示形式呈现出分布性、异构性的特点，不仅包括关系数据库、面向对象数据库等传统结构化数据，还包括Excel、CSV等不具有明确结构的特殊非结构化数据等，与此同时，其数据呈现了量大、更新快、可用性弱等大数据特点.然而使用无结构和半结构化文档组织和管理Excel等表单数据，存在着数据弱可控、弱可用、及访问效率差的问题.针对该类问题，本文以Excel文本为数据源，提出了一种新的面向Tabular库的关系数据模型并讨论了其上的查询及优化问题.首先，给出了Tabular表单数据的形式化定义，其次，设计PartiPath划分树实现表格的关系划分及结构转换，在关系模型的基础上，给出其数据模型及数据模式，再者，定义了表单数据上的基本查询问题及融合用户兴趣指数改进查询相似度指标，最后给出实验分析并作出总结.

关键词: Tabular库, 查询, 数据模型, PartiPath划分树, 关系模型

Abstract: With the rapid development of information technologies, data storage and representation of various sources, including not only the traditional structured data such as relational databases and object-oriented databases, but also those special unstructured data like Excel, CSV documents, manifest distributed and heterogeneous characteristics. Undoubtedly, all above data features high-volume, continuously-updating, low-usability, which falls into Big Data. However, the organization and management of Excel and other forms of data by using unstructured and semi-structured methods leads to a weakly-controllable, weakly-usable data structure with poor access efficiency. To solve this problem, this paper, taking Excel data source into consideration, aims to propose a new tabular-oriented relational data model and discusses Tabular querying and optimizing issues. Firstly, the formal definition of Tabular form data is given; secondly, PartiPath tree is designed to achieve structural transformation by tabular division and its relation schema as well; then its data model is presented. After that, four basic queries and their optimization by improved DICE with user interest similarity are described. Finally, the experiment was conducted and a conclusion was drawm.

Key words: Tabular repository, query, data model, PartiPath tree, relation model

中图分类号:

TP311

黄冬梅，孙乐，石少华，苏诚，赵丹枫. 面向Tabular库的数据模型及其查询问题[J]. 中国科学技术大学学报, 2016, 46(1): 56-65.

HUANG Dongmei, SUN Le, SHI Shaohua, SU Cheng, ZHAO Danfeng. Tabular-oriented data model and its query issues[J]. Journal of University of Science and Technology of China, 2016, 46(1): 56-65.

参考文献

［1］
Mayer-Schnberger V, Cukier K. Big Data: A Revolution That Will Transform How We Live, Work, and Think[M]. Boston: Houghton Mifflin Harcourt, 2013.
[2] China Argo News Letter, 2014, NO.2.
[3] Argo data center in China, http://www.argo.org.cn/
[4] Chui M, Brown B, Bughin J, et al. Big data: The next frontier for innovation, competition, and productivity[R]. McKinsey Global Institute, 2011.
[5] Hurst M. Layout and language: Challenges for table understanding on the web[EB/OL]. http://cgi.csc.liv.ac.uk/~wda2001/Papers/12_hurst_wda2001.pdf.
[6] Embley D W, Tao C, Liddle S W. Automating the extraction of data from HTML tables with unknown structure[J]. Data & Knowledge Engineering, 2005, 54(1): 3-28.
[7] Douglas S, Hurst M, Quinn D, et al. Using natural language processing for identifying and interpreting tables in plain text[J]. Proceedings of the Fourth Annual Symposium on Document Analysis and Information Retrieval, 1997, 21(2-4): 231-243.
[8] Gatterbauer W, Bohunsky P, Herzog M, et al. Towards domain-independent information extraction from web tables[C]// Proceedings of the16th International Conference on World Wide Web. Banff, Canada: ACM Press, 2007: 71-80.
[9] Ferrucci D, Lally A. UIMA: An architectural approach to unstructured information processing in the corporate research environment[J]. Natural Language Engineering, 2004, 10(3-4): 327-348.
[10] Pivk A, Sure Y, Cimiano P, et al. Transforming arbitrary tables into logical form with tartar[J]. Data & Knowledge Engineering, 2007, 60(3): 567-595.
[11] Pinto D, McCallum A, Wei X, et al. Table extraction using conditional random fields[C]// Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval. Tprpnto, Canada: ACM Press, 2003: 235-242.
[12] Duygulu P, Atalay V. A hierarchical representation of form documents for identification and retrieval[J]. International Journal on Document Analysis and Recognition, 1995, 5(1): 17-27.
[13] Tijerino Y A, Embley D W, Lonsdale D W, et al. Towards ontology generation from tables[J]. World Wide Web: Internet and Web Information Systems, 2005, 8(3): 261-285.
[14] Shigarov A O. Table understanding using a rule engine[J]. Expert Systems with Applications, 2015, 42(2): 929-937.
[15] Lopresti D, Nagy G. A tabular survey of automated table processing[C]// Graphics Recognition Recent Advances. Springer, 2000: 93-120.
[16] Wang X. Tabular abstraction, editing, and formatting[D]. University of Waterloo, Canada, 1996.
[17] Kumar T V V, Goel A, Jain N. Mining information for constructing materialised views[J]. International Journal of Information and Communication Technology, 2010, 2(4): 386-405.
[18] Frakes W B, Baeza-Yates R. Information Retrieval: Data Structure and Algorithms[M]. Upper Saddle River, USA: Prentice-Hall, 1992.

()
()

[1]	吴加禹，李永坤，许胤龙. 一种读写均衡的高性能键值存储系统[J]. 中国科学技术大学学报, 2020, 50(6): 825-831.
[2]	严俊涛，毕秀春，张曙光. 新形势下互联网金融对商业银行创新能力的影响[J]. 中国科学技术大学学报, 2019, 49(8): 645-654.
[3]	栾艳红，孙雪姣. 基于CP-net偏好的关系数据库的Top-k实现[J]. 中国科学技术大学学报, 2019, 49(2): 93-99.

面向Tabular库的数据模型及其查询问题

Tabular-oriented data model and its query issues

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

本文评价