基于ADMD融合策略的海洋大数据索引技术研究

doi:10.3969/j.issn.0253-2778.2015.10.003

中国科学技术大学学报 ›› 2015, Vol. 45 ›› Issue (10): 813-821.DOI: 10.3969/j.issn.0253-2778.2015.10.003

• 论著 • 上一篇

基于ADMD融合策略的海洋大数据索引技术研究

黄冬梅，孙乐，赵丹枫

上海海洋大学信息学院,上海 2 01306

收稿日期:2015-08-27 修回日期:2015-09-29 接受日期:2015-09-29 出版日期:2015-09-29 发布日期:2015-09-29
通讯作者: 黄冬梅
作者简介:黄冬梅(通讯作者)，女，1964年生，教授，研究方向：数据挖掘. E-mail:dmhuang@shou.edu.cn
基金资助:
国家自然科学基金项目（61272098）；上海市自然科学基金项目(13ZR1455800)

A composite index strategy for big marine data based on adaptive method of data merging strategy

HUANG Dongmei, SUN Le, ZHAO Danfeng

College of Information, Shanghai Ocean University, Shanghai 201306

Received:2015-08-27 Revised:2015-09-29 Accepted:2015-09-29 Online:2015-09-29 Published:2015-09-29

摘要/Abstract

摘要： 海洋数据具有多源、多类、多维、海量等特点，是一种典型的大数据，海洋大数据上的快速查询是该领域各类应用的基本需求.提高查询速度的关键是建立一个完善的索引结构，为此提出了一种基于时间间隔B+-tree和HSP-tree的多层索引架构ML-index(multi-layer index)，分别制定样本驱动的数据融合机制（adaptive method of data merging strategy）以确定分布式时态数据分区；并基于海洋数据特性、数据单元饱和度等，提出了一种自适应空间划分方法(adaptive space partition)，在此基础上建立HSP-tree作为辅助索引.实验验证在海洋数据模式下，提出的多层索引结构保证了海洋数据的查询速度，逼近线性的时间复杂度.

关键词: 海洋大数据, 时间间隔B+-tree索引, 自适应空间划分, AMDM

Abstract: Marine data fall easily into category of Big Data. A basic requirement for various marine monitoring applications is quick retrieval and the establishment of a sound index structure is of great importance. A multi-layer index (ML-index, for short) with regard to time interval B+-tree and hybrid space partition tree (HSP-tree, for short) was proposed. It employs the adaptive method of data merging strategy to optimize the primary key index (i.e. B+-tree). An adaptive space partition method was also proposed on the basis of data characteristics, and data unit capacity particular, for building secondary index, namely, HSP-tree. The experiment result shows that ML-index saves about 2/3 of the time in comparison with two state-of-the-art index methods.

Key words: marine data value function, time interval B+-tree, adaptive space partition, HSP-tree

中图分类号:

TP311

黄冬梅，孙乐，赵丹枫. 基于ADMD融合策略的海洋大数据索引技术研究[J]. 中国科学技术大学学报, 2015, 45(10): 813-821.

HUANG Dongmei, SUN Le, ZHAO Danfeng. A composite index strategy for big marine data based on adaptive method of data merging strategy[J]. Journal of University of Science and Technology of China, 2015, 45(10): 813-821.

参考文献

［1］
Mayer-Schnberger V, Cukier K. Big data: A Revolution That Will Transform How We Live, Work, and Think[M]. Houghton Mifflin Harcourt, 2013.
[2] 孟小峰, 慈祥. 大数据管理: 概念、技术与挑战[J]. 计算机研究已发展, 2013, 50(1): 146-169.
Meng X F, Ci X. Big data management: Concepts, techniques and challenges[J]. Journal of Computer Research and Development, 2013, 50(1): 146-169.
[3] 石绥祥, 雷波. 中国数字海洋: 理论与实践[M]. 北京: 海洋出版社, 2011.
[4] Liu Xiansan, Zhang Xin, Chi Tianhe, et al. Study on China digital ocean prototype system[C]// Proceedings of the 2009 WRI World Congress. Piscataway, USA: IEEE Press, 2009: 466-469.
[5] 周项敏, 王国仁. 基于关键位的高维空间划分策略[J]. 软件学报, 2004, 15(9): 1361-1374.
Zhou Xi M, Wang G R. Key dimension based high-dimensional data partition strategy[J]. Journal of Software, 2004, 15(9): 1361-1374.
[6] 黄冬梅, 杜艳玲, 贺琪. 混合云存储中海洋大数据迁移算法的研究[J]. 计算机研究与发展, 2014, 51(1): 199-205.
Huang D M, Du Y L, He Q. Migration algorithm for big marine data in hybrid cloud storage[J]. Journal of Integrative Plant Biology, 2014, 50(1): 199-205.
[7] Ren P, Liu W, Sun D, et al. Partition-based data cube storage and parallel queries for cloud computing[C]// Proceedings of the 9th International Conference on Natural Computation. Shengyang, China: IEEE Press, 2013: 1183-1187.
[8] Selvakumar C, Rathanam G J, Sumalatha M R. PDDS-Improving cloud data storage security using data partitioning technique[C]// Proceedings of the 3rd IEEE International Advance Computing Conference. Ghaziabad, India: IEEE Press, 2013: 7-11.
[9] 赵丹枫, 金顺福, 刘国华, 等. DAS模型下基于查询概率的密文索引技术[J]. 燕山大学学报, 2008, 32(6): 477-482.
Zhao D F, Jin S F, Liu G H, et al. A cryptograph index technology based on query probability in DAS model[J]. Journal of Yanshan University, 2008, 32(6): 477-482.
[10] 韩蕾, 孙徐湛, 吴志川, 等. MapReduce上基于抽样的数据划分最优化研究[J]. 计算机研究与发展, 2013, 50(S): 77-84.
Han L, Sun X Z, Wu Z C, et al. Optimization Study on sample based partition on MapReduce[J]. Journal of Computer Research and Development, 2013, 50(S): 77-84.

[11] Fox A, Eichelberger C, Hughes J, et al. Spatio-temporal indexing in non-relational distributed databases[C]// Proceeding of IEEE Conference on Big Data. Silicon Valley, IEEE Press, 2013: 291-299.
[12] Stantic B, Terry J, Topor R, et al. Indexing temporal data with virtual structure[C]// Proceedings of the 14th east European Conference on Advances in Databases and Information Systems. Springer, 2010: 591-594.
[13] Zhong Y Q, Fang J Y, Zhao X F. VegaIndexer: A Distributed composite index scheme for big spatio-temporal sensor data on cloud[C]// Proceedings of the IEEE Conference on Geoscience and Remote Sensing Symposium. Melbourne, Australia: IEEE Press, 2013: 1713-1716.
[14] Chen S, Ooi B C, Tan K L, et al. ST2B-tree: A self-tunable spatio-temporal b+-tree index for moving objects[C]// Proceeding of Conference on ACM Special Interest Group Conference on Management Of Data. Vancouver, Canada: IEEE Press, 2008: 29-42.
[15] Goil S, Nagesh H, Choudhary A. MAFIA: Efficient and scalable subspace clustering for very large data sets[C]// Proceedings of SIGKDD on Data Mining. San Diego: IEEE Press, 1999: 443-452.
[16] Kaufmann M, Amiri A, Vagenas P, et al., Timeline index: A unified data structure for processing queries on temporal data[C]// Proceedings of the ACM SIGMODInternational Conference on Management of Data. New York, USA: ACM Press, 2013:1173-1184.
[17] Chen S, Mario A, et al: ST2B-tree: a self-tunable spatio-temporal b+-tree index for moving objects. SIGMOD 2008:29-42.

()
()

[1]	吴加禹，李永坤，许胤龙. 一种读写均衡的高性能键值存储系统[J]. 中国科学技术大学学报, 2020, 50(6): 825-831.
[2]	蔡勇，陈红梅，. MapReduce环境下基于概念分层的概念格并行构造算法[J]. 中国科学技术大学学报, 2018, 48(4): 275-283.
[3]	王勇，黄志球，韦良芬，卢桂馥. 基于故障森林的组合测试故障定位研究[J]. 中国科学技术大学学报, 2018, 48(1): 28-34.
[4]	徐婉茹，杨盘隆. 移动广告用户传播能力评价与覆盖优化算法[J]. 中国科学技术大学学报, 2017, 47(7): 569-574.
[5]	常礼，马成学，高理富. 基于光电信号的玉米粒计数方法的研究[J]. 中国科学技术大学学报, 2015, 45(4): 275-279.
[6]	季铎，毕臣，蔡东风. 基于类别信息优化的潜在语义分析分类技术[J]. 中国科学技术大学学报, 2015, 45(4): 314-320.

基于ADMD融合策略的海洋大数据索引技术研究

A composite index strategy for big marine data based on adaptive method of data merging strategy

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

编辑推荐

Metrics

本文评价