MCDS: Large-scale mobile communication data computation on just a PC

doi:10.3969/j.issn.0253-2778.2016.01.006

Abstract

Abstract: Mobile data has the characteristics of high volume, variety, velocity and value. Mobile communication data is an important part of mobile data, and it has great research value. It is of tremendous significance to efficiently store and retrieve mobile data. At present, utilizing parallel technology to perform data mining has become the main stream, but the technology is very costly in terms of hardware, and code debugging and optimization of parallel algorithms is difficult. A mobile communication data processing system operational on a single PC was proposed. MCDS is based on GraphChi, and improves GraphChi from 3 aspects: data format, sharding mechanism and memory replacement algorithm. Experimental results verify the effectiveness of MCDS, and it provides a feasible experimental environment for mobile communication data mining.

Key words: mobile data, mobile data mining, mobile communication data, MCDS

CLC Number:

TP391

LIU Zhipeng. MCDS: Large-scale mobile communication data computation on just a PC[J]. Journal of University of Science and Technology of China, 2016, 46(1): 36-46.

References

［1］
Kang U, Tong H H, Sun J M, et al. Gbase: An efficient analysis platform for large graphs [J]. The VLDB Journal, 2012, 21(5): 637-50.
[2] Kang U, Tong H H, Sun J M, et al. Gbase: A scalable and general graph management system[C]// Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Diego, USA: ACM Press, 2011: 1091-1099.
[3] Malewicz G, Austern M H, Bik A J C, et al. Pregel: A system for large-scale graph processing[C]// Proceedings of the ACM SIGKDD International Conference on Management of Data. Calgary, Canada: ACM Press, 2010: 135-146.
[4] Low Y C, Bickson D, Gonzalez J, et al. Distributed GraphLab: a framework for machine learning and data mining in the cloud[J]. Proceedings of the VLDB Endowment, 2012, 5(8): 716-27.
[5] Gonzalez J E, Low Y C, Gu H J, et al. PowerGraph: Distributed graph-parallel computation on natural graphs[C]// Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation. Berkeley, USA: USENIX Association, 2012: 1-5.
[6] Kyrola A, Blelloch G, Guestrin C. GraphChi: Large-scale graph computation on just a PC[C]// Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation. Berkeley, USA: USENIX Association, 2012: 31-46.
[7] Han W S, Lee S, Park K, et al. TurboGraph: A fast parallel graph engine handling billion-scale graphs in a single PC[C]// Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM Press, 2013: 77-85.
[8] Holme P, Saram倞ki J. Temporal networks[J]. Physics Reports, 2012, 519(3): 97-125.
[9] Megiddo N, Modha D S. ARC: A Self-Tuning, Low Overhead Replacement Cache[C]// Proceedings of the 2nd USENIX Conference on File and Storage Technologies. San Francisco, USA: USENIX Association, 2003: 115-130.
[10] Eagle N, Pentland A. Reality mining: sensing complex social systems[J]. Personal and Ubiquitous Computing, 2006, 10(4): 255-68.
[11] Ficek M, Kencl L. Spatial extension of the reality mining dataset[C]// Proceedings of the International Conference on Mobile Adhoc and Sensor Systems. San Francisco: IEEE Press, 2010: 666-673.
[12] Pentland A. Reality Mining of Mobile Communications: Toward a New Deal on Data [M]. Springer, 2009.
[13] Kiukkonen N, Blom J, Dousse O, et al. Towards rich mobile phone datasets: Lausanne data collection campaign[C]// Proceedings of the International Conference on Pervasive Services. Berlin, Germany: ACM Press, 2010: 1-7.

()
()

[1]	NIE Tengfei, YU Haisuo, DU Shaofu. Agriculture supply chain optimization based on supply and demand uncertainty with government subsidy policies [J]. Journal of University of Science and Technology of China, 2017, 47(3): 267-273.
[2]	HU Chao, ZHANG Juyong. Shape approximation via sparse optimization [J]. Journal of University of Science and Technology of China, 2017, 47(9): 713-720.
[3]	LI Yan, ZHOU Shizhe, DENG Jiansong. Face sorting and stripe texture mapping of triangle mesh based on spectral decomposition [J]. Journal of University of Science and Technology of China, 2017, 47(9): 721-728.
[4]	HUO Jiamian, XIE Jingui. Modelling and forecasting of call center arrival process [J]. Journal of University of Science and Technology of China, 2017, 47(9): 770-777.
[5]	ZHAN Yongkun, YANG Wenfei, ZHANG Tianzhu. Relation aware network for weakly-supervised temporal action localization [J]. Journal of University of Science and Technology of China, 2021, 51(10): 753-765.
[6]	Liang Qipeng, Zhu Qiaohui, Kang Yu, Zhao YunBo. Estimation based approximating control for wireless networked control systems [J]. Journal of University of Science and Technology of China, 2021, 51(4): 327-334.
[7]	Li Huicong, Fu Xiaoming∗ . Anisotropic surface meshing using locally isometric embedding [J]. Journal of University of Science and Technology of China, 2020, 50(12): 1460-1471.
[8]	YU Jun, LI Yajie, CHENG Lilei, LIAN Shun, TAN Chang, DING Decheng, Liu Qi. Research and practice of plagiarism detection in program code assignments by college students [J]. Journal of University of Science and Technology of China, 2020, 50(8): 1048-1057.
[9]	ZHANG Wenjing, BAN Zhijie. Citation network’s influence maximization algorithm based on global influence [J]. Journal of University of Science and Technology of China, 2020, 50(8): 1058-1063.
[10]	REN Min, XU Ling, WANG Feng, WU Chao. Campus encyclopedia platform based on knowledge recommendation [J]. Journal of University of Science and Technology of China, 2020, 50(8): 1072-1076.
[11]	SUN Lei, ZHANG Yining, XUE Yanfang, QIAO Lishan, ZHANG Limei. Adaptive functional connectivity network learning and application in brain disorders identification [J]. Journal of University of Science and Technology of China, 2020, 50(8): 1102-1109.
[12]	BAI Jiangbo, YANG Yang, ZHANG Wensheng. A simulation of the synthetic aperture radar image based on improved CycleGAN [J]. Journal of University of Science and Technology of China, 2020, 50(8): 1181-1186.
[13]	XIN Shouyu, ZHENG Ruirui, ZHOU Yu, LIU Wenpeng, HE Jianjun. A one-shot learning algorithm using support set information during training [J]. Journal of University of Science and Technology of China, 2020, 50(8): 1187-1192.
[14]	WANG Zhihua, KANG Hongmei. Using splines on triangular mesh to solve PDE with nonhomogeneous boundary [J]. Journal of University of Science and Technology of China, 2020, 50(7): 901-905.
[15]	GONG Lejun , ZHOU Shehai, CHENG Yifei, GAO Zhihong, LI Huakang. Identification of PBMC-related cells of single cell RNA sequence data [J]. Journal of University of Science and Technology of China, 2020, 50(7): 1013-1018.