传统的遥感影像处理方法已无法有效应对当前遥感影像的3个“海量”问题,即日产量海量、单幅像素海量和可观测地物的类别及数据海量,使得多源海量遥感数据的利用率极其低下。为解决海量高分辨率遥感影像存储问题,提出了一种适用于云计算的高分辨率遥感影像存储组织结构,并对基于MapReduce框架的构建方法进行了详细的介绍。通过在Hadoop集群上对海量高分辨率遥感影像集进行的小影像集大文件构建方法实验与传统同类方式读取效率的对比,证明了本存储组织结构具有较高的扩展性,该小影像集大文件构建方法具有高效和高扩展的数据读写和处理能力,适合于作为处理海量高分辨率遥感影像的数据源。
Abstract
Traditional methods of processing remote sensing images could not effectively handle the mass daily production, mass pixel of single image, as well as the mass type and amount of objects. To solve the problem of image storage, we propose a structure for the storage of high-resolution remote sensing images in cloud computing environment, and expound the construction method based on MapReduce framework. We conducted experiments on large files of small image set in a Hadoop cluster and compared the image reading efficiency with that of traditional methods. The results proved that this storage structure has high scalability. Experiments also demonstrate this construction method has efficient reading/writing and processing ability.
关键词
云计算 /
高分辨率遥感影像 /
存储组织结构 /
MapReduce /
小影像集大文件 /
Hadoop
Key words
cloud computing /
high-resolution remote sensing image /
storage structure /
MapReduce /
large files of small image sets /
Hadoop
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 朱先强. 融合视觉显著特征的遥感图像检索研究[D]. 武汉:武汉大学, 2011. (ZHU Xian-qiang. Remote Sensing Imagery Retrieval Based on Integrating Visual Saliency Features[D]. Wuhan: Wuhan University, 2011.(in Chinese))
[2] WHITE T. Hadoop: The Definitive Guide[K]. US: O’Reilly Media, Inc. 2011.
[3] YANG C W, GOODCHILD M, HUANG Q Y, et al. Spatial Cloud Computing: How Can the Geospatial Sciences Use and Help Shape Cloud Computing [J]. International Journal of Digital Earth, 2011, 4(4):305-329.
[4] BLOWER J D. GIS in the Cloud Implementing a Web Map Service on Google App Engine [C]∥Proceedings of the 1st International Conference and Exhibition on Computing for Geospatial Research and Application, Washington, DC, USA, June 21-23, 2010: doi>10.1145/1823854.1823893.
[5] 康俊锋.云计算环境下高分辨率遥感影像存储与高效管理技术研究[D].杭州:浙江大学, 2011. (KANG Jun-feng. Technology of Efficient Management and Storage of High-resolution Remote Sensing Images in Cloud Computing Environment[D]. Hangzhou: Zhejiang University, 2011. (in Chinese))
[6] REN F, WANG J. Turning Remote Sensing to Cloud Services: Technical Research and Experiment[J]. Journal of Remote Sensing, 2012, 16(6):1331-1346.
[7] XIA Y, YANG X. Remote Sensing Image Data Storage and Search Method Based on Pyramid Model in Cloud[J]. Rough Sets and Knowledge Technology Lecture Notes in Computer Science, 2012, 7414: 267-275.
[8] CARY A, SUN Z G, HRISTIDIS V, et al. Experiences on Processing Spatial Data with MapReduce[C]∥Proceedings of 21st International Conference, SSDBM 2009, New Orleans, LA, USA, June 2-4, 2009: 302-319.
[9] LIU X, HAN J, ZHONG Y, et al. Implementing WebGIS on Hadoop: A Case Study of Improving Small File I/O Perfomance on HDFS[C]∥Proceedings of IEEE International Conference on Cluster Computing and Workshops. New Orleans, USA, August 31-September 4, 2009: 1-8.
[10]DEAN J, GHENMAWAT S. MapReduce: Simplified Data Processing on Large Clusters [C]∥Proceedings of Sixth Symposium on Operating System Design and Implementation, San Francisco, CA, USA, December 6-8, 2004: 10-23.
[11]GHEMAWAT S, GOBIOFF H, LEUNG S T. The Google File System [C]∥Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, Lake George, NY, USA, October 19-22, 2003: 29-43.
基金
国家自然科学基金项目(41271400);中央级公益性科研院所基本科研业务费(CKSF2014024/TB,CKSF2012044/TB,CKSF2014055/TB)
PDF(1802 KB)
