An agricultural data storage mechanism based on HBase. (23rd August 2019)

• Record Type: Journal Article
• Authors: Li, Changyun; Zhang, Qingqing; He, Pinjie; Wang, Zhibing; Chen, Li
• Is Part Of: International journal of information and communication technology. Volume 14:Number 4(2019)
• Page Start: 456
• Page End: 469
• Publication Date: 2019-08-23
• Subjects: agricultural big data -- data buffer structure -- HBase -- two-level indexing strategy; Information technology -- Periodicals; Computer science -- Periodicals; Telecommunication -- Periodicals; 004.05
• Languages: English
• ISSNs: 1466-6642

Abstract

With the development of agricultural space-time localisation, sensor network and cloud computing, the amount of agricultural data is increasing rapidly and the data structure becomes more complicated and changeable. Currently, the widely used agricultural database is the relational database. This database handles a large amount of data with very limited throughput and is not suitable for organisation and management of distributed data. HBase is a non-relational database of distributed file storage built on Hadoop platform. HBase is suitable for unstructured data storage database and it can handle large volume of data with high scalability. To better store agricultural big data in HBase, we propose a special agricultural data buffer structure which stores data based on the data category and a two-level indexing memory organisation strategy on HBase. The method proposed saves more than a quarter of the time compared to traditional buffering methods. Experimental results show the higher efficiency of the agricultural data buffer structure and memory-organisation strategy.