An IoT-oriented real-time storage mechanism for massive small files based on Swift. (12th February 2020)

• Record Type: Journal Article
• Authors: Zhu, Dongjie; Du, Haiwen; Wang, Yuhua; Peng, Xuan
• Is Part Of: International journal of embedded systems. Volume 12:Number 1(2020)
• Page Start: 72
• Page End: 80
• Publication Date: 2020-02-12
• Subjects: cloud storage -- internet of things -- IoT -- massive small files -- object storage -- Swift; Embedded computer systems -- Periodicals; 004.16
• Languages: English
• ISSNs: 1741-1068

Abstract

In the internet of things (IoT), large amounts of small files are generated from various structure sensors in cloud storage platforms. Real-time storage of massive small files will put great pressures to the traditional file system. By studying the real-time storage strategy of massive small files based on the object storage architecture Swift, we proved a unique aggregation storage strategy, called sequential data aggregation strategy (SDAS), for storage of small files. We designed a two-level index structure to improve writing rate by transferring randomly write to sequentially write. To improve overall data access efficiency and solve the performance bottleneck of proxy node, we utilise a file's potential relevance of timing to prefetch related files that are merged in other blocks. Simulation results show that relative to the original system, SDAS has shorter response time of writing operation, lower cost of index maintenance cost, more balanced node load and 30% reduction in disk overhead.