A Large-Scale Object-Based Active Storage Platform for Data Analytics in the Internet of Things

In this paper, we propose a large-scale object-based storage platform, named Gem, for data analytics in the Internet of Things (IoT). In Gem, a region covered by an IoT network is partitioned into sub-regions, each of which can be identified by a unique ID and known to all participants, which is automatic and economical. Gem can preserve object locality using type and location sensitive hashing, as well as dynamically distribute objects among a server cluster to keep load balancing. All data from the IoT can be stored as objects with attributes, methods and policies in Object Store Devices (OSDs). For some applications such as data analytics, application-specific operations are executed by OSDs, and only the results are returned to clients, rather than data files are read by the clients. Thus, the platform Gem is able to greatly reduce the overhead of data analytics applications in the IoT.