dualDup: A secure and reliable cloud storage framework to deduplicate the encrypted data and key

Cloud Storage Providers generally maintain a single copy of the identical data received from multiple sources to optimize the space. They cannot deduplicate the identical data when the clients upload the data in the encrypted form. To address this problem, recently, Duplicateless Encryption for Simple Storage (DupLESS) scheme is introduced in the literature. Besides, the data stored in the cloud is unreliable due to the possibility of data losses in remote storage environments. The DupLESS scheme, on the other hand, keeps both the key and the data on a single storage server, which is unreliable if that server goes down. In essence, the existing related works aim to handle either secure-deduplication or reliability limited to either key reliability or the data reliability. Hence, there is a need to develop a secure-deduplication mechanism that is not vulnerable to any malicious activity, semantically secures both data and key, and achieves the reliability. To address these problems, this paper proposes the dualDup framework that (a) optimizes the storage by eliminating the duplicate encrypted data from multiple users by extending DupLESS concept, and (b) securely distributes the data and key fragments to achieve the privacy and reliability using Erasure Coding scheme. The proposed approach is implemented in Python on the top of the Dropbox datacenter and corresponding results are reported. Experiments are conducted in a realistic environment. The results demonstrate that the proposed framework achieves reliability with an average storage overhead of 66.66% corresponding to the Reed-Solomon(3,2) codes. We validated through security analysis that the proposed framework is secure from insider and outsider adversaries. Moreover, dualDup framework provides all the aspects of deduplication, attack mitigation, key security and management, reliability, and QoS features as compared to other state-of-the-art deduplication techniques.