Archive storage system design for long-term storage of massive amounts of data

A dramatic shift is underway in how organizations use computer storage. This shift will have a profound impact on storage system design. The requirement for storage of traditional transactional data is being supplemented by the necessity to store information for long periods. In 2005, a total of 2,700 petabytes of storage was allocated worldwide for information that required long-term retention, and this amount is expected to grow to an estimated 27,200 petabytes by 2010. In this paper, we review the requirements for long-term storage of data and describe an innovative approach for developing a highly scalable and flexible archive storage system using commercial off-the-shelf (COTS) components. Such a system is expected to be capable of preserving data for decades, providing efficient policy-based management of the data, and allowing efficient search and access to data regardless of data content or location.