Pergamum: Replacing Tape with Energy Efficient, Reliable, Disk-Based Archival Storage

Appeared in Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST '08).

Abstract

As the world moves to digital storage for archival purposes, there is an increasing demand for reliable, low-power, cost-effective, easy-to-maintain storage that can still provide adequate performance for information retrieval and auditing purposes. Unfortunately, no current archival system adequately fulfills all of these requirements. Tape-based archival systems suffer from poor random access performance, which prevents the use of inter-media redundancy techniques and auditing, and requires the preservation of legacy hardware. Many disk-based systems are ill-suited for long-term storage because their high energy demands and management requirements make them cost-ineffective for archival purposes.

Our solution, Pergamum, is a distributed network of intelligent, disk-based, storage appliances that stores data reliably and energy-efficiently. While existing MAID systems keep disks idle to save energy, Pergamum adds NVRAM at each node to store data signatures, metadata, and other small items, allowing deferred writes, metadata requests and inter-disk data verification to be performed while the disk is powered off. Pergamum uses both intra-disk and inter-disk redundancy to guard against data loss, relying on hash tree-like structures of algebraic signatures to efficiently verify the correctness of stored data. If failures occur, Pergamum uses staggered rebuild to reduce peak energy usage while rebuilding large redundancy stripes. We show that our approach is comparable in both startup and ongoing costs to other archival technologies and provides very high reliability. An evaluation of our implementation of Pergamum shows that it provides adequate performance.

Available for download:

Bibtex entry

@inproceedings{
  author       = {Mark W. Storer and Kevin Greenan and Ethan L. Miller and Kaladhar Voruganti},
  title        = {Pergamum: Replacing Tape with Energy Efficient, Reliable, Disk-Based Archival
Storage},
  howpublished = {Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST
'08)},
  pages        = {1-16},
  month        = feb,
  year         = {2008},
}