Seminar: Adding Large-Stripe Parity to Archival Systems (Avani Wildani)

Digital archives are growing rapidly, necessitating stronger reliability measures than RAID to avoid data loss from device failure. Mirroring, a popular solution, is too expensive over time. We present a compromise solution that uses multi-level redundancy coding to reduce the probability of data loss from multiple simultaneous device failures. This approach handles small-scale failures of one or two devices efficiently while still allowing the system to survive rare-event, larger-scale failures of four or more devices.

In our approach, each disk is split into a set of fixed size disklets which are used to construct reliability stripes. To protect against rare event failures, reliability stripes are grouped into larger "über-groups," each of which has a corresponding "über-parity;"' über-parity is only used to recover data when disk failures overwhelm the redundancy in a single reliability stripe. Über-parity can be stored on a variety of devices such as NV-RAM and always-on disks to offset write bottlenecks while still keeping the number of active devices low.

Our calculations of failure probabilities found that the addition of über-groups allowed the system to absorb many more disk failures without data loss. Through discrete event simulation, we found that adding über-groups only negatively impacts performance when these groups need to be used for a rebuild. Since rebuilds using über-parity occur very rarely, they minimally impact system performance over time. Finally, we showed that robustness against rare events can be achieved for under 5% of total system cost.