Data deduplication is an essential and critical component of backup systems. Essential, because it reduces storage space requirements, and critical, because the performance of the entire backup operation depends on its throughput. Traditional backup workloads consist of large data streams with high locality, which existing deduplication techniques require to provide reasonable throughput.
We have developed Extreme Binning, a scalable deduplication technique for non-traditional backup workloads that are made up of individual files with no locality among consecutive files in a given window of time. Due to lack of locality, existing techniques perform poorly on these workloads. Extreme Binning exploits file similarity, and makes only one disk access for chunk lookup per file, which gives reasonable throughput. Multi-node backup systems built with Extreme Binning scale gracefully with the amount of input data; more backup nodes can be added to boost throughput. Each file is allocated using a stateless routing algorithm to only one node, allowing for maximum parallelization, and each backup node is autonomous with no dependency across nodes, making data management tasks robust with low overhead.
A partnership between academia and industry exploring and developing new technologies and techniques to improve the manageability, scalability, security, reliability, longevity, and performance of storage systems.
CRSS facilitates collaboration in research and education, and provides pathways to simplify direct transfer of university developed ideas, research results, and technology to its industrial sponsors, helping them to improve their competitive posture in the global marketplace.
CRSS conducts research in a wide range of storage-related fields and applied security, including archival storage, scalable distributed indexing and non-hierarchical file systems, large-scale distributed storage systems, file systems for next-generation storage devices, and data deduplication.
CRSS is looking for talented students who want to study for an M.S. or Ph.D. at UC Santa Cruz! Grad students at CRSS work closely with faculty and other students, as well as with local industry. Our graduates typically have multiple job offers, whether from industry, government, or academia.