Speaker

Dean Hildebrand (IBM Research)

Abstract

Heterogeneous and scalable access to remote data is a critical enabling feature of widely distributed collaborations. Parallel file systems provide high I/O throughput to large data stores, but are limited to particular operating system and hardware platforms, lack seamless integration and modern security features, and suffer from poor remote access performance.

To resolve these problems, NFSv4 architects are designing pNFS, a standard extension that provides direct storage access to a diversity of parallel file systems while preserving operating system and hardware platform independence. pNFS uses storage-specific layout drivers to distribute I/O across the bisectional bandwidth of a storage network. Storage-specific layout drivers allow universal storage protocol support and flexible security and data access semantics, but may diminish heterogeneity and transparency.

In this talk, I describe my research that analyzes the pNFS architecture. I then introduce Direct-pNFS, which bridges the transparency and heterogeneity gap between applications and data while matching (and even outperforming) the native parallel file system client across a range of workloads.