Eric Lalonde will give a talk titled Distributed File System Profiling With Computer Animation.

Achieving performance, reliability, and scalability introduces a unique set of challenges for distributed file systems. Debugging issues can be daunting given the scale of these systems. Recent work in distributed profiling has focused on tracking individual requests as they traverse the system. This is useful for building models that characterize workload, but is less appropriate for expressing performance relationships at a more abstract level. To support debugging, there must be away for developers to have a comprehensive view of relevent system activities, and be able to focus that view on problem areas.

We present a distributed file system profiling method that supports such a view. Our approach is based on multi-tiered profiling, with a focus on portability, scalability, and providing an intuitive view of system behavior in real-time. Those file system metrics most relevant to performance analysis are viewable, such as load distribution and I/O characterization. As a result, our system offers a view of how performance is effected by inter-nodal relationships and management policies. To measure the effectiveness of our tool, we use it to evaluate the Ceph parallel file system. In the process we have discovered several important performance issues in Ceph, ranging from inefficiencies in small I/O operations to instabilities in the Ceph messaging layer.