File and Storage Systems Benchmarking Workshop
The Storage and File Systems Benchmarking Workshop will be held May 19th at UC Santa Cruz. Attendance is by invitation only; if you haven't received an invitation but would like to attend, please contact one of the organizers.
Chair: Erez Zadok (Stony Brook University)
Hosts: Darrell Long and Ethan Miller (University of California, Santa Cruz)
Goal
A growing number of people in the community of storage and file system researchers and practitioners believe that the quality of benchmarking of those could be improved significantly. We often find that there is too little scientific or statistical method behind the current ad-hoc benchmarking, and many incompatible tools, leading to results that are hard to use meaningfully and compare. This workshop will bring together the best researchers and practitioners from industry and academia, representing all levels of the storage stack, along with statisticians and other interested parties. The main goals of the workshop are to educate everyone on the problems at hand, and try to reach some consensus as to the solutions that are needed. We expect a lot of interaction and discussion among the participants.
Directions
For information regarding maps and directions to the UCSC Campus, please visit this Web link. A customized Google Map to get you from the base of campus to the Engineering 2 building is also available. The workshop will be held in the Engineering 2 building, room 180 (the entrance to the room is at the far end of the ground floor in the picture).
We also have directions from the San Jose airport (SJC) to the Scotts Valley Hilton and from the Scotts Valley Hilton to UC Santa Cruz.
Parking
At the Main Entrance and also the West Entrance to campus, we will have signs posted that read "*SSRC Retreat*." Please follow these signs through campus to the Core West Parking Structure at the corner of Heller and McLaughlin. (The shortest route to the parking garage is through the west entrance of campus.)
UCSC's Transportation and Parking Services (TAPS) will have two attendants stationed on the 4th floor of the Core West parking garage. Please drive into the garage and follow the signs posted within the structure directing retreat attendees to drive up to the 4th floor to pick up your parking permits.
Important Note: You will need to tell the parking attendant that you are attending the SSRC Retreat. They will then provide you with a parking permit (at no cost to you) and direct you to park on the top floor of the structure. At this time the attendants will also give direction to guests on how to get to the Engineering Courtyard and Simularium.
The parking attendants will be present in the Core West Structure from 7:30 am - 9:30 am each morning. Should any of you arrive after the attendants have departed, you may be able to purchase a permit from the pay station located on the 2nd floor. However, these permits are only available for the pay stalls located on the 2nd floor during weekdays, otherwise you will need to return to the base of campus and purchase a parking permit at the Kiosk, or the TAPS Sales Office.
A link showing the location of the Core West Parking Structure can be found at http://maps.ucsc.edu/cdcorewest.html
Lastly, please know that you can always go to the parking & information kiosk located on the right side of the road after you enter the main campus entrance. They can provide you with a campus map and directions to the parking structure.
Subpages
You can edit the main page by editing one of these subpages.
Agenda: Monday, May 19, 2008
This is a tentative agenda; all of the listed speakers have been confirmed, but the schedule is subject to change as we have more speaker confirmations.
8:00 AM - 8:45 AM: Continental Breakfast (provided)
8:45 AM - 9:30 AM: Welcome
Welcome and Introduction (8:45 AM - 9:10 AM)
Ethan Miller (Univ. of California, Santa Cruz) and Erez Zadok (Stony Brook University)
slidesSummary of the Benchmarking Article in ACM Transactions on Storage (9:10 AM - 9:30 AM)
Avishay Traeger (Stony Brook University)
slides
9:30 AM - 10:30AM: Existing Benchmarks (1)
The iozone Benchmark (9:30 AM - 9:50 AM)
Don Capps (NetApp)
slidesFileBench (9:50 AM - 10:10 AM)
Spencer Shepler (Sun)
slidesSPECsfs Benchmark - The first 10 years and beyond (10:10 AM - 10:30 AM)
Don Capps (NetApp)
slides
10:30 AM - 10:50AM: Break
10:50 AM - 11:50AM: Traces
SNIA IOTTA TWG: Trace Collection and Availability (10:50 AM - 11:10 AM)
Geoff Kuenning (Harvey Mudd College)
slidesDataSeries: Fast, Efficient Trace File Storage and Analysis (11:10 AM - 11:30 AM)
Eric Anderson (HP)
slidesButtress: A Cautionary Tale (11:30 AM - 11:50 AM)
Eric Anderson (HP)
slides
11:50 AM - 1:00PM: Lunch (provided): discuss morning sessions
1:00 PM - 2:00PM: Industry Experiences (1)
Experiences with Storage Benchmarking in Virtualization (1:00 PM - 1:20 PM)
Richard McDougal, Devaki Kulkarni, Irfan Ahmad (VMware)
slides paperDevice Characterization Methodologies for Novel Storage Devices (1:20 PM - 1:40 PM)
Daniel Ellard and Jiri Schindler (NetApp)
slidesMicrosoft Clients and File Servers (1:40 PM - 2:00 PM)
Jeff Fuller (Microsoft)
slides
2:20 PM - 2:20 PM: Break
2:20 PM - 3:00PM: Industry Experiences (2)
ZFS Benchmarking (2:20 PM - 2:40 PM)
Eric Kustarz (Sun)
slidesBenchmarking Petabyte-scale Systems (2:40 PM - 3:00 PM)
Ethan Miller (Univ. of California, Santa Cruz)
3:00 PM - 3:20 PM: Break
3:20 PM - 4:00 PM: Statistical Methods for Benchmarking Storage
Round table discussion with Herbie Lee (UC Santa Cruz), Erez Zadok (Stony Brook University), Don Capps (NetApp), Sorin Faibish (EMC)
4:00 PM - 6:00 PM: Working Groups
- Benchmark standardization
- "Best practices" for storage benchmarking
- Trace collection and distribution (including anonymization)
6:00 PM - 6:30PM: Next Steps
Discussion of the next steps, including writing up the results from the workshop for ;login: and the workshop report
6:30 PM: Dinner (provided)
Dinner will be at the Hula Grill in downtown Santa Cruz.
Attendee List
This is a partial attendee list, and will be filled in as people respond.
Eric Anderson (HP) *
Don Capps (NetApp) *
Daniel Ellard (NetApp) *
Sorin Faibish (EMC) *
Jeff Fuller (Microsoft)
Gary Grider (Los Alamos National Laboratory)
Devaki Kulkarni (VMware)
Eric Kustarz (Sun) *
Chris Lumb (Data Domain)
Tim Moore (agami)
Brad Morrey (HP) *
Jim Pinkerton (Microsoft) *
Julian Satran (IBM)
Sanjay Sawhney (Symantec)
Jiri Schindler (NetApp) *
Spencer Shepler (Sun) *
Andrew Wilson (Sun)
Zheng Yang (Data Domain)
* Presenting
