sponsored byIEEEACMThe International Conference for High Performance 
Computing, Networking, Storage and Analysis
FacebookTwitterGoogle PlusLinkedInYouTubeFlickr

SCHEDULE: NOV 16-21, 2014

When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.

IndexFS: Scaling File System Metadata Performance with Stateless Caching and Bulk Insertion

SESSION: Best Practices in File Systems

EVENT TYPE: Papers, Best Student Paper Finalists, Best Paper Finalists

TIME: 2:30PM - 3:00PM


AUTHOR(S):Kai Ren, Qing Zheng, Swapnil Patil, Garth Gibson



The growing size of modern storage systems is expected to soon achieve and exceed billions of objects, making metadata operation critical to the overall performance. Many existing parallel and cluster file systems only focus on providing highly parallel access to file data, but lack a scalable metadata service. In this paper, we introduce a middleware design called IndexFS that adds support to existing file systems such as HDFS and PVFS for high-performance operations on metadata and small files. IndexFS uses a tabular-based architecture that incrementally partitions the namespace at per-directory basis, preserving disk locality for small directories. We also propose two client caching techniques: bulk insertion for creation intensive workloads and stateless metadata caching for hot spot mitigation. By combining these techniques, we scaled IndexFS to 128 servers for various metadata workloads. Experiments demonstrate that its out-of-core metadata throughput out-performs PVFS by 50% to an order of magnitude.

Chair/Author Details:

Mark Gary (Chair) - Lawrence Livermore National Laboratory

Kai Ren - Carnegie Mellon University

Qing Zheng - Carnegie Mellon University

Swapnil Patil - Carnegie Mellon University

Garth Gibson - Carnegie Mellon University

Add to iCal  Click here to download .ics calendar file

Add to Outlook  Click here to download .vcs calendar file

Add to Google Calendarss  Click here to add event to your Google Calendar

Paper provided by the ACM Digital Library

Paper also available from IEEE Computer Society