sponsored byIEEEACMThe International Conference for High Performance 
Computing, Networking, Storage and Analysis
FacebookTwitterGoogle PlusLinkedInYouTubeFlickr

SCHEDULE: NOV 16-21, 2014

When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.

Big Data Analytics on Object Stores: A Performance Study

SESSION: Poster Reception


TIME: 5:15PM - 7:00PM

AUTHOR(S):Lukas Rupprecht, Rui Zhang, Dean Hildebrand

ROOM:New Orleans Theater Lobby


Object Stores provide a cheap solution for storing, sharing, and accessing globally distributed, scientific datasets. Their scalability and high reliability makes them well suited as archive storage for data produced by HPC clusters. To make this archive active, i.e. to process the archived data, a separate analytics cluster is needed which adds overhead in terms of cost, maintenance, and performance as data needs to be copied between two systems. Running the analytics directly on the Object Store can greatly simplify the archive usage. We study the problems that arise when running an analytics system (Hadoop MapReduce) on top of an Object Store (OpenStack Swift). We conduct a set of detailed micro- and macro benchmarks and find that the static, consistent hashing based, object-node mapping adds significant overhead when object writes are not local. Additionally, repeated authentication calls slow down jobs which has a high performance impact, especially on interactive workloads.

Chair/Author Details:

Lukas Rupprecht - Imperial College London

Rui Zhang - IBM Corporation

Dean Hildebrand - IBM Corporation

Add to iCal  Click here to download .ics calendar file

Add to Outlook  Click here to download .vcs calendar file

Add to Google Calendarss  Click here to add event to your Google Calendar