- Home
- Register
- Attend
- Program
- Technical Program Overview
- SC14 Schedule
- Awards
- Birds-of-a-Feather Sessions (BOFs)
- Emerging Technologies
- Invited Talks
- Panels
- Papers
- Posters
- Scientific Visualization Showcase
- Tutorials
- Workshops
- Doctoral Showcase Program
- HPC Matters Plenary
- Keynote
- SC14 Archive
- SC14 Conference Program
- Tech Program Receptions
- Exhibit
- Engage
- Media
- Media Overview
- Media Releases
- Announcing the Second Test of Time Award Winner
- CDC to Present at Supercomputing 2014
- Finalists Compete for Coveted ACM Gordon Bell Prize in High Performance Computing
- Four Ways Supercomputing Is Changing Lives: From Climate Modeling to Manufacturing Consumer Goods
- Join the Student Cluster Competition
- New Orleans Becomes Home to Fastest Internet Hub in the World
- SC14 Announces New Plenary to Focus on the Importance of Supercomputers in Society
- SC14 Registration Opens, Technical Program Goes Live
- Supercomputing 2014 Recognizes Outstanding Achievements in HPC
- Supercomputing 2014 Sets New Records
- Supercomputing Invited Plenary Talks
- Supercomputing Unveils Ground-Breaking Innovations and the World’s Fastest Computer Network
- World’s Fastest Computer Network Coming to New Orleans
- SC14 Logo Usage
- SC14 Media Partners
- Social Media
- Newsletters
- SC14 Blog
- Opening Press Briefing
- SC Photograph and Film Acceptable Use Policy
- Media Registration
- Video Gallery
- SCinet
SCHEDULE: NOV 16-21, 2014
When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.
Big Data Analytics on Object Stores: A Performance Study
SESSION: Poster Reception
EVENT TYPE: Posters
TIME: 5:15PM - 7:00PM
AUTHOR(S):Lukas Rupprecht, Rui Zhang, Dean Hildebrand
ROOM:New Orleans Theater Lobby
ABSTRACT:
Object Stores provide a cheap solution for storing, sharing, and accessing globally distributed, scientific datasets. Their scalability and high reliability makes them well suited as archive storage for data produced by HPC clusters. To make this archive active, i.e. to process the archived data, a separate analytics cluster is needed which adds overhead in terms of cost, maintenance, and performance as data needs to be copied between two systems. Running the analytics directly on the Object Store can greatly simplify the archive usage. We study the problems that arise when running an analytics system (Hadoop MapReduce) on top of an Object Store (OpenStack Swift). We conduct a set of detailed micro- and macro benchmarks and find that the static, consistent hashing based, object-node mapping adds significant overhead when object writes are not local. Additionally, repeated authentication calls slow down jobs which has a high performance impact, especially on interactive workloads.
Chair/Author Details:
Lukas Rupprecht - Imperial College London
Rui Zhang - IBM Corporation
Dean Hildebrand - IBM Corporation
Click here to download .ics calendar file
