sponsored byIEEEACMThe International Conference for High Performance 
Computing, Networking, Storage and Analysis
FacebookTwitterGoogle PlusLinkedInYouTubeFlickr

SCHEDULE: NOV 16-21, 2014

When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.

I/O Monitoring in a Hadoop Cluster

SESSION: Poster Reception


TIME: 5:15PM - 7:00PM

AUTHOR(S):Carson L. Wiens, Joshua M. C. Long, Joel R. Ornstein

ROOM:New Orleans Theater Lobby


As data sizes for scientific computations grow larger, more of these types of computations are bottlenecked by disk input and output, rather than processing speed. Monitoring reads and writes, therefore, has become an important component of distributed computing clusters. Our team investigated several different possibilities for monitoring I/O on a Hadoop cluster, including the Splunk app for HadoopOps, Ganglia, and log file output. Each of these three methods were evaluated for compatibility, ease of use, and display. Surprisingly, despite the fact that Splunk HadoopOps is made specifically for Hadoop clusters, other monitoring programs and techniques still proved to be useful. Using our monitoring tools, we were also able to observe input and output behavior over different cluster architectures.

Chair/Author Details:

Carson L. Wiens - Los Alamos National Laboratory

Joshua M. C. Long - Los Alamos National Laboratory

Joel R. Ornstein - Los Alamos National Laboratory

Add to iCal  Click here to download .ics calendar file

Add to Outlook  Click here to download .vcs calendar file

Add to Google Calendarss  Click here to add event to your Google Calendar