sponsored byIEEEACMThe International Conference for High Performance 
Computing, Networking, Storage and Analysis
FacebookTwitterGoogle PlusLinkedInYouTubeFlickr

SCHEDULE: NOV 16-21, 2014

When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.

Parallel Deep Neural Network Training for Big Data on Blue Gene/Q

SESSION: Machine Learning and Data Analytics


TIME: 11:00AM - 11:30AM


AUTHOR(S):I-Hsin Chung, Tara Sainath, Bhuvana Ramabhadran, Michael Picheny, John Gunnels, Vernon Austel, Upendra Chauhari, Brain Kingsbury



Deep Neural Networks (DNNs) have recently been shown to significantly outperform existing machine learning techniques in several pattern recognition tasks. The biggest drawback to DNNs is the enormous training time - often 10x slower than conventional technologies. While training time can be mitigated by parallel computing algorithms and architectures, these algorithms often suffer from the cost of inter-processor communication bottlenecks. In this paper, we describe how to enable parallel DNN training on the IBM Blue Gene/Q (BG/Q) computer system using the data-parallel Hessian-free 2nd-order optimization algorithm. BG/Q, with its excellent inter-processor communication characteristics, is an ideal match for the HF algorithm. The paper discusses how issues regarding programming model and data-dependent imbalances are addressed. Results on large-scale speech tasks show that the performance on BG/Q scales linearly up to 4096 processes with no loss in accuracy, allowing us to train DNNs with millions of training examples in a few hours.

Chair/Author Details:

Hank Childs (Chair) - University of Oregon and Lawrence Berkeley National Laboratory

I-Hsin Chung - IBM Corporation

Tara Sainath - IBM Corporation

Bhuvana Ramabhadran - IBM Corporation

Michael Picheny - IBM Corporation

John Gunnels - IBM Corporation

Vernon Austel - IBM Corporation

Upendra Chauhari - IBM Corporation

Brain Kingsbury - IBM Corporation

Add to iCal  Click here to download .ics calendar file

Add to Outlook  Click here to download .vcs calendar file

Add to Google Calendarss  Click here to add event to your Google Calendar

Paper provided by the ACM Digital Library

Paper also available from IEEE Computer Society