- Home
- Register
- Attend
- Program
- Technical Program Overview
- SC14 Schedule
- Awards
- Birds-of-a-Feather Sessions (BOFs)
- Emerging Technologies
- Invited Talks
- Panels
- Papers
- Posters
- Scientific Visualization Showcase
- Tutorials
- Workshops
- Doctoral Showcase Program
- HPC Matters Plenary
- Keynote
- SC14 Archive
- SC14 Conference Program
- Tech Program Receptions
- Exhibit
- Engage
- Media
- Media Overview
- Media Releases
- Announcing the Second Test of Time Award Winner
- CDC to Present at Supercomputing 2014
- Finalists Compete for Coveted ACM Gordon Bell Prize in High Performance Computing
- Four Ways Supercomputing Is Changing Lives: From Climate Modeling to Manufacturing Consumer Goods
- Join the Student Cluster Competition
- New Orleans Becomes Home to Fastest Internet Hub in the World
- SC14 Announces New Plenary to Focus on the Importance of Supercomputers in Society
- SC14 Registration Opens, Technical Program Goes Live
- Supercomputing 2014 Recognizes Outstanding Achievements in HPC
- Supercomputing 2014 Sets New Records
- Supercomputing Invited Plenary Talks
- Supercomputing Unveils Ground-Breaking Innovations and the World’s Fastest Computer Network
- World’s Fastest Computer Network Coming to New Orleans
- SC14 Logo Usage
- SC14 Media Partners
- Social Media
- Newsletters
- SC14 Blog
- Opening Press Briefing
- SC Photograph and Film Acceptable Use Policy
- Media Registration
- Video Gallery
- SCinet
SCHEDULE: NOV 16-21, 2014
When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.
Parallel Deep Neural Network Training for Big Data on Blue Gene/Q
SESSION: Machine Learning and Data Analytics
EVENT TYPE: Papers
TIME: 11:00AM - 11:30AM
SESSION CHAIR: Hank Childs
AUTHOR(S):I-Hsin Chung, Tara Sainath, Bhuvana Ramabhadran, Michael Picheny, John Gunnels, Vernon Austel, Upendra Chauhari, Brain Kingsbury
ROOM:388-89-90
ABSTRACT:
Deep Neural Networks (DNNs) have recently been shown to significantly outperform existing machine learning techniques in several pattern recognition tasks. The biggest drawback to DNNs is the enormous training time - often 10x slower than conventional technologies. While training time can be mitigated by parallel computing algorithms and architectures, these algorithms often suffer from the cost of inter-processor communication bottlenecks. In this paper, we describe how to enable parallel DNN training on the IBM Blue Gene/Q (BG/Q) computer system using the data-parallel Hessian-free 2nd-order optimization algorithm. BG/Q, with its excellent inter-processor communication characteristics, is an ideal match for the HF algorithm. The paper discusses how issues regarding programming model and data-dependent imbalances are addressed. Results on large-scale speech tasks show that the performance on BG/Q scales linearly up to 4096 processes with no loss in accuracy, allowing us to train DNNs with millions of training examples in a few hours.
Chair/Author Details:
Hank Childs (Chair) - University of Oregon and Lawrence Berkeley National Laboratory
I-Hsin Chung - IBM Corporation
Tara Sainath - IBM Corporation
Bhuvana Ramabhadran - IBM Corporation
Michael Picheny - IBM Corporation
John Gunnels - IBM Corporation
Vernon Austel - IBM Corporation
Upendra Chauhari - IBM Corporation
Brain Kingsbury - IBM Corporation
Click here to download .ics calendar file
Click here to download .vcs calendar file
Click here to add event to your Google Calendar