BEGIN:VCALENDAR
PRODID:-//Microsoft Corporation//Outlook MIMEDIR//EN
VERSION:2.0
BEGIN:VEVENT
DTSTART:20141120T170000Z
DTEND:20141120T173000Z
LOCATION:388-89-90
DESCRIPTION;ENCODING=QUOTED-PRINTABLE:ABSTRACT: Deep Neural Networks (DNNs) have recently been shown to significantly outperform existing machine learning techniques in several pattern recognition tasks.  The biggest drawback to DNNs is the enormous training time - often 10x slower than conventional technologies. While training time can be mitigated by parallel computing algorithms and architectures, these algorithms often suffer from the cost of inter-processor communication bottlenecks. In this paper, we describe how to enable parallel DNN training on the IBM Blue Gene/Q (BG/Q) computer system using the data-parallel Hessian-free 2nd-order optimization algorithm. BG/Q, with its excellent inter-processor communication characteristics, is an ideal match for the HF algorithm. The paper discusses how issues regarding programming model and data-dependent imbalances are addressed. Results on large-scale speech tasks show that the performance on BG/Q scales linearly up to 4096 processes with no loss in accuracy, allowing us to train DNNs with millions of training examples in a few hours.
SUMMARY:Parallel Deep Neural Network Training for Big Data on Blue Gene/Q
PRIORITY:3
END:VEVENT
END:VCALENDAR