Friday 22 March 2019
Mattan Erez “Cross-Layer Architecture for Deep Learning”
16:00 CAB G51

Abstract: High-performance DNN inference and training is essential for the ongoing ML revolution. Training of DNNs requires massive memory capacity and bandwidth, and is generally a huge pain, especially for researchers. While significant research effort has been dedicated to inference accelerator, less work has been done on training, especially work that crosses the algorithmic and implementation layers. The result is a very limited number of high-cost accelerators available, in particular, with very expensive high-bandwidth memories. I will motivate and discuss some of our recent work on accelerating training (of CNNs) that combines understanding of and changes to the algorithm with matching hardware architecture modifications. Specifically, I will present a technique that reduces the bandwidth requirements of training modern CNNs like ResNet and Inception by 4x and a second technique that can speed up training of pruned networks by 2x or more even on current hardware. I will also discuss how accelerators pose new resource-sharing and QoS challenges and hope to motivate future collaborative research.

Bio: Mattan Erez is a Professor at the Department of Electrical and Computer Engineering at the University of Texas at Austin. His research focuses on improving the performance, efficiency, and scalability of computing systems through advances in memory systems, hardware architecture, software systems, and programming models. His current focus areas are architectures for machine learning, large-scale and high-performance computing, and memory systems. His work aims to improve cooperation across system layers and develop flexible and adaptive mechanisms for proportional resource usage. Mattan received a BSc in Electrical Engineering and a BA in Physics from the Technion, Israel Institute of Technology and his MS and PhD. in Electrical Engineering from Stanford University. He was awarded a Presidential Early Career Award for Scientists and Engineers from President Obama and received an Early Career Research Award from the Department of Energy and an NSF CAREER Awards