Course Information

Over the past few years, deep learning has become an important technique to successfully solve problems in many different fields, such as vision, NLP, robotics. An important ingredient that is driving this success is the development of deep learning systems that efficiently support the task of learning and inference of complicated models using many devices and possibly using distributed resources. The study of how to build and optimize these deep learning systems is now an active area of research and commercialization, and yet there isn’t a course that covers this topic.

This course is designed to fill this gap. We will be covering various aspects of deep learning systems, including: basics of deep learning, programming models for expressing machine learning models, automatic differentiation, memory optimization, scheduling, distributed learning, hardware acceleration, domain specific languages, and model serving. Many of these topics intersect with existing research directions in databases, systems and networking, architecture and programming languages. The goal is to offer a comprehensive picture on how deep learning systems works, discuss and execute on possible research opportunities, and build open-source software that will have broad appeal.

We will have two classes per week. The class will either be lecture or lab/discuss session. Each lecture will study a specific aspect of deep learning systems. The lab/discussion session will contain tutorials to implement that specific aspect and will include case studies of existing systems, such as Tensorflow, Caffe, Mxnet, PyTorch, and others.

Instructors

• Tianqi Chen
• Haichen Shen
• Arvind Krishnamurthy

Teaching Assistant

• Qiao Zhang

Logistics

• Time: Tue/Th 11:30am - 12:50pm
• Place: Mueller Hall 153
  • We will be using CSE 305 for some of the guest lectures, checkout the schedule for details
• Slack: Join https://uw-cse.slack.com dlsys channel for course discussions and announcements

Prerequisites

• Proficiency in Python, familar in C/C++
  • We will mainly be using python for case study the existing systems, and C/C++ for some of the background hacking.
• Basic knowledge of machine learning (e.g CSE 546)
• Prior knowledge in system (operating system/database) is useful but not required.

Homeworks and Grading

We will have assignments and a final project.

• Course project: 60%
• Homeworks: 30%
• Discussion participation: 10%

Acknowledgement

This course is created with help from DLsys seminar group at University of Washington.