Math 605 Theory of Machine Learning. Spring 2022.
This course is a 4-credit course, which means that in addition to the scheduled lectures/discussions, students are expected to do at least 9.5 hours of course-related work each week during the semester. This includes things like: completing assigned readings, participating in lab sessions, studying for tests and examinations, preparing written assignments, completing internship or clinical placement requirements, and other tasks that must be completed to earn credit in the course. PrerequisiteBasic prerequisites include probability theory, statistics, real analysis, and linear algebra. This is a theory class: although many tools will be reviewed in lectures, a strong mathematical background is necessary. DescriptionStatistical learning theory is a flourishing research field at the intersection of probability, statistics, computer science, and optimization that studies the performance of algorithms for making predictions based on training data. This course will provide an introduction to the theoretical analysis of prediction methods, focusing on statistical and computational aspects. The following topics will be covered: basics of statistical decision theory; concentration inequalities; supervised and unsupervised learning; empirical risk minimization; complexity-regularized estimation; generalization bounds for learning algorithms; VC dimension and Rademacher complexities; minimax lower bounds; online learning and optimization. It will focus on tools for the theoretical analysis of the performance of learning algorithms and the inherent difficulty of learning problems. Topics
TextThere will be no designated text. I will develop a set of notes drawing contents from those written by Philippe Rigollet (MIT), Percy Liang (Stanford), Peter Bartlett (UC Berkeley) and Martin Wainwright (UC Berkeley). Additional reading materials will be updated on this webpage. Grading
NotesThe students of this class from Spring 2021 have helped to scribed the notes which have formed an online book: Online Lecture Notes |