Math 535 Statistical Learning and Data Mining. Spring 2021.

  • Instructor: Xingye Qiao

  • Phone number: (607) 777-2593

  • Office: WH-134

  • Meeting time & location: Tuesday and Thursday 8:20–9:55 am at WH–G02.

  • Office hours: the 1/2-hours before and after the class meetings, or by appoitment.

This course is a 4-credit course, which means that in addition to the scheduled lectures/discussions, students are expected to do at least 9.5 hours of course-related work each week during the semester. This includes things like: completing assigned readings, participating in lab sessions, studying for tests and examinations, preparing written assignments, completing internship or clinical placement requirements, and other tasks that must be completed to earn credit in the course.

Prerequisite

  1. Scientific programming in a language such as R, Matlab, or Python.

  2. Linear regression and its inference

  3. Matrix algebra, preferrably including orthogonality, eigenvalues and eigenvectors, and singular value decomposition.

Description

This course is a survey of statistical learning and data mining methods. It will cover major statistical learning methods and concepts for both supervised and unsupervised learning. Topics covered include regression methods with sparsity or other regularizations, model selection, graphical models, statistical learning pipeline and best practice, introduction to classification, including discriminant analysis, logistic regression, support vector machines, and kernel methods, nonlinear methods, dimension reduction, including matrix factorization-based approaches - principal component analysis and non-negative matrix factorization-, multidimensional scaling, and independent component analysis, clustering, decision trees, random forest, boosting and ensemble learning.

In Spring 2021, this course will include a critical data science component. Data and data science are pervasive in our daily lives, and thus have immense cultural, social, and political influence. It is imperative to ask who is supported by data science, and who is oppressed by it, and how these outcomes impact the human experience, and whether we can create better models and practices of data science.

List of Topics

  • ordinary least square regression;

  • penalized regression: ridge regression; lasso; other sparse regression methods;

  • (optional) Gaussian graphical models;

  • model selection and assessment; model validation; cross-validation;

  • (optional) generalized linear models;

  • classification: Bayes classifier, linear/quadratic discriminant analysis, logistic regression, support vector machines

  • nonlinear methods: kernel methods, splines, smoothing, kernel ridge regression, GAM

  • (optional) deep neural networks;

  • dimension reduction: principal component analysis, nonnegative matrix factorizations, independent component analysis, multidimensional scaling

  • clustering: K-means, hierarchical clustering, Gaussian mixtures

  • Ensemble methods: classification and regression trees, bagging, random forest, boosting, Gradient Boosting

  • (optional) compressed sensing, etc.

  • (optional) Text as data.

Learning Outcomes

Students will learn how and when to apply statistical learning techniques, their comparative strengths and weaknesses, and how to critically evaluate the performance of learning algorithms. Students completing this course should be able to

  • process and visualize different data types,

  • apply basic statistical learning methods to build predictive models or perform exploratory analysis,

  • have basic understanding of the underlying mechanism of predictive models and evaluate and interpret such models,

  • properly tune, select and validate statistical learning models,

  • use analytical tools and software widely used in practice,

  • work both independently and in a team to solve problems, and

  • learn to present and communicate the findings effectively.

Recommended Texts

  • James, Witten, Hastie and Tibshirani, 2014. An Introduction to Statistical Learning with Applications in R. Book Home Page. The PDF file of the book can be downloaded for free. There is also a R library for this book.

  • Hastie, Trevor, Tibshirani, Robert, and Friedman, J. H. 2009. The elements of statistical learning: Data mining, inference, and prediction. New York, NY: Springer New York.

  • Hastie, Trevor, Tibshirani, Robert, and Wainwright, Martin. Statistical Learning with Sparsity: The Lasso and Generalizations. Chapman and Hall/CRC; 1 edition (May 7, 2015). The PDF file of the book can be downloaded for free.

  • B├╝hlmann, Peter, and van de Geer, Sara. Statistics for High-Dimensional Data. Springer-Verlag Berlin Heidelberg.

  • Boyd, Stephen, and Vandenberghe, Lieven. Convex Optimization. Cambridge University Press. The PDF file of the book can be downloaded for free.

  • D'Ignazio, C. and Klein, L.F., (2020). Data Feminism. MIT Press. Free copy online: https:data-feminism.mitpress.mit.edu/ Chapter 1 is required reading.

Piazza

Please use Piazza (www.piazza.com) for all communications with me rather than email. Piazza is a question-and-answer platform. It supports LaTeX, code formatting, embedding of images, and attaching of files. You are encouraged to ask questions when you have difficulty understanding a concept or working around a piece of code – you can even ask questions anonymously. Moreover, you can also answer questions from your classmates. I constantly monitor the answers and endorse those good answers.

Announcement will be sent to the class using Piazza.

Gradescope

We will use Gradescope to submit and grade homework. This will allow the instructor to efficient grade all the work and give feedback in a timely manner.

Mycourses(Blackboard)

Mycourses(Blackboard) will only be used for recording grades on assignments and exams and for distributing solutions. The code and lecture notes can also be found on blackboard.

Grading

  • Homework (30%): homework is assigned biweekly.

  • Midterm exam (32%): a midterm exam focusing on the theoretical part of the course will be administered.

  • Contest & Reports (30%): a group project will be assigned to each student. Successful completion of the project includes an initial report, a presentation and a final report.

  • Lecture attendance and participation (5%): attendance will be taken. Active participation on asking meaning questions and answering others’ questions will also count.

  • Data Feminism (3%): read chapter 1 of the book and participate in in-class discussion.

Homework Policy

There will be a deduction of 25% of the grade for homeworks that are not typeset using LaTeX. There will be a deduction of 15% of the grade for each day homeworks are late (the final grade for a late homework that is N days late will be 0.85^N times the real grade). Homeworks may be discussed with classmates but must be written and submitted individually.

Final Project

Students will compete against each other in a the Final Project. It can be completed in teams of 2 – 4 members. Grades will be based upon a progress report and a final report (one per team) as well as the contest results. Further details about the Final Project along with specific grading criteria will be given in a separate document and discussed in class.

Software

RStudio will be used for completion of the homework assignments.