Math 457 Introduction to Statistical Learning. Spring 2020.

• Instructor: Xingye Qiao

• Phone number: (607) 777-2593

• Office: WH-134

• Meeting time & location: Tuesday and Thursday 8:20–9:55 am at WH–G02.

• Office hours: Thursday 10–11:30 am

Prerequisite

1. Scientific programming in a language such as R, Matlab, or Python.

2. Linear regression and its inference

3. Matrix algebra, preferrably including orthogonality, eigenvalues and eigenvectors, and singular value decomposition.

Description

This course is a survey of statistical learning and data mining methods. It will cover major statistical learning methods and concepts for both supervised and unsupervised learning. Topics covered include regression methods with sparsity or other regularizations, model selection, graphical models, statistical learning pipeline and best practice, introduction to classification, including discriminant analysis, logistic regression, support vector machines, and kernel methods, nonlinear methods, dimension reduction, including matrix factorization-based approaches - principal component analysis and non-negative matrix factorization-, multidimensional scaling, and independent component analysis, clustering, decision trees, random forest, boosting and ensemble learning.

Learning Outcomes

Students will learn how and when to apply statistical learning techniques, their comparative strengths and weaknesses, and how to critically evaluate the performance of learning algorithms. Students completing this course should be able to

• process and visualize different data types,

• apply basic statistical learning methods to build predictive models or perform exploratory analysis,

• have basic understanding of the underlying mechanism of predictive models and evaluate and interpret such models,

• properly tune, select and validate statistical learning models,

• use analytical tools and software widely used in practice,

• work both independently and in a team to solve problems, and

• learn to present and communicate the findings effectively.

Recommended Texts

• James, Witten, Hastie and Tibshirani, 2014. An Introduction to Statistical Learning with Applications in R. Book Home Page. The PDF file of the book can be downloaded for free. There is also a R library for this book.

• Hastie, Trevor, Tibshirani, Robert, and Friedman, J. H. 2009. The elements of statistical learning: Data mining, inference, and prediction. New York, NY: Springer New York.

• Hastie, Trevor, Tibshirani, Robert, and Wainwright, Martin. Statistical Learning with Sparsity: The Lasso and Generalizations. Chapman and Hall/CRC; 1 edition (May 7, 2015). The PDF file of the book can be downloaded for free.

• Bühlmann, Peter, and van de Geer, Sara. Statistics for High-Dimensional Data. Springer-Verlag Berlin Heidelberg.

• Boyd, Stephen, and Vandenberghe, Lieven. Convex Optimization. Cambridge University Press. The PDF file of the book can be downloaded for free.

Piazza

Please use Piazza (www.piazza.com) for all communications with me rather than email. Piazza is a question-and-answer platform. It supports LaTeX, code formatting, embedding of images, and attaching of files. You are encouraged to ask questions when you have difficulty understanding a concept or working around a piece of code – you can even ask questions anonymously. Moreover, you can also answer questions from your classmates. I constantly monitor the answers and endorse those good answers.

Announcement will be sent to the class using Piazza.

Gradescope

We will use Gradescope to submit and grade homework. This will allow the instructor to efficient grade all the work and give feedback in a timely manner.

Mycourses(Blackboard)

Mycourses(Blackboard) will only be used for recording grades on assignments and exams and for distributing solutions. The code and lecture notes can also be found on blackboard.

Grading

• Homework (30%): homework is assigned biweekly.

• Midterm exam (30%): a midterm exam focusing on the theoretical part of the course will be administered.

• Contest & Reports (35%): a group project will be assigned to each student. Successful completion of the project includes an initial report, a presentation and a final report.

• Lecture attendance and participation (5%): attendance will be taken. Active participation on asking meaning questions and answering others’ questions will also count.

Homework Policy

There will be a deduction of 25% of the grade for homeworks that are not typeset using LaTeX. There will be a deduction of 15% of the grade for each day homeworks are late (the final grade for a late homework that is N days late will be 0.85^N times the real grade). Homeworks may be discussed with classmates but must be written and submitted individually.

Data Analysis Contest

Students will compete against each other in a Data Analysis Contest. The competition will begin on Tuesday, Feburary 20 and can be completed in teams of 2 – 4 members. Grades will be based upon a progress report and a final report (one per team) as well as the contest results. Further details about the contest along with specific grading criteria will be given in a separate document and discussed in class.

Software

Jupyter notebook will be used to illustrate both R and Python codes in class. I recommend installing Jupyter notebook via Anaconda.

Students are expected to write homework in LaTex. For users with no experience with LaTex, I suggest using the cloud LaTex editing service at overleaf.com.

Tentative schedule