Math 534 Practical Data Analysis, Fall 2025

• Instructor: Xingye Qiao

• Office: WH-122

• Meeting time & location: TR 9:45 – 11:15 am at WH–100E.

• Office hours: W 2:00 – 3:00 pm.

Prerequisite

• Scientific programming in a language such as R, Matlab, or Python. R and Python are strongly recommended.

• Previous coursework in statistics and data science.

Objectives

• Understand the major principles of practical data analysis

• Gain hands-on experience with practical aspects of data analysis.

• Cultivate critical thinking skills to critically assess the trustworthiness of data analyses.

• A Data Science and Statistics counterpart to the famous Missing Semester of Your CS Education (https://missing.csail.mit.edu/) course.

The Missing Semester for Data Science and Statistics

The hands-on competencies that every data scientist or statistician needs to thrive in industry, research, or applied work.

• Core Tooling & Environment – Use BashZSH for data handling, reproducible environments, and GitGitHub (incl. LFS) for version control.

• Data Acquisition & Engineering – Query databases (SQLite/MySQL), handle diverse formats, automate cleaning with provenance checks, and validate representativeness.

• Applied Modeling – Build pipelines (scikit-learn/tidymodels), engineer features without leakage, evaluate rigorously, and align models to practical needs.

• Experimental Design & Causal Inference – Design bias-controlled A/B tests, handle statistical pitfalls, apply robust methods, and critically assess causal claims.

• Statistical Inference & Uncertainty – Quantify and communicate uncertainty, use resampling for stability, and link assumptions to real-world limits.

• Visualization & Communication – Create advanced visualizations/dashboards, communicate clearly to varied audiences, and produce reproducible reports in Quarto.

• Project Organization – Structure projects for clarity, track metadata, and optimize pipelines for performance.

• Ethics & Responsibility – Address bias/fairness, frame problems ethically, and ensure analyses reflect reality and promote trust.

Recommended Texts

• Veridical Data Science, the Practice of Responsible Data Analysis and Decision Making, by Bin Yu and Rebecca L. Barter.

  • Free online book: https://vdsbook.com/

  • Print version: MIT Press Bookstore, Amazon, etc. (https://mitpress.mit.edu/9780262049191/veridical-data-science/)

• Practical Data Science with R, Second Edition, by Nina Zumel and John Mount

  • e-book and print (https://www.manning.com/books/practical-data-science-with-r-second-edition)

Slack

Course related communications will be done on Slack. Click here for access.

Gradescope and GitHub

We will use GitHub to serve all the coursrwork and use Gradescope to submit and grade homework. This will allow the instructor to efficient grade all the work and give feedback in a timely manner.

Brightspace

Brightspace will only be used for recording grades on assignments and exams and for distributing solutions. The code and lecture notes can also be found on Brightspace.

Grading

• Homework (55%): homework is assigned weekly.

• Project (35%)

• Quizzes and in-class participation (10%)

Homework Policy

There will be a deduction of 15% of the grade for each day homeworks are late (the final grade for a late homework that is N days late will be 0.85^N times the real grade). Homeworks may be discussed with classmates but must be written and submitted individually.

Project

Each student works on two projects — one as a Project Manager, defining goals, managing scope, and reviewing work, and one as a Data Scientist, executing analysis for another student’s project. The process includes public project pitches, observed consultation meetings, and constant feedback, adjustments, and clarifications via professional Slack communication, a DS-led final presentation and a PM follow-up email with assessment and feedback.

Software

RStudio and Google Colab will be used for completion of the homework assignments.

Tentative Schedule and Topics