IDS.160 / 18.S998 / 9.521 Spring 20

Mathematical Statistics: A non-asymptotic approach.


Topics covered

  1. Introduction to high dimensional phenomena.
  2. Sums of independent random variables.
    • Sub-Gaussian random variables. Chernoff bounds. Hoeffding's inequality.
    • Sub-Exponential random variables. Bernstein's inequality.
  3. Linear Regression.
    • Gaussian Sequence model. Sparsity. Thresholding.
    • High dimensional linear regression.
  4. Matrix estimation.
    • Perturbation analysis. Davis-Kahan sin-theta theorem. Community detection.
    • Covariance matrix estimation. Principal component analysis.
  5. Minimax lower bounds.
    • Distance between probability distributions.
    • Lower bounds for estimation and detection problems.
  6. Empirical processes
    • Statistical Learning. Prediction. Uniform Convergence.
    • Maximal Inequalities. Symmetrization. Covering, packing, chaining.
    • VC theory. Scale-sensitive dimensions.
  7. High-dimensional classification
    • Properties of Rademacher averages. Margin analysis for classification.
    • Linear classifiers. Neural networks.
  8. Nonparametric regression
    • Well-specified and misspecified models. Reproducing Kernel Hilbert Spaces.
    • Bias-variance. Local methods. The interpolation phenomenon.


Grading will be based on 4 problem sets (80%) and scribe notes (20%).