IDS.160 / 18.S998 / 9.521 Spring 20

Mathematical Statistics: A non-asymptotic approach.

Topics covered

1. Introduction to high dimensional phenomena.
2. Sums of independent random variables.
• Sub-Gaussian random variables. Chernoff bounds. Hoeffding's inequality.
• Sub-Exponential random variables. Bernstein's inequality.
3. Linear Regression.
• Gaussian Sequence model. Sparsity. Thresholding.
• High dimensional linear regression.
4. Matrix estimation.
• Perturbation analysis. Davis-Kahan sin-theta theorem. Community detection.
• Covariance matrix estimation. Principal component analysis.
5. Minimax lower bounds.
• Distance between probability distributions.
• Lower bounds for estimation and detection problems.
6. Empirical processes
• Statistical Learning. Prediction. Uniform Convergence.
• Maximal Inequalities. Symmetrization. Covering, packing, chaining.
• VC theory. Scale-sensitive dimensions.
7. High-dimensional classification
• Properties of Rademacher averages. Margin analysis for classification.
• Linear classifiers. Neural networks.
8. Nonparametric regression
• Well-specified and misspecified models. Reproducing Kernel Hilbert Spaces.
• Bias-variance. Local methods. The interpolation phenomenon.

Evaluation