A reading seminar on learning theory, diffusion models, and language models, with shared topics in scaling laws and neural network optimization.
Course information
- Background
- Probability, linear algebra, mathematical analysis, and machine learning.
- Format
- Introductory lectures by Yiping Lu, followed by student presentations and group discussion.
- Presentations
- Discuss topics and papers with Yiping Lu in advance. Share paper links and a brief reading guide before presenting.
- Contact
- yipinglu@bicmr.pku.edu.cn
Reference course: Statistical Learning (IEMS 402)
Schedule and materials
Lecture 1 · Foundations of Machine Learning Theory
- Risk decomposition, approximation, and the curse of dimensionality
- Concentration: Markov → Chebyshev → higher moments → Chernoff
- Uniform bounds, covering numbers, Rademacher complexity, and Maurey’s method
- Localized complexity: mean estimation with squared loss
- Kernel smoothing and RKHS
Lecture 2 · Foundations of Generative Models
- Computation in spaces of probability measures
- f divergences
- Optimal transport: duality and gradient flows
- Particle methods
- ELBO
- Continuous time stochastic processes
- Diffusion models and flow matching
Details of subsequent meetings will be announced.