Unconstrained convex optimization
- Convex sets and functions
- Differentiable functions, convexity, and optimization
- Gradient descent
- Convergence analysis of gradient descent
- Accelerated gradient descent: Heavy ball method and Nesterov's method
- Newton's method
- Quasi-Newton methods: BFGS
- Subgradients and subgradient descent
- Proximal methods
Constrained convex optimization
- Optimality conditions for constrained optimization problems
- Lagrangian duality
- KKT conditions
- Duality revisited: Convex conjugates and support functions
- Fenchel duality
- Algorithms for constrained optimization
- Dual ascent, dual decomposition, method of multipliers
- ADMM
- Distributed estimation using ADMM