ECE 3803: Optimization for Information Systems
Fall 2021
Overview
Lectures and Notes
Assignments
Course overview
Introduction to optimization
[
Slides
]
Least squares optimization
Regression and least squares
Solving the least squares problem
Understanding least squares: The singular value decomposition
Least squares in noise
Gradient descent for least squares
Convergence analysis of gradient descent
Nonlinear regression and kernels
Unconstrained convex optimization
Convex sets and functions
Convexity in optimization and introduction to algorithms
Line search methods and convergence of gradient descent revisited
Accelerated gradient descent
Newton's method and quasi-Newton methods
Subgradients and subgradient descent
Proximal algorithms
Constrained convex optimization
Lagrangian duality
The Karush-Kuhn-Tucker conditions
Algorithms for constrained optimization
A taxonomy of constrained optimization problems