Mark Davenport
Email: mdav (at) gatech (dot) edu
Office: Coda, Room S1117
Office Hours: Tuesdays from 2-3pm in Coda C1103 (Lindberg).

Chieh-feng Cheng
Email: ccheng71 (at) gatech (dot) edu

Jiannan Cui
Email: jcui67(at) gatech (dot) edu

This course is a general purpose, advanced DSP course designed to follow an introductory DSP course. The central theme of the course is the application of tools from linear algebra to problems in signal processing.

Download the syllabus.

An introductory course in digital signal processing covering concepts such as Fourier transforms, filtering, and sampling. Students should also be familiar with the fundamentals of linear algebra and should be very comfortable with the use of matrices to represent systems of equations -- some existing familiarity with eigenvalues, eigenvectors, and eigenvalue decompositions will be extremely helpful. While most of the course will adopt a deterministic perspective, many of the models and algorithms we will discuss also have alternative probabilistic interpretations, and hence familiarity with the basics of probability and statistics (especially random vectors) will be useful for gaining a deeper appreciation for the material. Finally, students should also have basic MATLAB programming skills.

There is no required text. Below is a list of books that the instructors have found helpful over the years for learning (and teaching) the material in this class.

Linear algebra and function spaces

Linear Algebra and its Applications by Strang (2006). (amazon).

Computational Science and Engineering by Strang (2007). (amazon).

Matrix Analysis by Horn and Johnson (2012). (amazon).

An Introduction to Hilbert Space by Young (1988). (amazon).

Mathematics of signal processing

Mathematical Methods and Algorithms for Signal Processing by Moon and Stirling (1999). (amazon).

Foundations of Signal Processing by Vetterli abd Kovacevic (2014). (amazon).

Statistical Signal Processing by Scharf (1991). (amazon).

Online resources

The Matrix Cookbook

A short, useful introduction to matrix calculus

Information about taking tests. (Consider reading before the first test.)

If you find anything else useful, let me know and I will post it here.

• Signal representations in vector spaces
  • Introduction to discretizing signals using a basis: The Shannon-Nyquist sampling theorem
  • Linear vector spaces, linear independence, and basis expansions
  • Norms and inner products
  • Orthobases and the reproducing formula
  • Parseval's theorem and the general discretization principle
  • Important bases: Fourier, discrete cosine, lapped orthogonal, splines, wavelets
  • Signal approximation in an inner product space
  • Gram-Schmidt and the QR decomposition
• Linear inverse problems
  • Introduction to linear inverse problems, examples
  • The singular value decomposition (SVD)
  • Least-squares solutions to inverse problems and the pseudo-inverse
  • Stable inversion and regularization
  • Weighted least-squares and linear estimation
  • Least-squares with linear constraints
• Matrix approximation using least-squares
  • Low-rank approximation of matrices using the SVD
  • Total least-squares
  • Principal components analysis
  • Signal and noise subspaces in array processing
• Computing the solutions to least-squares problems
  • Cholesky and LU decomposition
  • Structured matrices: Toeplitz, diagonal+low rank, banded systems
  • Large-scale systems: Steepest descent
  • Large-scale systems: The conjugate gradient method
• Low-rank updates for streaming solutions to least-squares problems
  • Recursive least-squares
  • The Kalman filter
  • Adaptive filtering using LMS
• Beyond least-squares (topics as time permits)
  • Approximation in non-Euclidean norms
  • Regularization using non-Euclidean norms
  • Recovering vectors from incomplete information (compressed sensing)
  • Recovering matrices from incomplete information (matrix completion)