EE61012: Convex Optimization in Control and Signal Processing

Welcome to this PG level subject.

  • Venue: N-227, Department of Electrical Engineering
  • Instructor: Prof. Ashish Hota
  • Office hours: Tuesday 5:30 - 6:30 pm or by appointment

Logistics of Grading

  • End Semester: 50
  • Mid Semester: 30
  • Homeworks: There will be two homework sets. Total weight of homeworks: 15 (5 each)
  • Class Participation: 5

Course Content (Tentative)

Here is a basic overview of the topics that are planned to be covered.

  1. Background (1-2 weeks)
  2. Convex Sets and Convex Functions (2 weeks)
  3. Convex Optimization Problems (1 week)
  4. Lagrangian Duality (1 week)
  5. Necessary and Sufficient Optimality Conditions (1 week)
  6. Algorithms for Convex Optimization, First Order Methods, (Sub)gradient methods, Primal-Dual Algorithms, ADMM (2 week)
  7. Application: Linear Regression, classification (1 week)
  8. Application: Compressive Sensing (1 week)
  9. Application: Model Predictive Control (1 week)
  10. Semidefinite Programming (SDP), Conic Duality (1-2 weeks)
  11. Application of SDP in Stability and Robust Controller Synthesis (1-2 weeks)

Software Packages

There are several dedicated environments that enable us to easily encode and solve convex optimization problems. We will demonstrate some examples in class using CVX/YALMIP.

  1. CVX.
  2. YALMIP.
  3. JuliaOPT.

Textbooks

There is no single textbook for this subject. We will discuss a variety of topics from different books. The first reference will be followed to a large extent. You are encouraged to refer the other texts below depending on your interests.

  1. Convex Optimization (freely available to download) by Boyd and Vandenberghe.
  2. Optimization III: Convex and Nonlinear Programming by Ben-Tal and Nemirovski. Lecture Notes.
  3. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers by Boyd and others. Monograph on ADMM.
  4. Linear Matrix Inequalities in Control by Carsten Scherer and Siep Weiland. More focus on applications of SDP in control.
  5. Lectures on Convex Optimization by Yurii Nesterov. Focused on algoithms for solving convex optimization problems and their convergence analysis.
  6. Lectures on Modern Convex Optimization by Ben-Tal and Nemirovski. More focus on Conic Optimization and SDP.
  7. Statistical Inference via convex Optimization by Juditsky and Nemirovski. More focus on applications in Signal Processing and Inference.
  8. Variational Analysis by Rockafellar and Wets. The Rigveda of the mathematics behind optimization.

References

Convex Optimization is widely applied in virtually all areas of Electrical, Electronics and Computer Sciences and Engineering, most notably in Control, Communications, Signal Processing, Power Systems and even VLSI. A large class of Machine Learning techniques involving solving optimization problems. Some reference papers in a variety of applications are linked below. To make the most of the course, you are encouraged to find a problem related to your research, formulate it as a convex optimization problem, and solve it.

  1. Convex Optimization in Signal Processing and Communications Edited volume. Several applications are considered. Also see the paper.
  2. Semidefinite programming duality and linear time-invariant systems, V. Balakrishnan and L. Vandenberghe, IEEE Transactions on Automatic Control, 48(1), 2003.
  3. Two part paper on Convex Optimization for Optimal Power Flow Problems. Part-I, and Part-II.