Lecture notes

• Introduction to optimization problems

• Introduction.

• Basic Linear Algebra, Analysis, Calculus

  • Vector Spaces, Linear Independence

  • Subspace, Linear Maps, Matrices

  • Inner Products, Vector Norms

  • Symmetric Matrices, Matrix Norms

  • Change of Basis, SVD

  • Analysis, Orderings

  • Continuity, Differentiability of Functions (recording link)

• Convex Analysis

  • Convex Sets

  • Convex Hull, Affine Sets

  • Cones, Simplex

  • Caratheodory, Radon, Helly's Theorems

  • Separating Hyperplane, Supporting Hyperplane, Extreme Points

  • Convex Functions

  • First Order and Second Order Conditions For Convex Functions

• First Order Conditions for Optimality

  • Convex Function Minima

  • First Order Necessary and Sufficient Conditions for Optimality

  • Radial Cone and Normal Cone, Some Theorems on Maxima of Convex Functions

• Lagrangian Duality

  • Convex Theorem on Alternatives

  • Lagrangian Duality

  • Saddle Point of Lagrangian

  • KKT Conditions

  • Examples

  • Barrier Functions for Interior Point Method

• Unconstrained Optimization and Descent Methods

  • Unconstrained Optimization

  • Descent Methods

  • Backtracking Line Search

  • Convergence of Gradient Descent

  • Conjugate Norm

  • Convergence of Steepest Descent

  • Newton's Method

  • Convergence of Newton's Method

• Linear Matrix Inequalities, Semidefinite Programming

  • Brief Overview of Semidefinite Programs and Solving Methods

References

• Lectures on Modern Convex Optimization by Aharon BenTal and Arkadi Nemirovski.

• Convex Optimization by Stephen Boyd and Lieven Vandenberghe, Cambridge University Press.

• Convex Analysis and Optimization by Dmitri Bertesekas, A. Nedic and A. Ozdaglar, Athena Scientific.

• Non-linear Programming by M. Bazaara, H. Sherali and C. Shetty, Wiley.

• Moments, Positive Polynomials and Their Applications by Jean Bernard Lasserre, Imperial College Press.