ECE 520/MATH 520 Lecture and Homework Schedule, Spring 2009	Version: January 07, 2009
Session	Date	Topic	Chapter	Homework Due
1	1/20	Introduction. Examples. Math preliminary: Notation, real vector spaces, linear independence, matrices.	1,2
2	1/22	Math preliminary: Inner product, norm. Eigenvalues and eigenvectors, quadratic forms, calculus of several variables, chain rule, Taylor series, gradient, level sets, directional derivative.	3,5
3	1/27	Definition of optimization problem and types of solutions. Quadratic problems. FONC.	6
4	1/29	SONC and SOSC. Basic iterative algorithms: form, basic properties, line search.	6
5	2/3	One-dimensional search methods: Golden section search, Newton's method, secant method.	7	1.5, 2.7, 3.6, 3.17a, 5.5, 5.8, 5.9, 6.2, 6.7, 6.17, 6.27
6	2/5	Multi-dimensional algorithms. Gradient methods: form, steepest descent, convergence.	8
7	2/10	Gradient methods: convergence of fixed step size algorithm, steepest descent algorithm. Order of convergence.	8
8	2/12	Newton's method: form, order of convergence. Properties of general algorithms.	9
9	2/17	Conjugate direction methods: form, properties, conjugate gradient formulas.	10	7.2a,b,d, 8.1, 8.5, 8.17, 8.23, 9.1, 9.4
10	2/19	Quasi-Newton methods: form, properties, rank one formula.	11
11	2/24	Quasi-Newton methods: DFP and BFGS.	11
12	2/26	Least squares problems: basic properties, grammian, examples.	12
13	3/3	RLS. Parameter identification.	12	10.1, 10.7, 10.9, 11.1, 11.10
14	3/5	Exam 1
15	3/10	Randomized search. Simulated annealing. Genetic algorithm: Intro, representation schemes, selection.	14
16	3/12	Genetic algorithm: evolution (crossover & mutation). Algorithms for constrained problems: projection, penalty method.	14,22
17	3/24	Constrained optimization: basic form with equality and inequality constraints. Intro to linear programs, geometric view, standard form.	15	12.3, 12.10, 12.12, 12.14, 12.15, 22.11a (Hint: solution is A^T(A.A^T)^{-1}.b; see Section 12.3)
18	3/26	Linear programming: converting to standard form. Linear equations, elementary row operations, basic solutions.	15,16
19	3/31	Basic feasible solutions. Fundamental theorem of LP. Pivoting, changing bases and canonical augmented matrix.	16
20	4/2	Moving from one BFS to an adjacent BFS. Reduced cost coefficients. Simplex algorithm. Matrix form of simplex.	16
21	4/7	Artificial problem and feasibility. Two phase algorithm. Duality: form, example.	16,17	15.1, 15.7, 15.10, 16.3, 16.4, 16.12, 16.13
22	4/9	Weak duality lemma, duality theorem, duality and Simplex algorithm, complementary slackness. Equivalence of feasibility and LP problems.	17
23	4/14	General equality constraints: basic form, example. Lagrange condition for scalar equality constraint.	17,19
24	4/16	General multivariable Lagrange condition. Tangent and normal space.	19
25	4/21	Minimizing quadratic subject to linear constraint (quadratic programming). Simple linear quadratic regulator problem. Second order conditions.	19	17.3, 17.6, 17.11, 19.6a, 19.11, 19.12a, 19.17a
26	4/23	Exam 2
27	4/28	General equality and inequality constraints: form, example. KKT condition with only inequality constraints. Examples.	20
28	4/30	KKT condition for equality and inequality constraints. SONC.	20
29	5/5	SOSC. Convex optimization problems.	20,21
30	5/7	Convex optimization problems.	21	19.6b,c, 19.12b,c, 19.17b, 20.9, 20.10, 20.15, 21.2, 21.12