An Introduction to Optimization, Fifth Edition

With Applications to Machine Learning

Edwin K. P. Chong, Wu-Sheng Lu, and Stanislaw H. Żak

John Wiley & Sons, Inc.
New York
Copyright © 2023
ISBN: 978-1-119-87763-9
ISBN-10: 1119877636
672 pages

From the back cover:

Praise for the Third Edition
"... guides and leads the reader through the learning path ... examples are stated very clearly and the results are presented with attention to detail."
MAA Reviews

Description

Accessible introductory textbook on optimization theory and methods, with an emphasis on engineering design, featuring MATLAB exercises and worked examples

Fully updated to reflect modern developments in the field, the Fifth Edition of An Introduction to Optimization fills the need for an accessible, yet rigorous, introduction to optimization theory and methods, featuring innovative coverage and a straightforward approach. The book begins with a review of basic definitions and notations while also providing the related fundamental background of linear algebra, geometry, and calculus.

With this foundation, the authors explore the essential topics of unconstrained optimization problems, linear programming problems, and nonlinear constrained optimization. In addition, the book includes an introduction to artificial neural networks, convex optimization, multi-objective optimization, and applications of optimization in machine learning.

Numerous diagrams and figures found throughout the book complement the written presentation of key concepts, and each chapter is followed by MATLABŽ exercises and practice problems that reinforce the discussed theory and algorithms.

The Fifth Edition features a new chapter on Lagrangian (nonlinear) duality, expanded coverage on matrix games, projected gradient algorithms, machine learning, and numerous new exercises at the end of each chapter.

An Introduction to Optimization includes information on:

The mathematical definitions, notations, and relations from linear algebra, geometry, and calculus used in optimization
Optimization algorithms, covering one-dimensional search, randomized search, and gradient, Newton, conjugate direction, and quasi-Newton methods
Linear programming methods, covering the simplex algorithm, interior point methods, and duality
Nonlinear constrained optimization, covering theory and algorithms, convex optimization, and Lagrangian duality
Applications of optimization in machine learning, including neural network training, classification, stochastic gradient descent, linear regression, logistic regression, support vector machines, and clustering.

An Introduction to Optimization is an ideal textbook for a one- or two-semester senior undergraduate or beginning graduate course in optimization theory and methods. The text is also of value for researchers and professionals in mathematics, operations research, electrical engineering, economics, statistics, and business.

Errata

An up-to-date errata is available.

Brief Table of Contents

(A more detailed table of contents is available.)

Preface

Part I. Mathematical Review

1 Methods of Proof and Some Notation
2 Vector Spaces and Matrices
3 Transformations
4 Concepts from Geometry
5 Elements of Calculus

Part II. Unconstrained Optimization

6 Basics of Set-Constrained and Unconstrained Optimization
7 One-Dimensional Search Methods
8 Gradient Methods
9 Newton's Method
10 Conjugate Direction Methods
11 Quasi-Newton Methods
12 Solving Linear Equations
13 Unconstrained Optimization and Neural Networks
14 Global Search Algorithms

Part III. Linear Programming

15 Introduction to Linear Programming
16 Simplex Method
17 Duality
18 Nonsimplex Methods
19 Integer Linear Programming

Part IV. Nonlinear Constrained Optimization

20 Problems with Equality Constraints
21 Problems With Inequality Constraints
22 Convex Optimization Problems
23 Algorithms for Constrained Optimization
24 Lagrangian Duality
25 Multiobjective Optimization

Part V Optimization in Machine Learning

26 Machine Learning Problems and Feature Engineering
27 Stochastic Gradient Descent Algorithms
28 Linear Regression and Its Variants
29 Logistic Regression for Classification
30 Support Vector Machines
31 K-Means Clustering

References

Index

Ordering information

Wiley has information on how to order the book.

Instructors only: The Instructor's Solutions Manual is available to Instructors who adopt the book. Please visit the solutions manual website (also linked from Wiley's companion site for the book) and register to receive access to the solutions manual.

Useful links:

Edwin K. P. Chong's ECE/MATH 520 website. Contains M-files for MATLAB implementation of algorithms.
For data sets and computer code involved in Part V, see Wu-Sheng Lu's Optimization website.
Stanislaw H. Zak's EE 580 website.

Professor Edwin Chong, Email

This document was last modified August 16, 2024.