An Introduction to Optimization, Fifth Edition
With Applications to Machine Learning

John Wiley & Sons, Inc.
New York
Copyright © 2023
ISBN: 9781119877639
ISBN10: 1119877636
672 pages
From the back cover:
Praise for the Third Edition
"... guides and leads the reader through the learning path ...
examples are stated very clearly and the results are presented with
attention to detail."
MAA Reviews
Description
Accessible introductory textbook on optimization theory and
methods, with an emphasis on engineering design, featuring MATLAB
exercises and worked examples
Fully updated to reflect modern developments in the field, the Fifth
Edition of An Introduction to Optimization fills the need for an
accessible, yet rigorous, introduction to optimization theory and
methods, featuring innovative coverage and a straightforward
approach. The book begins with a review of basic definitions and
notations while also providing the related fundamental background of
linear algebra, geometry, and calculus.
With this foundation, the authors explore the essential topics of
unconstrained optimization problems, linear programming problems,
and nonlinear constrained optimization. In addition, the book
includes an introduction to artificial neural networks, convex
optimization, multiobjective optimization, and applications of
optimization in machine learning.
Numerous diagrams and figures found throughout the book complement
the written presentation of key concepts, and each chapter is
followed by MATLABŪ exercises and practice problems that reinforce
the discussed theory and algorithms.
The Fifth Edition features a new chapter on Lagrangian (nonlinear)
duality, expanded coverage on matrix games, projected gradient
algorithms, machine learning, and numerous new exercises at the end
of each chapter.
An Introduction to Optimization includes information on:
 The mathematical definitions, notations, and relations from linear
algebra, geometry, and calculus used in optimization
 Optimization algorithms, covering onedimensional search, randomized
search, and gradient, Newton, conjugate direction, and quasiNewton
methods
 Linear programming methods, covering the simplex algorithm, interior
point methods, and duality
 Nonlinear constrained optimization, covering theory and algorithms,
convex optimization, and Lagrangian duality
 Applications of optimization in machine learning, including neural
network training, classification, stochastic gradient descent,
linear regression, logistic regression, support vector machines, and
clustering.
An Introduction to Optimization is an ideal textbook for a one or
twosemester senior undergraduate or beginning graduate course in
optimization theory and methods. The text is also of value for
researchers and professionals in mathematics, operations research,
electrical engineering, economics, statistics, and business.
Errata
An uptodate errata is available.
Brief Table of Contents
(A more detailed table of contents is available.)
 Preface
Part I. Mathematical Review
 1 Methods of Proof and Some Notation
 2 Vector Spaces and Matrices
 3 Transformations
 4 Concepts from Geometry
 5 Elements of Calculus
Part II. Unconstrained Optimization
 6 Basics of SetConstrained and Unconstrained Optimization
 7 OneDimensional Search Methods
 8 Gradient Methods
 9 Newton's Method
 10 Conjugate Direction Methods
 11 QuasiNewton Methods
 12 Solving Linear Equations
 13 Unconstrained Optimization and Neural Networks
 14 Global Search Algorithms
Part III. Linear Programming
 15 Introduction to Linear Programming
 16 Simplex Method
 17 Duality
 18 Nonsimplex Methods
 19 Integer Linear Programming
Part IV. Nonlinear Constrained Optimization
 20 Problems with Equality Constraints
 21 Problems With Inequality Constraints
 22 Convex Optimization Problems
 23 Algorithms for Constrained Optimization
 24 Lagrangian Duality
 25 Multiobjective Optimization
Part V Optimization in Machine Learning
 26 Machine Learning Problems and Feature Engineering
 27 Stochastic Gradient Descent Algorithms
 28 Linear Regression and Its Variants
 29 Logistic Regression for Classification
 30 Support Vector Machines
 31 KMeans Clustering
 References
 Index
Ordering information
Wiley has
information on how to order the book.
Instructors only: The Instructor's Solutions Manual is available
to Instructors who adopt the book. Please visit the
solutions
manual website (also linked from Wiley's companion site for the book)
and register to receive access to the solutions manual.
Useful links:
Professor Edwin Chong,
This document was last modified
August 16, 2024.