Calculus of Variations and Optimal Control Theory (Liberzon, 2011)
A unified graduate-level route from variational first-order conditions through the Pontryagin maximum principle, dynamic programming, viscosity solutions, and finite- and infinite-horizon LQR. It is GTK's main explanatory source for keeping conventions and logical dependencies consistent across the optimal-control branch.
Daniel Liberzon's 200-page 2011 prepublication text is unusually useful because it develops calculus of variations, Pontryagin's principle, HJB, and LQR with one notation and repeatedly explains how the methods connect. GTK uses it for derivations and counterexamples, not as a chapter-order template.
Coverage #
| Part | Printed pages | Role in GTK |
|---|---|---|
| Static optimization and classical variation | 1–46 | First and second variations, Euler–Lagrange equation, variable endpoints, constraints |
| From variational control to the maximum principle | 47–121 | Control constraints, needle variations, endpoint conditions, Hamiltonian properties, double integrator, singular controls, existence |
| HJB and nonsmooth value functions | 125–143 | Principle of optimality, HJB derivation, verification, PMP connection, viscosity interpretation |
| LQR | 145–160 | Finite-horizon feedback, Riccati differential equation, infinite-horizon limit and stability |
| Advanced connections | 161–180 | Characteristics, symplectic structure, Riccati inequalities, hybrid systems |
Load-bearing conclusions #
- The cost functional acts on a function , so the optimization problem is infinite-dimensional even when the state at each time is finite-dimensional.
- Euler–Lagrange and PMP are necessary conditions. They generate candidates; feasibility, endpoint conditions, second-order information, convexity, or a verification theorem must still establish optimality.
- HJB is a sufficient route when a sufficiently regular value function solves the PDE with the correct boundary condition. When the value function is nonsmooth, viscosity solutions preserve the right generalized statement.
- PMP costates are characteristics of HJB wherever the value function is differentiable: after matching sign conventions.
- Finite-horizon LQR is a boundary-value problem in time: integrate the Riccati differential equation backward from the terminal cost and the closed-loop state forward from the initial condition.
Visual audit #
Every physical page was included in the reading pass. The book's information-bearing figures are mostly geometric variational constructions, endpoint cones, phase portraits, switching curves, and value-function sketches. GTK does not copy them; it uses independently drawn SVGs tailored to the exact convention of each entry.