Pontryagin Maximum Principle

The Pontryagin maximum principle (PMP) turns an optimization over whole control functions into coupled differential equations plus an instantaneous optimization over the admissible input set. It is powerful precisely because it respects hard input constraints and discontinuous optimal controls. It is a necessary-condition engine, not a global-optimality certificate.

One sign convention, stated once #

Consider the minimization problem

$$J[u]=\Phi(t_f,x(t_f)) +\int_{t_0}^{t_f}L(t,x(t),u(t))dt, \qquad \dot x=f(t,x,u),\quad u(t)\in U.$$

GTK uses the maximum convention

$$\boxed{H(t,x,p,u,\lambda_0)=p^Tf(t,x,u)-\lambda_0L(t,x,u)},$$

where $p(t)\in\mathbb R^n$ is the costate and $\lambda_0\ge0$ is the cost multiplier. Many books instead define $H=L+p^Tf$ and minimize it. Both conventions are valid; mixing their adjoint, terminal, and optimization signs is not.

The necessary conditions #

For an optimal process $(x^*,u^*)$, under the theorem's regularity assumptions there exist $\lambda_0\ge0$ and an absolutely continuous costate $p(\cdot)$, not both identically zero, such that almost everywhere:

$$\dot x^*=\frac{\partial H}{\partial p} =f(t,x^*,u^*),$$

$$\boxed{\dot p=-\frac{\partial H}{\partial x} =-f_x(t,x^*,u^*)^Tp+\lambda_0L_x(t,x^*,u^*)},$$

and

$$\boxed{H(t,x^*,p,u^*,\lambda_0) =\max_{v\in U}H(t,x^*,p,v,\lambda_0).}$$

The state runs forward from $x(t_0)=x_0$. The costate is constrained at the terminal end. That makes the PMP system a two-point boundary-value problem rather than an ordinary initial-value simulation.

For a free terminal state and terminal cost $\Phi$, this convention gives

$$p(t_f)=-\lambda_0\nabla_x\Phi(t_f,x(t_f)).$$

If the terminal state must lie on a smooth manifold $\psi(t_f,x(t_f))=0$, then for a multiplier $\nu$,

$$p(t_f)=-\lambda_0\nabla_x\Phi -\left(\nabla_x\psi\right)^T\nu.$$

Equivalently, after subtracting the terminal-cost gradient, the costate is normal to the target manifold. If $t_f$ is free,

$$H(t_f)=\lambda_0\frac{\partial\Phi}{\partial t_f} +\nu^T\frac{\partial\psi}{\partial t_f}.$$

For an autonomous problem, time-independent terminal cost, and a time-independent target, $H(t_f)=0$; along a smooth autonomous extremal, $H$ is constant, so $H\equiv0$.

Why a costate appears #

The differential constraint couples a control perturbation at one time to the terminal state. The costate propagates the marginal value of that state perturbation backward. Formally, augment the running integrand with

$$p^T(f-\dot x).$$

Integration by parts moves the variation of $\dot x$ onto $p$. Choosing $p$ to cancel every interior $\delta x$ term produces the adjoint equation; the surviving endpoint term produces transversality; the remaining first-order control term produces pointwise Hamiltonian optimization.

Needle variations make this argument work even when $U$ is bounded and $u^*$ jumps. Instead of requiring a small-amplitude variation, replace $u^*$ by another admissible value on a short interval of width $\varepsilon$ and let $\varepsilon\downarrow0$.

Control-affine systems and switching functions #

For

$$\dot x=f_0(t,x)+G(t,x)u,$$

the Hamiltonian is

$$H=p^Tf_0+p^TGu-\lambda_0L(t,x,u).$$

If $L$ does not depend on $u$ and the scalar input satisfies $|u|\le u_{\max}$, define the switching function

$$\sigma(t)=p(t)^Tg(t,x(t)).$$

Then

$$u^*(t)=u_{\max}\mathop{\mathrm{sgn}}\sigma(t)$$

whenever $\sigma(t)\ne0$. Saturation is not a heuristic here: a nonzero linear functional reaches its maximum over an interval at an endpoint.

For a box $|u_i|\le \bar u_i$,

$$u_i^*=\bar u_i\mathop{\mathrm{sgn}}\sigma_i, \qquad \sigma=G^Tp.$$

For a Euclidean ball $\lVert u\rVert_2\le u_{\max}$,

$$u^*=u_{\max}\frac{G^Tp}{\lVert G^Tp\rVert_2}, \qquad G^Tp\ne0.$$

Thus a box selects a vertex componentwise, while a ball selects the boundary point aligned with the switching vector.

Singular arcs #

If $\sigma(t)=0$ only at isolated instants, it merely marks switches. If

$$\sigma(t)\equiv0$$

on an interval, first-order Hamiltonian maximization does not determine $u$. This is a singular arc. Differentiate the switching function along the state-costate dynamics until the control appears:

$$\sigma=0,quad \dot\sigma=0,quad \ldots,quad \sigma^{(r)}(x,p,u)=0.$$

Solving the first derivative that contains $u$ gives a singular candidate. It must also satisfy the input bounds and a higher-order condition such as the generalized Legendre–Clebsch condition. Setting $u=0$ merely because $\sigma=0$ is unjustified.

Normal and abnormal extremals #

If $\lambda_0>0$, scale all multipliers so $\lambda_0=1$; the extremal is normal. If $\lambda_0=0$, the running and terminal costs disappear from $H$ and the necessary conditions depend only on geometry and constraints. Such an abnormal extremal can be real; setting $\lambda_0=1$ without checking may exclude it.

The nontriviality condition

$$(\lambda_0,p(t))\ne(0,0)$$

prevents a vacuous multiplier solution that would satisfy every homogeneous equation.

Necessary does not mean optimal #

A PMP extremal can fail for several reasons:

• it violates a state or endpoint constraint;
• it is a local maximum or saddle of the cost;
• another extremal has lower cost;
• a singular arc violates a higher-order condition;
• the infimum is not attained at all.

For convex problems, PMP plus feasibility can become sufficient. One common sufficient setting has convex $U$, affine dynamics, convex running and terminal costs, and compatible convex endpoint constraints, expressed in the corresponding minimum-Hamiltonian convention.

PMP and HJB are the same geometry locally #

Let $V(t,x)$ be a differentiable value function for the minimization problem. HJB chooses

$$u^*\in\arg\min_{u\in U} \left[L+\nabla_xV^Tf\right].$$

With this article's maximum convention, set

$$p=-\nabla_xV.$$

Then maximizing $p^Tf-L$ is exactly the same pointwise choice. Along a smooth optimal trajectory, differentiating $p=-V_x$ recovers the adjoint equation. PMP follows one characteristic through state-costate space; HJB solves for the value over an entire state region. At switching surfaces $V$ is often nondifferentiable, which is why the viscosity-solution formulation matters.

Solving a PMP boundary-value problem #

1. Fix the formulation and Hamiltonian convention.
2. Write state, costate, maximization, and every transversality condition.
3. Determine bang, interior, and singular control laws from the maximization.
4. Guess unknown initial costates, switching times, and possibly $t_f$.
5. Integrate the state and costate and drive terminal residuals to zero.
6. Reject constraint-violating or inconsistent arcs.
7. Compare costs and use sufficiency, second variation, or HJB verification where available.

This shooting workflow is numerically sensitive because unstable costate directions amplify an initial guess. Direct transcription and collocation are often more robust computationally, but they discretize the same continuous necessary conditions and should be checked against their residuals and constraints.