Optimal-Control Problem Formulation

Optimal control does not begin with a Riccati equation or a Hamiltonian. It begins by saying exactly which trajectories are physically possible, which endpoint conditions must be met, and what scalar quantity distinguishes a better trajectory from a worse one. A beautifully solved wrong formulation is still the wrong controller.

The complete decision problem #

For a finite-horizon deterministic system, the standard Bolza problem is

$$\begin{aligned} \underset{u(\cdot),\,t_f}{\mathop{\mathrm{minimize}}}\quad J[u] &= \Phi\!\left(t_f,x(t_f)\right) +\int_{t_0}^{t_f}L\!\left(t,x(t),u(t)\right)\,dt,\\ \text{subject to}\quad \dot x(t) &= f\!\left(t,x(t),u(t)\right),\\ x(t_0)&=x_0,\\ u(t)&\in U(t,x(t)),\\ x(t)&\in X(t),\\ \psi\!\left(t_f,x(t_f)\right)&=0,\\ \eta\!\left(t_f,x(t_f)\right)&\le 0. \end{aligned}$$

Here $x(t)\in\mathbb R^n$ is the state, $u(t)\in\mathbb R^m$ the control, $f$ the dynamics, $L$ the running cost, and $\Phi$ the terminal cost. The set $U$ represents actuator limits and $X$ state constraints. The terminal equalities $\psi=0$ and inequalities $\eta\le0$ describe a fixed endpoint, a target manifold, or a target region.

The optimization variable is the whole function $u(\cdot)$, not the instantaneous number $u(t)$. The state is not an independent choice: once $x_0$ and an admissible $u(\cdot)$ are fixed, the differential equation determines $x(\cdot)$, provided the dynamics admit a unique solution.

Every field changes the answer #

An actuator command also needs units and semantics. If $u$ is voltage but the physical resource is propellant mass flow, $\int|u|dt$ is not automatically “fuel.” The plant model must contain the calibration that makes the integral proportional to the claimed resource.

Bolza, Lagrange, and Mayer forms #

The Lagrange form has only the integral cost,

$$J=\int_{t_0}^{t_f}L(t,x,u)\,dt,$$

whereas the Mayer form has only terminal cost,

$$J=\Phi(t_f,x(t_f)).$$

They are representational variants, not different theories. A Bolza problem becomes Mayer form by adding the accumulator state

$$\dot x_{n+1}=L(t,x,u),\qquad x_{n+1}(t_0)=0,$$

and using the terminal cost

$$\widetilde\Phi(t_f,\widetilde x(t_f)) =\Phi(t_f,x(t_f))+x_{n+1}(t_f).$$

Conversely, a differentiable terminal cost can sometimes be folded into an integral by differentiating $\Phi(t,x(t))$ along trajectories, although this is usually less convenient.

Three objectives that must not be conflated #

Minimum time #

With free $t_f$, fixed target $x(t_f)\in X_f$, and $L\equiv1$,

$$J=t_f-t_0=\int_{t_0}^{t_f}1\,dt.$$

There is no direct penalty for large control except the hard set $U$. Consequently the optimum often stays on the actuator boundary and changes sign at switching surfaces.

Minimum absolute effort #

For a scalar input,

$$J=\int_{t_0}^{t_f}\rho(t)|u(t)|\,dt,$$

and for independent channels,

$$J=\int_{t_0}^{t_f}\sum_{i=1}^{m}\rho_i(t)|u_i(t)|\,dt.$$

The corner at $u=0$ creates a dead zone in the Hamiltonian maximization, so optimal candidates are commonly bang-off-bang. Free terminal time can permit a sequence whose cost approaches an infimum while its duration diverges.

Quadratic regulation #

For a linear model $\dot x=Ax+Bu$,

$$J=\frac12x(t_f)^TS_fx(t_f) +\frac12\int_{t_0}^{t_f} \left[x(t)^TQ(t)x(t)+u(t)^TR(t)u(t)\right]dt.$$

When $Q\succeq0$, $S_f\succeq0$, and $R\succ0$, the control penalty is strictly convex. The unconstrained minimizer is smooth linear state feedback. This “quadratic effort” is a design metric; it is literal physical energy only when the variables and weights make the integrand a power.

Nondimensionalization and weights #

Terms added inside one scalar cost must have compatible dimensions. A practical normalization starts from meaningful state and input scales:

$$\bar x_i=\frac{x_i}{x_{i,\mathrm{scale}}},\qquad \bar u_j=\frac{u_j}{u_{j,\mathrm{scale}}}.$$

Then a dimensionless quadratic integrand can be written

$$L=\sum_i q_i\bar x_i^2+\sum_j r_j\bar u_j^2.$$

For diagonal physical-coordinate weights this corresponds to

$$Q_{ii}=\frac{q_i}{x_{i,\mathrm{scale}}^2},\qquad R_{jj}=\frac{r_j}{u_{j,\mathrm{scale}}^2}.$$

Multiplying the whole cost by a positive constant does not change the minimizer. Changing one relative weight does. That is why “large $Q$” has no meaning until the coordinate scaling and $R$ are known.

Fixed target versus tracking #

Tracking is most cleanly expressed in error coordinates. If a constant reference $(x_r,u_r)$ satisfies

$$0=f(x_r,u_r),$$

define $e=x-x_r$ and $v=u-u_r$. The target becomes $e=0$, and linearization gives

$$\dot e=Ae+Bv.$$

If the reference is time-varying, the error dynamics include $\dot x_r$ and generally cannot be obtained by merely replacing $x$ with $x-r$ in a regulator formula.

Existence is a separate question #

Necessary conditions can be derived even when no optimizer exists. An existence theorem normally needs some combination of:

• nonempty admissible trajectories and a reachable terminal set;
• regular dynamics with solutions defined for admissible controls;
• compactness or coercivity preventing controls from escaping to infinity;
• closed endpoint and path constraints;
• lower semicontinuity of the cost;
• convexity of the attainable velocity-cost set, or a relaxed-control formulation.

Formulation checklist #

Before applying PMP or HJB, write down:

1. state coordinates and their units;
2. the exact dynamics and validity region;
3. the admissible control function class and actuator geometry;
4. all state and endpoint constraints;
5. whether $t_f$ is fixed or free;
6. every running and terminal cost term, with units or normalization;
7. whether a minimizer is expected to exist;
8. whether the desired output is an open-loop extremal, feedback law, or implementable constrained controller.

That list is not bureaucracy. Every later equation depends on it.