Multivariable Differential Calculus

A derivative in several variables is not a list of slopes. It is the single linear map that best predicts the output change caused by every sufficiently small input change. Partial derivatives are coordinates of that map when the map exists; they are not, by themselves, proof that it exists.

Functions, domains, and limits #

A scalar field is a function $f:D\subseteq\mathbb R^n\to\mathbb R$. A vector-valued map has the form $F:D\subseteq\mathbb R^n\to\mathbb R^m$. Before differentiating, identify the actual domain. Roots, logarithms, denominators, and inverse trigonometric functions can create boundaries or holes that later affect tangent formulas and conservative-field tests.

The limit

$$\lim_{x\to a}f(x)=L$$

means that every sufficiently near point of the domain, regardless of its direction of approach, gives a value near $L$. Testing paths is therefore asymmetric:

• two paths with different limits disprove the multivariable limit;
• agreement along many selected paths does not prove the limit.

A reliable proof usually bounds $|f(x)-L|$ by an expression depending only on $\|x-a\|$.

For example,

$$f(x,y)=\frac{x^2y}{x^2+y^2},\qquad f(0,0)=0$$

satisfies

$$|f(x,y)|\le |y|\frac{x^2}{x^2+y^2}\le |y|\le \sqrt{x^2+y^2},$$

so $f(x,y)\to0$ as $(x,y)\to(0,0)$.

Differentiability is a linear approximation #

A map $F:\mathbb R^n\to\mathbb R^m$ is differentiable at $a$ if there is a linear map $DF(a)$ such that

$$F(a+h)=F(a)+DF(a)h+r(h), \qquad \frac{\|r(h)\|}{\|h\|}\to0.$$

For a scalar field $f:\mathbb R^n\to\mathbb R$,

$$Df(a)h=\nabla f(a)\cdot h.$$

For a vector-valued map, the matrix of $DF(a)$ is the Jacobian

$$J_F(a)= \begin{bmatrix} \partial F_1/\partial x_1 & \cdots & \partial F_1/\partial x_n\\ \vdots & \ddots & \vdots\\ \partial F_m/\partial x_1 & \cdots & \partial F_m/\partial x_n \end{bmatrix}_{x=a}.$$

A practical sufficient condition is that all first partial derivatives exist in a neighborhood of $a$ and are continuous at $a$. This is sufficient, not necessary.

Why partial derivatives are weaker #

The function

$$f(x,y)= \begin{cases} \dfrac{x^2y}{x^4+y^2},&(x,y)\ne(0,0),\\ 0,&(x,y)=(0,0) \end{cases}$$

has both coordinate partial derivatives equal to zero at the origin. Along $y=x^2$, however, $f(x,x^2)=1/2$. It is not even continuous there, so it cannot be differentiable. Coordinate-axis behavior did not control all directions.

Chain rule and total differential #

If $F:\mathbb R^n\to\mathbb R^m$ is differentiable at $a$, and $G:\mathbb R^m\to\mathbb R^p$ is differentiable at $F(a)$, then

$$D(G\circ F)(a)=DG(F(a))\,DF(a).$$

This is matrix multiplication because a small increment is transformed twice:

$$h \xmapsto{DF(a)} \Delta y \xmapsto{DG(F(a))} \Delta z.$$

For $z=f(x,y)$, $x=x(u,v)$, and $y=y(u,v)$,

$$\frac{\partial z}{\partial u} =f_x\frac{\partial x}{\partial u} +f_y\frac{\partial y}{\partial u}, \qquad \frac{\partial z}{\partial v} =f_x\frac{\partial x}{\partial v} +f_y\frac{\partial y}{\partial v}.$$

The total differential

$$df=f_x\,dx+f_y\,dy$$

is the linear part of the change, not an equality with the finite change $\Delta f$. More precisely,

$$\Delta f=df+o\!\left(\sqrt{(\Delta x)^2+(\Delta y)^2}\right).$$

A sensitivity calculation with units #

For an ideal resistor, electrical power is

$$P(V,R)=\frac{V^2}{R}.$$

At $V=12\ \mathrm V$ and $R=6\ \Omega$, the power is $24\ \mathrm W$. The two partial derivatives at that operating point are

$$P_V=\frac{2V}{R}=4\ \mathrm{W/V}, \qquad P_R=-\frac{V^2}{R^2}=-4\ \mathrm{W/\Omega}.$$

If the voltage rises by $0.1\ \mathrm V$ while resistance rises by $0.2\ \Omega$, the total differential predicts

$$\Delta P\approx P_V\Delta V+P_R\Delta R =4(0.1)-4(0.2)=-0.4\ \mathrm W.$$

The exact new power is $12.1^2/6.2\approx23.615\ \mathrm W$, a change of about $-0.385\ \mathrm W$. This is what the derivative does in an engineering model: it converts a vector of small input changes or tolerances into a first-order output change without re-evaluating every nearby case. A Jacobian does the same job for several outputs at once.

Gradient and directional derivatives #

For a unit vector $u$, differentiability gives

$$D_u f(a)=Df(a)u=\nabla f(a)\cdot u.$$

Cauchy–Schwarz then yields

$$|D_u f(a)|\le \|\nabla f(a)\|.$$

When $\nabla f(a)\ne0$, the steepest increase is in the gradient direction, the steepest decrease is opposite it, and the maximum rate is $\|\nabla f(a)\|$.

The unit-vector condition matters. If $v$ is not normalized, $\nabla f\cdot v$ is the derivative per unit of the parameter multiplying $v$, not per unit distance.

For a regular level surface $F(x,y,z)=c$, any tangent curve $r(t)$ satisfies

$$F(r(t))=c \quad\Longrightarrow\quad \nabla F(r(t))\cdot r'(t)=0.$$

Thus $\nabla F(a)$ is normal to the surface only when $\nabla F(a)\ne0$. At singular points the usual tangent-plane formula can fail or cease to determine a unique plane.

Taylor models and the Hessian #

If $f$ is twice continuously differentiable near $a$, then

$$f(a+h) =f(a)+\nabla f(a)\cdot h +\frac12 h^\mathsf T H_f(a)h +o(\|h\|^2),$$

where $H_f$ is the Hessian matrix.

At a stationary point, $\nabla f(a)=0$, the quadratic form $h^\mathsf T H_f(a)h$ gives the leading local shape:

• positive definite Hessian: strict local minimum;
• negative definite Hessian: strict local maximum;
• indefinite Hessian: saddle point;
• semidefinite or singular Hessian: the test is inconclusive.

“Inconclusive” does not mean “no extremum.” Both $x^4+y^4$ and $x^4-y^4$ have zero Hessian at the origin, but the first has a minimum and the second a saddle.

Unconstrained and constrained extrema #

For a differentiable function, an interior local extremum must satisfy $\nabla f=0$. Absolute extrema on a compact domain require more work:

1. find interior critical points;
2. analyze every boundary component;
3. include corners, endpoints, and singular boundary points;
4. compare the function values.

The boundary step is essential. A Hessian test sees only local interior behavior.

For a smooth constraint $g(x)=0$, a constrained local extremum $x_*$ with $\nabla g(x_*)\ne0$ satisfies

$$\nabla f(x_*)=\lambda\nabla g(x_*).$$

The reason is geometric: feasible velocities $v$ lie in the tangent space $\nabla g(x_*)\cdot v=0$, and at an extremum $\nabla f(x_*)\cdot v=0$ for every such $v$. Hence both gradients are normal to the same tangent space.

The multiplier equation is only a candidate condition. Classify candidates by comparing values or by restricting second-order changes to tangent directions.

Worked constrained example #

Interpret $x$ and $y$ as loads assigned to two channels whose required total is $x+y=3$. Suppose their normalized losses are $x^2$ and $2y^2$, so the second channel penalizes additional load more strongly. Minimize $f(x,y)=x^2+2y^2$ subject to $x+y=3$.

Set $g=x+y-3$. Then

$$(2x,4y)=\lambda(1,1),$$

so $2x=4y$, hence $x=2y$. Together with $x+y=3$, this gives $(x,y)=(2,1)$. Because the constraint is a closed affine line and $f$ is strictly convex, this candidate is the unique global minimum, with value $6$.

The strict-convexity argument is stronger than merely solving the multiplier equations: it supplies global uniqueness.

The optimum assigns more load to the less costly channel. At $(2,1)$ the marginal losses, $2x$ and $4y$, are equal. If they were unequal, moving a small amount of load from the higher-marginal-loss channel to the other would reduce the total. That marginal balance is the operational meaning of the multiplier condition.

Failure modes #

• Treating partial derivatives as proof of differentiability.
• Using several matching paths as proof of a limit.
• Writing a tangent plane to $F=c$ without checking $\nabla F\ne0$.
• Calling $\nabla f\cdot v$ a directional derivative without normalizing $v$.
• Applying the Hessian test at a nonstationary point.
• Declaring “no extremum” when the Hessian test is inconclusive.
• Solving only $\nabla f=0$ for an absolute-extremum problem and ignoring the boundary.
• Using Lagrange multipliers where the constraint gradient vanishes, or applying a second-order test in all directions instead of feasible tangent directions.