One Approximation Must Work in Every Direction
A directional derivative examines a function along one chosen line. Differentiability asks for more: near a point, can one linear rule approximate the function’s change for every sufficiently small displacement, regardless of its direction? This is a single approximation that must remain accurate as the displacement approaches zero along any path, not just along a fixed line.
Let \(U\) be open in \(\mathbb{R}^m\), let \(f:U\to\mathbb{R}^n\), and fix \(a\in U\). Since \(U\) is open, \(a+h\in U\) for every sufficiently small \(h\in\mathbb{R}^m\). We use the Euclidean norm in each space. A linear map \(L:\mathbb{R}^m\to\mathbb{R}^n\) gives a candidate first-order approximation to the change \(f(a+h)-f(a)\).
The condition says that the error becomes negligible compared with the size of the displacement. It does not say the error is exactly zero, nor that its size is bounded by a fixed multiple of \(\|h\|_2\) alone. The essential requirement is that the ratio of error to displacement tends to zero as \(h\to0\), with no restriction on the direction in which \(h\) approaches zero.
When \(n=1\), the norm in the numerator is absolute value and \(L\) is a real-valued linear map on \(\mathbb{R}^m\). When \(m=n=1\), every such map has the form \(L(h)=ch\), so this definition reduces to ordinary differentiability, with \(c=f'(a)\).
The Linear Approximation Is Unique
The definition asks whether there exists a suitable linear map. It is also important that there cannot be two different choices: otherwise the derivative would not be well-defined.
Proof. Suppose both \(L\) and \(M\) satisfy the definition. Fix any nonzero \(v\in\mathbb{R}^m\), and take \(h=tv\) with \(t>0\). As \(t\to0\), each approximation condition gives \(\|f(a+tv)-f(a)-L(tv)\|_2/t\to0\) and \(\|f(a+tv)-f(a)-M(tv)\|_2/t\to0\). By the triangle inequality, $$ \frac{\|L(tv)-M(tv)\|_2}{t} \leq \frac{\|f(a+tv)-f(a)-L(tv)\|_2}{t} + \frac{\|f(a+tv)-f(a)-M(tv)\|_2}{t}. $$ The right-hand side tends to zero. By linearity, the left-hand side is \(\|L(v)-M(v)\|_2\), independent of \(t\). Therefore \(\|L(v)-M(v)\|_2=0\), so \(L(v)=M(v)\). Both maps also send zero to zero, and \(v\) was arbitrary. Hence \(L=M\). \(\square\)
We can therefore refer to the unique map as the derivative of \(f\) at \(a\), often denoted \(f'(a)\) or \(Df(a)\). The notation emphasizes that the derivative is a linear map, rather than just a list of rates computed separately along selected lines.
Worked Example: An Affine Map
Let \(f:\mathbb{R}^2\to\mathbb{R}^2\) be \(f(x,y)=(2x-y,x+3y)\), and fix any \(a=(a_1,a_2)\). For \(h=(h_1,h_2)\), direct substitution gives $$ f(a+h)-f(a) = \bigl(2(a_1+h_1)-(a_2+h_2),\, (a_1+h_1)+3(a_2+h_2)\bigr) - (2a_1-a_2,\,a_1+3a_2) = (2h_1-h_2,\,h_1+3h_2). $$ Define \(L(h_1,h_2)=(2h_1-h_2,h_1+3h_2)\). This is linear, and the displayed calculation shows that the remainder is exactly zero for every \(h\). Thus the ratio in the definition is zero for every nonzero \(h\), and \(f\) is differentiable at \(a\) with derivative \(L\).
Differentiability Implies Continuity
A linear approximation has a direct consequence: the function’s values approach \(f(a)\) as the input approaches \(a\). We use the fact that a linear map between finite-dimensional Euclidean spaces is bounded. In this setting, that fact follows by writing a vector in standard coordinates and applying the triangle inequality.
Proof. Let \(L\) be the derivative and write \(f(a+h)-f(a)=L(h)+r(h)\), where \(\|r(h)\|_2/\|h\|_2\to0\). For the standard basis vectors \(e_1,\ldots,e_m\), linearity and the triangle inequality give $$ \|L(h)\|_2 = \left\|\sum_{j=1}^m h_jL(e_j)\right\|_2 \leq \sum_{j=1}^m |h_j|\,\|L(e_j)\|_2 \leq \left(\sum_{j=1}^m\|L(e_j)\|_2\right)\|h\|_2. $$ Thus \(\|L(h)\|_2\to0\) as \(h\to0\). Also, the remainder condition implies \(\|r(h)\|_2\to0\): for all sufficiently small nonzero \(h\), its ratio to \(\|h\|_2\) is at most \(1\), so \(\|r(h)\|_2\leq\|h\|_2\). Consequently, $$ \|f(a+h)-f(a)\|_2 \leq\|L(h)\|_2+\|r(h)\|_2\longrightarrow0. $$ This is continuity at \(a\). \(\square\)
Worked Example: A Quadratic Function
Let \(f(x,y)=x^2+xy+3y^2\) and \(a=(1,-1)\). For a displacement \(h=(u,v)\), first note that \(f(1,-1)=1-1+3=3\). Expanding each term, $$ \begin{aligned} f(1+u,-1+v) &=(1+u)^2+(1+u)(-1+v)+3(-1+v)^2\\ &=(1+2u+u^2)+(-1+v-u+uv)+(3-6v+3v^2)\\ &=3+u-5v+u^2+uv+3v^2. \end{aligned} $$ So the candidate linear map is \(L(u,v)=u-5v\), and the remainder is \(r(u,v)=u^2+uv+3v^2\). Since \(|uv|\leq (u^2+v^2)/2\), $$ |r(u,v)| \leq u^2+|uv|+3v^2 \leq \frac{9}{2}(u^2+v^2). $$ For every \((u,v)\neq(0,0)\), it follows that \(|r(u,v)|/\sqrt{u^2+v^2}\leq (9/2)\sqrt{u^2+v^2}\), which tends to zero. Therefore \(f\) is differentiable at \((1,-1)\), with derivative \(L(u,v)=u-5v\).
Directional Derivatives Follow from the Linear Approximation
If a function is differentiable, its directional derivatives are not independent pieces of information. Each one is obtained by applying the same derivative map to the chosen direction. For \(f:U\to\mathbb{R}^n\), define a directional derivative by the vector limit of the difference quotient, when that limit exists.
Proof. If \(v=0\), then \(a+tv=a\) for every \(t\), so the difference quotient is zero and \(L(v)=L(0)=0\). Now suppose \(v\neq0\). Write \(f(a+h)-f(a)=L(h)+r(h)\), with \(\|r(h)\|_2/\|h\|_2\to0\). For \(t\neq0\) sufficiently close to zero, set \(h=tv\). Linearity gives \(L(tv)=tL(v)\), and therefore $$ \frac{f(a+tv)-f(a)}{t} = L(v)+\frac{r(tv)}{t}. $$ The norm of the final term satisfies $$ \left\|\frac{r(tv)}{t}\right\|_2 = \frac{\|r(tv)\|_2}{|t|\|v\|_2}\,\|v\|_2 \longrightarrow 0, $$ because \(tv\to0\) and \(|t|\|v\|_2=\|tv\|_2\). Thus the difference quotient tends to \(L(v)\), as claimed. \(\square\)
The conclusion is stronger than merely knowing that directional derivatives exist: the directional rates must come from one linear map. In particular, as a function of \(v\), \(D_vf(a)\) must be linear. The existence of directional derivatives on their own does not guarantee this property, as the next example shows.
Worked Example: Directional Derivatives Do Not Ensure Differentiability
Define \(g:\mathbb{R}^2\to\mathbb{R}\) by $$ g(x,y)= \begin{cases} \dfrac{x^3}{x^2+y^2},&(x,y)\neq(0,0),\\ 0,&(x,y)=(0,0). \end{cases} $$ For any nonzero \(v=(p,q)\) and \(t\neq0\), substitution gives $$ g(tp,tq) = \frac{t^3p^3}{t^2p^2+t^2q^2} = \frac{tp^3}{p^2+q^2}. $$ Consequently, \(D_{(p,q)}g(0,0)=p^3/(p^2+q^2)\). The zero direction has derivative zero as well. Thus all directional derivatives exist.
But \(D_{(1,0)}g(0,0)=1\), \(D_{(0,1)}g(0,0)=0\), and \(D_{(1,1)}g(0,0)=1/2\). If \(g\) were differentiable at the origin, the theorem would imply that its directional derivatives come from a linear map. Additivity would then give \(D_{(1,1)}g(0,0)=D_{(1,0)}g(0,0)+D_{(0,1)}g(0,0)=1\), contradicting the calculated value \(1/2\). Hence \(g\) is not differentiable at the origin.
This function is nevertheless continuous there: for \((x,y)\neq(0,0)\), \(|g(x,y)|=|x|^3/(x^2+y^2)\leq |x|\leq\sqrt{x^2+y^2}\), and the final expression tends to zero at the origin. Thus continuity, like the existence of every directional derivative, is not by itself sufficient for differentiability.
Why the Remainder Condition Matters
A useful way to test a proposed derivative is to subtract the proposed linear term and estimate the remainder. A bound of the form \(\|r(h)\|_2\leq C\|h\|_2^2\), for a fixed constant \(C\), is enough: after division by \(\|h\|_2\), the bound becomes \(C\|h\|_2\), which tends to zero. The quadratic example used precisely this approach.
The main pitfall is checking only fixed lines. Directional derivatives take limits along individual lines through \(a\), whereas differentiability requires the approximation error to be small for all sufficiently small displacements. The direction of \(h\) may change as \(h\to0\). Differentiability therefore supplies a consistent linear first-order description; line-by-line rates alone may fail to do so.
Check Your Understanding
Use the definition and results in this tutorial to answer each question.
- In the definition of differentiability, what must happen to the ratio of the approximation error to \(\|h\|_2\)?
- Why can two distinct linear maps not both satisfy the definition of the derivative at the same point?
- How does the derivative map determine the directional derivative in a given direction \(v\)?
- What remainder bound in the quadratic example ensures that the error ratio tends to zero?
- Why does existence of all directional derivatives fail to establish differentiability in the final example?