When the Newton Update Breaks Down
Newton’s method uses the local slope of a function to estimate where a root lies. That slope is essential: the method divides the function value by its derivative. If the derivative is zero, the usual update is not defined. If the derivative is nonzero but very small, the update may be very large or sensitive to small errors.
These are related but distinct situations. A zero derivative can make a Newton step impossible, even when the function has a root elsewhere. A small derivative does not automatically make a step bad: the size of the step depends on the ratio of the function value to the derivative, not on the derivative alone.
The update is intended to find a root, a value \(x\) for which \(f(x)=0\). If \(f(x_n)=0\) already, the root has been found; one should stop rather than attempt a further update. This distinction matters especially when both \(f(x_n)\) and \(f'(x_n)\) are zero.
Why a Zero Derivative Prevents the Step
At \(x_n\), Newton’s method uses the tangent-line approximation \(f(x_n)+f'(x_n)(x-x_n)\). It asks where this linear expression is zero. If \(f'(x_n)\ne 0\), solving for \(x\) gives the Newton update. If \(f'(x_n)=0\), the expression is simply the constant \(f(x_n)\): it is either never zero, or it is zero for every \(x\). Neither case selects a unique next iterate.
This explains the failure directly. With zero slope, the tangent is horizontal, so it does not cross the horizontal axis unless it lies on that axis. If it lies on the axis, the current point is already a root; if not, the tangent offers no intersection to use as the next estimate.
Worked Example: A Horizontal Tangent Away from Any Root
Consider \(f(x)=x^2+1\), starting at \(x_0=0\). The function and its derivative are \(f(x)=x^2+1\) and \(f'(x)=2x\). Thus \(f(0)=1\), while \(f'(0)=0\).
The Newton formula would require \[ x_1=0-\frac{1}{0}, \] which is undefined. In the tangent-line equation, the approximation at zero is \(1+0(x-0)=1\), which never equals zero. In fact, \(x^2+1>0\) for every real \(x\), so this function has no real root to find. The calculation confirms that the method cannot take a step from the chosen point.
A zero derivative does not, by itself, tell us that the equation has no root. It tells us only that the current tangent-line step cannot be computed. A root may exist elsewhere, or the current point may already be a root.
Worked Example: A Root Exists, but the Starting Tangent Does Not Help
Let \(f(x)=x^3-1\) and \(x_0=0\). Then \(f(0)=-1\) and \(f'(x)=3x^2\), so \(f'(0)=0\). The proposed update is \[ x_1=0-\frac{-1}{0}, \] which is undefined. The tangent-line approximation is the constant \(-1\), so it has no zero.
Nevertheless, \(f(1)=1^3-1=0\), so the equation has the root \(x=1\). The failure is due to the starting point’s horizontal tangent, not the absence of a root. Substituting \(x=1\) verifies the root exactly.
A Small Derivative Can Produce a Large Step
When \(f'(x_n)\) is nonzero but close to zero, division is possible, but the quotient \(f(x_n)/f'(x_n)\) may be large. The resulting iterate can be far from the current one. A large step is not proof that the method has failed permanently, but it is a warning: the local tangent may be a poor guide over such a distance.
Worked Example: A Large Step from a Small Derivative
Take \(f(x)=x^2-1\) with \(x_0=\frac{1}{1000}\). The derivative is \(f'(x)=2x\), so \[ f(x_0)=\frac{1}{1{,}000{,}000}-1=-\frac{999{,}999}{1{,}000{,}000}, \qquad f'(x_0)=\frac{2}{1000}=\frac{1}{500}. \] The derivative is nonzero, so the update is defined. Compute it exactly: \[ \frac{f(x_0)}{f'(x_0)} = -\frac{999{,}999}{1{,}000{,}000}\cdot 500 = -\frac{999{,}999}{2000}, \] and therefore \[ x_1=\frac{1}{1000}-\left(-\frac{999{,}999}{2000}\right) =\frac{2+999{,}999}{2000} =\frac{1{,}000{,}001}{2000} =500.0005. \] The first step has moved from \(0.001\) to \(500.0005\). Substituting the result into the function gives \[ f(x_1) =\left(\frac{1{,}000{,}001}{2000}\right)^2-1 =\frac{999{,}998{,}000{,}001}{4{,}000{,}000}, \] which is a large positive residual, not a value close to zero. Thus the computed step is valid algebraically, but it has not improved the residual in this example.
The example illustrates the ratio in the update. Here the function value is near \(-1\), while the derivative is only \(0.002\); dividing a value of that size by a small slope produces a large displacement. The tangent is nearly horizontal and must travel a long distance to reach the axis.
Zero Over Zero: Stop If the Root Is Already Found
When both \(f(x_n)=0\) and \(f'(x_n)=0\), the written update has the indeterminate quotient \(0/0\). That does not mean Newton’s method should try to resolve the quotient. The equation has already been solved at \(x_n\); the appropriate action is to report the root and stop.
Worked Example: A Root with Zero Derivative
Let \(f(x)=x^3\), with \(x_0=0\). Then \(f(0)=0\) and \(f'(x)=3x^2\), so \(f'(0)=0\). Substitution into the update would give \(x_1=0-\frac{0}{0}\), which is undefined.
But \(f(0)=0^3=0\), so \(x_0=0\) is already an exact root. There is no need to compute \(x_1\). The correct check is the function value, not an attempt to assign a value to \(0/0\).
Why Small Derivatives Can Make Computations Sensitive
A small denominator can amplify errors in the quantities being divided. To see the mechanism, write the Newton correction as \(q=f/f'\), where the values are evaluated at the current point. For nonzero \(f'\), a small change in the numerator and denominator gives the first-order change \[ dq\approx \frac{df}{f'}-\frac{f\,df'}{(f')^2}. \] This follows by differentiating the quotient \(f/f'\). When \(f'\) is small, the factors \(1/f'\) and \(1/(f')^2\) can be large, so small errors in \(f\) or \(f'\) may noticeably change the correction. The approximation describes sensitivity; it does not imply that every small derivative causes a poor step.
There is an important qualification: the absolute size of \(f'\) depends on how the equation is scaled. Replacing \(f\) by \(g=cf\), where \(c\ne 0\) is a constant, changes both \(f\) and \(f'\) by the same factor. The Newton correction remains unchanged:
For example, if \(g(x)=10^{-6}(x^2-1)\), then at \(x_0=0.001\) both the function value and derivative are \(10^{-6}\) times the corresponding values for \(f(x)=x^2-1\). Their quotient, and hence the Newton step, is exactly the same. Therefore a derivative being numerically small in isolation is not enough to conclude that the step is unstable. One must consider the quotient, the function’s scale, and the effects of computational error.
A Practical Diagnosis
Before computing a Newton step, evaluate the function and derivative at the current iterate. The following distinctions keep the reasoning precise:
| Values at \(x_n\) | Interpretation | Action |
|---|---|---|
| \(f(x_n)=0\) | The current point is a root, whether or not \(f'(x_n)=0\). | Stop and report the root. |
| \(f(x_n)\ne 0\) and \(f'(x_n)=0\) | The standard Newton correction cannot be computed. | Do not divide by zero; choose another starting point or method. |
| \(f(x_n)\ne 0\) and \(f'(x_n)\ne 0\), but the derivative is small | The update exists; its size and sensitivity need attention. | Compute the correction and assess whether the step is plausible. |
In computation, exact equality may be difficult to decide when values are rounded. A program may use a threshold to flag a derivative that is too small for its numerical precision. Such a threshold depends on the scale of the problem and the arithmetic being used; there is no universal absolute cutoff that applies to every function. Moreover, any method that changes or limits a Newton step is a modified procedure, not the unaltered Newton update.
If \(f(x_n)=0\), the current point is a root, so stop rather than form a quotient.
If \(f'(x_n)=0\) while \(f(x_n)\ne0\), the Newton formula has no defined next iterate.
When \(f'(x_n)\ne0\), compute \(-f(x_n)/f'(x_n)\) and notice whether the proposed displacement is large relative to the current scale.
If the step cannot be formed or appears unreliable, do not treat it as a trustworthy root estimate; reconsider the starting point or use a suitable alternative root-finding approach.
A Common Mistake
A frequent error is to treat a zero derivative as if it simply produced a zero step. For \(f(x)=x^2+1\) at \(x_0=0\), one might incorrectly write \[ x_1=0-\frac{f(0)}{f'(0)}=0-\frac{1}{0}=0. \] This is invalid: division by zero is undefined, so the displayed expression does not equal zero. The correct conclusion is that the Newton update cannot be computed at this point. By contrast, for \(f(x)=x^3\) at zero, the quotient would be \(0/0\), but the correct response is still not to invent a step: verify \(f(0)=0\), recognize that the root has already been found, and stop.
The essential habit is to inspect the numerator and denominator separately before applying the formula. That simple check distinguishes a completed computation, a blocked iteration, and a potentially sensitive step.
Check Your Understanding
Answer each question by referring to the function value, derivative, and Newton update.
- For \(f(x)=x^2+1\) at \(x_0=0\), why is the Newton update undefined, and what does the tangent-line approximation look like?
- Can a function have a root even when Newton’s method cannot take a step from the chosen starting value? Explain using \(f(x)=x^3-1\) at zero.
- For \(f(x)=x^3\) at \(x_0=0\), why should one stop rather than evaluate the Newton quotient?
- What feature of the correction can make a nonzero but small derivative produce a large step?
- Why does multiplying a function by a nonzero constant leave its Newton correction unchanged?
