Why Newton’s Method Can Become So Fast
Newton’s method uses tangent lines to approximate a root of an equation. Near a suitable root, each new tangent-line step can do much more than merely reduce the error: it can make the new error roughly proportional to the square of the old one. If an error is about \(10^{-2}\), squaring it gives about \(10^{-4}\), a substantial improvement in one step.
This behavior is local. It describes iterations that are already sufficiently close to the root, and it relies on the function having a nonzero derivative there. It does not promise that every initial guess will lead to the root.
We will assume that \(f\) has a continuous second derivative on an interval around \(\alpha\). The simple-root condition ensures that the denominator in Newton’s formula stays away from zero if the interval is chosen small enough. The second-derivative condition controls how much the function’s slope can change nearby.
Deriving the Local Error Relation
Apply Taylor’s theorem to \(f\) at \(x_n\), evaluating at the root \(\alpha\). For some \(\xi_n\) between \(x_n\) and \(\alpha\),
Since \(f(\alpha)=0\) and \(\alpha-x_n=-e_n\), rearranging gives
Subtract the root \(\alpha\) from the Newton update. The result is \(e_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}-\alpha\). Combining this with the rearranged Taylor formula yields the exact one-step error relation:
The point \(\xi_n\) depends on the step. In particular, this formula is not usually an exact recurrence involving only \(e_n\) and derivatives evaluated at \(\alpha\). It does, however, immediately give a useful bound if the second derivative is bounded and the first derivative is bounded away from zero.
Why the Error Bound Implies Convergence
Here is a precise local argument. Suppose \(f\) has a continuous second derivative on an interval \(J\) around \(\alpha\), and \(f'(\alpha)\ne 0\). Continuity of \(f'\) lets us choose a smaller closed interval \(I\) around \(\alpha\) and a number \(m>0\) such that \(|f'(x)|\ge m\) throughout \(I\). Continuity of \(f''\) also gives a finite number \(M\ge 0\) such that \(|f''(x)|\le M\) on \(I\).
As long as \(x_n\) lies in \(I\), the error relation gives
Set \(C=\max\{1,M/(2m)\}\), so \(C>0\) and the same inequality holds with \(C\) in place of \(M/(2m)\). Choose a radius \(\delta>0\) small enough that the interval \([\alpha-\delta,\alpha+\delta]\) lies inside \(I\) and \(C\delta\le \frac12\). If \(|e_n|\le\delta\), then
Thus the next iterate remains within the chosen interval. Repeating the argument shows that every subsequent iterate remains there and that its error is at most half the preceding error. Therefore the errors tend to zero. At the same time, they satisfy the stronger quadratic bound \(|e_{n+1}|\le C|e_n|^2\). This proves local convergence with a quadratic error estimate, rather than merely asserting that small errors tend to shrink.
Choose a neighborhood where \(|f'|\ge m>0\) and \(|f''|\le M\).
The exact error relation gives \(|e_{n+1}|\le (M/(2m))|e_n|^2\).
Make the neighborhood small enough that the bound also gives \(|e_{n+1}|\le |e_n|/2\).
The errors stay in the neighborhood, decrease to zero, and obey the quadratic estimate at each step.
The Asymptotic Error Constant
The bound proves local convergence, but it does not tell us the precise leading factor in the error. As \(x_n\) approaches \(\alpha\), the points \(\xi_n\) in the error relation also approach \(\alpha\). Continuity of \(f''\) and \(f'\), together with \(f'(\alpha)\ne0\), therefore gives
Equivalently, the signed error has the local expansion
Here \(o(e_n^2)\) denotes a remainder whose ratio to \(e_n^2\) tends to zero as \(e_n\) tends to zero. If \(f''(\alpha)\ne0\), the magnitude of the ratio \(|e_{n+1}|/|e_n|^2\) tends to the positive constant \(\left|f''(\alpha)/(2f'(\alpha))\right|\). This is the usual precise meaning of quadratic convergence. If \(f''(\alpha)=0\), the quadratic bound still applies, but this leading constant is zero; the error may then shrink even faster.
Worked Example: Two Newton Steps for a Simple Root
Worked Example: Two Newton Steps for a Simple Root
Apply Newton’s method to \(f(x)=x^2-1\), starting from \(x_0=\frac65\), near the simple root \(\alpha=1\). Here \(f(1)=0\), \(f'(x)=2x\), and \(f'(1)=2\ne0\).
At the starting value,
The first Newton step is
Thus \(e_0=\frac15\) and \(e_1=\frac1{60}\). For the second step,
Therefore
The errors are \(e_1=\frac1{60}\) and \(e_2=\frac1{7320}\). The second step is checked by substituting \(x_1\) into Newton’s formula above; the resulting value satisfies \(x_2-1=\frac1{7320}\). In particular, the first error has fallen from \(\frac15\) to \(\frac1{60}\), and the next error is much smaller still.
Worked Example: Checking a Local Error Bound
Worked Example: Checking a Local Error Bound
For \(f(x)=x^2-1\), use the interval \(I=[\frac34,\frac54]\) around \(\alpha=1\). On this interval \(f'(x)=2x\), so \(|f'(x)|\ge\frac32\). Also \(f''(x)=2\), so \(|f''(x)|\le2\). We may take \(m=\frac32\), \(M=2\), and hence
For \(x_0=\frac65\), the initial error is \(e_0=\frac15\), which is less than \(\frac14\), the radius of this interval about \(\alpha\). The error bound predicts
From the first worked example, \(x_1=\frac{61}{60}\), so the actual error is \(|e_1|=\frac1{60}\). The bound checks out because \(\frac1{60}\le\frac{2}{75}\), as \(75\le120\) after cross-multiplication. Moreover, \(\frac{2}{75}<\frac14\), so this estimate keeps the next iterate inside the chosen interval. The bound is an upper limit, not a prediction that the actual error must equal it.
Worked Example: Comparing with the Asymptotic Constant
Worked Example: Comparing with the Asymptotic Constant
Consider \(f(x)=x^2+x-2\), which has the root \(\alpha=1\). It is simple because \(f'(x)=2x+1\) and \(f'(1)=3\ne0\). Also \(f''(1)=2\), so the predicted leading error constant is
Start at \(x_0=\frac{11}{10}\), so \(e_0=\frac1{10}\). Direct calculation gives
The Newton update is therefore
Thus \(e_1=\frac1{320}\), and the actual ratio is
This is close to, but not exactly, the limiting value \(\frac13\). The example checks the distinction between the asymptotic relation and an exact one-step formula: at this starting point the error is small, but it is not zero, so the higher-order remainder need not vanish.
A Common Pitfall: Treating the Leading Term as Exact
A tempting but incorrect step is to replace \(f''(\xi_n)\) and \(f'(x_n)\) in the exact error relation by \(f''(\alpha)\) and \(f'(\alpha)\), then claim an exact formula for every Newton step. Those replacements are justified only in the limit as the iterate approaches the root, not at an arbitrary nearby point.
For a concrete check, take \(f(x)=x^3-1\), \(\alpha=1\), and \(x_0=\frac{11}{10}\). Here \(f'(1)=3\), \(f''(1)=6\), so the proposed leading-term formula would give \(e_1=e_0^2=\frac1{100}\). But the actual Newton step is
Consequently \(e_1=x_1-1=\frac{16}{1815}\), which is not \(\frac1{100}\). The correction is to use the exact error relation with its intermediate point \(\xi_n\), or to use the derivative values at \(\alpha\) only as an asymptotic approximation when the errors are tending to zero.
Why the Conditions Matter
The quadratic error relation explains Newton’s rapid local behavior, but each part of the argument has a role. The condition \(f'(\alpha)\ne0\) lets us keep the Newton denominator away from zero nearby. A continuous second derivative lets us bound the curvature and apply Taylor’s theorem. Starting sufficiently close ensures the error estimate keeps the iterates in the neighborhood where those bounds hold.
These are local conclusions, not a guarantee about a distant starting point. They explain what happens once the iteration is close enough to a simple root under the stated smoothness assumptions. The central estimate is \(|e_{n+1}|\le C|e_n|^2\): sufficiently small errors are controlled by their square, which is why Newton’s method can converge so rapidly near a simple root.
Check Your Understanding
Use the error relation and its assumptions to answer the following questions.
- What two conditions define a simple root?
- How does Taylor’s theorem lead to the factor \(e_n^2\) in the Newton error relation?
- Why must the derivative be bounded away from zero in a neighborhood of the root?
- What does the asymptotic error constant say, and why is it not usually an exact one-step formula?
- If \(f''(\alpha)=0\), does the local quadratic error bound necessarily fail? Explain.
