Comparing Newton Runs From Different Starting Values

Why the Starting Value Matters

Newton’s method uses the tangent line at the current iterate to choose the next one. For a differentiable function \(f\), the update is

provided \(f'(x_n)\ne 0\). The formula depends on the current value, so changing the initial guess can change every later iterate. A run may approach one root, approach another, take more steps to meet a tolerance, or fail because the derivative vanishes.

Comparing runs therefore requires more than looking at their final answers. Record the iterates, check the residual \( |f(x_n)| \), and use the same stopping rule for each run. A small residual means that \(x_n\) nearly solves the equation in function value; it does not, by itself, guarantee that \(x_n\) is equally close to a root in the input variable.

A Comparison Framework

For each initial value, calculate enough of the sequence to see its behavior, and keep track of the same quantities. In particular, note which root the run approaches if convergence is apparent, how many iterations it takes to meet the chosen criterion, and whether the residual or step size behaves unexpectedly along the way.

Stopping rules based on steps and residuals answer different questions. A residual rule stops when \( |f(x_n)|\le\varepsilon_f \). A step rule stops when \( |x_n-x_{n-1}|\le\varepsilon_x \). These tests should not be treated as interchangeable: a small step can occur without a small residual, and a small residual can occur while the input is not yet as accurate as desired. The tolerances should be stated, and competing runs should be judged using the same rule.

Worked Example: Two Starting Values, Two Roots

Consider \(f(x)=x^3-2x\). Its roots are \(-\sqrt{2}\), \(0\), and \(\sqrt{2}\), since \(x^3-2x=x(x^2-2)\). The derivative is \(f'(x)=3x^2-2\), so Newton’s update, when \(3x^2-2\ne 0\), is

Start one run at \(x_0=\frac12\). The first update is

The next update is

Its residual is

Here \(235^3=12977875\), and \(4/235=220900/12977875\). Thus the residual is about \(0.01702\), not a value obtained by subtracting an incorrectly converted fraction. The iterate is close to the root \(0\), but the residual is not yet especially small.

Continue once more:

Substituting this value into \(f\) gives a residual of approximately \(0.000001233\). The run has moved very close to \(0\), and its residual has dropped substantially between steps 2 and 3.

Now start a second run at \(x_0=\frac32\). Its first iterate is

The residual at this iterate is exactly

The next iterate is

Since \(f(x)=x(x^2-2)\), this iterate has residual

This run is approaching \(\sqrt{2}\), while the run from \(\frac12\) is approaching \(0\). After two updates, the second run has a smaller residual, but the runs are targeting different roots. The comparison illustrates why the initial value can affect not only speed but also which solution is found.

Why Residuals Are Useful, but Not the Whole Story

The residual measures the value of the equation at an iterate. To relate it to distance from a root, suppose \(f(\alpha)=0\), and \(f\) is continuous on the interval between \(x\) and \(\alpha\) and differentiable inside it. The Mean Value Theorem gives a point \(\xi\) between \(x\) and \(\alpha\) such that

Because \(f(\alpha)=0\), this becomes

Consequently, if throughout that interval the derivative satisfies \(m\le |f'(t)|\le M\) for constants \(0<m\le M\), then

This proves that, in an interval where the derivative is bounded away from zero and also bounded above, residual and root error control one another up to those factors. Without such information, a small residual alone need not give a reliable estimate of distance to the root. Near a point where the derivative is small, the function can change slowly, so an iterate may have a modest residual while still being noticeably displaced.

Worked Example: Comparing Runs with a Common Tolerance

Use the two runs above and adopt the residual stopping rule \( |f(x_n)|\le 0.001 \). For the run from \(x_0=\frac12\), the residuals already computed include \(r_2\approx 0.01702\) and \(r_3\approx 0.000001233\). Since \(0.01702>0.001\) and \(0.000001233<0.001\), this run first meets the rule at step 3.

For the run from \(x_0=\frac32\), step 1 has residual about \(0.02756\), which exceeds the tolerance. Step 2 has residual about \(0.00019624\), which is below it. This run first meets the same rule at step 2. By this particular residual criterion, the run from \(\frac32\) takes fewer updates. That comparison does not mean it has found the same root as the other run: it is approaching \(\sqrt{2}\), not \(0\).

A useful comparison table keeps the iteration count, residual, and target root together:

The comparison is meaningful because both runs use the same function, residual definition, and tolerance. If the intended application requires a particular root, the target must also be part of the assessment. A fast run to an unwanted root is not a successful solution to that task.

Worked Example: A Starting Value That Causes Failure

For \(f(x)=x^3-2x\), the derivative vanishes when \(3x^2-2=0\), or \(x=\pm\sqrt{2/3}\). Start at \(x_0=\sqrt{2/3}\). Then

But \(f(x_0)\ne 0\): using \(x_0^2=2/3\),

Thus the Newton update would divide a nonzero number by zero, and the run cannot take its first step. This is a genuine breakdown, not convergence or a large residual. A practical comparison should report it as such rather than omit the starting value or replace it silently with a nearby one.

Interpreting a Set of Runs

Several features matter when comparing starting values:

• Root reached: Different initial values may lead to different roots, even for the same equation.
• Iterations to tolerance: Count updates using one stated stopping rule and one stated tolerance.
• Residual history: Residuals need not decrease at every step, so inspect the sequence rather than only its last entry.
• Step history: Small changes between iterates are useful information, but do not alone certify a small residual.
• Breakdown: If \(f'(x_n)=0\) while \(f(x_n)\ne0\), the Newton update is undefined.

Near a simple root, under the local conditions for Newton convergence, the method can converge quadratically. In practical terms, once the iterates are sufficiently close and the assumptions for that local result hold, errors may shrink rapidly. This local behavior does not guarantee that an arbitrary starting value will enter the relevant neighborhood. The observed iteration counts from different initial values therefore reflect both how quickly Newton’s method works near a root and how the starting values behave before getting there.

A Common Mistake: Comparing Only the Final Residual

A tempting conclusion is that the run with the smallest final residual is always the best run. That conclusion ignores the number of iterations, which root was reached, whether the run encountered an undefined update, and whether both runs were stopped using the same rule. For instance, in the example, a residual below \(0.001\) is reached in two steps from \(\frac32\) and three steps from \(\frac12\), but those runs approach different roots. If the goal is specifically to find \(0\), the faster run to \(\sqrt{2}\) is not the desired result.

A related computational mistake is to compare rounded intermediate values as though they were exact. When a later update depends strongly on earlier values, rounding too early can distort the result. Keep exact fractions when practical, or carry sufficient precision through the iterations; round only when reporting the final approximate values and state the precision used.