Tutorials › Calculus II › Taylor Derivation of Newton’s Method

Home  /  Tutorials  /  Taylor Methods in Computation

Taylor Methods in Computation · Tutorial 888 of 998

Taylor Derivation of Newton's Method

Learn how first-order Taylor linearization produces Newton’s root iteration and how to calculate and verify its steps.

Advanced 9 min read

What You'll Learn

  • Linearize a scalar function at a current approximation using its first-order Taylor polynomial.
  • Solve the linear model for its predicted zero to derive Newton’s update.
  • Identify the differentiability and nonzero-derivative conditions needed for a step.
  • Compute Newton iterates for polynomial and exponential equations from stated initial guesses.
  • Check each iterate in the tangent model and distinguish that check from having found an exact root.
  • Recognize a sign error in the Newton correction and understand why the method may fail.

Why Linearize an Equation?

Finding a zero of a nonlinear function means finding a number \(r\) such that \(f(r)=0\). Even when the equation cannot be solved conveniently by algebra, a local approximation can suggest a useful next estimate. The key is to replace the function, near a current estimate, by its first-order Taylor polynomial.

A first-order Taylor polynomial is a line. Its zero is often easier to calculate than a zero of the original function. Newton’s method repeatedly takes the zero of this local line as the next approximation. This explains both the method’s efficiency near a suitable root and its limitation: a local line is only an approximation to the function.

Definition: Given a differentiable function \(f\) and a current approximation \(x_n\) with \(f'(x_n)\ne 0\), the Newton update is $$ x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}. $$ Starting from a specified initial guess \(x_0\), repeated application of this rule produces a sequence of approximations, provided each required derivative is defined and nonzero.

Deriving the Newton Update

Suppose \(x_n\) is the current estimate and \(r\) is the root we hope to approximate. The first-order Taylor model of \(f\) centered at \(x_n\) is

$$ T_1(x)=f(x_n)+f'(x_n)(x-x_n). $$

This line agrees with \(f\) at \(x_n\) and has the same slope there. Newton’s idea is to estimate \(r\) by the zero of this line. Set \(T_1(x_{n+1})=0\) and solve for \(x_{n+1}\):

$$ f(x_n)+f'(x_n)(x_{n+1}-x_n)=0, $$ $$ f'(x_n)(x_{n+1}-x_n)=-f(x_n), $$ $$ x_{n+1}-x_n=-\frac{f(x_n)}{f'(x_n)}, $$ $$ x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}. $$

The division is possible only when \(f'(x_n)\ne 0\). The function must also be defined and differentiable at the current estimate. These are conditions for calculating a step; they do not ensure that the step is close to a root or that later steps will succeed.

1
Choose the current estimate.
Specify \(x_n\), at which \(f(x_n)\) and \(f'(x_n)\) can be evaluated.
2
Build the local line.
Use \(T_1(x)=f(x_n)+f'(x_n)(x-x_n)\).
3
Find the line’s zero.
Set \(T_1(x)=0\) and solve. The result is the next Newton iterate.

Why the Taylor Step Is a Reasonable Proposal

The derivation makes exact use of the Taylor polynomial, but the polynomial is generally not the function itself. If \(f\) is twice continuously differentiable on an interval containing \(x_n\) and \(r\), Taylor’s theorem gives some point \(\xi\) between \(x_n\) and \(r\) such that

$$ f(r)=f(x_n)+f'(x_n)(r-x_n)+\frac{f''(\xi)}{2}(r-x_n)^2. $$

If \(r\) is an exact root, then \(f(r)=0\). The first two terms are precisely the value of the first-order Taylor model at \(r\); the last term is the error in using that model there. When \(r\) is close to \(x_n\), the remainder contains the squared distance \((r-x_n)^2\). This explains why a tangent-line estimate can be informative near a root, especially when the function’s curvature is controlled.

It does not prove that Newton’s method converges from every initial guess. The approximation can be poor when the current estimate is far from a root, the derivative can be very small, or an iterate can leave the domain. The update is a local-model strategy, not a universal guarantee.

Geometric interpretation: At \(x_n\), draw the tangent line to the graph \(y=f(x)\). If its slope is nonzero, where that tangent crosses the horizontal axis is \(x_{n+1}\). The next step repeats this construction at the new estimate.

Worked Iterations

Worked Example: Approximating \(\sqrt{2}\)

Use Newton’s method on \(f(x)=x^2-2\), starting from \(x_0=1\). The equation \(f(x)=0\) is equivalent to \(x^2=2\). First differentiate and form the update:

$$ f'(x)=2x,\qquad x_{n+1}=x_n-\frac{x_n^2-2}{2x_n}. $$

At the initial estimate, \(f(1)=1-2=-1\) and \(f'(1)=2\). Thus

$$ x_1=1-\frac{-1}{2}=1+\frac12=\frac32. $$

At \(x_1=3/2\), \(f(x_1)=9/4-2=1/4\) and \(f'(x_1)=3\). Therefore

$$ x_2=\frac32-\frac{1/4}{3} =\frac32-\frac1{12} =\frac{18}{12}-\frac1{12} =\frac{17}{12}. $$

At \(x_2=17/12\), the function value is \(289/144-288/144=1/144\), and the derivative is \(17/6\). The next step is

$$ x_3=\frac{17}{12}-\frac{1/144}{17/6} =\frac{17}{12}-\frac1{408} =\frac{578}{408}-\frac1{408} =\frac{577}{408} \approx 1.414216. $$

Check the result in the original function:

$$ f(x_3)=\left(\frac{577}{408}\right)^2-2 =\frac{332929-332928}{166464} =\frac1{166464}. $$

The residual is small and positive, so \(x_3\) is a close approximation to a zero, not an exact zero. Each step was computed using the stated Newton update, and the final substitution checks the original equation.

Worked Example: One Step for a Cubic Equation

Apply one Newton step to \(f(x)=x^3-2x-5\), starting from \(x_0=2\). The derivative is

$$ f'(x)=3x^2-2. $$

At \(x_0=2\), evaluate both quantities needed for the update:

$$ f(2)=2^3-2(2)-5=8-4-5=-1, $$ $$ f'(2)=3(2^2)-2=12-2=10. $$

Since \(f'(2)=10\ne 0\), the step is defined:

$$ x_1=2-\frac{-1}{10} =2+\frac1{10} =2.1. $$

Check the tangent model at this proposed next estimate. Centered at \(2\), it is

$$ T_1(x)=-1+10(x-2), \qquad T_1(2.1)=-1+10(0.1)=0. $$

The update has exactly found the zero of the tangent model. It has not made \(2.1\) an exact root of the cubic: direct substitution gives

$$ f(2.1)=(2.1)^3-2(2.1)-5 =9.261-4.2-5 =0.061. $$

That nonzero value is consistent with the method: Newton’s step solves the local linear approximation, not necessarily the original nonlinear equation.

Worked Example: An Exponential Equation

Use one Newton step on \(f(x)=e^x-2\), with initial guess \(x_0=1\). The target equation is \(e^x=2\), and

$$ f'(x)=e^x. $$

At \(x_0=1\), \(f(1)=e-2\) and \(f'(1)=e\), which is nonzero. The next estimate is

$$ x_1=1-\frac{e-2}{e} =1-1+\frac2e =\frac2e \approx 0.735759. $$

The tangent model centered at \(1\) is \(T_1(x)=e-2+e(x-1)\). Substitution verifies the update:

$$ T_1\left(\frac2e\right) =e-2+e\left(\frac2e-1\right) =e-2+2-e =0. $$

The original function at that estimate is \(f(2/e)=e^{2/e}-2\), approximately \(0.087\), not zero. Thus the calculated value is a new estimate generated by linearization; further steps would be needed to improve it.

A Sign Error to Avoid

A frequent mistake is to lose the negative sign while solving the tangent-line equation. The line must be set equal to zero as

$$ f(x_n)+f'(x_n)(x_{n+1}-x_n)=0. $$

Solving this equation requires moving \(f(x_n)\) to the other side, which gives \(f'(x_n)(x_{n+1}-x_n)=-f(x_n)\). For a concrete check, take \(f(x)=x^2-2\) at \(x_0=1\). Then \(f(1)=-1\) and \(f'(1)=2\). The incorrect sign would give \(x_1=1+f(1)/f'(1)=1/2\). But the tangent model is \(T_1(x)=-1+2(x-1)\), and

$$ T_1\left(\frac12\right)=-1+2\left(-\frac12\right)=-2\ne 0. $$

The correct update gives \(x_1=1-(-1)/2=3/2\), and

$$ T_1\left(\frac32\right)=-1+2\left(\frac12\right)=0. $$

Checking the proposed value in the tangent model is a direct way to catch this sign error.

When the Update Cannot Be Used

Newton’s formula requires a finite, nonzero derivative at the current iterate. For example, if \(f(x)=x^2+1\) and \(x_0=0\), then \(f'(0)=0\), so the update would require division by zero and is undefined. More generally, even if the first step is defined, a later iterate might have zero derivative or fall outside the domain of \(f\).

A nonzero derivative is therefore a step-by-step condition, not a promise of success. The method also depends on the initial guess because the Taylor line changes with the point where it is constructed. An initial value whose tangent points away from the desired root can produce unhelpful iterates.

Key takeaway: Newton’s method is obtained by taking the zero of the first-order Taylor polynomial at the current estimate. Its update follows from solving a linear equation, while its usefulness depends on the quality of that local approximation and on the iterates remaining in situations where the formula is defined.

Check Your Understanding

Use the Taylor-model derivation, rather than memorizing the update alone, to answer these questions.

  1. For a differentiable \(f\), write the first-order Taylor model centered at \(x_n\), and show how setting it equal to zero produces the Newton update.
  2. For \(f(x)=x^2-3\) with \(x_0=2\), calculate \(x_1\) and verify that it makes the tangent model at \(x_0\) equal to zero.
  3. What condition on \(f'(x_n)\) is needed to calculate a Newton step, and why does that condition not guarantee convergence?
  4. For \(f(x)=x^2-2\) at \(x_0=1\), what incorrect value results from using \(x_0+f(x_0)/f'(x_0)\)? Check it in the tangent model.
  5. Explain why a Newton iterate can make the tangent model zero while leaving the original function value nonzero.

Previous

Next

Back to Tutorials