Why More Steps Can Give a Worse Answer
Forward Euler is often improved by reducing the step size. In exact arithmetic, smaller steps generally reduce the method’s discretization error on a fixed time interval. A computer, however, does not perform exact real arithmetic. It stores numbers with finite precision and rounds intermediate results.
Consequently, the result of a computer calculation has at least two distinct sources of error. Reducing the step size can reduce one while increasing the effect of the other. Understanding that competition explains why taking extremely small steps may fail to improve an Euler approximation.
If \(y(t_n)\) is the exact solution at a grid point, \(Y_n\) is the exact-arithmetic Euler value, and \(\widehat{Y}_n\) is the computer-computed value, then
The first term is discretization error; the second is roundoff error. This inequality is the triangle inequality, and it does not say the two errors must have the same sign or that they always add exactly. It does show why an improvement in the first term need not produce an improvement in the total.
The Two Errors Behave Differently
For the initial-value problem \(y'=f(t,y)\), forward Euler with step size \(h\) uses
Under suitable smoothness assumptions on the solution, the global discretization error of forward Euler on a fixed interval is typically proportional to \(h\) for small \(h\). This is its first-order convergence: halving \(h\) typically reduces the discretization error by about a factor of two once the method is in its asymptotic regime.
A computation with smaller \(h\), however, needs more steps to cover the same interval. On an interval of length \(T\), the number of steps is \(n=T/h\), when the step divides the interval. If each update introduces a small arithmetic perturbation, there are \(T/h\) opportunities for such perturbations to affect the answer. Their total effect depends on their signs and on how the differential equation amplifies errors, but it can grow as \(h\) decreases.
A Floating-Point Model
A useful idealized model for a basic floating-point operation is
where \(\circ\) is an arithmetic operation, \(\operatorname{fl}\) means “the computer’s rounded result,” and \(u\) is the unit roundoff. This model assumes the operation does not overflow or enter exceptional underflow cases. It describes relative rounding error for a single operation; it does not mean every Euler update has exactly the same error or that the errors necessarily point in the same direction.
For reasoning about a sequence of Euler updates, it is often convenient to collect the roundoff introduced at step \(k\) into an additive perturbation \(r_k\):
The perturbation \(r_k\) represents the net effect of rounding in that update, including relevant intermediate operations. Its size depends on the magnitudes of the numbers being computed, the arithmetic, and the implementation. The model is a tool for understanding error accumulation, not a claim that all real computations have a fixed absolute error per step.
How Per-Step Roundoff Accumulates
We can compare a rounded computation with the Euler calculation that would use exact arithmetic on the same grid. Suppose \(f\) is Lipschitz in its second variable on the region of interest: there is a constant \(L\geq 0\) such that
Write \(Y_k\) for the exact-arithmetic Euler values and let \(d_k=\widehat{Y}_k-Y_k\). Assume both computations start from the same initial value, so \(d_0=0\). Subtracting their update formulas gives
Apply the triangle inequality and then the Lipschitz condition:
This recurrence shows both the direct contribution of the new rounding perturbation and the possible amplification of errors already present. Iterating it, with \(d_0=0\), yields the bound
For completeness, the iteration follows by repeated substitution. The first step gives \(|d_1|\leq |r_0|\). The next gives \(|d_2|\leq(1+Lh)|r_0|+|r_1|\). Each further substitution multiplies earlier contributions by one more factor \(1+Lh\), producing the displayed sum.
If, for illustration, \(|r_j|\leq\rho\) at every step and \(L>0\), the geometric sum gives
On a fixed interval \(T=nh\), the factor \((1+Lh)^n\) is at most \(e^{LT}\), since \(1+x\leq e^x\) for \(x\geq 0\). Thus a simpler bound is
This bound is deliberately cautious: it assumes the perturbations could combine in the most unfavorable direction. In a real computation they may partly cancel. Still, it explains why a per-step roundoff level that is small can matter when the number of steps is very large.
Combining this kind of estimate with the usual first-order discretization estimate gives a useful qualitative picture. If the discretization contribution is approximately \(C h\), while accumulated roundoff behaves approximately like \(D u/h\), then a rough total-error model is
The constants \(C\) and \(D\) depend on the problem and calculation. This is not a universal formula for every Euler computation; it describes a common trade-off when per-step rounding effects have a scale related to machine precision \(u\). The first term falls with \(h\), while the second rises as the step count increases.
Worked Examples
Worked Example: Euler Discretization Error Without Roundoff
Consider \(y'=y\), \(y(0)=1\), and use four forward Euler steps to approximate \(y(1)\). Here \(h=(1-0)/4=0.25\). First compute in exact arithmetic, so any difference from the exact solution is discretization error, not roundoff.
The Euler update is \(Y_{k+1}=Y_k+0.25Y_k=1.25Y_k\). Starting from \(Y_0=1\), the successive values are
The exact solution is \(y(t)=e^t\), because its derivative is \(e^t\) and it satisfies \(y(0)=1\). Thus the exact endpoint is \(y(1)=e\approx2.718281828\), and the endpoint error is
The computed Euler value also checks against the update: \(1.25(1.953125)=2.44140625\). Since all the displayed updates are exact decimal arithmetic, their discrepancy from \(e\) comes from the Euler discretization. In a floating-point calculation, roundoff would be an additional contribution.
Worked Example: Rounding Can Erase Every Small Increment
Consider \(y'=0.014\), \(y(0)=0\), on \(0\leq t\leq1\). The exact solution is \(y(t)=0.014t\), so \(y(1)=0.014\). Euler is exact for a constant derivative when its arithmetic is exact, because each step adds exactly the change in the solution over that step.
To isolate the effect of finite precision, suppose this illustrative computer rounds every stored result to the nearest \(0.01\). With one Euler step, \(h=1\), the exact update is \(0+1(0.014)=0.014\), which rounds to \(0.01\). The computed endpoint is therefore \(0.01\), with absolute error
Now take ten steps, so \(h=0.1\). Each exact increment is \(0.1(0.014)=0.0014\). At the first step, \(0+0.0014=0.0014\) rounds to \(0.00\). At every later step the stored value is still \(0.00\), and adding \(0.0014\) again still rounds to \(0.00\). The computed endpoint is \(0.00\), with error
The endpoint time is \(10(0.1)=1\), as required. The exact solution still gives \(0.014\), and the rounded update rule indeed leaves the stored value at zero after all ten steps. In this simplified decimal-arithmetic example, the smaller step makes the result worse because each increment falls below the rounding resolution. Actual computer arithmetic uses binary formats and has more details, but the underlying issue is the same: small changes can be lost when added to stored values.
Worked Example: A Step-Size Trade-Off Model
Suppose a rough error estimate for a particular calculation is \(E(h)=h+10^{-8}/h\), with \(0<h\leq1\). This is an illustrative model with \(C=D=1\) and unit roundoff scale \(u=10^{-8}\), not a measured guarantee for a particular problem.
At \(h=10^{-2}\), the two terms are \(10^{-2}\) and \(10^{-8}/10^{-2}=10^{-6}\), so
At \(h=10^{-4}\), the terms are both \(10^{-4}\), giving
At the much smaller step \(h=10^{-6}\), the terms are \(10^{-6}\) and \(10^{-8}/10^{-6}=10^{-2}\), so
The model’s minimum can be checked by differentiating for \(h>0\):
Setting \(E'(h)=0\) gives \(h^2=10^{-8}\), hence \(h=10^{-4}\) in the positive domain. Also \(E'(h)<0\) when \(0<h<10^{-4}\), and \(E'(h)>0\) when \(h>10^{-4}\), confirming that this is the minimum. The example shows the model’s central point: beyond a certain refinement, the roundoff contribution can outweigh the reduction in discretization error.
What to Conclude from Refinement
A smaller step size is not inherently bad. It is useful while discretization error dominates and the arithmetic remains sufficiently resolved. The important point is that refinement is not an unlimited route to accuracy. Once roundoff or other numerical effects become significant, successive calculations may stop improving, fluctuate, or become worse.
The constants in an error model are problem-dependent. Error amplification, the scale of the solution and its derivative, the number of arithmetic operations, and the floating-point format all affect the outcome. The simple term \(D u/h\) is a guide to a possible trend, not a substitute for examining the actual computation.
A Common Mistake
A common mistake is to assume that because forward Euler has first-order convergence, halving the step size must always improve the computed answer. That conclusion confuses the exact-arithmetic method with its finite-precision implementation.
For the illustrative model above, reducing \(h\) from \(10^{-4}\) to \(10^{-6}\) reduces the discretization term from \(0.0001\) to \(0.000001\). But the roundoff term rises from \(0.0001\) to \(0.01\), and the model estimate increases from \(0.0002\) to \(0.010001\). The correction is to treat total error as having multiple sources: finer spacing helps only while its benefit is not outweighed by arithmetic effects.
The decimal example gives a more concrete warning. With rounding to the nearest \(0.01\), the exact Euler increment \(0.0014\) is real and mathematically meaningful, but it does not change the stored value from \(0.00\). Repeating an update that loses its increment does not recover the information that was rounded away.
Check Your Understanding
Answer these questions by distinguishing the mathematical Euler sequence from the sequence actually stored by a computer.
- What is the difference between discretization error and roundoff error?
- On a fixed interval of length \(T\), how does the number of Euler steps depend on the step size \(h\)?
- In the perturbation recurrence, what role does the Lipschitz constant \(L\) play?
- Why can a small per-step rounding perturbation become important when the step size is very small?
- For the model \(E(h)=Ch+Du/h\), which term decreases as \(h\) decreases, and which term increases?
