Why Rounded Data Need Care
A table or calculator may show function values with only a few decimal places. When inputs get close to a target, the displayed outputs can suggest that the function values are approaching a particular number. But a displayed decimal is usually a rounded report, not the exact value.
For example, a display of \(5.00\) could represent a value slightly below \(5.00\), exactly \(5.00\), or slightly above it. If several nearby inputs all produce the display \(5.00\), that is useful evidence, but it does not tell us that the true outputs are identical or exactly equal to \(5\).
What a Rounded Decimal Tells You
Suppose a value is rounded to \(d\) decimal places using the usual nearest-value convention. One unit in the last displayed decimal place is \(10^{-d}\). The maximum rounding error is half of that unit:
If \(r\) is the displayed value and \(y\) is the true value, this gives the bound
The inequality gives a safe enclosure for the true value. At an exact halfway point, the result depends on the tie-breaking convention, but the bound remains valid. For instance, a value displayed as \(2.37\) to two decimal places is within \(0.005\) of the true value. It is not automatically exactly \(2.37\).
| Displayed precision | Half of one last-place unit | What a report of \(r\) guarantees |
|---|---|---|
| Nearest whole number | \(0.5\) | \(|y-r|\leq 0.5\) |
| Nearest tenth | \(0.05\) | \(|y-r|\leq 0.05\) |
| Nearest hundredth | \(0.005\) | \(|y-r|\leq 0.005\) |
| Nearest thousandth | \(0.0005\) | \(|y-r|\leq 0.0005\) |
Equivalently, the true value lies in the interval from \(r-\frac12 10^{-d}\) to \(r+\frac12 10^{-d}\). The endpoints can be included to give a conservative bound, even if a particular rounding rule assigns a tie to only one of two neighboring displays.
Worked Example: Reading Nearby Values Rounded to Hundredths
Worked Example: Reading Nearby Values Rounded to Hundredths
A table gives \(f(x)=2x+1\) near the target input \(x=2\). Each output is rounded to the nearest hundredth. What value do the reports suggest the function values approach as the inputs get close to \(2\)?
First calculate the exact outputs for the listed inputs:
To the nearest hundredth, these become \(4.80\), \(4.98\), and \(5.00\). The inputs are respectively \(0.1\), \(0.01\), and \(0.001\) from \(2\). The displayed outputs move toward \(5.00\), so the data support \(5\) as an estimate of the approached value.
Check the precision of the final display. A report of \(5.00\) to the nearest hundredth allows an error of at most \(0.005\). The exact last output, \(4.998\), differs from \(5.00\) by \(0.002\), and indeed \(0.002\leq 0.005\). Thus the rounded report is consistent with the exact calculation.
This finite table is evidence for the estimate, not by itself a proof that all function values sufficiently near \(2\) behave this way.
Rounded Reports Are Intervals, Not Exact Points
A useful way to read rounded data is to replace each displayed value by its uncertainty interval. If the report is \(r\) to \(d\) decimal places, use the bound \(|y-r|\leq \frac12 10^{-d}\). This can show whether two true values must be ordered, or whether the rounding uncertainty is too large to decide.
Worked Example: Comparing Two Reports to the Nearest Thousandth
Two outputs near a target input are reported as \(0.997\) and \(1.004\), each to the nearest thousandth. Can the true second output be smaller than the true first output?
For three decimal places, the maximum error is
Therefore, the first true output lies between
and the second lies between
Even the largest possible first value, \(0.9975\), is below the smallest possible second value, \(1.0035\), since \(0.9975<1.0035\). So the second true output must be larger. The gap between the interval endpoints is \(1.0035-0.9975=0.006\), confirming that the intervals do not overlap.
Check: the reported values differ by \(1.004-0.997=0.007\), while the two reports together can account for at most \(0.0005+0.0005=0.001\) of difference in the opposite direction. The true increase must therefore be at least \(0.007-0.001=0.006\), matching the interval calculation.
Why a Fixed Number of Decimal Places Can Hide Behavior
If the precision does not increase, rounding leaves a fixed amount of uncertainty. Many different true values can produce the same displayed number. As a result, repeated reports of one value do not establish that the underlying outputs are constant, nor do they guarantee that the outputs approach the reported value.
Worked Example: One Display, Two Different Behaviors
Suppose a sequence of inputs \(x_n\) gets closer to a target, and outputs are recorded to the nearest hundredth. Imagine the true outputs alternate between \(1.234\) and \(1.226\).
Both numbers round to \(1.23\): \(1.234\) is \(0.004\) above \(1.23\), and \(1.226\) is \(0.004\) below \(1.23\). Each distance is at most \(0.005\), the maximum error for hundredth-place rounding. Thus every report can show \(1.23\), even though the true outputs alternate between two distinct values.
The true outputs do not approach a single value. The outputs at even indices remain \(1.234\), while those at odd indices remain \(1.226\), and these two values differ by \(1.234-1.226=0.008\). A sequence with two persistent, distinct values cannot settle arbitrarily close to one number. The display hides this difference because its precision is not fine enough to show it.
Check: each true value is inside the interval from \(1.23-0.005=1.225\) to \(1.23+0.005=1.235\), exactly as the shared display requires. The identical reports do not contradict the different true values.
Improving Precision Can Strengthen the Evidence
The previous example has a fixed rounding uncertainty. One way to make numerical evidence more informative is to report more decimal places as the inputs get closer to the target. The uncertainty then shrinks along with the reported values’ apparent changes.
The reason can be expressed precisely. Suppose \(y_n\) is the true output at the \(n\)th selected input, and \(r_n\) is its reported rounded value. Assume the rounding error is at most \(\varepsilon_n\), where \(\varepsilon_n\) gets arbitrarily small. If the reports get arbitrarily close to a number \(L\), then the true outputs also get arbitrarily close to \(L\).
Here is the complete justification. For any positive tolerance \(\eta\), choose a stage after which \(\varepsilon_n<\eta/2\). Because \(r_n\) approaches \(L\), choose a stage after which \(|r_n-L|<\eta/2\). Once both conditions hold, the triangle inequality gives
Thus the true outputs are within any chosen positive tolerance of \(L\) from that stage onward. The shrinking rounding error matters: if the error bound stays fixed, this argument cannot make it smaller than an arbitrary tolerance.
This result concerns a sequence of selected inputs and their outputs. It explains when rounded reports reliably track those outputs. A finite list of such reports still does not cover every input near a target, so it should be treated as evidence for a limit estimate rather than a complete proof about the function.
A Common Mistake: Treating the Display as Exact
A common error is to read every displayed value as exact and then claim that several identical outputs prove the limit equals that number. For example, if a calculator prints \(3.14\) for several nearby inputs, the mistaken conclusion might be: “The function values are all exactly \(3.14\), so the limit is \(3.14\).”
The correction is to account for the reporting precision. To the nearest hundredth, each display of \(3.14\) allows a true value within \(0.005\) of \(3.14\). The values could differ from one another and from \(3.14\), and a finite set of inputs cannot establish what happens at every sufficiently nearby input. A more careful statement is: “These reports are consistent with an approached value near \(3.14\), to the precision shown.”
How to Judge Rounded Evidence
When using rounded tables or calculator output, separate three questions:
Count the decimal places and calculate half of one unit in the last place.
Compare them as the inputs get closer to the target, while keeping the uncertainty intervals in view.
State an estimate supported by the data, but do not treat a finite table or a fixed-precision display as proof of a limit.
These steps help distinguish the pattern in the reported numbers from what is known about the unrounded function values. A strong estimate uses both the direction of the observed changes and the size of the possible rounding error.
Check Your Understanding
Use the rounding bounds in your answers. Treat each reported value as rounded to the stated precision.
- A value is reported as \(7.2\) to the nearest tenth. What is the maximum possible rounding error?
- A table repeatedly reports \(4.00\) to the nearest hundredth for inputs close to a target. What range of true values is consistent with any one report?
- Two outputs reported to the nearest hundredth are \(2.31\) and \(2.34\). Find the possible-value interval for each. Do the intervals overlap?
- Why do identical rounded reports for a finite set of inputs not prove that the function values are identical?
- In the result about shrinking errors, why is it important that the error bounds approach zero?
