Testing Models Against Endpoint Measurements

Why Check an Accumulation Model at Endpoints?

An accumulation model predicts how a stock changes over time from its starting value and its rate of change. For example, a model might describe the amount of water in a reservoir, inventory in a warehouse, or material stored in a tank. Comparing its predictions with measured stock values at selected times is a direct way to check whether the model is consistent with observations.

A mismatch is useful evidence, but it does not automatically reveal the cause. A prediction may be off because the starting stock was recorded incorrectly, because the modeled rate is inaccurate, or because the measurement itself contains error. Comparing several endpoints—and comparing the changes between them—helps separate these possibilities.

Model Predictions and Endpoint Discrepancies

Suppose a stock \(S(t)\) has initial value \(S_0\) at time \(t_0\), and its rate of change is modeled by \(r(t)\). The accumulation model predicts

Here \(M(t)\) is the model-predicted stock. The integral accumulates the modeled rate from the initial time to the endpoint. This formula applies when the rate is integrable over the interval and the stock changes according to that rate.

If a measurement at time \(t_i\) is \(y_i\), define its endpoint discrepancy as observed stock minus predicted stock:

A positive discrepancy means the observed stock is greater than the model prediction; a negative discrepancy means it is smaller. A discrepancy of zero means the observation agrees with the prediction at that endpoint. The sign convention matters: reversing the subtraction reverses the interpretation.

Why Comparing Endpoints Works

The distinction between a baseline discrepancy and an accumulated rate discrepancy follows from the accumulation equation itself. Let \(X(t)\) be the actual stock and suppose its actual rate is \(q(t)\), so that \(X(t)=X_0+\int_{t_0}^{t}q(u)\,du\). Assume a measurement at \(t_i\) is \(y_i=X(t_i)+\varepsilon_i\), where \(\varepsilon_i\) is its measurement error. The model prediction is \(M(t_i)=S_0+\int_{t_0}^{t_i}r(u)\,du\). Subtracting the prediction from the measurement gives

This identity separates three contributions: the initial-stock difference \(X_0-S_0\), the accumulated difference between actual and modeled rates, and measurement error. In particular, for two endpoints \(t_i<t_j\), subtracting their discrepancies cancels the baseline difference:

So, if measurement errors are small, a change in discrepancy between endpoints indicates a net rate mismatch over the intervening interval. It does not specify exactly when that mismatch occurred or what caused it. The endpoint comparison measures the accumulated difference over the interval, not the full rate history.

Compare Changes as Well as Stock Values

For consecutive observation times \(t_i\) and \(t_j\), compare the observed stock change \(y_j-y_i\) with the model-predicted change:

The initial value does not appear in this comparison. That is why interval changes help assess the rate component separately from the baseline. If the observed and predicted changes agree, but both endpoint discrepancies are similar, the mismatch may lie in the baseline. If the changes disagree, the model's accumulated rate over that interval does not match the observed change, subject to measurement uncertainty.

Check an Interval Before Naming the Cause

A discrepancy at a later endpoint may arise from a mismatch anywhere in the interval leading to that endpoint. Comparing successive endpoints can narrow down where the accumulated disagreement occurs. It cannot, by itself, reveal the detailed behavior of the rate between observations.

A Common Mistake: Calling Every Difference a Rate Error

Suppose the first example is interpreted by saying, “The day-zero measurement is \(3\) items above prediction, so the modeled rate must be \(3\) items per day too low.” This confuses stock, measured in items, with rate, measured in items per day. It also ignores the later endpoint comparisons.

If the rate really were \(3\) items per day too low while the baseline remained \(100\), this interpretation would predict \(100+(5+3)(2)=116\) items at day \(2\), not the observed \(113\). The correction is to compare interval changes: the model and measurements both increase by \(10\) items over the first two days and \(15\) items over the next three days. The repeated \(3\)-item stock difference is consistent with a baseline offset, not evidence by itself of a rate error.

Limits of Endpoint Evidence

Endpoint measurements are valuable checks, but conclusions must remain proportionate to the evidence. A few endpoints do not determine the rate at every moment. A rate could be too high during part of an interval and too low during another part, with the two errors canceling in the total accumulation. Consequently, matching endpoints do not prove that the model's rate was accurate throughout the interval.

Measurement uncertainty also matters. If each measurement has a known tolerance, discrepancies smaller than that tolerance may not indicate a meaningful mismatch. When measurement errors are unknown, it is safest to describe the observed pattern and the explanations it is consistent with, rather than assert a unique cause.