Communicating Uncertainty and Model Limitations

Why Uncertainty Belongs Beside the Estimate

A forecast is a conditional statement about possible outcomes, not a statement that one outcome is certain to occur. Yet a precise-looking number can easily be read as a promise. In a strategic decision, that misunderstanding can matter as much as an inaccurate calculation: a decision maker may treat a central estimate as guaranteed, overlook a fragile assumption, or mistake a sensitivity case for a likely outcome.

Communicating uncertainty well does not mean weakening every conclusion with vague disclaimers. It means explaining what the estimate represents, the assumptions on which it depends, how it could vary, and which limitations could change the decision. The objective is to make the evidence usable without implying more confidence than the analysis supports.

Definition: An estimate is a calculated or assessed value for a quantity of interest. A forecast is an estimate about a future quantity over a specified horizon. Uncertainty describes what is not known about the value or outcome, including variation in future outcomes and uncertainty in the information or model used. A model limitation is a material boundary or weakness in the data, assumptions, methods, or scope that can affect how the result should be interpreted.

A useful communication separates three things that are often blurred together:

The result: the number, range, or scenario output produced by the analysis.
The conditions: the assumptions, data, and time horizon under which that result was produced.
The interpretation: what the result supports, what it does not establish, and what could change it.

For example, “the modeled annual cost is $1.7 million” is more informative when followed by the modeled scenarios, their assigned probabilities, and the fact that those probabilities are assumptions rather than observed frequencies. A forecast should identify whether it is a central estimate, a scenario result, a range, or a probability statement. Those labels are not interchangeable.

Describe the Basis Before Interpreting the Number

Before presenting a result, state enough of its basis for a reader to understand its scope. At minimum, consider the following:

Quantity and units: What is being estimated—annual cost, quarterly revenue, number of events—and in what units?
Horizon and scope: Which time period, business activity, or population is included?
Data: What information supports the estimate, and how current and complete is it?
Assumptions: Which values or relationships are held fixed, and which are varied?
Method and uncertainty: Is the result a scenario-weighted value, a sensitivity case, or a range? Has a probability been assigned?
Limitations: What relevant effects are omitted or weakly represented, and could they affect the decision?

A limitation is material when knowing it could change the interpretation, the decision, or the confidence placed in the result. A long list of generic caveats is less useful than a short explanation of the limitations that matter. For each important limitation, describe its direction where it is reasonably known, its potential magnitude if it can be quantified, and what evidence or follow-up could reduce the uncertainty. If the direction or magnitude is not known, say so rather than implying that the effect is negligible.

Scenario-Weighted Estimates and What They Mean

Suppose a model represents a future outcome $Z$ by a finite set of mutually exclusive, collectively exhaustive scenarios. If scenario $j$ has outcome $z_j$ and assigned probability $p_j$, where each $p_j$ is nonnegative and the probabilities sum to one, the probability-weighted estimate is the sum of each outcome multiplied by its assigned probability.

E[Z]=\sum_{j=1}^{n}p_jz_j, \qquad \sum_{j=1}^{n}p_j=1

This formula follows by dividing the possible outcomes into the scenarios. The contribution to the average from scenario $j$ is its outcome $z_j$, counted in proportion to how often that scenario is assigned to occur, $p_j$. Adding those contributions gives the probability-weighted average. The result is meaningful as an expected value only under the stated scenario definitions and probabilities; it is not a claim that the outcome will equal that average.

This calculation does not determine whether the probabilities are credible. Those probabilities must come from a defensible basis, such as relevant data or explicitly elicited judgments, and the communication should identify that basis. If scenario probabilities have not been assigned, do not describe a simple average of scenarios as an expected value.

Worked Example: Reporting a Scenario-Weighted Cost

A hypothetical service provider estimates its cost from a one-year disruption using three mutually exclusive scenarios. The modeled costs are $500,000 with probability 0.60, $2,000,000 with probability 0.30, and $8,000,000 with probability 0.10. Assume the scenarios cover all modeled outcomes, the probabilities are accepted as the model inputs, and no discounting is applied because the amounts are one-year costs. What is the probability-weighted cost, and what should be communicated about it?

Method and calculation. First check that the scenario probabilities sum to one: $0.60+0.30+0.10=1.00$. The weighted contributions are:

0.60(\$500{,}000)=\$300{,}000,\qquad 0.30(\$2{,}000{,}000)=\$600{,}000,\qquad 0.10(\$8{,}000{,}000)=\$800{,}000.

Add the contributions: $\$300{,}000+\$600{,}000+\$800{,}000=\$1{,}700{,}000$. The probability-weighted cost is therefore $1,700,000.

For the separate question “what is the probability the cost exceeds $2,000,000?”, only the $8,000,000 scenario exceeds the threshold. The modeled probability is $0.10$, or 10%. The $2,000,000 scenario is equal to, not greater than, the threshold.

Interpretation and check. The $1,700,000 is an average under the stated scenario probabilities; it is not the most likely scenario cost (that is $500,000), and it is not a guaranteed cost. The scenario costs are hypothetical model inputs, not observed industry data. This presentation avoids the common mistake of reporting the weighted average as the amount the provider should expect to pay in that particular year.

Use Sensitivity Analysis Without Inventing Probabilities

A sensitivity analysis changes one or more assumptions and recalculates the result. A sensitivity case answers, “What would the model produce if these inputs changed?” It does not, by itself, answer, “How likely is this case?” A stress case can be useful even when no probability is assigned, provided it is labeled as a conditional calculation rather than a forecast of what will occur.

Worked Example: Showing the Effect of Two Input Changes

A hypothetical insurer forecasts annual premium revenue of $12,000,000, 900 claims, an average claim cost of $8,000, and fixed annual expenses of $1,800,000. Assume claim cost is calculated as claim count multiplied by average claim cost; revenue and fixed expenses do not change in the sensitivity case; and all amounts relate to the same year with no discounting. Calculate the baseline margin and the margin if both claim count and average claim cost are 10% higher. The sensitivity case is not assigned a probability.

Baseline. Claims cost is $900\times \$8{,}000=\$7{,}200{,}000$. The margin after claims and fixed expenses is: $ \$12{,}000{,}000-\$7{,}200{,}000-\$1{,}800{,}000=\$3{,}000{,}000$.

Change the inputs. A 10% increase in claim count gives $900(1.10)=990$ claims. A 10% increase in average claim cost gives $\$8{,}000(1.10)=\$8{,}800$. The sensitivity-case claims cost is $990\times \$8{,}800=\$8{,}712{,}000$.

Recalculate the margin. The sensitivity-case margin is $\$12{,}000{,}000-\$8{,}712{,}000-\$1{,}800{,}000=\$1{,}488{,}000$. The change from baseline is $\$3{,}000{,}000-\$1{,}488{,}000=\$1{,}512{,}000$, a decline of $\$1{,}512{,}000/\$3{,}000{,}000=0.504$, or 50.4%.

Check and interpretation. Increasing both multiplicative inputs by 10% increases claims cost by $1.10\times1.10-1=0.21$, or 21%; the calculated increase from $7,200,000 to $8,712,000 is also $1,512,000, or 21%. The margin falls by more than 21% because claims cost is only one part of the baseline margin. A common mistake is to add the two input changes and apply a 20% increase to claims cost. The correct combined factor is $1.10\times1.10=1.21$, not 1.20. The result describes the specified sensitivity case, not its likelihood.

Present Ranges and Limitations Precisely

A range can communicate how an estimate responds to uncertain inputs, but the range needs a label. A range formed from selected low and high assumptions is not automatically a confidence interval, a prediction interval, or a statement about the probability that the result will fall within those endpoints. Describe how the endpoints were constructed and whether they are meant to represent plausible values, deliberately adverse cases, or simply the tested input bounds.

Worked Example: Distinguishing an Input Range From a Probability Interval

A hypothetical automation project processes 800,000 transactions per year and is estimated to save $3.00 per transaction. Annual operating cost is $2,100,000. Assume the base estimate uses those values and compares annual savings with annual operating cost, with no discounting. The analysts also test transaction volumes from 720,000 to 880,000 and savings per transaction from $2.50 to $3.50. These are input bounds only; no probabilities have been assigned to the values within them. Calculate the base net benefit and the net-benefit range implied by the tested endpoints.

Base case. Annual savings are $800{,}000\times \$3.00=\$2{,}400{,}000$. Net benefit is $\$2{,}400{,}000-\$2{,}100{,}000=\$300{,}000$.

Low tested case. Using both lower inputs gives savings of $720{,}000\times \$2.50=\$1{,}800{,}000$. Net benefit is $\$1{,}800{,}000-\$2{,}100{,}000=-\$300{,}000$.

High tested case. Using both upper inputs gives savings of $880{,}000\times \$3.50=\$3{,}080{,}000$. Net benefit is $\$3{,}080{,}000-\$2{,}100{,}000=\$980{,}000$.

Check and interpretation. The tested range is $-\$300{,}000$ to $\$980{,}000$, and the base estimate of $300,000 lies within it. The result says the decision changes from a negative to a positive net benefit across the tested combinations. It does not say that the true result has any particular probability of being in that range. In particular, do not call these endpoints a “90% range” or a “confidence interval”; the model assigned no probabilities and constructed no such interval.

A material limitation should accompany this result if, for instance, the calculation excludes transition costs or assumes every transaction receives the stated saving. Those effects could reduce net benefit, but their amounts are not included in the computed range. A decision maker should not interpret the range as covering uncertainty that the analysis did not test.

Make Limitations Decision-Relevant

Model limitations can arise from incomplete or outdated data, uncertain parameter values, simplifying relationships, omitted activities, or a mismatch between the model’s scope and the decision. Distinguishing these types helps explain what the result can support. For example, additional historical data may help with a data limitation, while a structural limitation—such as assuming costs rise proportionally with volume—may require a different model or a separate scenario.

A useful limitation statement links the issue to its implication. “The model excludes implementation disruption; if disruption is material, first-year net benefit may be lower than shown” is more useful than “the model has limitations.” Where a limitation cannot be quantified, state that plainly and avoid presenting a precise adjustment as though it were measured. Where a limitation can be tested, show the test and its assumptions separately from the base estimate.

Keep the distinction between model uncertainty and outcome variability clear. Even a model with correctly implemented methods and complete input data would not make a future outcome certain. Conversely, a range of future outcomes does not capture uncertainty about whether the model itself has omitted an important driver. Both can matter to a decision, and the communication should not use one as a substitute for the other.

Common Communication Traps

Precision mistaken for certainty: A result shown to the dollar may still depend on approximate assumptions. Match displayed precision to the quality of the inputs and explain material uncertainty.
A sensitivity case presented as a likelihood: A stress calculation has no implied probability unless one has actually been assigned and justified.
A range presented without its construction: State which inputs vary and whether endpoints are tested bounds or probability-based limits.
An average confused with a typical outcome: A probability-weighted average can differ substantially from the most probable scenario, as the disruption example shows.
Omissions hidden in a caveat: Identify important excluded effects and explain how they could affect the decision, rather than relying on generic wording.

Key takeaway: Present the result together with its horizon, assumptions, uncertainty method, and material limitations. Say what the analysis supports—and what it does not—so a modeled number is not mistaken for a certain outcome.

Check Your Understanding

For each problem, identify the calculation and communicate what its result does and does not establish.

A hypothetical equipment plan has annual costs of $400,000 with probability 0.50, $1,200,000 with probability 0.35, and $3,000,000 with probability 0.15. The scenarios are mutually exclusive and exhaustive. Calculate the probability-weighted cost and the probability that cost exceeds $1,200,000. (Answer: weighted cost $1,070,000; probability of exceeding $1,200,000 is 0.15, or 15%.)
A hypothetical insurer forecasts $9,000,000 of annual revenue, 600 claims at an average cost of $10,000, and fixed expenses of $1,200,000. In a sensitivity case, claim count rises 5% and average claim cost rises 8%, while revenue and expenses are unchanged. Calculate the baseline and sensitivity-case margins and the percentage decline from baseline. (Answer: baseline margin $1,800,000; sensitivity margin $996,000; decline $804,000, or 44.7%.)
A hypothetical process handles 250,000 transactions annually, with base savings of $4.00 per transaction and annual cost of $900,000. Test transaction volume from 230,000 to 270,000 and savings per transaction from $3.50 to $4.20. Calculate base net benefit and the endpoint net-benefit range, assuming no probabilities are assigned. (Answer: base net benefit $100,000; tested range from $-\$95,000$ to $\$234,000$.)
A model assigns probabilities 0.70, 0.20, and 0.10 to annual costs of $100,000, $700,000, and $2,000,000. Calculate the weighted cost and state whether it is the most likely scenario cost. (Answer: weighted cost $410,000; no—the most likely scenario cost is $100,000.)
A forecast reports a $500,000 annual benefit, but the calculation excludes transition costs and assigns no probability to its low and high sensitivity cases. What two statements should accompany the estimate to prevent overinterpretation? (Answer: identify transition costs as an omitted effect that could reduce benefit, with its magnitude unquantified if unknown; label the sensitivity results as tested cases, not probability-based outcomes.)

What You'll Learn