Uncertainty in Dynamic Policy Counterfactuals

Why Policy Counterfactuals Have Sampling Uncertainty

A dynamic policy counterfactual is a model-based prediction: estimate behavioral parameters, change a payoff or transition rule, solve the resulting dynamic choice problem, and compare an outcome under the new policy with its baseline. The computed difference may look like a single number, but the parameters were estimated from a finite sample. A different sample could produce different parameter estimates, value functions, choices, and policy effects.

The uncertainty of interest is therefore not obtained by holding the estimated parameters fixed and varying simulated choices alone. It comes from sampling uncertainty in the estimated structural parameters, propagated through the entire counterfactual calculation. This tutorial develops three ways to do that: a delta-method approximation, parametric simulation, and bootstrap re-estimation.

Definition: Let $\theta$ denote the vector of structural parameters, and let $\hat{\theta}$ be its estimate from the observed sample. For a specified baseline and policy, define the policy effect as $g(\theta)$, the difference in a chosen outcome produced by solving the dynamic model under the policy and baseline using parameters $\theta$. The estimated policy effect is $g(\hat{\theta})$. Its sampling uncertainty is the variation in this estimate across hypothetical repeated samples from the same data-generating process.

Set Up the Counterfactual and Its Uncertainty

Before calculating an interval, specify the outcome, the population or initial-state distribution over which it is averaged, the policy change, and the time horizon. For example, an effect might be the change in expected discounted consumer value, or the change in the probability of choosing an action during the next year. A policy effect must compare the same target population and outcome under the baseline and policy scenarios.

Write the dynamic solution under scenario $a$ as $V^a(\theta)$, where $a$ identifies the baseline or policy. The solution includes the value functions and any simulated choice or state-transition quantities needed for the outcome. A typical estimated effect has the form

\hat{\Delta} = g(\hat{\theta}) = m\!\left(V^{\mathrm{policy}}(\hat{\theta}),\hat{\theta}\right) - m\!\left(V^{\mathrm{baseline}}(\hat{\theta}),\hat{\theta}\right),

where $m$ maps a solved model into the target outcome. Both scenario solutions depend on the same estimated parameters. Their uncertainty is consequently linked; it is generally wrong to treat the baseline and policy predictions as independent estimates.

The methods below rely on a correctly specified estimation and counterfactual procedure, an estimator $\hat{\theta}$ with a reliable estimate of its sampling covariance, and a policy effect that is well-defined near the estimate. For the usual delta-method approximation, $g$ must also be differentiable locally, and the estimator must have an approximately normal sampling distribution in the relevant sample size. The dynamic solution must be computed accurately enough that numerical error is small relative to sampling uncertainty. Constraints on parameters, such as probabilities or positive scale parameters, must be respected when drawing parameters.

Delta-Method Uncertainty

The delta method approximates the effect function locally by a linear function of the estimated parameters. Let $\widehat{\mathrm{Var}}(\hat{\theta})=\widehat{\Sigma}$ be the estimated covariance matrix of the parameter estimator, and let $\nabla g(\hat{\theta})$ be the gradient of the policy effect with respect to the parameters. The estimated variance of the policy effect is

\widehat{\mathrm{Var}}\{g(\hat{\theta})\} \approx \nabla g(\hat{\theta})^\top \widehat{\Sigma} \nabla g(\hat{\theta}).

The gradient describes how much the policy effect changes when each parameter changes slightly, with the other parameters held fixed. It can be computed analytically, by numerical derivatives, or by differentiating the model’s solution equations where that is feasible. Numerical derivatives require resolving the relevant model at perturbed parameter values; the perturbations must be large enough to avoid numerical noise but small enough to approximate local changes.

If the estimated effect is approximately normal, a two-sided approximate $100(1-\alpha)\%$ interval is $g(\hat{\theta})\pm z_{1-\alpha/2}SE$, where $SE$ is the square root of the estimated variance and $z_{1-\alpha/2}$ is the corresponding standard normal quantile. This interval uses both the parameter covariance and the sensitivity of the counterfactual to those parameters.

Worked Example: Delta-Method Standard Error

Suppose a hypothetical dynamic model estimates three parameters, and the estimated policy effect in thousands of dollars per household is $g(\hat{\theta})=0.420$. Numerical derivatives of the fully solved counterfactual give the gradient $(1.54,-1.54,0.135)^\top$. Suppose the estimated parameter covariance matrix is diagonal, with standard errors $0.04$, $0.03$, and $0.05$. The zero covariances are an explicit simplifying assumption for this example.

For a diagonal covariance matrix, the variance is the sum of each squared gradient component multiplied by the corresponding parameter variance:

\begin{aligned} \widehat{\mathrm{Var}}(\hat{\Delta}) &=1.54^2(0.04^2)+(-1.54)^2(0.03^2)+0.135^2(0.05^2)\\ &=2.3716(0.0016)+2.3716(0.0009)+0.018225(0.0025)\\ &=0.00379456+0.00213444+0.0000455625\\ &=0.0059745625. \end{aligned}

Thus $SE=\sqrt{0.0059745625}\approx 0.0773$ thousand dollars per household, or about $77.3. Using $1.96$ as the approximate 95% normal critical value gives

0.420\pm 1.96(0.0773) \approx [0.269,\;0.571] \quad\text{thousand dollars per household}.

Under the stated model and approximation, this interval expresses sampling uncertainty in the estimated policy effect: its endpoints are about $269 and $571 per household. It is not a 95% probability statement about the fixed effect being inside this particular interval.

Why the Delta-Method Formula Works

The justification is a first-order Taylor expansion around the true parameter $\theta$. If $g$ is differentiable and $\hat{\theta}$ is close to $\theta$, then

g(\hat{\theta}) \approx g(\theta)+\nabla g(\theta)^\top(\hat{\theta}-\theta).

If the parameter estimator has approximate covariance $\Sigma$, the variance of the linear term is

\mathrm{Var}\!\left\{\nabla g(\theta)^\top(\hat{\theta}-\theta)\right\} = \nabla g(\theta)^\top\Sigma\nabla g(\theta).

Replacing the unknown quantities by their estimates yields the delta-method variance formula. This argument also explains its limitation: it approximates the policy-effect function by a tangent plane. If the function is strongly curved over the range of plausible parameter values, or if the estimate is near a parameter boundary or a point where the model solution changes non-smoothly, the approximation may be poor.

Parametric Simulation Through the Value Functions

Parametric simulation propagates uncertainty by generating parameter vectors from an estimated sampling distribution, then repeating the counterfactual calculation for each vector. In a common large-sample implementation, draw $\theta^{(b)}$ from a multivariate normal distribution centered at $\hat{\theta}$ with covariance $\widehat{\Sigma}$. For every draw, solve the baseline and policy models using the same $\theta^{(b)}$, and calculate $\Delta^{(b)}=g(\theta^{(b)})$. Quantiles of the resulting effects provide an interval.

Using the same parameter draw for both scenarios preserves their dependence and usually reduces unnecessary simulation noise in the difference. Each draw must be passed through the value-function solution and the policy outcome calculation; drawing an effect directly from a normal distribution would discard the nonlinear mapping that this method is intended to capture.

Draw parameters.
Generate $\theta^{(b)}$ from the estimated sampling distribution, respecting parameter constraints.

Resolve both scenarios.
For the same draw, solve the baseline and policy dynamic models and calculate their target outcomes.

Form the effect distribution.
Store $\Delta^{(b)}=g(\theta^{(b)})$ and repeat for many draws.

Summarize uncertainty.
Use suitable quantiles of the simulated effects for an interval, and report the point estimate and units.

Worked Example: A Simulated Policy-Effect Interval

Consider a hypothetical policy that changes a recurring fee in a dynamic service-use model. The estimated effect is $0.420$ thousand dollars of discounted net benefit per household. A researcher draws 5,000 parameter vectors from the estimated multivariate normal distribution, solves both scenarios for every draw, and obtains 5,000 policy effects. Suppose the simulated 2.5th percentile is $0.270$ and the 97.5th percentile is $0.610$, both in thousands of dollars per household. The simulated median is $0.420$, and the simulated mean is $0.440$.

The percentile interval is therefore $[0.270,0.610]$ thousand dollars, or $270 to $610 per household. The mean and median differ, suggesting some asymmetry in the simulated effect distribution. That asymmetry is retained in this percentile interval, unlike the symmetric delta-method interval in the previous example.

This interval reflects uncertainty under the chosen parameter-draw distribution and the maintained dynamic model. It is not automatically robust to model misspecification, an unsuitable covariance estimate, or numerical errors in solving the model. If a normal parameter draw produces inadmissible parameter values, the researcher must use a valid parameterization or another justified sampling procedure rather than silently treating invalid draws as meaningful.

Bootstrap Intervals by Re-Estimation

A bootstrap assesses sampling variability by creating repeated data sets from the observed sample, re-estimating the model in each data set, and recomputing the counterfactual from each estimate. If observations are independent decision-makers, a nonparametric bootstrap typically resamples decision-makers with replacement. If the data contain repeated observations for each decision-maker, resample whole decision-maker histories rather than treating dependent periods as independent units.

For bootstrap replication $b$, let $\hat{\theta}^{*(b)}$ be the estimate from the resampled data. Re-solve both policy scenarios and record $\Delta^{*(b)}=g(\hat{\theta}^{*(b)})$. A percentile interval uses the empirical 2.5th and 97.5th percentiles of these effects for a nominal 95% interval. The model must be successfully estimated and solved in enough replications for those quantiles to be stable.

Worked Example: Bootstrap Percentile Interval

Suppose a hypothetical data set contains 800 independent households, each with an observed choice history. The researcher resamples 800 complete household histories with replacement for each of 999 bootstrap replications, re-estimates the dynamic choice model, and recomputes the same policy effect. Assume the original estimate is $0.420$ thousand dollars per household and the empirical 2.5th and 97.5th percentiles of the 999 bootstrap effects are $0.250$ and $0.630$.

The bootstrap percentile interval is $[0.250,0.630]$ thousand dollars per household, or $250 to $630. The procedure captures how re-estimation and the subsequent nonlinear counterfactual calculation vary across resampled data sets. It does not mean that the policy effect has a 95% probability of lying in this observed interval.

The resampling unit matters: resampling individual periods from these households would break the within-household choice histories and would not reproduce the stated sampling design. Bootstrap validity also depends on the sample being informative for the model and on the estimator and dynamic solver succeeding reliably across replications.

Choosing and Reporting an Approach

Approach	Main calculation	Strength	Important limitation
Delta method	Gradient and estimated parameter covariance	Fast and transparent when the effect is locally smooth	Can be inaccurate for strong curvature, skewness, or boundary behavior
Parametric simulation	Draw parameters, solve the model, and take effect quantiles	Retains nonlinear transformation of parameter uncertainty	Depends on the assumed parameter-draw distribution and covariance estimate
Bootstrap	Resample data, re-estimate, solve, and take effect quantiles	Repeats estimation and the complete counterfactual pipeline	Can be computationally demanding and depends on valid resampling units

A useful report names the policy, outcome, target population, horizon, point estimate, interval method, and units. It should also state whether the interval is based on a first-order approximation, parametric parameter draws, or bootstrap re-estimation. When computationally feasible, comparing methods can reveal sensitivity to nonlinearity or distributional assumptions; similar intervals are reassuring but do not establish that the structural model is correct.

Common Pitfall: Holding the Solved Model Fixed

A tempting but incorrect argument is: “The policy effect has been simulated, so it is known exactly; only the choices within the simulation need random draws.” That confuses simulation noise with sampling uncertainty. If the policy effect is computed from $\hat{\theta}$, uncertainty remains because another sample could produce a different $\hat{\theta}$. Randomizing choices while holding the estimated parameters fixed does not, by itself, propagate that parameter uncertainty.

The correction is to vary the estimated parameters according to their sampling uncertainty and then carry each variation through the model’s value-function solution and policy comparison. In a bootstrap, this requires re-estimation; in parametric simulation, it requires drawing parameter vectors. Any additional randomness from simulated outcomes can be included when it is part of the target procedure, but it should be distinguished from the uncertainty due to estimating $\theta$.

Key takeaway: A dynamic policy effect is a function of estimated structural parameters and model solutions. Quantifying its sampling uncertainty requires propagating parameter uncertainty through both counterfactual scenarios, using a local approximation, parameter simulation, or bootstrap re-estimation.

Check Your Understanding

Try these questions without looking back.

Why must baseline and policy outcomes generally be computed using the same parameter draw when forming a simulated policy effect?
In the delta method, what does the gradient measure, and what role does the parameter covariance matrix play?
A simulated 95% percentile interval is asymmetric around the point estimate. What feature of the method allows this?
For panel data with repeated choices by household, why might resampling individual periods give an invalid bootstrap procedure?
Explain why randomizing choices while holding estimated structural parameters fixed does not, on its own, quantify sampling uncertainty in the estimated policy effect.