Why Diagnose a Dynamic Choice Model?
A dynamic discrete choice model is useful only to the extent that its assumptions capture the observed sequence of choices and states. A model can fit an aggregate choice rate yet miss important patterns: choices may be wrong for particular states, actions may predict subsequent states poorly, or observed sequences may be rare under the model. Such discrepancies matter because predictions and policy counterfactuals depend on more than the overall fraction choosing each action.
Diagnostics compare features of the observed data with the corresponding implications of a fitted model. They are not a mechanical procedure that reveals which assumption is false. A mismatch can point toward an inadequate state description, payoff specification, transition law, or shock structure; several of these can generate similar symptoms. The goal is to identify where the model has difficulty and to guide further investigation.
The Sequential Structure of the Model
Let \(s_{it}\) denote decision-maker \(i\)'s state at period \(t\), and let \(a_{it}\) be the action chosen. Write \(p_\theta(a\mid s)\) for the model-implied probability of choosing \(a\) in state \(s\), with estimated parameters collected in \(\theta\). Write \(q_\theta(s'\mid s,a)\) for the probability of next state \(s'\), conditional on current state \(s\) and action \(a\).
For the diagnostics below, the maintained specification has a sufficient state: conditional on \(s_{it}\), the choice rule incorporates the relevant payoff-relevant history and shock information. The state transition follows \(q_\theta\) conditional on the current state and action. The model also specifies, or conditions on, the initial-state distribution. These are assumptions to assess, not facts guaranteed by fitting an estimator.
The order of events is essential. Given an initial state \(s_{i1}\), a joint path of choices and subsequent states has probability formed successively: first a choice, then its state transition, then the next choice, and so on. For a path through period \(T\), its conditional probability is
This factorization is a sequential construction of the joint path probability. It does not imply that choices are independent conditional on the entire realized state path. If a later state depends on an earlier action, observing that state can itself provide information about the action. Diagnostics for histories must preserve the choice-then-transition order rather than condition on a complete state path and multiply choice probabilities without justification.
Choice Frequencies Within States
A first check compares observed and predicted choice frequencies for substantively relevant state groups. If \(n_s\) observations have state \(s\), the model predicts \(n_s p_\theta(a\mid s)\) choices of action \(a\), conditional on those states. For continuous or high-dimensional states, groups can be defined using pre-specified bins or summaries, or predictions can be compared across ranges of the state. Grouping too finely produces noisy counts; grouping too broadly can hide systematic misfit.
Worked Example: Choice Frequencies in Two State Groups
Suppose a fitted model predicts action 1 with probability \(0.25\) in a low-demand state and \(0.75\) in a high-demand state. In a hypothetical sample, 120 decision periods are low-demand and 80 are high-demand. The observed action-1 counts are 39 and 51, respectively. Treat the fitted probabilities as fixed for this descriptive comparison.
Low demand. The predicted count is \(120(0.25)=30\). The observed count is 39, a difference of \(39-30=9\).
High demand. The predicted count is \(80(0.75)=60\). The observed count is 51, a difference of \(51-60=-9\).
Overall, the model predicts \(30+60=90\) action-1 choices, and the data contain \(39+51=90\). Thus the aggregate frequency matches exactly, while the conditional frequencies do not: the model underpredicts action 1 in low demand and overpredicts it in high demand. This pattern is more informative than the aggregate match, but it does not by itself establish whether the problem lies in payoffs, the state description, or another assumption.
Residual patterns across state groups can be useful even when a formal test is not the objective. A tendency for observed choices to exceed predictions at one end of a state range and fall below them at the other may indicate a misspecified state dependence or payoff function. Such an interpretation is a lead for diagnosis, not a unique conclusion: omitted state variables or differences in shock distributions may also alter conditional choice rates.
Transition Distributions
A separate diagnostic checks whether the model predicts what happens after each state-action pair. For a discrete next state \(s'\), compare the observed fraction transitioning to \(s'\) among observations with current state \(s\) and action \(a\) to \(q_\theta(s'\mid s,a)\). This evaluates the transition component directly. Because actions are chosen rather than randomly assigned, it is not a causal comparison of actions; it is a check of the specified conditional transition distribution in the observed state-action strata.
Worked Example: Checking a Binary State Transition
Suppose the next-period state is either low or high operating cost. For a particular current state, the model predicts a high-cost next state with probability \(0.40\) after action “maintain” and probability \(0.20\) after action “replace.” In hypothetical data, there are 30 maintain decisions, of which 18 are followed by high cost, and 20 replace decisions, of which 4 are followed by high cost.
After maintain, the observed high-cost fraction is \(18/30=0.600\). The model-predicted number is \(30(0.40)=12\), so the observed count is 6 above prediction.
After replace, the observed fraction is \(4/20=0.200\). The predicted count is \(20(0.20)=4\), exactly the observed count. The transition discrepancy is therefore concentrated in the maintain stratum in this sample.
This does not show that maintaining causes high cost more often than the model allows: decisions may reflect state or information not included in the comparison. It does show that the fitted transition distribution has difficulty matching the observed conditional transition frequency for the specified current state and action. Further checks should examine whether the current state captures relevant determinants of future cost and whether the transition law is flexible enough.
Choice Histories and Joint Path Checks
Some model implications concern histories rather than one-period margins. For example, a model may match the action frequency in each period and still fail to reproduce the frequency of particular action-state sequences. To evaluate a history, multiply the successive choice and transition probabilities in chronological order. If the initial state is random rather than conditioned on, include its model probability as well.
Worked Example: Probability of a Two-Period History
Consider decision-makers who begin in state \(s_1\). The model gives \(P(a_1=1\mid s_1)=0.60\), \(P(s_2=H\mid s_1,a_1=1)=0.80\), and \(P(a_2=0\mid s_2=H)=0.30\). The probability of the history “choose 1, transition to \(H\), then choose 0” is
For 250 independent decision-makers with this same initial state, the expected count of this history is \(250(0.144)=36\). Suppose 21 such histories are observed. The observed fraction is \(21/250=0.0840\), well below the model probability \(0.144\). This is a warning of possible path-level misfit, though sampling variation and parameter estimation uncertainty must be considered before judging its strength.
The calculation uses the transition probability conditional on the action that precedes it. It does not calculate the probability of a choice sequence conditional on the entire state path. That distinction matters whenever state evolution depends on choices.
In practice, one can select a set of histories or summary statistics before examining the results, then compare observed values with model-implied values. Alternatively, simulate complete paths from the fitted model using the same initial-state distribution and observation horizon as the data. Simulated distributions show how much variation the model itself predicts. If the model was estimated from the same sample, simulations or uncertainty calculations should account for parameter estimation where feasible; treating fitted probabilities as known can make diagnostic discrepancies appear more precise than they are.
How to Interpret a Mismatch
The most useful diagnostic summaries are tied to a specific model implication and conditioning set. A global choice share addresses a different question from a choice share by state, and both differ from a transition distribution or a complete history frequency. Reporting the definition of each comparison makes it possible to see which part of the specification is being challenged.
Potential warning signs include:
- Choice residuals vary systematically with a state variable, suggesting that payoffs, state dependence, or the state representation may be inadequate.
- Observed transitions depart from \(q_\theta(s'\mid s,a)\) in particular state-action groups, suggesting a transition-law problem or omitted predictors of state evolution.
- Choice patterns persist across periods more strongly than model simulations predict, raising concern about omitted persistent heterogeneity or serial dependence in choice shocks.
- Predictions are poor for groups with different histories despite similar recorded current states, suggesting that the recorded state may not summarize relevant history.
These are hypotheses to investigate, not one-to-one diagnoses. For example, apparent serial dependence can arise because a persistent determinant is omitted from the state; an omitted state can in turn make a payoff specification appear wrong. Earlier material on serially correlated choice shocks and state augmentation, discrete latent types, and state-space complexity provides relevant model-building concepts, but no single residual pattern settles the issue.
Uncertainty and Practical Checks
Observed frequencies fluctuate even when a model is correctly specified. Moreover, estimated parameters make predicted frequencies uncertain. A diagnostic should therefore distinguish a notable discrepancy from ordinary sampling variation. For independent individuals, simulation-based reference distributions can be formed by drawing paths under the fitted model and recomputing the same diagnostic summaries. In panel data, the simulation should preserve the relevant unit-level sequence structure; treating every decision period as an independent observation can understate uncertainty when periods within a decision-maker are dependent.
The simulation design should also match the target comparison. If the diagnostic conditions on observed initial states, simulate paths from those initial states. If the model is being checked unconditionally, simulate initial states from the specified initial-state distribution. When estimation uncertainty matters, re-estimating parameters in suitable resamples or incorporating parameter draws can reflect that additional source of variation.
A useful report states the observed statistic, its model-implied counterpart or reference distribution, the sample or strata used, and whether uncertainty from repeated observations and parameter estimation was addressed. Large departures can motivate revising the specification and rechecking its implications. They do not establish that a particular revision is correct, and repeated exploration across many groups should be treated cautiously.
Common Pitfall
A related mistake is to treat a matching aggregate choice share as evidence that the entire model fits. The first worked example shows why: overprediction and underprediction in different state groups can cancel exactly in the total. Diagnostics should examine multiple implications while keeping their conditioning sets and uncertainty clear.
Check Your Understanding
Answer these questions by referring to the model's conditioning structure.
- A model predicts the correct overall action-1 frequency but overpredicts it in one state group and underpredicts it in another. Why is the aggregate match insufficient?
- For a transition diagnostic, what variables should define the conditioning stratum, and why is this not automatically a causal comparison of actions?
- Suppose a next state records the action taken in the prior period. Why might conditioning on the full state path change the probability of an earlier action?
- What additional uncertainty arises when choice probabilities used in diagnostics are estimated rather than known?
- Name two different model features that could plausibly produce unusually persistent observed choice histories.