Likelihood Construction for Dynamic Choices

Why the Likelihood Has Several Parts

A dynamic choice observation is more than a sequence of actions. Each action is taken in a state, and the state may change after the action. A panel therefore provides evidence about both the conditional choice probabilities and the law governing state transitions. If the first observed state is random under the model, its probability or density also contributes to the likelihood.

The earlier material on choice-specific values and conditional choice probabilities supplies the model-implied probability of each observed action at its state. Here the task is to assemble those probabilities with the observed state history. Getting the factorization right matters: omitting a transition term discards information about the state process, while multiplying by a choice or transition term that is not part of the sampling design changes the likelihood being optimized.

The state can contain discrete variables, continuous variables, or both. Accordingly, \(\mu_\theta\) and \(q_\theta\) are probability masses for discrete states and densities for continuous states; for mixed states they are defined with respect to the appropriate underlying measure. A density evaluated at an observed value is not itself a probability of observing that exact continuous value. It is a density contribution to the likelihood, with units inverse to the units of the continuous variable.

Model Assumptions and the Panel Likelihood

The basic factorization uses three assumptions. First, conditional on the parameter, individuals’ panel histories are independent. Second, the choice probability at period \(t\), conditional on the information relevant to choice, is \(p_\theta(a_{it}\mid s_{it})\); the choice shocks have already been integrated into this probability. Third, the state follows a first-order transition law: conditional on the current state and action, the next state has distribution \(q_\theta(s_{i,t+1}\mid s_{it},a_{it})\), with no additional dependence on earlier history. These are modeling assumptions, not consequences of the arithmetic used to construct the likelihood.

Suppose states and actions are observed for \(T\) choice periods, and the state transitions from periods 1 through \(T-1\) are observed. The individual contribution to the full likelihood is

There is one initial-state term, one choice term for each observed action, and one transition term for each observed transition. There are \(T-1\) transitions in this observation scheme because no transition after the final choice is included. If the data also record the state after the final choice, that additional transition contributes a further \(q_\theta\) term.

For \(N\) independent individuals, the sample likelihood is the product of their contributions, \(L(\theta)=\prod_{i=1}^{N} L_i(\theta)\). The log likelihood is therefore a sum. Taking logarithms turns products into additions but does not change the parameter value that maximizes the likelihood, since the logarithm is a strictly increasing function on positive values.

Why the Factorization Holds

The chain rule expresses the probability or density of a history as a product of successive conditional distributions. For two choice periods, its ordering can be written as

The first-order state and choice assumptions reduce the last conditional choice distribution to \(p_\theta(a_2\mid s_2)\). Repeating the same chain-rule decomposition for each later period gives the general individual likelihood above. Thus the product form is justified by the assumed conditional structure of the model. It is not justified merely because the observations happen to be recorded in chronological order.

Worked Likelihoods

Initial States, Conditioning, and Data Design

Whether the initial-state term belongs in the likelihood depends on the statistical experiment being modeled. In a full likelihood, the model specifies how the first observed state was generated, and \(\mu_\theta(s_{i1})\) is included. If the analysis conditions on the initial state—for example, because the sample is treated as having been selected conditional on that state—the initial-state factor is omitted. The resulting conditional likelihood is a legitimate different likelihood, but it uses less information about parameters that govern the initial-state distribution.

The same logic applies to actions. If the likelihood is explicitly conditional on an observed action, its choice probability is excluded. If the action is part of the random history being modeled, its conditional choice probability belongs in the product. State clearly which observations are conditioned upon before writing the likelihood.

A transition factor belongs in the likelihood when the relevant next state is observed and its conditional law is part of the model. If the next state is continuous, use its density; if it is discrete, use its transition probability. If no post-choice state is recorded at the end of the panel, do not add a transition term for an unobserved next state. The choice at the final observed period still contributes its CCP.

Common Mistakes and Why Construction Matters

• Leaving out transitions. Multiplying only the CCPs treats observed state histories as if they supplied no information about the transition law. Include each observed transition term unless the likelihood is deliberately conditional on the relevant states.
• Using a transition probability for a continuous state. A continuous observation has a density contribution, not a point probability. Its units and scale should be understood when interpreting the likelihood.
• Including a term for an unobserved terminal transition. A transition after the last observed choice is not part of the stated likelihood unless the data include the next state or the observation scheme otherwise supplies it.
• Dropping the initial-state term without explanation. This is appropriate for a likelihood conditional on the initial state, but not automatically for a full likelihood that models how initial states arise.
• Multiplying by shock densities after using CCPs. The conditional choice probability already accounts for the choice shocks integrated out in the model. Do not add a shock-density factor unless the shocks themselves are observed and modeled as data.

Likelihood construction determines which aspects of the data inform estimation. Choice frequencies inform the CCPs, observed state changes inform transition parameters, and initial-state frequencies can inform parameters in the initial distribution. Keeping these contributions distinct makes the assumptions visible and helps diagnose an incorrectly specified objective.