Bayesian Estimation and Prior Choices

Why Prior Choice Matters in Shared-Parameter Models

A shared-parameter model connects an outcome process and a dropout process through latent subject-specific effects. The same latent effect can describe, for example, a tendency toward higher outcome values and a tendency toward earlier dropout. Because dropout can be related to unobserved outcomes through that effect, the association is not generally determined by the observed data alone. Bayesian inference makes this uncertainty explicit through a posterior distribution, but that distribution depends on both the likelihood and the prior.

The earlier material on joint likelihoods and EM estimation establishes the likelihood-based framework. Here the focus is the Bayesian extension: specifying priors for the fixed effects, random-effect covariance, and outcome-dropout association; combining them with the joint likelihood; and interpreting the resulting posterior uncertainty. A prior is not a device for making a difficult model automatically identifiable. Its scale and structure must be defensible and its influence examined.

Model and Posterior Distribution

Let \(Y_i\) denote the outcomes observed for subject \(i\), and let \(T_i\) denote the observed dropout information, such as a dropout time or a sequence of discrete-time dropout indicators. Let \(b_i\) be a \(q\)-dimensional latent random-effect vector shared by the two processes. Write \(\theta\) for outcome-model parameters, \(\gamma\) for dropout-model parameters, and \(\alpha\) for an association parameter. A common specification assumes

The conditional-independence statement means that, once the shared effects and model parameters are given, the outcome and dropout contributions factor. It does not assert that outcomes and dropout are independent marginally. Let \(f_Y(Y_i\mid b_i,\theta)\) and \(f_T(T_i\mid b_i,\gamma,\alpha)\) denote the appropriate conditional contributions, including the observation and dropout pattern. The joint likelihood with latent effects included is

where \(\phi_q(\cdot;0,D)\) is the multivariate normal density and \(\mathbf b=(b_1,\ldots,b_n)\). If the random effects are integrated out, subject \(i\)'s observed-data contribution is instead

These are two representations of the same model, not two different assumptions. Bayesian computation may retain the \(b_i\)'s as latent variables or work with an integrated likelihood. In either case, priors are needed for all unknown parameters. Denote their joint density by \(\pi(\theta,\gamma,\alpha,D)\). Bayes' rule gives a posterior density proportional to likelihood times prior:

Choosing Priors for Covariance and Association

A covariance matrix \(D\) must be positive definite. For multiple random effects, a useful parameterization separates standard deviations and correlations:

where \(\tau_k>0\) is the standard deviation of random effect \(k\), and \(\Omega\) is a correlation matrix. One practical prior family assigns weakly or moderately informative positive priors to the \(\tau_k\)'s and an LKJ prior to \(\Omega\). For example, \(\tau_k\sim\text{half-Normal}(0,A_k^2)\) and \(\Omega\sim\text{LKJ}(\eta)\). The half-Normal is the distribution of the absolute value of a centered normal variable; \(A_k\) sets its scale. The LKJ density is proportional to \(|\Omega|^{\eta-1}\). In two dimensions, \(\eta=1\) is uniform over the allowable correlation, while \(\eta>1\) gives increasing concentration toward zero correlation. Neither choice should be described as neutral without considering the dimension and parameterization.

An inverse-Wishart prior is another option for \(D\), but it couples marginal variances and correlations. That coupling can be undesirable when prior knowledge about standard deviations and correlations differs. A separation prior makes those components easier to inspect and communicate. Priors for outcome residual scales are also commonly positive, such as half-Normal or half-\(t\) priors; fixed effects can receive Normal priors on scientifically meaningful scales.

The association parameter needs particular care. In a discrete-time hazard model, for example,

a one-unit increase in \(b_{i1}\), holding the other terms fixed, multiplies the dropout odds in interval \(j\) by \(\exp(\alpha)\). Thus the meaning of a prior on \(\alpha\) depends on the scale of \(b_{i1}\). If that random effect is rescaled, the numerical value and interpretation of \(\alpha\) change. Specify the random-effect scale first, then choose a prior on \(\alpha\) by examining the implied odds multipliers or hazard changes over plausible values.

Posterior Uncertainty and a Short Derivation

Posterior summaries should reflect uncertainty in the association and covariance parameters, not just uncertainty in the outcome regression. A posterior mean or median summarizes the center of a marginal posterior; a credible interval summarizes posterior mass under the full model and chosen priors. The interval is conditional on the model specification and prior choices, so it is not a guarantee that the model is correct.

The basic posterior calculation follows directly from Bayes' rule. For a scalar parameter \(\alpha\), suppose the likelihood is \(L(\alpha)\) and its prior density is \(\pi(\alpha)\). The joint density of data and parameter is \(L(\alpha)\pi(\alpha)\). Integrating over \(\alpha\) gives the marginal data density \(m=\int L(a)\pi(a)\,da\); dividing by \(m\) normalizes the joint density and yields

This derivation also explains why a prior can matter most when the likelihood is relatively flat: the likelihood then changes little across values where the prior may vary substantially. In shared-parameter models, sparse dropout events, limited variation in dropout times, or weak separation of association from random-effect variance can leave the likelihood uninformative about \(\alpha\). Posterior concentration in such a setting may partly reflect prior information rather than strong empirical identification.

Assumptions, Checks, and Limits

The posterior is meaningful only relative to a coherent sampling model and prior specification. Before interpreting it, verify that the outcome and dropout contributions match the observed data structure, including which outcomes are available before dropout and how dropout indicators or times are encoded. Confirm that the conditional-independence assumption is the one intended: dependence between outcome and dropout is represented through the shared effects, not silently omitted. Check that covariance priors have support only on valid positive-definite matrices and that all chosen priors produce a proper posterior for the intended analysis.

Prior predictive checks are useful before fitting. Draw parameters and latent effects from the priors, generate hypothetical outcomes and dropout patterns, and ask whether the resulting ranges, dropout frequencies, and outcome-dropout associations are plausible. After fitting, compare posterior summaries under defensible alternative scales for \(\alpha\), random-effect standard deviations, and correlations. Large changes indicate that the data do not tightly determine those features. Posterior computation also requires numerical checks, but convergence and boundary diagnostics are a separate step; a plausible-looking posterior summary should not be taken as evidence that a simulation algorithm has sampled it reliably.

Bayesian estimation does not remove limitations of the model. If different combinations of \(\alpha\), \(D\), and dropout parameters imply similar observed-data distributions, posterior dependence can be strong and posterior inference can be sensitive to priors. Nor does a narrow posterior interval establish that the shared-effect distribution or link function is correct. Model checking and sensitivity to latent-effect distributions remain important.

Compared with maximum likelihood estimation using adaptive quadrature or EM estimation, Bayesian estimation combines the likelihood with explicit prior distributions and yields posterior distributions for parameters and latent effects. Likelihood-based methods do not require priors, though they still rely on the same substantive model and may have weakly identified parameters. Bayesian estimates can stabilize weakly informed components when prior information is defensible; they can also conceal that weakness if prior implications are not reported. Neither approach turns an untestable dropout assumption into a fact.

Common Pitfall