Bayesian Estimation of Pattern-Specific Distributions

Why Estimate a Distribution Within Each Pattern?

A pattern-mixture analysis describes outcome distributions conditional on dropout pattern. For example, participants who provide outcomes at every scheduled visit form one pattern, while participants who stop after the first visit form another. A pattern-specific distribution summarizes the observed longitudinal outcomes for members of that pattern; it is not automatically a summary for all participants.

This tutorial develops Bayesian estimation for a multivariate normal distribution within one pattern. The central benefit is that a prior can stabilize estimates when a pattern contains few participants, while posterior summaries express uncertainty about the pattern-specific mean and covariance. That benefit comes with a responsibility: the result can be sensitive to prior choices, and conditioning on a pattern does not by itself determine outcomes that were not observed.

Model: Index dropout patterns by $r$, and let $n_r$ be the number of participants in pattern $r$. For the outcome components available in that pattern, write participant $i$'s outcome vector as $Y_{ir}$, of dimension $q_r$. The pattern-specific normal model is $$ Y_{ir}\mid \mu_r,\Sigma_r \sim N_{q_r}(\mu_r,\Sigma_r),\qquad i=1,\ldots,n_r, $$ with conditional independence across participants. Here $\mu_r$ is the pattern-specific mean vector and $\Sigma_r$ is the pattern-specific covariance matrix. The likelihood uses the outcome components observed for that pattern. It does not, on its own, specify the distribution of components unobserved for its members.

The model assumes that, within a given pattern, participants' outcome vectors have a common mean and covariance and follow a multivariate normal distribution. It also assumes independent participant vectors conditional on those parameters. These are modeling assumptions, not consequences of Bayesian estimation. The normal model may be a useful approximation for continuous outcomes, but it can be unsuitable for strongly skewed, bounded, binary, ordinal, or count outcomes.

For clarity, first consider a pattern with the same set of $q_r$ outcome components available for every member. Let $q=q_r$, $n=n_r$, and omit $r$ from the notation while fitting this one pattern. Write the observed mean vector as $\bar{Y}=n^{-1}\sum_iY_i$, and the within-pattern scatter matrix as $S=\sum_i(Y_i-\bar{Y})(Y_i-\bar{Y})^\mathsf{T}$. The superscript $\mathsf{T}$ denotes transpose.

A Conjugate Prior and Its Interpretation

A standard conjugate prior for the mean vector and covariance matrix is the normal–inverse-Wishart prior. It allows a direct update from the observed pattern-specific sample summaries. The parameterization below fixes the meaning of the inverse-Wishart scale matrix $\Psi_0$, which matters when computing posterior covariance summaries.

Definition: The normal–inverse-Wishart prior is $$ \Sigma\sim \operatorname{IW}(\nu_0,\Psi_0),\qquad \mu\mid\Sigma\sim N_q\left(m_0,\frac{\Sigma}{\kappa_0}\right). $$ Here $\kappa_0>0$, $\Psi_0$ is positive definite, and $\nu_0>q-1$ gives a proper inverse-Wishart distribution. Under this convention its density is proportional to $$ |\Sigma|^{-(\nu_0+q+1)/2} \exp\left\{-\frac{1}{2}\operatorname{tr}(\Psi_0\Sigma^{-1})\right\}. $$ When $\nu_0>q+1$, the prior mean of $\Sigma$ exists and equals $\Psi_0/(\nu_0-q-1)$.

The vector $m_0$ is the prior center for the mean, and $\kappa_0$ controls how concentrated the conditional prior for $\mu$ is around that center: given $\Sigma$, its covariance is $\Sigma/\kappa_0$. The matrix $\Psi_0$ controls the prior for covariance, together with $\nu_0$. Prior values should be expressed in the same outcome units and time ordering as the data. A proper prior is not automatically a reasonable prior; its implications for plausible means and covariances should be checked before interpreting results.

With this prior, the likelihood for the $n$ observed vectors is proportional to $$ |\Sigma|^{-n/2} \exp\left\{-\frac{1}{2}\sum_{i=1}^n(Y_i-\mu)^\mathsf{T}\Sigma^{-1}(Y_i-\mu)\right\}. $$ It is a likelihood conditional on membership in this pattern. It contains no term for the probability of belonging to the pattern, so it does not estimate pattern frequencies or combine pattern-specific means into an overall mean.

Worked Example: Setting Up a Scalar Pattern-Specific Model

Suppose, hypothetically, that three participants in an early-dropout pattern have a continuous outcome measured at the one visit available to them: 4, 6, and 5 points. Consider a normal model $Y_i\mid\mu,\sigma^2\sim N(\mu,\sigma^2)$. Use the scalar version of the prior with $m_0=4$, $\kappa_0=1$, $\nu_0=4$, and $\Psi_0=2$.

The sample mean is $\bar{Y}=(4+6+5)/3=5$, and the scatter is $S=(4-5)^2+(6-5)^2+(5-5)^2=2$ points squared. The likelihood is proportional to $(\sigma^2)^{-3/2}\exp\{-\sum_i(Y_i-\mu)^2/(2\sigma^2)\}$. The prior on $\sigma^2$ is inverse-Wishart in dimension one, equivalently inverse-gamma under the corresponding parameterization; the conditional prior is $\mu\mid\sigma^2\sim N(4,\sigma^2)$.

Since $\nu_0=4>q+1=2$, the prior variance mean exists and is $E(\sigma^2)=\Psi_0/(\nu_0-2)=2/2=1$ point squared. This setup says the prior mean for the outcome is 4 points, with uncertainty about both the mean and variance. It does not assert that 4 is the true mean or that the observations are representative of other dropout patterns.

Posterior Updates

For the normal–inverse-Wishart model, the posterior has the same form as the prior. Let $m_n,\kappa_n,\nu_n,\Psi_n$ denote its parameters. The update incorporates the within-pattern scatter and the discrepancy between the observed mean and the prior mean.

Theorem: Under the independent multivariate normal likelihood and the normal–inverse-Wishart prior above, $$ \begin{aligned} \kappa_n&=\kappa_0+n, & \nu_n&=\nu_0+n,\\ m_n&=\frac{\kappa_0m_0+n\bar{Y}}{\kappa_n}, & \Psi_n&=\Psi_0+S+ \frac{\kappa_0n}{\kappa_n}(\bar{Y}-m_0)(\bar{Y}-m_0)^\mathsf{T}. \end{aligned} $$ Consequently, $$ \Sigma\mid Y_1,\ldots,Y_n\sim\operatorname{IW}(\nu_n,\Psi_n),\qquad \mu\mid\Sigma,Y_1,\ldots,Y_n\sim N_q\left(m_n,\frac{\Sigma}{\kappa_n}\right). $$

A short derivation explains the extra term in $\Psi_n$. The observed-data quadratic form separates into within-pattern scatter and the distance between the sample mean and the proposed mean. Combining that expression with the prior quadratic term gives the identity $$ \sum_i(Y_i-\mu)(Y_i-\mu)^\mathsf{T} +\kappa_0(\mu-m_0)(\mu-m_0)^\mathsf{T} = S+\frac{\kappa_0n}{\kappa_n}(\bar{Y}-m_0)(\bar{Y}-m_0)^\mathsf{T} +\kappa_n(\mu-m_n)(\mu-m_n)^\mathsf{T}. $$ The final term is the completed square in $\mu$; it gives the updated conditional normal distribution. The first two terms that do not involve $\mu$, added to $\Psi_0$, give $\Psi_n$. The likelihood contributes $n$ to both $\kappa_n$ and $\nu_n$. This derivation also shows why the prior-to-data disagreement affects posterior uncertainty about covariance as well as the posterior mean.

Worked Example: Computing a Bivariate Posterior

Suppose three hypothetical participants in one pattern have two continuous outcomes, measured at visits 1 and 2. Their vectors are $(-1,0)^\mathsf{T}$, $(0,1)^\mathsf{T}$, and $(1,2)^\mathsf{T}$, in outcome units. Use $m_0=(0,0)^\mathsf{T}$, $\kappa_0=1$, $\nu_0=5$, and $\Psi_0=\begin{pmatrix}2&0\\0&2\end{pmatrix}$. The prior is proper, and its covariance mean exists because $\nu_0=5>q+1=3$.

The sample mean is $\bar{Y}=(0,1)^\mathsf{T}$. The deviations from it are $(-1,-1)^\mathsf{T}$, $(0,0)^\mathsf{T}$, and $(1,1)^\mathsf{T}$, so $S=\begin{pmatrix}2&2\\2&2\end{pmatrix}$. The updated scalar parameters are $\kappa_n=1+3=4$ and $\nu_n=5+3=8$, while $m_n=(1(0,0)^\mathsf{T}+3(0,1)^\mathsf{T})/4=(0,0.75)^\mathsf{T}$.

The mean-discrepancy contribution is $$ \frac{1\cdot3}{4} \begin{pmatrix}0\\1\end{pmatrix} \begin{pmatrix}0&1\end{pmatrix} = \begin{pmatrix}0&0\\0&0.75\end{pmatrix}. $$ Therefore the posterior scale matrix is $$ \Psi_n= \begin{pmatrix}2&0\\0&2\end{pmatrix} +\begin{pmatrix}2&2\\2&2\end{pmatrix} +\begin{pmatrix}0&0\\0&0.75\end{pmatrix} = \begin{pmatrix}4&2\\2&4.75\end{pmatrix}. $$ In particular, the lower-right entry is $2+2+0.75=4.75$. The posterior is $\Sigma\mid Y\sim\operatorname{IW}(8,\Psi_n)$ and $\mu\mid\Sigma,Y\sim N_2((0,0.75)^\mathsf{T},\Sigma/4)$.

Because $8>q+1=3$, the posterior mean covariance exists: $E(\Sigma\mid Y)=\Psi_n/(8-2-1)=\Psi_n/5 =\begin{pmatrix}0.8&0.4\\0.4&0.95\end{pmatrix}$. The estimated posterior mean covariance between the two visits is $0.4$ outcome-units squared. This is a posterior expectation of the pattern-specific covariance matrix, not the covariance of the posterior distribution of $\mu$.

Posterior Summaries and Predictions

The posterior mean vector $m_n$ is also the posterior expectation of $\mu$, when the relevant moments exist. Uncertainty about that mean is not represented by a single covariance matrix alone: it depends on both uncertainty about $\Sigma$ and the conditional factor $1/\kappa_n$. Integrating out $\Sigma$ gives a multivariate Student $t$ distribution for $\mu$, with $\nu_n-q+1$ degrees of freedom, location $m_n$, and scale matrix $\Psi_n/\{\kappa_n(\nu_n-q+1)\}$.

Worked Example: A Credible Interval for the Second Visit Mean

Continue with the bivariate hypothetical data above. The marginal posterior for the second component $\mu_2$ is a Student $t$ distribution with $8-2+1=7$ degrees of freedom, location $0.75$ points, and scale squared $4.75/(4\cdot7)=0.1696$ points squared. Its scale is therefore $\sqrt{0.1696}=0.4119$ points.

Using $t_{0.975,7}=2.365$, an equal-tailed 95% posterior credible interval is $0.75\pm2.365(0.4119)=0.75\pm0.974$, or approximately $(-0.224,1.724)$ points. Conditional on this model and prior, after observing these data, the posterior probability that the pattern-specific mean at visit 2 lies in this interval is 0.95. This is a Bayesian posterior probability statement; it is not a claim that there is a 95% probability that a fixed parameter lies in an interval generated by repeated-sampling coverage.

The same posterior also gives a predictive distribution for a new participant's vector from this pattern, provided that the participant is governed by the same pattern-specific distribution. Integrating over both $\mu$ and $\Sigma$, the predictive distribution is multivariate Student $t$, with $\nu_n-q+1$ degrees of freedom, location $m_n$, and scale matrix $$ \frac{\kappa_n+1}{\kappa_n(\nu_n-q+1)}\Psi_n. $$ If $\nu_n-q+1>2$, its covariance matrix is $(\kappa_n+1)\Psi_n/\{\kappa_n(\nu_n-q-1)\}$. A predictive distribution includes variation among individual outcomes as well as uncertainty about model parameters, so it is wider than the posterior distribution for the mean.

Worked Example: Predicting a New Pattern Member

For the same two-visit pattern, the predictive degrees of freedom are 7. The predictive scale matrix is $(4+1)\Psi_n/\{4\cdot7\}=(5/28)\Psi_n$. Its location is $(0,0.75)^\mathsf{T}$, so the predictive distribution is centered at the posterior mean vector.

Since 7 is greater than 2, the predictive covariance is $(5/28)\Psi_n\cdot7/(7-2)=\Psi_n/4 =\begin{pmatrix}1&0.5\\0.5&1.1875\end{pmatrix}$. Thus predictive variances are 1 and 1.1875 outcome-units squared, and the predictive covariance is 0.5 outcome-units squared. These describe a new outcome vector conditional on membership in this pattern and on the fitted model; they are not population-wide summaries across dropout patterns.

Assumptions, Diagnostics, and Alternatives

The conjugate updates are exact for the stated model and prior. Before interpreting them, check that the data structure and assumptions make sense: the components have a consistent visit order, all vectors within the fitted pattern have the same dimension, and observations are independent across participants conditional on the parameters. If the pattern contains repeated records per participant, those records must be represented within that participant's vector rather than treated as independent participants.

Several features can signal that the analysis needs revision. A posterior mean that changes substantially under reasonable alternative priors indicates meaningful prior sensitivity. A very small $n_r$, compared with the dimension $q_r$, means the data may provide weak information about the covariance structure. A scatter matrix with limited rank is not necessarily a problem for a proper inverse-Wishart prior, but it is a warning that covariance estimates may be strongly influenced by that prior. Implausible implied correlations, variances on the wrong measurement scale, or a prior predictive distribution inconsistent with plausible outcomes also call for investigation. These checks assess the model and prior; they do not prove that either is correct.

A Bayesian fit differs from a likelihood-based fit in how uncertainty and prior information enter. A maximum-likelihood estimate uses the likelihood without a prior, while a Bayesian posterior combines likelihood and prior and supports probability statements about parameters conditional on the model. Weighted estimating equations for pattern-specific means target estimating-equation summaries and do not, by themselves, supply a joint posterior distribution for means and covariance matrices. Neither a prior nor a likelihood-based estimate resolves the broader issue that outcomes unobserved within a pattern are not directly learned from that pattern's observed components alone.

A common mistake is to interpret $\mu_r$ as the overall mean merely because it is estimated precisely. It is the mean conditional on pattern $r$, under the fitted model. Another is to interpret an inverse-Wishart scale matrix $\Psi_n$ as the posterior mean covariance; under the convention used here, the expectation is $\Psi_n/(\nu_n-q-1)$, when it exists. Finally, a posterior credible interval is not a frequentist confidence interval with a posterior probability attached after the data are observed. Its probability interpretation is conditional on the specified model and prior.

Key takeaway: Bayesian estimation provides a coherent joint posterior for a pattern-specific mean and covariance when the conditional normal model and prior are appropriate. Always report the pattern, outcome components, prior specification, and posterior summary; do not silently turn a pattern-conditional estimate into a population summary.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

For $q=3$, what condition on $\nu_0$ makes the inverse-Wishart prior proper, and what stronger condition gives a finite prior mean for $\Sigma$?
In the bivariate worked example, why is the posterior scale matrix's lower-right element 4.75 rather than 5.75?
What does a 95% posterior credible interval for $\mu_r$ mean, and what does it not mean?
Why does a posterior predictive covariance exceed the uncertainty about the mean in general?
Can a precise posterior estimate within one dropout pattern automatically be interpreted as an overall mean for all participants? Explain.

Solutions. (1) A proper inverse-Wishart prior requires $\nu_0>q-1=2$. Its covariance expectation is finite when $\nu_0>q+1=4$. (2) The terms are $2+2+0.75=4.75$: the prior scale contributes 2, within-pattern scatter contributes 2, and the mean-discrepancy term contributes $0.75$. (3) Under the specified model and prior, the posterior probability that the parameter lies in the interval is 0.95. It is not a statement that a fixed parameter has a 95% probability of lying in an interval under repeated sampling. (4) A future observation varies both because the model parameters are uncertain and because individual outcomes vary around those parameters. (5) No. It estimates a mean conditional on that pattern; an overall mean requires combining information across patterns under an appropriate marginalization and weighting scheme.