Comparing Shared-Parameter Structures

Why Compare Shared-Parameter Structures?

A shared-parameter model links a longitudinal outcome process and a dropout process through one or more latent subject-specific effects. Candidate specifications can differ in the random effects they include, how those effects enter the dropout hazard, or the strength and form of the outcome-dropout association. These choices can change both model fit and the estimated treatment effect. Comparing structures is therefore part of sensitivity analysis: the aim is to understand which conclusions depend on a particular specification, not to declare one model unquestionably true.

A fair comparison requires a common basis. Candidate models should use the same subjects, observed outcome records, dropout information, treatment coding, and likelihood convention. In particular, compare their observed-data likelihoods, which account for the latent effects by integrating them out. A likelihood conditional on estimated latent effects is not directly comparable to a marginal likelihood for the same data.

Definition: A shared-parameter structure specifies the latent effects in the joint outcome-dropout model and how those effects enter each process. For subject $i$, let $Y_i^{\mathrm{obs}}$ denote the observed outcome history, $D_i$ the observed dropout information, and $b_i$ the latent effect. A model $M$ has observed-data likelihood $$ L_M(\theta_M)=\prod_{i=1}^{n}\int f_M(Y_i^{\mathrm{obs}},D_i\mid b_i;\theta_M)\,g_M(b_i;\theta_M)\,db_i, $$ where $f_M$ describes the joint data conditional on the latent effect, $g_M$ is its distribution, and $\theta_M$ collects the model parameters. The comparison concerns how well each specified model represents the observed data, subject to its assumptions.

Common changes to structure include adding a random slope alongside a random intercept, changing which latent effects enter the dropout model, or allowing a different association between the outcome and dropout processes. Such changes may be scientifically plausible, but they do not all have the same inferential status. Some candidate models are nested, so one is obtained from the other by restrictions on parameters; others are non-nested alternatives. The method of comparison must match that relationship.

Likelihood-Based Criteria

Let $\ell_M$ be the maximized observed-data log likelihood for model $M$, and let $k_M$ be its number of freely estimated parameters. The Akaike information criterion (AIC) and Bayesian information criterion (BIC) are

\mathrm{AIC}_M=-2\ell_M+2k_M,\qquad \mathrm{BIC}_M=-2\ell_M+k_M\log(n).

Both reward improved likelihood and penalize additional parameters; lower values are preferred within the set being compared. AIC uses a penalty of 2 per parameter and is motivated by expected predictive information loss. BIC uses a penalty that increases with sample size and is commonly used as a more stringent complexity penalty. Neither is a probability that a model is true. Their absolute values have little meaning in isolation, and differences are interpretable only when the likelihoods are based on comparable data and conventions. The parameter count must include all estimated components, including latent-effect covariance parameters and dropout-process parameters.

Worked Example: Comparing AIC and BIC

Suppose three candidate structures are fitted to the same $n=240$ subjects using the same observed-data likelihood. Model A has $\ell_A=-812$ and $k_A=7$; model B has $\ell_B=-804$ and $k_B=10$; model C has $\ell_C=-800$ and $k_C=14$. Calculate both criteria.

For model A, $\mathrm{AIC}_A=-2(-812)+2(7)=1624+14=1638$. For B, $\mathrm{AIC}_B=1608+20=1628$. For C, $\mathrm{AIC}_C=1600+28=1628$. AIC therefore favors B and C equally among these three, with A 10 points higher.

Since $\log(240)\approx5.4806$, $\mathrm{BIC}_A=1624+7(5.4806)=1662.4$, $\mathrm{BIC}_B=1608+10(5.4806)=1662.8$, and $\mathrm{BIC}_C=1600+14(5.4806)=1676.7$. BIC slightly favors A over B and penalizes C more strongly. The criteria do not produce a single unambiguous winner: the choice depends partly on the goal and the scientific plausibility of the structures. These numbers do not establish that A, B, or C is the true dropout mechanism.

For nested models, a likelihood-ratio test can assess whether the additional parameters improve fit beyond what sampling variability would typically produce under the simpler model. If model 0 is nested in model 1, the statistic is

G^2=2(\ell_1-\ell_0).

Under the null hypothesis that the restrictions defining model 0 hold, $G^2$ is asymptotically distributed as chi-square with degrees of freedom equal to the number of independent restrictions, provided regularity conditions hold. These include an identifiable parameterization, an interior null value for the tested parameters, and suitable large-sample behavior. The test is not a general-purpose ranking tool for non-nested models.

Worked Example: A Nested-Model Likelihood-Ratio Test

Suppose model A above is nested in model B, which adds three ordinary association coefficients. Assume the null values of those coefficients are interior points of their parameter space, the models are identifiable, and the likelihood maximization is reliable. Test $H_0$, that all three added coefficients are zero, against $H_a$, that at least one is nonzero, at $\alpha=0.05$.

The likelihood-ratio statistic is $G^2=2[-804-(-812)]=2(8)=16$, with $df=3$. Under $H_0$, the reference distribution is approximately chi-square with three degrees of freedom, giving $p\approx0.00113$. Since $p<0.05$, the data provide evidence against the restrictions in $H_0$ for this comparison. This result supports the added coefficients under the stated model and regularity assumptions; it does not prove that model B is correctly specified or that its dropout assumptions are true.

If the added parameter were a variance constrained to be nonnegative, the null value zero would lie on the boundary. The usual chi-square reference distribution need not apply. A boundary-adjusted reference distribution or a suitable parametric bootstrap may be needed instead. Likewise, a likelihood-ratio test should not be used to compare models that are not nested.

Predictive Checks for the Joint Model

Likelihood criteria summarize fit with a single penalized number. Predictive checks ask a more targeted question: can a fitted model reproduce features of the observed outcomes and dropout process that matter scientifically? A check begins by defining a summary before interpreting the result. Examples include dropout rates across groups defined by prior observed outcome history, distributions of observed outcome changes, or dropout timing. The fitted model is then used to simulate replicated data under its assumptions, and the observed summary is compared with the simulated distribution.

For a maximum-likelihood fit, one practical approach is a parametric bootstrap: simulate data from the fitted model, refit or otherwise account for parameter uncertainty as appropriate, and form a reference distribution for the chosen summary. A Bayesian posterior predictive check uses draws from the posterior predictive distribution instead. In either case, the check is diagnostic rather than a formal proof of correctness. Checks should reflect the joint structure: assessing outcome trajectories alone can miss a poorly represented dropout process.

Worked Example: Checking Dropout Patterns by Prior Outcome

In a hypothetical study of 200 participants, divide subjects into four equal-sized groups according to their most recent observed outcome. The observed numbers who drop out during the next interval are 4, 8, 16, and 23 out of 50 in the four groups, corresponding to rates of $4/50=0.080$, $8/50=0.160$, $16/50=0.320$, and $23/50=0.460$. A fitted model is used to generate 1,000 replicated data sets.

Suppose the model's simulated 95% predictive interval for the dropout count in the highest-outcome group is 14 to 22, while the observed count is 23. The observed count lies just above that interval. If the other three observed group counts lie inside their respective predictive intervals, the main discrepancy is a possible underprediction of dropout among participants with the highest prior outcomes. This is a reason to investigate the shared-effect structure or other model features, not a stand-alone rejection rule. The interval is simulation-based under the fitted model; it does not cover uncertainty about every possible model specification.

A second plausible structure might yield a 95% predictive interval of 15 to 30 for that highest group and include the observed count. That is evidence that the second structure better reproduces this particular feature, but it does not establish that its latent mechanism is correct. The check is most informative when the grouping, summary, and practical concern are specified clearly rather than chosen only after a surprising result appears.

Observed-data prediction has an important limit in dropout problems: outcomes after dropout are not observed. A model can reproduce observed dropout times and available outcomes while still making different, unverifiable assumptions about the missing outcomes. Predictive checks can reveal conflicts with observed data, but cannot by themselves confirm the unobserved outcome distribution or identify the true shared-effect association. This distinction is central to sensitivity analysis.

Stability and Interpretation of Treatment Effects

Model selection should be connected to the scientific estimand. For each defensible structure, report the treatment-effect estimate, its standard error or interval, and its units. Compare the direction, magnitude, uncertainty, and practical interpretation across models. If conclusions change materially when a plausible shared-effect structure changes, that instability is itself an important result: it indicates that the treatment conclusion relies on assumptions about the joint outcome-dropout process.

Worked Example: Assessing Treatment-Effect Stability

Suppose the treatment effect is defined as the adjusted difference in mean outcome between treatment and control at a specified follow-up, in outcome points, with negative values favoring treatment. Three plausible shared-parameter structures give the following hypothetical estimates: model A, $-2.4$ with $SE=0.90$; model B, $-2.1$ with $SE=0.85$; model C, $-1.2$ with $SE=0.95$. Use a normal approximation to form 95% intervals as estimate $\pm1.96SE$.

For A, $-2.4\pm1.96(0.90)=-2.4\pm1.764$, giving $(-4.16,-0.64)$ points. For B, $-2.1\pm1.96(0.85)=-2.1\pm1.666$, giving $(-3.77,-0.43)$ points. For C, $-1.2\pm1.96(0.95)=-1.2\pm1.862$, giving $(-3.06,0.66)$ points.

All three point estimates favor treatment, but the estimated benefit ranges from 1.2 to 2.4 points, and model C's interval includes zero. Thus, the direction of the point estimate is stable while the estimated magnitude and evidence of a nonzero effect are not fully stable. These intervals describe uncertainty under each fitted model and the normal approximation; they do not account for uncertainty over which shared-parameter structure is appropriate.

Practical Comparison and Failure Modes

A sound comparison combines evidence rather than letting one numerical criterion decide. AIC or BIC can summarize relative fit and complexity; a likelihood-ratio test can address a specific nested restriction when its assumptions hold; predictive checks can expose mismatches in features that matter; and treatment-effect stability shows whether the substantive conclusion depends on structure. Interpretability also matters. A modest improvement in fit may not justify a difficult-to-explain association structure, while a structure with a clear scientific rationale may deserve attention even if it is not the numerical winner.

Before interpreting a comparison, verify that each fit converged to a credible maximum, that numerical integration or approximation is adequate, and that estimated variances and correlations are not signaling an unstable or boundary solution. A failed or questionable fit cannot be rescued by a favorable AIC. Also verify that the models use the same observed records and likelihood definition. If one model is fit to a different set of participants or a different outcome record, its information criteria do not provide a clean comparison.

Cross-validation can provide another predictive comparison, but it must respect the subject-level data structure: all records from one participant should remain together in a fold. Splitting repeated records from the same participant between training and validation sets can leak information about that participant's latent effect and exaggerate predictive performance. Predictive scoring must also be defined consistently for incomplete records; comparing scores that condition on different information can mislead.

Define the candidate structures.
State precisely which latent effects or association terms differ, and identify whether the models are nested.

Check comparability and fit quality.
Use the same observed data and likelihood convention; investigate convergence, boundary estimates, and numerical accuracy.

Use criteria matched to the question.
Compare AIC or BIC; use a likelihood-ratio test only for a suitable nested comparison with a valid reference distribution.

Inspect predictions and conclusions.
Check relevant replicated-data summaries and compare treatment-effect estimates, uncertainty, and interpretation across plausible structures.

A common mistake is to select the model with the smallest AIC and then report its treatment effect as though model uncertainty had disappeared. The reasoning is wrong because AIC is a relative criterion over the candidates considered; it neither certifies the selected structure nor incorporates uncertainty over all plausible structures into the reported interval. The correction is to report the comparison transparently, retain scientifically plausible alternatives, and describe how the treatment conclusion varies across them.

Key takeaway: Comparing shared-parameter structures is a sensitivity analysis, not a contest with a guaranteed winner. Evaluate comparable likelihoods, use nested-model tests only under their conditions, inspect predictions for meaningful discrepancies, and report whether the treatment-effect conclusion is stable across defensible assumptions.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

Two models use the same observed-data likelihood and data. Their maximized log likelihoods are $-500$ and $-494$, and the second model has two additional parameters. Calculate the likelihood-ratio statistic and AIC difference, defining the difference as second minus first.
Why is the usual chi-square likelihood-ratio test potentially invalid when the added parameter is a variance constrained to be nonnegative?
A model reproduces the observed dropout rates well but its treatment-effect estimate differs substantially from a plausible alternative structure. What should be reported?
Why can a predictive check not confirm the distribution of outcomes after dropout?

Solutions. (1) The likelihood-ratio statistic is $2[-494-(-500)]=12$. The AIC difference is $[-2(-494)+2(k+2)]-[-2(-500)+2k]=-12+4=-8$, so the second model has the lower AIC by 8 points. (2) Under the null, the variance equals zero, which is on the parameter-space boundary rather than in its interior; the standard chi-square approximation may not hold, so an appropriate boundary result or parametric bootstrap may be needed. (3) Report the estimate and uncertainty under both structures and explain that the treatment conclusion is sensitive to the shared-effect specification; good fit to one observed feature does not settle the structural choice. (4) Those outcomes are unobserved, so the check can compare simulations with observed outcomes and dropout information but cannot directly compare simulated post-dropout outcomes with data.