Canonical Model Assumptions and Diagnostics

Why Check the Canonical Model?

Group sequential calculations rely on a joint model for statistics at several analysis looks. The canonical model represents the score accumulated by each look as a sum of stage increments whose variances are measured by information. Its usefulness depends on more than the marginal distribution at each look: the increments must also have the appropriate means, variances, and dependence structure. A statistic can look approximately standard normal at every look while the joint model is still wrong.

The key diagnostic objects are therefore the score increments, not just the cumulative \(Z\) statistics. The earlier material on score statistics and asymptotically independent score increments motivates this representation. Here the emphasis is on the assumptions it requires and on what simulated or observed increments can tell us about them.

This is a model for the joint behavior of the sequential statistics, not a claim that every finite-sample score is exactly normal. It is commonly justified as an approximation in regular models. The information increments must be positive, and the score and information must be calculated under a correctly specified likelihood or a suitable valid approximation. The stages should contribute non-overlapping data increments; otherwise, the score increments can share data and be dependent. The stated distribution also assumes that the information scale is correctly calibrated.

The compact form above treats the information increments as fixed at the analysis schedule. If information is random or the schedule changes in response to accumulating data, an unconditional fixed-schedule canonical model does not follow automatically. A suitable conditional model may still be possible, but it needs its own justification. Similarly, nuisance-parameter estimation, model misspecification, clustering, or temporal dependence can affect the score variance or the dependence among increments.

What the Increment Assumptions Imply

Under the model, cumulative scores are sums of increments. Thus \(U_k=\sum_{j=1}^k\Delta U_j\), and cumulative information is \(I_k=\sum_{j=1}^k\Delta I_j\). Independence gives a direct way to derive the covariance structure of cumulative statistics.

This derivation also clarifies what to inspect. If standardized increments show serial dependence, unusual variance, or a systematic change in their mean, the assumed joint distribution may not describe the data well. Checking only the marginal mean and variance of each cumulative \(Z_k\) can miss such departures because cumulative statistics combine information from earlier stages.

Diagnostics for Simulated and Observed Increments

A diagnostic starts by putting increments on a comparable scale. Under \(H_0\), calculate \(Q_k=\Delta U_k/\sqrt{\Delta I_k}\). Under a specified alternative, the model gives \(E(Q_k)=\theta\sqrt{\Delta I_k}\), so a residual-style quantity is \(Q_k-\theta\sqrt{\Delta I_k}\). Examine whether these quantities are centered as expected, have roughly unit variance across repeated realizations, and are not systematically dependent across stages. Plots against look number, information increment, calendar time, and relevant design features can expose patterns that a single summary conceals.

Simulation is especially useful because many independent replicate trials can be generated under a specified model and analysis plan. For each replicate, calculate all stage score and information increments using the same method intended for the analysis. Compare empirical means, variances, correlations, and distributional shapes with their canonical targets. Observed data offer only one path through the sequence, so they can suggest a problem but provide much weaker evidence about repeated-sampling behavior.

For observed data, the same calculations can be informative, but interpretation requires care. Under a nonzero effect, standardized increments need not be centered at zero. If the purpose is to inspect departures from a specified alternative, compare them with the model’s expected drift as well as its variance. Substituting an effect estimate computed from the same increments can induce dependence among the resulting residuals; such residuals are exploratory, not independent null draws.

Using Replicate Simulations to Diagnose Departures

A stronger diagnostic compares the empirical behavior of increments across many simulated replicates. For each stage, estimate the mean and variance of \(Q_k\) across replicates, and estimate correlations between different stages. Under the null canonical model, the target means are zero, variances are one, and correlations between distinct increments are zero. These are reference values, not exact finite-simulation requirements: Monte Carlo estimates fluctuate, especially when the number of replicates is small.

The following hypothetical results illustrate how to interpret a discrepancy. Assume 5,000 simulated null trials with three planned increments. The empirical variance of the first standardized increment is 1.44, its empirical correlation with the second is 0.38, and the corresponding correlation between the second and third is 0.02. A variance of 1.44 suggests that the first-stage score may be more variable than its stated information predicts. The correlation of 0.38 suggests dependence between the first two increments. The near-zero later correlation is consistent with independence for that pair, but does not cancel the first discrepancy.

Comparison, Failure Modes, and Limits

Increment diagnostics complement, rather than replace, checks of the cumulative covariance matrix. A covariance check asks whether the joint covariance of the \(Z\) statistics matches the canonical pattern. Increment checks help localize why it may not: dependence between adjacent increments, incorrect variance at one stage, or a mean pattern inconsistent with the reference effect. Comparing the two views is useful because cumulative statistics can hide where a departure entered the sequence.

Simulation and observed-data diagnostics have different strengths. Simulation can assess repeated-sampling behavior under an explicitly specified model and schedule, but it cannot reveal misspecification that was omitted from the simulation. Observed increments reflect the actual data path, but one path cannot reliably estimate repeated-sampling variances or correlations. Neither approach turns model checking into proof that assumptions hold.

Several practical departures deserve attention:

• Overlapping or reused data: stage increments may share observations, creating dependence not represented by the model.
• Information miscalibration: an incorrect variance model can make standardized increments too variable or too concentrated.
• Unmodeled heterogeneity or clustering: dependence among individuals can alter both the score variance and the cross-stage relationship.
• Changing data collection or analysis: a shift in measurement, eligibility, or score calculation can create a stage pattern.
• Adaptive timing or random information: a fixed-schedule joint distribution may not apply without an appropriate conditional argument.

A common mistake is to say, “Every cumulative \(Z_k\) looks approximately standard normal, so the canonical model is validated.” That reasoning checks only marginal behavior. The correction is to inspect the joint structure, especially the score increments: the canonical model requires the specified variance and mean at each increment and independence across increments, not merely standard-normal-looking cumulative statistics.

Diagnostics are also limited by the number of stages and replicates, the quality of the score and information calculations, and the sensitivity of the chosen summaries. A small correlation estimate does not prove independence, and a large one may be unstable if based on few replicates. Graphs and numerical summaries should be interpreted alongside knowledge of the data-generating process and analysis implementation. If a material departure remains, the response is to investigate and justify a more suitable model or analysis, not to assume that the canonical approximation remains valid unchanged.