Canonical Statistics and Information Fractions

Why Information Fractions Matter

In a group sequential design, an interim Z statistic is not an independent new test. It summarizes accumulating evidence, so its value is related to the statistics at earlier looks. To evaluate the probability of crossing one or more efficacy boundaries under the null, we need both the timing of each look in terms of information and the joint distribution of the interim statistics. The canonical model gives a convenient representation of that dependence.

The information fraction describes how much of the planned final information has accrued at a particular look. It is not automatically the fraction of participants enrolled: that equality requires information to accumulate proportionally to sample size. The distinction matters when allocation, outcome variability, follow-up, or analysis precision changes over time.

Canonical Statistics and Their Assumptions

Definition: Let \(I_k\) be the cumulative statistical information at look \(k\), with \(0<I_1<\cdots<I_K\). In the canonical model, the cumulative score \(S_k\) has independent increments. For each interval, \(S_k-S_{k-1}\) is Gaussian with mean \(\psi(I_k-I_{k-1})\) and variance \(I_k-I_{k-1}\), where \(S_0=I_0=0\) and \(\psi\) is the effect parameter on the information scale. The interim statistic is \(Z_k=S_k/\sqrt{I_k}\).

This is an exact model when the score increments are Gaussian with the stated means and variances. More generally, the same canonical joint normal distribution may be used as an approximation when a suitable large-sample normal approximation applies. Independent increments alone are not enough to imply normality. If increments are independent and have the stated means and variances but are not Gaussian, the mean, variance, and covariance calculations below still follow, but the joint distribution need not be multivariate normal.

For a one-parameter normal-mean illustration, suppose the cumulative estimate \(\hat{\theta}_k\) has variance \(1/I_k\), and under the model its mean is \(\theta\). Then \(Z_k=\sqrt{I_k}\hat{\theta}_k\), and \(\psi=\theta\). In other settings, \(I_k\) can be the information for an adjusted treatment-effect estimate. The scale and units of \(I_k\) depend on the parameterization; the resulting Z statistic is dimensionless.

Theorem: Under the Gaussian independent-increments assumptions above, \((Z_1,\ldots,Z_K)\) is multivariate normal. For every look \(k\), \(E_\psi(Z_k)=\psi\sqrt{I_k}\) and \(\operatorname{Var}_\psi(Z_k)=1\). For \(i<j\), \(\operatorname{Cov}_\psi(Z_i,Z_j)=\operatorname{Corr}_\psi(Z_i,Z_j)=\sqrt{I_i/I_j}\).

Deriving the Dependence Across Looks

The joint distribution can be derived directly from the cumulative scores. Since \(S_k\) is the sum of independent increments up to look \(k\), its mean is \(\psi I_k\) and its variance is \(I_k\). For \(i<j\), write \(S_j=S_i+(S_j-S_i)\). The later increment is independent of \(S_i\) and has mean \(\psi(I_j-I_i)\) and variance \(I_j-I_i\).

E_\psi(Z_k)=\frac{E_\psi(S_k)}{\sqrt{I_k}} =\frac{\psi I_k}{\sqrt{I_k}} =\psi\sqrt{I_k}, \qquad \operatorname{Var}_\psi(Z_k)=\frac{I_k}{I_k}=1.

Independence of the later increment gives \(\operatorname{Cov}_\psi(S_i,S_j)=\operatorname{Var}_\psi(S_i)=I_i\). Dividing by the standardizing factors yields the covariance and correlation of the Z statistics:

\operatorname{Cov}_\psi(Z_i,Z_j) =\frac{\operatorname{Cov}_\psi(S_i,S_j)}{\sqrt{I_iI_j}} =\frac{I_i}{\sqrt{I_iI_j}} =\sqrt{\frac{I_i}{I_j}}.

The same covariance formula holds whether or not the increments are Gaussian, provided the independent-increments means and variances just stated hold. Gaussian increments add the further conclusion that the vector of Z statistics is multivariate normal. This qualification is important: a covariance matrix does not, by itself, determine a general nonnormal joint distribution.

Worked Example: Building a Three-Look Correlation Matrix

Suppose a hypothetical trial has planned cumulative information \(I_1=100\), \(I_2=225\), and \(I_3=400\). Assume Gaussian independent increments with variance equal to each information increment. Find the null correlations among the three interim Z statistics.

The information increments are \(100\), \(225-100=125\), and \(400-225=175\), all positive. Under \(H_0:\psi=0\), the means of all three Z statistics are zero, and each has variance one. The correlations are:

\rho_{12}=\sqrt{\frac{100}{225}}=0.6667,\qquad \rho_{13}=\sqrt{\frac{100}{400}}=0.5000,\qquad \rho_{23}=\sqrt{\frac{225}{400}}=0.7500.

Thus the null correlation matrix is

\begin{pmatrix} 1 & 0.6667 & 0.5000\\ 0.6667 & 1 & 0.7500\\ 0.5000 & 0.7500 & 1 \end{pmatrix}.

The later statistics are positively correlated with the earlier ones because they include the earlier accumulated evidence. Under the assumed Gaussian increment model, this matrix, together with the zero means, specifies the joint null distribution. If the increments were only known to be independent and to have the specified variances, the matrix would still describe their correlations, but it would not establish a joint normal distribution.

Information Fractions

Definition: The information fraction at look \(k\) is \(t_k=I_k/I_K\), the cumulative information at that look divided by the planned final information. Therefore \(0<t_1<\cdots<t_K=1\), and the canonical correlation can also be written as \(\operatorname{Corr}(Z_i,Z_j)=\sqrt{t_i/t_j}\) for \(i<j\).

Information is a measure of precision, not simply a count of observations. In a setting where \(I_k=1/\operatorname{Var}(\hat{\theta}_k)\), a more precise estimate has greater information. If the standard error is known or specified by the design, the corresponding information is \(1/SE^2\). In practice, the planned or appropriately updated information should be used consistently with the design’s statistical model.

Worked Example: Converting Standard Errors to Information Fractions

A hypothetical study plans three analyses with standard errors \(0.20\), \(0.14\), and \(0.10\) for the same treatment-effect estimate, on the same scale. Assume these are the planned standard errors under a model in which information is the inverse variance. Calculate the information fractions.

The information at each look is \(I_k=1/SE_k^2\). Thus \(I_1=1/(0.20)^2=25.00\), \(I_2=1/(0.14)^2=51.02\), and \(I_3=1/(0.10)^2=100.0\). The final planned information is \(100.0\), so:

t_1=\frac{25.00}{100.0}=0.2500,\qquad t_2=\frac{51.02}{100.0}=0.5102,\qquad t_3=\frac{100.0}{100.0}=1.

The second look has accumulated about 51% of the planned information, not exactly half. If the standard errors had been proportional to the inverse square root of sample size, these fractions could match sample size fractions. That proportionality should be checked, not assumed.

Information and Cumulative Alpha

For upper efficacy boundaries \(c_1,\ldots,c_K\), the cumulative null crossing probability through look \(k\) is the probability of crossing at that or an earlier look:

A(t_k)=P_0\!\left(\bigcup_{j=1}^{k}\{Z_j\ge c_j\}\right).

Here \(P_0\) denotes probability under \(H_0\). The incremental alpha spent between looks \(k-1\) and \(k\) is \(A(t_k)-A(t_{k-1})\), with \(A(t_0)=0\). Under the canonical model, calculating this probability requires the joint normal distribution with correlations determined by the information fractions. It is generally not correct to add the marginal tail probabilities \(P_0(Z_j\ge c_j)\), because crossing events at different looks are dependent.

A spending function specifies a target cumulative amount \(A(t)\) as a function of information fraction. It describes how much null crossing probability should have been used by a given information time. The boundary values are then assessed or calibrated so that the joint crossing probability agrees with those targets. The spending function itself does not make the interim Z statistics independent, and an information fraction alone does not determine a boundary.

Worked Example: Cumulative Crossing Probability at Two Looks

Consider a hypothetical two-look design under \(H_0\), with information fractions \(t_1=0.40\) and \(t_2=1.00\), and upper boundaries \(c_1=2.50\) and \(c_2=2.00\). Assume the canonical Gaussian model. Calculate the cumulative null crossing probability through the second look.

The correlation is \(\rho_{12}=\sqrt{0.40/1.00}=0.6325\). The probability of crossing by look 2 is the probability of crossing at look 1 plus the probability of crossing at look 2 without an earlier crossing:

P_0(Z_1\ge2.50)+P_0(Z_1<2.50,\ Z_2\ge2.00).

The first term is the standard normal upper tail \(1-\Phi(2.50)\), approximately \(0.00621\). To evaluate the second term, use the conditional normal distribution \(Z_2\mid Z_1=z\sim N(\rho_{12}z,1-\rho_{12}^2)\). Therefore:

\int_{-\infty}^{2.50} \phi(z)\left[ 1-\Phi\!\left( \frac{2.00-0.6325z}{\sqrt{1-0.6325^2}} \right) \right]\,dz \ \approx\ 0.0203.

Adding the two disjoint contributions gives a cumulative null crossing probability of approximately \(0.00621+0.0203=0.0265\), or 2.65%. This is the design’s cumulative type I error through the second look for these specified boundaries under the stated model. It is not the sum of the two marginal tail probabilities, because some paths that cross the second boundary would already have crossed the first.

Diagnostics, Alternatives, and Limitations

Before relying on canonical calculations, check that the information values refer to the same parameter and analysis scale at every look; that cumulative information is increasing; and that the planned information at the final look is the denominator used for every fraction. Also check that the increment assumptions are credible. In particular, the model requires independent increments with the specified variance structure, and a multivariate normal calculation additionally requires Gaussian increments or an adequate normal approximation.

Potential warning signs include information fractions that decrease across looks, correlations calculated from a different schedule than the actual analysis schedule, or standard errors that change for reasons not represented in the information update. If the observed-data analysis departs materially from the prespecified model, the canonical joint distribution may no longer describe the null path probabilities accurately. A normal approximation may also be poor with sparse data, highly discrete outcomes, or other settings where score distributions remain strongly nonnormal.

The main alternative is to model the joint distribution directly from the trial’s detailed data-generating and analysis model, rather than use the canonical approximation. That can better represent complicated outcomes or changing variance structures, but may require more modeling and computation. Treating interim tests as independent is simpler, but is generally inappropriate for accumulating evidence: it ignores the shared data and miscalculates the probability of crossing at least one boundary. Canonical calculations are useful precisely because they retain the principal dependence in a compact form, while their validity remains conditional on the stated assumptions.

Common pitfall: “The Z statistics have the canonical correlations, so they must be jointly normal.” This conclusion is too strong. Independent increments with the specified means and variances establish the stated means, variances, and covariances; they do not establish normality. To use multivariate normal crossing probabilities, verify Gaussian increments or justify a suitable normal approximation.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

At two looks, \(I_1=64\) and \(I_2=256\). Under the Gaussian canonical model, what is the correlation between \(Z_1\) and \(Z_2\)?
A planned estimate has standard errors \(0.25\) at an interim look and \(0.125\) at the final look. Under the inverse-variance information definition, what is the interim information fraction?
Why is the probability of crossing at either of two looks not generally the sum of the two marginal crossing probabilities?
What follows from independent increments with the stated means and variances if the increments are not Gaussian, and what does not follow?

Solutions. (1) \(\operatorname{Corr}(Z_1,Z_2)=\sqrt{64/256}=\sqrt{0.25}=0.5000\). (2) Interim information is \(1/(0.25)^2=16\); final information is \(1/(0.125)^2=64\). The fraction is \(16/64=0.2500\). (3) The crossing events are dependent because later statistics include earlier accumulated evidence. Adding marginal probabilities counts paths that cross at both looks more than once. (4) The specified means, variances, and covariance formula follow from the increment assumptions. Joint multivariate normality does not follow without Gaussian increments or an adequate normal approximation.