Variance Estimation and the Chi-Square Reference

Why the Covariance Matters

Gray’s test compares cumulative incidence for a specified event type across groups. Its score summarizes departures from the null hypothesis, while its covariance describes the variability and dependence of the score components under that null. The covariance is not an optional adjustment: it sets the scale of the score and determines the reference distribution used to assess evidence against the null.

The earlier material on Gray’s score and weight functions establishes how the score is constructed and how its time weights affect the comparison. Here the focus is what happens after that score has been calculated: how to estimate its covariance, why the resulting quadratic form is asymptotically chi-square, and when that approximation needs caution.

Definition: Let $U_n$ be a $d$-component Gray score vector for $n$ independent subjects, evaluated under the null hypothesis that the target-event cumulative incidence functions are equal across the groups being compared. Let $V_n$ estimate the covariance matrix of $U_n$ under that null. If $V_n$ is nonsingular, the score quadratic form is $$ Q_n = U_n^{\mathsf T} V_n^{-1} U_n. $$ The number of score components, $d$, is the number of linearly independent contrasts represented by the vector.

The notation above describes the general vector calculation. In a two-group comparison there is one independent score component, so $U_n$ and $V_n$ are scalars and $Q_n=U_n^2/V_n$. In higher dimensions, the off-diagonal entries of $V_n$ matter: they represent covariance between score components. Treating them as zero without justification changes the test.

Assumptions Behind the Covariance and Reference

The covariance estimate and chi-square approximation rely on a large-sample argument. In broad terms, the score must behave like a sum of subject-level contributions with mean zero under the null, and the estimated covariance must converge to the covariance of that score. A standard formulation is

U_n = \sum_{i=1}^{n} \xi_i + o_p(\sqrt{n}), \qquad \frac{V_n}{n} \overset{p}{\longrightarrow} \Sigma, \qquad \Sigma \text{ is positive definite}.

Here $\xi_i$ is the influence contribution of subject $i$ to the complete score procedure, and $\Sigma$ is the limiting covariance per subject. The notation $o_p(\sqrt{n})$ means that the remainder, divided by $\sqrt{n}$, converges in probability to zero. The contributions need not be directly observed: they are estimated from the data.

The usual justification also requires independent subjects, a correctly specified null comparison, adequate follow-up and censoring conditions for the Gray procedure, and enough observations and events for its large-sample approximation to be credible. Censoring must be noninformative in the sense required by the procedure—for example, independent within group, or independent conditional on variables incorporated in the analysis—and the probability of remaining uncensored must not collapse over the time region contributing to the score. The chosen weights and any estimated quantities used in the score must be sufficiently regular for the influence representation to apply. These are conditions for asymptotic validity, not guarantees that a small data set will yield a good approximation.

In particular, $\xi_i$ must reflect the whole score calculation. Gray’s procedure uses estimated quantities such as risk-set information and, depending on its formulation, weights or other nuisance estimates. Their estimation contributes to score variability. A variance formed only from the apparent target-event increments can omit this contribution and need not estimate the correct covariance. Use the covariance calculation paired with the chosen Gray score implementation, or recompute the entire procedure when estimating its influence contributions.

Estimating Covariance From Influence Contributions

If the score has an asymptotically linear representation, estimate each $\xi_i$ by $\hat{\xi}_i$, recomputing or accounting for all estimated quantities in the score. Center the estimated contributions to remove their finite-sample mean, and use their outer products:

\bar{\xi} = \frac{1}{n}\sum_{i=1}^{n}\hat{\xi}_i, \qquad \hat{V}_n = \sum_{i=1}^{n} (\hat{\xi}_i-\bar{\xi})(\hat{\xi}_i-\bar{\xi})^{\mathsf T}.

This estimates the covariance of the unnormalized score $U_n$, which is typically of order $n$. If software instead reports a covariance for $U_n/\sqrt{n}$, that covariance is typically of order one. The score and covariance must use the same scaling: multiplying the score by a constant requires multiplying its covariance by the square of that constant. The resulting quadratic form is unchanged if both are scaled consistently.

A practical alternative for a regular score is a delete-one jackknife. For each subject $i$, remove that subject and recompute the entire score, including the risk sets, weights, and estimated nuisance quantities. Call the result $U_{(-i)}$, and let $\bar{U}_{(-.)}=n^{-1}\sum_i U_{(-i)}$. The jackknife covariance estimate is

\hat{V}_{J} = \frac{n-1}{n}\sum_{i=1}^{n} \bigl(U_{(-i)}-\bar{U}_{(-.)}\bigr) \bigl(U_{(-i)}-\bar{U}_{(-.)}\bigr)^{\mathsf T}.

This is a data-based approximation to the variability induced by subject-level changes in the complete score. It is not a license to delete a subject from only the final sum while freezing all estimated parts. For a score with nonregular behavior, or an implementation for which the delete-one calculation is unstable, the jackknife may not be reliable; use a justified covariance method suited to that procedure.

Worked Example: Building a Covariance Matrix

Suppose a hypothetical analysis has two score components and four estimated, centered influence contributions from the complete Gray score procedure: $\hat{\xi}_1=(1,0)^{\mathsf T}$, $\hat{\xi}_2=(-1,0)^{\mathsf T}$, $\hat{\xi}_3=(0,1)^{\mathsf T}$, and $\hat{\xi}_4=(0,-1)^{\mathsf T}$. These simple values illustrate the calculation; four subjects would not ordinarily be enough to rely on an asymptotic test.

The contributions sum to zero, so their mean is zero. The covariance estimate is the sum of their outer products:

\hat{V}_n = \begin{pmatrix}1&0\\0&0\end{pmatrix} + \begin{pmatrix}1&0\\0&0\end{pmatrix} + \begin{pmatrix}0&0\\0&1\end{pmatrix} + \begin{pmatrix}0&0\\0&1\end{pmatrix} = \begin{pmatrix}2&0\\0&2\end{pmatrix}.

The diagonal entries estimate the variances of the two score components, and the off-diagonal entries estimate their covariance. Here the estimated covariance is zero because the specified contributions cancel across components; this should not be assumed in a different data set.

Worked Example: Delete-One Covariance for a Scalar Score

In a hypothetical two-group analysis, suppose the score recomputed after deleting each of four subjects is $5, 3, 6,$ and $2$. The mean delete-one score is $\bar{U}_{(-.)}=(5+3+6+2)/4=4$. The squared deviations from this mean sum to $(5-4)^2+(3-4)^2+(6-4)^2+(2-4)^2=1+1+4+4=10$.

The jackknife covariance estimate, now a variance, is $\hat{V}_{J}=(4-1)10/4=7.5$. This calculation only makes sense if each delete-one score was obtained by rerunning the full score procedure. The small hypothetical sample is for arithmetic illustration, not a recommendation to use the chi-square approximation with four subjects.

Why the Quadratic Form Is Chi-Square

Under the null and the regularity conditions above, a multivariate central limit theorem gives $U_n/\sqrt{n}$ an asymptotic normal distribution with mean zero and covariance $\Sigma$. If $V_n/n$ consistently estimates $\Sigma$, then the quadratic form uses an estimated version of the covariance to standardize the score. The justification is:

Obtain the score limit.
Under the null, $U_n/\sqrt{n}$ converges in distribution to a mean-zero multivariate normal vector $Z$ with covariance $\Sigma$.

Standardize the vector.
If $\Sigma$ is positive definite, then $Z^{\mathsf T}\Sigma^{-1}Z$ is the sum of squares of $d$ independent standard normal variables. By definition, that sum has a chi-square distribution with $d$ degrees of freedom.

Replace the limiting covariance.
Since $V_n/n$ converges in probability to $\Sigma$, the continuous mapping and Slutsky results allow the estimated covariance to replace $\Sigma$ in the quadratic form. Thus $Q_n$ converges in distribution to $\chi^2_d$.

The scaling cancels explicitly:

Q_n = \left(\frac{U_n}{\sqrt{n}}\right)^{\mathsf T} \left(\frac{V_n}{n}\right)^{-1} \left(\frac{U_n}{\sqrt{n}}\right) \ \overset{d}{\longrightarrow}\ \chi^2_d.

This is an asymptotic reference distribution under the null, not an exact finite-sample distribution. The p-value is the upper-tail probability of a chi-square variable with the appropriate degrees of freedom at least as large as the observed $Q_n$. A small p-value indicates that the observed score is relatively far from zero after accounting for its estimated covariance; it does not give the probability that the null is true.

Worked Example: A Scalar Score and Its Chi-Square Reference

Suppose a hypothetical two-group Gray comparison gives scalar score $U_n=3.20$ and estimated variance $V_n=2.56$. The groups are independent, the analysis uses a prespecified regular weight, and the full procedure’s covariance estimate is valid. Assume also that the sample is large enough for the asymptotic approximation to be credible and that the variance is positive.

The quadratic form is $Q_n=U_n^2/V_n=3.20^2/2.56=10.24/2.56=4.00$. There is one independent score component, so the reference distribution is $\chi^2_1$. The upper-tail p-value is $P(\chi^2_1\geq4.00)\approx0.0455$. At a 5% significance level, this is just below 0.05, providing evidence against equal target-event cumulative incidence under the stated test assumptions. It does not establish a causal effect or quantify the size of a risk difference.

Worked Example: A Vector Score With Correlated Components

For a hypothetical illustration of the matrix calculation, suppose a score vector is $U_n=(2,1)^{\mathsf T}$, and its estimated covariance is $\hat{V}_n=\begin{pmatrix}4&1\\1&2\end{pmatrix}$. The covariance matrix is positive definite because its determinant is $4(2)-1(1)=7>0$, and its first diagonal entry is positive.

The inverse is $\hat{V}_n^{-1}=\frac{1}{7}\begin{pmatrix}2&-1\\-1&4\end{pmatrix}$. Therefore,

Q_n = \begin{pmatrix}2&1\end{pmatrix} \frac{1}{7} \begin{pmatrix}2&-1\\-1&4\end{pmatrix} \begin{pmatrix}2\\1\end{pmatrix} = \frac{1}{7} \begin{pmatrix}2&1\end{pmatrix} \begin{pmatrix}3\\2\end{pmatrix} = \frac{8}{7} \approx 1.143.

If these components represent two independent score contrasts and the asymptotic conditions hold, the reference is $\chi^2_2$. The upper-tail probability is $\exp(-Q_n/2)=\exp(-4/7)\approx0.565$. This example shows why the off-diagonal covariance matters: the calculation uses the full matrix, not separate variance divisions.

Diagnostics, Failure Modes, and Alternatives

Before using the chi-square reference, inspect the covariance estimate and the score calculation that produced it. Useful checks include:

Positive variance and covariance rank. A variance estimate should not be negative, and the covariance matrix should have the rank expected from the independent score components. A zero or nearly zero eigenvalue may indicate redundant contrasts, sparse information, or a numerical problem.
Reasonable scaling. Confirm that the reported covariance is for the same version of the score—summed, averaged, or scaled by $\sqrt{n}$—used in the quadratic form. A mismatch can make the test statistic wrong by a large factor.
Recomputation of estimated quantities. With a delete-one or perturbation method, verify that each replicate recomputes every estimated element required by the Gray score. Holding such elements fixed can understate uncertainty.
Influential observations and instability. If a few delete-one scores differ sharply from the rest, the covariance may be dominated by a few subjects. Sparse target events, heavy censoring, or little information in part of the weighted time range can also make large-sample calibration unreliable.

If $V_n$ is singular because some score components are exactly redundant, reduce the score to a set of independent contrasts and use the rank of that reduced covariance as the degrees of freedom. A generalized inverse may be appropriate only when the estimable subspace and its rank are justified; it should not be used to conceal a singularity caused by inadequate information. If the covariance is unstable or the sample is small, a routine chi-square p-value may be poorly calibrated. A suitable resampling approach can be considered when its assumptions and implementation are justified, but it does not repair a misspecified score or invalid censoring conditions.

The main alternative is a componentwise, one-degree-of-freedom score test when the scientific hypothesis is genuinely one-dimensional. It is not equivalent to testing a vector of contrasts while ignoring their covariance. A likelihood-ratio or Wald test may be appropriate in a separately specified parametric model, but it targets that model’s parameterization and assumptions; it is not automatically a substitute for Gray’s score test.

Common mistake: “The score increments look like event counts, so I can estimate the variance by adding their squared deviations and use a chi-square reference regardless of the sample.” This can omit variability from estimated risk sets, weights, and other parts of Gray’s procedure, and it treats an asymptotic result as exact. Instead, use a covariance estimator that matches the complete score construction, verify its rank and scaling, and regard the chi-square calibration as approximate under the stated conditions.

Check Your Understanding

Answer these questions using the definitions and calculations above. Brief solutions follow.

Why must an estimated covariance account for more than just the target-event increments in the score?
A scalar score is $U_n=-2.4$, with estimated variance $V_n=1.44$. Calculate its quadratic form and state the degrees of freedom for a two-group comparison.
What does a zero eigenvalue of an estimated covariance matrix suggest, and what should be checked before choosing degrees of freedom?
Why must a delete-one covariance calculation rerun the complete Gray score procedure?
State the large-sample result that justifies the chi-square reference and one assumption that could undermine it.

Solutions. (1) Estimated risk-set quantities, weights, and other nuisance estimates can contribute to score variability. (2) $Q_n=(-2.4)^2/1.44=5.76/1.44=4.00$, with one degree of freedom. (3) It may indicate redundant score components, too little information, or numerical instability; check the score parameterization, covariance construction, and available information before using a reduced rank. (4) Deleting a subject can change the estimated quantities used throughout the score; freezing them omits part of the variability. (5) Under the null, $U_n/\sqrt{n}$ is asymptotically mean-zero normal and $V_n/n$ consistently estimates its covariance, yielding a chi-square quadratic form. Invalid censoring assumptions are one reason this argument may fail.