A Joint Test Of Several Regression Coefficients

Why Test Coefficients Jointly?

A regression analysis may ask whether a group of coefficients is jointly compatible with specified values. For example, a model may include two related predictors, and the scientific question may concern whether both slopes are zero. This is not the same question as whether either coefficient is individually significant: the joint test accounts for the covariance between the estimates and tests one combined null hypothesis.

With multiple imputation, each completed data set gives an estimate and a covariance matrix for the coefficients. A D1 test combines those estimates and their covariance information to produce a single approximate F test. Its purpose is to test several linear restrictions on the same parameter vector, such as \(R\beta=c\).

Definition: Suppose \(m\) imputed data sets give coefficient estimates \(\hat\beta_j\) and estimated covariance matrices \(U_j\), for \(j=1,\ldots,m\). To test \(H_0:R\beta=c\), let \(q\) be the number of independent restrictions, with \(R\) having full row rank \(q\). Define \(\hat\theta_j=R\hat\beta_j\), \(V_j=RU_jR^\mathsf{T}\), \(\bar\theta=m^{-1}\sum_j\hat\theta_j\), \(\bar V=m^{-1}\sum_jV_j\), and \(B=(m-1)^{-1}\sum_j(\hat\theta_j-\bar\theta)(\hat\theta_j-\bar\theta)^\mathsf{T}\). The D1 test uses \(d=\bar\theta-c\) and \(r_1=(1+1/m)\operatorname{tr}(B\bar V^{-1})/q\) to calculate an approximate F statistic and reference degrees of freedom.

The analysis model and the imputation procedure must target the same estimand, and the coefficient estimates and covariance matrices must be defined on the same scale in every imputation. The imputations should adequately represent uncertainty under the assumed missing-data and imputation models; the analysis model should be appropriate for the data. The test also requires a finite, invertible \(\bar V\). These conditions support the approximate reference distribution; they do not make the D1 p-value exact in finite samples.

The D1 Statistic and Its Reference Distribution

The pooled estimate of the restricted parameter vector is \(\bar\theta\). The within-imputation covariance \(\bar V\) measures estimated sampling variability within completed data sets, while \(B\) measures variation of the coefficient estimates across imputations. The D1 test starts with the multivariate Wald quadratic form \(d^\mathsf{T}\bar V^{-1}d\), then adjusts its scale to reflect the additional uncertainty from imputation.

F_{\mathrm{D1}}= \frac{d^\mathsf{T}\bar V^{-1}d}{q(1+r_1)}, \qquad r_1=\frac{1+1/m}{q}\operatorname{tr}(B\bar V^{-1}), \qquad df_1=q,\quad df_2=4+\frac{q+1}{q r_1}.

Under \(H_0\), the D1 reference distribution is approximately \(F_{q,df_2}\), and the p-value is its upper-tail probability, \(P(F_{q,df_2}\ge F_{\mathrm{D1}})\). The denominator degrees of freedom are generally non-integer. They depend on the relative between-imputation uncertainty \(r_1\): more between-imputation variation tends to lower \(df_2\), reflecting less precise calibration of the test.

The trace in \(r_1\) summarizes the between-imputation variability relative to within-imputation covariance across all \(q\) restrictions. Because it is a trace of a covariance-scaled matrix, it accounts for covariance among the restricted estimates rather than treating each coefficient as an independent scalar test. If \(r_1=0\), there is no estimated between-imputation variation; the displayed expression for \(df_2\) is then interpreted as having an infinite denominator df, so the reference is \(F_{q,\infty}\). For small positive \(r_1\), \(df_2\) can be large.

Worked Matrix Calculation

Worked Example: Testing Two Slopes Against Zero

Hypothetical data. A researcher fits the same regression model in \(m=5\) imputed data sets. The outcome is systolic blood pressure in mmHg, and two predictors are measured in fixed, stated units. The joint null is that both slopes are zero, so \(q=2\), \(R\) selects the two slopes, and \(c=(0,0)^\mathsf{T}\). Suppose the pooled estimates of these slopes are \(\bar\theta=(1,2)^\mathsf{T}\) mmHg per predictor unit. The average within-imputation covariance and the between-imputation covariance are

\bar V= \begin{pmatrix} 1 & 0.2\\ 0.2 & 1 \end{pmatrix}, \qquad B= \begin{pmatrix} 0.22244 & 0.044488\\ 0.044488 & 0.22244 \end{pmatrix}.

Both covariance matrices are symmetric and positive definite, and \(\bar V\) is invertible. The estimates, covariance matrices, and null values refer to the same coefficient scale in each imputation. The discrepancy is \(d=\bar\theta-c=(1,2)^\mathsf{T}\). The inverse within-imputation covariance is

\bar V^{-1}= \begin{pmatrix} 1.041667 & -0.208333\\ -0.208333 & 1.041667 \end{pmatrix}.

Multiplying this inverse by the discrepancy gives \(\bar V^{-1}d=(0.6250,1.8750)^\mathsf{T}\). Thus the Wald quadratic form is

d^\mathsf{T}\bar V^{-1}d =(1,2) \begin{pmatrix} 0.6250\\ 1.8750 \end{pmatrix} =0.6250+3.750=4.375.

To calculate the relative increase, use \(\operatorname{tr}(B\bar V^{-1})\). The matrices are proportional up to rounding, with \(B\) approximately \(0.22244\bar V\), so the trace is approximately \(2(0.22244)=0.44488\). Therefore,

r_1=\frac{1+1/5}{2}(0.44488)=0.26693.

The test statistic and denominator degrees of freedom are

F_{\mathrm{D1}}= \frac{4.375}{2(1+0.26693)} \approx 1.727, \qquad df_2=4+\frac{2+1}{2(0.26693)} \approx 9.62.

The upper-tail probability for \(F_{2,9.62}\) at 1.727 is approximately 0.229. At conventional significance levels, this is not strong evidence against the joint null. In context, the data do not provide sufficient evidence that the two regression slopes, considered jointly, differ from zero. This conclusion does not establish that both slopes are exactly zero.

Why the Adjustment Has This Form

A short justification helps distinguish the D1 calculation from an ordinary complete-data Wald test. If the restricted estimates were based on one complete data set and \(\bar V\) correctly described their covariance, the Wald quadratic form would be compared with a chi-square distribution with \(q\) degrees of freedom; equivalently, its value divided by \(q\) would be compared with an \(F_{q,\infty}\) reference. Across imputations, however, the estimates vary, and that variation contributes uncertainty not represented by \(\bar V\) alone.

The factor \((1+1/m)B\) is the between-imputation contribution used in Rubin pooling. The trace \(\operatorname{tr}(B\bar V^{-1})/q\) expresses its average size relative to within-imputation variability in the \(q\)-dimensional restriction space. Multiplication by \(1+1/m\) incorporates the finite number of imputations. The D1 approximation scales the Wald statistic by \(1+r_1\), and uses \(df_2=4+(q+1)/(q r_1)\) to calibrate the upper tail. Thus, the test retains a multivariate Wald numerator while allowing for uncertainty and finite degrees of freedom associated with imputation. This is an approximation, not an identity asserting that the statistic has an exact F distribution.

Worked Examples: Applying and Checking the Test

Worked Example: A Specified Nonzero Pair of Slopes

Hypothetical data. In \(m=10\) imputations, an analyst tests whether two slopes equal specified values \(c=(2,-1)^\mathsf{T}\), in their respective outcome-units-per-predictor-unit scales. Suppose \(\bar\theta=(2.8,-1.4)^\mathsf{T}\), \(\bar V=\begin{pmatrix}0.25&0\\0&0.16\end{pmatrix}\), and \(B=\begin{pmatrix}0.05&0\\0&0.04\end{pmatrix}\). The null consists of two independent restrictions, and the diagonal covariance matrices are positive definite. The discrepancy is \(d=(0.8,-0.4)^\mathsf{T}\), and \(\bar V^{-1}=\operatorname{diag}(4,6.25)\).

The quadratic form is \(0.8^2(4)+(-0.4)^2(6.25)=2.56+1.00=3.56\). Also, \(\operatorname{tr}(B\bar V^{-1})=0.05(4)+0.04(6.25)=0.45\), so \(r_1=(1+1/10)(0.45)/2=0.2475\). Consequently,

F_{\mathrm{D1}}=\frac{3.56}{2(1.2475)}\approx1.427, \qquad df_2=4+\frac{3}{2(0.2475)}\approx10.06.

The upper-tail p-value for \(F_{2,10.06}\) is approximately 0.285. The estimates are not sufficiently inconsistent with the specified pair of values to reject the joint null at the 5% level. The result tests the two restrictions together, not whether each estimate independently equals its specified value.

Worked Example: A Joint Test With a Large P-Value

Hypothetical data. In \(m=4\) imputations, suppose two regression coefficients have pooled estimates \(\bar\theta=(3,1)^\mathsf{T}\), and the null is \(c=(2,0)^\mathsf{T}\). Let \(\bar V=\begin{pmatrix}2&0\\0&1\end{pmatrix}\) and \(B=\begin{pmatrix}0.1&0\\0&0.2\end{pmatrix}\). The restrictions are independent, and \(\bar V\) is invertible. Here \(d=(1,1)^\mathsf{T}\) and \(\bar V^{-1}=\operatorname{diag}(0.5,1)\), giving \(d^\mathsf{T}\bar V^{-1}d=0.5+1=1.5\).

The trace is \(0.1(0.5)+0.2(1)=0.25\), so \(r_1=(1+1/4)(0.25)/2=0.15625\). Hence

F_{\mathrm{D1}}=\frac{1.5}{2(1.15625)}\approx0.6486, \qquad df_2=4+\frac{3}{2(0.15625)}=13.60.

The upper-tail p-value for \(F_{2,13.60}\) is approximately 0.538. There is little evidence against the two specified coefficient values considered jointly. A large p-value is not the probability that the null is true; it describes how often a statistic at least this large would occur under the null and the approximate reference model.

Diagnostics, Alternatives, and Limitations

Before interpreting a D1 result, verify that the same coefficients and restrictions were estimated in every imputation, and that the rows and columns of the covariance matrices are in the same order. Check that \(R\) has rank \(q\), that \(\bar V\) can be inverted, and that the matrices are symmetric and sensible for the coefficient scales. A singular or nearly singular \(\bar V\) can make the inverse and Wald statistic unstable; it can arise from redundant restrictions, collinear parameters, or insufficient information. Resolve the model or restriction problem rather than silently relying on an unstable inverse.

Also inspect the scale of \(r_1\), the number of imputations, and whether the coefficient estimates are broadly consistent across imputations. A large \(r_1\) signals substantial imputation-related uncertainty and often leads to a smaller denominator df. A very small number of imputations can make estimates of between-imputation variability noisy. The D1 approximation may be unreliable with highly non-normal estimates, poorly estimated covariance matrices, sparse or separated categorical outcomes, or analyses with inadequate imputations. A finite complete-data degrees-of-freedom setting may require an appropriate small-sample adjustment; the displayed formula is the standard D1 reference calculation and should not be treated as a universal correction for every finite-sample design.

The main alternatives answer related but not identical pooling questions. Separate scalar tests do not generally reproduce a joint test, because they do not use the full covariance among coefficients and do not directly test the combined null. D2 pools repeated test statistics, while D3 pools likelihood-ratio information; neither is obtained by averaging the D1 F statistics. For linear restrictions, D1 uses pooled coefficient estimates and covariance information. The earlier material on linear contrasts explains how to express and pool linear combinations; here those ideas are assembled into a joint test. A nonlinear restriction is not automatically covered by \(R\beta=c\); it requires a method suited to nonlinear contrasts.

Common mistake: invert the total pooled covariance and then apply the D1 inflation again. The D1 formula here uses \(\bar V^{-1}\) in the Wald quadratic and adjusts by \(1+r_1\). Substituting a total covariance into the quadratic and also applying that inflation changes the test and can count between-imputation uncertainty twice.
Common mistake: test each coefficient separately and infer the joint result. Separate p-values do not account for the full covariance structure and are not the D1 test of \(q\) restrictions.
Common mistake: read a large p-value as proof of equality. A failure to reject means the data do not provide sufficient evidence against the joint null; it does not establish that the specified values are true.

Key takeaway: For a D1 joint test, form the discrepancy \(d=\bar\theta-c\), calculate the quadratic form with the inverse average within-imputation covariance, and adjust it using \(r_1\). Report \(F_{\mathrm{D1}}\), both degrees of freedom, the upper-tail p-value, and the joint restrictions in context.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

What is the meaning of \(q\) in a D1 test, and why must the restrictions be independent?
If \(r_1=0.20\) and \(q=3\), calculate the D1 denominator degrees of freedom.
For a two-coefficient test, explain why separate scalar tests do not automatically equal the joint D1 test.
A D1 test gives a large p-value. State an appropriate conclusion and one conclusion that would be incorrect.
What should be checked if the average within-imputation covariance matrix cannot be inverted?

Solutions. (1) \(q\) is the number of independent linear restrictions, and it is the numerator degrees of freedom. Redundant restrictions do not add independent information and make the stated dimension incorrect. (2) \(df_2=4+(q+1)/(q r_1)=4+4/(3(0.20))=10.67\). (3) Separate tests do not use the full multivariate covariance structure as a single joint statistic, so their results need not match the test of all restrictions together. (4) The data do not provide strong evidence against the joint null under the approximate reference model; it would be incorrect to say that the null has been proved or has a high probability of being true. (5) Check for redundant restrictions, parameter-ordering errors, collinearity, and inadequate information; do not interpret a result based on an unstable or undefined inverse.