Why nuisance parameters change a test
A coefficient is rarely estimated in isolation. For example, a slope in a regression model is estimated alongside an intercept, and a treatment effect may be estimated alongside covariate effects. These other parameters are nuisance parameters for a particular test: they are part of the model, but they are not the target of the hypothesis.
If nuisance parameters were known, the information for the tested coefficient would be the corresponding block of the information matrix. When they must instead be estimated, some of the score variation for the target can be explained by variation in the nuisance score. The information remaining after accounting for that overlap is the efficient information. Its formula is a Schur complement, and it is the information that belongs in a test that allows the nuisance parameters to be estimated.
These formulas apply to the information for the whole sample, or to per-observation information, provided the same convention is used consistently. Here \(\mathcal I\) denotes Fisher information, the covariance matrix of the score under the model; in regular models it also equals the expected negative Hessian of the log-likelihood. The cross-blocks describe how the tested and nuisance scores vary together.
Why the Schur complement is the right information
The efficient score is obtained by subtracting from \(U_\psi\) the part that can be linearly predicted by \(U_\lambda\). The coefficient \(\mathcal I_{\psi\lambda}\mathcal I_{\lambda\lambda}^{-1}\) is exactly the covariance-based linear projection coefficient. This gives a score uncorrelated with the nuisance score, so its variance measures the information about \(\psi\) left after nuisance variation is accounted for.
Proof. Set \(A=\mathcal I_{\psi\lambda}\mathcal I_{\lambda\lambda}^{-1}\). Since score covariance equals Fisher information, $$ \operatorname{Cov}(U_{\mathrm{eff}},U_\lambda) =\mathcal I_{\psi\lambda}-A\mathcal I_{\lambda\lambda}=0. $$ Expanding the variance gives $$ \begin{aligned} \operatorname{Var}(U_{\mathrm{eff}}) &=\mathcal I_{\psi\psi}-A\mathcal I_{\lambda\psi} -\mathcal I_{\psi\lambda}A^\mathsf{T} +A\mathcal I_{\lambda\lambda}A^\mathsf{T}\\ &=\mathcal I_{\psi\psi} -\mathcal I_{\psi\lambda}\mathcal I_{\lambda\lambda}^{-1}\mathcal I_{\lambda\psi} =\mathcal I_{\mathrm{eff}}. \end{aligned} $$ For any nonzero \(v\in\mathbb R^q\), take \(w=(-\mathcal I_{\lambda\lambda}^{-1}\mathcal I_{\lambda\psi}v,\ v^\mathsf{T})^\mathsf{T}\), which is nonzero. Direct multiplication gives \(w^\mathsf{T}\mathcal I w=v^\mathsf{T}\mathcal I_{\mathrm{eff}}v\). Positive definiteness of \(\mathcal I\) therefore implies \(v^\mathsf{T}\mathcal I_{\mathrm{eff}}v>0\), proving the claim. \(\square\)
Another way to see the same adjustment is through matrix inversion. The \(\psi,\psi\) block of \(\mathcal I^{-1}\) is \(\mathcal I_{\mathrm{eff}}^{-1}\). Thus, when the nuisance parameters are estimated, the asymptotic covariance for the tested block of the full estimator is governed by the inverse efficient information, not by \(\mathcal I_{\psi\psi}^{-1}\).
Worked Example: A Gaussian Slope With an Unknown Intercept
Let \(Y_i\) be independent with \(Y_i\sim N(\alpha+\beta x_i,1)\), where the variance is known to be \(1\). Test \(H_0:\beta=0\); the intercept \(\alpha\) is a nuisance parameter. Use the observations \(x=(0,1,2)\) and \(y=(1,2,2)\).
The information for \((\alpha,\beta)\) is \(X^\mathsf{T}X\). Since \(\sum x_i=3\) and \(\sum x_i^2=5\), $$ \mathcal I= \begin{pmatrix}3&3\\3&5\end{pmatrix}, \qquad \mathcal I_{\mathrm{eff}}=5-\frac{3^2}{3}=2. $$
Under the null, the restricted estimate is \(\hat\alpha_0=\bar y=5/3\). The slope score at this restricted fit is $$ U_\beta=\sum_i x_i(y_i-\hat\alpha_0) =0\left(-\frac23\right)+1\left(\frac13\right)+2\left(\frac13\right)=1. $$ The intercept score is zero at the restricted fit, so this is also the efficient score there. The score statistic is \(1^2/2=0.5\).
As a check, the unrestricted slope estimate is \(\hat\beta=\sum_i(x_i-\bar x)(y_i-\bar y)/\sum_i(x_i-\bar x)^2=(1)/2\), and its variance is \(1/2\). The Wald statistic is \((1/2)^2/(1/2)=0.5\). In this known-variance Gaussian model, the likelihood-ratio statistic is also \(0.5\): the reduction in residual sum of squares is \((1)^2/2=0.5\). The agreement here is exact; in general GLMs the three statistics need not agree in a finite sample.
Using efficient information in the three tests
Suppose the null hypothesis is \(H_0:\psi=\psi_0\). Let \(\hat\lambda_0\) be the restricted maximum likelihood estimate of the nuisance parameter, and evaluate the score and information at \((\psi_0,\hat\lambda_0)\). Under regularity conditions, the nuisance score is zero at this fit. Consequently, the efficient score there equals \(U_\psi\), but its variance is still \(\mathcal I_{\mathrm{eff}}\). The adjustment remains essential even though the nuisance score itself is zero.
For clarity, standard asymptotic conclusions require a correctly specified, identifiable, regular model; an interior null parameter; fixed parameter dimension; nonsingular information; and the usual differentiability, moment, and likelihood regularity conditions that support consistent maximum likelihood estimation, a score central limit theorem, and a local quadratic likelihood expansion. In an independent-observation GLM, these conditions are understood to apply to the likelihood being used. The tests below are large-sample tests under these conditions.
The score result follows by expanding the nuisance score around the true null value. With sample size \(n\), write the information blocks at the null as \(n\mathcal I_{\psi\psi}\), \(n\mathcal I_{\psi\lambda}\), and so on. The restricted nuisance estimate satisfies, to first order, $$ \sqrt n(\hat\lambda_0-\lambda_0) =\mathcal I_{\lambda\lambda}^{-1}\frac{U_\lambda(\psi_0,\lambda_0)}{\sqrt n}+o_p(1). $$ A first-order Taylor expansion of the tested score at the restricted fit then gives $$ \frac{U_\psi(\psi_0,\hat\lambda_0)}{\sqrt n} =\frac{U_\psi(\psi_0,\lambda_0)}{\sqrt n} -\mathcal I_{\psi\lambda}\mathcal I_{\lambda\lambda}^{-1} \frac{U_\lambda(\psi_0,\lambda_0)}{\sqrt n}+o_p(1). $$ The limiting covariance of the right-hand leading term is \(\mathcal I_{\mathrm{eff}}\), by the projection calculation above. The multivariate central limit theorem therefore implies $$ U_\psi(\psi_0,\hat\lambda_0)^\mathsf{T} \mathcal I_{\mathrm{eff},n}^{-1} U_\psi(\psi_0,\hat\lambda_0) \xrightarrow{d}\chi^2_q, $$ where \(\mathcal I_{\mathrm{eff},n}\) is the total efficient information at the restricted fit.
For the Wald test, the full maximum likelihood estimator is asymptotically normal with covariance \(\mathcal I_n^{-1}\). The tested block of this inverse is \(\mathcal I_{\mathrm{eff},n}^{-1}\), so the usual Wald quadratic form for \(\hat\psi-\psi_0\) has a \(\chi^2_q\) limit. For the likelihood-ratio test, the local quadratic expansion of the log-likelihood reduces, after maximizing over nuisance increments under both hypotheses, to a quadratic in the tested increment with information \(\mathcal I_{\mathrm{eff}}\). Maximizing that quadratic gives twice the log-likelihood gain equal to the efficient-score quadratic form to first order. This establishes the same \(\chi^2_q\) limit for the likelihood-ratio statistic.
Worked Example: A Logistic Slope With an Intercept Nuisance Parameter
Consider two groups of 20 independent Bernoulli observations each, with \(x=0\) in group 0 and \(x=1\) in group 1. Fit \(\operatorname{logit}(p_i)=\alpha+\beta x_i\), and test \(H_0:\beta=0\). The observed success counts are 8 for \(x=0\) and 12 for \(x=1\).
Under the null, the pooled success proportion is \(20/40=0.5\), so \(\hat\alpha_0=0\). The intercept score is \(8+12-40(0.5)=0\); the slope score is \(12-20(0.5)=2\). Each observation has information weight \(p(1-p)=0.25\), giving $$ \mathcal I= \begin{pmatrix}10&5\\5&5\end{pmatrix}, \qquad \mathcal I_{\mathrm{eff}}=5-\frac{5^2}{10}=2.5. $$ Thus the score statistic is \(2^2/2.5=1.6\).
For comparison, the unrestricted fitted proportions are \(0.4\) and \(0.6\). Hence \(\hat\alpha=\operatorname{logit}(0.4)=\log(2/3)\) and \(\hat\beta=\operatorname{logit}(0.6)-\operatorname{logit}(0.4)=\log(9/4)\approx0.8109\). At these estimates the information matrix is $$ \begin{pmatrix}9.6&4.8\\4.8&4.8\end{pmatrix}. $$ Its efficient information for the slope is \(4.8-4.8^2/9.6=2.4\), so the Wald statistic is \(2.4(0.8109)^2\approx1.578\). The likelihood-ratio statistic is $$ 2\left\{16\log(0.8)+24\log(1.2)\right\}\approx1.611. $$ All three tests use one degree of freedom, but their finite-sample values differ.
Worked Example: A Joint Poisson Test With Two Slopes
For four independent Poisson counts, use \(\log(\mu_i)=\alpha+\beta_1x_{i1}+\beta_2x_{i2}\), with covariate pairs \((0,0),(1,0),(0,1),(1,1)\) and observed counts \(3,1,2,2\), respectively. Test \(H_0:\beta_1=\beta_2=0\), treating the intercept as nuisance.
At the restricted fit, the common fitted mean is \(\hat\mu=8/4=2\). The intercept score is \(\sum_i(y_i-2)=0\); the slope scores are \(U_{\beta_1}=1(1-2)+1(2-2)=-1\) and \(U_{\beta_2}=1(2-2)+1(2-2)=0\). For Poisson log-link regression the information is \(\sum_i\mu_i x_ix_i^\mathsf{T}\). With columns ordered as intercept, \(x_1\), and \(x_2\), this gives $$ \mathcal I= \begin{pmatrix}8&4&4\\4&4&2\\4&2&4\end{pmatrix}, \qquad \mathcal I_{\mathrm{eff}}= \begin{pmatrix}4&2\\2&4\end{pmatrix} -\begin{pmatrix}4\\4\end{pmatrix}\frac18 \begin{pmatrix}4&4\end{pmatrix} = \begin{pmatrix}2&0\\0&2\end{pmatrix}. $$ The joint score statistic is \((-1,0)\mathcal I_{\mathrm{eff}}^{-1}(-1,0)^\mathsf{T}=0.5\). It is compared with a \(\chi^2_2\) reference distribution asymptotically. With only four observations, that approximation need not be accurate; the calculation illustrates the adjustment, not a claim of reliable small-sample calibration.
A common mistake and what the adjustment means
A frequent error is to use \(\mathcal I_{\psi\psi}\) as the information for a test even though \(\lambda\) is estimated. That treats the nuisance parameter as if it were known. The correct information is smaller in the positive-semidefinite sense: the subtracted term \(\mathcal I_{\psi\lambda}\mathcal I_{\lambda\lambda}^{-1}\mathcal I_{\lambda\psi}\) is positive semidefinite. In the logistic example, using \(5\) rather than \(2.5\) would halve the score statistic and overstate the information about the slope.
There is a related subtlety: at a restricted maximum likelihood fit, \(U_\lambda=0\), so the efficient score numerically equals \(U_\psi\). It does not follow that the efficient information equals \(\mathcal I_{\psi\psi}\). The score value and its variance play different roles, and the nuisance adjustment remains in the variance. This is why a valid score test uses both the tested score at the restricted fit and the Schur-complement information.
Check Your Understanding
Use the definitions and calculations above to answer the following.
- In a partitioned information matrix, what does the cross-block \(\mathcal I_{\psi\lambda}\) represent?
- Why can the efficient score equal \(U_\psi\) at a restricted fit while the efficient information still differs from \(\mathcal I_{\psi\psi}\)?
- For scalar \(\psi\), write the efficient information when \(\lambda\) is vector-valued.
- In the Gaussian example, what is the score statistic if the nuisance intercept were known rather than estimated? Explain the information change.
- Why must the Wald covariance for a tested coefficient come from the full parameter covariance matrix?