Nuisance Parameters and Schur-Complement Information

Why nuisance parameters change the information available

A population design is often intended to estimate a particular parameter or parameter group, while other model parameters must also be estimated to describe the data adequately. Those other parameters are nuisance parameters for the target question: they are not the immediate design objective, but uncertainty about them can affect precision for the target. In pharmacometric models, for example, the target may be a fixed effect while random-effect or residual-error parameters are also estimated.

The population Fisher information matrix describes local information about all parameters jointly. Its diagonal target block alone answers a different question: how much information would be available about the target if the nuisance parameters were known. When nuisance parameters are estimated, cross-information between parameter groups can reduce the information effectively available for the target. The Schur complement gives that adjusted information.

Definition: Partition a parameter vector into parameters of interest $\psi$ and nuisance parameters $\lambda$, and partition the expected Fisher information matrix accordingly: $$ \mathcal{I} = \begin{pmatrix} \mathcal{I}_{\psi\psi} & \mathcal{I}_{\psi\lambda} \\ \mathcal{I}_{\lambda\psi} & \mathcal{I}_{\lambda\lambda} \end{pmatrix}. $$ If $\mathcal{I}_{\lambda\lambda}$ is nonsingular, the effective information for $\psi$ with $\lambda$ estimated is $$ \mathcal{I}_{\mathrm{eff},\psi} = \mathcal{I}_{\psi\psi} - \mathcal{I}_{\psi\lambda} \mathcal{I}_{\lambda\lambda}^{-1} \mathcal{I}_{\lambda\psi}. $$ This is the Schur complement of the nuisance block $\mathcal{I}_{\lambda\lambda}$ in the full information matrix.

Here, $\mathcal{I}_{\psi\psi}$ is the information about $\psi$ when $\lambda$ is fixed and known. The subtracted term accounts for the loss caused by estimating $\lambda$ jointly. The cross-blocks $\mathcal{I}_{\psi\lambda}$ and $\mathcal{I}_{\lambda\psi}$ describe how the score for one group relates to the score for the other. For a Fisher information matrix, $\mathcal{I}_{\lambda\psi}=\mathcal{I}_{\psi\lambda}^{T}$.

Why the Schur complement is the effective information

Assume a regular likelihood model at an interior parameter value, with differentiable likelihood, finite score covariance, and the usual equality between expected Fisher information and score covariance. Also assume the nuisance block is positive definite, so its inverse exists. Write the score as $(U_\psi,U_\lambda)$. Its covariance blocks are the corresponding Fisher information blocks. The part of $U_\psi$ that is linearly predictable from $U_\lambda$ does not supply independent information about $\psi$: it can be explained by movement in nuisance parameters.

Remove that predictable component by defining an adjusted score $U_{\mathrm{eff}}=U_\psi-AU_\lambda$, where $A$ is chosen so that the adjusted score is uncorrelated with the nuisance score. The condition and its solution are

\operatorname{Cov}(U_{\mathrm{eff}},U_\lambda) = \mathcal{I}_{\psi\lambda}-A\mathcal{I}_{\lambda\lambda}=0, \qquad A=\mathcal{I}_{\psi\lambda}\mathcal{I}_{\lambda\lambda}^{-1}.

The covariance of this adjusted score is its information. Substituting the value of $A$ gives

\begin{aligned} \operatorname{Var}(U_{\mathrm{eff}}) &= \mathcal{I}_{\psi\psi} -A\mathcal{I}_{\lambda\psi} -\mathcal{I}_{\psi\lambda}A^T +A\mathcal{I}_{\lambda\lambda}A^T \\ &= \mathcal{I}_{\psi\psi} -\mathcal{I}_{\psi\lambda}\mathcal{I}_{\lambda\lambda}^{-1} \mathcal{I}_{\lambda\psi} =\mathcal{I}_{\mathrm{eff},\psi}. \end{aligned}

The subtraction is positive semidefinite, so effective information cannot exceed the known-nuisance information in any direction. Under regularity and local large-sample approximation, the $\psi$-block of the inverse full information matrix is

\left(\mathcal{I}^{-1}\right)_{\psi\psi} = \mathcal{I}_{\mathrm{eff},\psi}^{-1}.

Thus, the inverse Schur complement approximates the covariance matrix for estimating $\psi$ when both parameter groups are estimated. It is a local information and precision statement, not a guarantee of finite-sample estimator performance.

Worked Example: One Target and One Nuisance Parameter

Suppose a hypothetical population design has the following expected Fisher information at the nominal parameter values. Let $\psi$ be a target effect measured in concentration units per dose unit, and let $\lambda$ be a nuisance parameter. The information entries are reported in the corresponding inverse-parameter units.

\mathcal{I}= \begin{pmatrix} 100 & 30\\ 30 & 25 \end{pmatrix}.

The nuisance block is $25$, which is positive, so it is invertible. The target information if the nuisance parameter were known is $100$. With $\lambda$ estimated, the effective information is

\mathcal{I}_{\mathrm{eff},\psi} =100-\frac{30(30)}{25} =100-36 =64.

Under the local large-sample approximation, the target variance bound is $1/64=0.015625$ in squared target units, giving a standard error of $1/\sqrt{64}=0.125$ target units. If $\lambda$ were known, the corresponding information-based standard error would be $1/\sqrt{100}=0.100$ target units. The difference reflects the nonzero cross-information: uncertainty in $\lambda$ reduces target precision.

Vector parameters and the meaning of the adjustment

When $\psi$ contains several parameters, the effective information is a matrix, not a single adjusted diagonal entry. The cross-block structure matters: nuisance estimation can alter both the information about individual target parameters and their joint dependence. The matrix subtraction should be performed as written, preserving parameter order and dimensions.

Worked Example: Two Targets and Two Nuisance Parameters

Consider a hypothetical information matrix with two target parameters $\psi=(\psi_1,\psi_2)$ and two nuisance parameters $\lambda=(\lambda_1,\lambda_2)$:

\mathcal{I}_{\psi\psi}= \begin{pmatrix}12&2\\2&8\end{pmatrix}, \quad \mathcal{I}_{\psi\lambda}= \begin{pmatrix}2&1\\1&2\end{pmatrix}, \quad \mathcal{I}_{\lambda\lambda}= \begin{pmatrix}4&0\\0&2\end{pmatrix}.

The nuisance block is positive definite. Its inverse is $\operatorname{diag}(1/4,1/2)$. First calculate the adjustment:

\begin{aligned} \mathcal{I}_{\psi\lambda}\mathcal{I}_{\lambda\lambda}^{-1} \mathcal{I}_{\lambda\psi} &= \begin{pmatrix}2&1\\1&2\end{pmatrix} \begin{pmatrix}1/4&0\\0&1/2\end{pmatrix} \begin{pmatrix}2&1\\1&2\end{pmatrix}\\ &= \begin{pmatrix}1.5&1.5\\1.5&2.25\end{pmatrix}. \end{aligned}

Therefore,

\mathcal{I}_{\mathrm{eff},\psi} = \begin{pmatrix}12&2\\2&8\end{pmatrix} - \begin{pmatrix}1.5&1.5\\1.5&2.25\end{pmatrix} = \begin{pmatrix}10.5&0.5\\0.5&5.75\end{pmatrix}.

The target parameters retain information, but less than if the nuisance parameters were known. For instance, the approximate variance for $\psi_1$, allowing $\psi_2$ and both nuisance parameters to be estimated, is the first diagonal element of $\mathcal{I}_{\mathrm{eff},\psi}^{-1}$. The determinant is $10.5(5.75)-0.5^2=60.125$, so this variance is $5.75/60.125\approx0.09563$, and its standard error is approximately $0.3092$ in $\psi_1$'s units. Reading $1/10.5$ as that variance would ignore the need to estimate the other target parameter.

Using effective information to compare designs

For design decisions, calculate the effective information at the same nominal parameter values and under the same parameterization for each candidate design. A design with greater $\mathcal{I}_{\psi\psi}$ need not have greater effective information: strong target–nuisance cross-information can offset a large known-nuisance block.

Worked Example: A Design With More Target Information Can Be Less Precise

Two hypothetical designs are being compared for one target parameter and one nuisance parameter. Assume both nuisance blocks are positive and both matrices are valid positive-definite information matrices:

\mathcal{I}_A= \begin{pmatrix}100&60\\60&50\end{pmatrix}, \qquad \mathcal{I}_B= \begin{pmatrix}64&16\\16&16\end{pmatrix}.

Design A has greater target information if the nuisance parameter is known: $100$ versus $64$. But the effective information for A is $100-60^2/50=28$, while for B it is $64-16^2/16=48$. The corresponding local standard errors are $1/\sqrt{28}\approx0.1890$ target units for A and $1/\sqrt{48}\approx0.1443$ target units for B. Therefore, B is more informative for the target when the nuisance parameter is estimated, despite having a smaller target diagonal block. This comparison concerns local precision only; it does not account for differences in cost, feasibility, or robustness to model misspecification.

Assumptions, diagnostics, and failure modes

Before using a Schur complement, check that the parameter partition reflects the scientific question, that the matrix is evaluated at the intended nominal values and design, and that the nuisance block can be inverted. The full information matrix should be symmetric and positive semidefinite; numerical asymmetry or materially negative eigenvalues can indicate calculation or coding problems. The nuisance block should be positive definite for the ordinary inverse formula. A nearly singular block may make the result highly sensitive to rounding and nominal values.

A zero cross-block gives $\mathcal{I}_{\mathrm{eff},\psi}=\mathcal{I}_{\psi\psi}$: locally, the target and nuisance scores are orthogonal, so estimating the nuisance group causes no first-order information loss. Otherwise, the subtraction is positive semidefinite. If a computed adjustment produces more effective information than known-nuisance information in some direction, check matrix ordering, transposes, units, and numerical calculations.

If $\mathcal{I}_{\lambda\lambda}$ is singular, the stated inverse formula is not defined. This can arise when nuisance parameters are redundant or locally unidentified. Do not silently replace the inverse with a generalized inverse and interpret the result as an ordinary Schur complement. First determine which parameter combinations are estimable, then consider a scientifically justified reparameterization or restriction; the relevant information calculation depends on that choice. Near-singularity also warrants caution even when a numerical inverse exists.

The alternative calculation $(\mathcal{I}^{-1})_{\psi\psi}$ can be useful when the full matrix is already inverted. Its inverse is the effective information, provided the relevant matrices are nonsingular. Using $\mathcal{I}_{\psi\psi}$ alone is appropriate only for a question that treats nuisance parameters as known (or when the cross-information is zero). Neither shortcut replaces checking the assumptions. The result is local to the nominal parameter values and model; it does not diagnose model misspecification or guarantee that the likelihood is well approximated by a quadratic.

Key takeaway: When nuisance parameters are estimated jointly, target information is the Schur complement $\mathcal{I}_{\psi\psi}-\mathcal{I}_{\psi\lambda}\mathcal{I}_{\lambda\lambda}^{-1}\mathcal{I}_{\lambda\psi}$, not simply the target diagonal block. The adjustment quantifies the information shared with nuisance-score directions.

Check Your Understanding

Answer each question, then compare your work with the solutions below.

For $\mathcal{I}=\begin{pmatrix}50&10\\10&8\end{pmatrix}$, calculate the effective information for the first parameter when the second is nuisance.
What does a zero target–nuisance cross-information block imply about effective versus known-nuisance information?
For a vector target, why is it generally incorrect to adjust only the diagonal entries of $\mathcal{I}_{\psi\psi}$?
What should be investigated if the nuisance information block is singular or nearly singular?

Solutions. (1) The nuisance block is $8$, so $\mathcal{I}_{\mathrm{eff}}=50-10^2/8=50-12.5=37.5$. (2) The adjustment is zero, so the effective information equals $\mathcal{I}_{\psi\psi}$; there is no first-order information loss from estimating the nuisance group. (3) The adjustment is a matrix product that can change off-diagonal as well as diagonal entries. Those off-diagonal entries affect joint estimation and the diagonal of the inverse effective information. (4) Check for redundant or unidentified nuisance parameter combinations and numerical instability. Establish which combinations are estimable and consider a scientifically justified reparameterization or restriction before applying an inverse-based formula.