Nuisance Parameters And Schur Complements

Why Nuisance Parameters Change the Information Picture

A pharmacometric model may contain parameters that are not the immediate target of a design decision but still have to be estimated. For example, a design may target a treatment effect while also estimating a baseline, an assay variance, or a random-effects parameter. Such parameters are nuisance parameters for that particular question: they matter to the model, but the quantity of interest is another parameter or parameter vector.

The diagonal entry of a Fisher information matrix for a target parameter does not, by itself, describe how precisely that target can be estimated when nuisance parameters are unknown. Off-diagonal entries describe information coupling: changes in the target can be partly offset by changes in nuisance parameters. The Schur complement accounts for this coupling and gives the effective information for the target in the local likelihood approximation.

Definition: Partition the parameter vector as $\theta=(\psi^\mathsf{T},\eta^\mathsf{T})^\mathsf{T}$, where $\psi$ is the parameter vector of interest and $\eta$ is the nuisance-parameter vector. Partition the Fisher information matrix at the parameter point and design under consideration as $$ \mathcal I = \begin{pmatrix} \mathcal I_{\psi\psi} & \mathcal I_{\psi\eta}\\ \mathcal I_{\eta\psi} & \mathcal I_{\eta\eta} \end{pmatrix}. $$ If $\mathcal I_{\eta\eta}$ is invertible, the effective information for $\psi$ after accounting for $\eta$ is the Schur complement $$ \mathcal I_{\psi\cdot\eta} = \mathcal I_{\psi\psi} - \mathcal I_{\psi\eta}\mathcal I_{\eta\eta}^{-1}\mathcal I_{\eta\psi}. $$

Here $\mathcal I_{\psi\psi}$ is the information block for the target considered on its own within the full matrix, and the subtracted term represents the loss associated with its information coupling to nuisance parameters. This is a matrix statement: $\mathcal I_{\psi\cdot\eta}$ has the same dimension as $\mathcal I_{\psi\psi}$. For one target parameter it is a scalar. Fisher information entries carry units determined by the parameters; in the examples below, parameters are standardized so the displayed information values can be compared directly.

Deriving the Schur Complement

The derivation uses the local quadratic approximation to the log likelihood, which is standard under regularity conditions. The likelihood must be sufficiently smooth near the parameter point, the model must be locally identifiable in the directions being considered, and the Fisher information must be a valid finite matrix. Write $A=\mathcal I_{\psi\psi}$, $B=\mathcal I_{\psi\eta}$, and $C=\mathcal I_{\eta\eta}$. For small parameter changes $\delta\psi$ and $\delta\eta$, the quadratic part of minus twice the log likelihood is proportional to

Q(\delta\psi,\delta\eta) = \delta\psi^\mathsf{T}A\delta\psi + 2\delta\psi^\mathsf{T}B\delta\eta + \delta\eta^\mathsf{T}C\delta\eta.

Assume $C$ is invertible. Complete the square in the nuisance change:

\begin{aligned} Q(\delta\psi,\delta\eta) &= \left(\delta\eta+C^{-1}B^\mathsf{T}\delta\psi\right)^\mathsf{T} C \left(\delta\eta+C^{-1}B^\mathsf{T}\delta\psi\right)\\ &\quad+ \delta\psi^\mathsf{T} \left(A-BC^{-1}B^\mathsf{T}\right)\delta\psi. \end{aligned}

To verify the identity, expanding the first term gives the nuisance quadratic $\delta\eta^\mathsf{T}C\delta\eta$, the cross term $2\delta\psi^\mathsf{T}B\delta\eta$, and the additional target term $\delta\psi^\mathsf{T}BC^{-1}B^\mathsf{T}\delta\psi$. The final term subtracts exactly that additional target term and adds $\delta\psi^\mathsf{T}A\delta\psi$, recovering $Q$.

For each fixed $\delta\psi$, the quadratic is minimized over $\delta\eta$ at $\delta\eta=-C^{-1}B^\mathsf{T}\delta\psi$. Substituting this value leaves $\delta\psi^\mathsf{T}(A-BC^{-1}B^\mathsf{T})\delta\psi$. Thus the Schur complement is the curvature remaining in the target direction when nuisance changes are allowed to adjust locally. It is not generally equal to the target block $A$.

Theorem: If the full information matrix is positive definite and $\mathcal I_{\eta\eta}$ is invertible, then the Schur complement $\mathcal I_{\psi\cdot\eta}$ is positive definite. Under the usual large-sample likelihood approximation, the $\psi$-by-$\psi$ block of $\mathcal I^{-1}$ is $\mathcal I_{\psi\cdot\eta}^{-1}$. Consequently, the approximate covariance for estimating $\psi$ while estimating $\eta$ is $\mathcal I_{\psi\cdot\eta}^{-1}$.

The covariance statement follows directly from block elimination. To solve $\mathcal I\binom{x}{y}=\binom{u}{0}$, the lower block equation gives $y=-\mathcal I_{\eta\eta}^{-1}\mathcal I_{\eta\psi}x$. Substitution into the upper equation gives $\mathcal I_{\psi\cdot\eta}x=u$, so $x=\mathcal I_{\psi\cdot\eta}^{-1}u$. This is precisely the target block of the inverse full matrix. When $\psi$ is scalar, its approximate variance is $1/\mathcal I_{\psi\cdot\eta}$, and its approximate standard error is the square root of that variance.

Worked Example: A Target Effect and an Unknown Baseline

Suppose a hypothetical local Fisher information matrix for a standardized treatment-effect parameter $\psi$ and a standardized baseline parameter $\eta$ is

\mathcal I= \begin{pmatrix} 100 & 60\\ 60 & 50 \end{pmatrix}.

The diagonal entries are positive, and the determinant is $100(50)-60^2=1{,}400>0$, so this symmetric matrix is positive definite. The nuisance block $50$ is invertible. The effective information for the treatment effect is

\mathcal I_{\psi\cdot\eta} = 100-\frac{60^2}{50} = 100-72 = 28.

The approximate variance is $1/28=0.03571$ standardized units squared, and the approximate standard error is $\sqrt{1/28}=0.1890$ standardized units. If the baseline were treated as known, the target block alone would give information $100$ and standard error $1/\sqrt{100}=0.1000$. That smaller standard error does not describe the situation in which the baseline must also be estimated. The cross-information of $60$ makes the treatment effect less precisely estimated.

Several Nuisance Parameters

When $\eta$ contains multiple parameters, their joint information and their mutual associations matter. It is not valid in general to adjust for each nuisance parameter separately and subtract a collection of scalar corrections. The matrix product in the Schur complement accounts for the full nuisance block, including its off-diagonal entries.

Worked Example: Adjusting for Two Nuisance Parameters

Consider a hypothetical information matrix for one standardized target parameter $\psi$ and two standardized nuisance parameters $\eta_1,\eta_2$:

\mathcal I= \begin{pmatrix} 40 & 6 & 8\\ 6 & 9 & 3\\ 8 & 3 & 16 \end{pmatrix}.

The nuisance block is $C=\begin{pmatrix}9&3\\3&16\end{pmatrix}$. Its determinant is $9(16)-3^2=135>0$, and its diagonal entries are positive, so it is positive definite and invertible. Its inverse is

C^{-1} = \frac{1}{135} \begin{pmatrix} 16&-3\\ -3&9 \end{pmatrix}.

The target-to-nuisance block is $B=(6\;\;8)$. Calculate the correction:

\begin{aligned} BC^{-1}B^\mathsf{T} &= \frac{1}{135}(6\;\;8) \begin{pmatrix} 16&-3\\ -3&9 \end{pmatrix} \binom{6}{8}\\ &= \frac{1}{135}(6\;\;8)\binom{72}{54} = \frac{864}{135} = 6.400. \end{aligned}

Therefore $\mathcal I_{\psi\cdot\eta}=40-6.400=33.60$. The approximate variance for $\psi$ is $1/33.60=0.02976$ standardized units squared, and its standard error is $0.1725$ standardized units. The nuisance block is positive definite and the Schur complement is positive, so the full matrix is positive definite by the block-matrix criterion. The calculation uses the covariance structure within the nuisance block; ignoring its off-diagonal entries would change the correction.

Comparing Candidate Designs

For design work, the Schur complement provides a target-specific comparison when nuisance parameters are estimated as part of the same model. A candidate design with a larger $\mathcal I_{\psi\psi}$ can still have a smaller effective information if it also has stronger coupling between the target and nuisance parameters. Comparing only the target diagonal block can therefore rank designs incorrectly for the target question.

Worked Example: More Raw Information, Less Effective Information

Two hypothetical designs provide information matrices for a standardized target $\psi$ and nuisance parameter $\eta$:

\mathcal I_A= \begin{pmatrix} 30&6\\ 6&3 \end{pmatrix}, \qquad \mathcal I_B= \begin{pmatrix} 40&18\\ 18&12 \end{pmatrix}.

Both matrices are positive definite: their determinants are $30(3)-6^2=54>0$ and $40(12)-18^2=156>0$, respectively, with positive diagonal entries. For Design A, $\mathcal I_{\psi\cdot\eta,A}=30-6^2/3=18$. For Design B, $\mathcal I_{\psi\cdot\eta,B}=40-18^2/12=40-27=13$. Thus Design B has the larger target block, $40$ rather than $30$, but its effective information is lower, $13$ rather than $18$.

The corresponding approximate standard errors are $1/\sqrt{18}=0.2357$ standardized units for Design A and $1/\sqrt{13}=0.2774$ standardized units for Design B. For this target and under these local information matrices, Design A gives greater adjusted information. This comparison is specific to the chosen target and the assumed nominal parameter point; it is not a claim that one design is better for every model quantity.

Assumptions, Diagnostics, and Limitations

Before interpreting an effective-information calculation, check that the matrix is a Fisher information matrix evaluated at a clearly specified model and parameter point, and that the partition matches the scientific question. The local covariance interpretation also relies on regular likelihood behavior and an adequate large-sample approximation. In nonlinear models, the information and resulting Schur complement can change with the nominal parameter values, so a calculation at one point is not automatically valid elsewhere.

Check symmetry and definiteness. A Fisher information matrix should be symmetric and positive semidefinite. Small numerical asymmetry may arise from computation, but substantial asymmetry or a negative effective-information value can indicate an implementation or numerical error.
Check the nuisance block. The displayed formula requires $\mathcal I_{\eta\eta}$ to be invertible. If it is singular, the stated inverse-based calculation is not defined. Investigate redundancy or non-identifiability, or reformulate the estimable parameter space before proceeding.
Watch for near-singularity. An invertible but ill-conditioned nuisance block can make the result highly sensitive to rounding and numerical error. Report adequate precision and use stable linear-system methods rather than explicitly forming an inverse in software.
Interpret a small Schur complement carefully. It signals weak local information about the target after nuisance adjustment. It does not, by itself, diagnose which model component caused the weakness or guarantee that a large-sample approximation will be accurate.

A common alternative is to invert the full information matrix and read the target variance from its corresponding diagonal block. Under the stated invertibility assumptions this gives the same result, because that block equals $\mathcal I_{\psi\cdot\eta}^{-1}$. The Schur complement is often easier to interpret as adjusted information, while full inversion can be convenient when uncertainties for all parameters are needed. Treating nuisance parameters as fixed instead is not an equivalent alternative: it ignores the uncertainty and coupling associated with estimating them.

The method is local and model-dependent. It does not remove uncertainty from model misspecification, make a poorly identified model estimable, or replace checking the likelihood approximation. Its role is precise: for a specified partition and a regular local information matrix, it quantifies the target curvature remaining after nuisance directions are allowed to adjust.

Check Your Understanding

Answer each question using the definitions and calculations in this tutorial. Solutions follow each question.

For $\mathcal I=\begin{pmatrix}25&5\\5&10\end{pmatrix}$, with the first parameter of interest and the second nuisance, calculate the effective information and approximate standard error. Solution: The nuisance block is $10$, so $\mathcal I_{\psi\cdot\eta}=25-5^2/10=22.50$. The approximate standard error is $1/\sqrt{22.50}=0.2108$ in the standardized parameter units.
Why can $\mathcal I_{\psi\psi}$ overstate information about the target when nuisance parameters are unknown? Solution: The target and nuisance parameters may be coupled through $\mathcal I_{\psi\eta}$. Nuisance changes can partly offset target changes in the local likelihood, so the effective information subtracts the nonnegative adjustment $\mathcal I_{\psi\eta}\mathcal I_{\eta\eta}^{-1}\mathcal I_{\eta\psi}$ when the nuisance block is positive definite.
A nuisance block is singular. May the inverse-based Schur complement formula be applied as written? Solution: No. $\mathcal I_{\eta\eta}^{-1}$ does not exist, so this formula is undefined. The parameterization or identifiable nuisance space must be examined before using an appropriate reformulation.
Two designs have target-block information 50 and 45, but effective information 12 and 20, respectively. Which gives the smaller approximate standard error for the target, and what are the values? Solution: The second design gives the smaller standard error: $1/\sqrt{20}=0.2236$, compared with $1/\sqrt{12}=0.2887$, in the standardized parameter units. The raw target-block entries alone would suggest the opposite ranking.