Adjusting Design Information for Nuisance Parameters

Why nuisance parameters change design information

A design can produce precise measurements of a primary parameter when other model parameters are held fixed, yet provide much less precision when those parameters must also be estimated. The issue is not whether the other parameters are scientifically unimportant. It is whether their effects on the observations can be distinguished from the effect of the parameter of interest. In a nonlinear mixed-effects (NLME) model, a design objective that ignores this distinction can overstate the information available for the scientific question.

This tutorial considers fixed effects divided into two groups: parameters of interest, denoted by the vector $\psi$, and nuisance parameters, denoted by the vector $\lambda$. Nuisance parameters are estimated in the model but are not the primary targets of the design objective. They may include other structural or covariate effects. Random effects and residual variability still enter the likelihood and the information calculation; the partition here concerns the fixed-effect parameters being estimated.

Definition: Write the Fisher information matrix for $(\psi,\lambda)$ in block form as $$ \mathcal I = \begin{pmatrix} \mathcal I_{\psi\psi} & \mathcal I_{\psi\lambda}\\ \mathcal I_{\lambda\psi} & \mathcal I_{\lambda\lambda} \end{pmatrix}. $$ When $\mathcal I_{\lambda\lambda}$ is invertible, the information for $\psi$ adjusted for simultaneous estimation of $\lambda$ is the Schur complement $$ \mathcal I_{\psi\mid\lambda} = \mathcal I_{\psi\psi} - \mathcal I_{\psi\lambda} \mathcal I_{\lambda\lambda}^{-1} \mathcal I_{\lambda\psi}. $$

The unadjusted block $\mathcal I_{\psi\psi}$ describes curvature in the $\psi$ directions if $\lambda$ were fixed. The subtracted term accounts for the extent to which changes in $\psi$ can be offset by changes in $\lambda$. Thus adjusted information is not generally obtained by simply selecting the rows and columns for the parameters of interest.

Derivation and interpretation

The Schur complement follows from the local quadratic approximation to the log-likelihood. Near its maximum, let $\delta\psi$ and $\delta\lambda$ be small changes in the two parameter groups. Under the usual regularity conditions for likelihood-based information, the negative quadratic change in log-likelihood is approximately

\frac{1}{2} \begin{pmatrix}\delta\psi\\\delta\lambda\end{pmatrix}^{\!T} \mathcal I \begin{pmatrix}\delta\psi\\\delta\lambda\end{pmatrix} = \frac{1}{2}\left[ \delta\psi^T\mathcal I_{\psi\psi}\delta\psi +2\delta\psi^T\mathcal I_{\psi\lambda}\delta\lambda +\delta\lambda^T\mathcal I_{\lambda\lambda}\delta\lambda \right].

To assess curvature for a change in $\psi$ when $\lambda$ is estimated, consider the nuisance-parameter change that makes the likelihood fit best for that $\delta\psi$. Assuming $\mathcal I_{\lambda\lambda}$ is positive definite, differentiate the expression inside the brackets with respect to $\delta\lambda$ and set the result to zero:

2\mathcal I_{\lambda\psi}\delta\psi +2\mathcal I_{\lambda\lambda}\delta\lambda=0, \qquad \delta\lambda^* = -\mathcal I_{\lambda\lambda}^{-1}\mathcal I_{\lambda\psi}\delta\psi.

Substitute this best-fitting nuisance change into the quadratic form. The cross-term and nuisance-only term together contribute $-\delta\psi^T\mathcal I_{\psi\lambda}\mathcal I_{\lambda\lambda}^{-1}\mathcal I_{\lambda\psi}\delta\psi$. The resulting curvature for $\delta\psi$ is therefore

\frac{1}{2}\delta\psi^T \left( \mathcal I_{\psi\psi} -\mathcal I_{\psi\lambda}\mathcal I_{\lambda\lambda}^{-1} \mathcal I_{\lambda\psi} \right)\delta\psi.

This establishes the adjusted-information formula. Equivalently, under a regular local likelihood approximation, the $\psi$-by-$\psi$ block of the covariance matrix for the joint estimator is $\mathcal I_{\psi\mid\lambda}^{-1}$. If $\lambda$ were known, the corresponding covariance approximation would instead be $\mathcal I_{\psi\psi}^{-1}$. Estimating nuisance parameters therefore cannot increase the local information about $\psi$: the subtracted matrix is positive semidefinite, so $\mathcal I_{\psi\mid\lambda}$ is no greater than $\mathcal I_{\psi\psi}$ in the positive-semidefinite ordering.

Conditions: The information matrix must describe the same likelihood or approximation used for the design, evaluated at specified parameter values or averaged over them. The likelihood should be sufficiently regular for the local information approximation to be meaningful. For the displayed inverse formula, $\mathcal I_{\lambda\lambda}$ must be invertible; for finite local variances of all parameters of interest, $\mathcal I_{\psi\mid\lambda}$ must also be positive definite. A singular or nearly singular matrix signals non-identification or weak identification, not a value to be silently inverted.

Worked Example: One Parameter of Interest and One Nuisance Effect

Suppose a hypothetical design has local information

\mathcal I= \begin{pmatrix} 100&30\\ 30&25 \end{pmatrix},

where the first parameter is the parameter of interest and the second is a nuisance fixed effect. The entries use the corresponding parameter scales, and the matrix is positive definite because its leading entry is positive and its determinant is $100(25)-30^2=1600>0$.

If the nuisance parameter were known, the information for the target would be $100$. When it is estimated, the adjusted information is

\mathcal I_{\psi\mid\lambda} =100-\frac{30^2}{25} =100-36=64.

The local variance approximation for the target is consequently $1/64=0.01563$, rather than $1/100=0.01000$ under the unrealistic assumption that the nuisance effect is known. If the target is expressed on a log scale, this variance is in squared log-scale units. The loss of information arises from the nonzero cross-information $30$: part of the pattern associated with changing the target can be explained by changing the nuisance effect.

Several parameters of interest

When $\psi$ contains multiple parameters, the Schur complement is a matrix. It captures both the adjusted information for each parameter and the adjusted dependence among them. Design criteria for the parameters of interest should be applied to this adjusted matrix. For example, a determinant-based objective can use $\det(\mathcal I_{\psi\mid\lambda})$ to assess joint information, while an average-variance objective can use $\operatorname{tr}(\mathcal I_{\psi\mid\lambda}^{-1})$, provided the adjusted matrix is positive definite. The interpretation of a criterion must match its parameter scales and the scientific objective.

Worked Example: Two Targets With One Nuisance Parameter

Consider a hypothetical information matrix with two parameters of interest and one nuisance parameter:

\mathcal I= \begin{pmatrix} 16&4&6\\ 4&9&3\\ 6&3&9 \end{pmatrix}.

The target block is $A=\begin{pmatrix}16&4\\4&9\end{pmatrix}$, the target-to-nuisance block is $B=\begin{pmatrix}6\\3\end{pmatrix}$, and the nuisance information is $C=9$. The adjusted matrix is $A-BC^{-1}B^T$:

\mathcal I_{\psi\mid\lambda} = \begin{pmatrix}16&4\\4&9\end{pmatrix} - \frac{1}{9} \begin{pmatrix}6\\3\end{pmatrix} \begin{pmatrix}6&3\end{pmatrix} = \begin{pmatrix}16&4\\4&9\end{pmatrix} - \begin{pmatrix}4&2\\2&1\end{pmatrix} = \begin{pmatrix}12&2\\2&8\end{pmatrix}.

The determinant of the unadjusted target block is $16(9)-4^2=128$; the determinant after adjustment is $12(8)-2^2=92$. Both matrices are positive definite, but the adjusted determinant is smaller, reflecting the information lost to joint estimation. The adjusted covariance approximation is

\mathcal I_{\psi\mid\lambda}^{-1} = \frac{1}{92} \begin{pmatrix}8&-2\\-2&12\end{pmatrix}.

For instance, the first target's approximate variance is $8/92=0.0870$ in squared units of that parameter. It would be incorrect to infer the target variances from the inverse of the unadjusted $2\times2$ block when the nuisance parameter is estimated simultaneously.

Applying the adjustment to an NLME design

In an NLME design calculation, the information matrix may come from a marginal likelihood or from a stated approximation to it. It reflects the specified structural model, random-effects distribution, residual model, planned observations, and nominal parameter values. Once that full information has been calculated, the adjustment is a block-matrix operation: identify the target and nuisance fixed effects, partition the matrix consistently, and form the Schur complement. The adjustment does not repair a misspecified likelihood or make a local approximation reliable far from the nominal values.

Worked Example: Comparing Two Hypothetical NLME Designs

Suppose two candidate sampling designs are evaluated at the same nominal parameter values using the same NLME likelihood approximation. The target is a population disposition parameter; a baseline effect is treated as a nuisance fixed effect. Assume both designs have the same number of subjects and comparable observation conditions. Their hypothetical information matrices, in target-then-nuisance order, are

\mathcal I_A= \begin{pmatrix}50&30\\30&20\end{pmatrix}, \qquad \mathcal I_B= \begin{pmatrix}40&10\\10&5\end{pmatrix}.

Both matrices are positive definite: their determinants are $50(20)-30^2=100$ and $40(5)-10^2=100$, respectively. If the baseline effect were incorrectly treated as known, design A would appear preferable because its target-block information is $50$, compared with $40$ for design B. With joint estimation, the adjusted information is

\mathcal I_{\psi\mid\lambda,A} =50-\frac{30^2}{20} =5, \qquad \mathcal I_{\psi\mid\lambda,B} =40-\frac{10^2}{5} =20.

The adjusted comparison favors design B: its target information is four times that of design A after accounting for the baseline effect. The corresponding local target variance approximations are $1/5=0.2000$ and $1/20=0.05000$, in squared units of the target parameter. Design A has more conditional target information but also much stronger target–baseline confounding. These calculations are hypothetical; they illustrate why the nuisance block and cross-block must be retained when assessing an NLME design.

Diagnostics, alternatives, and limitations

Before interpreting an adjusted-information result, check the matrix and its partition. The parameter order must match the block labels, and every block must use the same parameterization and likelihood approximation. A useful diagnostic is to inspect the eigenvalues or condition number of the full and adjusted matrices. A very small eigenvalue indicates a weakly informed direction; in that situation, a scalar summary such as a determinant can conceal an important practical weakness. For a single target and nuisance parameter, a large cross-information relative to the diagonal blocks is a warning that the two effects may be difficult to distinguish.

Three common approaches answer different questions:

Approach	What it represents	Key caution
Use $\mathcal I_{\psi\psi}$ alone	Information for targets conditional on nuisance parameters being fixed	Usually optimistic if nuisance parameters are estimated
Use $\mathcal I_{\psi\mid\lambda}$	Local information for targets with nuisance parameters estimated jointly	Depends on the full information and the local model assumptions
Use the full joint matrix	Information for all fixed effects, including targets and nuisance effects	A joint objective may reward precision in nuisance effects even when that is not the design goal

The Schur complement is a local, likelihood-based adjustment, not a universal guarantee of good finite-sample estimation. It can be sensitive to nominal parameter values, the chosen likelihood approximation, and the parameterization. Re-expressing parameters changes information scales, so designs should be compared using a consistent parameterization and a criterion aligned with the scientific question. If nuisance information is singular, the ordinary inverse in the formula does not exist. A generalized inverse may be considered only with a careful treatment of identifiable directions; it is not a routine substitute for diagnosing the design.

Common pitfall: “The target block has the largest information, so this design estimates the target best.” That conclusion assumes the other fixed effects are known. When they are estimated, changes in nuisance parameters can mimic changes in the target. Compute $\mathcal I_{\psi\mid\lambda}$, check that it is well-conditioned enough to interpret, and compare designs using the adjusted information relevant to the target.

Key takeaway: When nuisance fixed effects are estimated simultaneously with target parameters, the relevant local information is the Schur complement, not the target block alone. The cross-information quantifies how much target information is lost because nuisance effects can explain similar changes in the data.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

For a scalar target and scalar nuisance effect, explain why a nonzero cross-information reduces adjusted information.
If $\mathcal I_{\psi\psi}=36$, $\mathcal I_{\psi\lambda}=12$, and $\mathcal I_{\lambda\lambda}=8$, calculate the adjusted target information.
For two targets, what matrix operation adjusts the target block for one or more nuisance parameters?
Why is a singular nuisance-information block a warning rather than a reason to apply the ordinary Schur-complement formula?

Solutions. (1) The adjustment subtracts $\mathcal I_{\psi\lambda}^2/\mathcal I_{\lambda\lambda}$, which is positive when cross-information is nonzero and nuisance information is positive. (2) $36-12^2/8=36-18=18$. (3) Subtract $\mathcal I_{\psi\lambda}\mathcal I_{\lambda\lambda}^{-1}\mathcal I_{\lambda\psi}$ from $\mathcal I_{\psi\psi}$. (4) The required inverse does not exist, indicating that some nuisance direction is not locally identified; the ordinary formula is therefore undefined, and the design or identifiable parameterization needs investigation.

Approach	What it represents	Key caution
Use \(\mathcal I_{\psi\psi}\) alone	Information for targets conditional on nuisance parameters being fixed	Usually optimistic if nuisance parameters are estimated
Use \(\mathcal I_{\psi\mid\lambda}\)	Local information for targets with nuisance parameters estimated jointly	Depends on the full information and the local model assumptions
Use the full joint matrix	Information for all fixed effects, including targets and nuisance effects	A joint objective may reward precision in nuisance effects even when that is not the design goal