Why Information Depends on What Is Known
A Fisher information matrix describes how strongly the likelihood responds to changes in all model parameters together. The information for one parameter therefore depends not only on the curvature in that parameter’s direction, but also on whether changes in other parameters can produce similar changes in the likelihood. If those other parameters are known exactly, they cannot compete with the parameter of interest. If they must be estimated, their uncertainty and their association with the target parameter reduce the precision available for the target.
This distinction matters in model-informed design. A design may appear highly informative about a parameter when other parameters are treated as fixed, yet provide substantially less precision when the full model is estimated. The introductory material on nuisance parameters and Schur complements gives the block-matrix context; here the emphasis is on interpreting and comparing the two kinds of information.
The information blocks are evaluated at the same parameter value and under the same observation model. The target can be scalar or vector-valued; if \(\psi\) is scalar, \(I_{\psi\mid\lambda}\) is a number, while for a vector target it is a matrix. The nuisance block must be invertible for the displayed formula. In the regular, locally identifiable case, the full information matrix is positive definite, which ensures that the nuisance block and its Schur complement are positive definite as well.
The word “marginal” is sometimes used for the precision of the target parameter when the other parameters are estimated. Here it does not mean integrating nuisance parameters out of a Bayesian model or forming a marginal data distribution. It refers to the target’s covariance block in the joint parameter-estimation problem.
Why the Effective Information Is Smaller
The reduction has a score-based interpretation. Let \(U_\psi\) and \(U_\lambda\) denote the score vectors for the target and nuisance parameters. Under the usual regularity conditions, their covariance blocks equal the expected Fisher information blocks. If \(\lambda\) is known, the target score \(U_\psi\) is available without needing to account for uncertainty in a nuisance estimate. If \(\lambda\) is estimated, the part of \(U_\psi\) predictable from \(U_\lambda\) cannot cleanly distinguish a target-parameter change from a nuisance-parameter change.
The score component that remains after removing its linear association with the nuisance score is \(U_{\mathrm{eff}}=U_\psi-I_{\psi\lambda}I_{\lambda\lambda}^{-1}U_\lambda\). Its covariance is the effective information. Expanding that covariance gives the result:
The two negative terms arise from the cross-covariances between target and nuisance scores; the final positive term is the variance of the subtracted nuisance-score component. Together they leave the Schur complement. Equivalently, block inversion shows that the \(\psi\)-by-\(\psi\) block of the joint inverse information matrix is \(I_{\psi\mid\lambda}^{-1}\). Thus, in regular large-sample likelihood theory, the approximate covariance of the target estimate when nuisance parameters are estimated is \(I_{\psi\mid\lambda}^{-1}\), not \(I_{\psi\psi}^{-1}\).
For a scalar target, \(I_{\psi\lambda}I_{\lambda\lambda}^{-1}I_{\lambda\psi}\) is nonnegative, so effective information cannot exceed conditional information. In the vector case, the subtracted matrix is positive semidefinite: for any vector \(a\), its quadratic form is \((I_{\lambda\psi}a)^\mathsf{T}I_{\lambda\lambda}^{-1}(I_{\lambda\psi}a)\geq 0\). The reduction is zero only when the target and nuisance scores are locally uncorrelated in the relevant directions.
Worked Example: One Target and One Nuisance Parameter
Suppose a hypothetical local analysis uses dimensionless log parameters \(\psi\), a target log-clearance parameter, and \(\lambda\), a nuisance log-volume parameter. Its expected information matrix is $$ I= \begin{pmatrix} 100 & 60\\ 60 & 100 \end{pmatrix}. $$ Assume the matrix is evaluated at the specified nominal parameters under a regular model. It is positive definite because its leading diagonal entry is positive and its determinant is \(100(100)-60^2=6{,}400>0\).
If \(\lambda\) is known, the conditional information for \(\psi\) is \(I_{\psi\psi}=100\). The corresponding information-based standard error is \(1/\sqrt{100}=0.1000\) log units.
If \(\lambda\) is estimated, the effective information is $$ I_{\psi\mid\lambda} =100-\frac{60(60)}{100} =100-36 =64. $$ The corresponding standard error is \(1/\sqrt{64}=0.1250\) log units. Estimating the nuisance parameter therefore increases the approximate standard error from 0.1000 to 0.1250 log units in this hypothetical example. The information falls because the nonzero cross-information makes changes in log clearance partly difficult to distinguish from changes in log volume.
Reading the Reduction in a Larger Partition
With several nuisance parameters, the same formula accounts for their joint information and their associations with the target. It is important to use the inverse of the entire nuisance block, rather than adjust separately for each nuisance parameter while ignoring correlations among them. The inverse block reflects how well the nuisance directions can be distinguished from one another.
Worked Example: A Target With Two Nuisance Parameters
Consider a hypothetical model with target \(\psi\), a log-clearance parameter, and nuisance parameters \(\lambda_1\) and \(\lambda_2\), representing log-absorption and log-residual-scale parameters. At a regular nominal parameter point, suppose the expected information matrix is $$ I= \begin{pmatrix} 120 & 20 & 30\\ 20 & 80 & 0\\ 30 & 0 & 50 \end{pmatrix}. $$ The nuisance block is \(I_{\lambda\lambda}=\begin{pmatrix}80&0\\0&50\end{pmatrix}\), which is invertible, and the target-to-nuisance block is \(I_{\psi\lambda}=(20\;\;30)\). The matrix is positive definite because the nuisance block is positive definite and the Schur complement computed below is positive.
If both nuisance parameters are known, conditional information is \(I_{\psi\psi}=120\), giving an information-based standard error of \(1/\sqrt{120}=0.09129\) log units. If both are estimated, then $$ \begin{aligned} I_{\psi\mid\lambda} &=120- (20\;\;30) \begin{pmatrix}1/80&0\\0&1/50\end{pmatrix} \begin{pmatrix}20\\30\end{pmatrix}\\ &=120-\left(\frac{20^2}{80}+\frac{30^2}{50}\right)\\ &=120-(5+18)=97. \end{aligned} $$ The effective-information standard error is \(1/\sqrt{97}=0.1015\) log units. Thus the conditional calculation overstates the target’s information when both nuisance parameters must be estimated. Although the nuisance parameters have zero cross-information with each other in this example, each is associated with the target and contributes to the reduction.
Same Conditional Information, Different Effective Information
The diagonal entry \(I_{\psi\psi}\) alone is not enough to compare designs or model configurations for target precision. A design can have the same target-direction curvature as another design and still have a different effective information because the cross-information differs. The next hypothetical comparison isolates this point.
Worked Example: Comparing Two Local Information Matrices
Suppose two hypothetical designs each give conditional information 100 for a scalar target \(\psi\), and information 100 for one nuisance parameter \(\lambda\). Their local expected information matrices are $$ I_A=\begin{pmatrix}100&0\\0&100\end{pmatrix}, \qquad I_B=\begin{pmatrix}100&80\\80&100\end{pmatrix}. $$ Both are positive definite: the determinants are 10,000 and 3,600, respectively, and their diagonal entries are positive.
For design A, effective information is \(100-0^2/100=100\), so the approximate target standard error is \(1/\sqrt{100}=0.1000\). For design B, it is \(100-80^2/100=36\), so the standard error is \(1/\sqrt{36}=0.1667\). The conditional information is identical, but design B has stronger target–nuisance coupling and substantially less effective information. This illustrates why a comparison based only on \(I_{\psi\psi}\) can give an incomplete account of target precision.
Assumptions, Diagnostics, and Alternatives
These calculations concern expected Fisher information, usually evaluated locally at a specified parameter value. Their interpretation as approximate estimator covariance relies on a correctly specified, sufficiently regular likelihood and the usual large-sample likelihood approximation. Parameters should be locally identifiable, and the information matrix should be nonsingular for the parameterization being used. In nonlinear pharmacometric models, information can vary with the nominal parameter values, so a numerical result describes that local setting rather than guaranteeing the same precision throughout the parameter space.
A practical diagnostic is to inspect the blocks, the Schur complement, and the resulting standard errors together. A very small or negative computed Schur complement can signal near-nonidentifiability, numerical instability, an invalid information matrix, or an implementation error. Under a positive-definite full information matrix, the exact Schur complement is positive definite; a negative value is not a legitimate “negative amount of information.” Also check that the nuisance block is invertible and that all blocks come from the same parameter point and likelihood.
For scalar \(\psi\) and \(\lambda\), the reduction can be written as \(I_{\psi\lambda}^2/I_{\lambda\lambda}\). It is especially severe when cross-information is large relative to the information available for the nuisance parameter. This form also helps diagnose unstable calculations: a nuisance parameter with little information can have a large effect if its cross-information with the target remains substantial. Rescaling parameters changes numerical entries, so comparisons of matrix entries require a consistent parameterization; the corresponding uncertainty for a clearly defined target quantity is the more relevant interpretation.
| Approach | What it represents | When it is useful |
|---|---|---|
| Conditional information \(I_{\psi\psi}\) | Target information with nuisance parameters fixed as known | To describe a genuinely fixed-nuisance problem or an optimistic benchmark |
| Effective information \(I_{\psi\mid\lambda}\) | Target information after accounting for nuisance estimation | To assess target precision in the joint estimation problem |
| Target block of \(I^{-1}\) | Approximate target covariance when all parameters are estimated | To report uncertainty directly; it equals \(I_{\psi\mid\lambda}^{-1}\) |
Conditional information is appropriate only when nuisance parameters are in fact fixed at known values, or when the question explicitly asks for that hypothetical comparison. Treating estimated nuisance parameters as known generally understates uncertainty. Conversely, the effective-information calculation is an asymptotic local approximation; it does not by itself resolve small-sample bias, parameter boundaries, model misspecification, or uncertainty about the nominal parameter values. Profile-likelihood curvature provides a related likelihood-based way to examine target precision, while Bayesian marginal inference is a different procedure that also depends on prior distributions.
Common Mistakes
- Using the target diagonal as the answer in a joint estimation problem. That gives conditional information. Subtract the nuisance-adjustment term to obtain effective information.
- Calling the reduction a probability or a percentage without defining the comparison. It is a difference between information matrices; standard errors change through the inverse square root in a scalar problem.
- Inverting only diagonal entries of a correlated nuisance block. Use the inverse of the full \(I_{\lambda\lambda}\) block so that nuisance-parameter dependence is handled correctly.
- Interpreting a negative computed effective information as meaningful. With a valid positive-definite full information matrix, the Schur complement is positive. Recheck assumptions, matrix construction, and numerical precision.
- Equating “nuisance parameters estimated” with Bayesian marginalization. The effective Fisher information here is a frequentist likelihood-based quantity; it does not integrate over a prior distribution.
Check Your Understanding
For each problem, first identify whether it concerns conditional or effective information. Then compare your result with the worked solutions below.
- A hypothetical information matrix for scalar \(\psi\) and \(\lambda\) is \(\begin{pmatrix}80&40\\40&50\end{pmatrix}\). Find both kinds of information and the corresponding target standard errors.
- For \(I_{\psi\psi}=150\), \(I_{\psi\lambda}=(10\;\;20)\), and \(I_{\lambda\lambda}=\begin{pmatrix}50&0\\0&100\end{pmatrix}\), calculate effective information for the scalar target.
- Two designs have the same \(I_{\psi\psi}\), but one has larger target–nuisance cross-information and the same nuisance information. What is the expected direction of the change in effective information?
- Why is a negative Schur complement a warning rather than a valid conclusion that the target has negative information?
Solutions.
- Conditional information is \(80\), with standard error \(1/\sqrt{80}=0.1118\). Effective information is \(80-40^2/50=80-32=48\), with standard error \(1/\sqrt{48}=0.1443\). The matrix is positive definite because its determinant is \(80(50)-40^2=2{,}400>0\).
- The nuisance block is invertible. The adjustment is \(10^2/50+20^2/100=2+4=6\). Effective information is \(150-6=144\), which is positive; its information-based standard error is \(1/\sqrt{144}=0.08333\) in the target’s units.
- With the same positive nuisance information, larger cross-information increases the nonnegative adjustment term. Effective information therefore decreases, provided the compared matrices remain valid information matrices.
- A positive-definite full information matrix has a positive-definite Schur complement. A negative computed value indicates that the matrix may not be valid or consistently assembled, that an inversion or arithmetic step is wrong, or that numerical instability is present.