Fisher Information for Population Pharmacokinetic Designs

Why Fisher Information Matters for a Population Design

A proposed population pharmacokinetic design determines which observations could be collected: the number of subjects, their sampling times, and the measurements available from each subject. Before data are observed, it is useful to ask how informative that design is expected to be about the model parameters. Fisher information gives a model-based, local measure of that expected information. It is calculated from the probability distribution of possible data under the design, not from a particular realized data set.

For an NLME model, the distribution relevant to population-parameter estimation is ordinarily the marginal distribution of a subject’s observations. It averages over that subject’s random effects. This distinction matters: a calculation based on the conditional likelihood, as if each subject’s random effects were known, does not in general describe the information available for estimating population parameters from the observed data.

Definition: Let \(y_i\) denote the observations from subject \(i\), \(d_i\) that subject’s design (including sampling times and doses), and \(\theta\) the vector of population parameters. The marginal density is \(f_i(y_i\mid\theta,d_i)=\int f_i(y_i\mid\eta_i,\theta,d_i)g(\eta_i\mid\theta)\,d\eta_i\), where \(\eta_i\) is the subject-level random effect and \(g\) its population distribution. For independent subjects, the marginal log likelihood is \(\ell(\theta)=\sum_i\log f_i(y_i\mid\theta,d_i)\). The marginal score is \(s(\theta)=\partial\ell(\theta)/\partial\theta\). The expected Fisher information is \(\mathcal I(\theta)=E_\theta[s(\theta)s(\theta)^\mathsf{T}]\), with expectation over data generated from the marginal model at the specified design and parameter value.

The expectation is taken at a specified, usually nominal, value of \(\theta\). Thus expected information is design- and model-dependent, and in nonlinear models it can change with the nominal parameter values. It is not a probability distribution for \(\theta\), nor does it guarantee that a realized study will be informative.

Expected Information and Observed Information

For a realized data set, the observed information is the negative Hessian of the marginal log likelihood: \(J(\theta;y)=-\partial^2\ell(\theta)/\partial\theta\partial\theta^\mathsf{T}\). It describes the curvature of the likelihood at the particular data set and parameter value. Expected information averages this curvature over possible data sets: \(\mathcal I(\theta)=E_\theta[J(\theta;Y)]\), under the regularity conditions below. The two are related, but they are not interchangeable: observed information varies with the realized data, whereas expected information is a prospective model-based quantity.

The score-outer-product and negative-expected-Hessian definitions agree when the marginal density is sufficiently smooth, its support does not depend on \(\theta\), differentiation can be interchanged with integration, and the relevant expectations are finite. In particular, the model must be normalized and differentiable in a way that permits the calculations below.

The expected score is zero.
For a normalized marginal density, \(\int f(y\mid\theta)\,dy=1\). Differentiating under the integral gives \(\int \partial f(y\mid\theta)/\partial\theta\,dy=0\). Since \(s=(\partial f/\partial\theta)/f\), this is \(E_\theta[s(\theta)]=0\).

Differentiate the score.
For one marginal density, \(\partial^2\log f/\partial\theta\partial\theta^\mathsf{T}=(\partial^2 f/\partial\theta\partial\theta^\mathsf{T})/f-ss^\mathsf{T}\).

Take expectations.
Integrating the first term against \(f\) gives \(\int \partial^2 f/\partial\theta\partial\theta^\mathsf{T}\,dy=0\), by differentiating the normalized integral twice. Therefore \(E_\theta[\partial^2\log f/\partial\theta\partial\theta^\mathsf{T}]=-E_\theta[ss^\mathsf{T}]\). Negating both sides yields the equality of the two information definitions.

If these conditions fail—for example, at a parameter boundary or under a nonregular model—the equality or usual likelihood-based approximations may fail. One should not assume that a numerical Hessian is automatically a valid information matrix.

What the Matrix Says About Precision

For a parameter vector \(\theta\), information is a matrix. Its diagonal entries describe curvature in individual coordinate directions, while off-diagonal entries describe cross-curvature: how changes in one parameter direction interact locally with changes in another. The entries depend on parameter units, so their magnitudes should not be compared across differently scaled parameters without care.

Under regularity and adequate information, the maximum likelihood estimator is approximately multivariate normal in sufficiently large samples, with covariance matrix approximately \(\mathcal I(\theta)^{-1}\). In practice, this is a local approximation, often evaluated at an estimated or nominal parameter value. The square roots of the diagonal entries of the inverse provide approximate standard errors. Parameter dependence is read from the covariance or correlation matrix derived from that inverse—not from the off-diagonal information entry alone.

Key interpretation: A well-conditioned, nonsingular information matrix indicates that the likelihood has curvature in all parameter directions locally. A small eigenvalue, or a nearly singular matrix, indicates a weakly informed combination of parameters. The inverse information is an approximate covariance matrix only under the stated likelihood and asymptotic assumptions; it is not an exact finite-sample guarantee.

Worked Example: Marginal Information for Log Concentration and Elimination Rate

Suppose, purely hypothetically, four different subjects each contribute one log-concentration observation at times \(t=0,1,2,3\) hours. After integrating over each subject’s random effect, assume the marginal model is \(z_i=\alpha-kt_i+\epsilon_i\), with independent \(\epsilon_i\sim N(0,0.25)\). Here \(z_i\) is log concentration relative to a fixed reference concentration, \(\alpha\) is the population log-concentration intercept, and \(k\) is the elimination rate in inverse hours. The marginal variance is treated as known and constant. Under this simplified Gaussian marginal model, the design and distribution assumptions needed for the calculation are explicit.

For a normal observation with known variance \(0.25\) and mean \(\alpha-kt_i\), the score contribution is the derivative of its mean, \((1,-t_i)^\mathsf{T}\), multiplied by the residual and divided by \(0.25\). Since the residual has mean zero and variance \(0.25\), the expected information contribution is the outer product of the mean-derivative vector divided by \(0.25\). Summing the four independent contributions gives

\mathcal I(\alpha,k) =\frac{1}{0.25}\sum_{t\in\{0,1,2,3\}} \begin{pmatrix}1\\-t\end{pmatrix} \begin{pmatrix}1&-t\end{pmatrix} =4\begin{pmatrix}4&-6\\-6&14\end{pmatrix} =\begin{pmatrix}16&-24\\-24&56\end{pmatrix}.

The determinant is \(16(56)-(-24)^2=896-576=320\), so the matrix is nonsingular. Its inverse is

\mathcal I(\alpha,k)^{-1} =\frac{1}{320}\begin{pmatrix}56&24\\24&16\end{pmatrix} =\begin{pmatrix}0.175&0.075\\0.075&0.050\end{pmatrix}.

The local large-sample approximation therefore gives standard errors \(\sqrt{0.175}=0.4183\) for \(\alpha\) and \(\sqrt{0.050}=0.2236\) inverse hours for \(k\). The implied approximate correlation is \(0.075/\sqrt{(0.175)(0.050)}=0.8018\), positive. The negative off-diagonal entry of the information matrix does not mean the parameter estimates are negatively correlated: dependence is determined from the inverse information. These values are local approximations under the specified model, not exact uncertainty statements for four observations.

Further Examples and Practical Comparisons

Worked Example: Expected and Observed Information Can Differ

Consider a hypothetical marginal model for two log-concentration residuals \(r_1=1\) and \(r_2=\sqrt{2}\), measured in log-concentration units. Assume independent \(N(0,v)\) residuals, where \(v\) is an unknown variance parameter. This is a regular normal model for \(v>0\), and the residual sum of squares is \(S=r_1^2+r_2^2=1+2=3\).

Ignoring terms that do not depend on \(v\), the log likelihood is \(\ell(v)=-\frac{n}{2}\log v-\frac{S}{2v}\). Differentiating twice gives observed information \(J(v)=-\ell''(v)=-\frac{n}{2v^2}+\frac{S}{v^3}\). At \(n=2\) and \(v=1\), the observed information is \(-2/(2\cdot1^2)+3/1^3=2\), in inverse squared variance units.

For data generated under this model, \(E_v[S]=nv\). Therefore expected information is \(-n/(2v^2)+E_v[S]/v^3=n/(2v^2)\), which equals \(1\) at \(n=2,v=1\). The realized data yield observed information \(2\), while the model expectation is \(1\). This illustrates why expected information is useful before data collection and why observed information should be calculated from, not substituted for, the prospective expectation.

Worked Example: Information Adds Across Independent Subjects

Suppose a hypothetical study has five independent subjects, each contributing one marginal log concentration \(z_i\sim N(\alpha,0.04)\) at a common fixed design condition. Treat the variance as known and the model assumptions as correct. For one subject, the score for \(\alpha\) is \((z_i-\alpha)/0.04\), so its expected squared score is \(0.04/(0.04)^2=25\). Independence makes the total log likelihood a sum, and its expected information is \(5(25)=125\) in inverse squared log-concentration units.

The inverse information is \(1/125=0.008\), so the approximate standard error for \(\alpha\) is \(\sqrt{0.008}=0.08944\) log-concentration units. The addition of subject information follows from independent marginal likelihood contributions. It does not mean every additional observation contributes the same amount in a nonlinear model: contributions can differ with sampling time, parameter values, and the marginal distribution.

Expected information is not the only way to describe precision. Observed information uses the curvature at the actual data and is commonly used for local uncertainty estimates after fitting. Profile likelihood examines likelihood changes along a parameter of interest while other parameters are re-optimized; it can reveal asymmetry or nonquadratic shape that an information-based normal approximation misses. Resampling methods address repeated-sampling variability through simulated or resampled data rather than solely through local curvature. These approaches answer related but not identical questions.

Diagnostics, Limitations, and Common Pitfalls

A proposed information calculation should be checked against both its mathematics and the model that produced it. Verify that the marginal likelihood integrates over random effects, that independent-subject contributions are summed only when subjects are independent under the model, and that derivatives are taken with respect to the stated parameterization. Check symmetry and positive semidefiniteness of an expected score-outer-product matrix; a substantial numerical violation can signal inaccurate derivatives, integration error, or an implementation problem. A singular or ill-conditioned matrix signals that some parameter combinations are weakly informed locally, although it does not by itself identify the source of the difficulty.

Information calculations are often local: nonlinear likelihood curvature can change across parameter values. A design evaluated at one nominal value may therefore perform differently if the true values differ. Approximate standard errors based on inverse information can also be misleading with small samples, boundary estimates, weak identification, strong nonlinearity, or a misspecified random-effects or residual model. Expected information summarizes the implications of the assumed model; it cannot compensate for an implausible model.

Confusing expected with observed information. Expected information averages over potential marginal data sets at a fixed design and parameter value. Observed information is computed from the realized data.
Reading the information off-diagonal as a correlation. It is a cross-curvature entry. Approximate covariance and correlation come from the inverse information.
Using conditional information as if random effects were observed. Population inference from observed concentrations uses the marginal likelihood unless the individual random effects are genuinely known.
Calling a large diagonal entry proof of precise estimation. Correlation with other parameters matters; a nearly dependent parameter combination can remain poorly estimated. Inspect the full matrix and its inverse.

Key takeaway: Expected Fisher information is the expected score outer product under the marginal NLME likelihood. It is a prospective, parameter-dependent description of local information. Its inverse gives an approximate covariance matrix only under suitable regularity and large-sample conditions; dependence must be interpreted from that inverse.

Check Your Understanding

Answer each question using the definitions and assumptions stated above. Brief solutions follow the questions.

In an NLME model, what is integrated out to obtain a subject’s marginal likelihood?
State one condition needed for the expected score outer product to equal the negative expected Hessian.
If the off-diagonal entry of a two-parameter information matrix is negative, does that establish a negative approximate correlation between the estimators?
For five independent subjects with equal expected information \(J\) each, what is the total expected information?
Name one reason an inverse-information standard error may be unreliable.

Solutions: (1) Integrate over the subject-level random effect using its population distribution. (2) For example, the density must be differentiable and differentiation must be interchangeable with integration; the support should not depend on the parameter. (3) No. The approximate covariance is the inverse information, and correlation is calculated from that covariance. (4) It is \(5J\), because independent marginal log-likelihood contributions add. (5) Examples include a small sample, weak identification, a boundary estimate, strong nonlinearity, or model misspecification.