Deriving the Individual Fisher Information Matrix

From Concentration Sensitivities to Information

For an individual PK model, a sampling schedule specifies the observation times, while the model predicts a concentration at each time as a function of the individual parameter vector. The information contributed by a sample depends on how strongly its predicted concentration changes with those parameters and on how much residual noise obscures that change. Deriving the matrix makes both contributions explicit: sensitivities describe the model response, and residual variances determine the weights.

The matrix is local to a parameter value. In a nonlinear model, sensitivities can change with the parameter values, so a numerical information matrix is calculated at a specified nominal individual parameter vector. This is the conditional, individual-level calculation: it treats that individual’s parameters as the unknown vector of interest and does not integrate over a population distribution.

Each entry of \(J\) is a local rate of change in predicted concentration with respect to one parameter. Its units are concentration divided by the units of that parameter. The row orientation matters: observations index the rows and parameters index the columns. Under independent errors with known, parameter-independent variances, each row contributes an outer product of its sensitivities, scaled by the inverse of its variance.

Deriving the Matrix Under Independent Gaussian Errors

Assume the concentration observation at time \(t_i\) follows \(Y_i=C(t_i;\theta)+\varepsilon_i\), where the errors are mutually independent and \(\varepsilon_i\sim N(0,\sigma_i^2)\). For this derivation, each \(\sigma_i^2\) is known and does not depend on \(\theta\). Write \(\mathbf{Y}\) for the vector of observations, \(\boldsymbol{\mu}(\theta)\) for the vector of predicted concentrations, and \(R=\operatorname{diag}(\sigma_1^2,\ldots,\sigma_n^2)\) for the residual covariance matrix. The Jacobian of \(\boldsymbol{\mu}\) with respect to \(\theta\) is \(J\).

With these assumptions, the log-likelihood, apart from constants that do not depend on \(\theta\), is the weighted squared residual expression below. Differentiating it gives the score, the vector of first derivatives of the log-likelihood.

The Fisher information is the expected outer product of the score, with expectation taken under the model at \(\theta\). Since the residual vector has mean zero and covariance \(R\), the expectation simplifies directly. This derivation also shows why the residual variances appear as inverse weights.

The last expression is a sum of \(p\times p\) matrices. In the term \(\mathbf{j}_i^\mathsf{T}\mathbf{j}_i\), the sensitivity row is transposed to make an outer product. Thus each observation adds a positive-semidefinite contribution; a more precise observation, with smaller \(\sigma_i^2\), receives greater weight.

The same result is obtained from the expected negative Hessian of the log-likelihood under these assumptions. The observed Hessian at a particular data set need not equal its expectation: in a nonlinear model, it includes residual-dependent terms involving second derivatives of the mean. Fisher information is an expected, model-based quantity, not simply the negative Hessian evaluated at observed data.

Worked Calculations

Assumptions, Diagnostics, and Boundaries

The formula depends on its assumptions. Independent errors make \(R\) diagonal; Gaussian errors with parameter-independent variances give the stated likelihood and derivation. If the model uses correlated errors instead, the general known-covariance expression remains \(J^\mathsf{T}R^{-1}J\), but \(R\) is no longer diagonal and observations cannot be treated as separate inverse-variance contributions. If variances depend on the parameters, the likelihood also contains information from changes in the covariance. For a Gaussian model with differentiable covariance \(R(\theta)\), the expected information has additional covariance-derivative terms; the mean-only formula above is then incomplete.

Useful checks start before interpreting the matrix. Confirm that the sensitivity matrix has \(n\) rows and \(p\) columns, that each variance matches the observation to which it is assigned, and that residual variances are used rather than residual standard deviations in \(R\). Check that the matrix is symmetric and positive semidefinite, as \(J^\mathsf{T}R^{-1}J\) must be when \(R\) is positive definite. A materially negative eigenvalue in a computed matrix usually signals an implementation, rounding, or weighting error.

A singular or nearly singular matrix is a different warning. Singularity means the weighted sensitivity rows do not span all parameter directions; near-singularity means some directions are weakly distinguished. More observations alone do not guarantee full rank: if their sensitivity rows are proportional or nearly proportional, they add little or redundant information about separating parameters. Numerical conditioning and scaling also matter when parameter units differ greatly; a matrix should be assessed with its parameterization in mind.

The most common alternative calculation is to add unweighted sensitivity outer products, \(J^\mathsf{T}J\). That is correct only when all residual variances are equal, apart from a common scaling factor. Another alternative is to use the observed negative Hessian. It may be useful for likelihood-based calculations, but it is data-dependent and is not the same object as expected Fisher information in a nonlinear model. The sensitivity-matrix formula is especially transparent for design calculations because it shows how each proposed sample time and its residual precision contribute.