Information Under Fixed Covariate Values

Information at Specified Covariate Values

In a population design, subject characteristics and design choices can affect how informative an observation is about the population parameters. A measurement taken at one dose, visit time, or covariate value need not carry the same information as a measurement taken at another. This tutorial focuses on the information contributed when those covariate values are specified and treated as fixed by the design.

Let subject $i$ have response vector $Y_i$, fixed covariate vector $x_i$, and a conditional model indexed by population parameter vector $\theta$. The covariate value is held fixed when calculating the subject's information; the response remains random under the model. Information is therefore evaluated at both a parameter value and a design point, written $\mathcal I_i(\theta;x_i)$. In planning, $\theta$ is usually set to a nominal value, so the result is a local information calculation.

Definition: The subject-level expected Fisher information conditional on fixed covariates $x_i$ is the expected outer product of the conditional score: $$ \mathcal I_i(\theta;x_i) = E_{\theta}\left[ s_i(\theta;x_i)s_i(\theta;x_i)^{\mathsf T} \mid x_i \right], \qquad s_i(\theta;x_i) = \frac{\partial}{\partial\theta} \log f(Y_i\mid x_i;\theta). $$ When the usual regularity conditions hold, it is also the negative expected Hessian of the conditional log likelihood. The expectation is over possible responses at the specified $x_i$, not over a distribution of covariates.

The definition requires a correctly specified conditional likelihood that is differentiable in $\theta$, finite score second moments, and regularity conditions allowing differentiation under the expectation. The subject's response must have the stated conditional distribution given its covariates. For a cohort sum, subjects must be conditionally independent given their fixed design values, or the joint likelihood must otherwise account for dependence. These assumptions should be checked against the study design: for example, shared random effects across subjects would generally prevent a simple independent-subject sum.

Gaussian Subject-Level Information

A useful case is a Gaussian conditional model $Y_i\mid x_i\sim N(\mu_i(\theta;x_i),V_i(\theta;x_i))$, where $V_i$ is positive definite. Write $\mu_{i,a}$ and $V_{i,a}$ for derivatives with respect to parameter component $\theta_a$. The expected information has a contribution from changes in the mean and a contribution from changes in the covariance:

\mathcal I_{i,ab}(\theta;x_i) = \mu_{i,a}^{\mathsf T}V_i^{-1}\mu_{i,b} + \frac{1}{2}\operatorname{tr} \left( V_i^{-1}V_{i,a}V_i^{-1}V_{i,b} \right).

This is the Gaussian Fisher information formula. If a parameter changes only the mean, its covariance derivative is zero; if it changes only the covariance, its mean derivative is zero. The formula is evaluated using the subject's particular covariates, including any covariate dependence in $\mu_i$ or $V_i$. A planned design can therefore change information through the predicted mean, the variability, or both.

Worked Example: Mean Information at One Fixed Covariate Value

Suppose one subject has a scalar response with conditional model $Y_i\mid x_i\sim N(\beta_0+\beta_1x_i,9)$. Treat the variance as known, and calculate information about $(\beta_0,\beta_1)$ for a subject whose fixed covariate is $x_i=2$.

The mean derivative with respect to the parameter vector is $(1,2)^{\mathsf T}$, and the variance is $9$. For a Gaussian model with known variance, the information is the derivative outer product divided by that variance:

\mathcal I_i(\beta_0,\beta_1;2) = \frac{1}{9} \begin{pmatrix}1\\2\end{pmatrix} \begin{pmatrix}1&2\end{pmatrix} = \begin{pmatrix} 0.1111&0.2222\\ 0.2222&0.4444 \end{pmatrix}.

The entries are information for the intercept, the slope, and their cross-information. The calculation is conditional on $x_i=2$: it does not average over possible covariate values. The matrix is rank one because this single observation provides information along only one direction in the two-parameter mean vector.

From Fixed Subject Values to Cohort Information

When subjects are conditionally independent, the cohort log likelihood is the sum of the subject log likelihoods. Its score is consequently the sum of the subject scores. Conditional on the fixed covariates, each score has expectation zero under the model. Independence makes the expected cross-products between scores from different subjects zero, leaving the sum of their expected information matrices.

\mathcal I_{\mathrm{cohort}}(\theta;\mathbf{x}) = \sum_{i=1}^{N}\mathcal I_i(\theta;x_i), \qquad \mathbf{x}=(x_1,\ldots,x_N).

Here $N$ is the number of subjects and $\mathbf{x}$ is the complete fixed covariate design. If every subject has the same covariate value $x$ and the same conditional model, this reduces to $N\mathcal I(\theta;x)$. With different covariate values, each subject contributes the matrix evaluated at their own value. Summation is matrix addition: each element of the cohort matrix is the sum of that element across subjects.

Justification: Write the cohort score as $S=\sum_i s_i$. Then $E(SS^{\mathsf T}\mid\mathbf{x})=\sum_i E(s_is_i^{\mathsf T}\mid x_i)+\sum_{i\ne j}E(s_is_j^{\mathsf T}\mid\mathbf{x})$. Conditional independence and zero-mean scores imply $E(s_is_j^{\mathsf T}\mid\mathbf{x})=E(s_i\mid x_i)E(s_j^{\mathsf T}\mid x_j)=0$ for $i\ne j$. The remaining terms are precisely the subject-level information matrices.

Worked Example: Comparing Two Fixed Allocations

Continue with the model $Y_i\mid x_i\sim N(\beta_0+\beta_1x_i,9)$. Compare a cohort of five independent subjects with covariates $0,0,0,2,2$ to five subjects all at $x_i=2$. Calculate the cohort information for each fixed design.

For any one subject, the information is $\frac{1}{9}\begin{pmatrix}1&x_i\\x_i&x_i^2\end{pmatrix}$. At $x_i=0$, this is $\frac{1}{9}\begin{pmatrix}1&0\\0&0\end{pmatrix}$; at $x_i=2$, it is $\frac{1}{9}\begin{pmatrix}1&2\\2&4\end{pmatrix}$. For the first allocation, sum three contributions at zero and two at two:

\mathcal I_{\mathrm{cohort}} = \frac{1}{9} \left[ 3\begin{pmatrix}1&0\\0&0\end{pmatrix} + 2\begin{pmatrix}1&2\\2&4\end{pmatrix} \right] = \frac{1}{9} \begin{pmatrix}5&4\\4&8\end{pmatrix} = \begin{pmatrix} 0.5556&0.4444\\ 0.4444&0.8889 \end{pmatrix}.

For the second allocation, multiply the $x_i=2$ matrix by five:

\mathcal I_{\mathrm{cohort}} = \frac{5}{9} \begin{pmatrix}1&2\\2&4\end{pmatrix} = \begin{pmatrix} 0.5556&1.111&\\ 1.111&2.222 \end{pmatrix}.

The second design has larger slope-related entries, but its matrix is rank one: all observations have the same covariate value, so their mean derivatives point in the same parameter direction. The first design includes observations at two values and produces a full-rank matrix. This comparison is about the information matrices under the stated model and variance; it does not, by itself, establish which design is preferable for every scientific objective.

Local Evaluation and Parameter-Dependent Covariance

In nonlinear models, the derivatives and the covariance matrix generally depend on $\theta$. A design calculation therefore needs a specified nominal parameter vector $\theta^\ast$. The resulting $\mathcal I_i(\theta^\ast;x_i)$ describes local sensitivity of the likelihood near that value; it is not a parameter-free property of the covariate. If plausible parameter values vary substantially, calculations at more than one nominal value can reveal whether a design comparison is sensitive to those assumptions.

Worked Example: Mean and Variance Information at a Fixed Dose

Suppose one observation at fixed dose $x$ follows $Y\mid x\sim N(m, \sigma^2)$, where $m=\exp(\alpha+\beta x)$. Evaluate the information at $x=1$, $\alpha=\log 4$, $\beta=\log 2$, and $\sigma=2$, using parameter vector $(\alpha,\beta,\sigma^2)$.

At these values, $m=\exp(\log4+\log2)=8$, and the variance is $4$. The mean derivatives with respect to $\alpha$ and $\beta$ are $m$ and $xm$, respectively, so the mean derivative vector is $(8,8)^{\mathsf T}$. The covariance derivative with respect to $\sigma^2$ is $1$. The mean and variance parameter contributions have zero cross-information in this normal model. Thus:

\mathcal I_i = \begin{pmatrix} (8)(8)/4&(8)(8)/4&0\\ (8)(8)/4&(8)(8)/4&0\\ 0&0&1/(2\cdot4^2) \end{pmatrix} = \begin{pmatrix} 16&16&0\\ 16&16&0\\ 0&0&0.03125 \end{pmatrix}.

The last entry is information for the variance parameter $\sigma^2$, not for the standard deviation $\sigma$. If the parameter were instead $\sigma$, its derivative and information would have to be transformed accordingly. The example also illustrates why parameterization must match the requested information calculation.

Checks, Alternatives, and Limitations

A fixed covariate design is a list of the values at which subjects are modeled. It is not the same operation as averaging subject information over a population distribution of covariates. That distinction matters: a cohort with a specified allocation uses the sum at its actual design values. If covariates are not fixed by design, or if the target is an average over a covariate population, a different calculation is needed.

Expected Fisher information is also distinct from observed information. Expected information averages over all responses under the conditional model at the chosen covariates; observed information is calculated from the response actually obtained. Expected information is useful for planning, while observed information reflects the realized data. Neither should be substituted for the other without stating the purpose.

Before trusting a calculation, check that the covariates and their units are encoded as intended, the model includes their relevant effects, and the covariance used is the covariance of the full subject response vector. Repeated measurements from one subject are generally not independent merely because they occur at different times. Treating correlated measurements as independent can overstate information. Also check the nominal parameter values and the parameterization: derivatives calculated at the wrong value or for a differently scaled parameter produce a different information matrix.

A common mistake is to calculate information at a “typical” covariate value and multiply by the cohort size even when the planned subjects have different values. That is valid only if the subjects truly share that covariate value and model. For a heterogeneous fixed design, use $\sum_i\mathcal I_i(\theta;x_i)$; in general this is not equal to $N\mathcal I(\theta;\bar{x})$, because information can depend nonlinearly on covariates. Another mistake is to treat covariates as random while using a fixed-design calculation. State clearly whether the covariate values are conditioned upon or averaged over.

Key takeaway: Evaluate each subject's expected information at the specified covariate value and nominal parameter vector. Under conditional independence, sum those matrices to obtain information for the fixed cohort design.

Check Your Understanding

Answer each question using conditional information at the stated fixed design values.

For $Y_i\mid x_i\sim N(\theta x_i,4)$, with $\theta$ the only unknown parameter, find the information from one subject at $x_i=3$.
In that same model, a cohort has covariates $1,2,2$. Find the total information.
What condition allows the cohort information to be computed as the sum of subject-level matrices?
Explain why evaluating information at the mean covariate value is not generally equivalent to summing information at the observed fixed covariate values.

Solutions. For 1, the mean derivative is $x_i=3$, so $\mathcal I_i=3^2/4=2.25$. For 2, sum the three contributions: $(1^2+2^2+2^2)/4=9/4=2.25$. For 3, conditional independence of subjects given the fixed design values, together with zero-mean conditional scores, makes cross-subject score products have expectation zero. For 4, information can be a nonlinear function of the covariate; the sum of its values at the actual design points need not equal the number of subjects times its value at the average covariate.