Encoding Subject-Level Design Choices

Why Subject-Level Design Needs an Explicit Representation

A population Fisher matrix is built from information contributed by subjects, but subjects need not share the same covariates, number of observations, or measurement times. A design representation must preserve those differences. If it silently replaces them with one “typical” subject, the resulting information matrix may describe a different study from the one planned.

The goal here is to encode each subject’s design as an input to that subject’s likelihood and approximate Fisher information. The design is not the observed response data, and it is not a population parameter. It records the conditions under which a subject’s measurements are to be taken and interpreted.

Definition: A subject-level design record \(d_i\) specifies the planned inputs for subject \(i\), including relevant subject covariates, the number of observations \(m_i\), the measurement occasions, and, when needed, measurement types or other protocol features. Given model parameters \(\theta\), the record determines the dimensions and design dependence of the subject’s mean vector \(\mu_i(\theta;d_i)\) and covariance matrix \(\Sigma_i(\theta;d_i)\).

The semicolon in \(\mu_i(\theta;d_i)\) emphasizes that parameters and design play different roles: \(\theta\) is the quantity about which information is calculated, while \(d_i\) specifies the subject’s measurement conditions. In general, \(m_i\), the measurement times, and the covariates can vary with \(i\). Consequently, \(\mu_i\) has length \(m_i\), and \(\Sigma_i\) is an \(m_i\)-by-\(m_i\) matrix.

From a Design Record to a Population Matrix

Suppose subjects are independent under the model, conditional on their design records, and subject \(i\)'s response vector follows a multivariate normal approximation:

Y_i\mid d_i \sim N\!\left(\mu_i(\theta;d_i),\Sigma_i(\theta;d_i)\right).

Assume that each \(\Sigma_i\) is positive definite, and that the mean and covariance are differentiable with respect to the components of \(\theta\). The Gaussian approximation is being used for the subject-level likelihood; the design record may differ across subjects. Write \(\mu_{i,a}\) for the derivative of \(\mu_i\) with respect to parameter \(\theta_a\), and \(\Sigma_{i,a}\) for the corresponding derivative of \(\Sigma_i\).

For this model, the expected Fisher information contributed by subject \(i\) has entries

\mathcal{I}_{i,ab}(\theta;d_i) = \mu_{i,a}^{\mathsf T}\Sigma_i^{-1}\mu_{i,b} + \frac{1}{2}\operatorname{tr} \!\left( \Sigma_i^{-1}\Sigma_{i,a} \Sigma_i^{-1}\Sigma_{i,b} \right).

The first term is the contribution through changes in the mean; the second is the contribution through changes in the covariance. The terms are evaluated using that subject’s own design-dependent vectors and matrices. Under independence across subjects, the approximate population Fisher matrix is the sum of those contributions:

\mathcal{I}_{\mathrm{pop}}(\theta;d_1,\ldots,d_N) = \sum_{i=1}^{N}\mathcal{I}_i(\theta;d_i).

This sum does not require identical subjects or identical observation schedules. It does require the subject-level likelihoods to be correctly specified and the assumed independence across subjects to hold. If the model includes dependence between subjects, the joint likelihood and its information cannot in general be represented by this simple sum.

Justification of the Subject-Level Formula

A short derivation shows why subject-specific means and covariances enter separately. For one subject, let \(r_i=Y_i-\mu_i\). The Gaussian log likelihood, omitting terms constant in \(\theta\), is

\ell_i(\theta) = -\frac{1}{2}\log|\Sigma_i| -\frac{1}{2}r_i^{\mathsf T}\Sigma_i^{-1}r_i.

Differentiating with respect to \(\theta_a\) gives the score

U_{i,a} = \mu_{i,a}^{\mathsf T}\Sigma_i^{-1}r_i + \frac{1}{2} \left[ r_i^{\mathsf T}\Sigma_i^{-1}\Sigma_{i,a}\Sigma_i^{-1}r_i -\operatorname{tr}(\Sigma_i^{-1}\Sigma_{i,a}) \right].

The expected Fisher information is \(E(U_{i,a}U_{i,b}\mid d_i)\). Since \(E(r_i\mid d_i)=0\) and \(E(r_ir_i^{\mathsf T}\mid d_i)=\Sigma_i\), the product of the two linear-in-\(r_i\) terms has expectation \(\mu_{i,a}^{\mathsf T}\Sigma_i^{-1}\mu_{i,b}\). A linear term multiplied by a centered quadratic term has expectation zero: a centered multivariate normal vector has zero third moments. For the two centered quadratic terms, the Gaussian fourth-moment identity gives the trace term in the formula above. Thus both sources of information are evaluated at the subject’s own \(d_i\). Finally, independent subjects have a joint log likelihood equal to the sum of their log likelihoods, so their expected information matrices also add.

Encoding Unequal Covariates and Occasion Lists

Worked Example: Building Design Matrices for Unequal Schedules

Consider a hypothetical study with a response measured at scheduled times \(t\). Each subject also has a standardized baseline covariate \(x_i\). For illustration, use the mean model \(\mu_{ij}=\beta_0+\beta_1x_i+\beta_2t_{ij}\). Three subjects have the following hypothetical designs: subject A has \(x_A=-1\) and times 1 and 4; subject B has \(x_B=0\) and times 2, 6, and 10; subject C has \(x_C=1\) and time 3. Encode each design for the mean model.

For subject A, \(m_A=2\), and the two rows of the design matrix have columns for the intercept, \(x_i\), and time:

X_A= \begin{bmatrix} 1&-1&1\\ 1&-1&4 \end{bmatrix}, \qquad \mu_A=X_A \begin{bmatrix}\beta_0\\\beta_1\\\beta_2\end{bmatrix}.

For subject B, \(m_B=3\), so its mean vector has length three:

X_B= \begin{bmatrix} 1&0&2\\ 1&0&6\\ 1&0&10 \end{bmatrix}.

For subject C, \(m_C=1\), and \(X_C=\begin{bmatrix}1&1&3\end{bmatrix}\). The records retain both the distinct covariate values and the distinct occasion lists. In particular, \(X_A\), \(X_B\), and \(X_C\) do not have to have the same number of rows. If a covariance model is specified, its matrix for each subject must have the matching dimension: \(2\)-by-\(2\), \(3\)-by-\(3\), and \(1\)-by-\(1\), respectively.

The encoding assumes that the response model applies at each listed occasion and that the covariate values are the values intended for those subjects. The calculation does not invent unobserved occasions to make the schedules equal; it uses the occasions actually specified by each design record.

Worked Information Calculation with Heterogeneous Subjects

Worked Example: Summing Contributions with Unequal Covariates and Counts

Suppose, hypothetically, that responses are independent normal measurements with mean \(\beta_0+\beta_1x_i\) and known variance \(\sigma^2=4\) response-units squared. Subject A has \(x_A=-1\) and two observations, subject B has \(x_B=0\) and one observation, and subject C has \(x_C=2\) and two observations. Compute the approximate population Fisher matrix for \((\beta_0,\beta_1)\).

For known common variance and this linear mean model, the mean derivative for each observation is the row \((1,x_i)\). Therefore, the contribution is \(\mathcal{I}_i=\sigma^{-2}X_i^{\mathsf T}X_i\). Equivalently, summing the five observation rows gives

\mathcal{I}_{\mathrm{pop}} = \frac{1}{4} \begin{bmatrix} \sum_i m_i&\sum_i m_ix_i\\ \sum_i m_ix_i&\sum_i m_ix_i^2 \end{bmatrix}.

Here, \(\sum_i m_i=2+1+2=5\), \(\sum_i m_ix_i=2(-1)+1(0)+2(2)=2\), and \(\sum_i m_ix_i^2=2(1)+1(0)+2(4)=10\). Substitution gives

\mathcal{I}_{\mathrm{pop}} = \frac{1}{4} \begin{bmatrix} 5&2\\ 2&10 \end{bmatrix} = \begin{bmatrix} 1.25&0.5000\\ 0.5000&2.50 \end{bmatrix}.

The entries are information for the intercept and slope parameters in the parameterization used; their units depend on the units of the response and of \(x_i\). The calculation assumes independent measurements, the stated common known variance, and a correctly specified mean model. The different covariates and observation counts are incorporated directly rather than replaced by a common subject record.

Occasions Can Change the Covariance Input

Worked Example: Encoding Two Measurements with Time-Dependent Covariance

Consider a hypothetical subject measured at times \(t_1=1\) and \(t_2=4\). Suppose the mean at both times is a common parameter \(\theta\), and the specified covariance has variance \(4\) at each occasion and covariance \(2\exp(-|t_1-t_2|/3)\) between occasions. The occasion list is part of the design record, so it determines the off-diagonal entries of the covariance matrix.

The time gap is \(4-1=3\), so the covariance is \(2e^{-1}\approx0.7358\). Thus

\mu_i= \begin{bmatrix}\theta\\\theta\end{bmatrix}, \qquad \Sigma_i= \begin{bmatrix} 4&0.7358\\ 0.7358&4 \end{bmatrix}.

The covariance is positive definite because its eigenvalues are \(4+0.7358=4.7358\) and \(4-0.7358=3.2642\), both positive. The mean derivative is \(\mu_{i,\theta}=(1,1)^{\mathsf T}\), and the covariance does not depend on \(\theta\), so the covariance derivative is zero. The subject’s information about \(\theta\) is therefore

\mathcal{I}_{i,\theta\theta} = \mu_{i,\theta}^{\mathsf T}\Sigma_i^{-1}\mu_{i,\theta} = \frac{2}{4+0.7358} \approx 0.4223.

The result has units of inverse response-units squared when \(\theta\) is measured in response units. This example illustrates why the occasion list must accompany the observation count: two measurements at different times can imply a covariance structure different from another pair of times. It also checks positive definiteness before using the inverse covariance in the information calculation.

Diagnostics, Alternatives, and Limits

Before assembling the population matrix, verify that every subject record maps to a valid likelihood contribution. Useful checks include the following:

Dimension checks: confirm that \(\mu_i\) has \(m_i\) entries and \(\Sigma_i\) has \(m_i\) rows and columns in the same occasion order.
Occasion checks: verify units, ordering, duplicate times, and whether the schedule is planned or actually observed. A time recorded in hours must not be passed to a model expecting days.
Covariance checks: check symmetry and positive definiteness of each \(\Sigma_i\) at the nominal parameter values. A singular or indefinite matrix makes the stated Gaussian likelihood invalid or unusable as written.
Model checks: confirm that covariates and measurement types used in the design record are handled by the model and that the model’s assumptions are appropriate for the planned observations.
Independence checks: verify that cross-subject dependence is absent under the assumed model. Shared unmodelled effects or correlated sampling can invalidate the simple sum.

A common alternative is to encode one common schedule and one common covariate profile for every subject. That is appropriate only when it matches the actual design, or when the model and question explicitly justify that simplification. Another practical representation is a long table with one row per observation, containing the subject identifier, occasion, covariates, and measurement type. Such a table can be convenient for data handling, but the information calculation still needs to reconstruct each subject’s vector and covariance in a consistent order.

A frequent mistake is to pad short records with extra rows of zeros so that all subjects have the same number of observations, then treat those padded entries as measurements. Those rows are not data and change the likelihood if included. The correction is to retain each subject’s actual dimension, or to use a mathematically equivalent joint representation that explicitly accounts for missing observations. Similarly, using a common covariance matrix merely because subjects share a model name is wrong when the schedule changes the covariance entries.

The approximate matrix is local to the nominal parameter values at which the derivatives are evaluated. It is also conditional on the design records and on the Gaussian subject-level likelihood. This encoding does not by itself establish that a design is optimal, that parameters are identifiable, or that the approximation accurately represents a complex likelihood. Those conclusions require additional analysis. If observation availability depends on unobserved responses or on unmodelled subject processes, treating the observed schedule as fixed may also fail to represent the data-generating mechanism.

Key takeaway: Keep one design record for each subject, map it to that subject’s own mean vector and covariance matrix, and sum the resulting Fisher contributions only under the specified independence and likelihood assumptions. Unequal subject designs are a direct input to the calculation, not a reason to substitute an artificial average subject.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

If a subject has four scheduled observations, what are the dimensions of that subject’s mean vector and covariance matrix?
Why can the subject-level information matrices be summed under the assumptions stated here?
A design table records times in hours, but the covariance function expects days. What should be checked before calculating information?
For a common-mean model with covariance \(\Sigma\), what expression gives the mean-parameter information for one subject?
What is wrong with padding subjects who have fewer observations with zero-valued rows and treating those rows as measurements?

Solutions: (1) The mean vector has four entries and the covariance matrix is \(4\)-by-\(4\). (2) Conditional independence makes the joint log likelihood the sum of subject log likelihoods, so expected information adds. (3) Convert the times consistently and verify that the covariance model receives the intended units and occasion values. (4) If the mean is \(\theta\mathbf{1}\) and covariance is known and parameter-independent, the information is \(\mathbf{1}^{\mathsf T}\Sigma^{-1}\mathbf{1}\). (5) The added rows are not observed data; including them changes the likelihood and produces an information matrix for a different design.