Why the Covariance Structure Matters
A population PK model can assign random effects to several individual-specific parameters. The covariance structure determines which pairs of those effects are allowed to vary together in the model. It is therefore a substantive modeling choice: it controls the relationships represented in the population distribution, and it also determines how many covariance parameters must be estimated.
The earlier material on reading the omega matrix established that its diagonal entries are random-effect variances and its off-diagonal entries are covariances. Here the focus is how to restrict or include those entries systematically. The three common choices are a diagonal matrix, a block-diagonal matrix, and a full covariance matrix.
The covariance matrix is symmetric: the covariance of effect \(j\) with effect \(k\) equals the covariance of effect \(k\) with effect \(j\). Thus, each distinct off-diagonal covariance is one parameter, not two. Under the usual multivariate-normal random-effect model, \(\eta_i \sim N(0,\Omega)\), the covariance restrictions describe the dependence allowed by that distribution. Without a multivariate-normal assumption, zero covariance alone does not generally imply independence.
What Each Structure Allows
For \(p\) random effects, every structure includes a variance for each effect. The difference is which off-diagonal entries are free to be estimated. A fixed zero is a model restriction; it is not an estimate that happened to be small. For a block-diagonal structure, the chosen partition must be stated because it defines the allowed covariances.
| Structure | Allowed covariance entries | Number of covariance parameters |
|---|---|---|
| Diagonal | No off-diagonal covariances | \(p\) |
| Block-diagonal, block sizes \(b_1,\ldots,b_K\) | Covariances only within a block | \(\sum_{k=1}^{K} b_k(b_k+1)/2\) |
| Full | Every distinct pair of effects | \(p(p+1)/2\) |
These counts cover parameters in \(\Omega\) only. They do not include typical values, residual-error parameters, or other parts of the PK model. Whether a software implementation reports variances, standard deviations, or a transformed parameterization does not change the number of free covariance parameters in the model.
Counting the Parameters
A symmetric covariance block of size \(b\) has \(b\) diagonal entries and \(b(b-1)/2\) distinct off-diagonal entries. The total is \(b+b(b-1)/2=b(b+1)/2\). A block-diagonal matrix is made of such blocks, so adding their counts gives the total. A diagonal structure is the special case in which every block has size one; a full structure is the special case of a single block containing all \(p\) effects.
The same result follows by counting the upper triangle of a symmetric matrix: it contains \(p\) diagonal entries and \(p(p-1)/2\) entries above the diagonal. A diagonal structure keeps only the \(p\) diagonal entries. A block-diagonal structure keeps the upper triangles inside its blocks and fixes entries between blocks to zero.
Worked Example: Three Effects With Different Structures
Hypothetical setup. Suppose a model has three dimensionless random effects, \(\eta_{i1}, \eta_{i2}, \eta_{i3}\). Compare a diagonal structure, a block structure with effects 1 and 2 together and effect 3 alone, and a full structure.
For a diagonal structure, all three variances are estimated and all three distinct covariances are fixed at zero. The count is \(3\) covariance parameters.
For the specified block structure, the block sizes are 2 and 1. The count is \(\frac{2(2+1)}{2}+\frac{1(1+1)}{2}=3+1=4\). The allowed covariance is between effects 1 and 2. Covariances between effects 1 and 3 and between effects 2 and 3 are fixed at zero.
For a full structure, all three variances and all three distinct covariances are estimated. The count is \(\frac{3(3+1)}{2}=6\). The counts are therefore 3, 4, and 6, respectively. These are counts of free entries in \(\Omega\), not counts of all parameters in the PK model.
A Block Structure in Matrix Form
Consider a four-effect vector, with hypothetical effects on log-scale parameters. If effects 1 and 2 form one block and effects 3 and 4 form another, the covariance matrix has the form below. The zero submatrices express the restriction that every covariance across the two groups is fixed at zero.
Each block has size 2 and contributes \(2(2+1)/2=3\) free entries. The total is \(3+3=6\). A diagonal structure for the same four effects would have 4 parameters. A full structure would have \(4(4+1)/2=10\). The block structure allows two of the six possible pairwise covariances, while the other four are fixed to zero.
Worked Example: Counting a Five-Effect Partition
Hypothetical setup. A model has five random effects partitioned into blocks \(\{\eta_{i1},\eta_{i2}\}\), \(\{\eta_{i3},\eta_{i4}\}\), and \(\{\eta_{i5}\}\). Find the number of free covariance parameters and the covariance restrictions.
The block sizes are 2, 2, and 1. Their parameter counts are \(2(3)/2=3\), \(2(3)/2=3\), and \(1(2)/2=1\). Thus the block-diagonal structure has \(3+3+1=7\) free covariance parameters.
There are five variances. Within the first two blocks, one covariance is estimated in each block, adding two more parameters. Covariances between different blocks are fixed at zero, including every covariance involving effect 5 and effects 1 through 4. As checks, the diagonal count is 5 and the full count is \(5(6)/2=15\).
Assumptions and Practical Choice
Every covariance structure is an assumption about the population distribution of the random effects, not just a computational setting. A diagonal structure says that no pairwise random-effect covariance is represented. A block-diagonal structure says that covariances may occur within specified groups but not between groups. A full structure places no such zero restrictions among the effects.
Under a multivariate-normal random-effect distribution, a diagonal covariance matrix implies that the random effects are mutually independent. Under that same distribution, a block-diagonal covariance matrix implies independence between the blocks, while effects within a block may be dependent. These independence statements rely on the joint normal assumption; for other distributions, zero covariance need not mean independence.
The choice should follow the model and the information available. A full structure uses the most flexible covariance pattern, but it also requires the most covariance parameters. When there are many effects or limited information about some effects, estimating many covariances can be difficult. A diagonal structure is more parsimonious, but it rules out all pairwise covariance patterns. A block-diagonal structure can express a planned compromise, provided the grouping has a clear rationale and is specified before interpreting the fitted results.
Worked Example: Comparing a Four-Effect Model
Hypothetical setup. A four-effect model groups effects 1, 2, and 3 in one block and leaves effect 4 in a separate block. Compare the parameter counts with the diagonal and full alternatives.
The block sizes are 3 and 1, so the block-diagonal count is \(3(4)/2+1(2)/2=6+1=7\). The diagonal count is 4. The full count is \(4(5)/2=10\).
In the block structure, the three variances and three distinct covariances among effects 1, 2, and 3 are free, along with the variance of effect 4. The three covariances between effect 4 and effects 1, 2, and 3 are fixed to zero. Thus this partition saves three covariance parameters relative to the full model, while allowing three covariances that the diagonal model excludes.
This arithmetic identifies the model restrictions; it does not show that a chosen structure is supported by data. That requires assessing estimation behavior and the model in context.
Diagnostics, Failure Modes, and Alternatives
The parameter count is a useful first check, but it is not evidence that a structure is estimable or appropriate. Inspect whether variance and covariance estimates are finite and stable, whether uncertainty is large, and whether fitting reports warnings or fails to converge. A covariance estimate near a boundary, a non-positive-definite estimate, or strong sensitivity to starting values can signal that the data do not support the chosen parameterization. These are reasons to investigate, not automatic proof that one particular simpler structure is correct.
Also check that the written partition matches the matrix and the scientific rationale. A common specification error is to call a matrix block-diagonal while leaving an unintended cross-block covariance free. The opposite error is to fix a within-block covariance at zero while describing that block as unrestricted. For a proposed structure, list the blocks and count the free entries explicitly.
The main alternatives differ in flexibility and restrictions. A diagonal model is the most restrictive of the three, a full model is the least restrictive, and a specified block model lies between them in the covariance entries it allows. Two block structures with different partitions need not be nested: one may allow a covariance that the other fixes to zero, while excluding a different covariance. Therefore, do not assume that every pair of block structures can be compared as a simple progression from fewer to more restrictions.
A full structure is not automatically preferable simply because it is less restrictive. If covariance parameters are poorly supported, estimates may be unstable. Conversely, a diagonal structure can omit dependence that matters to the population distribution. Structure selection should consider the intended model, the available data, estimation diagnostics, and the consequences of the restrictions. No covariance structure by itself establishes a causal relationship between PK parameters.
Common Mistake
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A diagonal covariance structure has four random effects. How many free covariance parameters does it have?
- A block-diagonal structure has block sizes 3 and 2. How many free covariance parameters does it have, and how many distinct covariances are free?
- For three jointly multivariate-normal random effects, what does a diagonal covariance matrix imply? Would the same conclusion follow from zero covariances alone for an arbitrary joint distribution?
- A four-effect structure has blocks of sizes 2 and 2. How many covariance parameters does it have, and how many cross-block covariances are fixed at zero?
- Is every block-diagonal structure nested within every other block-diagonal structure? Explain briefly.
Solutions. (1) The diagonal count is \(p=4\), so there are 4 free variances and no free covariances. (2) The total is \(3(4)/2+2(3)/2=6+3=9\). The five variances are free, so \(9-5=4\) distinct within-block covariances are free. (3) Under joint multivariate normality, the diagonal covariance matrix implies mutual independence. For an arbitrary joint distribution, zero covariance alone does not imply independence. (4) The count is \(2(3)/2+2(3)/2=3+3=6\). There are \(2\times2=4\) pairs crossing from one block to the other, all fixed at zero. (5) No. Different partitions can allow different covariance entries, so neither structure necessarily contains all the free covariance parameters of the other.