E-Optimal Designs and Weakly Identified Directions

Why the Weakest Direction Matters

A design can provide abundant information about some model parameters while leaving another parameter, or a combination of parameters, poorly determined. This imbalance matters when the weak direction is scientifically important: precise estimates of several parameters do not compensate for uncertainty in a quantity that depends on the poorly informed combination. E-optimal design focuses directly on this weakest direction.

The starting point is the Fisher information matrix for a specified model, nominal parameter value, and sampling design. In NLME design calculations, this is the local information matrix under the assumptions used to calculate it. This tutorial treats that matrix as available from the earlier material on Fisher information for population pharmacokinetic designs; the focus here is how its eigenstructure defines a design objective.

The word “smallest” refers to information, not variance. A small minimum eigenvalue means there is a direction in parameter space along which the design supplies relatively little information. If the information matrix is positive definite, that direction is the eigenvector associated with its smallest eigenvalue. The eigenvector describes a linear combination of parameters; it need not point along one parameter axis.

Connection to Uncertainty in Parameter Combinations

For a locally regular model, the usual large-sample approximation to the covariance matrix of an efficient parameter estimator is \(M^{-1}\). This is an approximation, not an exact finite-sample statement. If \(c\) is a unit-length vector, the approximate variance of the corresponding linear combination \(c^\mathsf{T}\hat{\theta}\) is \(c^\mathsf{T}M^{-1}c\). The least-informed direction is the one with the largest such approximate variance.

Because \(M\) is symmetric and positive definite, it has an orthonormal eigenvector basis. Writing any unit vector \(c\) in that basis gives \(c=\sum_j a_jv_j\), where \(\sum_j a_j^2=1\). Therefore,

Equality holds when \(c\) is an eigenvector for the smallest eigenvalue. Thus, under the local covariance approximation, maximizing the smallest eigenvalue limits the largest variance among unit-length linear combinations in the chosen parameter coordinates. This is the central justification for E-optimality. It is a worst-direction criterion, not a claim that every individual parameter variance is minimized.

Comparing Candidate Designs

E-optimal comparison is direct: calculate the smallest eigenvalue for each candidate design using the same parameterization, nominal values, model, and information convention. Prefer the larger value when the design goal is to strengthen the weakest direction. The calculation does not require the weakest eigenvector to be the same for all candidates; in fact, comparing eigenvectors can reveal that the limiting combination changes between schedules.

When an Added Sample Strengthens the Weak Direction

If information contributions from observations are additive under the calculation being used, a candidate sample can be assessed by adding its contribution to the existing matrix and recomputing the smallest eigenvalue. A sample is not automatically valuable for E-optimality simply because it is sensitive to a parameter: it should add information in, or help disentangle, the current weak direction. Correlations between parameter sensitivities can matter as much as their individual magnitudes.

Assumptions, Scaling, and Diagnostics

An E-optimal comparison is meaningful only when the information matrices are calculated for comparable conditions. The model, nominal parameter values, parameterization, planned population and sampling structure, and assumptions used for the information calculation must be held consistent across candidate designs. The matrix should be symmetric and positive semidefinite; a strictly positive smallest eigenvalue requires positive definiteness. The covariance interpretation additionally relies on the local large-sample approximation being appropriate.

Parameter scaling deserves particular care. Eigenvalues depend on the units and coordinates of the parameter vector. Changing a parameter from one unit to another rescales its information and can change both the eigenvalues and the apparent weakest eigenvector. Therefore, do not compare E-criteria computed in inconsistent units or parameterizations. A common approach is to define scientifically meaningful parameter scales \(s_j\) and work with dimensionless coordinates \(z_j=(\theta_j-\theta_{0j})/s_j\). The resulting E-criterion then refers to weakest directions measured in those stated scales. The choice of scales is part of the design objective and should be reported.

Useful diagnostics include checking the smallest eigenvalue, the condition of the information matrix, and the associated eigenvector for each candidate. A zero or numerically negligible minimum eigenvalue indicates a locally unidentifiable or very weak direction under the calculation. If the two smallest eigenvalues are close, the minimum eigenvalue may be reasonably stable while its individual eigenvector is sensitive to small numerical changes; in that case, interpret the weak subspace rather than over-interpreting one vector. Eigenvector signs are arbitrary: \(v\) and \(-v\) describe the same direction.

E-optimality is local when the information depends on unknown parameter values: a schedule optimal at one nominal value need not be optimal elsewhere. It also summarizes the weakest direction but does not guarantee good precision for every scientifically important nonlinear function, robust behavior under model misspecification, or feasibility under operational constraints. Those questions require additional design checks. The E-criterion is most useful when protecting against a poorly informed parameter combination is itself a clear priority.