E-Optimality And The Weakest Direction

Why Focus on the Weakest Direction?

A design can provide substantial information overall while still leaving one combination of parameters poorly determined. For example, measurements may estimate each parameter separately with reasonable precision but provide little information about a particular combination of them. An average-variance summary can obscure that weakness. E-optimality instead asks how informative the design is in its least-informed parameter direction.

This criterion is useful when the design goal is to avoid a particularly imprecise direction in the parameter space. It is a local criterion: in a nonlinear model, the information matrix is evaluated at specified nominal parameter values, so the resulting ranking is conditional on those values and the assumed observation model.

Definition and Interpretation

Let $M(\xi)$ denote the Fisher information matrix for a design $\xi$, using a fixed parameterization and a fixed model. It is symmetric and positive semidefinite under the usual regularity conditions. Write its eigenvalues as $\lambda_1,\ldots,\lambda_p$, where $p$ is the number of parameters, and order them so that $\lambda_{\min}(M(\xi))$ is the smallest. For a fixed sample size, multiplying the per-observation information matrix by the sample size changes the criterion’s numerical value but not the ranking of designs.

Definition: An E-optimal design maximizes the smallest eigenvalue of its Fisher information matrix: $$ \xi_E \in \arg\max_{\xi\in\mathcal{D}} \lambda_{\min}\!\left(M(\xi)\right), $$ where $\mathcal{D}$ is the set of allowable designs. The smallest eigenvalue measures information in the least-informed unit-length parameter direction.

The direction associated with the smallest eigenvalue is its eigenvector. Under the usual local, large-sample approximation, the covariance matrix of an estimator is approximately the inverse of the total information matrix. Thus, the information matrix’s smallest eigenvalue corresponds to the largest eigenvalue of the approximate covariance matrix. E-optimality can therefore be understood as protecting the least-precise direction, not as making every individual parameter equally precise.

\operatorname{Cov}(\hat{\theta}) \approx [N M(\xi)]^{-1}, \qquad \lambda_{\max}\!\left([N M(\xi)]^{-1}\right) = \frac{1}{N\lambda_{\min}(M(\xi))}.

Here $N$ is the number of independent observations and $M(\xi)$ is information per observation. The approximation presumes a correctly specified, locally regular model, an interior parameter value, and a nonsingular information matrix. The equality follows because a symmetric positive-definite matrix and its inverse have the same eigenvectors, with reciprocal eigenvalues. When $M(\xi)$ is singular, its smallest eigenvalue is zero: at least one direction has no local information, and the usual full-parameter inverse-information covariance approximation is unavailable.

Why the Smallest Eigenvalue Is the Weakest Information

For any unit vector $v$, the scalar $v^\mathsf{T}M(\xi)v$ is the information in the direction $v$. To see how the smallest eigenvalue identifies the weakest direction, decompose a unit vector into the orthonormal eigenvectors $u_j$ of $M(\xi)$: $v=\sum_j a_j u_j$, where $\sum_j a_j^2=1$. Then

v^\mathsf{T}M(\xi)v = \sum_{j=1}^{p}\lambda_j a_j^2 \geq \lambda_{\min}(M(\xi))\sum_{j=1}^{p}a_j^2 = \lambda_{\min}(M(\xi)).

Equality holds when $v$ is a unit eigenvector associated with the smallest eigenvalue. Therefore,

\lambda_{\min}(M(\xi)) = \min_{\lVert v\rVert=1} v^\mathsf{T}M(\xi)v.

This variational characterization makes the E criterion’s interpretation precise: among all unit-length parameter directions, it maximizes the minimum directional information. The unit-length qualification matters. It uses the ordinary Euclidean length of the parameter vector, so changing parameter units can change which directions count as unit length and can alter the E-optimal ranking.

Worked Comparisons

Worked Example: A Balanced Design Versus a Weak Direction

Suppose two hypothetical designs for two parameters have total information matrices

M_A= \begin{pmatrix} 9&0\\ 0&1 \end{pmatrix}, \qquad M_B= \begin{pmatrix} 4&0\\ 0&4 \end{pmatrix}.

Both matrices are diagonal, so their eigenvalues are their diagonal entries. For design A, the eigenvalues are 9 and 1, giving $\lambda_{\min}(M_A)=1$. For design B, both eigenvalues are 4, giving $\lambda_{\min}(M_B)=4$. E-optimality selects design B because $4>1$.

The comparison also shows what the criterion protects. Under the inverse-information approximation, the covariance matrices are

M_A^{-1}= \begin{pmatrix} 1/9&0\\ 0&1 \end{pmatrix}, \qquad M_B^{-1}= \begin{pmatrix} 1/4&0\\ 0&1/4 \end{pmatrix}.

The largest directional variance is 1 for design A and $1/4=0.25$ for design B. Thus, design A is more informative in its first coordinate but much less informative in its second; design B provides a smaller worst-direction variance. These are hypothetical information and variance quantities, not measurements from a study.

Worked Example: Finding the Weakest Direction

Consider the hypothetical information matrix

M_C= \begin{pmatrix} 6&2\\ 2&3 \end{pmatrix}.

Its trace is 9 and its determinant is $6(3)-2(2)=14$. The eigenvalues solve $\lambda^2-9\lambda+14=0$, so they are $(9+5)/2=7$ and $(9-5)/2=2$. The smallest eigenvalue is 2.

To find its direction, solve $(M_C-2I)v=0$:

\begin{pmatrix} 4&2\\ 2&1 \end{pmatrix} \begin{pmatrix}v_1\\v_2\end{pmatrix} = \begin{pmatrix}0\\0\end{pmatrix}.

The equation $2v_1+v_2=0$ gives a direction proportional to $(-1,2)$. A unit eigenvector is $u_{\min}=(-1,2)^\mathsf{T}/\sqrt{5}$. Thus, it is this combination of the two parameter coordinates—not either coordinate alone—that has the least information. Its directional information is $u_{\min}^\mathsf{T}M_Cu_{\min}=2$, and its approximate variance under this total information matrix is $1/2=0.5$. All numbers are hypothetical.

A Simple Allocation Argument

A design can improve its smallest eigenvalue by directing observations toward a weak direction. The following stylized allocation shows the balancing principle when the information contributions are especially simple. Suppose each observation at condition A contributes $\operatorname{diag}(4,0)$, each observation at condition B contributes $\operatorname{diag}(0,1)$, and exactly 10 independent observations are allocated between the two conditions. Let $n_A+n_B=10$. Additivity of independent information gives the total matrix

M(n_A,n_B)= \begin{pmatrix} 4n_A&0\\ 0&n_B \end{pmatrix}, \qquad \lambda_{\min}(M)=\min(4n_A,n_B).

Worked Example: Allocating Observations to Improve the Weakest Direction

For this hypothetical allocation, $n_B=10-n_A$, so the objective is $\min(4n_A,10-n_A)$. The first quantity increases with $n_A$; the second decreases. They are equal when

4n_A=10-n_A, \qquad 5n_A=10, \qquad n_A=2,\quad n_B=8.

At this allocation, $M=\operatorname{diag}(8,8)$, so $\lambda_{\min}(M)=8$. If $n_A<2$, then $4n_A<8$, so the minimum is below 8. If $n_A>2$, then $10-n_A<8$, so again the minimum is below 8. Therefore the allocation $(2,8)$ maximizes the smallest eigenvalue, including when allocations must be integer-valued. The example assumes fixed observation count and the stated per-observation information contributions; it illustrates balancing, not a general rule that optimal designs always equalize eigenvalues.

Assumptions, Comparisons, and Diagnostics

The calculations above use information matrices for a common set of parameters, with the same parameterization and observation model across candidate designs. In applications, information must be calculated from the model and its assumptions, including the observation variance structure, and for a nonlinear model it is typically evaluated at nominal parameter values. Independent contributions may be added when the corresponding observations are independent, as in the allocation example. If a design has a zero smallest eigenvalue, it fails to identify at least one local direction; if the eigenvalue is merely very small, the matrix may be technically nonsingular but still imply poor precision.

E-optimality differs from the criteria already used to summarize information in other ways. D-optimality focuses on generalized variance, whereas E-optimality focuses on the worst directional information. A-optimality minimizes average parameter variance, whereas E-optimality targets the largest directional variance implied by the inverse information. The criteria can rank designs differently: improving a very weak direction may be worthwhile for E-optimality even if other directions become less precise. Which trade-off is desirable depends on the scientific purpose.

Several checks help determine whether an E-optimal analysis is behaving sensibly:

Inspect the smallest eigenvalue and its eigenvector. The eigenvalue summarizes the weakest information; the eigenvector reveals which parameter combination is weak.
Check for singularity or near-singularity. A zero eigenvalue indicates a locally unidentified direction. A very small eigenvalue signals potentially unstable precision, even if numerical software reports an inverse.
Check units and parameterization. The eigenvalue has information units tied to the parameter coordinates. Rescaling a parameter changes the numerical criterion and can change the ranking, so comparisons require a defensible common scale.
Check sensitivity to nominal values. In nonlinear models, an E-optimal design at one nominal parameter vector need not remain optimal elsewhere.

A further limitation is that the objective is controlled by the weakest eigenspace. If the smallest eigenvalue is repeated, there is more than one equally weak direction, and a unique weakest eigenvector need not exist. If eigenvalues are close, small changes in nominal values or numerical calculation can substantially change the reported eigenvector. The smallest eigenvalue remains a defined objective, but its associated direction should not be overinterpreted as uniquely determined in those cases.

Common pitfall: “Design A has a very large information eigenvalue, so it must be E-optimal.” This reasoning ignores the other eigenvalues. E-optimality uses the smallest one: a single poorly informed direction can determine the criterion, regardless of how large the information is in stronger directions.

E-optimality is therefore most useful when avoiding a poorly estimated parameter combination is a primary design goal and the parameter scales are meaningful. It does not guarantee good precision for every named parameter, protect against model misspecification, or remove dependence on nominal values. The eigenstructure should be considered alongside the scientific estimand and the assumptions used to calculate the information matrix.

Check Your Understanding

Answer each question, then compare with the short solutions below.

For $M=\operatorname{diag}(10,2)$, what is the E criterion value, and which coordinate direction is weakest?
Two designs have smallest information eigenvalues 3 and 5. Which is E-preferred, assuming the same parameterization and constraints?
Why does a zero smallest eigenvalue prevent the usual full-parameter inverse-information covariance approximation?
In the allocation example, what is the total information matrix and E criterion at $n_A=1,\ n_B=9$?
Why can rescaling a parameter change an E-optimal ranking?

Solutions. (1) The smallest eigenvalue is 2, and the second coordinate direction is weakest. (2) The design with smallest eigenvalue 5 is E-preferred because it has greater information in its least-informed direction. (3) A zero eigenvalue makes the information matrix singular, so its inverse does not exist; at least one local direction is unidentified. (4) The matrix is $\operatorname{diag}(4,9)$, and the E criterion is $\min(4,9)=4$. (5) E-optimality measures the smallest eigenvalue using unit-length directions in the chosen coordinates; rescaling changes those directions and eigenvalues, and may change the design ranking.