Worked Comparison Of Alphabetic Criteria

Why Compare Criteria Numerically?

Two candidate designs can contain different patterns of information about the same model parameters. A design may provide strong information overall while leaving one parameter direction relatively uncertain; another may distribute information more evenly. Alphabetic criteria reduce these patterns to numerical summaries, but each summary encodes a different objective. A comparison is therefore not complete until the criterion, its direction of preference, and the scientific target are all stated.

Here, two hypothetical information matrices will be compared using D-, A-, E-, and c-optimality. The calculations assume both matrices describe the same parameter vector, in the same parameterization and units, at the same nominal parameter values, under comparable design constraints. Those conditions matter: changing parameter scaling, the target contrast, or the experimental budget can change the numerical values and sometimes the ranking.

Some sources report D-error, \(\det(M)^{-1/p}\), instead of the determinant itself. D-error is smaller when information is better, so it reverses the direction of the D-value ranking without changing the preferred design. Likewise, E-optimality is sometimes expressed through the largest eigenvalue of \(M^{-1}\), which is \(1/\lambda_{\min}(M)\) and is minimized. We will keep to the conventions in the definition box.

What the Four Values Measure

D-optimality evaluates the determinant of information. Equivalently, it minimizes the determinant of the approximate covariance matrix, since \(\det(M^{-1})=1/\det(M)\). It summarizes joint precision across the full parameter vector, rather than precision in one named parameter. A-optimality sums the diagonal elements of \(M^{-1}\), so it minimizes the sum of approximate marginal variances in the chosen parameter coordinates. E-optimality protects the least-informed direction: the smallest eigenvalue of \(M\) corresponds to the largest eigenvalue of \(M^{-1}\), the greatest variance direction under the approximation.

For c-optimality, \(c\) specifies the target. If the target is the linear contrast \(c^\mathsf{T}\theta\), its approximate variance is \(c^\mathsf{T}M^{-1}c\). Changing \(c\) changes the question being answered. For example, \(c=(1,0)^\mathsf{T}\) targets the first parameter, while \(c=(0,1)^\mathsf{T}\) targets the second. These criteria are not interchangeable scores of general quality: a design can be preferred by one and not by another.

Worked Comparison of Two Hypothetical Matrices

Suppose two hypothetical designs produce the following information matrices for the same two parameters. All entries are in the same chosen parameter coordinates.

Both matrices are symmetric and positive definite. For \(M_1\), the diagonal entries are positive. For \(M_2\), its eigenvalues are \(12\) and \(8\), both positive. Thus both have inverses and the four criteria can be calculated. Positive definiteness is a substantive requirement here, not merely a convenience: a singular information matrix does not yield a finite covariance approximation for every parameter direction.

Why the Calculations Have These Forms

The interpretation of c-optimality follows directly from the covariance approximation. If the estimated parameter vector has approximate covariance \(M^{-1}\), the variance of a linear combination \(c^\mathsf{T}\hat{\theta}\) is obtained by applying the covariance rule for a linear transformation:

This also explains why a c-value must be accompanied by its contrast. Without \(c\), there is no specified target variance to evaluate. A-optimality sums the marginal variances because the trace of a covariance matrix is the sum of its diagonal entries. For E-optimality, diagonalizing a symmetric positive-definite matrix gives eigenvalues \(\lambda_1,\ldots,\lambda_p\); the inverse has the same eigenvectors and eigenvalues \(1/\lambda_1,\ldots,1/\lambda_p\). Thus the largest variance eigenvalue is \(1/\lambda_{\min}(M)\), which motivates maximizing the smallest information eigenvalue.

For D-optimality, \(\det(M^{-1})=1/\det(M)\), so maximizing the determinant of information minimizes the determinant of the approximate covariance matrix. In two dimensions, this determinant summarizes the area scale of the joint uncertainty ellipse. This derivation also shows why the D-value convention and D-error convention point in opposite numerical directions.

Reading the Ranking in Context

Three of the four rows favor Design 2, but that does not make it universally superior. If the scientific priority is precise estimation of the first parameter, the stated c-criterion favors Design 1. If the priority is joint precision, summed marginal variance, or protection of the weakest information direction, the displayed criteria favor Design 2. A decision should be tied to the purpose of the experiment, not made by counting how many criteria favor each design.

The table also illustrates why “more balanced” is not always the only goal. \(M_1\) has very high information in its first coordinate and low information in its second. \(M_2\) distributes information more evenly. The targeted first-parameter contrast benefits from the strong first-coordinate information in \(M_1\), while the other criteria reward the more balanced matrix in this particular comparison.

Assumptions, Diagnostics, and Limitations

These calculations are meaningful as design comparisons when the matrices use a common parameterization and scale, correspond to comparable assumptions and nominal parameter values, and reflect comparable design effort. The covariance interpretation additionally relies on the usual local and asymptotic approximation being adequate for the model and design under consideration. It is not a guarantee that realized finite-sample uncertainty will equal \(M^{-1}\).

Before accepting a ranking, check the following:

• Check symmetry and positive definiteness. A negative eigenvalue often signals an invalid information calculation or numerical error. A zero eigenvalue means at least one direction is not informed; the ordinary inverse-based A- and c-values are then unavailable or unbounded for affected targets.
• Check parameter ordering and units. An inverse or contrast computed after silently reordering parameters can give a plausible but incorrect answer. The entries of \(c\) must match the parameter order, and its definition must match the scientific target.
• Check scaling. A- and E-values depend on parameter coordinates, and the numerical c-value depends on how the contrast is expressed. The D-value also changes under reparameterization, although a common nonsingular linear reparameterization multiplies all candidate determinants by the same factor when applied consistently.
• Check numerical conditioning. When eigenvalues differ greatly in magnitude, rounding can distort an inverse or a small eigenvalue. Use stable numerical linear algebra and retain enough precision for the comparison.

A criterion compresses the information matrix and can hide details. D-optimality is not a promise of small variance for every individual parameter; A-optimality weights parameter coordinates through their variances; E-optimality may be conservative if its least-informed direction is not scientifically important; and c-optimality addresses only the stated contrast. If objectives conflict, the comparison should make the objective explicit and may motivate a compound criterion or additional scientific constraints, rather than treating one scalar as an all-purpose measure.

Common Mistake

A frequent error is to say, “The largest number is the best design,” after calculating all four values. That reasoning compares unlike quantities and also gets the preference direction wrong for A- and c-values. The correction is to report each value with its definition and direction: maximize \(\det(M)\) and \(\lambda_{\min}(M)\), but minimize \(\operatorname{tr}(M^{-1})\) and \(c^\mathsf{T}M^{-1}c\). Then state the target and assumptions that make the ranking relevant.