D-Efficiency Relative To A Benchmark

Why Compare Designs to a Benchmark?

A determinant summarizes the joint information in a parameter vector, but its raw value is difficult to interpret on its own. It depends on the number and scale of the parameters, and it has no direct meaning without a reference. Relative D-efficiency makes the comparison explicit: it asks how the determinant of a candidate design's information matrix compares with that of a stated benchmark design.

The benchmark might be a current design, a standard allocation, or a proposed design that is regarded as a target. It need not be optimal. A relative efficiency greater than one means the candidate has a larger information determinant than the benchmark under the stated comparison; it does not, by itself, establish that either design is best among all feasible designs.

Definition: Suppose two designs, $\xi$ and $\xi_B$, estimate the same $k$-dimensional parameter vector, and their information matrices $M(\xi)$ and $M(\xi_B)$ are positive definite and evaluated under the same model conditions. The relative D-efficiency of $\xi$ with respect to benchmark $\xi_B$ is $$ \operatorname{Eff}_D(\xi;\xi_B) = \left( \frac{\det M(\xi)}{\det M(\xi_B)} \right)^{1/k}. $$ It is often reported as a percentage by multiplying this value by $100\%$.

For a fair comparison, the matrices must describe the same parameter vector in the same parameterization and be evaluated at the same nominal parameter values when the information is local. The designs should also have the same resource level if the question is about allocation quality at equal cost. If they use different numbers of observations, the comparison must say whether it uses total information or information normalized to a common sample size. A different resource level can change the determinant even when the allocation pattern is unchanged.

What the Parameter-Dimension Root Means

Let $M_B=M(\xi_B)$ be the benchmark information matrix and $M=M(\xi)$ the candidate matrix. Since $M_B$ is positive definite, it has a positive definite square root. The matrix $M_B^{-1/2} M M_B^{-1/2}$ describes candidate information relative to the benchmark in transformed parameter directions. Let its positive eigenvalues be $\lambda_1,\ldots,\lambda_k$. The determinant identity gives

\frac{\det M}{\det M_B} = \det\!\left(M_B^{-1/2}MM_B^{-1/2}\right) = \prod_{j=1}^{k}\lambda_j.

Consequently,

\operatorname{Eff}_D(\xi;\xi_B) = \left(\prod_{j=1}^{k}\lambda_j\right)^{1/k}.

This is the geometric mean of the relative information eigenvalues. Thus, if candidate information is exactly $c$ times benchmark information in every direction, every $\lambda_j=c$, and relative D-efficiency is $c$, whatever the dimension. But the root does not make the measure generally invariant to changing the parameter dimension. If a new direction has a different relative information level, it changes the geometric mean. The root expresses the determinant ratio on an average multiplicative scale; it does not remove the effect of dimension.

The comparison is also unchanged by applying the same nonsingular linear reparameterization to both designs. If $J$ is the Jacobian for that transformation, each information matrix becomes $J^{T}MJ$, whose determinant is $(\det J)^2\det M$. The common factor cancels in the determinant ratio. This cancellation requires the same transformation for candidate and benchmark; it does not justify comparing matrices for different parameter vectors or incompatible parameterizations.

Worked Example: A Two-Parameter Candidate Against a Benchmark

Suppose a hypothetical two-parameter model has benchmark information matrix $M_B=\operatorname{diag}(4,9)$ and candidate information matrix $M=\operatorname{diag}(8,9)$. Both matrices are positive definite, refer to the same two parameters at the same nominal values, and represent the same experimental resource.

The determinants are $\det M_B=4(9)=36$ and $\det M=8(9)=72$. With $k=2$, $$ \operatorname{Eff}_D = \left(\frac{72}{36}\right)^{1/2} = \sqrt{2} \approx 1.414. $$

The candidate's relative D-efficiency is about $141.4\%$ of the benchmark's. In this example the relative information eigenvalues are $2$ and $1$: the candidate doubles information in one direction and leaves the other unchanged. Their geometric mean is $\sqrt{2}$, not $2$. The result describes the determinant-based joint comparison, not a doubling of information in every parameter direction.

Comparing More Than One Design to the Same Benchmark

When several candidates are assessed against a common benchmark, calculate each determinant ratio using that same benchmark, then take the root using the common parameter dimension. This makes the reference for every reported value explicit and allows the candidates to be ranked by their relative D-efficiencies. The calculation does not imply that the benchmark is optimal or that the highest-ranked candidate satisfies constraints not included in the comparison.

Worked Example: Ranking Two Three-Parameter Designs

Consider a hypothetical three-parameter problem with a positive definite benchmark matrix $M_B=\operatorname{diag}(2,4,8)$. Two candidate designs have $M_A=\operatorname{diag}(4,4,8)$ and $M_C=\operatorname{diag}(3,8,8)$. Assume all matrices use the same parameterization, nominal values, and resource level.

The benchmark determinant is $2(4)(8)=64$. For candidate A, $\det M_A=4(4)(8)=128$, so $$ \operatorname{Eff}_D(A;\xi_B) = \left(\frac{128}{64}\right)^{1/3} = 2^{1/3} \approx 1.260. $$

For candidate C, $\det M_C=3(8)(8)=192$, so $$ \operatorname{Eff}_D(C;\xi_B) = \left(\frac{192}{64}\right)^{1/3} = 3^{1/3} \approx 1.442. $$

Both candidates have determinants greater than the benchmark's, and candidate C has the larger relative D-efficiency: about $144.2\%$, compared with $126.0\%$ for A. The comparison is based on the determinant across all three parameters; it does not say that C has more information in every individual parameter direction than A.

Dimension, Replication, and Interpretation

The parameter dimension in the exponent is the number of parameters represented by the information matrices, not the number of observations or design points. The following hypothetical comparison shows why changing dimension can change the geometric mean even when the added direction has relative information equal to one.

Worked Example: Why the Root Is Not Dimension-Invariant

In a one-parameter comparison, suppose the candidate-to-benchmark information ratio is $2$. Then $$ \operatorname{Eff}_D=2^{1/1}=2. $$ Now consider a two-parameter comparison whose relative information eigenvalues are $2$ and $1$. The determinant ratio is $2(1)=2$, but the efficiency is $$ \operatorname{Eff}_D=(2)^{1/2}\approx 1.414. $$

The added direction contributes no relative gain or loss, yet the geometric mean changes from $2$ to about $1.414$, because the product is averaged over two directions rather than one. If instead the second eigenvalue were also $2$, the two-dimensional efficiency would be $(2\cdot2)^{1/2}=2$. A common proportional increase in every direction retains that common scale across dimensions; an arbitrary added direction need not.

For a design with $n$ independent replicates at a fixed allocation, information scales as $n$: $M_n=nM_1$. For a $k$-parameter model, this implies $\det(M_n)=n^k\det(M_1)$, so the relative D-efficiency scales linearly with the ratio of replicate counts when comparing the same allocation pattern. This is why an unequal-sample-size comparison using total information can reflect both allocation and sample size. To isolate allocation quality, compare information matrices on a common per-observation or common-resource basis.

Assumptions, Diagnostics, and Failure Modes

The determinant-ratio definition requires both matrices to be nonsingular and positive definite. If either matrix is singular, its determinant is zero and the ordinary inverse-based local covariance interpretation fails; the relative D-efficiency formula may be zero or undefined, and a finite positive comparison is not available. Singularity can indicate that the design does not locally identify all components of the stated parameter vector. A small determinant can also arise from very weak information in at least one direction, even if other directions are well informed.

Before reporting a result, check that the candidate and benchmark have the same parameter dimension, refer to the same estimands, use the same local nominal values when appropriate, and have comparable resource levels. Verify that the determinant calculation uses the full information matrices rather than only their diagonal entries: off-diagonal terms encode parameter dependence and affect the determinant. For numerical stability, compute log determinants and then calculate $$ \operatorname{Eff}_D = \exp\!\left( \frac{\log\det M-\log\det M_B}{k} \right). $$ This avoids avoidable overflow or underflow for matrices with extreme scales. A failed Cholesky factorization or a nonpositive computed determinant is a reason to investigate singularity, numerical conditioning, or an implementation error, not to report a routine finite efficiency without qualification.

If the benchmark is claimed to be D-optimal over a specified feasible set, then no feasible candidate under the same model, parameter values, and resource constraints should have relative D-efficiency greater than one. A computed value above one is a diagnostic: perhaps the benchmark is not actually optimal, or the matrices were evaluated under inconsistent assumptions or resource levels. If the benchmark is merely a reference design, a value above one is entirely possible and is interpreted as a larger determinant than that reference.

Quantity reported	What it tells you	What it does not tell you
Determinant ratio $\det M/\det M_B$	The total multiplicative change in determinant-based information.	A dimension-normalized information scale.
Relative D-efficiency	The geometric-mean relative information scale across $k$ directions.	The relative gain in each direction separately.
Relative efficiency above one	The candidate determinant exceeds the benchmark determinant under matched conditions.	That the candidate is globally optimal or superior on every design objective.

Common Mistakes and Appropriate Comparisons

Omitting the root. The raw determinant ratio is a total product over $k$ dimensions. Relative D-efficiency takes its $k$th root to report a geometric-mean relative information scale.
Claiming dimension invariance. The root does not generally eliminate the effect of changing $k$. Adding a direction changes the geometric mean unless its relative information scale matches the existing common scale.
Assuming every efficiency must be at most one. This is expected only when the benchmark is truly optimal over the same feasible set and under matched conditions. A routine benchmark can be outperformed.
Comparing different sample sizes without saying so. Total information includes resource differences. Normalize or explain the resource basis when the aim is to compare allocations.
Comparing different parameter vectors. A determinant ratio requires corresponding information matrices for the same estimands and dimension. The root cannot repair a mismatch.

The raw determinant is useful when its absolute value is the quantity of interest under a fixed parameterization and dimension. The normalized relative D-efficiency is easier to communicate across candidates in the same problem because it expresses the comparison on a geometric-mean scale and uses a stated reference. Neither summary replaces examination of design constraints, model assumptions, or the information pattern across directions. Other optimality criteria target different summaries of uncertainty; relative D-efficiency answers specifically the determinant-based joint-information question.

Key takeaway: State the benchmark, verify that both information matrices are positive definite and comparable, calculate their determinant ratio, and take the reciprocal-$k$th power. Interpret the result as a geometric-mean relative information scale, not as a direction-by-direction guarantee or a dimension-invariant quantity.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A two-parameter candidate has determinant $45$, while its benchmark has determinant $20$. Calculate the relative D-efficiency and interpret it.
A three-parameter candidate has relative information eigenvalues $1.5$, $1$, and $0.8$. Calculate its relative D-efficiency.
Why can a candidate have a relative D-efficiency above one without contradicting the definition?
What is wrong with comparing determinants for two designs that estimate parameter vectors of different dimensions and then using the dimension of one model for the root?
For a fixed allocation, total information doubles when the replicate count doubles. By what factor does relative D-efficiency change against a fixed-resource benchmark of the original replicate count?

Solutions. (1) The determinant ratio is $45/20=2.25$, so $\operatorname{Eff}_D=2.25^{1/2}=1.5$. The candidate is at $150\%$ of the benchmark's determinant-based relative information scale, under matched conditions. (2) The determinant ratio is the product $1.5(1)(0.8)=1.2$. Thus $\operatorname{Eff}_D=1.2^{1/3}\approx1.063$, or about $106.3\%$. (3) The benchmark may simply be a reference rather than an optimum. A candidate with a larger determinant has a relative efficiency above one. (4) The definition requires the same parameter dimension and corresponding estimands; a root based on one dimension cannot make incompatible parameter spaces comparable. (5) If candidate information is doubled in all directions while the benchmark is fixed, the determinant ratio is multiplied by $2^k$. Taking the $k$th root multiplies relative D-efficiency by $2$.

Quantity reported	What it tells you	What it does not tell you
Determinant ratio \(\det M/\det M_B\)	The total multiplicative change in determinant-based information.	A dimension-normalized information scale.
Relative D-efficiency	The geometric-mean relative information scale across \(k\) directions.	The relative gain in each direction separately.
Relative efficiency above one	The candidate determinant exceeds the benchmark determinant under matched conditions.	That the candidate is globally optimal or superior on every design objective.