Calculating Relative Efficiency From Standard Errors

Why Compare Relative Efficiency?

Two allocation schemes can estimate the same treatment contrast but produce estimates with different precision. A standard error expresses that precision in the units of the response; a variance ratio turns the comparison into a unit-free measure. Relative efficiency then gives a compact answer to questions such as: how much variance does one scheme have compared with another, and how many observations under one scheme would match the precision of the other?

The direction of the ratio must be stated. In this tutorial, “efficiency of B relative to A” compares the variance under A with the variance under B. With this convention, a value above one means B has the smaller variance and is more precise. The ratio does not, by itself, say how many additional observations are needed; that translation requires a sample-size scaling assumption and careful attention to the sample sizes at which the variances were calculated.

Definition: Suppose Plan A and Plan B estimate the same treatment contrast, with variances $V_A$ and $V_B$, respectively. Define the relative efficiency of B to A as $$ \mathrm{RE}_{B:A}=\frac{V_A}{V_B} =\left(\frac{SE_A}{SE_B}\right)^2, $$ where $SE_A=\sqrt{V_A}$ and $SE_B=\sqrt{V_B}$. Thus $\mathrm{RE}_{B:A}>1$ means B has lower variance than A; $\mathrm{RE}_{B:A}<1$ means A has lower variance than B. Both variances must refer to the same estimand and be evaluated on a clearly specified basis.

Model and Conditions for a Meaningful Comparison

A comparison is meaningful only if both plans target the same contrast. For example, if there are two treatment means $\mu_1$ and $\mu_2$, the contrast might be $\theta=\mu_1-\mu_2$. The estimates from A and B must both estimate this $\theta$, not different weighted averages or contrasts. The variance is measured in squared response units, while a standard error is measured in response units; their ratios are dimensionless.

For the two-independent-group examples below, use the model

Y_{ij}=\mu_i+\varepsilon_{ij},\qquad E(\varepsilon_{ij})=0,\qquad \operatorname{Var}(\varepsilon_{ij})=\sigma^2,

where observations are independent, the error variance is the same in both groups, and $j=1,\ldots,n_i$. The estimated difference is $\bar{x}_1-\bar{x}_2$, and its variance is $\sigma^2(1/n_1+1/n_2)$. Normal errors are needed for exact normal-theory inference in small samples, but not for this variance calculation itself. In practice, a standard error may use an estimated variance in place of $\sigma^2$. Whatever method supplies the standard errors, their ratio is comparable only when it applies to the same contrast under appropriately comparable assumptions.

A relative-efficiency calculation is descriptive of precision, not a test of whether the treatment means differ. It also does not establish that one allocation is preferable for every purpose: costs, feasibility, and other design goals are outside the ratio itself.

From Standard Errors to Relative Efficiency

Since variance is the square of standard error, the standard-error ratio must be squared. If $SE_A=4$ units and $SE_B=3$ units, for instance, the ratio $4/3$ is not the relative efficiency: the variance ratio is $(4/3)^2$. This distinction matters because a modest change in standard error corresponds to a different-sized change in variance.

Worked Example: Comparing Two Allocations at the Same Total Size

Suppose two independent groups have a common response standard deviation of 12 points. Both plans estimate the difference $\mu_1-\mu_2$, and both allocate 40 observations in total. Plan A assigns 20 observations to each group; Plan B assigns 30 to group 1 and 10 to group 2. Calculate and interpret the relative efficiency of B to A.

The model assumptions for this calculation are independent observations and a common variance $\sigma^2=12^2=144$ points squared in both groups. The same difference in means is the target under both plans. The variance formula for the difference is $\sigma^2(1/n_1+1/n_2)$.

For Plan A,

V_A=144\left(\frac{1}{20}+\frac{1}{20}\right) =144(0.10)=14.40\ \text{points}^2, \qquad SE_A=\sqrt{14.40}=3.795\ \text{points}.

For Plan B,

V_B=144\left(\frac{1}{30}+\frac{1}{10}\right) =144(0.1333\ldots)=19.20\ \text{points}^2, \qquad SE_B=\sqrt{19.20}=4.382\ \text{points}.

Therefore,

\mathrm{RE}_{B:A} =\frac{14.40}{19.20} =\left(\frac{3.795}{4.382}\right)^2 =0.7500.

The computed ratio is $V_A/V_B=0.75$, which means $V_B=V_A/0.75=1.333V_A$, or 33.3% higher. Thus A is more precise here. The standard errors, 3.795 and 4.382 points, tell the same story: B's is larger. The direction check is useful whenever interpreting a ratio.

Equivalent Sample Size

Relative efficiency can be translated into a sample size only if variance scales predictably as the total sample size changes. A common approximation is inverse proportionality: within a plan, keep the allocation proportions and other design features fixed, and suppose multiplying the total sample size by a factor divides the variance by that factor. This can hold for independent observations with fixed variance components and a fixed allocation pattern. It should not be assumed automatically for every design or variance estimator.

Suppose A is observed at total size $N_A$, with variance $V_A$, and B is observed at total size $N_B$, with variance $V_B$. If Plan A's allocation proportions are held fixed, its variance at a hypothetical size $n$ is

V_A(n)=V_A\frac{N_A}{n}.

To find the Plan A size that matches Plan B's observed variance, set $V_A(n)=V_B$ and solve. This derivation makes the baseline size explicit; the sample-size multiplier is applied to Plan A's own baseline $N_A$.

Write the scaled variance.
At size $n$, Plan A is assumed to have variance $V_A(N_A/n)$.

Set the target precision.
Matching Plan B means $V_A(N_A/n)=V_B$.

Solve for the equivalent size.
Rearranging gives $n=N_A(V_A/V_B)=N_A\mathrm{RE}_{B:A}$.

Result: If $V_A$ and $V_B$ are measured at the same baseline total size $N$, the Plan A size matching Plan B's variance is $N_{A,\mathrm{equiv}}=N\mathrm{RE}_{B:A}$. If their baseline sizes differ, use $N_{A,\mathrm{equiv}}=N_A\mathrm{RE}_{B:A}$, where $N_A$ is the size at which $V_A$ was measured. The matching size is a mathematical equivalent and may not be an integer.

Worked Example: Equivalent Size When Baseline Sizes Differ

Suppose Plan A was evaluated at $N_A=40$ observations and has variance $V_A=0.0500$ response-units squared. Plan B was evaluated at $N_B=90$ observations and has variance $V_B=0.04545$ response-units squared. Assume that, for Plan A, the allocation proportions stay fixed and its variance scales inversely with its total sample size. How large would Plan A need to be to match Plan B's reported variance?

First calculate the relative efficiency using the variances actually given:

\mathrm{RE}_{B:A}=\frac{0.0500}{0.04545}=1.100.

Plan B's variance is smaller, so a relative efficiency above one is consistent with B being more precise at these reported baselines. To find the equivalent size under A, scale from A's own baseline of 40:

N_{A,\mathrm{equiv}} =N_A\frac{V_A}{V_B} =40\left(\frac{0.0500}{0.04545}\right) =44.00.

The check is direct: Plan A's predicted variance at 44 observations is $0.0500(40/44)=0.04545$ response-units squared, matching Plan B's variance. The equivalent is 44 Plan A observations, not $90(1.100)=99$. The latter incorrectly applies the relative-efficiency multiplier to Plan B's baseline, even though the variance being scaled belongs to A. This comparison does not claim that 44 observations under A have the same cost or other features as 90 under B; it matches only the stated variance under the scaling assumption.

Equal Baselines and the Meaning of “Percent More Efficient”

When both variances are evaluated at the same total sample size $N$, the equivalent-size statement is especially easy to interpret. If $\mathrm{RE}_{B:A}=1.25$, Plan A would require $1.25N$ observations to reach Plan B's variance, under the inverse-size scaling assumption. This says 25% more Plan A observations than the shared baseline $N$, not that B uses 25% fewer observations in every possible comparison. The latter phrasing depends on which plan and sample size are treated as the reference.

Worked Example: Translating a Standard-Error Ratio Into an Equivalent Size

At the same baseline total size of 60 observations, two plans estimate the same contrast. Their standard errors are $SE_A=0.400$ units and $SE_B=0.3586$ units. Assume that, within Plan A, the allocation proportions and variance conditions remain fixed as the sample size changes, so its variance is inversely proportional to total size. Find the relative efficiency and the Plan A sample size matching B's variance.

The ratio of standard errors is $0.400/0.3586\approx1.115$. Squaring gives the variance ratio:

\mathrm{RE}_{B:A} =\left(\frac{SE_A}{SE_B}\right)^2 =\left(\frac{0.400}{0.3586}\right)^2 \approx 1.244.

The corresponding variances are $V_A=0.400^2=0.1600$ and $V_B=0.3586^2\approx0.1286$ units squared. Since both are measured at $N=60$, the equivalent Plan A size is

N_{A,\mathrm{equiv}} =60\left(\frac{0.1600}{0.1286}\right) \approx 74.65.

Thus, under the stated scaling assumption, about 74.7 observations under A would match the variance achieved by 60 under B; a practical design would need to choose an integer and recalculate its precision. B has about 1.244 times the relative efficiency of A according to the chosen convention. This is a precision comparison for this contrast, not a claim that all aspects of the plans differ by the same factor.

Common Mistakes and a Practical Checklist

The most common errors are directional or arise from applying a valid formula outside its conditions. In particular, a variance ratio is not a standard-error ratio, and an equivalent sample size must be scaled from the baseline belonging to the plan whose size is being changed.

Using the standard-error ratio without squaring it. The relative efficiency is a variance ratio, so use $(SE_A/SE_B)^2$, not $SE_A/SE_B$.
Reversing the interpretation. With $\mathrm{RE}_{B:A}=V_A/V_B$, a value above one favors B in precision, because $V_B<V_A$. State the ratio's direction before interpreting it.
Comparing different targets. A ratio of standard errors for different contrasts does not describe relative efficiency for one shared estimand.
Using the wrong baseline size. If baseline sizes differ, Plan A's equivalent size is $N_A V_A/V_B$, not $N_B V_A/V_B$. The latter is justified only when the relevant baseline sizes are equal, so that $N_A=N_B=N$.
Assuming inverse-size scaling without checking. Changing sample size while changing allocation proportions, dependence, or variance structure may invalidate the simple scaling formula.

A reliable calculation therefore names the target contrast, labels the two standard errors, squares their ratio, checks which plan has the smaller variance, and identifies the baseline size used in any sample-size translation. Relative efficiency is useful because it separates a precision comparison from the response's measurement units, while the equivalent-size interpretation puts that comparison on a practical scale. Neither interpretation is sound unless the ratio's direction and scaling assumptions are explicit.

Check Your Understanding

For each question, identify the ratio direction and any assumptions needed before interpreting the result.

Plan A has standard error 5 units and Plan B has standard error 4 units for the same contrast. Calculate $\mathrm{RE}_{B:A}$ and say which plan is more precise.
Two plans have variances $V_A=0.12$ and $V_B=0.15$, measured at the same total size. What is $\mathrm{RE}_{B:A}$, and which variance is larger?
Plan A has $N_A=50$ and $V_A=0.080$; Plan B has $N_B=80$ and $V_B=0.064$. Under inverse-size scaling for A, calculate the Plan A size matching B's variance. Which baseline size belongs in the calculation?
Why is it not enough to compare two standard errors if they estimate different treatment contrasts?
State one condition under which the variance of a plan can reasonably be scaled as $V(n)=V(N)N/n$, and one situation in which that scaling may fail.