External Benchmarks As Moment Constraints

From an External Benchmark to a Causal Restriction

When treatment assignment is not random, each person reveals only one potential outcome: the outcome under the treatment actually received. An external benchmark can provide information about an average of the potential outcomes that are missing in the study. The useful step is to express that information as a restriction on a moment in the study population, then combine it with observed study data and the known outcome scale.

A benchmark is not automatically evidence about the study population. Its relevance depends on whether the external data measure the right quantity and whether the populations can be connected under a defensible transportability assumption. The calculations below make those requirements explicit.

Assume the outcome is bounded by known limits \(L\leq Y(d)\leq U\) for \(d=0,1\). Consistency gives \(\mu_1=E[Y(1)\mid D=1]\) and \(\mu_0=E[Y(0)\mid D=0]\). The two missing means are \(m_{10}=E[Y(0)\mid D=1]\), the untreated potential-outcome mean among treated people, and \(m_{01}=E[Y(1)\mid D=0]\), the treated potential-outcome mean among controls. Decomposing the population by treatment status gives

This identity separates the observed quantities from the missing means. Without further information, both missing means are restricted only by the outcome scale. An external benchmark can narrow the possible values of one or both.

Turning a No-Treatment Benchmark into a Moment Constraint

Suppose a credible external source provides a benchmark for the mean outcome that the study target population would have under no treatment. Denote this mean by \(c_0=E[Y(0)]\). In practice the source may report a point value \(b\), or a defensible interval \([b_L,b_U]\). If the benchmark transports to the study population, the law of total expectation yields

Thus a point benchmark \(c_0=b\) implies \(m_{10}=[b-(1-\pi)\mu_0]/\pi\), provided \(\pi>0\). If the credible benchmark is an interval, the implied range is

The outcome-scale limits are included because every potential outcome must lie in \([L,U]\). If the lower endpoint exceeds the upper endpoint, the benchmark interval, observed data, and assumptions cannot all hold together. That is a compatibility warning, not a reason to silently alter the calculation.

A benchmark interval may reflect substantive uncertainty about transport, a range judged credible by subject-matter knowledge, or uncertainty in an estimate. These meanings should not be conflated. A sampling confidence interval has a repeated-sampling interpretation; inserting its endpoints into the bound calculation produces a sensitivity range conditional on that choice, not automatically a confidence interval for \(\tau\).

Deriving the Treatment-Effect Bounds

Let \([m_{10,L},m_{10,U}]\) be the feasible range for \(m_{10}\) obtained from the benchmark and the outcome scale. If no other restriction is available for \(m_{01}\), then \(L\leq m_{01}\leq U\). In the expression for \(\tau\), the coefficient of \(m_{10}\) is \(-\pi\), while the coefficient of \(m_{01}\) is \(1-\pi\). For \(0<\pi<1\), the smallest effect therefore uses the largest feasible \(m_{10}\) and smallest feasible \(m_{01}\); the largest effect uses the smallest feasible \(m_{10}\) and largest feasible \(m_{01}\). Hence

These endpoints are sharp under the stated restrictions: each endpoint can be attained by setting the missing means to the corresponding feasible extreme, while leaving the observed means and benchmark restriction satisfied. This claim assumes no further restrictions linking the two missing means or imposing additional structure on potential outcomes. Additional information, such as a restriction on \(m_{01}\), can tighten the interval too.

Using Benchmarks Within Covariate Strata

A single population-average benchmark may fail to transport if the external and study populations have different covariate distributions. A more defensible route can be to transport conditional no-treatment means within covariate strata, then combine them using the study population's covariate distribution. Let \(X\) denote measured covariates and suppose, for each relevant \(x\), the external information identifies \(c_0(x)=E[Y(0)\mid X=x]\) in the study target population. Then

where \(m_{10}(x)=E[Y(0)\mid D=1,X=x]\) and \(\mu_0(x)=E[Y\mid D=0,X=x]\). This gives a stratum-specific restriction on the missing mean when \(P(D=1\mid X=x)>0\). The population benchmark is then standardized over the target distribution of \(X\), not the external source's distribution. Strata without treated people cannot identify \(m_{10}(x)\) by this equation; the outcome limits still apply there.

Assumption Checks, Alternatives, and Limitations

The calculation is simple only after the substantive assumptions have been made explicit. In particular, check that the benchmark concerns the same outcome definition, measurement scale, follow-up period, and target population as the causal estimand. If it comes from observed untreated units, consider whether treatment selection in the external source means those units fail to represent the source population's \(Y(0)\). If transport is conditional on \(X\), assess whether the relevant covariates are measured comparably, whether the conditional relationship is credible, and whether the target has support in the external data.

A basic numerical diagnostic is feasibility. For example, with \(\pi=0.40\), \(\mu_0=50\), and \(Y(0)\in[0,100]\), the study-implied population mean \(E[Y(0)]=0.40m_{10}+30\) must lie in \([30,70]\). A claimed benchmark range of \([75,80]\) has no overlap with that feasible range. The conflict may indicate incompatible data, a coding or scale error, or a failed transportability assumption. It cannot be resolved by reporting bounds calculated from an impossible moment restriction.

This approach differs from using only outcome-scale limits: the scale limits bound each missing outcome, while the benchmark adds information about an average missing potential outcome. It also differs from imposing a direct bias limit or a shape restriction: those approaches constrain selection or potential-outcome behavior directly, whereas an external benchmark constrains a population moment. The methods can be combined only when their assumptions are jointly credible. The main limitation is that a precise external number can produce misleadingly narrow bounds if it is transported without justification. Conversely, a valid benchmark may still leave wide bounds when the other missing mean is weakly constrained.