Tutorials › Causal Inference › Empirical Welfare Maximization over Policy Classes

Causal Learning and Policy Decisions · Tutorial 964 of 972

Empirical Welfare Maximization over Policy Classes

Formulate empirical welfare maximization as a search over a fixed class of treatment rules, and calculate and compare their inverse-probability value scores.

Advanced 12 min read

What You'll Learn

  • Define a treatment policy class and its empirical welfare maximizer.
  • Construct inverse-probability scores for binary treatment policies.
  • State the identification conditions that justify using those scores.
  • Compare candidate rules using their estimated values on the same data.
  • Distinguish maximizing within a class from finding the unrestricted optimal rule.
  • Recognize why a high in-sample estimated value is not proof of high true value.

From an Optimal Rule to a Rule We Can Choose

An optimal treatment rule is defined in terms of the value it would achieve in the population. In practice, that value is unknown: we observe outcomes under the treatment actually received, not the outcomes each person would have had under both treatment options. A decision-maker also may not be willing or able to choose among every mathematically possible rule. The practical question is therefore often narrower: which rule in a prespecified set of candidate rules has the highest estimated value in the available data?

Empirical welfare maximization (EWM) formalizes this choice. The analyst defines a class of allowable treatment rules, calculates an estimated welfare for each rule, and selects a rule attaining the largest estimate. The class might contain a small set of rules selected in advance or all rules of a specified form. The important point here is that EWM maximizes over the stated class; it does not automatically recover the best rule among all conceivable policies.

Definition: A deterministic treatment policy is a function \(\pi(X)\) taking values in \(\{0,1\}\), where \(\pi(X)=1\) assigns treatment and \(\pi(X)=0\) assigns control for covariates \(X\). A policy class \(\Pi\) is a prespecified set of such rules. For a rule \(\pi\), its value, or welfare, is the population mean outcome that would result if everyone were assigned according to that rule: $$ V(\pi)=E\left[\pi(X)Y(1)+\{1-\pi(X)\}Y(0)\right]. $$

Here \(A\in\{0,1\}\) denotes the observed treatment assignment, and \(Y\) is the observed outcome. The potential outcomes \(Y(1)\) and \(Y(0)\) are the outcomes under treatment and control. Assume a larger outcome represents greater welfare; if the scientific goal is to minimize an outcome, it must first be expressed as a reward or utility for which larger is better.

The population-optimal rule and the empirical class-optimal rule are different objects. The former maximizes \(V(\pi)\) over all allowable rules. EWM instead chooses a maximizer over \(\Pi\), based on estimates from a sample. A restricted class can be useful for making decisions manageable, but the resulting rule is only guaranteed to be the best estimated rule within that class.

Inverse-Probability Scores for a Policy

To compare policies using observed data, let \(e(X)=P(A=1\mid X)\) be the treatment propensity. The inverse-probability score for an individual under policy \(\pi\) is $$ \Gamma(\pi)=\frac{A\pi(X)Y}{e(X)} +\frac{(1-A)\{1-\pi(X)\}Y}{1-e(X)}. $$

For someone assigned treatment, this score contributes an inverse-probability-weighted outcome only if the policy would also assign treatment. For someone assigned control, it contributes a weighted outcome only if the policy would assign control. The sample estimated value is the average score:

$$ \widehat V(\pi)=\frac{1}{n}\sum_{i=1}^{n} \left[ \frac{A_i\pi(X_i)Y_i}{e(X_i)} +\frac{(1-A_i)\{1-\pi(X_i)\}Y_i}{1-e(X_i)} \right]. $$

The score is justified under standard identification conditions. First, consistency means the observed outcome equals the potential outcome corresponding to the treatment received: \(Y=AY(1)+(1-A)Y(0)\). Second, conditional exchangeability requires \(\{Y(1),Y(0)\}\) to be independent of \(A\) given \(X\). Third, positivity requires \(0<e(X)<1\) for covariate values relevant to the policies being considered. In addition, the treatment assignment and outcome are defined so that one person's assignment does not alter another person's outcome.

In a randomized trial, the propensity may be known from the randomization design. In observational data, it is typically estimated from covariates; the score then uses the estimated propensity in place of \(e(X)\). The examples below treat the propensity as known, so the arithmetic illustrates the score itself without adding uncertainty from propensity estimation.

Why the Score Targets Policy Value

For a fixed policy \(\pi\), the conditional expectation of the treated part of the score is $$ E\left[\frac{A\pi(X)Y}{e(X)}\middle|X\right] =\frac{\pi(X)}{e(X)}E[AY\mid X] =\pi(X)E[Y(1)\mid X]. $$ The last equality follows because \(E[AY\mid X]=e(X)E[Y(1)\mid X]\), using consistency and conditional exchangeability. Similarly, $$ E\left[\frac{(1-A)\{1-\pi(X)\}Y}{1-e(X)}\middle|X\right] =\{1-\pi(X)\}E[Y(0)\mid X]. $$ Adding these expressions and averaging over \(X\) gives $$ E[\Gamma(\pi)]=E[\pi(X)Y(1)+\{1-\pi(X)\}Y(0)]=V(\pi). $$ Thus, under the stated conditions, the score has expectation equal to the value of a fixed policy. This justifies using its sample average as an estimated value.

The qualification “fixed policy” matters. If many policies are compared and the same data are used to select the largest score, the winning estimated value is not thereby an unbiased estimate of the selected policy’s true value. The scores can fluctuate, and selection favors policies with unusually high fluctuations as well as those with genuinely high value.

The Empirical Welfare Maximizer

Given data and a policy class \(\Pi\), the empirical welfare maximizer is any rule in the class whose estimated value is at least as large as that of every other rule in the class:

$$ \widehat\pi\in\underset{\pi\in\Pi}{\operatorname{arg\,max}}\ \widehat V(\pi). $$

If more than one rule ties for the largest estimate, each is a maximizer; a tie-breaking rule can be specified for implementation. The calculation itself is straightforward, but the definition of \(\Pi\) is consequential. It determines which policies can be selected and therefore what “best” means in the EWM objective.

1
Specify the policy class.
List or define the candidate rules before comparing their estimated values.
2
Calculate each score.
For every observation and candidate rule, include the weighted observed outcome only for the treatment arm that the rule assigns.
3
Average and select.
Average each rule’s scores over the sample and choose a rule with the largest average.

Worked Example: Comparing Four Group-Based Rules

Suppose a hypothetical sample of six people has two covariate groups, low and high. The treatment propensity is known to be \(0.5\) for everyone. The candidate class contains four rules: treat everyone, treat no one, treat only the low group, and treat only the high group. The observed data are:

PersonGroup\(A\)\(Y\)
1Low110
2Low06
3Low18
4High09
5High15
6High04

For a treatment assignment, the score contribution is \(Y/0.5=2Y\) if the person received treatment, and zero if they received control. For a control assignment, it is \(Y/0.5=2Y\) if the person received control, and zero if they received treatment.

For the rule “treat only the low group,” the contributions are \(20,12,16,18,0,8\). For example, person 2 is assigned control by this rule and was observed under control, so the contribution is \(6/0.5=12\); person 5 is assigned control but was observed under treatment, so the contribution is zero. Therefore, $$ \widehat V(\pi_{\text{low}}) =\frac{20+12+16+18+0+8}{6} =\frac{74}{6} =12.33. $$

For “treat only the high group,” the contributions are \(0,12,0,0,10,0\), giving \(22/6=3.667\). For “treat everyone,” only the treated observations contribute: \(20+16+10=46\), so the estimate is \(46/6=7.667\). For “treat no one,” the control observations contribute \(12+18+8=38\), so the estimate is \(38/6=6.333\).

Among these four candidates, the empirical welfare maximizer is “treat only the low group,” with estimated value 12.33 outcome units per person. This is a sample-based comparison under the stated score and assumptions. It does not establish that the low-group rule has the greatest true population value.

Inverse-Probability Weights When Propensities Differ

Propensities need not be equal across covariate groups. When treatment assignment is less likely for a group, outcomes from the treated members of that group receive larger inverse-probability weights. Conversely, control outcomes receive weights based on \(1-e(X)\). Correctly using the propensity for each person is essential; using a single pooled propensity when assignment probabilities vary can misrepresent the target value.

Worked Example: Comparing Rules Under Unequal Propensities

Consider four hypothetical observations. The known propensity is \(0.75\) in the low group and \(0.25\) in the high group. The candidate class again contains treat everyone, treat no one, treat only low, and treat only high.

PersonGroup\(e(X)\)\(A\)\(Y\)
1Low0.7518
2Low0.7504
3High0.2513
4High0.25010

For “treat only low,” person 1 contributes \(8/0.75=10.667\), person 2 contributes \(4/(1-0.75)=16\), and person 4 contributes \(10/(1-0.25)=13.333\). Person 3 contributes zero because the policy assigns control but the observed treatment was treatment. Thus $$ \widehat V(\pi_{\text{low}}) =\frac{10.667+16+13.333+0}{4} =10.00. $$

For “treat everyone,” the two treated observations contribute \(8/0.75=10.667\) and \(3/0.25=12\), while the control observations contribute zero. The estimate is \(22.667/4=5.667\). For “treat no one,” the contributions are \(4/0.25=16\) and \(10/0.75=13.333\), giving \(29.333/4=7.333\). For “treat only high,” the low-group control observation contributes 16 and the high-group treated observation contributes 12; the other two contributions are zero. The estimate is \(28/4=7.000\).

The highest estimated value in this class is again for “treat only low,” at 10.00 outcome units per person. The inverse weights are not optional adjustments: they are what makes the score’s expectation target the policy value rather than simply the average outcome among people whose observed assignment happened to match the policy.

A Class Can Change the Selected Rule

The class is part of the decision procedure, not merely a computational detail. A class that includes only a few candidate rules may be easier to describe and compare, but it can exclude a rule with greater population value. A richer class permits more choices, while also allowing more opportunities for sample noise to affect which rule wins.

For example, a class might consist of thresholds on one prespecified covariate, with a finite set of candidate cutoffs. EWM then compares those threshold rules only. It does not silently search over different covariates, combinations of covariates, or every possible treatment assignment. Those would be different classes and potentially different searches.

Worked Example: Selecting Among Threshold Policies

Suppose five hypothetical observations have covariate values \(X=1,2,3,4,5\), treatment propensity \(0.5\) for everyone, and the following observed assignments and outcomes. The prespecified class consists of never treating, treating everyone, treating when \(X\geq3\), and treating when \(X\geq5\).

\(X\)\(A\)\(Y\)
117
205
314
408
516

The treatment-arm contributions for the three treated observations are \(7/0.5=14\), \(4/0.5=8\), and \(6/0.5=12\). The control-arm contributions for the two control observations are \(5/0.5=10\) and \(8/0.5=16\).

Treating everyone gives \((14+8+12)/5=6.800\). Never treating gives \((10+16)/5=5.200\). The rule \(X\geq3\) treats observations 3 through 5 and controls observations 1 and 2; its contributions total \(10+8+12=30\), so its estimated value is \(30/5=6.000\). The rule \(X\geq5\) controls observations 1 through 4 and treats observation 5; its contributions total \(10+16+12=38\), so its estimated value is \(38/5=7.600\).

The empirical maximizer within this four-rule class is the threshold rule \(X\geq5\). The result means only that this rule has the highest inverse-probability estimated value among the four candidates on these data. It does not establish that 5 is a population-optimal cutoff, or that the same cutoff would win in new data.

Interpretation and Common Pitfalls

A useful report of an EWM result names the class, the score used, and the selected candidate. For example: “Among the four prespecified group-based rules, treating only the low group had the highest inverse-probability estimated value in this sample.” That statement accurately describes the optimization. It does not overstate the result as proof of causal benefit for every person assigned treatment, or as evidence that the selected rule is globally optimal.

  • Choosing a candidate after seeing which one scored best. Calling a class “prespecified” while defining it around the observed winner hides part of the search. The reported class should reflect the actual candidate-generating procedure.
  • Forgetting the denominator for the observed arm. A treated outcome is weighted by \(1/e(X)\); a control outcome is weighted by \(1/\{1-e(X)\}\). Swapping these weights changes the score.
  • Treating zero contributions as zero outcomes. A zero score contribution means the observed treatment arm did not match the policy assignment for that person. It does not mean that person’s unobserved outcome was zero.
  • Equating the sample winner with the population optimum. The EWM rule maximizes estimated value within \(\Pi\). Sampling variation and the limits of the class remain relevant to its true value.
  • Ignoring poor overlap. When a propensity is close to zero or one, the corresponding inverse weight can be large. Positivity is required for identification, and extreme weights can make estimated values highly variable.
Key takeaway: Empirical welfare maximization is the rule \(\widehat\pi\in\operatorname{arg\,max}_{\pi\in\Pi}\widehat V(\pi)\). Its meaning depends on the policy class, and its inverse-probability scores target policy value only under the assumptions supporting identification.

Check Your Understanding

Try these questions using the definitions and score in this tutorial.

  1. In your own words, distinguish the empirical welfare maximizer from a population-optimal treatment rule.
  2. For a person assigned treatment, what condition makes that person contribute to the inverse-probability score for a policy?
  3. State the role of conditional exchangeability and positivity in interpreting the score as an estimate of policy value.
  4. Suppose two policies have estimated values of 8.1 and 7.6 outcome units per person. What conclusion can be made if these are the only two policies in the prespecified class, and what cannot be concluded?
  5. Why can a policy’s largest in-sample estimated value fail to be an unbiased estimate of that selected policy’s true value?
Need help applying this method to your own data?

Our team supports biostatistics, statistical programming and pharmacometric analyses. Send us a request and we will reply by email.

Request an analysis
Clinical Biostats