Constrained Policy Learning With Heterogeneous Effects

From Estimated Effects to an Allocation

Learning which units are likely to benefit from treatment does not, by itself, decide whom to treat. A service may have capacity for only a fixed number of clients, or each subgroup may face its own staffing or budget limit. The allocation problem is to use estimated treatment gains to choose a feasible set of recipients, rather than to treat everyone whose predicted gain is positive.

Suppose \(X_i\) denotes the pre-treatment characteristics of unit \(i\), and \(Y_i(1)\) and \(Y_i(0)\) are its potential outcomes under treatment and no treatment. Define the conditional average treatment effect, or conditional gain, as \(\tau(x)=E[Y(1)-Y(0)\mid X=x]\). Assume higher outcomes are preferred; if the outcome is a cost to be reduced, the gain can instead be defined with the sign reversed. A policy \(\pi(x)\) assigns treatment when its value is 1 and no treatment when its value is 0.

The identity follows by conditioning on \(X\): under treatment assignment \(\pi(X)\), the conditional expected outcome is \(E[Y(0)\mid X]+\pi(X)\tau(X)\). The term \(E[Y(0)]\) does not depend on the policy, so maximizing policy value is equivalent to maximizing gain relative to no treatment. The optimization methods below take estimated gains as inputs; they do not establish that those estimates are accurate.

For causal interpretation, the gains must be identified from a suitable design or assumptions, such as consistency, no interference between units, and conditional exchangeability given measured covariates, with positivity where needed for estimation. The policy must use characteristics measured before treatment. In practice, let \(\hat{\tau}_i\) be the estimated gain for unit \(i\), obtained, for example, from a fitted CATE model. The allocation then optimizes these estimated gains, not unobserved individual treatment effects.

A Fixed Treatment Capacity

For a finite set of \(n\) units, write \(z_i=1\) if unit \(i\) is assigned treatment and \(z_i=0\) otherwise. If at most \(K\) units can be treated, the estimated-gain allocation is the solution to

With equal treatment costs and a single capacity limit, an optimal allocation treats up to \(K\) units with the largest positive estimated gains. If fewer than \(K\) gains are positive, it is optimal to leave some capacity unused. If the policy must treat exactly \(K\) units, it instead selects the \(K\) largest gains even if some are negative. Ties at the boundary can be resolved in any way that preserves feasibility; if the assignment system permits randomization, tied units can be randomized to meet a target proportion.

Why a Threshold Appears

A capacity-constrained allocation can be described by a threshold on estimated gain. The threshold is a shadow price: it represents the value, in outcome-gain units, of relaxing the capacity limit at the margin. For the at-most-\(K\) problem, introduce a multiplier \(\lambda\ge 0\) and form the Lagrangian

For fixed \(\lambda\), each unit contributes \((\hat{\tau}_i-\lambda)z_i\). Maximizing that contribution over \(z_i\in\{0,1\}\) selects the unit when \(\hat{\tau}_i>\lambda\), excludes it when \(\hat{\tau}_i<\lambda\), and allows either choice at equality. The multiplier is chosen to support a feasible allocation, with any boundary ties handled to meet capacity.

The last qualification matters. A constraint is binding when the allocation uses all \(K\) slots, but this does not alone imply that the multiplier is strictly positive. For example, if every estimated gain is zero and the allocation treats \(K\) units, the capacity binds, yet \(\lambda=0\) is valid: every feasible allocation has the same estimated gain. Strictly positive marginal value of additional capacity requires additional conditions, such as positive gains among units who could use the extra slots.

A brief exchange argument also shows why ranking works. If a feasible allocation treats unit \(j\) but leaves unit \(i\) untreated, where \(\hat{\tau}_i>\hat{\tau}_j\), switching treatment from \(j\) to \(i\) preserves the number treated and changes the objective by \(\hat{\tau}_i-\hat{\tau}_j>0\). Thus an allocation that treats a lower-gain unit while excluding a higher-gain unit cannot be optimal under a single equal-cost capacity constraint.

Subgroup-Specific Limits

A common cutoff may not enforce subgroup rules. If units belong to disjoint groups \(g=1,\ldots,G\), let \(K_g\) be the maximum number treatable in group \(g\), in addition to an overall capacity \(K\). The finite allocation problem becomes

The group limits can force the policy to leave an otherwise high-gain unit untreated while selecting a lower-gain unit from a group with available capacity. When the constraints are relaxed with multipliers, a unit in group \(g\) has adjusted gain \(\hat{\tau}_i-\lambda_0-\lambda_g\), where \(\lambda_0\) corresponds to the overall limit and \(\lambda_g\) to that group’s limit. In this multiplier description, groups can therefore have different effective thresholds. For the integer allocation problem, multipliers help explain the constraints, but one should not assume that a simple threshold rule automatically solves every combination of discrete constraints.

More generally, subgroup limits express operational or policy requirements, not necessarily a claim that the estimated gains are different across groups. Their consequences should be explicit: a group cap can alter who is treated even when another person has a higher estimated gain. If group definitions overlap or constraints interact in more complex ways, the allocation must be formulated with all relevant constraints rather than using separate rankings by group.

Weighted and Indivisible Resources

When people require different amounts of a scarce resource, a treatment-count limit is not enough. Let \(c_i>0\) be the resource required to treat unit \(i\), and let \(B\) be the total available resource. With indivisible assignments, the problem is

This is a binary knapsack problem. In general, sorting by estimated gain alone does not respect resource use, and sorting by gain per resource unit is not guaranteed to maximize total gain when recipients are indivisible. The optimization must account for feasible combinations. If treatment can instead be allocated fractionally or randomized in a way that yields fractional expected resource use, a relaxed problem may have a multiplier-based rule involving adjusted gain per unit of resource. That is a different assignment structure, not a justification for treating indivisible decisions as fractional.

Common Pitfalls and Practical Interpretation

• Confusing estimated-gain optimization with a causal guarantee. The optimizer finds the best feasible allocation according to the supplied \(\hat{\tau}_i\). Estimation error can change rankings, so this does not establish that the selected people have the largest true gains.
• Treating a binding constraint as proof of a positive multiplier. Binding means the limit is reached; the multiplier can still be zero when relaxing the limit has no positive marginal value.
• Applying one threshold despite subgroup limits. A common cutoff may exceed total capacity or violate group limits. Include every constraint in the allocation problem.
• Using a gain-per-cost ranking as an exact solution for indivisible assignments. It can be a heuristic, but, as the example shows, it need not maximize total estimated gain.
• Ignoring the distinction between “at most” and “exactly.” An at-most-capacity policy can leave slots unused when gains are nonpositive; an exactly-\(K\) rule must fill them.

The practical benefit of formulating the optimization explicitly is that it makes the policy’s objective and restrictions auditable. A decision-maker can see whether the rule prioritizes estimated gains, obeys every resource limit, and uses a threshold only where the assumptions justify one. Choices about the capacity, subgroup limits, outcome scale, and treatment costs are part of the policy definition and can change the resulting allocation.