Why Evaluate a Policy in a Randomized Trial?
A treatment policy assigns a treatment recommendation to each person based on that person’s characteristics. A randomized trial can evaluate a fixed policy even when participants were assigned to treatment independently of that policy. The key is to use the known assignment probabilities: participants whose observed assignment agrees with the policy provide direct information about its outcomes, and inverse-probability weighting accounts for how likely that agreement was.
The resulting estimate concerns the average outcome that would arise if the policy were applied to the target population represented by the trial. It is not simply the average outcome among participants who happened to receive the policy-recommended treatment. That subgroup may differ in its characteristics, and the chance of matching the policy can vary across participants.
The policy is fixed when its recommendations are specified without using the outcomes in the data used for evaluation. The policy might, for example, recommend treatment above a prespecified risk threshold. The target population must also be clear: when trial participants are regarded as a sample from a broader population, the value describes that broader population only to the extent that the trial sample represents it.
Assumptions and the Inverse-Probability Estimator
Write the known probability of assigning treatment \(a\) for someone with characteristics \(X_i\) as \(\pi_a(X_i)=P(A_i=a\mid X_i)\). In a trial with a constant treatment allocation probability, \(\pi_1(X_i)\) and \(\pi_0(X_i)\) do not vary with \(X_i\). In a stratified or otherwise covariate-dependent randomized design, they may vary, but are known from the design.
The estimator below relies on the following conditions. First, consistency means that the observed outcome equals the potential outcome under the treatment actually assigned: \(Y_i=Y_i(A_i)\). Second, the randomized assignment is independent of the potential outcomes conditional on the characteristics used by the design. Third, positivity requires \(\pi_{d(X_i)}(X_i)>0\) for every participant and recommended treatment in the target population. If the policy sometimes recommends an arm that could never be assigned under the trial design, its value cannot be evaluated from that trial by this method. For population-level standard errors and intervals below, participants are treated as independent draws from the target population, with independent trial assignments according to the design.
For each participant, define a policy score that is nonzero only when the randomized assignment matches the policy recommendation. Divide the observed outcome by the probability of that match. Averaging these scores gives the inverse-probability estimator:
Here \(\mathbb{1}\{\cdot\}\) equals 1 when its condition is true and 0 otherwise. A participant assigned contrary to the policy contributes zero to this particular policy score; that zero is a weighting device, not a claim that the participant’s outcome was zero under the recommended treatment.
The first equality uses randomization and consistency: when the assignment matches the recommendation, the observed outcome equals the recommended-treatment potential outcome; when it does not match, the indicator is zero. Taking the expectation over the target population then gives \(V(d)\). This establishes unbiasedness under the stated conditions, conditional on the participants’ characteristics and potential outcomes, for the average of their policy-recommended potential outcomes. With participants sampled from a broader population, the estimator also estimates that population’s mean value.
Worked Example: Estimating the Value of Treating Everyone
Suppose a hypothetical randomized trial has four participants and assigns treatment with known probability \(\pi_1(X_i)=0.25\) for each participant. Consider the fixed policy \(d(X_i)=1\) for everyone. The observed data are:
| Participant | Assignment \(A_i\) | Observed outcome \(Y_i\), points |
|---|---|---|
| 1 | 1 | 8 |
| 2 | 0 | 3 |
| 3 | 1 | 4 |
| 4 | 0 | 5 |
For participants 1 and 3, assignment matches the policy; their scores are \(8/0.25=32\) and \(4/0.25=16\) points. Participants 2 and 4 were assigned control, so each has score 0 for this policy. Therefore,
The estimate is 12 points, not the mean outcome among the two participants who received treatment, which is \((8+4)/2=6\) points. The inverse-probability estimate gives more weight to the observed treated outcomes because treatment assignment had probability only 0.25. Its unbiasedness is a repeated-randomization property under the assumptions, not a guarantee that this small sample estimate equals the policy’s true value.
Estimating Uncertainty
Let \(Z_i(d)=\mathbb{1}\{A_i=d(X_i)\}Y_i/\pi_{d(X_i)}(X_i)\) be participant \(i\)’s score. Under independent sampling and assignment, the scores are independent and identically distributed draws from the target population’s score distribution. Estimate their variance using the usual sample variance,
The standard error measures estimated sampling variability of the policy-value estimate, in the same units as the outcome. For a sufficiently large sample and a score distribution for which a normal approximation is reasonable, an approximate 95% confidence interval is \(\widehat V(d)\pm1.96\widehat{SE}\{\widehat V(d)\}\). It is an interval procedure with approximate 95% coverage under the model and sampling conditions; it does not mean there is a 95% probability that this fixed interval contains the value.
Worked Example: Standard Error for the Treat-All Policy
Use the policy scores from the four-participant example: \(32,0,16,0\) points, with estimated value 12 points. The deviations from 12 are \(20,-12,4,-12\) points. Their squared deviations sum to
The sample variance and standard error are therefore
A mechanical normal-approximation interval would be \(12\pm1.96(7.659)\), or approximately \([-3.012,27.012]\) points. With only four participants, the normal approximation is not dependable; this interval is included to show the calculation, not as a reliable inferential conclusion. Its lower endpoint also illustrates that a normal approximation can extend beyond a plausible range for an outcome.
Comparing Two Fixed Policies
When comparing two fixed policies \(d\) and \(g\), estimate each value with the same randomized trial data. For each participant, calculate both policy scores and subtract them. The estimated difference is the mean of these participant-level differences:
The standard error should be based on the sample variance of \(D_i\), divided by \(n\) and square-rooted. This paired calculation accounts for the fact that both policy scores are computed for the same participants. Calculating separate standard errors and ignoring the covariance between policy estimates can give an incorrect standard error for their difference.
Worked Example: Comparing Two Policies With Correct Match Scores
Consider six hypothetical participants in a trial with known assignment probabilities \(\pi_1(X_i)=\pi_0(X_i)=0.5\). Policies \(d\) and \(g\) may recommend different treatments for some participants. The table shows the observed assignment and outcome, each policy’s recommendation, and the resulting weighted score. A matching outcome is divided by 0.5; a nonmatch receives a score of zero.
| Participant | \(A_i\) | \(Y_i\), points | \(d(X_i)\) | Score \(Z_i(d)\) | \(g(X_i)\) | Score \(Z_i(g)\) |
|---|---|---|---|---|---|---|
| 1 | 1 | 10 | 1 | 20 | 0 | 0 |
| 2 | 0 | 8 | 0 | 16 | 1 | 0 |
| 3 | 1 | 6 | 1 | 12 | 1 | 12 |
| 4 | 0 | 7 | 0 | 14 | 0 | 14 |
| 5 | 0 | 5 | 1 | 0 | 0 | 10 |
| 6 | 1 | 9 | 0 | 0 | 1 | 18 |
For policy \(d\), participants 1 through 4 match its recommendations, giving scores \(10/0.5=20\), \(8/0.5=16\), \(6/0.5=12\), and \(7/0.5=14\). Participants 5 and 6 do not match \(d\), so their scores for \(d\) are zero. Thus
For policy \(g\), participants 3, 4, 5, and 6 match its recommendations. In particular, participant 5 contributes \(5/0.5=10\) points and participant 6 contributes \(9/0.5=18\) points. Therefore,
The estimated difference is \(10.333-9.000=1.333\) points in favor of \(d\). The participant-level differences are \(20,16,0,0,-10,-18\) points, whose mean is \(8/6=1.333\) points. Their squared deviations from the mean sum to
The sample variance of the differences is \(1069.333/5=213.867\) points squared, so the standard error of the estimated difference is
A normal-approximation 95% interval is \(1.333\pm1.96(5.970)\), or approximately \([-10.368,13.035]\) points. This small trial gives a very imprecise estimate: the interval includes both negative and positive differences. It does not establish that the policies have equal value, nor does the positive point estimate by itself establish that \(d\) is better.
Common Mistakes and Practical Interpretation
- Leaving out a matching participant. A participant contributes to a policy score whenever the assignment matches that policy, even if another policy recommends a different treatment. In the comparison above, participants 5 and 6 contribute to \(g\), with scores 10 and 18.
- Treating a nonmatch score of zero as an observed outcome under the recommended treatment. Zero marks that the participant supplies no direct outcome observation for that policy in this trial. The weighting identity, not a zero potential outcome, justifies the estimator.
- Using the wrong assignment probability. Divide by the probability of receiving the treatment recommended for that participant. If the policy recommends control, the denominator is \(\pi_0(X_i)\), not \(\pi_1(X_i)\).
- Ignoring pairing when comparing policies. Form \(D_i=Z_i(d)-Z_i(g)\) participant by participant and estimate the variance of those differences. This preserves the covariance induced by using the same participants.
- Interpreting a wide interval as evidence of no difference. A wide interval signals substantial uncertainty. It may contain differences that would matter in either direction.
The weighting can also make estimates unstable when a policy recommends an arm assigned with very low probability. Positivity rules out probabilities of zero, but probabilities close to zero can still produce large scores and large standard errors. The trial’s randomization design therefore affects not only whether a policy can be evaluated, but also how precisely its value can be estimated.
Check Your Understanding
Answer these questions using the definitions and calculations above.
- A policy recommends treatment for a participant, whose known treatment-assignment probability is 0.4. The participant is assigned treatment and has outcome 12 points. What is this participant’s policy score?
- Why does a participant who does not receive the policy-recommended treatment have a score of zero without implying that their recommended-treatment potential outcome is zero?
- State the positivity condition needed to evaluate a fixed policy from a randomized trial.
- When comparing two policies evaluated on the same participants, why should the standard error use participant-level score differences?
- What does a 95% normal-approximation confidence interval for a policy-value difference describe, and what does it not describe?