Why the Feedback Structure Matters
A treatment-allocation method can use only information that becomes available to it. In some decision problems, each unit produces an outcome for every possible action. In a clinical trial, a participant is generally assigned one treatment, so the trial observes that participant’s response under the assigned treatment, not the responses they would have had under the alternatives. These are different feedback structures, even if both settings involve the same treatments and outcome model.
The distinction concerns what is observed after an assignment; it does not, by itself, specify how assignments are chosen. In particular, a trial can adapt its assignment probabilities over time while still receiving bandit feedback. Conversely, a simulation can provide full-information feedback even if its actions were selected by a fixed rule.
The potential-outcomes notation describes outcomes under treatments, but it does not make all those outcomes observable. For an individual trial participant, the assigned treatment determines which potential outcome is observed. The remaining potential outcomes are counterfactual: they describe what would have happened to that same participant under treatments not received.
This is why “the trial has data on every treatment” does not mean “the trial has full-information feedback.” A randomized trial with many participants may observe outcomes in every treatment arm, but each participant still contributes an observed response for only their assigned arm. Outcomes from different participants do not fill in one person’s missing counterfactual outcomes.
What the Two Data Structures Contain
A full-information record for unit \(t\) contains the assigned treatment and an outcome for every treatment. A bandit-feedback record contains the assigned treatment and its observed outcome; unassigned potential outcomes are not recorded as observed values. The distinction applies unit by unit, not merely to whether a dataset has at least one outcome column for every treatment.
| Feature | Full-information feedback | Bandit feedback |
|---|---|---|
| Outcomes observed for a unit | One for every treatment | Only for the assigned treatment |
| Possible source | A simulator with known outcome vectors | A clinical trial in which each participant receives one treatment |
| Within-unit treatment comparison | Can compare recorded outcomes for the same unit | Cannot directly compare that unit’s unassigned outcomes |
| Arm-level information | Each unit can inform every treatment’s outcome model | Each unit directly informs only the assigned treatment’s outcome model |
The table’s comparison assumes that the relevant outcomes are actually recorded. A full-information simulator may supply the complete vector by construction. In a trial, by contrast, a participant’s unassigned outcomes cannot be recovered simply by adding more outcome columns to the dataset. Recorded covariates, predictions, or responses from other participants may be useful for modeling, but they are not that participant’s observed counterfactual outcomes.
Likelihood Contributions: What Can Be Updated?
Suppose that, conditional on treatment \(a\), the observed response has model density or probability mass function \(f_a(y;\theta_a)\), with parameter \(\theta_a\). For this simple likelihood representation, assume responses from different units are conditionally independent given their treatment-specific models, and that the assignment and outcome records are correctly coded. Under bandit feedback, a unit assigned treatment \(A_t\) contributes the likelihood factor \(f_{A_t}(Y_t(A_t);\theta_{A_t})\). No observed outcome factor for an unassigned treatment is available from that unit.
Under full-information feedback, a unit supplies an observed outcome under every treatment. If the treatment-specific outcomes are independent conditional on their parameters, the likelihood can be written as
That product across treatments requires an independence assumption for the outcome components, or an appropriate factorization of their joint model. If the outcomes for different treatments on the same unit are dependent, a joint distribution for the outcome vector is needed instead. Full feedback means the components are observed; it does not assert that they are independent.
The basic justification for the bandit likelihood is that an unobserved outcome cannot supply a response value to the likelihood. If an outcome under treatment \(b\) is unobserved, then summing its model probability over all possible values gives one:
For a continuous outcome, the corresponding integral is \(\int f_b(y;\theta_b)\,dy=1\). Thus, when the missing component is integrated out under its model, it contributes no response-specific evidence about \(\theta_b\). This does not mean the treatment model can never be informed indirectly through covariates or a joint model. It means the unobserved outcome itself has not been measured and cannot be treated as a recorded response.
Worked Example: Reading Outcome Vectors in a Simulator
Consider a hypothetical simulator with two treatments, A and B, and a binary outcome, where 1 denotes response and 0 denotes no response. For four simulated units, the simulator generates the following potential outcomes, and the allocation rule assigns the treatment shown.
| Unit | \(Y_t(A)\) | \(Y_t(B)\) | Assigned treatment | Observed assigned outcome |
|---|---|---|---|---|
| 1 | 1 | 0 | A | 1 |
| 2 | 0 | 1 | A | 0 |
| 3 | 1 | 1 | B | 1 |
| 4 | 0 | 1 | A | 0 |
With full-information feedback, the simulator reveals all four outcomes under A and all four under B. The response proportions are
With bandit feedback, the observed records are only the assigned outcomes: units 1, 2, and 4 under A, and unit 3 under B. The observed response proportions within those assigned groups are
These calculations use different information. The full-information proportions use each unit’s generated outcome under both treatments; the bandit proportions use only assigned responses. In this tiny hypothetical dataset the estimates differ, but that difference alone does not establish that either treatment model is wrong. The point is that the two records provide different numbers of observed outcomes per treatment.
Worked Example: Bernoulli Updating With Different Feedback
Suppose a hypothetical two-treatment model assigns each treatment an independent Bernoulli response probability, with a \(\operatorname{Beta}(1,1)\) prior for each probability. Five simulated units have potential outcome vectors \((Y_t(A),Y_t(B))\) equal to \((1,0),(0,1),(1,1),(0,1),(1,0)\). The assignments are A, B, A, B, A, respectively.
Under bandit feedback, the observed responses are 1, 1, 1 for A and 1, 1 for B. For a \(\operatorname{Beta}(1,1)\) prior, \(s\) observed successes and \(f\) observed failures give posterior \(\operatorname{Beta}(1+s,1+f)\). Therefore,
The posterior means are \(4/(4+1)=0.8000\) for A and \(3/(3+1)=0.7500\) for B. Those values reflect the observed assigned responses, all of which happen to be successes in this small sample.
If the simulator instead supplies all five outcome vectors, A has three successes and two failures, and B also has three successes and two failures. The corresponding posteriors are
Each posterior mean is \(4/(4+3)=0.5714\). Full feedback supplies information about both treatments for every simulated unit; bandit feedback supplies information only for each unit’s assigned treatment. These posterior calculations assume the stated independent Bernoulli models and priors.
Clinical Trials and Structural Nonobservation
In a clinical trial, a participant’s unassigned treatment outcomes are structurally unobserved: the protocol assigns one treatment, and the participant’s response under the other treatments is not part of the observed record. This differs from an outcome that should have been measured but was not recorded, such as a missed follow-up assessment after the assigned treatment. Both situations leave an outcome value unavailable, but only the latter is ordinary missing outcome data for the treatment actually received.
A randomized trial can estimate treatment-specific response distributions using responses observed from participants assigned to each arm, subject to the trial’s design and analysis assumptions. That does not turn the trial into a full-information setting. An arm-level mean or response rate combines observations across participants; it is not a complete vector of potential outcomes for any one participant.
Worked Example: Auditing a Hypothetical Trial Record
A hypothetical trial assigns six participants among treatments A and B. Four are assigned A, of whom three respond; two are assigned B, of whom one responds. Each participant has a response recorded for their assigned treatment, and no response is recorded under the unassigned treatment.
The observed response proportions are \(3/4=0.7500\) in A and \(1/2=0.5000\) in B. These use the observed denominators, four and two, respectively. The two missing unassigned outcomes per participant are not additional failures, and they cannot be entered as zeros: no response under those treatments was observed.
A sound data audit would verify that the assignment and the recorded response match for each participant, that these denominators count participants actually assigned to each treatment, and that the absence of unassigned outcomes is by design. If an assigned participant’s response were missing, that would require separate handling; it would not be repaired by pretending the unassigned potential outcome had been observed.
Diagnostics, Comparisons, and Limitations
Before fitting or updating an outcome model, check the data against the feedback mechanism that generated it. In a bandit-feedback trial, each participant should generally have one assigned treatment and at most one observed assigned response. In a full-information simulation, the record should contain a response under every modeled action for each unit. Unexpectedly complete outcome vectors in a clinical dataset deserve scrutiny: they may be predictions or imputations rather than observed outcomes.
- Check assignment-response alignment. Confirm that each observed response is linked to the treatment actually assigned to that participant.
- Check the source of every outcome value. Distinguish a measured response from an imputed value, a model prediction, or a simulated potential outcome.
- Check denominators by treatment. Under bandit feedback, each treatment’s directly observed response count is the number of participants with an observed response after assignment to that treatment.
- Separate structural nonobservation from missed follow-up. The unassigned potential outcome is not a failed measurement of an assigned treatment’s response.
The main alternative to bandit feedback is full-information feedback, as in a controlled simulator that generates every treatment outcome for every unit. Full information permits comparisons among a unit’s recorded outcomes, but it requires those outcomes to be available. Bandit feedback is the usual structure for treatment outcomes in an individual-level clinical trial and limits direct response information to the assigned treatment. Neither label alone says whether the assignment procedure is randomized, adaptive, or fixed.
Full-information data also do not automatically justify every comparison. If the generated outcomes for different treatments on a unit are related, uncertainty for a within-unit contrast depends on that relationship. And in a real trial, model-based predictions for unassigned outcomes do not become observed feedback merely because a model produces numerical values. The feedback distinction identifies what was recorded; it does not settle every modeling or causal question.
Common Mistakes
- Wrong: “The trial contains outcomes in both arms, so every participant provides full information.” Correction: Outcomes from different participants do not reveal the unassigned potential outcomes for any one participant.
- Wrong: “An unassigned outcome is missing, so I can code it as zero.” Correction: Zero is a measured outcome value; an unassigned potential outcome is unobserved and must not be recoded as a response or nonresponse.
- Wrong: “If a model predicts an unassigned outcome, the trial now has full feedback.” Correction: A prediction is model output, not an observed outcome.
- Wrong: “Full-information feedback means treatment outcomes are independent.” Correction: It means the outcomes are observed. Their dependence requires separate modeling assumptions.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A three-treatment simulator records all three potential outcomes for each of 20 units, regardless of assignment. What feedback structure does it provide?
- A trial has observed responses from participants assigned to all four treatment arms. Does each participant thereby provide full-information feedback? Explain.
- In a Bernoulli model, a participant assigned to A responds. Which treatment’s response model receives a directly observed response contribution from that participant under bandit feedback?
- A database contains a predicted outcome under an unassigned treatment. Is that outcome full-information feedback? Why or why not?
- For a treatment assigned to five participants, four responses are recorded and one assigned participant has no recorded response. How should that absent value be distinguished from the outcomes under treatments those five participants did not receive?
Solutions. (1) Full-information feedback: all three treatment outcomes are observed for each unit. (2) No. Each participant contributes an observed response only under their own assigned treatment; observing responses across all four arms is not the same as observing all four outcomes for each participant. (3) Treatment A’s model receives the direct response contribution, because A was assigned and its response was observed. (4) No. A prediction is not a measured outcome, so it does not change the feedback structure. (5) The absent response is an unrecorded outcome under the treatment actually assigned and is a missing-data issue; outcomes under the other treatments are structurally unobserved potential outcomes, not failed measurements of assigned responses.