Why the Assignment Rule Matters
An institution may describe an instrument as “random,” “rotating,” or “assigned by policy.” Those labels do not establish independence. The relevant question is how the assignment rule actually operated for the people included in the analysis. A lottery can be compromised by departures from its procedure; a rotation can be predictable; and a geographic or administrative rule can sort people with different underlying risks into different instrument groups.
For instrumental-variable analysis, independence concerns the instrument’s relationship with potential outcomes and potential treatment receipt, not just its relationship with a handful of recorded baseline variables. Institutional records, interviews with staff, and the timing of decisions can help evaluate the assignment mechanism. Comparisons of observed characteristics are useful diagnostics, but cannot reveal every relevant difference.
The condition is about the assignment process in the target analysis population. In an observational setting, investigators may need to justify conditional rather than unconditional independence: within each relevant stratum of \(X\), the instrument must not systematically sort units according to their potential treatment or outcome values. This tutorial focuses on how institutional rules inform that assessment. The rule itself is evidence about the mechanism; it is not a substitute for stating and evaluating the assumption.
Trace the Rule, Not Just Its Name
A useful audit reconstructs the sequence from eligibility to the instrument actually analyzed. Ask who could enter the assignment process, who made or implemented the allocation, what information was available at that moment, whether anyone could influence the assignment, and which assigned units were ultimately retained in the analysis. The chronology matters: a characteristic measured before assignment may affect who receives which instrument value; a selection decision made after assignment can change the composition of the observed groups.
Specify what \(Z\) represents in the data: for example, an offer, a queue position, or assignment to a particular provider. Do not substitute treatment receipt \(D\) for the instrument.
Check whether a lottery, schedule, threshold, or staff decision determined \(Z\). Establish whether that process occurred before treatment and outcome information could influence assignment.
Ask whether people, staff, or administrators could steer units into instrument categories, and whether a factor could affect both assignment and potential outcomes or treatment receipt.
Determine whether eligibility, consent, follow-up, or record availability was decided after \(Z\) was assigned. Conditioning on such selection can create a different relationship between \(Z\) and underlying characteristics.
Compare pre-assignment characteristics across instrument groups, then interpret those comparisons alongside the institutional evidence. Balance can reveal a concern; it cannot certify independence.
A common cause is a variable that helps determine instrument assignment and is also related to potential treatment or outcome values. For example, a clinician’s workload might affect which clinician a patient sees and also affect care quality. Sorting occurs when a unit or decision-maker can influence which assignment category the unit enters, perhaps using information about prognosis. Neither concern is removed merely because the institution has a formal written rule: implementation and exceptions need to be checked.
The direction of time helps organize the audit. Baseline characteristics measured before the assignment event can be examined as possible sorting factors or common causes. Variables recorded after assignment should not automatically be treated as baseline adjustment variables: they may reflect the assignment or decisions made in response to it. In particular, restricting analysis to people with observed follow-up needs scrutiny if follow-up depends on both assignment and underlying prognosis.
Worked Examples
Worked Example: A Sealed-Envelope Encouragement Lottery
Hypothetical data. A clinic has 400 eligible patients. Within a single enrollment period, staff use a shuffled set of sealed envelopes to assign 200 patients to receive an appointment reminder (\(Z=1\)) and 200 to usual scheduling (\(Z=0\)). Before opening an envelope, staff record age and a binary indicator of high baseline severity. The mean age is 52.1 years in the reminder group and 51.4 years in the usual-scheduling group; the pooled standard deviation is 12.0 years. High severity is recorded for 60 of 200 reminder-group patients and 64 of 200 usual-scheduling patients.
Assignment audit. The stated mechanism is a shuffled lottery, with assignment after eligibility and baseline recording. To support the claim, the clinic would need to verify that envelopes were genuinely shuffled, opened in sequence, and not selected or replaced according to patient characteristics. It should also check that staff did not alter eligibility or omit assigned patients based on the assignment result. If these procedures were followed, the account supports independence within the enrolled group; it does not establish independence for people who were never eligible or enrolled.
Observed balance calculation. The difference in mean age, reminder minus usual scheduling, is \(52.1-51.4=0.7\) years. A standardized difference using the stated pooled standard deviation is \(0.7/12.0=0.0583\). The high-severity proportions are \(60/200=0.300\) and \(64/200=0.320\), a difference of \(0.300-0.320=-0.020\), or 2 percentage points. These measured differences are small in this hypothetical sample. They are consistent with the lottery account, but they do not establish that all potential treatment and outcome values are independent of \(Z\); unrecorded factors and failures in implementation remain possible.
Worked Example: A Provider Rotation With Triage Exceptions
Hypothetical data. An emergency department uses a rotation among two clinicians as an instrument for the initial clinician assigned. In a sample of 100 patients assigned to Clinician A, 45 had high triage severity. Among 100 assigned to Clinician B, 20 had high triage severity. The written rule says to assign patients in rotation, but triage staff may bypass the sequence when a patient is considered especially urgent.
Observed difference. The high-severity proportions are \(45/100=0.45\) for A and \(20/100=0.20\) for B. The difference, A minus B, is \(0.45-0.20=0.25\), or 25 percentage points. This imbalance is a warning that the actual assignment may depend on severity, or on a factor associated with severity. It does not by itself prove the direction or full magnitude of dependence in potential outcomes.
Institutional assessment. The written rotation is not enough: the exception gives staff discretion at the point of assignment. Investigators should examine timestamps, triage records, the sequence of assignments, and documented reasons for bypasses. If urgent patients were preferentially sent to A, and urgency predicts potential outcomes, then the instrument groups may differ in outcome potentialities. If the assignment log shows that the imbalance instead arose by chance despite strict rotation, the evidence would be different. The observed table motivates investigation; it cannot decide between these explanations on its own.
Worked Example: Selection Into Follow-Up After Assignment
Hypothetical data. A study randomly assigns 1,000 eligible people to receive an invitation (\(Z=1\)) or no invitation (\(Z=0\)), with 500 assigned to each group. At baseline, each group has 200 people at high underlying health risk and 300 at lower risk. Follow-up is obtained for 300 people in the invitation group: 60 high-risk and 240 lower-risk. In the no-invitation group, follow-up is obtained for 450 people: 180 high-risk and 270 lower-risk.
Check the original assignment groups. Before follow-up selection, the high-risk proportion is \(200/500=0.40\) in each group. The stipulated random assignment makes the groups comparable in this characteristic before follow-up is imposed.
Check the analyzed sample. Among people with follow-up, the high-risk proportion is \(60/300=0.20\) for \(Z=1\) and \(180/450=0.40\) for \(Z=0\). The difference is \(0.20-0.40=-0.20\), or 20 percentage points. In the hypothetical scenario, follow-up rates are \(60/200=0.30\) for high-risk and \(240/300=0.80\) for lower-risk people assigned the invitation; in the no-invitation group they are \(180/200=0.90\) and \(270/300=0.90\), respectively.
The initial assignment can be random while the groups retained for analysis are no longer comparable in this measured risk factor. Here follow-up depends on assignment and risk, so restricting analysis to observed follow-up changes group composition. The assignment rule alone does not justify independence in a selected sample; investigators must account for how the analysis population was formed and what assumptions support that restriction.
Why a Lottery Supports Independence
A simple argument explains why a correctly implemented lottery is valuable. Let \(V\) denote the full vector of potential treatment and outcome values, and suppose assignment is made by a lottery that gives each unit the same probability of each instrument value, regardless of \(V\). For a binary instrument with probability \(q\) of \(Z=1\), this means \(P(Z=1\mid V)=q\) for every value of \(V\).
By the definition of conditional probability, the joint probability of instrument assignment and potential values factors as follows:
The same factorization holds for \(Z=0\), with probability \(1-q\). Thus \(Z\) and \(V\) are independent. If the lottery is conducted separately within pre-assignment strata \(X\), the same argument applies within each stratum when the assignment probability is constant across potential values there; it yields \(Z\perp V\mid X\). This justification depends on the lottery being implemented as specified and on the analyzed group being formed in a way that preserves the assignment mechanism. If selection after assignment depends on \(Z\) and \(V\), the argument for the selected sample no longer follows.
What Balance Checks Can and Cannot Say
A balance table compares observed pre-assignment characteristics across instrument groups. Large differences can expose sorting, a flawed implementation, or a common cause that the analysis needs to address. Small differences are reassuring only in a limited sense: they show that the measured characteristics examined appear similar in the available data. They do not establish balance on unmeasured characteristics, prove that staff followed a rule, or rule out selection into the analyzed sample.
A significance test of baseline differences is not a test of the independence assumption as a whole. A non-significant result can arise from limited sample size, and even exact equality on recorded covariates would say nothing directly about unrecorded potential outcomes. Conversely, a difference in a baseline characteristic can occur by chance under valid random assignment. Interpret such comparisons as diagnostics and connect them to the assignment history, rather than treating a particular p-value or threshold as a validity verdict.
| Institutional evidence | What it can support | What it cannot establish alone |
|---|---|---|
| Documented lottery with preserved assignment records | A credible account of random assignment if implementation and analysis inclusion are verified | Independence after selective loss to follow-up |
| Similar measured baseline characteristics | No obvious imbalance on the measured characteristics examined | Independence from unmeasured factors or potential values |
| Rotation with discretionary exceptions | A predictable schedule may inform the assignment process | Independence if exceptions depend on prognosis or urgency |
| Geographic or administrative assignment rule | A basis for investigating which factors determine assignment | Independence when location or administration also tracks underlying risk |
Common Mistake: Treating Balance as Proof
Wrong reasoning: “The instrument groups have similar average age and severity, so the instrument is independent.” This moves from similarity on a few observed variables to a claim about the full vector of potential treatment and outcome values. That conclusion does not follow.
Correction: Describe the balance results narrowly, then assess the assignment process and the formation of the analysis sample. Ask whether the rule was implemented as stated, whether staff or participants could sort into assignment categories, what common causes could remain, and whether post-assignment selection altered the groups. Independence is a substantive assumption supported by institutional design and evidence, not a property established by one table.
Check Your Understanding
Use the assignment process, not just the instrument’s label, to answer each question.
- A hospital says patients are assigned to clinicians by rotation, but nurses may override the order for patients who appear unstable. What records would help assess whether this creates sorting?
- Why does a small standardized difference in measured age not establish independence from potential outcomes?
- In a lottery, eligibility is determined before assignment, but follow-up depends on assignment and baseline risk. Why should the analyst examine independence again in the followed-up sample?
- What additional condition is needed to apply the lottery factorization argument within covariate strata \(X\)?
- Can an observed baseline imbalance prove that an instrument is dependent on potential outcomes? Explain why or why not.