Conditional Randomization Under Hidden Bias

Why Condition on the Matched-Set Treated Count?

Matching creates sets of units with similar observed covariates. Within a set, treatment may still depend on an unmeasured factor. Rosenbaum’s sensitivity model represents that remaining dependence by bounding how different two units’ treatment odds can be. To use the model for randomization inference, we need the treatment-assignment probabilities within each set, conditional on the number treated there.

Conditioning matters because many matching designs fix the number treated in each set, or analysis deliberately conditions on that count. Once the count is fixed, the relevant random outcomes are which units in the set are treated, not how many are treated. This conditional distribution is the basis for asking how much a test could change under an allowed amount of hidden bias.

Definition: In matched set $i$, let $n_i$ be the number of units and $Z_{ij}=1$ indicate that unit $j$ is treated. Let $u_{ij}\in[0,1]$ be an unobserved covariate and let $\Gamma\geq1$ bound hidden bias. Rosenbaum’s model specifies treatment odds as $$ \frac{P(Z_{ij}=1)}{P(Z_{ij}=0)}=\exp(\kappa_i)\Gamma^{u_{ij}}, $$ where $\kappa_i$ is a set-specific intercept. Thus, within a set, the ratio of any two units’ treatment odds is at most $\Gamma$. The conditional assignment distribution is the distribution of the treatment indicators given the treated count in that set.

The scale $u_{ij}\in[0,1]$ is a sensitivity-model device; it is not a measured covariate. At $\Gamma=1$, all units in a set have equal treatment odds, so conditioning on the treated count gives equal probability to every assignment with that count. Larger $\Gamma$ permits greater differences in odds. The model bounds hidden bias; it does not identify which unit has which value of $u_{ij}$.

Deriving the Conditional Assignment Distribution

Suppose first that, conditional on the hidden covariates, treatment indicators in a set follow independent Bernoulli assignment with the specified odds. Write $m_i=\sum_{j=1}^{n_i}Z_{ij}$ for the treated count. For a particular assignment vector $z_i$, its probability before conditioning is proportional to the product of the odds for treated units. Terms involving the probabilities of not being treated can be collected into a factor that does not depend on which units are treated once $m_i$ is fixed.

Equivalently, the Bernoulli probability of $z_i$ is proportional to $\prod_j \{\exp(\kappa_i)\Gamma^{u_{ij}}\}^{z_{ij}}$, with a common factor from the untreated terms. Conditioning on $\sum_j z_{ij}=m_i$ makes the intercept contribution $\exp(\kappa_i m_i)$ common to all eligible assignments, so it cancels in the normalization. Let $A_i$ be any subset of $m_i$ units assigned to treatment. The resulting probability is:

P(A_i\mid m_i,u_i) = \frac{\Gamma^{\sum_{j\in A_i}u_{ij}}} {\displaystyle\sum_{B:\,|B|=m_i}\Gamma^{\sum_{j\in B}u_{ij}}}.

Here the denominator sums over every subset $B$ of size $m_i$. If $m_i=1$, the assignments are the $n_i$ possible choices of a single treated unit, and the formula simplifies to:

P(Z_{ij}=1\mid m_i=1,u_i) = \frac{\Gamma^{u_{ij}}}{\sum_{k=1}^{n_i}\Gamma^{u_{ik}}}.

For two units $j$ and $k$, the ratio of their conditional probabilities of being the sole treated unit is $\Gamma^{u_{ij}-u_{ik}}$. Since $u_{ij}-u_{ik}\in[-1,1]$, that ratio lies between $1/\Gamma$ and $\Gamma$. For a general fixed treated count, compare two assignments that differ only by swapping one treated unit for one untreated unit: their probability ratio is also $\Gamma^{u_{ij}-u_{ik}}$. This shows directly how the odds-ratio bound controls relative assignment weights after conditioning.

Worked Example: One Treated Unit in a Three-Unit Set

Consider a hypothetical matched set of three units with exactly one treated unit. Let $\Gamma=4$, and suppose their hidden-bias scores are $u=(0,0.5,1)$. The relative assignment weights are $\Gamma^{u_j}$, giving $1$, $4^{0.5}=2$, and $4^1=4$. Their total is $1+2+4=7$.

Therefore the conditional probabilities that units 1, 2, and 3 are treated are, respectively, $1/7=0.1429$, $2/7=0.2857$, and $4/7=0.5714$. They sum to 1. The probability ratio for units 3 and 1 is $4/1=4$, the largest permitted ratio. If $\Gamma=1$, each unit would instead have probability $1/3$, regardless of its $u$ value.

Worked Example: Two Treated Units in a Three-Unit Set

Now suppose the same hypothetical set has exactly two treated units, with $\Gamma=4$ and $u=(0,0.5,1)$. The possible treated subsets are $\{1,2\}$, $\{1,3\}$, and $\{2,3\}$. Their weights are the products of the individual weights:

For $\{1,2\}$, the weight is $1\times2=2$. For $\{1,3\}$, it is $1\times4=4$. For $\{2,3\}$, it is $2\times4=8$. The normalizing sum is $2+4+8=14$, so the conditional probabilities are $2/14=0.1429$, $4/14=0.2857$, and $8/14=0.5714$, respectively.

The highest-weight assignment treats the two units with the largest hidden-bias scores. The probabilities sum to 1, as required. This example also shows why the multiple-treated formula must sum weights over subsets: simply assigning each unit an independent conditional probability would not enforce exactly two treated units.

From Assignment Probabilities to Worst-Case Inference

A randomization test evaluates a statistic over possible treatment assignments while holding the outcomes fixed under a sharp null hypothesis. The sharp null states that every unit’s outcome would be the same whether treated or untreated: $Y_{ij}(1)=Y_{ij}(0)$ for all units. Under that null, the observed outcomes can be used for every assignment in the reference distribution. The assignment probabilities, however, depend on the hidden scores $u_{ij}$, which are unknown.

For a specified statistic $T$, a specified tail direction, and a fixed $\Gamma$, each allowed configuration $u$ gives a conditional randomization distribution. Its tail probability is the probability, under that distribution, of a statistic at least as extreme as the observed statistic. The sensitivity analysis uses the largest such tail probability over all $u_{ij}\in[0,1]$. In this way, it asks whether the result remains unusual even under the allowed hidden-bias configuration most favorable to the null.

Definition: A worst-case randomization p-value at sensitivity level $\Gamma$ is the supremum of the randomization-test tail probability over all hidden-score configurations allowed by the model. For a finite number of assignments and the bounded score space $u_{ij}\in[0,1]$, the probabilities vary continuously with the scores, so the supremum is attained as a maximum.

The maximization must use one coherent score configuration to determine the probabilities of all assignments. It is not valid to choose a different hidden-score configuration separately for each assignment in the tail event. Across matched sets, the conditional assignment probabilities factor into a product if the assignment mechanism is independent across sets after conditioning on each set’s treated count. If that independence or factorization is not credible, the product distribution is not justified.

Worked Example: A Worst-Case Upper-Tail Probability

Consider a hypothetical matched set with three units, exactly one treated, and fixed outcomes under the sharp null of 0, 1, and 2 outcome units. Use the statistic $T$ equal to the outcome of the treated unit, and consider the upper-tail event $T\geq2$. Let $\Gamma=2$.

For any allowed scores, the probability of treating the unit with outcome 2 is $\Gamma^{u_3}/(\Gamma^{u_1}+\Gamma^{u_2}+\Gamma^{u_3})$. To maximize this probability, set $u_3=1$ and $u_1=u_2=0$, giving numerator $2$ and denominator $1+1+2=4$. This configuration is allowed, so the maximum probability is $2/4=0.5$. No other configuration can yield a larger value: the numerator cannot exceed 2, and each of the other two terms is at least 1. Thus the worst-case upper-tail probability for this statistic is 0.5. It would not be below a conventional 0.05 significance threshold.

This calculation demonstrates the role of the sensitivity model, not a general formula for test statistics. With multiple sets or a different statistic, the tail event may contain several assignments, and the configuration that maximizes its total probability may require an explicit optimization.

Assumptions, Diagnostics, and Alternatives

The conditional distribution is useful only when its assumptions match the design and analysis. Check that the matched-set memberships and treated counts are defined as intended, that the hidden-bias model’s odds-ratio bound is a plausible sensitivity scale, and that conditional assignment across sets can be treated as independent if the analysis combines sets by multiplying their probabilities. Randomization inference also requires the sharp null for the reference distribution and a statistic defined consistently for every eligible assignment.

Useful diagnostics include verifying that all assignment probabilities are nonnegative and sum to one, checking that $\Gamma=1$ gives equal probability to assignments with the same treated count, and checking that the largest relative assignment weight does not exceed the stated bound. For one-treated sets, inspect whether the probability ratio for any pair of units exceeds $\Gamma$. When numerical optimization is used, compare solutions from multiple starting values or a grid in small examples, and verify that reported worst-case probabilities come from a single feasible score configuration.

A common mistake is to treat $\Gamma$ as a probability multiplier, or to multiply the treated unit’s probability by $\Gamma$ without renormalizing. The model instead multiplies odds by $\Gamma^{u_{ij}}$; conditioning and normalization then produce assignment probabilities. Another mistake is to interpret the worst-case p-value as the probability that the null is true. It is a supremum of tail probabilities under the specified assignment models, conditional on the sharp null.

The conditional Rosenbaum model is an alternative to assuming equal assignment probabilities within matched sets, and it provides a structured sensitivity analysis rather than a single unqualified randomization distribution. It differs from modeling treatment directly using measured covariates: the hidden score is not estimated from the observed data, but varied over a bounded range. Its conclusions are therefore conditional on the chosen $\Gamma$ and the model’s structure. It cannot reveal the true hidden covariate, remove confounding, or establish that the bound is appropriate. The introductory material on Rosenbaum’s Gamma model gives the basic sensitivity interpretation; here the key additional step is deriving the distribution used for conditional inference.

Key takeaway: Given a fixed treated count, Rosenbaum’s model assigns each eligible treatment allocation a weight proportional to $\Gamma$ raised to the sum of its hidden scores. Worst-case inference evaluates the test over these distributions and takes the largest allowed tail probability.

Check Your Understanding

Answer each question, then compare with the brief solutions.

With $\Gamma=1$ and exactly two treated units in a four-unit set, what is the conditional probability of each eligible assignment?
For a one-treated set with $\Gamma=3$ and scores $u=(0,1)$, calculate the two treatment probabilities.
Why does the set-specific intercept $\kappa_i$ disappear after conditioning on the treated count?
What does a worst-case randomization p-value maximize, and over what quantities?
Why is it invalid to select a different hidden-score configuration for each assignment in a tail event?

Solutions. (1) There are $\binom{4}{2}=6$ eligible assignments; at $\Gamma=1$, each has probability $1/6$. (2) The weights are $1$ and $3$, so the probabilities are $1/(1+3)=0.25$ and $3/(1+3)=0.75$. (3) Every eligible assignment has exactly $m_i$ treated units, so its intercept factor is $\exp(\kappa_i m_i)$, common to all assignments and canceled by normalization. (4) It maximizes the test’s tail probability over the allowed hidden scores $u_{ij}\in[0,1]$, at the specified $\Gamma$. (5) A single unknown score configuration defines the assignment distribution for the whole test; changing it by assignment does not correspond to any one allowed model.

Conditional Randomization Under Hidden Bias

What You'll Learn