Variable-Ratio Matched-Set Sensitivity Analysis

Why Variable-Ratio Sets Need Their Own Calculation

Matched sets do not all have to contain the same number of controls, and some designs may also have more than one treated unit in a set. A sensitivity calculation must respect both features: the actual set size and the number treated in that set. Applying a matched-pair formula to a set with three or four units changes its assignment distribution and can give an incorrect tail probability.

The task here is set-specific. Under a sharp null hypothesis of no treatment effect, outcomes are held fixed while treatment labels are reassigned within each matched set, conditional on that set’s observed treatment count. A Rosenbaum sensitivity model allows those assignments to be unequal in probability, subject to a bound indexed by $\Gamma$. This tutorial derives those probabilities and shows how they produce the distribution of a test-statistic contribution from one set. Calculating how evidence from many sets is combined is a separate step.

Assignment Probabilities Conditional on the Set’s Treatment Count

Index matched sets by $j$, and units within set $j$ by $i=1,\ldots,n_j$. Let $k_j$ be the number of treated units in the set, and let $Z_{ij}=1$ if unit $i$ is treated and $Z_{ij}=0$ otherwise. The assignment is conditioned on $\sum_i Z_{ij}=k_j$; it is not a draw in which the number treated is allowed to vary.

Definition: In the Rosenbaum sensitivity model, each unit has a positive assignment weight $\lambda_{ij}$. The ratio of the largest to the smallest weight within a matched set is at most $\Gamma$, where $\Gamma\geq 1$. Conditional on exactly $k_j$ treated units, the probability of a particular treated subset $A$ is $$ P(A\mid \textstyle\sum_i Z_{ij}=k_j) = \frac{\prod_{i\in A}\lambda_{ij}} {\displaystyle\sum_{B:\,|B|=k_j}\prod_{i\in B}\lambda_{ij}}, \qquad |A|=k_j. $$

Only relative weights matter: multiplying every $\lambda_{ij}$ in a set by the same positive constant leaves these probabilities unchanged. Thus, for calculations, the weights can be rescaled so that $1\leq\lambda_{ij}\leq\Gamma$. One common representation is $\lambda_{ij}=\Gamma^{u_{ij}}$ with $0\leq u_{ij}\leq1$. At $\Gamma=1$, all weights are equal, so each of the $\binom{n_j}{k_j}$ possible treated subsets has probability $1/\binom{n_j}{k_j}$.

The model bounds relative assignment odds, not treatment probabilities without conditioning. Its interpretation is that, within a matched set, hidden differences in treatment odds may make one unit up to $\Gamma$ times as likely as another to be treated. It does not assert that the analyst knows the actual weights or identify a particular unmeasured confounder.

From Assignment Probabilities to a Set Contribution

Under a sharp null, each unit’s outcome is fixed under either treatment assignment. Let $q_{ij}$ be a fixed score for unit $i$ in set $j$. The set’s contribution to a linear test statistic is $T_j=\sum_i Z_{ij}q_{ij}$: for a given assignment, add the scores of the treated units. For a test based directly on a binary outcome, one may use $q_{ij}=Y_{ij}$, where $Y_{ij}$ is the fixed binary outcome under the sharp null. Other tests use different fixed scores, such as ranks.

For each possible treated subset $A$, the contribution is $t_j(A)=\sum_{i\in A}q_{ij}$. Its sensitivity-model distribution is obtained by assigning that value the probability of $A$ from the preceding formula. For example, the upper-tail probability at an observed contribution $t_{j,\mathrm{obs}}$, calculated for this set alone, is $P(T_j\geq t_{j,\mathrm{obs}})$, summed over all subsets whose score total is at least the observed one. Ties are included in this weak upper tail.

This calculation is conditional on the sharp null and the set’s fixed treatment count. A tail probability for one set is not, by itself, a combined test of a study with multiple sets.

Worked Example: One Treated Unit and Two Controls

Consider a hypothetical set with $n_j=3$, $k_j=1$, and fixed binary scores $(q_{1j},q_{2j},q_{3j})=(1,0,0)$. Suppose the observed treated unit is unit 1, and consider $\Gamma=2$ with weights $(2,1,1)$. The weights satisfy the sensitivity bound because their maximum-to-minimum ratio is 2.

There are three possible treated subsets, each containing one unit. Their unnormalized weights are $2$, $1$, and $1$, so the normalizing denominator is $2+1+1=4$. The assignment probabilities are therefore $P(\{1\})=2/4=1/2$, $P(\{2\})=1/4$, and $P(\{3\})=1/4$.

The corresponding statistic contributions are $1$, $0$, and $0$. Since the observed contribution is 1, the set-specific upper-tail probability is $P(T_j\geq1)=P(\{1\})=1/2$. Under $\Gamma=1$, the three assignments would instead each have probability $1/3$, giving an upper-tail probability of $1/3$. The increase reflects the specified unequal assignment weights, not a change in the scores.

Worked Example: Two Treated Units and Two Controls

For another hypothetical set, let $n_j=4$, $k_j=2$, with binary scores $(1,1,0,0)$. Suppose the observed treated units are 1 and 2, so $t_{j,\mathrm{obs}}=2$. Use $\Gamma=2$ and weights $(2,2,1,1)$. These weights meet the bound, and the six possible treated subsets must each contain exactly two units.

The unnormalized subset weights are $4$ for $\{1,2\}$; $2$ each for $\{1,3\}$, $\{1,4\}$, $\{2,3\}$, and $\{2,4\}$; and $1$ for $\{3,4\}$. The denominator is $4+2+2+2+2+1=13$. Thus the probabilities are respectively $4/13$, $2/13$, $2/13$, $2/13$, $2/13$, and $1/13$.

The score totals for those subsets are respectively $2,1,1,1,1,0$. Only the subset $\{1,2\}$ reaches the observed total of 2, so $P(T_j\geq2)=4/13\approx0.3077$. Under equal assignment probabilities ($\Gamma=1$), the corresponding tail probability is $1/6$, because only one of the six subsets has total 2. The calculation differs from the preceding example because both the set size and the treated count differ.

A Marginal Probability Bound and Its Derivation

The subset formula also gives a useful bound for the chance that a particular unit is treated. Suppose a set has $n$ units and $k$ treated units, with $0<k<n$. To maximize the probability that unit $i$ is treated, set its weight to $\Gamma$ and all other weights to 1. This configuration is allowed by the sensitivity model.

There are $\binom{n-1}{k-1}$ subsets containing $i$, each with weight $\Gamma$. There are $\binom{n-1}{k}$ subsets excluding $i$, each with weight 1. Dividing the total weight of subsets containing $i$ by the weight of all subsets gives the upper bound below. Reversing the weights—unit $i$ has weight 1 and all others have weight $\Gamma$—gives the lower bound.

\frac{k}{k+(n-k)\Gamma} \ \leq\ P(Z_i=1\mid \textstyle\sum_{\ell=1}^{n}Z_\ell=k)\ \leq\ \frac{k\Gamma}{k\Gamma+n-k}.

For the upper bound, for example, substitute the subset counts into the probability: $\Gamma\binom{n-1}{k-1}/\{\Gamma\binom{n-1}{k-1}+\binom{n-1}{k}\}$. Using $\binom{n-1}{k-1}/\binom{n-1}{k}=k/(n-k)$ yields $k\Gamma/(k\Gamma+n-k)$. The same calculation with the low-weight unit gives the lower bound. When $k=0$ or $k=n$, the treatment indicator is fixed at 0 or 1, respectively, so the displayed nondegenerate bound is not needed.

Worked Example: Checking a Marginal Bound

Suppose a hypothetical set has $n=5$ units, $k=2$ treated units, and $\Gamma=3$. For any particular unit, the marginal treatment probability must be between $2/\{2+3(3)\}=2/11\approx0.1818$ and $2(3)/\{2(3)+3\}=6/9\approx0.6667$.

For the upper extreme, give the unit weight 3 and the other four units weight 1. There are $\binom{4}{1}=4$ subsets containing it, with total weight $4(3)=12$, and $\binom{4}{2}=6$ subsets excluding it, with total weight 6. Its probability is $12/(12+6)=2/3$, matching the upper bound. The fixed-count condition remains essential: these marginal probabilities do not imply that the five units are treated independently.

Diagnostics, Alternatives, and Limitations

A reliable calculation begins by checking the design and the conditioning event. Record $n_j$ and $k_j$ for each set, verify that every enumerated subset has exactly $k_j$ units, and confirm that the largest-to-smallest weight ratio is no greater than the chosen $\Gamma$. The probabilities over all subsets must sum to 1. At $\Gamma=1$, they must all be equal; if $k_j=1$, there must be exactly $n_j$ possible assignments. These checks catch common coding and hand-calculation errors.

Scores also need to match the null hypothesis and test. Under a sharp null, outcomes or ranks used in the statistic are held fixed as treatment labels are reassigned. If the score itself would change under reassignment, treating it as fixed is not justified without defining a suitable sharp null and statistic. For a set with many units, direct enumeration involves $\binom{n_j}{k_j}$ subsets; when that number is large, a computational method must still preserve the fixed treatment count and correctly normalize the assignment probabilities.

The matched-pair calculation is the special case $n_j=2,k_j=1$, not a general substitute for variable-ratio enumeration. Stratified binary-outcome tests organize evidence by matched sets under their specified assignment model; this tutorial supplies the assignment and contribution calculation for a particular set rather than re-deriving those tests. Other approaches, such as regression adjustment, target different models and do not automatically provide the same conditional sensitivity analysis.

The model does not show that hidden bias exists, nor does a selected value of $\Gamma$ identify the strength of any real confounder. Its conclusions depend on the matched-set structure, the chosen statistic, the sharp null being tested, and the assumed bound on relative odds. A sensitivity analysis reports how inference behaves under these specified departures from equal assignment; it does not repair poor matching or establish causal validity by itself.

Common Mistakes

Allowing the treated count to vary. The conditional model assigns probability only to subsets of size $k_j$. Including subsets with a different number treated changes the experiment being analyzed.
Using the pair formula for a larger set. With $n_j$ units and $k_j$ treated, enumerate $\binom{n_j}{k_j}$ subsets and use their product weights.
Ignoring the normalizing denominator. Products of weights are unnormalized probabilities. Divide each by their sum over all eligible subsets.
Treating assignments as independent. Conditional on a fixed number treated, indicators within the set are dependent, even when the weights are equal.
Calling a single-set tail probability a study-wide result. It describes the contribution’s distribution for that set; evidence across sets requires an additional analysis.
Interpreting $\Gamma$ as an estimated confounder effect. It is a bound on relative treatment-assignment odds, not an effect estimate or a probability that confounding is present.

Key takeaway: For a variable-ratio matched set, condition on its actual treated count, assign each eligible treated subset probability proportional to the product of its unit weights, and calculate the statistic contribution for each subset. The set size and treatment count determine the enumeration; $\Gamma$ bounds how unequal the assignment weights may be.

Check Your Understanding

For each question, state the calculation and the reason for it.

A set has $n=4$, $k=1$, and $\Gamma=1$. How many assignments are possible, and what is the probability of each?
A set has $n=4$, $k=2$, and weights $(2,1,1,1)$. Find the probability that units 1 and 2 are the treated subset.
For a set with $n=6$, $k=2$, and $\Gamma=2$, give the lower and upper marginal treatment-probability bounds for one unit.
Why is it incorrect to treat the unit-level assignment indicators as independent after conditioning on exactly two treated units?

Solutions. (1) There are $\binom{4}{1}=4$ assignments, each with probability $1/4$, because $\Gamma=1$ makes all weights equal. (2) The subset $\{1,2\}$ has weight $2(1)=2$. The six pair weights sum to $2+2+2+1+1+1=9$, so its probability is $2/9$. (3) The bounds are $2/\{2+4(2)\}=1/5=0.2000$ and $2(2)/\{2(2)+4\}=1/2=0.5000$. (4) The fixed-count condition requires exactly two indicators to equal 1. If one unit’s treatment status is known, that information constrains the possible statuses of the other units; the indicators therefore cannot be independent under this conditioning.