Wilcoxon Sensitivity Bounds for Continuous Outcomes

Why Bound a Signed-Rank Test?

Matched-pair analyses of continuous outcomes can use both the direction and the relative magnitude of pair differences. The Wilcoxon signed-rank statistic does this by adding ranks for differences in a chosen direction. But matching on measured covariates does not rule out hidden differences within pairs. Rosenbaum’s Γ model asks how much an unmeasured factor could alter the odds that one member of a pair receives treatment, then bounds the test’s p-value over all assignment probabilities allowed by that model.

The result is a sensitivity bound, not a correction that identifies the effect of the hidden factor. A large worst-case p-value means the observed evidence could be less compelling under some allowed assignment mechanism; it does not show that such a mechanism occurred. The test below concerns the sharp null of no treatment effect for any individual, conditional on the matched pairs.

Statistic, Assignment Model, and Assumptions

Let \(m\) be the number of nonzero pair differences. Rosenbaum’s Γ model bounds the treatment assignment odds within each matched pair. Conditional on exactly one treated member in each pair, the probability that the member with a particular fixed outcome ordering is treated is between \(1/(1+\Gamma)\) and \(\Gamma/(1+\Gamma)\), where \(\Gamma\geq1\). The case \(\Gamma=1\) corresponds to equal assignment odds within every pair. Larger Γ permits greater hidden bias.

For the calculation, assume the pair assignments are conditionally independent and that the sharp null holds. Under the null, reversing which member of a pair is treated reverses that pair’s signed difference without changing its absolute value or rank. The model permits the probability of a positive sign to vary by pair, but constrains each such probability to the interval above. These assumptions make the ranks fixed and the uncertainty in \(W^+\) arise from the signs.

For a one-sided alternative that outcomes tend to be higher under treatment, the worst-case upper-tail p-value is the largest probability of observing a statistic at least as large as the observed \(w_{\mathrm{obs}}\), over all allowed pair assignment probabilities. The adverse choice for this upper tail sets the probability of a positive sign to its maximum, \(p_\Gamma=\Gamma/(1+\Gamma)\), for every nonzero pair. Thus the bounding distribution is

The worst-case upper-tail bound is \(P(W^+_\Gamma\geq w_{\mathrm{obs}})\). For a lower-tail alternative, use the same construction for negative signs: their probabilities are set to \(p_\Gamma\), equivalently positive signs have probability \(1/(1+\Gamma)\). Tail comparisons include equality, which matters when the statistic has a discrete distribution.

Why the Extremal Probability Gives a Bound

The justification is monotonicity. For the upper tail, write each sign indicator as \(B_i(p_i)=1\{U_i\leq p_i\}\), where the \(U_i\) are independent uniform random variables on \([0,1]\). If \(p_i\leq p_\Gamma\), then \(B_i(p_i)\leq B_i(p_\Gamma)\) for every value of \(U_i\). Since every rank is positive,

Consequently, the event that the left-hand statistic reaches an upper threshold is contained in the corresponding event for the right-hand statistic. Its probability is therefore no greater. Setting every probability to \(p_\Gamma\) maximizes the upper tail over the allowed assignments. For a lower-tail test, apply the same argument to the negative-sign indicators, because the event of an unusually small \(W^+\) is an event of many negative signs. This proves the stated worst-case bounds under the model assumptions.

Exact Calculation and a Sensitivity Comparison

With a modest number of nonzero pairs, enumerate the \(2^m\) possible sign patterns. For any pattern, add the ranks assigned positive signs and include its probability under the bounding distribution. For larger \(m\), a dynamic-programming calculation can update the probability of each attainable rank sum one pair at a time. Tied ranks can be fractional; multiplying all ranks by 2 converts half-integer ranks to integers without changing any tail probability.

Two-Sided Tests and Direction

For a two-sided test, one common convention doubles the smaller one-sided tail probability and caps the result at 1. A conservative sensitivity bound can be formed by bounding each tail separately. Let \(U_+\) be the worst-case probability \(P(W^+\geq w_{\mathrm{obs}})\), and let \(U_-\) be the worst-case probability \(P(W^+\leq w_{\mathrm{obs}})\). Then a valid bound using this convention is

The bound is conservative because the assignment probabilities that maximize the two tails need not be the same. It may therefore be larger than the largest two-sided p-value over any single allowed assignment mechanism. State the convention used, particularly when the discrete distribution puts substantial probability on the observed statistic.

Diagnostics, Alternatives, and Limitations

Before interpreting a bound, check that the analysis follows the design and the statistic’s assumptions. In particular, verify that each pair contains the intended one treated and one control unit, that outcomes and treatment orientation are recorded consistently, and that the sharp-null comparison is the scientific null of interest. Inspect the pair differences: a coding error, an unexpected cluster of ties, or many zero differences can materially affect the statistic. Confirm that ties use average ranks and that zeros are removed before ranks are assigned.

Exact enumeration is preferable when feasible. A normal approximation to the signed-rank sum can be inaccurate with few nonzero pairs, many tied ranks, or a tail probability near a decision threshold. Under the bounding distribution with ranks \(r_i\), its mean and variance are

These follow by adding the means and variances of independent Bernoulli-weighted ranks. They can support an approximation, but they do not remove discreteness or make a poor approximation exact. If used, report the approximation and any continuity correction; do not present it as an exact bound.

The signed-rank test uses the ordering of the magnitudes as well as their signs. A paired sign test uses only the number of positive and negative nonzero differences, so it may be less sensitive when magnitudes carry useful information, but it is less affected by the precise rank ordering. A paired t-test instead uses the numerical magnitudes and has different distributional requirements for its usual small-sample inference. None of these alternatives automatically addresses hidden bias: a sensitivity analysis requires specifying how assignment odds may differ within pairs.

A Γ-bound is conditional on the matched-pair design, the sharp null, and the assignment model; it does not account automatically for matching errors, outcome measurement problems, dependence across pairs, or uncertainty from selecting a matching procedure. It also does not identify the unmeasured covariate or tell whether the worst-case assignment probabilities are plausible. Report the chosen Γ values and the corresponding bounds, and interpret them as sensitivity of the test conclusion to a stipulated degree of hidden bias.

Common Pitfall