Calculating Alpha Recycling in a Testing Graph

Why the Arithmetic of Recycling Matters

A testing graph specifies both the initial significance level assigned to each hypothesis and where that level goes if the hypothesis is rejected. After a rejection, the remaining hypotheses may be tested at different levels from before. When the graph contains cycles, those updates require careful bookkeeping: part of a level can circulate back to a hypothesis before eventually reaching another one. The updated transition weight and the eventual fraction transferred are therefore not always the same number.

This tutorial focuses on calculating that updated allocation and using it to make rejection decisions. The graph and the testing family must be prespecified; the arithmetic does not authorize changing the family or the graph in response to observed results.

Definitions and the Update Rule

Definition: A testing graph has one vertex for each hypothesis $H_i$, an initial local significance level $\alpha_i$ for each vertex, and transition weights $g_{ij}$. The weight $g_{ij}$ is the fraction of level at $H_i$ directed to $H_j$ when $H_i$ is rejected. We use $g_{ii}$ for a self-loop. The weights are nonnegative and each row sums to 1. The initial levels satisfy $\sum_i\alpha_i\leq\alpha$, where $\alpha$ is the familywise significance level.

At each step, compare the p-value for a remaining hypothesis $H_i$ with its current level $\alpha_i$. Reject $H_i$ if $p_i\leq\alpha_i$; otherwise, do not reject it at that step. When $H_i$ is rejected, remove it and redistribute its level and update the transition weights among the hypotheses still under consideration. The local tests must be valid at their assigned levels, and the graph and update procedure must be specified in advance.

Update rule: If $H_i$ is rejected, then for each remaining hypothesis $H_j$, its new level is $$ \alpha'_j=\alpha_j+\alpha_i\frac{g_{ij}}{1-g_{ii}}. $$ For remaining vertices $H_j$ and $H_k$, the updated transition weight is $$ g'_{jk}=g_{jk}+\frac{g_{ji}g_{ik}}{1-g_{ii}}. $$ These formulas require $g_{ii}<1$. A self-loop is allowed, but all of the level must eventually leave the rejected vertex.

The level update and the weight update answer different questions. The first adds to each surviving hypothesis the portion it receives from the rejected hypothesis. The second describes how a surviving vertex’s future transitions change because paths through the removed vertex are no longer represented explicitly.

Why Self-Loops Require a Denominator

Suppose $H_i$ has a self-loop of weight $g_{ii}$, and it directs weight $g_{ij}$ to $H_j$. Starting with one unit of level at $H_i$, the first passage to $H_j$ receives $g_{ij}$. The amount that loops back is then sent through the same transitions again, yielding $g_{ii}g_{ij}$, then $g_{ii}^2g_{ij}$, and so on. For $g_{ii}<1$, the total fraction reaching $H_j$ is the convergent geometric series $$ g_{ij}+g_{ii}g_{ij}+g_{ii}^2g_{ij}+\cdots =\frac{g_{ij}}{1-g_{ii}}. $$

Multiplying this eventual fraction by the rejected hypothesis’s current level gives the level update. The same path argument explains the transition-weight update: a surviving source $H_j$ can reach $H_k$ either directly, with weight $g_{jk}$, or by going first to $H_i$, then from $H_i$ to $H_k$, including any repeated loops at $H_i$. The latter contribution is $g_{ji}g_{ik}/(1-g_{ii})$. Adding the two paths gives the stated formula.

This distinction prevents a common computational error: after an update, a transition weight such as $0.80$ need not mean that only 80% of the rejected hypothesis’s level is eventually transferred. If the updated row has a self-loop of $0.20$, its eventual exit fraction can be $0.80/(1-0.20)=1$.

Worked Calculations

Worked Example: A Rejection Creates a Self-Loop

Consider three hypothetical hypotheses with initial levels $\alpha_1=0.020$, $\alpha_2=0.020$, and $\alpha_3=0.010$, which sum to the familywise level $0.050$. The graph rows needed here are $H_1\to(H_2,H_3)=(0.50,0.50)$ and $H_3\to(H_1,H_2)=(0.40,0.60)$; both have no self-loop. Suppose the valid p-values are $p_1=0.015$, $p_2=0.041$, and $p_3=0.019$.

Since $0.015\leq0.020$, reject $H_1$. Its level is split equally between $H_2$ and $H_3$: $H_2$ receives $0.020(0.50)=0.010$, and $H_3$ receives $0.020(0.50)=0.010$. The new levels are therefore $\alpha_2=0.030$ and $\alpha_3=0.020$.

For the transition row from $H_3$, removing $H_1$ gives $$ g'_{32}=0.60+\frac{0.40(0.50)}{1-0}=0.80, \qquad g'_{33}=0+\frac{0.40(0.50)}{1-0}=0.20. $$ The updated row sums to 1. Since $p_3=0.019\leq0.020$, reject $H_3$. Its full current level eventually reaches $H_2$, because $$ 0.020\frac{0.80}{1-0.20}=0.020. $$ Thus $H_2$'s level becomes $0.030+0.020=0.050$. Since $p_2=0.041\leq0.050$, reject $H_2$. The updated weight to $H_2$ was $0.80$, not 1; its eventual exit fraction was 1 because the self-loop was accounted for separately.

Worked Example: Updating a Graph With a Nonzero Initial Self-Loop

In a second hypothetical graph, set $(\alpha_1,\alpha_2,\alpha_3)=(0.016,0.014,0.020)$. The initial levels again sum to $0.050$. Let the row from $H_1$ be $g_{11}=0.20$, $g_{12}=0.50$, and $g_{13}=0.30$, and let the row from $H_2$ be $g_{21}=0.40$, $g_{22}=0$, and $g_{23}=0.60$. Each row sums to 1. Suppose $p_1=0.010$, $p_2=0.021$, and $p_3=0.031$.

Because $0.010\leq0.016$, reject $H_1$. The fractions of its level that eventually reach $H_2$ and $H_3$ are $0.50/(1-0.20)=0.625$ and $0.30/(1-0.20)=0.375$. Hence the levels become $$ \alpha_2=0.014+0.016(0.625)=0.024, \qquad \alpha_3=0.020+0.016(0.375)=0.026. $$

Now update the row from $H_2$, using the same denominator $1-g_{11}=0.80$: $$ g'_{23}=0.60+\frac{0.40(0.30)}{0.80}=0.75, \qquad g'_{22}=0+\frac{0.40(0.50)}{0.80}=0.25. $$

This row also sums to 1. Since $0.021\leq0.024$, reject $H_2$. Its full level eventually reaches $H_3$, with transfer $0.024(0.75)/(1-0.25)=0.024$. Thus $H_3$'s level is $0.026+0.024=0.050$. Because $0.031\leq0.050$, reject $H_3$. The updated weight $0.75$ is not itself the full transfer fraction; the self-loop of $0.25$ makes the eventual fraction $0.75/(1-0.25)=1$.

Worked Example: Stopping When a Remaining Test Fails

For a third hypothetical graph, let the initial levels be $(0.025,0.015,0.010)$. Suppose $H_1$ directs weights $0.60$ to $H_2$ and $0.40$ to $H_3$; $H_2$ directs $0.50$ to $H_1$ and $0.50$ to $H_3$; and $H_3$ directs $0.25$ to $H_1$ and $0.75$ to $H_2$. There are no initial self-loops. The hypothetical p-values are $p_1=0.018$, $p_2=0.028$, and $p_3=0.052$.

Reject $H_1$, since $0.018\leq0.025$. The levels become $\alpha_2=0.015+0.025(0.60)=0.030$ and $\alpha_3=0.010+0.025(0.40)=0.020$. For $H_2$'s row, removing $H_1$ gives $g'_{23}=0.50+0.50(0.40)=0.70$ and $g'_{22}=0.50(0.60)=0.30$. Thus $p_2=0.028\leq0.030$, so reject $H_2$. Its level eventually reaches $H_3$: $0.030(0.70)/(1-0.30)=0.030$. The level for $H_3$ is now $0.020+0.030=0.050$.

Do not reject $H_3$, because $p_3=0.052>0.050$. The comparison is made with the updated level, not the initial level $0.010$; nevertheless the p-value still exceeds the applicable threshold. No rejection follows from this calculation.

Checking the Calculation and Its Scope

A reliable calculation keeps the levels and transition weights in separate records. After each rejection, update the levels of the surviving hypotheses, update the relevant graph rows, and test any remaining p-values against their current levels. A level is not a p-value, and a transition weight is not a test threshold.

Check the inputs.
Confirm the family, initial levels, transition matrix, and local p-values are fixed and that the initial levels sum to no more than $\alpha$.

Apply the rejection rule.
For each eligible hypothesis, compare its p-value with its current level. Reject only when $p_i\leq\alpha_i$.

Recycle and update.
For a rejected $H_i$, use the denominator $1-g_{ii}$ in both updates, then verify that each updated transition row sums to 1.

Continue or stop.
Test remaining hypotheses at their updated levels. If none meets its threshold, make no further rejection.

A useful diagnostic is to check that every updated row is nonnegative and sums to 1, allowing for rounding. If $g_{ii}=1$, no level exits that vertex and the displayed update is undefined; the graph cannot recycle level from that vertex under this rule. Calculations should retain adequate precision before rounding the reported levels and weights.

Graphical recycling is not the same as testing every hypothesis at nominal $\alpha$, nor is it the same as a fixed sequence. A fixed-sequence procedure follows a prespecified order and does not generally redirect levels according to a transition matrix. Holm and other stepwise procedures use their own ordered thresholds rather than a graph’s evolving levels. Graphical procedures are useful when prespecified logical routes for reallocating significance are scientifically meaningful, but their results depend on that design choice. A favorable graph cannot compensate for invalid local tests or a family chosen after inspecting the results.

Common Mistakes

Calling an updated weight the full transfer. A row with weight $0.80$ to another hypothesis and self-loop $0.20$ eventually sends $0.80/(1-0.20)=1$ of its outgoing level there. Keep the updated weight and eventual transfer fraction distinct.
Dropping the denominator. When the removed vertex has a self-loop, use $1-g_{ii}$ in both update formulas. Omitting it understates the eventual transfer and can produce rows that do not sum to 1.
Updating weights but not levels. A new transition matrix describes future routes; the level update separately determines the current test thresholds.
Continuing after a failed test as if it were rejected. Only a rejection triggers removal and recycling. A non-rejected hypothesis remains, and its level is not redistributed by that step.

Key takeaway: After rejecting $H_i$, calculate the level transfer as $\alpha_i g_{ij}/(1-g_{ii})$ and update each surviving transition weight with $g_{jk}+g_{ji}g_{ik}/(1-g_{ii})$. Then compare each remaining p-value with its newly calculated level.

Check Your Understanding

Use the graph-update rules to answer each question. The solutions are included below for checking.

A rejected hypothesis has current level $0.012$, self-loop weight $0.25$, and weight $0.45$ to $H_j$. How much level eventually reaches $H_j$?
When removing $H_i$, suppose $g_{jk}=0.30$, $g_{ji}=0.20$, $g_{ik}=0.40$, and $g_{ii}=0.20$. Calculate $g'_{jk}$.
A remaining hypothesis has current level $0.018$ and p-value $0.019$. Is it rejected under the stated rule?
Explain why an updated transition weight of $0.70$ with a self-loop of $0.30$ can represent eventual transfer of the full level.

Solutions. (1) The transfer is $0.012(0.45)/(1-0.25)=0.0072$. (2) $g'_{jk}=0.30+0.20(0.40)/(1-0.20)=0.30+0.10=0.40$. (3) No: $0.019>0.018$, so the hypothesis is not rejected. (4) The eventual exit fraction is $0.70/(1-0.30)=1$, since repeated circulation through the self-loop is included.