Analyzing a Two-Stage Network Experiment

From a Two-Stage Design to an Exposure Effect

In a network experiment, a person’s outcome may depend both on their own treatment and on treatment received by people connected to them. A two-stage randomized saturation design makes these treatment patterns part of the experiment: first, groups are assigned different treatment saturation levels; then, individuals are assigned treatment within groups. An analysis must carry that design through to the exposure definition, the estimand, the weights, and the uncertainty calculation.

The analysis below uses a simple exposure mapping: a person’s exposure is their own treatment status together with whether any of their in-group neighbors are treated. This is a deliberate simplification, not a universal description of network effects. The mapping must represent the treatment features relevant to the scientific question; otherwise, the resulting estimates concern the mapped categories, not necessarily the actual mechanisms.

For the mapping used here, the categories are \(D=(1,0)\), meaning treated with no treated neighbor; \(S=(0,1)\), meaning untreated with at least one treated neighbor; and \(C=(0,0)\), meaning untreated with no treated neighbor. The direct contrast at \(G_i=0\) is \(\mu(D)-\mu(C)\). The spillover contrast among untreated individuals is \(\mu(S)-\mu(C)\). These are effects of the specified exposure contrasts; they do not summarize every possible way treatment could affect a network.

Assumptions and Assignment Probabilities

Before estimating these contrasts, specify what makes the observed outcomes interpretable. The design must be known, including both stages of randomization. For the exposure-specific interpretation, assume consistency: when individual \(i\) receives exposure \(e\), their observed outcome is \(Y_i(e)\). Also assume the mapping is sufficient for the outcome: assignments producing the same mapped exposure do not cause additional outcome differences through omitted features of the assignment. Finally, assume no interference across the groups used in the design, unless cross-group connections are explicitly incorporated into the mapping. These assumptions are substantive; randomization alone does not make a poor mapping adequate.

For an exposure \(e\), define \(\pi_i(e)=P(E_i=e)\), using the full two-stage assignment procedure. Positivity for this analysis requires \(\pi_i(e)>0\) for every individual included in the target mean. Exposure probabilities can vary across individuals, so they should be calculated from the design rather than assumed equal.

Horvitz–Thompson Estimation

Once the mapping and probabilities are specified, an exposure mean can be estimated by weighting observed outcomes inversely by their exposure probabilities. This Horvitz–Thompson estimator targets the finite-population mean, rather than simply averaging the people observed in that exposure category.

The reason for the weighting is visible from a short expectation calculation. Under the randomization distribution and consistency, if \(E_i=e\), then the observed outcome is \(Y_i(e)\). Thus the weighted contribution for individual \(i\) is \(I(E_i=e)Y_i(e)/\pi_i(e)\). Its expectation is

Averaging over the \(N\) individuals gives \(E[\widehat{\mu}_{HT}(e)]=\mu(e)\). Subtracting two such estimates gives an unbiased estimator of the corresponding mean contrast. This result relies on correct exposure probabilities, positivity, and the consistency and mapping assumptions stated above; it does not claim that a particular estimate is free of randomization error.

Assessing Uncertainty With the Randomization

Individuals’ exposures are dependent: assigning one person to treatment changes the exposure possibilities for their neighbors, and the first-stage group assignment affects several people at once. Therefore, treating all observed individuals as independent observations does not reproduce the experiment’s uncertainty. A design-based assessment instead uses the actual assignment mechanism. For a small design, an exact randomization test can enumerate every permitted assignment.

For the example above, consider the sharp null hypothesis that no individual’s outcome changes under any assignment. Under this null, each individual’s observed outcome would be the same under all six possible assignments. A two-sided randomization test compares the observed absolute value of a statistic with its values under those assignments. The test’s p-value is the proportion of permitted assignments whose absolute statistic is at least as large as the observed one. This test concerns the sharp null; it is not automatically a test of only the average direct contrast.

Common Pitfalls and Interpretation

A common mistake is to calculate an unweighted average among people observed in each exposure category and interpret differences as finite-population exposure effects. In this example, the category counts happen to be one, two, and three, and the relevant probabilities are equal across individuals. In other designs, individuals can have different chances of entering a category. An unweighted category mean can then overrepresent individuals with higher exposure probabilities; the Horvitz–Thompson weights correct for that unequal chance under the stated assumptions.

A second mistake is to treat the network as if each person were independently randomized. Here the six complete assignments are the randomization units for the exact test; the six people do not have six independent exposure assignments. In larger studies, uncertainty methods must likewise reflect the two-stage design and the dependence induced by the network. A test or interval also needs a clearly stated target: inference about a sharp null, an average exposure contrast, or another quantity are not interchangeable.

Finally, a precise calculation cannot compensate for an unsuitable exposure mapping. If outcomes depend on the number of treated neighbors, for example, “at least one” may combine consequentially different situations. If connections cross groups, assuming no cross-group interference may also fail. Report the mapping and design assumptions alongside estimates so readers know which causal contrasts the analysis supports.