Independence and the Sampling Design

Why Independence Depends on the Study Design

A one-way ANOVA can compare group means only if its observations contain the amount of independent information assumed by the test. A data table may contain many rows, but rows are not necessarily independent observations: several may come from the same person, machine, household, or clinic. The study’s sampling and assignment structure—not the number of recorded values—determines what counts as an independent unit.

This matters because the usual one-way F test uses within-group variation to estimate uncertainty in the group means. If responses from related units tend to move together, the data contain less independent information than the row count suggests. Treating correlated rows as independent can make standard errors too small and the F statistic too large, increasing the chance of declaring a group difference when the evidence is weaker than it appears.

Definition: An observational unit is the entity on which a study makes an independent sampling or assignment decision. A recorded measurement is not necessarily an observational unit. In the usual one-way fixed-effects model, the response from unit $j$ in group $i$ is written $Y_{ij}=\mu_i+\varepsilon_{ij}$. The independence condition is that the errors $\varepsilon_{ij}$ are mutually independent across all observations, conditional on the group structure. This condition is separate from assumptions about the error distributions and variances.

For example, if a treatment is assigned separately to each of 24 independently prepared specimens, the specimens may be the observational units. If treatment is assigned to four factories and six specimens are measured within each factory, the factories—not the 24 specimens—are the independently assigned units for comparing treatments. Each specimen still provides a measurement, but specimens from the same factory may share conditions and therefore have related responses.

Independence is a feature of how the data were generated. A scatterplot or a statistical test cannot establish that the design produced independent observations. A useful first pass is to ask: Who or what was sampled? What entity received the treatment? Was each entity measured once, or repeatedly? Were observations grouped by a shared source such as a site, batch, family, or operator?

Sampling, Assignment, and the ANOVA Model

The usual one-way ANOVA compares $k$ groups. Its independence requirement concerns errors within a group and across groups. It is not enough that the group labels differ: if measurements in two groups come from the same person or shared cluster, their errors may be associated. Conversely, the fact that observations share a treatment label does not itself make them dependent; independently selected units can receive the same treatment.

Random sampling and random assignment have different roles. Random sampling can support generalization from sampled units to a target population. Random assignment can support causal comparisons between treatments by distributing systematic differences among treatment groups. Neither, by itself, guarantees independent outcomes. For example, assigning a treatment at the school level does not make students in the same school independent. The relevant structure is the set of units and the way outcomes can share influences.

Name the unit.
State what one row represents and what entity was independently sampled or assigned. Do not assume that one row equals one independent unit.

Trace shared sources.
Check whether multiple responses come from the same person, cluster, batch, household, site, or other common source.

Check repeat measurement.
If the same unit contributes responses to multiple groups or times, those responses are generally linked rather than independent.

Decide whether the usual F test fits.
Use it only when the observational units and design support independent errors. If dependence is part of the design, the analysis must account for it rather than count each measurement as a separate independent replicate.

Worked Example: Separately Randomized Specimens

A fictional materials laboratory compares three sealant formulations. It independently prepares 18 test coupons, assigns six coupons at random to each formulation, and measures one adhesion strength on each coupon. Each coupon comes from a separately prepared batch, and no coupon is tested more than once. Assess whether the observation structure supports the usual one-way ANOVA independence condition.

Method. Identify the experimental unit, then check whether any unit contributes more than one response or shares a sampling cluster with another unit.

Step 1: Count the experimental units. There are 18 coupons. The treatment is assigned to each coupon separately, so each coupon is an experimental unit. There are three groups, with six units per group.

Step 2: Count the responses per unit. Each coupon contributes exactly one adhesion measurement. No coupon appears in more than one group, and no coupon is measured repeatedly.

Step 3: Check for shared sources. The problem states that the coupons are separately prepared and does not identify a shared batch, operator, or other cluster. On those stated design facts, there is no identified source that links responses. The design therefore supports treating the 18 errors as independent, subject to the laboratory’s stated independence assumption.

Conclusion. Independence is plausible for the usual one-way ANOVA: six separate units contribute to each formulation group, and each unit contributes one response. This conclusion does not establish the other ANOVA assumptions. It also depends on the design description being accurate; for example, discovering that all six coupons for one formulation came from a single batch would require reassessing the independence claim.

Common mistake avoided. It would be incorrect to say that random assignment alone proves independence. Here the conclusion is supported by the separate experimental units, one measurement per unit, and absence of an identified shared cluster—not merely by the word “random.”

Repeated Measurements Are Linked

Repeated measurements occur when the same unit contributes more than one response. Examples include measuring the same patient under several conditions, testing the same machine at several settings, or recording one worker’s output over multiple shifts. The measurements may be useful, but they are not independent replications: stable features of the unit can affect all of its responses.

If a patient tends to have a high response under every condition, that patient’s measurements will tend to be high together. In a one-way ANOVA that treats each condition as a separate group, the repeated values from that patient are then associated across groups. The usual independent-errors model does not represent that association, so its usual F reference calculation is not justified by that design. The appropriate analysis must preserve the matching or repeated-measures structure.

Worked Example: Three Readings From Each Patient

In a hypothetical pilot study, four patients each provide a response under each of three monitoring settings. The recorded responses are as follows: Patient A: 8, 10, 11; Patient B: 12, 14, 15; Patient C: 10, 12, 13; Patient D: 14, 16, 17. The three positions in each row are settings 1, 2, and 3. A researcher proposes an ordinary one-way ANOVA with 12 observations, four in each setting. Assess the independence issue.

Method. Determine how many distinct units were measured and whether a unit appears in multiple groups.

Step 1: Identify the units. The observational units are the four patients. Each patient contributes three responses, one to each setting. Thus the table has 12 measurements but only four independently sampled people.

Step 2: Calculate the setting means. For setting 1, the total is $8+12+10+14=44$, so the mean is $44/4=11$. For setting 2, the total is $10+14+12+16=52$, so the mean is $52/4=13$. For setting 3, the total is $11+15+13+17=56$, so the mean is $56/4=14$.

Step 3: Trace the responses. Patient A contributes to all three means, as do Patients B, C, and D. A patient’s stable characteristics can affect all three responses. The 12 values therefore cannot be treated as 12 independent outcomes on the basis of this design.

Conclusion. The proposed ordinary one-way ANOVA treats repeated measurements from a patient as independent and does not reflect the design. The calculated means are descriptive summaries, but their differences do not make the ordinary independent-groups F test appropriate. An analysis that accounts for the repeated measurements is needed.

Common mistake avoided. Counting four values per setting and concluding “four independent observations per group” confuses the number of measurements with the number of independent units. There are four people, each measured three times.

Clusters and the Cost of Ignoring Correlation

A cluster is a group of units that share a setting or source, such as students within a school, patients within a clinic, or products made on the same production line. Responses within a cluster may be more alike than responses from different clusters. The resulting dependence is often called intracluster correlation.

The following calculation shows why counting all individuals as independent can overstate the information in a clustered sample. Assume equal cluster size $m$, the same response variance $\sigma^2$ for each individual, and a common within-cluster correlation $\rho$ for every distinct pair in the same cluster. Also assume different clusters are independent. These are simplifying assumptions for the calculation, not a universal model for all clustered data.

Derivation: For one cluster with responses $Y_1,\ldots,Y_m$, the cluster mean is $\bar{Y}=m^{-1}\sum_{j=1}^m Y_j$. Each variance is $\sigma^2$, and each distinct within-cluster covariance is $\rho\sigma^2$. Therefore, $$ \operatorname{Var}(\bar{Y}) =\frac{1}{m^2}\left(m\sigma^2+m(m-1)\rho\sigma^2\right) =\frac{\sigma^2}{m}\left[1+(m-1)\rho\right]. $$ If the $m$ responses were independent, the variance would instead be $\sigma^2/m$. Thus the variance is multiplied by the design effect $1+(m-1)\rho$. When $\rho>0$, a cluster of $m$ people provides less information than $m$ independent people.

This derivation explains a general warning about the usual one-way F test: positive within-cluster association can make the apparent amount of independent information too large. The design effect is a useful illustration, not a replacement for an analysis that respects the sampling design. In particular, an “effective sample size” calculated from it should not automatically be substituted for the actual sample size in an F test.

Worked Example: A Clinic-Level Program

A hypothetical health network assigns a new reminder program to four clinics and the existing program to four other clinics. Five patients are sampled at each clinic, and each patient contributes one response. Suppose, for illustration, that the response variance among patients is $\sigma^2=25$ and the within-clinic correlation is $\rho=0.40$. A proposed analysis treats all 40 patients as independent observations, with 20 patients in each program group. Explain the problem and quantify the effect of clustering under the stated model.

Method. Identify the level of treatment assignment, then calculate the design effect for the five patients sampled within a clinic.

Step 1: Find the assignment units. The reminder program is assigned to clinics, not individual patients. There are eight independently assigned clinics in total: four per program. The 40 patient responses are nested within those eight clinics.

Step 2: Calculate the design effect. The cluster size is $m=5$, and the stated within-clinic correlation is $\rho=0.40$. Therefore, $$ 1+(m-1)\rho=1+(5-1)(0.40)=1+1.60=2.60. $$

Step 3: Interpret the variance multiplier. Under the stated equal-size, common-correlation model, the variance of a five-patient clinic mean is 2.60 times the variance it would have if those five responses were independent. The corresponding standard-deviation multiplier is $\sqrt{2.60}\approx1.612$, since $\sqrt{2.60}$ is approximately 1.612.

Step 4: State what the calculation does and does not establish. The calculation illustrates why 20 patients per program cannot be treated as 20 independent treatment assignments. The treatment was assigned to four clinics per program; patient responses within a clinic may be correlated. The design effect does not provide a substitute F test or change the number of clinics. An analysis must account for clinic-level assignment and dependence.

Conclusion. The proposed ordinary one-way ANOVA on 40 supposedly independent patient responses ignores the clustering structure. Its independence condition is not supported merely because each patient was measured once.

Common mistake avoided. “One response per patient” does not prove all responses are independent. Patients in the same clinic share a treatment assignment and potentially a common environment, so the clinic structure matters even without repeated measurements on an individual.

Sampling Without Replacement and Other Design Clues

Independence can also be affected by how units are sampled. A simple random sample taken without replacement from a finite population is not exactly independent: after one unit is selected, the composition of the remaining population changes. If the sampling fraction is small, treating the draws as approximately independent can often be a reasonable approximation for some sampling calculations. That approximation should not be confused with exact independence, and it does not fix dependence caused by repeated measurements or clusters.

Other design clues deserve attention. Measurements taken in a time sequence may be linked if conditions change gradually. Siblings may share family influences. Multiple readings from one instrument may share calibration error. Conversely, units being observed at the same time or receiving the same treatment does not by itself establish dependence; the question is whether their errors are related under the study design.

Design feature	Independence concern	Question to ask
One response from each separately sampled unit	Often supports independent observations, subject to the sampling process	Are there unreported shared sources or links?
Same person measured in several groups or conditions	Repeated responses can be correlated	Does each person contribute to more than one group?
Several people sampled within each site or household	Members of a cluster may share influences	What level was sampled or assigned independently?
Sampling without replacement from a finite population	Draws are not exactly independent	Is an independence approximation justified by the sampling fraction?

Common Mistakes and a Design-Based Check

Equating rows with independent units. A row is a recorded observation; it is independent only if the design and model support that claim.
Assuming random assignment resolves all dependence. Individual randomization does not erase shared-person or shared-site effects, and cluster-level randomization makes the cluster the assignment unit.
Treating repeated values as new replications. Repeated measurements can improve understanding of within-unit change, but they do not become independent people or experimental units.
Using the design effect as a repair formula for every F test. The multiplier above illustrates loss of information under explicit assumptions; it is not a general replacement for modeling the design.
Confusing absence of detected correlation with independence. Independence must be justified from the sampling and assignment structure. A small data set may not reveal dependence reliably.

Key takeaway: Before using the usual one-way ANOVA F test, identify the unit that was independently sampled or assigned and trace every response back to that unit. Repeated observations and observations nested in shared clusters may violate the independent-errors condition, even when the data table contains many rows.

Check Your Understanding

For each design, decide what the independent unit is and whether the usual one-way ANOVA independence condition is supported as described.

A fictional greenhouse randomly assigns 30 separate plant pots to three nutrient mixtures, with one plant and one final height measurement per pot. No pots share a tray. What is the unit, and does the described design support independence? (Answer: The pot/plant is the unit; yes, independence is supported by the described separate randomization and one measurement per unit.)
A lab measures each of eight devices once under each of two calibration settings, then proposes an ordinary one-way ANOVA with 16 readings. What is the key independence issue? (Answer: The same eight devices contribute to both groups, so readings from a device are repeated and generally dependent; the ordinary independent-groups ANOVA is not justified.)
A study assigns a training method to six work teams, three teams per method, and records output from ten workers per team. What is the treatment-assignment unit, and why are there not 30 independent treatment assignments per method? (Answer: The team is the assignment unit; workers within a team share the assigned method and may be correlated, so there are three independently assigned teams per method, not 30.)
In a cluster model, each cluster has $m=4$ observations and the within-cluster correlation is $\rho=0.25$. Calculate the design effect and the standard-deviation multiplier relative to four independent observations. (Answer: Design effect $=1+(4-1)(0.25)=1.75$; standard-deviation multiplier $=\sqrt{1.75}\approx1.323$.)
A quality auditor samples 10 items without replacement from a batch of 1,000 items, measuring each item once. Are the sample observations exactly independent, and is an independent-observation approximation plausible based on the sampling fraction alone? (Answer: They are not exactly independent because sampling is without replacement; the sampling fraction is $10/1000=0.01$, so an independence approximation is plausible based on that small fraction, subject to other design features.)

What You'll Learn