Evaluating Independence from Study Design

Why Independence Depends on the Design

One-way ANOVA compares the means of several groups. Its calculations treat the observations as independent pieces of information. That condition is not checked by looking only at the response values: it depends on how the study selected, assigned, and measured its units. For example, measurements from different people may be independent, while two measurements from the same person are often related. Similarly, students in the same classroom may share conditions that students in different classrooms do not.

The key question is therefore not simply “How many values were recorded?” It is “Which units were independently sampled or assigned, and how many measurements came from each unit?” A study can record many responses but contain relatively few independent units.

Definition: In one-way ANOVA, the independence assumption is that the response errors for distinct observations are statistically independent, conditional on the group or treatment being compared. In the usual one-way model, \(Y_{ij}=\mu_i+\varepsilon_{ij}\), where \(Y_{ij}\) is observation \(j\) in group \(i\), \(\mu_i\) is that group’s mean, and \(\varepsilon_{ij}\) is its error. Independence requires the errors for different observations to be independent. The model also conventionally assumes errors have mean zero and a common variance; the independence question here is whether the study design supports treating distinct observations as independent.

An observational unit is the entity for which a response is recorded. An experimental unit is the smallest unit independently assigned to a treatment. They can be the same, as when individual patients are assigned separately and each contributes one response. They can differ, as when a classroom receives one assigned treatment but responses are recorded for every student in that classroom. In that case, the classroom—not each student—is the independently assigned unit.

Independence does not mean that every response must be numerically different, nor does it mean that group means must be equal. It means that knowing one observation does not provide information about the random error in another observation, once group membership is accounted for. A common source of dependence is a shared person, site, classroom, batch, or other unit that can make responses alike.

What Randomization Can Establish

When individuals are assigned separately to treatment groups using a genuine random procedure, that design supports treating the individuals’ responses as independent, provided each person contributes one response and the individuals do not affect one another in a way that links their outcomes. Random assignment also helps make treatment groups comparable. It does not, by itself, guarantee independence if several measurements come from the same individual or if treatment is assigned to groups of individuals together.

Random sampling and random assignment answer different questions. Random sampling concerns how units are selected from a population and supports generalizing to that population. Random assignment concerns how selected units are allocated to treatments and supports comparisons of those treatments. Neither phrase should be used as a substitute for checking what was actually measured and assigned. For example, if a researcher randomly selects four schools and measures 100 students in each, students within a school may still share influences.

Worked Example: Individually Randomized Participants

A hypothetical trial enrolls 24 adults. A computer lottery assigns 12 adults to each of two exercise programs. Each adult follows only the assigned program, and one change in resting heart rate, in beats per minute, is recorded per adult at the end of six weeks. Assume participants follow their assigned program independently and do not influence one another’s outcomes. Is independence a reasonable ANOVA assumption?

Each adult is both an observational unit and an experimental unit: treatment is assigned separately to each adult, and each adult supplies one response. The lottery makes the assignment individual rather than by household or class. Under the stated no-interference condition, there is no repeated measurement or shared assigned unit linking responses.

Thus there are 24 independently assigned units, 12 in each group. The design supports treating their response errors as independent for the one-way comparison. This judgment is based on the assignment and measurement structure, not on whether the 24 recorded changes happen to look different. It would need reconsideration if, for example, several participants were measured as a single household unit or each adult contributed multiple measurements treated as separate observations.

Repeated Measurements on the Same Unit

If the same person, animal, machine, or site contributes responses to multiple groups, those responses are usually related. Stable characteristics of that unit can make its measurements more alike than measurements from different units. Treating those repeated values as independent can therefore give an incorrect estimate of uncertainty.

A simple calculation shows why. Suppose each participant receives both of two conditions, and the response under each condition has variance \(\sigma^2\). Let the correlation between that participant’s two responses be \(\rho\). For \(n\) participants, the variance of the difference between the two sample means is

\operatorname{Var}(\bar{Y}_1-\bar{Y}_2) =\frac{\sigma^2}{n}+\frac{\sigma^2}{n} -2\frac{\rho\sigma^2}{n} =\frac{2\sigma^2(1-\rho)}{n}.

The subtraction appears because the two means share the same participants: their covariance reduces the variance of their difference when \(\rho>0\). The ordinary independent-groups expression omits that covariance and would instead be \(2\sigma^2/n\). The study’s structure determines which expression is relevant; the one-way independent-groups model is not made appropriate just by entering every response as a separate row.

Worked Example: Two Measurements per Participant

In a hypothetical crossover experiment, 12 participants each use both of two devices, and one response in milliseconds is recorded for each device. Suppose the response standard deviation under each device is 10 ms and the within-participant correlation between device responses is 0.60. What does the design imply for an independent-groups analysis?

The same 12 people supply values in both groups, so the two sets of 12 responses are paired, not independent. Using the stated variance and correlation, the variance of the difference between device means is

\frac{2(10^2)(1-0.60)}{12} =\frac{200(0.40)}{12} =\frac{80}{12} \approx 6.667\ \text{ms}^2.

If the responses were incorrectly treated as independent, the variance expression would be

\frac{10^2}{12}+\frac{10^2}{12} =\frac{200}{12} \approx 16.67\ \text{ms}^2.

These differ because the observations from the same participant have correlation 0.60. The values from the two devices cannot be counted as 24 independent responses. A repeated-measures analysis is designed for this kind of structure; the relevant conclusion here is that ordinary one-way ANOVA’s independent-observation model does not match the design.

Clustering and Shared Experimental Units

Dependence also arises when several observations come from the same cluster. Examples include students within classrooms, patients within clinics, plants within plots, or specimens from the same production batch. Members of a cluster may share conditions, so responses within a cluster tend to be related.

A particularly important case occurs when treatment is assigned to clusters but outcomes are recorded on individuals within them. The individuals are observational units, but the clusters are experimental units. If a researcher assigns a teaching method to a whole classroom, the 25 students in that classroom do not represent 25 independent treatment assignments. Counting each student as an independent replicate can exaggerate the amount of information in the study.

For equal-sized clusters, the effect of within-cluster correlation can be illustrated with a variance calculation. Suppose each cluster has \(m\) individuals, each response has variance \(\sigma^2\), and the correlation between any two responses in the same cluster is \(\rho\). The variance of a cluster’s mean is

\operatorname{Var}(\bar{Y}_{\text{cluster}}) =\frac{1}{m^2}\left(m\sigma^2+m(m-1)\rho\sigma^2\right) =\frac{\sigma^2}{m}\left[1+(m-1)\rho\right].

The factor \(1+(m-1)\rho\) is called the design effect in this equal-cluster setting. When \(\rho>0\), a cluster of \(m\) people contributes less information than \(m\) independent people. This calculation assumes equal cluster sizes and one common within-cluster correlation; it illustrates the dependence rather than supplying a general correction for every clustered study.

Worked Example: Treatment Assigned to Classrooms

A hypothetical study assigns a new lesson to four classrooms and the usual lesson to four other classrooms. Each classroom has five students whose quiz scores, in points, are recorded. Assume the within-classroom correlation is 0.20 and the marginal variance of an individual score is \(\sigma^2\). Is it appropriate to treat the 40 student scores as 40 independent observations in an ordinary one-way ANOVA?

Treatment was assigned to classrooms, so there are four independently assigned units per lesson, not 20 independent assignments per lesson. Students within a classroom also have correlation 0.20. With \(m=5\), the design effect is

1+(m-1)\rho=1+(5-1)(0.20)=1.80.

The variance-equivalent independent sample size for the 20 students in one lesson group is approximately

n_{\text{eff}}=\frac{20}{1.80}\approx 11.11.

This is a rough variance-equivalence calculation under the stated equal-size correlation model; it does not turn 11.11 students into independently randomized units. The actual independent treatment units remain the four classrooms in each group. Because the design has both cluster assignment and within-class dependence, an ordinary ANOVA treating all 40 students as independent does not reflect the study design.

A Design-Based Check for Independence

Before using one-way ANOVA, trace each recorded response back to the unit that generated it and the unit that received any assignment. A practical review is:

Identify the recorded unit.
Specify what each row represents: one person, one measurement occasion, one plot, or something else.

Identify the independently selected or assigned unit.
Determine whether treatment was assigned to each recorded unit or to a larger shared unit such as a class, clinic, or batch.

Look for repeated or linked observations.
Check whether a unit appears more than once, or whether several observations share a cluster, environment, or assignment.

Decide whether the one-way model matches.
Ordinary one-way ANOVA is defensible on independence grounds when distinct responses come from distinct independent units, with no design feature that links their errors. Otherwise, the analysis must account for the dependence.

Random assignment to individuals is useful evidence, but not a complete checklist. Repeated measures remain repeated even when treatment order is randomized. Likewise, randomly assigning classrooms does not make students in the same classroom independent. A design may also involve groups whose members can directly affect one another; in that situation, independence may fail even when each member was nominally assigned separately.

Independence is primarily a design assumption, so residual plots cannot establish it. A plot may reveal patterns in model errors, but an apparently pattern-free plot does not show that repeated observations or classmates are independent. The study description—who was measured, who was assigned, and which units were shared—provides the essential evidence.

Common Mistake and Why It Matters

A common mistake is to count every recorded value as an independent replicate. For example, a researcher might say, “We measured 40 students, so our sample size is 40,” even though the teaching method was assigned to eight classrooms and students within classrooms share influences. The correction is to distinguish the number of observations from the number and arrangement of independent units. Both matter, but they are not interchangeable.

Ignoring dependence can make the uncertainty in a group comparison appear smaller than it is, because correlated observations contain overlapping information. The arithmetic may still produce an \(F\) statistic and a p-value, but those calculations rely on the model’s assumptions. If the independence assumption is unsupported, the resulting inferential quantities need not have their stated interpretation.

Key takeaway: Decide whether ANOVA observations are independent by examining the study’s units and assignment structure. One response per independently assigned individual may support independence; repeated responses, cluster sampling, and treatment assigned to shared units call for recognizing dependence rather than counting every measurement as a separate independent replicate.

Check Your Understanding

For each situation, identify the independent unit and decide whether ordinary one-way ANOVA’s independence assumption is supported by the description.

A researcher randomly assigns 30 separate laboratory mice to three diets, with one mouse and one weight-change response per assigned unit. What design feature supports independence?
Each of 18 patients tries all three treatments, with one response recorded after each treatment. Why are the 54 measurements not 54 independent observations?
A school district assigns one reading program to each of six schools and records scores for 80 students per school. What is the experimental unit, and what dependence should be considered?
Explain why random assignment of classrooms to treatments does not make students within a classroom independent treatment units.
Why is a residual plot alone insufficient to establish independence?