Why Check Independence Before Comparing Groups?
A comparison between two groups is not determined only by how many observations were collected or whether the groups have different labels. It also depends on how observations are related. If several measurements come from the same person, household, clinic, or school, they may carry shared information. Treating such measurements as if they were independent can make the evidence appear more precise than it is.
The design tells us which observations might be linked; a probability model tells us whether they are statistically dependent. These are related but not identical ideas. In particular, repeated measurements on the same person are linked by design and should not be assumed independent without justification. They are often modeled as dependent, but sharing a person does not, by itself, prove dependence.
In an ordinary two-group analysis, the relevant assumption is commonly that the observations are independent within each group and that the groups are independent of one another. The design should make that assumption plausible, or there should be a model that explicitly accounts for the dependence. Independence is not established merely by placing measurements in separate columns or assigning different names to groups.
Design Clues: Pairing, Households, and Clusters
Start by identifying the actual observational units and tracing how each outcome was produced. In a study of people, ask whether each person contributes one outcome or several, whether participants share a household, and whether multiple participants were recruited from the same site. In a study of schools or clinics, identify whether the school or clinic—not each person—is the unit selected or assigned.
A paired design deliberately connects observations, as in a before-and-after measurement on each person or a matched set of participants. The earlier material on paired designs treats the within-pair difference as the natural quantity to analyze. That design feature does not amount to a mathematical proof that the two values in every pair are dependent. Rather, pairing warns us not to treat the two columns as independent samples unless a suitable model supports that treatment.
Shared households and clusters raise a different concern: observations from different people can still be related. Family members may share diet or environment; patients at one clinic may share staff or local conditions; students at one school may share policies and resources. A group comparison can therefore have dependence within a group, across groups, or both. For example, if both treatment groups include people from the same clinic, a clinic-level influence can link outcomes across treatment groups.
Random assignment and independent sampling are useful design features, but neither phrase should be treated as a magic guarantee. Assignment may be at the person level while outcomes remain correlated within households, or one person's treatment may affect another person's outcome. Likewise, sampling several people from each selected clinic creates a cluster structure. The relevant question is whether the outcome variables can reasonably be modeled as independent at the level used by the proposed analysis.
Worked Example: Before-and-After Measurements on the Same People
In a hypothetical pilot, six volunteers have a concentration measured before and after a dietary change, in milligrams per liter. The paired measurements are: 18 and 16, 21 and 20, 17 and 17, 24 and 21, 20 and 19, and 19 and 17. Calculate the change for each person and assess the independence issue.
Define each change as after minus before. The six changes are \(-2,-1,0,-3,-1,-2\) mg/L. Their mean is \( \bar{x}_{\text{change}} = (-2-1+0-3-1-2)/6 = -9/6 = -1.50 \) mg/L. This summary describes the observed within-person changes; it does not establish whether a probability model would regard the measurements as statistically dependent.
The design links each before value to the after value from the same volunteer. One plausible model includes a random person-specific level \(U_i\): \(X_i=\mu_{\text{before}}+U_i+\varepsilon_{i1}\) and \(Y_i=\mu_{\text{after}}+U_i+\varepsilon_{i2}\), where the errors are independent of each other and of \(U_i\). If the variance of \(U_i\) is 4 and each error variance is 1, then \(\operatorname{Cov}(X_i,Y_i)=4\), \(\operatorname{Var}(X_i)=\operatorname{Var}(Y_i)=5\), and the correlation is \(4/\sqrt{5\cdot5}=0.800\). Under this model, the shared person-level variation makes the measurements dependent.
But suppose each person's level is a fixed constant \(u_i\), not a random effect, and the two measurement errors are independent. Then \(X_i=u_i+\varepsilon_{i1}\) and \(Y_i=u_i+\varepsilon_{i2}\) are independent under that model: adding fixed constants does not create dependence between independent errors. The correct conclusion is therefore that repeated measurements are linked by design and should not automatically be treated as independent—not that same-person measurements must always be dependent.
A Shared-Effect Model Makes Dependence Visible
A useful way to understand household or cluster dependence is to represent a shared influence explicitly. Suppose two different people in the same household have outcomes \(Y_1=\mu_1+U+\varepsilon_1\) and \(Y_2=\mu_2+U+\varepsilon_2\). Here \(U\) is a random household effect shared by both people; the errors \(\varepsilon_1\) and \(\varepsilon_2\) are independent of one another and of \(U\). A nonzero variance in \(U\) produces covariance between the outcomes.
\(\operatorname{Cov}(Y_1,Y_2)=\operatorname{Cov}(U+\varepsilon_1,\ U+\varepsilon_2)\)
\(=\operatorname{Var}(U)+\operatorname{Cov}(U,\varepsilon_2)+\operatorname{Cov}(\varepsilon_1,U)+\operatorname{Cov}(\varepsilon_1,\varepsilon_2)\)
\(=\operatorname{Var}(U)\).
If both outcomes have error variance \(\sigma_\varepsilon^2\), their common variance is \(\operatorname{Var}(U)+\sigma_\varepsilon^2\). Their correlation is therefore \(\operatorname{Var}(U)/[\operatorname{Var}(U)+\sigma_\varepsilon^2]\). This calculation depends on the stated shared-effect model; a household label alone does not determine the amount of dependence.
Worked Example: Two Participants From Each Household
Imagine a hypothetical study measuring a health score for different people, with two participants recruited from each household. For illustration, suppose a participant's score is modeled as the overall mean plus a household effect \(U\) and an individual error. Let the household-effect variance be 9 score-points-squared and the error variance be 16 score-points-squared.
For two people in the same household, the shared-effect model gives \(\operatorname{Cov}(Y_1,Y_2)=\operatorname{Var}(U)=9\) score-points-squared. Each outcome has variance \(9+16=25\) score-points-squared, so the correlation is \(9/\sqrt{25\cdot25}=9/25=0.360\). Thus, under this illustrative model, outcomes from the same household are positively dependent even though the observations come from different people.
Now consider two people from different households, with independent household effects and independent individual errors. They do not share the \(U\) term, so their covariance under this model is zero and the outcomes are independent if the full model also makes their components independent. The household membership—not simply the fact that there are two group labels—is the design information that reveals which observations share a source of variation.
If a group comparison treats all participants as independent, it ignores the modeled connection between people in the same household. The calculation does not prescribe a particular alternative method; it shows why the independence assumption needs to match the sampling and measurement structure.
Group Separation Does Not Guarantee Group Independence
Two groups can contain different individuals yet remain statistically linked. This can happen when both groups are observed in the same clusters, when one person's outcome affects another's, or when both groups are selected through a common process that induces dependence. Conversely, two measurements on one person are not automatically dependent under every model. Group membership and dependence are distinct questions.
Worked Example: Two Treatment Groups Across the Same Clinics
Suppose a hypothetical trial assigns individual patients to treatment A or treatment B at each of four clinics. Each clinic contributes patients to both groups. Consider one patient's outcome from each treatment group at the same clinic. Use the illustrative model \(Y_A=\mu_A+U+\varepsilon_A\) and \(Y_B=\mu_B+U+\varepsilon_B\), with clinic-effect variance 6 and independent error variance 10 for each patient.
The two patients are different people and receive different treatments, but their outcomes share the clinic effect. Their covariance is \(\operatorname{Cov}(Y_A,Y_B)=\operatorname{Var}(U)=6\). Each outcome has variance \(6+10=16\), so their correlation is \(6/\sqrt{16\cdot16}=6/16=0.375\). Under this model, outcomes from the two treatment groups are not independent simply because the patients belong to different groups.
The group means can each combine patients from several clinics, so the full dependence pattern involves observations nested within clinics, including links across treatment groups. The calculation identifies the issue; it does not establish that every clinic has the same effect or that this particular model is correct. The study design and subject-matter knowledge must support the model used.
If instead each clinic contributed patients to only one treatment group, a clinic effect could still make observations within that group dependent. In that arrangement the groups might avoid sharing clinic effects, but within-group independence would still need scrutiny. Independence between groups and independence within groups are separate checks.
A Practical Independence Check
Use the design in a deliberate sequence before choosing an ordinary two-group procedure. The questions below focus on dependence, not on the other conditions that may be needed for a particular interval or test.
State whether one observation represents a person, a household, a clinic, a time point, or another unit. Count how many outcomes each unit contributes.
Look for repeated measurements, matched pairs, shared households, common schools or clinics, and other clustering. Ask whether a model or design justifies treating connected outcomes as independent.
Check whether participants are genuinely separate, whether clusters contribute to both groups, and whether one unit's treatment or outcome can influence another unit's outcome.
Say that independence is plausible under the stated design assumptions, that dependence is built into the design, or that the structure needs further modeling. Do not claim the observed data alone prove independence.
For example, independently sampled people from two nonoverlapping populations, with one measurement per person and no meaningful shared clusters or interference, may support an independent-groups model. If sampling is without replacement from a finite population, observations are not exactly independent; whether that dependence can be ignored depends on the sampling setting and the intended model. The design should be described rather than hidden behind the phrase “random sample.”
Common Mistakes and Why They Matter
- “Different columns mean independent samples.” A before-and-after data table has two columns, but each row may link measurements from one person. Check how rows were formed.
- “Different people must be independent.” People from one household or clinic can share influences. Check for clusters, not only repeated identities.
- “Same person proves dependence.” Repeated measurements are linked by design and often modeled as dependent, but dependence is a property of the joint distribution. A model with fixed individual means and independent errors can yield independent measurements.
- “Random assignment guarantees independence.” Assignment may be random while outcomes remain connected through households, clinics, or interference. Check the outcome-generating structure as well as assignment.
- “The two groups are independent, so everything is fine.” Observations may be dependent within each group even if the group-level samples are independent. Check both levels.
Ignoring dependence can distort the uncertainty attached to a group comparison. In particular, treating correlated observations as independent can make a sample look as though it contains more independent information than it does. The exact consequence depends on the dependence pattern, so it is not correct to assume that every cluster structure changes uncertainty in precisely the same direction or amount. The essential decision is whether the procedure's independence model is credible for the study design.
Check Your Understanding
For each situation, identify the independence concern and state what additional design or modeling information would help resolve it.
- A study records blood pressure before and after an intervention for each of 30 people. What does the repeated-measure design tell you, and what does it not prove?
- A survey samples two adults from each of 50 households. Under a shared-household-effect model, why might two adults from the same household have correlated outcomes?
- A trial assigns patients individually to two groups, with both groups represented at each clinic. Which observations might share a clinic-level influence?
- Explain why independent groups do not automatically imply independence among observations within each group.
- Give an example of a model under which two measurements from the same person could be independent, and state the relevant assumption about the errors.