What a Chi-Square Conclusion Can Say
A chi-square test can provide evidence that two categorical variables are associated, or that a categorical outcome has different distributions across groups. Turning that evidence into a sound conclusion requires more than repeating a software output. The conclusion should name the variables or groups, state what the data support at the chosen significance level, and respect how the data were collected.
The last point is essential. An association in an observational table does not, by itself, show that one variable caused changes in the other. A small p-value describes how surprising the observed table, or a more extreme one, would be under the test’s null model. It does not identify a cause, measure the importance of a relationship, or tell us whether the study design supports generalization to a wider population.
A Reliable Structure for the Conclusion
Before interpreting the result, make sure the test is appropriate for the study. The usual chi-square procedures assume that the observations are independent and that expected cell counts are adequate for the chi-square approximation. A common guideline is that every expected count should be at least 5. The way the observations were selected or assigned also matters: random sampling supports conclusions about the population sampled, while random assignment in an experiment can support causal conclusions about the assigned treatment, subject to the study being carried out appropriately.
Under the null hypothesis of independence, expected counts come from the row and column totals. For a cell in row \(i\) and column \(j\), the expected count is
This formula follows from the null model: if the row and column classifications are independent, the expected proportion in that cell is the row proportion multiplied by the column proportion. Multiplying that joint proportion by the total number of observations gives the expected count. The test statistic compares the observed counts \(O_{ij}\) with those expected counts:
The p-value is calculated using the chi-square distribution under the null hypothesis and the appropriate degrees of freedom. A small p-value is evidence against that null model; it is not the probability that the null hypothesis is true. The test statistic and p-value also do not, by themselves, describe which cells differ or how important the relationship is.
A concise conclusion generally includes four elements: the decision at the stated significance level; the evidence for association or differing distributions; the variables or outcome and groups in context; and a qualification about causation or generalization when the study design requires one. If a p-value is not statistically significant, say that the analysis did not find sufficient evidence of an association or difference. Do not say that it proved there is no relationship.
Identify the two variables in an independence test, or the categorical outcome and groups in a homogeneity test.
Report the chi-square statistic, degrees of freedom, p-value, and significance level when relevant; say whether the evidence is statistically significant at that level.
Describe evidence of association or evidence that the group distributions differ. Use observed proportions to clarify the pattern without implying that the test alone proves a cause.
State whether the data are observational or come from a randomized experiment, and limit causal and population claims accordingly.
Worked Conclusions
Worked Example: An Observational Association
In a hypothetical survey, 100 adults are classified by whether they report high or low caffeine intake and whether they report poor or adequate sleep. The counts are shown below. Suppose the survey used a random sample, each adult contributed one independent response, and all expected counts meet the guideline of at least 5.
| Caffeine intake | Poor sleep | Adequate sleep | Total |
|---|---|---|---|
| High | 40 | 10 | 50 |
| Low | 20 | 30 | 50 |
| Total | 60 | 40 | 100 |
Under independence, the expected counts in the high-intake row are \(50(60)/100=30\) for poor sleep and \(50(40)/100=20\) for adequate sleep. The low-intake row has those same expected counts. Thus every expected count is at least 5. The test statistic is
For a \(2\times2\) independence test, \(df=(2-1)(2-1)=1\). The p-value is approximately \(0.000045\), well below \(\alpha=0.05\). The data provide strong evidence of an association between reported caffeine-intake category and reported sleep-quality category among the population represented by the sample. In this sample, 40 of 50 adults with high intake (80%) reported poor sleep, compared with 20 of 50 adults with low intake (40%).
The conclusion must stop short of saying that high caffeine intake caused poor sleep. This was an observational survey: other factors, such as work schedules or health conditions, could be related to both variables. The small p-value is evidence against independence, not evidence that caffeine caused the difference.
Worked Example: Different Distributions After Random Assignment
In a hypothetical randomized experiment, 180 patients are assigned equally to one of three reminder strategies. At day 30, each patient is classified as adherent or not adherent to a prescribed schedule. The counts are:
| Assigned reminder strategy | Adherent | Not adherent | Total |
|---|---|---|---|
| A | 45 | 15 | 60 |
| B | 36 | 24 | 60 |
| C | 30 | 30 | 60 |
| Total | 111 | 69 | 180 |
For each strategy, the expected counts under equal adherence distributions are \(60(111)/180=37\) adherent and \(60(69)/180=23\) not adherent. All expected counts exceed 5. The chi-square statistic is
For a \(3\times2\) table, \(df=(3-1)(2-1)=2\). The p-value is approximately \(0.018\), below \(\alpha=0.05\). The data provide evidence that adherence proportions differ among the three assigned reminder strategies. In this experiment, adherence was 75% for strategy A, 60% for strategy B, and 50% for strategy C.
Because patients were randomly assigned, a causal conclusion about the effect of assignment to the reminder strategies can be justified if the experiment was conducted appropriately and the outcome data were handled without bias. A careful conclusion is that assignment to reminder strategy affected adherence in this experiment. The chi-square result does not establish why the strategies differed, and broader generalization depends on how representative the patients and setting are.
Worked Example: A Non-Significant Result Is Not Proof of No Association
In a hypothetical random sample of 100 residents, each person is classified by neighborhood type and whether they usually use public transit. There are 50 urban and 50 rural residents; 40 urban residents and 35 rural residents report usually using transit. Assume one independent response per person and adequate expected counts.
| Neighborhood type | Usually uses transit | Does not usually use transit | Total |
|---|---|---|---|
| Urban | 40 | 10 | 50 |
| Rural | 35 | 15 | 50 |
| Total | 75 | 25 | 100 |
The expected counts for each row are \(50(75)/100=37.5\) transit users and \(50(25)/100=12.5\) non-users. Each is at least 5. Therefore,
Here \(df=1\), and the p-value is approximately \(0.248\). Since \(0.248>0.05\), the result is not statistically significant at the 5% level. The sample does not provide sufficient evidence of an association between neighborhood type and usual transit use. It does not prove the variables are independent: the observed sample proportions are 80% for urban residents and 70% for rural residents, and a different sample could provide different evidence. As the data are observational, even a significant result would not, on its own, show that neighborhood type caused a difference in transit use.
Why the Study Design Matters
The same statistically significant table can support different kinds of conclusions depending on the data collection. In an observational study, people or units are classified as they are; researchers do not assign the explanatory category. A detected association may reflect the relationship of interest, other related variables, or both. Conclude that the variables are associated, not that one caused the other.
In a randomized experiment, researchers assign units to treatments at random. If the experiment is appropriately conducted, differences in outcomes can support a causal conclusion about the effects of assignment. Random assignment is important because it tends to balance other influences across treatment groups. It does not automatically make the participants representative of a larger population, however, so causal evidence within the experiment and generalization beyond it are separate questions.
Sampling design also affects the population named in a conclusion. A random sample from a defined population can support inference about that population, subject to the sampling and test assumptions. A convenience sample may describe the people observed without justifying the same population-level claim. Always match the wording to the actual sampling frame and design.
Common Mistakes
- Claiming causation from association. “Caffeine caused poor sleep” is not supported by the observational survey. Say that caffeine-intake category and sleep-quality category were associated in the sampled population.
- Calling a non-significant result proof of no relationship. A p-value above \(\alpha\) means the data did not provide sufficient evidence against the null at that threshold; it does not establish that the null is true.
- Reporting only “significant” or “not significant.” Name the variables or groups and describe what the result means in context. Include the direction or pattern using observed proportions when useful.
- Treating the p-value as the probability the null is true. The p-value is calculated under the null model; it is not the probability that the null hypothesis is correct.
- Generalizing beyond the study. A result from a particular sample or experiment does not automatically apply to every population or setting.
Check Your Understanding
For each question, focus on what the statistical result supports and what the design permits you to conclude.
- A random survey finds a statistically significant association between pet ownership and reported stress category. What may the conclusion say, and why should it avoid claiming that pet ownership caused the stress difference?
- A chi-square test comparing outcome distributions across three randomly assigned treatments has \(p=0.03\), with \(\alpha=0.05\). What does the result indicate about the distributions?
- A test of independence gives \(p=0.21\) at \(\alpha=0.05\). Why is “the variables are independent” too strong a conclusion?
- How does random assignment differ from random sampling in the kinds of conclusions each design can support?
- Why should a conclusion name the variables or groups instead of reporting only the p-value?