From a Research Question to a Report
Choosing a procedure is not a matter of spotting a familiar word and immediately selecting a test. A reliable comparison begins by clarifying what population quantity the question concerns, what kind of outcome was measured, and how the two groups were formed. Those decisions determine which methods are suitable and which conditions need checking.
A useful workflow also prevents errors in the final interpretation. For example, an interval for a difference in average recovery time answers a different question from an interval for a difference in recovery proportions. And when the same people are measured twice, treating the observations as two independent groups discards the pairing built into the design.
A Practical Selection Workflow
Use the following sequence before calculating a statistic. The workflow is deliberately ordered: a procedure cannot be chosen correctly until the parameter and design are clear.
Write what population quantity is being compared. Specify group 1 and group 2, or define the paired difference, so the sign of the comparison is unambiguous.
A quantitative outcome leads to a comparison of means. A binary outcome, such as yes/no, leads to a comparison of proportions. Do not treat a category label as a quantitative measurement.
Decide whether the groups are independent or whether observations are naturally paired, such as before-and-after measurements on the same person. For paired data, analyze within-pair differences.
Use a confidence interval when the goal is to estimate the size and direction of a difference. Use a significance test when the question asks whether the data provide evidence against a specified null value. The design and outcome type still determine the underlying procedure.
Check the design and relevant distribution or count conditions before relying on the procedure. Then state the estimate or test result, the direction of the difference, and what it means for the population in the setting described.
| Outcome and design | Target parameter | Typical comparison procedure |
|---|---|---|
| Quantitative outcome; independent groups | \(\mu_1-\mu_2\) | Two-sample t procedure; use Welch's method unless equal population variances are justified for a pooled method |
| Quantitative outcome; paired observations | \(\mu_d\), the mean within-pair difference | One-sample t procedure applied to the differences |
| Binary outcome; independent groups | \(p_1-p_2\) | Two-proportion interval or test, with the appropriate condition checks |
This map is for comparisons of two groups. If the question has a different structure, such as several outcome categories or more than two groups, do not force it into one of these rows. First recognize that the two-group method does not match the question.
Why Paired Data Require a Different Target
When each observation in one condition is linked to an observation in the other condition, the informative quantity is usually the difference within each pair. For example, measurements before and after a program on the same person are related, not independent. Let \(d_i\) be the after measurement minus the before measurement for person \(i\). The sample mean difference is \(\bar{d}\), and the sample standard deviation of the differences is \(s_d\).
The standard error for estimating the population mean difference is based on the variation among the \(d_i\)'s. This follows from the usual standard error for a mean: when the \(n\) pair-differences are independent observations from the population of differences, their sample mean has estimated standard deviation \(s_d/\sqrt{n}\).
The standard error is not calculated by pretending that the before and after measurements came from unrelated groups. Pairing can reduce variability when people with higher measurements in one condition also tend to have higher measurements in the other. The paired analysis incorporates that relationship through the observed differences.
Worked Examples
Worked Example: Estimating a Difference in Two Group Means
In a hypothetical evaluation, two independent random samples of adults use different appointment-reminder systems. The outcome is the number of hours from appointment arrival to completion. For system 1, \(n_1=16\), \(\bar{x}_1=6.8\) hours, and \(s_1=1.2\) hours. For system 2, \(n_2=14\), \(\bar{x}_2=5.9\) hours, and \(s_2=1.0\) hour. Estimate the difference in population mean completion time, system 1 minus system 2, with a 95% confidence interval.
Parameter and procedure. The outcome is quantitative, and each sampled adult is in only one system group, so the samples are independent. The target is \(\mu_1-\mu_2\), in hours. Use a two-sample Welch t interval. This method does not require the population variances to be equal.
Conditions. The scenario specifies independent random samples, and the groups are separate. The sample sizes are modest, so the interval also relies on each group's outcome distribution being reasonably free of severe skewness or extreme outliers. In an actual analysis, inspect the group data or appropriate plots for that condition; no raw observations are supplied here to verify it. For this hypothetical example, assume those checks reveal no severe departures.
The estimated difference is \(6.8-5.9=0.9\) hour. The estimated standard error is
The Welch degrees of freedom are approximately \(27.9\), giving a 95% critical value of about \(t^*=2.048\). Therefore,
Report. The estimated mean completion time for system 1 is 0.9 hour longer than for system 2. Under the stated sampling and distribution conditions, the 95% confidence interval for the population mean difference, system 1 minus system 2, is about 0.077 to 1.72 hours. This interval estimates a population difference; it does not establish that the reminder system caused the difference.
Worked Example: Testing a Mean Paired Difference
In a hypothetical pilot, ten volunteers record a symptom score before and after a program. Define each difference as after minus before, so a negative difference means a lower score after the program. The ten recorded differences are \(-2,-1,-3,0,-2,-1,-4,-2,-1,-2\) score points. Test at \(\alpha=0.05\) whether the population mean difference is less than zero.
Parameter and hypotheses. Each volunteer contributes a pair, so the target is \(\mu_d\), not the difference between two independent population means. The question specifies a decrease under the chosen subtraction order:
Conditions. The volunteers are distinct, so the differences should be independent across volunteers; the scenario must support that assumption. Because \(n=10\) is small, a paired t test requires the distribution of differences to be approximately normal, with no severe outliers. The listed differences range from \(-4\) to \(0\) and show no isolated extreme value, though in practice the full distribution should be inspected. Assume the sampling or recruitment process makes the volunteers reasonably representative of the population to which the conclusion is intended to apply.
The sum of the differences is \(-18\), so \(\bar{d}=-18/10=-1.8\) points. The sum of squared deviations from \(-1.8\) is \(11.6\), giving \(s_d=\sqrt{11.6/9}\approx1.1353\) points. The test statistic is
The lower-tail p-value from a t distribution with 9 degrees of freedom is less than 0.001. Since this is below \(\alpha=0.05\), reject \(H_0\).
Report. These hypothetical data provide strong evidence that the population mean after-minus-before symptom-score difference is below zero. In this setting, the evidence supports a lower mean score after the program among the population represented by the volunteers. A before-and-after comparison alone does not rule out other explanations for the change, so it should not automatically be described as proof that the program caused it.
Worked Example: Comparing Two Independent Proportions
In a hypothetical survey, 42 of 120 respondents in group 1 and 54 of 130 respondents in group 2 answer “yes” to a binary question. Test whether the population proportions differ, using a two-sided test at \(\alpha=0.05\). Assume the samples were independently selected and no respondent appears in both groups.
Parameter and hypotheses. The outcome is binary and the groups are independent. Let \(p_1\) and \(p_2\) be the population proportions answering yes. The target is \(p_1-p_2\), with
Conditions. The samples are assumed independent and respondents' answers should be independent within each sample. Under the null model of equal proportions, the pooled estimate is \(\hat{p}=(42+54)/(120+130)=96/250=0.384\). The expected success and failure counts are \(120(0.384)=46.08\), \(120(0.616)=73.92\), \(130(0.384)=49.92\), and \(130(0.616)=80.08\); all are comfortably above 10.
The sample proportions are \(\hat{p}_1=42/120=0.3500\) and \(\hat{p}_2=54/130\approx0.4154\). Under \(H_0\), the pooled standard error and test statistic are
The two-sided standard normal p-value is approximately \(0.288\). Because \(0.288>0.05\), do not reject \(H_0\).
Report. The observed sample proportion in group 1 is about 0.065 lower than in group 2, or 6.5 percentage points lower. The test does not provide convincing evidence at the 5% significance level that the population proportions answering yes differ. This does not show that the proportions are equal; it means the data do not establish a difference with this test and significance level.
Writing the Final Report
A complete report makes the reasoning visible without turning the conclusion into a list of calculations. For an estimate, include the parameter, point estimate, interval, units, and group order. For a test, include the hypotheses or the tested difference, the relevant statistic and p-value, and a conclusion in context. The wording should reflect the study design: random assignment can support causal conclusions under suitable conditions, while random sampling supports generalization to the sampled population. Neither feature should be assumed when it was not part of the design.
Keep the reference group consistent throughout. If the parameter is \(\mu_1-\mu_2\), a positive estimate means group 1 has a higher population mean than group 2; if the difference is reversed, the sign reverses. Likewise, for paired data, the meaning of a positive or negative difference depends on how \(d\) was defined. State that definition before interpreting the sign.
Common Mistakes
- Choosing a test before identifying the parameter. The correction is to state the population quantity first: a difference in means, a difference in proportions, or a mean paired difference.
- Treating paired observations as independent groups. If the same person or matched unit contributes both measurements, calculate one difference per pair and analyze those differences.
- Using a two-proportion method for a nonbinary outcome. A two-proportion procedure requires a yes/no outcome. For a quantitative outcome, the target is usually a difference in means instead.
- Reversing the group order in the conclusion. Write the subtraction order in the parameter and carry it unchanged into the estimate and interpretation.
- Calling a nonsignificant result proof of no difference. A large p-value is not evidence that the null is true. Report that the analysis did not find convincing evidence of a difference, and include an interval when estimation is the goal.
- Ignoring conditions because the calculation is available. A computed result is not automatically trustworthy. Check the design and the applicable distribution or count conditions before interpreting it.
Check Your Understanding
For each question, explain the decisions in the workflow rather than naming a procedure alone.
- A clinic compares average waiting times for patients at two locations using separate random samples. State the outcome type, group structure, target parameter, and a suitable procedure.
- The same 18 people have a blood-pressure measurement before and after a change in routine. Define a useful paired difference and state the parameter to analyze.
- A survey compares the proportion of people in two independent groups who report using a particular service. What parameter is targeted, and what null value is natural for a test of no difference?
- Why does a paired analysis use the standard deviation of within-pair differences rather than separate standard deviations for the two measurement occasions?
- A test gives a p-value of 0.18 at \(\alpha=0.05\). Write a conclusion that avoids claiming the population quantities are equal.