Diagnose Common Procedure-Selection Errors

Why Procedure-Selection Errors Matter

A statistical procedure is useful only when it matches the data and the question. A common mistake is to choose a method from a superficial feature of a study—such as the presence of two columns, a large total sample size, or numbers in the data—without asking what was measured and how the observations were collected. The result may be a precisely calculated answer to the wrong question.

For a comparison of two groups, check three things first: whether observations are paired or independent, whether the outcome is quantitative or categorical, and how many groups are actually being compared. These distinctions determine whether a mean-based or proportion-based method is suitable and what counts as a sample.

Key distinction: A group is a set of individuals defined by the study design or a category of interest. An observation is a recorded value on an individual. The sample size for a group is the number of independent individuals in that group—not the number of variables, measurements, or rows created by reshaping the data.

Error 1: Treating Paired Measurements as Independent

Measurements are paired when each value in one condition is naturally linked to a value in the other condition. The most familiar example is measuring the same person before and after an intervention. Other pairings arise when individuals are deliberately matched, such as matching two people on important characteristics. In a paired design, the analysis uses the within-pair differences. It does not treat the two measurements from a pair as though they came from unrelated individuals.

Let \(d_i\) be the difference for pair \(i\), defined in a consistent direction, such as after minus before. A paired procedure for a quantitative outcome targets the population mean difference, \(\mu_d\). Its one-sample t statistic has the form

t=\frac{\bar{d}-\mu_{d,0}}{s_d/\sqrt{n}},

where \(\bar{d}\) and \(s_d\) are the sample mean and sample standard deviation of the \(n\) differences, and \(\mu_{d,0}\) is the mean difference under the null hypothesis. This procedure requires independent pairs; within a pair, the two measurements are expected to be related. The distribution of differences must also be suitable for the t procedure, particularly when the sample is small.

Worked Example: Before-and-After Measurements

In a hypothetical exercise study, eight participants have their resting heart rate recorded before and after a program. The outcome is quantitative, measured in beats per minute. The differences, calculated as after minus before, are \(-1,-1,1,-2,-1,-1,1,-2\). Assume the participants were sampled independently and that the paired differences have no severe departures from the t model.

Tempting error: Treat the eight before measurements and eight after measurements as two independent samples. That would ignore that each person's two measurements belong together. A participant with a naturally high heart rate may have high values in both conditions; the within-person comparison accounts for this relationship.

Correct choice and calculation: Analyze the eight differences as one sample. Their sum is \(-6\), so \(\bar{d}=-6/8=-0.75\) beats per minute. The sum of squared deviations from \(\bar{d}\) is \(9.5\), giving

s_d=\sqrt{\frac{9.5}{8-1}}=1.165,\qquad SE=\frac{s_d}{\sqrt{8}}=\frac{1.165}{\sqrt{8}}=0.412.

To test \(H_0:\mu_d=0\), the statistic is

t=\frac{-0.75-0}{0.412}=-1.82,\qquad df=8-1=7.

The negative estimate means the sample's after-program heart rate is lower on average by 0.75 beats per minute. The statistic alone does not establish that the program changes the population mean; a test conclusion would also use the p-value and the chosen significance level. The relevant sample size is eight pairs, not sixteen independent participants.

Pairing is a feature of the design, not a formatting choice. Putting before and after values into two columns does not make them independent. Conversely, two groups are not paired merely because their measurements appear side by side in a table. If each person belongs to only one group and the study did not match individuals across groups, the groups are generally treated as independent, subject to the design and sampling assumptions.

Error 2: Using a Mean Procedure for a Categorical Outcome

A mean procedure is designed for a quantitative outcome: a measurement or count for which differences and averages have meaningful interpretations. If the outcome records a category—such as recovered/not recovered, test positive/test negative, or selected/not selected—the target is typically a proportion, not a mean of the category labels. Coding categories as 0 and 1 does not change the underlying question: the mean of a 0/1 variable is the proportion coded 1.

For two independent groups with a binary outcome, the sample proportions are \(\hat{p}_1=x_1/n_1\) and \(\hat{p}_2=x_2/n_2\), where \(x_j\) is the number of successes and \(n_j\) is the sample size in group \(j\). A comparison concerns the population proportions \(p_1\) and \(p_2\), often through their difference \(p_1-p_2\). Independence between groups and the relevant conditions for a proportion procedure must be checked before using its standard test or interval.

Worked Example: Comparing Two Recovery Proportions

A hypothetical trial assigns 100 participants to each of two independent treatment groups. Recovery by a specified date is recorded as yes or no. There are 40 recoveries in Group 1 and 28 in Group 2. Suppose assignment made the groups independent and the trial's design supports treating outcomes from different participants as independent.

Tempting error: Apply a two-sample t procedure by coding recovery as 1 and no recovery as 0, then describing the result as a comparison of means. Although the mean of each 0/1 column equals its recovery proportion, the question and procedure are about two categorical outcome proportions. Use a two-proportion procedure rather than a mean procedure.

The observed proportions and difference are

\hat{p}_1=\frac{40}{100}=0.40,\qquad \hat{p}_2=\frac{28}{100}=0.28,\qquad \hat{p}_1-\hat{p}_2=0.12.

Thus the observed recovery proportion is 12 percentage points higher in Group 1. For a two-sided test of \(H_0:p_1=p_2\), the pooled estimate under the null is \(\hat{p}= (40+28)/(100+100)=0.34\). The null-model standard error and test statistic are

SE=\sqrt{0.34(1-0.34)\left(\frac{1}{100}+\frac{1}{100}\right)} =\sqrt{0.004488}=0.0670, \qquad z=\frac{0.40-0.28}{0.0670}=1.79.

Under the null model, the expected numbers of recoveries are 34 in each group, and the expected numbers of non-recoveries are 66 in each group, so the usual large-count condition is satisfied. With a two-sided alternative, \(z=1.79\) gives a p-value of about 0.073. At significance level 0.05, this result is not statistically significant. It does not show that the population recovery proportions are equal; it means the data do not provide sufficient evidence against equality at that level.

Error 3: Confusing the Number of Groups With the Number of Observations

A two-group comparison has two groups, even if it contains dozens or thousands of observations. The total number of observations is usually \(n_1+n_2\), while each group has its own sample size. A large total does not turn a two-group design into a many-group design, and a table with many rows does not imply many groups.

The reverse mistake is also common: treating repeated measurements as if they increased the number of independent individuals. If 30 people are each measured twice, there are 30 paired individuals, not 60 independent people. For a paired analysis, there are 30 differences. For an independent two-group analysis, the sample sizes refer to the number of independent individuals in each group.

Worked Example: Two Groups, Forty-Eight Individuals

In a hypothetical study, 24 people receive Program A and a different 24 people receive Program B. Each person's improvement in a quantitative score is recorded once. The sample means are 5.2 and 2.1 points, and the sample standard deviations are 6 and 8 points, respectively. Assume the groups were formed independently, observations within each group are independent, and the data meet the conditions for comparing means.

Tempting error: Call this a 48-group study because there are 48 rows, or use \(n=48\) as the sample size for each group. There are two groups and 48 individuals in total; the group sample sizes are \(n_1=24\) and \(n_2=24\).

For a Welch two-sample t procedure, the estimated difference in means, Program A minus Program B, is \(5.2-2.1=3.1\) points. Its standard error is

SE=\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}} =\sqrt{\frac{6^2}{24}+\frac{8^2}{24}} =\sqrt{\frac{100}{24}}=2.041\text{ points}.

The test statistic for \(H_0:\mu_1-\mu_2=0\) is \(t=3.1/2.041=1.52\). The calculation uses the two separate group sample sizes, not 48 in place of either sample size. The estimate is in points, and the statistic compares that estimated difference with its standard error. Whether the evidence is statistically significant requires the appropriate degrees of freedom and p-value; the statistic by itself is not a conclusion.

Always identify the number of groups from the question and design before choosing a procedure. A study with three treatment arms is not a two-group study simply because each arm contains many people. A two-sample method compares two population means or two population proportions; it does not, by itself, answer a question about all pairwise differences among three or more groups. Do not create extra groups by counting measurements, and do not conceal additional groups by combining them without a scientific reason.

A Short Justification: Why Paired Data Use Differences

The paired calculation follows directly from the quantity the question asks about. For pair \(i\), let \(Y_{i,\mathrm{after}}\) and \(Y_{i,\mathrm{before}}\) be the two measurements and define \(d_i=Y_{i,\mathrm{after}}-Y_{i,\mathrm{before}}\). The average change among the \(n\) pairs is

\bar{d} =\frac{1}{n}\sum_{i=1}^{n} \left(Y_{i,\mathrm{after}}-Y_{i,\mathrm{before}}\right) =\bar{Y}_{\mathrm{after}}-\bar{Y}_{\mathrm{before}}.

The equality shows why the paired design estimates a mean change. But the standard error must also reflect how the two measurements move together. For each pair,

\operatorname{Var}(Y_{\mathrm{after}}-Y_{\mathrm{before}}) =\operatorname{Var}(Y_{\mathrm{after}}) +\operatorname{Var}(Y_{\mathrm{before}}) -2\operatorname{Cov}(Y_{\mathrm{after}},Y_{\mathrm{before}}).

If the paired measurements are positively associated, the covariance is positive and the variability of the differences can be smaller than it would be under an independence assumption. Treating paired measurements as independent discards this design information and generally uses the wrong standard error. The paired procedure works on the observed differences directly, so the relevant distributional condition concerns those differences.

A Reliable Diagnostic Check

Before calculating a test statistic or interval, pause and describe the study in one sentence: what is one individual, what outcome is recorded, and how are the groups or measurements related? Then classify the outcome and write down the population quantity the question asks about. This simple check catches many mismatches before arithmetic begins.

Identify the outcome.
A meaningful numerical measurement points toward a mean; a category such as yes/no points toward a proportion. Do not let numeric codes obscure a categorical outcome.

Check the design.
Ask whether the same individual or a matched partner contributes to both conditions. If so, identify the pairs and analyze differences for a quantitative outcome.

Count groups and individuals separately.
Record the number of groups and the sample size in each group. Repeated measurements do not create additional independent individuals.

Check assumptions and conditions.
Verify the independence and distribution or count conditions needed by the selected procedure. If they are not plausible, do not proceed as if they were satisfied.

Match the conclusion to the target.
State the comparison in context and with the correct units: a mean difference for a quantitative outcome or a proportion difference for a categorical outcome.

Common Mistakes and Their Corrections

“There are two columns, so these must be independent groups.” Two columns may hold paired measurements from the same individuals. Check the design, not the layout.
“The outcome is yes/no, but I can run a t procedure because I coded yes as 1.” The question is about proportions for a categorical outcome. Use a proportion method and check its conditions.
“There are 60 measurements, so the sample size is 60.” Repeated measurements on the same participants are not independent participants. Count the independent units and preserve the pairing.
“There are two treatments and many participants, so I have many groups.” The number of groups is two; the number of participants is the total sample size, split across those groups.
“The means differ, so the two-group t procedure must be right.” A difference in sample means does not establish that the outcome is quantitative in the relevant sense, that groups are independent, or that the procedure's conditions hold. Check all three before using it.

Key takeaway: Choose a procedure from the study design, outcome type, and target quantity—not from the number of columns or the appearance of the data table. Paired quantitative data call for analysis of differences; independent quantitative groups call for a comparison of means; independent groups with categorical outcomes call for a comparison of proportions.

Check Your Understanding

For each situation, identify the main procedure-selection issue and explain the correction.

A clinic measures each patient's pain score before and after treatment. Why is treating the two sets of scores as independent samples a mistake?
A survey compares the proportions of two groups who report having received a vaccine. Why is a mean procedure not the natural choice?
One study arm has 18 participants and another has 22. How many groups are there, and what are the two group sample sizes?
Forty participants each provide a measurement at two visits. How many independent participants are there, and what feature must be retained in an analysis of change?
A study compares three treatment groups on a quantitative outcome. Why does having a large total sample not make it a two-group comparison?