Start With the Question and the Sample Design
A campus survey can ask whether students support a proposal, whether support differs between two groups, how long students study, or whether average study time differs by housing status. These questions may use similar survey data, but they concern different population parameters. Choosing a procedure begins by identifying the outcome and the comparison the question asks for—not by choosing a familiar formula.
The examples below are hypothetical. Each illustrates one of four common targets: one population proportion, a difference between two population proportions, one population mean, or a difference between means in independent groups. A survey procedure supports conclusions about its target population only when the sampling and measurement design justify that inference.
| Question of interest | Parameter | Procedure family |
|---|---|---|
| What proportion of campus undergraduates support a proposal? | One population proportion, \(p\) | One-proportion z procedure |
| Does support differ between two student groups? | Difference in proportions, \(p_1-p_2\) | Two-proportion z procedure |
| What is the average weekly study time for a population? | One population mean, \(\mu\) | One-sample t procedure |
| Does average weekly study time differ between two separate groups? | Difference in means, \(\mu_1-\mu_2\) | Two-sample t procedure |
For each question, also decide whether the goal is estimation or testing. A confidence interval estimates plausible values of a parameter or difference. A significance test evaluates a specified null hypothesis against an alternative. The parameter and design determine the procedure family; the research question determines the form of the interval or the alternative hypothesis.
One Population Proportion
This family fits questions such as “What proportion of campus undergraduates support extended library hours?” There is one population and one binary outcome, such as support or no support. It does not fit a question about average hours of library use, which has a quantitative outcome.
Before using a z procedure, check that the survey is based on a suitable random sample or other defensible sampling design, that observations can be treated as independent, and that sampling without replacement involves no more than a small fraction of the population (often checked with the 10% condition). For a test, check that the expected numbers of successes and failures under \(H_0\) are sufficiently large; a common rule is at least 10 of each. For an interval, check the observed numbers of successes and failures. These conditions support the normal approximation used by the procedure.
Worked Example: Support for a Campus Shuttle
Suppose a hypothetical random sample of 240 campus undergraduates includes 147 who support extending late-night shuttle service. The question is whether more than half of all campus undergraduates support the change. Choose the procedure and carry out the test at \(\alpha=0.05\).
The response is binary, there is one population, and the question concerns a single proportion. The suitable procedure is a one-proportion z test. Let \(p\) be the proportion of all campus undergraduates who support the extension. The hypotheses are \(H_0:p=0.50\) and \(H_a:p>0.50\), because “more than half” gives a greater-than alternative.
Assume the sample was selected randomly and the campus undergraduate population is at least 2,400, so \(240\) is no more than 10% of it. The independence and sampling-fraction checks are then reasonable. Under the null, the expected counts are \(240(0.50)=120\) supporters and \(240(0.50)=120\) non-supporters, so the large-count condition is satisfied.
The sample proportion and test statistic are
The one-sided p-value is \(P(Z\ge 3.486)\approx 0.00025\), using the standard normal distribution under the null model. Since this is below \(0.05\), the result is evidence against \(H_0\) in the direction that more than half support the extension. The result does not establish that every campus subgroup has majority support.
Two Subgroup Proportions
Use this family when the response is categorical with two outcomes and the question compares percentages across two groups—for example, the proportion supporting a proposal among undergraduates versus the proportion among graduate students. The groups must be distinct and their observations independent for the standard independent-samples procedure. Check random sampling or an appropriate design in each group, the sampling fraction when sampling without replacement, and adequate counts. For a test of equal proportions, the pooled proportion is used to check and calculate the null standard error. For an interval, the two sample proportions are used separately.
Worked Example: Comparing Shuttle Support by Student Group
In a hypothetical survey, 72 of 120 sampled undergraduates and 48 of 100 sampled graduate students support the shuttle extension. The question is whether the support proportions differ between the two campus populations. Assume independent random samples were taken from the two groups, each sample is at most 10% of its population, and individual responses are independent.
The outcome is binary and the comparison is between two group proportions, so use a two-proportion procedure. Set group 1 to undergraduates and group 2 to graduate students. The sample proportions are
For a two-sided test, \(H_0:p_1-p_2=0\) and \(H_a:p_1-p_2\ne 0\). Under the null, the pooled estimate is \(\hat{p}_{\text{pool}}=(72+48)/(120+100)=120/220\approx0.5455\). The expected success and failure counts are all above 10: for undergraduates they are about 65.45 and 54.55, and for graduate students they are about 54.55 and 45.45. Thus, the large-count check for the test is satisfied.
The pooled standard error and test statistic are
The two-sided p-value is approximately \(0.075\), so at the 0.05 level the sample does not provide statistically significant evidence of a difference. This is not proof that the population proportions are equal.
For a 95% confidence interval, use the unpooled standard error:
The interval estimates the undergraduate support proportion minus the graduate support proportion. It is consistent with anything from a very small negative difference to an undergraduate advantage of about 25.1 percentage points; zero is among the plausible values.
One Population Mean
This procedure fits a question about one population average, such as the mean number of hours per week campus undergraduates study. It is not a proportion procedure: weekly study time is quantitative, and a mean—not a percentage—is the target. Check that observations arise from an appropriate sampling design and can reasonably be treated as independent. The t procedure also requires a population distribution that is approximately normal, or a sample large enough for the sampling distribution of \(\bar{x}\) to be reasonably well behaved; strong skewness or influential outliers can make a small-sample t procedure unsuitable.
Worked Example: Estimating Average Weekly Study Time
A hypothetical random sample of 36 campus undergraduates has a mean reported study time of 14.2 hours per week and sample standard deviation 5.4 hours. Assume the campus population is at least 360 undergraduates and inspection of the sample reveals no severe skewness or extreme outliers. Estimate the population mean with a 95% confidence interval.
The outcome is quantitative and there is one population mean, so use a one-sample t interval. The random sample and 10% sampling-fraction check support independence; the stated shape check makes the t procedure reasonable. The estimated standard error is
With \(df=36-1=35\), the 95% critical value is approximately \(t^*=2.030\). Therefore,
The interval estimates the mean weekly study time of campus undergraduates at about 12.37 to 16.03 hours. In repeated sampling under the stated conditions, 95% of intervals formed by this method would cover the population mean; this particular interval does not assign a probability to the fixed mean.
Means in Independent Groups
When the outcome is quantitative and the question compares averages in two separate groups, choose a two-sample t procedure. The parameter is \(\mu_1-\mu_2\), with group order stated explicitly so the sign of the difference has a clear interpretation. Check the design for independent observations within each group and independent groups from one another. Check the distribution condition in each group: the data should be reasonably well behaved, particularly for small samples.
The standard error for a difference in independent sample means follows from independence. If \(X_1\) and \(X_2\) denote the sample means, then \(\operatorname{Var}(X_1-X_2)=\operatorname{Var}(X_1)+\operatorname{Var}(X_2)\), because the covariance is zero. For independent random samples, \(\operatorname{Var}(X_i)=\sigma_i^2/n_i\). Replacing the unknown population variances with sample variances gives the estimated standard error:
The Welch two-sample t procedure uses this standard error and does not require equal population variances. A pooled procedure instead relies on a common-variance assumption. When that assumption is not justified, Welch’s procedure is the appropriate choice. This is an independent-groups setting, not a setting in which the same individuals contribute both measurements.
Worked Example: Comparing Study Time by Housing Status
A hypothetical campus survey compares weekly study hours for students living in residence halls and students living off campus. The independent samples have these summaries: residence-hall students, \(n_1=35\), \(\bar{x}_1=6.8\), \(s_1=3.6\); off-campus students, \(n_2=29\), \(\bar{x}_2=5.2\), \(s_2=4.2\). Assume the groups were sampled independently, each sample is less than 10% of its population, observations are independent within groups, and neither group has severe outliers or extreme skewness.
The outcome is quantitative and the groups consist of different students, so this calls for a two-sample t procedure. Define the difference as residence-hall mean minus off-campus mean. Since the sample standard deviations differ and equal population variances have not been justified, choose Welch’s procedure rather than a pooled t procedure. For a two-sided test of equal means, \(H_0:\mu_1-\mu_2=0\) and \(H_a:\mu_1-\mu_2\ne0\).
The observed difference and Welch standard error are
Welch’s approximate degrees of freedom are
Thus, \(t=1.6/0.9892\approx1.617\), and the two-sided p-value is approximately \(0.1115\). At the 0.05 level, this is not statistically significant evidence that the population mean weekly study hours differ by housing status. The positive sample difference is 1.6 hours, but it should not be described as a proven population difference or as an effect of housing.
Why the Design Matters
The decision tree is compact: identify whether the outcome is binary or quantitative, then determine whether the question concerns one population or a difference between two groups. But the design matters just as much as the variable type. A convenience sample of students who happen to pass a table may not represent all campus students, even if the sample is large. Likewise, two group averages require genuinely independent groups for the independent-samples method. If observations are naturally linked or come from repeated measurements on the same people, do not treat them as independent groups.
Keep the parameter’s units and direction visible in the final report. A proportion difference is expressed in proportion units or percentage points; a mean difference is expressed in the quantitative variable’s units. For a difference, state group 1 minus group 2 before interpreting a positive or negative estimate. In all cases, inference is about the defined population and depends on the survey design, not simply on the arithmetic.
Common Mistake
A common mistake is to choose a two-proportion test merely because a survey compares two groups. For example, comparing the average number of study hours among residence-hall and off-campus students is not a comparison of proportions. The outcome is quantitative, so the parameter is a difference in means and an independent-groups t procedure is the relevant family. Conversely, comparing the percentages in each group who support a policy calls for a two-proportion procedure. Always identify the outcome and parameter before choosing the procedure.
Check Your Understanding
For each question, name the parameter and the procedure family, then note one design or condition check that matters.
- A random sample of campus students is asked whether they favor a new dining plan. What procedure would estimate the proportion in favor?
- A survey compares the percentages of first-year students and seniors who use the campus gym weekly. What is the parameter, and which procedure family fits?
- A sample of 28 students reports weekly commuting time in minutes. The question asks for the mean commuting time among all campus students. What procedure family is appropriate, and what sample feature could make it questionable?
- A study compares average weekly work hours for students living on campus and off campus, with different students in each group. What is the parameter, and why is the independent-groups procedure relevant?
- In a two-group survey, the outcome is “supports the proposal: yes or no,” but the researcher reports an average of the yes/no codes. What is the mistake, and what quantity should be compared instead?