Checking Random Sampling and Independence

Why the Sampling Design Matters

A one-proportion confidence interval uses sample data to estimate a population proportion. Its reliability depends not only on the number of observed successes and failures, but also on how the observations were selected. A large sample can still give a misleading picture of a population if people enter the sample through a biased process. And even a well-chosen sample can require a different calculation if its observations are strongly dependent.

This tutorial focuses on two design checks for the usual one-proportion z interval: whether the sample was selected randomly from the population of interest, and whether the observations can reasonably be treated as independent. The success-failure condition is a separate check. Satisfying the design checks does not replace it.

Definition: A random sample is obtained using a chance-based selection procedure from a specified population. For a simple random sample of size \(n\), every possible group of \(n\) population members has the same chance of selection. Independent observations are observations for which learning the value of one does not change the probability distribution of another. These conditions support the usual one-proportion interval’s sampling model; they do not guarantee that a sample has no measurement error or nonresponse bias.

Random Sampling and Representativeness

The target population is the full group about which the estimate is intended to provide information. Random selection helps connect the observed sample to that population: in repeated samples selected by the same chance procedure, different population members have opportunities to be included. It does not mean that every sample will mirror the population perfectly. Random samples vary, and an unusual sample can occur by chance.

A random sample is different from a random assignment. Random sampling concerns how units are selected from a population and supports generalizing estimates to that population. Random assignment concerns how units are allocated to conditions in an experiment. For a one-proportion confidence interval, the relevant question is whether the observed units were selected in a way that supports inference to the stated population.

A voluntary online poll, for example, is not a random sample merely because many people respond. People decide whether to participate, and those who choose to respond may differ from those who do not. Similarly, a sample drawn randomly from a list that excludes part of the target population may fail to represent the full target. Random selection is a property of the design, not a label earned by a large sample size.

Worked Example: A Random Sample From a Defined Population

A clinic has a complete list of 2,400 eligible patients. A computer selects 180 distinct patient IDs at random from that list, and staff record whether each selected patient received a particular screening during the past year. The goal is to estimate the proportion of all 2,400 eligible patients who received the screening.

Random sampling. The computer selects IDs at random from a complete list of the target population. If the selection gives each group of 180 patients the same chance of being selected, this is a simple random sample from the stated population.

Independence check. The 180 IDs are selected without replacement, so the selected patients are not literally independent draws. Check the sampling fraction:

\frac{n}{N}=\frac{180}{2400}=0.075=7.5\%.

Since \(7.5\%\) is no more than \(10\%\), the ten-percent condition is met. The usual procedure treats the observations as sufficiently independent for this purpose. The design checks therefore support using the usual one-proportion interval, provided the separately checked success-failure condition also holds and the recorded outcomes are accurate.

Independence and the Ten-Percent Condition

When a sample is drawn without replacement from a finite population, selecting one unit changes which units remain available. Thus, successive observations are not strictly independent. For a simple random sample, however, this dependence is usually small when the sample is a small fraction of the population. The ten-percent condition is a practical rule for deciding when it is reasonable to use the standard interval’s independence approximation.

Ten-percent condition: When a simple random sample of size \(n\) is drawn without replacement from a finite population of size \(N\), the usual one-proportion interval treats observations as approximately independent if \(n \leq 0.10N\). Equivalently, the sample is no more than \(10\%\) of the population. This condition is not needed to handle dependence from sampling without replacement when sampling is genuinely with replacement, but the other design assumptions still need to be considered.

The condition is about the number sampled relative to the population, not the number of successes, the confidence level, or the size of the sample by itself. A sample of 100 may be a small fraction of a population of 100,000 but a large fraction of a population of 500. When the population size is known, calculate \(n/N\) directly. When it is not known, use a credible population count or sampling frame if available; do not assume the condition holds simply because the sample seems small.

Why the Ten-Percent Rule Is Reasonable

A short variance calculation shows why sampling without replacement causes less variability than independent sampling, and why the difference is modest for a small sampling fraction. Consider a fixed population of \(N\) units, each with a binary value \(Y\) equal to 1 for a success and 0 for a failure. Let \(p_N\) be the population proportion of successes, and draw a simple random sample without replacement. Write \(Y_j\) for the binary value on the \(j\)th draw.

Each draw has variance \(p_N(1-p_N)\). Two distinct draws have covariance \(-p_N(1-p_N)/(N-1)\): after one unit is selected, the remaining pool is slightly depleted of that unit’s value. Since the sample proportion is the average of the \(n\) draw values, adding the \(n\) variances and the \(n(n-1)\) covariance terms gives:

\begin{aligned} \operatorname{Var}(\hat{p}) &=\operatorname{Var}\!\left(\frac{1}{n}\sum_{j=1}^{n}Y_j\right)\\ &=\frac{1}{n^2}\left[n p_N(1-p_N) +n(n-1)\left(-\frac{p_N(1-p_N)}{N-1}\right)\right]\\ &=\frac{p_N(1-p_N)}{n}\left(\frac{N-n}{N-1}\right). \end{aligned}

The factor \((N-n)/(N-1)\) reduces the variance compared with the independent-draw expression \(p_N(1-p_N)/n\). For example, if \(n/N\) is about \(0.10\) and \(N\) is large, the factor is about \(0.90\), so the standard deviation is multiplied by about \(\sqrt{0.90}=0.949\). The usual interval ignores this modest reduction. The ten-percent condition is therefore an approximation rule: the observations are still dependent, but the effect on the standard error is limited under this sampling design.

Applying the Checks Before an Interval

The usual one-proportion z interval has the form:

\hat{p}\ \pm\ z^*\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}.

Here, \(z^*\) is the critical value for the chosen confidence level. The formula relies on a sampling model in which the sample proportion is centered on the population proportion and its variability is adequately represented by the estimated standard error. Random selection and reasonable independence support that model; the success-failure condition supports the normal approximation used by the z interval. These are distinct checks, and passing one does not establish the others.

Before using the interval, identify the target population and sampling method. If the sample is a simple random sample without replacement, record \(n\) and \(N\), then verify \(n \leq 0.10N\). If the sample is drawn with replacement, or observations arise from a genuinely independent sampling process, the ten-percent condition is not the relevant check for dependence from depletion. For other designs, such as selecting groups or repeatedly measuring the same individuals, do not assume the ordinary interval is justified merely because a fraction calculation passes.

Worked Example: A Sample That Is Too Large a Fraction

A small professional association has 1,500 members. An analyst selects a simple random sample of 240 distinct members without replacement and asks whether each member attended the annual meeting. The goal is to estimate the proportion of all association members who attended.

The sample is randomly selected from the stated population, so the random-sampling check is met. But the sampling fraction is:

\frac{n}{N}=\frac{240}{1500}=0.16=16\%.

Because \(16\%>10\%\), the ten-percent condition is not met. The observations are dependent enough that the usual interval’s independence approximation should not be used without adjustment. Random selection alone does not make observations independent, and a large sample count does not remove this issue. The analyst should use an inference method that accounts for sampling without replacement or obtain a sampling design with a smaller sampling fraction. This conclusion concerns the design condition, regardless of whether the success-failure condition passes.

Worked Example: Why an Opt-In Poll Is Not a Random Sample

A university has 30,000 students. It posts a poll asking whether students support a proposed change to the academic calendar. A link is sent to all students, and 900 choose to submit a response. The university wants to estimate the proportion of all students who support the change.

The number of responses is not enough to establish random sampling. Each student decides whether to respond, so inclusion is based on self-selection rather than a chance-based selection procedure. Students with stronger opinions may be more likely to submit a response; the respondents could therefore differ systematically from nonrespondents.

The arithmetic \(900/30{,}000=0.03=3\%\) does not repair this problem. The ten-percent condition addresses dependence caused by sampling without replacement in a random sample; it does not correct selection bias. As described, the usual one-proportion confidence interval is not justified for estimating the opinion of all 30,000 students. A genuinely random sample, along with attention to nonresponse and measurement, would provide a stronger basis for that estimate.

Other Dependence Problems and Common Mistakes

The ten-percent check is specifically useful for a simple random sample drawn without replacement from a finite population. It is not a universal test for independence. If researchers sample several patients from each of a few clinics, patients within a clinic may have more similar outcomes than patients from different clinics. If the same person is measured multiple times, those measurements are related. In either case, a small overall sample fraction does not remove the dependence created by the design. The usual one-proportion interval may need a method that accounts for clustering or repeated observations.

A related issue is nonresponse. A study may begin with a random sample but fail to obtain outcomes for many selected units. If response is associated with the binary outcome, the observed respondents may no longer represent the intended population. Random selection is important, but it does not automatically eliminate bias from missing responses, incomplete coverage, or inaccurate measurement.

Mistake: “The sample is random, so the observations must be independent.” A simple random sample without replacement creates dependence. Check the sampling fraction against the ten-percent condition.
Mistake: “The sample is less than 10% of the population, so the poll is representative.” The ten-percent condition addresses dependence, not self-selection or coverage bias. Assess how units entered the sample.
Mistake: “The sample size is large, so independence is not a concern.” Compare the sample size with the population size when sampling without replacement. A large sample can be an even larger fraction of a small population.
Mistake: “Passing the random-sampling check proves the whole interval is appropriate.” Random sampling and independence are only part of the conditions. Check the success-failure condition separately and consider how the data were measured.

Key takeaway: For a one-proportion interval, confirm that the sample design supports inference to the target population. For a simple random sample without replacement, check \(n \leq 0.10N\) to justify treating observations as approximately independent. This rule does not fix biased selection or other forms of dependence.

Check Your Understanding

For each situation, identify the design issue and explain whether the usual random-sampling and independence checks are supported.

A town randomly selects 250 of its 8,000 registered voters without replacement. Calculate the sampling fraction and assess the ten-percent condition.
A website allows any visitor to answer a question about a new product. Why does a large number of responses not establish random sampling?
A simple random sample of 500 people is drawn without replacement from a population of 4,000. Does the ten-percent condition hold? Show the calculation.
Researchers select a small percentage of residents, but take several residents from each selected household. Why might the ten-percent condition alone be insufficient?
In one or two sentences, distinguish what random sampling supports from what the ten-percent condition checks.