Deriving the Sampling Distribution of a Difference

Why Study the Difference’s Sampling Distribution?

A sample proportion varies from sample to sample. When two groups are sampled independently, their sample proportions both vary, so their difference varies as well. Understanding that variability is essential: it tells us how much a difference might fluctuate even when the population proportions stay fixed.

Here the focus is the probability model for the difference, not a confidence interval or hypothesis test. We will derive its center and variance under the independent Bernoulli model, then explain when a normal curve can approximate its sampling distribution.

The Model and the Quantity of Interest

Suppose group 1 has \(n_1\) independent observations, each with probability \(p_1\) of the outcome of interest. Group 2 has \(n_2\) independent observations, each with probability \(p_2\). Assume the observations in both groups are Bernoulli trials: each records either the outcome of interest (coded 1) or its absence (coded 0). Within each group, the trials share the same probability, and the two groups are independent of one another.

Let \(X_1\) and \(X_2\) be the numbers of outcomes of interest in the two samples. Their sample proportions are \(\hat{p}_1=X_1/n_1\) and \(\hat{p}_2=X_2/n_2\). The population quantity of interest is \(p_1-p_2\), and the statistic used to estimate it is \(\hat{p}_1-\hat{p}_2\). The order matters: reversing the groups reverses the sign of both differences.

Definition: Under the independent Bernoulli model, the difference between two sample proportions is the random variable \(\hat{p}_1-\hat{p}_2\). Its expected value is \(p_1-p_2\), and its variance is \(p_1(1-p_1)/n_1+p_2(1-p_2)/n_2\). These describe its center and sampling variability across repeated samples under the model.

The assumptions are part of the result, not optional details. If observations within a group are dependent, if the success probability changes across observations in a way the model does not represent, or if the samples are paired rather than independent, the variance formula may not apply as written. For a random sample from a finite population without replacement, sampling dependence can also matter when the sample is a substantial fraction of that population.

Deriving the Mean and Variance

Represent each observation in group 1 by an indicator \(Y_{1j}\), equal to 1 for the outcome and 0 otherwise, for \(j=1,\ldots,n_1\). Define \(Y_{2j}\) similarly for group 2. Under the model, \(P(Y_{ij}=1)=p_i\), so \(E(Y_{ij})=p_i\) and \(\operatorname{Var}(Y_{ij})=p_i(1-p_i)\). Each sample proportion is the average of its indicators.

\hat{p}_1-\hat{p}_2 = \frac{1}{n_1}\sum_{j=1}^{n_1}Y_{1j} - \frac{1}{n_2}\sum_{j=1}^{n_2}Y_{2j}

Expectation is additive, even for dependent random variables. Variance, however, includes a covariance term when random variables are dependent. Here the two samples are independent, so the covariance between their sample proportions is zero. Within each sample, the indicators are also independent, so the variance of their sum is the sum of their variances.

Find the expected value.
Each group 1 indicator has expectation \(p_1\), so \(E(\hat{p}_1)=p_1\); likewise, \(E(\hat{p}_2)=p_2\). Therefore \(E(\hat{p}_1-\hat{p}_2)=p_1-p_2\).

Find each sample proportion’s variance.
Since the independent indicators in group 1 each have variance \(p_1(1-p_1)\), \(\operatorname{Var}(\hat{p}_1)=n_1p_1(1-p_1)/n_1^2=p_1(1-p_1)/n_1\). Similarly, \(\operatorname{Var}(\hat{p}_2)=p_2(1-p_2)/n_2\).

Combine the variances.
Independence between the groups means \(\operatorname{Cov}(\hat{p}_1,\hat{p}_2)=0\). Thus \(\operatorname{Var}(\hat{p}_1-\hat{p}_2)=\operatorname{Var}(\hat{p}_1)+\operatorname{Var}(\hat{p}_2)\).

E(\hat{p}_1-\hat{p}_2)=p_1-p_2

\operatorname{Var}(\hat{p}_1-\hat{p}_2) = \frac{p_1(1-p_1)}{n_1} + \frac{p_2(1-p_2)}{n_2}

This calculation also explains the minus sign’s role. Subtracting the group 2 proportion reverses its direction, but multiplying a random variable by \(-1\) does not change its variance: \(\operatorname{Var}(-X)=\operatorname{Var}(X)\). So both groups contribute a nonnegative amount to the variance of the difference.

The standard deviation of this sampling distribution is its standard error under the model:

SE(\hat{p}_1-\hat{p}_2) = \sqrt{ \frac{p_1(1-p_1)}{n_1} + \frac{p_2(1-p_2)}{n_2} }

The variance and standard error depend on the population proportions, not on a particular pair of observed sample proportions. They describe repeated-sampling variation when \(p_1,p_2,n_1,\) and \(n_2\) are fixed.

Worked Example: Deriving the Center and Spread

Suppose two independent hypothetical samples follow the model, with \(n_1=80\), \(p_1=0.25\), \(n_2=120\), and \(p_2=0.40\). Find the expected difference and its standard error.

The difference is ordered as group 1 minus group 2, so its expected value is

E(\hat{p}_1-\hat{p}_2)=p_1-p_2=0.25-0.40=-0.15

Its variance is

\begin{aligned} \operatorname{Var}(\hat{p}_1-\hat{p}_2) &=\frac{0.25(1-0.25)}{80}+\frac{0.40(1-0.40)}{120}\\ &=\frac{0.1875}{80}+\frac{0.24}{120}\\ &=0.00234375+0.002000=0.00434375. \end{aligned}

Therefore, the standard error is \(\sqrt{0.00434375}\approx0.06591\). Under this model, the difference is centered at \(-0.15\), or 15 percentage points lower for group 1 than group 2, with a standard deviation of about 0.0659 in proportion units. The standard error describes sampling variation; it is not the standard deviation of individual binary outcomes.

When Is the Sampling Distribution Approximately Normal?

The exact sampling distribution is discrete: each sample proportion can take only certain values, so their difference can take only certain values too. A normal distribution is continuous. It can nevertheless provide a useful approximation when both sample proportions are based on enough successes and failures.

To see why, each sample proportion is an average of independent Bernoulli variables. For sufficiently large samples, the central limit theorem makes such averages approximately normal. The difference of two independent approximately normal quantities is also approximately normal. Thus, under the model and suitable sample sizes, the difference is approximately distributed as a normal random variable with mean \(p_1-p_2\) and the variance derived above.

\hat{p}_1-\hat{p}_2 \ \mathrel{\dot\sim}\ N\left( p_1-p_2,\ \frac{p_1(1-p_1)}{n_1}+\frac{p_2(1-p_2)}{n_2} \right)

The notation \(\mathrel{\dot\sim}\) indicates an approximation, not an exact equality. A common practical check is that each group has adequate expected counts of both outcomes: \(n_1p_1\), \(n_1(1-p_1)\), \(n_2p_2\), and \(n_2(1-p_2)\). A threshold such as 10 is often used as a rule of thumb, not a mathematical guarantee. How good the approximation is depends on the sample sizes and how close the probabilities are to 0 or 1. In applications, the population probabilities are usually unknown, so the observed counts are commonly examined as a practical diagnostic.

Worked Example: Checking the Approximation Conditions

Consider hypothetical independent samples with \(n_1=100\), \(p_1=0.30\), \(n_2=150\), and \(p_2=0.42\). Check the expected counts and describe the approximate distribution.

For group 1, the expected success and failure counts are \(100(0.30)=30\) and \(100(0.70)=70\). For group 2, they are \(150(0.42)=63\) and \(150(0.58)=87\). Each expected count exceeds 10, so the usual count check supports using a normal approximation, though it does not prove that the approximation is exact.

The center is \(0.30-0.42=-0.12\). The variance and standard error are

\begin{aligned} \operatorname{Var}(\hat{p}_1-\hat{p}_2) &=\frac{0.30(0.70)}{100}+\frac{0.42(0.58)}{150}\\ &=0.002100+0.001624\\ &=0.003724,\\ SE&=\sqrt{0.003724}\approx0.06102. \end{aligned}

Thus the sampling distribution is approximately \(N(-0.12,\,0.003724)\), where the second parameter is the variance. Its standard deviation is about 0.06102. The negative center means that, over repeated samples under these specified probabilities, group 1’s sample proportion tends to be lower than group 2’s by 0.12.

Worked Example: Larger Samples and a Rare Outcome

Two hypothetical independent groups have \(p_1=0.60\) and \(p_2=0.50\). Compare the standard error when both sample sizes are 100 with the standard error when \(n_1=100\) and \(n_2=200\). Then consider whether the normal approximation is reasonable if instead the probabilities were \(p_1=0.08\) and \(p_2=0.12\), with sample sizes \(n_1=50\) and \(n_2=60\).

With \(n_1=n_2=100\), the variance is

\frac{0.60(0.40)}{100}+\frac{0.50(0.50)}{100} =0.002400+0.002500=0.004900, \qquad SE=\sqrt{0.004900}=0.07000.

Increasing only the second sample size to 200 gives

\frac{0.60(0.40)}{100}+\frac{0.50(0.50)}{200} =0.002400+0.001250=0.003650, \qquad SE=\sqrt{0.003650}\approx0.06042.

The expected difference remains \(0.60-0.50=0.10\), but the standard error decreases because the group 2 contribution to the variance is smaller. More observations in one group reduce that group’s contribution; they do not eliminate the other group’s contribution.

For the rare-outcome scenario, the expected counts are 4 successes and 46 failures in group 1, and 7.2 successes and 52.8 failures in group 2. The success counts are small, so the usual count check does not support a normal approximation. The variance formula still follows from the independent Bernoulli model:

\begin{aligned} \operatorname{Var}(\hat{p}_1-\hat{p}_2) &=\frac{0.08(0.92)}{50}+\frac{0.12(0.88)}{60}\\ &=0.001472+0.001760=0.003232,\\ SE&=\sqrt{0.003232}\approx0.05685. \end{aligned}

The fact that a variance can be calculated does not mean a normal approximation is adequate. Here the exact distribution is discrete and can be skewed because the event is rare, so the approximation needs caution.

What the Result Does—and Does Not—Say

The derivation shows that the difference is centered on the population difference and that its variance is the sum of the two independent groups’ variance contributions. This is useful because it separates two questions: where the sampling distribution is centered, and how much it typically varies. It also clarifies why sample sizes matter: each group’s variance contribution is divided by that group’s sample size.

The normal approximation is a description of repeated-sampling behavior under stated assumptions. It does not say that an individual observed difference must be close to the population difference, nor does it establish that the two population proportions differ. Those are questions addressed by inference, which uses the sampling distribution for a particular purpose.

Common pitfall: Adding the sample proportions’ standard errors directly, or subtracting their variances because the statistic is a difference, is incorrect. For independent groups, add the variances; then take the square root to obtain the standard error. The minus sign changes the center to \(p_1-p_2\), but it does not make either group’s variance negative.

A second important caution is to check the groups separately. A large combined sample does not guarantee that both proportions are well approximated by normal distributions. If one group has very few expected successes or failures, the difference may not be well approximated by a normal curve even if the other group has many observations.

Check Your Understanding

Use the model and results above to answer these questions.

Under the independent Bernoulli model, what is the expected value of \(\hat{p}_1-\hat{p}_2\), and what population quantity does it estimate?
Why do the two variance contributions add even though the statistic subtracts \(\hat{p}_2\)?
For \(n_1=80\), \(p_1=0.20\), \(n_2=100\), and \(p_2=0.30\), calculate the expected difference and its variance.
For a group with \(n=40\) and \(p=0.05\), calculate the expected number of successes. What does this suggest about using the normal approximation?
Why is a calculated variance not, by itself, sufficient evidence that the sampling distribution is approximately normal?