Three Objects That Can Have Similar Names
A recurring source of error in statistical reasoning is to use one number as if it represented three different things: a random statistic before data are collected, the statistic's observed value after data are collected, and a fixed quantity describing the population. They may be connected, and sometimes they may even have the same numerical value, but they are not interchangeable.
For example, a sample mean is random before the sample is drawn because its value depends on which observations are selected. Once the observations are recorded, the sample mean is a particular number. The population mean is a separate quantity: it describes the population or its probability model and does not change from one sample to another under the usual fixed-parameter model.
Use \(X_1,\ldots,X_n\) to denote the random observations before sampling, and \(x_1,\ldots,x_n\) to denote their observed values. If \(T\) is a statistic, then \(T(X_1,\ldots,X_n)\) is the random statistic, while \(T(x_1,\ldots,x_n)\) is its realized value. For instance, \(\bar{X}\) denotes the random sample mean and \(\bar{x}\) its observed value.
An estimator is a statistic used to estimate a population quantity. The random estimator and the parameter are different kinds of objects: the estimator varies across samples, while the parameter is fixed in the model. Once a sample is observed, the estimator's realized value is an estimate. An estimate can be above, below, or equal to the parameter.
| Object | Before sampling | After observing the sample |
|---|---|---|
| Sample mean | Random statistic \(\bar{X}\) | Realized value \(\bar{x}\) |
| Population mean | Fixed quantity \(\mu\) | The same fixed quantity \(\mu\) |
| Sample variance | Random statistic \(S^2\) | Realized value \(s^2\) |
| Population variance | Fixed quantity \(\sigma^2\) | The same fixed quantity \(\sigma^2\) |
The table is not claiming that a population quantity is known. A parameter can be fixed but unknown. “Fixed” means that its value is not changed merely because a different random sample is drawn from the same model.
How to Classify a Quantity
A reliable classification asks two questions. First, can the quantity change if a new sample is drawn while the population model stays the same? If so, it is a statistic or a statistic's realized value, depending on whether sampling has occurred. Second, does the quantity describe the population model itself? If so, it is a population quantity. A number's size alone does not tell you which role it plays.
Identify the population feature, such as \(\mu\), \(p\), or \(\sigma^2\). Treat it as fixed under the usual repeated-sampling model.
For example, \(\bar{X}=(X_1+\cdots+X_n)/n\) depends on the sample and is random before the observations are known.
Calculate the realized value, such as \(\bar{x}=(x_1+\cdots+x_n)/n\), and label it as an estimate if it is being used to estimate a parameter.
Worked Example: A Random Sample Mean and Its Observed Value
A hypothetical sensor's recorded reading has the following population distribution: \(P(X=1)=1/4\), \(P(X=3)=1/2\), and \(P(X=8)=1/4\). Two readings \(X_1\) and \(X_2\) are drawn independently from this distribution. Identify the population mean and the random statistic, find the statistic's realized value if the readings are 1 and 8, and calculate the probability that the statistic equals that realized value.
Method. Calculate the population quantity from the model probabilities, define the statistic using random observations, and then distinguish its observed value from both the random statistic and the population mean.
Population mean. The model's mean is
This \(3.75\) is a fixed population quantity. It does not change when the two readings change.
Random statistic. The sample mean before observing the readings is
If the observed readings are \(x_1=1\) and \(x_2=8\), then the realized value is
Thus, \(\bar{X}\) is the random statistic, \(4.5\) is its realized value in this sample, and \(3.75\) is the population mean. The realized value \(4.5\) estimates \(\mu\), but it is not identical to \(\mu\).
Probability check. The sample mean equals \(4.5\) exactly when the two readings are 1 and 8 in either order. Independence gives
The probability is \(0.1250\), which confirms that \(4.5\) is a possible value of a random statistic, not a constant population mean. Common mistake avoided: treating the observed sample mean as the population mean simply because both describe “the average.”
Why a Statistic Is Random Before Sampling
The defining feature is dependence on the sample. If the sample can take different values, a function of those values can also take different values. For a sample mean based on independent observations with common finite mean \(\mu\), this relationship is also visible in its expectation.
This equality concerns the average value of the random statistic over repeated samples. It does not say that every realized value \(\bar{x}\) equals \(\mu\). For a particular sample, the estimate can differ from the parameter; the expectation describes the statistic's behavior across the sampling process. Independence is part of the stated sampling model, although linearity of expectation itself does not require independence.
A Statistic Can Equal a Parameter Without Becoming One
A numerical coincidence does not erase the distinction between the roles of two quantities. A sample statistic can equal a population quantity for one observed sample, or for some possible samples, while remaining random before sampling. Likewise, two different definitions can produce the same number in a particular case.
Worked Example: A Sample Variance That Differs From the Population Variance
A hypothetical population takes values 0, 2, and 4, each with probability \(1/3\). Three observations are drawn independently. For the observed data \(0,4,4\), calculate the population variance and the realized sample variance using denominator \(n-1\). Explain which quantity varies across samples.
Method. First compute the fixed population variance from its definition. Then apply the sample-variance formula to the recorded values, keeping the population and sample calculations separate.
Population quantity. The population mean is
The population variance is the probability-weighted mean squared deviation from 2:
Realized sample statistic. For the observed sample \(x_1=0\), \(x_2=4\), \(x_3=4\), the sample mean is
The squared deviations from this observed mean are
Their sum is \(96/9=32/3\). Dividing by \(n-1=2\) gives
The population variance is \(\sigma^2=8/3\), while the realized sample variance is \(s^2=16/3\). The random statistic \(S^2\) would change if a different sample were observed; \(\sigma^2\) remains fixed under this model. Common mistake avoided: substituting a sample's squared deviations into the population definition and then labeling the result \(\sigma^2\).
Estimates of a Population Proportion
Sample proportions provide a particularly clear example because the statistic and parameter often have similar names. In a Bernoulli model, each observation records a success or failure. The population success probability \(p\) is fixed, while the sample proportion of successes depends on which observations occur.
Worked Example: A Sample Proportion Is Not the Population Probability
A hypothetical quality-control process has a fixed probability \(p=0.4\) that a tested unit requires adjustment. Five units are tested independently, with \(X_i=1\) if unit \(i\) requires adjustment and \(X_i=0\) otherwise. The observed results are \(1,1,0,1,0\). Find the realized sample proportion and the probability that the random sample proportion equals this value.
Method. Count the observed successes to obtain the realized statistic, then use the Bernoulli sampling model to find the probability of that statistic value.
Realized value. There are three successes among five observations, so
This \(0.6\) is an estimate from the observed sample. It is not a replacement for the model's fixed success probability \(p=0.4\).
Probability check. The random sample proportion equals \(0.6\) precisely when there are three successes among five independent trials. There are \(\binom{5}{3}=10\) arrangements, and each has probability \((0.4)^3(0.6)^2\). Therefore,
The answer is \(0.2304\). Repeated samples could give other sample proportions, but the model parameter remains \(0.4\). Common mistake avoided: interpreting an observed proportion of \(0.6\) as proof that the population probability is \(0.6\).
Common Exam Traps
- Calling a statistic fixed before sampling. A statistic is a function of random observations, so its value can vary from sample to sample. The population quantity it may estimate is fixed under the model.
- Using “estimate” and “estimator” interchangeably. The estimator is the random statistic; the estimate is its realized value after observing data.
- Equating numerical equality with conceptual identity. If a sample statistic happens to equal a parameter, it remains a statistic and the parameter remains a population quantity.
- Assuming an estimate must be accurate in every sample. A statistic can differ substantially from the quantity it is intended to estimate. Its behavior across samples is a separate question from the value in one sample.
These distinctions matter whenever a problem asks what is random, what is observed, or what belongs to the population model. They also prevent a common ambiguity in written solutions: label a quantity as \(T(X_1,\ldots,X_n)\) before sampling, as \(T(x_1,\ldots,x_n)\) after observing data, and as \(\theta\) when referring to the population parameter.
Check Your Understanding
For each problem, distinguish the population quantity from the statistic and its realized value where requested.
- A population takes values 2 and 6 with probabilities \(3/4\) and \(1/4\). Two independent observations are drawn, and the observed values are 6 and 2. Find the population mean, the realized sample mean, and the probability that the random sample mean equals 4. (Answer: \(\mu=3\), realized mean \(=4\), \(P(\bar{X}=4)=0.3750\))
- A population takes values 1, 5, and 9 with equal probabilities. Two independent observations are drawn. Define the statistic as the sample maximum. For observed values 5 and 9, give the realized statistic, the population maximum, and the probability that the sample maximum is 9. (Answer: realized statistic \(=9\), population maximum \(=9\), probability \(=5/9=0.5556\); the equal numbers have different roles.)
- Independent Bernoulli observations have fixed success probability \(p=0.3\). For a sample of four with observed outcomes \(0,1,0,1\), find the realized sample proportion and the probability that the random sample proportion equals it. (Answer: realized proportion \(=0.5\), \(P(\hat{p}=0.5)=\binom42(0.3)^2(0.7)^2=0.2646\); \(p=0.3\))
- A population takes values 0, 2, and 4 with equal probabilities. For three independent observations \(0,2,4\), calculate the realized sample variance using denominator \(n-1\), and distinguish it from the population variance. (Answer: \(s^2=4\), while \(\sigma^2=8/3\))
- For a population with fixed mean \(\mu\), independent observations \(X_1,\ldots,X_4\) have finite expectation. Write the sample mean as a random statistic and state its expectation. Explain whether the result makes every realized sample mean equal to \(\mu\). (Answer: \(\bar{X}=(X_1+X_2+X_3+X_4)/4\), \(E[\bar{X}]=\mu\); no, the equality is about expectation across samples, not every realized value.)