How Unequal Sampling Weights Subjects Unequally

Why the Number of Observations Matters

When subjects contribute different numbers of measurements, an average over all recorded observations does not give every subject the same total influence. A subject with many measurements contributes many terms to the pooled sum; a subject with few measurements contributes fewer. This is an arithmetic fact about how the mean is constructed, not by itself evidence that the pooled mean is wrong.

To make the distinction precise, suppose a numeric variable is measured for \(S\) subjects. Subject \(i\) contributes \(n_i\) observations \(x_{i1},\ldots,x_{in_i}\), where each subject has at least one observation. Let \(N=\sum_{i=1}^{S}n_i\) be the total number of observations and let \(\bar{x}_i\) be subject \(i\)'s mean. The measurements must be on a common scale and represent the quantity being averaged. For example, measurements in milligrams per litre can be averaged with other measurements in the same units.

The two means summarize the same collection of subjects and measurements, but they give different total weights to subjects whenever the \(n_i\) differ. The pooled mean gives every observation weight \(1/N\). Therefore, subject \(i\)'s total weight is \(n_i/N\). In the equal-subject mean, subject \(i\)'s total weight is \(1/S\), and each of that subject's observations has weight \(1/(S n_i)\) in the resulting summary.

Algebra of the Unequal Weights

The pooled sum can be regrouped by subject. Within each subject, the sum of the measurements is \(n_i\bar{x}_i\). Substituting that identity into the pooled mean gives a weighted average of the subject means:

The weights \(n_i/N\) are nonnegative and sum to one, since \(\sum_i n_i/N=N/N=1\). Thus the pooled mean is a weighted average of subject means, with weights proportional to the number of observations each subject contributes. By contrast, \(\bar{x}_{\mathrm{subject}}=\sum_i(1/S)\bar{x}_i\) assigns the same weight to every subject.

A useful expression makes the direction of the difference explicit. Let \(\bar{n}=N/S\), the average number of observations per subject, and define the finite-subject covariance between counts and subject means as \(C_{n,\bar{x}}=(1/S)\sum_{i=1}^{S}(n_i-\bar{n})(\bar{x}_i-\bar{x}_{\mathrm{subject}})\). Expanding the product gives \(C_{n,\bar{x}}=(1/S)\sum_i n_i\bar{x}_i-\bar{n}\bar{x}_{\mathrm{subject}}\). Since \(\bar{x}_{\mathrm{pool}}=(\sum_i n_i\bar{x}_i)/(S\bar{n})\), it follows that:

This identity says that the pooled mean exceeds the equal-subject mean when subjects with more observations tend, in this set of subjects, to have higher subject means; it is lower when they tend to have lower means. If counts and subject means have zero finite-subject covariance, the summaries are equal, even if counts are unequal. Equal counts are another sufficient condition for equality, because then every pooled subject weight is \(1/S\). Neither condition should be mistaken for a claim about how the measurements were generated.

Worked Comparisons

Choosing and Checking a Summary

The pooled and equal-subject means answer distinct descriptive questions. The pooled mean answers: what is the mean among the recorded observations, giving each recorded value equal weight? The equal-subject mean answers: what is the average of the observed subject-specific means, giving each included subject equal weight? These are not competing arithmetic rules for one uniquely defined target. The intended target determines which weighting is suitable.

A pooled mean can be appropriate when the object of interest is the collection of recorded observations and equal weighting of those observations is intended. An equal-subject mean can be useful when the intended summary is the average of the included subjects' means. The latter does not automatically represent every subject in a broader population, and the former does not automatically represent a typical subject. Both are summaries of the data as observed; wider population interpretations require assumptions and a defined target.

A practical check is to calculate the counts \(n_i\) and means \(\bar{x}_i\) separately before reporting an overall average. Then verify the pooled calculation in both forms: \(\sum_{i,j}x_{ij}/N\) and \(\sum_i(n_i/N)\bar{x}_i\). Agreement checks the arithmetic and makes the weighting transparent. Compare this result with \(\sum_i\bar{x}_i/S\), and state clearly which one is being reported.

• Counts vary: inspect the subject weights \(n_i/N\); do not describe the pooled mean as equally weighted across subjects.
• Counts do not vary: the pooled and equal-subject means coincide, because every subject supplies the same fraction of all observations.
• Means differ but the summaries coincide: this can occur if the weighting differences balance, as when count and subject mean have zero finite-subject covariance.
• Values are missing or not comparable: first clarify which observations and subjects are included and whether the measured quantity has the same meaning and units across them. The identities above describe the chosen observed values; they do not settle whether those values form an appropriate target.

Common Mistakes and Limitations

A common mistake is to calculate the pooled mean correctly and then call it an equally weighted average of subjects. The wrong reasoning is: “Every measurement is included, so every subject counts equally.” The correction is that every measurement has equal weight \(1/N\), while a subject with \(n_i\) measurements has total weight \(n_i/N\). Equal weighting of subjects instead requires taking the mean of the subject-specific means.

A second mistake is to assume that the equal-subject mean is always more representative. It gives each included subject the same weight by definition, but that feature does not prove it is the scientifically relevant summary. Conversely, unequal subject weights do not make the pooled mean mathematically biased by themselves. The weighting may be exactly what the descriptive question calls for. The essential limitation is that neither calculation can determine the intended target on its own.

These formulas are descriptive identities. They do not account for within-subject dependence when calculating uncertainty, and they do not make observations from the same subject independent. A mean's weighting and the method used to quantify its uncertainty are separate issues.