Worked Example: Standard Errors Under Repeated Sampling

Why the Standard Error Changes With Repeated Measurements

A standard error describes the sampling variability of an estimator, not the spread of individual measurements. If a concentration is measured several times in each subject, the total number of measurements is not generally the number of independent pieces of information. Measurements from the same subject tend to share subject-specific influences, so their dependence affects the variability of a pooled mean.

The independence-based calculation is still useful as a reference: it shows what the standard error would be if every measurement contributed independent information. The correlation-adjusted calculation accounts for repeated measurements within subjects. The comparison makes the consequence of the independence assumption explicit.

For the formulas in this tutorial, assume that all measurements target the same population mean \(\mu\), have the same marginal variance \(\sigma^2\), and have the same within-subject correlation \(\rho\). Subjects are independent of one another. Within a subject, every pair of distinct measurements has covariance \(\rho\sigma^2\). This is an exchangeable correlation assumption: it treats all within-subject pairs alike. In repeated pharmacokinetic data, the formulas apply to replicate measurements at a common time or under a common condition, not automatically to concentrations at different times that have different expected values.

In the common case of positive within-subject correlation, \(0<\rho\leq 1\). The mathematical lower bound for an exchangeable correlation among \(k\) measurements is \(-1/(k-1)\), but the examples below use positive correlation. In practice, \(\sigma\) and \(\rho\) are often unknown and must be estimated. The calculations below first state their values as known, or as specified plug-in values, so the standard-error arithmetic is clear.

Deriving the Correlation Adjustment

The pooled mean averages \(mk\) measurements, but its variance depends on both the \(mk\) individual variances and the covariances between measurements from the same subject. Since independent subjects have zero cross-subject covariance, the variance of a sum can be calculated subject by subject.

There are \(k\) variance terms for each subject and \(k(k-1)\) ordered covariance terms: each of the \(k\) measurements covaries with the other \(k-1\). This gives the within-subject contribution in the second line. Taking the square root gives the standard error of the pooled mean.

The result also shows why positive correlation increases the standard error. The design effect is greater than 1 when \(\rho>0\) and \(k>1\). The naive calculation then reports less uncertainty than the correlation-adjusted calculation. When \(\rho=0\), the two standard errors agree; when \(\rho=1\), the repeated measurements within each subject provide no additional independent information about the mean beyond one measurement per subject.

Working From Observed Measurements

Often the marginal standard deviation is not supplied. A common descriptive calculation is to compute the sample standard deviation across the \(mk\) recorded measurements and use it in the naive formula \(s/\sqrt{mk}\). To make the comparison, the same estimate of marginal spread can be used in the correlation-adjusted formula, with a specified or separately estimated \(\rho\). This substitution is a plug-in calculation; it does not account for uncertainty in the estimates of \(\sigma\) or \(\rho\).

There is an additional practical caution: the ordinary sample variance about the pooled mean is not automatically an unbiased estimate of marginal variance under clustering, because the pooled mean itself is estimated from dependent observations. The simple calculation above demonstrates the usual naive plug-in comparison; it should not be taken as a general recommendation for estimating uncertainty from a real clustered data set. In an actual analysis, the method used to estimate variability should respect the subject-level dependence and the study design.

Quantifying the Cost of Assuming Independence

The design effect can be expressed as an equivalent number of independent measurements. If \(\sigma^2\) is the marginal variance, the variance of a mean based on \(n_{\mathrm{eff}}\) independent measurements would be \(\sigma^2/n_{\mathrm{eff}}\). Matching this to the correlated-measurement variance gives

This is an effective sample size for the variance of this mean under the assumed model. It is not a literal count of subjects, and it is not necessarily the appropriate sample size for every analysis. It summarizes how much information the pooled mean receives relative to the same number of independent measurements.

When the Formula Applies, and When It Does Not

The balanced formula is most useful when every subject contributes the same number of measurements, those measurements target a common mean, and a single within-subject correlation is a reasonable approximation. If each subject contributes a different number of observations, if the correlation changes with the time gap between measurements, or if measurements at different times have different expected concentrations, the design effect \(1+(k-1)\rho\) does not generally describe the data. In those cases, the covariance structure and the exact estimator matter; a single adjusted multiplier should not be applied without justification.

A useful alternative for balanced data is to calculate a mean for each subject and then average those subject means. Under equal numbers of observations per subject, this gives the same pooled mean. Under the stated covariance assumptions, the variance of one subject's mean is \(\sigma^2[1+(k-1)\rho]/k\). Averaging the \(m\) independent subject means gives variance \(\sigma^2[1+(k-1)\rho]/(mk)\), exactly the result derived above. This provides a subject-level way to understand the same standard error, rather than a different correction.

The equivalence depends on equal numbers of measurements and a common target mean. With unequal numbers of observations, the pooled mean gives greater weight to subjects who contribute more measurements, whereas the average of subject means gives equal weight to subjects. Those estimators need not target the same quantity, and their standard errors need not agree. More flexible approaches can account for unequal sampling and richer dependence, but they require specifying and checking a more detailed model; the balanced formula is not a substitute for that work.

Diagnostics and Common Pitfalls

Before using a correlation adjustment, check whether its assumptions are plausible. Confirm that measurements being combined represent the same target quantity, count the number of observations per subject, and examine subject-specific trajectories or replicate values for evidence of within-subject similarity. Where data permit, estimate or otherwise justify the within-subject correlation. A single assumed value can be useful for a sensitivity calculation, but it should not be presented as a data-supported estimate unless the analysis actually estimates it.

• Pitfall: counting every sample as an independent observation. The reasoning is that there are \(mk\) recorded values, so the standard error must be \(s/\sqrt{mk}\). The correction is that this formula assumes zero covariance between distinct observations; with positive within-subject correlation, multiply the naive standard error by \(\sqrt{1+(k-1)\rho}\) under the balanced assumptions.
• Pitfall: treating the design effect as a variance multiplier for the standard error itself. The design effect multiplies the variance. The standard error is multiplied by its square root.
• Pitfall: using one correlation for observations at different times without checking the target. If expected concentrations change over time, pooling them does not estimate a common-time mean. Define the estimand first and use a dependence structure suited to the observations being combined.
• Pitfall: assuming more repeats can replace more subjects. Repeats can improve precision when \(\rho<1\), but their marginal contribution shrinks as correlation increases. The effective sample size calculation quantifies this for the pooled mean; it does not imply that a study with few subjects can answer every question as well as a larger study.

A practical sensitivity check is to calculate the adjusted standard error across a plausible range of \(\rho\) values. If the conclusion changes substantially, the analysis depends strongly on the assumed within-subject correlation and that dependence should be reported. The standard error alone also does not address bias from selective sampling, dropout, or a mismatch between the pooled mean and the scientific target.