Empirical Resampling Of Covariate Records

Why Resample Covariate Records?

A virtual population needs covariate profiles for its simulated subjects. When a set of observed records is being used to represent the target covariate population, empirical resampling creates virtual subjects by selecting from those records. This is a direct way to carry observed profiles into a simulation without inventing new covariate values. The choice of resampling scheme matters: it determines whether records can recur, how much of the finite dataset appears in a cohort, and what population the cohort represents.

The earlier material on defining target covariate populations distinguishes the population a simulation is meant to represent from the data available to construct it. Resampling does not, by itself, establish that observed records represent the target population. It only specifies how records are drawn once the eligible source records and their intended role have been defined.

Definition: Let a finite dataset contain \(N\) eligible covariate records, indexed \(1,\ldots,N\). Record \(i\) is a complete profile, such as age, body weight, and renal function for one observed individual. Under equal-probability empirical resampling, each record has probability \(1/N\) at each draw. The resulting empirical distribution assigns probability \(1/N\) to each record, counting distinct records separately even when some have identical covariate values.

A virtual subject should receive a whole record, not separately sampled pieces of several records. If age, weight, and renal function belong to one row, the row is the resampling unit. Selecting fields independently could create combinations that were never observed and alter relationships among covariates. Empirical row resampling retains the observed profiles and their observed combinations; it cannot create a profile outside the set of records.

Sampling With and Without Replacement

With replacement, each draw is made from all \(N\) records, and a selected record remains available for later draws. For a cohort of \(M\) virtual subjects, the same source record can therefore be used multiple times. Conditional on the finite dataset, the draws are independent and identically distributed from its empirical distribution.

Without replacement, records are selected by simple random sampling: every subset of \(M\) distinct record identifiers has the same probability. This requires \(M\leq N\). A record can appear at most once, so \(M=N\) gives an exact census of the eligible records. Distinct records can nevertheless have identical covariate values; “without replacement” concerns record identities, not necessarily unique combinations of values.

Feature	With replacement	Without replacement
Can a source record recur?	Yes	No
Maximum cohort size	No limit imposed by \(N\)	\(M\leq N\)
Draw dependence	Draws are independent given the dataset	Selections are dependent
When \(M=N\)	May omit some records and repeat others	Includes every eligible record exactly once

Worked Example: Expected Coverage of a Resampled Dataset

Suppose a hypothetical dataset contains \(N=5\) eligible covariate records, and a virtual cohort of \(M=3\) subjects is drawn. Under sampling with replacement, calculate the probability that a particular record appears at least once and the expected number of distinct records represented.

At each draw, the probability of not selecting the specified record is \(4/5\). Independence of the three draws gives

P(\text{record absent from all 3 draws})=\left(\frac{4}{5}\right)^3=\frac{64}{125}.

Thus the probability that it appears at least once is

1-\frac{64}{125}=\frac{61}{125}=0.488.

Let \(U\) be the number of distinct source records represented. Write \(U\) as the sum of five indicators, one for each source record, equal to 1 if that record appears and 0 otherwise. By linearity of expectation, the expected number represented is

E(U)=5\left(\frac{61}{125}\right)=2.44\text{ records}.

The expected count need not be an integer. Under sampling without replacement, exactly 3 distinct records would be represented. Both schemes give each source record the same marginal chance of selection on any one draw, but they do not give the same cohort coverage or dependence structure.

Sampling Variability for a Finite Dataset

A useful comparison is the variability of the sample mean of a covariate. Treat the \(N\) observed records as a fixed finite population, with values \(x_1,\ldots,x_N\), finite-population mean \(\mu_N\), and finite-population variance \(S_N^2\) defined using denominator \(N-1\). For \(N>1\), simple random sampling without replacement of size \(M\), where \(1\leq M\leq N\), gives the following result.

Theorem: For \(N>1\), under simple random sampling without replacement from the \(N\) fixed records,

\mu_N=\frac{1}{N}\sum_{i=1}^{N}x_i,\qquad S_N^2=\frac{1}{N-1}\sum_{i=1}^{N}(x_i-\mu_N)^2,

and the sample mean \(\bar{x}_{\text{sample}}\) satisfies

E(\bar{x}_{\text{sample}})=\mu_N,\qquad \operatorname{Var}(\bar{x}_{\text{sample}})=\left(1-\frac{M}{N}\right)\frac{S_N^2}{M}.

For \(N=1\), the only possible sample contains the sole record, so the sample mean has variance zero; \(S_N^2\) is not defined in that case.

The factor \(1-M/N\) is the finite-population correction. It reduces the variance because selecting a record without replacement makes it unavailable for subsequent selections. At \(M=N\), the sample mean equals the finite-dataset mean exactly and its variance is zero.

Why the Finite-Population Correction Appears

A short derivation makes the assumptions behind the result explicit. Set \(z_i=x_i-\mu_N\), so that \(\sum_i z_i=0\). Let \(I_i\) equal 1 if record \(i\) is selected and 0 otherwise. Under simple random sampling without replacement, \(P(I_i=1)=M/N\), and for distinct records \(i,j\), \(P(I_i=1,I_j=1)=M(M-1)/(N(N-1))\). The centered sample mean is \(M^{-1}\sum_i I_i z_i\), and its expectation is zero because each record has the same inclusion probability.

For its variance, expand the square. Since \(\sum_{i<j}z_i z_j=-\frac12\sum_i z_i^2\), the expansion is

\begin{aligned} \operatorname{Var}(\bar{x}_{\text{sample}}) &=\frac{1}{M^2}\left[ \frac{M}{N}\sum_i z_i^2+ 2\frac{M(M-1)}{N(N-1)}\sum_{i<j}z_i z_j \right]\\ &=\frac{1}{M^2}\left[ \frac{M}{N}-\frac{M(M-1)}{N(N-1)} \right]\sum_i z_i^2\\ &=\frac{N-M}{MN}\,S_N^2 =\left(1-\frac{M}{N}\right)\frac{S_N^2}{M}. \end{aligned}

The derivation depends on equal-probability sampling of distinct records from a fixed finite dataset. It is not the variance formula for independent draws with replacement. Under replacement, the variance of the mean is \(\sigma_{\mathrm{emp}}^2/M\), where \(\sigma_{\mathrm{emp}}^2=N^{-1}\sum_i(x_i-\mu_N)^2\) is the variance under the empirical distribution. Since \(\sigma_{\mathrm{emp}}^2=(N-1)S_N^2/N\), the two formulas differ.

Worked Example: Comparing Mean Variability Under Both Schemes

Consider a hypothetical dataset of eight patient records with eGFR values, in mL/min/1.73 m², of 110 once, 95 twice, 80 zero times, 65 three times, and 50 twice. Calculate the finite-dataset mean and compare the variance and standard error of the mean for a cohort of \(M=4\) records drawn with and without replacement.

The finite-dataset mean is

\mu_N=\frac{1(110)+2(95)+0(80)+3(65)+2(50)}{8} =\frac{595}{8}=74.375\text{ mL/min/1.73 m}^2.

For the finite-population variance, the sum of squared deviations is

\sum_{i=1}^{8}(x_i-\mu_N)^2 =(110-74.375)^2+2(95-74.375)^2+3(65-74.375)^2+2(50-74.375)^2 =3571.875.

Therefore \(S_N^2=3571.875/7=510.2679\;(\text{mL/min/1.73 m}^2)^2\). Without replacement, the sample-mean variance and standard error are

\operatorname{Var}(\bar{x}_{\text{sample}}) =\left(1-\frac{4}{8}\right)\frac{510.2679}{4} =63.7835\;(\text{mL/min/1.73 m}^2)^2,\qquad SE=\sqrt{63.7835}=7.986\text{ mL/min/1.73 m}^2.

With replacement, the empirical-distribution variance is \(3571.875/8=446.4844\;(\text{mL/min/1.73 m}^2)^2\). Independent draws give

\operatorname{Var}(\bar{x}_{\text{sample}}) =\frac{446.4844}{4}=111.6211\;(\text{mL/min/1.73 m}^2)^2,\qquad SE=\sqrt{111.6211}=10.565\text{ mL/min/1.73 m}^2.

The difference is expected: sampling without replacement reduces variability in the cohort mean because selected records cannot be selected again. These standard errors describe random sampling from this fixed set of eight records; they do not quantify uncertainty about whether those records represent a broader target population.

Preserving Profiles and Checking the Target

The resampling unit should match the unit whose joint profile the simulation is intended to use. For ordinary equal-probability row resampling, attach all required covariate fields to the same selected record, maintain its record identifier for auditing, and check that missing or ineligible records were handled according to the target definition. The following example shows how one draw carries a complete profile forward.

Worked Example: Carrying Complete Records Into a Cohort

Suppose a hypothetical four-record dataset has these covariate profiles: A has eGFR 110 and weight 70 kg; B has eGFR 95 and weight 82 kg; C has eGFR 65 and weight 60 kg; and D has eGFR 50 and weight 90 kg. A with-replacement draw of six record identifiers is B, B, D, A, B, C. Calculate the two sample means and explain what the draw preserves.

The eGFR values are 95, 95, 50, 110, 95, and 65. Their mean is

\bar{x}_{\text{eGFR}}=\frac{95+95+50+110+95+65}{6} =\frac{510}{6}=85.00\text{ mL/min/1.73 m}^2.

The corresponding weights are 82, 82, 90, 70, 82, and 60 kg, giving

\bar{x}_{\text{weight}}=\frac{82+82+90+70+82+60}{6} =\frac{466}{6}=77.67\text{ kg}.

Each simulated subject retains the original pairing: for example, every selected B record has both eGFR 95 and weight 82 kg. Record B occurs three times, which is allowed with replacement. The empirical dataset means are 80.00 mL/min/1.73 m² for eGFR and 75.50 kg for weight, so this particular cohort differs from the dataset means by chance. A single resampled cohort is not expected to reproduce every empirical summary exactly.

Choosing a Scheme and Diagnosing Problems

The main choice is not a contest over which scheme is universally better; it is a decision about the target and the intended construction of the virtual cohort. Use with-replacement sampling when each virtual subject is intended as a separate draw from the empirical distribution and repeated use of source profiles is acceptable. Use without-replacement sampling when the cohort is meant to be a subset of distinct observed records, or when representing the entire finite dataset once is the goal. If the requested cohort is larger than the source dataset, without-replacement sampling cannot produce it.

Before resampling, check that the source records actually correspond to the target population in the relevant design, eligibility criteria, time period, and covariate coverage. A large resampled cohort can make empirical frequencies appear precise without adding any new information about the target. Repeating records increases cohort size, not the number of distinct observed profiles. Conversely, a without-replacement cohort smaller than the source dataset omits some records by design; it is not a census of the finite dataset.

Useful checks include the number of unique source identifiers selected, the frequency of each source record, whether every simulated profile matches one eligible source row, and whether key empirical summaries differ substantially from the intended target summaries. A marked discrepancy in one random cohort may be ordinary sampling variation, but a persistent mismatch across repeated runs can indicate an incorrect target frame, an unintended selection rule, or a coding error. These checks diagnose construction and representation; they cannot prove that the observed records are representative of an unobserved population.

Common mistake: treating replacement as new information. A cohort of thousands created from a few dozen source records still contains only those observed profiles. Repeated rows do not expand observed covariate support.
Common mistake: resampling columns independently. This can break observed pairings and create profiles not present in the data. Select complete rows when the target is represented by complete records.
Common mistake: using the wrong variance formula. The finite-population correction applies to simple random sampling without replacement. Independent replacement draws use the empirical-distribution variance divided by \(M\).
Common mistake: claiming a finite sample establishes target representativeness. Resampling reproduces the selected dataset’s empirical distribution in expectation under replacement; it does not correct selection bias or missing target subgroups.

Key takeaway: Empirical resampling represents the distribution of eligible observed records, not a distribution beyond them. State the target frame, choose replacement according to the intended cohort construction, and preserve each record as a complete covariate profile.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A dataset has four eligible records. Under replacement, what is the probability that a specified record is selected on all four draws? How many distinct records appear in a four-record sample without replacement?
Why does a sample without replacement of size \(M=N\) have zero variance for its sample mean?
Two covariates are recorded together in each source row. What feature of the data may be lost if each column is resampled independently?
A cohort with replacement contains 10,000 virtual subjects generated from 40 observed profiles. Does this mean 10,000 distinct observed profiles support the simulation? Explain.

Solutions: (1) The probability is \((1/4)^4=1/256=0.003906\). Without replacement, all four record identifiers appear, so there are exactly four distinct records. (2) The sample is the entire finite dataset, so its mean equals the fixed finite-dataset mean in every possible draw; its variance is therefore zero. (3) Independent column resampling can destroy the observed pairing or dependence between the covariates and may create combinations absent from the source rows. (4) No. The cohort reuses records; it contains at most 40 distinct source profiles and provides no new observed covariate support.