Why Mixed Covariate Sampling Needs a Joint Model
A virtual population may need to represent both categorical characteristics, such as genotype group or disease subtype, and continuous characteristics, such as body weight or age. Matching each variable’s marginal distribution is not enough if the target population specifies relationships between them. For example, sampling genotype and weight independently can produce the correct overall genotype proportions and weight distribution while losing a real difference in weight between genotype groups.
The key design choice is therefore not just how to sample each variable, but how to represent their joint distribution. A useful general construction is to sample categorical covariates first, then sample continuous covariates from distributions conditional on the sampled categories. This approach makes the dependence explicit and can be extended to multiple categorical and continuous variables.
The joint distribution can be factored as \(P(C=c_k)f(x\mid C=c_k)\), where \(f\) is a probability density for the continuous covariates. Thus, sampling from the joint distribution can proceed in two stages: draw a category from its target probabilities, then draw continuous values from the distribution associated with that category. If the target specifies independence, the conditional distributions are the same across categories. Otherwise, they must differ in the ways required to represent the target dependence.
Sampling Categorical Covariates
For a categorical variable with target probabilities \(p_1,\ldots,p_K\), the probabilities must be nonnegative and sum to one. One standard procedure draws \(U\sim\mathrm{Uniform}(0,1)\) and assigns category \(c_k\) when \(U\) falls in the corresponding cumulative-probability interval. This is the inverse-CDF method for a discrete distribution.
With \(n\) independent draws, the number assigned to category \(c_k\) is random, with expected count \(np_k\). The expected count is not a promise that the realized count will equal it. If a workflow instead enforces exact category counts, it creates a different sampling design; it should be used only when those fixed counts are part of the target specification.
Worked Example: Drawing Categories From Target Proportions
Suppose a hypothetical population has three genotype groups with target probabilities 0.20, 0.50, and 0.30. Four independent uniform draws are 0.08, 0.41, 0.62, and 0.94. Assign each draw to a genotype group.
The cumulative intervals are: group 1 for \(0\leq U<0.20\), group 2 for \(0.20\leq U<0.70\), and group 3 for \(0.70\leq U<1\). Thus, 0.08 gives group 1; 0.41 and 0.62 give group 2; and 0.94 gives group 3. The four sampled groups are 1, 2, 2, and 3.
For \(n=4\), the expected category counts are \(4(0.20)=0.8\), \(4(0.50)=2.0\), and \(4(0.30)=1.2\). The realized counts, 1, 2, and 1, need not equal these expectations. Their proportions are 0.25, 0.50, and 0.25, which differ slightly from the targets because of finite-sample variation.
Combining Categories With Continuous Covariates
To represent dependence, specify the continuous distribution within each category. The distributions may differ in their means, variances, shapes, or dependence among continuous variables. After sampling \(C\), draw \(X\) from \(X\mid C\). The resulting overall distribution of \(X\) is a mixture of the category-specific distributions, weighted by the category probabilities.
This mixture formula also gives a useful justification for the procedure: for any region \(A\) of continuous values, the probability of drawing \(X\) in \(A\) is the sum of the probabilities of being in each category and drawing a value in \(A\) within that category. In symbols, \(P(X\in A)=\sum_k P(C=c_k)P(X\in A\mid C=c_k)\), which is the law of total probability. The procedure therefore reproduces the specified joint distribution, provided both stages use the stated probabilities and conditional distributions.
Worked Example: Weight Conditional on a Categorical Covariate
Consider a hypothetical population in which 25% of individuals are in category P and 75% are in category N. Body weight, \(W\), is normally distributed within each category: \(W\mid P\sim N(55,8^2)\) kg and \(W\mid N\sim N(70,12^2)\) kg. To generate two individuals, use category uniforms 0.18 and 0.62, and standard-normal draws 1.25 and \(-0.50\), respectively.
The first uniform is below 0.25, so the first individual is in P. Its weight is \(55+8(1.25)=65.0\) kg. The second uniform is at least 0.25, so the second individual is in N. Its weight is \(70+12(-0.50)=64.0\) kg. Each continuous draw uses the parameters for the individual’s sampled category.
The overall mean weight implied by this mixed distribution is \(0.25(55)+0.75(70)=66.25\) kg. The category-specific means are different, so category and weight are dependent. Sampling category and weight independently from their marginal distributions would not preserve that association.
The overall variance can also be checked by separating within-category and between-category variation. The within-category contribution is \(0.25(8^2)+0.75(12^2)=124\) kg\(^2\). The between-category contribution is \(0.25(55-66.25)^2+0.75(70-66.25)^2=42.1875\) kg\(^2\). The total variance is \(166.1875\) kg\(^2\), giving an overall standard deviation of approximately \(12.89\) kg.
Several Covariates and Conditional Dependence
With multiple covariates, the same principle applies: express the joint distribution as a sequence of conditional distributions. For example, one might sample a categorical variable \(C\), then another categorical variable \(D\) conditional on \(C\), and then a continuous vector \(X\) conditional on both. This factorization preserves the specified dependence only when each conditional model reflects the target population.
If the continuous covariates are jointly normal within each category combination, their conditional means and covariance matrices can vary across combinations. Correlation among continuous covariates must be represented within those conditional distributions, rather than lost by drawing each variable separately. The earlier material on joint sampling of correlated covariates provides the multivariate normal sampling construction; here the essential point is to apply the appropriate construction after the categorical combination is known.
Worked Example: A Category and Correlated Continuous Covariates
Suppose a hypothetical population has category A with probability 0.40 and category B with probability 0.60. Within category A, age and weight have means 60 years and 75 kg, standard deviations 10 years and 15 kg, and correlation 0.50. For a draw in A, use \(Z_1=1\) and \(Z_2=-1\), where the two standard-normal inputs are independent. Use the correlated-normal construction below.
A category uniform of 0.30 selects A because \(0.30<0.40\). The sampled age is \(60+10(1)=70\) years. Since \(\sqrt{1-0.50^2}=\sqrt{0.75}\approx0.8660\), the sampled weight is \(75+15(0.50-0.8660)=69.51\) kg. The formula gives age and weight the specified correlation within category A; drawing them as unrelated univariate normals would not.
This draw is one possible value, not evidence that age and weight are negatively associated: the particular independent normal inputs happened to produce a weight below its category mean. Dependence is a property of the distribution across repeated draws, not a rule that every individual’s values move in the same direction.
Assumptions, Diagnostics, and Alternatives
A mixed sampling algorithm is only as credible as the target distribution it encodes. Before sampling, verify that category probabilities are valid, conditional distributions are defined for all categories that can be drawn, and the proposed dependence structure corresponds to the intended population. For conditional normal models, for example, covariance matrices must be valid and the normal approximation should be plausible for each covariate. A normal model for a strictly positive variable can produce negative values; changing to a suitable transformation or another distribution may be necessary.
After generating a sufficiently large sample, compare it with the target at more than one level. Check overall category proportions and continuous marginal summaries, but also check conditional means, spreads, quantiles, and correlations within categories. Cross-tabulations are useful when there are multiple categorical variables. A sample can match every marginal distribution and still fail to match the intended joint distribution.
Write down category probabilities and the conditional continuous distributions, including any required dependence among continuous covariates.
Use the categorical probabilities to draw each category. Keep in mind that independent draws produce random, not guaranteed, category counts.
For each individual, draw continuous values from the distribution corresponding to that individual’s sampled category or category combination.
Compare category proportions, conditional distributions, and joint summaries with their targets; investigate discrepancies larger than expected from Monte Carlo variation.
Empirical resampling of complete covariate records is an alternative when representative joint records are available: it retains the observed combinations without requiring a parametric conditional model. Its limitation is that it can reproduce only the patterns represented in those records. Independent marginal sampling is simpler, but it is appropriate only when independence is part of the target. A conditional parametric model is more flexible and can represent specified relationships, but its results depend on model form and parameter values.
A common mistake is to draw the categories and continuous variables independently because their marginal distributions are each correct. That reasoning overlooks the joint distribution: if the continuous distribution differs by category, independent draws erase that dependence. The correction is to sample the continuous values conditionally, or to use another joint sampling method that reproduces the same target dependence.
Check Your Understanding
Answer each question using the stated target distributions. Solutions follow the questions.
- A three-level categorical covariate has probabilities 0.10, 0.35, and 0.55. What category is selected by a uniform draw of 0.42 using the inverse-CDF rule?
- A continuous covariate has mean 40 in category A and mean 50 in category B. Explain why independently sampling the category and the continuous value from their overall marginal distributions does not preserve the target dependence.
- For \(n=200\) independent draws, a category has target probability 0.30. What is its expected count, and does that value guarantee the realized count?
- In a model with a categorical covariate and two correlated continuous covariates, what should be checked beyond the overall marginal distributions?
Solutions. 1. The cumulative cut points are 0.10 and 0.45, so 0.42 selects the second category. 2. The different category-specific means imply an association; independent sampling would not make the continuous distribution depend on the selected category. 3. The expected count is \(200(0.30)=60\), but random sampling does not guarantee exactly 60. 4. Check category proportions, conditional distributions of both continuous covariates, their within-category correlation, and any specified joint summaries.