Tutorials › Nonlinear Mixed-Effects Models (NLME) › Informative Sampling and Biased Pooled Means

Why Mixed-Effects Models · Tutorial 9 of 797

Informative Sampling and Biased Pooled Means

See how concentration- or status-dependent sampling can make a pooled mean reflect frequently sampled subjects more than the typical subject.

Beginner 10 min read

What You'll Learn

  • Define the equal-subject mean and the record-weighted pooled mean for repeated measurements.
  • Derive how the difference between these means depends on the association between sampling counts and subject means.
  • Work through examples where informative sampling shifts a pooled mean upward or downward.
  • Distinguish unequal sampling from informative sampling and identify the target each summary describes.
  • Use subject-level summaries and the sampling design to diagnose potential weighting problems.
  • Compare pooled means with subject-weighted and other design-aware summaries.

Why the Number of Records per Subject Matters

In a repeated pharmacokinetic dataset, one subject may contribute only a few concentration records while another contributes many. If every record receives equal weight, the second subject has more influence on the pooled mean. That may be appropriate when the target is the average of all recorded samples. It can be misleading when the target is the average concentration across subjects, especially if sampling frequency is related to concentration or clinical status.

The issue is not simply that the dataset has unequal numbers of records per subject. The key question is why those counts differ and what population quantity the analysis is intended to summarize. More frequent sampling of subjects with high concentrations, for example, gives those subjects extra weight in the pooled mean. If sampling frequency is instead associated with low concentrations, the same mechanism can pull the pooled mean downward.

Definition: Suppose subject \(i\) has \(m_i\) records in a defined analysis window, with mean concentration \(\bar{C}_i\). The equal-subject mean is \(\frac{1}{N}\sum_{i=1}^{N}\bar{C}_i\), where \(N\) is the number of subjects. The record-weighted pooled mean is \(\frac{\sum_{i=1}^{N}m_i\bar{C}_i}{\sum_{i=1}^{N}m_i}\). The first gives each subject equal weight; the second gives each record equal weight.

These summaries answer different questions. The equal-subject mean describes the mean of the subject-level means, giving each subject the same contribution. The record-weighted mean describes the mean among records in the dataset, giving each subject a contribution proportional to their number of records. Neither is inherently wrong; interpretation depends on the target.

For a fair comparison, the subject-level means must refer to a clearly defined, comparable quantity—for example, concentration summaries over the same nominal time window. If subjects contribute records at different times and concentration changes over time, unequal record counts can also change the time points represented in the pooled mean. The examples below hold the analysis window and meaning of each subject’s mean fixed, so they isolate the effect of unequal subject weights.

How Informative Sampling Changes the Mean

Sampling is informative for this comparison when the number of records contributed by a subject is associated with that subject’s concentration summary, either directly or through a related factor such as clinical status. A protocol that calls for extra measurements when concentrations are high is one direct route. More monitoring of patients with a clinical state associated with high concentrations is an indirect route. In either case, the record-weighted mean can place more weight on subjects whose means differ systematically from the rest.

The relationship can be shown exactly for a finite set of subjects. Define \(\bar{m}=\frac{1}{N}\sum_i m_i\), \(\bar{C}=\frac{1}{N}\sum_i\bar{C}_i\), and the finite-population covariance \(\operatorname{Cov}_N(m,\bar{C})=\frac{1}{N}\sum_i(m_i-\bar{m})(\bar{C}_i-\bar{C})\). Then:

$$ \frac{\sum_i m_i\bar{C}_i}{\sum_i m_i}-\bar{C} = \frac{\operatorname{Cov}_N(m,\bar{C})}{\bar{m}}. $$

To verify the identity, write \(m_i=(m_i-\bar{m})+\bar{m}\) and \(\bar{C}_i=(\bar{C}_i-\bar{C})+\bar{C}\). On summing their products, the terms involving a single deviation sum to zero, leaving \(\sum_i m_i\bar{C}_i=N\operatorname{Cov}_N(m,\bar{C})+N\bar{m}\bar{C}\). Since \(\sum_i m_i=N\bar{m}\), dividing by that total and subtracting \(\bar{C}\) gives the displayed result.

Thus, if subjects with higher means tend to contribute more records, the covariance is positive and the record-weighted mean exceeds the equal-subject mean. If higher means tend to contribute fewer records, the covariance is negative and the record-weighted mean is lower. If every subject contributes the same number of records, the covariance is zero and the two means are identical. Unequal counts alone do not establish the direction or size of the shift; their association with subject means matters.

Worked Example: More Records for Subjects with Higher Concentrations

Consider three hypothetical subjects. Their mean concentrations in a common analysis window are 2, 8, and 14 mg/L. They contribute 1, 3, and 6 records, respectively. Find and compare the equal-subject mean and the record-weighted pooled mean.

The equal-subject mean is:

$$ \frac{2+8+14}{3} = \frac{24}{3} = 8\ \text{mg/L}. $$

The record-weighted pooled mean is:

$$ \frac{1(2)+3(8)+6(14)}{1+3+6} = \frac{2+24+84}{10} = 11\ \text{mg/L}. $$

The pooled mean is 3 mg/L higher because the subject with the highest mean contributes the most records. As a check using the identity, \(\bar{m}=10/3\), and \(\operatorname{Cov}_N(m,\bar{C})=10\), so the difference is \(10/(10/3)=3\) mg/L. This is not a calculation error: the two summaries weight subjects differently.

Sampling Related to Clinical Status

Sampling frequency need not be triggered by a concentration value itself. Clinical status can affect the number of observations, and that status can be related to concentration. The resulting pooled mean still reflects the distribution of records, not an equal contribution from every subject.

Worked Example: Extra Monitoring in a Clinical Status Group

In a hypothetical study, four subjects are summarized over the same analysis window. Two clinically stable subjects have mean concentrations of 4 and 5 mg/L and contribute 2 records each. Two subjects in a clinical status associated with more intensive monitoring have means of 12 and 14 mg/L and contribute 8 records each. Calculate the two means.

The equal-subject mean is:

$$ \frac{4+5+12+14}{4} = \frac{35}{4} = 8.75\ \text{mg/L}. $$

For the record-weighted mean, the two stable subjects contribute \(2+2=4\) records, while the other two contribute \(8+8=16\). The pooled calculation is:

$$ \frac{2(4)+2(5)+8(12)+8(14)}{4+16} = \frac{8+10+96+112}{20} = 11.3\ \text{mg/L}. $$

The record-weighted mean exceeds the equal-subject mean by 2.55 mg/L. The example shows how a status-related sampling rule can give more influence to a group whose concentration summaries differ. It does not show that clinical status caused the concentration difference; it demonstrates how the sampling pattern affects the pooled summary.

The direction is not predetermined. If low concentrations or a clinical state associated with low concentrations lead to more intensive monitoring, the pooled mean can be lower than the equal-subject mean.

Worked Example: More Records for Subjects with Lower Concentrations

Suppose three hypothetical subjects have mean concentrations of 15, 10, and 5 mg/L, and contribute 1, 2, and 7 records, respectively. Find the equal-subject and record-weighted means.

The equal-subject mean is:

$$ \frac{15+10+5}{3} = 10\ \text{mg/L}. $$

The record-weighted mean is:

$$ \frac{1(15)+2(10)+7(5)}{1+2+7} = \frac{15+20+35}{10} = 7\ \text{mg/L}. $$

The pooled mean is 3 mg/L lower than the equal-subject mean. Here, the lowest-mean subject contributes the most records. Unequal sampling therefore does not always inflate a pooled concentration summary; it shifts the summary toward the subjects who contribute more records.

Choosing and Checking a Summary

Start by stating the target. If the question concerns the average across subjects, calculate subject-level summaries using a common definition and average those summaries with equal subject weights. If the question concerns the concentration of a randomly selected record from the collected dataset, the record-weighted mean directly describes that dataset. A pooled mean should not be described as the average subject’s concentration without checking how its weights arise.

Several practical checks help reveal when the distinction matters:

  • Tabulate the number of records and the subject-level summary for each subject. Look for a pattern in which subjects with larger or smaller values contribute more records.
  • Review the sampling protocol and reasons for extra measurements. Concentration-triggered measurements or status-related monitoring can create an association between record count and concentration.
  • Compare the record-weighted summary with an equal-subject summary. A substantial difference indicates that the weighting choice affects the reported mean, though it does not by itself establish why.
  • Check whether subjects’ records cover comparable times and windows. If they do not, changes in timing can alter the pooled mean in addition to changing subject weights.

A useful alternative is to summarize each subject first and then average the subject-level summaries. This directly targets the equal-subject mean, but it gives a subject with few records the same final weight as a subject with many; their summaries may therefore have different precision. If the goal is to represent a broader population or to address a sampling design, an analysis may instead require design-aware weights or a model that accounts for how measurements were obtained. Such approaches require assumptions about the sampling process and the target population; they are not automatically preferable just because the raw counts differ.

Summary approachWeightingPrimary interpretationImportant caution
Record-weighted pooled meanEach record has equal weightMean among the collected recordsSubjects with more records contribute more
Equal-subject meanEach subject-level summary has equal weightMean of subject-level summariesEach summary may be based on a different number of records
Design-aware or model-based summaryWeights or model reflect a specified target and sampling processDepends on the stated estimand and assumptionsRequires justification of the design or model assumptions
Common pitfall: “There are more records from this subject, so the pooled mean is more representative of the average subject.” That reasoning confuses the number of records with the number of subjects. If a subject contributes ten records, that subject receives ten times the weight of someone contributing one record in the pooled mean. To describe the average subject, define subject-level summaries and give subjects equal weight, or use another explicitly justified method for the intended target.

Informative sampling affects representation in the summary; it does not, by itself, make an individual concentration measurement inaccurate. Nor does unequal sampling alone prove that a pooled mean is biased for every possible target. The relevant question is whether the sampling pattern makes the record-weighted summary differ from the quantity the analysis intends to describe.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

  1. Two subjects have means of 6 and 12 mg/L and contribute 2 and 6 records. Calculate the equal-subject mean and the record-weighted mean.
  2. Can unequal record counts shift a pooled mean downward? Explain what pattern would produce that result.
  3. What does the equal-subject mean describe, and what does the record-weighted mean describe?
  4. Why is a table of record counts and subject-level means useful when assessing informative sampling?

Solutions. (1) The equal-subject mean is \((6+12)/2=9\) mg/L. The record-weighted mean is \((2(6)+6(12))/(2+6)=(12+72)/8=10.5\) mg/L. (2) Yes. If subjects with lower means contribute more records, they receive more weight and can pull the pooled mean downward, as in the worked example with means of 15, 10, and 5 mg/L. (3) The equal-subject mean averages subject-level summaries with each subject contributing equally; the record-weighted mean averages the collected records, so subjects with more records contribute more. (4) It helps show whether record counts are associated with subject-level values, a pattern that can shift the pooled mean relative to the equal-subject mean. Counts alone do not prove why that pattern arose.