Why Incomplete Follow-Up Matters
A study may begin with a clearly defined cohort but end with outcome information for only some of its members. People may leave a program, change providers, stop responding to surveys, or become unobservable in an administrative system. This is attrition: loss of study participants or entities from observation before the planned follow-up is complete.
Attrition always reduces the amount of observed information. It can also change who remains represented in the data. If the chance of remaining under observation is related to outcome risk, the retained records may describe a different population from the original cohort. A rate calculated only from retained records can then differ systematically from the rate for the cohort that entered the study.
These definitions assume the outcome is well-defined over a common, specified horizon for every member of the cohort. If a person leaves before that horizon and their eventual outcome is unknown, the observed data do not by themselves reveal whether that person had the outcome. This tutorial focuses on what attrition can do to the included population and to a rate based on retained records; it does not treat an unknown outcome as a known non-event.
How Retention Can Change an Observed Rate
First consider the population quantities underlying the rates. Let \(p=P(Y=1)\) be the original-cohort outcome probability, \(r_1=P(R=1\mid Y=1)\) the retention probability for outcome cases, and \(r_0=P(R=1\mid Y=0)\) the retention probability for non-cases. If \(P(R=1)>0\), the population outcome rate among retained entities is
The numerator is the probability of both an outcome and retention. The denominator is the probability of retention, obtained by adding the retained cases and retained non-cases. Dividing the first probability by the second gives the outcome probability conditional on retention.
This is a population formula. It also gives the ratio of expected retained case counts to expected retained total counts for a cohort of size \(N\). It is not generally the exact expected value of a finite-sample rate, because a finite-sample rate has a random denominator. That distinction matters when a sample is small or retention is uncertain.
Retention can be related to outcome risk in more than one way. If higher-risk groups are less likely to remain, the mix of risk groups changes even when, within each group, retention is unrelated to the outcome. Retention can also depend on outcome status itself. Both mechanisms can make the retained-record rate unrepresentative of the original-cohort rate.
Worked Example: Risk-Related Attrition Changes the Cohort Mix
A hypothetical study follows 200 members of a service program. Of 80 high-risk members, 20 experience the outcome and 60 do not. Of 120 low-risk members, 6 experience the outcome and 114 do not. In the high-risk group, 10 outcome cases and 30 non-cases remain; all 120 low-risk members remain. Assume retention within each risk group is unrelated to outcome status. Calculate and compare the original-cohort and retained-record rates.
Method. Calculate the event count and denominator in the original cohort, then repeat for retained records. Check the rates separately within each risk group to see whether a change in group composition explains the difference.
In the original cohort, the outcome count is \(20+6=26\), and the cohort size is \(80+120=200\). Thus the original-cohort rate is \(26/200=0.1300\).
Among retained high-risk members, there are \(10+30=40\) people and 10 outcomes. Among retained low-risk members, there are \(6+114=120\) people and 6 outcomes. Therefore, the retained-record rate is
The group-specific rates are unchanged: the high-risk rate is \(20/80=10/40=0.2500\), and the low-risk rate is \(6/120=0.0500\). What changed was the mix. The high-risk group made up \(80/200=0.4000\) of the original cohort but only \(40/160=0.2500\) of retained records. Because the high-risk group has the larger outcome rate, its smaller representation pulls down the retained-record rate.
Check. The retained counts add to 160, and the retained outcomes add to 16. The retained-record rate is therefore 0.1000, below the original-cohort rate of 0.1300. The common mistake is to infer that attrition changed the risk within each group: here it changed the group composition, not either group-specific rate.
Attrition Related Directly to Outcome Status
Even if a cohort has only one risk group, a retained-record rate can be biased when outcome cases and non-cases have different retention probabilities. In the formula above, if \(r_1=r_0\), the common retention factor cancels and the population retained rate equals \(p\). If \(r_1\ne r_0\), the factor does not generally cancel.
This does not mean the analyst can usually observe \(r_1\) and \(r_0\): if outcomes are unknown for people lost to follow-up, their outcome status is precisely what is missing. The formula is useful for diagnosing the mechanism and its direction when the relevant probabilities are known or posited, not as a calculation that can be made from retained records alone.
Worked Example: Outcome-Related Retention
A hypothetical cohort has 200 members. By the defined study horizon, 40 have the outcome and 160 do not. Twenty outcome cases and 144 non-cases remain under observation with their outcomes recorded. Calculate the original-cohort and retained-record rates, and verify the retained rate using the population formula with the stated retention proportions.
Method. Use the known full-cohort counts for the original rate and the retained counts for the observed rate. Then substitute the outcome-specific retention proportions into the formula. This comparison assumes the counts describe the same fixed-horizon outcome definition.
The original-cohort rate is \(40/200=0.2000\). The retained-record denominator is \(20+144=164\), so its outcome rate is
The outcome-case retention proportion is \(r_1=20/40=0.5000\). The non-case retention proportion is \(r_0=144/160=0.9000\). Substituting \(p=0.2000\) gives
The formula agrees with the retained counts. Since outcome cases are less likely to be retained than non-cases, the retained-record rate is below the original rate. The common mistake is to divide the 20 retained cases by all 200 original members; that gives \(0.1000\), but it treats the 36 people lost to follow-up as known non-cases. Their outcomes are not in the retained-record denominator, and their statuses cannot be silently assigned.
A Finite-Sample Rate Is a Random Ratio
The population formula describes a conditional probability, or equivalently a ratio of expected counts. It should not be described as the exact expected observed rate for every finite cohort. To see why, consider a fixed two-person cohort containing one certain case and one certain non-case. Suppose their retention events are independent, with probabilities 0.9000 for the case and 0.1000 for the non-case. Define the observed rate only when at least one person is retained.
Worked Example: Expected Rate Versus Ratio of Expected Counts
Find (1) the ratio of expected retained case count to expected retained total count, and (2) the exact expected retained-record rate conditional on at least one person being retained.
Method. Enumerate all possible retained sets. For the exact expectation, weight each defined rate by its probability and condition on a nonempty retained set.
The probability that both people are retained is \((0.9000)(0.1000)=0.0900\), giving rate \(1/2=0.5000\). The probability that only the case is retained is \((0.9000)(0.9000)=0.8100\), giving rate 1. The probability that only the non-case is retained is \((0.1000)(0.1000)=0.0100\), giving rate 0. The probability that neither is retained is \((0.1000)(0.9000)=0.0900\); the rate is undefined in that outcome.
The probability of at least one person being retained is \(0.0900+0.8100+0.0100=0.9100\). The exact conditional expected rate is therefore
By contrast, the expected retained case count is \(0.9000\), and the expected retained total count is \(0.9000+0.1000=1.0000\). Their ratio is \(0.9000/1.0000=0.9000\), not \(0.9396\). The denominator of the sample rate varies, so the expectation of that ratio need not equal the ratio of expected counts. The check is that the exact calculation excludes the no-retention outcome by conditioning on a defined rate; ignoring that condition would use the wrong denominator.
Diagnosing Attrition in a Predictive Data Set
A practical assessment begins by defining the cohort, the outcome horizon, and what it means for an entity to be retained. Then compare retention across characteristics available for the full cohort, especially variables associated with outcome risk. If high-risk members are retained less often, the retained data may underrepresent high-risk experience even if there is no further outcome-dependent loss within those groups.
Where outcomes are observed for some entities before they leave, comparing retention by those observed outcomes can reveal a warning sign. It does not establish the eventual outcome for those lost before the horizon. Likewise, similarity on recorded characteristics cannot prove that attrition is unrelated to unrecorded risk. A finding of no obvious difference in observed retention patterns is evidence to assess, not a guarantee that retained records represent the original cohort.
State which entities entered the cohort and the period over which the outcome is defined. Changing either definition changes the rate being estimated.
Count retained and lost entities overall and, where possible, by baseline risk characteristics and known outcome information.
Check whether the retained group has a different mix of characteristics associated with outcome risk. Separate within-group differences from changes in group proportions.
Report the denominator used and explain what is unknown for those lost. Do not present a retained-record rate as the original-cohort rate without an appropriate justification.
For predictive work, attrition can affect both the outcome distribution and the relationships between predictors and outcome. A model trained on retained records may learn patterns for the people who remain observable rather than for the intended deployment population. The issue is therefore not solved merely by having many rows: a large retained sample can still represent a systematically selected subset.
A clear analysis distinguishes three statements: how many entities were lost; how the retained and original populations differ on known characteristics; and what assumptions would be needed to generalize the observed outcome rate. These statements support a cautious interpretation without claiming that missing outcomes have been recovered.
Common Exam Traps
- Using the wrong denominator. The retained-record rate uses retained cases divided by all retained entities, not by the original cohort size.
- Treating loss as a non-event. An entity without an observed outcome is not automatically a non-case. State whether the outcome is actually known.
- Assuming attrition only reduces precision. It reduces observed sample size, but risk-related retention can also change the represented population and shift a rate.
- Confusing a ratio with an expectation. A ratio of expected counts is not generally the expected value of a finite-sample rate with a random denominator.
- Assuming equal rates in observed strata prove no bias. Unmeasured risk or outcome-related retention can still matter; observed comparisons do not establish that all lost entities are comparable.
Check Your Understanding
For each problem, state which population the rate describes. Round rates to four decimal places.
- A hypothetical cohort contains 300 people, 45 of whom experience an outcome. At the end of follow-up, 30 of those cases and 204 non-cases have recorded outcomes. Find the original-cohort rate and the retained-record rate. (Answer: Original cohort \(45/300=0.1500\); retained records \(30/(30+204)=0.1282\).)
- A cohort has 100 high-risk members with 30 outcomes and 200 low-risk members with 20 outcomes. Sixty high-risk members remain, including 18 cases; all low-risk members remain. Find the original and retained rates, and say what changed. (Answer: Original \(50/300=0.1667\); retained \(38/260=0.1462\); the high-risk group is less represented among retained members.)
- For a fixed-horizon outcome, suppose \(p=0.1200\), \(r_1=0.7500\), and \(r_0=0.9500\). Calculate the population rate among retained entities using the conditional-rate formula. (Answer: \((0.1200)(0.7500)/[(0.1200)(0.7500)+(0.8800)(0.9500)]=0.0900/0.9260=0.0972\).)
- A cohort has 250 members, including 50 outcome cases. Of the cases, 30 remain; of the 200 non-cases, 160 remain. Calculate both rates and explain the direction of the difference. (Answer: Original \(50/250=0.2000\); retained \(30/190=0.1579\); cases have lower retention than non-cases, so the retained rate is lower.)
- A small cohort consists of one certain case and one certain non-case. Their independent retention probabilities are 0.8000 and 0.2000, respectively. Find the exact expected retained rate conditional on at least one retention. (Answer: Both retained has probability \(0.1600\) and rate \(0.5000\); case only has probability \(0.6400\) and rate 1; non-case only has probability \(0.0400\) and rate 0. Conditional expectation \(=(0.1600(0.5000)+0.6400)/0.8400=0.7200/0.8400=0.8571\).)