Why Audit Counts and Risk Sets?
An Aalen-Johansen cumulative incidence estimate depends on two things at every event time: how many individuals are at risk immediately before that time, and how many experience each event cause then. If either quantity is wrong, the resulting increments can be wrong even when the estimator’s formula is applied correctly. Auditing the counts is therefore a practical check on the data-to-estimate calculation, not merely a tidy way to present it.
The audit follows each distinct recorded follow-up time in order. At each time, ask whether the risk set agrees with the observed event and censoring counts, and whether the next risk set can be obtained by removing those individuals and accounting for any new delayed entrants. This checks bookkeeping. It does not, by itself, establish that censoring is independent or that the event and censoring records are accurate.
Definitions and Counting Convention
Suppose each individual can experience one of \(K\) mutually exclusive event causes or be right-censored. Let \(T_i\) be the observed follow-up time for individual \(i\), and let \(J_i\) be the observed status: \(J_i=j\) means an event from cause \(j\), while \(J_i=0\) means censoring. At a distinct time \(t_k\), define \(Y_k\) as the number at risk immediately before \(t_k\), \(d_{kj}\) as the number of cause-\(j\) events at \(t_k\), and \(c_k\) as the number censored at \(t_k\). The total number of events at that time is \(d_k=\sum_{j=1}^K d_{kj}\).
This convention is essential when an event and censoring occur at the same recorded time. Both individuals were under observation just before that time, so both belong in the denominator \(Y_k\). A censoring at \(t_k\) is not removed from the risk set before calculating the event fraction at \(t_k\).
With delayed entry, let \(L_i\) be individual \(i\)'s entry time and use the convention that an individual is at risk at \(t\) if \(L_i<t\le T_i\). Then \(Y(t)=\sum_i I(L_i<t\le T_i)\). Individuals who enter after one distinct time and before the next contribute to the later risk set, not the earlier one. If entry occurs exactly at an event time, the study’s time-origin and entry convention must specify whether that individual is considered at risk then; the convention must be applied consistently.
For the Aalen-Johansen calculation, the cause-\(j\) increment at \(t_k\) is \(S(t_k-)\,d_{kj}/Y_k\), where \(S(t_k-)\) is the estimated event-free probability immediately before \(t_k\). The event-free estimate is updated by multiplying its previous value by \(1-d_k/Y_k\). A censor-only time has \(d_k=0\): it changes later risk sets, but contributes no event increment at that time.
A Count Audit in Practice
A useful audit starts from the person-level records, not from the already-computed estimate. Group the records by distinct observed time, count each event cause and censoring status, then compare the computed counts with the risk-set size implied by the records. The following sequence makes the checks explicit.
Include times with censoring as well as times with events, because censoring changes subsequent risk sets.
Record \(d_{k1},\ldots,d_{kK}\) and \(c_k\). Verify that each individual has only one recorded status at a given observed time.
Count individuals under observation immediately before the time, applying the stated entry convention if there is delayed entry.
Without entry between times, subtract all events and censorings to obtain the next risk set. With entry, add the eligible entrants at the appropriate time.
At event times, confirm that every cause-specific fraction uses \(Y_k\), and that the event-free update uses the total \(d_k\), not one cause count alone.
Worked Example: Reconciling a Small Competing-Risks Table
Consider hypothetical follow-up of 10 individuals, with two possible event causes. There is no delayed entry. The grouped records are shown below. The “risk set before” column is to be checked against the records, not assumed to be correct.
| Time | Risk set before | Cause 1 events | Cause 2 events | Censored |
|---|---|---|---|---|
| 1 | 10 | 2 | 1 | 1 |
| 2 | 6 | 1 | 0 | 2 |
| 3 | 3 | 0 | 1 | 0 |
At time 1, \(d_1=2+1=3\), so the number remaining under observation afterward is \(10-3-1=6\). This agrees with the listed risk set at time 2. At time 2, \(d_2=1+0=1\), giving \(6-1-2=3\), which agrees with the listed risk set at time 3. After time 3, \(3-1-0=2\) individuals remain under observation. Across the three times, the counts account for \(3+1+1=5\) events and \(1+2+0=3\) censorings; \(5+3+2=10\), matching the starting cohort.
For illustration, the cause-1 cumulative incidence update also uses these audited counts. Starting with \(S(0)=1\) and \(F_1(0)=0\), at time 1 the event-free estimate becomes \(1(1-3/10)=0.7000\), and \(F_1(1)=0+1(2/10)=0.2000\). At time 2, the cause-1 increment is \(0.7000(1/6)=0.1167\), so \(F_1(2)=0.3167\), while \(S(2)=0.7000(1-1/6)=0.5833\). At time 3 there is no cause-1 event, so its increment is zero. The example shows why a count audit matters: the risk-set transitions and each cause-specific numerator have to agree with the underlying records.
Why the Risk-Set Identity Holds
The no-entry identity follows directly by tracking each person’s contribution to the risk set. Just before \(t_k\), every person counted in \(Y_k\) is either still under observation after \(t_k\), has an event at \(t_k\), or is censored at \(t_k\). These categories are disjoint under the one-status-per-person convention. The latter two groups have sizes \(d_k\) and \(c_k\), so removing them leaves \(Y_k-d_k-c_k\) individuals.
This identity assumes there is no new entry between \(t_k\) and \(t_{k+1}\). If \(e_{k+1}\) people enter after \(t_k\) but before \(t_{k+1}\), and no other enrollment changes occur, then
The same reasoning applies to every interval. A negative reconstructed risk set, or a risk set larger than the prior one without entry, signals inconsistent counts, an incorrect ordering, or a mismatch in the way entry and time ties were handled.
Worked Example: Events and Censoring Tied at One Time
In hypothetical data, \(Y_k=8\) individuals are under observation just before day 4. At day 4, one has a cause-1 event, two have cause-2 events, and two are censored. Thus \(d_k=1+2=3\), and the correct post-time count is \(8-3-2=3\).
The event-free update uses all three events over the full risk set: \(S(4)=S(4-)(1-3/8)=S(4-)(5/8)\). The cause-1 cumulative incidence increment is \(S(4-)(1/8)\), and the cause-2 increment is \(S(4-)(2/8)\).
A mistaken procedure might remove the two censored individuals first and use \(8-2=6\) as the event denominator. That would produce a total event fraction \(3/6=0.5000\) rather than the correct \(3/8=0.3750\). It also leaves only \(8-2-3=3\) people after the time, so the post-time count does not justify the incorrect denominator: the denominator must represent those at risk immediately before day 4. Tied censoring affects the later risk set, not the denominator for events recorded at the same time.
Worked Example: Auditing Delayed Entry
Suppose a hypothetical study enrolls seven individuals, but only five have entered before time 1. The other two enter after time 1 and before time 2. At time 1, among the five at risk, there is one cause-1 event and one censoring. The post-time count is \(5-1-1=3\). Adding the two eligible entrants gives \(Y_2=3+2=5\).
At time 2, suppose one individual has a cause-2 event and one is censored. Then \(d_2=1\), \(c_2=1\), and the count remaining after time 2 is \(5-1-1=3\). The complete transition checks are therefore
Using the original cohort size, seven, as the time-1 risk set would be incorrect: two individuals had not yet entered and were not under observation then. Conversely, omitting the two entrants from the time-2 risk set would also be incorrect. The entry records and the risk-set rule must agree at every time.
What an Audit Can and Cannot Establish
A successful reconciliation supports the internal consistency of the grouped counts and the denominators supplied to the estimator. It does not prove that a recorded event cause is correct, that follow-up times were measured accurately, or that the censoring assumption required for estimation holds. The independent censoring condition is a statistical assumption about the relationship between censoring and future event experience; arithmetic agreement cannot verify it.
The audit is also useful for locating the source of a discrepancy. A mismatch in \(Y_k\) may result from excluding someone censored at \(t_k\), including someone who entered later, or failing to carry a person forward after a censor-only time. A mismatch in \(d_k\) may reflect an omitted cause or a status coded as both event and censoring. Checking the individual records at the first time where the identities fail is usually more informative than inspecting only the final cumulative incidence.
| Approach | What it checks | Limitation |
|---|---|---|
| Person-level reconstruction | Who is at risk and what status each person has at each time | Requires access to correctly recorded individual-level times, statuses, and entry times |
| Grouped count reconciliation | Whether \(Y_k\), event counts, censoring counts, and transitions agree | Cannot identify which individual record caused a discrepancy without returning to person-level data |
| Checking only the final estimate | Whether the final output is numerically available | May conceal an incorrect risk set or event count at an earlier time |
These checks complement rather than replace other approaches. Person-level reconstruction is the strongest way to investigate discrepancies; grouped counts offer a compact repeatable audit once the records have been validated. A formula-level check of the estimator can confirm that the grouped counts were used consistently, but it cannot show that those counts match the actual records.
Common Mistakes and Diagnostics
- Removing tied censorings before counting events. This gives an artificially small denominator. Include everyone with an event or censoring at \(t_k\) in the risk set immediately before \(t_k\).
- Skipping censor-only times. They do not create a cumulative-incidence jump, but they reduce the risk set for later times and must be included in the reconciliation.
- Using one cause’s event count in the event-free update. The event-free probability is reduced by all causes at that time: use \(d_k=\sum_j d_{kj}\).
- Applying a no-entry identity when there is delayed entry. Add eligible entrants before calculating the later risk set, using the prespecified entry convention.
- Treating a successful count check as evidence for independent censoring. It is only a bookkeeping check; the censoring assumption requires substantive assessment and study-design knowledge.
A practical warning sign is a risk set that increases without recorded entry, a total event-plus-censoring count larger than the risk set, or a next risk set that cannot be obtained from the preceding one. When any of these occurs, pause before interpreting the estimate. Check the first failing time, its tied statuses, the entry records, and whether the displayed risk set is a before-time or after-time count.
Check Your Understanding
Answer each question, then compare with the solutions below.
- At a time with \(Y_k=12\), there are two cause-1 events, one cause-2 event, and three censorings. What is the post-time risk-set count if there is no entry before the next time?
- Why are individuals censored at \(t_k\) included in the risk set used for event fractions at \(t_k\)?
- A censor-only time has \(Y_k=9\) and \(c_k=2\). What happens to the risk set afterward, and does the cumulative incidence jump at that time?
- With delayed entry, the prior post-time count is 4 and three people enter before the next time. At that next time, one event and one censoring occur from a risk set of 7. How many remain afterward?
Solutions. (1) The total event count is \(d_k=2+1=3\), so the post-time count is \(12-3-3=6\). (2) They are observed immediately before \(t_k\); censoring removes them after that time, not before its event denominator is formed. (3) The risk set becomes \(9-0-2=7\), while the cumulative incidence does not jump because there is no event at that time. (4) The risk set at the next time is \(4+3=7\); after the event and censoring it is \(7-1-1=5\).