Tutorials › Competing Risks and Multi-State Models › Martingale Residuals From the Compensator

Counting Processes and Martingale Theory · Tutorial 443 of 699

Martingale Residuals From the Compensator

Learn why subtracting a predictable compensator from a counting process removes its conditional drift, and how to verify the result in concrete event-history settings.

Advanced 10 min read

What You'll Learn

  • State the conditions under which a compensated counting process is an integrable martingale.
  • Derive the conditional mean-zero increment property from the compensator identity.
  • Distinguish a realized residual increment from its conditional expectation.
  • Calculate compensator residuals for Poisson, survival, and competing-transition processes.
  • Recognize why estimated compensators and unconditional centering need not give martingales.

Why Subtract a Compensator?

A counting process increases when events occur, so its increments usually have a systematic upward drift. The compensator records that predictable accumulation of event occurrence. Subtracting it leaves a process whose increments have conditional mean zero: given the observed history, the residual has no remaining predictable drift. This is the precise sense in which a compensated counting process is a martingale.

The distinction between a realized increment and its conditional expectation matters. A residual increment can be positive or negative on a particular event-history path. The martingale property says that its conditional expectation, given the past, is zero; it does not say every realized increment equals zero or that residual increments are independent.

Definition: Let \(N(t)\), \(t\geq 0\), be an adapted counting process with \(N(0)=0\), and let \(\mathcal{F}_t\) describe the information available by time \(t\). A compensator \(A(t)\) is a predictable, increasing process with \(A(0)=0\) such that \(M(t)=N(t)-A(t)\) is a martingale, under the required integrability conditions. The process \(M(t)\) is the compensated counting process, also called its martingale residual process.

For a true, integrable martingale on a finite observation horizon, require \(E|M(t)|<\infty\) at each time and \(E\{M(t)\mid\mathcal{F}_s\}=M(s)\) whenever \(s\leq t\). For the counting-process setting here, a standard sufficient setting is that \(N(t)\) and \(A(t)\) have finite expectations on the horizon and that the Doob–Meyer decomposition gives \(N=M+A\). If only local integrability is available, the compensated process may be a local martingale rather than a true martingale; the conditional-expectation identity must not be asserted without the corresponding integrability.

Deriving the Conditional Mean-Zero Property

The Doob–Meyer decomposition for a counting process writes \(N(t)=M(t)+A(t)\), where \(A\) is predictable and increasing and \(M\) is a martingale. Thus \(M=N-A\). The defining martingale condition has an equivalent increment form: for all \(s\leq t\),

$$ E\{M(t)-M(s)\mid\mathcal{F}_s\}=0. $$

Substituting \(M=N-A\) gives the result we will use throughout:

Theorem: If \(N-A\) is an integrable martingale, then for \(s\leq t\), $$ E\{N(t)-N(s)\mid\mathcal{F}_s\} = E\{A(t)-A(s)\mid\mathcal{F}_s\}. $$ Equivalently, \(E\{[N(t)-N(s)]-[A(t)-A(s)]\mid\mathcal{F}_s\}=0\). The compensator increment may itself be random after time \(s\); it is its conditional expectation that matches the conditional expected count increment.

The same conclusion can be seen from the compensator identity. For any bounded predictable process \(H(u)\), the expected accumulated weighted count and compensator agree: \(E\int H(u)\,dN(u)=E\int H(u)\,dA(u)\), when these expectations are finite. Take \(H(u)=\mathbf{1}_B\mathbf{1}_{(s,t]}(u)\), where \(B\in\mathcal{F}_s\). This is predictable, and the identity becomes

$$ E\{\mathbf{1}_B[N(t)-N(s)]\} = E\{\mathbf{1}_B[A(t)-A(s)]\}. $$

Since this holds for every event \(B\) in \(\mathcal{F}_s\), it is exactly the defining characterization of equality of the two conditional expectations given \(\mathcal{F}_s\). Subtracting the right side from the left yields the conditional mean-zero increment property. This argument explains why the result is conditional on the observed past, not merely an unconditional statement that the two processes have matching means.

Worked Examples

Worked Example: A Homogeneous Poisson Count

Suppose, hypothetically, events follow a homogeneous Poisson process with rate \(0.4\) events per day. Its compensator is \(A(t)=0.4t\) events. Consider an interval from day \(3\) to day \(5\), and let \(\mathcal{F}_3\) contain the event history through day 3.

The compensator increment is \(A(5)-A(3)=0.4(5-3)=0.8\) events. Poisson independent increments imply that the number of events in those two days is independent of \(\mathcal{F}_3\), with expected value \(0.8\). Therefore,

$$ E\{M(5)-M(3)\mid\mathcal{F}_3\} =E\{N(5)-N(3)-0.8\mid\mathcal{F}_3\} =0.8-0.8=0. $$

If exactly one event is observed during that interval, the realized residual increment is \(1-0.8=0.2\) events. That nonzero value does not contradict the martingale property: the property concerns the conditional expectation over possible future intervals, not the particular interval that occurred.

Worked Example: One Subject at Risk

Consider a hypothetical subject who is event-free at time \(s\), with a constant event hazard of \(0.2\) per month. Observe the next two months, with no loss to follow-up before the end of that interval. Let \(u\) denote time since \(s\). The at-risk indicator is \(Y(s+u)=1\) until the event and 0 afterward, and the compensator increment is \(A(s+2)-A(s)=\int_0^2 0.2Y(s+u)\,du\), measured in expected events.

Conditional on being event-free at \(s\), the event time has an exponential distribution with rate \(0.2\) per month. The count increment is 1 if an event occurs within two months and 0 otherwise, so

$$ E\{N(s+2)-N(s)\mid\mathcal{F}_s\} =1-e^{-0.2(2)} =1-e^{-0.4} \approx 0.3297\text{ events}. $$

The subject remains at risk at elapsed time \(u\) with conditional probability \(e^{-0.2u}\). Therefore the conditional expected compensator increment is

$$ E\{A(s+2)-A(s)\mid\mathcal{F}_s\} =\int_0^2 0.2e^{-0.2u}\,du =1-e^{-0.4} \approx 0.3297\text{ events}. $$

The two conditional expectations agree, so the residual increment has conditional expectation zero. Notice that the compensator increment on an individual path is not always \(0.4\) events: if the event occurs after \(u\) months, it is \(0.2u\); if no event occurs within two months, it is \(0.4\). Replacing the random at-risk indicator by 1 for the entire interval would incorrectly ignore that the subject leaves the risk set at the event.

Worked Example: Two Competing Transition Types

Suppose, hypothetically, a subject is in an initial state at time \(s\), and can make either of two transitions out of that state. While the subject remains in the initial state, the transition-specific intensities are constant: \(\lambda_1=0.2\) per year and \(\lambda_2=0.1\) per year. The subject leaves the risk set at the first transition. Over the next year, let \(N_1\) count transitions of type 1 and let \(A_1\) be its compensator.

The total rate of leaving the initial state is \(0.2+0.1=0.3\) per year, so the probability of still being at risk \(u\) years after \(s\) is \(e^{-0.3u}\). The probability that the first transition is type 1 and occurs within one year is the integral of its event density:

$$ E\{N_1(s+1)-N_1(s)\mid\mathcal{F}_s\} =\int_0^1 0.2e^{-0.3u}\,du =\frac{0.2}{0.3}(1-e^{-0.3}) \approx 0.1728\text{ transitions}. $$

The type-1 compensator accumulates at rate \(0.2\) only while the subject remains in the initial state. Its conditional expectation is therefore

$$ E\{A_1(s+1)-A_1(s)\mid\mathcal{F}_s\} =\int_0^1 0.2e^{-0.3u}\,du \approx 0.1728\text{ transitions}. $$

Thus \(N_1-A_1\) has a conditional mean-zero increment. The same calculation for type 2 gives \((0.1/0.3)(1-e^{-0.3})\approx0.0864\) transitions for both its expected count and expected compensator increment. The calculation depends on using the total rate \(0.3\) for remaining in the initial state, not just the type-1 rate.

Assumptions, Diagnostics, and Common Errors

The argument applies to a specified filtration and a compensator appropriate to that filtration. In applications, the event process must be adapted, the compensator predictable, and the decomposition integrable on the time range where a true martingale claim is made. If an intensity \(\lambda(t)\) is used, it is the conditional event rate given the history just before \(t\); for an at-risk process \(Y(t)\), a typical compensator has the form \(A(t)=\int_0^t Y(u)\lambda(u)\,du\). The at-risk process and intensity must represent the observation and event mechanism being modeled. Examples above also assume the stated exponential or constant-rate structure and, for the competing transitions, that only one first transition can occur.

A useful diagnostic question is whether the proposed \(A\) reflects the information available just before each possible event. If it uses future information, it is not a predictable compensator. Another warning sign is an at-risk indicator that remains 1 after a terminal event or censoring time when observation has ended. A mathematically correct conditional-mean identity for the wrong risk process does not describe the intended event history.

The compensated process differs from two tempting alternatives. The raw count \(N(t)\) retains its upward drift. Subtracting the unconditional mean function \(E\{N(t)\}\) may center the process at each fixed time, but does not generally make its increments have conditional expectation zero given \(\mathcal{F}_s\). The compensator is history-adapted and predicts the accumulated event occurrence relative to the filtration; that conditional role is what gives \(N-A\) its martingale property.

A common mistake is to say, “The compensator increment is the conditional expected count increment, so \(A(t)-A(s)\) is already known at time \(s\).” The first part is imprecise and the second is generally false. The correct statement is \(E\{N(t)-N(s)\mid\mathcal{F}_s\}=E\{A(t)-A(s)\mid\mathcal{F}_s\}\). The future compensator increment can be random, as the at-risk example shows. Another mistake is to infer that residual increments are independent or have constant variance. Neither property follows from the martingale condition alone.

Finally, the result is exact for the compensator of the process under the stated model. In practice, an intensity is often estimated and substituted into a compensator. The resulting residual may be useful for assessing a model, but it is not automatically an exact martingale: estimation, model misspecification, and the chosen filtration can affect its behavior. Martingale residuals also need not be bounded or have a fixed scale, so a zero conditional mean is not by itself a complete diagnostic of model fit.

Key takeaway: For a counting process with integrable compensator \(A\), subtracting \(A\) removes the predictable conditional drift: \(M=N-A\) satisfies \(E\{M(t)-M(s)\mid\mathcal{F}_s\}=0\). This is a statement about conditional expectations, not about individual paths or independent increments.

Check Your Understanding

Answer each question, then compare your reasoning with the short solutions.

  1. A counting process has compensator \(A(t)=0.3t\) events. What is the compensator increment from time 4 to time 7?
  2. In the two-month survival example, why is the expected compensator increment not calculated as \(0.2(2)\) without accounting for the at-risk indicator?
  3. Does a realized residual increment of \(-0.4\) events contradict the martingale property? Explain.
  4. Why does subtracting \(E\{N(t)\}\) not generally establish the conditional mean-zero increment property?

Solutions. (1) \(A(7)-A(4)=0.3(7-4)=0.9\) events. (2) The subject stops contributing to the compensator after the event, so the risk indicator can fall to 0 before two months. Its expected contribution is \(0.2\int_0^2e^{-0.2u}\,du\approx0.3297\) events, not \(0.4\). (3) No. A pathwise residual can be nonzero; the requirement is that its conditional expectation given the past is zero. (4) An unconditional mean does not generally account for the history available at time \(s\). Conditional mean-zero increments require matching conditional expectations of the count increment and its compensator increment.

Clinical Biostats