From an Outcome Concept to a Measurement Rule
A protocol may ask whether a treatment strategy reduces severe asthma exacerbations. That concept is clinically meaningful, but it does not yet tell an emulation what to measure. Does an exacerbation mean an emergency visit, a course of oral corticosteroids, or either? Which records establish that it occurred? What dates count as being within follow-up? If a patient has several exacerbations, is the outcome the first one or the total number? Without explicit answers, two analysts can apply the same protocol and produce different outcomes.
The earlier material on choosing follow-up horizons and endpoints distinguishes the time horizon from the outcome itself. Here the task is to make the endpoint operational: specify a measurement source, an ascertainment window, an event definition, and rules for recurrent events where relevant. These rules should describe the same clinical construct under every treatment strategy. Outcome definition is a protocol decision, not a choice to make after seeing which rule produces a preferred result.
The value \(Y^a\) depends on the outcome rule, not just on a clinical label. For example, “hospitalization for heart failure within 180 days” needs a source for hospitalization and a rule for what counts as heart failure. A date window also needs a boundary convention: “days 0 through 180, inclusive” is more precise than “within six months.” If a different source, event rule, or window is used for another strategy, the resulting values no longer represent the same endpoint in the same way.
Four Elements to Specify
- Measurement source. Identify where evidence of the outcome will come from, such as inpatient discharge records, a registry, laboratory results, or adjudicated clinical records. If several sources are combined, say how records from them are reconciled.
- Ascertainment window. State when observations count, including the start point and endpoint. Use the time origin and horizon established by the protocol, and make the inclusion of boundary dates explicit.
- Event definition. State the criteria that make an observation qualify. Depending on the endpoint, criteria may include a diagnosis code, a procedure, a laboratory threshold, or a combination of evidence. A broad label alone is not a reproducible rule.
- Recurrent-event rule. Decide whether the outcome is the first event, any event, a number of episodes, or another summary. If multiple records can describe one episode, specify how they are grouped and what makes a later event a new episode.
These decisions should be clinically defensible and feasible with the intended data. More detail is not automatically better: an elaborate rule that cannot be applied consistently to the available records is not an operational definition. Conversely, a simple rule should not silently substitute a convenient data field for the outcome concept of interest.
Worked Example: Defining a First Hospitalization Outcome
Suppose a hypothetical protocol compares two post-discharge care strategies and asks whether a patient has an unplanned readmission within 30 days after discharge. For this example, day 0 is the discharge date, and the window is days 1 through 30 inclusive.
Source and event rule. Use inpatient discharge records. Count a new inpatient admission as a qualifying readmission if it is recorded as unplanned and the principal diagnosis is in a prespecified set of diagnoses for the condition of interest. A planned admission does not qualify. The protocol must list the diagnosis criteria rather than relying only on the phrase “related admission.”
Outcome value. Let \(Y=1\) if at least one qualifying admission begins on days 1 through 30, and \(Y=0\) otherwise, assuming the record source provides outcome information across that window. A qualifying admission beginning on day 30 counts; one beginning on day 31 does not.
Applying the rule. In hypothetical records, Patient A has a qualifying admission beginning on day 12, so \(Y=1\). Patient B has an admission on day 8 that is classified as planned and no other qualifying admission, so \(Y=0\). Patient C has no qualifying admission through day 30, so \(Y=0\) under the stated observation assumption. These assignments follow from the rule, not from an informal judgment about whether a patient seemed unwell.
This is a first-or-any-event binary endpoint: it records whether at least one event occurred, not how many admissions occurred or how long the patient remained in hospital. Those are different outcome definitions.
Defining Recurrent Events Without Ambiguity
For a condition that can recur, “number of events” is incomplete unless the protocol explains what counts as one event. A single episode may generate several claims, visits, or diagnoses; counting each record as a separate event can inflate the apparent burden. Alternatively, grouping records too broadly can combine clinically distinct episodes. The protocol should set the grouping rule before analysis.
A recurrent-event definition should specify whether the summary is a count of episodes during the window, an indicator for one or more episodes, or another explicitly defined measure. It should also state the date used to locate an episode in the window—for example, the first qualifying encounter date—and how events near the window boundary are treated. This tutorial defines the outcome; it does not prescribe how the resulting recurrent-event data should be analyzed.
Worked Example: Counting Recurrent Exacerbation Episodes
Consider a hypothetical outcome of acute exacerbation episodes during days 1 through 60. The source is emergency-department and urgent-care records. An encounter qualifies if it has a prespecified acute-exacerbation diagnosis. To avoid counting several records for one episode as separate episodes, define a new episode when a qualifying encounter date is more than 14 elapsed days after the date of the previous episode’s first qualifying encounter. This rule is stated in elapsed date differences; it does not require counting event-free calendar days.
Patient D has qualifying encounter dates on days 4, 19, and 36. The date differences are \(19-4=15\) days and \(36-19=17\) days. Since both differences are greater than 14 days, each later encounter begins a new episode. Patient D has 3 episodes.
Patient E has qualifying encounters on days 4 and 18. The difference is \(18-4=14\) days, which is not greater than 14. Under this rule, the second record is grouped with the existing episode, and the count remains 1. If Patient E instead has another qualifying encounter on day 19, the difference from day 4 is \(19-4=15\) days, so that encounter begins a second episode.
This illustration exposes why a boundary rule needs exact wording and arithmetic. “More than 14 elapsed days apart” assigns day differences of 15 or more to a new episode; it is not interchangeable with “more than 14 event-free calendar days.” The protocol must use one precise rule and apply it consistently.
Why Comparable Assessment Matters
Outcome assessment should use the same definition and, as far as possible, comparable opportunities for detection under each strategy. Otherwise a difference in recorded outcomes can reflect different measurement as well as a difference in the underlying clinical event. The basic point can be shown with a short derivation.
Let \(Y\) denote whether the true event occurred and \(Y^*\) whether the event is recorded by the chosen measurement process. Suppose, for illustration, that the probability of recording an event when it truly occurred is sensitivity \(Se_a\) under strategy \(a\), and the probability of a false positive is \(1-Sp_a\), where \(Sp_a\) is specificity. Then:
The first term counts recorded events among people with the true event; the second counts false positives among those without it. If the true event probability is \(0.10\) in both strategies, specificity is 1 in both, but sensitivity is \(0.90\) under one strategy and \(0.60\) under the other, the recorded event probabilities are \(0.90(0.10)=0.09\) and \(0.60(0.10)=0.06\). The recorded difference is 0.03 even though the true probabilities are equal. This is a measurement illustration, not a causal-effect estimate.
In practice, measurement opportunities can differ if one strategy leads to more visits, tests, or contact with clinicians. For a laboratory-defined outcome, for instance, more frequent testing can increase the chance of detecting an abnormal value. A protocol can address this concern by specifying a common assessment schedule or a source and event rule that do not depend on differential testing, when feasible. If comparability cannot be assured, the limitation should be explicit rather than hidden in the endpoint label.
Worked Example: A Laboratory Threshold and Unequal Testing
Suppose a hypothetical protocol defines acute kidney injury as a serum creatinine increase of at least \(0.3\) mg/dL from a prespecified baseline value during days 1 through 30. The source is laboratory records, and an event date is the first test date meeting the threshold. The same baseline definition, threshold, and window are to be used for both strategies.
For Patient F, the baseline creatinine is \(1.0\) mg/dL. A result of \(1.2\) mg/dL on day 10 is an increase of \(1.2-1.0=0.2\) mg/dL, so it does not qualify. A result of \(1.3\) mg/dL on day 20 is an increase of \(1.3-1.0=0.3\) mg/dL, so it meets the threshold and the outcome is recorded as present.
For Patient G, the baseline is also \(1.0\) mg/dL, but the only follow-up test is \(1.2\) mg/dL on day 30. The observed result does not meet the threshold. It would be incorrect to infer from this rule alone that no qualifying abnormality occurred between tests: the endpoint is based on available laboratory measurements, and the patient had fewer opportunities for detection. If testing frequency differs systematically across strategies, the recorded outcome may not be comparably ascertained. The protocol should anticipate that limitation and choose a feasible, defensible measurement approach.
Diagnostics, Alternatives, and Limitations
Before applying an outcome rule, check whether another person could reproduce it from the protocol and the specified data. Useful diagnostic questions include:
- Are the start and end dates explicit, and is it clear whether events on those dates count?
- Could two records for one clinical episode be counted twice? If so, is there a prespecified grouping rule?
- Does the source capture the event similarly under each strategy, or could visits and testing differ?
- Are the criteria specific enough to distinguish qualifying from nonqualifying events, including planned versus unplanned care where relevant?
- Does the proposed endpoint match the clinical concept, or merely what happens to be easy to extract?
Several approaches can be reasonable, but they answer different questions. A first-event indicator is straightforward and focuses on whether any event occurred by the end of follow-up; it discards information about subsequent episodes. A recurrent-event count retains episode burden but requires careful episode grouping and can be strongly influenced by a small number of people with many events. A composite endpoint can capture several clinically relevant events in one indicator, but it needs a complete list of components and can obscure which component accounts for a result. Reporting component-specific outcomes can clarify that composition, though it does not eliminate measurement problems.
No single source is universally best. Administrative records may provide broad coverage but depend on coding and care-seeking. Laboratory records can provide objective values but only when tests are taken. Adjudication can apply detailed clinical criteria but requires access to relevant records and consistent review procedures. The choice should balance clinical meaning, reproducibility, coverage, and comparable assessment.
A common mistake is to treat “no record of an event” as automatically equivalent to “the event did not occur.” That conclusion is justified only if the source and observation process support it. A second mistake is to let each strategy have a different event definition—for example, counting a diagnosis code under one strategy and a laboratory threshold under the other. The correction is to define one outcome rule and apply it to both strategies, while documenting any unavoidable differences in data capture. Detailed rules for incomplete follow-up and missing outcome information require separate prespecification.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A protocol defines a 90-day outcome as “infection.” Name two details needed to make this endpoint operational.
- Under a recurrent-event rule, a new episode starts only when qualifying encounter dates are more than 10 elapsed days apart. An encounter occurs on day 3 and another on day 13. Are there one or two episodes?
- Why can a laboratory-defined endpoint be recorded more often under a strategy that results in more frequent testing, even if the underlying event rate is the same?
- What information is lost when the outcome is “at least one event” rather than the number of recurrent episodes?
Solutions. (1) Specify, for example, the measurement source and precise qualifying infection criteria; also state the 90-day window boundaries and the date rule. (2) The elapsed difference is \(13-3=10\) days. Because 10 is not more than 10, the second encounter does not start a new episode under the stated rule, so there is one episode. (3) More tests create more opportunities to detect a threshold-crossing result, so recorded outcome frequency can differ because of ascertainment rather than only because of underlying event occurrence. (4) The indicator distinguishes no event from one or more events but does not distinguish one episode from several.