Why Overall Balance Can Hide a Temporal Gap
A trial can enroll similar numbers of participants in its two treatment arms overall and still have very little representation of one treatment during part of enrollment. This matters when prognosis, clinical practice, or outcome measurement changes over calendar time: an analysis that compares treatments across the whole trial may then rely heavily on comparisons between participants enrolled at different times. The earlier material on positivity under temporal allocation shifts explains why near-absent treatment options can undermine comparisons. Here the task is diagnostic: locate and describe sparse treatment representation along the enrollment timeline.
No single summary answers every question. Period-specific counts show how many participants actually received each treatment. The allocation probabilities recorded by the randomization system show how strongly the design favored one treatment at each assignment. Outcome-support summaries show how much observed outcome information is available in each treatment-by-period cell. These summaries should be considered together, not collapsed into one overall balance statistic.
A period might be a prespecified calendar block, such as a month or quarter, or a clinically meaningful enrollment epoch. The choice should be made with the trial’s enrollment rate and expected changes in mind: very wide periods can hide short-lived gaps, while very narrow periods create many cells with tiny counts. Record the period definitions and report the number of participants in each period, including periods with no participants in one arm.
Three Complementary Views of Support
Start with the observed counts \(n_{1k}\) and \(n_{0k}\). A zero count is an unambiguous warning that no within-period comparison is available from the observed assignments. A small positive count is also important: estimates for that cell may be unstable, and a few participants can have substantial influence. There is no universal count threshold that makes a period adequate. The relevant number depends on the intended analysis, outcome variability, and the size of effects the trial needs to estimate.
Next inspect the allocation probabilities \(\pi_i\) within each period. For each period, report a range or quantiles, and the proportion of participants whose probability was near an extreme. For example, a prespecified diagnostic might report the shares with \(\pi_i\leq 0.10\) and \(\pi_i\geq 0.90\). Those cutoffs are descriptive conventions, not universal pass/fail rules. A probability near zero means the treatment was rarely available at that assignment; a probability near one means the comparison treatment was rarely available.
If inverse-probability weights are used, extremes can also indicate potential instability. For a participant assigned to treatment, the unstabilized weight for treatment is \(1/\pi_i\); for a participant assigned to comparison, it is \(1/(1-\pi_i)\). Thus, as \(\pi_i\) approaches zero or one, the weight for the less-favored assignment can become large. This is a warning about limited information, not a diagnosis that weighting is necessarily invalid.
Finally, summarize the observed outcome in each treatment-by-period cell. Useful summaries include the cell count, missing outcome count, median, spread, and observed range or selected quantiles, all in the outcome’s units. These summaries reveal whether a cell has little outcome information and whether the observed outcome values differ in their coverage. But outcome ranges are not a substitute for treatment-assignment support: overlapping ranges do not guarantee adequate counts, and disjoint ranges do not by themselves establish why the distributions differ.
Worked Example: Balanced Totals, Sparse Periods
Hypothetical data. A trial enrolls participants in three prespecified periods. Treatment is coded \(A_i=1\). The counts are:
| Enrollment period | Treatment, \(n_{1k}\) | Comparison, \(n_{0k}\) | Total |
|---|---|---|---|
| Early | 32 | 8 | 40 |
| Middle | 30 | 30 | 60 |
| Late | 8 | 32 | 40 |
| Total | 70 | 70 | 140 |
Overall, the trial has \(70\) participants in each arm, so the overall counts are balanced. Within the early period, however, only \(8\) participants received comparison; within the late period, only \(8\) received treatment. The corresponding within-period proportions are \(8/40=0.200\) for comparison early and \(8/40=0.200\) for treatment late. The middle period has \(30/60=0.500\) in each arm.
The appropriate conclusion is not that the whole trial lacks treatment representation. Rather, the overall balance conceals sparse cells at opposite ends of enrollment. If the outcome changes over time, the early and late within-period comparisons may be less precise than the middle-period comparison, and an overall estimate may depend on how the analysis accounts for time. Counts alone do not quantify bias or determine whether a particular model is adequate.
Worked Example: Reading Allocation-Probability Extremes
Hypothetical data. In one enrollment period, six successive participants have recorded probabilities of treatment \(0.05,\ 0.10,\ 0.20,\ 0.80,\ 0.90,\ 0.95\). Treat these as fixed probabilities for this calculation, with independent random assignments. The expected treatment count is the sum of the six probabilities:
The expected comparison count is \(6-3.00=3.00\). That apparent balance does not mean both assignments were similarly available at every enrollment. Two of six probabilities are at most \(0.10\), and two of six are at least \(0.90\): each share is \(2/6=0.333\), or \(33.3\%\).
The largest possible unstabilized weight for a treatment-assigned participant in this list is \(1/0.05=20.0\). The largest comparison weight is \(1/(1-0.95)=20.0\). These large values show how the least-favored assignment can receive substantial weight. They are not weights actually observed unless the corresponding participant received that assignment. The range and tail shares therefore identify a design-support concern even though the expected arm counts are equal.
A Simple Justification for Probability Summaries
For a period with fixed assignment probabilities and independent Bernoulli assignments, write \(n_{1k}=\sum_{i\in k}A_i\), where \(A_i\) has mean \(\pi_i\) and variance \(\pi_i(1-\pi_i)\). Linearity of expectation gives the expected count. Independence gives the variance:
In the six-participant example, the variance is \(0.595\), calculated as \(0.0475+0.0900+0.1600+0.1600+0.0900+0.0475\). This calculation explains why a sum of probabilities describes an expected count, not a guaranteed realized count. When adaptive probabilities depend on trial history, report the logged probabilities as they were generated and interpret their summaries as records of the assignment opportunities offered. Do not treat the fixed-probability variance formula as an exact variance calculation for every adaptive design.
Worked Example: Outcome Support Within Periods
Hypothetical data. Suppose the outcome is hospital length of stay, measured in days. The following summaries are observed:
| Period | Treatment: count; range; median | Comparison: count; range; median |
|---|---|---|
| Early | 12; 3–18 days; 8 days | 3; 5–12 days; 7 days |
| Middle | 10; 4–16 days; 9 days | 10; 3–17 days; 9 days |
| Late | 3; 8–24 days; 16 days | 12; 7–25 days; 15 days |
In the early period, the observed ranges overlap from 5 through 12 days, but the comparison cell has only three participants. In the late period, the ranges overlap from 8 through 24 days, but the treatment cell has only three participants. The middle period has ten participants in each arm and broadly similar observed ranges. Thus, range overlap does not remove the sparse-cell concern.
The medians are descriptive: for example, the early-period treatment median is 8 days and the comparison median is 7 days. They do not establish a treatment effect. The summaries also need missingness counts; if outcomes are unobserved for some assigned participants, the cell count and the number contributing an outcome summary are not the same. Neither outcome ranges nor medians reveal whether unmeasured prognostic factors are balanced.
Practical Diagnostic Workflow
Choose periods that preserve clinically relevant temporal structure and have enough enrollment to make summaries interpretable. State the calendar boundaries and the reason for the choice.
For every period, report the total and the treatment and comparison counts. Highlight zero or small cells rather than relying on an overall arm total.
Within each period, describe the allocation-probability range, quantiles, and proportions near prespecified extremes. Check for missing or implausible logged probabilities.
Report outcome-observed counts, missingness, and selected distribution summaries by treatment and period. Keep these descriptions separate from causal interpretations.
Identify which periods offer weak within-period representation and explain what that means for precision or reliance on modeling. Do not infer that an analysis is unbiased or biased from a diagnostic alone.
Comparisons, Failure Modes, and Limits
An overall arm-count table is simpler, but it cannot locate temporal gaps: the first worked example has perfectly equal totals and sparse cells at both ends. A single allocation-probability average can also conceal extreme individual probabilities, as the second example shows. Period-specific displays retain the information needed to see these patterns. Conversely, making too many tiny periods can make every display look sparse; a coarser summary can be more stable but may conceal shorter shifts. Reporting a second, transparently chosen grouping can help assess whether conclusions depend on the bin width, but it does not eliminate the need to explain the primary grouping.
A fitted allocation-probability model may be useful in other analyses, but a model-based summary is not a replacement for checking the probabilities actually used by the randomization system. If allocation probabilities are missing, incorrectly recorded, or reconstructed from an incomplete history, extreme-value summaries may be misleading. Likewise, observed outcome support is necessarily limited by the outcomes that were measured and available; it cannot establish support for unobserved potential outcomes.
- Common mistake: “The overall treatment totals match, so overlap is adequate.” Equal totals say nothing about whether both arms are represented in each part of enrollment. Check period-specific counts and probabilities.
- Common mistake: “The observed outcome ranges overlap, so sparse assignment is no longer a problem.” Overlapping ranges do not create participants in an empty or tiny treatment-by-period cell. Report counts alongside outcome summaries.
- Common mistake: “Any probability below a chosen cutoff proves a positivity violation.” Cutoffs help flag potentially weak support; they are not universal thresholds. Interpret them with the count pattern, design, intended analysis, and degree of extrapolation required.
- Common mistake: “A diagnostic proves the treatment effect is biased.” These summaries describe observed representation and information. They do not, by themselves, quantify bias, prove a causal assumption, or select an analysis model.
The practical value of these diagnostics is that they make the data’s temporal support visible before an analysis is interpreted. They can show where estimates may rely on few participants, where weighting may be sensitive to extreme assignment probabilities, and where outcome summaries are thin. Their limits are equally important: a favorable display cannot rule out unmeasured differences, model misspecification, or outcome missingness, and a warning does not by itself determine how an analysis should be changed.
Check Your Understanding
Answer each question using the stated quantities. Brief solutions follow the questions.
- Two periods contain 25 participants each. In the first, treatment and comparison counts are 20 and 5; in the second, they are 5 and 20. What are the overall totals, and what does the period-specific display reveal?
- In a period, four participants have treatment probabilities \(0.10,\ 0.30,\ 0.70,\ 0.90\). Calculate the expected treatment count if probabilities are fixed and assignments independent. How many probabilities are at or beyond the cutoffs \(0.10\) and \(0.90\)?
- A treatment and comparison cell have overlapping outcome ranges, but the comparison cell contains two participants. Does the range overlap establish adequate representation? Explain.
- Give one reason to avoid extremely narrow calendar periods and one reason not to use only a single broad period.
Solutions. (1) The overall treatment total is \(20+5=25\), and the comparison total is \(5+20=25\). Equal totals conceal sparse cells: only five participants received comparison in the first period and only five received treatment in the second. (2) The expected treatment count is \(0.10+0.30+0.70+0.90=2.00\). One probability is at or below \(0.10\), and one is at or above \(0.90\); each is \(1/4=25.0\%\) of the four assignments. (3) No. Overlapping observed ranges do not increase the count of two in the comparison cell, and the range is a descriptive outcome summary rather than proof of adequate assignment support. (4) Extremely narrow periods can create many tiny cells and unstable summaries. A single broad period can hide shorter temporal gaps or changes in representation.