Diagnosing Sparse Treatment Histories

Why Tabulate Treatment Options Within Histories?

An overall treatment count can look reassuring while hiding a lack of observed treatment choices for particular kinds of people. In longitudinal data, the relevant comparison is made at each treatment decision among individuals with the history that precedes that decision. A table of treatment options across recurring histories makes that comparison concrete: it shows which histories occur, how often each treatment option is observed within them, and where the data are too sparse to provide much empirical information.

This is a descriptive diagnostic. A zero cell can flag a possible support problem, but it does not by itself establish that a treatment option is impossible in the population. Conversely, seeing each option at least once does not establish that the data support precise estimates or that the causal assumptions hold.

Definition: At decision time \(t\), let \(A_t\) be the treatment, taking values in a finite set \(\mathcal{A}_t\). Let \(H_t\) denote the observed history available just before that decision, including the measured covariate history \(\bar L_t=(L_0,\ldots,L_t)\) and the prior treatment history \(\bar A_{t-1}=(A_0,\ldots,A_{t-1})\). For individuals \(i\) eligible and observed at time \(t\), define the cell count \(N_t(h,a)=\sum_i I(H_{it}=h,A_{it}=a)\) and the history count \(n_t(h)=\sum_i I(H_{it}=h)\). When \(n_t(h)>0\), the empirical treatment proportion is \(\hat p_t(a\mid h)=N_t(h,a)/n_t(h)\). A sparse cell has a small count relative to the intended analysis; an empty cell has count zero.

The table should include every treatment option being considered, including options with count zero. Otherwise, a missing option can disappear from view. It should also make the risk set explicit: for example, whether the denominator includes only people eligible to receive treatment at that visit, and how missing treatment or covariate histories are handled. A history is the information available before the decision, not information measured afterward.

Building and Reading the Table

For each decision time, group individuals by their observed pre-decision history and count each treatment option within each group. With discrete covariates, recurring histories can be represented directly as table rows. With many variables or follow-up times, the number of distinct histories can grow quickly; the table may then contain many histories with only one or two observations. That pattern is itself important: a large overall sample does not guarantee repeated observations at the histories needed for a comparison.

A useful display shows the history, its total count, the count for every treatment option, and the corresponding empirical proportions when the total is nonzero. Report counts as well as proportions. A proportion of 1.00 based on 1 person is not equivalent in evidential strength to a proportion of 1.00 based on 100 people.

Worked Example: A Binary Decision With a Zero Cell

At one visit, a study has 40 eligible participants and a binary treatment \(A_t\in\{0,1\}\). Consider two recurring pre-decision histories, \(h_1\) and \(h_2\). The observed counts are:

History	Total \(n_t(h)\)	\(A_t=0\)	\(A_t=1\)	Empirical proportion for \(A_t=1\)
\(h_1\)	20	12	8	\(8/20=0.40\)
\(h_2\)	20	20	0	\(0/20=0\)

For \(h_1\), both options occur, with empirical proportions \(12/20=0.60\) and \(8/20=0.40\). For \(h_2\), treatment 1 is absent from the observed data. This is a direct warning that the sample provides no within-\(h_2\) treated observations. It is not proof that treatment 1 has zero probability in the target population: finite sampling could omit an option that has positive probability.

The overall treated count is 8 out of 40, so the overall treatment proportion is \(0.20\). That single number does not reveal that all eight treated participants came from \(h_1\). The history-specific table is necessary to see the pattern.

What Can a Zero Count Tell Us?

A simple sampling calculation clarifies why a zero cell is a warning rather than a population conclusion. Suppose a particular history \(h\) occurs \(m\) times, and, conditional on that history, each participant independently receives option \(a\) with the same probability \(p>0\). Then the number receiving \(a\) has a binomial distribution with parameters \(m\) and \(p\). In particular, the chance that the option is not observed at all is:

P\{N_t(h,a)=0\mid n_t(h)=m\}=(1-p)^m.

This calculation relies on the stated independent, common-probability sampling model within the history. It is not a universal probability model for every longitudinal study; for example, clustered sampling or unequal treatment probabilities may require a different calculation. Under the stated conditions, however, it quantifies how much a small history count limits what can be learned from an empty cell.

Result: If treatment option \(a\) has conditional probability \(p>0\) in a history represented by \(m\) independent observations with a common conditional probability, then the probability of observing no instances of \(a\) is \((1-p)^m\). This probability decreases as \(m\) increases, but is positive for every finite \(m\) when \(0<p<1\).

Derivation. For each of the \(m\) observations, the probability of not receiving \(a\) is \(1-p\). By independence, the probability that all \(m\) observations fail to receive \(a\) is the product of these probabilities, \((1-p)^m\). If \(0<p<1\), this product is positive for finite \(m\), and it decreases as \(m\) increases because \(0<1-p<1\). Thus an empty cell is compatible with a positive population probability. The calculation also shows why a zero among very few observations is less surprising than a zero among many observations. \(\square\)

Worked Example: How Sample Size Changes the Zero-Count Probability

Suppose option \(a\) truly has conditional probability \(p=0.10\) in a particular history. Under the independent, common-probability conditions above, if that history occurs \(m=5\) times, the probability of seeing no one receive \(a\) is:

$$ (1-0.10)^5=0.90^5=0.59049. $$

An empty cell is therefore quite plausible with only five observations, even though the option has positive probability. If the history occurs \(m=30\) times, the probability is:

0.90^{30}\approx 0.0424.

The same zero count is less expected with 30 observations than with 5. Neither calculation identifies the true conditional probability from the table; each interprets the chance of a zero under a specified probability and sampling model.

Pooling Histories Can Hide Sparse Cells

Combining histories can make the overall table appear to contain both treatment options even when some histories contain only one. The arithmetic behind this is straightforward: an overall proportion is a weighted average of the history-specific proportions, with weights determined by the history counts. That average does not preserve the within-history pattern.

Result: If a set of histories \(h\) is pooled and \(n_t(h)>0\), the pooled empirical proportion for option \(a\) is \(\sum_h N_t(h,a)/\sum_h n_t(h)=\sum_h [n_t(h)/\sum_g n_t(g)]\hat p_t(a\mid h)\). This identity follows by substituting \(N_t(h,a)=n_t(h)\hat p_t(a\mid h)\) for each history.

The pooled proportion is therefore a weighted average, but the average can conceal a zero in one or more component histories. This is why a table should retain the histories relevant to the treatment decision rather than report only a marginal treatment count. Choosing which histories to combine is a substantive modeling decision, not a cosmetic way to make cell counts larger.

Worked Example: An Overall 50–50 Split That Conceals Empty Cells

Suppose two histories each occur 10 times. In \(h_1\), all 10 participants receive treatment 1; in \(h_2\), all 10 receive treatment 0.

History	Total	Treatment 0	Treatment 1
\(h_1\)	10	0	10
\(h_2\)	10	10	0
Pooled	20	10	10

The pooled treatment-1 proportion is \(10/20=0.50\). Within \(h_1\), it is \(10/10=1\); within \(h_2\), it is \(0/10=0\). The pooled result is the weighted average \((10/20)(1)+(10/20)(0)=0.50\), but neither history contains both options. The aggregate count therefore cannot substitute for the history-specific counts.

Practical Diagnostics and Limits

Interpret sparse-cell tables as a map of where the observed data are thin. Patterns that warrant attention include empty treatment-option cells; cells represented by only one or a few individuals; histories appearing at one visit but not another; and many distinct histories with very small totals. Check that apparent zeros are not artifacts of data processing, such as incorrect eligibility rules, inconsistent coding of treatment, or missing values silently excluded from the table.

Where histories are too numerous to print, report a transparent summary of the distribution of history counts and identify the smallest cells, while preserving the treatment-option counts for the histories driving the analysis. Do not merge histories merely to eliminate zeros without considering whether the merged groups are meaningfully comparable. A table is also descriptive of the observed sample: selection, measurement error, missingness, and study design can all affect which cells appear.

These tables complement, rather than replace, longitudinal propensity-score overlap plots and weight diagnostics. A table shows the actual counts behind discrete history-option combinations; a propensity-score plot summarizes fitted probabilities, and weight diagnostics show consequences of the fitted weighting procedure. A well-behaved weight distribution cannot fill an unobserved treatment-option cell, while a sparse table alone does not diagnose every source of model misspecification. No one diagnostic establishes causal identification.

The central limitation is that empirical support and population support are different claims. A zero cell says that the option was not observed in that history in this sample. A nonzero cell says only that it was observed at least once. Counts and proportions help assess how much direct empirical information is available; they do not establish that all required causal assumptions hold or that estimates will be stable.

Key takeaway: At each treatment decision, tabulate every treatment option within the observed pre-decision histories that define the comparison. Read zero and small cells as evidence about the data available in the sample, not as definitive statements about population probabilities.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A history occurs 8 times, and an option has conditional probability \(0.25\) under the independent, common-probability model. What is the probability that the option is observed zero times?
Two histories each have 10 observations. One has treatment-option counts 10 and 0; the other has counts 0 and 10. What are the pooled counts, and what does pooling conceal?
Why should a table report counts as well as empirical proportions?
What does an empty cell establish, and what does it not establish?

Solutions. (1) The probability is \((1-0.25)^8=0.75^8\approx0.1001\). (2) The pooled counts are 10 for each option, out of 20. Pooling conceals that each history contains only one of the two options. (3) Counts distinguish, for example, a proportion of 1 based on one observation from a proportion of 1 based on many observations; the proportions alone do not show that difference. (4) An empty cell establishes that the option was not observed in that history in the analyzed sample. It does not prove that the option has zero probability in the population.