Attrition Bounds for Panel Mean Outcomes

Why Bound a Mean Under Attrition?

When a panel outcome is missing for some members of the target population, the mean among observed respondents need not equal the population mean. The observed mean describes those whose outcomes were recorded; the target mean also depends on the unobserved outcomes. If the outcome has a known lower and upper limit, those limits let us calculate a range of population means consistent with the observed data, without pretending to know the missing values.

These ranges are identification bounds: they describe which population means are compatible with the stated information and assumptions. They are not, by themselves, confidence intervals. Their width reflects how little is known about the missing outcomes, while sampling uncertainty is a separate issue.

Definitions and Worst-Case Bounds

Fix a panel wave and a target population of units. Let \(Y\) denote each unit’s outcome at that wave, and let \(R=1\) if the outcome is observed and \(R=0\) if it is missing. Assume the outcome is known to lie in the support \([L,U]\), where \(L\) and \(U\) are finite and \(L\leq U\). Write \(\mu=E[Y]\) for the target population mean and \(p=P(R=1)\) for the response proportion.

Definition: When \(0<p<1\), define the observed-outcome mean \(\mu_1=E[Y\mid R=1]\) and the missing-outcome mean \(\mu_0=E[Y\mid R=0]\). The target mean decomposes as \(\mu=p\mu_1+(1-p)\mu_0\). The support assumption implies \(L\leq\mu_0\leq U\), so the worst-case bounds are \(p\mu_1+(1-p)L\leq\mu\leq p\mu_1+(1-p)U\).

The decomposition follows by partitioning the target population into observed and missing outcomes. The worst-case lower endpoint assigns every missing outcome its smallest possible value, \(L\); the upper endpoint assigns every missing outcome \(U\). These are deliberately conservative assignments. The observed outcomes themselves are not changed.

\begin{aligned} \mu &=E[Y\mid R=1]P(R=1)+E[Y\mid R=0]P(R=0)\\ &=p\mu_1+(1-p)\mu_0, \qquad 0<p<1. \end{aligned}

The condition \(0<p<1\) matters: both conditional means in this decomposition must be defined. At the endpoints, handle the target mean directly. If \(p=0\), no outcomes are observed, so the support alone gives \(L\leq\mu\leq U\). If \(p=1\), every outcome is observed and \(\mu=\mu_1\); there is no missing-outcome mean to bound.

Worked Example: Worst-Case Bounds for a Bounded Score

Suppose a hypothetical panel has a target population of 160 people at a follow-up wave. Outcomes are scores known to lie between 0 and 100 points. Scores are observed for 120 people, and their mean is 68 points. Calculate the support-based bounds for the target population mean.

The response proportion is \(p=120/160=0.75\), so \(0<p<1\). The observed mean is \(\mu_1=68\) points, and the support is \([L,U]=[0,100]\). The missing group contains 40 people, but its mean is not observed. The support only establishes \(0\leq\mu_0\leq100\).

Substitute into the bound formula:

\begin{aligned} 0.75(68)+0.25(0) &\leq \mu \leq 0.75(68)+0.25(100),\\ 51 &\leq \mu \leq 76. \end{aligned}

Thus the target mean is between 51 and 76 score points under the support assumption alone. These endpoints are attainable if all 40 missing scores are respectively 0 or 100; the bounds do not claim that either extreme is likely.

Why the Bounds Hold

For \(0<p<1\), the support assumption gives \(L\leq\mu_0\leq U\). Multiplying all parts of this inequality by \(1-p\), which is positive, preserves the inequalities. Adding the fixed term \(p\mu_1\) then gives the claimed bounds on \(\mu\).

\begin{aligned} L\leq\mu_0\leq U &\ \Longrightarrow\ (1-p)L\leq(1-p)\mu_0\leq(1-p)U\\ &\ \Longrightarrow\ p\mu_1+(1-p)L \leq \mu \leq p\mu_1+(1-p)U. \end{aligned}

This also explains why the endpoints are sharp given only the stated information. Assigning every missing outcome \(L\) produces the lower endpoint while leaving the observed outcomes unchanged; assigning every missing outcome \(U\) produces the upper endpoint. Without additional information or assumptions, neither endpoint can generally be moved inward.

Directional Bounds Under Mean-Monotone Selection

The worst-case bounds do not use any relationship between response status and outcome. A defensible directional assumption about selection can narrow the range. Here, the assumption is stated directly in terms of group means. It must be justified by substantive knowledge; it does not follow merely from observing attrition.

Definition: For \(0<p<1\), positive mean-monotone selection assumes \(\mu_0\leq\mu_1\): the mean outcome among people with missing outcomes is no greater than the mean among respondents. Negative mean-monotone selection assumes \(\mu_1\leq\mu_0\). These are assumptions about conditional means, not claims that every missing individual has a lower or higher outcome than every respondent.

Under positive mean-monotone selection, combine \(L\leq\mu_0\leq\mu_1\) with the population-mean decomposition. Under negative mean-monotone selection, combine \(\mu_1\leq\mu_0\leq U\). The resulting ranges are narrower than, or equal to, the worst-case bounds because each adds a restriction on the missing-group mean.

\begin{array}{ll} \text{Positive mean-monotone selection:} & p\mu_1+(1-p)L\leq\mu\leq\mu_1,\\[4pt] \text{Negative mean-monotone selection:} & \mu_1\leq\mu\leq p\mu_1+(1-p)U, \end{array} \qquad 0<p<1.

For the first upper endpoint, setting \(\mu_0=\mu_1\) gives \(p\mu_1+(1-p)\mu_1=\mu_1\). For the second lower endpoint, the same equality gives \(\mu_1\). The other endpoints follow by setting \(\mu_0=L\) or \(\mu_0=U\), respectively.

The directional assumptions also require \(0<p<1\), since both \(\mu_1\) and \(\mu_0\) must be defined. If \(p=0\), neither group mean in the assumption is defined, and the support-only bounds \(L\leq\mu\leq U\) remain. If \(p=1\), the mean is observed for the whole target population, so \(\mu=\mu_1\), with no missing-group selection assumption needed.

Worked Example: Positive Mean-Monotone Selection

In a hypothetical follow-up survey, 60% of the target population has an observed outcome. The outcome is a rating from 0 to 100 points, and the observed respondents’ mean is 72 points. Suppose subject-matter knowledge supports the assumption that the missing group’s mean rating is no greater than the respondents’ mean. Calculate both the worst-case bounds and the positive mean-monotone bounds.

Here \(p=0.60\), \(\mu_1=72\), \(L=0\), and \(U=100\), with \(0<p<1\). The worst-case bounds are:

\begin{aligned} 0.60(72)+0.40(0) &\leq\mu\leq 0.60(72)+0.40(100),\\ 43.2&\leq\mu\leq83.2. \end{aligned}

Under the additional assumption \(0\leq\mu_0\leq72\), the lower endpoint stays at 43.2 points and the upper endpoint becomes:

0.60(72)+0.40(72)=72, \qquad\text{so}\qquad 43.2\leq\mu\leq72.

The tightened upper endpoint follows from the assumption, not from the observed response rate or score data alone. If the assumption is not credible, the wider support-based range is the appropriate one to report.

Worked Example: Negative Mean-Monotone Selection

Consider a different hypothetical panel wave. Outcomes are measured in minutes and lie between 0 and 100. Seventy percent of the target population has an observed outcome, with an observed mean of 40 minutes. Suppose a substantive argument supports the assumption that the missing group’s mean is at least 40 minutes. Calculate the corresponding bounds.

The conditions are \(p=0.70\), \(\mu_1=40\), \(L=0\), \(U=100\), and \(0<p<1\). The worst-case bounds are:

\begin{aligned} 0.70(40)+0.30(0) &\leq\mu\leq 0.70(40)+0.30(100),\\ 28&\leq\mu\leq58. \end{aligned}

Negative mean-monotone selection requires \(40\leq\mu_0\leq100\). Therefore:

40\leq\mu\leq0.70(40)+0.30(100)=58.

The target mean is bounded between 40 and 58 minutes under this assumption. Without the directional assumption, the lower bound would remain 28 minutes. The calculation does not establish that attrition caused longer outcomes; it only shows the implications of the stated selection restriction.

Using Panel Data and Interpreting the Result

In a finite panel, the same arithmetic can be applied to the mean for a specified wave and target cohort. If \(N\) cohort members are in the target population and \(n\) have observed outcomes, let \(r=n/N\) and let \(\bar{Y}_{\mathrm{obs}}\) be their observed mean. Given a support \([L,U]\), the finite-population mean must lie in:

r\bar{Y}_{\mathrm{obs}}+(1-r)L \leq \bar{Y}_{\mathrm{target}} \leq r\bar{Y}_{\mathrm{obs}}+(1-r)U.

This is a direct accounting identity plus support restriction for that cohort. For example, if \(N=100\), \(n=80\), and the observed outcomes sum to 4,800, then \(r=0.80\) and \(\bar{Y}_{\mathrm{obs}}=4{,}800/80=60\). With support \([0,100]\), the bounds are \(0.80(60)+0.20(0)=48\) and \(0.80(60)+0.20(100)=68\). Thus the finite-cohort mean lies between 48 and 68 outcome units, subject to the stated support.

In applications, be precise about the target population and wave. An attrition bound for the original cohort’s mean at follow-up is not automatically a bound for the mean among current respondents or for a different cohort. Also check that the proposed support is valid for every target-population outcome, not just the observed values. An incorrect support can make the resulting range invalid.

When the data are a sample from a larger population, plug-in endpoints do not account for sampling uncertainty in the response proportion or observed mean. Statistical inference for the endpoints or identified set requires additional methods. Report the identification range and any uncertainty assessment separately, and state the support and selection assumptions used.

Common Mistakes

Treating the respondent mean as the target mean. This is only justified with additional assumptions that make the missing group’s mean equal to the observed group’s mean. Response alone does not establish that equality.
Using the directional bound without stating its assumption. A narrower interval is not free information. Positive selection requires \(\mu_0\leq\mu_1\); negative selection requires \(\mu_1\leq\mu_0\).
Calling an identification range a confidence interval. The worst-case range describes means compatible with the support and observed information. It does not have a confidence level unless a separate inferential procedure is applied.
Using conditional means at an endpoint. The decomposition and directional bounds using \(\mu_1\) and \(\mu_0\) apply when \(0<p<1\). At \(p=0\), use the support-only range; at \(p=1\), the population mean is the observed mean.

Key takeaway: With a known support, the observed share and observed mean yield sharp worst-case bounds for a target mean. A mean-monotone selection assumption can tighten one side of the range, but only when its direction is stated and substantively defensible.

Check Your Understanding

Answer each question using the definitions and assumptions in this tutorial.

A panel outcome lies in \([10,50]\), 75% of outcomes are observed, and the observed mean is 30. Calculate the worst-case bounds for the target mean.
For \(0<p<1\), state the mean restriction required for positive mean-monotone selection and give the resulting upper bound.
Why are conditional means \(\mu_1\) and \(\mu_0\) not used in the bound formula when \(p=0\)? What range is available in that case?
If \(p=1\), what is the target population mean, and why is there no missing-outcome bound to calculate?
Explain why an identification bound is not automatically a confidence interval.