Two Different Sources of Uncertainty
A likelihood interval can be narrow and still leave an important question unanswered: does the observable data distinguish the parameter value being reported from alternatives? Sampling uncertainty concerns how estimates vary across repeated samples when a model is specified. Identification concerns whether that model’s restrictions connect observable-data distributions to a unique parameter value. These are different questions, and more data directly address only the first.
For example, a sample may estimate a particular feature of the data-generating distribution very precisely, while the parameter of interest is linked to that feature only through a strong assumption. The resulting interval can reflect sampling variation accurately under the assumption and still be uninformative about whether the assumption is appropriate.
This definition concerns the model and the distribution of observables, not whether two parameter values happen to give the same likelihood at one realized data set. Likelihood values can coincide by chance. Observational equivalence is stronger: the probability assigned to every possible observable outcome is the same under the two parameter values.
Identification Is Not Sampling Precision
In likelihood inference, a confidence interval is constructed under a specified model. Its sampling interpretation concerns repeated samples generated under that model and its assumptions. Identification is a prerequisite for learning a unique parameter from the observable distribution; it is not a measure of the interval’s width.
If different parameter values imply the same observable-data distribution, no increase in sample size can distinguish those values using those observables alone. It may make an identified feature of the distribution increasingly precise. To identify a parameter that is not uniquely determined, the analysis needs additional restrictions or information, such as a known measurement property or a restriction on a nuisance parameter. Those additions can be useful, but their role should be made explicit.
Worked Example: A Precisely Estimated Sum, but an Unidentified Component
Suppose the hypothetical observations \(Y_1,\ldots,Y_{100}\) are independent with \(Y_i\sim N(\alpha+\beta,1)\). The variance is known to be \(1\), and the only way \(\alpha\) and \(\beta\) enter the observable distribution is through their sum. Suppose the observed sample mean is \(\bar{y}=3.42\). What is estimated precisely, and is \(\beta\) identified?
Write \(\eta=\alpha+\beta\). Under the stated independent normal sampling model, the likelihood depends on the parameters through \(\eta\):
The sample mean estimates the identifiable combination \(\eta\). Its standard error is \(1/\sqrt{100}=0.1\). Using the 95% normal critical value \(1.96\), an interval for \(\eta\) is
This is a 95% confidence interval for the sum \(\alpha+\beta\), under the specified model. It is not an interval for \(\beta\). For every real number \(c\), the parameter values \((\alpha,\beta)=(3.42-c,c)\) give the same value of \(\eta\), and therefore the same observable-data distribution. In particular, the data cannot distinguish \((\alpha,\beta)=(3.42,0)\) from \((0,3.42)\).
If the additional restriction \(\alpha=0\) is imposed, then \(\beta=\eta\), so the same numerical interval becomes an interval for \(\beta\), conditional on that restriction. The interval’s width comes from sample size and the assumed variance; the interpretation as an interval for \(\beta\) comes from the restriction. Without it, \(\beta\) is not identified.
Assumptions Can Select Among Equivalent Values
The preceding example illustrates a general logic. A restriction can shrink the set of parameter values compatible with the model, sometimes to a single value. That does not mean the data independently established the restriction. It means the parameter is identified within the restricted model. A careful report separates what the observed distribution identifies from what follows only after a modeling choice.
The same distinction applies when the parameter of interest is identified only after specifying a measurement process. A model may identify a probability of observing a recorded outcome, while a target probability about an underlying condition depends on additional properties of the measurement system.
Worked Example: A Recorded Positive Rate and an Assumed Sensitivity
In a hypothetical screening sample, each of \(n=200\) independent people has an unobserved condition indicator. Let \(\theta\) be the population prevalence. Suppose the screening test has no false positives, and its sensitivity \(s\) is the probability of a positive result among people with the condition. Assume the test results are independent across people. Thus the observed positive probability is \(q=\theta s\). There are 68 positive results. First estimate \(q\); then consider what is learned about \(\theta\).
Under this model, the positive-result indicators are independent Bernoulli random variables with probability \(q\). The likelihood for the observed count \(K=68\) is proportional to \(q^{68}(1-q)^{132}\), so the data identify \(q\), not automatically its two factors. The maximum-likelihood estimate is
For a simple large-sample illustration of sampling uncertainty, the estimated standard error is
A normal-approximation 95% interval is \(0.3400\pm1.96(0.03350)\), or approximately \([0.2743,0.4057]\). The approximation is reasonable here because the fitted expected counts are \(200(0.34)=68\) positives and \(200(0.66)=132\) negatives, both large. The interval describes sampling uncertainty for \(q\), under the independent Bernoulli model; it does not by itself identify \(\theta\).
For any \(q\) between 0 and 1, all pairs \((\theta,s)\) with \(\theta s=q\), \(0\leq\theta\leq1\), and \(0\leq s\leq1\) are observationally equivalent. At the fitted value \(q=0.34\), for example, \((\theta,s)=(0.425,0.8)\) and \((0.68,0.5)\) both imply a positive probability of \(0.34\). The data alone cannot choose between them.
If sensitivity is assumed known to be \(s=0.8\), then \(\theta=q/0.8\). The estimate is \(0.34/0.8=0.425\), and transforming the endpoints gives an approximate interval
This interval is conditional on the known-sensitivity assumption and the no-false-positives model. Its endpoints are within the allowed range for a prevalence. If, instead, external knowledge only supports \(0.7\leq s\leq0.9\), then the same confidence limits for \(q\) imply a wider range for \(\theta\). Since \(\theta=q/s\), the smallest value uses the smallest \(q\) and largest \(s\), and the largest uses the largest \(q\) and smallest \(s\):
This last range incorporates the stated sensitivity bounds rather than pretending sensitivity is known exactly. Its confidence coverage follows from coverage for \(q\), provided the measurement model and sensitivity bounds hold. Neither interval accounts for violations of the no-false-positives assumption.
Restrictions and Sensitivity Analysis
In applications, a parameter may be linked to observables through an unknown bias, measurement error, or other nuisance feature. A restriction on that feature may yield point identification; a bound may yield only a range of compatible target values. It is often helpful to report how conclusions change across defensible restrictions, rather than presenting one assumption-dependent interval as if it were determined by the data alone.
Worked Example: A Bounded Measurement Bias
Suppose hypothetical measurements \(Y_1,\ldots,Y_{64}\) are independent with \(Y_i\sim N(\theta+b,1)\), where the variance is known and \(b\) is a constant measurement bias. The observable mean is \(\eta=\theta+b\). The observed sample mean is \(\bar{y}=2.10\). Calculate a 95% interval under \(b=0\), and then allow the bias to satisfy \(-0.2\leq b\leq0.2\).
The standard error of \(\bar{y}\) is \(1/\sqrt{64}=0.125\). Under the normal sampling assumptions, the 95% interval for the observable mean \(\eta\) is
If \(b=0\), then \(\eta=\theta\), and this interval is also the interval for \(\theta\). That interpretation depends on the assumption of no bias. If the bias is unknown but bounded, then \(\theta=\eta-b\). Combining every value of \(\eta\) in the interval with every allowed value of \(b\) gives
This wider interval accounts for the stated bias range as well as sampling uncertainty. If the bound on \(b\) is correct, it covers \(\theta\) whenever the interval covers \(\eta\). If \(b\) is unrestricted, then for any proposed \(\theta\) a value of \(b\) can be chosen to produce the same \(\eta\); no finite interval for \(\theta\) follows from these observations and this model alone.
A Short Identification Check
The examples share a simple justification. Let \(P_\theta\) denote the distribution of observable data under parameter \(\theta\), and let \(g(\theta)\) be the target. If two values \(\theta\) and \(\theta'\) satisfy \(P_\theta=P_{\theta'}\), then no procedure based only on the observable data can distinguish them through their sampling distribution. Consequently, point identification of \(g(\theta)\) requires that every such equivalent pair have the same target value:
This condition is also sufficient for the target to be identified within the model: each observable-data distribution then corresponds to one value of \(g\), even if it corresponds to several full parameter vectors. In the normal sum example, \(g(\alpha,\beta)=\alpha+\beta\) satisfies the condition, but \(g(\alpha,\beta)=\beta\) does not. In the screening example, \(q=\theta s\) satisfies it, while \(\theta\) does not unless sensitivity is restricted.
Common Mistakes and Practical Interpretation
- Equating a narrow interval with strong identification. A narrow interval can precisely estimate an observable combination while leaving the target parameter unidentified. Check which parameter the likelihood actually distinguishes.
- Treating an assumption-based interval as assumption-free. If an interval for a target requires a known sensitivity, zero bias, or another restriction, state that condition alongside the interval.
- Calling equal likelihood values for one sample observational equivalence. Identification concerns equality of the full observable-data distributions, not an accidental tie in likelihood for the realized sample.
- Assuming more observations solve nonidentification. Larger samples can reduce sampling uncertainty for identified features. They cannot separate parameter values that imply exactly the same observable distribution under the model.
A useful report therefore names the observable feature estimated, states the restrictions that connect it to the target, and distinguishes sampling uncertainty from uncertainty about those restrictions. A confidence interval is not the probability that the fixed parameter lies inside it, and its nominal coverage under a model does not show that the model’s identifying assumptions hold.
Check Your Understanding
For each question, distinguish information supplied by the observable distribution from information supplied by a modeling restriction.
- In the model \(Y_i\sim N(\alpha+\beta,1)\), which parameter combination is identified, and why can increasing the sample size not identify \(\beta\) when \(\alpha\) is unrestricted?
- What is the difference between two parameter values giving equal likelihood for one observed sample and being observationally equivalent?
- In the screening example, which assumption allows the interval for the positive probability to be converted into an interval for prevalence?
- If a measurement bias is restricted to a narrower range, what happens to the confidence set for the target, and what remains conditional on that restriction?
- State the condition for a target \(g(\theta)\) to be identified when the full parameter vector may not be identified.