Bayesian Priors Under Nonidentification

When the Data Cannot Distinguish Between Values

Bayesian analysis combines a likelihood with a prior distribution. When the likelihood varies with a parameter, observed data can update beliefs about it. But if distinct parameter values imply exactly the same distribution of observed data, the data cannot distinguish among those values. A posterior distribution over them may still be calculated; its existence does not mean the observed data identified the parameter.

A missing binary outcome gives a direct example. The response rate and the outcome rate among respondents are observable features of the data. The outcome rate among nonrespondents is not. The population outcome rate depends on all three, so posterior conclusions about it can depend strongly on the prior distribution assigned to the unobserved rate.

Definition: Let $Y\in\{0,1\}$ be a binary outcome and let $R=1$ indicate that $Y$ is observed, with $R=0$ indicating that it is missing. Write $\pi=P(R=1)$, $q=P(Y=1\mid R=1)$, and $p_0=P(Y=1\mid R=0)$. The population outcome probability is $$ \mu=P(Y=1)=\pi q+(1-\pi)p_0. $$ The observed-data distribution identifies $\pi$ and $q$, but it contains no information about $p_0$ without additional assumptions or information.

For this tutorial, assume independent, identically distributed individuals from one population, and that $R$ and $Y$ follow the probabilities just defined. Outcomes are recorded whenever $R=1$, and no outcome is recorded when $R=0$. We impose no missing-at-random restriction or other constraint linking $p_0$ to $q$. These assumptions give a simple model in which the source of nonidentification is explicit.

The Observed Likelihood and the Flat Direction

Suppose $n$ individuals are enrolled, $r$ have observed outcomes, and $s$ of those observed outcomes equal 1. The observed data consist of response indicators for everyone and outcomes only for respondents. Their likelihood, apart from a combinatorial factor that does not depend on the parameters, is

L(\pi,q,p_0)\propto \pi^r(1-\pi)^{n-r}q^s(1-q)^{r-s}.

There is no $p_0$ in this likelihood. In particular, for any two values $p_0^{(a)}$ and $p_0^{(b)}$ in $[0,1]$, holding $\pi$ and $q$ fixed gives

\frac{L(\pi,q,p_0^{(a)})}{L(\pi,q,p_0^{(b)})}=1.

This equality is exact, not an approximation caused by a small sample. No increase in the number of observations under this same observed-data scheme can create a likelihood difference between those two values of $p_0$. The earlier material on structural nonidentification distinguishes this situation from weak identification, where the likelihood does vary but may do so only slightly.

Key result: If the prior makes $p_0$ independent of $(\pi,q)$, then the posterior distribution of $p_0$ is exactly its prior distribution. The observed likelihood updates $\pi$ and $q$, but does not update $p_0$. Since $\mu=\pi q+(1-\pi)p_0$, posterior conclusions about $\mu$ nevertheless depend on the prior for $p_0$.

The independence condition matters. A prior may associate $p_0$ with $\pi$ or $q$. Then data that update $\pi$ or $q$ can indirectly change the posterior distribution of $p_0$. Such a change comes from the prior association combined with learning about the associated parameter; it does not mean the likelihood directly distinguishes values of $p_0$.

p(p_0\mid \text{data})\propto p(p_0)\int L(\pi,q)\,p(\pi,q\mid p_0)\,d\pi\,dq.

Here $p(p_0)$ is the prior marginal density and $p(\pi,q\mid p_0)$ describes the prior dependence. If the prior factorizes as $p(p_0)p(\pi,q)$, the integral is constant in $p_0$, leaving its prior marginal unchanged. With dependence, the integral need not be constant.

Worked Example: The Missing-Group Rate Is Not Updated

Consider a hypothetical sample of $n=100$ individuals. Outcomes are observed for $r=60$, and $s=18$ of those observed outcomes equal 1. For a transparent calculation, first condition on the observable rates $\pi=60/100=0.60$ and $q=18/60=0.30$, treating them as fixed. Assign $p_0$ a $\operatorname{Beta}(1,1)$ prior, uniform on $[0,1]$, independently of $\pi$ and $q$.

The observed-data likelihood factor involving these fixed rates is $0.60^{60}0.40^{40}\times0.30^{18}0.70^{42}$. It is the same for every $p_0$. Thus the posterior for $p_0$ remains $\operatorname{Beta}(1,1)$, with posterior mean $0.50$.

The implied population rate is $\mu=0.60(0.30)+0.40p_0=0.18+0.40p_0$. Its posterior mean, under this conditional calculation, is $0.18+0.40(0.50)=0.38$, or 38%. This is not a data-driven estimate of the missing-group rate: the value $0.50$ is the mean of the chosen prior.

How Prior Choice Changes a Population Conclusion

The following calculation keeps $\pi=0.60$ and $q=0.30$ fixed to isolate prior influence. It compares two proper priors for $p_0$. Proper means that the prior integrates to one; it does not mean that its assumptions have empirical support in this dataset.

Worked Example: Comparing Posterior Means and a Threshold Probability

Continue with the hypothetical data above. Compare a uniform $\operatorname{Beta}(1,1)$ prior with a $\operatorname{Beta}(1,4)$ prior, which places more prior weight near smaller values of $p_0$. Because the likelihood is constant in $p_0$ and the priors are independent of $\pi,q$, each posterior for $p_0$ equals its prior.

Under $\operatorname{Beta}(1,4)$, the mean of $p_0$ is $1/(1+4)=0.20$. The corresponding posterior mean of the population rate is

Calculation: $E(\mu\mid\text{data})=0.18+0.40(0.20)=0.26$, or 26%.

Under $\operatorname{Beta}(1,1)$, the same calculation gives $0.18+0.40(0.50)=0.38$, or 38%. The 12-percentage-point difference is entirely attributable to the different prior distributions in this conditional comparison; the observed likelihood does not favor one $p_0$ value over another.

For a threshold question, $\mu>0.35$ is equivalent to $0.18+0.40p_0>0.35$, hence $p_0>0.425$. Under the uniform prior, the posterior probability is $1-0.425=0.575$. Under $\operatorname{Beta}(1,4)$, the survival probability is $(1-0.425)^4=0.575^4\approx0.1093$. Thus the posterior probability that the population rate exceeds 35% is 57.5% under one prior and 10.9% under the other, with the same observed data. Neither probability is the probability that a null hypothesis is true.

These are conditional calculations, not a complete posterior analysis of uncertain $\pi$ and $q$. A full model would also specify priors for them and propagate their posterior uncertainty into $\mu$. That added uncertainty does not remove the prior dependence along the unidentified $p_0$ direction.

Prior Dependence Can Also Produce Indirect Updating

It would be too broad to say that the posterior of $p_0$ always equals its prior. The exact statement requires prior independence from the parameters the data do identify. The next finite-prior example shows how learning about $q$ can affect $p_0$ when the prior links them.

Worked Example: Updating Through a Prior Association

Suppose, hypothetically, the prior assigns probability $1/2$ to each of two states: state A has $(q,p_0)=(0.20,0.10)$, and state B has $(q,p_0)=(0.40,0.90)$. Let $\pi$ be fixed and common to both states. In a sample of 60 respondents, 18 outcomes equal 1. Conditional on $q$, the likelihood is proportional to $q^{18}(1-q)^{42}$.

The likelihood ratio for B versus A is

\frac{L_B}{L_A} =\left(\frac{0.40}{0.20}\right)^{18} \left(\frac{0.60}{0.80}\right)^{42} =2^{18}(0.75)^{42} \approx 1.483.

With equal prior probabilities, Bayes' rule gives $P(B\mid\text{data})=1.483/(1+1.483)\approx0.597$. Therefore the posterior probability of $p_0=0.90$ is about 59.7%, compared with its prior probability of 50%.

The data distinguish the two states through their different $q$ values, not through their $p_0$ values. The prior ties high $q$ to high $p_0$, so the posterior movement in $p_0$ is inherited from that assumed association. If the prior instead paired each $q$ with both $p_0$ values equally, the same likelihood update to $q$ would not favor either missing-group rate.

Derivation, Diagnostics, and Limits

The central posterior statement follows directly from Bayes' rule. Under prior independence, write the prior density as $p(\pi,q)p(p_0)$. Since the likelihood factors as $L(\pi,q)$, the joint posterior is proportional to

L(\pi,q)p(\pi,q)p(p_0).

Integrating this expression over $\pi$ and $q$ leaves $p(p_0)$ multiplied by a quantity that does not depend on $p_0$. Normalizing the resulting density therefore returns $p(p_0\mid\text{data})=p(p_0)$. This establishes that unchanged posterior beliefs about $p_0$ are a consequence of both the flat likelihood and the factorized prior.

When reviewing an analysis, useful diagnostics include checking whether the likelihood or posterior output changes along values of $p_0$ that imply the same observed-data distribution, and comparing substantively plausible priors. If reported posterior precision for $p_0$ appears to come from a large sample, verify whether it actually comes from a restrictive prior, a model constraint, or external information. Also check prior dependence: apparent updating of $p_0$ can be transmitted through an association with identified parameters.

Parameterization is another source of confusion. Since $\mu=\pi q+(1-\pi)p_0$, choosing a prior for $\mu$ is also a choice about $p_0$, conditional on $\pi,q$. For fixed $\pi,q$, the allowable range is $\pi q\leq\mu\leq\pi q+1-\pi$. A prior must respect that range. Assigning a familiar named prior independently to $\mu$ is not generally equivalent to assigning that same named prior to $p_0$; equivalent probability models require transforming the prior distribution correctly.

Bayesian analysis is not the only way to express what the observed data establish. The earlier material on sharp bounds for a missing binary outcome gives the range of population rates consistent with observed rates when $p_0$ may be anywhere in $[0,1]$: $\pi q\leq\mu\leq\pi q+(1-\pi)$. A prior can yield a posterior distribution within that range, but a concentrated posterior does not by itself show that the likelihood narrowed the range. Conversely, bounds do not supply a probability distribution over values inside the range. The approaches answer different questions.

A Bayesian posterior remains useful when its assumptions are explicit: it can combine observed data with defensible prior information, and it can quantify uncertainty conditional on the model and prior. Its limitation here is equally important. If no observed-data information or justified restriction informs $p_0$, a posterior conclusion about that rate—and about the component of $\mu$ it determines—cannot be presented as if it were learned from the observed outcomes. Report prior sensitivity and identify which conclusions change.

Common Pitfall

Common mistake: “The sample is large, so the posterior for the missing-group outcome rate must be data-driven.” A large sample can estimate $\pi$ and $q$ very precisely, but the likelihood remains constant in $p_0$ at every sample size. With an independent prior, the posterior for $p_0$ is exactly the prior. The correction is to inspect which parameters actually enter the observed likelihood and to separate likelihood learning from prior assumptions and prior-induced dependence.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

In the model above, which two probabilities are identified by the observed-data distribution, and which probability is not?
Suppose $\pi=0.50$, $q=0.20$, and $p_0$ has an independent $\operatorname{Beta}(1,1)$ prior. What is the conditional posterior mean of $\mu$?
Why can the posterior for $p_0$ change when the prior associates it with $q$, even though the likelihood contains no $p_0$?
For $\pi=0.60$, $q=0.30$, what range of $\mu$ is consistent with $0\leq p_0\leq1$?

Solutions. (1) The response rate $\pi$ and respondent outcome rate $q$ are identified; the nonrespondent outcome rate $p_0$ is not. (2) The prior mean of $p_0$ is $0.50$, so $E(\mu\mid\text{data})=0.50(0.20)+0.50(0.50)=0.35$, or 35%, under the stated conditional calculation. (3) The likelihood updates $q$; if the prior links $q$ and $p_0$, that update changes the posterior distribution of $p_0$ indirectly. (4) The minimum is $0.60(0.30)=0.18$, and the maximum is $0.18+0.40=0.58$. Thus the compatible population rates range from 18% to 58%.