Conditioning and Truncation Within Exponential Families

Conditioning Changes the Normalizer

An exponential-family model describes a distribution through a canonical statistic and a log-partition function. If observations are retained only when they fall in a specified region, the conditional law still has the same exponential tilt by that statistic. What changes is the normalizer: it must now normalize the tilted measure over the retained region rather than over the entire sample space.

This distinction is useful in truncated-data models, selection settings, and conditional likelihood calculations. The central requirement is that the conditioning event or region be fixed as the parameter varies and have positive probability under the law being conditioned. A region whose definition changes with the parameter requires separate care.

Definition: Let \(\nu\) be a fixed measure and suppose a family has density \(f_\eta(x)=\exp\{\eta^T T(x)-A(\eta)\}\) with respect to \(\nu\), where \(\eta\) is the natural parameter, \(T\) is a \(d\)-dimensional statistic, and \(A(\eta)=\log\int \exp\{\eta^T T(x)\}\,\nu(dx)\) is finite. For a fixed measurable region \(B\), define \[ Z_B(\eta)=\int_B \exp\{\eta^T T(x)\}\,\nu(dx). \] Where \(0<Z_B(\eta)<\infty\), the conditional law given \(X\in B\) has density \[ f_{\eta,B}(x)=\mathbf{1}_B(x)\exp\{\eta^T T(x)-A_B(\eta)\}, \qquad A_B(\eta)=\log Z_B(\eta), \] with respect to \(\nu\).

The statistic \(T(x)\) and natural parameter \(\eta\) have not changed. The new log-partition function \(A_B\) is the log of the integral over \(B\), not the original integral over the whole space. The indicator \(\mathbf{1}_B(x)\) records the restricted support. If the original density is written with a carrier \(h(x)\) relative to another base measure, take \(\nu(dx)=h(x)\,\mu(dx)\); the same formula applies.

Why the Conditional Formula Is Correct

Write \(P_\eta(B)\) for the probability of the fixed event under the original family. Direct integration gives

P_\eta(B) =\int_B \exp\{\eta^T T(x)-A(\eta)\}\,\nu(dx) =\exp\{A_B(\eta)-A(\eta)\}.

Consequently, on \(B\), dividing the original density by \(P_\eta(B)\) cancels \(A(\eta)\) and replaces it by \(A_B(\eta)\). Outside \(B\), the conditional density is zero. This proves the representation and also gives a practical identity for calculating the conditional normalizer from the original one.

Theorem: Suppose \(B\) is measurable and fixed, \(A(\eta)\) is finite, and \(0<P_\eta(B)\leq 1\). Then the conditional distribution of \(X\) given \(X\in B\) is an exponential-family distribution with the same statistic \(T\), natural parameter \(\eta\), and log-partition function \[ A_B(\eta)=A(\eta)+\log P_\eta(B). \] Equivalently, \(A_B(\eta)=\log\int_B \exp\{\eta^T T(x)\}\,\nu(dx)\). Its support is restricted to \(B\), up to sets of \(\nu\)-measure zero.

To verify the stated identity, the integral defining \(Z_B(\eta)\) is finite because it equals \(\exp\{A(\eta)\}P_\eta(B)\), and it is positive because \(P_\eta(B)>0\). Thus the conditional density displayed in the definition is well-defined and integrates to one. The formula for \(A_B\) follows by taking logarithms of \(P_\eta(B)=\exp\{A_B(\eta)-A(\eta)\}\). \(\square\)

Worked Examples

Worked Example: A Normal Distribution Truncated Above a Fixed Threshold

Suppose \(X\sim N(\mu,1)\), and retain only observations with \(X>1\). Write the model in canonical form and find the conditional log-partition function.

With \(\eta=\mu\), the density with respect to Lebesgue measure is \[ (2\pi)^{-1/2}\exp\{-x^2/2+\eta x-\eta^2/2\}. \] Thus \(T(x)=x\), \(\nu(dx)=(2\pi)^{-1/2}\exp(-x^2/2)\,dx\), and \(A(\eta)=\eta^2/2\). The restricted integral is

Z_{(1,\infty)}(\eta) =\int_1^\infty (2\pi)^{-1/2}\exp(-x^2/2+\eta x)\,dx =\exp(\eta^2/2)\bigl[1-\Phi(1-\eta)\bigr].

For the last equality, complete the square: \(-x^2/2+\eta x=-(x-\eta)^2/2+\eta^2/2\), then use the standard normal upper-tail probability. Therefore \(A_{(1,\infty)}(\eta)=\eta^2/2+\log[1-\Phi(1-\eta)]\). At \(\eta=0\), this is \(\log[1-\Phi(1)]\), and the conditional density is \(\phi(x)/[1-\Phi(1)]\) for \(x>1\). This is the standard normal density divided by the probability of the retained region.

Worked Example: A Binomial Count Conditioned to Be Positive

Let \(X\sim\operatorname{Binomial}(3,p)\), with \(0<p<1\), and condition on \(X\geq1\). Find the conditional normalizer and calculate the conditional probabilities when \(p=0.2\).

Set \(\eta=\log\{p/(1-p)\}\). The canonical statistic is \(T(x)=x\), the carrier at \(x=0,1,2,3\) is \(\binom{3}{x}\), and \(A(\eta)=3\log(1+e^\eta)\). Restricting the normalizing sum to \(x=1,2,3\) gives

Z_{\{1,2,3\}}(\eta) =\sum_{x=1}^3\binom{3}{x}e^{\eta x} =(1+e^\eta)^3-1, \qquad A_{\{1,2,3\}}(\eta)=\log\bigl((1+e^\eta)^3-1\bigr).

For \(p=0.2\), \(e^\eta=p/(1-p)=0.25\). The original probability of a positive count is \(1-(0.8)^3=0.488\). The three unconditioned probabilities are \(P(X=1)=3(0.2)(0.8)^2=0.384\), \(P(X=2)=3(0.2)^2(0.8)=0.096\), and \(P(X=3)=(0.2)^3=0.008\). Dividing each by \(0.488\) gives approximately \(0.7869\), \(0.1967\), and \(0.0164\), respectively; they sum to \(1.0000\) up to rounding. The statistic remains \(x\); only the normalizing sum excludes the zero-count outcome.

Worked Example: An Exponential Distribution Truncated Below a Fixed Point

Let \(X\) have an exponential distribution with rate \(\lambda>0\), and condition on \(X>2\). Derive its canonical form and the conditional density.

Write \(\eta=-\lambda<0\). Relative to Lebesgue measure, the original density is \((-\eta)\exp(\eta x)\mathbf{1}_{[0,\infty)}(x)\). Taking the carrier to be \(\mathbf{1}_{[0,\infty)}(x)\), its statistic is \(T(x)=x\), and \(A(\eta)=-\log(-\eta)\), since \(\int_0^\infty e^{\eta x}\,dx=1/(-\eta)\). For the retained region \(B=(2,\infty)\),

Z_B(\eta)=\int_2^\infty e^{\eta x}\,dx =\frac{e^{2\eta}}{-\eta}, \qquad A_B(\eta)=2\eta-\log(-\eta).

The conditional density is therefore \(\mathbf{1}_{(2,\infty)}(x)(-\eta)\exp\{\eta(x-2)\}\). Its integral is \((-\eta)\int_2^\infty e^{\eta(x-2)}\,dx=1\), because \(\eta<0\). In rate notation, this is \(\lambda e^{-\lambda(x-2)}\) for \(x>2\). The displayed form keeps the original statistic \(x\), with the factor involving the threshold absorbed into the new normalizer.

Conditional Means and Covariances

The restricted log-partition function also encodes moments under the conditional law. The derivative identities below require differentiability of the restricted integral; they do not follow merely from writing down a formal exponential-family expression.

Theorem: Let \(T=(T_1,\ldots,T_d)^T\), and suppose \(\eta\) lies in an open set on which \(Z_B(\eta)\) is finite and positive. At a parameter value \(\eta_0\), assume there is an \(\epsilon>0\) such that \[ \int_B \exp\{\eta_0^T T(x)+2\epsilon\|T(x)\|\}\,\nu(dx)<\infty. \] Then, in a neighborhood of \(\eta_0\), \(A_B\) is twice differentiable and \[ \nabla A_B(\eta)=E_\eta[T(X)\mid X\in B], \qquad \nabla^2 A_B(\eta)=\operatorname{Cov}_\eta(T(X)\mid X\in B). \]

To prove the result, choose \(\eta\) sufficiently close to \(\eta_0\) that \(\|\eta-\eta_0\|<\epsilon\). The integrand defining \(Z_B(\eta)\) is bounded by \(\exp\{\eta_0^TT+\epsilon\|T\|\}\). Each of its first and second parameter derivatives adds at most a factor \(\|T\|\) or \(\|T\|^2\). For \(r=1,2\), the quantity \(s^r e^{-\epsilon s}\) is bounded for \(s\geq0\), so these derivative integrands are bounded by a constant times \(\exp\{\eta_0^TT+2\epsilon\|T\|\}\). The assumed integrability therefore supplies a common integrable bound. Differentiation under the integral follows from the dominated convergence theorem.

In particular, \(\partial Z_B/\partial\eta_i=\int_B T_i e^{\eta^TT}\,d\nu\). Since \(A_B=\log Z_B\), the chain rule gives \[ \frac{\partial A_B}{\partial\eta_i} =\frac{\int_B T_i e^{\eta^TT}\,d\nu}{Z_B(\eta)} =E_\eta[T_i(X)\mid X\in B]. \] Differentiating once more yields \[ \frac{\partial^2 A_B}{\partial\eta_i\partial\eta_j} =\frac{\int_B T_iT_j e^{\eta^TT}\,d\nu}{Z_B(\eta)} -\frac{\int_B T_i e^{\eta^TT}\,d\nu}{Z_B(\eta)} \frac{\int_B T_j e^{\eta^TT}\,d\nu}{Z_B(\eta)}. \] The right-hand side is the conditional covariance of \(T_i(X)\) and \(T_j(X)\), proving the Hessian identity. \(\square\)

Scope and Common Pitfalls

A fixed region means the same measurable set \(B\) is used for every parameter value. The conditional parameter domain consists of those original natural parameters for which the restricted integral is finite and positive. For a fixed event of positive probability under every parameter in the original domain, this condition holds throughout that domain.

Using the original normalizer after conditioning. The original \(A(\eta)\) normalizes over the full sample space. The conditional density needs \(A_B(\eta)\), which normalizes over \(B\). The relationship is \(A_B(\eta)=A(\eta)+\log P_\eta(B)\).
Changing the statistic unnecessarily. Truncation does not require replacing \(T(x)\); the same statistic appears in the conditional exponential tilt. An algebraically equivalent recoding may be possible, but it is not needed to derive the conditional law.
Ignoring zero-probability events. If \(P_\eta(B)=0\), conditioning on \(X\in B\) is not defined by the usual conditional-density formula at that parameter value.
Treating parameter-dependent support as fixed truncation. If the region itself changes with \(\eta\), the indicator and potentially the integration domain vary with the parameter. The fixed-region derivation above does not, by itself, justify differentiating such a model.

The practical check is to integrate the proposed conditional density over \(B\). In canonical form, that calculation must produce \(\int_B e^{\eta^TT(x)}\,\nu(dx)=e^{A_B(\eta)}\), so the density integrates to one. This also makes clear why the original statistic can be retained even though the conditional distribution has a different normalizer and a restricted support.

Check Your Understanding

Try each question by identifying the restricted integral before simplifying it.

For a family with log-partition function \(A(\eta)\), express \(A_B(\eta)\) in terms of \(A(\eta)\) and \(P_\eta(X\in B)\).
Why must the conditioning region be fixed as \(\eta\) varies for the representation proved here to apply directly?
For a Poisson random variable with canonical parameter \(\eta=\log\lambda\), write the restricted normalizing sum when conditioning on \(X\geq1\).
State the conditional mean and covariance identities involving derivatives of \(A_B\), and give the regularity purpose of the integrability condition in the theorem.
A proposed conditional density uses \(A(\eta)\) rather than \(A_B(\eta)\). What normalization error should you check for?