Deriving the Truncated Regression Likelihood

Why Truncation Changes the Likelihood

Suppose an outcome is recorded only when it exceeds a known threshold. A study might, for example, include only transactions above a prespecified value. The observed outcomes are not an ordinary sample from the full outcome distribution: by construction, values at or below the threshold are absent. A likelihood for the observed sample must therefore describe outcomes conditional on passing the inclusion rule.

This differs from censoring, where an observation can remain in the data while its exact value is hidden or recorded only as a bound. With truncation, units failing the rule do not appear in the observed sample. Here we derive the likelihood for one specific rule: an outcome is included if and only if it is greater than a known threshold.

Model and Inclusion Rule

For observation \(i\), let \(Y_i\) be the outcome and \(x_i\) a vector of observed regressors, including an intercept if appropriate. Assume the latent outcome follows a Gaussian regression model,

Y_i=x_i^\top\beta+\varepsilon_i,\qquad \varepsilon_i\mid x_i\sim N(0,\sigma^2),\qquad \sigma>0.

Write \(\mu_i=x_i^\top\beta\). Thus, conditional on \(x_i\), \(Y_i\sim N(\mu_i,\sigma^2)\). Let \(c\) be the known truncation threshold, and suppose \(i\) is observed exactly when \(Y_i>c\). The observed data contain the selected outcomes and their regressors; outcomes that fail this rule are absent.

Assumptions for the likelihood: The conditional outcome distribution is Gaussian with mean \(x_i^\top\beta\) and variance \(\sigma^2\); the threshold \(c\) is known; inclusion occurs precisely when \(Y_i>c\); and selected outcomes are observed exactly. For the product likelihood below, observations are conditionally independent given their regressors. The likelihood is conditional on the observed regressors and on inclusion in the sample.

Let \(\phi(z)=(2\pi)^{-1/2}\exp(-z^2/2)\) be the standard normal density and let \(\Phi(z)\) be its cumulative distribution function. The probability that an observation with regressor vector \(x_i\) is included is

P(Y_i>c\mid x_i) =1-\Phi\!\left(\frac{c-\mu_i}{\sigma}\right) =\Phi\!\left(\frac{\mu_i-c}{\sigma}\right).

The last equality uses the symmetry of the standard normal distribution. This probability varies with the regression mean: when \(\mu_i\) is far above \(c\), inclusion is likely; when it is far below \(c\), inclusion is unlikely.

Deriving the Conditional Density

For an observed value \(y_i>c\), the relevant density is the density of \(Y_i\) conditional on the event \(Y_i>c\), not its unconditional Gaussian density. The conditional-density rule divides the joint density for an outcome in a small interval by the probability of the conditioning event. More formally, for \(y_i>c\), the conditional density is the limit of \(P(y_i\leq Y_i<y_i+\Delta\mid Y_i>c,x_i)/\Delta\) as \(\Delta\) tends to zero. The numerator is the unconditional probability of that small interval, since the interval lies above \(c\). Its density limit is \(f(y_i\mid x_i)\). Dividing by the probability of inclusion gives the result:

f(y_i\mid x_i,Y_i>c) = \frac{f(y_i\mid x_i)}{P(Y_i>c\mid x_i)} = \frac{ \frac{1}{\sigma}\phi\!\left(\frac{y_i-\mu_i}{\sigma}\right) }{ \Phi\!\left(\frac{\mu_i-c}{\sigma}\right) }, \qquad y_i>c.

The denominator is the normalizing probability. It ensures that the conditional density integrates to one over the values that can occur in the observed sample:

\int_c^\infty \frac{f(y\mid x_i)}{P(Y_i>c\mid x_i)}\,dy = \frac{P(Y_i>c\mid x_i)}{P(Y_i>c\mid x_i)} =1.

This also shows why the ordinary Gaussian density alone is insufficient: over the selected range \(y>c\), it integrates to the inclusion probability, not to one.

Worked Example: One Selected Outcome

Suppose \(Y\mid x\sim N(2,1)\), the known threshold is \(c=1\), and the observed outcome is \(y=1.5\). The observation is selected because \(1.5>1\). Calculate its conditional density contribution.

The unconditional density at \(y=1.5\) has standardized value \(z=(1.5-2)/1=-0.5\), so

f(1.5\mid x)=\phi(-0.5) =\frac{1}{\sqrt{2\pi}}\exp(-0.5^2/2) \approx 0.3521.

The inclusion probability is

P(Y>1\mid x) =1-\Phi((1-2)/1) =\Phi(1) \approx 0.8413.

Therefore, the selected-sample density contribution is

f(1.5\mid x,Y>1) =\frac{0.3521}{0.8413} \approx 0.4185.

This is a density, not the probability of observing the exact value \(1.5\). The division adjusts the Gaussian density to the distribution among outcomes above the threshold.

The Truncated Regression Likelihood

For a sample of \(n\) selected observations, conditional independence gives a product of the individual conditional densities. Let \(\mu_i=x_i^\top\beta\), and suppose every observed \(y_i\) satisfies \(y_i>c\). The likelihood for \(\beta\) and \(\sigma\), conditional on the observed regressors and inclusion, is

Truncated Gaussian regression likelihood: For a known common threshold \(c\),

L(\beta,\sigma) = \prod_{i=1}^n \frac{ \frac{1}{\sigma}\phi\!\left(\frac{y_i-x_i^\top\beta}{\sigma}\right) }{ \Phi\!\left(\frac{x_i^\top\beta-c}{\sigma}\right) }, \qquad y_i>c,\quad \sigma>0.

The denominator for each observation is its own inclusion probability, evaluated at that observation’s regressors.

Taking logarithms converts the product into a sum and makes the selection adjustment explicit. With \(z_i=(y_i-x_i^\top\beta)/\sigma\), the log-likelihood is

\ell(\beta,\sigma) = \sum_{i=1}^n \left[ -\log\sigma-\frac{1}{2}\log(2\pi) -\frac{(y_i-x_i^\top\beta)^2}{2\sigma^2} -\log\Phi\!\left(\frac{x_i^\top\beta-c}{\sigma}\right) \right].

The first three terms are the Gaussian log density. The final term subtracts the log probability of selection. Because that probability depends on \(\beta\) and \(\sigma\), it is part of the likelihood’s parameter-dependent structure and cannot be dropped as a constant.

Worked Example: Likelihood for Two Selected Observations

Consider two independent observations under a hypothetical model with \(c=0\), \(\sigma=1\), and \(\mu_i=x_i^\top\beta\). At the parameter values being evaluated, let \(\mu_1=0.5\), \(\mu_2=1\), and suppose the selected outcomes are \(y_1=0.2\) and \(y_2=1.4\). Both exceed the threshold.

For the first observation, the unconditional density is \(\phi(0.2-0.5)=\phi(-0.3)\approx0.3814\), and the inclusion probability is \(\Phi(0.5)\approx0.6915\). Its conditional contribution is therefore

\frac{\phi(-0.3)}{\Phi(0.5)} \approx \frac{0.3814}{0.6915} \approx 0.5516.

For the second observation, the unconditional density is \(\phi(1.4-1)=\phi(0.4)\approx0.3683\), and the inclusion probability is \(\Phi(1)\approx0.8413\). Its conditional contribution is

\frac{\phi(0.4)}{\Phi(1)} \approx \frac{0.3683}{0.8413} \approx 0.4377.

Multiplying the two conditional contributions gives the sample likelihood. Using the displayed rounded contributions, \(0.5516\times0.4377\approx0.2414\). The log-likelihood, calculated from the unrounded contributions, is approximately \(-1.421\). In full, it is the sum of each Gaussian log density minus the log of its inclusion probability. The denominator differs across observations because their regression means differ.

Why the Normalizer Affects Parameter Comparisons

The adjustment is not merely a rescaling that can be ignored when comparing parameter values. Changing \(\beta\) changes the fitted means and therefore changes both the Gaussian density in the numerator and the chance of passing the threshold in the denominator. The next calculation isolates that role by comparing two candidate means for one selected value.

Worked Example: Comparing Two Candidate Means

Suppose \(c=0\), \(\sigma=1\), and the observed outcome is \(y=0.5\). Compare its conditional density contribution at candidate means \(\mu=0\) and \(\mu=1\). The same observed value exceeds the threshold under both candidates.

At \(\mu=0\), the numerator is \(\phi(0.5)\approx0.3521\), while the inclusion probability is \(\Phi(0)=0.5\). Thus

f(0.5\mid \mu=0,Y>0) =\frac{\phi(0.5)}{\Phi(0)} \approx\frac{0.3521}{0.5000} \approx0.7041.

At \(\mu=1\), the numerator is \(\phi(-0.5)\approx0.3521\), but the inclusion probability is \(\Phi(1)\approx0.8413\). Hence

f(0.5\mid \mu=1,Y>0) =\frac{\phi(-0.5)}{\Phi(1)} \approx\frac{0.3521}{0.8413} \approx0.4185.

The numerators are equal because the candidate means are equally far from the observed value. The conditional contributions differ because the probability of inclusion differs. These are likelihood contributions at two parameter values, not probabilities that either parameter value is true.

Likelihood Interpretation and a Common Mistake

Maximum likelihood estimation for this model chooses parameter values that maximize the conditional likelihood, or equivalently its log. The selected observations contain information about the outcome distribution, but the selection rule changes how that information is represented: the model must account for how likely each observation was to appear at all. The likelihood written here is conditional on the observed regressors and on sample inclusion. It does not, by itself, use the number of units that failed the threshold when that count is available from a separate sampling design.

Common mistake: use only \(\prod_i f(y_i\mid x_i)\), reasoning that the observed values follow the Gaussian regression model.
Correction: the observed values follow the Gaussian distribution conditional on \(Y_i>c\). Divide each density by \(P(Y_i>c\mid x_i)\), which is \(\Phi((x_i^\top\beta-c)/\sigma)\). Without this normalizer, the density over the selected range does not integrate to one, and parameter comparisons ignore the selection rule.
Another useful check: every observation used in this likelihood must satisfy \(y_i>c\). A value at or below the threshold is incompatible with the stated exact-observation rule.

Key takeaway: In a Gaussian regression sample observed only above a known threshold, each likelihood contribution is the ordinary Gaussian density divided by the model-implied probability of exceeding that threshold. The denominator is observation-specific whenever fitted means vary with regressors.

Check Your Understanding

Use the conditional-density argument and likelihood formula to answer these questions.

For \(Y\mid x\sim N(\mu,\sigma^2)\) observed only when \(Y>c\), write the inclusion probability using \(\Phi\).
Explain why the Gaussian density alone does not integrate to one over the range of outcomes present in the selected sample.
Write the likelihood contribution for one observed \(y_i>c\) in terms of \(x_i^\top\beta\), \(\sigma\), \(\phi\), and \(\Phi\).
In the two-observation example, why are the two normalizing probabilities different?
What is wrong with interpreting a likelihood contribution as the probability of observing an exact continuous outcome?