Gaussian Conditional Density Models

Why Model a Conditional Density?

For a continuous treatment, a regression model for the conditional mean does not by itself provide the quantity needed for inverse-probability weighting. The denominator is a conditional density evaluated at the dose actually received. A Gaussian regression model supplies that density by modeling both the conditional mean and, usually, the conditional variance.

Let $A_i$ be the continuous treatment received by unit $i$, and let $H_i$ denote the measured covariates used to model treatment assignment. In a longitudinal setting, $A_i$ and $H_i$ can be replaced by the treatment and covariate history at time $t$; the conditional density is then evaluated at that time's observed treatment, given the history available just before assignment. The earlier material on continuous-treatment weights as conditional density ratios establishes the role of this denominator. Here the focus is how to estimate and evaluate it.

Definition: A Gaussian conditional density model specifies the distribution of $A_i$ given $H_i=h$ as $$ A_i \mid H_i=h \sim N\!\left(m(h;\beta),\sigma^2(h;\gamma)\right), $$ where $m(h;\beta)$ is a modeled conditional mean and $\sigma^2(h;\gamma)>0$ is a modeled conditional variance. In the common-variance model, $\sigma^2(h;\gamma)=\sigma^2$ for every $h$. The fitted conditional density evaluated at the observed dose is $$ \widehat f(A_i\mid H_i)= \frac{1}{\sqrt{2\pi\widehat{\sigma}^2(H_i)}} \exp\!\left\{-\frac{[A_i-\widehat m(H_i)]^2}{2\widehat{\sigma}^2(H_i)}\right\}. $$

A density is not a probability at a single point: for a continuous treatment, $P(A_i=a\mid H_i)=0$. The fitted density describes probability per unit of treatment, and probabilities are obtained by integrating it over intervals. Its numerical value can exceed 1 when the treatment is measured on a sufficiently fine scale. In a density ratio, however, numerator and denominator use the same treatment scale, so their units cancel.

Fitting a Linear Gaussian Conditional Model

A standard specification uses a linear conditional mean, $m(H_i;\beta)=x_i^\mathsf{T}\beta$, where $x_i$ is a chosen vector of covariate features, including an intercept if appropriate. It assumes a common conditional variance. For independent units, the conditional likelihood is the product of normal densities given the observed covariates. Independence here is an assumption about the sampling units; repeated treatment records from one person generally require accounting for their dependence when estimating uncertainty.

Result: Suppose $A_1,\ldots,A_n$, conditional on fixed design vectors $x_1,\ldots,x_n$, are independent with $A_i\mid x_i\sim N(x_i^\mathsf{T}\beta,\sigma^2)$. If the design matrix $X$ has full column rank and the residual sum of squares at its minimum is positive, the maximum-likelihood estimates are $$ \widehat\beta=(X^\mathsf{T}X)^{-1}X^\mathsf{T}A,\qquad \widehat{\sigma}^2=\frac{1}{n}\sum_{i=1}^n(A_i-x_i^\mathsf{T}\widehat\beta)^2. $$

Derivation. Write $A=(A_1,\ldots,A_n)^\mathsf{T}$ and $S(\beta)=\sum_i(A_i-x_i^\mathsf{T}\beta)^2$. Up to a constant, the log-likelihood is

\ell(\beta,\sigma^2)=-\frac n2\log(\sigma^2)-\frac{S(\beta)}{2\sigma^2}.

For fixed $\sigma^2>0$, maximizing this expression over $\beta$ is equivalent to minimizing $S(\beta)$. Full column rank makes the least-squares minimizer unique, and solving its normal equations $X^\mathsf{T}X\widehat\beta=X^\mathsf{T}A$ gives the displayed estimate. For fixed $\beta$, differentiation with respect to $\sigma^2$ gives

\frac{\partial \ell}{\partial \sigma^2} =-\frac{n}{2\sigma^2}+\frac{S(\beta)}{2(\sigma^2)^2}.

Setting this derivative to zero gives $\sigma^2=S(\beta)/n$. The second derivative there is negative, and the log-likelihood tends downward at either boundary when the minimized sum of squares is positive. Substituting the minimizing $\widehat\beta$ gives the stated maximum-likelihood estimate. The divisor is $n$, not the residual degrees of freedom; the latter divisor gives an unbiased variance estimate in the linear model, but not the maximum-likelihood estimate. $\square$

Worked Example: Estimating the Mean and Variance

Suppose four independent observations have covariate values $x=0,1,2,3$ and treatment doses $A=1.1,2.8,5.2,6.9$. Fit $A_i\mid x_i\sim N(\beta_0+\beta_1x_i,\sigma^2)$.

The sample means are $\bar x=1.5$ and $\bar A=4.0$. The slope estimate is $$ \widehat\beta_1= \frac{\sum_i(x_i-\bar x)(A_i-\bar A)}{\sum_i(x_i-\bar x)^2} =\frac{9.9}{5}=1.98, $$ so $\widehat\beta_0=\bar A-\widehat\beta_1\bar x=4-1.98(1.5)=1.03$. The fitted means at the four covariate values are $1.03,3.01,4.99,6.97$. Thus the residuals are $0.07,-0.21,0.21,-0.07$, and $$ SSE=0.07^2+(-0.21)^2+0.21^2+(-0.07)^2 =0.0049+0.0441+0.0441+0.0049=0.098. $$ The maximum-likelihood variance estimate is $0.098/4=0.0245$, and the estimated standard deviation is $\sqrt{0.0245}\approx0.1565$. These estimates define the fitted conditional normal density for any dose and covariate value.

Evaluating Densities at Observed Doses

Once the model is fitted, evaluate the normal density formula at each unit's observed dose and covariates. In the common-variance case, define the standardized residual $r_i=(A_i-\widehat m(H_i))/\widehat\sigma$. Then

\widehat f(A_i\mid H_i)= \frac{1}{\widehat\sigma\sqrt{2\pi}}\exp(-r_i^2/2).

The density is largest at the fitted conditional mean and decreases as the observed dose moves farther away in standard deviation units. A very small value indicates that the fitted treatment model regards the observed dose as unusual for that covariate profile. For numerical stability, it is often useful to calculate the log density, $\log\widehat f(A_i\mid H_i)=-\log(\widehat\sigma\sqrt{2\pi})-r_i^2/2$, and exponentiate only when a density on its original scale is needed.

Worked Example: Calculating a Fitted Conditional Density

Suppose a fitted model gives $\widehat m(H_i)=2.5$ and $\widehat\sigma=0.5$, while the observed dose is $A_i=3$. The standardized residual is $r_i=(3-2.5)/0.5=1$. Therefore, $$ \widehat f(3\mid H_i)= \frac{1}{0.5\sqrt{2\pi}}\exp(-1/2) \approx 0.7979(0.6065)=0.4839. $$ This is a density in inverse dose units, not a 48.39% probability. For example, an approximate probability for a narrow interval of width $0.1$ around 3 is $0.4839(0.1)=0.0484$, using the local density as an approximation.

For a continuous-treatment weight, the denominator density is typically paired with a numerator density, such as a marginal treatment density $g(A_i)$. The following identity explains why that ratio changes the conditional treatment distribution.

Result: Let $f(a\mid h)$ be a conditional treatment density and $g(a)$ a target density. If $f(a\mid h)>0$ wherever $g(a)>0$, then weighting the conditional density by $g(a)/f(a\mid h)$ gives the target density $g(a)$: $$ f(a\mid h)\frac{g(a)}{f(a\mid h)}=g(a). $$ Consequently, for any integrable function $q$, $$ \int q(a)\frac{g(a)}{f(a\mid h)}f(a\mid h)\,da =\int q(a)g(a)\,da. $$

Justification. The positivity condition makes the ratio defined wherever the target assigns positive density. Multiplying the ratio by the conditional density cancels $f(a\mid h)$ on that support, leaving $g(a)$. Integrating both sides after multiplying by $q(a)$ gives the second equality, provided the integrals exist. This is the density-ratio argument underlying weighting; in practice $f$ is replaced by its fitted Gaussian estimate, so model error and small fitted densities affect the resulting weights. $\square$

Worked Example: A Conditional Density in a Weight Ratio

Suppose for a particular covariate profile the fitted denominator is $N(2,1)$, and the numerator density is $N(2,0.64)$. At observed dose $A_i=3$, the denominator is $$ \widehat f(3\mid H_i)=\frac{1}{\sqrt{2\pi}}\exp(-1/2)\approx0.24197. $$ The numerator density is $$ g(3)=\frac{1}{0.8\sqrt{2\pi}}\exp\!\left(-\frac{(3-2)^2}{2(0.64)}\right) \approx 0.49868\exp(-0.78125)\approx0.2283. $$ The density ratio is therefore $0.2283/0.24197\approx0.9435$. This calculation illustrates evaluation at the observed dose; the numerator and denominator must refer to the same dose scale.

Assumptions, Diagnostics, and Limitations

The simple Gaussian model is most defensible when the conditional dose distribution is approximately symmetric and bell-shaped, the modeled mean captures systematic changes across $H_i$, and a common variance is adequate. Likelihood-based formulas also rely on independent units conditional on the design, correct specification of the conditional distribution, and a well-defined treatment measurement. A linear mean is a modeling choice, not a consequence of normality; important nonlinearities or interactions must be represented if they are needed.

Check residual shape. Examine standardized residuals for strong skewness, heavy tails, or separated clusters. A normal-looking mean fit does not guarantee a normal conditional distribution.
Check residuals against fitted values and covariates. A fan-shaped spread suggests changing conditional variance; curvature or systematic patterns suggest mean-model misspecification.
Check observed and fitted dose ranges. Predictions that assign extremely small densities to observed doses can produce unstable density ratios and large weights. Inspect the resulting weights as well as the density model.
Respect the treatment's support. A normal model assigns positive density to every real number. If doses are restricted to be positive or lie within fixed bounds, that support mismatch can make the Gaussian model implausible, especially near a boundary.

A Gaussian conditional model is compact and interpretable, but its convenience is not evidence that it fits. A model for the conditional mean alone is not an alternative density model unless it also specifies the distributional spread and shape. Other conditional-density approaches may be preferable when symmetry, common variance, or support assumptions fail; the essential task remains to estimate the conditional density at each observed dose. No parametric fit can repair a lack of covariate-specific treatment support: if observed doses are rare in a profile, the denominator may be unstable even when the fitted curve appears smooth.

Common Mistake

Pitfall: Treating $\widehat m(H_i)$ as the treatment probability, or treating $\widehat f(A_i\mid H_i)$ as the probability that the exact dose occurred. A conditional mean is not a density, and a continuous variable has zero probability at an exact point. The correction is to specify a full conditional distribution, estimate its parameters, and evaluate its density at the observed dose. Then use that density in the ratio required by the weighting analysis.

Check Your Understanding

Work through the questions, then check the solutions below.

In a common-variance linear Gaussian treatment model, why does the maximum-likelihood variance estimator divide the residual sum of squares by $n$, rather than by the residual degrees of freedom?
A fitted conditional model has mean 4 and standard deviation 2. Calculate the density at an observed dose of 6.
Why can a fitted density exceed 1 without being an invalid probability?
What does a fan shape in residuals versus fitted values suggest, and why can it matter for weighting?

Solutions. (1) Differentiating the log-likelihood with respect to $\sigma^2$ gives the maximum at $SSE/n$; dividing by residual degrees of freedom instead targets an unbiased variance estimate, not the likelihood maximum. (2) The standardized residual is $(6-4)/2=1$, so the density is $\exp(-1/2)/(2\sqrt{2\pi})\approx0.1210$. (3) A density is probability per unit of measurement, not a point probability; its integral over the full support, rather than its value at one point, equals 1. (4) A fan shape suggests nonconstant conditional variance. A common-variance model can then misstate the observed-dose density for some profiles, which can distort density ratios and weight stability.