Why Regularity Conditions Matter
Likelihood methods use derivatives of the log likelihood to describe how the data distinguish nearby parameter values. Those derivatives lead to the score, information, and familiar large-sample approximations. But a derivative calculation is not automatically valid just because a likelihood formula can be written down. In particular, the set of possible observations may change with the parameter, or the true parameter may lie at a boundary where ordinary two-sided perturbations are unavailable.
Regularity conditions are assumptions that make the standard calculus of likelihood work: they justify differentiating and integrating in the required order, ensure that the score has the expected mean and that information is finite, and keep the true parameter away from problematic boundaries. They are sufficient conditions for standard results, not a checklist that every valid likelihood method must satisfy. When a condition fails, the likelihood can still be useful, but standard conclusions must be checked rather than assumed.
These requirements have distinct roles. Smoothness makes local derivatives meaningful. Parameter-independent support avoids extra terms from a moving edge of the integration region. Finite information makes the score variability well-defined and prevents the usual curvature calculations from involving infinite quantities. An interior true parameter allows small perturbations in every parameter direction. For standard asymptotic results, further conditions—such as identification and suitable control of the likelihood away from \(\theta_0\)—may also be needed.
What the Conditions Justify
For one observation, write the score as \(s(Y;\theta)=\partial\log f(Y;\theta)/\partial\theta\) in a scalar model. If the support is fixed and differentiation can be passed under the integral, then \(\int f(y;\theta)\,dy=1\) implies that the score has mean zero. Indeed, differentiating the normalization gives \(\int \partial f(y;\theta)/\partial\theta\,dy=0\), and wherever \(f>0\), \(\partial f/\partial\theta=f\,s\). Thus \(E_\theta[s(Y;\theta)]=0\).
With twice differentiable densities and a second valid interchange, the same reasoning underlies the information matrix equality. In the scalar case, \[ \frac{\partial^2 \log f}{\partial\theta^2} = \frac{\partial^2 f/\partial\theta^2}{f} - \left(\frac{\partial\log f}{\partial\theta}\right)^2. \] Taking expectations and using \(\int \partial^2 f/\partial\theta^2\,dy=0\) gives \(E_\theta[-\partial^2\log f/\partial\theta^2]=E_\theta[s^2]\). In a vector model, the corresponding identity is \(E_\theta[-\partial^2\log f/\partial\theta\partial\theta^\mathsf{T}] =E_\theta[ss^\mathsf{T}]\). These equalities require the differentiation and integration steps to be justified; they are not purely formal consequences of differentiating a log likelihood.
Finite information is not the same as having a finite likelihood or a finite score for each observed data point. It concerns the expected squared score under the model. Similarly, an interior true parameter means that some open neighborhood of \(\theta_0\) lies in the parameter space; it does not mean that every possible estimate must be interior.
Worked Example: A Regular Normal Location Model
Suppose \(Y_i\) are independent with \(Y_i\sim N(\theta,4)\), where the variance is known and \(\theta\in\mathbb{R}\). Consider the hypothetical observations \(1,2,3\), so \(n=3\).
The density is positive for every real observation at every \(\theta\), so its support is the fixed set \(\mathbb{R}\). It is differentiable in \(\theta\), and every real value of \(\theta\), including the true value, is interior to the parameter space. For one observation, \[ \log f(y;\theta)=-\tfrac12\log(8\pi)-\frac{(y-\theta)^2}{8}, \qquad s(y;\theta)=\frac{y-\theta}{4}. \] The score has mean zero because \(E_\theta[Y-\theta]=0\). Its information is \[ I(\theta)=E_\theta\!\left[\left(\frac{Y-\theta}{4}\right)^2\right] =\frac{E_\theta[(Y-\theta)^2]}{16} =\frac{4}{16} =0.25. \] It is finite and positive.
For the three observations, the sample score is \[ \sum_{i=1}^3 s(y_i;\theta) =\frac{(1-\theta)+(2-\theta)+(3-\theta)}{4}. \] At \(\theta=2\), this is \(({-1}+0+1)/4=0\). The total information is \(3(0.25)=0.75\). This example satisfies the central local regularity conditions, though those checks alone are not a proof of every possible asymptotic result.
Parameter-Dependent Support: The Uniform Model
A parameter-dependent support is a central warning sign. Consider \(Y_i\) with a uniform distribution on \([0,\theta]\), for \(\theta>0\), using the density \(f(y;\theta)=1/\theta\) for \(0\leq y\leq\theta\) and zero otherwise. The support itself changes with \(\theta\), so the fixed-support condition fails. Within the support, the log density is \(-\log\theta\), whose derivative is \(-1/\theta\). That derivative does not capture how the support edge moves.
Worked Example: A Uniform Likelihood at Its Support Edge
Take the hypothetical observations \(0.20,0.55,0.88\). For a specified density version that includes both endpoints, the likelihood is \[ L(\theta)= \begin{cases} \theta^{-3},&\theta\geq 0.88,\\ 0,&\theta<0.88. \end{cases} \] For \(\theta\geq0.88\), this decreases as \(\theta\) increases. Therefore its maximum is attained at the smallest allowed value, \(\hat{\theta}=0.88\), in the same units as the observations. Numerically, \(L(0.88)=1/(0.88)^3=1/0.681472\approx1.467\).
The endpoint convention matters. If instead the density is specified to be positive only for \(0\leq y<\theta\), the observation \(y=0.88\) is outside the support at \(\theta=0.88\), and the likelihood there is zero. For every \(\theta>0.88\), the likelihood is \(\theta^{-3}\), which approaches \(0.88^{-3}\) as \(\theta\) decreases toward \(0.88\), but never attains that value. Under this open-endpoint version there is no maximum likelihood estimator for these exact data, only a supremum. Density versions that differ only at an endpoint are equivalent as probability distributions, but their pointwise likelihoods can differ for data exactly at that endpoint.
The score calculation also reveals the problem. For an observation strictly inside the support, the derivative of the log density is \(-1/\theta\), so its expectation under the uniform distribution is \(-1/\theta\), not zero. Differentiating only the density value while ignoring the moving endpoint incorrectly treats the normalization integral as if its limits were fixed. Thus the usual score-mean argument cannot be applied. The information matrix equality and standard likelihood conclusions likewise need not follow from the formal interior derivative alone.
A Moving Boundary in a Shifted Exponential Model
A parameter-dependent support can also produce a likelihood with a sharp edge even when the parameter itself is interior. Consider the shifted exponential density \(f(y;\theta)=\exp[-(y-\theta)]\) for \(y\geq\theta\), and zero otherwise, with \(\theta\in\mathbb{R}\). Its support is \([\theta,\infty)\), which moves with \(\theta\). The model is differentiable in \(\theta\) for observations strictly above the support boundary, but that pointwise fact does not make the model regular.
Worked Example: A Score That Misses the Moving Support
Let the hypothetical observations be \(1.2,1.7,2.0\). Their likelihood is \[ L(\theta)= \begin{cases} \exp[-(1.2+1.7+2.0)+3\theta],&\theta\leq1.2,\\ 0,&\theta>1.2. \end{cases} \] For \(\theta\leq1.2\), this increases with \(\theta\), since its derivative is \(3L(\theta)>0\). The maximum is attained at \(\hat{\theta}=1.2\), the smallest observation. The parameter space is all of \(\mathbb{R}\), so a true value such as \(\theta_0=0\) is interior; nevertheless, the support condition fails.
For an observation strictly above \(\theta\), the log density is \(-(y-\theta)\), and its derivative with respect to \(\theta\) is \(1\). Thus the pointwise score is \(1\) with probability one, giving \(E_\theta[s]=1\), not zero. The missing term is the contribution of the moving lower integration limit: integrating the derivative of the density over \(y\geq\theta\) gives \(1\), while differentiating the lower limit contributes \(-1\). Together they give zero for the derivative of the normalized integral. Ignoring that boundary contribution would lead to an invalid score calculation.
This example distinguishes parameter interiority from support regularity. An interior \(\theta_0\) allows two-sided changes in the parameter space, but it does not prevent the data support from shifting as \(\theta\) changes. The likelihood can have a maximizer at the sample minimum, yet the ordinary smooth score argument is not valid at the support edge.
Boundary Parameters and the Scope of the Conditions
The interior-point condition matters even when the support is fixed. For example, in a Bernoulli model with success probability \(p\in[0,1]\), the support \(\{0,1\}\) does not depend on \(p\). For \(0<p<1\), the score for one observation is \[ s(Y;p)=\frac{Y}{p}-\frac{1-Y}{1-p}, \] and the information is \(I(p)=1/[p(1-p)]\). As \(p\) approaches zero, this information diverges. At \(p=0\), the model assigns probability one to \(Y=0\), and the ordinary two-sided score formula is not defined there. The true parameter is on the boundary, not in an open neighborhood contained in \([0,1]\).
A boundary parameter does not imply that inference is impossible; it means that the standard interior, smooth likelihood approximation may not apply. The same warning applies to the uniform and shifted exponential examples for a different reason: their supports depend on the parameter. Some models violate multiple conditions at once, so the conditions should be checked separately.
A Practical Regularity Check
Before relying on an ordinary likelihood calculation, make the checks explicit. State the parameter space and true value, write the support, and then assess differentiability and information. Finally, ask whether the operation used in the derivation—especially exchanging differentiation and integration—is justified. This check helps distinguish a legitimate likelihood formula from a valid standard inference argument.
Check whether \(\theta_0\) is interior to \(\Theta\). A boundary value may require one-sided or otherwise nonstandard reasoning.
Write the set of possible observations as a function of \(\theta\). If that set changes with \(\theta\), ordinary fixed-support score arguments can miss boundary terms.
Verify the needed derivatives exist and that a suitable domination or integrability condition permits differentiation under the integral.
Confirm that the score has finite second moments and that information is nonsingular when required. Also assess identification and any other assumptions of the particular result being used.
A common mistake is to differentiate the log density on the interior of its support and immediately apply the information matrix equality. For the shifted exponential model, that procedure gives a score of \(1\), whose expectation is not zero; the discrepancy is a clue, not an algebra error. The correction is to inspect the support and the validity of differentiation under the integral. More generally, a finite-looking derivative is not sufficient evidence that the regular likelihood identities hold.
Check Your Understanding
Use the assumptions and examples above to answer the following questions.
- Why does fixed support help justify the zero-mean score identity?
- For the uniform model with observations \(0.20,0.55,0.88\), why does the closed-endpoint density version attain its likelihood maximum at \(0.88\), while the open-endpoint version does not?
- In the shifted exponential example, which term is omitted if one differentiates the density only for \(y>\theta\)?
- Can a true parameter be interior while the model still has parameter-dependent support? Give the example and identify its support.
- For a Bernoulli model with \(p\in[0,1]\), state what happens to \(I(p)\) as \(p\) approaches zero and why \(p=0\) fails the interior-point condition.