Small-Sample Calibration of Likelihood-Ratio Tests

Why Adjust a Likelihood-Ratio Test?

Wilks’ theorem gives a useful large-sample reference distribution for a likelihood-ratio (LR) statistic: under regularity conditions and the null hypothesis, it approaches a chi-square distribution. In a modest sample, however, the statistic’s finite-sample distribution can differ systematically from that limiting reference. When the leading discrepancy is a predictable inflation in the statistic, Bartlett correction rescales it before applying the same chi-square calibration.

The correction is not a new test statistic with a different number of restrictions, and it does not make an approximate test exact. It is a higher-order refinement: a model-specific adjustment designed to remove a leading finite-sample error in regular likelihood problems. Its validity depends on conditions that support the relevant likelihood expansions.

Definition: For a null hypothesis imposing $q$ regular restrictions, let $W=2\{\ell(\hat{\theta})-\ell(\hat{\theta}_0)\}$ be the LR statistic, where $\ell(\hat{\theta})$ is the maximized unrestricted log likelihood and $\ell(\hat{\theta}_0)$ is the maximized log likelihood subject to the null. If, under the null, its expectation has the expansion $$ E_0(W)=q\left(1+\frac{b}{n}\right)+O(n^{-2}), $$ then $b$ is the leading Bartlett coefficient under this convention. The Bartlett-corrected statistic is $$ W_B=\frac{W}{1+b/n}. $$ Under suitable higher-order regularity conditions, $W_B$ has a more accurate chi-square approximation than $W$, with reference distribution $\chi^2_q$.

Conditions and Interpretation

The starting point is the same regular likelihood setting used for Wilks’ theorem. The observations are typically independent and identically distributed from a correctly specified, identifiable model. The parameter is finite-dimensional and fixed as $n$ grows; the true parameter is interior; and the information matrix is nonsingular. The log likelihood must be sufficiently smooth, with suitable moments and derivatives for the likelihood and distributional expansions. The null restrictions must be regular, so that their number $q$ determines the degrees of freedom in Wilks’ theorem.

A Bartlett coefficient is model-specific. It may depend on nuisance parameters, in which case those parameters must be handled according to the correction’s derivation—for example, by a consistent plug-in estimate when justified. Neither the LR statistic nor a numerical sample size alone determines $b$. An analyst needs a coefficient derived for the likelihood at hand, or another valid method of obtaining it.

The adjustment’s intuition is easiest to see when $b>0$. The uncorrected statistic is, on average under the null, larger than $q$ by a leading relative amount of about $b/n$. Dividing by $1+b/n$ reduces that inflation. When $b<0$, the factor is below one and the correction instead increases the statistic. In either case, the degrees of freedom remain $q$.

Worked Example: Deriving a Bartlett Coefficient for an Exponential Model

Suppose independent observations $X_1,\ldots,X_n$ have an exponential distribution with unknown mean parameter $\theta>0$, with density $f(x\mid\theta)=\theta^{-1}\exp(-x/\theta)$ for $x>0$. Test $H_0:\theta=\theta_0$ against an unrestricted alternative. The model is smooth and identifiable for $\theta>0$; the null value is assumed to be interior. This is a one-restriction test, so $q=1$.

The log likelihood is $\ell(\theta)=-n\log\theta-\sum_i X_i/\theta$, and its unrestricted maximum occurs at $\hat{\theta}=\bar{X}$. Substituting the restricted and unrestricted estimates gives

W=2n\left\{\log\left(\frac{\theta_0}{\bar{X}}\right)+\frac{\bar{X}}{\theta_0}-1\right\}.

Under $H_0$, let $Y=\sum_i X_i/\theta_0$. Then $Y$ has a gamma distribution with shape $n$ and rate $1$, $E_0(Y)=n$, and $\bar{X}/\theta_0=Y/n$. The standard gamma expectation $E[\log Y]=\psi(n)$, where $\psi$ is the digamma function, therefore gives

E_0(W)=2n\left\{\log n-\psi(n)\right\}.

Using the large-$n$ expansion $\psi(n)=\log n-\frac{1}{2n}-\frac{1}{12n^2}+O(n^{-4})$, we obtain

E_0(W)=1+\frac{1}{6n}+O(n^{-3}).

Thus the leading coefficient is $b=1/6$. This derivation identifies the adjustment for this particular model; it is not a coefficient to transfer to other likelihoods.

Applying the Adjustment

Once the model-specific coefficient is available, the calculation is direct: compute the usual LR statistic, divide by $1+b/n$, and compare the result with a chi-square distribution on $q$ degrees of freedom. The uncorrected and corrected statistics use the same $q$. The following example shows the arithmetic and how the resulting p-value can change.

Worked Example: Corrected and Uncorrected Calibration in a Small Sample

In a hypothetical sample of $n=8$ exponential observations, test $H_0:\theta_0=1$ against a two-sided alternative. Suppose the observed sample mean is $\bar{X}=1.4$. The exponential-model derivation above supplies $b=1/6$; the null value is positive and interior, and the observations are assumed independent from the stated exponential model.

There is one restriction, so the reference distribution is $\chi^2_1$. The uncorrected statistic is

W=2(8)\left\{\log(1/1.4)+1.4-1\right\} =16\{-0.33647+0.40000\} =1.0165.

The Bartlett factor is $1+b/n=1+1/(6\cdot 8)=1.02083$, giving

W_B=\frac{1.0165}{1.02083}=0.9957.

Using the $\chi^2_1$ reference distribution, the uncorrected p-value is approximately $0.313$, while the corrected p-value is approximately $0.318$. Both exceed $0.05$, so neither calibration rejects $H_0$ at the five-percent level. The correction makes a modest difference here; it does not change the conclusion. These are chi-square-calibrated p-values, not exact finite-sample p-values.

The adjustment can also matter when a statistic is closer to a decision threshold. The next calculation illustrates the mechanics for a hypothetical regular model with two restrictions. Its coefficient is stipulated as an output of that model’s derivation; it is not inferred from the observed LR statistic.

Worked Example: A Two-Restriction Test Near the Decision Threshold

Suppose a hypothetical regular likelihood has $n=20$ independent observations and a null hypothesis imposing $q=2$ restrictions. For this likelihood, assume the derived Bartlett coefficient is $b=0.8$. The unrestricted and restricted maximized log likelihoods are $-41.30$ and $-44.90$, respectively. The model and null are assumed to meet the regularity conditions for the correction.

First compute the LR statistic from the log likelihoods:

W=2\{-41.30-(-44.90)\}=2(3.60)=7.20.

The correction factor and adjusted statistic are

1+\frac{b}{n}=1+\frac{0.8}{20}=1.04, \qquad W_B=\frac{7.20}{1.04}=6.923.

For two degrees of freedom, the chi-square survival probability at $x$ is $\exp(-x/2)$. The uncorrected p-value is therefore $\exp(-7.20/2)=0.02732$; the corrected p-value is $\exp(-6.923/2)=0.03136$. Both are below $0.05$, so both calibrations reject the null at the five-percent level, although the corrected p-value is larger. The two degrees of freedom have not changed: the adjustment rescales the statistic, not the number of restrictions.

When No Correction Is Needed

Not every likelihood-ratio test has a nonzero leading finite-sample bias. In some models, the uncorrected statistic already has an exact chi-square distribution under the null. In that case, applying a nonzero adjustment would make the calibration worse, not better.

Worked Example: An Exact Normal-Mean LR Test

Let $X_1,\ldots,X_9$ be independent $N(\mu,2^2)$ observations, where the variance is known. Test $H_0:\mu=0$ against $H_a:\mu\ne0$. The null value is interior, the model is identifiable and smooth, and there is one restriction. Suppose the observed sample mean is $\bar{x}=1.0$ measurement unit.

The unrestricted estimate is $\hat{\mu}=\bar{x}$. The normal log likelihood implies

W=\frac{n(\bar{x}-0)^2}{\sigma^2} =\frac{9(1.0)^2}{2^2} =2.25.

Under the null, $\sqrt{n}\bar{X}/\sigma$ is exactly standard normal. Its square is exactly $\chi^2_1$, so this LR statistic has an exact $\chi^2_1$ null distribution in this model. Its p-value is approximately $0.134$. There is no leading finite-sample discrepancy of the kind the Bartlett adjustment is designed to remove; the appropriate correction is $b=0$.

Limits and Common Pitfalls

A Bartlett correction refines a regular large-sample approximation. It does not repair a failure of the underlying likelihood theory. In particular, the usual justification should not be assumed in problems with boundary parameters, unidentified parameters under the null, weak identification, or other nonstandard likelihood limits. Such settings can have reference distributions other than $\chi^2_q$, so rescaling $W$ and retaining that reference distribution may be invalid.

The correction is also not a replacement for checking model specification, independence, or the assumptions used to derive $b$. If the coefficient depends on unknown nuisance parameters, simply plugging in an arbitrary value does not establish the higher-order result. At very small $n$, the factor $1+b/n$ may be an unreliable approximation; when it is zero or otherwise unsuitable, the correction cannot sensibly be applied in this form.

Mistake: changing the degrees of freedom. The number of restrictions $q$ determines the chi-square degrees of freedom. Bartlett correction rescales $W$; it does not replace $q$ with a new value.
Mistake: using a universal coefficient. The coefficient comes from the model and test’s higher-order expansion. A value derived for one likelihood does not automatically apply to another.
Mistake: treating the corrected p-value as exact. The correction improves an asymptotic calibration under stated conditions. It does not make the p-value an exact finite-sample probability.
Mistake: applying the correction to a nonstandard limit. If the null lies on a boundary or parameters are unidentified under the null, Wilks’ usual chi-square reference may already fail. The Bartlett adjustment described here does not resolve that failure.

The practical value of Bartlett correction is that a known leading finite-sample distortion can be removed with a transparent, model-specific rescaling. Its usefulness depends on respecting both parts of the procedure: derive or justify the coefficient, and verify that the regular chi-square framework applies.

Check Your Understanding

Use the definitions and examples above to answer the following questions.

In a regular test with $q=3$ restrictions, what determines the chi-square degrees of freedom after Bartlett correction?
If $E_0(W)=q(1+b/n)+O(n^{-2})$, write the corrected statistic and explain what leading discrepancy it addresses.
Why can the exponential-model coefficient $b=1/6$ not be assumed for a different likelihood?
In the two-restriction example, why did the corrected p-value increase even though the degrees of freedom stayed fixed?
Name two types of nonstandard problems in which the usual Bartlett correction may not be justified.