Profile Likelihood With an Estimated Variance

Why variance estimation and stopping both matter

When a trial stops at an interim analysis, the observed effect estimate is selected by the monitoring rule: stopping depends on interim data. If the endpoint variance is also estimated, uncertainty in the treatment-effect estimate is not captured by treating the estimated standard error as fixed. A profile likelihood handles the unknown variance by maximizing over it. But using the usual fixed-sample likelihood-ratio cutoff does not, by itself, account for the selection induced by stopping.

Here the treatment effect is a mean difference represented by independent, normally distributed patient-level contrasts, such as paired treatment-control differences. The sequential design examines the accumulating contrasts at prespecified sample sizes and stops according to a prespecified rule. We focus on a confidence interval calibrated conditional on the observed stopping stage. This is one specific coverage target; unconditional coverage across stages is a different target.

Model and assumptions: Let \(X_1,X_2,\ldots\) be independent observations with \(X_i\sim N(\theta,\sigma^2)\), where \(\theta\) is the treatment effect in the units of the endpoint and \(\sigma>0\) is unknown. Analyses occur at prespecified sample sizes \(n_1<\cdots<n_J\). The stopping stage \(K\) is determined by a prespecified rule using the interim data, such as estimated-variance test statistics and efficacy or futility boundaries. The normal model, independent observations, analysis schedule, and stopping rule are assumed correctly specified.

At the observed stopping stage \(K=k\), the available sample has size \(n=n_k\), sample mean \(\bar{x}\), and sample standard deviation \(s\). The likelihood uses all observations collected through that stage. The stopping decision is a deterministic function of the observed data and the design; it does not add a parameter-dependent factor to the likelihood of the observed path. The selection does, however, change the sampling distribution of statistics conditional on \(K=k\). We account for that change in the calibration step.

Profiling out the unknown variance

For the \(n\) observed normal outcomes, the log likelihood, apart from a constant, is

\ell(\theta,\sigma^2) =-\frac{n}{2}\log(\sigma^2) -\frac{1}{2\sigma^2}\sum_{i=1}^n(X_i-\theta)^2.

The unrestricted maximum occurs at \(\hat{\theta}=\bar{x}\) and \(\hat{\sigma}^2=(n-1)s^2/n\). For a proposed value \(\theta_0\), maximize the likelihood over \(\sigma^2\) while holding \(\theta=\theta_0\). The corresponding variance estimate is

\hat{\sigma}^2(\theta_0) =\frac{1}{n}\sum_{i=1}^n(X_i-\theta_0)^2 =\frac{(n-1)s^2+n(\bar{x}-\theta_0)^2}{n}.

The profile likelihood-ratio statistic is twice the log of the unrestricted maximum likelihood divided by the maximum likelihood subject to \(\theta=\theta_0\). Substituting the two variance estimates gives

W_k(\theta_0) =2\{\ell(\hat{\theta},\hat{\sigma}^2)-\ell(\theta_0,\hat{\sigma}^2(\theta_0))\} =n\log\left(1+\frac{n(\bar{x}-\theta_0)^2}{(n-1)s^2}\right).

Definition: \(W_k(\theta_0)\) is the profile likelihood-ratio statistic for the candidate treatment effect \(\theta_0\), calculated from the data available at stopping stage \(k\). It profiles out the unknown variance rather than substituting a fixed known value for \(\sigma^2\). The subscript \(k\) also reminds us that its calibration must correspond to the observed stopping stage.

This formula requires \(n\geq 2\) and \(s>0\). It is a statistic calculated from the stopped sample, not a stopping-adjusted critical value. In a fixed-sample analysis, \(W_k\) is often compared with a chi-square reference distribution with one degree of freedom. After sequential stopping, that reference distribution need not give the intended coverage, even though the formula for \(W_k\) is still the same.

Worked Example: Profiling the Variance at an Early Stop

Suppose a hypothetical trial stops at its second look with \(n=20\) paired contrasts. The observed mean contrast is \(\bar{x}=1.8\) endpoint units and the sample standard deviation is \(s=4.0\) endpoint units. Calculate the profile likelihood-ratio statistic for \(H_0:\theta=0\).

The normal model is being used for the independent patient-level contrasts; \(n=20\), and the observed \(s\) is positive. Thus the estimated-variance profile formula applies:

W_2(0) =20\log\left(1+\frac{20(1.8-0)^2}{19(4.0)^2}\right) =20\log\left(1+\frac{64.8}{304}\right).

Since \(64.8/304=0.2131579\), this is \(20\log(1.2131579)\approx 3.865\). The statistic is dimensionless; the effect estimate and standard deviation were both measured in endpoint units. This number alone does not determine whether zero is excluded from a stopping-adjusted interval: that requires a cutoff calibrated for this design and the observed stage.

Calibrating for the observed stopping stage

A stage-conditional interval is constructed by testing each candidate value \(\theta_0\) using the distribution of \(W_K(\theta_0)\) conditional on stopping at the observed stage. Because the variance is unknown, that conditional distribution can depend on nuisance parameters. A conservative, explicit calibration uses a cutoff large enough to control rejection probability for every admissible treatment effect and variance.

Definition: For a design with stopping stage \(K\), define a stage-\(k\) cutoff \(c_{k,\alpha}\) as a number satisfying \(P_{\theta,\sigma}\{W_k(\theta)\leq c_{k,\alpha}\mid K=k\}\geq 1-\alpha\) for every \(\theta\) and \(\sigma>0\) for which \(P_{\theta,\sigma}(K=k)>0\). Equivalently, one can take the supremum over those parameters of the conditional \(1-\alpha\) quantiles. The resulting cutoff is design-specific; it is not generally the usual chi-square cutoff.

The definition gives a precise calibration target. In practice, the conditional quantiles can be evaluated numerically from the specified normal model and stopping rule, or estimated by repeated simulation under candidate parameter values, retaining runs that stop at stage \(k\). The nuisance parameters must be varied or conservatively bounded. Using a single plug-in variance estimate to choose the cutoff is an approximation, not a guarantee of the stated uniform conditional coverage.

For a candidate \(\theta_0\), reject \(H_0:\theta=\theta_0\) at stage \(k\) when \(W_k(\theta_0)>c_{k,\alpha}\). Inverting these tests yields

C_k=\{\theta_0:W_k(\theta_0)\leq c_{k,\alpha}\}.

By the definition of the cutoff, for each admissible \(\theta\) and \(\sigma\), \(P_{\theta,\sigma}\{\theta\in C_k\mid K=k\}\geq 1-\alpha\). Thus the interval has at least the nominal coverage conditional on reaching stage \(k\), under the stated model and calibration. Conditioning matters: this statement does not claim that the probability the fixed parameter lies in the observed interval is \(1-\alpha\).

Worked calculations and interval inversion

Worked Example: Using a Stage-Specific Cutoff

Consider a hypothetical design that stops at stage \(k=2\). Suppose its prespecified, nuisance-conservative conditional calibration gives \(c_{2,0.05}=5.00\). At that stage, let \(n=20\), \(\bar{x}=1.8\) endpoint units, and \(s=4.0\) endpoint units, as in the preceding calculation. Is \(\theta_0=0\) in the inverted 95% interval?

The stage-adjusted test compares \(W_2(0)\approx3.865\) with \(5.00\). Since \(3.865\leq5.00\), the test does not reject zero, so zero belongs to the interval. By contrast, the common fixed-sample reference cutoff \(3.841\) would give \(3.865>3.841\) and exclude zero. The two decisions differ because the illustrative sequential cutoff is calibrated for the stopping design and stage. The value \(5.00\) is hypothetical, not a universal cutoff.

Worked Example: A Different Stage and Estimated Variance

In a second hypothetical trial, suppose the final available sample has \(n=10\), \(\bar{x}=3.0\) endpoint units, and \(s=\sqrt{10}\approx3.1623\) endpoint units. For \(H_0:\theta=0\), calculate the statistic and use an illustrative stage-specific cutoff of \(7.40\).

Using the unrounded value \(s^2=10\), the calculation is

W_k(0) =10\log\left(1+\frac{10(3.0)^2}{9(10)}\right) =10\log(2) \approx 6.931.

The statistic is less than \(7.40\), so zero is retained in the interval obtained by inverting this stage-specific test. If instead the displayed rounded standard deviation \(s=3.162\) is substituted, the statistic is approximately \(6.932\); either calculation leads to the same decision. The cutoff is again an explicitly hypothetical design-calibration output, not a general rule.

Worked Example: Inverting a Calibrated Test to Find Limits

Suppose a hypothetical design's stage-specific cutoff is \(c_{k,0.05}=4.80\), and the observed stopped sample has \(n=12\), \(\bar{x}=2.0\) endpoint units, and \(s=3.0\) endpoint units. Because this example uses a fixed cutoff for all candidate values, solve \(W_k(\theta_0)\leq4.80\) directly.

Rearranging the profile statistic gives

|\bar{x}-\theta_0| \leq s\sqrt{\frac{n-1}{n}\{\exp(c_{k,0.05}/n)-1\}}.

Substituting the values, the half-width is \(3.0\sqrt{(11/12)\{\exp(4.80/12)-1\}} =3.0\sqrt{(11/12)(0.491825)} \approx2.014\) endpoint units. The inverted interval is therefore approximately \([2.0-2.014,\;2.0+2.014]=[-0.014,\;4.014]\) endpoint units. Its slight extension below zero is permitted by this unconstrained treatment-effect model. As with the other calibration values, \(4.80\) is hypothetical and applies only if justified for the specified design and stage.

Why the calibration changes, and what to check

The profile statistic accounts for estimating \(\sigma^2\), but stopping changes which datasets are observed at a given stage. For example, if a trial stops early for efficacy when an interim statistic is unusually large, an endpoint statistic at that stage is not distributed as it would be under a fixed sample size chosen in advance. The cutoff must reflect the monitoring boundaries, look schedule, variance estimation, and the event \(K=k\).

A useful justification for inversion is direct. Fix a true value \(\theta\). The event that the reported confidence set misses it is exactly the event that its test rejects it:

\{\theta\notin C_K\}=\{W_K(\theta)>c_{K,\alpha}\}.

Conditional on any stage with positive probability, the calibrated cutoff makes the probability of the right-hand event at most \(\alpha\). Hence the conditional probability of inclusion is at least \(1-\alpha\). This argument depends on using the correct conditional distribution and on the assumed model; an inaccurate model or stopping rule invalidates the guarantee.

Approach	What it does	Main limitation
Fixed-sample profile likelihood interval	Uses the estimated-variance profile statistic and a fixed-sample reference cutoff.	Does not generally control coverage conditional on the observed stopping stage.
Stage-conditional calibrated profile interval	Uses the same profile statistic but calibrates its cutoff under the stopping rule, conditional on the observed stage.	Needs the design rule and nuisance-aware calibration; conditional intervals may be conservative.
Unconditional stopping-adjusted interval	Calibrates coverage over the distribution of stopping stages rather than separately within each stage.	Targets a different coverage criterion; it need not have nominal coverage conditional on every stage.

Practical diagnostics include checking that the simulated or numerically evaluated stopping frequencies match the specified rule, that stage-specific conditional samples are large enough for stable tail quantiles, and that calibration explores a sufficiently broad range of nuisance values. A very rare stopping stage can make conditional calibration unstable or highly conservative. If no runs stop at a stage for a particular parameter configuration, its conditional distribution is undefined there; the cutoff definition applies only where the stage has positive probability.

A common mistake is to calculate \(W_k(\theta_0)\), compare it with the usual fixed-sample chi-square cutoff, and conclude that profiling has already corrected for early stopping. Profiling addresses the unknown variance; it does not correct the distributional selection caused by the stopping rule. Use a cutoff calibrated for the stopping design and the coverage target, and report whether that target is conditional on the observed stage or unconditional across stages.

Key takeaway: The estimated-variance profile likelihood statistic is \(n\log\{1+n(\bar{x}-\theta_0)^2/[(n-1)s^2]\}\). After stopping, a confidence interval requires a reference distribution calibrated to the stopping rule and the intended coverage target, not merely the fixed-sample cutoff.

Check Your Understanding

Answer each question, then compare with the solutions below.

For a normal sample with unknown variance, what quantity is maximized over when forming the profile likelihood for a candidate treatment effect?
A stopped-sample statistic is below the usual fixed-sample cutoff. Is that alone sufficient to establish stage-conditional coverage? Why or why not?
For \(n=12\), \(\bar{x}=2.0\), and \(s=3.0\), calculate \(W_k(0)\) to two decimal places.
What coverage statement does a cutoff calibrated conditional on \(K=k\) support?
Name two design or model features that must be represented in stage-specific calibration.

Solutions. (1) Maximize the likelihood over the nuisance variance \(\sigma^2\) while holding the candidate \(\theta_0\) fixed. (2) No. The fixed-sample cutoff need not match the statistic's distribution conditional on the stopping stage. (3) \(W_k(0)=12\log\{1+12(2)^2/[11(3)^2]\}=12\log(1.484848)\approx4.74\). (4) Under the model, conditional on reaching stage \(k\), the probability the interval contains the true treatment effect is at least \(1-\alpha\). (5) Examples include the analysis schedule, stopping boundaries, variance estimation, and the assumed outcome distribution.