Parametric Bootstrap Predictive Envelopes

Why Use a Predictive Envelope?

A residual plot or a grouped outcome table shows what happened in the observed data, but it does not by itself show how much variation a fitted GLMM would typically produce. A parametric bootstrap predictive envelope adds that reference: it displays the range of a chosen residual summary or outcome summary across data sets generated under the fitted model. An observed feature outside the envelope is evidence of a discrepancy relative to that model and simulation scheme—not, by itself, proof of which model assumption failed.

An envelope is useful when a single test statistic would hide the pattern of misfit. For example, an envelope across several fitted-value bins can show where outcome counts are unexpectedly high or low. A residual envelope can show whether an observed residual pattern is larger than the patterns produced by the fitted model. The conclusions depend on specifying the model, the replication scheme, and the summary being compared.

Model and Envelope Definitions

Suppose the observed responses are \(Y=(Y_1,\ldots,Y_n)\), with fixed covariates and cluster membership. A GLMM specifies a conditional response distribution given random effects \(b\), and a distribution for those random effects. Write its parameter vector as \(\theta\), including fixed-effect and random-effect distribution parameters. The model also specifies the link and any offsets or exposure terms. Let \(\hat{\theta}\) denote the estimate from the observed data.

For the usual parametric bootstrap, assume that the fitted GLMM is the data-generating model for the check, that its conditional response distributions and random-effects distribution are correctly specified, and that the same design and observation structure can be simulated. The procedure uses \(\hat{\theta}\) as a plug-in value. It therefore approximates sampling variation under the fitted model; it does not account for every uncertainty about whether that model is true.

The previous material on conditional versus unconditional replications distinguishes two simulation targets. Conditional replications hold estimated cluster effects fixed and simulate outcomes given them; unconditional replications also draw new random effects from the fitted random-effects distribution. These targets answer different questions. State which is used: a conditional envelope checks outcome variation given the estimated effects, while an unconditional envelope includes variation from the model’s random-effects distribution.

When summaries involve fitted values, residuals, or estimated parameters, decide whether each bootstrap data set will be refitted. The observed summary and every replicate summary must be calculated by the same rule. Refitting is usually important when the diagnostic itself depends on fitted quantities: otherwise, the bootstrap summaries can omit the fitting step that shaped the observed residuals. For a direct outcome-count envelope based on fixed groups and fitted probabilities, one may instead hold the original fitted parameters fixed, provided the target is explicitly the plug-in predictive distribution.

Constructing the Envelope

Choose the summary before examining the bootstrap results. Examples include counts in prespecified fitted-value bins, residual means by covariate group, or a maximum absolute residual. Simulate \(B\) data sets under the stated fitted model. For each replicate, apply the chosen rule, including refitting if specified, and record \(T^{(b)}=(T_1^{(b)},\ldots,T_J^{(b)})\). For a central pointwise 95% envelope, the limits for component \(j\) are the empirical 2.5th and 97.5th percentiles of its \(B\) simulated values. Compare the observed \(T_j(Y)\) with those limits.

The envelope is pointwise: each interval targets the marginal distribution of one component. It does not generally have 95% probability of covering all \(J\) observed components together. Looking at many pointwise intervals increases the chance that at least one will be crossed even when the model is adequate. A simultaneous envelope addresses that issue by calibrating a maximum deviation across components.

Justification and Simultaneous Limits

The basis of a pointwise envelope is the bootstrap distribution of a summary under the fitted model. If model parameters were known and the simulation exactly reproduced the data-generating process, the observed data and independent simulated data sets would have the same distribution. Thus their summary values would be exchangeable, and an interval covering the central 95% of that distribution would contain a new summary about 95% of the time, apart from discreteness and Monte Carlo error. With estimated parameters, the procedure is a plug-in approximation: estimating \(\theta\) can change the distribution, so this coverage is not an exact guarantee in general.

For a simultaneous envelope, first estimate a center \(m_j\) and scale \(s_j\) for each component from the bootstrap summaries, such as the bootstrap mean and standard deviation. For replicate \(b\), calculate the maximum absolute standardized deviation \(M^{(b)}=\max_j |(T_j^{(b)}-m_j)/s_j|\). Let \(c\) be the empirical 95th percentile of these maxima. Display limits \(m_j-cs_j\) and \(m_j+cs_j\). Since every component’s standardized deviation is no greater than the maximum, a simulated vector with \(M^{(b)}\le c\) lies inside all these limits. This calibration aims for 95% joint coverage under the bootstrap distribution, subject to plug-in and Monte Carlo approximation. Components with zero or extremely small simulated spread need special handling; division by zero does not define a standardized deviation.

Diagnostics, Alternatives, and Limitations

A useful envelope needs more than a plotted band. Report the GLMM and its fitted parameters or fitting method, the number of replicates, the conditional or unconditional scheme, the refitting rule, the definition of the summaries, and the quantile convention. If the envelope is estimated from few replicates, its limits can be noisy. For a 95% interval from \(B=199\), only about five simulated values are expected in each 2.5% tail; \(B=999\) gives about 25 per tail. Increasing \(B\) reduces simulation error but does not repair a misspecified model or an unsuitable diagnostic.

Check for fitting failures and implausible replicate behavior. If some simulated data sets produce convergence failures, separation, boundary estimates, or undefined summaries, the successful replicates may no longer represent the intended bootstrap distribution. Do not silently discard such replicates. Investigate whether the fitting procedure is unstable, state the failure rate, and use a principled, consistent rule for handling failures. Also check that observed and replicate summaries use identical bins, residual definitions, and covariate grouping.

The main alternatives answer related but different questions. A pointwise envelope is easy to interpret locally, but does not adjust for inspecting many summaries. A simultaneous envelope controls the joint range for a specified vector, but can be wider and depends on how that vector is defined. A single discrepancy statistic compresses the data into one number and may be easier to summarize, but can conceal where the model misses. Conditional simulations isolate outcome variation given estimated cluster effects; unconditional simulations additionally reflect random-effect variation. Neither target is universally preferable: the choice follows the diagnostic question.

A parametric bootstrap is a frequentist plug-in procedure based on a fitted model. It is not a posterior predictive check, which averages predictions over a posterior distribution of parameters. It also cannot verify the random-effects distribution or response distribution merely by generating data from them: the envelope evaluates the observed data against assumptions already built into the generator. Sparse outcomes, few clusters, weakly identified variance components, and boundary estimates can make the bootstrap approximation fragile. Interpret an outlying summary as a reason to investigate the particular pattern, not as an automatic instruction to add a specific model component.

Common Mistake

Wrong reasoning: “All observed summaries lie inside their 95% pointwise intervals, so the whole model has passed a 5% test.” This treats separate intervals as a joint 95% region and treats a diagnostic as a binary certification. With many summaries, even a well-fitting model may produce one pointwise crossing; conversely, an envelope may miss a discrepancy that the chosen summaries do not capture.

Correction: Name the target and limits precisely. For a family of summaries, use a simultaneous envelope when joint containment is the goal, and explain that its calibration is still conditional on the fitted model and bootstrap procedure. Regardless of envelope type, examine the shape and location of discrepancies and use subject-matter and design knowledge to interpret them.