Why extreme predictor values need careful diagnosis
A continuous predictor can take values far beyond the range of most observations. If one such observation has a binary outcome different from the outcomes at ordinary predictor values, a scatterplot may suggest an unusually strong classification pattern. The pattern might reflect genuine separation across a range of predictor values; it might instead be driven almost entirely by one influential observation. Those possibilities have different implications for interpreting the fitted slope and its uncertainty.
In a generalized linear mixed model (GLMM), the distinction also depends on the clustering structure. Outcomes from the same cluster share a random effect, so a pattern that appears after pooling clusters may differ from the pattern within clusters. The appropriate diagnosis combines plots, model-based fitted probabilities, and influence checks; no single plot or software warning settles the question.
For a continuous predictor \(x\), begin by examining the observed outcomes against \(x\), using separate symbols or panels for clusters where useful. Look for overlap: both outcome values occurring over comparable predictor ranges. Then note gaps, sparse regions, and isolated observations at the ends of the range. An isolated point is not automatically an error, and a broad monotone pattern is not automatically a well-supported effect. The key question is how much of the apparent classification is supported by observations near that point and in other clusters.
A scatterplot diagnoses the data pattern, not the fitted model by itself. In a GLMM, \(\operatorname{logit}^{-1}(\mathbf{x}_{ij}^{\mathsf T}\boldsymbol{\beta})\) is not generally the marginal probability \(P(Y_{ij}=1\mid\mathbf{x}_{ij})\): the latter averages over the random-effects distribution. Conditional predictions, which include an estimated cluster effect, answer a different question from predictions averaged over clusters. State which predictions are being inspected.
Worked Example: An Isolated High Value Amid Overlap
Consider hypothetical observations from one cluster, with a continuous predictor \(x\) and binary outcome \(y\): \((0,0),(1,0),(2,1),(3,0),(4,1),(5,1),(12,1)\). The point at \(x=12\) is far from the other six observations. Does the plot establish complete separation by \(x\)?
No. The interior observations already include both outcomes: at \(x=2\) and \(x=4\) the outcome is 1, while at \(x=3\) it is 0. A threshold cannot place all 1s on one side and all 0s on the other, because the 1 at \(x=2\) lies below the 0 at \(x=3\). The point at \(x=12\) may strengthen the apparent positive trend, but it does not remove that overlap.
The appropriate conclusion is limited: the extreme point may be influential, and the observed data do not show complete separation by a threshold in \(x\). This does not establish that the point is erroneous, nor does it guarantee that a multi-predictor or cluster-conditional model has no separation. Other predictors and cluster effects must be considered in the fitted model.
What an extreme observation contributes to a logistic fit
A useful way to understand potential influence is to examine the conditional likelihood with the random effects held fixed. This is a local diagnostic for the fixed-effect coefficient, not a replacement for the marginal GLMM likelihood, which integrates over the random-effects distribution. For a Bernoulli observation, the conditional log-likelihood contribution is
Differentiating with respect to \(\boldsymbol{\beta}\) gives the score contribution \(\mathbf{x}_{ij}(y_{ij}-p_{ij})\). Differentiating again gives the negative Hessian, or observed information contribution, \(p_{ij}(1-p_{ij})\mathbf{x}_{ij}\mathbf{x}_{ij}^{\mathsf T}\). The random-effect term belongs in \(\eta_{ij}\), even though the derivative of \(\eta_{ij}\) with respect to \(\boldsymbol{\beta}\) is \(\mathbf{x}_{ij}\).
For a model with intercept and slope for \(x\), \(\mathbf{x}_{ij}=(1,x_{ij})^{\mathsf T}\). The slope-slope entry of this information contribution is \(p_{ij}(1-p_{ij})x_{ij}^2\). The \(x^2\) factor means an extreme value can have a large effect on the slope's information and score. But the weight \(p_{ij}(1-p_{ij})\) is small when the fitted probability is near 0 or 1. Thus, extremeness alone does not tell us the size or direction of influence; the fitted probability, outcome, and the rest of the design all matter.
Worked Example: Checking a Point's Slope Information
Suppose a hypothetical conditional logistic fit, with random effects held at their fitted values, has \(\hat{\eta}=-1+0.20x\). Compare the slope-information contributions at \(x=12\) and \(x=2\). These calculations describe this stated fit; they do not claim that it was estimated from a particular dataset.
At \(x=12\), \(\hat{\eta}=-1+0.20(12)=1.4\), so \(\hat{p}=1/(1+\exp(-1.4))\approx0.8022\). The slope-information contribution is \(0.8022(1-0.8022)(12^2)=0.8022(0.1978)(144)\approx22.85\).
At \(x=2\), \(\hat{\eta}=-1+0.20(2)=-0.6\), so \(\hat{p}=1/(1+\exp(0.6))\approx0.3543\). The contribution is \(0.3543(1-0.3543)(2^2)=0.3543(0.6457)(4)\approx0.9151\).
The high-\(x\) observation contributes much more to this entry of the conditional information matrix in this particular fit. This number alone is not a complete influence measure: the score also depends on \(y-\hat p\), and the fitted model and remaining observations jointly determine the estimate. In particular, a large information contribution should not be read as proof that the point is either harmless or responsible for separation.
Separate a stable pattern from a single-point effect
A practical diagnosis asks whether the pattern persists when attention is restricted to the ordinary range of \(x\), and whether it appears within more than one cluster. A continuous predictor that separates outcomes over a broad, populated range presents a different evidential pattern from one whose apparent predictive power comes from a single endpoint observation. For multi-predictor models, a one-dimensional plot is only a marginal view: separation may depend on a combination of predictors even when each separate plot shows overlap.
Fitted probabilities are another useful view. Plot conditional fitted probabilities against \(x\), preferably with observed outcomes shown and clusters identified. Probabilities close to 0 or 1 across a broad range can be consistent with strong classification; an abrupt change driven by one isolated \(x\) deserves scrutiny. In a random-intercept model, show or inspect cluster-conditional predictions as appropriate, and distinguish them from predictions averaged over the random-effects distribution. Estimated random effects are themselves uncertain, especially in small clusters.
Worked Example: Why Clusters Change the Plot's Interpretation
Suppose two hypothetical clusters each contain an observation at \(x=8\). In cluster A the outcome is 1; in cluster B it is 0. For illustration, consider a fitted random-intercept logistic model with \(\hat\eta=-1+0.20x+\hat b_j\), where \(\hat b_A=1\) and \(\hat b_B=-1\). The values are illustrative conditional predictions, not estimates from real data.
For cluster A, \(\hat\eta_A=-1+0.20(8)+1=1.6\), giving \(\hat p_A=1/(1+\exp(-1.6))\approx0.8320\). For cluster B, \(\hat\eta_B=-1+0.20(8)-1=-0.4\), giving \(\hat p_B=1/(1+\exp(0.4))\approx0.4013\).
The same extreme predictor value is paired with different outcomes in the two clusters. A pooled plot could make the predictor appear weakly discriminating at \(x=8\), while the cluster-conditional fitted probabilities differ because the model includes cluster effects. The example shows why cluster labels and conditional predictions matter; it does not show that the random effects have been estimated precisely or that the model is adequate. Inspect the number and range of observations in each cluster before interpreting the conditional pattern.
Display \(y\) against the continuous predictor, mark clusters, and identify overlap, gaps, and isolated endpoint observations.
State whether predictions are conditional on fitted cluster effects or averaged over the random-effects distribution. Check whether extreme probabilities occur broadly or are concentrated around a sparse endpoint.
Refit after a justified deletion or perturbation, and compare the continuous-predictor coefficient, uncertainty, and predicted probabilities. Also check whether the assumed functional form is plausible over the observed range.
Say whether evidence of separation is broad or appears driven by a few observations, and note any limitations from sparse clusters or extrapolation.
Diagnostics, alternatives, and limitations
Deletion checks are informative when used as sensitivity analyses. Refit the same model after removing a suspect observation only when the deletion is clearly identified as a diagnostic, not as an unreported way to obtain a preferred estimate. Compare the slope and its uncertainty, conditional predictions, and whether separation diagnostics change. In clustered data, also consider whether one entire cluster is influential; deleting one row and deleting a cluster answer different questions. The analysis must preserve the dependence structure in each refit.
A change in the slope after deleting one extreme point is evidence that the estimate is sensitive to that observation, not proof that the observation is invalid. Conversely, a small change does not establish that the model is correctly specified. Inspect data provenance, units, coding, and measurement range. Consider whether a nonlinear association is scientifically plausible, but do not add curvature merely to hide an inconvenient point; any alternative form must be justified and checked against the observed data.
These diagnostics complement, rather than replace, separation checks based on the design and fitted likelihood. The earlier material on geometric separation and score, information, and profile-likelihood diagnostics addresses model-wide evidence. A scatterplot for one predictor cannot rule out separation involving several predictors, factor contrasts, or cluster-specific patterns. A convergence warning is also not itself proof of separation. The earlier material on distinguishing separation from convergence failure is relevant when warnings and plots disagree.
| Approach | What it helps assess | Important limitation |
|---|---|---|
| Outcome scatterplot, marked by cluster | Overlap, gaps, and whether a pattern is supported across the predictor range | Shows one predictor at a time and does not diagnose every multivariable separating direction |
| Conditional fitted-probability plot | How the fitted model classifies observations within clusters | Depends on estimated random effects and can be unstable in small clusters |
| Marginal fitted probabilities | Predicted outcome probabilities averaged over the random-effects distribution | Do not describe cluster-specific predictions; averaging can obscure cluster patterns |
| Deletion or perturbation refit | Sensitivity of estimates and predictions to a particular point or cluster | Is a sensitivity analysis, not a license to discard valid data |
| Design, score, or profile diagnostics | Evidence of separation or weak identification in the fitted model | Do not by themselves explain whether the cause is a genuine broad pattern or a single extreme observation |
The central limitation is that diagnostics are conditional on the observed design and the model's assumptions. Sparse observations at the predictor range's edge make both the slope and predictions there uncertain; fitted probabilities can look decisive even when their uncertainty is substantial. Neither a plot nor a single influence statistic substitutes for examining the model, the data, and the cluster structure together.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- Among observations at ordinary values of \(x\), both outcomes occur. One observation at a very large \(x\) has outcome 1. Does that single fact establish complete separation by \(x\)?
- For a logistic model with an intercept and slope, what is the conditional information contribution for the slope from an observation with predictor \(x_{ij}\)?
- Two observations at the same \(x\) have different outcomes but belong to different clusters. Why should a diagnostic plot identify the clusters?
- After deleting one extreme observation for a sensitivity check, the fitted slope changes substantially. What conclusion is justified, and what conclusion is not?
Solutions. (1) No. Complete separation requires a separating pattern across the relevant observations; the outcome overlap at ordinary \(x\)-values contradicts a threshold separation in this one-predictor example. The extreme point may still be influential. (2) It is \(p_{ij}(1-p_{ij})x_{ij}^2\), obtained from the slope-slope entry of \(p_{ij}(1-p_{ij})\mathbf{x}_{ij}\mathbf{x}_{ij}^{\mathsf T}\), where \(\mathbf{x}_{ij}=(1,x_{ij})^{\mathsf T}\). (3) Cluster-specific random effects can shift conditional probabilities, so pooling can conceal or misrepresent within-cluster patterns; the same \(x\) need not imply the same conditional probability. (4) The estimate is sensitive to that observation under the specified model. The change does not prove the observation is erroneous, nor does it by itself establish separation.