Why a Change of Units Can Help—and What It Cannot Do
A continuous predictor measured near 1,000 may be inconvenient to use alongside an intercept: the corresponding design-matrix columns can be highly correlated, making numerical calculations less stable. Centering or changing units can improve that conditioning. But if the observed binary outcomes are separated by a linear predictor, changing coordinates does not change which observations are on which side of its decision boundary. The distinction is important: better numerical behavior is not the same as removing separation.
The discussion uses a binary-outcome generalized linear mixed model with a logit link. For observation \(i\) in cluster \(j\), write \(Y_{ij}\in\{0,1\}\) and, conditional on random effects \(b_j\), \(P(Y_{ij}=1\mid b_j)=\operatorname{logit}^{-1}(\eta_{ij})\), where \(\eta_{ij}=x_{ij}^{T}\beta+z_{ij}^{T}b_j\). The random effects have a specified distribution, commonly \(b_j\sim N(0,G)\). A coordinate change must preserve the linear predictors and, when random slopes are present, the corresponding random-effects distribution.
Affine Changes Preserve the Fixed-Effect Pattern
Consider a fixed-effects part with an intercept and one continuous predictor \(x\): \(\eta=\beta_0+\beta_1x\). Let \(x^*=(x-c)/s\), where \(c\) is a chosen center and \(s\ne 0\) is a chosen scale. Since \(x=c+s x^*\), the same linear predictor can be written \(\eta=\beta_0^*+\beta_1^*x^*\), with \(\beta_0^*=\beta_0+c\beta_1\) and \(\beta_1^*=s\beta_1\).
Because the transformation matrix is invertible when \(s\ne0\), every original coefficient pair has exactly one transformed pair, and conversely. More importantly, the two coefficient representations give the same value of \(\eta\) for every observation. They therefore give the same fitted probabilities and the same likelihood at corresponding parameter values. The signs of the observations’ linear predictors, and hence the separating pattern, have not changed.
Worked Examples
Worked Example: Centering Improves Conditioning but Leaves Separation
Consider a hypothetical binary-outcome data set with three observations: \((x,Y)=(1000,0),(1001,0),(1002,1)\). A fixed-effects logistic model includes an intercept and \(x\). The predictor values are nearly equal and large, so inspect the design cross-product matrix using columns \(1\) and \(x\).
The column sums and sums of squares are \(3\), \(3003\), and \(3{,}006{,}005\). Thus \(X^TX=\begin{pmatrix}3&3003\\3003&3006005\end{pmatrix}\). Its determinant is \(3(3{,}006{,}005)-3003^2=9{,}018{,}015-9{,}018{,}009=6\). The diagonal entries are large relative to this determinant, reflecting strong dependence between the intercept column and the uncentered predictor column.
Center at \(c=1001\), so \(x^*=x-1001\) takes values \(-1,0,1\). The centered design cross-product matrix is \(X^{*T}X^*=\begin{pmatrix}3&0\\0&2\end{pmatrix}\). The columns are now orthogonal in this small data set, which is a substantial improvement in this aspect of numerical conditioning.
Nevertheless, the outcomes are completely separated: the direction \(x-1001.5\) is negative for both observations with \(Y=0\) and positive for the observation with \(Y=1\). For any \(t>0\), take \(\eta=t(x-1001.5)\). At \(x=1000,1001,1002\), the linear predictors are respectively \(-1.5t,-0.5t,0.5t\), so the signs continue to match the observed outcomes for every \(t>0\). In centered coordinates, \(\eta=-0.5t+t x^*\), which gives those same three values. Centering has improved the design calculation, not removed the separating direction.
Worked Example: Rescaling Changes Coefficients, Not Predictions
Use the same three hypothetical observations and set \(x^*=(x-1001)/2\). The transformed predictor values are \(-0.5,0,0.5\). In the original coordinates, the separating linear predictor \(\eta=t(x-1001.5)\) has intercept \(-1001.5t\) and slope \(t\).
Here \(c=1001\) and \(s=2\), so the mapped coefficients are \(\beta_0^*=\beta_0+c\beta_1=-1001.5t+1001t=-0.5t\) and \(\beta_1^*=s\beta_1=2t\). At \(x^*=-0.5,0,0.5\), the transformed predictor gives \(-0.5t+2t(-0.5)=-1.5t\), \(-0.5t\), and \(-0.5t+2t(0.5)=0.5t\). These exactly match the original values.
Thus, scaling \(x\) into units twice as large changes the reported slope from \(t\) to \(2t\), but does not change any fitted probability. For example, the first probability remains \(\operatorname{logit}^{-1}(-1.5t)\) in either coordinate system. The divergent movement along \(t\) remains available after rescaling.
Random Slopes Require a Consistent Transformation
The same principle applies to a random intercept and random slope. Suppose the cluster-specific contribution is \(b_{0j}+b_{1j}x\). With \(x=c+s x^*\), it becomes \((b_{0j}+c b_{1j})+(s b_{1j})x^*\). To preserve the same conditional linear predictor, define \(b_j^*=(b_{0j}+c b_{1j},s b_{1j})^T\). If the original random-effects covariance is \(G\), the transformed covariance must be \(G^*=TGT^T\), where \(T=\begin{pmatrix}1&c\\0&s\end{pmatrix}\).
Worked Example: Transforming a Random-Effects Covariance
Suppose, hypothetically, that the random intercept and slope have covariance \(G=\begin{pmatrix}4&1\\1&0.25\end{pmatrix}\), and center at \(c=10\) without rescaling, so \(s=1\). Then \(T=\begin{pmatrix}1&10\\0&1\end{pmatrix}\). The transformed random intercept is \(b_{0j}^*=b_{0j}+10b_{1j}\), while the slope is unchanged.
The transformed intercept variance is \(\operatorname{Var}(b_{0j}+10b_{1j})=4+2(10)(1)+10^2(0.25)=49\). The covariance of transformed intercept and slope is \(\operatorname{Cov}(b_{0j}+10b_{1j},b_{1j})=1+10(0.25)=3.5\), and the slope variance remains \(0.25\). Therefore \(G^*=\begin{pmatrix}49&3.5\\3.5&0.25\end{pmatrix}\).
If the transformed covariance is used, the conditional linear predictor and model are preserved. But imposing a zero intercept-slope covariance in both coordinate systems would not preserve this example: its transformed covariance is \(3.5\), not zero. A covariance restriction can therefore make “the same model after centering” an incorrect claim. The random-effects structure must be transformed along with the predictor, or the change may alter the model rather than merely its coordinates.
Why the Invariance Holds
For a fixed-effects logistic model, the conditional log-likelihood contribution of an observation is \(y\eta-\log(1+\exp(\eta))\). It depends on the coefficients only through \(\eta\). Under the invertible affine mapping above, each observation has exactly the same \(\eta\) before and after transformation. Hence every likelihood contribution, and their sum, is identical at corresponding parameter values. Any path of coefficients producing increasingly favorable probabilities for the observed outcomes has a mapped path with the same probabilities and likelihood values.
The same argument carries through a GLMM when both fixed and random components are reparameterized to preserve each conditional linear predictor and the random-effects distribution. The conditional likelihood is then unchanged for each random-effects value, and integration over the correspondingly transformed distribution preserves the marginal likelihood. This is a coordinate argument, not a claim that every definition or diagnostic of separation in every mixed model reduces to the fixed-effects logistic case.
Identify the intercept, the continuous fixed-effect terms, and any random-slope terms involving the predictor.
Use \(x=c+s x^*\) and transform fixed and random coefficients so that the predictor remains identical for every observation and cluster.
Transform its covariance as \(G^*=TGT^T\), and check that imposed covariance restrictions still describe the same family of distributions.
Assess whether numerical calculations became more stable, then separately assess whether the observed outcomes still permit a separating direction.
Diagnostics, Alternatives, and Limits
After rescaling, a software fit may converge more smoothly, produce fewer rounding warnings, or yield a better-conditioned information matrix. Those are useful signs that numerical computation has improved, but they do not establish that a finite unpenalized estimate exists. Recheck the outcome pattern against the transformed predictor, and compare the relevant design columns and fitted linear predictors. If coefficients continue growing in magnitude while fitted probabilities approach zero or one for outcomes they match, separation remains a concern even if the optimizer’s messages look better.
Rescaling differs fundamentally from remedies that change the estimation target or information available. Collecting observations in the previously unrepresented parts of predictor space can change the separating pattern. Proper priors or penalization can constrain coefficient growth and yield finite regularized estimates, but these change the estimation procedure and interpretation. Collapsing factor levels is not an equivalent coordinate change and can discard distinctions in the data. Standard GLM material covers the general logic of reparameterization; here the relevant caution is to preserve the GLMM random-effects structure as well.
The invariance claim has limits. It requires a one-to-one transformation of the included predictor terms. Setting \(s=0\) discards the predictor and is not invertible. Dropping the intercept while centering, changing nonlinear terms inconsistently, or imposing different random-effects covariance constraints can change the model space. A transformation can also improve conditioning only modestly, depending on the design and the other predictors. It cannot supply missing outcome patterns, guarantee optimizer success, or establish that a particular mixed-model likelihood has a finite maximizer.
Common Mistake
- Wrong reasoning: “The model converged after standardizing \(x\), so separation must have been fixed.”
- Correction: Convergence behavior concerns the numerical algorithm; separation concerns the likelihood and the observed outcome pattern. An invertible rescaling preserves the linear predictors under coefficient mapping, so it cannot itself remove a separating direction.
- Another pitfall: Centering a random-slope predictor but retaining an incompatible covariance restriction may silently change the random-effects model. Transform the random effects and covariance, then verify the restrictions.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- For \(x^*=(x-20)/5\), express the coefficients \(\beta_0^*,\beta_1^*\) in terms of \(\beta_0,\beta_1\) so that \(\beta_0+\beta_1x=\beta_0^*+\beta_1^*x^*\).
- A data set has a separating direction before centering. Under what condition on a transformation \(x^*=(x-c)/s\) does the separation-invariance argument apply?
- Why can an intercept and slope column become less correlated when \(x\) is centered, even though the outcomes remain separated?
- A random intercept and slope are transformed after centering \(x\). What must happen to their covariance matrix?
Solutions. (1) Since \(x=20+5x^*\), \(\beta_0^*=\beta_0+20\beta_1\) and \(\beta_1^*=5\beta_1\). Substitution gives \(\beta_0+20\beta_1+5\beta_1x^*=\beta_0+\beta_1x\). (2) Require \(s\ne0\), retain the corresponding model terms, and map coefficients invertibly; in a GLMM also preserve the random-effects structure. (3) Centering can make the predictor column sum to zero, reducing its cross-product with the intercept column. That changes the numerical geometry of the columns, not the sign pattern of the matched linear predictors. (4) Transform it by \(G^*=TGT^T\), with the same transformation \(T\) applied to the random effects, and check whether any covariance restrictions remain compatible.