Resolving Numerical Instability in Weighted Survival Fits

Why weighted survival fits become unstable

A weighted survival model can fail to converge, report enormous coefficients, or produce fitted event probabilities close to zero or one. These symptoms have more than one possible cause. A poorly scaled covariate or an unstable calculation may be a computational problem; sparse events, separation, or little weighted support for some histories may instead make the model weakly identified or leave it without a finite unpenalized estimate. Changing the optimizer cannot create information that the weighted data do not contain.

For pooled logistic survival models, each person-period row contributes a binary event indicator $D_{it}$, model covariates $X_{it}$, and a nonnegative analysis weight $W_{it}$. The weighted log-likelihood is

\ell(\beta)=\sum_{i,t} W_{it}\left\{D_{it}\log(p_{it})+(1-D_{it})\log(1-p_{it})\right\}, \qquad \operatorname{logit}(p_{it})=X_{it}^{\mathsf T}\beta .

For a weighted Cox fit, numerical work centers on the weighted risk-set sums and the weighted score. In both models, observations with large weights can dominate the fit. A useful diagnostic is the effective sample size of a collection of nonnegative weights:

\operatorname{ESS}=\frac{\left(\sum_k W_k\right)^2}{\sum_k W_k^2}.

This summary is not a substitute for examining event counts and risk sets, but it flags when a large nominal sample is effectively driven by very few weighted observations.

Separate numerical trouble from limited information

Start with the data structure before changing the fitting algorithm. Confirm that event indicators, follow-up intervals, time variables, covariates, and weights are coded as intended; inspect event counts by time and important covariate patterns. For Cox models, verify that event times have nonempty risk sets and that the risk-set sums are finite. For pooled logistic models, check whether some predictor patterns contain only events or only non-events. A model with sparse events or separation can have very flat curvature or no finite maximum, even when the software is functioning correctly.

Next inspect the fitting output and the fitted values. Warning signs include failure to meet the convergence criterion, unusually large coefficients or standard errors, a nearly singular information matrix, fitted probabilities numerically indistinguishable from zero or one, and coefficients that change markedly after small changes in starting values. Summarize the weights by their minimum, maximum, quantiles, and ESS, and identify which observations contribute most of the total weight. Then check whether those observations actually contribute events or informative risk-set comparisons.

Diagnostic principle: A numerical remedy is credible when it improves computation while leaving the model and its substantive fit essentially unchanged. If only a few weighted observations support a coefficient, or the data are separated, stable software output does not by itself establish that the estimate is well determined.

Two useful results about weights and curvature

Result: In weighted pooled logistic regression, multiplying every analysis weight by the same constant $c>0$ leaves the set of coefficient maximizers unchanged, provided the original objective has a maximizer.

To prove this, let $\ell(\beta)$ be the weighted log-likelihood above. Replacing every $W_{it}$ by $cW_{it}$ gives $\ell_c(\beta)=c\ell(\beta)$. Since $c>0$, for any two coefficient vectors $\beta_1,\beta_2$, $\ell_c(\beta_1)\geq\ell_c(\beta_2)$ exactly when $\ell(\beta_1)\geq\ell(\beta_2)$. Thus the maximizing set is unchanged. The score and Hessian are both multiplied by $c$, so the root of the score is unchanged as well. This is a numerical scaling device, not a cure for separation, poor overlap, or a small effective sample size. Variance estimates and software conventions about weight interpretation still require care. $\square$

The same common-scale invariance holds for the usual weighted Cox score. In notation with counting-process increments $dN_i(t)$, at-risk indicators $Y_i(t)$, and covariates $X_i(t)$, define

S^{(k)}(\beta,t)=\sum_j Y_j(t)W_j e^{\beta^{\mathsf T}X_j(t)}X_j(t)^{\otimes k}, \qquad U(\beta)=\sum_i\int W_i\left\{X_i(t)-\frac{S^{(1)}(\beta,t)}{S^{(0)}(\beta,t)}\right\}dN_i(t).

Here $X^{\otimes 0}=1$ and $X^{\otimes 1}=X$. Replacing all $W_i$ by $cW_i$ multiplies each $S^{(k)}$ by $c$, so the ratio in the score is unchanged; the outer event weight multiplies the entire score by $c$. Therefore the score roots are unchanged. This argument presumes the risk-set denominators are positive at event times.

Result: For a weighted pooled logistic model with finite covariates and positive weights, the negative Hessian at a finite coefficient vector is $$ -\nabla^2\ell(\beta)=X^{\mathsf T}\operatorname{diag}\{W_kp_k(1-p_k)\}X. $$ It is positive definite exactly when the rows with positive $W_kp_k(1-p_k)$ span the full coefficient space.

For one row, differentiating the Bernoulli log-likelihood contribution gives score $W_k(D_k-p_k)X_k$, because $\partial p_k/\partial\beta=p_k(1-p_k)X_k$. Differentiating again gives $-W_kp_k(1-p_k)X_kX_k^{\mathsf T}$. Summing rows proves the displayed Hessian formula. For any vector $v$,

v^{\mathsf T}\{-\nabla^2\ell(\beta)\}v =\sum_k W_kp_k(1-p_k)(X_k^{\mathsf T}v)^2.

This quantity is positive for every nonzero $v$ exactly when no nonzero $v$ is orthogonal to every row with positive curvature weight; equivalently, those rows span the coefficient space. If probabilities approach zero or one, $p_k(1-p_k)$ approaches zero and contributes little curvature. If covariates are nearly redundant among influential rows, the Hessian can be nearly singular. Either condition can make coefficients unstable. $\square$

Worked Examples

Worked Example: Rescaling Weights Without Changing the Fit

Suppose four observations have weights $1,1,1,100$. Their sum is $103$, their squared-weight sum is $1+1+1+10{,}000=10{,}003$, and hence

\operatorname{ESS}=\frac{103^2}{10{,}003}=\frac{10{,}609}{10{,}003}\approx 1.061.

The four rows therefore carry roughly the weight concentration of just over one equally weighted observation. Dividing the weights by $100$ gives $0.01,0.01,0.01,1$. The ESS is unchanged, since both numerator and denominator are divided by $100^2$. By the common-scale result, the pooled logistic coefficient maximizer is also unchanged. This rescaling can help avoid unnecessarily large objective values, but it does not reduce the dominance of the fourth observation or resolve weak support. The maximum weight and the covariate and event pattern of that observation still need investigation.

Worked Example: Separation Is Not an Optimizer Failure

Consider a pooled logistic model with an intercept and binary predictor $X$. In the $X=0$ group, all 10 rows have $D=0$; in the $X=1$ group, all 10 rows have $D=1$. Let the positive weights be $a_0$ for each $X=0$ row and $a_1$ for each $X=1$ row. Set the intercept to $-c$ and the coefficient of $X$ to $2c$, where $c>0$. Then the linear predictors are $-c$ and $c$, so the fitted probabilities are $\operatorname{expit}(-c)$ and $\operatorname{expit}(c)$.

The weighted log-likelihood along this coefficient sequence is

10a_0\log\{1-\operatorname{expit}(-c)\} +10a_1\log\{\operatorname{expit}(c)\} =10(a_0+a_1)\log\{\operatorname{expit}(c)\}.

For every finite $c$, $\operatorname{expit}(c)<1$, so this value is negative. As $c$ increases without bound, $\operatorname{expit}(c)\to1$, and the log-likelihood approaches $0$, its supremum. No finite coefficient attains that supremum. Consequently, an unpenalized maximum-likelihood estimate does not exist; increasing iteration limits or changing starting values cannot produce a finite solution. In real data, near-separation can look similar: very large coefficients and fitted probabilities near the boundaries may accompany a finite but poorly determined estimate.

Worked Example: Stabilizing a Cox Risk-Set Calculation

Suppose a Cox risk-set calculation has two linear predictors, $800$ and $799$. Directly computing $e^{800}+e^{799}$ overflows in ordinary floating-point arithmetic, even though the ratio of the two risk contributions is well defined. Factor out the largest predictor:

\log(e^{800}+e^{799}) =800+\log(1+e^{-1}) \approx 800.3133.

The normalized contributions are $1/(1+e^{-1})\approx0.7311$ and $e^{-1}/(1+e^{-1})\approx0.2689$, which sum to one. In general, for finite values $\eta_j$,

\log\left(\sum_j e^{\eta_j}\right) =m+\log\left(\sum_j e^{\eta_j-m}\right), \qquad m=\max_j\eta_j.

Each exponent on the right is at most one, avoiding overflow. This log-sum-exp calculation preserves the risk-set comparison; clipping the predictors or their exponentials would instead alter it. The example diagnoses a computation problem, not a claim that a large linear predictor is substantively plausible: inspect the covariate scale and fitted model as well.

Remedies, alternatives, and limits

Apply remedies in an order that preserves the intended analysis where possible. First correct data, coding, and risk-set construction errors. Next improve numerical representation: center continuous covariates, scale them to comparable units, use stable log-sum-exp calculations, and consider multiplying all weights by a common positive constant. Centering changes the intercept parameterization but not fitted linear predictors when the intercept is included. For example, if $x_c=x-1001$, then

\alpha+\beta x=(\alpha+1001\beta)+\beta x_c.

The slope and fitted values are preserved, while the intercept corresponds to a more central covariate value. After such changes, refit and compare fitted hazards, coefficients on their corresponding scales, and convergence diagnostics.

If instability persists, examine weighted event support, covariate overlap, and collinearity. A smaller or substantively justified model may be preferable to retaining unsupported terms. Penalized likelihood can produce finite, more stable estimates under separation, but it changes the estimating procedure and generally changes the target estimate; report the penalty and its consequences rather than presenting it as a purely computational fix. Weight truncation likewise changes the weights and generally trades some variance reduction for bias. Neither is equivalent to common rescaling.

Approach	What it addresses	What to keep in mind
Centering, scaling, stable arithmetic, common weight rescaling	Avoidable conditioning or overflow	Does not add information or cure separation
Simplifying a weakly supported model	Redundant or unsupported predictors	Requires substantive justification; the model changes
Penalization	Separation or extreme coefficient estimates	Changes the estimation procedure and should be reported
Weight truncation	Influence from extreme weights	Changes the analysis weights and can introduce bias

A weighted Cox model and a pooled logistic survival model are different model choices, not interchangeable numerical repairs. A Cox fit uses risk-set comparisons and a proportional-hazards structure; a pooled logistic fit specifies interval-specific event probabilities on the logit scale. Switching between them may change the fitted model and interpretation, as well as the numerical behavior. Diagnose instability in the chosen model before treating a switch as a solution.

Common Pitfalls

Calling every failure “convergence trouble.” The optimizer may be accurately revealing separation, near-collinearity, or little weighted event support. Inspect the data pattern and curvature.
Clipping fitted probabilities or exponentiated predictors without accounting for the change. This changes contributions to the objective or risk sets; use stable algebra instead.
Assuming a large nominal sample ensures a stable weighted fit. A very small ESS or a few influential event rows can dominate the estimate.
Treating truncation as harmless stabilization. Unlike common rescaling, truncation changes individual weights and may change the fitted estimate and its interpretation.

Key takeaway: First ask whether the failure is computational, weakly identified, or caused by a model with no finite unpenalized estimate. Preserve the model when fixing arithmetic or scale; clearly label any remedy that changes weights or the estimation procedure.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

What happens to the pooled logistic coefficient maximizer if every positive weight is multiplied by 5?
A fitted pooled logistic model has several probabilities near zero and a nearly singular Hessian. Name two distinct data or model features that could explain this.
Why is clipping a Cox linear predictor at 50 not equivalent to a stable risk-set calculation?
Does weight truncation have the same invariance property as multiplying every weight by a common constant?

Solutions. (1) The maximizer is unchanged because the entire log-likelihood is multiplied by the positive constant 5. (2) Possible explanations include separation or near-separation, sparse events, nearly redundant covariates among influential rows, and curvature concentrated in very few weighted observations. (3) Clipping changes the relative exponentiated risk contributions; log-sum-exp subtracts a common maximum and therefore preserves their ratios. (4) No. Truncation changes some weights but not others, so it does not multiply the objective by one common positive constant and can change the maximizer.