Likelihood Displacement and Its Local Quadratic Approximation

Why Measure Displacement?

A subject-weight perturbation changes how strongly selected subjects contribute to a fitted selection-model likelihood. The resulting parameter change is useful, but it does not by itself say how much the fit has moved in likelihood terms: a given parameter change can matter greatly in a sharply curved likelihood and little in a relatively flat one. Likelihood displacement measures the change in the unperturbed log likelihood at the perturbed estimate. Its local quadratic approximation combines the perturbation’s effect on the score with the curvature of the original likelihood.

For a selection model, let $\ell_i(\boldsymbol{\theta})$ denote subject $i$’s observed-data log-likelihood contribution, including the subject’s observed outcomes and missingness or selection information. Write the total observed-data log likelihood as $\ell(\boldsymbol{\theta})=\sum_{i=1}^n \ell_i(\boldsymbol{\theta})$. The parameter vector $\boldsymbol{\theta}$ may contain outcome-model and selection-model parameters. The subject weights multiply these full subject contributions.

Definition: Let $\hat{\boldsymbol{\theta}}$ be an interior, locally maximizing fit at unit weights, and let $\hat{\boldsymbol{\theta}}(\boldsymbol{w})$ be the corresponding fit after subject weights $\boldsymbol{w}=(w_1,\ldots,w_n)$ are applied. The likelihood displacement is $$ LD(\boldsymbol{w})=2\left\{\ell(\hat{\boldsymbol{\theta}})-\ell\big(\hat{\boldsymbol{\theta}}(\boldsymbol{w})\big)\right\}. $$ Both log-likelihood evaluations in this definition use the unperturbed likelihood $\ell$. The perturbed likelihood is used to obtain the perturbed fit, not to evaluate the displacement.

This distinction matters: comparing optimized values of two different weighted likelihoods would combine a change in the objective function with a change in the estimate. Likelihood displacement instead evaluates how the perturbed estimate fares under the same baseline likelihood. Near a strict local maximum, small displacements are nonnegative. If the baseline estimate is a global maximum, the displacement is nonnegative for any parameter value at which the likelihood is defined.

From Weighted Scores to a Local Quadratic

Represent a small weight change by $\boldsymbol{w}(\epsilon)=\boldsymbol{1}+\epsilon\boldsymbol{d}$, where $\boldsymbol{1}$ is the vector of unit weights, $\boldsymbol{d}$ specifies the direction of change, and $\epsilon$ is a scalar measuring its size. Define the subject score $\boldsymbol{u}_i(\boldsymbol{\theta})=\partial\ell_i(\boldsymbol{\theta})/\partial\boldsymbol{\theta}$, the baseline score $\boldsymbol{U}(\boldsymbol{\theta})=\sum_i\boldsymbol{u}_i(\boldsymbol{\theta})$, and the perturbation score $\boldsymbol{s}_{\boldsymbol{d}}=\sum_i d_i\boldsymbol{u}_i(\hat{\boldsymbol{\theta}})$. The observed information at the baseline fit is $\boldsymbol{J}=-\partial^2\ell(\hat{\boldsymbol{\theta}})/\partial\boldsymbol{\theta}\partial\boldsymbol{\theta}^{\mathsf T}$.

Assume the baseline score is zero, the estimate is an interior local maximum, and $\boldsymbol{J}$ is nonsingular and positive definite. Assume also that the perturbed score equation has a locally unique solution that varies smoothly with $\epsilon$. Differentiating that equation at $\epsilon=0$ gives the first-order movement of the estimate.

\boldsymbol{U}\big(\hat{\boldsymbol{\theta}}(\epsilon)\big) +\epsilon\sum_{i=1}^n d_i\boldsymbol{u}_i\big(\hat{\boldsymbol{\theta}}(\epsilon)\big)=\boldsymbol{0}, \qquad -\boldsymbol{J}\left.\frac{d\hat{\boldsymbol{\theta}}(\epsilon)}{d\epsilon}\right|_{\epsilon=0} +\boldsymbol{s}_{\boldsymbol{d}}=\boldsymbol{0}.

The first equation is the weighted score equation. In the second, the derivative of the baseline score with respect to the parameter is $-\boldsymbol{J}$, and the derivative of the weighted perturbation term at $\epsilon=0$ is $\boldsymbol{s}_{\boldsymbol{d}}$. Thus the initial direction of parameter movement is $\boldsymbol{J}^{-1}\boldsymbol{s}_{\boldsymbol{d}}$.

Local approximation: If the score equation is sufficiently smooth and its parameter Jacobian is nonsingular, the implicit function theorem gives a differentiable local fitted path. With a twice-differentiable path whose second derivative is locally bounded, $$ \hat{\boldsymbol{\theta}}(\epsilon)-\hat{\boldsymbol{\theta}} =\epsilon\boldsymbol{J}^{-1}\boldsymbol{s}_{\boldsymbol{d}}+O(\epsilon^2). $$ If the baseline log likelihood has locally bounded third derivatives, then $$ LD(\boldsymbol{w}(\epsilon)) =\epsilon^2\boldsymbol{s}_{\boldsymbol{d}}^{\mathsf T} \boldsymbol{J}^{-1}\boldsymbol{s}_{\boldsymbol{d}}+O(\epsilon^3). $$

The remainder orders require care. For example, twice continuous differentiability of the fitted path with bounded second derivative supports the $O(\epsilon^2)$ parameter remainder. A suitable twice-differentiable score map with nonsingular parameter Jacobian provides this through the implicit function theorem. A bounded third derivative of $\ell$ in a neighborhood supports the cubic likelihood remainder. Under weaker conditions, one should use little-$o$ statements instead: differentiability of the fitted path gives a parameter remainder $o(\epsilon)$, and a continuous likelihood Hessian gives a displacement remainder $o(\epsilon^2)$. Differentiability and local uniqueness alone do not justify upgrading these to $O(\epsilon^2)$ and $O(\epsilon^3)$.

Justifying the Quadratic Form

The approximation follows by expanding the baseline log likelihood at its stationary point. Put $\Delta\boldsymbol{\theta}=\hat{\boldsymbol{\theta}}(\epsilon)-\hat{\boldsymbol{\theta}}$. Since the baseline score vanishes, the linear Taylor term is zero. Under locally bounded third derivatives,

\ell(\hat{\boldsymbol{\theta}}+\Delta\boldsymbol{\theta}) =\ell(\hat{\boldsymbol{\theta}}) -\frac{1}{2}\Delta\boldsymbol{\theta}^{\mathsf T}\boldsymbol{J}\Delta\boldsymbol{\theta} +O(\|\Delta\boldsymbol{\theta}\|^3).

Multiplying the likelihood difference by two gives $LD=\Delta\boldsymbol{\theta}^{\mathsf T}\boldsymbol{J}\Delta\boldsymbol{\theta}+O(\|\Delta\boldsymbol{\theta}\|^3)$. Substituting the first-order parameter movement gives the stated quadratic expression. It is nonnegative because $\boldsymbol{J}$ is positive definite. The approximation is a local measure of movement in the baseline likelihood geometry; it is not a significance test or a probability that a subject is influential.

Worked Example: A Two-Parameter Perturbation

Suppose a hypothetical selection-model fit has parameters $\boldsymbol{\theta}=(\beta,\gamma)^{\mathsf T}$, where $\beta$ is measured in log-odds units and $\gamma$ is a selection-model coefficient. At the fit, assume the observed information and a specified perturbation score are $\boldsymbol{J}=\begin{pmatrix}8&2\\2&5\end{pmatrix}$ and $\boldsymbol{s}_{\boldsymbol{d}}=(1,-2)^{\mathsf T}$. The information is positive definite: its leading diagonal entry is $8>0$, and its determinant is $8(5)-2(2)=36>0$. These local quantities are treated as coming from a smooth, interior fitted likelihood with a nonsingular information matrix.

The inverse information is $\boldsymbol{J}^{-1}=\frac{1}{36}\begin{pmatrix}5&-2\\-2&8\end{pmatrix}$. Therefore, \[ \boldsymbol{J}^{-1}\boldsymbol{s}_{\boldsymbol{d}} =\frac{1}{36}(9,-18)^{\mathsf T} =(0.25,-0.50)^{\mathsf T}. \] For a small perturbation of size $\epsilon=0.10$, the first-order parameter change is approximately $\Delta\boldsymbol{\theta}=0.10(0.25,-0.50)^{\mathsf T}=(0.025,-0.050)^{\mathsf T}$.

The quadratic displacement coefficient is \[ \boldsymbol{s}_{\boldsymbol{d}}^{\mathsf T}\boldsymbol{J}^{-1}\boldsymbol{s}_{\boldsymbol{d}} =(1,-2)(0.25,-0.50)^{\mathsf T}=0.25+1.00=1.25. \] Thus $LD\approx 1.25(0.10)^2=0.0125$ log-likelihood displacement units. Equivalently, using the parameter change, $\boldsymbol{J}\Delta\boldsymbol{\theta}=(0.10,-0.20)^{\mathsf T}$, and $\Delta\boldsymbol{\theta}^{\mathsf T}\boldsymbol{J}\Delta\boldsymbol{\theta}=0.025(0.10)+(-0.050)(-0.20)=0.0125$. This is a local approximation, not an exact refit result.

Worked Example: Comparing Two Subject Directions

In another hypothetical fit, suppose $\boldsymbol{J}=\begin{pmatrix}10&0\\0&4\end{pmatrix}$, and two subjects have baseline casewise scores $\boldsymbol{u}_1=(2,0)^{\mathsf T}$ and $\boldsymbol{u}_2=(0,1)^{\mathsf T}$. Consider reducing one subject’s weight by a small amount: for subject $i$, use direction $\boldsymbol{d}=-\boldsymbol{e}_i$, where $\boldsymbol{e}_i$ is one in position $i$ and zero elsewhere. The assumptions for the local calculation are a smooth interior fit and positive definite information, both satisfied by the stated matrix.

For subject 1, $\boldsymbol{s}_{\boldsymbol{d}}=-\boldsymbol{u}_1=(-2,0)^{\mathsf T}$, so the coefficient is \[ (-2,0)\begin{pmatrix}0.1&0\\0&0.25\end{pmatrix}(-2,0)^{\mathsf T}=0.40. \] For subject 2, $\boldsymbol{s}_{\boldsymbol{d}}=(0,-1)^{\mathsf T}$, so the coefficient is \[ (0,-1)\begin{pmatrix}0.1&0\\0&0.25\end{pmatrix}(0,-1)^{\mathsf T}=0.25. \]

For the same small reduction, $\epsilon=0.20$, the approximate displacements are $0.40(0.20)^2=0.0160$ for subject 1 and $0.25(0.20)^2=0.0100$ for subject 2. Under this shared weight-change scale, subject 1 produces the larger local displacement. This ranking is conditional on the chosen parameterization and weight scale; it does not establish that subject 1 is more important for every scientific target.

Worked Example: An Exact Quadratic Illustration

Consider a hypothetical one-parameter local likelihood with baseline log likelihood $\ell(\theta)=C-\frac{1}{2}10(\theta-2)^2$, where $C$ is a constant and the baseline estimate is $\hat{\theta}=2$. Its observed information is $J=10>0$. Suppose the aggregate perturbation adds $\epsilon g(\theta)$, where $g(\theta)=3(\theta-2)$. This smooth illustrative objective has a unique maximizer for every $\epsilon$, and its perturbed score equation is $-10(\theta-2)+3\epsilon=0$.

Solving gives $\hat{\theta}(\epsilon)=2+0.3\epsilon$. The perturbation score at the baseline is $s_d=g'(2)=3$, so the first-order formula gives $\Delta\theta=\epsilon J^{-1}s_d=\epsilon(3/10)=0.3\epsilon$, exactly. At $\epsilon=0.20$, the fitted value is $2.06$. Evaluating the baseline likelihood there, \[ LD=2\{\ell(2)-\ell(2.06)\} =2\{C-[C-5(0.06)^2]\}=0.0360. \] The quadratic approximation also gives $\epsilon^2s_d^2/J=(0.20)^2(9/10)=0.0360$. It is exact here because the baseline likelihood is quadratic and the perturbed score produces a linear parameter path.

Use, Diagnostics, and Limits

The expression $\boldsymbol{s}_{\boldsymbol{d}}^{\mathsf T}\boldsymbol{J}^{-1}\boldsymbol{s}_{\boldsymbol{d}}$ is useful because it separates two features: the perturbation score indicates how the specified weight changes push the fit, while inverse information accounts for how readily the fitted parameters move. It applies to a chosen direction of weight change. It does not, by itself, identify a universally worst perturbation direction or measure the effect on a particular treatment contrast; those require different questions and calculations.

Compare this local diagnostic with exact case deletion. Exact deletion refits after setting a subject’s weight to zero, which is a finite change from one to zero. The quadratic approximation describes behavior near unit weights, so using it for deletion extrapolates beyond its local derivation. The earlier material on first-order case-deletion approximation and the earlier material on weighted likelihood perturbations address related but distinct approximations. When deletion is the target, an exact refit is an important benchmark if it is computationally feasible.

Diagnostics should check both the fitted model and the approximation. Confirm that the baseline solution is interior, the score is close to zero, and the observed information is nonsingular and positive definite in the parameterization used. Check that small perturbations preserve valid weights and that the optimizer converges to the same local solution rather than jumping to another mode. A nearly singular information matrix can make the calculated movement highly sensitive to numerical error. Refit at several small positive and negative values of $\epsilon$, when permitted, and compare the exact displacement with its quadratic approximation: agreement should improve as $|\epsilon|$ decreases under the stated smoothness conditions. Large disagreement, unstable parameter paths, or a nonpositive computed quadratic form can indicate numerical error, a failed local-maximizer assumption, or a coding or sign error.

The diagnostic also depends on the parameterization and the scale assigned to $\boldsymbol{d}$. Rescaling a perturbation direction rescales its quadratic displacement by the square of that factor. Consequently, comparisons are meaningful only when the weight changes are defined on comparable scales. Finally, this is a likelihood-based local measure for the fitted model. It does not assess whether the selection model is scientifically adequate, whether its missingness assumptions are credible, or whether a parameter change is important for a particular clinical decision.

Common Pitfall

Wrong reasoning: “The quadratic formula is positive, so it is the exact displacement for deleting a subject.” Correction: Positivity establishes a property of the local quadratic form under positive definite information; it does not make the approximation exact for a finite weight change. Exact deletion changes a weight by one full unit, so check it with a refit rather than treating a local expansion as an identity.
Wrong reasoning: “A differentiable fitted path automatically gives a second-order parameter remainder and a cubic displacement remainder.” Correction: Differentiability generally gives a first-order expansion with $o(\epsilon)$ remainder. The stronger $O(\epsilon^2)$ path remainder needs stronger smoothness, and the cubic likelihood remainder needs suitable third-order smoothness.

Key takeaway: For a small subject-weight change in direction $\boldsymbol{d}$, the leading likelihood displacement is $\epsilon^2\boldsymbol{s}_{\boldsymbol{d}}^{\mathsf T}\boldsymbol{J}^{-1}\boldsymbol{s}_{\boldsymbol{d}}$. Interpret it locally, state the regularity supporting its remainder, and verify it against refits when perturbations are not small.

Check Your Understanding

Answer each question, then compare with the brief solutions.

Why does the likelihood displacement evaluate both parameter values using the unperturbed likelihood?
If $\boldsymbol{s}_{\boldsymbol{d}}=\boldsymbol{0}$ and $\boldsymbol{J}$ is positive definite, what does the quadratic approximation say at order $\epsilon^2$?
For $J=5$, $s_d=2$, and $\epsilon=0.10$ in a one-parameter problem, calculate the approximate displacement.
What smoothness justifies an $O(\epsilon^2)$ fitted-parameter remainder and an $O(\epsilon^3)$ displacement remainder?

Solutions. (1) It isolates the change in the estimate under a common baseline likelihood, rather than mixing that change with a change in the objective. (2) The quadratic coefficient is zero, so the leading second-order term vanishes; higher-order behavior is not determined by that term. (3) $LD\approx\epsilon^2s_d^2/J=(0.10)^2(4/5)=0.00800$. (4) A sufficiently smooth perturbed score equation with nonsingular parameter Jacobian and bounded second derivative of the fitted path supports the parameter remainder; locally bounded third derivatives of the baseline log likelihood support the cubic displacement remainder.