Local Influence Under Structured Perturbations

From Individual Changes to a Direction of Influence

An influence diagnostic often asks what happens when one case is changed or removed. Local influence asks a broader question: which small, coordinated change in a specified perturbation scheme most alters the fitted likelihood? The scheme matters. A direction may change several case weights together, shift selected responses, or modify selected covariates. Local curvature summarizes the effect of such changes near the unperturbed fit.

The resulting direction is a local diagnostic, not a declaration that the observations are erroneous. It identifies combinations worth examining, while the substantive plausibility of those changes must be assessed separately.

Definition: Let $\ell(\beta;\omega)$ be a twice continuously differentiable log-likelihood, with regression parameter $\beta\in\mathbb{R}^p$ and perturbation parameter $\omega\in\mathbb{R}^m$. At the baseline $\omega_0$, let $\hat\beta(\omega)$ be a local maximizer of $\ell(\beta;\omega)$, and define the profile log-likelihood $p(\omega)=\ell(\hat\beta(\omega);\omega)$. The local profile-curvature matrix is the Hessian of the negative twice profile log-likelihood, $\mathcal{C}=-2\nabla^2p(\omega_0)$. For a unit direction $d\in\mathbb{R}^m$, the signed curvature is $d^\mathsf{T}\mathcal{C}d$; its absolute magnitude is $|d^\mathsf{T}\mathcal{C}d|$.

A constant or a linear function of $\omega$ does not change a Hessian. Thus, if a displacement is centered by subtracting its baseline value and tangent plane, its curvature matrix is still $\mathcal{C}$. The eigenvector interpretation below concerns this local second-order change; it does not require treating the signed curvature as a nonnegative distance.

Deriving the Profile Curvature

Write all derivatives in the following display at $(\hat\beta(\omega_0),\omega_0)$, and suppose the baseline fit is an interior local maximum. Let $J=-\ell_{\beta\beta}$ be the observed information matrix and $\Delta=\ell_{\beta\omega}$ the $p$-by-$m$ matrix of mixed derivatives. Also write $G=\ell_{\omega\omega}$. The required local conditions are that these derivatives exist and are continuous near the baseline, and that $J$ is nonsingular. In particular, $J$ is positive definite at a strict local maximum with nonsingular observed information.

Theorem: Under these conditions, the local maximizer can be followed differentiably near $\omega_0$, and the profile-curvature matrix is $$ \mathcal{C}=-2\left(G+\Delta^\mathsf{T}J^{-1}\Delta\right). $$

Proof. The fitted parameter satisfies the score equation $\ell_\beta(\hat\beta(\omega);\omega)=0$. The implicit function theorem applies because the derivative of this equation with respect to $\beta$ is $\ell_{\beta\beta}=-J$, which is nonsingular. Differentiate the score equation with respect to $\omega$:

-J\frac{\partial\hat\beta}{\partial\omega}+\Delta=0, \qquad \frac{\partial\hat\beta}{\partial\omega}=J^{-1}\Delta.

By the chain rule and the score equation, $p_\omega=\ell_\omega$: the term involving $\ell_\beta$ vanishes at the fitted parameter. Differentiating again gives

\nabla^2p(\omega_0) =\ell_{\omega\omega} +\ell_{\omega\beta}\frac{\partial\hat\beta}{\partial\omega} =G+\Delta^\mathsf{T}J^{-1}\Delta.

Since the curvature matrix is $-2\nabla^2p(\omega_0)$, the stated formula follows. $\square$

For a unit direction $d$, a small change $\omega=\omega_0+td$ has second derivative $-\tfrac12 d^\mathsf{T}\mathcal{C}d$ for the profile log-likelihood at $t=0$. The real symmetric matrix $\mathcal{C}$ has an orthonormal eigenbasis. By the Rayleigh quotient theorem, the maximum of $|d^\mathsf{T}\mathcal{C}d|$ over unit vectors is the largest absolute eigenvalue, attained by a corresponding eigenvector. This follows by expanding $d$ in the eigenbasis: if its coordinates are $a_j$ and the eigenvalues are $\lambda_j$, then $d^\mathsf{T}\mathcal{C}d=\sum_j\lambda_j a_j^2$, with $\sum_j a_j^2=1$. Its absolute value is at most $\max_j|\lambda_j|$, and equality is attained by an eigenvector with that eigenvalue.

Case Weights and Structured Directions

For case-weight perturbations, let $\ell(\beta;\omega)=\sum_{i=1}^n\omega_i\ell_i(\beta)$, with baseline weights $\omega_0=\mathbf{1}$. The case log-likelihood contributions $\ell_i$ are held fixed as the weights vary. Consequently $G=0$, and the $i$th column of $\Delta$ is the case score $s_i=\partial\ell_i/\partial\beta$. Thus

\mathcal{C}=-2\Delta^\mathsf{T}J^{-1}\Delta.

This matrix is negative semidefinite when $J$ is positive definite. For a structured change $\omega=\omega_0+B a$, where $a\in\mathbb{R}^q$ is the perturbation coordinate and $B$ specifies how it changes the case weights, the curvature in $a$-coordinates is $B^\mathsf{T}\mathcal{C}B$. Restricting the calculation to $B$ makes the diagnostic answer a specified scientific question rather than searching over arbitrary changes.

Worked Example: Two Poisson Case Weights

Suppose $Y_1,Y_2$ are independent Poisson observations with a common log mean $\beta$. Their weighted log-likelihood, omitting terms independent of $\beta$, is $\ell(\beta;\omega)=\sum_{i=1}^2\omega_i(y_i\beta-e^\beta)$. Take observed counts $y_1=0,y_2=2$, and baseline weights $(1,1)$.

The baseline score is $\sum_i(y_i-e^\beta)=2-2e^\beta$, so $\hat\beta=0$. The observed information is $J=2e^{\hat\beta}=2$. The two case scores are $y_i-e^{\hat\beta}$, giving $\Delta=(-1,\ 1)$. There is no direct second derivative with respect to the weights, so $G=0$. Therefore

\mathcal{C} =-2\Delta^\mathsf{T}J^{-1}\Delta =-\begin{pmatrix}1&-1\\-1&1\end{pmatrix}.

The direction $d=(1,-1)^\mathsf{T}/\sqrt{2}$ has eigenvalue $-2$, while $(1,1)^\mathsf{T}/\sqrt{2}$ has eigenvalue $0$. Hence the largest absolute curvature is $2$, in the contrast direction that increases one weight as it decreases the other. The zero-curvature common-weight direction is consistent with the fact that multiplying the entire log-likelihood by a positive common weight does not change its maximizer.

Response and Covariate Perturbations

Unlike case weights, response and covariate perturbations can change the likelihood directly even when $\beta$ is held fixed. Their $G$ term must therefore be calculated; omitting it can change both the magnitude and sign of the curvature.

Worked Example: Perturbing Gaussian Responses

Consider $Y\sim N(X\beta,I)$, with known variance one, and perturb the response vector to $y+\omega$. The log-likelihood, up to a constant, is $\ell(\beta;\omega)=-\tfrac12(y+\omega-X\beta)^\mathsf{T}(y+\omega-X\beta)$. Let $X$ be the two-row intercept-only design, $X=(1,1)^\mathsf{T}$, and take $y=(1,3)^\mathsf{T}$.

At $\omega_0=0$, $\hat\beta=2$, and $J=X^\mathsf{T}X=2$. Differentiating the score $X^\mathsf{T}(y+\omega-X\beta)$ with respect to $\omega$ gives $\Delta=X^\mathsf{T}=(1,1)$. The direct response Hessian is $G=-I_2$. Thus

\Delta^\mathsf{T}J^{-1}\Delta =\frac12\begin{pmatrix}1&1\\1&1\end{pmatrix}, \qquad \mathcal{C} =-2\left[ -I_2+\frac12\begin{pmatrix}1&1\\1&1\end{pmatrix} \right] =\begin{pmatrix}1&-1\\-1&1\end{pmatrix}.

The leading eigenvalue is $2$, with unit eigenvector $(1,-1)^\mathsf{T}/\sqrt{2}$. A common response shift is absorbed by the fitted intercept, producing zero curvature; a contrast between the two responses cannot be absorbed and produces the largest curvature.

Worked Example: Perturbing a Poisson Covariate

Consider independent Poisson observations with log means $\eta_i=\beta_0+\beta_1x_i$. Use covariates $x=(0,0,1,1)$ and responses $y=(0,2,1,3)$. The fitted values $\hat\beta_0=0,\hat\beta_1=b=\log 2$ give means $\mu=(1,1,2,2)$. The residuals are $(-1,1,-1,1)$, so both score components are zero: their sum is $0$, and their covariate-weighted sum is $0$. The information matrix is

J=\sum_i\mu_i \begin{pmatrix}1\\x_i\end{pmatrix} \begin{pmatrix}1&x_i\end{pmatrix} =\begin{pmatrix}6&4\\4&4\end{pmatrix}, \qquad J^{-1}=\frac18\begin{pmatrix}4&-4\\-4&6\end{pmatrix}.

Perturb only the third covariate, replacing $x_3$ by $x_3+\omega$. The third case has residual $y_3-\mu_3=-1$, covariate vector $(1,1)^\mathsf{T}$, and fitted slope $b$. Differentiating its score contribution with respect to $\omega$ gives the mixed-derivative column $\Delta=(-2b,-1-2b)^\mathsf{T}$. Its direct second derivative is $G=-\mu_3b^2=-2b^2$. Therefore

\Delta^\mathsf{T}J^{-1}\Delta =\frac34+b+b^2, \qquad \mathcal{C} =-2\left\{\frac34+b+b^2-2b^2\right\} =-2\left(\frac34+b-b^2\right).

For $b=\log 2$, the signed curvature is approximately $-1.93$, and its absolute magnitude is approximately $1.93$. Here there is only one perturbation coordinate, so its unit direction is the leading direction automatically. The calculation illustrates why the direct term $G$ matters: the covariate change alters the linear predictor and hence the likelihood even before the coefficients are refitted.

Using Curvature Carefully

For a perturbation scheme with several coordinates, inspect the eigenvector associated with the eigenvalue of largest absolute value, then translate its coordinates back into the original perturbations. If the scheme is $\omega=\omega_0+B a$, the eigenvector describes a combination of the columns of $B$, not necessarily a simple change to one case. Scaling also matters: changing the units of a perturbation coordinate changes the numerical curvature. Define comparable, scientifically meaningful units before ranking directions.

Curvature is local: it describes a second-order change near $\omega_0$, not the result of a large perturbation. A large eigenvalue flags a direction for follow-up, but does not by itself establish model failure, data error, or causal importance.

Common mistake: dropping $G$. The reasoning that “all influence comes from refitting $\beta$” ignores direct changes to the likelihood. The correction is to compute both $\ell_{\omega\omega}$ and the refitting term $\Delta^\mathsf{T}J^{-1}\Delta$.
Common mistake: reversing the sign. The profile Hessian is $G+\Delta^\mathsf{T}J^{-1}\Delta$, so the Hessian of negative twice the profile log-likelihood is $-2$ times that sum. Check the definition of the displacement before interpreting a signed eigenvalue.
Common mistake: choosing the largest signed eigenvalue. Curvature is being ranked by absolute magnitude here. A large negative eigenvalue can be more influential than a small positive one.
Common mistake: treating the leading direction as a culprit. An eigenvector identifies a sensitive combination under the chosen perturbation scheme; it does not prove that the corresponding cases are incorrect.

Key takeaway: Derive curvature for the perturbation actually under study. The observed information and mixed derivatives describe how refitting responds; the direct perturbation Hessian records how the likelihood changes without refitting. The leading absolute eigenvalue and its eigenvector summarize the strongest local direction.

Check Your Understanding

Use the definitions and derivations above to answer the following questions.

In the profile-curvature formula, what does $G$ measure, and why is it zero for a linear case-weight perturbation?
For a structured perturbation $\omega=\omega_0+B a$, what matrix gives curvature in the $a$-coordinates?
Why does the largest absolute eigenvalue, rather than necessarily the largest eigenvalue, identify the strongest signed-curvature magnitude?
In the Gaussian response example, which response direction is absorbed by the intercept, and which direction has nonzero curvature?
Why can rescaling a perturbation coordinate change its reported curvature without changing the underlying fitted model?