Directional Curvature for Prespecified Subject Changes

Why Evaluate a Prespecified Direction?

A maximum-curvature direction identifies the perturbation pattern that produces the greatest local likelihood displacement among all unit directions. But an analyst may have a more specific question: what happens if one subject is downweighted, or if a named subgroup is downweighted together? Directional curvature answers that question using the same local geometry, while keeping the perturbation pattern fixed in advance. It is a targeted diagnostic, not a search for the most influential direction.

Here the perturbation coordinates are subject weights in the observed selection likelihood. Write that likelihood as \(\ell(\theta;\omega)\), where \(\theta\) is the model parameter vector and \(\omega\) is the vector of perturbation weights. The unperturbed setting is \(\omega_0\), usually the vector of ones. Let \(\hat{\theta}\) be the fitted parameter at \(\omega_0\). The calculations below concern small changes around this fit; they do not, by themselves, predict the exact result of deleting a subject.

Definition: A prespecified perturbation direction is a fixed vector \(d\) in the coordinates of \(\omega\). We take \(d\) to have Euclidean length one, so a small movement along it is \(\omega(a)=\omega_0+ad\). The directional curvature \(C_d\) is the magnitude of the second derivative, at \(a=0\), of the likelihood displacement along this path. It measures local displacement per squared unit of movement in the stated weight coordinates.

Formula and Assumptions

Define likelihood displacement by comparing the perturbed fit with the original likelihood surface:

LD(\omega)=2\{\ell(\hat{\theta};\omega_0)-\ell(\hat{\theta}_{\omega};\omega_0)\},

where \(\hat{\theta}_{\omega}\) is the maximum-likelihood estimate under perturbation \(\omega\). Thus the displaced estimate is evaluated using the baseline likelihood \(\ell(\cdot;\omega_0)\). This definition makes \(LD(\omega_0)=0\); near a well-behaved interior maximum, the displacement is nonnegative.

Let \(H\) be the Hessian matrix of second derivatives of \(\ell\) with respect to \(\theta\), evaluated at \((\hat{\theta},\omega_0)\). Let \(\Delta\) be the matrix of mixed second derivatives, with rows indexed by parameters and columns by perturbation coordinates:

H=\left.\frac{\partial^2\ell}{\partial\theta\,\partial\theta^{\mathsf T}}\right|_{(\hat{\theta},\omega_0)}, \qquad \Delta=\left.\frac{\partial^2\ell}{\partial\theta\,\partial\omega^{\mathsf T}}\right|_{(\hat{\theta},\omega_0)}.

The usual local-curvature calculation assumes that the likelihood is twice continuously differentiable near the fitted point, that \(\hat{\theta}\) is an interior stationary point, and that \(H\) is nonsingular and negative definite. The perturbation must also be defined on a neighborhood of \(\omega_0\). For example, weights at one are locally interior to the nonnegative-weight region. If a perturbation coordinate is already on a boundary, a two-sided curvature may not describe an allowable change.

Result: For a unit direction \(d\), the local directional curvature is \(C_d=2\left|d^{\mathsf T}\Delta^{\mathsf T}H^{-1}\Delta d\right|\). Because \(H\) is negative definite, the quantity inside the absolute value is nonpositive; equivalently, \(C_d=d^{\mathsf T}Bd\), where \(B=-2\Delta^{\mathsf T}H^{-1}\Delta\) is positive semidefinite.

Why the Formula Holds

The formula follows by differentiating the perturbed likelihood score and then using a quadratic approximation at the original fit. This short derivation also explains why curvature uses a second-order, rather than a first-order, measure of change.

Differentiate the score equation.
At each nearby fit, the parameter score is zero: \(\ell_\theta(\hat{\theta}_{\omega};\omega)=0\). Differentiating with respect to \(a\) at \(a=0\) gives \(H\dot{\theta}+\Delta d=0\), where \(\dot{\theta}=\left.\frac{d\hat{\theta}_{\omega(a)}}{da}\right|_{a=0}\). Since \(H\) is nonsingular, \(\dot{\theta}=-H^{-1}\Delta d\).

Approximate the baseline likelihood loss.
The baseline score at \(\hat{\theta}\) is zero. For a small parameter displacement \(\delta\theta\), Taylor expansion gives \(\ell(\hat{\theta}+\delta\theta;\omega_0)=\ell(\hat{\theta};\omega_0)+\frac12\delta\theta^{\mathsf T}H\delta\theta+o(\|\delta\theta\|^2)\).

Substitute the displacement into \(LD\).
Along the path, \(\delta\theta=a\dot{\theta}+o(a)\). Therefore \(LD(\omega(a))=-a^2\dot{\theta}^{\mathsf T}H\dot{\theta}+o(a^2)=-a^2d^{\mathsf T}\Delta^{\mathsf T}H^{-1}\Delta d+o(a^2)\). Its second derivative at zero is \(-2d^{\mathsf T}\Delta^{\mathsf T}H^{-1}\Delta d\), which yields the stated absolute-magnitude formula.

Since \(H^{-1}\) is negative definite, \(d^{\mathsf T}\Delta^{\mathsf T}H^{-1}\Delta d\leq 0\). Hence \(B\) is positive semidefinite and the absolute value simply records nonnegative curvature. The sign of the direction does not change its curvature: \(C_{-d}=C_d\). In particular, locally, downweighting and upweighting the same coordinate have equal curvature when they are represented by opposite directions. Their finite changes need not have equal effects.

Worked Directional Calculations

Worked Example: Downweighting One Subject

Suppose a hypothetical fitted selection model has two parameters and three subject-weight coordinates. At the fit, take \(H=\begin{pmatrix}-100&0\\0&-25\end{pmatrix}\) and \(\Delta=\begin{pmatrix}4&-2&1\\3&1&-2\end{pmatrix}\). Assume these are the observed derivatives of a smooth likelihood at an interior maximum. The Hessian is negative definite because its diagonal entries are negative, so the local formula applies. Find the curvature for subject 1 alone.

The unit direction for subject 1 is \(d=e_1=(1,0,0)^{\mathsf T}\). Its mixed-derivative column is \(\Delta d=(4,3)^{\mathsf T}\), and \(H^{-1}=\begin{pmatrix}-0.01&0\\0&-0.04\end{pmatrix}\). Thus

C_{e_1} =2\left|(4,3) \begin{pmatrix}-0.01&0\\0&-0.04\end{pmatrix} \begin{pmatrix}4\\3\end{pmatrix}\right| =2|-0.16-0.36| =1.04.

In these weight coordinates, subject 1's unit perturbation has curvature 1.04. This is a local measure of likelihood displacement, not a percentage change in the treatment effect or a measure of exact case-deletion impact. Replacing \(e_1\) with \(-e_1\), the opposite infinitesimal change, gives the same curvature.

Worked Example: A Prespecified Two-Subject Downweighting

Using the same hypothetical derivatives, suppose subjects 1 and 2 are a subgroup of interest, and the intended change downweights them equally while leaving subject 3 unchanged. For a unit-length direction, take \(d=(1,1,0)^{\mathsf T}/\sqrt{2}\); the negative of this vector represents the downweighting orientation if increasing weights are positive. The direction is normalized because its squared length is \(1/2+1/2=1\). The sign choice does not affect curvature.

From the columns of \(\Delta\), \(\Delta d=(2/\sqrt{2},4/\sqrt{2})^{\mathsf T}=(\sqrt{2},2\sqrt{2})^{\mathsf T}\). Substitution gives

C_d =2\left|-\frac{(\sqrt{2})^2}{100} -\frac{(2\sqrt{2})^2}{25}\right| =2| -0.02-0.32| =0.68.

The equal subgroup direction has curvature 0.68, less than subject 1's 1.04. This does not mean that each subgroup member is individually less influential; the result concerns the specified joint pattern, including its equal relative changes and its unit-length normalization.

Worked Example: Comparing With Maximum Curvature

For these same derivatives, calculate \(B=-2\Delta^{\mathsf T}H^{-1}\Delta\). This produces

B= \begin{pmatrix} 1.04&0.08&-0.40\\ 0.08&0.16&-0.20\\ -0.40&-0.20&0.34 \end{pmatrix}.

For instance, the off-diagonal entry \(B_{13}\) is \(-2\{(4)(1)(-0.01)+(3)(-2)(-0.04)\}=-0.40\). A unit direction's curvature is its quadratic form \(d^{\mathsf T}Bd\). The eigenvalues of this matrix are approximately \(1.244\), \(0.296\), and \(0\), so the maximum curvature is approximately \(1.244\). The largest eigenvalue is the maximum of \(d^{\mathsf T}Bd\) over all unit directions; its eigenvector is the corresponding maximum-curvature direction.

The subject-1 direction reaches \(1.04/1.244\approx 0.836\), or about 84% of the maximum curvature. The subgroup direction reaches \(0.68/1.244\approx 0.547\), or about 55%. These ratios compare local curvatures under the same coordinate scaling. They do not give probabilities that a subject is influential, and they do not imply that the named directions are close to the maximizing eigenvector.

Reading the Diagnostic and Checking Its Limits

A directional calculation is most informative when its direction has a concrete interpretation fixed before examining the diagnostic—for example, a prespecified subgroup or a particular subject whose record has a substantive reason for review. The vector and its normalization should be reported, since curvature depends on the units and scaling of perturbation coordinates. For case weights, unit Euclidean length means a unit change in the vector of weights, not a unit change in every subject's weight.

Several checks help identify a calculation that may be going wrong:

Check the fitted point. The parameter score should be close to zero, and \(H\) should be symmetric and negative definite for an ordinary, well-identified interior maximum. A nearly singular Hessian can make \(H^{-1}\), and therefore curvature, numerically unstable.
Check dimensions and coordinates. If there are \(p\) parameters and \(m\) perturbation coordinates, \(H\) is \(p\) by \(p\), \(\Delta\) is \(p\) by \(m\), and \(B\) is \(m\) by \(m\). The direction \(d\) has length \(m\) and should have unit length under the stated convention.
Check signs and nonnegativity. With a negative-definite \(H\), computed directional curvatures should be nonnegative up to numerical rounding. A substantial negative value usually indicates a sign error, a Hessian evaluated with the wrong convention, or a failure of the assumed local maximum conditions.
Check local adequacy. Curvature is a second-order approximation at the baseline fit. A very large finite weight change, an optimizer that moves to a different mode, or a boundary-constrained estimate may not be described by this local calculation.

Directional curvature differs from both exact case deletion and the first-order case-deletion approximation. Exact deletion refits after removing a subject and can capture nonlocal changes, but costs additional fitting and answers a finite-change question. A first-order approximation uses local score information to approximate a deletion-related parameter change. Curvature instead quantifies local likelihood displacement along a stated perturbation direction. Maximum-curvature analysis searches over directions; prespecified directional curvature evaluates only the chosen one. These diagnostics complement one another rather than substitute for one another.

A common mistake is to compare unnormalized directions as though their curvatures measured the same amount of perturbation. If \(d\) is replaced by \(c d\), the quadratic form becomes \(c^2d^{\mathsf T}Bd\); doubling the vector quadruples the reported value. The correction is to normalize directions consistently, and to state the weight coordinates and scaling. Another mistake is to read equal curvature for \(d\) and \(-d\) as evidence of equal finite upweighting and downweighting effects. The formula is local and second order, so it loses this directional sign information.

Key takeaway: For a prespecified unit subject-weight direction \(d\), compute \(C_d=d^{\mathsf T}Bd\), with \(B=-2\Delta^{\mathsf T}H^{-1}\Delta\). Report the direction, its scaling, the curvature, and—when useful—its comparison with the maximum eigenvalue. Interpret the result as local likelihood-displacement sensitivity, not as an exact deletion effect.

Check Your Understanding

Try each question, then check the concise solutions below.

If \(d\) is a unit direction with curvature \(0.30\), what is the curvature for \(-d\), and why?
For a direction \(v\) of length 2 with \(v^{\mathsf T}Bv=1.20\), what is the curvature of the corresponding unit direction?
Why is a negative directional curvature a warning when \(H\) is negative definite?
What does a directional curvature of 0.70 establish, and what does it not establish about exact case deletion?

Solutions. (1) It is \(0.30\), because \((-d)^{\mathsf T}B(-d)=d^{\mathsf T}Bd\). (2) The unit vector is \(d=v/2\), so its curvature is \(d^{\mathsf T}Bd=1.20/4=0.30\). (3) Since \(H^{-1}\) is negative definite, \(B=-2\Delta^{\mathsf T}H^{-1}\Delta\) is positive semidefinite, so every quadratic form \(d^{\mathsf T}Bd\) must be nonnegative; a materially negative result suggests a numerical or sign problem, or violated assumptions. (4) It establishes the second-order local displacement measure for that specified, normalized direction. It does not establish the exact displacement after a finite deletion, which requires refitting or another finite-change analysis.