Log-Likelihood Scores and Information

From an Intensity Model to Local Likelihood Curvature

An intensity model describes how the instantaneous event rate changes with time and the observed history. Once that rate depends on an unknown finite-dimensional parameter, the likelihood turns the model into an estimation problem: the score records the local direction of change in log-likelihood, while information describes its local curvature. These quantities are central to likelihood estimation and uncertainty assessment, but their formulas depend on differentiating both the event contributions and the cumulative exposure contribution.

The focus here is that differentiation. In particular, when a rate is not an exponential function of the parameter, the second derivative of its logarithm contributes to observed information. Omitting that term, or counting an outer-product term twice, gives incorrect curvature.

Definition: Let \(N_r(t)\), for \(r=1,\ldots,R\), be event-counting processes with predictable at-risk indicators \(Y_r(t)\) and intensities \(Y_r(t)\lambda_r(t;\beta)\). The parameter \(\beta\) is a \(p\)-dimensional column vector in an open parameter space. Write \(\dot{\lambda}_r(t;\beta)\) for the \(p\)-dimensional gradient of \(\lambda_r\) with respect to \(\beta\), and \(\ddot{\lambda}_r(t;\beta)\) for its \(p\)-by-\(p\) Hessian. The score is the gradient of the log-likelihood, and the observed information is the negative Hessian of the log-likelihood.

Likelihood, Score, and Observed Information

For a fixed observation interval \([0,\tau]\), the parameter-dependent part of the multiplicative-intensity log-likelihood is

\ell(\beta) = \sum_{r=1}^{R} \left\{ \int_0^\tau \log \lambda_r(t;\beta)\,dN_r(t) - \int_0^\tau Y_r(t)\lambda_r(t;\beta)\,dt \right\}.

This expression conditions on the observed predictable risk processes and any covariate history. It omits terms that do not depend on \(\beta\), including the contribution from \(Y_r\) at event times when that contribution is parameter-independent. It is a suitable likelihood contribution when the relevant event intensities are correctly specified, the observation and censoring mechanisms contribute no additional parameter-dependent terms, and the observed path is supported by the model.

Assume the rates are positive wherever the processes are at risk and are twice continuously differentiable in \(\beta\). Assume also that differentiation can be interchanged with the displayed integrals, and that the integrals and derivatives are finite. For the formulas below, the risk indicators and covariate histories are held fixed with respect to \(\beta\). If they themselves depend on the parameter, their derivatives must also be included.

Differentiating the event term uses \(\nabla_\beta\log\lambda_r=\dot{\lambda}_r/\lambda_r\); differentiating the integrated rate gives \(\int Y_r\dot{\lambda}_r\,dt\). Thus the score vector is

U(\beta) = \frac{\partial\ell(\beta)}{\partial\beta} = \sum_{r=1}^{R} \left\{ \int_0^\tau \frac{\dot{\lambda}_r(t;\beta)}{\lambda_r(t;\beta)}\,dN_r(t) - \int_0^\tau Y_r(t)\dot{\lambda}_r(t;\beta)\,dt \right\}.

The first integral adds a contribution at each observed event. The second subtracts the rate accumulated over time at risk. A score value of zero in every parameter direction is the likelihood equation for an interior maximum; it does not by itself establish that a maximum exists or is unique.

For the second derivative, use \(\nabla_\beta^2\log\lambda_r =\ddot{\lambda}_r/\lambda_r -\dot{\lambda}_r\dot{\lambda}_r^\mathsf{T}/\lambda_r^2\). The observed information \(J(\beta)=-\partial^2\ell(\beta)/\partial\beta\partial\beta^\mathsf{T}\) is therefore

J(\beta) = \sum_{r=1}^{R} \left\{ \int_0^\tau \left[ \frac{\dot{\lambda}_r(t;\beta)\dot{\lambda}_r(t;\beta)^\mathsf{T}} {\lambda_r(t;\beta)^2} - \frac{\ddot{\lambda}_r(t;\beta)}{\lambda_r(t;\beta)} \right]dN_r(t) + \int_0^\tau Y_r(t)\ddot{\lambda}_r(t;\beta)\,dt \right\}.

There is exactly one outer-product term in the event contribution. It comes from differentiating \(\dot{\lambda}_r/\lambda_r\); the other event term, \(-\ddot{\lambda}_r/\lambda_r\), comes from the curvature of \(\log\lambda_r\). The integrated-rate contribution has a positive sign in observed information because the log-likelihood contains the negative integrated rate.

Why the Expected Information Has a Simpler Form

Observed information depends on the realized event times through \(dN_r(t)\). Under the model, the intensity property gives the expectation of an integral against \(N_r\) by replacing \(dN_r(t)\) with its compensator \(Y_r(t)\lambda_r(t;\beta)\,dt\). Subject to the integrability conditions that justify taking expectations, this yields the expected Fisher information

\mathcal{I}(\beta) = \operatorname{E}_{\beta}\{J(\beta)\} = \sum_{r=1}^{R} \operatorname{E}_{\beta} \left[ \int_0^\tau Y_r(t) \frac{\dot{\lambda}_r(t;\beta)\dot{\lambda}_r(t;\beta)^\mathsf{T}} {\lambda_r(t;\beta)} \,dt \right].

To see the cancellation, the expected event contribution to \(J(\beta)\), conditional on the predictable history, is the integral of \(Y_r\{\dot{\lambda}_r\dot{\lambda}_r^\mathsf{T}/\lambda_r-\ddot{\lambda}_r\}\). The expected integrated-rate contribution is the integral of \(Y_r\ddot{\lambda}_r\). The two Hessian terms cancel, leaving the outer product divided by the rate. Taking expectations gives the displayed expression. The result is positive semidefinite, but it need not be positive definite: some parameter directions may be unsupported by the observed risk and covariate patterns.

Worked Example: A Constant Exponential Rate

Suppose a single event process has intensity \(Y(t)e^\beta\), where the rate is measured per person-year. In a hypothetical dataset, there are \(d=12\) events over a total of \(T=8\) person-years at risk. The risk history is treated as fixed, and the rate is positive and differentiable for every real \(\beta\). Ignoring parameter-free terms, the log-likelihood is

\ell(\beta)=d\beta-Te^\beta.

Differentiating gives \(U(\beta)=d-Te^\beta\) and \(J(\beta)=Te^\beta\). At the trial value \(\beta=\log(1.2)\), the fitted rate is \(1.2\) events per person-year, so

U\{\log(1.2)\}=12-8(1.2)=2.4, \qquad J\{\log(1.2)\}=8(1.2)=9.6.

The positive score means that, locally at this value, increasing \(\beta\) increases the log-likelihood. Solving \(U(\beta)=0\) gives \(\widehat{\beta}=\log(d/T)=\log(1.5)\), corresponding to an estimated rate of \(1.5\) events per person-year. At that estimate the observed information is \(8(1.5)=12\). Here observed and expected information agree because the second derivative does not depend on the observed event times.

Worked Example: A Log-Linear Rate With a Covariate

Consider two hypothetical covariate groups with \(x=0\) and \(x=1\), and rate \(\lambda(t;\beta)=\exp(\beta_0+\beta_1x)\). The first group has 4 events and 5 person-years at risk; the second has 6 events and 3 person-years at risk. These rates are positive and smooth, and the covariate and risk histories do not depend on \(\beta\). Let \(z=(1,x)^\mathsf{T}\); then \(\dot{\lambda}=\lambda z\) and \(\ddot{\lambda}=\lambda zz^\mathsf{T}\). Substituting into the general formulas gives the groupwise score contribution \(z(d-T\lambda)\) and observed-information contribution \(T\lambda zz^\mathsf{T}\).

Evaluate at \(\beta_0=\log(0.8)\) and \(\beta_1=\log(1.5)\). The group rates are \(0.8\) and \(0.8(1.5)=1.2\) events per person-year. The score is

U(\beta) = \begin{pmatrix}1\\0\end{pmatrix}(4-5(0.8)) + \begin{pmatrix}1\\1\end{pmatrix}(6-3(1.2)) = \begin{pmatrix}0\\0\end{pmatrix} + \begin{pmatrix}2.4\\2.4\end{pmatrix} = \begin{pmatrix}2.4\\2.4\end{pmatrix}.

The observed information is

J(\beta) = 5(0.8) \begin{pmatrix}1&0\\0&0\end{pmatrix} + 3(1.2) \begin{pmatrix}1&1\\1&1\end{pmatrix} = \begin{pmatrix}7.6&3.6\\3.6&3.6\end{pmatrix}.

The nonzero score shows that this trial parameter is not a likelihood stationary point. The information matrix describes local curvature in the intercept and covariate directions; it is not itself a standard error or a confidence interval.

Worked Example: A Nonlinear Rate and the Log-Rate Curvature

Let \(\lambda(t;\beta)=\beta^2\), with \(\beta>0\), over fixed total exposure \(T\). This parameterization is positive and twice differentiable on its open parameter space. Suppose a hypothetical path has \(d=5\) events and \(T=4\) person-years. The log-likelihood is \(\ell(\beta)=2d\log\beta-T\beta^2\), so its score and observed information are

U(\beta)=\frac{2d}{\beta}-2T\beta, \qquad J(\beta)=\frac{2d}{\beta^2}+2T.

The general event-information integrand confirms the event term. Here \(\dot{\lambda}=2\beta\) and \(\ddot{\lambda}=2\), so \[ \frac{\dot{\lambda}^2}{\lambda^2}-\frac{\ddot{\lambda}}{\lambda} = \frac{4}{\beta^2}-\frac{2}{\beta^2} = \frac{2}{\beta^2}. \] There is one such contribution per event, followed by the integrated contribution \(2T\). At \(\beta=1.5\), the arithmetic is

U(1.5)=\frac{10}{1.5}-2(4)(1.5)=-5.333, \qquad J(1.5)=\frac{10}{1.5^2}+2(4)=12.44.

The negative score indicates that decreasing \(\beta\) locally raises the log-likelihood at this trial value. The positive observed information indicates downward curvature in this parameter direction at that point. If the outer-product term were incorrectly counted twice, the event term would be \(4/\beta^2\), contradicting the direct second derivative \(2d/\beta^2+2T\).

Assumptions, Checks, and Common Failure Modes

The formulas are only as useful as the likelihood and differentiation assumptions behind them. Before interpreting a score or information matrix, check that the model describes the observed event types, that the at-risk indicators are predictable, and that the likelihood includes any parameter-dependent entry, censoring, initial-state, or observation-process contributions relevant to the analysis. The fixed-risk formulas above do not account for omitted parameter-dependent terms.

Several practical checks can expose errors:

Differentiate the log-likelihood directly. For a simple parameterization, compare the resulting derivatives with the general score and information formulas.
Check dimensions and signs. A score is a \(p\)-vector and information is a \(p\)-by-\(p\) matrix. The integrated-rate term enters observed information with a positive sign.
Check numerical derivatives. At an interior parameter value, small finite differences of the log-likelihood should agree approximately with the analytic score; differences of the score should agree with the negative observed information.
Inspect curvature and identifiability. A singular or nearly singular information matrix can signal redundant parameters or inadequate variation in risk and covariates. A negative eigenvalue of observed information at a proposed maximum is a warning that the point may not be a local maximum.
Check the model’s support. If an observed event has zero modeled rate, the log-likelihood contribution is not finite. The positivity and support conditions cannot be repaired by differentiating the formula as if that event were possible.

A common mistake is to replace observed information by expected information before evaluating the realized data, or to assume the two are always equal. Expected information averages over event histories under the model; observed information uses the actual event contributions. They coincide in some special cases, such as the constant exponential-rate example, but need not coincide for general rates.

Another frequent error is to differentiate \(\log\lambda\) as though its second derivative were always \(-\dot{\lambda}\dot{\lambda}^\mathsf{T}/\lambda^2\). The correct identity includes \(\ddot{\lambda}/\lambda\). For a log-linear rate \(\lambda(t;\beta)=\exp\{a(t)+c(t)^\mathsf{T}\beta\}\), with \(a(t)\) and \(c(t)\) not depending on \(\beta\), \(\ddot{\lambda}=\dot{\lambda}\dot{\lambda}^\mathsf{T}/\lambda\), so those terms cancel in the event part of observed information; that convenient cancellation does not hold for every parameterization.

The score and information are local likelihood summaries. They do not guarantee a unique estimate, correct model specification, or reliable large-sample approximation. Expected information is a model-based curvature summary; observed information is a data-specific one. Choosing between them for a particular inferential procedure requires additional considerations beyond differentiation.

A Compact Derivation Checklist

Write the parameter-dependent likelihood.
Keep the event integrals and integrated rates, and identify which risk and covariate terms are fixed with respect to \(\beta\).

Differentiate once.
Use \(\nabla_\beta\log\lambda=\dot{\lambda}/\lambda\) for event contributions and \(\nabla_\beta\lambda=\dot{\lambda}\) for integrated rates.

Differentiate again and negate.
Use \(\nabla_\beta^2\log\lambda=\ddot{\lambda}/\lambda-\dot{\lambda}\dot{\lambda}^\mathsf{T}/\lambda^2\); retain one outer product in observed information.

Check the result.
Compare with a direct derivative in a tractable case, verify dimensions and signs, and assess whether the fitted information is nonsingular.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

For one process with intensity \(Y(t)\exp(\beta)\), write the score and observed information in terms of the event count \(d\) and total exposure \(T\).
For \(\lambda(\beta)=\beta^2\), \(\beta>0\), state the event contribution to observed information per event and explain which derivative creates it.
In the two-group log-linear example, what does the nonzero score at the stated parameter values tell you?
Why does expected information have no \(\ddot{\lambda}\) term under the stated model?
Name one warning sign of weak local information and one condition required for the displayed formulas to apply.

Solutions. (1) \(U(\beta)=d-Te^\beta\) and \(J(\beta)=Te^\beta\). (2) The contribution is \(2/\beta^2\); it is the negative second derivative of \(\log(\beta^2)\), equivalently \(\dot{\lambda}^2/\lambda^2-\ddot{\lambda}/\lambda\). (3) The trial value is not a stationary point because the score is not zero. (4) Taking expectation replaces \(dN_r\) by \(Y_r\lambda_r\,dt\), making the event contribution’s \(-Y_r\ddot{\lambda}_r\) cancel the integrated contribution \(+Y_r\ddot{\lambda}_r\). (5) Near-singularity of the information matrix is a warning; examples of required conditions include twice differentiable positive rates on event support and fixed, parameter-independent risk histories.