Why Scores and Information Matter
A likelihood describes how compatible observed genetic data are with different parameter values. Maximum-likelihood estimation identifies a parameter value that maximizes that likelihood; the score and information describe the likelihood’s local behaviour around a value. The score indicates the direction and rate of change, while information describes curvature and, in expectation, how much precision the data can provide.
These quantities are useful for interpreting likelihoods, checking a proposed estimate, and understanding why some genetic samples estimate a parameter more precisely than others. They are not themselves probabilities that a parameter is true. Their interpretation depends on the model and its assumptions.
Definitions and Regularity Conditions
Let \(Y\) denote the observed data, let \(\theta\) be a scalar parameter, and write \(L(\theta;Y)\) for the likelihood and \(\ell(\theta;Y)=\log L(\theta;Y)\) for the log-likelihood. For a vector parameter, the derivatives below become a gradient vector and a matrix of second derivatives. Differentiating the log-likelihood is convenient because independent likelihood contributions multiply, whereas their log-likelihood contributions and scores add.
A positive score means the log-likelihood increases as the scalar parameter increases at the value being evaluated; a negative score means it decreases locally. At an interior maximum of a differentiable log-likelihood, the score is zero. This first-derivative condition is necessary, not sufficient: a zero score alone does not establish that the point is a maximum. Positive observed information in a scalar model means the log-likelihood has negative curvature there.
Standard score and information identities require regularity conditions. In particular, the likelihood should be differentiable in the parameter, the possible data outcomes should not change with the parameter, and differentiation and expectation must be interchangeable. At a regular interior parameter value these conditions imply \(E_\theta[U(\theta)]=0\) and \(I(\theta)=\operatorname{Var}_\theta\{U(\theta)\}\). Boundary values, such as an allele frequency of zero or one, may violate these conditions; the identities should not be applied there without checking.
Worked Example: Hardy–Weinberg Genotype Counts
Worked Example: Hardy–Weinberg Genotype Counts
Suppose a hypothetical sample contains 100 unrelated individuals, with genotype counts 36, 48, and 16 for zero, one, and two copies of an allele. Assume Hardy–Weinberg equilibrium, independent sampled individuals, and a common allele frequency \(p\), with \(0<p<1\). The genotype probabilities are \((1-p)^2\), \(2p(1-p)\), and \(p^2\). Find the score and observed information at \(p=0.40\), and assess the score at that value.
Ignoring the multinomial coefficient, which does not depend on \(p\), the log-likelihood is
Differentiating gives
At \(p=0.40\), the substituted values give
Thus the log-likelihood has zero slope at \(p=0.40\). This agrees with the allele-frequency maximum-likelihood estimate from these counts, \(\hat p=(48+2(16))/(2(100))=0.40\). The observed information is
The positive curvature measure is consistent with a local maximum of the log-likelihood. It quantifies the observed curvature at this parameter value; it is not a probability or a standard error by itself.
Expected Information for Hardy–Weinberg Data
The same model illustrates how expected information is calculated. For one individual, let \(G\) be the number of copies of the allele, so \(G\in\{0,1,2\}\). Under Hardy–Weinberg equilibrium, \(G\) has the distribution of the sum of two independent Bernoulli allele indicators with success probability \(p\). The individual score can therefore be written
The mean of \(G\) is \(2p\), so \(E_p[U_1(p)]=2p/p-2(1-p)/(1-p)=0\). Since the two allele indicators are independent, \(\operatorname{Var}_p(G)=2p(1-p)\). Also \(U_1(p)=(G-2p)/\{p(1-p)\}\), and hence
For \(n\) independent individuals, scores add and expected information adds, giving \(I_n(p)=2n/\{p(1-p)\}\). At \(n=100\) and \(p=0.40\), this is \(200/0.24=833.3\), matching the observed information in the worked example at the fitted value. This equality for that sample is not required in general: observed information varies with the data, while Fisher information averages over possible data sets under the model.
Worked Example: Logistic Genotype-Specific Penetrance
Worked Example: Logistic Genotype-Specific Penetrance
Consider a hypothetical sample of three unrelated individuals with genotype dosages \(g=0,1,2\), respectively, and with all three having the binary phenotype of interest. Assume independent Bernoulli outcomes conditional on genotype and the logistic penetrance model \(\operatorname{logit}(\pi_i)=\alpha+\beta g_i\), where \(\pi_i\) is the probability of the phenotype for individual \(i\). Evaluate the score and information at \(\alpha=-1\), \(\beta=0.5\).
For an outcome \(y_i\in\{0,1\}\), the Bernoulli log-likelihood contribution is \(y_i\log(\pi_i)+(1-y_i)\log(1-\pi_i)\). The logit model gives \(\partial\ell/\partial\eta_i=y_i-\pi_i\), where \(\eta_i=\alpha+\beta g_i\). By the chain rule, the score components are \(\sum_i(y_i-\pi_i)\) for \(\alpha\) and \(\sum_i g_i(y_i-\pi_i)\) for \(\beta\).
The fitted penetrances at the evaluation point are \(\operatorname{logit}^{-1}(-1)\), \(\operatorname{logit}^{-1}(-0.5)\), and \(\operatorname{logit}^{-1}(0)\). Using their unrounded values, approximately \(0.268941\), \(0.377541\), and \(0.500000\), the scores are
Both components are positive, so at this parameter pair the log-likelihood increases locally when either parameter is increased while the other is held fixed. This statement is local; it does not say that both parameters can be changed independently without affecting the likelihood’s joint shape.
For logistic regression, the negative Hessian and expected information have the same form: sum the weights \(\pi_i(1-\pi_i)\) times the outer products of \((1,g_i)\) with itself. Here the three weights are approximately \(0.196612\), \(0.235004\), and \(0.250000\), so
The off-diagonal entries reflect the fact that intercept and genotype effect are informed by overlapping data. The information matrix describes joint local curvature, not two unrelated measures of precision.
Worked Example: Observed Versus Expected Penetrance Information
Worked Example: Observed Versus Expected Penetrance Information
In a hypothetical group of 20 individuals with the same genotype, 10 have the phenotype. Assume independent Bernoulli outcomes with a common penetrance \(\rho\), where \(0<\rho<1\). Evaluate the score, observed information, and Fisher information at \(\rho=0.40\).
The log-likelihood, up to a parameter-independent constant, is \(10\log\rho+10\log(1-\rho)\). Its score and observed information are
Substituting the stated value gives
For \(n\) Bernoulli observations, the expected information is \(I(\rho)=n/\{\rho(1-\rho)\}\). Therefore \(I(0.40)=20/(0.40\times0.60)=83.3333\). The observed and expected information differ because the observed information depends on the number of affected individuals, whereas the expected information averages over possible outcomes. Here the score is positive, indicating a locally increasing log-likelihood at \(\rho=0.40\); it is not zero because that value is not the sample proportion, which is \(10/20=0.50\).
How Information Relates to Precision
Information measures how sharply the likelihood tends to change near a parameter value. In regular models with sufficiently informative data, the inverse Fisher information is the large-sample covariance approximation for maximum-likelihood estimates. For a scalar parameter, larger information generally corresponds to smaller approximate variance; for multiple parameters, the inverse information matrix accounts for their dependence. This is a local, model-based approximation, not a guarantee of precision in every realized sample.
Information also depends on the design and the data-generating model. More independent observations typically add information. In genotype–phenotype models, however, the distribution of genotypes, the range of observed genotype dosages, and the penetrances all affect how much information is available about a genetic effect. If genotype values do not vary, for example, an intercept and a genotype slope cannot generally be separated from those observations alone; the information matrix may be singular.
Common Mistakes
- Confusing score with information. The score is a first derivative and can be positive or negative; information is based on curvature or its expectation. A large score is not itself a measure of precision.
- Treating observed and expected information as identical. They need not agree in a particular sample. State which quantity is being calculated and use the observed outcomes for \(J\), or an expectation under the model for \(I\).
- Rounding fitted probabilities too early. Calculate scores and information using unrounded model probabilities, then round the final result. Early rounding can change reported digits and make arithmetic inconsistent.
- Assuming zero score proves a maximum. A zero first derivative is only a stationary-point condition. Curvature and the parameter space also matter, especially at boundaries.
- Ignoring model assumptions. The formulas rely on the specified likelihood, including assumptions such as independent outcomes and the stated genotype or penetrance model. If these assumptions fail, the calculated information may not describe the actual sampling behaviour.
Check Your Understanding
Use the definitions and examples above to answer these questions.
- What does a positive score for an allele frequency mean at the parameter value where it is evaluated?
- Why can observed information differ from Fisher information in a particular penetrance sample?
- Under the Hardy–Weinberg model, why does expected information add across independent individuals?
- In the logistic penetrance example, what does a nonzero off-diagonal information entry indicate?
- Give one reason the standard information identities might fail at a parameter-space boundary.