Additive Contributions From Multiple Numerical Predictors

Building an Additive Linear Predictor

A GLM can use several numerical covariates to describe how the distribution of an observation depends on its measured characteristics. The linear predictor combines those covariates into one number, \(\eta_i\), for observation \(i\). Understanding how the pieces add is useful both for calculating \(\eta_i\) and for explaining what a fitted coefficient contributes.

For this tutorial, suppose there are \(n\) observations and \(p\) numerical covariates. Write \(x_{ij}\) for the value of covariate \(j\) on observation \(i\), and \(\beta_j\) for its coefficient. The intercept is \(\beta_0\). The covariates may differ across observations, while the coefficients are shared across them.

Definition: With an intercept and \(p\) numerical covariates, the linear predictor for observation \(i\) is \(\eta_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\cdots+\beta_px_{ip}\). The contribution of covariate \(j\) to \(\eta_i\) is the product \(\beta_jx_{ij}\). The intercept contributes the constant \(\beta_0\).

Each product \(\beta_jx_{ij}\) is an observation-specific contribution: it depends on the coefficient and the covariate value for that observation. The intercept is not a contribution from an observed covariate. It is the constant term in the predictor. To calculate \(\eta_i\), include the intercept and every specified covariate contribution, with each coefficient’s sign.

In GLM notation, the predictor is linked to the mean through the model’s link function: \(g(\mu_i)=\eta_i\). Thus, the sum of contributions is on the linear-predictor scale. It is not necessarily the mean itself, nor a probability or expected count. To obtain the mean, one uses the inverse link, \(\mu_i=g^{-1}(\eta_i)\).

Calculating Contributions Observation by Observation

A reliable calculation has three parts: list the intercept, calculate each coefficient-times-value product, and add those signed quantities. Keeping the products visible makes it easier to catch a missing term or a sign error.

Worked Example: Three Covariates in a Poisson GLM

Suppose a Poisson GLM for the expected number of visits has the log link and linear predictor \(\eta_i=0.4+0.03x_{i1}+0.20x_{i2}-0.10x_{i3}\), where \(x_{i1}\) is age in years, \(x_{i2}\) is the number of prior visits, and \(x_{i3}\) is distance in tens of kilometres. For one patient, the values are 50, 2, and 3. Find each contribution, \(\eta_i\), and the model mean.

The intercept contributes \(0.4\). The age contribution is \(0.03(50)=1.5\). The prior-visits contribution is \(0.20(2)=0.4\). The distance contribution is \(-0.10(3)=-0.3\). Therefore,

\eta_i=0.4+1.5+0.4-0.3=2.0.

For a Poisson GLM with the log link, \(g(\mu_i)=\log(\mu_i)=\eta_i\), so \(\mu_i=\exp(\eta_i)\). Here \(\mu_i=\exp(2.0)\approx 7.39\) visits. The value 2.0 is the linear predictor; the expected count is approximately 7.39.

The sign of a contribution comes from the product of the coefficient and the covariate value. A positive covariate value paired with a negative coefficient gives a negative contribution. A negative contribution lowers the total predictor relative to what it would be if that term were zero; it does not mean that the covariate value itself is negative.

Worked Example: A Negative Coefficient and a Binary Response

Consider a logistic GLM with \(\eta_i=-1+0.8x_{i1}-0.5x_{i2}\), where \(x_{i1}\) is a numerical symptom score and \(x_{i2}\) is a numerical treatment dose. For a case with \(x_{i1}=2\) and \(x_{i2}=3\), calculate the contributions, the predictor, and the fitted probability.

The intercept contributes \(-1\). The symptom-score contribution is \(0.8(2)=1.6\). The treatment-dose contribution is \(-0.5(3)=-1.5\). Adding them gives

\eta_i=-1+1.6-1.5=-0.9.

For the logistic link, \(\operatorname{logit}(\mu_i)=\eta_i\), and the inverse link is \(\mu_i=1/(1+\exp(-\eta_i))\). Hence,

\mu_i=\frac{1}{1+\exp(0.9)}\approx 0.289.

The dose contributes \(-1.5\) to \(\eta_i\); it does not subtract 1.5 from the fitted probability. The probability is obtained only after applying the inverse-logit function to the total predictor.

What Happens When One Covariate Changes?

The additive form gives a direct way to compare two predictor values when all other covariates are held fixed. The coefficient for a numerical covariate determines the change in \(\eta\) for a one-unit increase in that covariate. This statement concerns the linear predictor, not necessarily the response mean.

Result: Fix observation-specific covariate values except \(x_{ij}\), and hold all coefficients fixed. If \(x_{ij}\) changes from \(a\) to \(b\), the resulting change in the linear predictor is \(\beta_j(b-a)\). In particular, a one-unit increase changes \(\eta_i\) by \(\beta_j\).

Proof. Before the change, the predictor is the intercept plus the contributions from all covariates, including \(\beta_j a\). After the change, that term is \(\beta_j b\), while every other term is unchanged. Subtracting the first total from the second cancels the intercept and the unchanged contributions:

\eta_{\text{after}}-\eta_{\text{before}} =(\beta_0+\sum_{k\ne j}\beta_kx_{ik}+\beta_jb) -(\beta_0+\sum_{k\ne j}\beta_kx_{ik}+\beta_ja) =\beta_j(b-a).

Setting \(b=a+1\) gives a change of \(\beta_j\). \(\square\)

Worked Example: Comparing Two Covariate Values

Suppose a model has \(\eta_i=0.6+0.4x_{i1}-0.2x_{i2}+0.05x_{i3}\). Compare a case with covariates \((x_{i1},x_{i2},x_{i3})=(1,4,10)\) to a case with \((3,4,10)\). Only \(x_{i1}\) changes.

For the first case, the contributions are \(0.6\), \(0.4(1)=0.4\), \(-0.2(4)=-0.8\), and \(0.05(10)=0.5\). Thus,

\eta_{\text{first}}=0.6+0.4-0.8+0.5=0.7.

For the second case, the \(x_{i1}\) contribution is \(0.4(3)=1.2\), while the other terms are unchanged:

\eta_{\text{second}}=0.6+1.2-0.8+0.5=1.5.

The difference is \(1.5-0.7=0.8\). The covariate increased by \(3-1=2\), and its coefficient is \(0.4\), so the result also follows from \(0.4(2)=0.8\).

The same subtraction works when several covariates change. If the covariate values for two profiles are \(x_{1},\ldots,x_{p}\) and \(x'_{1},\ldots,x'_{p}\), with the coefficients and intercept unchanged, then the predictor difference is the sum of the changes in their contributions.

Result: For two profiles evaluated with the same coefficients, the difference between their linear predictors is \(\eta'-\eta=\sum_{j=1}^{p}\beta_j(x'_j-x_j)\). The intercept cancels because it has the same value in both predictors.

Proof. Write each predictor as the intercept plus the \(p\) coefficient-value products, and subtract:

\eta'-\eta =\left(\beta_0+\sum_{j=1}^{p}\beta_jx'_j\right) -\left(\beta_0+\sum_{j=1}^{p}\beta_jx_j\right) =\sum_{j=1}^{p}\beta_j(x'_j-x_j).

The equality follows by cancellation of \(\beta_0\) and distributing subtraction across the finite sums. \(\square\)

Why Additive Contributions Matter

Separating the total predictor into terms makes a model easier to audit and explain. For any observation, one can check which covariate contributes most to the numerical value of \(\eta_i\), which terms raise it, and which lower it. These are statements about the model’s specified linear predictor. They do not by themselves establish causal effects or describe how a covariate was measured.

The contributions can also clarify coefficient interpretation. For example, if \(x_{ij}\) increases by one unit and the other covariates remain fixed, \(\eta_i\) changes by \(\beta_j\). What that change means for the mean depends on the link function. With a log link, the mean is transformed exponentially; with a logistic link, the mean is a probability transformed through the inverse-logit. Thus, adding a contribution to \(\eta_i\) is not generally the same as adding that amount to \(\mu_i\).

Common Mistakes

Omitting the intercept. The total is not just the sum of covariate products when the model includes an intercept. Include \(\beta_0\) once.
Dropping a negative sign. A term such as \(-0.2x_{ij}\) contributes \(-0.2x_{ij}\), not \(0.2x_{ij}\). Preserve the coefficient’s sign when multiplying.
Adding coefficients instead of contributions. The contribution for an observation is \(\beta_jx_{ij}\), not \(\beta_j\) alone. The coefficient alone gives the change in \(\eta\) per one-unit increase in the covariate.
Treating \(\eta_i\) as the response mean. The predictor is on the link scale. Apply the inverse link to find \(\mu_i\), if the question asks for the mean.
Assuming equal changes in \(\eta\) mean equal changes in the mean. The inverse link may be nonlinear, so the same predictor change can correspond to different changes in the mean at different starting values.

Key takeaway: For each observation, calculate \(\beta_jx_{ij}\) for every numerical covariate, add those products to the intercept, and interpret the result as \(\eta_i\). A coefficient describes the change in \(\eta_i\) per one-unit covariate increase with other covariates fixed; the inverse link determines what that means on the response scale.

Check Your Understanding

Use the additive contributions to answer each question.

For \(\eta_i=1+0.3x_{i1}-0.4x_{i2}\), calculate the intercept and covariate contributions and the total predictor when \(x_{i1}=5\) and \(x_{i2}=2\).
In a model with coefficient \(-0.25\) for a numerical covariate, what is the change in \(\eta\) when that covariate increases by 4 units and all others are fixed?
Explain why a negative covariate contribution does not mean that the covariate value itself is negative.
Two profiles differ only in a covariate whose coefficient is \(0.6\). The second profile’s value is 3 units larger. Find the difference between their linear predictors.
In a GLM with a log link, should the sum of the intercept and covariate contributions be reported as the expected count, or should it be transformed? Explain.