Complementary Log-Log Regression

From a Binary Probability to a Link

For a binary response, a regression model must produce a conditional probability between zero and one. A link function transforms that probability to a scale on which a linear predictor can take any real value. The complementary log-log link is one such link. Its inverse converts a real-valued linear predictor back to a probability.

Let \(Y_i\) be a binary response and let \(\pi_i=P(Y_i=1\mid x_i)\), where \(x_i\) is the covariate vector for observation \(i\). In a complementary log-log regression, the mean is linked to a linear predictor \(\eta_i=x_i^\mathsf{T}\beta\). The link is nonlinear, so a change in \(\eta_i\) should not be read as a fixed change in probability.

Definition: For a probability \(0<\pi<1\), the complementary log-log link is \(g(\pi)=\log[-\log(1-\pi)]\). Its inverse is \(g^{-1}(\eta)=1-\exp[-\exp(\eta)]\), for \(\eta\in\mathbb{R}\). In a complementary log-log regression, \(g(\pi_i)=\eta_i=x_i^\mathsf{T}\beta\), so the fitted probability is \(\hat\pi_i=1-\exp[-\exp(x_i^\mathsf{T}\hat\beta)]\).

The inner quantity \(-\log(1-\pi)\) is positive whenever \(0<\pi<1\), so its logarithm is defined. It is useful to give this transformed quantity a name:

Definition: Define \(H(\pi)=-\log(1-\pi)\). On this scale, the complementary log-log link is \(g(\pi)=\log H(\pi)\), and therefore \(H(\pi)=\exp[g(\pi)]\). For a fitted mean, write \(H_i=-\log(1-\pi_i)\); this is a transformed cumulative-hazard quantity. Here it is an algebraic transformation of a binary probability, not by itself a claim about event times.

Why the Inverse Has This Form

The inverse follows directly by solving the link equation. This also verifies that every real linear predictor gives a valid probability and that every probability strictly between zero and one has a finite link value.

Result: The map \(g(\pi)=\log[-\log(1-\pi)]\) is a one-to-one map from \((0,1)\) onto \(\mathbb{R}\), with inverse \(g^{-1}(\eta)=1-\exp[-\exp(\eta)]\).

Solve for the inner quantity.
Set \(\eta=\log[-\log(1-\pi)]\). Exponentiating gives \(\exp(\eta)=-\log(1-\pi)\).

Solve for the probability.
Multiply by \(-1\) and exponentiate: \(1-\pi=\exp[-\exp(\eta)]\). Thus \(\pi=1-\exp[-\exp(\eta)]\).

Check the range and the reverse composition.
For finite \(\eta\), \(\exp(\eta)>0\), so \(0<\exp[-\exp(\eta)]<1\), which implies \(0<g^{-1}(\eta)<1\). Conversely, for \(0<\pi<1\), \(H=-\log(1-\pi)>0\), and \(g^{-1}(\log H)=1-\exp(-H)=\pi\).

The restrictions matter: probabilities of exactly zero or one correspond to limits of the link, not finite link values. In a fitted model with finite coefficients and finite covariates, the inverse link returns a probability strictly inside the unit interval.

Worked Example: Converting a Probability to Its Predictor

Suppose a covariate profile has probability \(\pi=0.40\). Find its complementary log-log predictor and check the inverse calculation.

First calculate the transformed quantity: \(-\log(1-\pi)=-\log(0.60)=0.5108256\). The predictor is therefore \(\eta=\log(0.5108256)\approx-0.6717\).

To check, exponentiate the predictor: \(\exp(-0.6717)\approx0.5108\). Applying the inverse link gives \(1-\exp(-0.5108)\approx1-0.6000=0.4000\). The small discrepancy in the final digits comes from rounding the predictor and intermediate value.

Linear Predictors and Fitted Probabilities

As in other generalized linear models, the regression structure is linear in the covariates on the link scale. For example, with one quantitative covariate \(x\), an intercept and slope give \(\eta(x)=\beta_0+\beta_1x\). The probability is not linear in \(x\); it is obtained by applying the inverse link to that predictor.

Worked Example: Fitting Probabilities at Three Covariate Values

Consider the specified model \(\eta(x)=-1+0.7x\). Calculate the fitted probabilities at \(x=0\), \(x=1\), and \(x=2\).

At \(x=0\), \(\eta=-1\), so \(H=\exp(-1)=0.3679\). The fitted probability is \(1-\exp(-0.3679)\approx0.3078\).

At \(x=1\), \(\eta=-0.3\), so \(H=\exp(-0.3)=0.7408\). The fitted probability is \(1-\exp(-0.7408)\approx0.5233\).

At \(x=2\), \(\eta=0.4\), so \(H=\exp(0.4)=1.4918\). The fitted probability is \(1-\exp(-1.4918)\approx0.7750\).

The calculations use the same sequence at every profile: form the linear predictor, exponentiate it to obtain \(H\), then calculate \(1-\exp(-H)\). The resulting probability increments are not equal, even though \(x\) increases by one each time.

What a Change in the Predictor Means

The predictor has a particularly direct interpretation on the \(H\) scale. If a covariate profile changes the predictor by \(\Delta\), then the transformed quantity is multiplied by \(\exp(\Delta)\). The probability itself changes through the inverse link and does not generally have a constant additive or multiplicative change.

Theorem: Suppose two covariate profiles have predictors \(\eta_1\) and \(\eta_2=\eta_1+\Delta\), and let \(H_j=-\log(1-\pi_j)\) for their probabilities. Then \(\frac{H_2}{H_1}=\exp(\Delta)\). Their probabilities satisfy \(\pi_1=1-\exp(-H_1)\) and \(\pi_2=1-\exp[-\exp(\Delta)H_1]\).

To prove the result, the link equation gives \(H_1=\exp(\eta_1)\) and \(H_2=\exp(\eta_2)\). Substitution of \(\eta_2=\eta_1+\Delta\) yields \[ \frac{H_2}{H_1} =\frac{\exp(\eta_1+\Delta)}{\exp(\eta_1)} =\exp(\Delta). \] Applying the inverse relation \(\pi=1-\exp(-H)\) to each transformed quantity gives the two probability expressions in the theorem. Thus the multiplicative interpretation applies exactly to \(H\), not directly to \(\pi\). \(\square\)

In a model with a quantitative covariate \(x\) and slope \(\beta_1\), increasing \(x\) by \(c\) changes the predictor by \(\Delta=c\beta_1\). Consequently, \(H\) is multiplied by \(\exp(c\beta_1)\). For a binary covariate, the same statement compares its two coded values, holding the other covariates fixed. The coefficient is not itself a probability difference.

Worked Example: Interpreting a Predictor Change

Use the model \(\eta(x)=0.4-0.8x\), and compare the explicitly specified profiles \(x=0\) and \(x=0.5\). Find both fitted probabilities and the ratio of their transformed quantities.

At \(x=0\), \(\eta_0=0.4\), so \(H_0=\exp(0.4)=1.4918\), and \(\pi_0=1-\exp(-1.4918)\approx0.7750\).

At \(x=0.5\), \(\eta_{0.5}=0.4-0.8(0.5)=0\), so \(H_{0.5}=\exp(0)=1\), and \(\pi_{0.5}=1-\exp(-1)\approx0.6321\).

The predictor change is \(0-0.4=-0.4\). Therefore \(\frac{H_{0.5}}{H_0}=\frac{1}{\exp(0.4)}=\exp(-0.4)\approx0.6703\). Equivalently, the transformed quantity falls by about \(32.97\%\). The probability falls by about \(0.7750-0.6321=0.1429\), or 14.29 percentage points; that probability difference is not the coefficient \(-0.8\), nor is it the transformed-quantity ratio.

A derivative gives a local version of the probability effect. Writing \(\pi(\eta)=1-\exp[-\exp(\eta)]\), differentiation by the chain rule gives \(\frac{d\pi}{d\eta}=\exp(\eta)\exp[-\exp(\eta)]\). Since \(H=\exp(\eta)\) and \(1-\pi=\exp(-H)\), this is also \(\frac{d\pi}{d\eta}=H(1-\pi)\). For a covariate \(x\) with slope \(\beta_1\), the derivative is \(\frac{d\pi}{dx}=\beta_1H(1-\pi)\). It depends on the profile through \(H\) and \(\pi\), so it is not a single constant effect across observations.

Result: Under the complementary log-log link, a predictor shift \(\Delta\) multiplies \(-\log(1-\pi)\) by \(\exp(\Delta)\). The probability is recovered only after applying the nonlinear inverse link.

Common Mistakes and Practical Interpretation

The transformation \(H=-\log(1-\pi)\) is sometimes called a cumulative-hazard scale because it has that form in event-time settings. In this tutorial it is defined from a binary probability alone. The connection to a discrete-time hazard has its own assumptions and interpretation; the algebraic link definition does not establish them.

Confusing the link with the inverse link. The link maps probability to predictor: \(g(\pi)=\log[-\log(1-\pi)]\). The inverse maps predictor to probability: \(1-\exp[-\exp(\eta)]\).
Reading a coefficient as a risk difference. A coefficient change \(\Delta\) multiplies \(H\) by \(\exp(\Delta)\); it does not add \(\Delta\) to the probability.
Calling \(\exp(\beta_1)\) an odds ratio. It is a multiplier for \(H\) for a one-unit increase in \(x\), not an odds ratio and not generally a risk ratio.
Dropping a negative sign. The correct transformed quantity is \(-\log(1-\pi)\), which is positive. Omitting the minus sign makes the logarithm of the transformed quantity invalid for \(0<\pi<1\).
Leaving the comparison profile unspecified. To calculate numerical predictors, state the covariate values being compared and substitute them into the full linear predictor.

Check Your Understanding

Use the link, inverse, and transformed-scale interpretation to answer these questions.

For \(\pi=0.25\), write the complementary log-log predictor exactly in terms of logarithms. What is its approximate value?
If a predictor changes from \(\eta\) to \(\eta+0.5\), by what factor does \(H=-\log(1-\pi)\) change?
For a model with slope \(\beta_1=-0.3\), describe the effect of increasing \(x\) by two units on \(H\).
Why is a complementary log-log coefficient not generally a constant probability difference?
What additional interpretation is not justified merely by defining \(H=-\log(1-\pi)\) for a binary response?