Cauchit Regression And Heavy-Tailed Probability Changes

From a Linear Predictor to a Probability

In a binary regression model, the linear predictor can take any real value, but the fitted mean must be a probability between zero and one. A link function connects these scales. The cauchit link uses the quantiles of a standard Cauchy distribution: it converts a probability to the Cauchy quantile at that probability. Its inverse converts a linear predictor back to a probability.

This choice has a practical consequence for extreme predictors. For example, a large positive predictor does produce a probability near one, but the change in probability is not a constant multiple of the predictor. The calculations below show how to interpret the coefficient and how to evaluate the probability directly.

Definition: The standard Cauchy distribution has cumulative distribution function $F_C(t)=\frac{1}{2}+\frac{1}{\pi}\arctan(t)$, for $t\in\mathbb{R}$. The cauchit link is its quantile function: $$ g(p)=F_C^{-1}(p)=\tan\!\left(\pi\left(p-\frac{1}{2}\right)\right),\qquad 0<p<1. $$ Its inverse link is $$ g^{-1}(\eta)=\frac{1}{2}+\frac{1}{\pi}\arctan(\eta),\qquad \eta\in\mathbb{R}. $$

For binary responses, write $Y_i\sim\operatorname{Bernoulli}(p_i)$, where $p_i=P(Y_i=1)$. In a cauchit regression model, $g(p_i)=\eta_i$, with linear predictor $\eta_i=\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta}$. Thus the fitted probability at predictor value $\eta_i$ is

p_i=\frac{1}{2}+\frac{1}{\pi}\arctan(\eta_i).

The formula gives a valid probability for every finite real $\eta_i$. It also gives $p_i=1/2$ when $\eta_i=0$: the center of the standard Cauchy distribution is zero, and its median is zero.

Why This Is a Valid Link

A link for a binary mean must pair probabilities in $(0,1)$ with real-valued linear predictors. The cauchit link does this because the standard Cauchy distribution function is continuous and strictly increasing, and its values range from zero to one as its argument runs from negative to positive infinity.

Result: The cauchit link is a one-to-one map from $(0,1)$ onto $\mathbb{R}$, and the displayed inverse link maps $\mathbb{R}$ into $(0,1)$. Its inverse derivative is $$ \frac{d}{d\eta}g^{-1}(\eta)=\frac{1}{\pi(1+\eta^2)}. $$

Proof. Differentiating the standard Cauchy distribution function gives

F_C'(t)=\frac{1}{\pi(1+t^2)}>0 \qquad\text{for every }t\in\mathbb{R}.

Therefore $F_C$ is strictly increasing and continuous. Since $\arctan(t)$ tends to $-\pi/2$ as $t$ tends to $-\infty$, and to $\pi/2$ as $t$ tends to $+\infty$, $F_C(t)$ tends to zero and one, respectively. Hence its range is $(0,1)$ and it has an inverse on that range. Solving $p=\frac12+\frac1\pi\arctan(t)$ for $t$ gives $t=\tan(\pi(p-\frac12))$, which is the stated link. Conversely, substituting any real $\eta$ into $\frac12+\frac1\pi\arctan(\eta)$ gives a value strictly between zero and one. Finally, differentiating that inverse expression gives $1/(\pi(1+\eta^2))$. $\square$

Interpreting Cauchit Coefficients

Because $g(p_i)=\eta_i$, a coefficient describes a change in the Cauchy quantile of the response probability. If $x_{ij}$ is a predictor with coefficient $\beta_j$, increasing $x_{ij}$ by one unit while holding the other predictors fixed changes $\eta_i$ by $\beta_j$. Equivalently, it shifts the Cauchy quantile corresponding to $p_i$ by $\beta_j$.

This is not a constant change in probability. The probability-scale effect depends on the starting predictor. Differentiating the inverse link shows that, for a continuous predictor $x_j$,

\frac{\partial p_i}{\partial x_{ij}} = \frac{\beta_j}{\pi(1+\eta_i^2)}.

The derivative is largest in magnitude around $\eta_i=0$, where the fitted probability is $1/2$. At large absolute predictors it is smaller. The coefficient therefore determines the direction of the probability change through its sign, but not a single probability difference that applies to every observation.

Worked Example: Interpreting a Cauchit Slope

Suppose a fitted model has $\eta=-1+2x$, where $x$ is a binary exposure indicator. Interpret the slope and calculate the fitted probabilities at $x=0$ and $x=1$.

Quantile-scale interpretation. Changing $x$ from zero to one increases the linear predictor by $2$. Thus the Cauchy quantile associated with the response probability increases by two units.

At $x=0$, $\eta=-1$, so $$ p(0)=\frac12+\frac1\pi\arctan(-1) =\frac12-\frac14 =\frac14. $$ At $x=1$, $\eta=1$, so $$ p(1)=\frac12+\frac1\pi\arctan(1) =\frac12+\frac14 =\frac34. $$ The fitted probability difference for this particular contrast is $3/4-1/4=1/2$. That difference is a consequence of the selected intercept and slope; it is not what the slope of two means for all possible starting values.

Calculating Probabilities at Extreme Predictors

For a large positive predictor, the inverse link is close to one; for a large negative predictor, it is close to zero. The probabilities can still be calculated without approximating the link by a different model. For $a>0$, the identity $\arctan(a)=\pi/2-\arctan(1/a)$ gives

g^{-1}(a)=1-\frac{1}{\pi}\arctan(1/a), \qquad g^{-1}(-a)=\frac{1}{\pi}\arctan(1/a).

These formulas are useful for checking small probabilities in the tails. They also make the symmetry of the fitted probabilities explicit: positive and negative predictors of equal magnitude give probabilities that sum to one.

Result: For every real $\eta$, $g^{-1}(-\eta)=1-g^{-1}(\eta)$.

Proof. The arctangent is an odd function, so $$ g^{-1}(-\eta) =\frac12+\frac1\pi\arctan(-\eta) =\frac12-\frac1\pi\arctan(\eta) =1-\left(\frac12+\frac1\pi\arctan(\eta)\right) =1-g^{-1}(\eta). $$ Thus the two probabilities sum to one. $\square$

Worked Example: Fitted Probabilities at Large Predictors

Calculate the cauchit fitted probabilities for $\eta=10$, $\eta=-10$, $\eta=100$, and $\eta=-100$. Keep enough precision to show how close the extreme fitted probabilities are to zero or one.

For $\eta=10$, $$ g^{-1}(10)=\frac12+\frac{\arctan(10)}{\pi} \approx\frac12+\frac{1.471128}{3.141593} \approx 0.968274. $$ By symmetry, $g^{-1}(-10)=1-0.968274\approx0.031726$.

For $\eta=100$, use $\arctan(100)=\pi/2-\arctan(0.01)$: $$ g^{-1}(100) =1-\frac{\arctan(0.01)}{\pi} \approx 1-\frac{0.00999967}{3.141593} \approx 0.996817. $$ Consequently, $g^{-1}(-100)\approx 1-0.996817=0.003183$. Even at a predictor of magnitude 100, the fitted probabilities are close to, but not equal to, zero or one. The formula remains well-defined for any finite predictor.

Worked Example: A Finite Probability Change

Suppose the current linear predictor is $1$, and a one-unit increase in a continuous predictor has coefficient $0.5$. Calculate the fitted probability before and after the increase, and compare the finite change with the local derivative at the starting value.

The predictor moves from $1$ to $1+0.5=1.5$. Initially, $$ p_{\text{start}}=g^{-1}(1) =\frac12+\frac{\pi/4}{\pi} =0.75. $$ After the increase, $$ p_{\text{new}} =g^{-1}(1.5) =\frac12+\frac{\arctan(1.5)}{\pi} \approx\frac12+\frac{0.982794}{3.141593} \approx0.812832. $$ The finite probability change is approximately $0.812832-0.75=0.062832$.

At the starting predictor, the derivative with respect to the predictor itself is $1/[\pi(1+1^2)]=1/(2\pi)$. Multiplying by the coefficient gives a local probability slope of $0.5/(2\pi)\approx0.079577$ per unit increase in the covariate. This is a local rate, not the finite change over a full unit; the actual change is about $0.062832$.

What the Heavy-Tailed Link Means in Practice

The cauchit link uses the standard Cauchy distribution, whose quantiles extend far into the tails. On the probability scale this means that extreme linear predictors can correspond to probabilities that are very close to zero or one without reaching either boundary at a finite predictor. The calculations at 10 and 100 illustrate the conversion directly.

The link does not make a coefficient a probability difference, a risk ratio, or an odds ratio. Its direct interpretation is a shift in Cauchy quantiles; probabilities must be recovered with the inverse link. For a particular covariate contrast, calculate the two linear predictors, transform both, and then take their difference if a probability difference is wanted.

Common Mistake

Wrong reasoning: “A coefficient of $0.5$ means the probability increases by 0.5.” That treats the coefficient as a direct probability-scale effect.
Correction: A coefficient of $0.5$ adds $0.5$ to the Cauchy-quantile-scale predictor for a one-unit covariate increase. The probability change depends on the starting predictor. Apply $g^{-1}(\eta)=\frac12+\frac1\pi\arctan(\eta)$ at both predictor values to obtain the actual change.

A second useful check is that finite predictors must give probabilities strictly between zero and one. A computed value of exactly zero or one for a finite $\eta$ signals a calculation or numerical-rounding issue, not the exact cauchit inverse link.

Key takeaway: In cauchit regression, the linear predictor is a standard Cauchy quantile of the fitted probability. Coefficients shift that quantile, while the inverse link determines the probability change; evaluate it at the actual predictor values, especially for extreme predictors.

Check Your Understanding

Use the inverse link and the coefficient interpretation to answer each question.

What is the cauchit fitted probability when the linear predictor is zero, and why?
A coefficient is $1.2$. What does a one-unit increase in its predictor change directly: the probability or the Cauchy quantile?
Calculate the fitted probability for $\eta=-1$.
Without a calculator, use symmetry to obtain the fitted probability at $\eta=-10$ from the fitted probability at $\eta=10$.
Explain why a coefficient cannot generally be read as a fixed probability difference.