Covariance with Monotone Functions

Why Monotonicity Determines the Covariance Sign

A random variable and a function of that same variable are dependent by construction. Their covariance can often be determined without calculating a joint distribution: if larger values of $X$ always correspond to at least as large values of $g(X)$, their centered values tend to have the same sign. The precise result uses an independent copy of $X$ to make that intuition rigorous.

This result is useful when a direct covariance calculation would require several moments or a lengthy sum or integral. It also gives a sign check: if $g$ is nondecreasing on the values that can occur, a computed negative covariance signals an error or a failure of the assumptions.

Definition: Let $X$ be a random variable and let $g$ be a measurable real-valued function. Suppose $E|X|<\infty$, $E|g(X)|<\infty$, and $E|Xg(X)|<\infty$. Then $$ \operatorname{Cov}(X,g(X))=E[Xg(X)]-E[X]E[g(X)]. $$ The function $g$ is nondecreasing on a set $A$ if $g(x_1)\leq g(x_2)$ whenever $x_1,x_2\in A$ and $x_1\leq x_2$.

The integrability conditions ensure that the expectations in the covariance formula exist and are finite. A convenient sufficient condition, when $E[X^2]$ is finite, is that $E[g(X)^2]$ is finite: the Cauchy–Schwarz inequality then gives $E|Xg(X)|<\infty$. The theorem below uses the direct integrability conditions in the definition and does not require a particular distribution, such as a normal or uniform distribution.

The Independent-Copy Result

Theorem: Suppose $X$ satisfies the integrability conditions above. If $g$ is nondecreasing on some measurable set $A$ with $P(X\in A)=1$, then $$ \operatorname{Cov}(X,g(X))\geq 0. $$

A set with probability one is enough: the theorem only needs monotonicity for values that $X$ actually takes, apart from events of probability zero. In particular, the standard topological support of a distribution—the smallest closed set with probability one—is one possible choice of $A$, if $g$ is nondecreasing there. This is not the same as collecting only values that individually have positive probability or a positive density. For example, a singular continuous distribution may have no atoms and no density while still having a nonempty topological support. More generally, the theorem only requires a probability-one set on which $g$ is nondecreasing.

Here is the independent-copy justification. Let $X'$ be an independent random variable with the same distribution as $X$, and apply the same function $g$ to it. Both $X$ and $X'$ belong to $A$ with probability one. Thus, whenever $X\geq X'$, nondecreasingness gives $g(X)\geq g(X')$; when $X\leq X'$, it gives $g(X)\leq g(X')$. In either case, the two differences have the same sign or one is zero.

(X-X')\bigl(g(X)-g(X')\bigr)\geq 0 \quad\text{with probability one.}

The integrability assumptions also allow us to take expectations and expand the product. Independence gives $E[Xg(X')]=E[X]E[g(X')]$, and $X'$ has the same distribution as $X$. The other cross term factors in the same way. Therefore:

\begin{aligned} E\!\left[(X-X')\bigl(g(X)-g(X')\bigr)\right] &=E[Xg(X)]+E[X'g(X')]\\ &\quad-E[Xg(X')]-E[X'g(X)]\\ &=2E[Xg(X)]-2E[X]E[g(X)]\\ &=2\operatorname{Cov}(X,g(X)). \end{aligned}

The left side is the expectation of a nonnegative random variable, so it is nonnegative. Dividing by $2$ proves the theorem. This argument establishes the sign without requiring a density, a probability table, or a special distribution.

Check the expectations.
Confirm that the covariance exists; the theorem assumes $E|X|$, $E|g(X)|$, and $E|Xg(X)|$ are finite.

Check monotonicity where it matters.
Find a probability-one set on which $g$ is nondecreasing. It need not be nondecreasing at values that $X$ cannot take.

Conclude the sign or calculate the value.
The theorem gives a nonnegative sign. If the question asks for the numerical covariance, compute $E[Xg(X)]-E[X]E[g(X)]$ and check that the result agrees.

Worked Applications

Worked Example: A Nonlinear Function of a Discrete Variable

A hypothetical screening score $X$ has the following distribution: $P(X=0)=0.2$, $P(X=1)=0.5$, and $P(X=3)=0.3$. Let $g(x)=x^2$. Find $\operatorname{Cov}(X,g(X))$, and use monotonicity to check its sign.

Method and assumptions. This is a finite distribution, so all relevant expectations exist. Although $x^2$ is not nondecreasing on the entire real line, its values on the probability-one set $A=\{0,1,3\}$ are $0,1,9$, in nondecreasing order. The theorem therefore predicts a nonnegative covariance.

Step 1: Calculate $E[X]$. $$ E[X]=(0)(0.2)+(1)(0.5)+(3)(0.3)=0+0.5+0.9=1.4. $$

Step 2: Calculate $E[g(X)]$. Since $g(X)=X^2$, $$ E[g(X)]=(0^2)(0.2)+(1^2)(0.5)+(3^2)(0.3)=0+0.5+2.7=3.2. $$

Step 3: Calculate $E[Xg(X)]$. Here $Xg(X)=X^3$, so $$ E[Xg(X)]=(0^3)(0.2)+(1^3)(0.5)+(3^3)(0.3)=0+0.5+8.1=8.6. $$

Step 4: Apply the covariance formula. $$ \operatorname{Cov}(X,g(X))=8.6-(1.4)(3.2)=8.6-4.48=4.12. $$ The covariance is $4.12$, which is nonnegative as the theorem predicts. Common mistake avoided: checking that $x^2$ is increasing on all real numbers. It is enough here that it is nondecreasing on the values $X$ can take.

Worked Example: A Continuous Variable With a Cubic Transformation

A hypothetical instrument reading $X$ is uniformly distributed on $[-1,2]$, so its density is $f_X(x)=1/3$ for $-1\leq x\leq 2$ and $0$ otherwise. For $g(x)=x^3$, find $\operatorname{Cov}(X,g(X))$ and check the sign by the theorem.

Method and assumptions. The variable is bounded, so the expectations of $X$, $g(X)$, and $Xg(X)$ are finite. The function $x^3$ is increasing on the real line. The theorem predicts a nonnegative covariance.

Step 1: Calculate $E[X]$. $$ E[X]=\frac{1}{3}\int_{-1}^{2}x\,dx =\frac{1}{3}\left[\frac{x^2}{2}\right]_{-1}^{2} =\frac{1}{3}\left(2-\frac{1}{2}\right)=\frac{1}{2}. $$

Step 2: Calculate $E[g(X)]$. $$ E[g(X)]=E[X^3] =\frac{1}{3}\int_{-1}^{2}x^3\,dx =\frac{1}{3}\left[\frac{x^4}{4}\right]_{-1}^{2} =\frac{1}{3}\left(4-\frac{1}{4}\right)=\frac{5}{4}. $$

Step 3: Calculate $E[Xg(X)]$. Since $Xg(X)=X^4$, $$ E[Xg(X)]=E[X^4] =\frac{1}{3}\int_{-1}^{2}x^4\,dx =\frac{1}{3}\left[\frac{x^5}{5}\right]_{-1}^{2} =\frac{1}{3}\left(\frac{32}{5}+\frac{1}{5}\right)=\frac{11}{5}. $$

Step 4: Calculate and check the covariance. $$ \operatorname{Cov}(X,X^3)=\frac{11}{5}-\left(\frac{1}{2}\right)\left(\frac{5}{4}\right) =\frac{11}{5}-\frac{5}{8} =\frac{88-25}{40}=\frac{63}{40}=1.575. $$ The result is $1.575$, consistent with the theorem. Common mistake avoided: using $E[X^3]$ as $E[Xg(X)]$. The product is $X\cdot X^3=X^4$, so the fourth moment is needed.

Worked Example: Monotonicity Only on the Values That Occur

A hypothetical classification variable $X$ takes the value $1$ with probability $1/4$ and $3$ with probability $3/4$. A score transformation satisfies $g(1)=2$ and $g(3)=5$, but also has $g(2)=100$. Find $\operatorname{Cov}(X,g(X))$ and decide whether the theorem applies.

Method and assumptions. Only the values 1 and 3 occur, so $A=\{1,3\}$ has probability one. On that set, $g(1)=2\leq 5=g(3)$, so $g$ is nondecreasing there. It is not nondecreasing on the entire real line, since $2<3$ but $g(2)=100>g(3)=5$. All expectations are finite because $X$ has only two possible values and the corresponding function values are finite.

Step 1: Calculate the marginal expectations. $$ E[X]=(1)\left(\frac{1}{4}\right)+(3)\left(\frac{3}{4}\right) =\frac{1}{4}+\frac{9}{4}=\frac{5}{2}, $$ $$ E[g(X)]=(2)\left(\frac{1}{4}\right)+(5)\left(\frac{3}{4}\right) =\frac{1}{2}+\frac{15}{4}=\frac{17}{4}. $$

Step 2: Calculate the product expectation. $$ E[Xg(X)]=(1)(2)\left(\frac{1}{4}\right)+(3)(5)\left(\frac{3}{4}\right) =\frac{1}{2}+\frac{45}{4}=\frac{47}{4}. $$

Step 3: Calculate the covariance. $$ \operatorname{Cov}(X,g(X)) =\frac{47}{4}-\left(\frac{5}{2}\right)\left(\frac{17}{4}\right) =\frac{47}{4}-\frac{85}{8} =\frac{94-85}{8}=\frac{9}{8}=1.125. $$ The covariance is positive, as predicted. The value $g(2)$ does not affect the calculation because $P(X=2)=0$. Common mistake avoided: rejecting the theorem because $g$ is not nondecreasing everywhere, rather than checking its behavior on a probability-one set.

When the Result Is Strict, and When It Does Not Apply

The theorem guarantees a nonnegative covariance, not necessarily a positive one. If $g$ is constant over all values that $X$ takes, then $g(X)$ is constant and the covariance is zero. More generally, a nondecreasing function can have flat sections, so distinct values of $X$ may map to the same value of $g(X)$. In contrast, if $g$ is strictly increasing on a probability-one set and $X$ is not almost surely constant, then the covariance is positive. Indeed, the independent-copy product in the proof is positive whenever $X\ne X'$; for a nonconstant $X$, that event has positive probability.

The monotonicity direction matters. A decreasing function reverses the order of the values, so the differences $X-X'$ and $g(X)-g(X')$ have opposite signs. Under the same integrability conditions, the independent-copy argument then gives $\operatorname{Cov}(X,g(X))\leq 0$. If the function is neither nondecreasing nor nonincreasing on a probability-one set, this sign result does not follow; a direct calculation may be needed.

Common Mistake: Treating a Decreasing Function as Increasing

Suppose $X$ takes the value $0$ with probability $0.7$ and $1$ with probability $0.3$, and let $g(x)=-x$. A mistaken argument says that $g(X)$ increases whenever $X$ increases, so the covariance should be nonnegative. The correction is that $g$ is decreasing, not increasing.

Calculate the expectations: $$ E[X]=(0)(0.7)+(1)(0.3)=0.3,\qquad E[g(X)]=E[-X]=-0.3, $$ $$ E[Xg(X)]=E[-X^2]=-(0^2)(0.7)-(1^2)(0.3)=-0.3. $$ Therefore, $$ \operatorname{Cov}(X,-X)=-0.3-(0.3)(-0.3)=-0.3+0.09=-0.21. $$ The negative result is correct. As a check, $\operatorname{Cov}(X,-X)=-\operatorname{Var}(X)$, and this distribution has $\operatorname{Var}(X)=(0.3)(0.7)=0.21$.

In an exam calculation, separate the sign argument from the numerical calculation. First verify integrability and the direction of monotonicity on a probability-one set. Then use the theorem to predict the sign. If a value is requested, calculate $E[Xg(X)]$, $E[X]$, and $E[g(X)]$ explicitly, and compare the result with that prediction. This two-part check catches errors such as omitting a probability weight, using the wrong power in a product moment, or assuming that a function is increasing merely because it is positive.

Key takeaway: For an integrable random variable $X$, a measurable nondecreasing function $g$ on a probability-one set gives $\operatorname{Cov}(X,g(X))\geq 0$, provided the covariance expectations are finite. The independent-copy identity explains why; it does not require $X$ to have a density or $g$ to be monotone away from values that can occur.

Check Your Understanding

For each problem, check the relevant monotonicity and expectation conditions before calculating.

A discrete variable $X$ equals $-1$ with probability $0.6$ and $2$ with probability $0.4$. Let $g(x)=x^2$. Calculate $\operatorname{Cov}(X,g(X))$ and explain why the nonnegative-covariance result applies on the values of $X$. (Answer: $2.16$; $x^2$ takes the ordered values $1$ and $4$ on $\{-1,2\}$.)
Let $X$ be uniform on $[0,2]$ and $g(x)=2x+1$. Calculate $\operatorname{Cov}(X,g(X))$ using the covariance formula and verify its sign. (Answer: $2/3$.)
A variable $X$ takes $0,1,4$ with probabilities $0.25,0.50,0.25$, respectively. Let $g(x)=\sqrt{x}$ for $x\geq0$. Calculate $\operatorname{Cov}(X,g(X))$. (Answer: $1$.)
A discrete variable $X$ takes $2,5,8$ with probabilities $0.2,0.5,0.3$. A measurable function has $g(2)=4$, $g(5)=4$, and $g(8)=10$, with arbitrary values elsewhere. Calculate the covariance and state why the values elsewhere do not affect the theorem. (Answer: $4.86$; $g$ is nondecreasing on the probability-one set $\{2,5,8\}$, and other values are never sampled.)
A Bernoulli variable $X$ has $P(X=1)=0.3$ and $P(X=0)=0.7$. For $g(x)=-x$, calculate the covariance and explain why the nonnegative-covariance theorem does not apply. (Answer: $-0.21$; $g$ is decreasing, not nondecreasing.)

What You'll Learn