Covariance Signs for Monotone Functions

Why Monotonicity Determines a Covariance Sign

If larger values of a random variable tend to produce larger values of a function of that variable, it is natural to expect the two quantities to have nonnegative covariance. This intuition can be made exact without assuming a particular distribution or deriving its density. The key is to compare two independent draws of the same random variable: when the function is nondecreasing, the two differences always have the same sign.

The conclusion concerns $X$ and $g(X)$, which are generally dependent because one is a function of the other. It does not say that every pair of dependent random variables has positive covariance, nor that every increasing transformation has a covariance of a particular numerical size. The sign follows from monotonicity, while the value depends on the distribution and the function.

Definition: A function $g$ is nondecreasing on an interval $I$ if $x\le y$ implies $g(x)\le g(y)$ for all $x,y\in I$. It is nonincreasing if $x\le y$ implies $g(x)\ge g(y)$. A random variable $X$ is square integrable if $E[X^2]<\infty$.

For a precise statement, suppose $X$ takes values in $I$ with probability one and $g:I\to\mathbb{R}$ is measurable. A convenient sufficient condition for the covariance to exist is $E[X^2]<\infty$ and $E[g(X)^2]<\infty$. The theorem below uses the slightly more direct conditions $E|X|<\infty$, $E|g(X)|<\infty$, and $E|Xg(X)|<\infty$. These ensure that the means and the product expectation in the covariance are finite.

Definition: When the expectations are finite, the covariance of $X$ and $g(X)$ is $$ \operatorname{Cov}(X,g(X))=E[Xg(X)]-E[X]E[g(X)]. $$

The Independent-Copy Identity

Let $X'$ be an independent random variable with the same distribution as $X$, defined on a product probability space if necessary. Then $(X-X')$ and $(g(X)-g(X'))$ move in the same direction whenever $g$ is nondecreasing. The following identity turns that pointwise observation into a covariance result.

Theorem: Suppose $X'$ is an independent copy of $X$, $g(X')$ is defined using the same measurable function $g$, and $E|X|<\infty$, $E|g(X)|<\infty$, and $E|Xg(X)|<\infty$. Then $$ \operatorname{Cov}(X,g(X)) =\frac12 E\big[(X-X')(g(X)-g(X'))\big]. $$

Proof. Expand the product. Each term is integrable: the two same-draw terms have absolute expectation $E|Xg(X)|$, and independence gives $E|Xg(X')|=E|X|E|g(X)|<\infty$, with the same argument for $X'g(X)$. Thus linearity of expectation and independence may be used:

\begin{aligned} E\big[(X-X')(g(X)-g(X'))\big] &=E[Xg(X)]+E[X'g(X')]\\ &\quad-E[Xg(X')]-E[X'g(X)]\\ &=2E[Xg(X)]-2E[X]E[g(X)]\\ &=2\operatorname{Cov}(X,g(X)). \end{aligned}

Here $X'$ has the same distribution as $X$, so $E[X'g(X')]=E[Xg(X)]$ and $E[g(X')]=E[g(X)]$. Dividing by two proves the identity. $\square$

Theorem: Under the integrability conditions above, if $g$ is nondecreasing on an interval containing $X$ almost surely, then $\operatorname{Cov}(X,g(X))\ge 0$. If $g$ is nonincreasing there, then $\operatorname{Cov}(X,g(X))\le 0$.

Proof. For any $x,y\in I$, a nondecreasing $g$ satisfies $(x-y)(g(x)-g(y))\ge 0$: when $x\ge y$, both factors are nonnegative, and when $x\le y$, both are nonpositive. Consequently, the integrand in the independent-copy identity is nonnegative almost surely, and its expectation is nonnegative. If $g$ is nonincreasing, the two differences have opposite signs, so their product is nonpositive almost surely and the expectation is nonpositive. $\square$

The proof needs monotonicity only on an interval that contains $X$ almost surely; $g$ need not be monotone outside that interval. The integrability conditions matter too: a pointwise sign for the product does not by itself make an undefined covariance meaningful.

Worked Examples Across Distributions

Worked Example: A Uniform Variable and a Quadratic Function

Let $X\sim \operatorname{Uniform}(0,1)$ and $g(x)=x^2$. Since $x^2$ is nondecreasing for $x\ge 0$, the theorem predicts a nonnegative covariance. All moments used below are finite. Direct calculation gives

E[X]=\int_0^1 x\,dx=\frac12,\qquad E[g(X)]=E[X^2]=\int_0^1 x^2\,dx=\frac13,

E[Xg(X)]=E[X^3]=\int_0^1 x^3\,dx=\frac14.

Therefore,

\operatorname{Cov}(X,X^2) =E[X^3]-E[X]E[X^2] =\frac14-\frac12\cdot\frac13 =\frac14-\frac16 =\frac1{12}>0.

The restriction to nonnegative values is important here: $x^2$ is not nondecreasing on the whole real line. The theorem applies because the support of this $X$ lies in $[0,1]$, where the required monotonicity holds.

Worked Example: A Poisson Variable and Its Square

Let $X\sim\operatorname{Poisson}(\lambda)$, where $\lambda>0$, and take $g(x)=x^2$. The support is the set of nonnegative integers, so $g$ is nondecreasing on the support. A Poisson variable has finite moments of every positive integer order, so the covariance is defined.

Using the factorial moments $E[X]=\lambda$, $E[X(X-1)]=\lambda^2$, and $E[X(X-1)(X-2)]=\lambda^3$, first write

X^2=X(X-1)+X,\qquad E[X^2]=\lambda^2+\lambda.

The identity $X^3=X(X-1)(X-2)+3X(X-1)+X$ gives

E[X^3]=\lambda^3+3\lambda^2+\lambda.

It follows that

\begin{aligned} \operatorname{Cov}(X,X^2) &=E[X^3]-E[X]E[X^2]\\ &=(\lambda^3+3\lambda^2+\lambda)-\lambda(\lambda^2+\lambda)\\ &=2\lambda^2+\lambda>0. \end{aligned}

This example illustrates that the theorem does not require a continuous distribution. Its argument applies equally to discrete random variables.

Worked Example: A Normal Variable and an Increasing Exponential

Let $X\sim N(\mu,\sigma^2)$, with $\sigma^2>0$, and let $g(x)=e^{tx}$ for a fixed $t>0$. This function is strictly increasing. Normal exponential moments are finite, including $E|Xe^{tX}|$, so the covariance exists.

Completing the square in the normal density gives

E[e^{tX}]=e^{\mu t+\sigma^2t^2/2}.

In the integral for $E[Xe^{tX}]$, the product of the normal density and $e^{tx}$ can be written as $e^{\mu t+\sigma^2t^2/2}$ times the density of $N(\mu+\sigma^2t,\sigma^2)$. The integral of $x$ against that shifted normal density is its mean. Hence

E[Xe^{tX}]=(\mu+\sigma^2t)e^{\mu t+\sigma^2t^2/2}.

Substitution into the covariance definition yields

\begin{aligned} \operatorname{Cov}(X,e^{tX}) &=E[Xe^{tX}]-E[X]E[e^{tX}]\\ &=(\mu+\sigma^2t)e^{\mu t+\sigma^2t^2/2} -\mu e^{\mu t+\sigma^2t^2/2}\\ &=\sigma^2t\,e^{\mu t+\sigma^2t^2/2}>0. \end{aligned}

The result agrees with the sign theorem, and the calculation shows how the distribution determines the covariance's magnitude.

Worked Example: A Decreasing Function Reverses the Sign

Let $X\sim\operatorname{Exponential}(1)$ and $g(x)=e^{-x}$ for $x\ge 0$. The function is decreasing, so the covariance should be nonpositive. The density of $X$ is $e^{-x}$ for $x\ge 0$, and direct integration gives

E[X]=1,\qquad E[e^{-X}]=\int_0^\infty e^{-2x}\,dx=\frac12,

E[Xe^{-X}]=\int_0^\infty xe^{-2x}\,dx=\frac14.

The integral in the last line is $1/4$, since integration by parts gives $\int_0^\infty xe^{-2x}\,dx=[-xe^{-2x}/2]_0^\infty+\frac12\int_0^\infty e^{-2x}\,dx=1/4$. Therefore,

\operatorname{Cov}(X,e^{-X}) =\frac14-(1)\left(\frac12\right) =-\frac14.

The negative value matches the nonincreasing case of the theorem.

Strictness, Scope, and a Common Mistake

If $g$ is strictly increasing on an interval containing $X$ almost surely, then the product $(X-X')(g(X)-g(X'))$ is strictly positive whenever $X\ne X'$, and is zero when $X=X'$. Thus, if $X$ is not almost surely constant, then $P(X\ne X')>0$, and the covariance is strictly positive. To see why a nonnegative random variable that is positive with positive probability has positive expectation, note that its positive event is the union of the events on which it is at least $1/k$, for positive integers $k$; at least one of those events must have positive probability. The same reasoning gives a strictly negative covariance for a strictly decreasing $g$ when $X$ is nonconstant.

For a merely nondecreasing function, the covariance can be zero even when $X$ varies. For example, if $X\sim\operatorname{Uniform}(0,1)$ and $g(x)=0$ for $x\in[0,1]$, then $\operatorname{Cov}(X,g(X))=0$. A flat function on the values taken by $X$ does not create variation to covary with $X$.

Common mistake: “$g$ is increasing, so $\operatorname{Cov}(X,g(X))$ is positive for every random variable.” This argument can fail if $g$ is increasing only over part of the real line, while $X$ can take values outside that part. For instance, $g(x)=x^2$ decreases on negative values and increases on positive values, so it is not monotone on an interval containing the support of a normal variable. The correction is to check monotonicity over the values $X$ can actually take, and to verify that the expectations defining covariance are finite. Even when these conditions hold, nondecreasing gives a nonnegative sign; strict positivity needs additional conditions, such as strict increase and a nonconstant $X$.

Key takeaway: Under suitable integrability, the sign of $\operatorname{Cov}(X,g(X))$ follows from whether $g$ is nondecreasing or nonincreasing on the range of $X$. The independent-copy identity makes the reason explicit: the differences $X-X'$ and $g(X)-g(X')$ have matching signs for a nondecreasing function and opposite signs for a nonincreasing one.

Check Your Understanding

Use the independent-copy identity and check the conditions in each case.

State sufficient moment conditions for the independent-copy identity for $\operatorname{Cov}(X,g(X))$.
If $g$ is nonincreasing on an interval containing $X$ almost surely, what sign does the theorem give for the covariance? Explain using the two differences.
For $X\sim\operatorname{Uniform}(0,1)$, is $g(x)=3x+2$ nondecreasing on the support? What sign should $\operatorname{Cov}(X,g(X))$ have?
Why does strict increase of $g$ not imply a strictly positive covariance when $X$ is almost surely constant?
Why is the monotonicity of $x^2$ on the real line insufficient to apply the theorem directly to a normal $X$?