Unizor - Random Variables - Problem 7

Notes to a video lecture on http://www.unizor.com


Random Variables
Problems 7 (Covariance)

As always, try to solve any problems presented on this Web site just by yourself and check against the answers provided.
Only then study the suggested solutions.

Problem

Consider two random variables,ξ and η, not necessarily independent, each taking no more than two different values and having known mutual distribution of probabilities.

Prove that, if covariance between them is zero, then either they are independent random variables or one of them is a constant and can take only one possible value with probability 1.

Solution:

Assume values of ξ are a, b and values of η are c, d.
Let's organize their mutual probabilities into a table:

	η=c	η=d
ξ=a	p	q
ξ=b	u	v

So, for instance,
P{ξ=a AND η=c} = p
It's important to notice that
p+q+u+v=1

Having zero covariance implies the following equation:
Cov(ξ,η) = E(ξ·η)−E(ξ)·E(η) = 0
Or, equivalently,
E(ξ·η)=E(ξ)·E(η)

Let's calculate each component separately.
E(ξ·η) = (acp+adq+bcu+bdv)
E(ξ) = a(p+q)+b(u+v)
E(η) = c(p+u)+d(q+v)

So, we have an equation:
acp+adq+bcu+bdv = [a(p+q)+b(u+v)] · [c(p+u)+d(q+v)]

At this time we suggest to multiply the left side by a multiplier p+q+u+v that is equal to 1 and do the tedious job of opening all the parenthesis, canceling and grouping.

Then, after many transformations we will come up to an equation
(a−b)(c−d)(pv−qu)=0

There are three case when this equation is satisfied.

Case 1 (trivial): a = b
This means, ξ is a constant.

Case 2 (trivial): c = d
This means, η is a constant.

Case 3: pv = qu
This is a more interesting case that will lead us toindependence between ξ and η.
To prove independence, we have to prove that all conditional probabilities for our random variables to take different values equal to corresponding unconditional ones or, equivalently, that the probability of them to take simultaneously some of their values equals to a product of corresponding unconditional probabilities.
Let's prove the latter for ξtaking value of a and η taking value c.
Here is how.
Let's take an obvious equation
1 = p+q+u+v
and multiply it by p:
p = p²+pq+pu+pv
Since we consider a case whenpv = qu, replace pv with qu in the last equation:
p = p²+pq+pu+qu = (p+q)(p+u)

Notice that
p = P(ξ=a AND η=c)
p+q = P(ξ=a)
p+u = P(η=c)

Therefore, we have proven that
P(ξ=a AND η=c) = P(ξ=a) · P(η=c)

Analogously, we can prove all equations needed to proveindependence between ξ and η.

End of proof.