Unizor - Probability - Expectation of Product

Notes to a video lecture on http://www.unizor.com


Independent Random Variables
Expectation of Product

Our goal in this lecture is to prove that expectation of a product of two independent random variables equals to a product of their expectations.

First of all, intuitively, this fact should be obvious, at least, in some cases.
When an expectation of arandom variable is a value, around which results of random experiments are concentrated (like a temperature of a healthy person), product of results of two different experiments (product of temperatures of two different healthy persons) tend to concentrate around product of their expectations.
In some other cases, when such a concentration does not take place (like flipping a coin), that same rule of multiplicative property of an expectation is still observed.

A very important detail, however, differentiates property of a sum of two random variables from their product. The expectation of a sum always equals to a sum of expectations of its component. With a product the analogous property is true only in case the components areINDEPENDENT random variables.

Let's approach this problem more formally and prove this theorem.

Consider the following tworandom experiments (sample spaces) and random variablesdefined on their elementary events.

Ω₁=(e₁,e₂,...,e_M )
with corresponding measure of probabilities of theseelementary events
P=(p₁,p₂,...,p_M )
(that is, P(e_i )=p_i - non-negative numbers with their sum equaled to 1)
and random variable ξ defined for each elementary event as
ξ(e_i) = x_i where i=1,2,...M

Ω₂=(f₁,f₂,...,f_N )
with corresponding measure of probabilities of theseelementary events
Q=(q₁,q₂,...,q_N )
(that is, Q(f_j )=q_j - non-negative numbers with their sum equaled to 1)
and random variable η defined for each elementary event as
η(f_j) = y_j where j=1,2,...N

Separately, the expectations of these random variables are:
E(ξ) = x₁·p₁+x₂·p₂+...+x_M·p_M
E(η) = y₁·q₁+y₂·q₂+...+y_N·q_N

To calculate the expectation of a product of these random variables, let's research what values and with whatprobabilities this product can take.
Since every value of ξ can be observed with every value of η, we can conclude that all the values of their product are described by all values x_i·y_jwhere index i runs from 1 to Mand index j runs from 1 to N.

Let's examine the probabilistic meaning of a product of tworandom variables defined on two different sample spaces.
Any particular value x_i·y_j is taken by a new random variableζ=ξ·η defined on a new combined sample spaceΩ=Ω₁×Ω₂ that consists of all pairs of elementary events(e_i , f_j ) with the corresponding combined measure ofprobabilities of these pairs equal to
R(e_i , f_j ) = r_ij
where index i runs from 1 to Mand index j runs from 1 to N.

Thus, we have defined a newrandom variable ζ=ξ·η defined on a new sample space Ω ofM·N pairs of elementary eventsfrom two old spaces Ω₁ and Ω₂as follows
ζ(e_i , f_j ) = x_i·y_j
with probability r_{i j}

Before going any further, let's examine very important properties of probabilities r_ij.
We have defined r_ij as aprobability of a random experiment described by asample space Ω₁ resulting inelementary event e_i and, simultaneously, a random experiment described by asample space Ω₂ resulting inelementary event f_j.
Incidentally, if events from these two sample spaces areindependent,
r_ij = p_i·q_j
because, for independentevents, probability of their simultaneous occurrence equals to a product of probabilities of their separate individual occurrences.

Keeping in mind the above properties of probabilities r_ij, we can calculate the expectationof our new random variable ζ.
E(ζ) = E(ξ·η) =
= (x₁·y₁)·r₁₁+...+(x₁·y_N)·r_1N +
+ (x₂·y₁)·r₂₁+...+(x₂·y_N)·r_2N +
...
+ (x_M·y₁)·r_M1+...+(x_M·y_N)·r_MN

On the other hand, let's calculate the product of expectations of our random variable ξ and η:

E(ξ)·E(η) =
=(x₁·p₁+...+x_M·p_M)·
·(y₁·q₁+...+y_N·q_N) =
= (x₁·y₁)·p₁q₁+...+(x₁·y_N)·p₁q_N +
+ (x₂·y₁)·p₂q₁+...+(x₂·y_N)·p₂q_N +
...
+ (x_M·y₁)·p_Mq₁+...+(x_M·y_N)·p_Mq_N

Obviously, if random variablesξ and η are INDEPENDENT, probability r_ij of ξ to take valuex_i and, simultaneously, η to take value y_j equals to a product of corresponding probabilitiesp_i·q_j. In this case expressions for E(ξ·η) and E(ξ)·E(η) are identical.

That proves that forINDEPENDENT random variables mathematical expectation of their product equals to a product of their mathematical expectations.
End of proof.