Unizor - Probability - Random Variables - Problems 3

Problem 3.1.
Given a random variable. Then a new random variable is formed as a product of a constant by this random variable.
Prove that an expectation (mean) of a product of a constant by a random variable equals to a product of that constant by an expectation of that random variable:
E(a·ξ) = a·E(ξ)

Proof
Recall that an expectation of a random variable is a weighted average of its values with corresponding probabilities as weights.
Assume that random variable ξ takes values x1, x2,...xN with probabilities, correspondingly, p1, p2,...pN.
Then
E(ξ) = x1·p1 + x2·p2 +...+ xN·pN
Random variable a·ξ takes values a·x1, a·x2,...a·xN with probabilities, correspondingly, p1, p2,...pN.
Then
E(a·ξ) = a·x1·p1+a·x2·p2+...+a·xN·pN =
= a·(x1·p1 + x2·p2 +...+ xN·pN) = a·E(ξ)

Problem 3.2.
Given a random variable. Then a new random variable is formed as a product of a constant by this random variable.
Prove that a variance of a product of a constant by a random variable equals to a product of a square of that constant by a variance of that random variable:
Var(a·ξ) = a^2·Var(ξ)

Proof
Recall that a variance of a random variable is an expectation of a square of deviation of this random variable from its expected value (mean). As a formula, it looks like this:
Var(ξ) = E{[(ξ−E(ξ)]^2}
Assume that random variable ξ takes values x1, x2,...xN with probabilities, correspondingly, p1, p2,...pN.
Let the expectation (mean) of this random variable be E(ξ)=μ.
Then
Var(ξ) = (x1−μ)^2·p1 + (x2−μ)^2·p2 +...+ (xN−μ)^2·pN
Random variable a·ξ takes values a·x1, a·x2,...a·xN with probabilities, correspondingly, p1, p2,...pN.
As we know from a previous problem,
E(a·ξ) = a·E(ξ) = a·μ
Then
Var(a·ξ) = (a·x1−a·μ)^2·p1 + (a·x2−a·μ)^2·p2+...+ (a·xN−a·μ)^2·pN
= a^2·(x1−μ)^2·p1 + a^2·(x2−μ)^2·p2 +...+ a^2·(xN−μ)^2·pN =
= a^2·Var(ξ)

Problem 3.3.
Given a random variable. Then a new random variable is formed as a product of a constant by this random variable.
Prove that a standard deviation of a product of a constant by a random variable equals to a product of an absolute value of that constant by a standard deviation of that random variable:
σ(a·ξ) = |a|·σ(ξ)

Proof
Recall that a standard deviation of a random variable is a (non-negative) square root of its variance.
Therefore,
σ(a·ξ) = √Var(a·ξ) = √a2·Var(ξ) = |a|·√Var(ξ) = |a|·σ(ξ)
Notice the absolute value around a constant a. Even if this constant is negative, a standard deviation is always non-negative, as is a variance of a random variable.

Problem 3.4.
Given N independent identically distributed random variables ξ1, ξ2,...ξN, each having the expectation (mean) μ and the standard deviation σ.
Calculate the expectation, variance and standard deviation of their average
η = (ξ1 + ξ2 +...+ ξN)/N

Answer
E(η) = μ
Var(η) = σ2/N
StdDev(η) = σ/√N

Hint
Use the additive property of an expectation relative to a sum of any random variables and an additive property of a variance relative to a sum of independent random variables.

IMPORTANT CONSEQUENCE

Averaging of independent identically distributed random variables produces a new random variable with the same mean (expectation) as the mean of original random variables, but the standard deviation of that average from its mean is decreasing by a factor of √N as the number N of random variables participating in the averaging in increasing.

Therefore, the process of averaging produces more and more precise estimate of the expectation of a random variable as the number of random variable grows. This is a basis of most statistical calculations.

For example, if one person measures a length of some object, there is always an error of his measuring. His result is a true length plus some error introduced by his measurement (positive or negative). So, we can consider his measure as a random variable with certain mean - a true length of an object and a random error with some standard deviation that depends on his measurement tool and accuracy. Thus, if measure with a ruler the length of a table with a true length of 1 meter, we can say that the error might be no greater then 1 centimeter, and our results will be from 99 to 101 centimeters.

But if this person measures the length of this object 100 times and the results of these measurements are averaged, the final result will be much closer to the real value of the length of the object since the mean of this average is the same as the mean of every measurement, that is the true length of an object, but the standard deviation around this true length will be √100=10 times smaller than that of each measurement. That assures the precision of our evaluation of the length of the table to be within 1 millimeter from the real length.