Unizor - Statistics - Averages

Unizor - Creative Minds through Art of Mathematics - Math4Teens


Statistical Averages

This lecture is, actually, a refresher of things studied in the course of Theory of Probabilities, but applied to Statistics.

As we noted, the purpose of Mathematical Statistics is, using the past observations, to evaluate the probability of certain events. More precisely, using the results of random experiments, we need to know the distribution of probabilities of a random variable.

In many cases we can assume that the random variable we deal with is distributed Normally, and the Normal distribution, as we know is fully defined by its mathematical expectation and variance. In other, not Normal, cases of distribution these two parameters also play a very important role, representing a concentration point of the value of a random variable and its spread around this concentration point. So, it's not surprising that evaluation of these parameters is a very important statistical task.

The typical approach to evaluation of mathematical expectation is the calculation of arithmetic average of the sample data. Let's examine a validity of this approach in case we conduct certain number of experiments with our random variable, providing the experiments are independent of each other and the conditions of experiments are not changing.

To make these two requirements less abstract, let's exemplify cases when these conditions are not present.
For instance, you roll the dice. If, for the next roll, you position the die exactly on the number you got on a previous roll, you establish a connection between the rolls and experiments are not independent.
As far as conditions of experiment changing, we can examine the bow and arrow target shooting at an outside range with wind affecting the results. Obviously, conditions are changing unpredictably with the wind and the results will not exactly correspond to the level of skills of participants.

So, we assume independence of the random experiments and stability of conditions.

Assume we conduct N experiments with the same random variable ξ and obtain N values Xi (i = 1, 2,...N).
How can we interpret the arithmetic average of these data
m = (X1+X2+...+XN)/N ?

The simplest approach is to consider a set of independent random variables ξ1, ξ2,...ξN distributed identically with ξ. Now assume a new random variable
η = (ξ1+ξ2+...+ξN)/N
Obviously, our average m of the N results of experiments with random variable ξ can be considered as a single result of an experiment with a random variable η.
It is an extremely important point that this approach is the base for all subsequent logical statements, because there is no better approximation of the random variable with unknown distribution of probabilities than its real experimental value.

But why a single result of an experiment with η is better than N results of experiments with ξ? The reason is simple - random variable η has the same mean value as ξ, but much smaller variance, that is its values have a much smaller spread around it's mean value and, therefore, are a better approximation of this mean value.
The proof of this is based on the known properties of mathematical expectation and variance of a sum of random variables discussed in the lectures Probability - Random Variables - Expectation of Sum and Probability - Random Variables - Variance of Sum. 
In particular, expectation of a sum of random variables equals to sum of their expectations and variance of a sum of independent random variables equals to sum of their variances.

In our case, if expectation of random variable ξ is μ = E(ξ), expectation of random variable η is
E(η)=E[(ξ1+ξ2+...+ξN)/N]=
=[E(ξ1)+E(ξ2)+...+E(ξN)]/N=
(since all ξi are independent and identically distributed with ξ)
= N·E(ξ)/N = E(ξ) = μ
So, expectations of ξ and η are the same.

If variance of ξ is σ² = Var(ξ), variance of η is
Var(η)=Var[(ξ1+...+ξN)/N]=
=Var(ξ1+...+ξN)/N²=
(since all ξi are independent and identically distributed with ξ)
= N·Var(ξ)/N² = 
=Var(ξ)/N = σ²/N
So, variance of η is N times smaller than variance of ξ.

So, the distribution of random variable η is more tightly concentrated around its mean value than the distribution of ξ around exactly the same mean value. This is the main reason why average of N observations of values of ξ is a good approximation for the mean value of ξ. And the quality of this approximation increases with N→∞.

In addition, it makes sense to add another important detail. With the number of experiments N growing, the distribution of random variable η is more and more resembling the Normal distribution (see Probability - Normal Distribution - Normal is a Limit in this course).
That allows us, knowing sample mean and sample variation, to evaluate the range of the value of μ=E(ξ) with some certainty.