Unizor - Probability - Normal Distribution - Normal is the Limit

Let's start with formulating again the Central Limit Theorem of the theory of probabilities that illustrates the importance of the Normal distribution of probabilities.

In non-rigorous terms, the Central Limit Theorem states that, given certain conditions, the average of a large number of random variables behaves approximately like a normal random variable.

One of the simplest sufficient conditions, for example, is the requirement about random variables participating in averaging to be independent, identically distributed with finite expectation and variance.
Throughout the history of development of the theory of probabilities the Central Limit Theorem was proven for weaker and weaker conditions. Even if individual variables are not completely independent, even if their probability distributions are not the same, the theorem can still be proven. Arguably, the history of the development of the theory of probabilities is the history of proving the Central Limit Theorem under weaker and weaker conditions.

Rigorous proof of this theorem, even in the simplest case of averaging independent identically distributed random variables, is outside of the scope of this course. However, the illustrative examples are always possible.

Consider a sequence of N independent Bernoulli experiments with a probability of success equal to 1/2. Their results are the random variables ξi, each taking two values, 0 and 1, with equal probability of 1/2 (index i is in the range from 1 to N).

Now consider their sum
η = ξ1+ξ2+...+ξN

From the lecture about Bernoulli distribution we know that random variable η can take any value from 0 to N with the probability to take a value K (K is in the range from 0 to N) equal to
P(η=K) = CNK·pK·qN−K
where p is the probability of SUCCESS and q=1−p is the probability of FAILURE.

Let's graph the distribution of probabilities of our random variable η for different values of N with probabilities p = q = 1/2.

N=1: η = ξ1
The graph of the distribution of probabilities of η is
zero to the left of x=0,
1/2 on [0,1],
1/2 on [1,2] and
zero after x=2.

N=2: η = ξ1+ξ2
The graph of the distribution of probabilities of η is
zero to the left of x=0,
1/4 on [0,1],
1/2 on [1,2],
1/4 on [2,3] and
zero after x=3.

N=3: η = ξ1+ξ2+ξ3
The graph of the distribution of probabilities of η is
zero to the left of x=0,
1/8 on [0,1],
3/8 on [1,2],
3/8 on [2,3],
1/8 on [3,4] and
zero after x=4.

N=4: η = ξ1+ξ2+ξ3+ξ4
The graph of the distribution of probabilities of η is
zero to the left of x=0,
1/16 on [0,1],
4/16 on [1,2],
6/16 on [2,3],
4/16 on [3,4],
1/16 on [4,5] and
zero after x=5.

N=5: η = ξ1+ξ2+ξ3+ξ4+ξ5
The graph of the distribution of probabilities of η is
zero to the left of x=0,
1/32 on [0,1],
5/32 on [1,2],
10/32 on [2,3],
10/32 on [3,4],
5/32 on [4,5],
1/32 on [5,6] and
zero after x=6.

N=6: η = ξ1+ξ2+ξ3+ξ4+ξ5+ξ6
The graph of the distribution of probabilities of η is
zero to the left of x=0,
1/64 on [0,1],
6/64 on [1,2],
15/64 on [2,3],
20/64 on [3,4],
15/64 on [4,5],
6/64 on [5,6],
1/64 on [6,7] and
zero after x=7.

The above graphs obviously more and more resemble the bell curve. When squeezed by a factor of N (to transform a sum of N random variables into their average), all the graphs will be greater than zero only in a segment [0,1] and inside this segment, as N grows, the graphs will be closer and closer to a bell curve.