Unizor - Probability - Random Variables - Introduction

Historically, the theory of probabilities was developed from attempts to study the games and analyze the chances of winning. The concept of random variables has the same roots. Not only people tried to evaluate the chances of winning, they also bet money on these games and wanted to understand how much to bet on this or that game. So, they were dealing with some numerical quantity associated with chances of winning or losing a game. This numerical quantity associated with the results (elementary events) of the game (random experiment) is an example of a random variable. The elementary event can be considered as a qualitative result of an experiment, while the random variable describes its quantitative result.

Consider an example of betting an amount of $1 in a game of flipping a coin you play against a partner. Let's assume you are betting on "heads", then a coin is flipped. If you guessed correctly and a coin indeed falls on "heads", your partner pays you $1. If you guess incorrectly, you give $1 to him. This seems to be a fair game with equal chances to win and lose. Let's associate a numerical value - positive amount of winning or negative amount of losing - with each elementary event occurring in this game, that is introduce a function with numerical values defined on a set of elementary events. The value of this function for an elementary event "heads" equals to 1 and the value for an elementary event "tails" equals to −1. This is an example of a random variable.

Basically, any function defined on a set of elementary events that takes real values depending on results of certain random experiment is a random variable. Let's consider some examples.

Consider a game of roulette. A dealer spins a small ball on a wheel divided into partitions with 36 numbers from 1 to 36 and two additional partitions with 0 and 00 (American version). You can bet on any number from 1 to 36 (among other options which we do not consider for this example). Assume, you bet $1 on number 23. If the ball stops on this number, you win and a dealer pays you $36. If the ball stops on any other number, including 0 or 00, you lose you bet of $1. The random variable thus is defined as having a value of 36 on the elementary event "Ball stops in a partition with number 23 in it" and having a value of −1 for all other elementary events.

Since in this course of theory of probabilities we consider only random experiments with finite number of elementary events, all our random variables are defined on the finite set of elements and take finite number of values. The values are always real numbers. The elementary events might or might not have equal chances to occur. Therefore, if there are N elementary events that our random experiment results in, there are N values (not necessarily different) of the random variable defined on this set of elementary events.

Assume, our sample space Ω consists of N elementary events E1, E2,..., EN. Generally speaking, they might not necessarily have equal chances to occur, so let's assume that the probability of occurrence of elementary event Ei is Pi (index i changes from 1 to N), where all these probabilities are not negative and, if added together, sum up to 1.
Random variable is a function with real values defined on each of the elementary events.
Let's use a Greek letter ξ to denote this random variable. Values taken by a random variable ξ depend on the results of the random experiment.
If the experiment results in E1, which can happen with a probability P1, the random variable ξ takes the value of X1.
If the experiment results in E2, which can happen with a probability P2, the random variable ξ takes the value of X2.
etc.
X1=ξ(E1),
X2=ξ(E2),
...
XN=ξ(EN).

So, we have a random variable ξ that takes different values Xi with different probabilities Pi, depending on the results of a random experiment. To analyze this random variable, we don't really need to think much about concrete elementary events (arguments of a function that we call the random variable) and concentrate only on probabilities of the values that our random variable takes. Thus, we can say that in the game of roulette with 38 partitions on a wheel (numbers from 1 to 36, 0 and 00), betting on number 23, we deal with a random variable that takes a value of 36 with the probability 1/38 (winning by correctly guessing the number where the ball stops spinning) and a values of −1 with the probability 37/38 (losing).

Unizor - Probability - Conditional - Problems 3

We recommend to attempt solving these problems prior to listening to the lecture or reading answers and proofs provided.
Also assume that all probabilities mentioned in the problems are not equal to zero, that is we are excluding impossible events.
Wherever possible, try to represent the problem graphically with a set of elementary events.
The usage of a word "random" assumes equal chances to occurrences of elementary events unless otherwise specified.

Problem 3.1.
We roll two dice. What is the probability that the sum of two rolled numbers equals to 7 if it's known that the first dice shows a number greater than 3?
Answer: 1/6

Problem 3.2.
We roll two dice. What is the probability that the number on the second dice is greater than 3 if it's known that the numbers on both dice are different?
Answer: 1/2

Problem 3.3.
A student is preparing to an exam. Exam includes 100 questions. A student had time to prepare only for the first 70 questions. During the exam students are randomly divided into 2 groups and original 100 questions were also divided among these groups: group 1 got questions from #1 to #50, group 2 got questions from #51 to #100.
Assume that our student can answer correctly any question he had prepared for (those from #1 to #70), but the one he had not prepared for he answers incorrectly.
He randomly picks a question from those given to his group.
(a) What is the probability of answering it correctly if it's known that he was selected into group 1?
(b) What is this probability on condition that he is selected into group 2?
(c) What is the total (unconditional) probability of answering the question correctly?
Answer: 0.7

Unizor - Probability - Conditional - Problems 2

We recommend to attempt solving these problems prior to listening to the lecture or reading answers and proofs provided.
Also assume that all probabilities mentioned in the problems are not equal to zero, that is we are excluding impossible events.

Problem 2.1.
There are 3 white and 2 black socks in a box. You randomly pulled one sock and it happened to be white.
What is the probability of randomly pulling a second white sock?
Solve the problem in more than one way, try to use the concepts of conditional probability

Answer
1/2

Problem 2.2.
There is a computer game that has 2 levels. Experience shows that the probability of passing the level 1 during the first day equals to 50%=1/2. The probability of mastering both levels in the first day equals to 10%=1/10.
What is the probability of passing the level 2 during the first day for children who have managed to pass the level 1 on this day?

Answer
1/5

Problem 2.3.
There are three categories of people living in some place: 20% Jewish, 30% Muslims and 50% atheists.
The food restrictions among Jews and Muslims are similar, but not identical. Also, even among people of the same religion there are differences in interpretation of the laws.
Assume that 90% of Jewish people consider some food X as prohibited, while only 80% of Muslims agree with them. Atheists do not have any restrictions on food.
You invite a random person from a street for dinner. What is the probability that he would not eat the food X because he considers it prohibited?

Answer
0.42

Unizor - Probability - Conditional - Problems 1

We recommend to attempt to solve these problems prior to listening to the lecture or reading answers and proofs provided.
Also assume that all probabilities mentioned in the problems are not equal to zero, that is we are excluding impossible events.

Problem 1.1.
Let A and B be two independent events defined in the sample space Ω.
Prove that the product of the probability of both events to occur by the probability of neither event to occur equals to a product of the probability of occurrence of only event A (but not B) by the probability of occurrence of only event B
(but not A).
In symbolic form using the set theory operations, prove that
P(A∩B)·P[Ω∖ (A∪B)] =
P(A∖ B)·P(B∖ A)
or, equivalently,
P(A∩B)·P[NOT (A∪B)] =
P[A∩(NOT B)]·P[B∩(NOT A)]

Problem 1.2.
Prove "geometrically" (see above) and probabilistically that if event A is independent of event B then event NOT A is also independent of event B.

Problem 1.3.
Prove "geometrically" and probabilistically that if event A is independent of event B then event A is also independent of event NOT B.

Problem 1.4.
Prove "geometrically" and probabilistically the following identity
P(A|B) + P[(NOT A)|B)] = 1

Problem 1.5.
Is the following equality always true (in other words, is it an identity)?
P(A|B) + P[A|(NOT B)] = 1

Unizor - Probability - Definition - Problems 1

There are 8 people in the room:
A - male, 19 years of old;
B - male, 18 years of old;
C - female, 18 years of old;
D - male, 20 years of old;
E - male, 24 years of old;
F - female, 28 years of old;
G - male, 33 years of old;
H - female, 16 years of old.
Our random experiment consists of selecting one of them.

(a) What are the set and its elements representing the result of this random experiment and its elementary events? What is the measure of probability of an entire sample space?
Answer:
{A,B,C,D,E,F,G,H}; Pa=1

(b) What is the measure of probability allocated to each elementary event?
Answer:
Pb=1/8

(c) What is the subset that represents the event "Randomly selected person is a female"? What is its measure of probability?
Answer:
{C,F,H}; Pc=3/8

(d) What is the subset that represents the event "Randomly selected person is a male"? What is its measure of probability?
Answer:
{A,B,D,E,G}; Pd=5/8

(e) What is the subset that represents the event "Randomly selected person is older than 20 years"? What is its measure of probability?
Answer:
{E,F,G}; Pe=3/8

(f) What is the subset that represents the event "Randomly selected person is a female AND older than 20 year"? What is its measure of probability?
Answer:
{C,F,H}∩{E,F,G}={F}; Pf=1/8

(g) What is the subset that represents the event "Randomly selected person is a person"? What is its measure of probability?
Answer:
{A,B,C,D,E,F,G,H}; Pg=1

(h) What is the subset that represents the event "Randomly selected person is younger then 10 year old"? What is its measure of probability?
Answer:
{}=∅; Ph=0

(i) What is the subset that represents the event "Randomly selected person is NOT younger then 10 year old"? What is its measure of probability?
Answer:
{A,B,C,D,E,F,G,H}; Pi=1

(j) What is the subset that represents the event "Randomly selected person is NOT younger then 25 year old AND is NOT a male"? What is its measure of probability?
Answer:
{F,G}∩{C,F,H}={F}; Pj=1/8

(k) What is the subset that represents the event "Randomly selected person is NOT younger then 25 year old OR is a male"? What is its measure of probability?
Answer:
{F,G}∪{A,B,D,E,G}=
={A,B,D,E,F,G}; Pk=6/8=3/4

(l) What is the subset that represents the event "Randomly selected person is NOT a male OR NOT older than 20 years old"? What is its measure of probability?
Answer:
{C,F,H}∪{A,B,C,D,H}=
={A,B,C,D,F,H}; Pl=6/8=3/4

(m) What is the subset that represents the event "Randomly selected person is NOT a male AND NOT older than 20 years old"? What is its measure of probability?
Answer:
{C,F,H}∩{A,B,C,D,H}=
={C,H}; Pm=2/8=1/4

Unizor - Probability - Conditional - Bayes Theorem

Let's start with the Bayes Theorem itself and its proof, and then discuss its applications. The theorem is very simple, but its applications are very far reaching.

Bayes Theorem
There are two events that are the results of a random experiment - A and B.
P(A) is the probability of event A to occur.
P(B) is the probability of event B to occur.
P(A|B) is the conditional probability of event A to occur if event B has already occurred.
P(B|A) is the conditional probability of event B to occur if event A has already occurred.
Then the following equality is true
P(A|B) = P(B|A) · P(A) / P(B)

Proof
By definition of conditional probability,
P(A|B) = P(A∩B) / P(B) and
P(B|A) = P(A∩B) / P(A).
Let's resolve the second equation for P(A∩B):
P(A∩B) = P(B|A) · P(A)
Now substitute this expression for P(A∩B) into a formula for P(A|B) above.
P(A|B) = P(B|A) · P(A) / P(B)
End of proof.

Consider a case when an entire sample space of elementary events is divided among two subsets with no common elements:
X and Y. Then any event W can be represented as a union of two non-intersecting parts W = (W·X)+(W·Y) and the measure of an event W is equal to a sum of measures of its two non-intersecting parts:
P(W) = P(W·X)+P(W·Y)
Using the formula of conditional probability P(A|B)=P(A·B)/P(B), we can rewrite the equation for P(W) above as
P(W) =
= P(X)·P(W|X) + P(Y)·P(W|Y)
This is called a formula of total probability. It is important here that events X and Y are complementary, that is
X·Y=∅ and X+Y=Ω.

Now we can use the Bayes formula to determine a conditional probability P(X|W) and P(Y|W):
P(X|W) = P(X·W)/P(W) = P(X)·P(W|X)/P(W) and
P(Y|W) = P(Y·W)/P(W) = P(Y)·P(W|Y)/P(W)

The formulas above show how, knowing probabilities of occurrence of certain conditions X and Y and the conditional probabilities of certain event W under these conditions, we can calculate its total probability P(W) and then determine which condition X and Y actually occurred with what conditional probability P(X|W) and P(Y|W) under a conditi the on that event W did occur.

Unizor - Probability - Independent Events

It's natural to consider a concept of independent events as related to a concept of conditional probability. After all, "independence" of a random experiment from certain condition means that, no matter what that condition is, the outcomes from our experiment are the same and their chances to occur are also the same as if this condition never was imposed.

Let's call a random event A independent of a random event B if and only if the probability of A is the same as the conditional probability of A under condition of occurrence of an event B:
P(A) = P(A|B).

Mini Theorem 1
Independence is a symmetrical property of the events.
From independence of one event from another follows their mutual independence.
More precisely, if a random event A is independent of a random event B then a random event B is independent of a random event A.
Proof
By definition of conditional probability,
P(B|A) = P(A∩B) / P(A) and P(A|B) = P(A∩B) / P(B).
Let's resolve the second equation for P(A∩B) and substitute it into the first.
P(A∩B) = P(B) · P(A|B)
P(B|A) = P(B) · P(A|B) / P(A)
Now we can use the independence of a random event A of a random event B, which means that P(A|B)=P(A) and the right side of the previous equation equals to
P(B) · P(A) / P(A) = P(B).
Therefore,
P(B|A)=P(B),
which means that a random event B is independent of a random event A.
End of proof.

Using the above theorem, we can always replace the words mutually independent events with just independent events.

Mini Theorem 2
If random events A and B are independent then
P(A∩B) = P(A) · P(B)
Proof
By definition of conditional probability,
P(B|A) = P(A∩B) / P(A).
Since our events are independent,
P(B|A) = P(B).
Therefore,
P(B) = P(A∩B) / P(A) and
P(A∩B) = P(A) · P(B).
End of proof.

Mini Theorem 3
This is a converse theorem to a previous one.
If P(A∩B) = P(A) · P(B)
then random events A and B are independent.
Proof
By definition of conditional probability,
P(A|B) = P(A∩B) / P(B).
Since the probability of intersection of these events equals to a product of their respective probabilities,
P(A|B) = P(A) · P(B) / P(B).
Therefore,
P(A|B) = P(A),
which is a definition of independence.
End of proof.

Based on the mini-theorems 1, 2 and 3 above, we can equivalently define events A and B as independent, if they satisfy the following rule:
Probability of their intersection is equal to a product of their respective probabilities, that is
P(A∩B) = P(A) · P(B)