Introduction

Terms Definition: The probability of an outcome is the proportion of times the outcome would occur if we observed the random process an infinite number of times (Law of large numbers) Frequency Interpretation: The frequency of the event occurring over many trials is equal to the expected probability Subjective Interpretation: The probability of the event occurring is how much a person is willing to bet on that event occurring when: Win a unit of currency if the event occurs Loss the stake if the event does not occur There is no “right” probability for the event, as it is up the individual person Notations Sample space, denoted S: The set of all possible outcomes of an experiment Example: Flip a coin, S{Head, Tail} Sets and subsets: Set is a group of objects (letters, numbers, ect), while subset is a set within a set. An empty set is denoted ∅ Event, denoted E: Event/events of interest The complement of E is referred to as Ec and contains all the outcomes in the sample space not covered by E Cardinality: Total number of possible outcomes in sample space, denoted |S| = n Mutually exclusive events (Additive Rule): P(E∪F), union, A or B Both E and F (Multiplicative Rule): P(E∩F), intersection, A intersect B (A and B)

Probability rules Rule 1: probability always between 0 and 1, denoted: 0 ≤P ≤1 Rule 2 (Additive): When events are mutually exclusive (they cannot occur at the same time), the probability is calculated by summing the probability of the different events, e.g. P(E1) + P(E2), etc. Deduction formula: P(E)+P(F)=P(E∪F)+P(E∩F), or equivalently P(E∪F)=P(E)+P(F)-P(E∩F). If E and F are mutually exclusive, then: P(E∪F)=P(E)+P(F)-P(Ø)=P(E)+P(F). The adaptive additive law considers the probability of E1 and E2 both occurring, e.g. what is the probability of throwing 3,4,6 or an even number with a die? P(3,4,5) = 1/2 P(even number) = 1/2 P(3,4,6 Or even number) = ½ + ½ = 1 (This is incorrect, since 4 is both in E1 and E2 meaning that the events are not mutually exclusive). Therefore, need to take into account the probability of E1 and E2 occurring at the same time by subtracting the probability of that: P(3,4,5 And even) = P(4) = 1/6 The adaptive Additive law: P(E1 or E2)=P(E1)+P(E2)-P(E1 and E2) In above example: P(3,4,5 Or even)=1/2+1/2-1/6=5/6 Rule 3 (Multiplicative): When two events both occur (e.g. are not mutually exclusive), the probability is calculated as the product of the two events, P(E1) * P(E2) – it also means that one event is conditional on the other, e.g. one event occurring conditional on the other P(E1 and E2) = P(E1) * P (E2 | E1), where P(E2 | E1) is the probability of E2 occurring given that E1 has occurred. Conditional probability An event can change probability based on some conditions. By rearranging the multiplicative law, we get a formulate for the conditional probability of an event occurring P(E1 and E2)=P(E1)*P(E2 ┤|E1) into P(E2│E1)=P(E1 ∩ E2)/(P(E1))

The numerator is the is the probability that E1 and E2 occur. When we know that E1 has occurred, we reduce the sample space to E1 (hence the denominator), meaning that if E2 occurs, it must be in the intersection between E1 and E2. An example of a conditional probability: The probability of drawing an ace of hearts in a card deck is 1/52, but you get the information that a heart has already be drawn, the probability is now 1/13. What is the probability of drawing an ace of heart, when a heart has been drawn? P(heart) = 13/52 = 1/4 P(ace) = 4/52 P(heart and ace) = 1/52 P(Ace┤|Heart)=P(Ace and heart)/P(heart) =((1/52))/((1/4) )= 1/13 = 0.077

Extended the concept of conditional probability, for two events B and A, the overall probability of A can be found using the following formula (a concept of the Law of Total Probability) P(A)=P(A|B)P(B)+P(A ┤| B^c)P(B^{c)=P(A∩B)+P(A∩B}c) This essentially means that the probability of A is given by probability of A given that B has occurred plus probability of A given that B has not occurred.

Independent Events Two events are independent if: P(E∩F)=P(E)*P(F), that is if the knowledge that F has occurred does not affect the probability that E occurs – thus the occurrence of E is independent of whether or not F occurs. Can also be checked with the following formula: P(E│F)= (P(E∩F))/(P(F))=P(E) If it is known that events E and F are independent, knowledge on P(E) and P(F) can also be used to calculate P(E and F))

Bayes Rules P(A│B)= (P(B│A)P(A))/(P(B)),where P(A│B)= (P(A∩B))/(P(B)) by definition of conditional probability By multiplying the formula by P(B) we get: P(A│B)P(B)= P(B│A)*P(A), and thus the Bayes formula provides a useful way to go between P(A│B) and P(B│A) The denominator P(B) is sometimes also written fully using the Law of total Probability:

			P(A│B)=  (P(B│A)*P(A))/(P(B|A)*P(A)+P(B ┤| A^c)*P(A^c))

Definitions

Examples

Exercises

\[ P(E_1 \cap E_2) = P(E_1) \cdot P(E_2 \mid E_1), \quad \text{where } P(E_2 \mid E_1) = \frac{P(E_1 \cap E_2)}{P(E_1)} \] \[ \binom{n}{k}=\frac{n!}{(n-x)!(x)!} \]

\[ \binom{n}{k} = \frac{n!}{(n-x)!(x)!} \]