Probability

Probabity and Statistics
Author

Anushka Dhiman

Published

January 9, 2025

Overview

Probability is defined as a quantitative measure of uncertainty – a numerical value that conveys the strength of our belief in the occurrence of an event.

The probability of an event is always a number between 0 and 1 both 0 and 1 inclusive.

If an event’s probability is nearer to 1, the higher is the likelihood that the event will occur; the closer the event’s probability to 0, the smaller is the likelihood that the event will occur.

If the event cannot occur, its probability is 0. If it must occur (i.e., its occurrence is certain), its probability is 1.

Random experiment

An experiment is random means that the experiment has more than one possible outcome and it is not possible to predict with certainty which outcome that will be. For instance, in an experiment of tossing an ordinary coin, it can be predicted with certainty that the coin will land either heads up or tails up, but it is not known for sure whether heads or tails will occur. If a die is thrown once, any of the six numbers, i.e., 1, 2, 3, 4, 5, 6 may turn up, not sure which number will come up.

(i) Outcome A possible result of a random experiment is called its outcome for example if the experiment consists of tossing a coin twice, some of the outcomes are HH, HT etc.

(ii) Sample Space A sample space is the set of all possible outcomes of an experiment. In fact, it is the universal set S pertinent to a given experiment. The sample space for the experiment of tossing a coin twice is given by S = {HH, HT, TH, TT}

The sample space for the experiment of drawing a card out of a deck is the set of all cards in the deck.

Event

An event is a subset of a sample space S. For example, the event of drawing an ace from a deck is A = {Ace of Heart, Ace of Club, Ace of Diamond, Ace of Spade}

Event A or B

If A and B are two events associated with same sample space, then the event A or B is same as the event A ∪ B and contains all those elements which are either in A or in B or in both.

Further more, P(A∪B) denotes the probability that A or B (or both) will occur.

Event A and B

If A and B are two events associated with a sample space, then the event A and B is same as the event A∩ B and contains all those elements which are common to both A and B.

Further more, P (A ∩ B) denotes the probability that both A and B will simultaneously occur.

The Event A but not B (Difference A – B)

An event A – B is the set of all those elements of the same space S which are in A but not in B, i.e.,

A – B = A ∩ B’.

Mutually exclusive

Two events A and B of a sample space S are mutually exclusive if the occurrence of any one of them excludes the occurrence of the other event.

Hence, the two events A and B cannot occur simultaneously, and thus P(A∩B) = 0.

Consider the experiment of throwing a die once.

E = The event of getting a even number.

F = The event of getting an odd number

The events are mutually exclusive events because E ∩ F = φ.

Note that For a given sample space, there may be two or more mutually exclusive events.

Exhaustive events

If \(E_1, E_2, \ldots, E_n\) are \(n\) events of a sample space \(S\) and if

\[ E_1 \cup E_2 \cup E_3 \cup \ldots \cup E_n = \bigcup_{i=1}^{n} E_i \]

then \(E_1, E_2, \ldots, E_n\) are called exhaustive events.

In other words, events E1 , E2 , …, En of a sample space S are said to be exhaustive if atleast one of them necessarily occur whenever the experiment is performed.

Let’s consider the example of rolling a die.

We have a sample space S = {1, 2, 3, 4, 5, 6}.

We define the two events

Event A : a number less than or equal to 4 appears

Event B : a number greater than or equal to 4 appears

which implies A : {1, 2, 3, 4}, B = {4, 5, 6}

Hence, A ∪ B = {1, 2, 3, 4, 5, 6} = S

Such events A and B are called exhaustive events.

Classical definition

If all of the outcomes of a sample space are equally likely, then the probability that an event will occur is equal to the ratio :

\(\frac{\text{Number of favourable outcomes}}{\text{Total number of outcomes in the sample space}}\)

Suppose that an event E can happen in h ways out of a total of n possible equally likely ways.

Then, the classical probability of occurrence of the event is denoted by

\[ P(E) = \frac{\text{Number of favourable outcomes}}{\text{Total number of outcomes in the sample space}} \]

The probability of non-occurrence of the event \(E\) is denoted by \(P(\text{not }E)\) or \(P(E')\), where \(E'\) is the complement of \(E\). It is calculated as:

\[ P(\text{not }E) = 1 - P(E) \]

Thus,

\[ P(E) + P(\text{not }E) = 1 \]

The event “not \(E\)” is denoted by \(E'\) (complement of \(E\)).

Therefore,

\[ P(E') = 1 - P(E) \]

Axiomatic approach to probability

Let S be the sample space of a random experiment.

The probability P is a real valued function whose domain is the power set of S, i.e., P (S)

and range is the interval [0, 1] i.e. P : P (S) → [0, 1]

satisfying the following axioms.

  1. For any event E, \[ 0 \leq P(E) \leq 1\]

  2. P(S) = 1

  3. If E and F are mutually exclusive events, then P (E ∪ F) = P (E) + P (F).

It follows from (iii) that P (φ) = 0.

Let \(S\) be a sample space containing elementary outcomes \(w_1, w_2, \ldots, w_n\), i.e.,

\[ S = \{w_1, w_2, \ldots, w_n\} \]

It follows from the axiomatic definition of probability that:

  1. \(0 \leq P(w_i) \leq 1\) for each \(w_i \in S\)

  2. \(P(w_1) + P(w_2) + \ldots + P(w_n) = 1\)

  3. For any event \(A\) containing elementary events \(w_i\), the probability of \(A\) is the sum of the probabilities of these elementary events. Thus, if \(A = \{w_i\}\), then \(P(A) = P(w_i)\). However, if \(A\) contains multiple elementary events, say \(A = \{w_1, w_2, \ldots, w_k\}\), then:

\[ P(A) = P(w_1) + P(w_2) + \ldots + P(w_k) \]

Let’s consider a fair coin is tossed once P (H) = P (T) = \(\frac{1}{2}\) satisfies the three axioms of probability.

Now suppose the coin is not fair and has double the chances of falling heads up as compared to the tails, then P (H) = \(\frac{2}{3}\) and P (T) = \(\frac{1}{3}\) .

This assignment of probabilities are also valid for H and T as these satisfy the axiomatic definitions.

Probabilities of equally likely outcomes:

Let a sample space of an experiment be \(S = \{w_1, w_2, \ldots, w_n\}\) and suppose that all the outcomes are equally likely to occur, i.e., the chance of occurrence of each simple event must be the same. Thus,

\[ P(w_i) = p \text{ for all } w_i \in S, \text{ where } 0 \leq p \leq 1 \]

Since the sum of probabilities of all outcomes must equal 1:

\[ \sum_{i=1}^{n} P(w_i) = 1 \]

This implies:

\[ p + p + \ldots + p \text{ (n times)} = 1 \]

\[ \Rightarrow np = 1 \]

Therefore,

\[ p = \frac{1}{n} \]

Let \(S\) be the sample space and \(E\) be an event, such that \(n(S) = n\) and \(n(E) = m\). If each outcome is equally likely, then it follows that:

\[ P(E) = \frac{m}{n} = \frac{\text{Number of outcomes favourable to } E}{\text{Total number of possible outcomes}}\]