Conditional probability

byMarco Taboga, PhD

A conditional probability is a probability assigned to an event after receiving some information about other relevant events.

Aims

The aims of this lecture are to:

• introduce the concept of conditional probability in a mathematically sound manner;

• provide two proofs of the formula used to compute conditional probabilities;

• discuss some of the subtleties involved in the formula, including potential division-by-zero problems.

The mathematical setting

Let be asample space and let denote theprobability assigned to theevents.

Suppose that, after assigning probabilities to the events in, we receive new information about the things that will happen (the possible outcomes).

In particular, suppose that we are told that the realized outcome will belong to a set.

How should we revise the probabilities assigned to the events in, to properly take the new information into account?

The answer to this question is provided by the conditional probability, the revised probability assigned to an event after learning that the realized outcome will be an element of.

A gradual approach

Despite being an intuitive concept, conditional probability is quite difficult to define in a mathematically rigorous way.

We take a gradual approach in this lecture:

• we first discuss conditional probability for the very special case in which all thesample points are equally likely;

• we then give a more general definition;

• finally, we refer the reader to other lectures where conditional probability is defined in even more abstract ways.

The case of equally likely sample points

Suppose that a sample space has a finite number of sample points:

Suppose also that each sample point is assigned the same probability:

Counting favorable cases

In such a simple space, the probability of a generic event is obtained aswhere denotes the cardinality of a set, that is, the number of its elements.

In other words, the probability of an event is obtained in two steps:

1. counting the number of "cases that are favorable to the event", that is, the number of elements belonging to;

2. dividing the number thus obtained by the number of "all possible cases", that is, the number of elements belonging to.

For example, if, then

First proof of conditional probability formula

When we learn that the realized outcome will belong to a set, we still apply the rule

However, the number of all possible cases is now equal to the number of elements of because only the outcomes belonging to are still possible.

Furthermore, the number of favorable cases is now equal to the number of elements of because the outcomes in are no longer possible.

As a consequence,

By dividing numerator and denominator by, we obtain

Therefore, when all sample points are equally likely, conditional probabilities are computed as

Example

Let us make an example.

Suppose that we toss a die. Six numbers (from to can appear face up, but we do not yet know which one of them will appear.

The sample space is

Each of the six numbers is a sample point and is assigned probability.

Define the event as follows:where the event could be described as "an odd number appears face up".

Now define the event as follows:where the event could be described as "a number greater than three appears face up".

The probability of isSuppose we are told that the realized outcome will belong to. How do we have to revise our assessment of the probability of the event, according to the rules of conditional probability?

First of all, we need to compute the probability of the event:

Then, the conditional probability of given is

Comments about the formula

In the next section, we will show that the conditional probability formulais valid also for more general cases (i.e., when the sample points are not all equally likely).

However, this formula already allows us to understand why defining conditional probability is a challenging task.

In the conditional probability formula, a division by is performed. This division is impossible when is azero-probability event (i.e.,).

If we want to be able to define also when, then we need to give a more complicated definition of conditional probability. We will return to this point later.

A more general approach

In this section we give a more general definition of conditional probability, by taking an axiomatic approach.

First, we list the mathematical properties that we would like conditional probability to satisfy.

Then, we prove that the conditional probability formula introduced above satisfies these properties.

Mathematical properties

The conditional probability is required to satisfy the following properties:

1. Probability measure. has to satisfy all theproperties of a probability measure.

2. Sure thing..

3. Impossible events. If (, the complement of with respect to, is the set of all elements of that do not belong to), then.

4. Constant likelihood ratios on. If, and, then

These properties are very intuitive:

1. Property 1 requires that also conditional probability measures satisfy the fundamental properties that any other probability measure needs to satisfy.

2. Property 2 says that the probability of a sure thing must be: since we know that only things belonging to the set can happen, then the probability of must be.

3. Property 3 says that the probability of an impossible thing must be: since we know that things not belonging to the set will not happen, then the probability of the events that are disjoint from must be.

4. Property 4 is a bit more complex: it says that if is - say - two times more likely than before receiving the information, then remains two times more likely than, also after receiving the information because all the things in and remain possible (can still happen) and, hence, there is no reason to expect that the ratio of their likelihoods changes.

Second proof of conditional probability formula

When the above properties are satisfied, then the conditional probability formula can be used.

Tackling division by zero

In the previous section we have generalized the concept of conditional probability. However, we have not been able to define the conditional probability for the case in which. This case is discussed in the lectures:

• Conditional probability with respect to a sigma-algebra;

• Conditional probability distributions.

The law of total probability

We end this lecture by stating an important formula that allows us to write the probability of an event as a weighted sum of conditional probabilities.

Let, ..., be events having the following characteristics:

1. they are mutually disjoint: whenever;

2. they cover all the sample space:

3. they have strictly positive probability: for any.

The events, ..., are called a partition of.

Thelaw of total probability states that, for any event, the following holds:which can, of course, also be written as

Solved exercises

Some solved exercises on conditional probability can be found below.

Exercise 1

Consider a sample space comprising three possible outcomes,,:

Suppose the three possible outcomes are assigned the following probabilities:

Define the events

and denote by the complement of.

Compute, the conditional probability of given.

Exercise 2

Consider a sample space comprising four possible outcomes,,,:

Suppose the four possible outcomes are assigned the following probabilities:

Define two events

Compute, the conditional probability of given.

Exercise 3

The Census Bureau has estimated the following survival probabilities for men:

1. probability that a man lives at least 70 years: 80%;

2. probability that a man lives at least 80 years: 50%.

What is the conditional probability that a man lives at least 80 years given that he has just celebrated his 70th birthday?

How to cite

Please cite as:

Taboga, Marco (2021). "Conditional probability", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/conditional-probability.