Conditional probability

A conditional probability is the probability that one event occurs, given that another event already has. The concept of conditional probability allows us to revise our probability models given new information. We use the notation

$$P(A|B)$$

to mean "the probability that A occurs after B already has." Here are some Venn diagrams to illustrate the meaning of conditional probability.

First, imagine an experiment with events A and B. There are twelve possible outcomes, including six ways to obtain outcome A and seven ways to obtain B, with three of those shared. In this scenario, there are two outcomes in our sample space that aren't outcomes A or B.

In the sample space, the probabilities of A and B are

$$P(A) = \frac{6}{12} \phantom{00000} P(B) = \frac{7}{12}$$

That's just the number of dots in A divided by 12 dots total in Ω, and so on.

If we have only numerical probabilities:

Now let's do the same conditional probability problem, but this time we won't know the number of elements in each set, only the numerical probabilities of events $A$, $B$ and $A \cap B$. Here's the Venn diagram:

Independence of events

Events may or may not be independent. We might want to know whether the occurrence of one event affects the occurrence of another. Two events, A and C, are independent if the occurrence of one does not affect on the probability of occurrence of the other. That means

$$P(A) = P(A|C) \: \: \color{#E90F89}{\text{or}} \: \: P(C) = P(C|A)$$

In the first equation, we see that the probability that event $A$ occurs is the same as the probability that $A$ occurs after $B$. In other words, it doesn't matter whether $B$ occurs first or not — they're uncoupled or independent. The second equation says the same thing. Here are two examples of a two-dice experiment to illustrate how we can check for independence:

Example 1

Let's say that a fair coin, with outcomes "heads" (H) and "tails" (T) is tossed three times, and answer these questions:

1. Calculate the probability of getting three heads, P(HHH).
2. Calculate the probability of getting exactly one head.
3. Calculate the probability of seeing at least one more head after a heads outcome is already observed.

Solution: For (a), we observed that tossing of the three coins, either together or one after another, produced three independent outcomes. No one coin affects any other. So we're looking for the probability P(HHH), which is

$$P(HHH) = P(H) \cdot P(H) \cdot P(H) = \left( \frac{1}{2} \right)^3 = \frac{1}{8},$$

so $P(HHH) = \frac{1}{8}.$

(b) Now for the probability of observing exactly one heads (H), we're looking for either of the outcomes HTT, THT or TTH. Just like in the previous calculation, the probabilities of each are $P(HTT) = P(THT) = P(TTH) = \frac{1}{8}.$

The probability of tossing one of these is the sum of three probabilities:

$$P(\text{one heads}) = \frac{1}{8} + \frac{1}{8} + \frac{1}{8} = \frac{3}{8}.$$

(c) Now the third probability is conditional. First, the probability of rolling at least one heads can be thought of in this way.

Example 2

Let's say that in your city it is sunny $(S)$ one-third of the days. When it is sunny, there is a 50% chance that traffic will be heavy, but when it's rainy $(!S)$, that chance reduces to 25%. If it's rainy and there is heavy traffic, you will arrive late to work 50% of the time. Otherwise, on sunny days with light traffic, you arrive to work late 1/8 of the time. When it's rainy with no traffic or sunny with traffic, than chance of you being late is 1/4. Now pick any random day and caclulate:

1. The probability that it is sunny and there is heavy traffic if you are not late.
2. The overall probability that you'll be late for work.
3. Given that you arrived late, the probability that it rained that day.