xaktly | Probability

Independence

The basic flow of the probability pages goes like this:


When are events dependent of one-another


... and when are they not? This is an important question. Generally, two events are said to be independent if.

$$P(A \cap B) = P(A)P(B)$$

That is, if the probability of the intersection of two sets is simply the product of the individual probabilities, then the two events occur independently. That's a fine definition of independence, but it's not very intuitive. Let's look at it another way, using conditional probabilities. Consider the conditional probability P(A|B). If we know that P(A ∩ B) = P(A)P(B), then we have

$$ \require{cancel} \begin{align} P(A|B) &= \frac{P(A \cap B)}{P(B)} \\[5pt] P(A|B) &= \frac{P(A)\cancel{P(B)}}{\cancel{P(B)}} \\[5pt] P(A|B) &= P(A) \end{align}$$

which means that the conditional probability P(A|B) is just P(A). In other words, it doesn't matter whether B occured first or not; the two events are independent. One obvious caveat here is that P(B) > 0, of course.

We can go through the same process for P(B|A). Recall that an intersection like $P(A \cap B)$ is indifferent to order:

$$P(A \cap B) = P(B \cap A)$$

So now we have three statements of what it means for events A and B to be independent:

$$ \begin{align} &1. \; P(A \cap B) = P(A)P(B) \\[5pt] &2. \; P(A|B) = P(A) \\[5pt] &3. \; P(B|A) = P(B) \end{align}$$

The first we consider to be the definition of independence, but the other two are easier to interpret. They are all equivalent, however, so if one of these conditions fails, then they all fail and the system is not independent.

Independence of events

Events A and B are independent if $P(A \cap B) = P(A)P(B).$ We can express this idea in terms of conditional probabilities:

$$ \begin{align} P(A|B) &= \frac{P(A\cap B)}{P(B)}, \;\; P(B) \gt 0 \\[5pt] P(A|B) &= \frac{P(A)\cancel{P(B)}}{\cancel{P(B)}} \\[5pt] P(A|B) &= P(A) \end{align}$$

means that A is independent of B.

$$ \begin{align} P(B|A) &= \frac{P(A\cap B)}{P(A)}, \;\; P(A) \gt 0 \\[5pt] P(B|A) &= \frac{P(B)\cancel{P(A)}}{\cancel{P(A)}} \\[5pt] P(B|A) &= P(B) \end{align}$$

means that B is independent of A.

Symbol for independence

The symbol for "is independent of" is $\unicode{0x2AEB},$ a sort of double-perpendicular ( ⊥ ) symbol, if you remember your geometry.

$A \unicode{0x2AEB} B$   means that sets A and B are independent. Note that if $A \unicode{0x2AEB} B,$ then $B \unicode{0x2AEB} A.$

Examples – independence


Example 1. Consider two outcomes of a probability experiment, A and B, where

$$ \begin{align} P(A) = 0.32 \\[5pt] P(B) = 0.44 \\[5pt] P(A \cap B) = 0.14 \end{align}$$

Are outcomes A and B independent?

Here we just check to see that

$$P(A \cap B) = P(A) P(B)$$

We get:

$$ \begin{align} P(A) P(B) \stackrel{?}{=} 0.14 \\[5pt] 0.32 \cdot 0.44 \stackrel{?}{=} 0.14 \\[5pt] 0.1408 \approx 0.14 \end{align}$$

So to within our ability to determine (that is, we're only allowed two significant digits here), the two sides are equal, so we conclude that the outcomes are independent.

Example 2. Consider two outcomes of a probability experiment, A and B, where

$$ \begin{align} P(A) = 0.40 \\[5pt] P(B) = 0.54 \\[5pt] P(A \cap B) = 0.30 \end{align}$$

Are outcomes A and B independent?

This time we find:

$$ \begin{align} P(A) P(B) \stackrel{?}{=} 0.30 \\[5pt] 0.40 \cdot 0.54 \stackrel{?}{=} 0.30 \\[5pt] 0.216 \ne 0.30 \end{align}$$

So these events are not independent.


Example 3. Are events A and B, with P(A) = 0.1 and P(B) = 0.2 independent?

In this case we just don't have enough information to tell. We'd either need to have a conditional probability like P(A|B) or we need to know the probability of {A ∩ B}.

Example 4 – die roll


Let's say we roll a six-sided die and we identify two outcomes:

A = {3, 4, 5, 6}   and   B = {4, 5}.

Are these outcomes independent?


Solution: Well, for a six sided die, the four possible events of set A give us a probability of

$$P(A) = \frac{4}{6} = \frac{2}{3}.$$

Likewise

$$P(B) = \frac{2}{6} = \frac{1}{3}.$$

Now the intersection of these sets is {4, 5}, for a probability of $\frac{1}{3}.$ So we have:

$$ \begin{align} P(A) P(B) &\stackrel{?}{=} P(A \cap B) \\[5pt] \frac{2}{3} \cdot \frac{1}{3} &\stackrel{?}{=} \frac{1}{3} \\[5pt] \frac{2}{9} &\ne \frac{1}{3} \end{align}$$

So these outcomes are not independent. We can also think about it using conditional probabilities. Consider the conditional probability P(A|B). If event B is obtained, then event A is guaranteed because both elements of B are in A. Once B is obtained, A is inevitable, so we have

$$P(A|B) = 1.$$

which is not the same as P(A), so our system fails the independence test this way, too.

Finally, if we take P(B|A) we find that the probability of obtaining B after A has been obtained is ½. That's not the same as P(B).

Example 5 – another die roll


Let's say we roll a six-sided die and we identify two outcomes:

A = {4, 5}   and   B = {1, 3, 5}.

Notice that the second outcome is like saying "the value showing on the die is odd." Are these outcomes independent?


Solution: The probabilities of events A and B are:

$$ \begin{align} P(A) &= \frac{2}{6} = \frac{1}{3} \\[5pt] P(B) &= \frac{3}{6} = \frac{1}{2} \end{align}$$

Now these sets intersect only at 5, so $P(A \cap B) = \frac{1}{6}.$ Now let's check for independence:

$$ \begin{align} P(A) P(B) &\stackrel{?}{=} P(A \cap B) \\[5pt] \frac{1}{3} \cdot \frac{1}{2} &\stackrel{?}{=} \frac{1}{6} \\[5pt] \frac{1}{6} = \frac{1}{6} \end{align}$$

So these outcomes are independent. Let's also show this with conditional probabilities.

$$ \begin{align} P(A|B) &= \frac{P(A \cap B)}{P(B)} \\[5pt] &= \frac{1/6}{1/2} = \frac{1}{3} \\[5pt] &= P(A) \end{align}$$

and

$$ \begin{align} P(B|A) &= \frac{P(A \cap B)}{P(A)} \\[5pt] &= \frac{1/6}{1/3} = \frac{1}{2} \\[5pt] &= P(B) \end{align}$$

So the sets are independent by all versions of the test. It's not necessary to use them all. I just did here for drill.

Now let's do something interesting. Given that sets A and B are independent, what about A and !B (the complement of B)?.

Notice that P(!B) is the same as P(B), and P(A ∩ B) is still ⅙, so we expect the result to be the same, and we find that A is independent of !B. In fact, it turns out that if A is independent of B, then:

  • $A \unicode{0x2AEB} !B$
  • $!A \unicode{0x2AEB} B$
  • $!A \unicode{0x2AEB} !B$

You can work these out by considering, for example, that if P(A|B) = P(A), then P(!A|B) = 1 - P(A|B), and so on.

Proof that if $A \unicode{0x2AEB} B$, then $A \unicode{0x2AEB} !B$


We assume that $P(A \cap B) = P(A) P(B).$

Now we can decompose P(A) like this:

$$P(A) = P(A \cap B) + P(A \cap !B).$$

That is, there are only two possibilities for A: It results in the case where B is true and where B is not. We've simply summed those probabilities to the total probability of A.

Now we have

$$ \begin{align} P(A) &= P(A \cap B) + P(A \cap !B) \\[5pt] &= P(A)P(B) + P(A \cap !B) \end{align}$$

Now we can rearrange to

$$P(A \cap !B) = P(A) - P(A)P(B).$$

Then a little algebra reveals:

$$ \begin{align} P(A \cap !B) &= P(A)P(B) \\[5pt] &= P(A)(1 - P(B)) \\[5pt] &= P(A)P(!B) \end{align}$$

So we have our conclusion that

$$A \unicode{0x2AEB} !B$$


Practice problems

1.

Given that P(A) = 0.3, P(A|B) = 0.4 and P(B) = 0.5, calculate

  1. P(A ∩ B)
  2. P(B|A)
  3. P(!A|B)
  4. P(A|!B)

and determine whether events A and B are independent.

Solution
  1. Using the definition of conditional probability, we get:

$$ \begin{align} P(A \cap B) &= P(A|B)P(B) \\[5pt] &= 0.4(0.5) \\[5pt] &= 0.2 \end{align}$$

  1. We'll use the result from (a).

$$ \begin{align} P(B|A) &= \frac{P(A \cap B)}{P(A)} \\[5pt] &= \frac{0.2}{0.3} = \frac{2}{3} \end{align}$$

  1. Recall that $P(B) = P(B \cap A) + P(B \cap !A).$ Then we have $P(B \cap !A) = P(B) - P(B \cap A).$ Now,

$$ \begin{align} P(!A|B) &= \frac{P(!A \cap B)}{P(B)} \\[5pt] &= \frac{P(B) - P(A \cap B)}{P(B)} \\[5pt] &= \frac{0.5 - 0.2}{0.5} = \frac{3}{5} \end{align}$$

  1. First,

$$P(A|!B) = \frac{P(A \cap !B)}{P(!B)},$$

where $P(!B) = 1 - P(B),$ and

$P(A \cap !B) + P(A \cap B) = P(A),$ so we have

$$ \begin{align} P(A|!B) &= \frac{P(A) - P(A \cap B)}{P(!B)} \\[5pt] &= \frac{0.3 - 0.2}{1 - 0.5} \\[5pt] &= \frac{1}{5} = 0.20 \end{align}$$

To determine whether A and B are independent we must show that $P(A \cap B) = P(A) \cdot P(B).$ We have

$$P(A) \cdot P(B) = 0.3(0.5) = 0.15,$$ which isn't the same as $P(A \cap B),$ so the events aren't independent.

2.

A single, fair, six-sided die is thrown. Let A = {2} and B = {2, 4, 6}. Are A and B independent?

Solution

The universal set for this experiment is

$$\Omega = \{ 1, 2, 3, 4, 5, 6 \}$$

Now $P(A) = \frac{1}{6},$ and $P(B) = \frac{1}{2}.$

If two events are independent, then

$$P(A \cap B) = P(A) \cdot P(B).$$

For this experiment, $(A \cap B) = \{ 2 \},$ so

$$P(A \cap B) = \frac{1}{6}.$$

We also have

$$P(A) \cdot P(B) = \frac{1}{6} \frac{1}{2} = \frac{1}{12}.$$

Now we have $P(A \cap B) \color{red}{\ne} P(A)P(B),$ so these events are not independent.

3.

The table below shows the numbers of people who were married by age range and by gender.

Age: ≤ 18 19 - 30 > 31 Total
M 43 293 114 450
F 82 299 71 452
Total 125 592 185 902

Determine whether the events F (is female) and T (was a teenager at marriage) are independent.

Solution

The condition for independence is

$$P(F \cap T) = P(F) \cdot P(T).$$

First calculate the elementary probabilities:

$$ \begin{align} P(F) &= \frac{452}{902} = 0.501 \\[5pt] P(T) &= \frac{125}{902} = 0.138 \\[5pt] P(F \cap T) &= \frac{82}{902} = 0.091 \end{align}$$

Now calculate the product of the probabilities and compare:

$$P(F) \cdot P(T) = 0.501(0.138) = 0.069$$

Now we have $P(F \cap T) \color{red}{\ne} P(F)P(T),$ so these events are not independent.

Creative Commons License   optimized for firefox
xaktly.com by Dr. Jeff Cruzan is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. © 2012, Jeff Cruzan. All text and images on this website not specifically attributed to another source were created by me and I reserve all rights as to their use. Any opinions expressed on this website are entirely mine, and do not necessarily reflect the views of any of my employers. Please feel free to send any questions or comments to jeff.cruzan@verizon.net.