The existence of gambling for many centuries is evidence of long-running interest in probability. But a good understanding of probability transcends mere gambling. The mathematics of probability are very important for understanding all kinds of important topics.
In this section we will consider probability for discrete random variables.
Discrete in this sense means that a variable can take on one of only a few specific values. A good example is a coin. When laying flat, only one side can possibly be showing at a time. Another is a die (singular of dice), which can show numbers 1-6 only, and only one of those at a time. In the section on continuous probability we'll consider continuous random variables, but we're not there yet.
Why is probability important?
Our universe is driven mostly by random events, so it's very important to understand randomness and the probability of any event occurring in such a universe. Here are a few examples of where you might need to understand probability, but there are many, many others.
1. Experimental science
Any experimental measurement, no matter how carefully performed, is affected by random errors or “noise.” Shown at right is data from a high resolution far-infrared spectroscopy experiment. The "peak" represents absorption of a very small amount of far-infrared light by the C3 molecule. You can easily see the noise (roughness) in the signal. Random errors follow the laws of probability, which form the basis of how we estimate the effect of those errors on our results.
For example, we might measure a length and report it as 3.45 ± 0.03 meters, where the 0.03 is a measure of the “average” random error present in the measurement. Just how we estimate that average error comes from a study of probability.
Source: Giesen et al., Astrophys. J., 551:L181-L184 (2001)
2. Chemistry & Physics
Whether a chemical reaction takes place depends on a number of factors, like whether reactants collide (necessary for a reaction to occur), with what kinetic energy they collide, and in what orientation they collide (see the illustration). Because in any ensemble (group) of reacting molecules there will be a wide and randomly-occurring range of speeds, paths and orientations, these processes are best understood using the laws of probability.
There is a whole field in physics/chemistry called statistical mechanics, based on probability theory, that derives the laws of thermodynamics from a study of the behavior of large ensembles of atoms & molecules.
3. Medicine
The laws of probability are crucial in the medical sciences. Among other things, they are important for developing effective tests for diseases and in testing for the presence of drugs and other substances.
In testing the effectiveness of drugs, researchers must carefully employ the laws of probability and statistics. There are legendary cases of reliance upon a drug to treat some disease which later was proven to be completely ineffectual by careful probability-based analysis.
Much of the work of public health professionals is backed up by a solid knowledge of probability to prove or disprove cause-and-effect relationships.
Image: Wikipedia Commons
Discrete random variables
A discrete random variable is one that can only take on one of a set of specific values at a time.
- A die, for example, can land (randomly) showing the numbers 1, 2, 3, 4, 5 or 6 only
- A coin can land with only one of two sides showing at a time.
- One card drawn from a deck of playing cards can have only one of 52 unique values.
Discrete Probability
Discrete events are those with a finite number of outcomes, e.g. tossing dice or coins. For example, when we flip a coin, there are only two possible outcomes: heads or tails. When we roll a six-sided die, we can only obtain one of six possible outcomes, 1, 2, 3, 4, 5, or 6. Discrete probabilities are simpler to understand than continuous probabilities, so that's where we'll begin.
Let's look at flipping a coin first. The probability, we'll call it P, of obtaining an outcome (heads or tails) is 1 chance in 2, or 1:2, or just ½.
The possible elementary outcomes of our experiment (coin flipping) form a set {H, T}. If we call P the probability function and H and T the two possible outcomes, then P(H) = ½, and P(T) = ½. When we flip a coin, we have to get either H or T, so the total probability is 1. Here, of course, we need to say that we're ruling out the unlikely event that the coin will land in such a way that it sticks on its edge. When we flip a coin, we make the reasonable assumption that there are only two possible outcomes, and the one we get can only be one of those, or ½ of the total.
So the probability of obtaining either outcome H or outcome T from our experiment (flipping the coin) can be written:
$$P(H) + P(T) = 1$$
In other words, the sum of all possible discrete outcomes is one. Note that this is only true when outcomes H and T are mutually exclusive, i.e. when they can't occur at the same time. The story would be different if we could get heads and tails at the same time. (We disregard the very unlikely event that the coin lands on its edge.)
The sum of all possible discrete outcomes of a probability experiment is one.
Two important rules of probability
We can write down two important rules of probability now. The first says that or an outcome of a probability experiment to be defined, it must have a finite positive probability. If the probability is zero, it can never happen and we don't have to worry about it, and negative probabilities don't make any sense.
If you haven't seen summation notation before, rule (2) translates like this: the sum of all n outcomes (from one to n, labeled with the index i) is one. It means that for a particular experiment, like flipping a coin, the sum of the probabilities of all outcomes (½ for heads, ½ for tails) must equal 1. It's another way of saying that something has to happen.
Now let's take a look at something more complicated, rolling dice ...
$$(1) \phantom{0000} P(E_i) \gt 0$$
The probability of the ith event (Ei) must be greater than zero.
$$(2) \phantom{00} \sum_{i = 1}^n P(E_i) = 1$$
The sum of all probabilities of all n possible events must be 1. That is, the probability the something happens is 1.
Rolling dice
Here are the 36 possible outcomes of rolling two distinguishable (one white, one black) dice.
The probability of rolling a three, for example, is just the number of ways of forming a total of three divided by the total number of possibilities (36). For distinguishable dice, we can roll a (1, 2) and a (2, 1), where the bold number corresponds to the white die. The probability of rolling a 3 is then 2/36 = 1/18. It's the same calculation for any other roll. Notice that there are six ways of rolling a 7 for a probability of 6/16 = 1/6, making it the most probable result.
If we insist that in rolling a 7, we roll a 1 on the white die and a 6 on the black, then the probability of that roll is 1/36. If we want a (6, 1) combination but we don't care which die comes up 1 or 6, the probability is 2/36 = 1/18. This is the case with indistinguishable dice (e.g. both white or both black).
Practice
Determine the probability of the following outcomes of rolling two distinguishable dice. Roll over the question to see its answer.
More complex events
Continuing with the distinguishable dice from above, let's now define some more complicated events. An even will be something that can happen in our probability experiment. It can be an elementary outcome,
such as rolling a 3 with two dice, or something more complicated, such as the probability of rolling a 3 OR a 4. Here are a few examples of how we might define a few events for two dice:
| Event | Description | Elementary outcomes | Probability |
|---|---|---|---|
| A | Dice add to 3 | (1,2),(2,1) | 2/36 = 1/18 |
| B | Dice add to 6 | (1,5),(5,1),(2,4),(4,2),(3,3) | 5/36 |
| C | White die = 1 | (1,1),(1,2),(1,3),(1,4),(1,5),(1,6) | 6/36 = 1/6 |
| D | No 4 on either die | (1,1),(1,2),(1,3),(1,5),(1,6), (2,1),(2,2),(2,3),(2,5),(2,6), (3,1),(3,2),(3,3),(3,5),(3,6), (5,1),(5,2),(5,3),(5,5),(5,6), (6,1),(6,2),(6,3),(6,5),(6,6) |
25/36 |
Combining events
It's very often an advantage to combine two events. In the table below we'll combine some of the events (A, B, C, D) from the table above, and introduce some new notation while we're at it. This notation will be particularly helpful a bit later on in our study of discrete probability and Bayesian probability.
We're most interested in when two outcomes can occur together, like outcome A AND B, or when one or the other occurs: outcome A OR B.
Symbols defined
| Symbol | Meaning |
| ∩ | AND |
| ∩ | OR |
| ! | NOT |
| Combination of events | Definition | Notation |
|---|---|---|
| A and B | A and B both occur | $A \cap B$ |
| A or B | Either A or B occurs | $A \cup B$ |
| not A | Event A does not occur | $!A$ |
| A and not C | A occurs and C does not | $A \cap !C$ |
| not B or not C | Either B does not occur or C does not | $!B \cup !C$ |
An event either occurs or it does not. The sum of the probabilities of an event occurring and not occurring is 1:
$$P(A) + P(!A) = 1$$
Combining events: Example 1
What is the total probability of rolling a 1 on either die, rolling two dice at a time?
First, we'll establish some events. We'll let event A be rolling a 1 on the white die, and event B be rolling a 1 on the dark die.
The diagram above shows all 36 possible combinations of rolling two distinguishable dice. In red along the left side are the six ways that a 1 can be rolled on the white die, and across the top are the ways that 1 can be rolled on the dark die.
Now the probability of event A is P(A) = 6/36 = 1/6. Likewise, the probability of event B is P(B) = 6/36 = 1/6.
$$P(A) = P(B) = \frac{1}{6}$$
Now it would seem that the probability of rolling a 1 on either the white or dark die would be the sum of these
two probabilities because by adding the two events with the word "or," we're increasing our chance of success.
But we have to be careful not to over count the 1,1 outcome. We can't double count it as both an outcome of event A and event B, so the total number of ways of success is 11, not 12, and the total probability is 11/36, not 1/3.
Symbolically we write the probability of either event A or event B occurring as P(A ∪ B), and
$$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$
where P(A ∩ B) is the probability that both events happen together. This takes care of our overlap.
Probability that either of two events occur
The probability, P(A ∪ B), that either event A or event B occurs is
$$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$
where P(A) and p(B) are the probabilities of events A and B, and P(A ∩ B) is the probability that both A and B occur together.
Combining events: Example 2
Let event A be rolling a sum of 3 on two dice, and let event B be rolling a sum of 5 on two dice. Calculate the probability of rolling a sum of either a 3 or a 5. The possibilities are illustrated below. The ways of rolling 3 and 5 are highlighted in blue and red, respectively.
In this situation, P(A) = 2/36 = 1/18, and P(B) = 4/36 = 1/9. Our experience up to this point suggest (correctly) that P(A ∪ B) = P(A) + P(B) - P(A ∩ B). But P(A ∩ B) is the probability that both A and B occur together. In this case, the figure shows that they can clearly not occur together.
Either we roll a sum of 3 or we roll a sum of 5. We can't do both. These events are called mutually exclusive, and P(A ∩ B) = 0.
So the total probability is
$$ \begin{align} P(A \cup B) &= P(A) + P(B) - P(A \cap B) \\[5pt] &= \frac{1}{18} + \frac{1}{9} \\[5pt] &= \frac{3}{18} = \frac{1}{6} \end{align}$$
Notice that P(A ∩ B) = 0, reflecting the fact that A and B cannot occur simultaneously.
Mutually-exclusive events
Events A and B are mutually exclusive if it is the case that, if A occurs, B cannot possibly occur, and if B occurs, A cannot possibly occur.
$$P(A \cap B) = 0$$
Another way of looking at it
This Venn diagram is a nice illustration of how combining event probabilities and mutual exclusion works.
$$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$
$$P(A \cup B) = P(A) + P(B)$$
Combining events: Example 3
Rolling two dice: Let event A be that the sum of the dice is even, event B be that at least one of the dice shows a 6, and event C when the values of the two dice are equal. In the figure, A is highlighted in yellow, and B in blue. Note that some outcomes share yellow and blue. And C outcomes are outlined in red. We'll take a look at a variety of combinations of these events and their probabilities in the table below the figure.
The table below summarizes the probabilities of several event combinations. Let's step throught logic of each in these paragraphs:
P(A) is the probability that the sum of the dice is even. There are 18 such outcomes highlighted in yellow above. 18/36 = ½, so half of all rolls of two dice, on average, will yield an even number. Notice that we could call the probability of rolling an odd number P(!A) = ½, and because the two events are mutually exclusive, P(A) + P(!A) = 1.
P(B) is the probability that at least one die is a 6. For that we can simply count the number of rolls that contain one or two 6's (including 6,6) to get 11/36. We have to remember not to double count the 6,6 roll.
P(C) is the probability that each of the two dice show the same number. Those six outcomes are along the diagonal of the figure and P(C) = ⅙.
P(A ∩ B) is the probability that the sum of two dice is even and that at least one of them is a 6.
For that probability, simply count outcomes that share blue and yellow color in the figure; there are five such rolls: (26), (46), (66), (62) and (64). Remember that (46) and (64) are different rolls because we're using distinguishable dice. The probability is 5/36.
P(B ∩ C) is the probability that at least one die shows a 6 and that the two values are equal. In this case, there's only one such outcome, (66), so the probability is 1/36.
P(A ∪ B) is the probability that either the sum of two dice is even or that at least one die shows a 6. "OR" probabilities like this are calculated by summing the probabilities of each event occuring alone and subtracting the probability of each occuring together: P(A ∪ B) = P(A) + P(B) - P(A ∩ B), so the probability is ½ + 11/36 - 5/36 = ⅔.
Finally, P(B ∪ C) = P(B) + P(C) - P(B ∩ C) = 11/36 + 1/6 - 1/36 = 4/9. There is a 4-in-9 chance of rolling at least one six or having both dice show the same number.
The results are tabulated below.
| Event | Expression | Probability |
|---|---|---|
| $A$ | $P(A)$ | $\frac{18}{36} = \frac{1}{2}$ |
| $B$ | $P(B)$ | $\frac{11}{36}$ |
| $C$ | $P(C)$ | $\frac{6}{36} = \frac{1}{6}$ |
| $A \: \text{and} \: B$ | $P(A \cap B)$ | $\frac{5}{36}$ |
| $B \: \text{and} \: C$ | $P(A \cap B)$ | $\frac{1}{6} \cdot \frac{1}{6} = \frac{1}{36}$ |
| $A \: \text{or} \: B$ | $P(A) + P(B) - P(A \cap B)$ | $\frac{1}{2} + \frac{11}{36} - \frac{5}{36} = \frac{24}{36} = \frac{2}{3}$ |
| $B \: \text{or} \: C$ | $P(B) + P(C) - P(B \cap C)$ | $\frac{11}{36} + \frac{6}{36} - \frac{1}{36} = \frac{16}{36} = \frac{4}{9}$ |
Practice problems
| 1. |
A card game requires that each hand contain five cards, drawn randomly from a standard deck of 52 cards. How many unique hands are possible? Remember that the order of the cards doesn't matter. SolutionThere are 52 unique cards in the deck. The number of possible hands is $52 \cdot 51 \cdot 50 \cdot 49 \cdot 48.$ But the order of the cards drawn doesn't matter. There are 5! = 5·4·3·2·1 ways of ordering five cards, so we have do divide that into our number: $$P = \frac{52 \cdot 51 \cdot 50 \cdot 49 \cdot 48}{5!} = 2,598,960.$$ There are over 2.5 million possible 5-card hands. |
| 2. |
How many unique 7-digit product codes are possible if the first two places of the code are letters, the next three places are numbers and the last two places can be either a letter or a number. Assume that numbers run from 0 to 9, and letters from A,B,C ... to X,Y,Z. SolutionThere are 26 letters, and we allow them to repeat. There are 10 digits, and where letters or digits are possible, there are 36 possibilities. The number of possible codes is $$N = (26)(26)(10)(10)(10)(36)(36) = 876,095,000$$ There are over 875 million possible codes. |
| 3. |
Two fair, 6-sided dice are thrown. Let A be the event that the sum of the dice is even. Let B be the event that at least one of the dice lands on 6, and let C be the event that the numbers on the two dice are equal. Calculate:
Solution
|
| 4. |
A hospital staff comprises 132 people. 55 are males and 76 are females. 41 are medical doctors (MDs) and 90 are registered nurses (RNs). From this group, calculate the probabilities of finding
SolutionsBeing a nurse or a doctor (in this group) are mutually-exclusive outcomes, as are being male and female. These are simple AND probabilities. First we have P(♂) $= \frac{55}{132} = 0.417$ Then we can calculated probabilities a-d: P(♂ ∩ MD) = (0.417)(0.311) = 13% Notice that to within our rounding error here, the probabilities, which span all possible combinations of outcomes, add to 100%. Here is a tree diagram of these probabilities, another way of looking at this situation that might help.
|
| 5. |
The birthday paradoxHow many people must be in a group in order for the probability that two people have the same birthday to be 50% ? SolutionsThis is a common problem in probability theory that can be really enlightening if you work to understand it. The most common answer is 366 people (if we count Feb. 29), but that's incorrect, because it assumes we're asking about a specific day, but we're not — just that two share a common birthday. The trick is not to calculate P(B), the probability of sharing a birthday, but P(!B), the probability of not sharing a birthday, then realizing that P(B) + P(!B) = 1 to solve for P(B) in the end. The probability that person 1 shares her own birthday is 1, which we'll write as $$\frac{365}{365}$$ The next person has 364 of 365 chances of not having the same birthday, so the probability that two people will not share the same day will be the product, $$\frac{365}{365} \cdot \frac{364}{365} = 99.7\%$$ For subsequent people, we continue the calculation. The probability that n people don't share the same birthday is $$P(!B) = \frac{365}{365} \cdot \frac{364}{365} \cdot \frac{363}{365} \cdot \dots \cdot \frac{366 - n}{365}$$ Now we need to find the $n$ for which the probability drops under 50%. That takes some trial and error, but the answer is 23 people. I found it by setting up a spreadsheet that, row-by-row, multiplies the terms to calculate P(!B) and P(B). Here is a graph of P(B) vs. n:
|
Glossary of probability terms
| Term | Definition |
|---|---|
| Random | without any order or definite aim – a list of random numbers will have no discernable pattern. True randomness is often difficult to achieve. |
| Discrete random variable | A variable that can have only one of a set of discrete values, like the side of a six-sided die that lands on top, or the face of a coin that shows after a fair toss. Contrast this to a continuous random variable like height of a human, which can vary over an infinitely continuous scale. |
| Experiment | In probability theory, an "experiment" is one or more trials of some process (like flipping a coin, rolling a die) that yields a specific outcome (like "heads" or "5"). An experiment may include one or more elementary outcomes. |
| Elementary outcome | An elementary outcome is one of the set of possible outcomes of a single discrete probability experiment. The set of elementary outcomes for rolling a six-sided die is $o = \{1, 2, 3, 4, 5, 6\}.$ |
| Probability | The probability of an outcome of an experiment is the likelihood that it will occur, usually expressed as a fraction. The probability of flipping a coin and having it land heads-up is 1 out of 2 or ½, or 50%. |
| And ( ∩ ) | The word "and" in probability means "occuring together." On rolling two fair six-sided dice, we can as what the probability of rolling a 1 AND a 2 is, or P(1) ∩ P(2). In this case, the probability of rolling a 1 is 1/6, and the probability of then rolling a 2 is 1/6. Multiplication of these probabilities gives 1/36, but we could roll this the other way around, as 2 then 1, so we sum those probabilities to get 2/36 = 1/18. |
| Or ( ∪ ) | The word "or" in probability means that we are looking for either of two outcomes. For example, the probability of rolling a 1 on a six-sided die is 1/6. Likewise, the probability of rolling a 2 is 1/6. The probability of rolling either a 1 or 2 is the sum, 2/6 = 1/3. |
| Mutually exclusive | Two elementary outcomes of a probability experiment are mutually exclusive if, when the first occurs, the second cannot. For example, if one outcome of rolling two dice is that the two rolls are the same (1-1, 2-2, 3-3, and so on), and another outcome is that the sum of the dice is odd, then these outcomes are mutually exclusive. It is impossible for the sum of doubles to be odd, and it is impossible for an odd sum to come from doubles. |
xaktly.com by Dr. Jeff Cruzan is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. © 2012, Jeff Cruzan. All text and images on this website not specifically attributed to another source were created by me and I reserve all rights as to their use. Any opinions expressed on this website are entirely mine, and do not necessarily reflect the views of any of my employers. Please feel free to send any questions or comments to jeff.cruzan@verizon.net.