Speaking the language of probability

More things in the universe are governed completely by random action than you might realize. Yet everything still works. We describe that randomness using the language, axioms and theorems of probability. To study probability is to study

Probability experiments & outcomes

In probability theory, an experiment is something that we can do repeatedly and that can have more than one possible outcome (with only one outcome, we don't need probability, right?). We can list all possible outcomes, and we generally do so in set notation, such as {a, b, c}, using curly braces.

In fact, sets, set notation and the manipulation of sets will be a key feature of our exploration of probability.

For any probability experiment, we'll want to build a model that predicts the probability of any given outcome. We'll do that by building up our knowledge from simple to more complex experiments.

Sample spaces

The sample space of a probability experiment is the set containing all possible outcomes. We often label the sample space, with the (capital) Greek letter omega (Ω). We'll use basic set notation like Ω = {H, T} for flipping a coin (H = heads, T = tails). It's also nice once in a while to use a box diagram like the one shown below. It's just a box listing all of the possible outcomes, and labeled Ω.

Here are representations of the sample space for tossing a single tetrahedral (four-sided) die. (When such a die is tossed, one number shows on three sides at the peak of the pyramid.)

A tetrahedral die. "Die" is the singular of "dice."

Probability notation

For each outcome in a sample space, we'll want to assign a probability value: what is the likelihood that outcome x will occur if the experiment is run once? Let's take a coin flip as our example. The sample set is Ω = {H, T}. We'll call the probability that the coin comes up heads P(H) and the probability that it comes up tails P(T).

Notice that the two outcomes are mutually exclusive, so the probability of the coin landing with tails up is the same as the probability that it does not land with heads up. We can write this situation like this:

$$P(T) = P(!H),$$

The probability P(!H) can be read as "the probability of not tossing heads," where the exclamation point means and is pronounced "not." It is called the compliment of P(H), and can also be written as P(H^C).

Appropriateness of sample spaces

This is a bit of an absurd example, but it will illustrate a worthwhile point: we need to pare our sample set down to observations that are relevant.

Consider the experiment of flipping a coin, with outcomes heads (H) or tails (T). Now let's consider flipping a coin while watching the cat. Sometimes the coin comes up heads while the cat is in the room and sometimes it's tails. Sometimes the cat isn't in the room when either H or T occur. Our possibilities are:

Tree diagrams

Sometimes a tree diagram is very useful for mapping out all of the possible outcomes of a probability experiment. Here is a tree diagram showing all possibilities of rolling a tetrahedral (four-sided) die twice — or rolling two dice that can be distinguished from one another, say one blue one and one red one.

For the first roll, there are four possibilities, the set $\Omega = \{1, 2, 3, 4\}.$ Each of those has an additional four possibilities for the second roll, giving the complete set of 16 possibilities:

$$ \Omega = \begin{Bmatrix} 11, & 12, & 13, & 14, \\ 21, & 22, & 23, & 24, \\ 31, & 32, & 33, & 34, \\ 41, & 42, & 43, & 44, \end{Bmatrix}$$

Obviously, to add any more rolls to this tree would make it complicated and difficult to draw, but drawing a simple tree can often help us to establish a pattern and guess the rest by inductive reasoning. It's easy to see, for example, that adding another roll would bring the number of possibilities to 64. We'll use trees like this a lot in examples in other sections.

Example 1

What is the probability of drawing a number card (2, 3, ..., 9, 10) from a standard deck of 52 playing cards?

We can simplify the sample space a little bit here by just considering one of the four possible suits of cards. It looks like this:

There are 14 different cards in each suit, three of which are face cards, and one is an ace (which we could call a 1, but I'll keep it separate from the number cards here).

Example 2

A fair coin is tossed three times. Calculate the probabilities that

1. At least one head (H) is obtained
2. Exactly one head is obtained

The easiest way to find these probabilities is to sketch a tree diagram like this one.

Now we can easily see that in this sample space (Ω), there are eight possible outcomes. Another way to sketch it is like this:

Example 3

Let's invent a new dice game. Each of two players gets one roll of a single die. If the highest of the two dice is 4 or below, player 1 wins. If the highest roll is 5 or 6, then player 2 wins. Each player can bet on the outcome in advance of the rolls. Who is likelier to win, player 1 or player 2?

Here's a table of all possible outcomes for two rolls:

Sample spaces: discrete vs. continuous

There are two kinds of probability sample spaces, discrete and continuous. Let's illustrate them with examples.

Discrete sample spaces

Consider making two rolls with a tetrahedral die (or equivalently one roll of two equivalent but distinguishable dice at the same time). Here is a table of the possible outcomes:

For this experiment, there are a finite number of possible outcomes. That number is either 16 or 10, depending on whether the order of the numbers matters. That is, is the outcome (1, 2) different than (2, 1)? That will depend on how the experiment is defined.

What's important in this context is that the number of possible outcomes is finite, and the set defined by the grid is exhastive (includes all possible outcomes). Now let's compare that to a continuous probability example.

Continuous sample spaces

Think about a dart board of sorts that is 1 meter on a side (1 m × 1 m). Imagine we throw darts at the board (and that we're incapable of missing it altogether). Each dart will have a real-number set of (x, y) coordinates to mark its position on the board. Now this experiment has an infinitely large set of possible outcomes.

Venn diagrams and set operations

Venn diagrams can be very useful for visualizing what's going on as we combine and manipulate probability sets. We'll start with a rectangle to represent our whole probability sample space, Ω.

Within that space, we can specify some subsets, like A and B below. Subset A might be all of the ways of rolling a sum of seven with two cubic dice, and B might all possible ways of rolling a sum of six, just as an example. They are both fully contained within the universe (Ω) of possible two-die rolls, but they are mutually exclusive: Either a seven is rolled or a six, but the two sets can't overlap. We say that the intersection of the two, $A \cap B,$ is the empty set, { } or ∅.

In the case above, sets A and B are said to be disjoint because they have no elements in common.

Now let's imagine that A is the set of ways to roll a total of seven with two cubic dice, and B is the set of rolls containing at least one 2. These sets can overlap because we can roll a 5-2 or a 2-5. That overlap is given by the intersection of sets A and B, $A \cap B = \{52, 25\}.$ The Venn diagram is shown below.

Practice problems