Discrete random variables & expectations

Discrete random variables (DRVs) are used to describe many kinds of situations. Discrete is the opposite of continuous. A discrete variable can take on only one of some known list of values – discrete values, while a continuous variable can have any value within some range (which could be an infinite range). Let's start with a simple and familiar example.

Sum of two six-sided dice

The outcomes and probabilities of this probability experiment are

There are eleven discrete outcomes in this experiment,

$$X = \{2, 3, 4, \dots , 11, 12\},$$

where $X$ is our discrete random variable and the set of $p_i$ are the probabilities of the i^th value of $X_i.$ No values of X between any of its discrete possibilities are allowed.

The probabilities here can be thought of as the weights in a weighted average. They have the advantage, however, of being normalized – they sum to 1. So in all such cases the weighted average of all possible outcomes is

Variance $(\sigma^2)$ of a discrete random distribution

The variance of a distribution is

$$\sigma^2 = \frac{1}{n - 1} \sum_{i = 1}^n (X_i - \bar X)^2$$

By analogy with the mean of a discrete random distribution (DRD) above, we have the variance of a DRD:

$$\sigma^2 = \sum_{i = 1}^n (X_i - \bar X)^2 p_i$$

Here's an example calculation using our sum-of-two-dice data from above.

The standard deviation of the distribution is

$$ \begin{align} \sigma &= \sqrt{\sigma^2} \\[5pt] &= \sqrt{5.83} \\[5pt] &= 2.42 \end{align}$$

Practice 1

Fill in any missing probabilities and calculate the mean and standard deviation of these discrete random distributions. Be sure to check the solutions to make sure you're getting it.

1. These are data from a 2021 survey of United States households, inquiring about the number of members in the houshold from 1 to 7 or more.

   No. persons	1	2	3	4	5	6	>6
   Prob. (%)	28.45	35.03		12.39	5.83	2.03	1.24

   Fill in the missing probability and calculate the mean and standard deviation of this distribution.

   Solution

   
   

   The missing value must be 0.1503 or 15.03% because we assume that the probabilities are normalized (sum to one). The mean is

   $$ \begin{align} \bar X &= 1(0.2845) + 2(0.3503) + 3(0.1503) \\[5pt] &+ 4(0.1239) + 5(0.0583) + 6(0.0203) \\[5pt] &+ 7(0.00124) = 2.4317 \end{align}$$

   Notice that the mean of a discrete random distribution doesn't have to be one of the allowed discrete values.

   The variance is

   $$ \begin{align} \sigma^2 &= (1 - 2.4317)^2(0.2845) + (2 - 2.4317)^2(0.3503) \\[5pt] &+ (3 - 2.4317)^2(0.1503) + (4 - 2.4317)^2(0.1239) \\[5pt] &+ (5 - 2.4317)^2(0.0583) + (6 - 2.4317)^2(0.0203) \\[5pt] &+ (7 - 2.4317)^2(0.00124) = 1.9035 \end{align}$$

   Taking the square root of the variance gives us $\sigma = 1.3797.$ So our mean is $\bar X = 2.43 ± 1.38$. We'd conclude from this data that 68% of households (± one standard deviation from the mean) have between 1 and 4 members, inclusive.

   Note that we made one small approximation here in lumping all families with 7 or mor members into the 7-member category. Give the low probability for such a household, the error introduced should be small.

   
   

2. The number of traffic accidents per month at a certain busy intersection follows this distribution:

   Accidents	0	1	2	3	4
   Probability	0.465	0.382	0.135	0.015

   Fill in the missing probability and calculate the mean and standard deviation of this distribution.

   Solution

   
   

   The missing value must be 0.003 or 0.3% because we assume that the probabilities are normalized (sum to one). The mean is

   $$ \begin{align} \bar X &= 0(0.465) + 1(0.382) + 2(0.135) \\[5pt] &+ 3(0.015) + 4(0.003) = 0.709 \end{align}$$

   The variance is

   $$ \begin{align} \sigma^2 &= (0 - 0.709)^2(0.465) + (1 - 0.709)^2(0.382) \\[5pt] &+ (2 - 0.709)^2(0.135) + (3 - 0.709)^2(0.015) \\[5pt] &+ (4 - 0.709)^2(0.003) = 0.602 \end{align}$$

   Taking the square root of the variance gives us $\sigma = 0.776.$ So our mean is $\bar X = 0.709 ± 0.776$. We'd conclude from this data that in 68% of months the number of accidents lies in the interval [0, 2].

Transformations of DRVs

We'll often need to multiply and divide discrete random variables by a constant, and to add or subtract two variables. But when we do this, we're maniplating whole distributions, characterized by their means and variances.

Multiplication and division.

It's only really necessary to consider multiplication here because division is just multiplication by the reciprocal. Consider this distribution. It's made up, but it will do the job. It approximates a normal distribution, but that's not necessary for this purpose. Its mean is $\bar X ± \sigma = \bar X = 5.0 ± 0.1.456$ .

When a discrete random distribution is multiplied by 2, its mean and variance are also multiplied by 2. Thus the whole distribution is stretched in the left ↔ right direction.

Adding and subtracting

When we add and subtract discrete random variables, we add/subtract both means and variances (not standard deviations). For example, if we have random variables

$$ \begin{align} x_1 ± \sigma_1 \\[5pt] x_2 ± \sigma_2, \end{align}$$

then the sum is

$$x_1 + x_2 ± \sqrt{\sigma_1^2 + \sigma_2^2}$$

Why we add variances

... and why we do it when adding or subtracting distributions

Here is the underlying mathematics of why we add variances and take the square root of the sum to find the standard deviation of any combination of distributions.

We begin with the expectation value of anything in a discrete distribution, say the expectation of the random variable $X$, which we write $E(X)$.

$$E(X) = \sum_{i=1}^n X p_i(X)$$

That would just be the mean of the distribution. The standard deviation is then

$$E[(X - \mu)^2] = \sum_{i=1}^n (X - \mu) p_i(X)$$

Now we often want to express that in a way that is more useful for what comes next, so see if you can follow this process. Here the term $'\text{Var}(X)'$ will mean "the variance of $X$."

$$ \begin{align} \text{Var}(X) &= E[(X - \mu)^2] \\[5pt] &= E(X^2 -2 X \mu - \mu^2) \\[5pt] &= E(X^2) - 2 \mu E(X) + E(\mu^2) \tag{1} \\[5pt] &= E(X^2) - 2 \mu^2 + \mu^2 \\[5pt] \text{Var}(X) &= E(X^2) - \mu^2 \tag{2} \end{align}$$

In equation (1) we've used the fact that $E(X) = \mu$. In the last relationship, $\text{Var}(X) = E(X^2) - \mu^2$ (2) is a handy one.

The

Practice 2

1. Let $A = 2.36 ± 0.74$ and $B = 9.76 ± 2.18$, where the numbers after ± are one standard deviation. Calculate the following:

   1. $A + B$
   2. $A - B$
   3. $A + 4B$
   4. $B - \frac{A}{3}$