Expressing confidence in a measurement

When we make a measurement, we usually want to report the result along with an estimate of the likely error. Sometimes we call it the margin of error. That error is an estimate of the width of the distribution of very many measurements of the same type. Often we simply use the standard deviation, $\sigma$. Now we'd like to go further and express our results using confidence intervals of our choosing. That means we can describe our results with sentences like:

• The mean, 45.45 ± 2.20, derived from this data with a 95% confidence interval means that we can be 95% confident that the true mean is between 43.25 and 47.65;

The basic idea

The basic idea of a confidence interval isn't a great mystery. This figure shows the standard normal distribution (top graph), defined as the normal distribution with center $\mu = 0$ and standard deviation $\sigma = 1$, and which encloses an area of 1.

Confidence intervals – definitions

Division of a normal distribution into sections of width $\mu ± n\sigma$, where $n$ is 1, 2, 3, ... , $\mu$ is the mean and $\sigma$ is the standard deviation, should be familiar to you. The area under the normal distribution curve, which represents all of the probability for a particular measurement or event, can be conveniantly divided into chunks of width $\mu ± \sigma$, $\mu ± 2\sigma$, and so on.

If we express our mean as $\mu ± \sigma$, then what we're really saying is that we are 68% confident that our estimate of $\mu$ is in the interval $[\mu - \sigma, \mu + \sigma]$. Take a look at the normal curve below and notice that this means there is a 32% chance that the estimate of $\mu$ is outside of this interval.

If we broaden our interval to $\mu ± 2\sigma$ then there is a 95% probability that our estimate of $\mu$ is in the interval $[\mu - 2\sigma, \mu + 2\sigma]$, and only a 5% chance that it is outside of that interval.

Choosing a confidence interval

A 95% confidence interval centered around the estimate of a mean looks like the figure below. The blue area constitutes 95% of the total probability of whatever sample we take. There is only a 5% chance, on average, that our mean lies outside of this interval.

We need to have a mathematical way (a "formalism") for expressing and calculating confidence intervals. Check out the new labeling of the normal distribution below.

Practice 1 – Z-scores from area

Use your calculator or a standard normal table to calculate the Z-scores for the given α levels.

Example 1

Suppose that a research firm estimates the percent of adults living in a city who have cell phones less than 1 year old. Four hundred adults are selected randomly and surveyed. Of the 400 people, 132 responded "yes." Use a 95% confidence interval to estimate the proportion of adults in the city who have newer cell phones.

Solution: First let's check the three conditions. This is a simple random sample of 400 people, surely fewer than 10% of the population of a "city," and we have $n \hat p = 400 \bigg( \frac{132}{400}\bigg) = 132 \gt 10$ and $n(1-\hat p)$ is even larger. So all conditions are met and we can proceed.

The proportion of new phone owners is

$$\hat p = \frac{132}{400} = 0.33$$

The Z-score for a 95% confidence interval is found using the invNorm( ) function on a TI-84 calculator:

invNorm(0.025, 0, 1) = 1.96

A handy table

Common critical Z values for proportions

This table collects some common critical Z values (Z^*)for solving proportion problems. Often you'll use some of them so much you'll memorize them. You might also consider storing one or two in a memory slot on your calculator.

Notice the confidence intervals for Z-scores of 3.0 and 1.0, which are 97.5% and 68%, respectively. These are familiar divisions of the normal probability distribution — the 68-95-97.5 rule.

Example 2

A random sample of 1000 community college students taken from all CC students in the United States reveals that females constitute 60% of the student body. Calculuate the 90% confidence interval around this result.

Solution: First check the conditions. This is a large survey of 1000 students, but surely fewer than 10% of the U.S. population of CC students. The large sample and large proportion $(\hat p = 0.60)$ ensures that the large-count conditions will hold, and the sample was random. We're good.

The proportion of new phone owners is

$$\hat p = 0.60$$

The Z-score for a 90% confidence interval is found using the invNorm( ) function on a TI-84 calculator:

invNorm(0.05, 0, 1) = 1.6449

Then the margin of error is

$$ \begin{align} \text{MOE} &= Z_{\frac{\alpha}{2}} \sqrt{\frac{\hat p (1 - \hat p)}{n}} \\[5pt] &= 1.6449 \frac{0.60(1-0.60)}{1000} \\[5pt] &= 0.0255 \end{align}$$

Practice 2 – Confidence intervals for proportions

1. Let's say we want to determine the proportion of M&Ms^® that are blue. In which of the following situations are we able to construct a confidence interval?

   1. We take a sample of 25 candies and find 8 to be blue.
   2. We look at one bag containing about 60 candies and observe that 12 are blue.
   3. We want to determine the proportion of blue M&Ms in one small bag (about 60 candies). To do so we take a simple random sample (SRS) of 25 candies from the bag and find 9 to be blue.
   4. We randomly select 100 candies from a day's worth of factory production and find 23 to be blue.
   Solution

   1. We take a sample of 25 candies and find 8 to be blue.

      We don't know the size of the parent population, so we don't know what fraction of that population a sample of 25 represents.

   2. We look at one bag containing about 60 candies and observe that 12 are blue.

      This is a sample of only one bag – presumably chosen randomly. We'd hope for a sample of many bags, up to 10% of some larger population of bags, from which we'd calculate an average proportion of blue M&Ms and its associated standard error.

   3. We want to determine the proportion of blue M&Ms in one small bag (about 60 candies). To do so we take a simple random sample (SRS) of 25 candies from the bag and find 9 to be blue.

      25 candies out of 60 is about 42% of the total population of 60. That's not a sample from which we can make reliable inferences. We need to keep the number sampled down to 10% of the total or fewer.

   4. We randomly select 100 candies from a day's worth of factory production and find 23 to be blue.

      This is a valid sample. The number, 100, is surely less than 10% of all candies produced in a day, and it's a random sample. The proportion of 0.23, coupled with the relatively large sample, ensures that we'll pass the large-count conditions.




2. A marketing firm is hired to estimate the proportion of adults in a large metropolitan area who have and use cell phones. They randomly select 500 residents of the area and ask a binary question: "Do you own a cell phone?" 402 of 500 people responded "yes." Using a 95% confidence level (&alpha = 0.05), calculate the mean and 95% confidence interval for the true proportion of metro-area residence who own cell phones.

   Solution

   
   

   First we should look at the sample quality. 500 is a large sample, but surely less than 10% of the adults in a large metro area. The sample is an SRS, and we can see without doing a calculation that the large-count conditions will hold.

   We have

   $$\hat p = \frac{402}{500} = 0.804$$

   The critical Z value for a 95% confidence interval is $Z^* = 1.96$, so our margin of error is

   $$ \begin{align} \text{MOE} &= Z_{\frac{\alpha}{2}} \sqrt{\frac{\hat p (1 - \hat p)}{n}} \\[5pt] &= ±1.96 \, \sqrt{\frac{0.804(1-0.804)}{500}} \\[5pt] &= ±0.035 \end{align}$$

   
   

   So our 95% confidence interval would be best expressed as $[0.804 - 0.035, 0.804 + 0.035]$ or $[0.769, 0.839]$. So we can say with 95% confidence that the true proportion of residents who own cell phones is between 77% and 84%.




3. The chief financial officer (CFO) of a company wants to estimate the proportion of accounts due that are more than 30 days overdue. She surveys 500 of more than 1 million accounts and finds that 280 are more than 30 days overdue. Calculate a 90% confidence interval for the true proportion of accounts due that are late. Interpret the confidence interval.

   Solution

   
   

   First we should look at the sample quality. 500 is a large sample, but less than 10% of one million accounts due. The sample is an SRS, and we can see without doing a calculation that the large-count conditions will hold, both because of the sample size and the fairly large point estimate of the proportion,

   $$\hat p = \frac{280}{500} = 0.56$$

   The critical Z value for a 90% confidence interval is $Z^* = 1.645$, so our margin of error is

   $$ \begin{align} \text{MOE} &= Z_{\frac{\alpha}{2}} \sqrt{\frac{\hat p (1 - \hat p)}{n}} \\[5pt] &= ±1.645 \, \sqrt{\frac{0.56(1-0.56)}{500}} \\[5pt] &= ±0.036 \end{align}$$

   
   

   So our 90% confidence interval would be best expressed as $[0.56 - 0.036, 0.56 + 0.036]$ or $[0.524, 0.596]$. So we can say with 90% confidence that the true proportion of accounts due at this firm that are late is between 52% and 60%. That seems like a large proportion!




4. A campaign staff believes that 35% of all voters favor their candidate.

   1. How large of a simple random sample is required so that the margin of error of a polling estimate seeking to explore that belief is no more than ±3% at the 95% confidence level?

      Solution

      
      

      We know that the margin of error is

      $$\text{MOE} = Z_{\frac{\alpha}{2}} \sqrt{\frac{\hat p (1 - \hat p)}{n}}$$

      Solving for the number of people sampled or surveyed (n) gives:

      $$n = \frac{Z_{\frac{\alpha}{2}}}{\text{MOE}^2} \hat p (1 - \hat p)$$

      
      

      which gives $n = 495$.

      So in order to achieve a margin of error of 0.03 (3%) with a 95% confidence level and a point estimate of the proportion of $\hat p = 0.35$, we'd need to survey at least 495 people in a random sample.

   
   

   2. Suppose that we want to take a poll such that we are 99% confident within ±3% of the true value. Calculate the sample size that would be required to achieve this result.

      Solution

      
      

      This calculation is the same as the one above, except that $Z_{\frac{\alpha}{2}} = 2.576$, yielding n = 651. It makes sense that the sample size would have to be larger to obtain a higher degree of confidence.




5. A simple random sample (SRS) of 200 college students is chosen from the student population of a large university. Of those sampled, 35 identified themselves as minority students. Calculate the 90% confidence interval for the true proportion of self-identified minority students at this university.

   Solution

   
   

   First we should look at the sample quality. 200 is a fairly large sample, but certainly less than 10% of the student population of a large university. The sample is an SRS, and we can check the large-count conditions by calculating the point estimate of the proportion:

   $$\hat p = \frac{35}{200} = 0.175$$

   The large count conditions are

   $$ \begin{align} n \hat p &-= 200(0.175) = 35 \gt 10 \\[5pt] n (1 - \hat p) &= 200(0.825) = 165 \gt 10 \end{align}$$

   ... so we're good to go.

   
   

   The critical Z value for a 90% confidence interval is $Z_{\frac{\alpha}{2}} = 1.645$, so our margin of error is

   $$ \begin{align} \text{MOE} &= Z_{\frac{\alpha}{2}} \sqrt{\frac{\hat p (1 - \hat p)}{n}} \\[5pt] &= ±1.645 \, \sqrt{\frac{0.175(1-0.175)}{200}} \\[5pt] &= ±0.044 \end{align}$$

   
   

   So our 90% confidence interval would be best expressed as $[0.175 - 0.044, 0.175 + 0.044]$ or $[0.131, 0.219]$. So we can say with 90% confidence that the true proportion of students at this university who self identify as a minority is between 13% and 22%.

Confidence intervals for sample means

When our sample data suggest a mean rather than a proportion, there are two kinds of problems we can encounter:

• Population mean is known (rare)
• Population mean is unknown (usually the case)

When we calculate a confidence interval, we're usually trying to make an estimate of a population parameter and to place it between some limits – plus or minus our margin of error. Thus we don't really know very much about the parameter and its variance. We only really know about our sample.

If we know the popuulation mean and our sample is of sufficient size and quality (n < 0.1N, n ≥ 30), we can assume that Z-scores are distributed normally and we can use them to calculate confidence intervals.

Those Z-scores are

$$Z = \frac{\bar X - \mu}{\frac{\sigma}{\sqrt{n}}},$$

where $\sigma / \sqrt{n}$ is the standard error. If the standard deviation of the population is not known, then we can't use it to properly calculate normally-distributed Z-scores, so we need another method. What we do usually have is the standard deviation of the sample. We can substitute the sample standard deviation for the population $\sigma$ in these cases, but with the small cost that the Z-scores (which we'll call t values) are distributed according to a distribution called the t-distribution.

The critical t value, $t^*$, is

$$t_{\frac{\alpha}{2}} = \frac{\bar X - \mu}{\frac{s}{\sqrt{n}}},$$

where $s$ is the sample standard deviation, which we ought to know because we took the sample. These t-values are distributed according to the well-known t-distribution.

The t-distribution differs from the normal distribution in some important ways. First, the normal distribution, $N(\mu, \sigma)$, is a function of two parameters, the mean, $\mu$, and the standard deviation, $\sigma$. But the t-distribution, $t(\mu, \sigma, df)$ is also a function of the size of the sample, n, or more precisely the number of degrees of freedom, $df = n - 1$. Here are normal and t-distributions (df = 3) on the same graph.

T vs. Normal distribution

The T-distribution is similar to the normal distribution except that the latter is a function of the known mean $\mu$ and its standard deviation $\sigma$. The T-distribution does not assume knowledge of those parameters, and is instead a function of the number of degrees of freedom of the problem — the number of data points minus one.

For a small number of degrees of freedom, the "wings" of the T-distribution are wider than those of the Normal distribution. But as the number of degrees of freedom increases, the two distributions converge.

Move the slider under the graph to change the number of degrees of freedom between 1 and 21 to see that convergence.

Black = T-distribution

Red = Normal distribution

* Note on the simulation: Calculation of the T-distribution requires evaluation of the Gamma function, $\Gamma(X)$, for which I've used a bit of an approximation. That approximation breaks down slightly for larger degrees of freedom, and that's why the T-distribution curve (black) pops up above the Normal distribution for larger numbers of degrees of freedom; it shouldn't. My point was more to illustrate the differences in the wings, so that's fine with me.

Example 3

A count of Giardia lamblia, the parasite that causes Giardiasis, in one-hundred 1 mL samples of water yields a mean of 30 parasites per mL of water with a standard deviation of 8.3. Several counts of a 0.5L sample yielded a near-normal sampling distribution. Calculate the 95% confidence interval for this distribution and determine if a count of 40 in another sample lies outside of that interval.

Solution: In this problem, we know the sample standard deviation, $\sigma = 8.3$, so we can take the Z-score approach. The distribution is normal with mean $\mu ± \sigma = 30 ± 8.3$. The Z-score for a 95% confidence interval can be calculated on a TI-84 calculator using the invNorm( ) function:

invNorm(0.025, 0, 1) = 1.96

Then the margin of error is

$$ \begin{align} \text{MOE} &= Z_{\frac{\alpha}{2}} \frac{\sigma}{\sqrt{n}} \\[5pt] &= 1.96 \frac{8.3}{\sqrt{100}} \\[5pt] &= 1.63 \end{align}$$

Example 4

Consider a pain relief treatment for which we can measure pain (say by interviewing a patient) on a ten-point scale: 1 = no pain, 10 = extreme pain. Let's say that the following pain levels were recorded in one experiment.

Solution: In

invNorm(0.025, 0, 1) = 1.96

Then the margin of error is

$$ \begin{align} \text{MOE} &= Z_{\frac{\alpha}{2}} \frac{\sigma}{\sqrt{n}} \\[5pt] &= 1.96 \frac{8.3}{\sqrt{100}} \\[5pt] &= 1.63 \end{align}$$

Practice 3 – Confidence intervals for means

1. Adult mayflies live between 30 minutes and one day (0.5h to 24 h), depending on the species. Data for one species was collected by tracking the movement of ten mayflies. The measured lifespans (in hours) were

   17.68	13.69	11.22	11.05	13.86
   14.47	14.50	13.47	10.04	13.10

   Calculate a 95% confidence interval for the mean lifetime of thiis species of mayfly.

   Solution

   
   

   First let's enter the data into a calculator list and use that to calculate the average (our point estimate) and its standard deviation.

   

   Then run 1-variable statistics on the data to get the mean and standard deviation, $\bar x ± \sigma = 13.31 ± 2.17$.

   

   
   

   Now we can use those and the sample size (n = 10) to calculate the margin of error and confidence interval:

   The critical t value is

   invNorm(0.025, 9) = -2.2622

   $$ \begin{align} \text{MOE} &= t_{\frac{\alpha}{2}} \frac{\sigma}{\sqrt{n}} \\[5pt] &= 2.2622 \, \frac{2.17}{\sqrt{10}} \\[5pt] &= 1.55 \end{align}$$

   So the 95% confidence interval is [13.31 - 1.55, 13.31 + 1.55] = [11.76, 14.86].




2. For a sample of 10 dairy cows that had recently given birth, the mean protein content in 40 oz. (1.18 L) of milk was 3.42 g with a sample standard deviation of σ = 0.41 g. Calculate a 95% confidence interval for the mean protein content of the milk.

   Solution

   
   

   As is typical, we don't know the standard deviation of the parent distribution of cows, so we'll use the sample standard deviation and the t-distribution (with 9 degrees of freedom) rather than the normal distribution.

   The critical t-value for a confidence level of 95% (α = 0.05) and nine degrees of freedom can be obtained from a t-table or by using a TI-84 calculator:

   invT(0.025, 9) = -2.2622

   
   

   The margin of error is

   $$ \begin{align} \text{MOE} &= t_{\frac{\alpha}{2}} \frac{\sigma}{\sqrt{n}} \\[5pt] &= 2.2622 \, \frac{0.41}{\sqrt{10}} \\[5pt] &= 0.293 \end{align}$$

   So the 95% confidence interval is [3.42 - 0.293, 3.42 + 0.293] = [3.13, 3.71].




3. A random survey of 1500 teen-agers (ages 13-18) showed that the average number of selfies (defined as any picture in which the phone owner both appeared in and took the photo) taken each year was 422 with a standard deviation of 183. Use this information to form a conclusion about all teens with a 90% confidence interval.

   Solution

   
   

   We don't know the standard deviation of the parent distribution of teens (why would we?), so we'll use the sample standard deviation, σ = 183, and the t-distribution (with 1499 degrees of freedom) rather than the normal distribution.

   The critical t-value for a confidence level of 90% (α = 0.10) and 1499 degrees of freedom can be obtained from a t-table or by using a TI-84 calculator:

   invT(0.05, 1499) = -1.6459

   
   

   The margin of error is

   $$ \begin{align} \text{MOE} &= t_{\frac{\alpha}{2}} \frac{\sigma}{\sqrt{n}} \\[5pt] &= 1.6459 \, \frac{183}{\sqrt{1500}} \\[5pt] &= 7.8 \end{align}$$

   So the 95% confidence interval is [183 - 8, 183 + 8] = [175, 191]. We can conclude with 95% confidence that the average teen takes between 175 and 191 selfies per year.




4. A committe is set up to study whether there might be significant waste of prospective jurors' time in the judicial system. They take a random survey from jury records, asking 83 jurors how long they waited in the courthouse building before being called for service. The sample average was 7 hours with a standard deviation of 4 hours. How many people would the survey need to include to have a margin of error of 1 hour at the 95% confidence level?

   Solution

   
   

   These problems can be a little tricky because we have to make some assumptions in order to err on the side of caution. If we rearrange the margin of error expression to solve for sample number, we get

   $$\text{MOE} = t^* \, \frac{s_x}{\sqrt{n}} \, \longrightarrow \, n = \bigg( \frac{t^* s_x}{\text{MOE}} \bigg)^2$$

   Our MOE will be 1 hour, and we'll choose $t^*$ using 82 degrees of freedom. That's a conservative choice because the sample number we calculate will be larger.

   invT(0.025, 82) = -1.9893

   
   

   We'll also use the sample standard deviation (4 hours) obtained from the original study, again so that our calculated sample number is larger than what might actually be necessary. Then we have

   $$ \begin{align} n &= \bigg( \frac{t^* s_x}{\text{MOE}} \bigg)^2 \\[5pt] &= \bigg( \frac{1.9893(4)}{1} \bigg)^2 \\[5pt] &= 63.32 \end{align}$$

   We'll round that up to a sample size of 64 individuals.




5. A university bookstore wants to estimate the average amount that a student spends on books each year. They'd like to determine the total to within \$100 with 90% confidence. They can roughly estimate that the annual amount ranges between \$100 and \$1,900. How many students should be included in a random sample in order to make this estimate?

   Solution

   
   

   First we rearrange the margin of error expression for a normal distribution to solve for sample number, we get

   $$\text{MOE} = Z_{\frac{\alpha}{2}} \, \frac{\sigma}{\sqrt{n}} \, \longrightarrow \, n = \bigg( \frac{Z_{\frac{\alpha}{2}} \sigma}{\text{MOE}} \bigg)^2$$

   Our MOE will be 100, and we'll choose $Z_{\frac{\alpha}{2}}$ for a 90% confidence interval:

   invNorm(0.05, 0, 1) = -1.6449

   Here we can estimate the standard deviation by dividing the range of the data by 4:

   
   

   $$\sigma \approx \frac{2000 - 100}{4} = 475$$

   This rough rule is known as the range rule and is only a rough estimate of σ. Now we can estimate $n$:

   $$ \begin{align} n &= \bigg( \frac{Z_{\frac{\alpha}{2}} \sigma}{\text{MOE}} \bigg)^2 \\[5pt] &= \bigg( \frac{1.6449(475)}{100} \bigg)^2 \\[5pt] &= 61.05 \end{align}$$

   We'll round that down to a sample size of 61 individuals.

Interpretations of the confidence interval

There are two ways we can interpret a confidence interval in the context of the data and measurements. Let's assume here a 90% confidence interval. The first is

We have 90% confidence that any measurement of the same sample size will fall within the confidence interval.

That is, if we repeated the sample measurement a large number of times, then we expect that, on average, 90 of 100 measurements would lie within the confidence interval.

The second way is a little more subtle.

The confidence intervals around 90% of point estimates will include the "true" mean.

We can show that graphically like this.

Work flow

Here is the basic work flow for calculating a confidence interval.