How to describe the precision of a measurement

In making any measurement, it's nice to be able to describe the quality of that measurement. What we're interested in here is precision, the ability of a measurement technique to be reproduced with only small errors.

These graphs represent simulated measurements of a quantity (maybe a length or a mass) with a "true value" of 6. The green circles are instances of each measurement. In the top graph, four measurements produced a value of 3, three produced a value of 4, and so on. The average of all 25 measurements is 6.12, pretty close to our true value. The number after the ± sign will be know as the standard deviation, and it represents the precision of those measurements – how spread out they are. More on this later, but it will turn out that about 68% of all of the measurements will fall in the range between 6.12 - 2.09 and 6.12 + 2.09.

The 25 measurements in the lower graph are more tightly clustered about the true value. They are more precise. This is represented mathematically by the smaller standard deviation, which shows that 68% of all measurements in this set will lie between 6.08 - 0.81 and 6.08 + 0.81, a more narrow range than the other set.

On this page we'll learn how to calculate the standard deviation (and related variance) of sets of data like these.

The standard deviation, $\sigma$

The standard deviation is a measure of the width of the underlying distribution of measurements. The narrower the distribution, the more tightly clustered the measurements will be about the mean and the more precise those measurements are.

Roll over or tap the image to compare the widths of a precise and less precise distribution.

Calculating the standard deviation

Mathematically, the standard deviation, usually signified by the Greek letter sigma ($\sigma$) is the average value of the distance of each measurement from the mean. It looks like this:

$$\sigma^2 = \frac{1}{N} \sum_{n=1}^N \, (x_n - \bar x)^2$$

You should study that equation a bit. It isn't as intimidating as it looks. If you don't know about summation notation, the Greek capital letter sigma ( $\Sigma$ ) simply means to add up $N$ things starting from the first one (we count them with the variable $n$, starting at $n = 1$) and ending at the $N$^th. There are $N$ total measurements.

Example 1

Calculating a simple standard deviation

Let's say we have a set of ten measurements, $x_1$ through $x_{10}$, like this:

$$ \begin{align} x = \{ &1.5, 2.0, 2.5, 2.9, \\ &3.1, 3.2, 3.4, 3.4, 4.7 \} \end{align}$$

In order to calculate the standard deviation (and to have it mean something) we need the mean of those measurements, $\bar x$. That's found by adding them up and dividing by the number of measurements, N = 10. In summation notation that looks like:

$$\bar{x} = \frac{1}{10} \sum_{n = 1}^{10} x_n = 2.88$$

It's a good practice to keep some extra digits while doing the calculation, and pare them back to match the number of significant digits in the data once the calculation is complete. That way we avoid some round-off error. Now recall that the variance (the square of $\sigma$) is

$$\sigma^2 = \frac{1}{n} \sum_{n = 1}^N (x_n - \bar{x})^2$$

To help us do the calculation, it's convenient to make a table:

What about when $N$ is small?

We must accept that when the number of measurements is small, both an average and its standard deviation has a diminished meaning. If, for example, I want to measure the average height of American females, and I do that by measuring the heights of two women and averaging the results, I hope you wouldn't take that seriously as the average height of ALL women in America. The same is true of the standard deviations calculated from those two data sets.

When the number of measurements is small OR when the sample does not represent an entire population, we customarily divide the sum of squares of $x_n - \bar x$ not by $N$, but by $N-1$

The so-called sample variance, $\sigma^2$ is

$$\sigma^2 = \frac{1}{N - 1} \sum_{n = 1}^N (x_n - \bar{x})^2$$

Example 2

Find the mean and standard deviation of 20 test scores

Let's say we gave a test and the twenty resulting scores formed the set:

$$ \begin{align} x = \{ &90, 98, 86, 91, 73, \\ &73, 85, 81, 79, 74, \\ &75, 88, 74, 97, 77, \\ &90, 85, 83, 98, 80 \} \end{align}$$

In order to analyze these, it's best to construct a table something like this:

Standard deviations and the Gaussian distribution

When the Gaussian or normal probability distribution is divided into standard deviations, we find that the total probability enclosed between $\pm \sigma$ is about 68%, as shown in the graph. That is, we would expect 68% of all measurements approximating the mean to lie between $\pm \sigma$.

Likewise, two standard deviations enclose about 95% of the total probability, and three sigmas enclose about 99.7% of the total probability. In other words, it's very unlikely to find a measurement greater than three standard deviations from the mean or less than $3 \sigma$ from it.

These results come from calculus, as the Gaussian function is an integral-defined function.

The standard deviation, your calculator and spreadsheets

For the most part, standard deviations aren't calculated by hand as they've been in these examples. Calculators often have a statistics mode that can be used to calculate means, standard deviations and other properties of data sets.

Practice problems

Calculate the mean and sample standard deviation of the mean for the following data sets:

Some Python code

Here is some simple Python code that will calculate the standard deviation (both population and sample) of a data set stored in the list variable x[ ]. This little program is a great one to write and enhance with different data-analysis features along the way. Soon you'll have a comprehensive 1-dimensional data set analysis tool.

        __author__='Doc'
        #!/usr/local/bin/phython
        #include "Python.h"
        
        import math  #For the square-root function
        
        x = [88.1, 92.3, 90.9, 97.2, 68.9, 55.3, 94.2, 82.3, 88.7, 94.0,
             84.1, 85.3, 80.9, 79.2, 78.9, 20.0, 85.3, 70.7, 72.9, 89.6,
             90.2, 93.4, 71.1, 77.8, 65.8, 87.4, 88.3, 92.9, 89.0, 70.6,
             58.2, 95.0, 82.3, 79.4, 90.2, 85.5, 80.3, 81.4, 72.7, 84.0]
             
        nData = len(x)
        sum = 0.0
        
        for i in range (0, nData):   #Calculate sum
            sum += x[i]
            
        mean = sum/nData
        sum = 0.0
        
        for i in range (0, nData):
            sum += (x[i] - mean)**2  #Calculate sum squares of differences
            
        variance_sample = sum/(nData - 1)
        variance_pop = sum / nData
            
        standardDeviation_sample = math.sqrt(variance_sample)
        stardardDeviation_pop = math.sqrt(variance_pop)
        
        print('mean    = %8.2f +/i %8.2f (sample)' %
               mean, standardDeviation_sample))
        print('mean    = %8.2f +/i %8.2f (population)' %
               mean, standardDeviation_pop))
        print('variance = %8.2f' % (variance))
        

The output of this program would be:

            mean     = 80.61 +/- 13.80 (sample)
            mean     = 80.61 +/- 13.63 (population)
            variance = 190.5