xaktly | Matrix and Vector Algebra

Dot Product

The angle between vectors

The dot product (also called the scalar product) gives us the angle between any two vectors. It's one of the most important relationships between vectors. In this section we'll define the dot product and show how it gives the angle between vectors for two- and three-dimensional vectors.

The dot product gives the angle between two vectors of any dimension. It's a tricky concept, but it's true. You can calculate the angle between 27-dimensional vectors.

Here's the definition of the dot product for two-dimensional vectors like v1 = (x1, y1):

Dot product of 2-D vectors

$$ \begin{align} \vec v_1 &= (x_1, y_1) \\[3pt] \vec v_2 &= (x_2, y_2) \\[5pt] \vec v_1 \cdot \vec v_2 &= x_1 x_2 + y_1 y_2 \end{align}$$

Notice the the result of the dot product is not another vector – it's just another number or a scalar. That's why we often call the dot product the scalar product. We can easily extend the definition of the dot product to vectors of any dimension, as long as the two have the same dimension.

Dot product of 2-D vectors

$$ \begin{align} \vec v_1 &= (a_1, a_2, \dots , a_n) \\[3pt] \vec v_2 &= (b_1, b_2, \dots , b_n) \\[5pt] \vec v_1 \cdot \vec v_2 &= a_1 b_1 + a_2 b_2 + \dots + a_n b_n \end{align}$$

Finally, the dot product can be thought of as a row vector (or a 1 × n matrix) multiplied by a column vector (an n × 1 matrix). Here's a picture of that for 2-D vectors:

$$ \begin{align} v_1 &\cdot v_2 = (x_1, y_1)\left( \begin{matrix} x_2 \\ y_2 \end{matrix} \right) = x_1 x_2 + y_1 y_2 \\[5pt] &\phantom{0000000} \color{#E90F89}{(1 \times 2)(2 \times 1) = 1 \times 1 \; \text{(scalar)}} \end{align}$$

The dot product is defined only for vectors of the same dimension.

Practice problems

Calculate the dot product of these vectors. Roll over or tap each problem for the solution.

You can think of matrix multiplication like taking lots of dot products

If matrix multiplication gets the best of you, it might help to think of it like taking a bunch of dot products, row vectors from the left matrix with column vectors with the right-hand matrix. Here's a 3-D matrix multiplication problem:

We'll think of the matrix on the left as a vertical list of row vectors (a column vector of row vectors ... but that's confusing), called a, b and c. The matrix on the right is a horizontal list of column vectors, x, y and z.

Now if we multiply the two matrices, we're just taking a series of dot products like this:

Now take a look at that matrix. The element in the second row, second column is just the dot product of the second row vector, b, and the second column vector, y. It's the same for each element in the result matrix. Here is the expanded result:

The connection between the dot product and the angle between vectors

Consider two vectors, a and b. Here they are:

The first thing we'll do is to translate them to the origin to make things easier. Remember that we can always do this because the only things about vectors that are important are their length and direction.

Now if we remember vector subtraction, we can construct the vector a - b. Let's also draw in angle θ between a and b:

We begin with the law of cosines, where |v1| is the length of vector v1, |v2| is the length of vector v2, and so on:

$$|v_1 - v_2| = |v_1|^2 + |v_2|^2 - 2|v_1||v_2| cos(\theta)$$

Now let's take a look at that left side, |v1 - v2|2. We can use the coordinates of the endpoints of v1 and v2, which are the coordinates of both ends of v1 = v2 to get

$$|v_1 - v_2|^2 = \left(\sqrt{(x_2 - x_1)^2 + (y_2 - y_1)^2}\right)^2$$

We're just using the distance formula with those coordinates. Now on the right we have the square of a square root, so the left side is just

$$= (x_2 - x_1)^2 + (y_2 - y_1)^2$$

Now for the right side of our original equation. If we remember that the square of the length of a vector is

$$|v_1|^2 = \left(\sqrt{x_1^2 + y_1^2} \right)^2$$

we get

$$= x_1^2 + y_1^2$$

So we have |v1|2 = x12 + y12 and |v2|2 = x22 + y22. Now we can rewrite both sides, and expand the binomials on the left to get:

$$ \begin{align} (x_2 &- x_1)^2 + (y_2 - y_1)^2 \\ &= x_1^2 + y_1^2 + x_2^2 + y_2^2 - 2|v_1||v_2| cos(\theta) \end{align}$$

Notice that the squared terms containing purely x's or y's are represented on both sides of the equation, so we can eliminate them:

$$ \begin{align} x_2^2 &- 2x_1x_2 + x_1^2 + y_2^2 - 2y_1y_2 + y_1^2 \\ &= x_1^2 + y_1^2 + x_2^2 + y_2^2 - 2|v_1||v_2| cos(\theta) \end{align}$$

Finally, lots of those terms subtract to zero, and all of the remaining -2's can be divided away

$$-2x_1x_2 - 2y_1y_2 = -2|v_1||v_2| cos(\theta)$$

to give

$$x_1x_2 + y_1y_2 = |v_1||v_2| cos(\theta)$$

Now all that remains is to recognize that x1x2 + y1y2 is just the definition of the dot product, so we've arrived at the relationship between the dot product of two vectors and the angle between them:

$$v_1 \cdot v_2 = |v_1||v_2| cos(\theta)$$

Properties of the dot product

Here are some key properties of the dot product that will come in handy when doing calculations. They can all be proved fairly easily using either the basic definition of the dot product, $\vec{a} \cdot \vec{b} = a_1 b_1 + a_2 b_2 + \dots + a_n + b_n,$ or the angle definition, $\vec{a} \cdot \vec{b} = |\vec{a}||\vec{b}| cos(\theta).$

Property Explanation
$\vec{a} \cdot \vec{a} = |\vec{a}|^2$ The dot product of a vector with itself is equal to the square of its length.
$\vec{0} \cdot \vec{a} = 0$ The dot product of a vector with the zero vector, $\vec{0} = (0, 0, \dots \ 0),$ is zero because the length of the zero vector is zero.
$\vec{a} \cdot \vec{b} = \vec{b} \cdot \vec{a}$ The dot product is commutative because multiplication is commutative.
$\vec{a} \cdot (\vec{b} + \vec{c}) = \vec{a} \cdot \vec{b} + \vec{a} \cdot \vec{c}$ The dot product is distributive.
$(k \vec{a}) \cdot \vec{b} = k(\vec{a} \cdot \vec{b})$ (k is a constant) Scalar multiplication and the dot product are commutative and associative.

Example 1

Find the angle between vectors $v_1 = (-1, 2)$ and $v_2 = (2, 5)$

Solution: We begin by writing the dot-product equation and rearranging it to find the angle:

$$v_1 \cdot v_2 = |v_1||v_2| cos(\theta)$$

Solving for the angle, we get

$$\theta = cos^{-1} \left( \frac{v_1 \cdot v_2}{|v_1||v_2|} \right)$$

Now we just have to fill in the information. The dot product is the product of the x-coordinates plus the product of the y's:

$$v_1 \cdot v_2 = (-1)(2) + 2(5) = 8$$

The lengths of the vectors are found using the distance formula:

$$ \begin{align} |v_1| &= ((-1)^2 + 2^2)^{1/2} = \sqrt{5} \\ |v_2| &= (2^2 + 5^2)^{1/2} = sqrt{29} \end{align}$$

And finally the angle:

$$\theta = cos^{-1}\left( \frac{8}{\sqrt{5\cdot 29}} \right) = 48.4˚$$

Example 2

Find a vector of length 1 that is perpendicular to $(1, 0, 1).$

Solution: This kind of problem will have two main steps. First we'll find one vector that is perpendicular to (1, 0, 1), but we'll have to remember that there are an infinite number of such vectors – all of different length. The second step will be to adjust the length of that vector to one – to make it a unit vector. Here's a picture of this 3-dimensional vector, which lives in the x-z plane:

Recall that the angle between perpendicular vectors (one known and one unknown in this case) has to be 90˚, and that the cosine of 90˚ is zero, so we'll start there:

$$cos(\theta) = \left( \frac{v_1 \cdot v_2}{|v_1||v_2|} \right)$$

We can get rid of the denominator because for this expression to equal zero, all that is required is that the dot product be zero:

$$\frac{v_1 \cdot v_2}{|v_1||v_2|} = 0$$

Now the dot product between v = (1, 0, 1) and an unknown vector (x, y, z) of the same dimension is:

$$ \begin{align} 1(x) + 0(y) + 1(z) &= 0 \\ x + z &= 0 \\ x &= -z \end{align}$$

So all that is required of this vector is that y = 0 and x = -z.

$$v_{\perp} = (1, \, 0, \, -1)$$

The vectors (3, 0, -3) and (4.5, 0, 4.5) would work just as well. They differ only in length, but they're perpendicular to (1, 0, 1). What we need to do now is to adjust the length of (1, 0, 1) to 1. First we need to find its current length:

$$|(1, \;, 0, \; -1)| = (1^2 + 0^2 + (-1)^2)^{1/2} = \sqrt{2}$$

Now to adjust the length we just divide each component of our vector by that length. You can check for yourself that the length of this new vector is 1.

$$v_{\perp} = \left( \frac{1}{\sqrt{2}}, \, 0, \, \frac{-1}{\sqrt{2}} \right)$$

The process of adjusting the length of a vector to 1, or making it a unit vector, is called normalization.

Finally, if (a, 0, -a) is perpendicular to (1, 0, 1), then so is (-a, 0, a). It also satisfies the requirement that x = -z, it just points in the opposite direction.

The diagram above shows our two vectors. They lie in the x-z plane. Notice that there are many other vectors perpendicular to (1, 0, 1) outside of that plane.

Normalization of a vector

To normalize a vector (set its length to one), calculate the length of the vector, then divide each component of the vector by that length.

Example 3

Find a vector of length 1 orthogonal to vectors $v_1 = (1, -1, 2)$ and $v_2 = (-1, 2, 2).$

Solution: First, the word orthogonal replaces perpendicular when we're working in dimensions higher than three, and it's often used with three. It means the same thing: the angle between orthogonal vectors is 90˚, but we use a different word because it's difficult to visualize a 90˚ angle in higher dimensions.

Our vectors are

$$ \begin{align} v_1 &= (1, \, -1, \, 2) \\ v_2 &= (-1, \, 2, \, 2) \end{align}$$

Here's a 3-D plot of these. The difference between this example and the last is that these vectors don't lie in one of the coordinate planes, so we'll have to approach finding orthogonal vectors differently.

Any two vectors can define a plane, as ours do. We seek a vector that's orthogonal to both, so we can write these two dot products:

$$ \begin{align} (1, \, -1, \, 2) \cdot (a, \, b, \, c) &= 0 \\ a - b + 2c &= 0 \end{align}$$


$$ \begin{align} (1, \, 2, \, 2) \cdot (a, \, b, \, c) &= 0 \\ -a + 2b + 2c &= 0 \end{align}$$

If we add those two equations we can eliminate a to find a relationship between b and c

Now we have b = -4c, and we can plug that into either dot-product result to find a = -6c.

$$ \begin{align} b &= -4c \\ a &= -6c \end{align}$$

If we let c = 1, we have one of many versions of our desired vector, differing in length.

$$(a, \, b, \, c) = (-6, \, -4, \, 1) \; or \; (6, \, 4, \, -1)$$

Notice that there are actually two vectors perpendicular to both v1 and v2, the negatives of each other.

Now we need to normalize our vectors, first by finding the length:

$$((-6)^2 + (-4)^2 + 1^2)^{1/2} = \sqrt{53}$$

... then by dividing by that length to find two vectors in the same directions, but of length 1:

Finally, here is our picture again with the two orthogonal unit vectors sketched in.

Practice problems

For each problem calculate the unit vector orthogonal to each of the vectors given. Remember that a unit vector has a length of 1.

1. $(2, -3)$
2. $(1, 1, -1)$ and $(1, -1, 1)$
3. $(1, 2, -1)$ and $(-1, -1, 0)$
4. $(2, -2, 1)$ and $(1, 0, 0)$
5. (1, -1, -2, 1), (-2, -1, 1, 2), (2, 3, 4, 1)
6. (2, 1, -2, -1), (1, 1, -1, 2), (0, 1, 1, 1)

Calculate the angle (in degrees) between these pairs of vectors.

7. $(1, 2), \; (-1, 3)$
8. $(1, -1), \; (-1, 1)$
9. $(1, 2, -1), \; (2, 2, -3)$
10. $(1, 1, 2, 1), \; (-2, 1, -1, 1)$

Creative Commons License   optimized for firefox
xaktly.com by Dr. Jeff Cruzan is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. © 2012, Jeff Cruzan. All text and images on this website not specifically attributed to another source were created by me and I reserve all rights as to their use. Any opinions expressed on this website are entirely mine, and do not necessarily reflect the views of any of my employers. Please feel free to send any questions or comments to jeff.cruzan@verizon.net.