Optimizing with constraints

As a first example, here's a simple min/max problem you've probably encountered before. Let's say that we want to have a rectangle with a perimeter no larger than 100 units. What are the dimensions of the largest such rectangle we could make with this constraint? The question consists of a function for which we need to find a maximum — the area function, and a constraint — a limit on the perimeter.

First, the relevant equations are for the area:

$$A = xy$$

And our constraint, which we'll call $g(x,y)$, is

$$g(x,y) = 2x + 2y \le 100$$

This isn't a difficult problem to solve without any calculus. First solve the constraint equation for $y$:

$$ \begin{align} 2x + 2y &= 100 \\[5pt] x + y &= 50 \\[5pt] y &= 50-x \end{align}$$

Now plugging that into the area equation, we get the area as a function of the single variable $x$:

$$A(x) = x(50-x) = -x^2 + 50x$$

This is the equation of a downward-opening parabola of the form $y = ax^2 + bx + c$ which has a maximum value at $-b/2a$, where $a = -1$ and $b = 50$, so the $x$-coordinate of that vertex is

$$\frac{-b}{2a} = \frac{-50}{2(-1)} = 25$$

So our optimum dimensions are $25 \times 25$ units². That matches our intuition that a square is the rectangular figure that encloses the maximum area for a given perimeter.

Now let's do it with the Lagrange Multiplier method. If we look at the area equation as a function of $x$ and $y$, we can plot a family of level curves that form a contour plot. Take a look:

Example 1

Suppose that the maximum size of a package to be mailed is calculated by adding the girth (perimeter of the rectangle formed by the two smallest dimensions) to the length (longest dimension) of the box. What are the dimensions of the largest package that could be mailed if the girth-plus-length measure must be less than or equal to 108 inches.

Solution: If we let the dimensions of our box be $x, y$ and $z$, then the volume is

$$V = xyz$$

The constraint, which we'll call $g(x,y,z)$, will be

$$2x + 2y + z \le 108$$

We'll just solve for the maximum dimensions, the equality.

The gradient of the volume is

$$\nabla V = (yz,xz,xy)$$

and the gradient of the constraint is $\nabla g = (2, 2, 1)$. This gives us four equations from which to puzzle out the maximum of $(x,y,z)$:

Example 2

Suppose we want to design a cylindrical can with top and bottom to contain food. We'd like to use the minimum of material (metal) to enclose a given volume. Use the method of Lagrange Multipliers to show that the minimum surface area of the can occurs when its diameter is the same as its height.

Solution: The area of the cylinder consists of the area of two disks and the area of the rectangle that wraps around to form the side of the cylinder:

$$A(r,h) = 2\pi r^2 + 2 \pi rh$$

The constraint on the area is that the volume, $V = \pi r^2 h$, is constant ($V$).

The relevant gradients are

$$ \begin{align} \nabla A &= (4 \pi r + 2 \pi h, 2 \pi r) \\[5pt] \nabla V &= (2 \pi rh, \pi r^2) \end{align}$$

Example 3

Here's a more abstract example where we'll consider just the graphs of our function and constraint. The function is

$$f(x,y) = 9+x^2-y^2$$

And the constraint is the projection of the circle $x^2 + y^2 = 4$ on that function, the red curve shown below.

We'll write out constraint as $g(x,y) = x^2 - y^2 - 4 = 0$. Our relevant gradients are

$$ \begin{align} \nabla f(x,y) &= (2x, -2y) \\[5pt] \nabla g(x,y) &= (2x, 2y) \end{align}$$

These yield three equations in $x, y$ and $\lambda$,

$$ \begin{align} 2x &= \lambda 2x \tag{1} \\[5pt] -2y &= \lambda 2y \tag{2} \\[5pt] x^2 + y^2 &= 4 \tag{3} \end{align}$$

Example 4

Find the maximum volume of a right circular cone inscribed in a sphere.

Solution: Here's the setup. We take a sphere of radius $R$ and inscribe a cone with bottom radius $r$ and height $h$, as shown.

The function we're trying to maximize is the volume of the cone,

$$V(r,h) = \frac{1}{3} \pi r^2 h$$

The constraint is that the cone be inside the sphere, which has volume

$$g(R) = \frac{4}{3} \pi R^3$$

we need to get these two functions to have the same set of variables. We can use the small triangle in the figure and the Pythagorean theorem to do that:

$$ \begin{align} R^2 &= (h-R)^2 + r^2 \\[5pt] \cancel{R^2} &= h^2 - 2hR + R^2 + r^2 \\[5pt] 2hR &= h^2 + r^2 \\[5pt] R &= \frac{h^2 + r^2}{2h} \end{align}$$

So we can convert $g(R)$ to $g(r,h)$:

$$g(r,h) = \frac{4}{3} \pi \left( \frac{h^2+r^2}{2h} \right)^3$$

Now we can find our gradients:

$$\nabla V = \left( \frac{2}{3}\pi r h, \frac{1}{3} \pi r^2 \right)$$

The gradient of the constraint is a little tougher:

$$ \begin{align} \frac{\partial g}{\partial r} &= 4 \pi \left( \frac{h^2+r^2}{2h} \right)^2 \frac{r}{h} \\[5pt] \frac{\partial g}{\partial h} &= 4\pi \left( \frac{h^2+r^2}{2h} \right)^2 \cdot \frac{2h(2h)-(h^2+r^2)(2)}{(2h)^2} \\[5pt] &= 2 \pi \left( \frac{h^2+r^2}{2h} \right) \left( \frac{h^2-r^2}{2h^2} \right) \\[5pt] \nabla g &= \left( 4 \pi \left( \frac{h^2+r^2}{2h} \right)^2 \frac{r}{h}, \; 2 \pi \left( \frac{h^2+r^2}{2h} \right) \cdot \frac{h^2-r^2}{2h^2} \right) \end{align}$$

Practice problems

1. Find the critical points of $f(x,y) = x+3y$ on the curve $x^2 + y^2=5$.

   Solution

   
   

   We'll call the constraint equation $g(x,y)$. The relevant gradients are

   $$ \begin{align} \nabla f &= (1, 3) \\[5pt] \nabla g &= (2x, 2y) \end{align}$$

   Our set of coupled equations is

   $$ \begin{align} 1 &= 2 \lambda x \tag{1}\\[5pt] 3 &= 2 \lambda y \tag{2}\\[5pt] x^2 + y^2 &= 5 \tag{3} \end{align}$$

   From equations (1) and (2) we have $2 \lambda = \frac{1}{x} = \frac{3}{y}$, so $y = 3x$. Plugging this into equation (3) gives

   $$ \begin{align} x^2 + 9x^2 &= 5 \\[5pt] 10 x^2 &= 5 \\[5pt] x^2 &= \frac{1}{2} \; \rightarrow \; x = \pm \frac{1}{\sqrt{2}} \end{align}$$

   Then we have

   $$y = \pm \frac{3}{\sqrt{2}}$$

   
   

   So we have four candidate critical points:

   $$\left( \frac{1}{\sqrt{2}}, \frac{3}{\sqrt{2}} \right), \left( \frac{1}{\sqrt{2}}, -\frac{3}{\sqrt{2}} \right), \\ \left( -\frac{1}{\sqrt{2}}, \frac{3}{\sqrt{2}} \right), \left( -\frac{1}{\sqrt{2}}, -\frac{3}{\sqrt{2}} \right)$$

   Let's calculate the value of the function (a plane) at these points:

   $$ \begin{align} f\left( \frac{1}{\sqrt{2}}, \frac{3}{\sqrt{2}} \right) &= \frac{10}{\sqrt{2}} \phantom{00} \text{max}\\[5pt] f\left( \frac{1}{\sqrt{2}}, -\frac{3}{\sqrt{2}} \right) &= \frac{-8}{\sqrt{2}}\\[5pt] f\left( -\frac{1}{\sqrt{2}}, \frac{3}{\sqrt{2}} \right) &= \frac{8}{\sqrt{2}}\\[5pt] f\left( -\frac{1}{\sqrt{2}}, -\frac{3}{\sqrt{2}} \right) &= \frac{-10}{\sqrt{2}} \phantom{00} \text{min} \end{align}$$




2. Find the critical points of $f(x,y,z) = 2x - 2y - 3z$ on the surface $z = x^2+y^2$.

   Solution

   
   

   We'll call the constraint equation $g(x,y,z) = x^2 + y^2 - z$. The relevant gradients are

   $$ \begin{align} \nabla f &= (2,-2,-3) \\[5pt] \nabla g &= (2x, 2y, -1) \end{align}$$

   Our set of coupled equations is

   $$ \begin{align} 2 &= 2 \lambda x \tag{1}\\[5pt] -2 &= 2 \lambda y \tag{2}\\[5pt] -3 &= - \lambda z \tag{3}\\[5pt] z &= x^2 + y^2 \tag{4} \end{align}$$

   If we solve equations 1, 2 and 3 for $\lambda$ and equating the resulting expressions for $x, y$ and $z$, we get

   $$\frac{1}{x} = \frac{-1}{y} = \frac{3}{z}$$

   We can solve for $x$ and $y$ in terms of $z$:

   $$x = \frac{3}{z} \phantom{0000} y = \frac{-z}{3}$$

   
   

   and plug those into equation (4) to find $z$:

   $$ \require{cancel} \begin{align} z &= x^2 + y^2 \\[5pt] &= \frac{z^2}{9} + \frac{z^2}{9} \\[5pt] \cancel{z} &= \frac{2z^{\cancel{2}}}{9} \; \rightarrow \; z = \frac{9}{2} \end{align}$$

   Then it's easy to find $x$ and $y$ from our equations above:

   $$ \begin{align} x &= \frac{\frac{9}{2}}{3} = \frac{9}{6} = \frac{3}{2} \\[5pt] y &= \frac{\frac{-9}{2}}{3} = \frac{-9}{6} = \frac{-3}{2} \end{align}$$

   Now the value of our function at the critical point

   $$(x, y, z) = \left( \frac{3}{2}, \frac{-3}{2}, \frac{9}{2} \right)$$

   is

   $$f \left( \frac{3}{2}, \frac{-3}{2}, \frac{9}{2} \right) = \frac{9}{4} + \frac{9}{4} - \frac{18}{4} = 0$$




3. Find the critical points of $f(x,y,z) = e^{xyz}$ on the surface $x^2 + 2y^2 + z^2 = 4$.

   Solution

   
   

   The relevant gradients are

   $$ \begin{align} \nabla f &= \left( yz e^{xyz}, xz e^{xyz}, xy e^{xyz} \right) \\[5pt] \nabla g &= (2x, 4y, 2z), \end{align}$$

   where we're calling $g$ the function name of the constraint. Our four equations relating $x, y, z$ and $\lambda$ are

   $$ \begin{align} yz e^{xyz} &= 2 \lambda x \; \rightarrow \; \lambda = \frac{yz}{2x} e^{xyz} \\[5pt] yz e^{xyz} &= 2 \lambda x \; \rightarrow \; \lambda = \frac{xz}{4y} e^{xyz} \\[5pt] yz e^{xyz} &= 2 \lambda x \; \rightarrow \; \lambda = \frac{xy}{2z} e^{xyz} \\[5pt] x^2 &+ 2y^2 + z^2 - 4 = 0 \end{align}$$

   Now we can equate terms:

   $$\frac{yz}{2x} e^{xyz} = \frac{xz}{4y} e^{xyz} = \frac{xy}{2z} e^{xyz}$$

   Now we can solve for $x$ and $y$ in terms of $z$.

   $$ \begin{align} \frac{yz}{2x} \cancel{e^{xyz}} &= \frac{xz}{4y} \cancel{e^{xyz}} \phantom{00} \frac{xz}{4y} \cancel{e^{xyz}} = \frac{xy}{2z} \cancel{e^{xyz}} \\[5pt] \cancel{2} \cancel{y} z^2 &= \cancel{2} x^2 \cancel{y} \phantom{00000} 2 \cancel{y} z^2 = 4 \cancel{x}y^2 \\[5pt] z^2 &= x^2 \phantom{000000000000} z^2 = 2 y^2 \\[5pt] x &= z \phantom{00000000000000} y = \frac{z}{\sqrt{2}} \end{align}$$

   
   

   Then we insert these versions of $x$ and $y$ into the fourth equation:

   $$ \begin{align} x^2 + 2y^2 + z^2 &= 4 \\[5pt] z^2 + \frac{\cancel{2}z^2}{\cancel{2}} + z^2 &= 4 \\[5pt] 3z^2 &= 4 \\[5pt] z &= \frac{2}{\sqrt{3}} \end{align}$$

   This we have

   $$x = \frac{2}{\sqrt{3}}, \; y = \frac{2}{\sqrt{6}}, \; z = \frac{2}{\sqrt{3}}$$

   Finally,

   $$f\left( \frac{2}{\sqrt{3}}, \frac{2}{\sqrt{6}}, \frac{2}{\sqrt{3}} \right) = e^{\frac{8}{3 \sqrt{6}}}$$




4. Find the lowest temperature on the plane $x+y+z=1$ if the temperature in a region of space surrounding the plane is $T = x^2 + y^2 + z^2$.

   Solution

   
   

   The relevant gradients are

   $$ \begin{align} \nabla T = (2x, 2y, 2z) \\[5pt] \nabla g = (1, 1, 1) \end{align}$$

   Our equations relating $x, y, z$ and $\lambda$ are

   $$ \begin{align} 2x &= \lambda \\[5pt] 2y &= \lambda \\[5pt] 2z &= \lambda \\[5pt] x+y+z &= 1 \end{align}$$

   
   

   $$2x = 2y = 2z$$

   $$x = y = z$$

   
   

   Now if $x = y = z$ and $x+y+z=1$, then $x = y = z = \frac{1}{3}$.

   Finally,

   $$T \left( \frac{1}{3},\frac{1}{3},\frac{1}{3} \right) = \frac{1}{9} + \frac{1}{9} + \frac{1}{9} = \frac{1}{3}$$




5. A rectangular sheet of aluminium of a fixed perimeter, $p$, is rolled into a cylinder. What dimensions give the maximum volume enclosed by the cylinder?

   Solution

   
   

   

   The dimensions of the rectangle are the circumference of the cylinder multplied by its height. The perimeter is $P = 4 \pi r + 2h$. The gradients are

   $$ \begin{align} \nabla V &= (2 \pi rh, \pi r^2) \\[5pt] \nabla g &= (4 \pi, 2) \end{align}$$

   We solve for $r$ and $h$ using

   $$ \begin{align} 2 \pi rh &= 4 ]\lambda \pi \tag{1} \\[5pt] \pi r^2 &= 2 \lambda \tag{2} \\[5pt] 4 \pi r + 2h &= P \tag{3} \end{align}$$

   
   

   From equations (1) and (2) we get

   $$\lambda = \frac{rh}{2} \phantom{000} \lambda = \frac{\pi r^2}{2}$$

   If we equate these, eliminating $\lambda$, we get

   $$h = \pi r$$

   Plugging into the constraint equation gives

   $P = 6 \pi r$, so $h = \frac{P}{6}$ and $r = \frac{P}{6\pi}$.

   The maximum volume of our cyclinder, in terms of $P$, is

   $$V = \pi \left( \frac{P}{6 \pi} \right)^2 \left( \frac{P}{6} \right) = \frac{\pi P^3}{36}$$




6. What point on the surface $\frac{1}{x}+\frac{1}{y}+\frac{1}{z}=1$ is closest to the origin?

   Solution

   
   

   In this problem, the function we need to minimize is the distance from our point — let's call it $P = (x,y,z)$ — to the origin. For that we'll use the distance formula:

   $$ \begin{align} d &= \sqrt{(x-x_o)^2 + (y-y_o)^2 + (z-z_o)^2} \\[5pt] &= \sqrt{(x-0)^2 + (y-0)^2 + (z-0)^2} \\[5pt] &= \sqrt{x^2 + y^2 + z^2} \end{align}$$

   The gradients are:

   $$ \begin{align} \nabla f &= \bigg( \frac{1}{\cancel{2}} \sqrt{x^2+y^2+z^2}(\cancel{2}x) ,\\[5pt] &\frac{1}{\cancel{2}} \sqrt{x^2+y^2+z^2}(\cancel{2}y), \\[5pt] &\frac{1}{\cancel{2}} \sqrt{x^2+y^2+z^2}(\cancel{2}z) \bigg) \\[5pt] &= \sqrt{x^2+y^2+z^2}\, (x,y,z) \\[5pt] \nabla g &= \bigg( \frac{-1}{x^2}, \frac{-1}{y^2}, \frac{-1}{z^2} \bigg) \end{align}$$

   
   

   Our Lagrange equations are

   $$ \begin{align} x \sqrt{x^2+y^2+z^2} &= \frac{-\lambda}{x^2} \; \rightarrow \; \lambda = x^3 \sqrt{x^2+y^2+z^2} \\[5pt] y \sqrt{x^2+y^2+z^2} &= \frac{-\lambda}{y^2} \; \rightarrow \; \lambda = y^3 \sqrt{x^2+y^2+z^2} \\[5pt] z \sqrt{x^2+y^2+z^2} &= \frac{-\lambda}{z^2} \; \rightarrow \; \lambda = z^3 \sqrt{x^2+y^2+z^2} \\[5pt] \frac{1}{x}&+\frac{1}{y}+\frac{1}{z}=1 \end{align}$$

   Setting all of these terms equal and dividing out the square roots gives

   $$x^3 = y^3 = z^3 \; \rightarrow \; x = y = z$$

   Combining this result with the constraint gives $x=y=z=3$. The minimum distance is therefore

   $$d = \bigg( 3 \cdot \frac{1}{9} \bigg)^{\frac{1}{2}} = \sqrt{\frac{3}{9}} = \frac{\sqrt{3}}{3}$$