Calculus of Variations Part 1: Establishing the Basis

Calculus of variations is an extremely useful and amazing tool in physics, math, computer science, and a variety of fields. Similar to how regular calculus is focused around functions and differentials, this field focuses on functionals and variations. A functional $F[y]$ takes in a function and spits out a number. The following are examples of functionals.

$S[y]=\int_a^b\sqrt{1+{y'}^2}\textup{d}x$ (arc length of $y$ from $a$ to $b$ )

$N[\phi]=\frac{1}{\int_{-\infty}^\infty\phi\textup{d}x}$ (normalization factor for $\phi$ i.e $\int_{-\infty}^\infty N[\phi]\phi\textup{d}x=1$ which is useful in quantum)

$P[f]=f(\pi)$ (value of $f$ at $x=\pi$ )

In some sense, this is just notational and should not be thought about too intensely.

A very important result of calculus is the ability to find minimums and maximums or more generally extrema. Now consider the differential of a function in regular calculus $\textup{d}f=\lim_{h\rightarrow0}f(x+h)-f(x)$ . The corresponding idea in calculus of variations is the variation of a functional denoted and defined $\delta F[h]=F[y+h]-F[y]$ . $h$ and $y$ here are functions. There exists an extremum for a function if $\textup{d}f=0$ and analogously a functional if $\delta F[h]=0$ . However, finding a function that minimizes a functional is a lot less straightforward than finding a number in a range. One cannot just systematically test all possibilities. In this case, it helps to talk about specific forms of functionals. The following is a VERY common functional.

$F[y]=\int_a^b f(x,y,y')\textup{d}x$

$y(a)=A, y(b)=B$

The most common is arc length where $f=\sqrt{1+y'^2}$

Let’s look at the variation of this functional.

$\delta F[h]=\int_a^b f(x,y+h,y'+h')-f(x,y,y')\textup{d}x$

Here, $h(a)=h(b)=0$ in order to preserve the boundary conditions for $y(x)$ . We know that when $h$ is small, the following is true.

$\int_a^b f(x,y+h,y'+h')-f(x,y,y')\textup{d}x=\int_a^b f_y(x,y,y')h+f_{y'}(x,y,y')h'\textup{d}x$

Here, we will use the following theorem (the proof is at the bottom of the post).

Theorem: Let $h(x)$ be any continuous function such that $h(a)=h(b)=0$ . Assume then that, for all possible $h(x)$ , the following is true.

$\int_a^b \alpha(x)h(x)+\beta(x)h'(x)\textup{d}x=0$

This implies $\beta'(x)=\alpha(x)$ . (Highly recommend looking at the proof of this on the bottom before continuing).

Going back to our original integral, we see that $\alpha(x)=f_y$ and $\beta(x)=f_{y'}$ . This means $\frac{\mathrm{d} }{\mathrm{d} x}f_{y'}=f_y$ . This is more formally written below and is called Euler’s equation.

$\frac{\partial f}{\partial y}-\frac{\mathrm{d} }{\mathrm{d} x}\frac{\partial f}{\partial y'}=0$

However, sometimes not all variables are used so there exists 3 other forms of this.

If $f$ does not depend on $y$ , then $\frac{\partial f}{\partial y}=0$ so Euler’s equation reduces to the following.

$\frac{\mathrm{d} }{\mathrm{d} x}\frac{\partial f}{\partial y'}=0 \textup{ (}y\textup{-independent)}$

If $f$ does not depend on $y'$ , then $\frac{\partial f}{\partial y'}=0$ so Euler’s equation becomes

$\frac{\partial f}{\partial y}=0\textup{ (}y'\textup{-independent)}$

If $f$ does not depend on $x$ , then we have to use some clever tricks to come to a simpler equation because $y$ still depends on $x$ . Let’s multiply the left side of the equation by $y'$ which doesn’t change the value because it equals 0 anyway (I alternate between prime and differential notation and put terms in weird places but it will become clear why later).

$\frac{\partial f}{\partial y}-\frac{\mathrm{d} }{\mathrm{d} x}\frac{\partial f}{\partial y'}=\frac{\partial f}{\partial y}\frac{\partial y}{\partial x}-y'\frac{\mathrm{d} }{\mathrm{d} x}\frac{\partial F}{\partial y'}$

From here, we make a simple change to the equation.

$\frac{\partial f}{\partial y}\frac{\partial y}{\partial x}-y'\frac{\mathrm{d} }{\mathrm{d} x}\frac{\partial f}{\partial y'}=\frac{\partial f}{\partial y}\frac{\partial y}{\partial x}+\frac{\partial f}{\partial y'}\frac{\partial y'}{\partial x}-\frac{\partial f}{\partial y'}\frac{\partial y'}{\partial x}-y'\frac{\mathrm{d} }{\mathrm{d} x}\frac{\partial f}{\partial y'}$

Because we know $x$ has no influence on $f$ , the first two terms form a chain rule.

$\frac{\partial F}{\partial y}\frac{\partial y}{\partial x}+\frac{\partial F}{\partial y'}\frac{\partial y'}{\partial x}-\frac{\partial F}{\partial y'}\frac{\partial y'}{\partial x}-y'\frac{\mathrm{d} }{\mathrm{d} x}\frac{\partial F}{\partial y'}=\frac{\mathrm{d} F}{\mathrm{d} x}-\frac{\partial F}{\partial y'}\frac{\partial y'}{\partial x}-y'\frac{\mathrm{d} }{\mathrm{d} x}\frac{\partial F}{\partial y'}$

Looking a little harder, it becomes clear that the last two terms are the result of a product rule.

$\frac{\mathrm{d} F}{\mathrm{d} x}-\frac{\partial F}{\partial y'}\frac{\partial y'}{\partial x}-y'\frac{\mathrm{d} }{\mathrm{d} x}\frac{\partial F}{\partial y'}=\frac{\mathrm{d} F}{\mathrm{d} x}-\frac{\mathrm{d} }{\mathrm{d} x}\left[y'\frac{\partial f}{\partial y'}\right]=\frac{\mathrm{d} }{\mathrm{d} x}\left[F-y'\frac{\partial f}{\partial y'}\right]$

Because it equals 0, we come to the final equation.

$F-y'\frac{\partial f}{\partial y'}=c \textup{ (}x\textup{-independent)}$

Altogether, this is a VERY powerful equation that is used in a huge range of physics and mathematics but it is understandable if there doesn’t seem to be any obvious use right now. This post was simply to establish a basis and derive the above equation. In the next post, I will give examples and uses of the equation which might make more clear what the purpose of the above steps were. In fact, I will be able to summarize 2 different past posts using this equation. If you want to get a head start, try proving that a line minimizes the distance between two points with this equation (Hint: use the arc length equation mentioned earlier and the $x$ -independent equation).

If you want to know more or see where I learned it from, read the book “Calculus of Variations” by Gelfand and Fomin (amazing book). You can find it in the Books section of this site.

Theorem: Let $h(x)$ be any continuous function such that $h(a)=h(b)=0$ . Assume then that, for all possible $h(x)$ , the following is true.

$\int_a^b \alpha(x)h(x)+\beta(x)h'(x)\textup{d}x=0$

This implies $\beta'(x)=\alpha(x)$

Proof:

Lemma: Let $h(x)$ be any continuous function such that $h(a)=h(b)=0$ . Assume then that, for all possible $h(x)$ , the following is true.

$\int_a^b\alpha(x)h'(x)\textup{d}x=0$

Then this implies $\alpha(x)=c$
Proof of Lemma: Define $c$ as the constant such that the following is true.

$\int_a^b (\alpha(x)-c)\textup{d}x=0$

Then let $h(x)$ be defined as

$h(x)=\int_a^x (\alpha(z)-c)\textup{d}z$

A simple calculation shows that this function satisfies the boundary conditions. Now we see that the following integral reduces nicely.

$\int_a^b (\alpha(x)-c)h'(x)\textup{d}x=\int_a^ba(x)h'(x)-ch'(x)\textup{d}x=0-c(h(b)-h(a))=0$

We can also reduce it a different way.

$\int_a^b (\alpha(x)-c)h'(x)\textup{d}x=\int_a^b (\alpha(x)-c)^2\textup{d}x$
We see that both of these statements can only hold true if $\alpha(x)-c=0$ because the square restricts the possibilities to all positive values. This means $\alpha(x)=c$ .