← All notes

Calculus of Variations

A quick and dirty introduction to calculus of variations

Calculus of Variations

Derivatives answer a simple question1:

How much does an output change when we perturb an input in a given direction?

That “perturb it and watch what happens” idea scales up nicely:

  • from numbers,
  • to vectors,
  • to functions (this is where calculus of variations lives).

Functions of a real variable

Let f:R→Rf:\mathbb{R}\to\mathbb{R}. The derivative at xx can be viewed as: push xx a tiny amount ε\varepsilon in a direction δx\delta x, and measure the rate of change at ε=0\varepsilon=0:

f′(x) δx=ddε f(x+ε δx)∣ε=0.f'(x)\,\delta x = \left.\frac{d}{d\varepsilon}\, f(x+\varepsilon\,\delta x)\right|_{\varepsilon=0}.

If you pick δx=1\delta x = 1, you recover the usual definition:

f′(x)=ddε f(x+ε)∣ε=0.f'(x)=\left.\frac{d}{d\varepsilon}\, f(x+\varepsilon)\right|_{\varepsilon=0}.

So f′(x)f'(x) is the linear map that takes a small input change δx\delta x and predicts the corresponding output change.


Functions on vector spaces

Let f:Rk→Rf:\mathbb{R}^k\to\mathbb{R}. The right generalization of “derivative” is the differential (a linear map):

df(x)[δm]=ddε f(x+ε δm)∣ε=0,δm∈Rk.df(x)[\delta m] = \left.\frac{d}{d\varepsilon}\, f(x+\varepsilon\,\delta m)\right|_{\varepsilon=0}, \qquad \delta m\in\mathbb{R}^k.

This is the directional derivative in direction δm\delta m.

In Euclidean space, the differential is represented by the gradient:

df(x)[δm]=∇f(x)⋅δm.df(x)[\delta m] = \nabla f(x)\cdot \delta m.

So:

  • df(x)[⋅]df(x)[\cdot] is a linear functional (takes a vector and returns a scalar),
  • ∇f(x)\nabla f(x) is the vector that represents that functional via the dot product.

From vectors to functions: functionals

Now we level up.

A functional is a map that takes a function as input and returns a real number: J[u]∈RJ[u]\in\mathbb{R}.

Here uu might be a function u:[a,b]→Ru:[a,b]\to\mathbb{R}, or u:Ω⊂Rn→Ru:\Omega\subset\mathbb{R}^n\to\mathbb{R}.

A variation is a small perturbation of the function:

uε=u+εv,u_\varepsilon = u + \varepsilon v,

where vv is a “test direction” (often assumed smooth, and often with v(a)=v(b)=0v(a)=v(b)=0 if boundary values are fixed).

First variation (the analogue of the directional derivative)

δJ[u](v)=ddε J[u+εv]∣ε=0.\delta J[u](v) = \left.\frac{d}{d\varepsilon}\, J[u+\varepsilon v]\right|_{\varepsilon=0}.

This is the calculus of variations version of df(x)[δm]df(x)[\delta m].

We say uu is a stationary point of JJ (a candidate minimizer/maximizer) if:

δJ[u](v)=0for all admissible v.\delta J[u](v)=0 \quad \text{for all admissible } v.

That condition is the analogue of “gradient equals zero”.


The classic setup: integral functionals

A very common functional looks like:

J[u]=∫abL(x,u(x),u′(x)) dx,J[u] = \int_a^b L\big(x, u(x), u'(x)\big)\,dx,

where LL is called the Lagrangian.

Let’s compute the first variation.

  1. Perturb uu as uε=u+εvu_\varepsilon = u + \varepsilon v. Then:
uε′=u′+εv′.u_\varepsilon' = u' + \varepsilon v'.
  1. Differentiate under the integral sign:
δJ[u](v)=ddε∣ε=0∫abL(x,u+εv,u′+εv′) dx=∫ab(∂L∂uv+∂L∂u′v′) dx.\delta J[u](v) = \left.\frac{d}{d\varepsilon}\right|_{\varepsilon=0} \int_a^b L\big(x, u+\varepsilon v, u'+\varepsilon v'\big)\,dx = \int_a^b \left( \frac{\partial L}{\partial u}v + \frac{\partial L}{\partial u'}v'\right)\,dx.
  1. Remove the derivative on v′v' using integration by parts:
∫ab∂L∂u′v′ dx=[∂L∂u′v]ab−∫abddx(∂L∂u′)v dx.\int_a^b \frac{\partial L}{\partial u'}v'\,dx = \left[\frac{\partial L}{\partial u'}v\right]_a^b - \int_a^b \frac{d}{dx}\left(\frac{\partial L}{\partial u'}\right)v\,dx.

If boundary values are fixed, we require v(a)=v(b)=0v(a)=v(b)=0, so the boundary term vanishes. Then:

δJ[u](v)=∫ab(∂L∂u−ddx(∂L∂u′))v dx.\delta J[u](v) = \int_a^b \left( \frac{\partial L}{\partial u} - \frac{d}{dx}\left(\frac{\partial L}{\partial u'}\right) \right)v\,dx.

For this to be zero for all admissible vv, the only way out is:

∂L∂u−ddx(∂L∂u′)=0\boxed{ \frac{\partial L}{\partial u} - \frac{d}{dx}\left(\frac{\partial L}{\partial u'}\right) = 0 }

This is the Euler–Lagrange equation.

That is the main engine of classical calculus of variations.


Boundary conditions (quick sanity)

If the endpoints are fixed: u(a),u(b)u(a), u(b) prescribed, then v(a)=v(b)=0v(a)=v(b)=0.

If an endpoint is free, the boundary term does not automatically vanish, and you get a natural boundary condition:

∂L∂u′∣x=a=0or∂L∂u′∣x=b=0,\left.\frac{\partial L}{\partial u'}\right|_{x=a} = 0 \quad\text{or}\quad \left.\frac{\partial L}{\partial u'}\right|_{x=b} = 0,

depending on which end is free (details depend on the exact constraints).


A tiny example (so this feels real)

Minimize “smoothness energy”:

J[u]=∫ab12(u′)2 dx,with u(a)=α,  u(b)=β.J[u] = \int_a^b \frac{1}{2}(u')^2\,dx, \quad \text{with } u(a)=\alpha,\; u(b)=\beta.

Here L=12(u′)2L = \frac12 (u')^2. Then:

∂L∂u=0,∂L∂u′=u′.\frac{\partial L}{\partial u} = 0, \qquad \frac{\partial L}{\partial u'} = u'.

Euler–Lagrange gives:

0−ddx(u′)=−u′′=0⇒u(x)=cx+d.0 - \frac{d}{dx}(u') = -u'' = 0 \quad\Rightarrow\quad u(x) = cx + d.

So the minimizer is a straight line between the boundary values. This matches the intuition: among all curves that hit the endpoints, the least “bendy” one is linear.


Multi-dimensional version (PDE form)

If u:Ω⊂Rn→Ru:\Omega\subset\mathbb{R}^n\to\mathbb{R} and

J[u]=∫ΩL(x,u(x),∇u(x)) dx,J[u] = \int_\Omega L(x, u(x), \nabla u(x))\,dx,

then the Euler–Lagrange equation becomes:

∂L∂u−∇⋅(∂L∂(∇u))=0.\boxed{ \frac{\partial L}{\partial u} - \nabla\cdot\left(\frac{\partial L}{\partial (\nabla u)}\right) = 0. }

So calculus of variations is one of the cleanest “factories” for producing PDEs.


What to remember

  • A derivative is “rate of change under perturbation”.
  • A gradient is how that derivative looks in Euclidean coordinates.
  • A functional J[u]J[u] eats a function and outputs a scalar.
  • The first variation δJ[u](v)\delta J[u](v) is the functional analogue of a directional derivative.
  • Setting δJ[u](v)=0\delta J[u](v)=0 for all admissible vv yields the Euler–Lagrange equation.

That’s basically the whole game. Everything else is: “what kind of JJ?”, “what constraints?”, and “how do I solve the resulting ODE/PDE?”.

Footnotes

  1. sContent is rephrased with ChatGPT. ↩

Written by Rudresh Veerkhare.