% !TEX root = lectures.tex
%!TEX encoding = UTF-8 Unicode
%\input{lectureheader.tex}

\section{Lagrangians and the Calculus of Variations}
\label{sec:lagrangians}
\bigskip

\subsection{Calculus of Variations and the Euler-Lagrange Equation}

The usual minimization problem one faces involves taking a function
$J(y)$, then finding the single value $y$ for which $J$ is either a
maximum or minimum. In multivariate calculus one also learns to solve
problems where you minimize for multiple variables, $J(y_1,y_2,\cdots
y_n)$, and finding the point $(y_1\cdots y_n)$ in $n-$dimensional
space that maximizes or minimizes the function. Here, we consider what
seems to be a much more ambitious problem. Imagine you have a function
$J(y(x),y'(x);x)$, and you wish to find the extrema for an infinite
number of values of $y$, i.e. $y$ at each point $x$. The function $J$
will not only depend on $y$ at each point $x$, but also on the slope
at each point, plus an additional dependence on $x$. Note we are NOT
finding an optimum value of $x$, we are finding the set of optimum
values of $y$ at each point $x$, or equivalently, finding the function
$y(x)$.

One treats the function $y(x)$ as being unknown while minimizing
\[
J=\int_{x_1}^{x_2}dx~f\{y(x),y'(x);x\}.
\]
Thus, we are minimizing $J$ with respect to an infinite number of
values of $y(x_i)$ at points $x_i$. As an additional criteria, we will
assume that $y(x_1)$ and $y(x_2)$ are fixed, and that that we will
only consider variations of $y$ between the boundaries. The dependence
on the derivative, $y'=dy/dx$, is crucial because otherwise the
solution would involve simply finding the one value of $y$ that
minimized $f$, and $y(x)$ would equal a constant if there were no
explicit $x$ dependence. Furthermore, $y$ wouldn't need to be
continuous at the boundary.

The Euler equation is a differential equation for $y(x)$, that when
solved, provides the required solution. For an extrema
\begin{eqnarray}
\delta J&=&\int_{x_1}^{x_2}dx~\left\{ \frac{\partial f}{\partial
  y}\delta y(x)+\frac{\partial f}{\partial y'}\delta y'(x)\right\}=0.
\end{eqnarray}
For ANY small $\delta y(x)$ at any point $x$, the change should be
zero if one is at an optimum function $y(x)$. Integrating the second
term by parts,
\begin{eqnarray}
\delta J&=&\int_{x_1}^{x_2}dx~\left\{ \frac{\partial f}{\partial
  y}-\frac{d}{dx}\left(\frac{\partial f}{\partial
  y'}\right)\right\}\delta y(x)\\ \nonumber &&+\left.\frac{\partial
  f}{\partial y'}\delta y\right|_{x_2}-\left.\frac{\partial
  f}{\partial y'}\delta y\right|_{x_1}\\ \nonumber &=&0.
\end{eqnarray}
Because $y$ is not allowed to vary at the endpoints, $\delta
y(x_1)=\delta y(x_2)=0$, so the middle line can be ignored.  Also,
this relation must hold for ANY $\delta y$, so one can write the Euler
equations (or sometimes called the Euler Lagrange equations)
\begin{equation}
\label{eq:el}
\frac{\partial f}{\partial y}=\frac{d}{dx}\frac{\partial f}{\partial
  y'}.
\end{equation}
This will yield a differential equations for $y$. Combined with the
boundary conditions,
\begin{equation}
y(x_1)=y_1, ~~~y(x_2)=y_2,
\end{equation}
one can now solve the differential equations for $y$. Because $f$ is
only a function of $f'$, not $f''$ or $f'''$, the $d/dx$ in Euler
Lagrange equation can lead to terms involving $f''$, but not
$f'''$. Thus, the equation will be a second-order differential
equation (not a linear equation usually), and the two boundary
conditions are sufficient to determine the entire equation.

\example\label{ex:brachiostone}\noindent Consider a particle
constrained to move along a path (like a bead moving without friction
on a wire) and you need to design a path from $x=y=0$ to some final
point $x_f,y_f$. Assume there is a constant force in the $x$
direction, $F_x=mg$. Design the path so that the time the bead travels
is a minimum.

{\bf Solution:} The net time is
\[
T=\int \frac{d\ell}{v}=\int_0^{x_f}
dx~\frac{\sqrt{1+y'^2}}{\sqrt{2gx}}={\rm minimum}.
\]
Here we made use of the fact that $d\ell=\sqrt{dx^2+dy^2}$ and that
the velocity is determined by $KE=mv^2/2=mgx$. The Euler equations can
be applied if you first define the function as
\begin{eqnarray*}
f(y,y';x)&=&\frac{\sqrt{1+y'^2}}{\sqrt{x}}.
\end{eqnarray*}
The equations are then
\begin{eqnarray*}
\frac{d}{dx}\frac{\partial f}{\partial y'}&=&0.
\end{eqnarray*}
The simplification ensued from $f$ not having any dependence on
$y$. This yields the differential equation
\begin{eqnarray}
\frac{y'}{x^{1/2}(1+y'^2)^{1/2}}&=&(2a)^{-1/2},
\end{eqnarray}
because $\partial f/\partial y'$ must be a constant, which with some
foresight we label $(2a)^{-1/2}$. One can now solve for $y'$,
\begin{eqnarray*}
(y')^2&=&2ax(1+y'^2)\\ \nonumber
  y'&=&\sqrt{\frac{x}{2a-x}},\\ \nonumber y(x)&=&\int_0^x
  dx'~\frac{\sqrt{x'}dx'}{\sqrt{2a-x'}}=\int_0^x
  dx'~\frac{x'dx'}{\sqrt{2ax'-x'^2}}\\ \nonumber
  &=&\frac{1}{2}\int_0^x\frac{(2x'-2a)dx'}{(2ax'-x'^2)^{1/2}}+a\int_0^x\frac{dx'}{\sqrt{2ax'-x'^2}}\\ \nonumber
  &=&\frac{-1}{2}\int_0^{2ax-x^2}\frac{du}{\sqrt{u}}+a\int_0^x\frac{dx'}{\sqrt{a^2-(x'-a)^2}}\\ &=&-\sqrt{2ax-x^2}+a\cos^{-1}(1-x/a).
\end{eqnarray*}
This turns out to be the equation for a {\it cycloid} or a {\it
  brachiostone}. If you rolled a wheel of radius $a$ down the $y$ axis
and followed a point on the rim, it would trace out a cycloid. Here,
the constant $a$ must be chosen to match the boundary condition,
$y_2=y(x_2)$. You can see the textbook for more details, plus you get
a chance to work with cycloids in the exercises at the end of this
chapter.

\exampleend

\subsection{Auxiliary Constraints}

Sometimes an auxiliary constraint is added to the problem (beyond
fixing the end poits $y_1$ and $y_2$). Just ahead, we will work on the
example of a hanging chain. The shape of the curve minimizes the
potential energy, under the constraint of a fixed length of
chain. Before presenting such an example we first review the method of
Lagrange multipliers as a method for finding minima or maxima under
constraints.

Imagine a function $f(x_1,x_2\cdots x_n)$ for which you wish to find
the minima. Additionally, you are given a constraint
\begin{eqnarray}
C(x_1\cdots x_n)=0
\end{eqnarray}
The usual condition for a a minimum is
\begin{eqnarray}
\frac{\partial f}{\partial x_i}=0{\rm ,~~or~}\nabla f=0.
\end{eqnarray}
which would be $n$ equations for the $n$ variables. The gradient of a
scalar is a vector, so you should think of $\nabla$ as
$\vec{\nabla}$. However, the solution will likely not satisfy the
constraint, i.e. the point at which $f(x_1\cdots x_n)$ has an extrema,
may not be a point where $C(x_1\cdots x_n)=0$.

A necessary condition for the solution is that
\begin{equation}
\nabla f\cdot\vec{\epsilon}=0,
\end{equation}
for any infinitesimal vector $\vec{\epsilon}$ if $\vec{\epsilon}$
satisfies the condition
\begin{equation}
\delta C=\nabla C\cdot\vec{\epsilon}=0.
\end{equation}
That is to say if I take a small step in a direction that doesn't
change the constraint, then $f$ must not change if it is an
extrema. Not changing the constraint implies the step is orthogonal to
$\nabla C$. As there are $n$ dimensions of $x$, the vector $\nabla C$
defines one direction, and $\vec{\epsilon}$ can be in any of the $n-1$
directions orthogonal to $\nabla C$. If $\nabla
f\cdot\vec{\epsilon}=0$ for ANY of the $n-1$ directions of
$\vec{\epsilon}$ orthogonal to $\nabla C$, then
\begin{equation}
\nabla f ~||~ \nabla C.
\end{equation}
Because the two vectors are parallel you can say there must exist some
constant $\lambda$ such that
\begin{equation}
\label{eq:lagrangemultiplier}
\nabla(f-\lambda C)=0.
\end{equation}
Here, $\lambda$ is known as a Lagrange multiplier. Satisfying
Eq. (\ref{eq:lagrangemultiplier}) is a necessary, but not a sufficient
condition. One could add a constant to the constraint and the gradient
would not change. One must find the correct value of $\lambda$ that
satisfies the constraint $C=0$, rather than $C=$ some other
constant. The strategy is then to solve
Eq. (\ref{eq:lagrangemultiplier}) then adjust $\lambda$ until one
finds the $x_1\cdots x_n$ that gives $C(x_1\cdots x_n)=0$.

The method of Lagrange multipliers is counter-intuitive to one's
intuition to use the constraint to reduce the dimensionality of the
problem. Normally, minimizing a function of $n$ variables, leads to
$n$ equations and $n$ unknowns. A constraint could be used, by
substitution, to replace the $n$ variables with $n-1$
variables. Instead, we add an unknown parameter, $\lambda$, and change
the equation to $n+1$ equations with $n+1$ unknowns, with the extra
unknown being the Lagrange multiplier $\lambda$. Often, it is rather
easy to solve for $x_1\cdots x_n$. Then one is left with the usually
difficult problem of finding $\lambda$, often requiring the solution
of a transcendental equation.

\example As an example of using Lagrange multipliers for a standard
optimization formula we attempt to maximize the following function,
\[
F(x_1\cdots x_n)=-\sum_{i=1}^n x_i\ln(x_i),
\]
with respect to the $n$ variables $x_i$. With no constraints, each
$x_i$ would maximize the function for
\begin{eqnarray*}
\frac{d}{dx_j}~\left[-\sum_i
  x_i\ln(x_i)\right]&=&0\\ -\ln(x_j)-1&=&0,~~~~x_j=e^{-1}.
\end{eqnarray*}
Now, we repeat the problem but with two constraints,
\[
\sum_ix_i=1~,~~~~\sum_ix_i\epsilon_i=E.
\]
Here, $\epsilon_i$ and $E$ are fixed constants. We go forward by
finding the extrema for
\begin{eqnarray*}
G(x_1\cdots x_n)&=&F-\alpha\sum_i x_i-\beta\sum_i\epsilon_ix_i =\sum_i
\left\{-x_i\ln(x_i)-\alpha x_i-\beta\epsilon_ix_i\right\}.
\end{eqnarray*}
There are two Lagrange multipliers, $\alpha$ and $\beta$,
corresponding to the two constraints. One then solves for the extrema
\begin{eqnarray*}
\frac{d}{dx_j}G&=&0\\ &=&-\ln(x_j)-1-\alpha-\beta\epsilon_j,\\ x_j&=&\exp\left\{-1-\alpha-\beta\epsilon_j\right\}.
\end{eqnarray*}
For any given $\alpha$ and $\beta$ this provides a solution for
constraining $\sum_i x_i$ and $\sum_i\epsilon_ix_i$ to some values,
just not the values of unity and $E$ that you wish. One would then
have to search for the correct values by adjusting $\alpha$ and
$\beta$ until the constraint are actually matched by solving a
transcendental equation. Although this can be complicated, it is
certainly less expensive than searching over all $N$ values of
$x_i$. This particular example corresponds to maximizing the entropy
for a system, $S=-\sum_i x_i\ln(x_i)$, where $x_i$ is the probability
of the system being in a particular discrete level $i$ that has energy
$\epsilon_i$. One wishes to maximize the entropy subject to the
constraints that the probabilities sum to unity and the average energy
has some given value. The result that $x_i\sim e^{-\beta\epsilon_i}$
demonstrates the origin of the Boltzmann factor, with the inverse
temperature $\beta=1/T$.

\exampleend

Lagrange multipliers also assist with the Euler-Lagrange equation. If
one breaks an interval $x_1<x<x_2$ into a large number
$n\rightarrow\infty$ points separated by $dx$, the Euler-Lagrange
equation involves finding the $n$ values $y_i$ at each point so that
$\sum_i dx f\left\{y_i,y'_i=(y_{i+1}-y_{i-1})/(2dx)\right\}$ is
maximized for some given function $f$. If an additional auxiliary
constraint is added, also some function of the $n$ values $y_i$, one
can use the method of Lagrange multipliers. In the constraint can also
be written as some function of $C(y_i,y'_i)$, then one simply adds a
term $\lambda C(y,y')$ to the function $f$ and uses the Euler-Lagrange
equation to find the extrema of.
\begin{eqnarray}
J&=&\int_{x_1}^{x_2}dx~f\left\{y(x),y'(x),x\right\}-\lambda
C\left\{y(x),y'(x),x\right\},
\end{eqnarray}
the one difference being that
\begin{equation}
f\left\{y(x),y'(x),x\right\}\rightarrow
f\left\{y(x),y'(x),x\right\}-\lambda C\left\{y(x),y'(x),x\right\}
\end{equation}
in Eq. (\ref{eq:el}).

\example Consider a chain of length $L$ and mass per unit length
$\kappa$ that hangs from point $x=0,y=0$ to point $x_f,y_f$. The shape
must minimize the potential energy. Find general expressions for the
shape in terms of three constants which must be chosen to match
$y(0)=0, y(x_f)=y_f$ and the fixed length. Equivalently, one finds the
function $y(x)$ that provides an extrema for the integral,

{\bf Solution:} One must minimize
\[
\int d\ell~\kappa gy-\lambda\int d\ell= \int_0^{x_f}
dx~\sqrt{1+y'^2}\kappa gy-\lambda \int_0^{x_f} dx\sqrt{1+y'^2}.
\]
Here $\lambda$ is the Lagrange multiplier associated with constraining
the length of the chain. The constrained length $L$ appears nowhere in
the expression. Instead, one solves for form of the answer, then
adjusts $\lambda$ to give the correct length. For the purposes of the
Euler-Lagrange minimization one considers the function
\begin{eqnarray}
f(y,y';x)&=&\kappa gy\sqrt{1+y'^2}-\lambda\sqrt{1+y'^2}.
\end{eqnarray}
Because $\lambda$ is an unknown constant and because minimizing a
function multiplied by a constant is the same as minimizing the
function, we can equivlently minimize the integral using the function
\begin{eqnarray}
\tilde{f}(y,y';x)&=&y\sqrt{1+y'^2}-\tilde{\lambda}\sqrt{1+y'^2},\\ \nonumber
\tilde{\lambda}&\equiv&\frac{\lambda}{\kappa g}.
\end{eqnarray}
The Euler-Lagrange equations then become
\begin{eqnarray*}
\frac{d}{dx}\left\{
\frac{y'}{\sqrt{1+y'^2}}y-\tilde{\lambda}\frac{y'}{\sqrt{1+y'^2}}
\right\}&=&\sqrt{1+y'^2}.
\end{eqnarray*}
Here, we will guess at the form of the solution,
\begin{eqnarray*}
y'&=&\sinh[(x-x_0)/a],~~y=a\cosh[(x-x_0)/a]+y_0.
\end{eqnarray*}
Plugging into the Euler-Lagange equations,
\begin{eqnarray*}
\frac{d}{dx}\left\{(a\cosh[(x-x_0)/a]+y_0)\frac{\sinh[(x-x_0)/a]}{\cosh[(x-x_0)/a]}-\tilde{\lambda}\frac{\sinh[(x-x_0)/a]}{\cosh[(x-x_0)/a]}\right\}&=&\cosh[(x-x_0)/a],\\ \nonumber
\frac{d}{dx}\left\{(y_0-\tilde{\lambda})\tanh[(x-x_0)/a]\right\}=0.
\end{eqnarray*}
This solution works if $y_0=\tilde{\lambda}$. So the general form of
the solution is
\[
y=\tilde{\lambda}+a\cosh[(x-x_0)/a].
\]
One must find $\tilde{\lambda}$, $x_0$ and $a$ to satisfy three
conditions, $y(x=0)=0$, $y(x=x_f)=y_f$ and that the length is $L$. For
a hanging chain $a$ is positive. A solution with negative $a$ would
represent a maximum of the potential energy. A remarkable property of
the solution is that once you define the length and the end-point
positions $y_1$ and $y_2$, the solution does not depend on $\kappa$ or
$g$. Thus, the shape of the chain would be the same if you took it to
the moon. These solutions are known as {\it
  catenaries},\\ \href{http://en.wikipedia.org/wiki/Catenary}{http://en.wikipedia.org/wiki/Catenary}.

\exampleend

\subsection{Lagrangians}

Lagrangians represent a powerful method for solving problems that
would be nearly impossible by direct application of Newton's third
law, $\vec{F}=m\vec{a}$. The method works well for problems where a
system is well described by a few \textit{generalized coordinates}. A
generalized coordinate might be the angle describing the position of a
pendulum. This one angle takes the place of using $x$ and $y$ to
describe the position of the pendulum, then applying a clumsy
constraint.

The Lagrangian equations of motion can be derived from a principle of
least action, where the action $S$ is defined as
\begin{equation}
S=\int dt~ L(q,\dot{q},t),
\end{equation}
where $q$ is some coordinate that describes the orientation of a
system and the Lagrangian $L$ is defined as
\begin{equation}
L=T-U,
\end{equation}
the difference of the kinetic and potential energies. Minimizing the
action through the Euler-Lagrange equations gives the Lagrangian
equations of motion,
\begin{equation}
\frac{d}{dt}\frac{\partial L}{\partial \dot{q}}=\frac{\partial
  L}{\partial q}.
\end{equation}
We begin with two simple examples, neither of which gains from the
Lagrangian approach.

\example Consider a particle of mass $m$ connected to a spring with
stiffness $k$. Derive the Lagrangian equations of motion.\\ {\bf
  Solution:}
\begin{eqnarray*}
L&=&\frac{1}{2}m\dot{x}^2-\frac{1}{2}kx^2,\\ \frac{d}{dt}\frac{\partial
  L}{\partial \dot{x}}&=&\frac{\partial L}{\partial
  x},\\ m\ddot{x}&=&-kx.
\end{eqnarray*}

\example Derive the Lagrangian equations of motion for a pendulum of
mass $m$ and length $\ell$.
\begin{eqnarray*}
L&=&\frac{m}{2}\ell^2\dot{\theta}^2-mg\ell(1-\cos\theta),\\ \frac{d}{dt}\frac{\partial
  L}{\partial \dot{\theta}}&=&\frac{\partial L}{\partial
  \theta},\\ m\ell^2\ddot{\theta}&=&-mg\ell\sin\theta,\\ \ddot{\theta}&=&-\frac{g}{\ell}\sin\theta,\\ \ddot{\theta}&\approx&-\frac{g}{\ell}\theta.
\end{eqnarray*}
\exampleend

\subsection{Proving Lagrange's Equations of Motion from Newton's Laws}

Lagrange's equations of motion can only be applied for the following
conditions:
\begin{enumerate}
\item The potential energy is a function of the generalized
  coordinates $q_i$, but not of $\dot{q}_i$.
\item The relation between the original coordinates $x,y,z\cdots$ and
  the generalized coordinates does not depend on $\dot{q}_i$,
  e.g. $x(q,t)$ not $x(q,\dot{q},t)$.
\item Any constraints used to reduce the number of degrees of freedom
  are functions of $\vec{q}$, but not of $\dot{\vec{q}}$.
\item The motion is not dissipative (no damping or friction).
\end{enumerate}
Going forward with the proof, consider $x_i(q_1,q_2\cdots,t)$ and look
at the l.h.s. of Lagrange's equations of motion.
\begin{eqnarray}
\label{eq:lagrangederivation1}
\frac{\partial T}{\partial\dot{q}_j}&=&\sum_i\frac{\partial
  T}{\partial\dot{x}_i}\frac{\partial\dot{x}_i}{\partial\dot{q_j}}
+\sum_i\frac{\partial T}{\partial x_i}\frac{\partial
  x_i}{\partial\dot{q_j}}\\ \nonumber &=&\sum_i
m\dot{x}_i\frac{\partial \dot{x}_i}{\partial\dot{q_j}}\\ \nonumber
&=&\sum_i m\dot{x}_i\frac{(\delta x_i/\delta t)|_{{\rm fixed~}q_{j'\ne
      j}}}{\delta q_j/\delta t}\\ \nonumber
&=&\sum_im\dot{x}_i\frac{\delta{x}_i|_{{\rm fixed~}q_{j'\ne
      j}}}{\delta q_j}\\ \nonumber &=&\sum_i m\dot{x}_i\frac{\partial
  x_i}{\partial q_j}.
\end{eqnarray}
In the first line we used the fact that $T$ does not depend on
$x$. Continuing with taking the derivative of $U$,
\begin{eqnarray}
-\frac{\partial U}{\partial\dot{q}_j}&=&-\sum_i\frac{\partial
  U}{\partial x_i}\frac{\partial x_i}{\partial\dot{q}_j}=0.
\end{eqnarray}
In the first line above we used the fact that $U$ does not depend on
$\dot{x}$ then we used the second condition that $x$ does not depend
on $\dot{q}$. Adding the two pieces together, then taking the
derivative w.r.t. time,
\begin{eqnarray}
\nonumber
\frac{d}{dt}\frac{\partial}{\partial\dot{q}}(T-U)&=&\sum_im\ddot{x}_i\frac{\partial
  x_i}{\partial q_j} +\sum_i
m\dot{x}_i\frac{\partial\dot{x}_i}{\partial q_j}.
\end{eqnarray}

Now, we consider the r.h.s. of Lagrange's equations. Because the
kinetic energy depends only on $\dot{x}$ and not $x$, and because the
potential depends on $x$ but not $\dot{x}$,
\begin{eqnarray}
\label{eq:lagrangederivation2}
\frac{\partial}{\partial q_j}(T-U)&=&\sum_i\frac{\partial
  T}{\partial\dot{x}_i}\frac{\partial\dot{x_i}}{\partial q_j}
-\sum_i\frac{\partial U}{\partial x_i}\frac{\partial x_i}{\partial
  q_j}\\ \nonumber &=&\sum_i
m\dot{x}_i\frac{\partial\dot{x_i}}{\partial q_j} -\sum_i\frac{\partial
  U}{\partial x_i}\frac{\partial x_i}{\partial q_j}
\end{eqnarray}
Using the fact that $m\ddot{x}_i=-(\partial/\partial x_i)U$, one can
see that the bottom expressions in Eq.s (\ref{eq:lagrangederivation1})
and (\ref{eq:lagrangederivation2}) are identical,
\begin{equation}
\frac{d}{dt}\frac{\partial}{\partial\dot{q}_i}(T-U)=\frac{\partial}{\partial
  q_i}(T-U).
\end{equation}

\subsection{Lagrangian Examples}

Two examples are presented here. In the first, there are two
generalized coordinates, but the two equations of motion can be
reduced to one through conservation laws (angular momentum in this
case). In the second, there is a time-dependent constraint.

\example Consider a cone of half angle $\alpha$ standing on its tip at
the origin. The surface of the cone is defined as
\[
r=\sqrt{x^2+y^2}=z\tan \alpha.
\]
Find the equations of motion for a particle of mass $m$ moving along
the surface under the influence of a constant gravitational force,
$-mg\hat{z}$. For generalized coordinates use the azimuthal angle
$\phi$ and $r$.

{\bf Solution:}\\ The kinetic energy is
\begin{eqnarray*}
T&=&\frac{1}{2}mr^2\dot{\theta}^2+\frac{1}{2}m(\dot{r}^2+\dot{z}^2)\\ &=&\frac{1}{2}mr^2\dot{\theta}^2+\frac{1}{2}m\dot{r}^2\left(1+\cot^2\alpha\right)\\ &=&\frac{1}{2}mr^2\dot{\theta}^2+\frac{1}{2}m\dot{r}^2\csc^2\alpha.
\end{eqnarray*}
The potential energy is
\[
U=mgr\cot\alpha,
\]
so Lagrange's equations give
\begin{eqnarray*}
\frac{d}{dt}\left(mr^2\dot{\theta}\right)&=&0,\\ \frac{d}{dt}\left(m\csc^2\alpha
\dot{r}\right)&=&mr\dot{\theta}^2-mg\cot\alpha,\\ \ddot{r}&=&r\dot{\theta}^2\sin^2\alpha-g\cos\alpha\sin\alpha
\end{eqnarray*}
The first equation is a statement of the conservation of angular
momentum with $L=mr^2\dot{\theta}$, so the second equation can also be
expressed as
\[
\ddot{r}=\frac{L^2\sin^2\alpha}{m^2r^3}-g\sin\alpha\cos\alpha.
\]

\example A bead slides along a wire bent in the shape of a parabola,
\[
z=\frac{1}{2}kr^2,~~r^2=x^2+y^2.
\]
Also, the parabolic wire is rotating about the $z$ axis with angular
velocity $\omega$. Derive the equations of motion. Are there any
stable configurations?

{\bf Solution:}\\ Using the fact that
\[
\dot{z}=\dot{r}\frac{\partial z}{\partial r}=kr\dot{r},
\]
the kinetic and potential energies are
\begin{eqnarray*}
T&=&\frac{1}{2}m\left(\dot{r}^2+\dot{z}^2+r^2\omega^2\right)\\ &=&\frac{1}{2}m\left(\dot{r}^2+(kr\dot{r})^2+r^2\omega^2\right),\\ U&=&mgkr^2/2.
\end{eqnarray*}
The equations of motion are then
\begin{eqnarray*}
\frac{d}{dt}\left\{m\dot{r}(1+k^2r^2)\right\}&=&-mgkr+mk^2\dot{r}^2r+m\omega^2r,\\ \ddot{r}&=&\frac{-gkr+\omega^2r-k^2\dot{r}^2r}{1+k^2r^2}
\end{eqnarray*}
For a stable configuration, there needs to be a solution with
$\dot{r}=0$ and $\ddot{r}=0$. This can only happen at $r=0$, and then
for the acceleration to be inward for small deviations of $r$ one
needs to have $gk>\omega^2$. If $\omega^2>gk$ the bead will move
outward indefinitely.

\subsection{Small Vibrations and Normal Modes}

Two examples are provided for solving for normal modes. These are
solutions with multiple generalized coordinates, where the motion is
that of simple harmonic motion. However, the motion is only simple for
a particular set of coordinates $q_1$ and $q_2$,
\begin{eqnarray}
q_1&=&A\cos(\omega_1 t),\\ \nonumber q_2&=&B\cos(\omega_2 t),
\end{eqnarray}
while it is not necessarily simple in other coordinates. For example
if $x=q_1+q_2$, and $y=q_1-q_2$, the $x$ and $y$ motions will contain
mixtures of multiple frequencies. For many problems, or in the limit
of small vibrations about a minimum, there is some coordinate system
where the motion is simple. These are normal modes. Characterizing the
normal modes involves finding the frequencies, $\omega_i$, and the
coordinate system where the motion is simple for each coordinate. This
involves finding the direction, or the linear combination of $x_i$
that form the coordinates $q_i$ in which the motion is that of a
single oscillator in each coordinate.

For a first example, we consider a system of springs, where we write
the Lagrangian, then find the normal modes. For the second example, a
double pendulum is considered. In this case, one must first make a
small angle expansion before finding the modes. In principle, problems
could have the same number of normal modes a degrees of freedom. For
example, a system of 7 particles moving in three dimensions has 21
degrees of freedom. However, some of the degrees of freedom do not
have oscillatory behavior. For example, for a rigid body in free
space, the angles describing the orientation evolve, but do not
oscillate. Also, the center-of-mass coordinates of a system of
particles isolated from outside particles moves at constant
velocity. One can also describe these as normal modes, but acknowledge
that their characteristic frequency is zero, as there are no restoring
forces.

\example
\begin{figure}
\centerline{\includegraphics[width=0.5\textwidth]{figs/springs}}
\caption{\label{fig:springs} For Example
  \ref{sec:lagrangians}.\arabic{examplecounter}, three masses
  connected by two springs. The center-of-mass motion is unaffected by
  the springs.}
\end{figure}
Consider two springs, whose relaxed lengths are $\ell$, connected to
three masses as depicted in Fig. \ref{fig:springs}. Describe the two
normal modes of the motion. We can write the Lagrangian as
\begin{eqnarray*}
\mathcal{L}&=&\frac{m}{2}\dot{x}_1^2+m\dot{x}_2^2+\frac{m}{2}\dot{x}_3^2
-\frac{k}{2}(x_2-x_1-\ell)^2-\frac{k}{2}(x_3-x_2-\ell)^2.
\end{eqnarray*}
There are three coordinates, thus there are three equations of motion,
\begin{eqnarray*}
m\ddot{x}_1&=&-k(x_1-x_2+\ell)\\ 2m\ddot{x}_2&=&-k(x_2-x_1-\ell)-k(x_2-x_3+\ell)\\ &=&-k(2x_2-x_1-x_3)\\ m\ddot{x}_3&=&-k(x_3-x_2+\ell).
\end{eqnarray*}
This is a bit complicated because the center-of-mass motion does not
easily separate from the three equations. Instead, choose the
following coordinates,
\begin{eqnarray*}
X&=&\frac{x_1+2x_2+x_3}{4},\\ q_1&=&x_1-x_2+\ell,\\ q_3&=&x_3-x_2-\ell.
\end{eqnarray*}
In these coordinates the potential energy only involves two
coordinates,
\begin{eqnarray*}
U&=&\frac{k}{2}(q_1^2+q_3^2).
\end{eqnarray*}
To express the kinetic energy express $x_1, x_2$ and $x_3$ in terms of
$X$, $q_1$ and $q_3$,
\begin{eqnarray*}
x_1&=&(3q_1-q_3-4\ell+4X)/4,\\ x_2&=&(4X-q_1-q_3)/4,\\ x_3&=&(3q_3-q_1+4\ell+4X)/4.
\end{eqnarray*}
The kinetic energy and Lagrangian are them
\begin{eqnarray*}
T&=&\frac{m}{2}\frac{1}{16}(3\dot{q}_1-\dot{q}_3+4\dot{X})^2
+m\frac{1}{16}(4\dot{X}-\dot{q}_1-\dot{q}_3)^2
+\frac{m}{2}\frac{1}{16}(3\dot{q}_3-\dot{q}_1+4\dot{X})^2\\ &=&\frac{3m}{8}(\dot{q}_1^2+\dot{q}_3^2)-\frac{m}{4}\dot{q}_1\dot{q}_3
+2m\dot{X}^2,\\ \mathcal{L}&=&\frac{3m}{8}(\dot{q}_1^2+\dot{q}_3^2)-\frac{m}{4}\dot{q}_1\dot{q}_3
+2m\dot{X}^2-\frac{k}{2}q_1^2-\frac{k}{2}q_3^2.
\end{eqnarray*}
The three equations of motion are then,
\begin{eqnarray*}
\frac{3}{4}m\ddot{q}_1-\frac{1}{4}m\ddot{q}_3&=&-kq_1,\\ \frac{3}{4}m\ddot{q}_3-\frac{1}{4}m\ddot{q}_1&=&-kq_3,\\ 4M\ddot{X}&=&0.
\end{eqnarray*}
The last equation simply states that the center-of-mass velocity is
fixed. One could obtain the same result by summing the equations of
motion for $x_1$, $2x_2$ and $x_3$ above. The second two equations are
more complicated. To solve them, we assume a form
\begin{eqnarray*}
q_1&=&Ae^{i\omega t},\\ q_3&=&Be^{i\omega t},
\end{eqnarray*}
Because this is a linear equation, we can multiply the solution by a
constant and it will still be a solution. Thus, we can set $B=1$, then
solve for $A$, effectively solving for $A/B$. Putting this guess into
the equations of motion,
\begin{eqnarray*}
-\frac{3}{4}\frac{A}{B}\omega^2+\frac{1}{4}\omega^2&=&-\omega_0^2\frac{A}{B},\\ \frac{3}{4}\omega^2+\frac{1}{4}\frac{A}{B}\omega^2&=&-\omega_0^2.
\end{eqnarray*}
This is two equations and two unknowns, $\omega^2$ and
$A/B$. Substituting for $A/B$ gives a quadratic equation,
\begin{eqnarray*}
\omega^4-3\omega_0^2\omega^2+2\omega_0^4&=&0,\\ \omega_0^2&\equiv&k/m.
\end{eqnarray*}
The two solutions are
\begin{eqnarray*}
(1)~~\omega&=&\omega_0,~~~A=-B,\\ (2)~~\omega&=&\omega_0\sqrt{2},~~~A=B.
\end{eqnarray*}
The first solution corresponds to the two outer masses moving in
opposite directions, in sync, with the middle mass fixed. The second
solution has both outer masses moving in the same direction, but with
the center mass moving opposite. These two solutions are referred to
as normal modes, and are characterized by their frequency and by the
linear combinations of coordinates that oscillate together. In
general, the solution is a linear combination of normal modes, which
usually results in a chaotic looking motion. However, once the
solution is expressed in terms of the normal modes, each of which
oscillates independently in a simple manner, one can better understand
the motion. Further, the frequencies of these modes represent the
natural resonant frequencies of the system. This is important in the
construction of many structures, such as bridges or vehicles.

\example Consider a double pendulum confined to the $x-y$ plane, where
$y$ is vertical. A mass $m$ is connected to the ceiling with a
massless string of length $\ell$. A second mass $m$ hangs from the
first mass with an identical massless string of the same length. Using
$\theta_1$ and $\theta_2$ to describe the orientations of the strings
relative to the vertical axis, find the Lagrangian and derive the
equations of motion, both for arbitrary angles and in the small-angle
approximation. Finally, express the equations of motion in the limit
of small oscillations.

{\bf Solution:}\\ The kinetic and potential energies are:
\begin{eqnarray*}
T&=&\frac{1}{2}m\ell^2\dot{\theta}_1^2
+\frac{1}{2}m\left\{(\ell\dot{\theta}_1\cos\theta_1+\ell\dot{\theta}_2\cos\theta_2)^2
+(\ell\dot{\theta}_1\sin\theta_1+\ell\dot{\theta}_2\sin\theta_2)^2\right\}\\ &=&\frac{1}{2}m\ell^2\left\{2\dot{\theta}_1^2+\dot{\theta}_2^2+2\dot{\theta}_1\dot{\theta}_2\cos(\theta_1-\theta_2)
\right\},\\ U&=&mg\ell(1-\cos\theta_1)+mg\left[\ell(1-\cos\theta_1)+\ell(1-\cos\theta_2)\right]\\ &=&mg\ell(3-2\cos\theta_1-\cos\theta_2)
\end{eqnarray*}
Lagrange's equations for $\theta_1$ lead to
\begin{eqnarray*}
m\ell^2\frac{d}{dt}\left\{2\dot{\theta}_1+\dot{\theta}_2\cos(\theta_1-\theta_2)\right\}&=&
-m\ell^2\dot{\theta}_1\dot{\theta}_2\sin(\theta_1-\theta_2)
-2mg\ell\sin\theta_1,\\ 2\ddot{\theta}_1+\ddot{\theta}_2\cos(\theta_1-\theta_2)+\dot{\theta}_2^2\sin(\theta_1-\theta_2)
&=&-2\omega_0^2\sin\theta_1,\\ \omega_0^2&\equiv& g/\ell,
\end{eqnarray*}
and the equations for $\theta_2$ are
\begin{eqnarray*}
m\ell^2\frac{d}{dt}\left\{\dot{\theta}_2+\dot{\theta}_1\cos(\theta_1-\theta_2)\right\}&=&
m\ell^2\dot{\theta}_1\dot{\theta}_2\sin(\theta_1-\theta_2)-mg\ell\sin\theta_2,\\ \ddot{\theta}_2+\ddot{\theta_1}\cos(\theta_1-\theta_2)&=&
-\omega_0^2\sin\theta_2.
\end{eqnarray*}
For small oscillations, one can only consider terms linear in
$\theta_1$ and $\theta_2$ or their derivatives,
\begin{eqnarray}
\label{eq:doublependulum}
2\ddot{\theta}_1+\ddot{\theta}_2&=&-2\omega_0^2\theta_1,\\ \nonumber
\ddot{\theta}_1+\ddot{\theta}_2&=&-\omega_0^2\theta_2.
\end{eqnarray}
To find the solutions, assume they are of the form
$\theta_1=Ae^{i\omega t}, \theta_2=Be^{i\omega t}$. Solve for $\omega$
and $A/B$, noting that $B$ is arbitrary.

Plug in the desired form and find
\begin{eqnarray*}
e^{i\omega t}(-2\omega^2A-\omega^2B)&=&e^{i\omega
  t}(-2\omega_0^2A),\\ e^{i\omega
  t}(-\omega^2A-\omega^2B)&=&e^{i\omega t}(-\omega_0^2B).
\end{eqnarray*}
We can treat $B$ as arbitrary and set it to unity. When we find $A$,
it is the same as $A/B$ for arbitrary $B$. This gives the equations
\begin{eqnarray*}
2\omega^2A+\omega^2&=&2\omega_0^2A,\\ \omega^2A+\omega^2&=&\omega_0^2.
\end{eqnarray*}
This is two equations and two unknowns. Solving them leads to a
quadratic equation with solutions
\begin{eqnarray*}
A/B&=&\pm\frac{1}{\sqrt{2}},\\ \omega^2&=&\frac{\omega_0^2}{1\pm
  1/\sqrt{2}}.
\end{eqnarray*}
Again, these two solutions are the normal modes, and the general
solution is a sum of the two solutions, with two arbitrary
constants. For the angles $\theta_1$ and $\theta_2$ are:
\begin{eqnarray*}
\theta_1&=&\frac{A_+}{\sqrt{2}}e^{i\omega_+t},
~\theta_2=A_+e^{i\omega_+t},\\ \theta_1&=&\frac{-A_-}{\sqrt{2}}e^{i\omega_-t},
~\theta_2=A_-e^{i\omega_-t},\\ \omega_{\pm}&=&\omega_0\sqrt{\frac{1}{1\pm
    1/\sqrt{2}}}.
\end{eqnarray*}
One can also express the solution in vector notation, with the vectors
having arbitrary amplitudes $A_+$ and $A_-$,
\begin{eqnarray*}
\theta_+&=&\left(\begin{array}{c}
  \frac{1}{\sqrt{2}}\\ 1\end{array}\right)A_+e^{i\omega_+t},\\ \theta_-&=&\left(\begin{array}{c}
    \frac{-1}{\sqrt{2}}\\ 1\end{array}\right)A_-e^{i\omega_-t}.
\end{eqnarray*}
Here, the upper/lower components of the vector describe
$\theta_1/\theta_2$ respectively.

\exampleend

{\bf Aside:} (not applied in this course)\\ These problems can be
treated as linear algebra exercises. Linear algebra is not used in
this course, but nonetheless we describe how this works for the
curious student. In the limit of small vibrations, the equations of
motion can be expressed in the form,
\begin{eqnarray*}
M\ddot{q}&=&-Kq,
\end{eqnarray*}
a form that looks like the spring equation. However, $q$ is an
$n-$dimensional vector and $M$ and $k$ are $n\times n$ matrices. In
the double pendulum example, the dimensionality is 2 and the $q$
refers to the $\theta_1$ and $\theta_2$, and the matrices for $M$ and
$K$ can be read off Eq. (\ref{eq:doublependulum}),
\begin{eqnarray*}
M&=&\left(\begin{array}{cc}
  2&1\\ 1&1\end{array}\right)~,\hspace*{40pt}
  K=\left(\begin{array}{cc}
    2\omega_0^2&0\\ 0&\omega_0^2\end{array}\right).
\end{eqnarray*}
Multiplying both sides of the equation by the inverse matrix $M^{-1}$,
\begin{eqnarray*}
\ddot{q}&=&-\left(M^{-1}K\right)q.
\end{eqnarray*}
Here,
\begin{eqnarray*}
M^{-1}&=&\left(\begin{array}{cc}
  1&-1\\ -1&2\end{array}\right),\\ M^{-1}K&=&\left(\begin{array}{cc}
    2&-1\\ -2&2\end{array}\right)\omega_0^2.
\end{eqnarray*}
One can find a transformation, basically a rotation, that transforms
to a frame where $M^{-1}K$ is diagonal. In this coordinate system the
diagonal components of $M^{-1}K$ represent the squared frequencies of
the normal modes,
\[
M^{-1}K\rightarrow -\left(\begin{array}{cc}
  \omega_+^2&0\\ 0&\omega_-^2\end{array}\right)~,
\]
and are known as ``eigen'' frequencies. The corresponding unit
vectors,
\[
\left(\begin{array}{c} 1\\0\end{array}\right)~{\rm
    and}~\left(\begin{array}{c} 0\\1\end{array}\right)~{\rm
      in~the~new~coordinate~system},
\]
can be rotated back into the original frame, and become the solutions
for the normal modes. These are then called ``eigenvectors'', which
are the same as the normal modes. Finding the eigenfrequencies is
performed by realizing that the determinant of a matrix is unchanged
by the rotation between coordinate systems. Writing the equations of
motion as an eigenvalue problem,
\begin{eqnarray}
\left[A-\lambda_i\mathbb{1}\right]u_i&=&0,~~~A\equiv
M^{-1}K,~\lambda_i\equiv \omega^2_i.
\end{eqnarray}
In the coordinate system where $M^{-1}K$ is diagonal, and the forms
for $u_i$ are simple this requires that in that system, the diagonal
elements of $M^{-1}K$ are the eigenvalues, $\omega_i^2$. For each
$\omega^2_i$, the determinant $|A-\lambda_i\mathbb{1}|$ must
vanish. This is then true in any coordinate system,
\begin{eqnarray}
{\rm det}\left[A-\lambda\mathbb{1}\right]&=&0,
\end{eqnarray}
which for a $2\times 2$ matrix becomes
\begin{eqnarray}
\label{eq:eigen}
\left|
\begin{array}{cc}
A_{11}-\lambda&A_{12}\\ A_{21}&A_{22}-\lambda
\end{array}
\right|&=&0,\\ A_{11}A_{22}-\lambda A_{11}-\lambda
A_{22}+\lambda^2-A_{21}A_{12}&=&0.
\end{eqnarray}
One can solve a quadratic equation for $\lambda$, which gives two
eigenvalues corresponding to $\omega_+^2$ and $\omega_-^2$ found
above. Choosing one of the eigenvalues, one can insert one of the
eigenvalues $\lambda_i$ into Eq. (\ref{eq:eigen}) and solve for $u_i$,
then choose the other eigenvalue and solve for the other corresponding
vector.

If this were a 3-dimensional set of equations, the determinant would
include terms like $\lambda^3$ and would become a cubic equation with
three eigenvalues. One would then solve for three eigenvectors. If one
has a system with dimensionality $n>2$, one usually resorts to solving
the problem numerically due to the messiness of the algebra. The main
programming languages all have packages which readily diagonalize
matrices and find eigenvectors and eigenvalues.
\subsection{Conservation Laws}

Energy is conserved only when the Lagrangian has no explicit
dependence on time, i.e. $L(q,\dot{q})$, not $L(q,\dot{q},t)$. To show
this, we first define the Hamiltonian,
\begin{eqnarray}
\label{eq:Hdef}
H&=&\sum_i\left(\dot{q}_i\frac{\partial
  L}{\partial\dot{q}_i}\right)-L.
\end{eqnarray}
After showing that $H$ is conserved, i.e. $(d/dt)H=0$, we then show
that $H$ can be identified with the total energy, $H=T+V$.

One can see that $H$ is conserved by applying first using the chain
rule for $(d/dt)H$ in Eq. (\ref{eq:Hdef}), then applying Lagrange's
equations,
\begin{eqnarray}
\frac{d}{dt}H&=&\sum_i\left\{\ddot{q}_i\frac{\partial
  L}{\partial\dot{q}_i}+\dot{q}_i\frac{d}{dt}\left(\frac{\partial
  L}{\partial\dot{q}_i}\right)-\frac{\partial
  L}{\partial\dot{q}_i}\ddot{q}_i-\frac{\partial L}{\partial
  q_i}\dot{q}_i\right\}\\ \nonumber
&=&\sum_i\left\{\ddot{q}_i\frac{\partial
  L}{\partial\dot{q}_i}+\dot{q}_i\frac{\partial L}{\partial
  q_i}-\frac{\partial L}{\partial\dot{q}_i}\ddot{q}_i-\frac{\partial
  L}{\partial q_i}\dot{q}_i\right\}\\ \nonumber &=&0.
\end{eqnarray}
These steps assumed that $L$ had no explicit time dependence, i.e. $L$
is a function of $q$ and $\dot{q}$, but not of $t$.

Next, we show that $L$ can be identified with the energy. Because $V$
does not depend on $\dot{q}$,
\begin{equation}
H=\sum_i\frac{\partial T}{\partial\dot{q}_i}\dot{q}_i-T+V.
\end{equation}
If the kinetic energy has a purely quadratic form in terms of
$\dot{q}$,
\begin{equation}
\label{eq:Hquadq}
T=\sum_{ij}A_{ij}(q)\dot{q}_i\dot{q}_j,
\end{equation}
the Hamiltonian becomes
\begin{eqnarray}
H&=&\sum_{ij}2A_{ij}(q)\dot{q}_i\dot{q}_j-\sum_{ij}A_{ij}(q)\dot{q}_i\dot{q}_j+V\\ \nonumber
&=&T+V.
\end{eqnarray}
The proof that $H$ equals the energy hinged on the fact that the
kinetic energy was quadratic in $\dot{q}$. This can be attributed to
time-reversal symmetry. Because the Cartesian coordinates $x_i$ do not
depend on $\dot{q}_i$ or on time, $\dot{x}_i=(\partial x_i/\partial
q_j)\dot{q}_j$. Thus, the kinetic energy, $T=m\dot{x}_i^2/2$, should
be proportional to two powers of $\dot{q}$, which validates the
assumption in Eq. (\ref{eq:Hquadq}).

Here, energy conservation is predicated on the Lagrangian not having
an explicit time dependence. Without an explicit time dependence the
equations of motion are unchanged if one translates a fixed amount in
time because the physics does not depend on when the clock starts. In
contrast, the absolute time becomes relevant if there is an explicit
time dependence. In fact, conservation laws can usually be associated
with symmetries. In this case the translation symmetry in time leads
to energy conservation.

For another example of how symmetry leads to conservation laws,
consider a Lagrangian for a particle of mass $m$ moving in a
two-dimensional plane where the generalized coordinates are the radius
$r$ and the angle $\theta$. The kinetic energy would be
\begin{equation}
T=\frac{1}{2}m\left\{\dot{r}^2+r^2\dot{\theta}^2\right\},
\end{equation}
and if the potential energy $V(r)$ depends only on the radius $r$ and
not on the angle, Lagrange's equations become
\begin{eqnarray}
\frac{d}{dt}(m\dot{r})&=&-\frac{\partial V}{\partial
  r}+m\dot{\theta}^2r,\\ \nonumber \frac{d}{dt}(mr^2\dot{\theta})&=&0.
\end{eqnarray}
The second equation implies that $mr^2\dot{\theta}$ is a
constant. Indeed, it is the angular momentum which is conserved for a
radial force. Here, the conservation of angular momentum is associated
with the independence of the physics to changes in $\theta$, or in
other words, rotational invariance. Once one knows the fact that
$L=mr^2\dot{\theta}$ is conserved, it can be inserted into the
equations of motion for $\dot{r}$,
\begin{equation}
m\ddot{r}=-\frac{\partial V}{\partial r}+\frac{L^2}{mr^3}.
\end{equation}
This is related to Noether's theorem
\href{http://en.wikipedia.org/wiki/Noether's_theorem}{http://en.wikipedia.org/wiki/Noether's\_theorem},
named after Emmy Noether,
\href{http://en.wikipedia.org/wiki/Emmy_Noether}{http://en.wikipedia.org/wiki/Emmy\_Noether}. Simply
stated, if the Lagrangian $L$ is independent of $q_i$, one can see
that the quantity $\partial L/\partial\dot{q}_i$ is conserved,
\begin{equation}
\label{eq:noether}
\frac{d}{dt}\frac{\partial L}{\partial\dot{q}_i}=0.
\end{equation}

Another easy example is in Cartesian coordinates where the potential
depends only on $x$ and $y$ but not on $z$. In that case, there is a
translational symmetry. From Eq. (\ref{eq:noether}), this translates
into conservation of the momentum in the $z$ direction.

\example Consider a pair of particles of mass $m_1$ and $m_2$ where
the potential is of the form
\[
U(\vec{r}_1,\vec{r}_2)=V_a(|m_1\vec{r}_1+m_2\vec{r}_2|/(m_1+m_2))+V_b(|\vec{r}_1-\vec{r}_2|).
\]
Using symmetry arguments alone, are there any conserved components of
the momentum? or the angular momentum??\\

There is no translational invariance, hence there are no conserved
components of the momentum. However, there is rotational invariance
about any axis that goes through the origin. Hence, there is angular
momentum conservation in all three directions. Symmetry arguments are
great ways to recognize the existence of conserved quantities, but
actually expressing them in terms of coordinates can be tricky. For
instance, you may need to write the Lagrangian in terms of angles.

\exampleend

\subsection{Numerically Solving Differential Equations}

Lagrangians lead to equations of motion for the generalized
coordinates. Often, these equations are not analytically solvable, but
are readily addressed computationally. Here, we demonstrate how one
would solve such an equation numerically. So, we first imagine we have
a differential equation for $y(t)$, and for this example we assume the
equation has no derivatives higher than second order. If the boundary
conditions fix $y(0)$ and $y'(0)$, this should be sufficient to
determine $y(t)$ for all future times. Of course, computers need to
discretize the time steps. So assume there are time steps separated by
$\Delta t$ and values,
\begin{eqnarray}
t_n&=&n\Delta t,~=0,\Delta t, 2\Delta t, 3\Delta t\cdots\\ \nonumber
y_n&=&y(t_n).
\end{eqnarray}
Our first step is to solve for $y_{-1}$ and $y_1$ given $y_0$ and
$y'_0$. Approximating the derivatives over the finite intervals,
\begin{eqnarray}
y(0)&=&y_0,\\ \nonumber y'(0)&=&\frac{y_1-y_{-1}}{2\Delta
  t},\\ \nonumber y''(0)&=&\frac{1}{\Delta
  t}\left\{\frac{y_1-y_0}{\Delta t}-\frac{y_0-y_{-1}}{\Delta
  t}\right\} =\frac{y_1-2y_0+y_{-1}}{\Delta t^2}.
\end{eqnarray}
One needs to solve for the three unknowns $y_{-1},y_0$ and
$y_1$. Here, $y(0)$ and $y'(0)$ are given, but $y''(0)$ is not, so a
third equation is needed to solve for the three unknowns. This is
provided by the equations of motion from the Lagrangian. Solving 3
equations and 3 unknowns is thus a fair fight, and one can determine
$y_{-1}\rightarrow y_1$. Once knows these three values, one can use
the differential equation to solve for $y_2$ using $y_0$ and
$y_1$. One can then iteratively find $y_n$ for any $n$ given $y_{n-1}$
and $y_{n-2}$.

\example

Write an algorithm for solving the driven harmonic oscillator,
\[
y''(t)+2\beta y'(t)+\omega_0^2 y(t)=(F_0/m)\cos\omega t.
\]
given $y'(0)=6$ m/s, $y(0)=0.3$ m, and given $m=0.5$ kg, $F_0=400$ N,
and a spring constant $k=2000$ N/m. Let the driving force have a
period of 2.5 s, and the damping rate be $\beta=0.15$ s$^{-1}$.

Let's first solve for $y$ with indices $a,b,c=-1,0,1$, using the
differential equation and the boundary conditions. The BC state that
$y_b=0.3$ m and $(y_c-y_a)=2\Delta t y'_b$, where $y'_b=6$
m/s. Writing the differential equation,
\begin{eqnarray*}
\frac{y_c-2y_b+y_a}{\Delta t^2}+\beta\frac{y_c-y_a}{\Delta
  t}+\omega_0^2 y_b&=&(F_0/m)\cos\omega t_b,
\end{eqnarray*}
one can substitute for $y_a$ because the boundary condition for
$y'_b=y'_0$ gives $y_a=y_{-1}$,
\begin{eqnarray*}
y_a&=&y_c-2\Delta ty'_b,
\end{eqnarray*}
which gives an equation for $y_c$, where everything else in known,
\begin{eqnarray*}
\frac{2y_c-2y_b-2y'_b\Delta t}{\Delta t^2}+2\beta
y'_b+\omega_0^2y_b&=&f_b,\\ \nonumber y_c&=&y_b+y'_b\Delta
t+\left[(f_b/2)-(\omega_0^2y_b/2)-\beta y'_b\right](\Delta t)^2.
\end{eqnarray*}
Here, $f_b\equiv (F_0/m)\cos\omega t$.

Now, that one knows $y_0$ and $y_1$, lets write the differential
equation for three consecutive points $y_a$, $y_b$ and $y_c$, and
solve for $y_c$ in terms of the other two (rather than using the BC as
was done above). Again, beginning with the differential equation,
\begin{eqnarray*}
\frac{y_c-2y_b+y_a}{\Delta t^2}+\beta\frac{y_c-y_a}{\Delta
  t}+\omega_0^2 y_b&=&(F_0/m)\cos\omega
t_b,\\ y_c\left(\frac{1}{\Delta t^2}+\frac{\beta}{\Delta t}\right)
&=&f_b-\omega_0^2y_b+\frac{2y_b-y_a}{\Delta t^2}+\frac{\beta
  y_b}{\Delta
  t},\\ y_c&=&\frac{f_b-\omega_0^2y_b+\frac{2y_b-y_a}{\Delta
    t^2}+\frac{\beta y_b}{\Delta t}}{\frac{1}{\Delta
    t^2}+\frac{\beta}{\Delta t}},\\ &=&\frac{f_b\Delta
  t^2-\omega_0^2y_b\Delta t^2+2y_b-y_a+\beta y_b\Delta
  t}{1+\beta\Delta t}.
\end{eqnarray*}
Using $a=0, b=1, c=2$, this gives $y_2$. Then one can repeat with
$a=1, b=2$ and $c=3$ to find $y_3$, and iterate to find all $n$.

{\bf Solution}:\\ The code might look something like this:
{\tt \begin{verbatim} void main(){ const double PI=4.0*atan(1.0);
    double m=0.5,k=2000.0; double omega0=sqrt(k/m), omega=2.0*PI/2.5,
    beta=0.15; double ya,yb,yc,Dt=0.0025; // Dt = Delta t double
    F0=400,yprime0; // Solve for a,b,c=-1,0,1, solve for yc yb=0.3;
    yprime0=6.0; fb=(F0/m); // Use BC to solve for y_1=yc, from
    lecture notes yc=yb+yprime0*Dt
    +(0.5*fb-0.5*omega0*omega0*yb-beta*yprime0)*Dt*Dt; printf("t=0,
    y=%g\n",yb);

   // Now interatively solve for yc, beginning with yc=y_2
   for(n=0;n<1000;n++){ ya=yb; yb=yc; t=(n+1)*Dt;
     fb=(F0/m)*cos(omega*t);
     yc=(fb*Dt^2-omega0^2*yb*Dt*Dt+2*yb-ya+beta*yb*Dt)/(1+beta*Dt);
     printf("t=%g, y=%g\n",Dt*n,yb); } }
\end{verbatim}}
\hrulefill

\subsection{Exercises}

\begin{enumerate}

\item Consider a hill whose height $y$ is given as a function of the
  horizontal coordinate $x$. Consider a segment of the hill from $x=0$
  to $x=L$ with initial height $y(x=0)=0$ and whose final height is
  $y(x=L)=-h$. Transforming the last equation in Example
  \ref{ex:brachiostone} for a downward vertical force rather than a
  horizontal force,
\[
x=-\sqrt{-2ay-y^2}+a\arccos(1+y/a).
\]
Consider a wheel of radius $a$ rolling along the bottom of the $x$
axis. Mark a point on the top of the wheel, which is originally at the
origin, $x=y=0$, when the top of the wheel touches the origin. As the
wheel rolls by an angle $\theta$ the marked point moves due to both
the translation and the rotation of the wheel. The $y$ coordinate of
the marked point is
\[
y=-a(1-\cos\theta),
\]
whereas the $x$ coordinate is
\[
x=a\theta-a\sin\theta.
\]
The first term is due to the horizontal translation of the axis, while
the second term arises from the rotation of the wheel. Re-express
these two equations to find $x(y)$.

\item Consider a chain of length $L$ that hangs from two supports of
  equal height stretched from $x=-X$ to $x=+X$. The general solution
  for a catenary is
\[
y=\lambda+a\cosh[(x-x_0)/a],
\]
\begin{enumerate}
\item Using symmetry arguments, what is $x_0$?
\item Express the length $L$ in terms of $X$ and $a$.
\item Numerically solve the transcendental equation above to find $a$
  in terms of $L=10$ m and $X=4$ m.
\end{enumerate}

\item Consider a mass $m$ connected to a spring with spring constant
  $k$. Rather than being fixed, the other end of the spring oscillates
  with frequency $\omega$ and amplitude $A$. For a generalized
  coordinate, use the displacement of the mass from its relaxed
  position and call it $y=x-\ell-A\cos\omega t$. In this system the
  potential energy of the spring is $ky^2/2$.
\begin{enumerate}
\item Write the kinetic energy in terms of the generalized coordinate.
\item Write down the Lagrangian.
\item Find the equations of motion for $y$.
\end{enumerate}

\item Consider a bead of mass $m$ on a circular wire of radius
  $R$. Assume a force $kx$ acts on the bead, where $x$ and $y$ axes
  run through the center of the circle. Using $\theta$ as the
  generalized coordinate (measured relative to the $x$ axis),
\begin{enumerate}
\item Write the Lagrangian in terms of $\theta$.
\item Find the equations of motion.
\end{enumerate}

\item Consider a pendulum of length $\ell$ with all the mass $m$ at
  its end. The pendulum is allowed to swing freely in both
  directions. Using $\phi$ to describe the azimuthal angle about the
  $z$ axis and $\theta$ to measure the angular deviation of the
  pendulum from the downward direction, address the following
  questions:
\begin{enumerate}
\item If the pendulum is initially moving horizontally with velocity
  $v_0$ and angle $\theta_0=90^\circ$ (horizontal), use energy and
  angular momentum conservation to find the minimum angles of
  $\theta_{\rm min}$ subtended by the pendulum. (Note that the angle
  will oscillate between $90^\circ$ and the minimum angle.
\item Write the Lagrangian using $\theta$ and $\phi$ as generalized
  coordinates.
\item Write the equations of motion for $\theta$ and $\phi$.
\item Rewrite the equations of motion for $\theta$ using angular
  momentum conservation to eliminate and reference to $\phi$.
\item Find the value of $L$ required for the stable orbit to be at
  $\theta=45^\circ$.
\item For the steady orbit found in (e) consider small perturbations
  of the orbit. Find the frequency with which the pendulum oscillates
  around $\theta=45^\circ$.
\end{enumerate}

\item Consider a mass $m$ that is connected to a wall by a spring with
  spring constant $k$. A second identical mass $m$ is connected to the
  first mass by an identical spring. Motion is confined to the $x$
  direction.
\begin{enumerate}
\item Write the Lagrangian in terms of the positions of the two masses
  $x_1$ and $x_2$.
\item Solve for the equations of motion.
\item Find two solutions of the type
\begin{eqnarray*}
x_1&=&Ae^{i\omega t},~~~x_2=Be^{i\omega t}.
\end{eqnarray*}
Solve for $A/B$ and $\omega$. Express your answers in terms of
$\omega_0^2=k/m$.
\end{enumerate}

\item Consider two masses $m_1$ and $m_2$ interacting according to a
  potential $V(\vec{r}_1-\vec{r}_2)$.
\begin{enumerate}
\item Write the Lagrangian in terms of the generalized coordinates
  $\vec{R}_{\rm cm}=(m_1\vec{r}_1+m_2\vec{r}_2)/(m_1+m_2)$ and
  $\vec{r}=\vec{r}_1-\vec{r}_2$, and their derivatives.
\item Using the independence of the Lagrangian with respect to
  $\vec{R}_{\rm cm}$, find expressions for the conserved total
  momentum.
\end{enumerate}

\end{enumerate}
%\end{document}


\documentclass[12pt,a4paper]{article}
\usepackage{amsmath}
\usepackage{graphicx}
\begin{document}
\title{Optimal state observer for linear systems Lecture Notes}
\author {Fadi Younes}
\maketitle
\newpage
\section*{Overview}
A state observer provides internal state estimation by measuring the Input and output of real given system. The system states are compulsory to solve control engineering problems like state feedback stabilization of system. It is used to reconstruct the system state from output measurement only if system is observable.
\section*{Linear time invariant (LTI) system}
It is assumed that the state of linear time invariant (LTI) physical discrete-time system satisfies, \cite{khalil2002nonlinear}\\
\begin{equation}
x(k+1) = Ax(k) + Bu(k)
\end{equation}
\begin{equation}
y(k) = Cx(k) + Du(k)
\end{equation}\\
Where plant state is represented by x (k) at time ‘k’, u (k) is an input and y (k) is the output of the system. Merely such equations defines that current inputs and current states is solely responsible to determine the future states and current output of plant. However same equations can be used to analyze the continuous time linear invariant system with continuous time step. System observation can be determined by observable matrix and if it is observable then y (k) can be utilized to get states of the system.
\section*{Observer design}
The state observer model of given system is derived from equation 1 and 2. The successive calculated values of input and output of plant is ensured by adding additional term in above equations, so that state converges to plant. The state observer called Luenberger observer, constructed by adding the following term in above equation no 1 \cite{bernat2015multi}.\\
\begin{align*}
L[y(k)-\hat{y(k)}]
\end{align*}
Where observer output is subtracted from plant output and then multiplied by ‘L’ matrix. Then it summarized as,
\begin{equation}
\hat{x(k+1)} = A\hat{x(k)} + L[y(k)-\hat{y(k)}]+ Bu(k)
\end{equation}
\begin{equation}
\hat{y(k)} = C\hat{x(k)} + Du(k)
\end{equation}
The designed observer is stable if and only if error e (k) approaches to zero when discrete time step ‘k’ leads to infinity.
\begin{equation}
e(k)= \hat{x(k)}-x(k)
\end{equation}
\section*{Asymptotic stability}
The Luenberger observer satisfies if and only if error e (k+1) approaches to zero when discrete time step ‘k’ leads to infinity  \cite{pasand2019luenberger}.\\
\begin{equation}
e(k+1)= (A-LC)e(k)
\end{equation}
The asymptotic stability of Luenberger observer is associated with Eigenvalue values of (A-LC) matrix. If eigenvalues lies in inside the unit circle then it is asymptotic stable.
\section*{Optimal minimal order state observer}
The reduced order or minimal order state observer are called optimal observer that gives us least mean square estimation error in all 'n-m' dimensional filter. where n represent the system state dimension to be estimated and 'm' shows that no of available independent outputs of system. Meanwhile kalman filter has dynamic order greater than reduced order observer \cite{pasand2019luenberger}\cite{gautheir1992simple}.
The design of optimum minimum order observer is considered with noise inputs at correlated time steps. Correlated noise is a system noise which included in the estimation problem
\cite{ciccarella199310}.
\section*{Luenberger State Observer}
The control input is added to the system equations for control purpose of whole system. In which, the observer output is supplied back to input of the plant and observer as well by ‘K’ gain matrix \cite{pasand2019luenberger} \cite{hadj2001estimation}.
\begin{equation}
u(k)= -K\hat{x(k)}
\end{equation}
\begin{figure}[h]
  \centering
  \includegraphics[width=4.5 in]{s.PNG}
  \caption{Schematic block diagram}\label{1}
\end{figure}
Finally we achieved Observer equation\\
\begin{equation}
\hat{x(k+1)} = A\hat{x(k)} + L[y(k)-\hat{y(k)}]- BK\hat{x(k)}
\end{equation}
\begin{equation}
\hat{y(k)} = C\hat{x(k)} - DK\hat{x(k)}
\end{equation}
As shown in Figure \ref{1} \cite{vinodh2013comparison}, compact form of the equations summarized as
\begin{equation}
\hat{x(k+1)} = (A- BK)\hat{x(k)} + L[y(k)-\hat{y(k)}]
\end{equation}
\begin{equation}
\hat{y(k)} = (C- DK)\hat{x(k)}
\end{equation}
Here, K and L can be selected independently to avoid any stability loss to the system.  Usually observer poles from (A-LC) are selected for rapid convergence than the state feedback poles from (A-BK).

\section*{Inference}
Lecture notes has discussed the state estimation problem in linear time invariant discrete-time-system using minimal order Luenberger State Observer.
\bibliographystyle{ieeetran}
\bibliography{ref_observer}
\end{document}

student_lecture_notes/state observer (Fadi Younes)/observer_notes.tex


\section{Genetic Algorithm}
	Genetic algorithms are a population based heuristic method that simulates nature's evolutionary system to evolve a starting population of random solutions to obtain very good (possibly optimal) solutions at the end of the process. To do this a random starting population is generated, then each solution is evaluated with a fitness function. After that some solutions, usually the best ones, are chosen to create offsprings that will replace worst solutions. This way the algorithm performs a sort of natural selection keeping the best solutions and improving them over generations.

	As the problem is a variation of the popular Traveling Salesman Problem (TSP), it will be traded as such, so operators and solution representation are often the one used in literature for that problem. Also the important thing in the solution is the order of visited nodes, so crossover and mutations are order preserving, this means that they don't generate many new connections but try te keep connected the nodes that were already connected before.

	\subsection{Encoding}
		To represent a TSP solution the path representation has been used, it consists in a sequence of $N$ nodes, where $N$ is the total number of nodes, sorted from the starting one to the last visited on encoded path.

		For example the path $0 \rightarrow 3 \rightarrow 1 \rightarrow 4 \rightarrow 2 \rightarrow 0$ is encoded as:
		\[[0, 3, 1, 4, 2]\]
		Note that there are some symmetries using this representation:
		\begin{enumerate}
			\item \textit{Rotation symmetries}: By rotating the path we obtain a solution that is equivalent but simply starts from a different node, e. g. [0,3,1,4,2] and [3,1,4,2,0]. These have been removed by fixing starting node to 0.
			\item \textit{Direction symmetry}: By reversing all the elements but the first of a path we get a representation of the inverse path, e.g. [0,3,1,4,2] and [0,2,4,1,3]. Dealing with this symmetry is however not efficient and so it's not been removed.
		\end{enumerate}

	\subsection{Fitness}
		As fitness concerns I took a different approach from usual genetic algorithms and considered the sum of costs in path as the fitness and I will try to minimize it.
		This choice was taken because other considered fitness functions had drawbacks, mainly because most of them were in the form $upperBound - costs$ but upper bounds are not so strict and then a proportional choice based on fitness would yield a very similar probability for each individual (because $maxFitness - minFitness$ is much lower than $upperBound - anyFitness$).

		By setting the fitness function equal to the sum of path costs it it directly proportional to objective function resulting in a more correct reward based on fitness.

	\subsection{Select}
		The select operator is a little tricky because of the fact that solutions with lower fitness should have higher probabilities, so both Montecarlo and Linear ranking methods can't be used. N-Tournament is a valid alternative, but it was discarded because if a subset of $k$ solutions is chosen, the $k-1$ worst solutions are never used for crossover and this reduces (slightly) diversification.

		The final choice is a probability calculated based on position in an array of solutions sorted by fitness. The selected solution is the one with index
		\[index = \left \lfloor{N * r^{2}}\right \rfloor\]
		where N is the total number of nodes and r is a random number $\in [0, 1)$

		This way the solutions with lower cost are more likely to be chosen than the others, but each solution might be selected.

	\subsection{Crossover}
		The crossover operator takes a certain number of parents and creates an offspring that inherits parents properties. In the project was used the Order Crossover (OX), that from 2 parents creates one child, and works like this:

		\begin{figure}[h]
		\includegraphics[width=0.6\textwidth]{ox_crossover}
		\centering
		\caption{Example of Order Crossover (OX)}
		\end{figure}

		\begin{enumerate}
			\item Select a random subsequence of consecutive alleles from parent 1. (underlined)
			\item Drop the subsequence down to Child and mark out these alleles in Parent 2.
			\item Starting on the right side of the subsequence, grab alleles from parent 2 and insert them in Child at the right edge of the subsequence. Since 8 is in that position in Parent 2, it is inserted into Child first at the right edge of the subsequence.\\
				Notice that alleles 1, 2 and 3 are skipped because they are marked out and 4 is inserted into the 2nd spot in Child.
		\end{enumerate}

	\subsection{Mutation}
		Only one type of mutation is used and it is the 2-opt, an algorithm that takes a substring of the path representation and reverse it. The idea is to take a route that crosses over itself and reorder it so that it does not.
		This is good for the main purpose of mutations (prevent genetic drift) as well as for improving solutions.

		\begin{figure}[h]
		\includegraphics[width=0.5\textwidth]{2-opt}
		\centering
		\caption{Example of 2-opt mutation}
		\end{figure}

		To control the mutation there is a parameter MUTATION\_CHANCE that sets the probability that this mutation happens.

	\subsection{Generational Replacement}
		When a population advance to next generation we need to replace the old bad solutions with newly created offsprings. This is done by elitist, so only the best solutions are kept and all the others are replaced by new solutions. The percentage of survivors (and replaced solutions) is set by the parameter SURVIVAL\_RATIO.

	\subsection{Improvements}
		After implementing all the elements of the algorithm and doing some basic tests I found out that the convergence was fast and after a few generations the solution was already quite good and it was rarely improved after. This means that algorithm was iterating through a lot of generation with very few improving solutions, so I decided to introduce random restarts reducing generations count and considering the best solution from all evaluated populations. This approach worked out well to find more reliably good solutions but slowed computation down. To make up for the loss in performance multithread has been used running genetic algorithm on many random starting populations in parallel. It is possible to control the amount of populations evaluated by setting the parameter POPULATIONS.

		Another thing that has been tested is to modify the mutation chance at runtime based on the number of non-improving generations, but this didn't lead to significative improvement of solutions.


\subsection{Algebras over semirings}\label{subsec:algebras_over_semirings}

Algebras are usually defined for fields or at least commutative rings. We extend this to semirings for the purposes of polynomial semirings.

\begin{definition}\label{def:multilinear_function}\mimprovised
  Generalizing \hyperref[def:semimodule/homomorphism]{linear maps}, if \( M_1, \ldots, M_n \) and \( N \) are \( R \)-modules, we say that the function
  \begin{equation*}
    f: M_1 \times \ldots \times M_n \to N
  \end{equation*}
  is \term{multilinear} (\term{bilinear} for \( n = 2 \)) if it is linear in each component. That is, for every tuple
  \begin{equation*}
    (x_1, \ldots, x_n) \in M_1 \times \cdots \times M_n,
  \end{equation*}
  and for every index \( k = 1, \ldots, n \), the following function is linear:
  \begin{equation*}
    y \mapsto f(x_1, \ldots, x_{k-1}, y, x_{k+1}, \ldots, x_n)
  \end{equation*}
\end{definition}

\begin{definition}\label{def:algebra_over_semiring}\mimprovised
  An \term{algebra} over a \hyperref[def:semiring/commutative]{commutative semiring} \( R \) is an \( R \)-\hyperref[def:semimodule]{semimodule} \( M \) with an \hyperref[def:magma/associative]{associative} \hyperref[def:multilinear_function]{bilinear} vector multiplication operation. This makes \( M \) a nonunital ring. By default, we will also assume that \( M \) has a multiplicative unit, although nonunital algebras as just as valid as nonunital rings.

  As in the case of general rings, by \enquote{\( M \) is commutative}, we will mean that vector multiplication is commutative. Furthermore, although we assume it by default, if needed, we will distinguish between associative and non-associative algebra.

  We identify every element \( t \) of \( R \) with its canonical embedding \( t \cdot 1_M \) in \( M \), and thus we can also regard \( R \) as a sub-semiring of \( M \).

  Algebras have the following metamathematical properties:
  \begin{thmenum}
    \thmitem{def:algebra_over_semiring/theory} The \hyperref[def:first_order_theory]{first-order theory} for algebras extends the \hyperref[def:semimodule/theory]{theory of commutative semimodules}. We add a new \hyperref[rem:first_order_formula_conventions/infix]{infix} binary function symbol \( \odot \) to the language, and add to the theory all semiring axioms from \fullref{def:semiring/theory} for \( + \) and \( \odot \). We must also add axioms ensuring that \( \odot \) is bilinear. Additivity follows from distributivity, hence it remains to account for homogeneity. Using the notation of \fullref{def:semimodule/theory}, this amounts to the following axiom schemas:
    \begin{align*}
      m_r(x) \odot y &= m_r(x \odot y), \\
      x \odot m_r(y) &= m_r(x \odot y).
    \end{align*}

    \thmitem{def:algebra_over_semiring/homomorphism} A \hyperref[def:first_order_homomorphism]{first-order homomorphism} between two \( R \)-algebras \( M \) and \( N \) is a linear map that also preserves vector multiplication.

    \thmitem{def:algebra_over_semiring/submodel} The set \( A \subseteq M \) is a \hyperref[thm:substructure_is_model]{submodel} of \( M \) if it is a \hyperref[def:monoid/submodel]{submodule} of \( M \) that is closed under algebra multiplication. We say that \( A \) is a \term{subalgebra}.

    As for general submodules, \fullref{rem:span_over_different_semirings} shows how it is important to be unambiguous about over which semiring we consider the subalgebra.

    As a consequence of \fullref{thm:positive_formulas_preserved_under_homomorphism}, the image of an \( R \)-algebra homomorphism is a subalgebra of its range.

    \thmitem{def:algebra_over_semiring/trivial} The \hyperref[thm:substructures_form_complete_lattice/bottom]{trivial} semimodule is the \hyperref[def:pointed_set/trivial]{trivial pointed set} \( \set{ 0 } \).

    \thmitem{def:algebra_over_semiring/category} We denote the category of algebras over \( R \) by \( \cat{Alg}_R \) and the subcategory of commutative algebras by \( \cat{CAlg}_R \).
  \end{thmenum}
\end{definition}

\begin{proposition}\label{thm:semiring_is_algebra}
  Every \hyperref[def:semiring]{semiring} \( R \) is an \( R \)-\hyperref[def:algebra_over_semiring]{algebra} with both scalar and vector multiplication given by the multiplication in \( R \).

  This extends \fullref{thm:semiring_is_semimodule}.
\end{proposition}
\begin{proof}
  Follows from \fullref{thm:semiring_is_semimodule} by noting that bilinearity follows from left distributivity in \( R \).
\end{proof}

\begin{proposition}\label{thm:semiring_is_natural_number_algebra}
  The categories \( \hyperref[def:semiring/category]{\cat{SRing}} \) of semirings and \( \hyperref[def:algebra_over_semiring/category]{\cat{Alg}_\BbbN} \) of natural number algebras are \hyperref[rem:category_similarity/isomorphism]{isomorphic}.

  Compare this result to \fullref{thm:commutative_monoid_is_semimodule} and \fullref{thm:ring_is_integer_algebra}.
\end{proposition}
\begin{proof}
  Follows from \fullref{thm:commutative_monoid_is_semimodule} by noting that, as in the proof of \fullref{thm:semiring_is_algebra}, distributivity implies bilinearity.
\end{proof}

\begin{proposition}\label{thm:functions_over_algebra}
  For a set \( A \) and an \( R \)-\hyperref[def:algebra_over_semiring]{algebra} \( N \), the set of all functions from \( A \) to \( N \) is itself an \( R \)-algebra with the following operations:
  \begin{thmenum}
    \thmitem{thm:functions_over_algebra/addition} Pointwise addition
    \begin{equation*}
      [f + g](x) \coloneqq f(x) + g(x)
    \end{equation*}

    \thmitem{thm:functions_over_algebra/scalar_multiplication} Pointwise scalar multiplication
    \begin{equation*}
      [t \cdot f](x) \coloneqq t \cdot f(x)
    \end{equation*}

    \thmitem{thm:functions_over_algebra/vector_multiplication} Pointwise vector multiplication
    \begin{equation*}
      [f \odot g](x) \coloneqq f(x) \cdot g(x)
    \end{equation*}

    In practice, we use juxtaposition \( fg \) or \( f \cdot g \) instead of \( f \odot g \).
  \end{thmenum}

  If \( A \) is also an \( R \)-algebra, we denote the set of all \( R \)-\hyperref[def:algebra_over_semiring/homomorphism]{algebra homomorphisms} by \( \hom(A, N) \).

  This result extends \fullref{thm:functions_over_semimodule}.
\end{proposition}
\begin{proof}
  By \fullref{thm:functions_over_model_form_model}, \( N \) is both an \( R \)-semiring and an \( R \)-semimodule. Compatibility comes from left distributivity in \( N \).
\end{proof}

\begin{definition}\label{def:multi_index}\mimprovised
  A \term{multi-index} over the \hyperref[def:set]{plain set} \( \mscrK \) is a member of the \hyperref[def:free_semimodule]{free \( \BbbN \)-semimodule} \( \mscrK^{\oplus \BbbN} \) over \( \mscrK \). We endow \( \mscrK^{\oplus \BbbN} \) with the \hyperref[def:norm]{norm}
  \begin{equation*}
    \norm{ \alpha } \coloneqq \sum_{k \in \mscrK} \alpha_k
  \end{equation*}
  and the \hyperref[def:partially_ordered_set]{partial order}
  \begin{equation*}
    \alpha \leq \beta \T{if and only if} \qforall {k \in \mscrK} \alpha_k \leq \beta_k.
  \end{equation*}

  Multi-indices are \hyperref[def:weighted_set/multiset]{multisets} with extra structure.
\end{definition}

\begin{definition}\label{def:polynomial_algebra}
  Fix a \hyperref[def:semiring/commutative]{commutative semiring} \( R \) and a set \( \mscrX \) of \hyperref[def:formal_language/symbol]{symbols}, which we will call \term{indeterminates}.

  Let \( \mscrM \) be the \hyperref[def:free_semimodule]{free \( R \)-semimodule} over \( \mscrX \), written \hyperref[rem:additive_magma]{multiplicatively}. We will call the members of \( \mscrM \) \term{monomials}. Using a \hyperref[def:multi_index]{multi-index} \( \gamma \) over \( \mscrX \), every monomial can be written as
  \begin{equation*}
    \prod_{X \in \mscrX} X^{\gamma_X},
  \end{equation*}
  where \( \gamma_X \) are the coefficients in \( R^{\oplus \mscrX} \) of the monomial.

  The \term{polynomial algebra} or \term{polynomial semiring} \( R[\mscrX] \) for the given indeterminates is the \hyperref[def:free_semimodule]{free \( R \)-semimodule} over \( \mscrM \). That is, a polynomial \( p \in R[\mscrX] \) is an \( R \)-linear combination of monomials, and we denote polynomials by
  \begin{equation}\label{eq:def:polynomial_algebra/p}
    p(\mscrX) = \sum_\gamma a_\gamma \prod_{X \in \mscrX} X^{\gamma_X}.
  \end{equation}

  We call \( a_\gamma \) the \term{coefficients} of the polynomial. We use the components of the multi-index as powers in the monomials, but we use \( \gamma \) itself as an index for the coefficient \( a_\gamma \). Unfortunately, multi-indices are sometimes confusing, but often their brevity outweighs the possible confusion.

  We do not ignore the structure of \( \mscrM \). We conflate exponentiation in \( \mscrM \) in the sense of \fullref{def:monoid/exponentiation} with exponentiation in \( R[\mscrX] \) in the sense of \fullref{def:semiring/exponentiation}. Multiplication in \( \mscrM \) motivates us to define multiplication in \( R[\mscrX] \) via a convolution of the coefficients. We define the product of \( p(X) \) from \eqref{eq:def:polynomial_algebra/p} with
  \begin{equation}\label{eq:def:polynomial_algebra/q}
    q(\mscrX) = \sum_\gamma b_\gamma \prod_{X \in \mscrX} X^{\gamma_X}
  \end{equation}
  as
  \begin{equation}\label{eq:def:polynomial_algebra/pq}
    [pq](\mscrX) \coloneqq \sum_\gamma \parens*{ \sum_{\delta + \eta = \gamma} a_\delta b_\eta } \prod_{X \in \mscrX} X^{\gamma_X}.
  \end{equation}

  We simultaneously use multi-indices as vectors with pointwise summation (i.e. \( \delta + \eta = \gamma \)) and as indices of coefficients (i.e. \( a_\delta \) and \( b_\eta \)).

  We avoid writing the embedding \( \iota: \mscrX \to R[\mscrX] \), but it is sometimes beneficial to denote it explicitly, for example in \fullref{thm:polynomial_algebra_universal_property}.
\end{definition}

\begin{theorem}[Polynomial algebra universal property]\label{thm:polynomial_algebra_universal_property}
  Fix a \hyperref[def:semiring/commutative]{commutative semiring} \( R \) and a set \( \mscrX \) of indeterminates. The \hyperref[def:polynomial_algebra]{polynomial algebra} \( R[\mscrX] \) is the unique up to a unique isomorphism commutative \hyperref[def:algebra_over_semiring]{algebra} that satisfies the following \hyperref[rem:universal_mapping_property]{universal mapping property}:
  \begin{displayquote}
    For every commutative \( R \)-algebra \( M \) and every function \( e: \mscrX \to M \), there exists a unique \( R \)-algebra homomorphism \( \Phi_e: R[\mscrX] \to M \) such that the following diagram commutes:
    \begin{equation}\label{eq:thm:polynomial_algebra_universal_property/diagram}
      \begin{aligned}
        \includegraphics[page=1]{output/thm__polynomial_semiring_universal_property.pdf}
      \end{aligned}
    \end{equation}
  \end{displayquote}

  The function \( e \) evaluates each indeterminate in \( M \), while \( \Phi_e \) substitutes this value in every polynomial. We call \( \Phi_e \) the \term{substitution homomorphism} corresponding to the \term{variable assignment} \( e \). We can parameterize this by the evaluation functions to obtain the functional evaluation homomorphism
  \begin{equation*}
    \begin{aligned}
      &\Phi: R[\mscrX] \to \fun(M^\mscrX, M) \\
      &\Phi(p) \coloneqq (e \mapsto \Phi_e(p))
    \end{aligned}
  \end{equation*}

  We call the values of \( \Phi \) \term{polynomial functions}. Given elements \( x_1, \ldots, x_n \) of \( M \), we write
  \begin{equation*}
    p(x_1, \ldots, x_n)
  \end{equation*}
  rather than
  \begin{equation*}
    \Phi(p)(x_1, \ldots, x_n).
  \end{equation*}

  Via \fullref{rem:universal_mapping_property}, \( R[\anon*] \) becomes \hyperref[def:category_adjunction]{left adjoint} to the \hyperref[def:concrete_category]{forgetful functor}
  \begin{equation*}
    U: \cat{CAlg}_R \to \cat{Set}.
  \end{equation*}

  The action of \( R[\anon*] \) on morphisms is given by \( \Phi \).
\end{theorem}
\begin{proof}
  For every indeterminate \( X \), we want
  \begin{equation*}
    \Phi_e(\iota(X)) = f(X).
  \end{equation*}

  This suggests defining \( \Phi_e \) for the polynomial
  \begin{equation*}
    p(\mscrX) = \sum_\gamma a_\gamma \prod_{X \in \mscrX} \iota(X)^{\gamma_X}
  \end{equation*}
  as the evaluation
  \begin{equation*}
    \Phi_e(p) \coloneqq \sum_\gamma a_\gamma \prod_{X \in \mscrX} f(X)^{\gamma_X}.
  \end{equation*}

  We discuss well-definedness of infinitary operations in direct sums in \fullref{rem:binary_operation_syntax_trees/infinite/direct_sum}.
\end{proof}

\begin{remark}\label{rem:polynomials_over_infinitely_many_indeterminates}
  As we saw in \fullref{def:polynomial_algebra} and \fullref{thm:polynomial_algebra_universal_property}, there is no formal problem in defining polynomial algebras over infinitely many indeterminates.

  There is a problem, however. Polynomials in one indeterminate, which we will call univariate in accordance to \fullref{def:multi_valued_function/arguments}, have a \hyperref[def:well_ordered_set]{well-ordering} on their monomials, induced by the degree of their monomials. This is defined and discussed in \fullref{subsec:univariate_polynomials}.

  Polynomials in more than one variable do not have a well-ordering by default. If the indeterminates themselves are well-ordered, as is the case for finitely many indeterminates, we may introduce, for example, a \hyperref[def:lexicographic_order]{reverse lexicographic order} on the monomials. Furthermore, for finitely many variables, \fullref{thm:def:polynomial_algebra/iterated} allows us to use \hyperref[rem:induction/peano_arithmetic]{natural number induction} on the number of variables in order to prove statements about multivariate polynomial rings.

  For infinitely many, especially uncountably many variables, however, the theory is seriously crippled by the lack of the tools described above. For this reason, only polynomials in finitely many variables are often considered.
\end{remark}

\begin{proposition}\label{thm:def:polynomial_algebra}
  The following are basic properties of \hyperref[def:polynomial_algebra]{polynomial semirings}:
  \begin{thmenum}
    \thmitem{thm:def:polynomial_algebra/empty} If \( \mscrX \) is empty, \( R[\mscrX] \cong R \).

    \thmitem{thm:def:polynomial_algebra/iterated} The polynomial algebras \( R[\mscrX] \) and \( R[\mscrX \setminus \set{ X_0 }][X_0] \) are isomorphic for any \( X_0 \in \mscrX \) (in case \( \mscrX \) has more than one member).

    In particular,
    \begin{equation*}
      R[X_1, \ldots, X_{n-1}][X_n] \cong R[X_1, \ldots, X_n].
    \end{equation*}

    \thmitem{thm:def:polynomial_algebra/entire} The univariate \hyperref[def:polynomial_algebra]{polynomial semiring} \( R[X] \) is \hyperref[def:divisibility/zero]{entire} if and only if \( R \) is entire.

    \thmitem{thm:def:polynomial_algebra/units} If \( R \) is entire, the \hyperref[def:divisibility/unit]{units} in \( R[X_1, \ldots, X_n] \) are precisely the (embeddings of) the units of \( R \).
  \end{thmenum}
\end{proposition}
\begin{proof}
  \SubProofOf{thm:def:polynomial_algebra/empty} Trivial.

  \SubProofOf{thm:def:polynomial_algebra/iterated} Polynomials in \( R[\mscrX] \) have the form
  \begin{equation*}
    p(\mscrX) = \sum_{k=0}^\infty \sum_\gamma \parens*{ a_{(k,\gamma)} \prod_{\mathclap{X \in \mscrX \setminus \set{ X_0 }}} X^{\gamma_X} } X_0^k,
  \end{equation*}
  where \( \gamma \) is a \hyperref[def:multi_index]{multi-index} on \( \mscrX \).

  Due to associativity, commutativity and distributivity, this can be rewritten as
  \begin{equation*}
    p(\mscrX) = \sum_{k=0}^\infty \parens*{ \sum_\gamma a_{(k,\gamma)} \prod_{\mathclap{X \in \mscrX \setminus \set{ X_0 }}} X^{\gamma_X} } X_0^k.
  \end{equation*}

  This shows how \( R[\mscrX] \) can be embedded into \( R[\mscrX \setminus \set{ X_0 }][X_0] \). This embedding is surjective because the coefficients \( a_\gamma \) range through \( R \). Therefore, the embedding is an isomorphism.

  \SubProofOf{thm:def:polynomial_algebra/entire}

  \SufficiencySubProof Since \( R \) is an \( R \)-subalgebra of \( R[X] \), if the latter is entire, so is the former.

  \NecessitySubProof Suppose that \( R \) is entire and that \( R[X] \) isn't. Then there exist nonzero polynomials \( p(X) \) and \( q(X) \) such that \( p(X) q(X) = 0 \). If \( a_n \) is the leading coefficient of \( p(X) \) and \( b_m \) --- of \( q(X) \), the leading coefficient of \( p(X) q(X) \) is \( a_n b_m \). Since \( p(X) q(X) \) is the zero polynomial, \( a_n b_m = 0 \), which contradicts the assumption that \( R \) is entire.

  Therefore, \( R[X] \) is entire.

  \SubProofOf{thm:def:polynomial_algebra/units} As in \fullref{thm:def:polynomial_algebra/entire}, it is sufficient to prove the statement for one indeterminate.

  Clearly every constant is invertible as a constant polynomial.

  Now suppose that \( p(X) q(X) = 1 \). By definition of multiplication, the product has only one nonzero coefficient. Since \( R \) is entire, it follows that both \( p(X) \) and \( q(X) \) have only one nonzero coefficient, and are hence constants.
\end{proof}

\begin{example}\label{ex:def:polynomial_algebra}
  We list several examples of \hyperref[def:polynomial_algebra]{polynomials} over semirings.
  \begin{thmenum}
    \thmitem{ex:def:polynomial_algebra/natural_numbers} Consider the polynomial \( p(X) \coloneqq aX^2 + bX + c \) in \( \BbbN[X] \). A function from the set \( \set{ X } \) to \( \BbbN \) corresponds to an element of \( \BbbN \), and hence evaluating the polynomial is done by simply replacing \( X \) symbolically in \( p \) and then evaluating the obtained \hyperref[rem:binary_operation_syntax_trees]{syntax tree}.

    We seek the roots of \( p(X) \). We will only formally define roots in \fullref{def:polynomial_root}; for the purposes of the example, a root is a natural number \( n \) such that \( \Phi_n(p) = 0_R \).

    By \fullref{thm:fundamental_theorem_of_algebra} and \fullref{def:algebraically_closed_field/exactly_n_roots}, \( p \) has two roots in the \hyperref[def:set_of_complex_numbers]{complex plane}. That is, we regard \( \BbbC \) as an algebra over \( \BbbN \) and use \fullref{thm:polynomial_algebra_universal_property} to obtain a polynomial function on \( \BbbC \). Furthermore, over the complex numbers the roots can be explicitly found using
    \begin{equation*}
      \frac {-b \pm \sqrt{b^2 - 4ac}} {2a}.
    \end{equation*}

    Finding a root of \( p \) over the natural numbers cannot be done in general, however. If \( p(n) = 0 \), by the ordering of the natural numbers we have
    \begin{equation*}
      p(n) = an^2 + bn + c \geq c,
    \end{equation*}
    and hence \( c \) must necessarily be \( 0 \). If \( c = 0 \), then zero is a root of the polynomial \( p(X) = aX^2 + bX \).

    Now let \( n \) be any root of \( p \). We have
    \begin{equation*}
      an^2 + bn \geq bn,
    \end{equation*}
    and hence \( bn \) must also be \( 0 \). Thus, either \( b = 0 \) or \( n = 0 \). If we want a root other than \( n \), both \( a \) and \( b \) must be \( 0 \).

    Therefore, the only natural number solution to the quadratic equation is \( 0 \), and it is only a solution if \( c = 0 \).

    \thmitem{ex:def:polynomial_algebra/tropical} Consider again the polynomial \( p(X) \coloneqq aX^2 + bX + c \) over \( \BbbN \), but this time evaluated over the \hyperref[def:tropical_semiring]{\( \min \)-plus semiring} \( (\BbbN \cup \set{ \infty }, \min, +) \).

    Expressed via the standard natural number operations, this polynomial becomes
    \begin{equation*}
      \min\set{ 2X + a, X + b, c }.
    \end{equation*}

    This allows us to express certain optimization problems via polynomials.

    This polynomial has a root if and only if \( a = b = c \). Roots in the tropical semiring are not very interesting, however.
  \end{thmenum}
\end{example}

\begin{proposition}\label{thm:generators_via_polynomials}
  For a set \( A \) in an \( R \)-\hyperref[def:algebra_over_semiring]{algebra} \( M \), the \hyperref[def:algebra_over_semiring/submodel]{generated subalgebra} of \( A \), defined as the \( R \)-subalgebra generated by \( A \) in the sense of \fullref{def:first_order_generated_substructure}, equals the set
  \begin{equation*}
    \bigcup \set[\Big]{ R[a_1, \ldots, a_n] \given* a_1, \ldots, a_n \in A }
  \end{equation*}
  obtained by evaluating all multivariate polynomials over \( R \) with elements of \( A \).

  The \( \BbbN \)-subalgebras of \( M \) correspond to \hyperref[def:semiring/submodel]{sub-semirings} and the \( M \)-subalgebras correspond to \hyperref[def:semiring_ideal/generated]{ideals}.

  Compare this result to \fullref{thm:span_via_linear_combinations} for modules.
\end{proposition}
\begin{proof}
  Similar to \fullref{thm:span_via_linear_combinations}.
\end{proof}

\begin{proposition}\label{thm:adjoining_elements_to_semiring}
  Let \( R \subseteq S \) be \hyperref[def:semiring/commutative]{commutative semirings} and let \( A \subseteq S \) be an arbitrary subset.

  Fix a set \( \mscrX \) of indeterminates and a bijective function \( e: \mscrX \to A \) and consider the \hyperref[thm:polynomial_algebra_universal_property]{evaluation homomorphism}
  \begin{equation*}
    \Phi_e: R[\mscrX] \to S.
  \end{equation*}

  The image \( R[A] \) of \( \Phi_e \) is the smallest super-semiring of \( R \) that contains \( A \).

  We say that \( R[A] \) is obtained by \term{adjoining} the elements of \( A \) to \( R \).
\end{proposition}
\begin{proof}
  Follows from \fullref{thm:generators_via_polynomials}.
\end{proof}

\begin{example}\label{ex:adjoining_root}
  Continuing \fullref{ex:def:polynomial_algebra/natural_numbers}, consider the polynomial equation
  \begin{equation*}
    X + 1 = 0.
  \end{equation*}

  It has no natural number root as a consequence of \eqref{eq:def:peano_arithmetic/PA2}.

  It does have an integer root, however, \( -1 \). We can \hyperref[thm:adjoining_elements_to_semiring]{adjoin} \( -1 \) to the semiring \( \BbbN \) to obtain the semiring \( \BbbN[-1] \). But this latter semiring is (isomorphic to) \( \BbbZ \).

  Therefore, \( \BbbZ \) is the smallest extension of \( \BbbN \) that contains a root to the polynomial \( X + 1 \).

  This example extends to the theory of \hyperref[def:transcendetal_element]{transcendental and algebraic} elements of fields.
\end{example}

\begin{definition}\label{def:formal_power_series}\mimprovised
  If we extend the concept of \hyperref[def:polynomial_algebra]{polynomials} to allow countably many nonzero terms, we obtain a set \( R\Bracks{\mscrX} \) which we call the \term{formal power series} over \( R \) with indeterminates from the set \( \mscrX \).

  The evaluation homomorphism defined in \fullref{thm:polynomial_algebra_universal_property} is problematic, however, since algebraic operations are finitary by nature. This is discussed in \fullref{rem:binary_operation_syntax_trees/infinite}, along with how sometimes we can make sense of infinitary algebraic operations.
\end{definition}


\chapter{Vectors}
\label{chp:vector}
\label{sec:vector}

Pick two points \(A\) and \(B\).

The vector \(AB\) tells us how to go from \(A\) to \(B\).

The vector \(AB\) is the subtraction \( B - A \).

We draw the vector \(AB\) as a straight arrow from \(A\) to \(B\).

If we know \(A\) and \(B\), then we can find \(AB\).

If we are at \(A\), then following \(AB\) leads us to \(B\).
We notate this fact as \( A + AB = B \).
We write the point on the left and the vector on the right:
write \( A + AB \), and don't write \( AB + A \).

The zero vector \(0\) tells us how to get to the same point:
\(A + 0 = A\).

The length of a vector \(v\) is written \(\norm{v}\).

A unit vector is a vector whose length is one.

\section*{Adding two vectors}

If \(AB\) tells us how to go from \(A\) to \(B\)
and \(BC\) tells us how to go from \(B\) to \(C\),
then \(AC = AB + BC\)
tells us how to go from \(A\) to \(C\).

We can see that \( a + b = b + a \) by drawing a parallelogram.
We say that vector addition is commutative
because commuting (swapping) the arguments doesn't change the result.

\section*{Negating a vector}

The negation of a vector \(v\) is \(-v\).
It is the vector such that the sum \(v + (-v)\) is the zero vector \(0\).

If we know the vector \(AB\), then we know \(BA\).
We write \(BA = -AB\).
If \(AB\) tells us how to go from \(A\) to \(B\),
then \(BA\) tells us how to go from \(B\) to \(A\).
\(BA\) is the reverse of \(AB\).

\(BA\) is also called the negation of \(AB\) because \(AB + BA = 0\) is the zero vector.
\(A + 0 = A\).

Negating a vector preserves its length:
\(\norm{-AB} = \norm{BA} = \norm{AB}\).

\section{Vector spaces}

A vector space is a set of vectors.

An example vector space is the set of all two-dimensional Euclidean vectors.
We can imagine this vector space as the set of all arrows that we can draw on an unbounded flat sheet of paper.
We don't care about where the arrow is.
We only care about its length and direction.
If two arrows have the same length and direction,
then they represent the same vector.

An example vector space is the space of all two-dimensional Euclidean vectors.

\section{Collinearity and linear combination}

Two vectors \(a\) and \(b\) are collinear
iff they have the same direction or the opposite direction.
If we place them so that their origins coincide, they form a straight line segment.

The zero vector is collinear with every vector.
% This simplifies the definition of basis later.
% Now we can write \enquote{non-collinear} instead of \enquote{non-zero non-collinear}.

See also Wikipedia\footnote{\url{https://en.wikipedia.org/wiki/Collinearity}}.

\paragraph{Linear combination of two vectors}

Let \(p\) and \(q\) be vectors.
A linear combination of \(p\) and \(q\) is a vector \( ap+bq \)
where \(a\) and \(b\) are real numbers.
In other words, we say that \(r\) is a linear combination of \(p\) and \(q\)
iff there exists \(a,b\in \Real\) such that \(r = ap+bq\).

\section{External resources}

See Wikipedia%
\footnote{\url{https://en.wikipedia.org/wiki/Euclidean_vector}}%
\footnote{\url{https://en.wikipedia.org/wiki/Vector_(mathematics_and_physics)}}%
.

\section{What?}

\paragraph{Finding the angle between two vectors}

For every pair of vectors,
we can always find a plane such that both of them lie on that plane.
The plane defines the angle.

\paragraph{Deciding orthogonality by dot product}

Let \(a\) be a vector.

Let \(b\) be vector.

Let the angle from \(a\) to \(b\) is \(\theta\). Positive means counterclockwise.

The dot product is \(a \cdot b = \norm{a} \cdot \norm{b} \cdot \cos \theta\).

Note that the dot symbol \(\cdot\) is overloaded.
It may mean real multiplication,
vector scaling,
vector dot product,
or something else.
Its meaning depends on the things around it.

If \(\norm{a} \cdot \norm{b} \neq 0\) and \(a \cdot b = 0\),
then \(a\) and \(b\) are orthogonal.
Two orthogonal vectors form a right angle.


\section{Backpropagation} \label{sec:backpropagation}
Last section showed how learning in a neural network is a minimization problem on the loss function and that it is solved by repeatedly updating the network's parameters using the gradient of the loss. But how exactly is the gradient calculated? The loss function is a surface in very high dimensions, calculated by averaging some distance function between the network's output and the true output for all inputs on the dataset; and the output of the network is a mapping calculated by passing the input through possible thousands, millions, or more units, where each one can apply a nonlinearity to its output. In summary, the loss surface is extremely complex and the same should be expected for its gradient.

To make it simpler to understand how the gradient is calculated, the procedure will be shown only for the weights and biases parameters, since those are present in practically all neural networks (although sometimes the bias is omitted in some units). This section will suppose a neural network with $L+1$ layers, where layer $0$ is the input, layer $L$ is the output and values in between are hidden layers.

Essentially, calculating the gradients consists of a smart application of the chain rule of calculus. Recall that the chain rule is a way of calculating the derivative of composite functions, this means that for a function $y$ that depends on $t$, and $t$ that depends on $x$, then the chain rule allows for calculating the derivative of $y$ with respect to $x$ by compounding how $x$ changes $t$ and how $t$ changes $y$. For the case of single variable functions the chain rule is given by \autoref{eq:chain_rule} \cite[p. 406]{calculusIII2016}.
\begin{equation} \label{eq:chain_rule}
    \frac{dy}{dx} = \frac{dy}{dt} \frac{dt}{dx}
\end{equation}

But for the case of neural networks, the loss function is dependent on all the activations of the output layer, and those are dependent on their weights, biases, and possibly other parameters, besides being dependent on activations of the previous layer. So it is necessary to use the multi-variable generalization of the chain rule shown in \autoref{eq:chain_rule_general} \cite[p. 412]{calculusIII2016}, here it is necessary to compound the effect of $x$ for all the variables $(t_0, t_1, \dots t_n)$ that $y$ is dependent on.
\begin{equation} \label{eq:chain_rule_general}
    \frac{\partial y}{\partial x} = \sum_{i}^{n}{\frac{\partial y}{\partial t_i} \frac{\partial t_i}{\partial x}}
\end{equation}

Having the chain rule in mind, it is also useful to define an additional term $\delta$ that represents the partial derivative of the loss with respect to the weighted input of a unit. For unit $i$ on layer $l$ this term is given by \autoref{eq:neuron_delta}.
\begin{equation} \label{eq:neuron_delta}
    \delta_{i}^{(l)} = \frac{\partial J}{\partial z_i^{(l)}}
\end{equation}

And lastly, note that by differentiating Equations \ref{eq:activation_again} and \ref{eq:dense_weighted_input}, the following relations are obtained.
\begin{equation*}
    \frac{\partial z_i^{(l+1)}}{\partial a_i^{(l)}} = w_{ij}^{(l+1)}
    \qquad
    \qquad
    \frac{\partial z_i^{(l)}}{\partial b_i^{(l)}} = 1
    \qquad \qquad
    \frac{\partial z_i^{(l)}}{\partial w_{ij}^{(l)}} = a_j^{(l-1)}
    \\[2pt]
\end{equation*}

The main idea of this algorithm is to derive $\delta$ for all layers, then use these values to calculate the gradient terms for all parameters, the first step is to calculate $\delta$ in the last layer. For the following derivation, consider $\hat{\bm{y}}$ as the network output, and notice that it is the same as the activations of the output layer $\bm{a}^{(L)}$. By using this knowledge, \autoref{eq:delta_last_layer} is derived as follows.
\begin{align}
    \frac{\partial J}{\partial z_i^{(L)}} = \delta_{i}^{(L)} &=
    \frac{\partial J}{\partial \hat{y_i}} \frac{\partial \hat{y_i}}{\partial z_i^{(L)}} \nonumber \\[10pt]
    %
    &= \frac{\partial J}{\partial \hat{y_i}} \frac{\partial a_i^{(L)}}{\partial z_i^{(L)}} \nonumber \\[10pt]
    %
    &= \frac{\partial J}{\partial \hat{y_i}} \frac{\partial}{\partial z_i^{(L)}} {f\left(z_i^{(L)}\right)} \nonumber \\[10pt]
    %
    \delta_{i}^{(L)} &= \frac{\partial J}{\partial \hat{y_i}} f'\left(z_i^{(L)}\right) \label{eq:delta_last_layer}
\end{align}

Recall that the loss function $J$ and activation function $f$ should both be continuous and differentiable, and since they are chosen when building the network their derivatives are known. The values $\hat{y_i}$ and $z_i^{(L)}$ are also known since they are calculated by the network and can be easily stored during training. This means that \autoref{eq:delta_last_layer} can be used to calculate all $\delta$ values in the last layer.

By using \autoref{eq:delta_last_layer} it is also possible to find an expression to calculate all the other $\delta$ values, the following derivation shows how this can be done by writing $\delta^{(l)}$ in terms of $\delta^{(l+1)}$ as shown in \autoref{eq:delta_hidden_layer}.
\begin{align}
    \frac{\partial J}{\partial z_j^{(l)}} = \delta_j^{(l)} &= \sum_{i}{
        \frac{\partial J}{\partial z_i^{(l+1)}}
        \frac{\partial z_i^{(l+1)}}{\partial a_j^{(l)}}
        \frac{\partial a_j^{(l)}}{\partial z_j^{(l)}}
    } \nonumber \\[10pt]
    %
    &= \sum_{i}{
        \delta_i^{(l+1)}
        \frac{\partial z_i^{(l+1)}}{\partial a_j^{(l)}}
        \frac{\partial a_j^{(l)}}{\partial z_j^{(l)}}
    } \nonumber \\[10pt]
    %
    &= \sum_{i}{
        \delta_i^{(l+1)}
        w_{ij}^{(l+1)}
        \frac{\partial a_j^{(l)}}{\partial z_j^{(l)}}
    } \nonumber \\[10pt]
    %
    \delta_j^{(l)} &= \sum_{i}{
        \delta_i^{(l+1)}
        w_{ij}^{(l+1)}
        f'\left( z_j^{(l)} \right)
    } \label{eq:delta_hidden_layer}
    %
\end{align}

Notice how the algorithm works, first the input is feedforwarded through the network to obtain the output $\hat{\bm{y}}$, this value is used to calculate $\delta^{(L)}$, that is then \textit{backpropagated} through the network in order to calculate $\delta^{(l)}$ for all previous layers. This process gives the name \textit{Backpropagation} to the algorithm.

Now for calculating the gradients using $\delta$. Notice that for this case, where only the weights and biases are being considered, the gradient depends on the change $\partial J$ with respect to $\partial b_i^{(l)}$ and $\partial w_{ij}^{(l)}$ for all units and layers.

The following derivation applies the chain rule to obtain the relation in \autoref{eq:gradient_bias} for the partial derivatives of the loss with respect to all the biases parameters.
\begin{align}
    \frac{\partial J}{\partial b_i^{(l)}} &= \sum_{k}{
        \frac{\partial J}{\partial z_k^{(l)}}
        \frac{\partial z_k^{(l)}}{\partial b_i^{(l)}}
    } \nonumber \\
    %
    &= \frac{\partial J}{\partial z_i^{(l)}} \frac{\partial z_i^{(l)}}{\partial b_i^{(l)}} +
    \sum_{k \neq i}{
        \frac{\partial J}{\partial z_k^{(l)}}
        \cancelto{0}{\frac{\partial z_k^{(l)}}{\partial b_i^{(l)}}}
    } \nonumber \\[10pt]
    %
    &= \frac{\partial J}{\partial z_i^{(l)}} \nonumber \\[10pt]
    %
    \frac{\partial J}{\partial b_i^{(l)}} &= \delta_{i}^{(l)} \label{eq:gradient_bias}
\end{align}

The derivation for \autoref{eq:gradient_bias} first breaks the partial derivative of the cost in terms of the weighted inputs $\bm{z}^{(l)}$ in the layer where the bias is present. This is enough since the bias can not influence any previous layers and all influences in the next layers are already captured in the change $\partial J$ with respect to the weighted inputs $\partial z_k^{(l)}$. Since it is also known that the bias does not influence any other unit in the layer, the derivation could have been made directly without breaking the derivative into a sum of all the terms in the layer, but the whole process was shown here for completion sake.

A similar rationale can be used for the weight parameters, obtaining the relation seen in \autoref{eq:gradient_weight}.
\begin{align}
    \frac{\partial J}{\partial w_{ij}^{(l)}} &= \sum_{k}{
        \frac{\partial J}{\partial z_k^{(l)}}
        \frac{\partial z_k^{(l)}}{\partial w_{ij}^{(l)}}
    } \nonumber \\
    %
    &= \frac{\partial J}{\partial z_i^{(l)}} \frac{\partial z_i^{(l)}}{\partial w_{ij}^{(l)}} +
    \sum_{k \neq i}{
        \frac{\partial J}{\partial z_k^{(l)}}
        \cancelto{0}{\frac{\partial z_k^{(l)}}{\partial w_{ij}^{(l)}}}
    } \nonumber \\[10pt]
    %
    &= \frac{\partial J}{\partial z_i^{(l)}} a_{j}^{(l-1)} \nonumber \\[10pt]
    %
    \frac{\partial J}{\partial w_{ij}^{(l)}} &= \delta_{i}^{(l)} a_{j}^{(l-1)} \label{eq:gradient_weight}
\end{align}

\autoref{eq:gradient_weight} was the last piece of the puzzle, together with Equations \ref{eq:delta_last_layer}, \ref{eq:delta_hidden_layer}, and \ref{eq:gradient_bias}, it can be applied to calculate the gradient for all parameters in the network, and this allows for gradient descent to update the parameters and minimize the loss function. From data to model, a complete procedure for a machine to learn by itself.

There are still some more concepts that will be briefly explored in the next section. One further detail to mention about backpropagation is the fact that the derivations in this section were only made for the weights and biases parameters, what about possible others? There are many different types of additional parameters, but the idea with backpropagation is that the $\delta$ values are already calculated for all the units in the network, any new parameter $\theta$ must simply have a correlation with some of these values in order to obtain $\partial J$ in terms of $\partial\theta$ and apply gradient descent for updates. Modern libraries and frameworks already abstract most of these calculations for the programmer via underlying procedures of automatic differentiation, for example, Tensorflow \cite{tensorflow2015} provides a \texttt{GradientTape} object to automatically watch and calculate the gradients for any desired parameter.