\documentclass[11pt]{amsart}
\usepackage{amsmath,amsfonts,amsthm,amssymb, amsaddr}

\title{Maxwell's Equations}

\author{Joe Bentley}

\date{\today}

\begin{document}

\maketitle

\newpage

\section{Electrostatics}

Consider the electric field $\mathbf{E}(\mathbf{r})$ at a point $\mathbf{r}$ due to a charge distribution $\rho(\mathbf{r'})$. The point $\mathbf{r}$ is the point at which we want to measure the electric field, and the point $\mathbf{r'}$ is the point at which the source of the charge we are considering resides. The infinitesimal charge at that point is given by,

\begin{align*}
  dq' = \rho(\mathbf{r'}) dV'
\end{align*}

The infinitesimal volume $dV'$ can also be written as $dV' = d^3\mathbf{r'}$. Coulomb's law gives us that the infinitesimal electric field at point $\mathbf{r}$ is,

\begin{align*}
  d\mathbf{E}(\mathbf{r}) &= \frac{dq'}{4\pi\epsilon_0 {|\mathbf{r}-\mathbf{r'}|}^2} \frac{(\mathbf{r} - \mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|} \\
  &= \frac{\rho(\mathbf{r'})(\mathbf{r}-\mathbf{r'})}{4\pi\epsilon_0{|\mathbf{r}-\mathbf{r'}|}^3} dV' \\
  \mathbf{E}(\mathbf{r}) &= \frac{\rho(\mathbf{r'})(\mathbf{r}-\mathbf{r'})}{4\pi\epsilon_0{|\mathbf{r}-\mathbf{r'}|}^3} dV'
\end{align*}

In the second line we substitute in our expression for an infinitesimal charge element $dq' = \rho(\mathbf{r'}) dV'$. The divergence of the electric field is then given by,

\begin{align*}
  \nabla\cdot\mathbf{E}(\mathbf{r}) &= \int\frac{\rho(\mathbf{r'})}{4\pi\epsilon_0} \nabla r \cdot \frac{(\mathbf{r}-\mathbf{r'})}{{|\mathbf{r}-\mathbf{r'}|}^3} d^3\mathbf{r'}
  &= \int \frac{\rho(\mathbf{r'})}{4\pi\epsilon_0} 4\pi\delta^3(\mathbf{r} - \mathbf{r'}) d^3\mathbf{r'}
\end{align*}

In the distributions notes we showed that $\nabla\cdot\frac{\mathbf{r}}{r^3}=4\pi\delta^3(\mathbf{r})$, which is what we wrote on the second line of the last equation. By cancelling the values of $4\pi$ and then integrating the delta function over volume, we obtain our first Maxwell equation,

\begin{align*}
  \nabla\cdot\mathbf{E}(\mathbf{r}) = \frac{\rho(\mathbf{r})}{\epsilon_0}
\end{align*}

If we integrate both sides of this over a volume $V$, bounded by a surface $S = \partial V$ and apply the divergence theorem we get,

\begin{align*}
  \int_V \nabla\cdot\mathbf{E} dV = \int_V \frac{\rho(\mathbf{r})}{\epsilon_0} dV
\end{align*}

Which gives us an equation known as Gauss' law,

\begin{align*}
  \oint_S \mathbf{E}\cdot d\mathbf{S} = \frac{Q}{\epsilon_0}
\end{align*}

We can show that the curl of the electric field is zero, $\nabla\times\mathbf{E}=0$, since we can write the electric field as the gradient of a scalar field $\mathbf{E} = -\nabla\phi$, where $\phi(r)$ is given by,

\begin{align*}
  \phi(r) = \int_V \frac{\rho(\mathbf{r'})d^3\mathbf{r'}}{4\pi\epsilon_0 |\mathbf{r} - \mathbf{r'}|}
\end{align*}

We therefore know that the curl of the electric field must be zero, from the vector identity that the curl of a gradient of a scalar field is always zero $\nabla\times(\nabla\phi) = \mathbf{0}$. However this is only true in the static case, where there is no time dependence in the electric or magnetic fields. We will explore the electromagnetic case later.

\section{Magnetostatics}

The Biot-Savart law gives us that the magnetic field $\mathbf{B}(\mathbf{r})$ due to a current density distribution $\mathbf{j}(\mathbf{r'})$ is given by,

\begin{align*}
  \mathbf{B}(\mathbf{r})=\frac{\mu_0}{4\pi}\int_V\frac{\mathbf{j}(\mathbf{r'})\times(\mathbf{r}-\mathbf{r'})}{{|\mathbf{r}-\mathbf{r'}|}^3} d^3\mathbf{r'}
\end{align*}

The cross product shows us that the magnetic field $\mathbf{B}(\mathbf{r})$ will be perpendicular to both the current density $\mathbf{j}(\mathbf{r'})$ (and thus the current) and the radial direction $(\mathbf{r} - \mathbf{r'})$.

We can rewrite this by noting that if a vector $\mathbf{c}$ is constant in space, then we can write,

\begin{align*}
  \nabla\times(\phi\mathbf{c}) = \nabla\phi\times\mathbf{c}+\phi\nabla\times\mathbf{c} = \nabla\phi\times\mathbf{c}
\end{align*}

The second term $\phi\nabla\times\mathbf{c} = 0$ because the curl of a constant vector is zero, since it is unchanging in space so the derivative of each component will be zero. In this case our constant vector is given by,

\begin{align*}
  \mathbf{c} = \frac{(\mathbf{r}-\mathbf{r'})}{{|\mathbf{r}-\mathbf{r'}|}^3}
\end{align*}

and then by applying this to the expression inside the integral in the Biot-Savart law,

\begin{align*}
  \mathbf{j}(\mathbf{r'})\times\frac{(\mathbf{r}-\mathbf{r'})}{{|\mathbf{r}-\mathbf{r'}|}^3} &= -\frac{(\mathbf{r}-\mathbf{r'})}{{|\mathbf{r}-\mathbf{r'}|}^3}\times \\
  &= \left(\nabla_r \frac{1}{|\mathbf{r}-\mathbf{r'}|}\right)\times\mathbf{j}(\mathbf{r'}) \\
  &= \nabla_r\times\left(\frac{\mathbf{j}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}\right)
\end{align*}

By applying these results, the Biot-Savart law may be rewritten as,

\begin{align*}
  \mathbf{B}(\mathbf{r}) = \frac{\mu_0}{4\pi}\int_V\nabla_r\times\frac{\mathbf{j}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|} d^3\mathbf{r'}
\end{align*}

It follows that we can write the magnetic field as,

\begin{align*}
  \mathbf{B}(\mathbf{r}) = \nabla\times\mathbf{A}(\mathbf{r})
\end{align*}

where,

\begin{align*}
  \mathbf{A}(\mathbf{r}) = \frac{\mu_0}{4\pi}\int_V\frac{\mathbf{j}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}d^3\mathbf{r'}
\end{align*}

We immediately see from our vector identities, that taking the divergence of the magnetic field is taking the divergence of the curl of a vector field, which our vector identities show is zero, $\nabla\cdot(\nabla\times\mathbf{A})=0$. Therefore the divergence of the magnetic field is zero,

\begin{align*}
  \nabla\cdot\mathbf{B}(\mathbf{r}) = 0
\end{align*}

which is another of Maxwell's equations.

The curl of the magnetic field can be calculated using the vector identity,

\begin{align*}
  \nabla\times\mathbf{B}=\nabla\times(\nabla\times\mathbf{A})=\nabla(\nabla\cdot\mathbf{A})-\nabla^2\mathbf{A}
\end{align*}

First we will calculate the laplacian of the vector field, as this is much easier than the first term,

\begin{align*}
  \nabla_r^2\mathbf{A}(\mathbf{r})&=\frac{\mu_0}{4\pi}\int_V\left(\nabla_r^2\frac{1}{|\mathbf{r}-\mathbf{r'}|}\right)\mathbf{j}(\mathbf{r'})d^3\mathbf{r'} \\
  &= \frac{\mu_0}{4\pi}\int_V -4\pi\delta^3(\mathbf{r}-\mathbf{r'})\mathbf{j}(\mathbf{r'})d^3\mathbf{r'} \\
  &= -\mu_0\mathbf{j}(\mathbf{r'})
\end{align*}

Next we need to calculate the first term. First we will try and work out a nice form for the divergence of $\mathbf{A}$,

\begin{align*}
  \nabla_r\cdot\int_V\frac{\mathbf{j}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}d^3\mathbf{r'}=\int_V\nabla_r\cdot\left[\frac{\mathbf{j}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}\right]d^3\mathbf{r'}
\end{align*}

Much as with the curl of a scalar field multiplied by a constant vector, we can write $\nabla\cdot(\phi\mathbf{c})=(\nabla\phi)\cdot\mathbf{c}+\phi\nabla\cdot\mathbf{c}=(\nabla\phi)\cdot\mathbf{c}$. We know that the second term is zero, since the divergence of a constant vector must be zero due to the fact that vector is unchanging in space. We can therefore write,

\begin{align*}
  \int_V\nabla_r\left[\frac{\mathbf{j}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}\right]d^3\mathbf{r'}=\int_V\left[\nabla_r\frac{1}{|\mathbf{r}-\mathbf{r'}|}\right]\cdot\mathbf{j}(\mathbf{r'})d^3\mathbf{r'}
\end{align*}

We want to know this in terms of $\nabla_{r'}$ instead of $\nabla_r$. We can write,

\begin{align*}
  \nabla_{r'} \frac{1}{|\mathbf{r}-\mathbf{r'}|} = \frac{(\mathbf{r}-\mathbf{r'})}{{|\mathbf{r}-\mathbf{r'}|}^3} = -\nabla_r\frac{1}{|\mathbf{r}-\mathbf{r'}|}
\end{align*}

Using this we can write,

\begin{align*}
  \nabla_r\cdot\int_v\frac{\mathbf{j}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}d^3\mathbf{r}=-\int_V\left[\nabla_{r'}\frac{1}{|\mathbf{r}-\mathbf{r'}|}\right]\cdot\mathbf{j}(\mathbf{r'})d^3\mathbf{r'}
\end{align*}

Here we can use the vector identity, $\nabla\cdot(\phi\mathbf{A})=(\nabla\phi)\cdot\mathbf{A}+\phi\nabla\cdot\mathbf{A}$, so that we can write,

\begin{align*}
  \nabla_{r'}\cdot\left[\frac{\mathbf{j}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}\right] = \left[\nabla_{r'}\frac{1}{|\mathbf{r}-\mathbf{r'}|}\right]\cdot\mathbf{j}(\mathbf{r'}) + \frac{1}{|\mathbf{r}-\mathbf{r'}|}\nabla_{r'}\cdot\mathbf{j}(\mathbf{r'})
\end{align*}

Subtituting this back into our integral we obtain,

\begin{align*}
  \nabla_r\cdot\int\frac{\mathbf{j}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}d^3\mathbf{r'}&=-\int_V\nabla_{r'}\cdot\left[\frac{\mathbf{j}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}\right]d^3\mathbf{r'} + \int_V\frac{\nabla_{r'}\cdot\mathbf{j}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}d^3\mathbf{r'} \\
  &=-\oint_S\frac{\mathbf{j}(\mathbf{r'})\cdot d\mathbf{S'}}{|\mathbf{r}-\mathbf{r'}|} + \int_V\frac{\nabla_{r'}\cdot\mathbf{j}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}d^3\mathbf{r'}
\end{align*}

For the first term we have used the divergence theorem to convert the volume integral into a surface intagral at infinity. Since $\mathbf{j}(\mathbf{r'})=\mathbf{0}$ along the surface, this integral will be zero.

From the continuity equation we have that,

\begin{align}
  \label{eq:continuity}
  \nabla_{r'}\cdot\mathbf{j}(\mathbf{r'}, t) = -\frac{\partial\rho(\mathbf{r'}, t)}{\partial t} = 0
\end{align}

This is zero as charge (and thus current) is always conserved.

Since we have already shown that the first integral is zero, and this shows that the second integral is zero, we therefore have that in the static case,

\begin{align*}
  \nabla_{r'}\cdot\mathbf{A}(\mathbf{r}) = \frac{\mu_0}{4\pi}\int_V\nabla_r\cdot\left[\frac{\mathbf{j}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}\right]d^3\mathbf{r'} = 0
\end{align*}

Therefore, in the case where the magnetic and electric fields are constant in time, we have that,

\begin{align*}
  \nabla\times\mathbf{B}(\mathbf{r})&=\nabla(\nabla\cdot\mathbf{A})-\nabla^2\mathbf{A} \\
  &=\mu_0\mathbf{j}(\mathbf{r})
\end{align*}

In the general, non static case, we calculate the divergence of $\mathbf{A}$ as,

\begin{align*}
  \nabla_r\cdot\mathbf{A}(\mathbf{r}, t)&=\frac{\mu_0}{4\pi}\int_V\nabla_r\cdot\left[\frac{\mathbf{j}(\mathbf{r'}, t)}{|\mathbf{r}-\mathbf{r'}|}\right]d^3\mathbf{r'} \\
  &=\frac{\mu_0}{4\pi}\int_V\frac{\nabla_{r'}\cdot\mathbf{j}(\mathbf{r'}, t)}{|\mathbf{r}-\mathbf{r'}|}d^3\mathbf{r'} \\
  &=-\frac{\mu_0}{4\pi}\frac{\partial}{\partial t}\int_V\frac{\rho(\mathbf{r'}, t)}{|\mathbf{r}-\mathbf{r'}|}d^3\mathbf{r'}
\end{align*}

In the third line we have used the continuity equation, eq.~\ref{eq:continuity}. By taking the gradient of this we get the result we need,

\begin{align*}
  \nabla_r\left(\nabla_r\cdot\mathbf{A}(\mathbf{r}, t)\right) &=\frac{\mu_0}{4\pi}\frac{\partial}{\partial t}\int_{V'}\frac{\rho(\mathbf{r'}, t)}{{|\mathbf{r}-\mathbf{r'}|}^3}d^3\mathbf{r'} \\
  &= \mu_0\epsilon_0\frac{\partial\mathbf{E}(\mathbf{r}, t)}{\partial t}
\end{align*}

We therefore have, in the general case where the magnetic and electric field need not be constant,

\begin{align*}
  \nabla\times\mathbf{B}=\mu_0\mathbf{j}+\mu_0\epsilon_0\frac{\partial\mathbf{E}}{\partial t}
\end{align*}

\section{Faraday's Law}

Our final (and easiest) equation to find is the non-static form of $\nabla\times\mathbf{E}$. This is a very simple equation to find as it is not much more than a restatement of an empirical law known as Faraday's law.

Faraday's law tells us that the emf induced in a loop $C$ is equal to the rate of change of the magnetic flux through the loop,

\begin{align*}
  \epsilon &= -\frac{d\Phi_B}{dt}
\end{align*}

Where the magnetic flux is defined as $\Phi_B = \int_S\mathbf{B}\cdot d\mathbf{S}$. The emf is also defined by the integral of the electric field around the closed loop,

\begin{align*}
  \epsilon &= \oint_C\mathbf{E}\cdot d\mathbf{r}
\end{align*}

Using these with Faraday's law we have that,

\begin{align*}
  \oint_C\mathbf{E}\cdot d\mathbf{r}=-\frac{\partial}{\partial t}\int_S\mathbf{B}\cdot d\mathbf{S}=-\int_S\frac{\partial\mathbf{B}}{\partial t}\cdot d\mathbf{S}
\end{align*}

By applying Stokes' theorem we have,

\begin{align*}
  \int_S(\nabla\times\mathbf{E})\cdot d\mathbf{S} = -\int_S\frac{\partial\mathbf{B}}{\partial t}\cdot d\mathbf{S}
\end{align*}

Therefore we have our final Maxwell equation, which is now valid for the non-static case,

\begin{align*}
  \nabla\times\mathbf{E}=-\frac{\partial\mathbf{B}}{\partial t}
\end{align*}

\section{Helmholtz's Theorem}

Any vector field $\mathbf{V}(\mathbf{r})$ which is defined on all space and falls off at infinite at least as fast as $1 / r^2$ can be written as the sum of the gradient of some scalar field $\phi$ and the curl of some vector field $\mathbf{A}$.

To prove this, consider the fields $\phi(\mathbf{r})$ and $\mathbf{V}(\mathbf{r})$, defined by,

\begin{align*}
  \phi(\mathbf{r})&=\nabla\cdot\mathbf{B}(\mathbf{r}) \\
  \mathbf{A}(\mathbf{r})&=\nabla\times\mathbf{B}(\mathbf{r}) \\
\end{align*}

where,

\begin{align*}
  \mathbf{B}(\mathbf{r})=\frac{1}{4\pi}\int\frac{\mathbf{V}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|} d^3r'
\end{align*}

It therefore follows from our vector identities that,

\begin{align*}
  -\nabla\phi + \nabla\times\mathbf{A} &= -\nabla(\nabla\cdot\mathbf{B})+\nabla\times(\nabla\times\mathbf{B}) \\
                                       &= -\nabla^2\mathbf{B}
\end{align*}

By looking at the definition of $\mathbf{B}(\mathbf{r})$ and applying the Laplacian,

\begin{align*}
  \nabla^2\mathbf{B}(\mathbf{r})&=\frac{1}{4\pi}\nabla^2\int\frac{\mathbf{V}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}d^3r' \\
                                &=\frac{1}{4\pi}\int\mathbf{V}(\mathbf{r'})\nabla_r^2\frac{1}{|\mathbf{r}-\mathbf{r'}|}d^3r' \\
                                &=\frac{1}{4\pi}\int\mathbf{V}(\mathbf{r'})\left[-4\pi\delta^3(\mathbf{r}-\mathbf{r'})\right]d^3r' \\
                                &=-\mathbf{V}(\mathbf{r})
\end{align*}

In the third line we just used that the Laplacian of $1/|r|$ is given by the delta function, which we have shown in other notes.

We have therefore proved that,

\begin{align*}
  -\nabla\phi+\nabla\times\mathbf{A}=-\nabla^2\mathbf{B}=\mathbf{V}(\mathbf{r})
\end{align*}

A vector field $\mathbf{V}(\mathbf{r})$ with $\nabla\cdot\mathbf{V}=0$ is known as solenoidal, and with $\nabla\times\mathbf{V}=0$ is known as irrotational. Since if we take the divergence, we get $\nabla\cdot(\nabla\times\mathbf{A})=0$ or if we take the curl $\nabla\times\nabla\phi=\mathbf{0}$, we can express a vector field as the sum of a solenoidal and irrotational part. In $\mathbf{V}=-\nabla\phi+\nabla\times\mathbf{A}$, $-\nabla\phi$ is the irrotational part, as when the curl is taken this part goes to zero, and $\nabla\times\mathbf{A}$ is the solenoidal part, as when the divergence is taken that part goes to zero.

Using some vector calculus we can show that $\phi(\mathbf{r})$ and $\mathbf{A}(\mathbf{r})$ can be written in terms of the divergence $D(\mathbf{r})=\nabla\cdot\mathbf{V}(\mathbf{r})$, and the curl $\mathbf{C}(\mathbf{r})=\nabla\times\mathbf{V}(\mathbf{r})$,

\begin{align*}
  \phi(\mathbf{r})&=\frac{1}{4\pi}\int\frac{D(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}d^3r' \\
  \mathbf{A}(\mathbf{r})&=\frac{1}{4\pi}\int\frac{\mathbf{C}(\mathbf{r'})}{|\mathbf{r}-\mathbf{r'}|}d^3r'
\end{align*}

Therefore Helmholtz shows that any vector field $\mathbf{V}(\mathbf{r})$ which is defined on all space and falls off at infinity at least as fast as $1/r^2$ is uniquely defined by its divergence and curl,

\begin{itemize}
  \item If $\mathbf{V}$ is solenoidal, and therefore $\phi=0$, then $\mathbf{V}$ can be written as just the curl of $\mathbf{A} \to \mathbf{V}=\nabla\times\mathbf{A}(\mathbf{r})$
  \item If $\mathbf{V}$ is irrotational, and therefore $\mathbf{A}=\mathbf{0}$, then $\mathbf{V}$ can be written as just the gradient of $\phi \to \mathbf{V}=-\nabla\phi$.
\end{itemize}

\section{Example: Maxwell's Equations}

Writing out the differential form of Maxwell's equations,

\begin{alignat*}{2}
  &\nabla\cdot\mathbf{E}=\frac{\rho}{\epsilon_0}\qquad&&\nabla\times\mathbf{E}=-\frac{\partial\mathbf{B}}{\partial t} \\
  &\nabla\cdot\mathbf{B}=0\qquad&&\nabla\times\mathbf{B}=\mu_0\mathbf{j}+\mu_0\epsilon_0\frac{\partial\mathbf{E}}{\partial t}
\end{alignat*}

we notice that two of the equations involve a source term. These equations are $\nabla\cdot\mathbf{E}$ and $\nabla\times\mathbf{B}$, which contain the source terms $\rho$, and $\mathbf{j}$ respectively. They both contain a term which is the source of the field.

Consider the two other equations which do not have a source term. Firstly we have $\nabla\cdot\mathbf{B}=0$, this means that $\mathbf{B}$ is solenoidal, and therefore as we have shown by Helmholtz's theorem, $\mathbf{B}$ can be written as the curl of another vector field $\mathbf{A}$, $\mathbf{B}=\nabla\times\mathbf{A}$.

Next we have $\nabla\times\mathbf{E}$. As we have just shown we can write the $\mathbf{B}=\nabla\times\mathbf{A}$, which allows us to write,

\begin{align*}
  \nabla\times\mathbf{E} = -\frac{\partial\mathbf{B}}{\partial t} = -\frac{\partial}{\partial t}(\nabla\times\mathbf{A}) = -\nabla\times\frac{\partial\mathbf{A}}{\partial t}
\end{align*}

By grouping both terms onto one side we can see that,

\begin{align*}
  \nabla\times\left(\mathbf{E}\times\frac{\partial\mathbf{A}}{\partial t}\right) = \mathbf{0}
\end{align*}

This means that the part between the parenthesis is irrotational, and thus from Helmholtz this can be written as the gradient of a scalar field, $\mathbf{E} + \frac{\partial\mathbf{A}}{\partial t} = -\nabla\phi$.

We have shown that we can write the electric and magnetic field in terms of a scalar and vector potential $\phi$ and $\mathbf{A}$,

\begin{align*}
  \mathbf{B}&=\nabla\times\mathbf{A}\\
  \mathbf{E}&=-\nabla\phi-\frac{\partial\mathbf{A}}{\partial t}
\end{align*}

\end{document}


\subsection{Finite (Galois) fields}

Finite number of elements.

\subsubsection{Integers mod \(p\) field}

\subsubsection{Characteristic of a field}


\chapter{Quicksort recurrences}

\section{On the average number of checks}

Let $c_{n}$ be the number of checks occuring on the computation
of Quicksort algorithm on a vector of length $n$. After some manipulation
starting from a general relation over $c_{n}$, which is derived from
Quicksort algorithm implementation directly, the following
recurrence relation can be derived:
\begin{equation}
    \label{eq:quicksort-checks-recurrence}
    \frac{c_{n}}{n + 1} = \frac{2}{n + 1} + \frac{1}{n} c_{n - 1}
\end{equation}
one unfolding step allows us to rewrite term $\frac{1}{n} c_{n - 1}$,
rewriting the starting relation as:
\begin{displaymath}
    \frac{c_{n}}{n + 1} = \frac{c_{n - 2}}{n - 1} + \frac{2}{n} + \frac{2}{n + 1}
\end{displaymath}
Doing $3$ more unfolding steps from the above recurrence yield the
new recurrence:
\begin{displaymath}
    \frac{c_{n}}{n + 1} = \frac{2}{n - 3} + \frac{c_{n - 5}}{n - 4} + \frac{2}{n + 1} + \frac{2}{n} + \frac{2}{n - 1} + \frac{2}{n - 2}
\end{displaymath}
To reach previous recurrence, the following subterms have been unfolded,
according \autoref{eq:quicksort-checks-recurrence}:
\begin{displaymath}
    \left \{ \frac{1}{n} c_{n - 1} : \frac{c_{n - 2}}{n - 1} + \frac{2}{n}, \quad \frac{c_{n - 4}}{n - 3} : \frac{2}{n - 3} + \frac{c_{n - 5}}{n - 4}, \quad \frac{c_{n - 3}}{n - 2} : \frac{2}{n - 2} + \frac{c_{n - 4}}{n - 3}, \quad \frac{c_{n - 2}}{n - 1} : \frac{2}{n - 1} + \frac{c_{n - 3}}{n - 2}\right \}
\end{displaymath}


\subsection{Seasonal and non-seasonal trends}

We can model the process as:

\(y_t=\mu_t +f(t)+\epsilon_t \)


\chapter{CT Convolution}

\section{Review CT LTI systems and superposition property}

Recall the superposition property of LTI systems. If a CT system is LTI then the superposition property holds. Given a system where
\[
x_i(t) \mapsto y_i(t) \; \forall\; i
\]
then
\[
\sum\limits_{i} a_i x_i(t) \mapsto \sum\limits_{i} a_i y_i(t)
\]

Superposition enables a powerful problem reduction strategy. The overall idea for is that if:

\begin{itemize}
\item we can write an aribtrary signal as a sum of simple signals, and
\item we can determine the response to the simple signals, then
\item we can easily express the output due to the input using superposition
\end{itemize}

This will be a recurring pattern in this course. In this lecture, the simple signals are weighted, time shifts of one signal, the delta function, $\delta(t)$.

\section{Convolution Integral}

To derive this we start with the sifting property of the CT impulse function (from lecture 2)
\[
\int\limits_{a}^{b} x(t)\delta(t-t_0) \; dt = x(t_0)
\]
for any $a < t_0 < b$. A slight change of variables ($t_0 \rightarrow \tau$) and limits ($a \rightarrow -\infty$ and $b \rightarrow \infty$) gives:
\[
x(t) = \int\limits_{-\infty}^{\infty} x(\tau)\delta(t-\tau) \; d\tau
\]
showing that we can write any CT signal as an infinite sum (integral) of weighted and time-shifted impluse functions.

Let $h(t)$ be the CT {\it impulse response}, the output due to the input $\delta(t)$, i.e. $\delta(t) \mapsto h(t)$. Then if the system is time-invariant: $\delta(t-\tau) \mapsto h(t-\tau)$ and by superposition if the input is writen as
\[
x(t) = \int\limits_{-\infty}^{\infty} x(\tau)\delta(t-\tau) \; d\tau
\]
then the output is given by
\[
  y(t) = \int\limits_{-\infty}^{\infty} x(\tau)h(t-\tau) \; d\tau = x(t) * h(t)
\]
This is called the \emph{convolution integral} \index{CT Convolution}.

It is worth pausing here to see the signifigance. For a LTI CT system, if I know it's impulse response $h(t)$, I can find the response due to \textbf{any} input using convolution. For this reason the impulse response is another way to represent an LTI system.

\section{Graphical View of the Convolution Integral.}

Lets break the convolution expression down into pieces. In it's general form the convolution of two signals $x_1(t)$ and $x_2(t)$ is
\[
x_1(t) * x_2(t) = \int\limits_{-\infty}^{\infty} x_1(\tau)x_2(t-\tau) \; d\tau
\]

Suppose $x_1(t)$ and $x_2(t)$ are signals that look like
\begin{center}
  \includegraphics[scale=1]{graphics/convolution-explain1.pdf}
\end{center}

Then $x_1(\tau)$ and $x_2(-\tau)$ look like
\begin{center}
  \includegraphics[scale=1]{graphics/convolution-explain2.pdf}
\end{center}
The signal $x_2(t-\tau)$ is $x_2(-\tau)$ shifted by $t$ (since $x_2(-\tau+t)= x_2(t-\tau)$) and then looks like
\begin{center}
  \includegraphics[scale=1]{graphics/convolution-explain3.pdf}
\end{center}
Then the integrand of convolution is the product $x_1(\tau)x_2(t-\tau)$ whose plot depends of the value of $t$. Some examples, where the individual signals are dashed and their product is in bold:
\begin{center}
  \includegraphics[scale=1]{graphics/convolution-explain4.pdf}
\end{center}
Then convolution is the total integral of the product (bold curves above) for that value of $t$. For the example above we see the integral will be zero for $t$ less than $t_0$ since the two signals do not overlap and their product is zero. For $t_0 < t < t_1$ the signals overap and the product is non-zero, and the effective bounds of integration are $[t_0,t]$. For $t > t_1$ the signals again overap and the product is non-zero, but the effective bounds of integration are $[t_0,t_1]$.

\section{Examples of CT Convolution}

\begin{example}[$u(t) * u(t)$] Consider the convolution of two unit step functions.
  \[
  u(t) * u(t) = \int\limits_{-\infty}^{\infty} u(\tau)u(t-\tau) \; d\tau
  \]
  The product $u(\tau) u(t-\tau)$ is non-zero only when $t\geq 0$ as illustrated here
  \begin{center}
  \includegraphics[scale=1]{graphics/convolution-step.pdf}
\end{center}
  The convolution integral is then the shaded area
  \[
u(t) * u(t) = \left\{ \begin{array}{lc}
  0 & t< 0\\
  \int\limits_{0}^{t} d\tau = t  & t \geq 0\\
\end{array}\right.
\]
Combining this back into a single expression gives:
\[
u(t) * u(t) = tu(t)
\]
Thus the convolution of two step signals is a ramp signal.\\$\blacksquare$
\end{example}

\begin{example}[$u(t) * e^{-at}u(t)$] Let $x_1(t) = u(t)$ and $x_2(t) = e^{-at}u(t)$ for constant $a\in\mathbb{C}$, then
  \[
u(t) * e^{-at}u(t) = \int\limits_{-\infty}^{\infty} u(\tau)e^{-a(t-\tau)}u(t-\tau) \; d\tau
\]
Similar to the previous example, the product $u(\tau) e^{-a(t-\tau)} u(t-\tau)$ is non-zero only when $t\geq 0$
  \begin{center}
  \includegraphics[scale=1]{graphics/convolution-expstep.pdf}
  \end{center}
  The convolution integral is then the shaded area
  \[
u(t) * e^{-at}u(t) = \left\{ \begin{array}{lc}
  0 & t< 0\\
  \int\limits_{0}^{t} e^{-a(t-\tau)} d\tau = \frac{1-e^{-at}}{a}  & t \geq 0\\
\end{array}\right.
\]
Combining this back into a single expression gives:
\[
u(t) * e^{-at}u(t) = \frac{1-e^{-at}}{a}u(t)
\]
$\blacksquare$
\end{example}

\begin{example}[Convolution with a delta function] Let $x_1(t) = \delta(t)$ and $x_2(t)$ be an arbitrary signal. Then
  \[
  \delta(t) * x_2(t) = \int\limits_{-\infty}^{\infty} \delta(\tau)x_2(t-\tau) \; d\tau
  \]
  By the sifting property of the delta function this evaluates to
  \[
  \delta(t) * x_2(t) = x_2(t)
  \]
  or in other words convolution with a delta function just results in the signal it was convolved with. That is it acts like the identity function, with respect to convolution.\\
  $\blacksquare$
\end{example}

The following table lists several convolution results.

\begin{center}
  Short Table of Representative Convolution Integrals
  \vspace{1em}

\bgroup
\def\arraystretch{2}
\setlength\tabcolsep{2em}
\begin{tabular}{|c|c|c|}
  \hline
  $x_1(t)$ & $x_2(t)$ & $x_1(t) * x_2(t)$\\
  \hline
  \hline
  $e^{a t}u(t)$ & $u(t)$ & $\frac{1-e^{a t}}{-a}u(t)$\\
  $u(t)$ & $u(t)$ & $tu(t)$\\
  $e^{a_1 t}u(t)$ & $e^{a_2 t}u(t)$ & $\frac{e^{a_1 t}-e^{a_2 t}}{a_1 - a_2}u(t)$ for $a_1 \neq a_2$\\
  $e^{a t}u(t)$ & $e^{a t}u(t)$ & $te^{a t}u(t)$\\
  $te^{a_1 t}u(t)$ & $e^{a_2 t}u(t)$ & $\frac{e^{a_2 t}-e^{a_1 t} + (a_1-a_2)te^{a_1 t}}{(a_1 - a_2)^2}u(t)$ for $a_1 \neq a_2$\\
  $e^{a_1 t}\cos(\beta t + \theta)u(t)$ & $e^{a_2 t}u(t)$ & $\frac{\cos(\theta - \phi)e^{a_2 t} - e^{a_1 t}\cos(\beta t + \theta - \phi)}{\sqrt{(a_1 + a_2)^2 + \beta^2}}u(t)$\\
  & & $\phi = \arctan\left( \frac{-\beta}{a_1 + a_2}\right)$\\
\hline
\end{tabular}
\egroup

\end{center}

\section{Properties of CT Convolution}
There are several useful properties of convolution. We do not prove these here, but it is not terribly difficult to do so. Given signals $x_1(t)$, $x_2(t)$, and $x_3(t)$:

\begin{description}
\item [Communative Property] The ordering of the signals does not matter.
  \[
x_1(t) * x_2(t) = x_2(t) * x_1(t)
  \]
\item [Distributive Propery] Convolution is distributed over addition.
  \[
  x_1(t) * \left[x_2(t) + x_3(t)\right] = \left[x_1(t) * x_2(t) \right] + \left[x_1(t) * x_3(t) \right]
  \]
\item [Associative Property] The order of convolution does not matter.
    \[
  x_1(t) * \left[x_2(t) * x_3(t)\right] = \left[x_1(t) * x_2(t) \right] * x_3(t)
  \]
\item [Time Shift] Given $x_3(t) = x_1(t) * x_2(t)$ then for time shifts $\tau_1, \tau_2 \in \mathbb{R}$
  \[
  x_1(t-\tau_1) * x_2(t-\tau_2) = x_3(t-\tau_1 - \tau_2)
  \]
\item [Multiplicative Scaling] Given $x_3(t) = x_1(t) * x_2(t)$ then for constants $a,b \in \mathbb{C}$
  \[
  \left[a\, x_1(t)\right] * \left[b\, x_2(t)\right] = a\, b\, x_3(t)
  \]
\end{description}

These properties can be used in combination with a table like that above to compute the convolution of a wide variety of signals without evaluating the integrals.

\begin{example} Here is a simple example. Let $x_1(t) = e^tu(t)$ and $x_2(t) = 2\delta(t) + 5e^{-3t}u(t)$.
  \[
  x_1(t) * x_2(t) =  e^tu(t) * \left[2\delta(t) + 5e^{-3t}u(t)\right]
  \]
  Using the distributive property
  \[
  x_1(t) * x_2(t) =  2\left[\delta(t) * e^tu(t)\right]  + 5\left[e^tu(t) * e^{-3t}u(t)\right]
  \]
  Using previously derived results involving the delta function and the table row 3
  \[
  x_1(t) * x_2(t) = 2 e^t\, u(t) + 5\left[ \frac{e^t-e^{-3t}}{4}\right]u(t)
  \]
  Doing some simplification gives the result
  \[
  x_1(t) * x_2(t) = \left[ \frac{13}{4}e^t-\frac{5}{4}e^{-3t}\right]u(t)
  \]

$\blacksquare$
\end{example}

\begin{example} Here is a more complicated example. Let $x_1(t) = 2e^{-5t}u(t-1)$ and $x_2(t) = \left(1-e^{-t}\right)u(t)$.
  \[
  x_1(t) * x_2(t) = \left[2e^{-5t}u(t-1)\right] * \left[\left(1-e^{-t}\right)u(t)\right]
  \]
  We first rewrite $e^{-5t}u(t-1)=e^{-5}e^{-5(t-1)}u(t-1) = e^{-5}e^{-5t}u(t)\Big|_{t=t-1}$ so that we can remove the time shift
  \[
  x_1(t) * x_2(t) = 2e^{-5}\left[e^{-5t}u(t)\right] * \left[\left(1-e^{-t}\right)u(t)\right]\Big|_{t=t-1}
  \]
  We now apply the distributive property
  \[
x_1(t) * x_2(t) = 2e^{-5}\left[\left(e^{-5t}u(t) * u(t)\right) - \left(e^{-5t}u(t)* e^{-t}u(t)\right)\right]\Big|_{t=t-1}
  \]
  Using the table rows 1 and 3 we get
  \[
  x_1(t) * x_2(t) = 2e^{-5}\left[\frac{1}{5}\left(1-e^{-5t}\right)u(t) + \frac{1}{4}\left(e^{-5t} - e^{-t}\right)u(t)\right]\Big|_{t=t-1}
  \]
  Combining terms we simplify to
  \[
x_1(t) * x_2(t) = 2e^{-5}\left[\frac{1}{5} - \frac{1}{4}e^{-t} + \frac{1}{20}e^{-5t} \right]u(t)\Big|_{t=t-1}
\]
Replacing the time shift gives the final result
\[
x_1(t) * x_2(t) = 2e^{-5}\left[\frac{1}{5} - \frac{1}{4}e^{-(t-1)} + \frac{1}{20}e^{-5(t-1)} \right]u(t-1)
\]
which can be cleaned up a bit more by distributing the leading term
\[
x_1(t) * x_2(t) =\left[\frac{2}{5}e^{-5} -\frac{1}{2}e^{-(t+4)} +\frac{1}{10}e^{-5t}\right]u(t-1)
\]

$\blacksquare$
\end{example}


\chapter{Cardinals}
An ordinal measures a total ordering.
However, it does not do a fantastic job at measuring size.
For example, there is a bijection between the elements of $\omega$ and $\omega+1$:
\[
	\begin{array}{rccccccc}
		\omega+1 = & \{ & \omega & 0 & 1 & 2 & \dots & \} \\
		\omega = & \{ & 0 & 1 & 2 & 3 & \dots & \}.
	\end{array}
\]
In fact, as you likely already know,
there is even a bijection between $\omega$ and $\omega^2$:
\[
	\begin{array}{l|cccccc}
		+ & 0 & 1 & 2 & 3 & 4 & \dots \\ \hline
		0 & 0 & 1 & 3 & 6 & 10 & \dots \\
		\omega & 2 & 4 & 7 & 11 & \dots & \\
		\omega \cdot 2 & 5 & 8 & 12 & \dots & & \\
		\omega \cdot 3 & 9 & 13 & \dots & & & \\
		\omega \cdot 4 & 14 & \dots & & & &
	\end{array}
\]
So ordinals do not do a good job of keeping track of size.
For this, we turn to the notion of a cardinal number.

\section{Equinumerous sets and cardinals}
\begin{definition}
	Two sets $A$ and $B$ are \vocab{equinumerous}, written $A \approx B$,
	if there is a bijection between them.
\end{definition}

\begin{definition}
	A \vocab{cardinal} is an ordinal $\kappa$ such that
	for no $\alpha < \kappa$ do we have $\alpha \approx \kappa$.
\end{definition}
\begin{example}[Examples of cardinals]
	Every finite number is a cardinal.
	Moreover, $\omega$ is a cardinal.
	However, $\omega+1$, $\omega^2$, $\omega^{2015}$ are not,
	because they are countable.
\end{example}
\begin{example}[$\omega^\omega$ is countable]
	Even $\omega^\omega$ is not a cardinal,
	since it is a countable union
	\[ \omega^\omega = \bigcup_n \omega^n \]
	and each $\omega^n$ is countable.
\end{example}
\begin{ques}
	Why must an infinite cardinal be a limit ordinal?
\end{ques}

\begin{remark}
	There is something fishy about the definition of a cardinal:
	it relies on an \emph{external} function $f$.
	That is, to verify $\kappa$ is a cardinal I can't just look at $\kappa$ itself;
	I need to examine the entire universe $V$ to make sure
	there does not exist a bijection $f : \kappa \to \alpha$ for $\alpha < \kappa$.
	For now this is no issue, but later in model theory
	this will lead to some highly counterintuitive behavior.
\end{remark}

\section{Cardinalities}
Now that we have defined a cardinal, we can discuss the size
of a set by linking it to a cardinal.

\begin{definition}
	The \vocab{cardinality} of a set $X$
	is the \emph{least} ordinal $\kappa$ such that $X \approx \kappa$.
	We denote it by $\left\lvert X \right\rvert$.
\end{definition}
\begin{ques}
	Why must $\left\lvert X \right\rvert$ be a cardinal?
\end{ques}
\begin{remark}
	One needs the well-ordering theorem (equivalently, choice)
	in order to establish that such an ordinal $\kappa$ actually exists.
\end{remark}
Since cardinals are ordinals, it makes sense to ask whether $\kappa_1 \le \kappa_2$,
and so on.
Our usual intuition works well here.
\begin{proposition}[Restatement of cardinality properties]
	Let $X$ and $Y$ be sets.
	\begin{enumerate}[(i)]
		\ii $X \approx Y$ if and only $\left\lvert X \right\rvert = \left\lvert Y \right\rvert$,
		if and only if there's a bijection from $X$ to $Y$.
		\ii $\left\lvert X \right\rvert \le \left\lvert Y \right\rvert$
		if and only if there is an injective map $X \injto Y$.
	\end{enumerate}
\end{proposition}
Diligent readers are invited to try and prove this.

\section{Aleph numbers}
\prototype{$\aleph_0 = \omega$, and $\aleph_1$ is the first uncountable ordinal.}
First, let us check that cardinals can get arbitrarily large:
\begin{proposition}
	We have $\left\lvert X \right\rvert < \left\lvert \PP(X) \right\rvert$ for every set $X$.
\end{proposition}
\begin{proof}
	There is an injective map $X \injto \PP(X)$
	but there is no injective map $\PP(X) \injto X$ by \Cref{lem:cantor_diag}.
\end{proof}

Thus we can define:
\begin{definition}
	For a cardinal $\kappa$, we define $\kappa^+$ to be the least cardinal above $\kappa$,
	called the \vocab{successor cardinal}.
\end{definition}
This $\kappa^+$ exists and has $\kappa^+ \le \left\lvert \PP(\kappa) \right\rvert$.

Next, we claim that:
\begin{exercise}
	Show that if $A$ is a set of cardinals, then $\cup A$ is a cardinal.
\end{exercise}

Thus by transfinite induction we obtain that:
\begin{definition}
	For any $\alpha \in \On$, we define the \vocab{aleph numbers} as
	\begin{align*}
		\aleph_0 &= \omega \\
		\aleph_{\alpha+1} &= \left( \aleph_\alpha \right)^+ \\
		\aleph_{\lambda} &= \bigcup_{\alpha < \lambda} \aleph_\alpha.
	\end{align*}
\end{definition}

Thus we have the sequence of cardinals
\[
	0 < 1 < 2 < \dots < \aleph_0 < \aleph_1 < \dots < \aleph_\omega < \aleph_{\omega+1} < \dots.
\]
By definition, $\aleph_0$ is the cardinality of the natural numbers,
$\aleph_1$ is the first uncountable ordinal, \dots.

We claim the aleph numbers constitute all the cardinals:
\begin{lemma}[Aleph numbers constitute all infinite cardinals]
	If $\kappa$ is a cardinal then
	either $\kappa$ is finite (i.e.\ $\kappa \in \omega$) or
	$\kappa = \aleph_\alpha$ for some $\alpha \in \On$.
\end{lemma}
\begin{proof}
	Assume $\kappa$ is infinite, and take $\alpha$ minimal with $\aleph_\alpha \ge \kappa$.
	Suppose for contradiction that we have $\aleph_\alpha > \kappa$.
	We may assume $\alpha > 0$, since the case $\alpha = 0$ is trivial.

	If $\alpha = \ol\alpha + 1$ is a successor, then
	\[ \aleph_{\ol\alpha} < \kappa < \aleph_{\alpha}
		= (\aleph_{\ol\alpha})^+ \]
	which contradicts the definition of the successor cardinal.

	If $\alpha = \lambda$ is a limit ordinal, then $\aleph_\lambda$ is the
	supremum $\bigcup_{\gamma < \lambda} \aleph_\gamma$.
	So there must be some $\gamma < \lambda$ with $\aleph_\gamma > \kappa$,
	which contradicts the minimality of $\alpha$.
\end{proof}

\begin{definition}
	An infinite cardinal which is not a successor cardinal
	is called a \vocab{limit cardinal}.
	It is exactly those cardinals of the form $\aleph_\lambda$,
	for $\lambda$ a limit ordinal, plus $\aleph_0$.
\end{definition}

\section{Cardinal arithmetic}
\prototype{$\aleph_0 \cdot \aleph_0 = \aleph_0 + \aleph_0 = \aleph_0$}
Recall the way we set up ordinal arithmetic.
Note that in particular, $\omega + \omega > \omega$ and $\omega^2 > \omega$.
Since cardinals count size, this property is undesirable, and
we want to have
\begin{align*}
	\aleph_0 + \aleph_0 &= \aleph_0 \\
	\aleph_0 \cdot \aleph_0 &= \aleph_0
\end{align*}
because $\omega + \omega$ and $\omega \cdot \omega$ are countable.
In the case of cardinals, we simply ``ignore order''.

The definition of cardinal arithmetic is as expected:
\begin{definition}[Cardinal arithmetic]
	Given cardinals $\kappa$ and $\mu$, define
	\[ \kappa + \mu
		\defeq
		\left\lvert
		\left( \left\{ 0 \right\} \times \kappa \right)
		\cup
		\left( \left\{ 1 \right\} \times \mu \right)
		\right\rvert
	\]
	and
	\[
		\kappa \cdot \mu
		\defeq
		\left\lvert \mu \times \kappa \right\rvert
		.
	\]
\end{definition}

\begin{ques}
	Check this agrees with what you learned in pre-school
	for finite cardinals.
\end{ques}

\begin{abuse}
	This is a slight abuse of notation since we are using
	the same symbols as for ordinal arithmetic,
	even though the results are different ($\omega \cdot \omega = \omega^2$
	but $\aleph_0 \cdot \aleph_0 = \aleph_0$).
	In general, I'll make it abundantly clear whether I am talking
	about cardinal arithmetic or ordinal arithmetic.
\end{abuse}
To help combat this confusion, we use separate symbols for ordinals and cardinals.
Specifically, $\omega$ will always refer to $\{0,1,\dots\}$ viewed as an ordinal;
$\aleph_0$ will always refer to the same set viewed as a cardinal.
More generally,
\begin{definition}
	Let $\omega_\alpha = \aleph_\alpha$ viewed as an ordinal.
\end{definition}

However, as we've seen already we have that $\aleph_0 \cdot \aleph_0 = \aleph_0$.
In fact, this holds even more generally:

\begin{theorem}[Infinite cardinals squared]
	Let $\kappa$ be an infinite cardinal.
	Then $\kappa \cdot \kappa = \kappa$.
\end{theorem}
\begin{proof}
	Obviously $\kappa \cdot \kappa \ge \kappa$,
	so we want to show $\kappa \cdot \kappa \le \kappa$.

	The idea is to try to repeat the same proof
	that we had for $\aleph_0 \cdot \aleph_0 = \aleph_0$,
	so we re-iterate it here. We took the ``square'' of
	elements of $\aleph_0$, and then
	\emph{re-ordered} it according to the diagonal:
	\[
	\begin{array}{l|cccccc}
		  & 0 & 1 & 2 & 3 & 4 & \dots \\ \hline
		0 & 0 & 1 & 3 & 6 & 10 & \dots \\
		1 & 2 & 4 & 7 & 11 & \dots & \\
		2 & 5 & 8 & 12 & \dots & & \\
		3 & 9 & 13 & \dots & & & \\
		4 & 14 & \dots & & & &
	\end{array}
	\]
	We'd like to copy this idea for a general $\kappa$;
	however, since addition is less well-behaved for infinite ordinals
	it will be more convenient to use $\max\{\alpha,\beta\}$
	rather than $\alpha+\beta$.
	Specifically, we put the ordering $<_{\text{max}}$
	on $\kappa \times \kappa$ as follows:
	for $(\alpha_1, \beta_1)$ and $(\alpha_2, \beta_2)$ in $\kappa \times \kappa$
	we declare $(\alpha_1, \beta_1) <_{\text{max}} (\alpha_2, \beta_2)$ if
	\begin{itemize}
		\ii $\max \left\{ \alpha_1, \beta_1 \right\} < \max \left\{ \alpha_2, \beta_2 \right\}$ or
		\ii $\max \left\{ \alpha_1, \beta_1 \right\} = \max \left\{ \alpha_2, \beta_2 \right\}$ and $(\alpha_1, \beta_1)$
		is lexicographically earlier than $(\alpha_2, \beta_2)$.
	\end{itemize}
	This alternate ordering (which deliberately avoids referring
	to the addition) looks like:
	\[
	\begin{array}{l|cccccc}
		  & 0 & 1 & 2 & 3 & 4 & \dots \\ \hline
		0 & 0 & 1 & 4 & 9 & 16 & \dots \\
		1 & 2 & 3 & 5 & 10 & 17 & \dots \\
		2 & 6 & 7 & 8 & 11 & 18 & \dots \\
		3 & 12 & 13 & 14 & 15 & 19 & \dots \\
		4 & 20 & 21 & 22 & 23 & 24 & \dots \\
		\vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \ddots \\
	\end{array}
	\]

	Now we proceed by transfinite induction on $\kappa$.
	The base case is $\kappa = \aleph_0$, done above.
	Now, $<_{\text{max}}$ is a well-ordering of $\kappa \times \kappa$,
	so we know it is in order-preserving bijection with some ordinal $\gamma$.
	Our goal is to show that $\left\lvert \gamma \right\rvert \le \kappa$.
	To do so, it suffices to prove that for any $\ol\gamma \in \gamma$,
	we have $\left\lvert \ol\gamma \right\rvert < \kappa$.

	Suppose $\ol\gamma$ corresponds to the point $(\alpha, \beta) \in \kappa \times \kappa$
	under this bijection.
	If $\alpha$ and $\beta$ are both finite
	then certainly $\ol\gamma$ is finite too.
	Otherwise, let $\ol\kappa = \max \{\alpha, \beta\} < \kappa$;
	then the number of points below $\ol\gamma$ is at most
	\[
		\left\lvert \alpha \right\rvert \cdot \left\lvert \beta \right\rvert
		\le \ol\kappa \cdot \ol\kappa
		= \ol\kappa
	\]
	by the inductive hypothesis.
	So $\left\lvert \ol\gamma \right\rvert \le \ol\kappa < \kappa$ as desired.
\end{proof}

From this it follows that cardinal addition and multiplication is really boring:
\begin{theorem}[Infinite cardinal arithmetic is trivial]
	Given cardinals $\kappa$ and $\mu$,
	one of which is infinite, we have
	\[ \kappa \cdot \mu = \kappa + \mu
	= \max\left\{ \kappa, \mu \right\}.\]
\end{theorem}
\begin{proof}
	The point is that both of these are less than the square of the maximum.
	Writing out the details:
	\begin{align*}
		\max \left\{ \kappa, \mu \right\}
		&\le \kappa + \mu \\
		&\le \kappa \cdot \mu \\
		&\le \max \left\{ \kappa, \mu \right\}
			\cdot \max \left\{ \kappa, \mu  \right\} \\
		&= \max\left\{ \kappa, \mu \right\}. \qedhere
	\end{align*}
\end{proof}

\section{Cardinal exponentiation}
\prototype{$2^\kappa = \left\lvert \PP(\kappa) \right\rvert$.}
\begin{definition}
	Suppose $\kappa$ and $\lambda$ are cardinals.
	Then
	\[ \kappa^\lambda
		\defeq \left\lvert \mathscr F(\lambda, \kappa) \right\rvert.
	\]
	Here $\mathscr F(A,B)$ is the set of functions from $A$ to $B$.
\end{definition}

\begin{abuse}
	As before, we are using the same notation for
	both cardinal and ordinal arithmetic. Sorry!
\end{abuse}

In particular, $2^\kappa = \left\lvert \PP(\kappa) \right\rvert > \kappa$,
and so from now on we can use the notation $2^\kappa$ freely.
(Note that this is totally different from ordinal arithmetic;
there we had $2^\omega = \bigcup_{n\in\omega} 2^n = \omega$.
In cardinal arithmetic $2^{\aleph_0} > \aleph_0$.)

I have unfortunately not told you what $2^{\aleph_0}$ equals.
A natural conjecture is that $2^{\aleph_0} = \aleph_1$; this is called the
\vocab{Continuum Hypothesis}.
It turns out that this is \emph{undecidable} -- it is not possible
to prove or disprove this from the $\ZFC$ axioms.

\section{Cofinality}
\prototype{$\aleph_0$, $\aleph_1$, \dots\ are all regular, but $\aleph_\omega$ has cofinality $\omega$.}

\begin{definition}
	Let $\lambda$ be an ordinal (usually a limit ordinal),
	and $\alpha$ another ordinal.
	A map $f : \alpha \to \lambda$ of ordinals is called \vocab{cofinal}
	if for every $\ol\lambda < \lambda$, there is some $\ol\alpha \in \alpha$
	such that $f(\ol\alpha) \ge \ol\lambda$.
	In other words, the map reaches arbitrarily high into $\lambda$.
\end{definition}
\begin{example}
	[Example of a cofinal map]
	\listhack
	\begin{enumerate}[(a)]
		\ii The map $\omega \to \omega^\omega$ by $n \mapsto \omega^n$ is cofinal.
		\ii For any ordinal $\alpha$, the identity map $\alpha \to \alpha$ is cofinal.
	\end{enumerate}
\end{example}

\begin{definition}
	Let $\lambda$ be a limit ordinal.
	The \vocab{cofinality} of $\lambda$, denoted $\cof(\lambda)$,
	is the smallest ordinal $\alpha$ such that there is a cofinal map
	$\alpha \to \lambda$.
\end{definition}
\begin{ques}
	Why must $\alpha$ be an infinite cardinal?
\end{ques}

Usually, we are interested in taking the cofinality of a cardinal $\kappa$.

Pictorially, you can imagine standing at the bottom of the universe and looking
up the chain of ordinals to $\kappa$.
You have a machine gun and are firing bullets upwards, and you want to get arbitrarily
high but less than $\kappa$.
The cofinality is then the number of bullets you need to do this.

We now observe that ``most'' of the time, the cofinality of a cardinal is itself.
Such a cardinal is called \vocab{regular}.
\begin{example}[$\aleph_0$ is regular]
	$\cof(\aleph_0) = \aleph_0$, because no finite subset of
	$\aleph_ 0 = \omega$ can reach arbitrarily high.
\end{example}
\begin{example}[$\aleph_1$ is regular]
	$\cof(\aleph_1) = \aleph_1$.
	Indeed, assume for contradiction that some countable
	set of ordinals $A = \{ \alpha_0, \alpha_1, \dots \} \subseteq \aleph_1$
	reaches arbitrarily high inside $\aleph_1$.
	Then $\Lambda = \cup A$ is a \emph{countable} ordinal,
	because it is a countable union of countable ordinals.
	In other words $\Lambda \in \aleph_1$.
	But $\Lambda$ is an upper bound for $A$, contradiction.
\end{example}
On the other hand, there \emph{are} cardinals which are not regular;
since these are the ``rare'' cases we call them \vocab{singular}.
\begin{example}[$\aleph_\omega$ is not regular]
	Notice that $\aleph_0 < \aleph_1 < \aleph_2 < \dots$ reaches
	arbitrarily high in $\aleph_\omega$, despite only having $\aleph_0$ terms.
	It follows that $\cof(\aleph_\omega) = \aleph_0$.
\end{example}

We now confirm a suspicion you may have:
\begin{theorem}
	[Successor cardinals are regular]
	If $\kappa = \ol\kappa^+$ is a successor cardinal,
	then it is regular.
\end{theorem}
\begin{proof}
	We copy the proof that $\aleph_1$ was regular.

	Assume for contradiction that for some $\mu \le \ol\kappa$,
	there are $\mu$ sets reaching arbitrarily high in $\kappa$ as a cardinal.
	Observe that each of these sets must have cardinality at most $\ol\kappa$.
	We take the union of all $\mu$ sets, which gives an ordinal $\Lambda$
	serving as an upper bound.

	The number of elements in the union is at most
	\[ \#\text{sets} \cdot \#\text{elms}
		\le \mu \cdot \ol\kappa = \ol\kappa \]
	and hence $\left\lvert \Lambda \right\rvert \le \ol\kappa < \kappa$.
\end{proof}

\section{Inaccessible cardinals}
So, what about limit cardinals?
It seems to be that most of them are singular: if $\aleph_\lambda \ne \aleph_0$ is a limit ordinal,
then the sequence $\{\aleph_\alpha\}_{\alpha \in \lambda}$ (of length $\lambda$) is certainly cofinal.

\begin{example}[Beth fixed point]
	Consider the monstrous cardinal
	\[ \kappa = \aleph_{\aleph_{\aleph_{\ddots}}}. \]
	This might look frighteningly huge, as $\kappa = \aleph_\kappa$,
	but its cofinality is $\omega$ as it is the limit of the sequence
	\[ \aleph_0, \aleph_{\aleph_0}, \aleph_{\aleph_{\aleph_0}}, \dots \]
\end{example}

More generally, one can in fact prove that
\[ \cof(\aleph_\lambda) = \cof(\lambda). \]
But it is actually conceivable that $\lambda$ is so large
that $\left\lvert \lambda \right\rvert = \left\lvert \aleph_\lambda \right\rvert$.

A regular limit cardinal other than $\aleph_0$ has a special name: it is \vocab{weakly inaccessible}.
Such cardinals are so large that it is impossible to prove or disprove their existence in $\ZFC$.
It is the first of many so-called ``large cardinals''.

An infinite cardinal $\kappa$ is a strong limit cardinal if
\[ \forall \ol\kappa < \kappa \quad 2^{\ol\kappa} < \kappa \]
for any cardinal $\ol\kappa$.  For example, $\aleph_0$ is a strong limit cardinal.
\begin{ques}
	Why must strong limit cardinals actually be limit cardinals?
	(This is offensively easy.)
\end{ques}
A regular strong limit cardinal other than $\aleph_0$
is called \vocab{strongly inaccessible}.

\section\problemhead
\begin{problem}
	Compute $\left\lvert V_\omega \right\rvert$.
	\begin{hint}
		$\sup_{k \in \omega} \left\lvert V_k \right\rvert$.
	\end{hint}
\end{problem}

\begin{problem}
	Prove that for any limit ordinal $\alpha$, $\cof(\alpha)$ is a \emph{regular} cardinal.
	\begin{hint}
		Rearrange the cofinal maps to be nondecreasing.
	\end{hint}
\end{problem}

\begin{sproblem}
	[Strongly inaccessible cardinals]
	\label{prob:strongly_inaccessible}
	Show that for any strongly inaccessible $\kappa$,
	we have $\left\lvert V_\kappa \right\rvert = \kappa$.
\end{sproblem}

\begin{problem}
	[K\"onig's theorem]
	Show that \[ \kappa^{\cof(\kappa)} > \kappa \] for every infinite cardinal $\kappa$.
\end{problem}


\subsection{Geodesics}

How do we have straight line on a curve? eg going round equator, but not going via uk.

Take start direction and find tangent vectors. geodesic is where tangent vectors stay parallel.


\subsection{Markov Logic networks (MLNs)}
Markov Logic networks \cite{markovLogicNetworks} are defined as a collection of first order logical formulas \(F_j\) (for \(j\in 1\ldots K\)), each associated with a real number \(w_j\), the {\em weight} of the rule. Assume given a set of constants \(\{c_1,\ldots, c_L\}\) (in a 1-1 relationship with intended domain of interpretation), and \(a_1,\ldots, a_N\), an enumeration of the ground atoms (built from the given constants and the predicates in the formulas). Then the joint probability of the network given the truth values \(X_1, \ldots, X_n\) (for the ground atoms \(a_i\)) is given by:
\[ joint(X_1,\ldots, X_n)= (1/Z)\times \prod_{F_j} e^{w_j n_j}
\]
\noindent where \(n_j\) is the number of true groundings of \(F_j\) for the given assignment. \(Z\) is the normalization factor.

We can represent this MLN as a PCC agent \mcode{joint(X1,...,Xn)} using an interpreter for quantifier-free first-order formulas written in CCP (see the agent {\tt v/2}).
For any formula \(F_i\) let \(F_i^1, \ldots, F_i^{p_i}\) enumerate its \(p_i\) groundings. For any ground formula \(F\), let \([F]\) stand for the formula obtained by replacing each occurrence of \(a_k\) by the variable \(X_k\) (for all \(k\)).

The predicate \mcode{joint/n} has a single rule with \(n+2\times\Pi_{j\in 1\ldots K} p_j\) atoms in the body. The \(n\) \mcode{b(Xi)} agents each probabilistically guess the value of \mcode{Xi} (drawn from the domain \mcode{\{true,false\}}. Thus computation will result in one probabilistic branch for each of these valuations, each with a probability factor of \(0.5^{n}\) contributed by the \mcode{b/1} agents (hence these factors will get normalized out). The subsquent pairs of agents force a a value for \(\mbox{\tt Y}_{\mbox{\tt i}}^{\mbox{\tt j}}\) (for {\tt i} ranging from {\tt 1} to {\tt K} and for {\tt j} ranging from {\tt 1} to \(\mbox{\tt p}_{\mbox{\tt i}}\) ) by evaluating the formula \(\mbox{\tt [F}_{\mbox{\tt i}}^{\mbox{\tt j}}\mbox{\tt ]}\) (for the given valuation provided by \(\mbox{\tt X}_{\mbox{\tt i}}\)), and contribute a probability factor for this choice.
\begin{lstlisting}[mathescape=true]
joint(X$_1,\ldots, $X$_n$) ->
  b(X$_1$), ..., b(X$_n$),
  Y$_1^1 \sim $ true/exp(w$_1$)+ false/1, v([F$_1^1$],Y$^1_1$),
  ...,
  Y$_1^{p_1}\sim$ true/exp(w$_1$)+ false/1 , v([F$_1^{p_1}$],Y$_{p_1}^1$),
  ...,
  Y$_K^1\sim$ true/exp(w$_K$)+ false/1,  v([F$_K^1$],Y$_K^1$)
  ...,
  Y$_K^{p_K}\sim$ true/exp(w$_K$)+ false/1,  v([F$_K^{p_k}$],Y$_K^{p_K}$),

b(X) -> X$\sim$true/0.5 + false/0.5.
\end{lstlisting}
\noindent Note that the variables \(\mbox{\tt Y}_{\mbox{\tt i}}\) are local to the body (existentially quantified).

The valuation predicate {\tt v/2} is straightforward to define -- it simply evaluates the first argument based on the values of the \(\mbox{\tt X}_{\mbox{\tt i}}\) supplied at the leaves, using standard CCP idioms.
\begin{lstlisting}[mathescape=true]
v(and(A,B),V) -> v(A,VA), v(B,VB), and(VA,VB,V).
v(or(A,B),V) -> v(A,VA), v(B,VB), or(VA,VB,V).
v(not(A),V) -> v(A, VA), not(VA,V).
v(v(X),V) -> X=V.
and(false, \_, Y) -> Y=false.
and(\_, false, Y) -> Y=false.
and(true,true,Y) -> Y=true.
or(true, \_, Y) -> Y=true.
or(\_, true, Y) -> Y=true.
or(false,false,Y) -> Y=false.
not(true,Y) -> Y=false.
not(false,Y)->Y=true.
\end{lstlisting}

\begin{example}\label{ex_mln}
Following the example given by Richardson et al. \cite{markovLogicNetworks}, we consider the following three formulas $F_1$, $F_2$ and $F_3$: \emph{``Smoking causes cancer''} $F_1=\neg sm(X) \vee ca(X)$ with weight $w_1=1.5$;
\emph{``If two people are friends, either both smoke or neither does''} $F_2 = \neg fr(X, Y) \vee sm(X) \vee \neg sm(Y)$ and $F_3= \neg fr(X, Y) \vee \neg sm(X) \vee sm(Y) $ with weights $w_2=w_3=1.1$.
\begin{figure}[h!]
\centering
\begin{tikzpicture}[scale=0.53]
\scriptsize
\tikzstyle{every node}=[draw,shape=ellipse,fill=gray!30];
\node (x1) at (0, 2) {$fr(a,a)$};
\node (x2) at (2, 0) {$ca(a)$};
\node (x3) at (4, 2) {$sm(a)$};
\node (x4) at (6, 0) {$fr(b,a)$};
\node (x5) at (6, 4) {$fr(a,b)$};
\node (x6) at (8, 2) {$sm(b)$};
\node (x7) at (10, 0) {$ca(b)$};
\node (x8) at (12, 2) {$fr(b,b)$};
\draw [-]  (x1) -- (x3);
\draw [-]  (x2) -- (x3);
\draw [-]  (x3) -- (x4);
\draw [-]  (x3) -- (x5);
\draw [-]  (x3) -- (x6);
\draw [-]  (x6) -- (x8);
\draw [-]  (x6) -- (x7);
\draw [-]  (x6) -- (x4);
\draw [-]  (x6) -- (x5);
%\coordinate [label=right:$c_1$] (p) at (5.5,2.8);
%\coordinate [label=right:$c_2$] (p) at (5.5,1.3);
%\coordinate [label=right:$c_3$] (p) at (1.6,2.3);
%\coordinate [label=right:$c_4$] (p) at (2.2,1.2);
%\coordinate [label=right:$c_6$] (p) at (8.6,1.2);
%\coordinate [label=right:$c_5$] (p) at (9.3,2.3);
\end{tikzpicture}
\caption{MLN of Example \ref{ex_mln}}
\label{mln}
\end{figure}
In Figure \ref{mln} we can see the graph of the ground Markov network defined by formulas $F_1$, $F_2$ and $F_3$ and the constants \emph{Anna} ($a$) and \emph{Bob} ($b$).

The equivalent PCC agent is defined as follows. Assume the ground atomic formulas are enumerated as:
\lstinline|fr(a,b)|,
\lstinline|fr(a,a)|,
\lstinline|sm(a)|,
\lstinline|sm(b)|,
\lstinline|fr(b,b)|,
\lstinline|ca(a)|,
\lstinline|fr(b,a)| and
\lstinline|ca(b)|. Then we have:
\begin{lstlisting}[mathescape=true]
joint(X$_1$,...,X$_8$) $\rightarrow$
  b(X$_1$), ... , b(X$_8$),
  Y$_1^1$$\sim$ true/e$^{w_1}$+ false/1,
  v(or(not(X$_3$), X$_6$),Y$_1^1$),
  Y$_1^2$$\sim$ true/e$^{w_1}$+ false/1,
  v(or(not(X$_4$), X$_8$),Y$_1^2$),
  Y$_2^1$$\sim$ true/e$^{w_2}$+ false/1,
  v(or(not(X$_1$),or(X$_3$, not(X$_4$)),Y$_2^1$),
  Y$_2^2$$\sim$ true/e$^{w_2}$+ false/1,
  v(or(not(X$_2$),or(X$_3$, not(X$_3$)),Y$_2^2$),
  Y$_2^3$$\sim$ true/e$^{w_2}$+ false/1,
  v(or(not(X$_5$),or(X$_4$, not(X$_4$)),Y$_2^3$),
  Y$_2^4$$\sim$ true/e$^{w_2}$+ false/1,
  v( or(not(X$_7$),or(X$_4$, not(X$_3$)),Y$_2^4$),
  Y$_3^1$$\sim$ true/e$^{w_3}$+ false/1,
  v(or(not(X$_1$),or(not(X$_3$), X$_4$)),Y$_3^1$),
  Y$_3^2$$\sim$ true/e$^{w_3}$+ false/1,
  v(or(not(X$_2$),or(not(X$_3$), X$_3$)),Y$_3^2$),
  Y$_3^3$$\sim$ true/e$^{w_3}$+ false/1,
  v(or(not(X$_5$),or(not(X$_4$), X$_4$)),Y$_3^3$),
  Y$_3^4$$\sim$ true/e$^{w_3}$+ false/1,
  v(or(not(X$_7$),or(not(X$_4$), X$_3$)),Y$_3^4$).
\end{lstlisting}
\end{example}

It is important to notice that the PCC representation of a MLN has un-normalised weights for its random variables. It is easy to find a normalisation constant that allow to recover the original MLN probability distribution over complete truth assignment of the variables (corresponding to successful executions of the PCC agent) since in the PCC formulation the probability of an execution is the product of the probability over the full set of variables (in each execution we sample all the random variables). So we consider a random variable of the form:
\begin{lstlisting}[mathescape=true]
Y$_i^j$$\sim$ true/e$^{w_i}$+false/e$^0$
\end{lstlisting}
as:
\begin{lstlisting}[mathescape=true]
Y$_i^j$ $\sim$ true/$\frac{e^{w_i}}{e^{w_i}+e^0 }$+false/$\frac{e^0}{e^{w_i}+e^0 }$
\end{lstlisting}
 and at the end of the computation we multiply every execution probability for the following normalisation constant:
 $$Z=\prod_{i=1}^k \prod_{j=1}^{p_i} (e^{w_i} +e^0)=\prod_{i=1}^k (e^{w_i} +e^0)^{p_i}$$

\begin{theorem}
Given a MLN $\mathcal{M}$ defined over a set of clauses $F_j$ and its PCC translation $\mathcal{P}$
the probability distribution $pd_{\mathcal{M}}$ defined by $\mathcal{M}$ over the complete truth assignments of the ground atoms  is equal to the probability distribution $pd_{\mathcal{P}}$ defined by $\mathcal{P}$ over the complete truth assignments of the ground atoms.
\end{theorem}
\begin{proof}
Given a input $x_1, \ldots ,x_n$ where $x_1, \ldots , x_n$ are the truth values of the ground atoms $a_1,\ldots, a_n$ generated by the MLN, we want to prove that the probability of $pd_{\mathcal{P}}$ is equal to $pd_{\mathcal{M}}$ on this input. Given the definition of MLN, we have that $pd_{\mathcal{M}}(x_1, \ldots ,x_n)=\prod_{F_j} e^{w_j n_j}$.

Each execution of the PCC agent provide a complete instantiation of the variables $Y_i^j$ by definition of the predicate $jt$. We have thus that the truth value of each grounding formula $[F]_i^j$ is defined: \lstinline[mathescape=true]{v([F]$_i^j$,Y$_i^j$)}.

The probability of such combination (and the corresponding execution) is the product of the factors associated with each choice: we have a factor $e^{w_i}$ for each \lstinline[mathescape=true]{v([F]$_i^j$,true)} used and $1$ for each \lstinline[mathescape=true]{v([F]$_i^j$,false)} used. Only one of these combinations will succeed, since the input $x_1, \ldots ,x_n$ determines uniquely only one consistent instantiation of the $[F]_i^j$.
Thus there is only one sampling execution that will succeeds, since every input $x_1,\ldots,x_N$ is consistent with only one instantiation of the $[F]_i^j$.
\end{proof}

%We can define a the same encoding of MLNs in PCCs also in SLPs with minor modifications that we omit due to lack of space. It is important to notice that the SLP formalisation of MLN corresponds to an un-normalised and impure SLP program: un-normalised because the sum of the weights for clauses whose heads share the same predicate could be greater than $1$, and impure since there are rules that don't have a weight.