\documentclass[]{revision-notes}

\addbibresource{QT.bib}

\title{PHYS11019: Quantum Theory}
\subtitle{Integrals. SO MANY INTEGRALS.}
\author{James Shaw}
\date{Spring 2018}

\begin{document}
\input{titlepage}

\tableofcontents % prints Table of Contents

\chapter{Back to Basics --- Quantum Mechanics Style}
\section{Double Slit Experiment}

The double slit experiment demonstrates some of the most important points of quantum mechanics.
Feynman was a particular fan of it. We make $P_1 = \abs{\phi_1}^2$ the probability of a particle passing through the first slit and likewise for $P_2$.
Classically, we'd expect that $$P_{1\text{ or }2} = P_1 + P_2 = \abs{\phi_1}^2 + \abs{\phi_2}^2,$$ which is all well and good but isn't what happens when we approach the quantum world.

We define two states, $\ket{i}$ and $\ket{f}$, the initial and final states, and two intermediate states $\ket{1}, \ket{2} \in \mathcal{H}$ for each of the slits.
These are vectors in a Hilbert space, and as such are linear superposable and have dual basis vectors $\bra{i}, \bra{f}, \bra{1}, \bra{2} \in \mathcal{H}^\ast$.
These define the normalisation ($\braket{\psi} = 1$) and are used to great effect elsewhere.

The probabilities in this formalism are defined by
\begin{align*}
  P(i \to f) &= \abs{\bra{f}\ket{i}}^2\\
   &= \abs{\bra{f}\ket{1}\bra{1}\ket{i} + \bra{f}\ket{2}\bra{2}\ket{i}}^2\\
  &= \abs{\bra{f}\ket{1}\bra{1}\ket{i}}^2 + \abs{\bra{f}\ket{2}\bra{2}\ket{i}} + 2 \Re(\bra{f}\ket{1}\bra{1}\ket{i}^\ast \bra{f}\ket{2}\bra{2}\ket{i})
\end{align*}
where the completeness of the Hilbert space has been used.
This asserts that \[ \hat{\mathbbm{1}} = \sum_{\text{states}\, k} \dyad{k}\] where \( \ket{k}\in\mathcal{H} \) are orthonormal basis vectors.
This is what is meant by ``inserting a complete set of states''.
Orthonormality of discrete eigenvectors (eigenfunctions, eigenstates\ldots) is defined by \[ \braket{n}{m} = \delta_{mn}\] and in the continuous limit is defined by \[\braket{x}{x^\prime} \delta(x-x^\prime).\]

We can change the basis of a set of states with \[\ket{\bar{n}} = \sum_{m} \ket{m}\braket{m}{\bar{n}} \] where \(\braket{m}{n} = U_{mn}\) where $U \in \mathrm{SL}(n)$.

\section{Operators and Observables}
An observable \(\hat\xi = \sum_n \xi_n \op{n}\) is an operator which satisfies the following properties:
\begin{enumerate}
  \item That there are states \(\ket{n}\) which are eigenstates of $\hat \xi$ such that $\xi_n \in \mathbb{R}$ are the eigenvalues.
  This comes from the spectral theorem and representation.
  \item It is hermitian, that is to say \[\mel{\phi}{\hat\xi^\dagger}{\psi} =\mel{\phi}{\hat\xi}{\psi}\] which requires that the eigenvalues $\xi_n$ are real by the spectral theorem.
  \item It is complex-linear, such that \[ \hat\xi(c_1 \ket{\psi_1} + c_2 \ket{\psi_2} ) = c_1\hat\xi \ket{\psi_1} + c_2\hat\xi \ket{\psi_2} \] for $c_i \in \mathbb{C}$. This comes from the definition of the Hilbert space.
  \item It commutes with other observables, such that if $\hat{\chi}$ is an observable, then $\commutator{\hat\chi}{\hat\xi}=0$.
\end{enumerate}

When we measure an observable, we get a collapse of the quantum states.
This is shown through the use of the \emph{projection} operator, \[\hat{P}_n = \ketbra{n}\] which has the effect of throwing away all other states in a composite state.
For example, \[ \hat{P}_1 (c_1\ket{1} + c_2\ket{2}) = c_1\ket{1}\braket{1} + c_2\ket{1}\bra{1}\ket{2} = c_1 \ket{1} \] which seems nice and obvious in the maths, but the physics behind it is \emph{weird}.

Degeneracy also makes this weird, but fundamentally it's just adding in more sums until everything is nicely in one state.

\section{Squish It All Together}
We move to the continuous variables of position and momentum space.
This is equivalent to having infinite slits in your diffraction grating.

\subsection{Position}
Position space is defined such that \[ \int_a^b \psi(x) \ket{x} \dd{x}\] with the orthonormality condition \[\braket{x}{x^\prime}=\delta(x-x^\prime). \]
This means we can define the projection on to position space as a function of the position, i.e.~\[\braket{x}{\psi} = \psi(x).\]
The position operator is then \[ \hat{x} = \int x \ketbra{x} \dd{x} \] which is also known as the spectral resolution, and we've used the continuum analogue of the identity operator
\[ \hat{\mathbbm{1}} = \int \ketbra{x}\dd{x}. \]

\subsection{Momentum}
We use the fourier transform to find the momentum space.
We define states \(\ket{k}\) conventionally such that \[ \ket{k} = \frac{1}{\sqrt{2\pi}} \int e^{ikx} \ket{k} \dd{x}\] which allows us to identify that
\begin{align*}
  \braket{x}{k} &= \frac{1}{\sqrt{2\pi}}e^{ikx} & \braket{k}{x} &= \frac{1}{\sqrt{2\pi}}e^{-ikx},
\end{align*}
in turn showing that it is a unitary transformation between bases.

It's worth noting that \[ \tilde\psi(k) = \mathcal{F}\qty[\psi(x)].\]

\subsection{Working Between The Two}
We use Feynman's trick to find the action of the momentum operator on the position basis, given by
\begin{align*}
  \hat{k} \ket{x} &= \frac{1}{\sqrt{2\pi}} \int k e^{-ikx} \ket{k} \dd{k}\\
  &= \frac{i}{\sqrt{2\pi}} \pdv{x} \int e^{-ikx} \ket{k} \dd{k}\\
  &= i \pdv{x} \ket{x}
\end{align*}
which gives us the final results
\begin{align*}
  \mel{x}{\hat{k}}{\psi} &= -i\pdv{x} \psi & \mel{k}{\hat{x}}{\tilde\psi} &= i\pdv{k}\tilde\psi.
\end{align*}

\subsection{Canonical Commutation Relation}
The fundamental result of quantum mechanics is given by the canonical commutation relation, \[ \commutator{\hat{x}}{\hat{k}} = i \] which more practically is used as \[\commutator{\hat{x}}{\hat{p}} = i\hbar. \]

\section{Time is a Continuum. Lunchtime, doubly so}
We now increase the number of gratings, and take it to its continuum limit.
Firstly, we define a system with \(N\) gratings arranged one after another.
This gives us states \( \ket{x_i, t_i} \) where the \(i\) labels the grating, so \(t_i\) is the time the \(i\)-th grating is passed.
Using the Hilbert completeness relation, this gives us \[\braket{f}{i} = \int \braket{f}{x_1,t_1} \braket{x_1,t_1}{i}\dd{x_1}\] for \(N=1\).
Increasing \(N\) further gives us \[\braket{f}{i} = \idotsint \braket{f}{x_1,t_1} \braket{x_1,t_1}{x_2,t_2} \bra{x_2, t_2}\cdots\ket{x_n,t_n}\braket{x_n, t_n}{f}\dd{x_1}\cdots\dd{x_n}\] where the increasing number of integrals increases the precision, but makes actual calculations difficult.

Defining \(\ket{i} = \ket{x_0, t_0} \) and \( \ket{f} = \ket{x_{N+1}, t_{N+1}} \) allows us to evenly space the gratings a distance \( \varepsilon \) from each other with
\begin{alignat*}{3}
  t_n &= t_0 + n\varepsilon & \qquad \text{where}&\qquad& \varepsilon &= \frac{t_{N+1} - t_0}{N+1}
\end{alignat*}
which then allows us to write the transition amplitude as
\begin{align*}
  \braket{x_b, t_b}{x_a, t_a} &= \qty(\prod_{n=1}^{N}\int\dd{x_n})\qty(\prod_{n=1}^{N+1}\braket{x_{n}, t_{n}}{x_{n-1}, t_{n-1}}).
\end{align*}

To make the maths neater, we define a new measure which compactifies this notation for the \(N \to \infty \) limit.
This is \[ \braket{x_b, t_b}{x_a, t_a} =  \eval{\int_{x_a}^{x_b}\DD{x}\braket{x_b, t_b}{x_a, t_a}}_{x(t)}. \]

\chapter{Tiny Little Things Sort Of Moving Around Not Actually That Fast\ldots{} Maybe}
\section{Classically Speaking, Actions Are Awkward}
The Lagrangian definition of the action is \[ S[x(t)] = \int_{t_a}^{t_b} L(x, \dot{x}, t) \dd{t} \]
where \(L = T-V \) is the classical Lagrangian (of a point particle in some generalised co-ordinate) and \(S[x(t)]\) is a functional of the position. We also have canonical momenta \[ p = \pdv{L}{\dot{x}} \] which coincide with regular momenta in most cases, rather like the energy function.

We define the functional derivative
\[ \eval{\frac{\delta S}{\delta x}}_{x = \bar x} = 0 \] which is a statement of Hamilton's principle, defining the classical trajectory where the endpoints are fixed. What it actually \emph{means} is that we vary the action along a path by an infinitesimal using \[ \delta S = S[x + \delta x] - S[x] \] and Taylor expand. This leads to the Euler-Lagrange equation for the classical trajectory, \[ \dv{t}(\pdv{L}{\dot x}) - \pdv{L}{x} = 0 \] which defines the classical trajectory \(\bar{x} \) on which the \emph{classical action}
\(S_{\mathrm{cl}} = S[\bar x(t)]\) is defined. This will come back later.

\section{Here Come The Irish With Their Principle Functions}
If we don't fix the later-in-time endpoint in space, we derive Hamilton's principle function.
The variation of \( (x_b, t_b) \mapsto (x_b, t_b) + (\delta x_b, \delta t_b)\) (where we then take \(\variation{t_b} = 0\)) gives us that this variation is \begin{align*}
  \variation{S_{\mathrm{cl}}} &= \eval{\pdv{L}{\dot{x}}\variation{x}}^{t_b}\\
  &= p(t_b)\variation{x}\\
  \implies \pdv{S_{\mathrm{cl}}}{x} &= p_b.
\end{align*}

Doing the equivalent variation for variation-in-time for the endpoint in space, we get the Hamilton-Jacobi equation for the Hamiltonian, \[\variation{S_{\mathrm{cl}} = - E_b \variation{t_b}}\] or otherwise written,
\[E(x_b, \pdv{S_{\mathrm{cl}}}{x_b}, t_b) + \pdv{S_{\mathrm{cl}}}{t_b} = 0\] which \emph{defines} Hamilton's principle function, which we've suspiciously also named \(S_{\mathrm{cl}}\).

\section{Just how probable are these paths, then?}
We go back to the transition amplitudes.
Through some slightly iffy arguments involving ``intuition'', we define the path amplitudes as
\begin{align*}
  \eval{\braket{x_a, t_a}{x_b, t_b}}_{x(t)} &\sim \exp[i\int_{t_a}^{t_b} \!\!\!\phi(x, \dot x, t) \dd{t}]\\
  &= e^{\nicefrac{i}{\hbar} S[x(t)]}.
\end{align*}
which we can get an expression for the full transition with
\begin{align*}
  \braket{x_b, t_b}{x_a, t_a} &= \int_{x_a}^{x_b}\!\DD{x} e^{\nicefrac{i}{\hbar} S[x(t)]}.
\end{align*}

Now, this still has the funny \(\DD{x}\) thing, which is defined as
\[\int_{x_a}^{x_b}\!\DD{x} = \lim_{N\to \infty} A_N \prod_{n=1}^N \int_{-\infty}^{\infty} \dd{x_{n}} \] where \(A_N = (\nu(\varepsilon))^{N+1}\) is a normalisation factor equal across each discrete step.
Ideally, it should make the entire thing converge, but we now need to find what \(\nu(\varepsilon)\) is.

\section{Where's a potential when you need one?}
We take a discrete time approximation for the Lagrangian of a free particle, that is to say
\[ L = T = \frac{1}{2} m \dot{x}^2 \approx \frac{1}{2} m \qty(\frac{x_1 - x_0}{\varepsilon})^2.\]
Turning this into a transition amplitude gives us (remembering that we've discretized time again)
\begin{align*}
 \braket{x_b, t_b}{x_a, t_a} &= \lim_{N\to \infty} A_N \prod_{n=1}^N \int_{-\infty}^{\infty} \dd{x_{n}} \exp[\frac{i\varepsilon}{\hbar} \frac{m}{2} \sum_{n=1}^{N+1}\qty(\frac{x_n - x_{n-1}}{\varepsilon})^2] \intertext{which, using results from \autoref{sec:nested-gaussians}, gives us}
 \braket{x_b, t_b}{x_a, t_a} &= \lim_{N\to \infty} A_N \qty(\frac{2 \varepsilon \hbar}{m})^{\nicefrac{N}{2}} \prod_{n=1}^N \int_{-\infty}^{\infty} \dd{x^\prime_{n}} \exp[i \sum_{n=1}^{N+1}\qty(\frac{x^\prime_n - x^\prime_{n-1}}{\varepsilon})^2]\\
 &= \lim_{N\to \infty} A_N \qty(\frac{2\pi i \varepsilon \hbar}{m})^{\nicefrac{N}{2}} \frac{1}{\sqrt{N+1}}\exp[\frac{i\qty(x^\prime_b - x^\prime_{a})^2}{\varepsilon(N+1)}]\\
 &= \lim_{N\to\infty}  A_N \qty(\frac{2\pi i \varepsilon \hbar}{m})^{\nicefrac{N+1}{2}} \sqrt{\frac{m}{2\pi i \hbar (N+1)\varepsilon}}\exp[\frac{im}{2\hbar}\frac{\qty(x_b - x_{a})^2}{\varepsilon(N+1)}].
\end{align*}
Now we must remember what we were trying to do --- find the normalisation \(\nu(\varepsilon)\) for the period \(T = t_b - t_a = (N+1)\varepsilon \).
To do this, we impose the requirement that
\[ A_N \qty(\frac{2\pi i \varepsilon \hbar}{m})^{\nicefrac{N+1}{2}} = 1\]
 which is equivalent to
 \[\nu(\varepsilon) = \sqrt{\frac{2\pi i \varepsilon \hbar}{m}},\]
  allowing us to simplify the above result into
\begin{equation}\label{eqn:free-particle}
  \braket{x_b, t_b}{x_a, t_a} = \sqrt{\frac{m}{2\pi i \hbar T}}\exp[i\frac{m}{2\hbar}\frac{\qty(x_b - x_{a})^2}{T}].
\end{equation}

Now, it can be worked out easily that for a free particle,
\[ S_{\mathrm{cl}} = \frac{m}{2} \frac{(x_b - x_a)^2}{T}\] which is suspiciously similar to \autoref{eqn:free-particle}, so we make an identification that
\begin{alignat*}{3}
  \braket{x_b, t_b}{x_a, t_a} &= F_0(T)e^{i\nicefrac{S_{\mathrm{cl}}}{\hbar}} & \qq{where} && F_0(T) &= \sqrt{\frac{m}{2\pi i \hbar T}}
\end{alignat*}
which is a \emph{very} helpful notation, since the prefactor is now independent of the endpoints.

You may notice that \( \nu(\varepsilon) \) diverges for infinitesimal \(\varepsilon \), which is a little inconvenient, but we basically ignore it, as that choice of \(\nu\) is only the first order term.

\section{Oscillation of its own accord}
For a harmonic oscillator, the ``discretize time'' method becomes awkward to do; however, there's another thing we can try.
Firstly, the Lagrangian for the harmonic oscillator is
\[ L = \frac{1}{2} m \dot{x}^2 - \frac{1}{2}m \omega x^2\] from which we can calculate the classical action,
\[ S_{\mathrm{cl}} = \frac{m\omega}{2\sin(\omega T)} \qty[(x_a^2 + x_b^2)\cos(\omega T) - 2x_a x_b]\]
and the Newtonian equation of motion, \[ \ddot{x} + \omega^2 x = 0. \]

We will write \( x = \bar{x} + \eta \) where \(\eta(t_a) = \eta(t_b) = 0\) is the deviation from the classical path (allowing us to use the variational principle).
We find that
\begin{align*}
  S[x] = S[\bar{x} + \eta] &= \frac{m}{2}\int_{t_a}^{t_b} \qty[ \qty(\dot{\bar{x}} + \dot\eta)^2 - \omega^2(\bar{x} + \eta)^2] \dd{t}\\
                    &= S[\bar{x}] + S[\eta] + m\int_{t_a}^{t_b} \dot{\bar{x}}\dot\eta - \omega^2\bar{x}\eta\dd{t}\\
                    &= S[\bar{x}] + S[\eta] + \cancel{m[\eta \dot{\bar{x}}]_{t_a}^{t_b}} - m \int_{t_a}^{t_b}\eta \cancel{(\ddot{\bar{x}} + \omega^2 \bar{x}) }\dd{t}\\
                    &= S[\bar{x}] + S[\eta]
\end{align*}
which is, in fact, true for any Lagrangian quadratic in \(x\).
We can also argue that \[ \int_{x_a}^{x_b} \DD{x} = \int_{0}^{0} \DD{\eta}\] since \(\dd{x} = \dd{\eta} \), which looks a bit weird, but it's a useful statement.
This gives us our transition amplitude
\begin{align*}
  \braket{x_b, t_b}{x_a, t_a} &= \int_{x_a}^{x_b} e^{\nicefrac{i}{\hbar} S[x(t)]} \DD{x}\\
  &= e^{\nicefrac{i}{\hbar} S_{\mathrm{cl}}} \int_{0}^{0} e^{\nicefrac{i}{\hbar} S[\eta]} \DD{\eta}\\
  &\equiv F_\omega(T)e^{\nicefrac{i}{\hbar} S_{\mathrm{cl}}}
\end{align*}
which people solve in various ways, such as in \cite{feynman2010quantum}. Apparently the easiest is to do it by inspection, where we notice that since \( F_\omega (T) \) has no position dependence, we can also write it as
\begin{align*}
  F_\omega(T) &= \braket{0, T}{0, 0}\\
              &= \int_\mathbb{R} \bra{0,T}\ket{x,t}\bra{x,t}\ket{0,0} \dd{x}\\
              &= \int_\mathbb{R} \dd{x} F_\omega(T - t)e^{\nicefrac{i}{\hbar} S_{\mathrm{cl}}(T-t)}\,F_\omega(t)e^{\nicefrac{i}{\hbar} S_{\mathrm{cl}}(t)}\\
              &= F_\omega(T - t)\, F_\omega(t) \int_\mathbb{R} \dd{x} e^{\nicefrac{i}{\hbar} (S_{\mathrm{cl}}(T-t) + S_{\mathrm{cl}}(t))}
\end{align*}
at which point we use the definition of the classical action from earlier, notice that it's quadratic in \(x \) making it all a Fresnel integral, and abuse the nested Gaussians formula from \autoref{sec:nested-gaussians}.
This gives us
\begin{equation*}
  F_\omega(T) = F_\omega(T - t)\, F_\omega(t) \sqrt{\frac{2\pi i \hbar}{m\omega} \frac{\sin\omega(T-t) \sin\omega t}{\sin \omega T}}
\end{equation*}
which we can look at and notice that \( F_\omega (T) \) should probably be
\[ F_\omega (T) = \sqrt{\frac{m\omega}{2\pi i \hbar} \frac{1}{\sin \omega T}} \] if we want the two other terms to cancel out. There's no systematic way of doing this really, though taking the \( T \gg t \) limit should result in the same.

\section{Driven to Distraction}
The forced harmonic oscillator has the Lagrangian
\[ L = \frac{1}{2} m (\dot{x}^2 - \omega^2 x^2) + J(t)x \]
and equation of motion for the classical path
\[ \ddot{\bar{x}} + \omega^2 \bar{x} = \frac{J}{m}. \]

This leaves the classical action \(S_\mathrm{cl}[\bar{x} + \eta]_{J} \), which helfully splits into independent terms like the unforced case, so it all works out rather nicely and we get
\begin{equation*}
  \braket{x_b, t_b}{x_a, t_a} = F_\omega(T)e^{\frac{i}{\hbar} S_\mathrm{cl}[\bar{x}]_{J}}
\end{equation*}
where we must calculate \(S_\mathrm{cl}\).

\section{Shiftying Everything in Time on Our Terms}
We look at a generic Lagrangian, \[ L= \frac{1}{2} m \dot{x}^2 - V(x, t) \] which we can Legendre transform into the Hamiltonian formalism and do the path integral to get
\[ \braket{x_b, t_b}{x_a, t_a} = \int \DD{x} \int \DD{p} \exp[\frac{i}{\hbar} \int_{t_a}^{t_b} \dd{t} (p\dot{x} - H(x, p, t))] \]
which comes from the infinitesimal amplitude
\begin{align*}
  \braket{x_{n+1}, t_{n+1}}{x_n, t_n} &= \sqrt{\frac{m}{2\pi i \varepsilon\hbar}} \exp[\frac{i\varepsilon}{\hbar}\qty{\frac{m}{2}\qty(\frac{x_{n+1} - x_n}{\varepsilon})^2 - V(x_n, t_n)}] \\
  &= \int_{-\infty}^\infty \frac{1}{2\pi\hbar}\exp[\frac{ip_n}{\hbar}(x_{n+1} - x_n) - \frac{i\varepsilon}{2m\hbar}p_n^2 - \frac{i\varepsilon}{\hbar}V(x_n, t_n)]\dd{p}\\
  &= \int_{-\infty}^\infty \mel{x_{n+1}}{\exp[-\frac{i\varepsilon}{2m\hbar}p_n^2]}{p_n} \mel{p_n}{\exp[ -\frac{i\varepsilon}{\hbar}V(x_n, t_n)]}{x_{n}}\dd{p}\\
  &=\mel{x_{n+1}}{\exp[-\frac{i\varepsilon}{2m\hbar}\hat{p}_n^2]\exp[ -\frac{i\varepsilon}{\hbar}V(\hat{x}_n, t_n)]}{x_{n}}
\end{align*}
which has successfully been moved into an operators form, and looks rather suggestive.
We can use the Baker-Campbell-Hausdorff formula \[ e^{\hat{A}}e^{\hat{B}} = e^{\hat{A} + \hat{B} + \frac{1}{2}\commutator{\hat{A}}{\hat{B}}} + \cdots \]
to first order to turn this into the infinitesimal time evolution operator \[\braket{x_{n+1}, t_{n+1}}{x_{n}, t_{n}} = \mel{x_{n+1}}{e^{-\frac{i\varepsilon}{\hbar{}} \hat{H}(t_n)}}{x_{n}}.\]
This \emph{defines} the time evolution operator for finite time for a time independent Hamiltonian,
\[ \ket{x, t} = \hat{U}(t, t_0)\ket{x} = e^{-\frac{i}{\hbar{}} (t - t_0) \hat{H}(t)}\ket{x}.\]

For time dependent Hamiltonians, we have (by the chain rule) that
\[ i\hbar \pdv{t}\hat{H}(t, t_0) = \hat{H}(t)\hat{U}(t, t_0). \]
\(\hat{U}\) is fundamentally changes in basis.

\section{Herren Schr\"odinger und Heisenberg kommen}
Using the time evolution operator, we can find derive the time-dependent Schr\"odinger equation, \[ i\hbar\pdv{t}\psi(x, t) = \hat{H}\psi(x, t) .\]

By defining a time dependent state vector \[ \ket{\psi, t} = \hat{U}(t, t_0) \ket{\psi} \] we introduce the Schr\"odinger picture of quantum mechanics where the bases are fixed and the states change, in opposition to the Heisenberg picture of fixed states and changing bases.
The choice of picture is a choice of where we put the time dependence, and does not change the physics of the problem.

In the Schr\"odinger picture, operators are time independent, so all evolution is done in-state. Contrastingly, in the Heisenberg picture, the operators are time \emph{dependent}, and are given by \[ \hat{x}(t) = \hat{U}^\dagger(t, t_0)\, \hat{x}\, \hat{U}(t, t_0) \] which is clearly the same if applied to a state. This time dependence gives the Heisenberg equation of motion almost immediately, which is
\begin{align*}
  i\hbar \pdv{t}\hat{O}(t) &= \commutator{\hat{O}(t)}{\hat{H}(t)}.
\end{align*}

We say that an operator is conserved if \(\pdv{\hat{O}}{t} = \commutator{\hat{O}(t)}{\hat{H}(t)} = 0 \).

\section{A British Miller (who happened to be really good at Maths) Cometh}
Moments of recognition might've triggered at some point that \( \braket{x_a, t_a}{x_b, t_b} \) is actually a Green's function of the Sch\"odinger equation, i.e.~
\begin{align*}
  \qty(\hat{H} - i\hbar\pdv{t}) \braket{x, t}{x^\prime, t^\prime} &= -i\hbar \delta(t- t^\prime) \delta(x - x^\prime)
\end{align*}
although this isn't quite true, as the time parameter isn't fixed automatically.
Specifically, this is the retarded Green's function, where we must impose causality, giving us
\begin{equation*}
  G(x-x^\prime, t - t^\prime) =
  \begin{cases}
    \braket{x, t}{x^\prime, t^\prime} & t > t^\prime \\
    0 & t < t^\prime
  \end{cases}
\end{equation*}
which is kind of useful, since we can now use Green's function methods for finding states.

\section{Energy Eigenbasis}
We can use the eigenstates of the Hamiltonian as a pretty nifty basis, since the discretized energy levels means that everything is nicely sums. Specifically, if
\begin{align*}
  \hat{H}\ket{n} &= E_n\ket{n}\\
  u_n(x) &= \braket{x}{n}
\end{align*}
the practical upshot is that for \(t > 0 \), the colourful function is
\begin{align*}
  G(x-y, t) \equiv \braket{x, t}{y, 0} &= \mel{x}{e^{-\nicefrac{i}{\hbar}\,t \hat{H}}}{y}\\
  &= \sum_n e^{-\nicefrac{i}{\hbar}\,t E_n} u_n(x)\,u_n^\ast(y).
\end{align*}

The trace of the energy eigenbasis gives us \[ \int_{-\infty}^{\infty} \braket{x,t}{x,0} \dd{x} = \sum_n e^{-\nicefrac{itE_n}{\hbar}} \] which is really somewhat useful.\footnote{In the notes, at this point, there follows a good few examples which I simply cannot be bothered to type out, more's the pity. They don't actually seem to be that illuminating, though the Aharonov-Bohm effect is an interesting thing --- a magnetic field which by rights shouldn't effect the path of the particle does because integration.}

\section{Transition Elements}
The transition amplitude discovered several times now leads us to ask ``what is the weighted average of the paths on an operator?''.
We discover that it is
\begin{align*}
  \ev{x(t)}_S &\equiv \int_{x_a}^{x_b} x(t) \, e^{\frac{i}{\hbar} S[x]} \DD{x}\\
  &= \int_{-\infty}^\infty x \int_x^{x_b} \exp[\frac{i}{\hbar} \int_t^{t_b} L \dd{t^{\prime\prime}}]\DD{x^{\prime\prime}} \int^x_{x_a}\exp[\frac{i}{\hbar} \int^t_{t_a} L \dd{t^{\prime}}]\DD{x^{\prime}}\dd{x}\\
  &= \mel{x_b, t_b}{\hat{x}(t)}{x_a, t_b}
\end{align*}
which connects the path integrals to the traditional operator version.

Similarly, \[ \ev{f(x(t))}_S \equiv \int_{x_a}^{x_b} f(x(t))\, e^{\frac{i}{\hbar} S[x]} \DD{x} = \mel{x_b, t_b}{f(\hat{x}(t))}{x_a, t_b}\]
which is going to come in handy.

\section{Time Ordered Products}
It's worth noting, as it hasn't been stated previously, that positions at different times do not commute in the Heisenberg (or any) picture, that is to say \( \commutator{\hat{x}(t)}{\hat{x}(t^\prime)} \neq 0 \) for \( t \neq t^\prime \).
This motivates us to write
\[ \ev{x(t) x(t^\prime)}_S = \mel{x_b, t_b}{T(\hat{x}(t), \hat{x}(t^\prime))}{x_a, t_b} \]
where we define the \emph{time ordered product} as \[ T(\hat{x}(t), \hat{x}(t^\prime)) \equiv \theta(t - t^\prime)\hat{x}(t)\hat{x}(t^\prime) + \theta(t^\prime - t)\hat{x}(t^\prime)\hat{x}(t) \] which has the operators ordered from latest to earliest.

We can apply this to more general states \( \ket{\phi} \& \ket{\psi} \), giving
\begin{align*}
  \mel{\psi}{T(\hat{x}(t_1)) \cdots \hat{x}(t_2))}{\phi} &= \int \dd{x_a} \int \dd{x_b} \psi^\ast (x_b, t_b) \phi(x_a, t_a)  \ev{x(t_1)\cdots x(t_n)}_S
\end{align*}
which are also known as correlation functions.

\section{Another Austrian, Another Theorem, Classical Physics?}
By considering the spatial derivatives of the transition elements, we derive Ehrenfest's theorem,
\begin{equation*}
  \ev{\ddot{x}}_S = - \frac{1}{m} \ev{\pdv{V}{x}}_S
\end{equation*}
which looks a lot like Newton's 2nd law, and handily works like it too.

\chapter{Making It All A Bit Less Accurate}
\section{Perturbation Theory from Path Integrals}
Finding exact solutions is tricky (or impossible), and usually not worth it.
However, we can usually use perturbation theory by splitting the action into a solvable part and a perturbation, such as
\begin{align*}
  S[x(t)] &= S_0[x(t)] + S_1[x(t)]
\end{align*}
and then we can then consider (for slowly varying potentials), we can write
\begin{alignat*}{3}
  S_0[x(t)] &= \int_{t_a}^{t_b} \frac{1}{2}m\dot{x}^2 \dd{t} & \qq{and} &&  S_1[x(t)] &= -\int_{t_a}^{t_b} V(x(t), t) \dd{t}\intertext{or}
  S_0[x(t)] &= \int_{t_a}^{t_b} \frac{1}{2}m\dot{x}^2 - U(x) \dd{t} & \qq{and} &&  S_1[x(t)] &= -\int_{t_a}^{t_b} \tilde{V}(x(t), t) \dd{t}
\end{alignat*}
where we split \(V(x,t) = U(x) + \tilde{V}(x,t)\) where \(U\) is a solvable, time independent potential, often that of the Harmonic oscillator.

Substituting this separation into the transition amplitude gives
\begin{align*}
  \braket{x_b, t_b}{x_a, t_a} &= \int_{x_a}^{x_b} \exp[\frac{i}{\hbar}(S_0[x(t)] + S_1[x(t)])] \DD{x}\\
  &=  \int_{x_a}^{x_b} e^{\frac{i}{\hbar}S_0[x(t)]}\sum_{n=0}^\infty \frac{1}{n!}\qty(\frac{i}{\hbar}S_1[x(t)])^2 \DD{x}\\
  &= \braket{x_b, t_b}{x_a, t_a}_0 + \sum_{n=1}^\infty  \frac{1}{n!}\qty(\frac{i}{\hbar})^2 \prod_{k=1}^{n} \int_{t_a}^{t_b} \dd{t_n} \int_{x_a}^{x_b} \DD{x} \prod_{m=1}^{n}V(x(t_m), t_m) e^{\frac{i}{\hbar} S_0[x(t)] }
\end{align*}
where the last one uses the time ordered product.
Interpreting the last of those units is interesting, as
\begin{align*}
  \int_{x_a}^{x_b} \DD{x} \prod_{m=1}^{n}V(x(t_m), t_m) e^{\frac{i}{\hbar} S_0[x(t)] } = \int \dd{x_1} \cdots \int \dd{x_n} &\braket{x_b, t_b}{x_{n}, t_{n}}_0 V(x_n, t_n)  \\
  &\times \braket{x_{n}, t_{n}}{x_{n-1}, t_{n-1}}_0 \\
  & \times \cdots \times V(x_1, t_1) \braket{x_{2}, t_{2}}{x_{1}, t_{1}}_0
\end{align*}
which means that the perturbation can be thought of as the sum of situations where the particle is scattered through interaction with the potential an increasing number of times with each unit.

The \(x\) and \(t\) integrals are needed to sum over all paths (as is the basic conceit of the path integral) and the \( \frac{1}{n!}\) unit ensures that we don't overcount for each of the time ordered units.
There's an identity\footnote{which I genuinely don't see the use of} which is that
\begin{equation*}
  \frac{1}{n!}\int_{t_a}^{t_b} \dd{t_n} \cdots \int_{t_a}^{t_b} \dd{t_1} V(t_1) \cdots V(t_n) = \int_{t_a}^{t_b} \dd{t_n} \int_{t_a}^{t_n} \dd{t_{n-1}} \cdots \int_{t_a}^{t_2} \dd{t_1} V(t_1) \cdots V(t_n)
\end{equation*}
for\ldots reasons\ldots
Using this identity, we can get rid of the factorials and find that
\begin{align*}
    \braket{x_b, t_b}{x_a, t_a} &= \braket{x_b, t_b}{x_a, t_a}_0 + \sum_{n=1}^\infty \qty(-\frac{i}{\hbar})^n \prod_{k=1}^{n} \int_{t_a}^{t_{n+1}} \dd{t_n} \int_{x_a}^{x_b} \DD{x} \prod_{m=1}^{n}V(x(t_m), t_m) e^{\frac{i}{\hbar} S_0[x(t)] }.
\end{align*}

Some rewriting can occur, it all makes a meal out of getting to a statement which feels sort of intuitive, \begin{equation*}
  \braket{x_b, t_b}{x_a, t_a} = \braket{x_b, t_b}{x_a, t_a}_0  - \frac{i}{\hbar} \int_{-\infty}^\infty \dd{x} \int_{t_a}^{t_b} \dd{t} \braket{x_b, t_b}{x, t}_0 V(x, t) \braket{x, t}{x_a, t_a}.
\end{equation*}
This looks a little funky, but it's fine really; look at it as a weird application of the Green's function at different times.
It's worth noting that the last term (the \( \braket{x, t}{x_a, t_a}\)) is in relation to the full action, not just the unperturbed, so this is actually a hugely telescoping thing, which we can choose to use as many or as few terms of it as wanted.

We can apply this for a general wavefunction as
\begin{align*}
  \braket{x, t}{\psi} &= \braket{x, t}{\psi}_0 - \frac{i}{\hbar}\int_{-\infty}^\infty \dd{x^\prime} \int_{t_0}^t \dd{t} \braket{x, t}{x^\prime, t^\prime}_0 V(x^\prime, t^\prime) \braket{x^\prime, t^\prime}{\psi}
\end{align*}
which really emphasises the Green's function nature.

\section{Shooting Things At A Target Which Doesn't Move}
We invent a scattering operator, the \(S\) matrix, defined by
\begin{align*}
  \lim_{t_b \to \infty} \lim_{t_a \to -\infty} \braket{\vec{r}_b, t_b}{\vec{r}_a, t_a} &=   \lim_{t_b \to \infty} \lim_{t_a \to -\infty} \mel{\vec{r}_b}{\hat{U}(t_b, t_a)}{\vec{r}_a}\\
  &\equiv \mel{\vec{r}_b}{\hat{S}}{\vec{r}_a}\\
  &= \lim_{T \to \infty} \braket{\vec{r}_b, T}{\vec{r}_a, 0}
\end{align*}
which means we define in (at \( r_a \)) and out (at \( r_b \)) states at infinite time, but have the choice to start our internal clock whenever.

In the region outside the potential, we assume that the incident particle is a free particle with Lagrangian \[L_0 (\vec{r}) = \frac{m}{2} \qty(\dot{x}_1^2 + \dot{x}_2^2 + \dot{x}_3^2).\]

We then calculate to increasing orders in perturbation theory, the first order of which is
\begin{align*}
  A_1 &= - \frac{i}{\hbar} \int_{-\infty}^\infty \dd[3]{r} \int_{0}^{T} \dd{t} \braket{\vec{r}_b, t_b}{\vec{r}, t}_0 V(x, t) \braket{\vec{r}, t}{\vec{r}_a, t_a}\\
      &= - \frac{i}{\hbar} \int_{-\infty}^{\infty} \dd[3]{r} \int^T_0 \dd{t} \qty(\frac{m}{2\pi i \hbar (T-t)})^{\nicefrac{3}{2}}\exp{\frac{im\abs{\vec{r}_b - \vec{r}}^2}{2\hbar (T - t)}} V(\vec{r})\\
      &\mspace{300mu}\times\qty(\frac{m}{2\pi i \hbar t})^{\nicefrac{3}{2}}\exp{\frac{im\abs{\vec{r} - \vec{r}_a}^2}{2\hbar (T - t)}}.
\end{align*}
Solving that integral is a little tricky.
It requires a pretty complicated statement which you can probably just look up.
We set \( R_a = \abs{\vec{r}_a - \vec{r}} \) and \( R_b = \abs{\vec{r}_b - \vec{r}} \) and thus get
\begin{equation*}
A_1 = -\frac{i}{\hbar}\qty(\frac{m}{2\pi i\hbar{}})^{\nicefrac{5}{2}} \frac{1}{T^{\nicefrac{3}{2}}} \int^\infty_{-\infty} \qty(\frac{1}{R_a} + \frac{1}{R_b}) V(\vec{r}) \exp[\frac{im}{2\hbar T}(R_a + R_b)^2] \dd[3]{r}
\end{equation*}

We continue our quest to rewrite this ever less accurately by assuming that the potential is short range and thus expanding the statements
\[ R_i = \abs{\vec{r}_i - \vec{r}} \approx r_i - \vec{n}_i \vdot \vec{r} \]
which, with substitution and expansion, gives
\begin{multline*}
  A_1 = -\frac{i}{\hbar}\qty(\frac{m}{2\pi i\hbar{}})^{\nicefrac{5}{2}} \frac{1}{T^{\nicefrac{3}{2}}} \qty(\frac{1}{r_a} + \frac{1}{r_b}) \exp[\frac{im}{2\hbar T}(r_a + r_b)^2] \\
  \times\int^\infty_{-\infty}  V(\vec{r}) \exp[\frac{im}{\hbar T}(r_a + r_b)(\vec{n}_a + \vec{n}_b)\vdot{\vec{r}}] \dd[3]{r}
\end{multline*}

\subsection{Transfers}
By assuming that most of the time the particle is a free particle, we can assume its speed is \[ u = \frac{r_a + r_b}{T} \] which then makes
\begin{align*}
  p &= m\frac{r_a + r_b}{T} & E &= \frac{1}{2} m \qty(\frac{r_a + r_b}{T})^2
\end{align*}
and the vector momentum is thus \[ \vec{p}_i = -p \vec{n}_i \qq{for} i \in \{a, b\}. \]
The momentum transfer (i.e.~the change in momentum due to the interactions) is
\begin{align*}
  \vec{p}_a - \vec{p}_b &= -m\frac{r_a + r_b}{T} (\vec{n}_a + \vec{n}_b)\\
  &\equiv \hbar \vec{k}
\end{align*}

This gives us \emph{YET ANOTHER} rewriting,
\begin{align*}
  A_1 &= -\frac{i}{\hbar}\qty(\frac{m}{2\pi i\hbar{}})^{\nicefrac{5}{2}} \frac{1}{T^{\nicefrac{3}{2}}} \qty(\frac{r_a + r_b}{r_a r_b}) \exp[\frac{i}{\hbar}ET] \int^\infty_{-\infty}  V(\vec{r}) \exp[i\vec{k}\vdot{\vec{r}}] \dd[3]{r}
\end{align*}
and we define (for the sake of easier writing later)
\begin{align*}
  \tilde{V}(\vec{k}) &= \int^\infty_{-\infty}  V(\vec{r}) \exp[i\vec{k}\vdot{\vec{r}}] \dd[3]{r}
\end{align*}
which also emphasises that this is just the Fourier transform of the potential --- wave-particle duality is becoming a thing.

\subsection{Transition Probability}
The transition probability (from the starting position to a specific endpoint)
\begin{equation*}
  P(a \to b) = \abs{A_1}^2 = \frac{1}{\hbar^2} \qty(\frac{m}{2\pi\hbar})^5 \frac{1}{T^3} \qty(\frac{r_a + r_b}{r_a r_b})^2 \abs{\tilde{V}(\vec{k})}^2.
\end{equation*}

For a no-scatter situation --- where the particle doesn't interact with the potential --- we use the free particle transition amplitudes
\begin{align*}
  A_0 = \braket{\vec{r}_c, T}{\vec{r}_a, 0}_0 &= \qty(\frac{m}{2\pi i \hbar T})^{\nicefrac{3}{2}} \exp[i\frac{m}{2\hbar T}(\vec{r}_c - \vec{r}_a)^2]\\
  \implies P(a \to c) = \abs{A_0}^2 &= \qty(\frac{m}{2\pi i \hbar T})^{3}.
\end{align*}

Putting these together to get the transition ratio gives
\begin{align*}
  \frac{P(a \to b)}{P(a \to c)} &= \qty(\frac{m}{2\pi\hbar^2})^2 \qty(\frac{r_a + r_b}{r_a r_b})^2 \abs{\tilde{V}(\vec{k})}^2.
\end{align*}

\subsection{Cross Section}
The differential cross section (in the Born approximation) is the rate of scattering against an infinitesimal area \( \dd{\sigma}\) into an infinitesimal solid angle \( \dd{\Omega} \).
This gives us the statement that
\begin{align*}
  \frac{P(a \to b)}{P(a \to c)} &= \qty(\frac{r_a + r_b}{r_a r_b})^2 \dv{\sigma}{\Omega}\\
  \implies\qquad \dv{\sigma}{\Omega} &= \qty(\frac{m}{2\pi\hbar^2})^2 \abs{\tilde{V}(\vec{k})}^2.
\end{align*}

If \(V(\vec{r}) \) only depends on r (i.e.~is a central potential), then
\begin{align*}
  \tilde{V}(\vec{k}) &= \int_{-\infty}^{\infty} V(r) \exp i \vec{k}\vdot\vec{r} \\
  &= \frac{4\pi}{k}\int_0^\infty r\,V(r) \,\sin(kr)\dd{r}.
\end{align*}
In order to get this to converge, we will sometimes have to modify this to
\begin{align*}
  \tilde{V}(\vec{k}) &= \lim_{\mu \to 0} \int_0^\infty r\,V(r) \,\sin(kr) e^{-\mu r}\dd{r}
\end{align*}
where \( k = \frac{1}{\hbar} \abs{\vec{p}_a - \vec{p}_b} = \frac{2p}{\hbar}\sin(\nicefrac{\theta}{2}) \).

\begin{example}
  For any \(V(r) \propto -\frac{1}{r}\), we can use the forced-convergence trick to find that
  \begin{align*}
    \int_0^\infty r\,V(r)\,\sin(kr) \dd{r} &\propto -\frac{1}{k}
  \end{align*}
  and hence
  \begin{align*}
    \dv{\sigma}{\Omega} &\propto \frac{1}{16E^2}\frac{1}{\sin^4(\nicefrac{\theta}{2})}
  \end{align*}
  which, in the case of the Coulomb potential, happens to coincide with the Rutherford cross section.
\end{example}

\section{Shooting Beams at Each Other}
Two particles with masses \(m_1\) and \(m_2 \) interact with each other with a potential from either (effectivley pairwise).
The system has a two-variable Lagrangian
\[ L = \frac{1}{2}m_1 \abs{\pdv{\vec{r}_1}{t}}^2 + \frac{1}{2}m_2 \abs{\pdv{\vec{r}_2}{t}}^2 - V(\vec{r}_1 - \vec{r}_2). \]
We shuffle this into the centre-of-mass frame, defining the postition of the centre of mass,
\begin{align*}
  \vec{R} &= \frac{m_1 \vec{r}_1 + m_2 \vec{r}_2}{M} & M &= m_1 + m_2 \intertext{and relative motion within the system}
  \vec{r} &= \vec{r}_1 - \vec{r}_2 & \mu &= \frac{ m_1 m_2 }{m_1 + m_2}
\end{align*}
to give us a new Lagrangian
\[ L = \frac{1}{2} M \abs{\pdv{\vec{R}}{t}}^2 + \frac{1}{2}\mu \abs{\pdv{\vec{r}}{t}}^2 - V(\vec{r}). \]
In the centre-of-mass frame, \(\vec{R} = 0\), and \(L = \frac{1}{2}\mu \abs{\pdv{\vec{r}}{t}}^2 - V(\vec{r})\) which is the same as for the fixed-target, so we can use the same result as earlier, but define a new function
\begin{align*}
  \dv{\sigma}{\Omega} &= \qty(\frac{\mu}{2\pi\hbar^2})^2 \abs{\tilde{V}(\vec{k})}^2\\
  &\equiv \abs{f(\theta, \phi)}^2
\end{align*}
which we call the scattering amplitude.

We use the Born-Oppenheimer approximation where we assume one particle much more massive than the other, and this recovers the fixed-particle result.

If we have two indistinguishable particles, then there are two indistinguishable outcomes of the collision, resulting in
\begin{align*}
  \dv{\sigma}{\Omega} &= \abs{f(\theta) + f(\pi - \theta)}^2.
\end{align*}
This allows us to distinguish between fermions and bosons, since the result will be different due to the spin statistics theorem (fermions are antisymmetric).

\section{Operator?}
We can develop perturbation with the time evolution operator.
We define a new picture, the Dirac picture, which is the same as the Heisenberg picture but is done with respect to \( S_0 \) instead of \(S\), such that
\begin{equation*}
  i\hbar \pdv{t} \hat{U}_0 = \hat{H}_0 \hat{U}_0
\end{equation*}
for a time dependent Hamiltonian, and
\begin{equation*}
  \hat{U}_0 = \exp[\frac{i}{\hbar} (t - t_0) \hat{H}_0].
\end{equation*}
This allows us to define a new set of Heisenberg operators, for example
\[ \hat{x}_0 (t) = \hat{U}_0^\dagger (t, t_0) \, \hat{x} \,  \hat{U}_0(t, t_0)\]

\section{Time Dependent Transition}
Going all of the way back to the statement for the perturbative transition amplitude series, we can put the whole thing into the operator formalism, such that
\begin{align*}
  \hat{U}(t_a, t_b) &= \hat{U}_0 (t_b, t_a) + \sum_{n=1}^\infty (-\frac{i}{\hbar})^n \int^{t_b}_{t_a} \dd{t_n} \cdots \int^{t_2}_{t_a} \dd{t_1}\\
  &\mspace{100mu}\times \hat{U}_0(t_b, t_n) V(\hat{x}, t_n) \hat{U}_0(t_n, t_{n-1}) \cdots V(\hat{x}, t_1) \hat{U}_0(t_1, t_a)\\
  &= \hat{U}_0 (t_b, t_a) -\frac{i}{\hbar} \int^{t_b}_{t_a} \dd{t_n} \hat{U}_0(t_b, t) V(\hat{x}, t) \hat{U}(t, t_{a})\\
\end{align*}

Now we remember that in the energy basis, \(\hat{H}_0\) is a diagonal operator, which is useful, since \[ \braket{m,t}{n^\prime, t^\prime} = e^{-\frac{i}{\hbar}(t - t^\prime)E_n} \delta_{mn}.\]

Using this for the transition gives
\begin{align*}
  \braket{b, t_b}{a, t_a} &= \mel{b}{\hat{U}(t_b, t_a)}{a}\\
  &= \mel{b}{\hat{U}_0(t_b, t_a)}{a} - \frac{i}{\hbar} \int_{t_a}^{t_b} \mel{b}{\hat{U}(t_b, t)V(\hat{x}, t)\hat{U}(t, t_a)}{a} \dd{t} + \cdots\\
  &= e^{-\frac{i}{\hbar}(t_b - t_a)E_a}\delta_{ab} - e^{-\frac{i}{\hbar}(E_bt_b - E_at_a)} \int_{t_a}^{t_b} \dd{t} e^{\frac{i}{\hbar}t (E_b - E_a)}V_{ba}(t)+ \cdots
\end{align*}
which corresponds to more and more interactions you add more potential interactions at different times (as number of terms increases).

Using this statement, we can see the first order interaction probability is
\begin{align*}
   P(a \to b) &= \frac{1}{\hbar^2} \abs{\tilde{V}_{ba}}^2\intertext{where}
   \tilde{V}_{ba}(t) &= \int_{t_a}^{t_b} \dd{t}V_{ba}(t)
\end{align*}
where we have defined, by convention, that
\begin{align*}
 V_{ab} &\equiv \mel{a}{V(\hat{x}, t)}{b} = \int_{-\infty}^\infty u_m^\ast(x) V(x, t) u_n(x).
\end{align*}

If the perturbation is time independent, we derive Fermi's golden rule for the transition rate, \[ R = \frac{2\pi}{\hbar}\abs{V_{ba}}^2 \rho(E_b) \] which is covered extensively elsewhere.

\chapter{Richard Feynman Sticks His Boot In}
We go back to classical mechanics for a bit, in order to motivate the point of the exercise.
A forced \emph{an}harmonic oscillator has an action \(S = S_0[x]_J\footnote{I'm not certain about this notation, but I know for sure and certain I don't like the one in the notes.} + S_1[x]\) with Lagrangian \[L = L_{\text{Driven SHO}} - \frac{\lambda}{4}x^4 \] and
\[ \qty(\pdv[2]{t} + \omega^2)x = \frac{J}{x} - \frac{\lambda}{m}x^3.\]

\section{GREEEEEEEN *waves fist*}
We have the homogeneous equation \[ \qty(\pdv[2]{t} + \omega^2) \bar{x}_0 = 0 \] with logical (Neumann) boundary conditions.
With this in mind, we build a Green's function \(\Delta(t, t^\prime)\) such that \[\qty(\pdv[2]{t} + \omega^2) \Delta(t, t^\prime) = -\delta(t - t^\prime) \] with boundary conditions \( \Delta(t_a, t^\prime) = \Delta(t_b, t^\prime) = 0 \).

We call \(\Delta\) the Feynman Propagator, and solve it as
\begin{multline*}
  \Delta(t, t^\prime) = \frac{1}{\omega \sin\omega T}\big[\theta(t - t^\prime)\sin\omega(t_b - t)\,\sin\omega(t^\prime-t_a) \\+ \theta(t^\prime - t)\sin\omega(t_b - t^\prime)\,\sin\omega(t-t_a)\big]
\end{multline*}
which means that (as usual) the full solution for the classical solution is
\begin{align*}
  \bar{x}(t) = \bar{x}_0(t) + \frac{1}{m}\int_{t_a}^{t_b} \Delta(t, t^\prime) \qty(-J(t^\prime) + \lambda \bar{x}(t^\prime)^3) \dd{t^\prime}.
\end{align*}
You may notice the similarities between this and the previously derived quantum perturbation theories.
This is intentional.
The equation maps out ``trees of four-point interactions'' somehow.

\section{And again, it all becomes wibbly and weird}
We take our forced harmonic oscillator and sort it out so that
\begin{align*}
  S_0[x]_J &= \int_{t_a}^{t_a} \qty(\frac{m}{2} (\dot{x}^2 - \omega^2 x^2) + Jx)\dd{t} & S_1[x] &= -\frac{\lambda}{4} \int_{t_a}^{t_b} x^4 \dd{\tau}.
\end{align*}
This makes our transition (using functional derivatives)
\begin{align*}
  \mel{x_b, t_b}{T(\hat{x}(t_1)\cdots\hat{x}(t_m))}{x_a, t_a}_J &= \int x(t_1)\cdots x(t_m)\,e^{\frac{i}{\hbar}S[x]_J} \DD{x}\\
  &= \int x(t_1)\cdots x(t_m)\,e^{\frac{i}{\hbar}S[x]_0 + \int_{t_a}^{t_b} J(t) x(t) \dd{t}} \DD{x}\\
  &= \qty(\frac{\hbar}{i})^m \frac{\delta^m}{\delta J(t_1)\,\cdots\,\delta J(t_m)} \braket{x_b, t_b}{x_a, t_a}_J.
\end{align*}

\subsection{Wibblier and Weirder}
We expand \(\exp[\frac{i}{\hbar} S_1[x]] = \exp[-\frac{i}{\hbar} \frac{ \lambda }{4}\int_{t_a}^{t_b} \dd{\tau} x(\tau)^4]\) to get a perturbation expansion for the anharmonic oscillator
\begin{align*}
  \braket{x_b, t_b}{x_a, t_a}_J &= \int_{x_a}^{x_b} \DD{x} \exp[\frac{i}{\hbar}(S_0[x]_J + S_1[x])]\\
  &= \sum_{n=0}^{\infty} \frac{1}{n!} \qty(-\frac{i\lambda}{4\hbar})^n \int_{t_a}^{t_b} \dd{\tau_1} \cdots \int_{t_a}^{t_b} \dd{\tau_n}  \int_{x_a}^{x_b} \DD{x} x(\tau_1)^4 \cdots x(\tau_2)^4 e^{\frac{i}{\hbar} S_0[x]_J} \\
  &= \sum_{n=0}^{\infty} \frac{1}{n!} \qty(-\frac{i\lambda}{4\hbar})^n \qty[\prod_{k=1}^n \int_{t_a}^{t_b} \dd{\tau_k} \qty{\frac{\hbar}{i}\fdv[k]{J(\tau_k)}}^4] \int_{x_a}^{x_b} \DD{x}  e^{\frac{i}{\hbar} S[x]_J}\\
  &= \exp[-\frac{i\lambda}{4\hbar} \int_{t_a}^{t_b} \dd{\tau} \qty(\frac{\hbar}{i}\fdv{J(\tau)})^4] F_\omega(T) \exp[\frac{i}{\hbar} S[\bar{x}]_J].
\end{align*}

If we make the assumptions required to use the Feynman Propagator, we can find a general expression for the correlation functions
\begin{multline*}
  \frac{1}{F_\omega(T)} \mel{0, t_b}{T(\hat{x}(t_1)\cdots\hat{x}(t_n))}{0, t_a} \\= \qty(\frac{\hbar}{i})^m \frac{\delta^m}{\delta J(t_1)\,\cdots\,\delta J(t_m)} \exp[-\frac{i\lambda}{4\hbar} \int_{t_a}^{t_b} \dd{\tau} \qty(\frac{\hbar}{i}\fdv{J(\tau)})^4]\\
  \times \exp[\frac{i}{\hbar} \int_{t_a}^{t_b}\int_{t_a}^{t_b}J(t)\Delta(t, t^\prime)J(t^\prime)\dd{t^\prime}\dd{t}].
\end{multline*}

\section{Special Results}
We have Wick's Theorem for the \( n\)-point correlation function (time ordered product of \(n\) position operators) which is that for a pure forced simple harmonic oscillator,
\begin{align*}
  \frac{1}{F_\omega (T)} \mel{0, t_b}{T(\hat{x}(t_1) \cdots \hat{x}(t_n))}{0, t_a}_{J=0} &= \qty(\frac{i\hbar}{m})^{\nicefrac{m}{2}} \sum_{\text{all pairings}}\Delta(t_{i_1}, t_{i_2})\cdots \Delta(t_{i_{m-1}}, t_{i_m}).
\end{align*}

By considering the first order of the first exponential (the anharmonic term), we can derive the first of the Feynman rules.
It's worth noting that in the integrals, we only care about terms which keep a \( J(\tau) \) after functional differentiation, so the entire thing reduces to \[\frac{1}{F_\omega(T)} \mel{0, t_b}{T(\hat{x}(t_1) \hat{x}(t_2) \hat{x}(t_3) \hat{x}(t_4)) }{0, t_a}_{J=0} = -i \frac{3!\, \lambda}{\hbar}\int_{t_a}^{t_b} \dd{t} \prod_{i=1}^{4}(\frac{i\hbar}{m}\Delta(t, t_i))\]
which represents an interaction (or a crossing point on a Feynman diagram) since they all four propagators go terminate or start at \(t\).

\section{Doing It Over All Time}
None of these integrals converge for all times, since \(\Delta(t, t^\prime)\) doesn't have a limit and oscillates \emph{wildly}.
We have three choices in this case, which are all roughly equivalent:
\begin{enumerate}
  \item Take \( t = -i\tau \) as is the case in Statistical Mechanics\footnote{which is one of the examples which were skipped earlier}
  \item Rotate \(t\) in the complex plane, such that \( t \mapsto t - i\epsilon \)
  \item Dampen the path integral by mapping \[ S^\prime = S + i\epsilon \int_{-\infty}^\infty x^2 \dd{t} \] which is called the `\(i\epsilon\) prescription'. In this prescription we have \( (\omega^\prime)^2 = \omega^2 - i \epsilon \).
\end{enumerate}

For the propagator over all time, we define a new, cooler version which does everything the old one does and converges.
With all of the rewriting we've done, this feels like an anticlimax, but we find that
\begin{align*}
  \lim_{t_b\to \infty}\lim_{t_a\to -\infty}\Delta(t, t^\prime) &= \frac{1}{2i\omega}\qty(e^{-i\omega(t-t^\prime)} \theta(t-t^\prime) + e^{i\omega(t-t^\prime)} \theta(t^\prime - t))\\
  &= \Delta_F(t- t^\prime).
\end{align*}

\chapter{Lots of Little Things Moving Around Really Rather Quickly\ldots{} Maybe}
\section{The Klein-Gordon equation}
In special relativity\footnote{I've skipped the four-vector stuff}, we know that the energy of a particle is given by
\[ E^2 = p^2 c^2 + m^2 c^4 \]
which mean that if we use the standard substitutions (\(E \mapsto i\hbar \pdv{t}\) and \(\vec{p}\mapsto i\hbar\grad\)) which allows us to derive
\begin{align*}
  m^2 c^4 \phi(\vec{r}, t) - \hbar^2 c^2 \laplacian \phi(\vec{r}, t) &= -\hbar^2 \pdv[2]{t}\phi(\vec{r}, t)\\
  \implies \qquad (\hbar^2\partial^2 + m^2 c^2)\phi(\vec{r}, t) &= 0
\end{align*}
which is the Klein-Gordon equation for the free particle. This is solvable with a 4-vector plane wave expression \[\phi(x) = e^{-ik\vdot x} \equiv e^{-i\frac{p}{\hbar}\vdot x}.\]

This has a couple of problems.
Firstly, our basic idea that \(\hbar^2( \omega^2 - c^2 \vec{k}^2 ) = m^2 c^4\) implies that there are negative energy particles; we'll deal with that later.

Secondly, we can derive the probability current for the Schr\"odinger equation \[0 = \psi^\ast (\text{SE})  - \psi (\text{SE})^\ast \] which expands to
\[\pdv{\rho}{t} + \div\vec{j} = 0\]
where we find that
\begin{align*}
  \rho &= \psi^\ast \psi & \vec{j} &= -\frac{i\hbar}{2m}\qty(\psi^\ast \grad\psi - \psi \grad\psi^\ast)
\end{align*}
and for the Klein Gordon equation,
\begin{align*}
  \rho &= \frac{i\hbar}{2mc^2}\qty(\phi^\ast \pdv{t}\phi - \phi \pdv{t}\phi^\ast) & \vec{j} &= -\frac{i\hbar}{2m}\qty(\phi^\ast \grad\phi - \phi \grad\phi^\ast)
\end{align*}
which has the issue that \(\rho\), the probability density, is not positive definite, so we can't interpret it \emph{as} a probability density.

\section{Yet another thing named after Dirac}
To combat the two negative things, we try to find a different relativistic wave equation which is linear in \( \pdv{t}\).
We start with the Schr\"odinger equation again, in the form
\[ i\hbar\pdv{t}\psi(x) = \hat{H}\psi(x) \]
which we make a reasonably radical ansatz for, and get
\begin{align*}
  i\hbar\pdv{t}\psi(\vec{r}, t) &= \qty[c \vec{\alpha}\vdot \vec{\hat{p}} + \beta m c^2]\psi(\vec{r}, t) \\
  &= \hat{H}\psi(\vec{r}, t)
\end{align*}
where \(\vec{\alpha}\) and \(\beta\) are independent of all derivatives, so commute with all terms except each other.
We also require that
\[ \hat{H}^2 \psi = (c^2\vec{\hat{p}}^2 + m^2 c^4)\psi\]
which after rearranging to compare with the previous ansatz gives us the anticommutator relations
\begin{align*}
  \anticommutator{\alpha^i}{\alpha^j} &= \delta^{ij} & \anticommutator{\alpha^i}{\beta} &= 0
\end{align*}
and note that \(\beta^2 = 1\).
From these requirements, we guess that \(\alpha\) and \(\beta\) must actually be hermitian \(n \times n\) matrices. We also note that they must be traceless, and have eigenvalues of \(\pm 1\).
We use the smallest representation of \(\alpha\) and \(\beta\) which is in four dimensional matrices,
\begin{align*}
  \beta &= \mqty(I&0\\0&-I) & \vec{\alpha}^i &= \mqty(0 & \sigma^i \\ \sigma^i & 0)
\end{align*}
where \(\sigma^i\) are the Pauli matrices,
\begin{align*}
  \sigma^1 &= \mqty(\pmat{1}) & \sigma^2 &= \mqty(\pmat{2}) & \sigma^3 &= \mqty(\pmat{3})
\end{align*}
 which are unitary, hermitian and satisfy the anti-commutation relation required.

 The Hamiltonian is now a 4D matrix, so we must make a wavefunction to match. We give it a \emph{spinor}\footnote{pronounced ``spinner''} which is simply a four component column matrix (not a vector) in the format
 \begin{align*}
   \psi(\vec{r}, t) &= \mqty(\psi_1(\vec{r},t)\\ \psi_2(\vec{r},t)\\ \psi_3(\vec{r},t)\\ \psi_4(\vec{r},t)).
 \end{align*}

\subsection{Continuity}
We go back to the continuity equation, and doing the same thing with the Dirac equation as we did with the others, we find that
\begin{align*}
  \rho &= \psi^\dagger \psi & \vec{j} &= \psi^\dagger \vec{\alpha} \psi & j^\mu &= \mqty(\rho, \vec{j})
\end{align*}
which gives the covariant form of the continuity equation as \[\partial_\mu j^\mu = 0. \]
This is now perfectly well defined for the probability density (huzzah!).

\section{Just Leave It Alone. It's The Only Thing To Do}
Free particle \(\implies \) plane wave, as always.
However, we need a spinor version, so we invent a new spinor, \(w(p)\) which gives us
\begin{align*}
  \psi(\vec{r}, t) &= \exp(-\frac{i}{\hbar}p\vdot x)\,w(p)\\
  &= \exp(-\frac{i}{\hbar}(cp^0 t - \vec{p}\vdot\vec{r}))\,w(p).
\end{align*}
This leads us to a slightly different form of the Dirac equation,
\begin{align}
  p^0 w (p) &= (\vec{\alpha} \vdot \vec{p} + \beta m c)\, w(p)\label{eqn:pspace}
\end{align}
 called the \(p\)-space Dirac equation.

 This can be solved elegantly using a slightly funky formalism where
 \[ w(p) = \mqty(\phi(p)\\\chi(p))\] which is a four-component spinor in terms of two two-component spinors.
 This makes \autoref{eqn:pspace}
 \begin{align*}
   p^0 \mqty(\phi\\\chi) &= \mqty(mc & \vec{\sigma}\vdot \vec{p} \\ \vec{\sigma}\vdot \vec{p} & -mc ) \mqty(\phi\\\chi)
 \end{align*}
 which shows this off as an eigenvalue equation, which is nice, because we know how to solve them.
 Whatever your method of choice, you can find that
 \begin{align*}
   \chi &= \frac{\vec{\sigma}\vdot\vec{p}}{p^0 + mc}\phi\\
   p_0 \phi &= \qty(mc + \frac{(\vec{\sigma}\vdot \vec{p})^2}{p^0 + mc})\phi
 \end{align*}
which apparently shows that
\[(p^0)^2 = m^2c^2 + \vec{p}^2\]
but I'm unconvinced.

\subsection{Can a particle have Feng Shui?}
As usual, we identify \(p^0 = \nicefrac{E}{c}\) with the energy of the particle.
We thus write the positive energy solutions as
\begin{align*}
  w^{(i)}(p) &= \mqty(\phi^{(i)}\\\frac{c \vec{\sigma}\vdot\vec{p} }{E + mc^2}\phi^{(i)}) & \phi^{(1)} &= \mqty(1\\0) & \phi^{(2)} &= \mqty(0\\1)&  i \in { 1, 2 }
\end{align*}
where we have conventionally chosen the spinors (which have to be linearly independent, and preferably orthogonal).

This gives us that
\begin{align*}
  w^{(1)}(p) &= \begin{pmatrix}
    1 \\ 0\\
    \frac{cp^3}{E+mc^2} \\ \frac{c(p^1 + i p^2)}{E+mc^2}
  \end{pmatrix} &
  w^{(2)}(p) &= \begin{pmatrix}
    0\\ 1\\
    \frac{c(p^1 + i p^2)}{E+mc^2} \\ \frac{-cp^3}{E+mc^2}
  \end{pmatrix}
\end{align*}
where it's worth noticing that \( \vec{\sigma}\vdot \vec{p} \) is not a scalar, since \(\vec{\sigma}\) is a horrible abuse of notation for the three Pauli matrices.

\subsection{Negative Energy}
The negative energy solutions (\(p^0 = - \nicefrac{E}{C}\)) are written in terms of \(\chi\), and conventionally we find that
\begin{align*}
  w^{(i)}(-p) &= \mqty(\frac{c \vec{\sigma}\vdot\vec{p} }{E + mc^2}\chi^{(i-2)}\\\chi^{(i-2)}) & \chi^{(1)} &= \mqty(0\\1) & \chi^{(2)} &= \mqty(1\\0) &  i \in { 3, 4 }.
\end{align*}
We choose the negative energy states to have negative momentum because of the Dirac Sea interpretation of particle physics in a faintly vector like object which can be iterated over.

This gives us that
\begin{align*}
  w^{(3)}(-p) &= \begin{pmatrix}
    \frac{c(p^1 + i p^2)}{E+mc^2} \\ \frac{-cp^3}{E+mc^2}\\
    0\\ 1
  \end{pmatrix} &
  w^{(4)}(-p) &= \begin{pmatrix}
    \frac{cp^3}{E+mc^2} \\ \frac{c(p^1 + i p^2)}{E+mc^2}\\
    1 \\ 0
  \end{pmatrix}
\end{align*}
which represent the negative energy states of a spin-\(\nicefrac{1}{2}\) particle.

\section{Let's stop it all moving around, just to keep track}
When \(\vec{p} = 0\), the \(w\) spinors become orthonormal, and we conventionally write them as
\begin{align*}
  w^{(1)} &= \mqty(1\\0\\0\\0) &
  w^{(2)} &= \mqty(0\\1\\0\\0) &
  w^{(3)} &= \mqty(0\\0\\0\\1) &
  w^{(4)} &= \mqty(0\\0\\1\\0)
\end{align*}
and the positive-energy solutions become
\begin{align*}
  \psi^{(i)} &= e^{-i\frac{mc^2}{\hbar}t}\, w^{(i)}
\end{align*}
which means that they are degenerate, and there must be some other operator in the good quantum numbers for the rest frame.

We call this new operator \[\hat{\Sigma}^3 = \smqty(\dmat{1,-1,1,-1}) \] of which the four \(w\) spinors are all eigenvectors\footnote{Eigenspinors? What \emph{is} a spinor? How is it special? This entire section has some very questionable notation.} with values \(\pm 1\).
That this is in fact part of a wider definition that
\begin{align}
  \hat{\Sigma}^i &\equiv \mqty(\dmat{\sigma^i,\sigma^i})
\end{align}
is what leads us to believe that these are spin-\(\nicefrac{1}{2}\) particles.
This comes from us defining \(\hat{s}^i = \nicefrac{1}{2} \hbar \hat{\Sigma}^i \) which is exactly the spin operator.
This only commutes in the rest frame, as does the angular momentum \(\hat{\vec{L}} = \hat{\vec{r}}\times \hat{\vec{p}} \), so neither of these are conserved, or good quantum numbers.
However, the total angular momentum \( \hat{\vec{J}} \) commutes with the Dirac Hamiltonian in all frames.

\subsection{Helicity}
We can define a different operator which also commutes with the Hamiltonian, the Helicity operator
\begin{align*}
  \hat{h}(\vec{p}) &= \mqty(\frac{\vec{\sigma}\vdot \vec{p}}{\abs{\vec{p}}} & 0 \\ 0 & \frac{\vec{\sigma}\vdot \vec{p}}{\abs{\vec{p}}}) \\
  &\propto \hat{\vec{\Sigma}} \vdot \frac{\vec{p}}{\abs{\vec{p}}}
\end{align*}
which has eigenvalues \(\pm 1 \) --- this is the particle's spin along the direction of motion.

\section{The Dirac Sea}
Dirac proposed that all negative energy states are filled, with each energy level holding two electrons --- a spin up and a spin down.
The Pauli exclusion principle means that positive energy electrons can't just transition into a negative energy state without that state already being empty.
The negative energy states are unobservable, and are completely full in a complete vaccuum.

This is also a model for pair production.
The negative energy electron takes \(2(E+mc^2)\) in energy to jump up into its relative positive energy state, leaving behind a `hole' in the sea, which in this model is the positron (discovered after the model).

Quantum field theory fixes the issue that this model doesn't work for bosons.

\section{Covariant Dirac Equation}
The Dirac Equation can be rewritten from
\begin{align*}
  i\hbar \pdv{t} \psi(\vec{r}, t) &= \qty(-i\hbar c \vec{\alpha}\vdot\grad{} + \beta{} mc^2)\,\psi(\vec{r}, t)\intertext{to}
  0 &= \qty[i\hbar\qty(\gamma^0 \pdv{x^0} + \gamma^i \pdv{x^i}) - mc] \psi(\vec{r}, t)
\end{align*}
where we have defined
\begin{align*}
  \gamma &\equiv (\beta, \beta\vec{\alpha}).
\end{align*}

We also now define the Feynman Slash notation, \[\slashed{a} \equiv \gamma^\mu a_\mu \] which is the contraction with the \(\gamma\) spinor.
This allows us to reduce this expression to \[ (i \hbar \slashed{\partial}- mc) \psi(x) = 0\] which is really very simple.

The plane wave solutions reduce to \[ \psi(x) = e^{-\nicefrac{i}{\hbar}p\vdot x} u(p, s) \] where this satisfies
\[ (\slashed{p} - mc)\, u(p, s) = 0 \]
and we've made new, less awkward to write, positive energy spinors \[ u(p, s) \equiv w^{(s)}(p).\]

The negative energy states are the same, with \[ \psi(x) = e^{\nicefrac{i}{\hbar}p\vdot x} v(p, s) \] where this satisfies
\[ (\slashed{p} + mc)\, v(p, s) = 0 \] and we've defined \[v(p, s) \equiv w^{(s + 2)}(-p).\]

The \(\gamma\) matrices satisfy the same anticommutation relations as we required earlier for \(\alpha\) and \(\beta\),
\[ \anticommutator{\gamma^\mu}{\gamma^\nu} = 2 \eta^{\mu\nu} \]
which is the requirements for a `Clifford Algebra'.

We can see from
\begin{align*}
  \gamma^i &= \mqty(0 & \sigma^i\\ - \sigma^i & 0) & \gamma^0 &= \mqty(I&0\\0&-I)
\end{align*} that \(\gamma^0\) is hermitian and \(\gamma^i \) is anti-hermitian, which we need.

We also define
\begin{align*}
  \bar{\psi} &= \psi^\dagger \gamma^0
\end{align*}
which new notation is called the covariant formulation.

We finally get to \[ j^\mu = \bar{\psi} \gamma^\mu \psi\] which conveniently transforms as a 4-vector.

\begin{note}
And from here, it's the Pauli equation for electromagnetic interactions, which is just more Spinors. It's all a bit of a nightmare, but isn't too bad to work through.
\end{note}

\include{includes/appendices}
\printbibliography
\end{document}


\lab{Profiling and Optimizing Python Code}{Profiling}
\objective{Identify which portions of the code are most time consuming using a
profiler. Optimize Python code using good coding practices and just-in-time compilation with Numba.}
\label{lab:ProfilingCode}

The best code goes through multiple drafts.
In a first draft, you should focus on writing code that does what it is supposed
to and is easy to read. After writing a first draft, you may find that your code
does not run as quickly as you need it to. Then it is time to \emph{optimize}
the most time consuming parts of your code so that they run as quickly as possible.

In this lab we will optimize the function \li{qr1()} that computes the QR
decomposition of a matrix via the modified Gram-Schmidt algorithm
(see Lab \ref{lab:QRdecomp}).

\begin{lstlisting}
import numpy as np
from scipy import linalg as la

def qr1(A):
    ncols = A.shape[1]
    Q = A.copy()
    R = np.zeros((ncols, ncols))
    for i in range(ncols):
        R[i, i] = la.norm(Q[:, i])
        Q[:, i] = Q[:, i]/la.norm(Q[:, i])
        for j in range(i+1, ncols):
            R[i, j] = Q[:, j].dot(Q[:, i])
            Q[:,j] = Q[:,j]-Q[:, j].dot(Q[:, i])*Q[:,i]
    return Q, R
\end{lstlisting}

\section*{What to Optimize}
Python provides a \emph{profiler} that can identify where code spends most of
its runtime. The output of the profiler will tell you where to begin your
optimization efforts.

In IPython\footnote{If you are not using IPython, you will need to use the
\li{cProfile} module documented here: \url{https://docs.python.org/2/library/profile.html}.},
you can profile a function with \li{\%prun}.
Here we profile \li{qr1()} on a random $300 \times 300$ array.

\begin{lstlisting}
In [1]: A = np.random.rand(300, 300)
In [2]: %prun qr1(A)
\end{lstlisting}

On the author's computer, we get the following output.

{\scriptsize
\begin{verbatim}
         97206 function calls in 1.343 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.998    0.998    1.342    1.342 profiling_hw.py:4(qr1)
    89700    0.319    0.000    0.319    0.000 {method 'dot' of 'numpy.ndarray' objects}
      600    0.006    0.000    0.012    0.000 function_base.py:526(asarray_chkfinite)
      600    0.006    0.000    0.009    0.000 linalg.py:1840(norm)
     1200    0.005    0.000    0.005    0.000 {method 'any' of 'numpy.ndarray' objects}
      600    0.002    0.000    0.002    0.000 {method 'reduce' of 'numpy.ufunc' objects}
     1200    0.001    0.000    0.001    0.000 {numpy.core.multiarray.array}
     1200    0.001    0.000    0.002    0.000 numeric.py:167(asarray)
        1    0.001    0.001    0.001    0.001 {method 'copy' of 'numpy.ndarray' objects}
      600    0.001    0.000    0.022    0.000 misc.py:7(norm)
      301    0.001    0.000    0.001    0.000 {range}
        1    0.001    0.001    0.001    0.001 {numpy.core.multiarray.zeros}
      600    0.001    0.000    0.001    0.000 {method 'ravel' of 'numpy.ndarray' objects}
      600    0.000    0.000    0.000    0.000 {method 'conj' of 'numpy.ndarray' objects}
        1    0.000    0.000    1.343    1.343 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
\end{verbatim}
}

The first line of the output tells us that executing \li{qr1(A)} results in
almost 100,000 function calls. Then we see a table listing these functions along
with data telling us how much time each takes. Here, \li{ncalls} is the number
of calls to the function, \li{tottime} is the total time spent in the function,
and \li{cumtime} is the amount of time spent in the function including calls
to other functions.

For example, the first line of the table is the function \li{qr1(A)} itself.
This function was called once, it took 1.342s to run, and 0.344s of that was
spent in calls to other functions. Of that 0.344s, there were 0.319s spent on
89,700 calls to \li{np.dot()}.

With this output, we see that most time is spent in multiplying matrices.
Since we cannot write a faster method to do this multiplication, we may want to
try to reduce the number of matrix multiplications we perform.

\section*{How to Optimize}
Once you have identified those parts of your code that take the most time,
how do you make them run faster?
Here are some of the techniques we will address in this lab:

\begin{itemize}
\item Avoid recomputing values
\item Avoid nested loops
\item Use existing functions instead of writing your own
\item Use generators when possible
\item Avoid excessive function calls
\item Write Pythonic code
\item Compiling Using Numba
\item Use a more efficient algorithm
\end{itemize}

You should always use the profiling and timing functions to help you decide
when an optimization is actually useful.

\begin{problem}
In this lab, we will perform many comparisons between the runtimes of various
functions. To help with these comparisions, implement the following function:

\begin{lstlisting}
def compare_timings(f, g, *args):
    """Compares the timings of 'f' and 'g' with arguments '*args'.

    Inputs:
        f (callable): first function to compare.
        g (callable): second function to compare.
        *args (any type): arguments to use when callings functions
            'f' and 'g'
    Returns:
        comparison (string): The comparison of the runtimes of functions
            'f' and 'g' in the following format :
                Timing for <f>: <time>
                Timing for <g>: <time>
            where the values inside <> vary depending on the inputs)
\end{lstlisting}
Hint: You can gain access to the name of many functions by using its
\li{func_name} method. However, this method does not exist for all functions
we will be interested in timing. Therefore, even though it is not as clean, use
\li{str(f)} to print a string representation of f.
\end{problem}

\subsection*{Avoid Recomputing Values}
In our function \li{qr1()}, we can avoid recomputing \li{R[i,i]} in the outer
loop and \li{R[i,j]} in the inner loop.
The rewritten function is as follows:
\begin{lstlisting}
def qr2(A):
    ncols = A.shape[1]
    Q = A.copy()
    R = np.zeros((ncols, ncols))
    for i in range(ncols):
        R[i, i] = la.norm(Q[:, i])
        Q[:, i] = Q[:, i]/R[i, i] # this line changed
        for j in range(i+1, ncols):
            R[i, j] = Q[:, j].dot(Q[:, i])
            Q[:,j] = Q[:,j]-R[i, j]*Q[:,i] # this line changed
    return Q, R
\end{lstlisting}

Profiling \li{qr2()} on a $300 \times 300$ matrix produces the following output.

{\scriptsize
\begin{verbatim}
         48756 function calls in 1.047 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.863    0.863    1.047    1.047 profiling_hw.py:16(qr2)
    44850    0.171    0.000    0.171    0.000 {method 'dot' of 'numpy.ndarray' objects}
      300    0.003    0.000    0.006    0.000 function_base.py:526(asarray_chkfinite)
      300    0.003    0.000    0.005    0.000 linalg.py:1840(norm)
      600    0.002    0.000    0.002    0.000 {method 'any' of 'numpy.ndarray' objects}
      300    0.001    0.000    0.001    0.000 {method 'reduce' of 'numpy.ufunc' objects}
      301    0.001    0.000    0.001    0.000 {range}
      600    0.001    0.000    0.001    0.000 {numpy.core.multiarray.array}
      600    0.001    0.000    0.001    0.000 numeric.py:167(asarray)
      300    0.000    0.000    0.012    0.000 misc.py:7(norm)
        1    0.000    0.000    0.000    0.000 {method 'copy' of 'numpy.ndarray' objects}
      300    0.000    0.000    0.000    0.000 {method 'ravel' of 'numpy.ndarray' objects}
        1    0.000    0.000    1.047    1.047 <string>:1(<module>)
      300    0.000    0.000    0.000    0.000 {method 'conj' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 {numpy.core.multiarray.zeros}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
\end{verbatim}
}

Our optimization reduced almost every kind of function call by half, and reduced the total run time by 0.295s.

Some less obvious ways to eliminate excess computations include moving computations out of loops, not copying large data structures, and simplifying mathematical expressions.

\subsection*{Avoid Nested Loops}
For many algorithms, the temporal complexity of an algorithm is determined by its loops. Nested loops quickly increase the temporal complexity.
The best way to avoid nested loops is to use NumPy array operations instead of iterating through arrays.
If you must use nested loops, focus your optimization efforts on the innermost loop, which gets called the most times.

\begin{problem}
The code below is an inefficient implementation of the LU
algorithm. Write a function \li{LU_opt()} that is an optimized
version of \li{LU()}. Look for ways to avoid recomputing values
and avoid nested loops by using array slicing instead.
Print a comparison of the timing of the
original function and your optimized function using your
\li{compare_timings} function.
\begin{lstlisting}
def LU(A):
    """Returns the LU decomposition of a square matrix."""
    n = A.shape[0]
    U = np.array(np.copy(A), dtype=float)
    L = np.eye(n)
    for i in range(1,n):
        for j in range(i):
            L[i,j] = U[i,j]/U[j,j]
            for k in range(j,n):
                U[i,k] -= L[i,j] * U[j,k]
    return L,U
\end{lstlisting}
\end{problem}

\subsection*{Use Existing Functions Instead of Writing Your Own}
If there is an intuitive operation you would like to perform on an array, chances are that NumPy or another library already has a function that does it.
Python and NumPy functions have already been optimized, and are usually many times faster than the equivalent you might write.
We saw an example of this in Lab \ref{lab:NumPyArrays} where we compared NumPy array multiplication with our own matrix multiplication implemented in Python.

\begin{problem} \label{prob:add}
Without using any builtin functions, implement that following function:
\begin{lstlisting}
def mysum(X):
    """ Return the sum of the elements of X.
    Inputs:
        X (array) - a 1-D array
    """
\end{lstlisting}
Perform a comparision of \li{mysum} to Python's builtin \li{<<sum>>} function and
NumPy's \li{<<np.sum>>} using your \li{compare_timings} function.
\end{problem}

\subsection*{Use Generators When Possible}
When you are iterating through a list, you can often replace the list with a \emph{generator}.
Instead of storing the entire list in memory, a generator computes each item as it is needed.
For example, the code
\begin{lstlisting}
>>> for i in range(100):
...     print i
\end{lstlisting}
stores the numbers 0 to 99 in memory, looks up each one in turn, and prints it.
On the other hand, the code
\begin{lstlisting}
>>> for i in xrange(100):
...     print i
\end{lstlisting}
uses a generator instead of a list.
This code computes the first number in the specified range (which is 0), and prints it.
Then it computes the next number (which is 1) and prints that.

In our example, replacing each \li{range} with \li{xrange} does not speed up \li{qr2()} by a noticeable amount.

Though the example below is contrived, it demonstrates the benefits of using generators.
\begin{lstlisting}
# both these functions will return the first iterate of a loop of length 10^8.
def list_iter():
    for i in range(10**8):
        return i

def generator_iter():
    for i in xrange(10**8):
        return i

>>> compare_timings(list_iter,generator_iter)
Timing for <function list_iter at 0x7f3deb5a4488>: 1.93316888809
Timing for <function generator_iter at 0x7f3deb5a4500>: 1.19209289551e-05
\end{lstlisting}

It is also possible to write your own generators.
Say we have a function that returns an array. And say we want to iterate through this array later in our code. In situations like these, it is valuable to consider turning your function into a generator instead of returning the whole list. The benefits of this approach mirror the benefits of using \li{xrange} instead of {range}. The only thing that needs to be adjusted is to change the \li{return} statement to a \li{yield} statement. Here is a quick example:
\begin{lstlisting}
def return_squares(n):
    squares = []
    for i in xrange(1,n+1):
        squares.append(i**2)
    return squares

def yield_squares(n):
    for i in xrange(1,n+1):
        yield i**2
\end{lstlisting}
When yield is called, the single value is returned and all the local variables for the function are stored away until the next iteration. To iterate step-by-step through a generator, use the generator's \li{<<next>>} method.
\begin{lstlisting}
>>> squares = yield_squares(3)
>>> squarees.next()
1
>>> squares.next()
4
>>> squares.next()
9
\end{lstlisting}

We can also easily iterate through a generator using a for loop.
\begin{lstlisting}
>>> for s in squares:
...    print s,
...
1 4 9
\end{lstlisting}

\begin{problem}
Write a generator that yields the first \li{n} Fibonacci numbers.
\end{problem}

If you are interested in learning more about writing your own generators, see \url{https://docs.python.org/2/tutorial/classes.html#generators} and \url{https://wiki.python.org/moin/Generators}.

\subsection*{Avoid Excessive Function Calls}
Function calls take time.
Moreover, looking up methods associated with objects takes time.
Removing ``dots'' can significantly speed up execution time.

For example, we could rewrite our function to reduce the number of times we need to look up the function \li{la.norm()}.

\begin{lstlisting}
def qr2(A):
    norm = la.norm # this reduces the number of function look ups.
    ncols = A.shape[1]
    Q = A.copy()
    R = np.zeros((ncols, ncols))
    for i in range(ncols):
        R[i, i] = norm(Q[:, i])
        Q[:, i] = Q[:, i]/R[i, i]
        for j in range(i+1, ncols):
            R[i, j] = Q[:, j].dot(Q[:, i])
            Q[:,j] = Q[:,j]-R[i, j]*Q[:,i]
    return Q, R
\end{lstlisting}
Once again, an analysis with \li{\%prun} reveals that this optimization does not help significantly in this case.

\subsection*{Write Pythonic Code}
Several special features of Python allow you to write fast code easily.
First, list comprehensions are much faster than for loops. These are particularly useful when building lists inside a loop.
For example, replace
\begin{lstlisting}
>>> mylist = []
>>> for i in xrange(100):
...     mylist.append(math.sqrt(i))
\end{lstlisting}
with
\begin{lstlisting}
>>> mylist = [math.sqrt(i) for i in xrange(100)]
\end{lstlisting}
We can accomplish the same thing using the \li{map()} function, which is even faster.
\begin{lstlisting}
>>> mylist = map(math.sqrt, xrange(100))
\end{lstlisting}
The analog of a list comprehension also exists for generators, dictionaries, and sets.

Second, swap values with a single assignment.
\begin{lstlisting}
>>> a, b = 1, 2
>>> a, b = b, a
>>> print a, b
2 1
\end{lstlisting}

Third, many non-Boolean objects in Python have truth values.
For example, numbers are \li{False} when equal to zero and \li{True} otherwise.
Similarly, lists and strings are \li{False} when they are empty and \li{True} otherwise.
So when \li{a} is a number, instead of
\begin{lstlisting}
>>> if a != 0:
\end{lstlisting}
use
\begin{lstlisting}
>>> if a:
\end{lstlisting}

Lastly, it is more efficient to iterate through lists by iterating over the elements instead of iterating over the indices.
\begin{lstlisting}
# Bad
for i in xrange(len(my_list)):
    print my_list[i],

# Good
for x in my_list:
    print x,
\end{lstlisting}

However, there are situations where you will need to know the indices of the elements over which you are iterating. In these situations, use \li{enumerate}.

\begin{problem}
Using \li{\%prun}, find out which portions of the code below require the most runtime. Then, rewrite the function using some of the optimization techniques we have discussed thus far.
\begin{lstlisting}
def foo(n):
    my_list = []
    for i in range(n):
        num = np.random.randint(-9,9)
        my_list.append(num)
    evens = 0
    for j in range(n):
        if j%2 == 0:
            evens += my_list[j]
    return my_list, evens
\end{lstlisting}

Hint: If you are unsure where to begin optimizing, walk through the code line by line to determine what the code is accomplishing. Then, write your own function to perform the same task in a more efficient way using the optimization techniques we have discuessed.
\end{problem}

\subsection*{Compiling Using Numba}
Though it is much easier to write simple, readable code in Python, it is also much slower than compiled languages such as C. Compiled languages, in general, are much faster.
Numba is a tool that you can use to optimize your code. Numba uses \emph{just-in-time} (JIT) compilation.

This means that the code is compiled right before it is executed. We will discuss this process a bit later in this section.
The API for using Numba is incredibly simple. All one has to do is import Numba and add the \li{@jit} function decorator to your function. The following code would be a Numba equivalent to Problem \ref{prob:add}.
\begin{lstlisting}
from numba import jit
@jit
def numba_sum(A):
    total = 0
    for x in A:
        total += x
    return total
\end{lstlisting}

Though this code looks very simple, a lot is going on behind the scenes. Without getting into too many details, one of the reasons compiled languages like C are so much faster than Python is because they have explicitly defined datatypes. The main strategy used by Numba is to speed up the Python code by assigning datatypes to all the variables. Rather than requiring us to define the datatypes explicitly as we would need to in any compiled language, Numba attempts to \emph{infer} the correct datatypes based on the datatypes of the input.

In the code above, for example, say that our array \li{A} was an array of integers. Though we have not explicitly defined a datatype for the variable \li{total}, Numba will infer that the datatype for total should also be an integer.

Once all the datatypes have been inferred and assigned, the code is translated to machine code by the LLVM library. Numba will then cache this compiled version of our code. This means that we can bypass this whole inference and compilation process the next time we run our function.

\subsubsection*{More Control Within Numba}
Though the inference engine within Numba does a good job, it's not always perfect. There are times that Numba is unable to infer all the datatypes correctly.

If you add the keyword argument, \li{nopython=True} to the \li{jit} decorator, an error will be raised if Numba was unable to convert everything to explicit datatypes.

If your function is running slower than you would expect, you can find out what is going on under the hood by calling the \li{inspect_types()} method of the function. Using this, you can see if all the datatypes are being assigned as you would expect.

\begin{lstlisting}
# Due to the length of the output, we will leave it out of the lab text.
>>> numba_sum.inspect_types()
\end{lstlisting}

If you would like to have more control, you may specify datatypes explicity as demonstrated in the code below.

In this example, we will assume that the input will be doubles. Note that is necessary to import the desired datatype from the Numba module.

\begin{lstlisting}
from numba import double
# The values inside 'dict' will be specific to your function.
@jit(nopython=True, locals=dict(A=double[:], total=double))
def numba_sum(A):
    total = 0
    for i in xrange(len(A)):
        total += A[i]
    return total
\end{lstlisting}
Notice that the jit function decorator is the only thing that changed. Note also that this means that we will not be allowed to pass an array of integers to this function. If we had not specified datatypes, the inference engine would allow us to pass arrays of any numerical datatype. In the case that our function sees a datatype that it has not seen before, the inference and compilation process would have to be repeated. As before, the new version will also be cached.

\begin{problem}
% TODO Change this problem to something more compelling.
The code below defines a Python function which takes a matrix to the $n$th power.
\begin{lstlisting}
def pymatpow(X, power):
    """ Return X^{power}.

    Inputs:
        X (array) - A square 2-D NumPy array
        power (int) - The power to which we are taking the matrix X.
    Returns:
        prod (array) - X^{power}
    """
    prod = X.copy()
    temparr = np.empty_like(X[0])
    size = X.shape[0]
    for n in xrange(1, power):
        for i in xrange(size):
            for j in xrange(size):
                tot = 0.
                for k in xrange(size):
                    tot += prod[i,k] * X[k,j]
                temparr[j] = tot
            prod[i] = temparr
    return prod
\end{lstlisting}

\begin{enumerate}
\item Create a function \li{numba_matpow} that is the compiled version of \li{pymatpow} using Numba.
\item Write a function \li{numpy_matpow} that performs the same task as \li{pymatpow} but uses \li{np.dot()}. Compile this function using Numba.
\item Compare the speed of \li{pymatpow},  \li{numba_matpow} and the \li{numpy_matpow} function. Remember to time \li{numba_matpow} and \li{numpy_matpow} on the second pass so the compilation process is not part of your timing. Perform your comparisons using your \li{compare_timings} function.
\end{enumerate}

NumPy takes products of matrices by calling BLAS and LAPACK, which are heavily optimized linear algebra libraries written in C, assembly, and Fortran.
\end{problem}

\begin{warn}
NumPy's array methods are often faster than a Numba equivalent you could code yourself.
If you are unsure which method is fastest, time them.
\end{warn}

\subsection*{Use a More Efficient Algorithm}

The optimizations discussed thus far will speed up your code at most by a constant.
They will not change the complexity of your code.
In order to reduce the complexity (say from $O(n^2)$ to $O(n \log(n))$), you typically need to change your algorithm.
We will address the benefits of using more efficient algorithms in Problem \ref{prob:tridiag}.

A good algorithm written with a slow language (like Python) is faster than a bad algorithm written in a fast language (like C).
Hence, focus on writing fast algorithms with good Python code, and only Numba when and where it is necessary. In other words, Numba will not always save you from a poor algorithm design.

\begin{comment}
\begin{problem}
Optimize the following function using techniques described in this lab:
\begin{lstlisting}
# TODO: COME UP WITH SOME ALGORITHM TO DO HERE!!
\end{lstlisting}
It should also include a list of changes, the reasoning behind the changes, and the effect of the changes on runtime. On the author's computer, computing the LU-decomposition on a 1000x1000 matrix took over 2 and a half minutes. The optimized version took a little over a second.

Hint: The best way to approach this problem is to analyze what each piece of code is actually doing. Then, determine if there is a more efficient way to accomplish the same task. Specifically, look for ways to use array operations instead of for loops, ways to replace blocks of code with built-in Python functions, and ways to avoid recomputing values.
\end{problem}
\end{comment}

The correct choice of algorithm is more important than a fast implementation.
For example, suppose you wish to solve the following tridiagonal system.
\[\begin{bmatrix}
b_1 & c_1 & 0 & 0 & \cdots & \cdots & 0 \\
a_2 & b_2 & c_2 & 0 & \cdots & \cdots & 0 \\
0 & a_3 & b_3 & c_3 & \cdots & \cdots & 0 \\
\vdots & \vdots & \vdots & \vdots & \ddots & \ddots & \vdots \\
\vdots & \vdots & \vdots & \vdots & \ddots & \ddots & c_{n-1} \\
0 & 0 & 0 & 0 & \cdots & a_n & b_n
\end{bmatrix}
\begin{bmatrix}
x_1\\
x_2\\
x_3\\
\vdots\\
\vdots\\
x_n
\end{bmatrix}
=
\begin{bmatrix}
d_1\\
d_2\\
d_3\\
\vdots\\
\vdots\\
d_n
\end{bmatrix}\]
One way to do this is with the general \li{solve} method in SciPy's \li{linalg} module.
Alternatively, you could use an algorithm optimized for tridiagonal matrices.
The code below implements one such algorithm in Python. This is called the Thomas algorithm.

\begin{lstlisting}
def pytridiag(a,b,c,d):
    """Solve the tridiagonal system Ax = d where A has diagonals a, b, and c.

    Inputs:
        a, b, c, d (array) - All 1-D NumPy arrays of equal length.
    Returns:
        x (array) - solution to the tridiagonal system.
    """
    n = len(a)

    # Make copies so the original arrays remain unchanged
    aa = np.copy(a)
    bb = np.copy(b)
    cc = np.copy(c)
    dd = np.copy(d)

    # Forward sweep
    for i in xrange(1, n):
        temp = aa[i]/bb[i-1]
        bb[i] = bb[i] - temp*cc[i-1]
        dd[i] = dd[i] - temp*dd[i-1]

    # Back substitution
    x = np.zeros_like(a)
    x[-1] = dd[-1]/bb[-1]
    for i in xrange(n-2, -1, -1):
        x[i] = (dd[i]-cc[i]*x[i+1])/bb[i]

    return x
\end{lstlisting}

\begin{problem} \label{prob:tridiag}
\leavevmode
\begin{enumerate}
\item Write a function \li{numba_tridiag} that is a compiled version of \li{pytridiag}.
\item Compare the speed of your new function with \li{pytridiag} and \li{scipy.linalg.solve}.
When comparing \li{numba_tridiag} and \li{pytridiag}, use a $1000000 \times 1000000$ sized systems.
When comparing \li{numba_tridiag} and the SciPy algorithm, use a $1000 \times 1000$ systems. You may use the code below to generate the arrays, \li{a}, \li{b}, and \li{c}, along with the corresponding tridiagonal matrix \li{A}.
\end{enumerate}
\begin{lstlisting}
def init_tridiag(n):
    """Initializes a random nxn tridiagonal matrix A.

    Inputs:
        n (int) : size of array

    Returns:
        a (1-D array) : (-1)-th diagonal of A
        b (1-D array) : main diagonal of A
        c (1-D array) : (1)-th diagonal of A
        A (2-D array) : nxn tridiagonal matrix defined by a,b,c.
    """
    a = np.random.random_integers(-9,9,n).astype("float")
    b = np.random.random_integers(-9,9,n).astype("float")
    c = np.random.random_integers(-9,9,n).astype("float")

    # Check for and change 0 values in arrays
    a[a==0] = 1
    b[b==0] = 1
    c[c==0] = 1

    # Create tridiagonal matrix A from a,b,c
    A = np.zeros((b.size,b.size))
    np.fill_diagonal(A,b)
    np.fill_diagonal(A[1:,:-1],a[1:])
    np.fill_diagonal(A[:-1,1:],c)
    return a,b,c,A
\end{lstlisting}
Note that an efficient tridiagonal matrix solver is implemented by \li{scipy.sparse.linalg.spsolve()}.
\end{problem}

\section*{When to Stop Optimizing}
You don't need to apply every possible optimization to your code.
When your code runs acceptably fast, stop optimizing. There is no need spending valuable time making optimizations once the speed is sufficient.

Moreover, remember not to prematurely optimize your functions. Make sure the function does exactly what you want it to before worrying about any kind of optimization.

% TODO: get these lab references correct and replace.
\begin{problem}
Optimize a function you wrote in a previous lab using the techniques discussed in this lab. Consider optimizing one of the following:
\begin{enumerate}
\item Householder triangularization or Hessenburg decomposition % (Lab \ref{lab:QRdecomp}).
\item Givens triangularization % (Lab \ref{lab:qr-applications})
\item Image Segmentation % (Lab \ref{lab:ImgSeg_eigenvalues})
\item Eigenvalue Solvers % (Lab \ref{lab:EigSolve})
\end{enumerate}
 Compare the timings of the function before and after optimization using your \li{compare_timings} function. Write a short paragraph describing what you did to optimize your function.
\end{problem}


% !TEX root = main.tex

%-------------------------------------------------
\section{The Markov and Chebyshev inequalities}\label{sec:conc}

%-----------------------------
\subsection{Markov's inequality}

If the distribution of a random variable is not known, probabilities can be estimated using the moments of the distribution. A simple upper bound on the tail probabilities of non-negative random variables is provided by \emph{Markov's inequality}.

% theorem: markov inequality
\begin{theorem}[Markov's inequality]
Let $X$ be a non-negative random variable. Then for every $a>0$,
\[
\prob(X\geq a) \leq \frac{\expe(X)}{a}.
\]
\end{theorem}

% proof
\begin{proof}
Let $I_A$ be the indicator function of event $A = \{\omega: X(\omega)\geq a\}$.
\[
I_A(\omega) = \left\{\begin{array}{ll} 0 & \text{if}\ X(\omega) < a, \\ 1 & \text{if}\ X(\omega)\geq a. \end{array}\right.
\]
\bit
\it If $\omega\in A$ then $X(\omega)   \geq a = a I_A(\omega)$.
\it If $\omega\notin A$ then $X(\omega)\geq 0 = a I_A(\omega)$.
\eit
In either case we have $X(\omega)\geq a I_A(\omega)$ so by the monotonicity of expectation,
\[
\expe(X) \geq a\expe(I_A) = a\prob(A) \equiv a\prob(X\geq a).
\]
Hence $\prob(X\geq a) \leq \expe(X)/a$ as required.
\end{proof}

% example
\begin{example}
A fair die is rolled once. Use Markov's inequality to find an upper bound on the probability that we observe a score of at least $5$.
\end{example}

\begin{solution}
Let $X$ be the number shown on the die. Then $X$ is a non-negative random variable with expectation $\expe(X) = 7/2$.
Markov's inequality yields the upper bound
\[
\prob(X\geq 5) \leq \frac{\expe(X)}{5} = \frac{7}{10}.
\]
In this example we know that $\prob(X\geq 5)=1/3$, which illustrates that Markov's inequality provides only crude bounds on tail probabilites. Indeed, for the probability $\prob(X\geq 3)$ Markov's inequality yields
\[
\prob(X\geq 3) \leq \frac{\expe(X)}{3} = \frac{7}{6}.
\]
This tells us nothing useful, because we know that $\prob(X\geq 3)\leq 1$.
\end{solution}
%-----------------------------
\subsection{Chebyshev's inequality}

An upper bound on the absolute deviation of a random variable from its mean is provided by \emph{Chebyshev's inequality}.

\begin{corollary}[Chebyshev's inequality]
Let $X$ be any random varible with finite expectation. Then for all $\epsilon>0$,
\[
\prob(|X-\expe(X)|\geq \epsilon) \leq \frac{\var(X)}{\epsilon^2}.
\]
\end{corollary}
\begin{proof}
Take the non-negative random variable $\big[X-\expe(X)\big]^2$ in Markov's inequality with $a=\epsilon^2$:
\begin{align*}
\prob(|X-\expe(X)|\geq \epsilon)
	& = \prob\big[\big(X-\expe(X)\big)^2\geq \epsilon^2\big]
	\leq	\frac{\expe\big[\big(X-\expe(X)\big)^2\big]}{\epsilon^2}
	= \frac{\var(X)}{\epsilon^2},
\end{align*}
as required.
\end{proof}

\begin{example}
Suppose that $\expe(X) = 0$ and $\var(X) = 1$. Find an integer value $k$ such that $\prob(|X|\geq k) \leq 0.01$.
\begin{solution}
$X$ is not non-negative, so we can not use Markov's inequality here. By Chebyshev's inequality,
\[
\prob(|X|\geq k) = \prob(|X-\expe X|\geq k) \leq \frac{\var(X)}{k^2} = \frac{1}{k^2} %\leq \frac{1}{100}
\]
so can take any $k\geq 10$.
\end{solution}
\end{example}

% example
\begin{example}
Let $X$ be a continuous random variable with expected value $3.6$ and standard deviation $1.2$.
\par Show that $\prob(1.2\leq X\leq 6.0) \geq 0.75$.
\begin{solution} % <<<
\par
The event $\{1.2 \leq X \leq 6.0\}$ can be written as $\{|X-3.6| \leq 2.4\}$, and by Chebyshev's inequality,
\[
\prob(|X-3.6|> 2.4) \leq \frac{\var(X)}{2.4^2} = \frac{1.2^2}{2.4^2} = \frac{1}{4},
\]
so $\prob(1.2 \leq X \leq 6.0) \geq 0.75$, as required.
\end{solution}
\end{example}

\begin{exercise}
\begin{questions}

\question
What does Chebyshev's inequality tell us about the probability that the value taken by a random variable deviates from its expectation by five or more standard deviations?
\begin{answer}
For any random variable $X$ with finite variance $\sigma^2$,
\[
\prob(|X-\mu|\geq 5\sigma) \leq \frac{\sigma^2}{(5\sigma)^2} = \frac{1}{25} = 0.04.
\]
Thus for any distribution, we are very unlikely to observe values that are more than 5 standard deviations away from the mean.
\end{answer}

\question
A fair coin is tossed $n$ times. Does Chebyshev's inequality ensure that the observed number of heads will not deviate from $n/2$ by more than $100$ with a probability of at least $0.99$, provided that $n$ is sufficiently large?
\begin{answer}
No. Let $X_i=1$ if a head occurs on the $i$th toss and $X_i=0$ otherwise, and let $S_n = \sum_{i=1}^n X_i$ be the total number of heads after $n$ tosses. Because the coin is fair, $\expe(S_n)=n/2$ and $\var(S_n)=n/4$. Applying Chebyshev's inequality to $S_n$ with $\epsilon=100$:
\[
\prob(|S_n - n/2| > 100) \leq \var(S_n)/100^2
\]
But $\var(S_n)\to\infty$ as $n\to\infty$, so the probability that $S_n$ deviates from $n/2$ by a fixed amount cannot be bounded using Chebyshev's inequality.
\end{answer}

\question
Let $X$ be a random variable with mean $\mu\neq 0$ and variance $\sigma^2$. The \emph{relative deviation} of $X$ from its mean is defined by $\displaystyle D  = \left|\frac{X-\mu}{\mu}\right|$. Use Chebyshev's inequality to show that
$
\displaystyle \prob(D \geq \epsilon) \leq \left(\frac{\sigma}{\mu\epsilon}\right)^2.
$
\begin{answer}
By Chebyshev's inequality,
\[
\prob(D\geq \epsilon)
	= \prob\left(\left|\frac{X-\mu}{\mu}\right|\geq \epsilon\right)
	= \prob\big(|X-\mu|\geq |\mu|\epsilon)
	\leq \frac{\sigma^2}{\mu^2 \epsilon^2}
\]
\end{answer}

\question
\begin{parts}
\part\label{qu:pt:cheb_bern}
Let $X_1,X_2,\ldots,X_n$ be independent with each $X_i\sim\text{Bernoulli}(p)$. Using the fact that $p(1-p)\leq 1/4$ for all $0<p<1$, show that for any $\epsilon > 0$,
\[
\prob\left(\left|\frac{1}{n}\sum_{i=1}^n X_i - p\right|\geq\epsilon\right) \leq \frac{1}{4n\epsilon^2}.
\]
\begin{answer} % <<<
For the $\text{Binomial}(n,p)$ distribution, Chebyshev's inequality yields
\[
\prob\left(\left|\frac{S_n}{n}-p\right|\geq\epsilon\right) \leq \frac{p(1-p)}{n\epsilon^2}
\]
and the result follows because $p(1-p)\leq 1/4$.
\end{answer}
\part
Let $A$ be an event associated with a random experiment. Suppose that the experiment is repeated $n$ times. Let $X_i$ be the indicator variable of the event that $A$ occurs during the $i$th trial. Then $X_i\sim\text{Bernoulli}(p)$ where $p=\prob(A)$, and the sample mean $\frac{1}{n}\sum_{i=1}^n X_i$ is the \emph{relative frequency} of $A$ over these $n$ trials. What does the upper bound derived in part \ref{qu:pt:cheb_bern} say about the relative frequency of event $A$ as $n\to\infty$.
\begin{answer} % <<<
The probability that the relative frequency $\displaystyle\frac{1}{n}\sum_{i=1}^n X_i$ of $A$ differs from its true probability $p=\prob(A)$ by more than a fixed amount (however small), tends to zero as the number of trials increases to infinity.
\end{answer}
\end{parts}

\end{questions}
\end{exercise}


%	\includegraphics[scale=0.95]{competences}

	\section{Calculer}

	Calculer astucieusement en détaillant les calculs
	\begin{questions}

%		\question[2]  $\num{5.5} + 4 + \num{2.5} + 8$
%		\fillwithdottedlines{2cm}
%		%
%		\begin{solution}
%
%		\end{solution}

		\question[2]  $\num{3.3} + \num{7.4} + \num{2.7} + \num{2.6} + 8 $
		\fillwithdottedlines{2cm}
		\begin{solution}

		\end{solution}

		\question[2]  $\num{3.2} + \num{7.5} + \num{2.8} + \num{5.5} + 15 $
		\fillwithdottedlines{2cm}
		\begin{solution}

		\end{solution}

		\question[2]  $\num{5} \times 25 \times 11 \times \num{4} \times 2$
		\fillwithdottedlines{2cm}
		\begin{solution}

		\end{solution}

		\question[2]  $\num{2.5} \times 4 \times 3 \times 7$
		\fillwithdottedlines{2cm}
		\begin{solution}

		\end{solution}

		\question[2]  $\num{12.5} \times 25 \times \num{8} \times 4$
		\fillwithdottedlines{2cm}
		\begin{solution}

		\end{solution}

	\newpage

		\question[2]  $\num{5} \times 50  \times \num{4} \times 2$
		\fillwithdottedlines{2cm}
		\begin{solution}

		\end{solution}

		\question[2]  $\num{3} \times 45 \times 20 \times \num{5} $
		\fillwithdottedlines{2cm}
		\begin{solution}

		\end{solution}

		\question[2]  $(2 + 5) \times (7 + 2) $
		\fillwithdottedlines{2cm}
		\begin{solution}

		\end{solution}

		\question[2]  $2 + 8 \times 7 + 2 $
		\fillwithdottedlines{2cm}
		\begin{solution}

		\end{solution}

		\question[2]  $(2 + 8 \times 7) \times 2 $
		\fillwithdottedlines{2cm}
		\begin{solution}

		\end{solution}

	\end{questions}


\section{Other attacks}
Attacks against discrete logarithm involve finding the integer $x$ for a given $\alpha$ and $\beta$ in $G$ such that:
$$\beta = \alpha \circ \alpha \circ \dots \circ \ alpha = \alpha^x$$

In 1997, a lower bound for discrete logarithms was given: any generic algorithm that solves with high probability the problem must perform at least $\Omega(p^{1/2})$ group operations, where $p$ is the largest prime dividing the cardinality of the group. The same result holds for non cyclic groups and Diffie-Hellman protocol.

The exact difficulty of computing this is still unknown: despite the existence of algorithms, there could be better or more powerful ones which still are undiscovered (similarly to integer factorization).

Generic algorithms are methods which only rely on the group operation $\circ$, without needing further algebraic structure. This property makes them applicable in any cyclic group.

\subsection{Brute force search}
Brute-force is the most naive and costly generic algorithm to obtain the discrete logarithm $\log_\alpha\beta$: all powers of the generator $\alpha$ are computed \textbf{successively}, until the result equals $\beta$.

For a random logarithm $x$, on average the correct solution is expected to be found after checking \textit{half of the possible values}. This gives a complexity of $O(|G|)$, linear in the cardinality of the group.

To avoid this kind of attack to be successful, therefore, the cardinality of groups must be sufficiently large, without any other particular measure.

In the case of $\mathbb{Z}^*_p$ with $p$ prime, approximately $\frac{p-1}{2}$ tests are required to compute the discrete logarithm, meaning that $|G| = p - 1$ should be in the order of at least $2^{80}$ to make a brute force infeasible using the modern hardware.

\subsection{Shank's method}
Shank's method is another generic algorithm which allows to reduce the time of a brute force search, at the trade off of occupying more storage. It is a meet-in-the-middle attack, storing intermediate values by trying to successively crack the multiple steps of discrete logarithm encryption.

The procedure is based on rewriting the discrete logarithm in a two-digit representation:
$$x = x_gm = x_b \qquad 0 \leq x_g, x_b < m$$
The value $m$ is chosen to be approximately the square root of the cardinality of the set, i.e.\ $m = \lceil{\sqrt{|G|}}\rceil$. Discrete logarithm can then be stated as $\beta = \alpha^x = \alpha^{x_gm + x_b}$ which leads to:
$$\beta \cdot (\alpha^{-m})^{x_g} = \alpha^{x_b}$$
The core idea of the algorithm is using a divide-and-conquer approach to find $x_a$ and $x_b$ separately, in two phases:
\begin{enumerate}
	\item Baby-step, where all the values $\alpha^{x_b}$ with $0 \leq x_b < m$ are computed and stored with approximately $m \approx \sqrt{|G|}$ operations;
	\item Giant-step, in which the program checks for all $x_g$ in $0 \leq x_g < m$ whether this condition is fulfilled:
	$\beta \cdot (\alpha^{-m})^g \stackrel{?}{=} \alpha^{x_b}$ for some stored entry $\alpha^{x_b}$ that was computed during the previous step.
\end{enumerate}
In case of a match, i.e.\ $\beta \cdot (\alpha^{-m})^{x_g, 0} = \alpha^{x_b, 0}$ for some pair $(x_{g, 0}, x_{b, 0})$ the discrete logarithm is:
$$x = x_{g, 0}m + x_{b, 0}$$
Further speed up the method can be obtained using efficient lookup schemes, such as \textbf{hash tables} which allow constant time search.

This algorithm has a total computational time and space of $O(\sqrt{|G|})$, which means that in a group of order $2^{80}$ an attacker would approximately need $2^{40}$ operations, easily obtainable with modern hardware.

Direct consequence of this is that groups need to have a cardinality of at least $|G| \geq 2^{160}$, to make Shank's method have a complexity of $2^{80}$ and therefore be infeasible. In case of prime groups $G = \mathbb{Z}^*_p$, $p$ must have a length of at least 160 bit; using prime groups is advised since otherwise Shank's method would have an even smaller complexity.

\subsection{Pollard's Rho method}
Pollard's Rho method has the same computational time as Shank's algorithm, yet constant space requirements. It is based on the \textbf{birthday paradox}, concept related to the higher likelihood of collisions found between random attack attempts and a fixed degree of permutations.

The basic idea consists in pseudo-randomly generate ($\rho$ represents randomness) group elements of the form $\alpha^i \cdot \beta^j$, keeping track of values $i$ and $j$, to then continue until obtaining a collision:
$$\alpha^{i_1} \cdot \beta^{j_1} = \alpha^{i_2} \cdot \beta^{j_2}$$
Substituting $\beta = \alpha^x$ and comparing the exponents on both sides of the equation, the collision leads to the following formula:
$$i_1 + xj_1 \equiv i_2 + jx_2 \mod |G| \qquad \rightarrow \qquad x \equiv \frac{i_2 - i_1}{j_1 - j_2} \mod |G|$$
The second part allows to find the discrete logarithm. Furthermore, additional methods to speed up computations such as the Extended Euclidean Algorithm and Floyd's cycle-finding algorithm.

Its computational time is approximately $O(\sqrt{n})$, while used together with Silver-Pohlig-Hellman it can achieve $O(\sqrt{p})$ where $p$ is the largest prime factor of $n$.

The practical importance of Rho's method is it being the best-known algorithm in \textit{elliptic curve group}s, making 160 bit operands very popular within this cryptography; however, it is not the most powerful attack for discrete logarithm and it does not work well in a distributed environment.

Seminar - Algebraic Methods and Algorithms in Cryptology  /chapters/attacks.tex