\section{Universal cover}

The following two theorems summarize our results about covering spaces.

\begin{theorem}[Galois Correspondence of Covering Spaces]
  There is a 1-1 correspondence between the subgroups of $\Gal(Y|X)$ and covers of $X$ that lie between $Y$ and $X$, given by the following maps
  \begin{align*}
    \{\mbox{ subgroups of $\Gal(Y|X)$ } \} &\longleftrightarrow  \{\mbox{ covers lying between $Y$ and $X$ } \}\\
    H &\longmapsto Y/H \\
    \Gal(Y|Z) & \longmapsfrom Z
  \end{align*}
  Under this correspondence, the normal subgroups of $\Gal(X|Y)$ correspond to Galois covers of $X$.
\end{theorem}

\begin{theorem}[Monodromy action of the fundamental group]
  \label{theorem:fundamentalGroupQuotient2}
  If $p:Y \rightarrow X$ is a Galois cover with $p(y) = x$, then there is a short exact sequence of groups
  \begin{equation*}
    \begin{tikzcd}
      1 \ar[r] & \pi_1(Y,y) \ar[r, "p_*"] & \pi_1(X,x) \ar[r,"M"] &  \Gal(Y|X) \ar[r] & 1
    \end{tikzcd}
  \end{equation*}
  where the map $p_*$ is sending a loop $\gamma$ in $Y$ to $p(\gamma)$ and $M$ is the monodromy action.
\end{theorem}

We will assume the following for now without proof.
\begin{theorem}[Existence of universal cover]
  For any space $X$, there exists a space $\widetilde{X}$ with a covering map $P:\widetilde{X} \rightarrow X$ satisfying the following properties:
  \begin{enumerate}
    \item $P$ is Galois.
    \item $\pi_1(\widetilde{X})$ is trivial. (We say that $\widetilde{X}$ is simply connected.)
    \item Every cover $Y$ of $X$ lies between $\widetilde{X}$ and $X$.
  \end{enumerate}
  Such a space $\widetilde{X}$ is called the \emph{universal cover} of $X$.
\end{theorem}

Several corrolaries follow immediately from this statement.

\subsection{Classification of all covers}

\begin{proposition}
  $\Gal(\widetilde{X} | X) \cong \pi_1(X,x)$.
\end{proposition}
\begin{proof}
  Apply Theorem \ref{theorem:fundamentalGroupQuotient2} to the cover $P:\widetilde{X} \rightarrow X$.
\end{proof}

\begin{theorem}[Galois correspondence \#2]
  \label{theorem:GaloisCorrespondence2}
  There is a 1-1 correspondence between the subgroups of $\pi_1(X,x)$ and covers of $X$, given by the following maps
  \begin{align*}
    \{\mbox{ subgroups of $\pi_1(X,x)$ } \} &\longleftrightarrow  \{\mbox{ covers of $X$ } \}\\
    H &\longmapsto \widetilde{X}/H \\
    \pi_1(Z,z) & \longmapsfrom Z
  \end{align*}
  Under this correspondence, the normal subgroups of $\pi_1(X,x)$ correspond to Galois covers of $X$.
\end{theorem}

\subsection{Subgroups of a free group}
Denote by $F_n$ the free group with $n$ generators.
\begin{theorem}
  Every subgroup of a finitely generated free group is free.
\end{theorem}
\begin{proof}
  Let $G = \mathrm{Free}(S)$ be a finitely generated free group with $|S|=k$.
  Let $X$ be a wedge of $k$ circles, so that $\pi_1(X) = G$.
  \begin{figure}[H]
    \centering
    \begin{tikzpicture}[scale=0.5]
     \input{images/bouquet5.tex}
    \end{tikzpicture}
    \caption{Wedge of 5 circles $\bigvee \limits_5 S^1$ has fundamental group $F_5$.}
  \end{figure}
  Every cover of $X$ is a graph, and the fundamental group of a graph is a free group (as there are no faces).
  Hence, by Theorem \ref{theorem:GaloisCorrespondence2} every subgroup of $G$ is free.
\end{proof}

\begin{theorem}
  If $F_m \triangleleft F_n$ then $n - 1 | m - 1$ and the index of $F_m$ inside $F_n$ is $(m-1)/(n-1)$.
\end{theorem}
\begin{proof}
  A normal subgroup $H$ of $G$ corresponds to a Galois cover $X$ of $\bigvee \limits_n S^1$, of some degree, say $d$.
  As $X$ is a cover, one can check by simple edge conting, that $X$ is free group on $(n-1)d + 1$ generators.
  The result follows.
\end{proof}

\subsection{Universal property of the universal cover}

We say that a space $Y$ is simply connected if $\pi_1(Y) \cong \set{\mathbb{1}}$.
\begin{proposition}
  Let $p:Y \rightarrow X$ be a Galois cover such that $Y$ is simply-connected.
  Further suppose that $p_2:Z \rightarrow X$ is another cover then $Z$ is a subcover of $Y$.
  \begin{equation*}
    \begin{tikzcd}
      Y \ar[rd, dashed, "\exists"] \ar[dd, "p"']\\
        & Z \ar[ld,"p_2"] \\
      X
    \end{tikzcd}
  \end{equation*}
\end{proposition}
\begin{proof}
  We will explicitly construct a map $p_1: Y \rightarrow Z$.
  Pick vertices $y_0$ in $Y$, $x_0$ in $X$, and $z_0$ in $Z$ such that $p(y_0) = x_0 = p_2(z_0)$.
  We define $p_1(y_0) = z_0$.

  Let $y$ be a vertex in $Y$. Let $\gamma$ be a path in $Y$.
  We push down $\gamma$ to X then lift it up to $X$ and set $p_1(y) = d_1 \widetilde{p(\gamma)}$.
  \begin{equation*}
    \begin{tikzcd}
      \gamma \ar[dd, mapsto]\\
        & \widetilde{p(\gamma)}  \\
      p(\gamma) \ar[ur, mapsto]
    \end{tikzcd}
  \end{equation*}

  We need to check that the map defined this way is well-defined i.e. it does not depend on the choice of the path $\gamma$.
  Suppose there are two paths $\gamma_1$ and $\gamma_2$ connecting $y_0$ to $y$.
  Then $\gamma_1^{-1} \gamma_2$ is a loop at $y$ and hence is (upto conjugation) a boundary of a face.
  It follows from this that the lifts of both $\gamma_1$ and $\gamma_2$ have the same endpoints.
\end{proof}

And so a universal cover of $X$ is any cover $\widetilde{X}$ of $X$ that is simply-connected.
There is a theorem of existence of the universal space of a general space, but in practice we simply construct these spaces by hand.

\begin{ex}
  The universal cover of $S^1$ is $\bbr^1$.
\end{ex}

\begin{ex}
  The universal cover of a cylinder (and hence also a Mobius strip) is $\bbr^1 \times [0,1]$.
\end{ex}

\begin{ex}
  The universal cover of a torus (and hence also a Klein bottle) is $\bbr^2$.
\end{ex}

\begin{ex}
  The universal cover of a real projective space is $S^2$.
\end{ex}

\begin{ex}
  The universal cover of a bouquet of $k$ circles is the Cayley graph of the free group with $k$ generators.
  \begin{figure}[H]
  \centering
    % \includegraphics[width=0.5\textwidth]{example-image}
    \includegraphics[width=0.5\textwidth]{CayleyF2.png}
    \caption{Universal cover of $F_2$}
  \end{figure}
\end{ex}

\begin{ex}
  A universal cover of a graph $X$ (space with no faces) can be constructed as follows:
  \begin{enumerate}
    \item Pick a vertex $x$.
    \item $\widetilde{X}$ is a graph whose vertices are paths in $X$ that start at $x$.
    \item There is an edge from $\gamma$ and $\gamma'$ if $\gamma = \gamma' \cdot e$ for some edge $e$.
    \item The covering map $P:\widetilde{X} \rightarrow X$ is the map $\gamma \mapsto d_1 \gamma$.
  \end{enumerate}
  \begin{qbox}
    Show that the $\widetilde{X}$ defined above is a simply-connected space and that $P$ is a covering map, and hence $\widetilde{X}$ is a universal cover.
  \end{qbox}
  This method can be generalized to spaces with faces. In this case, instead of taking all paths we need to take equivalence classes of paths with respect to boundaries of faces.
  \begin{equation*}
    \widetilde{X}_x = \set{ \mbox{paths starting at } x} / \mbox{faces}
  \end{equation*}
\end{ex}

\begin{remark}
  Finally, here is a comparison of the Galois correspondences for algebra and topology from Terrence Tao's blog \cite{TerrenceTao}.
  \begin{figure}[H]
  \centering
    \includegraphics[width=0.8\textwidth]{TerryTao1.png}
    \includegraphics[width=0.8\textwidth]{TerryTao2.png}
  \end{figure}

\end{remark}

\begin{thebibliography}{9}
\bibitem{Szamuely}
Tam\'as Szamuely, \emph{Galois Groups and Fundamental Groups}, Cambridge Studies in Advanced Mathematics, 2007.

\bibitem{TerrenceTao}
Terrence Tao, \emph{Trying to understand the Galois correspondence},\\ \url{https://terrytao.wordpress.com/2018/08/28/trying-to-understand-the-galois-correspondence/}.
\end{thebibliography}

% In this section, we'll prove that for every (connected) space $Y$ and a vertex $y \in Y$, there exists a simply-connected space $\widetilde{Y}_y$ with a covering map
% \begin{align*}
%   \widetilde{Y}_y \rightarrow Y
% \end{align*}
%
% By Theorem \ref{theorem:liftingOfCovers} this cover is universal, in the sense that every cover $Z \rightarrow Y$ lies between $\widetilde{Y}_y$ and $Y$.
% \begin{equation*}
%   \begin{tikzcd}
%     & Z \ar[d] \\
%     \widetilde{Y}_y \ar[ur,"\exists !",dashrightarrow] \ar[r, "p'"']& Y
%   \end{tikzcd}
% \end{equation*}
%
% The following lemmas are more or less true by definition.
% \begin{lemma}
%   The fundamental group of a graph (=space with only vertices and edges) is a free group.
% \end{lemma}
%
% \begin{lemma}
%   Every cover of a graph is a graph.
% \end{lemma}


The previous chapter developed a mathematical relationship between the position of a point $P$ in a scene (expressed in world frame coordinates $P_W$), and the corresponding point $p$ in pixel coordinates that gets projected onto the image plane of the camera. This relationship was derived based on the pinhole camera model, and required knowledge about the camera's intrinsic and extrinsic parameters. Nonetheless, even in the case where all of these camera parameters are known it is still impossible to reconstruct the depth of $P$ with a single image (without additional information).
However, in the context of robotics, recovering 3D information about the structure of the robot's environment through computer vision is often a very important task (e.g. for obstacle avoidance). Two approaches for using cameras to gather 3D information are therefore presented in this chapter, namely \textit{stereo vision} and \textit{structure from motion}\cite{SiegwartNourbakhshEtAl2011}\cite{ForsythPonce2011}.

\notessection{Stereo Vision and Structure From Motion}
Recovering scene structure from images is extremely important for mobile robots to safely operate in their environment and successfully perform tasks. While a number of other sensors can also be used to recover 3D scene information, such as ultrasonic sensors or laser rangefinders, cameras capture a broad range of information that goes beyond depth sensing. Additionally, cameras are a well developed technology and can be an attractive option for robotics based on cost or size.

Unfortunately, unlike sensors that are specifically designed to measure depth like laser rangefinders, the camera's projection of 3D data onto a 2D image makes it impossible to gather some information from a single image\footnote{Unless you are willing to make some strong assumptions, for example that you know the physical dimensions of the objects in the environment.}. Techniques for extracting 3D scene information from 2D images have therefore been developed that leverage \textit{multiple} images of a scene. Examples of such techniques include \textit{depth-from-focus} (uses images with different focuses), \textit{stereo vision} (uses images from different viewpoints), or \textit{structure from motion} (uses images captured by a moving camera).

\subsection{Stereo Vision}
Stereopsis (from \textit{stereo} meaning solidity, and \textit{opsis} meaning vision or sight) is the process in visual perception leading to the sensation of depth from two slightly different projections of the world onto the retinas of the two eyes. The difference in the two retinal images is called horizontal \textit{disparity}, retinal disparity, or binocular disparity, and arise from the eyes' different positions in the head. It is the disparity that makes our brain fuse (perceive as a single image) the two retinal images, making us perceive the object as one solid object. For example, if you hold your finger vertically in front of you and alternate closing each eye you will see that the finger jumps from left to right. The distance between the left and right appearance of the finger is the disparity.

Computational stereopsis, or \textit{stereo vision}, is the process of obtaining depth information of a 3D scene via images from two cameras which look at the same scene from different perspectives. This process consists of two major steps: fusion and reconstruction. Fusion is a problem of correspondence, in other words how do you correlate each point in the 3D environment to their corresponding pixels in \textit{each} camera. Reconstruction is then a problem of \textit{triangulation}, which uses the pixel correspondences to determine the full position of the source point in the scene (including depth).

\subsubsection{Epipolar Constraints}
As previously mentioned, the first step in the stereo vision process is to fuse the two (or more) images and generate point correspondences\footnote{This generally assumes that the perspective of each image is only a slight variation from the other, such that the features appear similarly in each.}. This task can be quite challenging, and erroneously matching features can lead to large errors in the reconstruction step. Therefore, several techniques are leveraged to make this task simpler. The most important simplifying technique is to impose an \textit{epipolar constraint}.

\begin{figure}[ht]
  \begin{center}
	\includegraphics[width=.8\textwidth]{tex/figs/ch09_figs/stereo.png}
  \end{center}
  \caption{The point $P$ in the scene, the optical centers $O$ and $O'$ of the two cameras, and the two images $p$ and $p'$ of $P$ all lie in the same plane, referred to as the epipolar plane. The lines $l$ and $l'$ are the epipolar lines of the points $p$ and $p'$, respectively. Note that if the point $p$ is observed in one image, the corresponding point in the second image must lie on the epipolar line $l'$!}
  \label{fig:epi}
\end{figure}

Consider the images $p$ and $p'$ of a point $P$ observed by two cameras with optical centers $O$ and $O'$ (see Figure \ref{fig:epi}). These five points all belong to the \textit{epipolar plane} defined by the two intersecting rays $OP$ and $O'P$.
In particular, the point $p$ lies on the line $l$ where the epipolar plane and the image plane intersect. The line $l$ is referred to as the \textit{epipolar line} associated with the point $p$, and it passes through the point $e$ (referred to as the \textit{epipole}).
Based on this geometry, if $p$ and $p'$ are images of the same point $P$, then $p$ must lie on the epipolar line $l$ and $p'$ must lie on the epipolar line $l'$.

Therefore, when searching for correspondences between $p$ and $p'$ for a particular point $P$ in the scene it makes sense to restrict the search to the corresponding epipolar line. This is referred to as an \textit{epipolar constraint}, and greatly simplifies the correspondence problem by restricting the possible candidate points to a line rather than the entire image (i.e. a one dimensional search rather than a two dimensional search).
Mathematically, the epipolar constraints can be written as:
\begin{equation}
\overline{Op} \cdot [\overline{OO'} \times \overline{O'p'}] = 0,
\end{equation}
since $\overline{Op}$, $\overline{O'p'}$, and $\overline{OO'}$ are coplanar. Assuming the world reference frame is co-located with camera 1 (with an origin at point $O$) this constraint can be written as:
\begin{equation} \label{eq:epiconst}
    p^\top  F p'=0,
\end{equation}
where $F$, referred to as the \textit{fundamental matrix}, has seven degrees of freedom and is singular. For a derivation of the epipolar constraint see Section 7.1 from Forsyth et al.\cite[]{ForsythPonce2011}. Additionally, the matrix $F$ is only dependent on the intrinsic camera parameters for each camera and the geometry that defines their relative positioning, and can be assumed to be constant. The expression for the fundamental matrix in terms of the camera intrinsic parameters is:
\begin{equation}
    F = K^{-\top}EK'^{-1}, \quad E = \begin{bmatrix}
    0 & -t_3 & t_2 \\
    t_3 & 0 & -t_1 \\
    -t_2 & t_1 & 0
    \end{bmatrix}R,
\end{equation}
where $K$ and $K'$ are the intrinsic parameter matrices for cameras 1 and 2 respectively, and $R$ and $t = [t_1, t_2, t_3]^\top $ are the rotation matrix and translation vector that map camera 2 frame coordinates into camera 1 frame coordinates.
Note that with the epipolar constraint defined by the fundamental matrix \eqref{eq:epiconst}, the epipolar lines $l$ and $l'$ can be expressed by $l = Fp'$ and $l' = F^\top p$. Additionally, it can be shown that $F^\top e = Fe' = 0$ where $e$ and $e'$ are the epipoles in the image frames of cameras 1 and 2, since by definition the translation vector $t$ is parallel to the coordinate vectors of the epipoles in the camera frames. This in turn guarantees that the fundamental matrix $F$ is singular.

If the parameters $K$, $K'$, $R$, and $t$ are not already known, the fundamental matrix $F$ can be determined in a manner similar to the intrinsic parameter matrix $K$ in the previous chapter. Suppose a number of corresponding points $p^h = [u, v, 1]^\top $ and $(p^h)'= [u',v',1]^\top $ are known and are expressed as homogeneous coordinates. Each pair of points has to satisfy the epipolar constraint \eqref{eq:epiconst}, which can be written as:
\begin{equation*}
\begin{bmatrix}
u & v & 1
\end{bmatrix} \begin{bmatrix}
F_{11} & F_{12} & F_{13} \\
F_{21} & F_{22} & F_{23} \\
F_{31} & F_{32} & F_{33}
\end{bmatrix} \begin{bmatrix}
u' \\ v' \\ 1
\end{bmatrix} = 0
\end{equation*}
This expression can then be equivalently expressed by reparameterizing the matrix $F$ in vector form $f$ as:
\begin{equation}
\begin{bmatrix}
uu' & uv' & u & vu' & vv' & v & u' & v' & 1
\end{bmatrix}f = 0
\end{equation}
where $f = [F_{11}, \:F_{12} , \: F_{13}, \:F_{21}, \:F_{22}, \:F_{23}, \:F_{31}, \:F_{32}, \:F_{33}]^\top $. For $n$ known correspondences $(p,p')$ these constraints can be stacked to give:
\begin{equation}
    Wf = 0,
\end{equation}
where $W \in \R^{n \times 9}$.
Given $n \geq 8$ correspondences, an estimate $\tilde{F}$ of the fundamental matrix estimate is given by:
\begin{equation} \label{eq:fopt}
\begin{split}
\min_{f} \:\:& \lVert Wf \rVert^2, \\
\text{s.t.} \:\:& \lVert f \rVert^2 = 1.
\end{split}
\end{equation}
Note that the estimate $\tilde{F}$ computed by \eqref{eq:fopt} is not guaranteed to be singular. A second step is therefore taken to enforce this additional condition. In particular it is desirable to find the matrix $F$ that is closest to the estimate $\tilde{F}$ that has a rank of two:
\begin{equation}
\begin{split}
   \min_F \:\:& \lVert F-\tilde{F}\rVert^2, \\
\text{s.t.} \:\:& \text{det}(F) = 0,
\end{split}
\end{equation}
which can be accomplished by computing a singular value decomposition of the matrix $\tilde{F}$.

\subsubsection{Image Rectification}
Given a pair of stereo images, epipolar rectification is a transformation of each image plane such that all corresponding epipolar lines become colinear and parallel to one of the image axes, for convenience usually the horizontal axis. The resulting rectified images can be thought of as acquired by a new stereo camera obtained by rotating the original cameras about their optical centers. The great advantage of the epipolar rectification is the correspondence search becomes simpler and computationally less expensive because the search is done along the horizontal lines of the rectified images. The steps of the epipolar rectification algorithm are illustrated in Figure \ref{fig:rect}. Observe that after the rectification, all the epipolar lines in the left and right image are colinear and horizontal.
For an in-depth discussion on algorithms for image rectification see \cite{Fusiello2000}\cite{LoopZhang1999}.
\begin{figure}[ht]
  \begin{center}
	\includegraphics[width=0.7\textwidth]{tex/figs/ch09_figs/rectification.png}  \end{center}
  \caption{Epipolar rectification example from Loop et al. (1999).}
  \label{fig:rect}
\end{figure}

\subsubsection{Correspondence Problem}
Epipolar constraints and image rectification are commonly used in stereo vision to address the problem of correspondence, which is the problem of determining the pixels $p$ and $p'$ from two different cameras with different perspectives that correspond to the same scene feature $P$. While these concepts make finding correspondences easier, there are still several challenges that must be overcome. These include challenges related to feature occlusions, repetitive patterns, distortions, and others.

\subsubsection{Reconstruction Problem}

\begin{figure}[ht]
\centering
\includegraphics[width=0.9\textwidth]{tex/figs/ch09_figs/triangulation.png}
\caption{Triangulation with rectified images (horizontal view on the left, top-down view on the right).}
\label{fig:recttri}
\end{figure}
In a stereo vision setup, once a correspondence between the two images is identified it is possible to reconstruct the 3D scene point based on \textit{triangulation}. This process of triangulation has already been covered by the discussion on the epipolar geometry. However if the images have also be rectified such that the epipolar lines become parallel to the horizontal image axis the triangulation problem becomes simpler. This occurs, for example, when the two cameras have the same orientation, are placed with their optical axes parallel, and are separated by some distance $b$ called the \textit{baseline} (see Figure \ref{fig:recttri}).

In Figure \ref{fig:recttri}, a point $P$ on the object is described as being at coordinate $(x,y,z)$ with respect to the origin located in the left camera at point $O$. The horizontal pixel coordinate in the left and right image are denoted by $p_u$ and $p'_u$ respectively. Based on the geometry the depth of the point $P$ can be computed from the properties of similar triangles:
\begin{align}
  \frac{z}{b} &= \frac{z-f}{b-p_u+p'_u},
\end{align}
which can be algebraically simplified to:
\begin{equation}
    z = \frac{bf}{p_u-p'_u},
\end{equation}
where $f$ is the focal length. Generally a small baseline $b$ will lead to larger depth errors, but a large baseline $b$ may cause features to be visible from one camera but not the other. The difference in the image coordinates, $p_u-p'_u$, is referred to as \textit{disparity}. This is an important term in stereo vision, because it is only by measuring disparity that depth information can be recovered. The disparity can also be visually represented in a \textit{disparity map} (for example see Figure \ref{fig:disparity}), which is simply a map of the disparity values for each pixel in an image. The largest disparities occur from nearby objects (i.e. since disparity is inversely proportional to $z$).
\begin{figure}[ht]
\centering
\includegraphics[width=0.95\textwidth]{tex/figs/ch09_figs/disparitymap.png}
\caption{Disparity map from a pair of stereo images. Notice that the lighter values of the disparity map represent larger disparity, and correspond to the point in the scene that are closer to the cameras. The black points represent points that were occluded from one of the images and therefore no correspondence could be made. Images from Scharstein et al. (2003) \nocite{ScharsteinSzeliski2003}.}
\label{fig:disparity}
\end{figure}

\subsection{Structure From Motion (SFM)}
The structure from motion (SFM) method uses a similar principle as stereo vision, but uses \textit{one} camera to capture multiple images from different perspectives while moving within the scene. In this case, the intrinsic camera parameter matrix $K$ will be constant, but the extrinsic parameters (i.e. the rotation matrix $R$ and relative position vector $t$) will be different for each image.
\begin{figure}[ht]
  \begin{center}
	\includegraphics[width=0.75\textwidth]{tex/figs/ch09_figs/sfm.png}  \end{center}
  \caption{A depiction of the structure from motion (SFM) method. A single camera is used to take multiple images from different perspectives, which provides enough information to reconstruct the 3D scene.}
  \label{sfm}
\end{figure}
Consider a case where $m$ images of $n$ fixed 3D points are taken from different perspectives. This would involve $m$ homography matrices $M_k$ and $n$ 3D points $P_j$ that would need to be determined by leveraging the relationships:
\begin{equation*}
    p_{j,k}^h = M_k P^h_j, \quad j = 1,\dots,n, \quad k=1,\dots,m.
\end{equation*}

However, SFM also has some unique disadvantages, such as an ambiguity in the absolute scale of the scene that cannot be determined. For example a bigger object at a longer distance and a smaller object at a closer distance may yield the same projections.

One application of the SFM concept is known as \textit{visual odometry}. Visual odometry estimates the motion of a robot by using visual inputs (and possible additional information). This approach is commonly used in practice, for example by rovers on Mars, and is useful because it not only allows for 3D scene reconstruction but also to recover the motion of the camera.


\subsection{Exponential separation of $\braket{X}$-Isomorphism-QMDD}

In this section, we separate $\braket{X}$-Isomorphism-QMDD from QMDD by giving a quantum state which requires $2^{\Omega(\sqrt{n})}$ as a QMDD, but has an $\braket{X}$-Isomorphism-QMDD with only $\mathcal O(n)$ nodes.
By $\braket{X}$-Isomorphism-QMDD, we mean that the only isomorphisms that are allowed to appear on the diagram's Isomorphism nodes are of the form $A_1\otimes\cdots\otimes A_n$ where $A_i$ is either $I=\begin{smallmat}1 & 0 \\ 0 & 1\end{smallmat}$ or $X=\begin{smallmat}0 & 1 \\ 1 & 0\end{smallmat}$.

Duris et al. show the following lower bound on the size of nondeterministic branching programs.
\begin{theorem}[\v{D}uri\v{s} et al.\cite{vdurivs2004multi}]
	\label{thm:random-vector-space-hard-for-bdd}
	The characteristic function $f_V$ of a randomly chosen vector space $V$ in $\mathbb F_2^n$ needs a (non-) deterministic branching program of size $2^{\Omega(n)}/(2n)$ with high probability.
\end{theorem}
For us, it suffices to use the fact that the bound holds for deterministic branching programs, because it implies the following.
% this means that the uniform superposition over $A_n$ is a quantum state which has a large QMDD.
\begin{theorem}
	For a random vector space $S\subseteq \{0,1\}^n$, the uniform superposition $\ket{S}$ has QMDDs of size $2^{\Omega(n)}/(2n)$, with
	\begin{align}
		\ket{S}= \frac{1}{\sqrt{|S|}} \sum_{x\in S}\ket{x}
	\end{align}
\end{theorem}
\begin{proof}[Proof sketch]
	The idea is that BDDs are QMDDs taking values in $\{0,1\}$.
	Conversely, whenever the amplitudes of a state $\ket{\phi}$ have values only in $\{0,z\}$ for some $z\in \mathbb C$, then, up to a phase, we have $\ket{\phi}=\ket{S}$, for some set of bitstrings $S\subseteq\{0,1\}^n$.
	In this case, the QMDD has the same structure as the BDD of the indicator function $f_S$, namely, the weights on its edges are all in $\{0,1\}$, and they have the same number of nodes.
	By taking $S$ to be a random vector space, the result follows from \autoref{thm:random-vector-space-hard-for-bdd} because all BDDs are branching programs.
\end{proof}

On the other hand, these states are compactly represented by $\braket{X}$-Isomorphism QMDDs, because they are stabilizer states.
For context, we note that the theorem proved by \v{D}uri\v{s} et al. is much stronger than what we need.
Namely, they show that, even if the qubits do not need to be ordered, and even if the diagram is allowed to flip nondeterministic coins, then it still holds that almost all vector spaces have exponential-size diagrams.

\paragraph{Vector spaces and stabilizer states}
A \emph{vector space} of $\{0,1\}^n$ is a set $S\subseteq\{0,1\}^n$ which contains the bitstring $0\in S$, and is closed under bitwise XOR, i.e., for each $a,b\in S$, it holds that $(a\oplus b)\in S$.
Each vector space has a basis, and has exactly $2^k$ elements for some $1\leq k\leq n$.
The uniform superposition over $S$ is the $n$-qubit state $\ket{S}_n$,
\begin{align}
	\ket{S}_n =\frac{1}{\sqrt{|S|}} \sum_{x\in S}\ket{x}
\end{align}
\begin{theorem}
If $S$ is a vector space, then $\ket{S}$ is a stabilizer state.
\end{theorem}
\begin{proof}
	For $n=1$, the statement holds trivially.

	For $n>1$,

	(Zelf denk ik dat de volgende route het snelst is: Een vector space bestaat uit de oplossingen van een stelsel lineaire vergelijkingen. Neem de eerste variabele, $x_1$. Dan kan je kijken naar $x_1:=0$ en $x_1:=1$, en dan krijg je weer twee stelsels lineaire vergelijkingen. Maar die zijn "hetzelfde", als ze allebei oplossingen hebben, namelijk de ene krijg je door de oplossingen van de ander te XORen met een slim gevonden bitstring. Deze uitleg is heel slecht).
\end{proof}


%---------------------------Edge Ratio-----------------------------
\section{Edge Ratio}

The edge ratio of a triangle is:
\[
\frac{L_{\max}}{L_{\min}}.
\]

\trimetrictable{edge ratio}%
{$1$}%                                      Dimension
{$[1,1.3]$}%                                Acceptable range
{$[1,DBL\_MAX]$}%                           Normal range
{$[1,DBL\_MAX]$}%                           Full range
{$1$}%                                      Square
{\cite{pebay:03}}%                          Citation
{v\_tri\_edge\_ratio}%                            Verdict function name


\subsection{Probability Calculation}
\def\E{\mathbb{E}}
\begin{frame}{Universal Hashing}{Probability Calculation}
  \begin{columns}
    \begin{column}{0.65\textwidth}
      \begin{itemize}
        \item<2->
          We just discuss the discrete case
        \item<3->
          Probability space {\color{MainA}$\Omega$} with {\color{MainA}elementary (simple) events}
        \item<4->
          Events {\color{MainA}$e$} have probabilities $\ldots$\\
          {\color{MainA}\[\sum_{e \in \Omega} P(e) = 1\]}
        \item<5->
          The probability for a subset of events
          {\color{MainA}$E \subseteq \Omega$} is
          {\color{MainA}\[P(E) = \sum_{e \in E} P(e) \mid e \in E\]}
      \end{itemize}
    \end{column}
    \begin{column}{0.35\textwidth}
      \onslide <6->
      \begin{table}[!h]
        \caption{throwing a dice}
        \label{tab:probabilities:rolling_dice}
        \begin{tabularx}{0.5\linewidth}{c|c}
          {\color{MainA}$e$} & {\color{MainA}$P(e)$}\\
          \midrule
          1 & $\sfrac{1}{6}$\\
          2 & $\sfrac{1}{6}$\\
          3 & $\sfrac{1}{6}$\\
          4 & $\sfrac{1}{6}$\\
          5 & $\sfrac{1}{6}$\\
          6 & $\sfrac{1}{6}$\\
        \end{tabularx}
      \end{table}
    \end{column}
  \end{columns}
\end{frame}

%-------------------------------------------------------------------------------

\begin{frame}{Universal Hashing}{Probability Calculation}
  \begin{columns}
    \begin{column}{0.65\linewidth}
      \textbf{Example:}
      \begin{itemize}
        \item<2->
          Rolling a dice twice ({\color{MainA}$\Omega = \{1,\dots,6\}^2$})
        \item<3->
          Each event {\color{MainA}$e \in \Omega$} has the probability
          {\color{MainA}$P(e) = \sfrac{1}{36}$}
        \item<4->
          {\color{MainA}$E =$} if both results are even,
          then {\color{MainA}$P(E)=$}
      \end{itemize}
    \end{column}
    \onslide<3->
    \begin{column}{0.35\linewidth}
      \begin{table}[!h]
        \caption{throwing a dice twice}
        \label{tab:probabilities_rolling_dice_twice}
        \begin{tabularx}{0.8\linewidth}{c|c}
          {\color{MainA}$e$} & {\color{MainA} $P(e)$}\\
          \midrule
          $(1, 1)$ & $\sfrac{1}{36}$\\
          $(1, 2)$ & $\sfrac{1}{36}$\\
          $(1, 3)$ & $\sfrac{1}{36}$\\
          $\dots$ & $\dots$\\
          $(6, 5)$ & $\sfrac{1}{36}$\\
          $(6, 6)$ & $\sfrac{1}{36}$\\
        \end{tabularx}
      \end{table}
    \end{column}
  \end{columns}
\end{frame}

%-------------------------------------------------------------------------------

\begin{frame}{Universal Hashing}{Probability Calculation}
  \begin{columns}
    \begin{column}{0.55\linewidth}
      \textbf{Example:}
      \begin{itemize}
      \setlength\itemsep{1em}
      \item<1->
        Random variable
        \begin{itemize}
        \item<2->
          Assigns a number to the result of an experiment
        \item<3->
          For example: {\color{MainA}$X =$}
          Sum of results for rolling twice
        \item<4->
          {\color{MainA}$X = 12$} and {\color{MainA}$X \geq 7$}
          are regarded as events
        \item<5->
          Example 1: {\color{MainA}$P(X = 2) = $}
        \item<6->
          Example 2: {\color{MainA}$P(X = 4) = $}
        \end{itemize}
      \end{itemize}
    \end{column}
    \onslide<4->
    \begin{column}{0.45\linewidth}
      \begin{table}[!h]
        \caption{throwing a dice twice}
        \label{tab:probabilities:rolling_dice_twice2}
        \begin{tabularx}{0.95\linewidth}{c|cc}
          {\color{MainA}$e$} & {\color{MainA}$P(e)$} &
          {\color{MainA}$X$}\\
          \midrule
          $(1, 1)$ & $\sfrac{1}{36}$ & 2\\
          $(1, 2)$ & $\sfrac{1}{36}$ & 3\\
          $(1, 3)$ & $\sfrac{1}{36}$ & 4\\
          $\dots$ & $\dots$ & $\dots$\\
          $(6, 5)$ & $\sfrac{1}{36}$ & 11\\
          $(6, 6)$ & $\sfrac{1}{36}$ & 12\\
        \end{tabularx}
      \end{table}
    \end{column}
  \end{columns}
\end{frame}

%-------------------------------------------------------------------------------

%% \begin{frame}{Universal Hashing}{Probability Calculation}
%%   \begin{columns}
%%     \begin{column}{0.55\linewidth}
%%       \textbf{Example:}
%%       \vspace{1em}
%%       \begin{itemize}
%%         \item
%%           {\color{MainA}$E = \{e_{(i,\,j)} \in \Omega
%%           \mid X \;\mathrm{mod}\; 2 = 0\}$}\\
%%           {\color{MainA}$P(E) =$}
%%       \end{itemize}
%%     \end{column}
%%     \begin{column}{0.45\linewidth}
%%       \begin{table}[!h]
%%         \caption{Throwing a dice twice}
%%         \label{tab:probabilities:rolling_dice_twice3}
%%         \begin{tabularx}{0.95\linewidth}{c|cc}
%%           {\color{MainA}$e_{(i,\,j)}$} & {\color{MainA}$P(e_{(i,\,j)})$} &{\color{MainA} $X = i + j$}\\
%%           \midrule
%%           $(1, 1)$ & $\sfrac{1}{36}$ & 2\\
%%           $(1, 2)$ & $\sfrac{1}{36}$ & 3\\
%%           $(1, 3)$ & $\sfrac{1}{36}$ & 4\\
%%           $\dots$ & $\dots$ & $\dots$\\
%%           $(6, 5)$ & $\sfrac{1}{36}$ & 11\\
%%           $(6, 6)$ & $\sfrac{1}{36}$ & 12\\
%%         \end{tabularx}
%%       \end{table}
%%     \end{column}
%%   \end{columns}
%% \end{frame}

%-------------------------------------------------------------------------------

\begin{frame}{Universal Hashing}{Probability Calculation}
  \onslide<1->
  \textbf{Expected value} is defined as  {\color{MainA}$\E(X)  = \sum \left(k \cdot P(X = k)\right)$}
  \vspace*{0em}
  \begin{itemize}
    \item <2->
      Intuitive: the weighted average of possible values of {\color{MainA}$X$}, where
      the weights are the probabilities of the values
  \end{itemize}
  \vspace*{-1.0em}
  \onslide <3->
 \begin{columns}
   \begin{column}{0.5\linewidth}
     \begin{table}[!h]
       \small{
     \caption{throwing a dice once}
    \label{tab:probabilities:value_rolling_dice_once}
    \begin{tabularx}{0.25\linewidth}{c|cc}
      {\color{MainA}$X$} & {\color{MainA}$P(X)$}\\
      \midrule
      1 & $\sfrac{1}{6}$\\
      2 & $\sfrac{1}{6}$\\
      3 & $\sfrac{1}{6}$\\
      4 & $\sfrac{1}{6}$\\
      5 & $\sfrac{1}{6}$\\
      6 & $\sfrac{1}{6}$\\
    \end{tabularx}}
  \end{table}
   \end{column}
   \begin{column}{0.5\linewidth}
     \begin{table}[!h]
       \small{
    \caption{throwing a dice twice}
    \label{tab:probabilities:value_rolling_dice_twice}
    \begin{tabularx}{0.275\linewidth}{c|cc}
      {\color{MainA}$X$ }&{\color{MainA} $P(X)$}\\
      \midrule
      2 & $\sfrac{1}{36}$\\
      3 & $\sfrac{2}{36}$\\
      4 & $\sfrac{3}{36}$\\
      $\dots$ & $\dots$\\
      11 & $\sfrac{2}{36}$\\
      12 & $\sfrac{1}{36}$\\
    \end{tabularx}}
   \end{table}
   \end{column}
 \end{columns}
 \begin{itemize}
 \vspace*{-1.0em}
 \item<4-> Example "rolling once": \onslide<5->
  {\color{MainA}$\E(X) = 1 \cdot \frac{1}{6} + 2 \cdot \frac{1}{6}
    + \dots + 6 \cdot \frac{1}{6} = 3.5$}
 \item<6-> Example "rolling twice": \rlap{\onslide<7->{\color{MainA}$\E(X) = 2 \cdot \frac{1}{36} + 3 \cdot \frac{2}{36}    + \dots + 12 \cdot \frac{1}{36} = 7$}}
   \end{itemize}
\end{frame}

%% \begin{frame}{Universal Hashing}{Probability Calculation}
%%   \textbf{Expected value:}
%%   {\color{MainA}\[E(X)
%%     = \sum \left(k \cdot P(X = k)\right)\]}
%%   \begin{itemize}
%%     \item
%%       The weighted average of all possible resulting values {\color{MainA}$X$}
%%     \item
%%       The weight factor is the result value {\color{MainA}$X$} itself
%%   \end{itemize}
%% \end{frame}

%% %-------------------------------------------------------------------------------

%% \begin{frame}{Universal Hashing}{Probability Calculation}
%%   \vspace*{-1.5em}
%%   \begin{table}[!h]
%%     \caption{Throwing a dice once}
%%     \label{tab:probabilities:value_rolling_dice_once}
%%     \begin{tabularx}{0.25\linewidth}{c|cc}
%%       {\color{MainA}$X$} & {\color{MainA}$P(X)$}\\
%%       \midrule
%%       1 & $\sfrac{1}{6}$\\
%%       2 & $\sfrac{1}{6}$\\
%%       3 & $\sfrac{1}{6}$\\
%%       4 & $\sfrac{1}{6}$\\
%%       5 & $\sfrac{1}{6}$\\
%%       6 & $\sfrac{1}{6}$\\
%%     \end{tabularx}
%%   \end{table}
%%   \onslide<1>
%%   Throwing the dice once:
%%   {\color{MainA}\[E(X) = 1 \cdot \frac{1}{6} + 2 \cdot \frac{1}{6}
%%     + \dots + 6 \cdot \frac{1}{6} = 3.5\]}
%% \end{frame}

%% %-------------------------------------------------------------------------------

%% \begin{frame}{Universal Hashing}{Probability Calculation}
%%   \vspace*{-1.5em}
%%   \begin{table}[!h]
%%     \caption{Throwing a dice twice}
%%     \label{tab:probabilities:value_rolling_dice_twice}
%%     \begin{tabularx}{0.275\linewidth}{c|cc}
%%       {\color{MainA}$X$ }&{\color{MainA} $P(X)$}\\
%%       \midrule
%%       2 & $\sfrac{1}{36}$\\
%%       3 & $\sfrac{2}{36}$\\
%%       4 & $\sfrac{3}{36}$\\
%%       $\dots$ & $\dots$\\
%%       11 & $\sfrac{2}{36}$\\
%%       12 & $\sfrac{1}{36}$\\
%%     \end{tabularx}
%%   \end{table}
%%   Throwing the dice twice:
%%   {\color{MainA}\[E(X) = 2 \cdot \frac{1}{36} + 3 \cdot \frac{2}{36}
%%     + \dots + 12 \cdot \frac{1}{36} = 7\]}
%% \end{frame}

%% %-------------------------------------------------------------------------------

\begin{frame}{Universal Hashing}{Probability Calculation}
   \textbf{Sum of expected values:}
   for arbitrary discrete random variables {\color{MainA}$X_1,\dots,X_n$} we can write:
     {\color{MainA}\[\E\left(X_1+\dots+X_n\right)
       = \E(X_1) + \dots + \E(X_n)\]}
   \onslide<2->
   \textbf{Example:} throwing two dice
   \begin{itemize}
     \item<3->
       {\color{MainA}$X_1$}: result of dice {\color{MainA}$1$}: {\color{MainA}$\E(X_1) = 3.5$}
     \item<4->
       {\color{MainA}$X_2$}: result of dice {\color{MainA}$2$}: {\color{MainA}$\E(X_2) = 3.5$}
     \item<5->
       {\color{MainA}$X = X_1 + X_2$}:  total number
     \item<5-> Expected number when rolling two dices:
       {\color{MainA}\[\E(X)
         = \E(X_1 + X_2)
         = \E(X_1) + \E(X_2) = 3.5 + 3.5 = 7\]}
   \end{itemize}
\end{frame}

%% %-------------------------------------------------------------------------------

\begin{frame}{Universal Hashing}{Probability Calculation}
  \onslide<1->
  \begin{block}{Corollary:}
    The probability of the event $E$ is $p = P(E)$.
    Let $X$ be the occurences of the event $E$ and $n$ be the number
    of executions of the experiment. Then {\color{MainA}$\E(X) = n \cdot P(E) = n \cdot p$}
  \end{block}
  \onslide<2->
  \begin{example}[Rolling the dice 60 times]
    \[\E\left(\text{occurences of 6}\right) = \frac{1}{6} \cdot 60 = 10\]
  \end{example}
\end{frame}

%-------------------------------------------------------------------------------

\begin{frame}{Universal Hashing}{Probability Calculation}
  \begin{proof}[Proof Corollary:]
    Indicator variable: $X_i$\\
    \vspace*{-1.5em}
    \onslide<2->
    \begin{align*}
      X_i &=
        \left\lbrace
          \begin{array}{ll}
            1, & \text{if event occurs}\\
            0, & \text{else}
          \end{array}
          \right. \hspace{1.5em}\\
        \Rightarrow \; X = \sum_{i=1}^{n} X_i
    \end{align*}
    \vspace*{-1.0em}
    \onslide<3->
    \begin{align*}
      \E(X) &= \E\left(\sum_{i=1}^{n} X_i\right)
        = \sum_{i=1}^{n} \E(X_i)
        \stackrel{\text{def. $\E$-value}}{=}
        \sum_{i=1}^{n} p = n \cdot p
    \end{align*}
    \qedhere
    \onslide<4->
    \vspace*{-1.0em}
    \begin{align*}
        \text{Def. $\E$-value: }  \E(X_i) &= \; 0 \cdot P(X_i = 0) + 1 \cdot P(X_i = 1) &= \; P(X_i = 1) \\[0.5em]
    \end{align*}
  \end{proof}
\end{frame}


\documentclass[../main.tex]{subfiles}

\begin{document}
        \section{Vector-Valued Functions}
        \begin{definition}{Vector-Valued Functions}{}
                A vector valued function is a function \(\vec{r}:I \to \mathbb{R}^n\), where \(I \subseteq \mathbb{R}\). A vector valued function in \(\mathbb{R}^3\) may be written in the form
                \[
                \vec{r}(t)=x(t)\i + y(t)\j + z(t)\k
                \]
        \end{definition}
        \begin{definition}{Continuity}{}
                A vector valued function \(\vec{r}(t)=x(t)\i + y(t)\j + z(t)\k
\) is \emph{continuous} on \(I\) if and only if \(x(t)\), \(y(t)\), and \(z(t)\) are all continuous on \(I\).
        \end{definition}
        \begin{definition}{Velocity \& Acceleration}{}
                Given some vector valued function \(\vec{r}(t)\), there exists a corresponding \emph{velocity} \(\vec{v}(t)\) equal to \(\vec{r}'(t)\) and a corresponding \emph{acceleration} \(\vec{a}(t)\) equal to \(\vec{r}''(t)\), or equivalently \(\vec{v}'(t)\). The magnitude of \(\vec{v}(t)\) is a scalar quantity sometimes called the \emph{speed} of \(\vec{r}\).
        \end{definition}
        A stright line is described by the parameterization
        \begin{equation*}
                \vec{r}(t)=\vec{r}_0 + t\vec{v}
        \end{equation*}
        but may also be equivalently parameterized like
        \begin{equation*}
                \vec{r}(t)=\vec{r}_0 + 2t\vec{v}
        \end{equation*}
        which describes the same curve, but has a different velocity. Generally, the parameterization of a curve is not unique.

        A helix is descibed by the parameterization
        \begin{equation*}
                \vec{r}(t) = \cos{t}\i + \sin{t}\j + t\k
        \end{equation*}
        which corresponds to uniform circular motion in the x-y plane coupled with downward motion in the z direction with constant velocty, resulting in a curve with a helical shape. \(z(t)\) may be modified to change the ``compression" of the helix. For instance,
        \begin{equation*}
                \vec{r}(t) = \cos{t}\i + \sin{t}\j + t^3\k
        \end{equation*}
        is much less compressed at large values of \(z\), and highly compressed near \(0\).
        \begin{example}{}{}
        Parameterize the curve of intersection of \(x^2+y^2=9\) with \(z = x + y\).
        \tcblower
        In the x-y plane, the path is circular, so we have
        \begin{align*}
                x &= 3\cos{t}\\
                y &= 3\sin{t}
        \end{align*}
        and due to the fact that the curve lies on the surface \(z = x + y\), it must be that
        \[z = 3\cos{t} + 3\sin{t}\]
        which is a parameterization on the interval \([0,2\pi]\).
        \end{example}
        \begin{example}{}{}
        Parameterize the intersection of \(z = x^2 + y^2\) and \(x = 2y\).
        \tcblower
        The intersection of these curves is a parabola, so we may simply set
        \[y=t\]
        which then yields equations for \(x\) and \(z\)
        \begin{align*}
                x &= 2t\\
                z &= 5t^2
        \end{align*}
        \end{example}
        \begin{definition}{Differentiation Rules}{}
                The differentiation of vector valued functions proceeds by differentiating each of the components, so the normal rules of differentiation apply, namely
                \[(\vec{u}+\vec{v})' = \vec{u}' + \vec{v}'\]
                \[(\vec{u}\cdot\vec{v})' = \vec{u}'\cdot\vec{v}+\vec{u}\cdot\vec{v}'\]
                and similarly for the cross product
                \[(\vec{u}\times\vec{v})' = \vec{u}'\times\vec{v}+\vec{u}\times\vec{v}'\]
        \end{definition}
\end{document}


\section{The Dirac Delta Function}
\subsection{Definition}
We want to define a generalised function $\delta(x-\xi)$ with the following properties:
$$\forall x\neq\xi,\delta(x-\xi)=0,\int_{-\infty}^\infty\delta(x-\xi)\,\mathrm dx=1$$
So $\delta(x-\xi)$ can be thought as a ``function'' with an infinite spike at $x=\xi$.
Of course, it would be ridiculous to use it really as a function.
Almost always, we use it in conjunction with an integral, so we can take it as a linear operator having the property that
$$\left( \int_{-\infty}^\infty\mathrm dx\,\delta(x-\xi) \right)f(x)=\int_{-\infty}^\infty\delta(x-\xi)f(x)\,\mathrm=f(\xi)$$
\begin{note}
    The $\delta$ function is some sort of ``generalised function'', or ``distribution'' which admits rigorous mathematical formulation.
    However, this will not be discussed here.
\end{note}
We want the $\delta$ function to represent a unit point source or an impulse in physical situations.
Loosely, we can take $\delta$ as the ``limit'' of a family of well-defined functions.
For example, we can consider
$$\delta_\epsilon(x)=\frac{1}{\epsilon\sqrt{\pi}}\exp\left( -\frac{x^2}{\epsilon^2} \right)$$
So we can interpret $\delta$ as saying
$$\int_{-\infty}^\infty\delta(x)f(x)\,\mathrm dx=\lim_{\epsilon\to 0}\int_{-\infty}^\infty\delta_\epsilon(x)f(x)\,\mathrm dx=f(0)$$
which, as one can verify, works for sufficiently nice $f$.
This is known as the Gaussian approximation.
There are some other (discrete) choices of $\delta$ too, for example,
$$\delta_n(x)=\frac{n}{2}1_{|x|\le 1/n},\delta_n(x)=\frac{\sin(nx)}{\pi x}=\frac{1}{2\pi}\int_{-n}^ne^{ikx}\,\mathrm dk,\delta_n(x)=\frac{n}{2}\operatorname{sech}^2(nx)$$
\subsection{Properties}
We interpret the integral of $\delta$ to be the Heaviside function
$$H(x)=\begin{cases}
    1\text{, for $x\ge 0$}\\
    0\text{, for $x<0$}
\end{cases}=\int_{-\infty}^x\delta(t)\,\mathrm dt$$
One can verify that the integral of $\delta_n(x)=n\operatorname{sech}^2(x)/2$, that is $(\tanh(nx)+1)/2$, tends to $H(x)$ as $n\to\infty$.\\
We are gonna do something more sacrilegeous, that would be
$$\int_{-\infty}^\infty\delta^\prime(x-\xi)f(x)\,\mathrm dx=-\int_{-\infty}^\infty\delta(x-\xi)f^\prime(x)\,\mathrm dx=-f^\prime(\xi)$$
for a sufficiently nice $f$.
\footnote{And a sufficiently nice crowd of students who does not have access to life-threatening weapons. Cure yourself by checking out some rigorous theories formulated by Dirac, Schwartz and Temple.}
\begin{example}
    For the Gaussian approximation,
    $$\delta_\epsilon^\prime(x)=-\frac{2x}{\epsilon^3\sqrt{\pi}}e^{-x^2/\epsilon^2}$$
    which one can plot and have an idea of what the heck is going on with $\delta^\prime$.
\end{example}
Also, we have the sampling property
$$\int_a^bf(x)\delta(x-\xi)\,\mathrm dx=\begin{cases}
    f(\xi)\text{, for $\xi\in(a,b)$}\\
    0\text{, otherwise}
\end{cases}$$
Also $\delta$ is even and $\delta^\prime$ is odd.
In addition we have the scaling property
$$\int_{-\infty}^\infty f(x)\delta(a(x-\xi))\,\mathrm dx=\frac{1}{|a|}f(\xi)$$
and its advanced version:
If $g$ has $n$ isolated zeros as $x_1,\ldots,x_n$ with $g^\prime(x_i)\neq 0$ for all $i$, then
$$\delta(g(x))=\sum_{i=1}^n\frac{\delta(x-x_i)}{|g^\prime(x_i)|}$$
\begin{example}
    Take $g(x)=x^2-1$, then
    \begin{align*}
        \int_{-\infty}^\infty f(x)\delta(x^2-1)\,\mathrm dx&=\int_{1-\epsilon}^{1+\epsilon}\frac{f(x)}{2|x|}\delta(x-1)\,\mathrm dx+\int_{-1-\epsilon}^{-1+\epsilon}\frac{f(x)}{|2x|}\delta(x+1)\\
        &=\frac{f(1)+f(-1)}{2}
    \end{align*}
\end{example}
There is also this isolation property: $g(x)\delta(x)=g(0)\delta(x)$ given that $g$ is continuous at $0$.
\begin{example}
    We have
    $$\int_0^\infty\delta^\prime(x^2-1)x^2\,\mathrm dx=-\frac{1}{4}$$
\end{example}
\subsection{Eigenfunction Expansions}
For $-L\le x<L$, if we want to represent
$$\delta(x)=\sum_{n\in\mathbb Z}c_ne^{in\pi x/L}$$
Then the coefficients are
$$c_n=\frac{1}{2L}\int_{-L}^L\delta(x)e^{-in\pi x/L}\,\mathrm dx=\frac{1}{L}\implies \delta(x)=\frac{1}{2L}\sum_{n\in\mathbb Z}e^{in\pi x/L}$$
We obvious want to check that it has compatible properties.
Indeed, if $f(x)=\sum_{n\in\mathbb Z}d_ne^{in\pi x/L}$ on $[-L,L)$, then
\begin{align*}
    \langle f,\delta\rangle&=\int_{-L}^Lf^*(x)\delta(x)\,\mathrm dx\\
    &=\frac{1}{2L}\sum_{n\in\mathbb Z}d_n\int_{-L}^Le^{-in\pi x/L}e^{in\pi x/L}\,\mathrm dx\\
    &=\sum_{n\in\mathbb Z}d_n\\
    &=f(0)
\end{align*}
Note that we only defined $\delta$ on $[-L,L)$, so we can use the Fourier series to extend it periodically to the whole real line and obtain what is called a Dirac comb:
$$\sum_{m\in\mathbb Z}\delta(x-2mL)=\sum_{n\in\mathbb Z}e^{in\pi x/L}$$
For general eigenfunctions $\{y_n\}$, suppose we have
$$\delta(x-\xi)=\sum_{n=1}^\infty a_ny_n(x)$$
for $x,\xi\in [a,b]$, then the coefficients are
\begin{align*}
    a_n&=\left. \int_a^bw(x)y_n(x)\delta(x-\xi)\,\mathrm dx \middle/ \int_a^bwy_n^2\,\mathrm dx\right.\\
    &=\left. w(\xi)y_n(\xi) \middle/ \int_a^bwy_n^2\,\mathrm dx\right.=w(\xi)Y_n(\xi)
\end{align*}
where $Y_n$ is the normalised eigenfunctions.
So
$$\delta(x-\xi)=w(\xi)\sum_{n=1}^\infty Y_n(\xi)Y_n(x)=w(x)\sum_{n=1}^\infty Y_n(\xi)Y_n(x)$$
since, by the isolation property, $w(x)\delta(x-\xi)/w(\xi)=\delta(x-\xi)$.
In other words,
$$\delta(x-\xi)=w(x)\int_{n=1}^\infty\frac{y_n(\xi)y_n(x)}{N_n},N_n=\int_a^bwy_n^2\,\mathrm dx$$
\begin{example}
    Consider the Fourier sine series with $y(0)=y(1)=0$ and $y_n(x)=\sin n\pi x$, then we have
    $$\delta(x-\xi)=2\sum_{n=1}^\infty\sin(n\pi\xi)\sin(x\pi x)$$
    for $\xi\in (0,1)$.
    Integrate both sides over $[0,1]$ with $\xi=1/2$ gives
    $$\frac{\pi}{4}=\sum_{n=1}^\infty\frac{(-1)^{m+1}}{2m-1}$$
\end{example}
Another interesting observation to make is that if we integrate the series in previous example twice, we obtained a Green's function we've seen before:
$$G(x,\xi)=2\sum_{n=1}^\infty\frac{\sin(n\pi x)\sin(n\pi\xi)}{(n\pi)^2}$$


\section*{Ex.3.4}
\subsection*{Is multiply--Shift strongly universal}

We want to show that it is not by showing it does not hold for $c=1$. We do this by showing that the distinct keys does not hash independently.

So assume that it is strongly universal, then by definition  for any distinct keys $x,y$ and any possibly non-distinct hash values $q,r$ the probability $P((h_b(x) = q) \cap (h_b(y) = r)) = \frac{1}{m^2}>0$. As strong universality holds for any key, we pick $x=0, q=1$ and $y\neq 0$. Furthermore; it is easy to see that $h_b(0)=0$ for all values of $b$. As we assumed $h$ is strongly universal, then the keys hash independently and
$$
P((h_b(0) = 1) \cap (h_b(y) = r)) = P(h_b(0) = 1) P(h_b(y) = r) = 0 P(h_b(y) = r) = 0.
$$

Hence by assuming strong universality we have now shown
$$
0< \frac{1}{m^2} = P((h_b(0) = 1) \cap (h_b(y) = r)) = 0 P(h_b(y) = r) = 0.
$$

which is a contradiction.

As our only assumption is that $h$ is strongly universal, that assumption must be false. Hence we have proven that $h$ is \emph{not} strongly universal.


\documentclass[a4paper]{article}
\usepackage[top=35mm,left=3cm,right=3cm]{geometry}
\usepackage{graphicx}
\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{color}
\usepackage{verbatim}
\usepackage[colorlinks=true,
            linkcolor=blue,
            citecolor=blue,
            urlcolor=blue]{hyperref}

\newcommand{\todo}[1]{\textcolor{red}{\begin{center}TODO : #1\end{center}}}
\newcommand{\selfref}[1]{\href{#1}{#1}}
\setlength{\parindent}{0cm} % no indentation

\begin{document}

\title{Rate Gyro Model}
\maketitle

\section{Dynamics}

The dynamics of a rate gyroscope  takes the
form\footnote{A. Lawrence, Modern Inertial Technoogy:
              Navigation, Guidance, and Control (2nd edition),
              Mechanical Engineering Series, 1998,
              Chapter 7, Rate Gyro Dynamics (page 100)}
\begin{equation}
\label{eq:dyn:gyro}
    I_a \ddot{\theta} + c \dot{\theta} + K_{tb} \theta = H \omega,
\end{equation}
where $\theta$ is the gimbal angle, $\omega$ is the input angular velocity,
$H$ is the wheel angular momentum, $I_a$ is the gimbal moment of inertia about
the output axis (OA), $c$ is the damping constant about the OA and $K_{tb}$ is
the torsion bar spring constant. The output of the gyro is
\begin{equation}
\label{eq:gyr:output}
    \hat{y} = \theta K_{po} + B,
\end{equation}
where $K_{po}$ is the pickoff sensitivity and $B$ is the measurement bias.
The idea of a rate gyroscope is that by measuring $\hat{y}$ one can get
information about $\omega$. \\

Assume that the value of the bias $B$ is known
and consider the debiased gyro output $y = \hat{y} - B$.
Clearly, ${y}$ satisfies a differential equation similar to $\theta$:
\begin{eqnarray*}
    {y} &=& K_{po}\theta ,\\
    \dot{{y}} &=& K_{po} \dot{\theta},\\
    \ddot{{y}} &=& K_{po} \ddot{\theta} = \frac{K_{po}}{I_a}
    (H\omega - c\dot{\theta} - K_{tb}\theta)
    = \frac{K_{po}}{I_a} \left(H\omega - \frac{c}{K_{po}}
      \dot{{y}} - \frac{K_{tb}}{K_{po}} y \right),
\end{eqnarray*}
or
\begin{equation}
\label{eq:gyro:harm1}
    \ddot{{y}} + \frac{c}{I_a} \dot{{y}} + \frac{K_{tb}}{I_a} {y}
    = \frac{H K_{po}}{I_a} \omega \,.
\end{equation}
If the gyroscope is well-designed, its dynamics will be critically damped with
a gain close to $1$. This will make $y$ converge quickly to $\omega$ (more
precisely, $y(t)-\omega(t)\rightarrow 0$, $t\rightarrow 0$ when $\omega$
changes slowly or is constant). \\

By introducing
$a_1 = c/I_a$, $a_2 = K_{tb}/I_a$, $b_1 = H/(I_aK_{po})$,
we can rewrite~\eqref{eq:gyro:harm1} in the ``standard'' form
\begin{equation}
\label{eq:out:gyro}
    \ddot{y} + a_1 \dot{y} + a_2 y = b_1 \dot{u}\,.
\end{equation}
Here we also introduced $\dot{u} = \omega$, which is meant to emphasize that
the quantity to be estimated is the angular displacement $u$ instead of the
angular velocity ($\omega$). Now, the right-hand side is critically damped
with a gain of $1$ if the following holds:
\begin{equation}
\label{eq:cd}
\begin{split}
    \textrm{gain} = \frac{b_1}{a_2} = 1
    \quad , \quad
    D = a_1^2 - 4 a_2 = 0 .
\end{split}
\end{equation}
Thus, if we are interested in estimating the parameters, we only need to
estimate one parameter as the others can then be determined from~\eqref{eq:cd}.

\section{Simulation}

The behavior of the gyro is given by the ODE described by \eqref{eq:out:gyro}
with the initial condition $y(t_0) = y_0$, $\dot{y}(t_0) = \dot{y}_0$,
were $y_0,\dot{y}_0$ are the known initial conditions (usually, they would be
$(0,0)$, assuming a resting robot). \\

In order to simulate this ODE, we first transform it to a (multi-dimensional)
first order ODE. Second, we will need to deal with the issue that
the angular velocity ($\dot{u}$) is not available, just the angles ($u$). \\

To deal with this second issue, we integrate
($\ref{eq:out:gyro}$) respect to the time and get
\begin{equation}
    \dot{y} + a_1 y + a_2 \int y = b_1 u .
\end{equation}
Next, by introducing  $w_1 = \int y$ and $w_2 = y$, we  rewrite this system as:
\begin{equation} \begin{split}
    \dot{w}_1 &= w_2\,, \\
    \dot{w}_2 &= - a_1 w_2 - a_2 w_1 + b_1 u\,,
\end{split} \end{equation}
where $w_1(t_0) = w_2(t_0) = 0$. This can be further transformed into the
\begin{equation}
    \dot{w} = E w + F u
\end{equation}
first order ODE form, where $w = [w_1,w_2]^T$ and
\begin{equation}
    E = \left[ \begin{array}{cc}
            0 & 1 \\
            -a_2 & -a_1
        \end{array} \right]
        \quad , \quad
        F =
        \left[ \begin{array}{c}
            0 \\
            b_1
        \end{array} \right] .
\end{equation}

\end{document}


\section{Matrix Iteration Method}\label{sec:mim}
The matrix iteration method assumes that the natural frequencies are distinct
and well separated such that \(\omega_1 < \omega_2 < \dots < \omega_n\).
The iteration starting by selecting a trial vector \(\vec{\mathbf{X}}_{1}\),
which is then pre-multiplied by the dynamical matrix \([D]\).
The resulting column vector is then normalized, usually by making one of its
components equal to unity.
The normalized column vector is pre-multiplied by \([D]\) to obtain a third
column vector, which is normalized in the same way as before and become still
another trial column vector.
The process is repeated until the successive column vectors converge to a common
vector: the fundamental eigenvector.
According to the expansion theorem, any arbitrary \(n\)-dimensional vector
\(\vec{\mathbf{X}}_{1}\) can be expressed as linear combination of the \(n\)
orthogonal eigenvectors of the system \(\vec{\mathbf{X}}^{(i)}\).
\begin{equation}\label{eq:expasniontheorem}
	\sum_{i = 1}^{n} = c_{i}\,\vec{\mathbf{X}}^{(n)}
\end{equation}
where \(c_{i}\) are unknown constant number to be determined.
As view before, it is possible pre-multiply \(\vec{\mathbf{X}}_{1}\) by matrix
\([D]\) obtaining:
\begin{equation}\label{eq:equationmultiply}
	[D] \vec{\mathbf{X}}_{1} = 	c_{1}\, [D] \,\vec{\mathbf{X}}_{(1)} +
								c_{2}\, [D] \,\vec{\mathbf{X}}_{(2)} + \dots +
  							c_{n}\, [D] \,\vec{\mathbf{X}}_{(n)}\\
\end{equation}
In according with the equation \(\lambda\,[I]\,\vec{X} = [D]\,\vec{X}\), then
we obtain
\(\vec{X}^{(n)} = \lambda\,[I]\,\vec{X}^{(n)} = \frac{1}{\omega^{2}_{n}}
\vec{X}^{(n)}\), thus after having substituted it in the equation
\eqref{eq:equationmultiply} one comes to the:
\begin{align}\label{eq:equationsubs}
  	[D]\,\vec{\mathbf{X}}_{1} &= \vec{\mathbf{X}}_{2}\\[0.75em]
      						&= \frac{c_{1}}{\omega^{2}_{1}}\,\vec{\mathbf{X}}^{(1)} +
		\frac{c_{2}}{\omega^{2}_{2}}\,\vec{\mathbf{X}}^{(2)} + \dots +
		\frac{c_{n}}{\omega^{2}_{n}}\,\vec{\mathbf{X}}^{(n)}
\end{align}
By repeating the process we obtain, after \(r_{th}\) iteration,
\begin{align}\label{eq:equationrth}
  	[D]\,\vec{\mathbf{X}}_{r} &= \vec{\mathbf{X}}_{r+1}\\[0.75em]
      						&= \frac{c_{1}}{\omega^{2r}_{1}}\,\vec{\mathbf{X}}^{(1)} +
		\frac{c_{2}}{\omega^{2r}_{2}}\,\vec{\mathbf{X}}^{(2)} + \dots +
		\frac{c_{n}}{\omega^{2r}_{n}}\,\vec{\mathbf{X}}^{(n)}
\end{align}
Since the natural frequencies are are assumed \(\omega_1 < \omega_2 < \dots <
\omega_n\), a sufficiently large value of \(r\) yields \(\frac{1}{\omega_1^{2r}}
>> \frac{1}{\omega_2^{2r}} >> \dots >> \frac{1}{\omega_n^{2r}}\). Thus the first
term in right-hand side of equation \eqref{eq:equationrth} becomes the only
significant one.
Which means that the \((r+1)_{th}\) trial vector becomes identical to the
fundamental modal vector to within a multiplicative constant
\(\vec{\mathbf{X}}_{r} = \frac{c_1}{\omega^{2(r-1)}_{1}}\vec{\mathbf{X}}^{(1)}\).
Then the fundamental natural frequency \(\omega_1\) can be found by taking the
ratio of any two corresponding components in the vectors \(\vec{\mathbf{X}}_{r}\)
and \(\vec{\mathbf{X}}_{r+1}\):
\begin{equation}
  \omega_1^{2} \simeq \frac{\vec{\mathbf{X}}_{n,r}}{\vec{\mathbf{X}}_{n,r+1}}
\end{equation}
where \(\vec{\mathbf{X}}_{n,r}\) and \(\vec{\mathbf{X}}_{n,r+1}\) are the
\(n_{th}\) elements of vector \(\vec{\mathbf{X}}_{r}\) and
\(\vec{\mathbf{X}}_{r+1}\), respectively.
\subsection{Intermediate natural frequencies}
Once the first natural frequency \(\omega_1\) and the corresponding eigenvector
\(\vec{\mathbf{X}}^{(1)}\) are determined, it is possible proceed to find the
higher natural frequencies and the corresponding mode shapes.
To find the eigenvector \(\vec{\mathbf{X}}^{(i)}\), the previous eigenvector
\(\vec{\mathbf{X}}^{(i-1)}\) is normalized with respect to the mass matrix such
that \(\vec{\mathbf{X}}^{(i-1)^\top}\,[m]\,\vec{\mathbf{X}}^{(i-1)} = 1\).
The deflated matrix \([D_i]\) is the constructed as is used.
\begin{equation}
  [D_i] =
  [D_i] - \lambda_{i-1}\vec{\mathbf{X}}^{(i-1)}\,\vec{\mathbf{X}}^{(i-1)^\top}[m]
\end{equation}
where \([D_1] = [D]\).
Once \([D_i]\) is constructed the iterative scheme
\begin{equation}
  \vec{\mathbf{X}}^{(r+1)} = [D_i]\,\vec{\mathbf{X}}^{(r)}
\end{equation}
%
\subsection{Result}
\label{ssec:resultmim}
The whole procedure is performed for the case of free damping where natural
frequencies obtained: \(\omega_{1} = 8.27843\) [\si{\radian\per\second}],
\(\omega_{2}= 27.41224\) [\si{\radian\per\second}], \(\omega_{3} = 41.85486\)
[\si{\radian\per\second}].
The mode shapes results are shown in \eqref{eq:mimmodefree}.
\begin{equation}\label{eq:mimmodefree}
 [\mathbf{U}] = \begin{bmatrix*}[r]
	1.00000 & 1.00000 & 1.00000 \\
	0.86540 & 0.21230 &-0.85499 \\
	0.61945 &-0.30643 &-0.30390 \\
	 \end{bmatrix*}
\end{equation}
%
In the case of proportional damping natural frequencies are:
\(\omega_{1} = 8.27799\) [\si{\radian\per\second}], \(\omega_{2} = 27.40863\)
[\si{\radian\per\second}], \(\omega_{3} = 41.91576\) [\si{\radian\per\second}].
The mode shapes results are shown in \eqref{eq:mimmodeprop}.
 \begin{equation}\label{eq:mimmodeprop}
 [\mathbf{U}] = \begin{bmatrix*}[r]
	1.00000 & 1.00000 & 1.00000 \\
	0.86499 &-0.48006 &-2.46147 \\
	0.61907 &-1.28528 & 2.16954 \\
	 \end{bmatrix*}
\end{equation}
\subsection{Observation}\label{ssec:observationmim}
Although it is theoretically necessary to have \(r\rightarrow\infty\) for the
convergence of the method, in practice only a finite number of iterations
suffices to obtain a reasonably good estimate of \(\omega_{1}\).
The actual number of iterations necessary to find the value of \(\omega_1\) to
within a desired degree of accuracy depends on how closely the arbitrary trial
vector \(\vec{\mathbf{X}}_{1}\) resembles the mode and how well \(\omega_1\)
and \(\omega_2\) are separated.
The required number of iterations is less if \(\omega_2\) is very large compared
to \(\omega_{1}\).
The method has a distinct advantage in that any computational errors made do not
yield incorrect results. Any error made in pre-multiplying
\(\vec{\mathbf{X}}_{i}\) by \([D]\) results in a vector other than the desired
one, \(\vec{\mathbf{X}}_{i+1}\).
But this wrong vector can be considered as a new trial vector.
This may delay the convergence but does not produce wrong results.


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{tikz}
\setlength{\parindent}{0pt}

\newtheorem*{theorem}{Theorem}
\newtheorem*{definition}{Definition}
\newtheorem*{lemma}{Lemma}
\newtheorem*{corollary}{Corollary}
\newtheorem{example}{Example}
\newtheorem*{trick}{Trick}
\newtheorem*{question}{Question}

\title{Lecture 10: Second Derivative Test}
\author{}
\date{}

\begin{document}

\maketitle

How do we know the global maximum or minimum values of a function?

Similar to single-variable functions, the global max/min value of a
multivariable function can be the function value of:
\begin{itemize}
  \item Either a local max/min point.
  \item Or the boundary or infinity.
\end{itemize}

We know that local max/min points are critical points. Then how do we determine
the type of a critical point? Except the method we talked about in the previous
lecture, there is a systematic way to do it, called
\textbf{Second Derivative Test}.

\section{A Simple Case: Quadratic Functions}

Suppose that we have a quadratic multivariable function
\begin{equation*}
  z = f(x, y) = ax^2 + bxy + cy^2, a \neq 0
\end{equation*}
We can calculate its partial derivatives and find that $(0, 0)$ is a critical
point of the function:
\begin{gather*}
  \frac{\partial f}{\partial x} = 2ax + by \\
  \frac{\partial f}{\partial y} = 2cy + bx \\
  \frac{\partial f}{\partial x}(0, 0) = 0 \\
  \frac{\partial f}{\partial y}(0, 0) = 0 \\
\end{gather*}
The value of the function on the critical point $(0, 0)$ is
\begin{equation*}
  z = f(0, 0) = a \cdot 0^2 + b \cdot 0 \cdot 0 + c \cdot 0^2 = 0
\end{equation*}
Let's try to find the type of this critical point.
\begin{equation*}
  \begin{split}
    z &= ax^2 + bxy + cy^2 \\
      &= a(x^2 + \frac{b}{a}xy) + cy^2 \\
      &= a(x^2 + \frac{b}{a}xy + \frac{b^2}{4a^2}y^2) + (c - \frac{b^2}{4a})y^2 \\
      &= a(x + \frac{b}{2a}y)^2 + \frac{1}{4a}(4ac - b^2)y^2 \\
      &= \frac{1}{4a}[4a^2(x + \frac{b}{2a}y)^2 + (4ac - b^2)y^2] \\
  \end{split}
\end{equation*}
We can see that the quadratic function is actually a sum of two terms:
\begin{itemize}
  \item $4a^2(x + \frac{b}{2a}y)^2$, which is always greater than or equal to 0.
  \item $(4ac - b^2)y^2$, which is either always greater than or equal to 0, or
    always less than or equal to 0, depending on the sign of $(4ac - b^2)$.
\end{itemize}
Therefore, we can discuss different cases here:
\begin{itemize}
  \item If $4ac - b^2 < 0$, the two terms would have different signs, no matter
    what sign the factor $\frac{1}{4a}$ has. Therefore, starting from the
    critical point $(0, 0)$, in one direction (perpendicular to the direction
    represented by the positive term) the function value would increase, and in
    another direction (perpendicular to the direction represented by the
    negative term) the function value would decrease. Therefore, the critical
    point $(0, 0)$ is actually a \textbf{saddle point}.
  \item If $4ac - b^2 = 0$, the function is actually $a(x + \frac{b}{2a}y)^2$.
    Therefore, the function would have degenerate critical points along a
    direction, which is perpendicular to the direction represented by the left
    term, and it means along the direction the function value won't change.
  \item If $4ac - b^2 > 0$, both of the two terms are either $\leq 0$, or
    $\geq 0$, depending on the sign of the factor $\frac{1}{4a}$. Therefore,
    \begin{itemize}
      \item When $a > 0$, both terms are positive, then the critical point
        $(0, 0)$ is a \textbf{local minimum point}.
      \item When $a < 0$, both terms are negative, then the critical point
        $(0, 0)$ is a \textbf{local maximum point}.
    \end{itemize}
\end{itemize}

\section{General Cases: Second Derivative Test}

\subsection{Second Derivatives}

\textbf{Second derivatives} are partial derivatives of partial derivatives of
multivariable functions. For multivariable functions with two independent
variables, there are four second derivatives:
\begin{gather*}
  \frac{\partial^2 f}{\partial x^2} = f_{xx} \\
  \frac{\partial^2 f}{\partial x \partial y} = f_{xy} \\
  \frac{\partial^2 f}{\partial y \partial x} = f_{yx} \\
  \frac{\partial^2 f}{\partial y^2} = f_{yy} \\
\end{gather*}

A property of second derivatives is:
\begin{equation*}
  \frac{\partial^2 f}{\partial x \partial y} = \frac{\partial^2 f}{\partial y \partial x}
\end{equation*}
which means the order of partial derivatives taken doesn't affect the result.

\subsection{Rules of Second Derivative Test}

At a critical point $(x_0, y_0)$ of a multivariable function $f$, let
\begin{gather*}
  A = f_{xx}(x_0, y_0) \\
  B = f_{xy}(x_0, y_0) \\
  C = f_{yy}(x_0, y_0) \\
\end{gather*}
The conclusions are as follows:
\begin{itemize}
  \item If $AC - B^2 < 0$, then the critical point is a saddle point.
  \item If $AC - B^2 = 0$, then the type of the critical point is unknown.
  \item If $AC - B^2 > 0$, then
    \begin{itemize}
      \item If $A > 0$, then the critical point is a local minimum point.
      \item If $A < 0$, then the critical point is a local maximum point.
    \end{itemize}
\end{itemize}

We can verify that the second derivative test is consistent with the conclusions
we get from the simple quadratic function case in the previous section.
\begin{gather*}
  z = f(x, y) = ax^2 + bxy + cy^2 \\
  f_x = 2ax + by \\
  f_y = 2cy + bx \\
  f_{xx} = 2a \\
  f_{xy} = b \\
  f_{yx} = b \\
  f_{yy} = 2c \\
\end{gather*}
Therefore
\begin{gather*}
  A = f_{xx} = 2a \\
  B = f_{xy} = b \\
  C = f_{yy} = 2c \\
  AC - B^2 = 4ac - b^2 \\
\end{gather*}

\subsection{Ideas behind Second Derivative Test}

The reason why the second derivative test holds can be explained by quadratic
approximation. According to quadratic approximation, the function value around
a point $(x_0, y_0)$ can be approximated as
\begin{gather*}
  f \approx f(x_0, y_0) + f_x(x - x_0) + f_y(y - y_0) + \frac{1}{2}f_{xx}(x - x_0)^2 + f_{xy}(x - x_0)(y - y_0) + \frac{1}{2}f_{yy}(y - y_0)^2
\end{gather*}

For critical points, $f_x = f_y = 0$, hence the approximation formula becomes
\begin{gather*}
  f \approx f(x_0, y_0) + \frac{1}{2}f_{xx}(x - x_0)^2 + f_{xy}(x - x_0)(y - y_0) + \frac{1}{2}f_{yy}(y - y_0)^2
\end{gather*}
Therefore, the general case reduces to a quadratic function case.

Quadratic approximation also explains why we cannot draw any conclusion about
the critical point when $AC - B^2 = 0$, i.e. the degenerate cases. In those cases,
the function values along the direction are actually affected by the high-order
terms that we ignored in the quadratic approximation. Therefore, we do not
really know what the shape of the function graph is around the critical point.

\begin{example}
  Find the global maximum and minimum values of the function
  $f(x, y) = x + y + \frac{1}{xy}, x > 0, y > 0$.

  Solution: \\
  \begin{gather*}
    f_x = 1 - \frac{1}{x^2y} \\
    f_y = 1 - \frac{1}{xy^2} \\
    f_{xx} = \frac{2}{x^3y} \\
    f_{xy} = \frac{1}{x^2y^2} \\
    f_{yy} = \frac{2}{xy^3} \\
  \end{gather*}
  Solving the system of equations
  \begin{equation*}
    \begin{cases}
      f_x = 1 - \frac{1}{x^2y} = 0 \\
      f_y = 1 - \frac{1}{xy^2} = 0 \\
    \end{cases}
  \end{equation*}
  we find that there are only one critical point $(1, 1)$, whose value is $3$.

  According to the second derivative test,
  \begin{gather*}
    A = f_{xx}(1, 1) = 2 \\
    B = f_{xy}(1, 1) = 1 \\
    C = f_{yy}(1, 1) = 2 \\
    AC - B^2 = 3 > 0 \\
    A > 0 \\
  \end{gather*}
  Therefore, the critical point $(1, 1)$ is a local minimum point.

  Then let's check the boundary and infinity:
  \begin{gather*}
    \lim_{x \to 0}f(x, y) = \infty \\
    \lim_{y \to 0}f(x, y) = \infty \\
    \lim_{x \to \infty}f(x, y) = \infty \\
    \lim_{y \to \infty}f(x, y) = \infty \\
  \end{gather*}

  Therefore, the global minimum point is the only local minimum point $(1, 1)$,
  whose value is $3$, and the global maximum point is the boundary or infinity,
  where the function value is $\infty$.
\end{example}

\end{document}


\chapter{Some Basic Properties of Bessel Functions}\label{chap:Besselfunctions}
Below, we collect some important properties of Bessel functions that have been heavily used in solving the radiation problem with a cylindrical boundary condition.
Detailed properties of Bessel functions may be found in Ref.~\cite{Watson1995}.

Differential equations define Bessel functions are in the following general form,
\begin{align}
x^2\sdd{R(x)}{x}+x\dd{R(x)}{x}+(x^2-m^2)R(x)=0,
\end{align}
where $R(x)$ is a Bessel function with index $m$.

Recurrence relations for the first kind of Bessel functions:
\begin{align}
J_m(x)=\frac{m+1}{x}J_{m+1}(x)+\dd{J_{m+1}(x)}{x}=\frac{m-1}{x}J_{m-1}-\dd{J_{m-1}(x)}{x}.
\end{align}

Derivatives of Bessel functions:
\begin{align}
J_m^\prime(x)&=\frac{1}{2}(J_{m-1}(x)-J_{m+1}(x))\\
{H^{(1)}_m}^\prime (x) &=\frac{1}{2}({H^{(1)}_{m-1}}^\prime (x)-{H^{(1)}_{m+1}}^\prime(x)).
\end{align}


\subsection{Partial Fractions}
\noindent
If we have a function of two polynomials $f(x) = \frac{P(x)}{Q(x)}$, it's often easier to break this quotient into a sum of parts where the denominator is a linear or quadratic factor and the numerator is always a smaller degree than the denominator.

\ifodd\includeBackgroundReviewExamples\input{./backgroundReview/algebraPreCalc/partialFractions_example.tex}\fi

\noindent
One natural way to find these small denominators comes from the linear factors of the denominator where we keep quadratics with complex roots. This way, when making a common denominator, we get back the original big denominator. However, there are a few special cases we have to take care of.

\input{./backgroundReview/algebraPreCalc/linearFactors.tex}
\input{./backgroundReview/algebraPreCalc/repeatedLinearFactors.tex}
\input{./backgroundReview/algebraPreCalc/quadraticFactors.tex}
\input{./backgroundReview/algebraPreCalc/repeatedQuadraticFactors.tex}
\input{./backgroundReview/algebraPreCalc/improperFractions.tex}


\section{Expectation Maximization}
\smallskip \hrule height 2pt \smallskip

A clever method for maximizing marginal likelihood, where you alternate between computing an expectation and a maximization.

It is not magic: it is still optimizing a non-convex function with lots of local optima.  The computations are just easier.

\begin{itemize}
	\item as in GMM, the objective is:
		$$ \argmax_{\theta} \prod_i P(x^j; \theta) = \argmax \prod_j \sum_{i=1}^k P(y^j=i, x^j; \theta) $$
	\item \textbf{E step:} Compute the expectations to "fill in" the missing $y$ values according to the current parameters. \hfill \\
		For all examples $j$ and values $i$ for $y$, compute: $P(y^j = i | x^j, \theta)$.
	\item \textbf{M step:}
		Re-estimate the parameters with "weighted" MLE estimates:
		Set
			$$\theta = \argmax_{\theta} \sum_j \sum_{i=1}^k P(y^j = i | x^j, \theta) log P(y^j = i, x^j | \theta )$$
	\item this is especially useful when the E and M steps have closed form solutions.
\end{itemize}


\section{Comparison with integer factorization}
While computing discrete logarithms within a field and computing the integer factorization of a composite number are distinct problems, they share some characteristics:
\begin{itemize}
	\item Both concern cases of the small subgroup problem for finite commutative groups;
	\item Both are NP-hard problems, despite the existence of efficient algorithms on quantum computers;
	\item Both have been extensively used to construct strong encryption systems still valid today.
\end{itemize}

Furthermore, some algorithms to break integer factorization also have practical application in solving discrete logarithms, such as index-calculus. Shank and Pollard, on the other hand, have both developed different methods to solve the two problems.

Solving the discrete logarithm problem would actually solve the integer factorization problem, and vice versa, for the two following reasons:
\begin{enumerate}
	\item One system can be reduced to the other, since they both work on groups using the modulo operation;
	\item Both are assumed to be in the same computation class of NP, therefore solving any NP-hard problem would imply there exist an efficient way to consequently solve all the others.
\end{enumerate}

Since the RSA cryptography extensively rely on integer factorization, it is considered equally hard to break as the discrete logarithm. Both algorithms, therefore, are considered strong enough for safe encryption purposes and have similar performance.

The nature of Diffie-Hellman, however, makes it susceptible to man-in-the-middle attacks, since it does not authenticate involved parts during the exchange and requires additional confirmation.

RSA, on the other hand, allows digital signatures, although still needs the exchange of a public key beforehand.

\section{Final considerations}
Discrete logarithm has been proved to be secure against both active and passive attacks: the only effective way to break this problem would be to come up with new algorithms, since the key length can be arbitrarily increased to resist modern hardware.

However, there are some problems:
\begin{itemize}
	\item Is Decisional Diffie-Hellman equally hard as discrete logarithm, when dealing with preprocessing attacks?
	\item Is quantum computing an efficient way to reduce computational time?
	\item Would attacks on elliptic curves also influence discrete logarithm?
\end{itemize}

Having these open questions allow space for urther research and development, while still ensuring computation infeasibility.

Seminar - Algebraic Methods and Algorithms in Cryptology  /chapters/conclusion.tex