\subsection{Multi-class Object Classification}
% Describe here the CNN architecture.
We use for classification of point cloud data Convolutional Neural Network (CNN).
Our neural network architecture is represented in Figure \ref{fig:cnn}.
The network model is designed for multi-class classification of
point cloud objects.
% Convolutional neural network provides a promising result of object model with spatial data.
CNN effectively learns the feature recognition of planes, corners, and the edges of the object model.
Our network model is designed with 4 convolutional layers where each layer filter is expected to
recognize planes, corners, edges, and distribution of the objects per orientation.
Other layers learn the global label for the input grid from the knowledge passed by each layer.

\textbf{Input preparation:}
The 3D points \textit{(x,y,z)} of the object are projected into 2D points \textit{(x,y) or (x',y')} (Sec 3.2).
These projected object points are placed on a grid of height 7 and length 10.
This grid is divided into cells of 0.1x0.1 resulting in 70x100 cells and the information about the number of points in each cell is the input to the model.
This input on plotting as pixels will result in the object.

\textbf{Input layer:}
The network model takes a fixed-size input of NxM point counts in grid-cells. We have
used $N=70$ and $M=100$ for our model requirements. The input layer forwards each object input to the 1st layer
of convolutional.

\textbf{Convolutional Layer (input,f,s):}
There are 4 convolutional layers in our architecture to learn features on each orientation.
The first convolutional layer feed on two-dimensional input of size $N \times  M$ from
the input layer, a filter size=f and number of filters=s to be applied.

The filter-size provided to convolutional layer is $5 \times 5$ with 16 such filters to be applied on the input.
The convolutional layer uses filter f to apply convolution operation on input to create a feature map as output.
The convolutional layer learns about the spatial planes, edges, and distribution in the object,
the convolutional layer passes the output to the rectified linear unit (ReLU) before feeding
it forwards to next layers.

\textbf{Max Pooling layer:}
The pooling layer is added to reduce the dimension of the data by a factor of d.
The size of pooling layer is  $d \times  d$, in our model d = 2.
The max pooling layer is applied after each alternative convolutional
layer, it reduces the dimension of the feature map with their maximums.

\textbf{Dropout Layer:}
Dropout layer is used for reducing overfitting. As the network is executed on the iterative model with a feed forward network.
The model network tends to overfit on the data. Dropout layer drops random samples from the feature
map while training.

\textbf{Fully Connected Layer:}
As fully connected layers contain n output neurons. Each neuron of the output is a learned linear
combination of all the outputs from the previous layer, passed through a nonlinearity.
We use ReLUs to save the final output layer, where the number of outputs corresponds
to the number of class labels and a softmax nonlinearity is used to provide a probabilistic output.

% Our Network model starts with an input layer feeding to convolution layer with ReLu then
% another convolutional layer with ReLu and max pool layer then passing input to another batch of convolutional layers
% with Relu and convolutional layer with Relu and max pool, then the feature map is flattened to be feed-forwarded to
% fully connected layer with ReLu, then random samples are dropped with dropout layers
% and finally fed to a fully connected layer and softmax to classify the object into multiclass.

The combination of the above layers is used by some of the state-of-the-art models which have high accuracy on 2-D image inputs.
In order to build the model for this challenge, we have used the above layers, because the input to the model is in the form of pixels.
% Different configurations had different resulted in different models.
%To come up with the state-of-the-art architecture,

\textbf{Model:}
Single objects data is used as input for training the model and it predicts the object out of 28 classes. We started implementing a simple 2 layered convolutional neural network,
which uses two convolutional layers with rectified linear unit and a max pool layer,
drop out and a couple of fully connected layers to classify point cloud objects based upon classes.
This architecture was effective but
did not give acceptable results.
%%%%%decide%%%%%% the test accuracy on single objects is low
Further experimenting with layers explained the learning of pixel input by convolutional layer.

The experiment with 4 layered convolutional network has better performance, as they were able to learn planes, edges, and distribution of the point cloud object.
A couple of 2 convolutional layers with alternative ReLU added before output and max-pooling layer between them shows the best results while training and testing on individual objects.

The architecture was added with 2 fully connected layers with a single drop out layer to reduce overfitting.
This experimented architecture showed the best result with the training set and the evaluation platform.

\begin{figure*}[!h]
     \begin{center}
       \includegraphics[width=0.8\textwidth]{./images/object_net.pdf}
       \caption{Overview of Convolutional Neural Network Layers with 4 Convolutional Layers}
       \label{fig:cnn}
     \end{center}
\end{figure*}

\subsection{CNN with 3D point cloud Input Data}
We experimented with 3D CNN as
% dataset could be transformed into the input of 3D CNN being a 3D point cloud dataset.
3D point cloud could be transformed into the input of 3D CNN.
Our approach was to convert point clouds into a set of voxels by voxelization.
Voxelization is the process of conversion of a geometric object from its continuous geometric representation into a set of voxels that best approximates the continuous
object. We used the process to fit a bounding box voxel around the point cloud to form a voxel grid.

We designed for 4 layered 3D CNN architecture with 2 fully connected layers, a dropout layer, and a softmax layer at the end.
The voxel-grid is feed into the network as input and network predict the object out the 28 classes on which network is trained.
The 3D CNN model gave the best performance when trained on a minimal number of class attributes but failed to perform when trained on all the class.
The major reason we tracked for low performance of 3D CNN with a large number of output classes was the density of the object and its distribution.

The performance could be improved by experimenting with lower voxel size to increase more voxels for input.
This method is high resources dependent as the input size grows from several megabytes to several gigabytes, making harder to work on a standard system.
Reviewing the performance between 2D CNN and 3D CNN, the 2D CNN model was providing better performance and precision.


\vssub
\subsection{~Wave crest and height space-time extremes} \label{sub:space_time_ext}
\conthead{\ws (ISMAR, NCEP)}{Barbariol, F., Benetazzo, A., Alves, J.H.G.M.}

Space-Time (ST) extreme waves are modeled in WAVEWATCH III based on the Euler Characteristics (EC) approach, which states that for a given multi-dimensional (2-D space + time), statistically homogeneous and stationary Gaussian random wave field, the probability of exceedance of the maximal sea surface elevation is approximated by means of the mean value of the EC \citep{Fed12eu}. The ST extreme elevation model used here was formulated by \cite{Fed12sp} for Gaussian sea waves, and extended to second-order nonlinear spatial wave fields by \cite{Fed13} and spatio-temporal fields by \cite{Beet15}. The proposed ST extreme linear model was assessed with numerical simulations \citep{Baet15,Baet16}, while the extension to second-order nonlinear waves was verified using stereo imaging \citep{Fed13,Beet15}. According to those models, the probability of exceedance of the second-order nonlinear ST maximal crest height $\eta_{2ST_m}$ is approximated (for large threshold $z_2$ with respect to the standard deviation of the surface elevation $\sigma$) as:
\begin{equation}
P(\eta_{2ST_m} > z_2) \approx \left[N_{3D} \left( \frac{z_1}{\sigma} \right)^{2} + N_{2D} \left( \frac{z_1}{\sigma} \right) + N_{1D} \right] \exp{ \left( -\frac{z_1^2}{2 \sigma^2} \right)},
\label{eq_STE1}
\end{equation}
where the nonlinear threshold $z_2$ is related to its linear approximation $z_1$ via the Tayfun quadratic equation using the steepness parameter $\mu$, strictly valid in deep waters, which accounts for bandwidth effects. Parameters $N_{3D}$, $N_{2D}$, and $N_{1D}$ express the average number of 3D, 2D, and 1D waves within the ST region, respectively, and are determined from the moments $m_{ijl}$ of the directional wave spectrum $S(k, \theta)$ defined as follows:
\begin{equation}
m_{ijl}=\int k_{x}^{i}k_{y}^{j}\omega^{l}S(k,\theta)dkd\theta.
\label{eq_STE2}
\end{equation}

The average number of waves in Eq. (\ref{eq_STE1}) also depends on the size of the spatio-temporal domain, namely the spatial dimension $X$ along the mean direction of wave propagation, the spatial dimension $Y$ orthogonal to the mean direction of wave propagation, and the duration $D$. The expected value $\bar{\eta}_{2ST_m}$ (output parameter \textbf{STMAXE}, in meters) of the random variable $\eta_{2ST_m}$ is given by
\begin{multline}
\bar{\eta}_{2ST_m}=\mathbb{E}\left\{\eta_{2ST_m} \right\} = \\
	\sigma \left[(h_1+\frac{\mu}{2}h_1^{2})+
\gamma \left( h_1-\frac{2N_{3D}h_1+N_{2D}}{N_{3D} h_1^{2}+N_{2D} h_1+N_{1D}}\right)^{-1} (1+\mu h_1) \right],
\label{eq_STE3}
\end{multline}
where $\gamma \approx 0.5772$ is the Euler-Mascheroni constant, and $h_1$ is the dimensionless (with respect to the standard deviation $\sigma$) most probable (mode) extreme value, which is the largest solution of the implicit equation in $h$
\begin{equation}
\left[N_{3D} h^{2} + N_{2D} h + N_{1D} \right] \exp{ \left( -\frac{h^2}{2} \right)}=1.
\label{eq_STE4}
\end{equation}

The standard deviation $\sigma_{2_m}$ (output parameter \textbf{STMAXD}, in meters) of the crest height $\eta_{2ST_m}$ is given by:
\begin{equation}
\sigma_{2_m}=std(\eta_{2ST_m})=\sigma \frac{\pi}{\sqrt{6}} \left( h_1-\frac{2N_{3D}h_1+N_{2D}}{N_{3D} h_1^{2}+N_{2D} h_1+N_{1D}}\right)^{-1} (1+\mu h_1) .
\label{eq_STE5}
\end{equation}

The expected value of the ST extreme crest-to-trough wave height is obtained using the Quasi-Determinism (QD) model, which predicts the mean shape of ST wave groups close to the apex of their development. According to the QD model the expected value of the crest-to-trough height $\bar{H}_{1_{cm}}$ (output parameter \textbf{HCMAXE}, in meters) of the wave with linear extreme crest height $\bar{\eta}_{1ST_m}$ is expressed as
\begin{equation}
\bar{H}_{1_{cm}}=\mathbb{E}\left\{H_{1_{cm}} \right\} = \bar{\eta}_{1ST_m}(1-\psi_1^* / \sigma^2),
\label{eq_STE6}
\end{equation}
where $\psi_1^* < 0$ is the value of the first minimum of the temporal autocovariance function computed from the spectrum as
\begin{equation}
\psi_1(\tau) = \int S(\omega) \cos{(\omega \tau)} d \omega,
\label{eq_STE7}
\end{equation}
and $\bar{\eta}_{1ST_m}  \psi_1^*/\sigma^2 < 0$ is the expected displacement of the wave trough preceding or following the expected linear extreme crest height $\bar{\eta}_{1ST_m}$, which is computed using Eq. (\ref{eq_STE3}) after letting the wave steepness $\mu=0$. For a given linear group, the height $\bar{H}_{1_{cm}}$ is generally smaller than the maximum expected wave height $\bar{H}_{1_m}$ (output parameter \textbf{HMAXE}, in meters), which is computed as
\begin{equation}
\bar{H}_{1_m}=\mathbb{E}\left\{H_{1_m} \right\} = \bar{\eta}_{1ST_m}\sqrt{2(1-\psi_1^* / \sigma^2)}.
\label{eq_STE8}
\end{equation}

The effect on wave heights of second-order nonlinearities is generally small, particularly in narrow band seas, and it will be neglected in the present implementation to reduce the computational cost. Uncertainty of estimates of $\bar{H}_{1_{cm}}$ (output parameter \textbf{HCMAXD}, in meters) and $\bar{H}_{1_m}$ (output parameter \textbf{HMAXD}, in meters) are determined using the standard deviation of $\eta_{1ST_m}$ ($\sigma_{1_m}$, which is computed using Eq. (\ref{eq_STE5}) after letting the wave steepness $\mu=0$) as follows:
\begin{eqnarray}
std({H}_{1_m})&=&\sigma_{1_m}\sqrt{2(1-\psi_1^* / \sigma^2)}, \nonumber \\
std({H}_{1_{cm}})&=&\sigma_{1_m}(1-\psi_1^* / \sigma^2).
\label{eq_STE9}
\end{eqnarray}

To activate the computation of wave crest and height space-time extremes in WAVEWATCH III, the user has to specify values of the {\F MISC} namelist parameters {\code STDX}, {\code STDY} and {\code STDT} in {\file ww3\_grid.inp} different to -1 (the latter is a default value that avoids computer overheads when these parameters are not wanted). {\code STDX} and {\code STDY} are spatial dimensions over which extremes are calculated. {\code STDT} is the time length over which extremes are calculated. If {\code STDX} and {\code STDY} are left at default values (-1), but {\code STDT} has a namelist value different to default (e.g., greater than 0), then extreme values are provided over time, for a point. Conversely, if {\code STDT} is kept at default (-1) and {\code STDX} and {\code STDY} are greater than zero, instantaneous extreme values are computed over space. When all three parameters are greater than zero, space-time probabilities and values are computed.

Wave crest and height space-time extremes outputs follow the standard WAVEWATCH III parameter framework, and have to be specified as namelists or flags in {\file ww3\_shel.inp} or {\file ww3\_multi.inp}, in which case they are included in the standard gridded binary output files during a model run. Consequently, they also have to be specified in gridded output post-processors for obtaining a final human-readable form. Space-time extremes output parameters available in WAVEWATCH III are provided in Table \ref{tab:ste_parm}.

\begin{table}[h]
\begin{center} \begin{tabular}{|l|c|c|} \hline \hline
Internal Label & User-Interface Label & Description \\ \hline
STMAXE  &   MXE   & Max surface elev (STE) \\
STMAXD  &   MXES  & STD of max crest (STE) \\
HMAXE   &   MXH   & Max wave height (STE) \\
HCMAXE  &   MXHC  & Max wvhgt from crest (STE) \\
HMAXD   &   SDMH  & STD of MXH (STE) \\
HCMAXD  &   SDMHC & STD of MXHC (STE) \\ \hline
\end{tabular}
\end{center}
\caption{User-defined parameters in the computation of wave crest and height space-time extremes.}
\label{tab:ste_parm} \botline \end{table}


\section{Examples of convolution filters and performance}
In this section, we will give a brief description how convolution operations are used for image processing.
One useful description can be found in the following link:

\href{http://aishack.in/tutorials/image-convolution-examples/}{http://aishack.in/tutorials/image-convolution-examples/}

Convolutions is a technique for general signal processing. People studying electrical/electronics will tell you the near-infinite sleepless nights these convolutions have given them. Entire books have been written on this topic. And the questions and theorems that need to be proved are insurmountable. But for computer vision, we'll just deal with some simple things.

A convolution lets you do a very wide variety of things, like calculating derivatives, detecting edges, apply ing blurs, etc. And all of this is done with a "convolution kernel".

\subsection{Calculation with convolutions}
The most direct way to compute a convolution would be to use multiple for loops. But that causes a lot of repeated calculations. And as the size of the image and kernel increases, the time to compute the convolution increases quite drastically.

Techniques haves been developed to calculate convolutions rapidly. One such technique is using the Discrete Fourier Transform. It converts the entire convolution operation into a simple multiplication. Fortunately, you don't need to know the math to do this in OpenCV. It automatically decides whether to do it in frequency domain (after the DFT) or not.

\subsection{Image convolution examples}
A convolution is very useful for signal processing in general. There is a lot of complex mathematical theory available for convolutions. For digital image processing, you don't have to understand all of that. You can use a simple matrix as an image convolution kernel and do some interesting things!

\subsection{Line detection by 1D Laplacian}
With image convolutions, you can easily detect lines. Figure \ref{line:detect1} represents the  four convolutions to detect horizontal, vertical and lines at 45 and 135 degrees:
\begin{figure}[!ht]
	\centering
	\includegraphics[width=.4\textwidth]{6DL/figures/line}
	\caption{Four convolutions to detect horizontal, vertical and lines at degrees 0,90,45,135.}
	\label{line:detect1}
\end{figure}
Figure \ref{line:detect2}, \ref{line:detect4}, \ref{line:detect6} and \ref{line:detect8} plot the  0,90,45,135 lines detection that I got on an image.
%I looked for horizontal lines on the house image. The result I got for this image convolution was:
\begin{figure}[!ht]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/image.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/line-detection-0-k.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/line-detection-0.png}
	}
	\caption{~~A horizontal line detection done with convolutions.}
	\label{line:detect2}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

%\begin{figure}[H]
%	\centering
%	\subfigure[image]{
%		%\label{fig:subfig:a} %% label for first subfigure
%		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena.png}
%	}
%	%\subfigure[filter]{
%		%\label{fig:subfig:b} %% label for second subfigure
%	%	\includegraphics[width=1in]{xu-0401/line-detection-0-k.png}
%	%}
%	\subfigure[result]{
%		%\label{fig:subfig:b} %% label for second subfigure
%		\includegraphics[width=2.6in]{6DL/figures/xu-0401/lena_line_0.png}
%	}
%	\caption{~~A horizontal line detection for Lena.}
%	\label{line:detect3}
%	%\label{fig:subfig} %% label for entire figure
%\end{figure}
In Figure \ref{line:detect4}, the black background is the original result, the white background is obtained by subtracting the original result from 255, the same below.
\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/image.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/line-detection-90-k.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/line-detection-90.png}
	}
	\caption{~~A vertical line detection done with convolutions.}
	\label{line:detect4}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

%\begin{figure}[H]
%	\centering
%	\subfigure[image]{
%		%\label{fig:subfig:a} %% label for first subfigure
%		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena.png}
%	}
%	%\subfigure[filter]{
%		%\label{fig:subfig:b} %% label for second subfigure
%	%	\includegraphics[width=1.0in]{xu-0401/line-detection-90-k.png}
%	%}
%	\subfigure[result]{
%		%\label{fig:subfig:b} %% label for second subfigure
%		\includegraphics[width=2.6in]{6DL/figures/xu-0401/lena_line_90.png}
%	}
%	\caption{~~A vertical line detection for Lena.}
%	\label{line:detect5}
%	%\label{fig:subfig} %% label for entire figure
%\end{figure}

\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/image.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/line-detection-45-k.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/line-detection-45.png}
	}
	%\label{fig:subfig} %% label for entire figure
	\caption{~~A 45 degree line detection done with convolutions.}
	\label{line:detect6}
\end{figure}

%\begin{figure}[H]
%	\centering
%	\subfigure[image]{
%		%\label{fig:subfig:a} %% label for first subfigure
%		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena.png}
%	}
%	%\subfigure[filter]{
%		%\label{fig:subfig:b} %% label for second subfigure
%	%	\includegraphics[width=1.0in]{xu-0401/line-detection-45-k.png}
%	%}
%	\subfigure[result]{
%		%\label{fig:subfig:b} %% label for second subfigure
%		\includegraphics[width=2.6in]{6DL/figures/xu-0401/lena_line_45.png}
%	}
%	\caption{~~A 45 degree line detection done with convolutions}
%	\label{line:detect7}
%	%\label{fig:subfig} %% label for entire figure
%
%\end{figure}

\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/image.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/line-detection-135-k.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/line-detection-135.png}
	}
	\caption{~~A 135 degree line detection done with convolutions.}
	\label{line:detect8}
\end{figure}

%\begin{figure}[H]
%	\centering
%	\subfigure[image]{
%		%\label{fig:subfig:a} %% label for first subfigure
%		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena.png}
%	}
%	%\subfigure[filter]{
%%		%\label{fig:subfig:b} %% label for second subfigure
%%		\includegraphics[width=1.0in]{xu-0401/line-detection-135-k.png}
%	%}
%	\subfigure[result]{
%		%\label{fig:subfig:b} %% label for second subfigure
%		\includegraphics[width=2.6in]{6DL/figures/xu-0401/lena_line_135.png}
%	}
%	\caption{~~A 135 degree line detection done with convolutions.}
%	\label{line:detect9}
%	%\label{fig:subfig} %% label for entire figure
%\end{figure}

\subsection{Edge detection by 2D Laplacian operator}
The laplacian is the second derivative of the image. It is extremely sensitive to noise, so it isn't used as much as other operators. Unless, of course you have specific requirements.
\begin{figure}[!htbp]
	\centering
	\includegraphics[width=.5\textwidth]{6DL/figures/conv-laplacian.jpg}\\
\end{figure}

Here's the result of the convolution kernel without diagonals:
\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/image.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/laplace-k.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/laplace.png}
	}
	%\label{fig:subfig} %% label for entire figure
\end{figure}
\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena.png}
	}
	%	\subfigure[filter]{
	%		%\label{fig:subfig:b} %% label for second subfigure
	%		\includegraphics[width=1.0in]{xu-0401/laplace-k.png}
	%	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=2.6in]{6DL/figures/xu-0401/lena_laplace.png}
	}
	\caption{~~A laplace operator done with convolutions}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

Below is the result of the convolution kernel with diagonals.
\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/image.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/laplace-diag-k.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/laplace-diag.png}
	}
	%\label{fig:subfig} %% label for entire figure
\end{figure}
\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena.png}
	}
	%	\subfigure[filter]{
	%		%\label{fig:subfig:b} %% label for second subfigure
	%		\includegraphics[width=1.0in]{xu-0401/laplace-diag-k.png}
	%	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=2.6in]{6DL/figures/xu-0401/lena_laplace_diag.png}
	}
	\caption{~~A laplace operator including diagonals are done with convolutions}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

\subsection{The Laplacian of Gaussian}
The laplacian alone has the disadvantage of being extremely sensitive to noise. So, smoothing the image before a laplacian improves the results we get. This is done with a 5x5 image convolution kernel.
\begin{table}[!htbp]
	\centering
	\begin{tabular}{|l|l|l|l|l|}
		\hline
		0  & 0  & -1 & 0  & 0  \\ \hline
		0  & -1 & -2 & -1 & 0  \\ \hline
		-1 & -2 & 16 & -2 & -1 \\ \hline
		0  & -1 & -2 & -1 & 0  \\ \hline
		0  & 0  & -1 & 0  & 0  \\ \hline
	\end{tabular}
\end{table}

Below is the result by applying this image convolution.
\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/image.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/Laplacian-of-Gaussian-k.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/Laplacian-of-Gaussian.png}
	}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena.png}
	}
	%	\subfigure[filter]{
	%		%\label{fig:subfig:b} %% label for second subfigure
	%		\includegraphics[width=1.0in]{xu-0401/Laplacian-of-Gaussian-k.png}
	%	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=2.6in]{6DL/figures/xu-0401/lena_laplace_gaussian.png}
	}
	\caption{~~A Laplacian of Gaussian operator done with convolutions}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

\subsection{Other examples with ReLU activation}
\begin{figure}[H]
	\centering
	\subfigure[input image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=.25\textwidth]{6DL/figures/xu-0401/obama.png}
	}
	\quad
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=.25\textwidth]{6DL/figures/xu-0401/Laplacian-of-Gaussian-k.png}
	}
	\quad
	\subfigure[convolution result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=.25\textwidth]{6DL/figures/xu-0401/o_conv.png}
	}
	\\
	\subfigure[result after ReLU]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=.25\textwidth]{6DL/figures/xu-0401/o_relu.png}
	}
	\quad
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=.25\textwidth]{6DL/figures/xu-0401/blur--.png}
	}
	\quad
	\subfigure[result after average]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=.25\textwidth]{6DL/figures/xu-0401/o_avg.png}
	}
	%\caption{~~A Laplacian of Gaussian operator done with convolutions}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

%\begin{figure}[H]
%	\centering
%	\subfigure[image]{
%		%\label{fig:subfig:a} %% label for first subfigure
%		\includegraphics[width=.25\textwidth]{6DL/figures/xu-0401/trump.png}
%	}
%	\quad
%	\subfigure[filter]{
%		%\label{fig:subfig:b} %% label for second subfigure
%		\includegraphics[width=.25\textwidth]{6DL/figures/xu-0401/Laplacian-of-Gaussian-k.png}
%	}
%	\quad
%	\subfigure[filter]{
%		%\label{fig:subfig:b} %% label for second subfigure
%		\includegraphics[width=.25\textwidth]{6DL/figures/xu-0401/t_conv.png}
%	}\\
%	\subfigure[ReLU]{
%		%\label{fig:subfig:b} %% label for second subfigure
%		\includegraphics[width=.25\textwidth]{6DL/figures/xu-0401/t_relu.png}
%	}
%	\quad
%	\subfigure[filter]{
%		%\label{fig:subfig:b} %% label for second subfigure
%		\includegraphics[width=.25\textwidth]{6DL/figures/xu-0401/blur--.png}
%	}
%	\quad
%	\subfigure[average]{
%		%\label{fig:subfig:b} %% label for second subfigure
%		\includegraphics[width=.25\textwidth]{6DL/figures/xu-0401/t_avg.png}
%	}
%	%\caption{~~A Laplacian of Gaussian operator done with convolutions}
%	%\label{fig:subfig} %% label for entire figure
%\end{figure}

\iffalse
\subsection{Some other examples}

\subsubsection{Edge detection}
The above kernels are in a way edge detectors. Only difference between the kernels is that they have separate components for horizontal and vertical lines. A way to "combine" the results is to merge the convolution kernels. The new image convolution kernel looks like this:
\begin{table}[!htbp]
	\centering
	\begin{tabular}{|l|l|l|}
		\hline
		-1 & -1 & -1 \\ \hline
		-1 & 8  & -1 \\ \hline
		-1 & -1 & -1 \\ \hline
	\end{tabular}
\end{table}

Below result I got with edge detection:

\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/image.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/edge-detection-k.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/edge-detection.png}
	}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena.png}
	}
%	\subfigure[filter]{
%		%\label{fig:subfig:b} %% label for second subfigure
%		\includegraphics[width=1.0in]{xu-0401/edge-detection-k.png}
	%}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=2.6in]{6DL/figures/xu-0401/lena_edge_detection.png}
	}
	\caption{~~A edge detection done with convolutions}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

\subsubsection{The Sobel Edge Operator}
The above operators are very prone to noise. The Sobel edge operators have a smoothing effect, so they're less affected to noise. Again, there's a horizontal component and a vertical component.
\begin{figure}[!htbp]
	\centering
	\includegraphics[width=.4\textwidth]{6DL/figures/sobel}\\
\end{figure}

On applying horizontal component in image , the result was:\\

\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/image.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/sobel-detection-0-k.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/sobel-detection-0.png}
	}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena.png}
	}
%	\subfigure[filter]{
%		%\label{fig:subfig:b} %% label for second subfigure
%		\includegraphics[width=1.0in]{xu-0401/sobel-detection-0-k.png}
	%}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=2.6in]{6DL/figures/xu-0401/lena_sobel_0.png}
	}
	\caption{~~A horizontal sobel edge operator done with convolutions}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

On applying vertical component in image , the result was:
\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/image.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/sobel-detection-90-k.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/sobel-detection-90.png}
	}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena.png}
	}
%	\subfigure[filter]{
%		%\label{fig:subfig:b} %% label for second subfigure
%		\includegraphics[width=1.0in]{xu-0401/sobel-detection-90-k.png}
	%}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=2.6in]{6DL/figures/xu-0401/lena_sobel_90.png}
	}
	\caption{~~A vertical sobel edge operator done with convolutions}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

\subsubsection{Simple box blur}\footnote{The following examples are from the website, http://aishack.in/tutorials/image-convolution-examples/ }
Here's a first and simplest example. This convolution kernel has an averaging effect. So you end up with a slight blur. The image convolution kernel is:
\begin{table}[!htbp]
	\centering
	\begin{tabular}{|l|l|l|}
		\hline
		1/9 & 1/9 & 1/9 \\ \hline
		1/9 & 1/9 & 1/9 \\ \hline
		1/9 & 1/9 & 1/9 \\ \hline
	\end{tabular}
\end{table}

Note that the sum of all elements of this matrix is 1.0. This is important. If the sum is not exactly one, the resultant image will be brighter or darker.

Here's a blur that I got on an image:

\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/image.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/blur--.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/blur.png}
	}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/blur--.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena_blur.png}
	}
	\caption{~~A simple blur done with convolutions}
	%\label{fig:subfig} %% label for entire figure
\end{figure}

\subsubsection{Gaussian blur}
Gaussian blur has certain mathematical properties that makes it important for computer vision. And you can approximate it with an image convolution. The image convolution kernel for a Gaussian blur is in Table \ref{Gaussianblur}. The result in shown in Figure \ref{lena:Gaussian}.
\begin{table}[!ht]
	\centering
	\begin{tabular}{|l|l|l|l|l|l|l|}
		\hline
		0 & 0  & 0   & 5   & 0   & 0  & 0 \\ \hline
		0 & 5  & 18  & 32  & 18  & 5  & 0 \\ \hline
		0 & 18 & 64  & 100 & 64  & 18 & 0 \\ \hline
		5 & 32 & 100 & 100 & 100 & 32 & 5 \\ \hline
		0 & 18 & 64  & 100 & 64  & 18 & 0 \\ \hline
		0 & 5  & 18  & 32  & 18  & 5  & 0 \\ \hline
		0 & 0  & 0   & 5   & 0   & 0  & 0 \\ \hline
	\end{tabular}
	\caption{The image convolution kernel for a Gaussian blur.}
	\label{Gaussianblur}
\end{table}
\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/image.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/g-blut-k.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/Gaussian-blur.png}
	}
	%\caption{A Gaussian blur done with convolutions}
\end{figure}

\begin{figure}[H]
	\centering
	\subfigure[image]{
		%\label{fig:subfig:a} %% label for first subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena.png}
	}
	\subfigure[filter]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.0in]{6DL/figures/xu-0401/g-blut-k.png}
	}
	\subfigure[result]{
		%\label{fig:subfig:b} %% label for second subfigure
		\includegraphics[width=1.3in]{6DL/figures/xu-0401/lena_G_blur.png}
	}
	\caption{~~A Gaussian blur done with convolutions}
	\label{lena:Gaussian}
\end{figure}

\fi

\subsection{Summary}
These examples show  how different convolution kernels modify an image in different ways. Note that the filter plays an important role in detecting edges on an image.


\chapter[Bonus: A hint of p-adic numbers]{Bonus: A hint of $p$-adic numbers}
This is a bonus chapter meant for those
who have also read about \textbf{rings and fields}:
it's a nice tidbit at the intersection of algebra and analysis.

In this chapter, we are going to redo most of the previous chapter
with the absolute value $\left\lvert - \right\rvert$
replaced by the $p$-adic one.
This will give us the $p$-adic integers $\ZZ_p$,
and the $p$-adic numbers $\QQ_p$.
The one-sentence description is that these are
``integers/rationals carrying full mod $p^e$ information''
(and only that information).

In everything that follows $p$ is always assumed to denote a prime.
The first four sections will cover the founding definitions
culminating in a short solution to a USA TST problem.
We will then state (mostly without proof) some more
surprising results about continuous functions $f \colon \ZZ_p \to \QQ_p$;
finally we close with the famous proof of the Skolem-Mahler-Lech theorem
using $p$-adic analysis.

%In this whole post, $p$ is always a prime.
%Much of this is based off of Chapter 3A from
%\emph{Straight from the Book}.

\section{Motivation}
Before really telling you what $\ZZ_p$ and $\QQ_p$ are,
let me tell you what you might expect them to do.

In elementary/olympiad number theory,
we're already well-familiar with the following two ideas:
\begin{itemize}
	\ii Taking modulo a prime $p$ or prime $p^e$, and
	\ii Looking at the exponent $\nu_p$.
\end{itemize}

Let me expand on the first point.
Suppose we have some Diophantine equation.
In olympiad contexts, one can take an equation modulo $p$
to gain something else to work with.
Unfortunately, taking modulo $p$ loses some information:
the reduction $\ZZ \surjto \ZZ/p$ is far from injective.

If we want finer control, we could consider instead
taking modulo $p^2$, rather than taking modulo $p$.
This can also give some new information (cubes modulo $9$, anyone?),
but it has the disadvantage that $\ZZ/p^2$ isn't a field,
so we lose a lot of the nice algebraic properties that we
got if we take modulo $p$.

One of the goals of $p$-adic numbers is that we can get around
these two issues I described.
The $p$-adic numbers we introduce is going to have the following properties:
\begin{enumerate}
	\ii \textbf{You can ``take modulo $p^e$ for all $e$ at once''.}
	In olympiad contexts, we are used to picking a particular modulus and then
	seeing what happens if we take that modulus.
	But with $p$-adic numbers, we won't have to make that choice.
	An equation of $p$-adic numbers carries enough information
	to take modulo $p^e$.
	\ii \textbf{The numbers $\QQ_p$ form a field},
	the nicest possible algebraic structure:
	$1/p$ makes sense.
	Contrast this with $\ZZ/p^2$, which is not even an integral domain.
	\ii \textbf{It doesn't lose as much information}
	as taking modulo $p$ does:
	rather than the surjective $\ZZ \surjto \ZZ/p$ we have an
	\emph{injective} map $\ZZ \injto \ZZ_p$.
	\ii \textbf{Despite this, you ``ignore'' some ``irrelevant'' data}.
	Just like taking modulo $p$, you want to zoom-in on
	a particular type of algebraic information,
	and this means necessarily
	losing sight of other things.\footnote{To draw an analogy: the equation
	$ a^2 + b^2 + c^2 + d^2 = -1$
	has no integer solutions, because, well, squares are nonnegative.
	But you will find that this equation has solutions modulo any prime $p$,
	because once you take modulo $p$ you stop being able to
	talk about numbers being nonnegative.
	The same thing will happen if we work in $p$-adics:
	the above equation has a solution in $\ZZ_p$ for every prime $p$.}
\end{enumerate}
So, you can think of $p$-adic numbers as the right tool to use
if you only really care about modulo $p^e$ information,
but normal $\ZZ/p^e$ isn't quite powerful enough.

To be more concrete, I'll give a poster example now:
\begin{example}
	[USA TST 2002/2]
	For a prime $p$, show the value of
	\[ f_p(x) = \sum_{k=1}^{p-1} \frac{1}{(px+k)^2} \pmod{p^3} \]
	does not depend on $x$.
	\label{ex:token}
\end{example}
Here is a problem where we \emph{clearly} only care
about $p^e$-type information.
Yet it's a nontrivial challenge to do the
necessary manipulations mod $p^3$ (try it!).
The basic issue is that there is no good way to deal with
the denominators modulo $p^3$
(in part $\ZZ/p^3$ is not even an integral domain).

However, with $p$-adic analysis we're going to be able
to overcome these limitations and give a ``straightforward'' proof
by using the identity
\[
	\left( 1 + \frac{px}{k} \right)^{-2}
	= \sum_{n \ge 0} \binom{-2}{n} \left( \frac{px}{k} \right)^n.
\]
Such an identity makes no sense over $\QQ$ or $\RR$
for convergence reasons,
but it will work fine over the $\QQ_p$, which is all we need.

\section{Algebraic perspective}
\prototype{$-1/2 = 1 + 3 + 3^2 + 3^3 + \dots \in \ZZ_3$.}
We now construct $\ZZ_p$ and $\QQ_p$.
I promised earlier that a $p$-adic integer will let you look at
``all residues modulo $p^e$'' at once.
This definition will formalize this.

\subsection{Definition of $\ZZ_p$}
\begin{definition}
	[Introducing $\ZZ_p$]
	A \vocab{$p$-adic integer} is a sequence
	\[ x = (x_1 \bmod p, \; x_2 \bmod{p^2}, \; x_3 \bmod{p^3}, \; \dots) \]
	of residues $x_e$ modulo $p^e$ for each integer $e$,
	satisfying the compatibility relations
	$x_i \equiv x_j \pmod{p^i}$ for $i < j$.

	The set $\ZZ_p$ of $p$-adic integers forms a ring
	under component-wise addition and multiplication.
\end{definition}

\begin{example}
	[Some $3$-adic integers]
	Let $p=3$.
	Every usual integer $n$ generates
	a (compatible) sequence of residues modulo $p^e$ for each $e$,
	so we can view each ordinary integer as $p$-adic one:
	\[ 50 = \left( 2 \bmod 3, \; 5 \bmod 9, \;
		23 \bmod{27}, \; 50 \bmod{81}, \; 50 \bmod{243}, \; \dots \right). \]
	On the other hand, there are sequences of residues
	which do not correspond to any usual integer despite
	satisfying compatibility relations, such as
	\[ \left( 1 \bmod 3, \; 4 \bmod 9, \;
		13 \bmod{27}, \; 40 \bmod{81}, \; \dots \right) \]
	which can be thought of as $x = 1 + p + p^2 + \dots$.
\end{example}
In this way we get an injective map
\[ \ZZ \injto \ZZ_p \qquad
	n \mapsto \left( n \bmod p, n \bmod{p^2}, n \bmod{p^3}, \dots \right) \]
which is not surjective.
So there are more $p$-adic integers than usual integers.

(Remark for experts:
those of you familiar with category theory might recognize
that this definition can be written concisely as
\[ \ZZ_p \defeq \varprojlim \ZZ/p^e \ZZ \]
where the inverse limit is taken across $e \ge 1$.)

\begin{exercise}
	Check that $\ZZ_p$ is an integral domain.
\end{exercise}

\subsection{Base $p$ expansion}
Here is another way to think about $p$-adic integers using ``base $p$''.
As in the example earlier, every usual integer can be written in base $p$,
for example
\[ 50 = \ol{1212}_3 = 2 \cdot 3^0 + 1 \cdot 3^1 + 2 \cdot 3^2 + 1 \cdot 3^3. \]
More generally, given any $x = (x_1, \dots) \in \ZZ_p$,
we can write down a ``base $p$'' expansion in the sense
that there are exactly $p$ choices of $x_k$ given $x_{k-1}$.
Continuing the example earlier, we would write
\begin{align*}
	\left( 1 \bmod 3, \; 4 \bmod 9, \;
	13 \bmod{27}, \; 40 \bmod{81}, \; \dots \right)
	&= 1 + 3 + 3^2 + \dots \\
	&= \ol{\dots1111}_3
\end{align*}
and in general we can write
\[ x = \sum_{k \ge 0} a_k p^k = \ol{\dots a_2 a_1 a_0}_p \]
where $a_k \in \{0, \dots, p-1\}$,
such that the equation holds modulo $p^e$ for each $e$.
Note the expansion is infinite to the \emph{left},
which is different from what you're used to.

(Amusingly, negative integers also have infinite base $p$ expansions:
$-4 = \ol{\dots222212}_3$, corresponding to
$(2 \bmod 3, \; 5 \bmod 9, \; 23 \bmod{27}, \; 77 \bmod{81} \dots)$.)

Thus you may often hear the advertisement that a $p$-adic integer
is an ``possibly infinite base $p$ expansion''.
This is correct, but later on we'll be thinking of $\ZZ_p$ in
a more and more ``analytic'' way,
and so I prefer to think of this as
\begin{moral}
	$p$-adic integers are Taylor series with base $p$.
\end{moral}
Indeed, much of your intuition from generating functions $K[[X]]$
(where $K$ is a field) will carry over to $\ZZ_p$.

\subsection{Constructing $\QQ_p$}
Here is one way in which your intuition from generating functions carries over:
\begin{proposition}
	[Non-multiples of $p$ are all invertible]
	The number $x \in \ZZ_p$ is invertible if and only if $x_1 \ne 0$.
	In symbols,
	\[ x \in \ZZ_p^\times \iff x \not\equiv 0 \pmod p. \]
\end{proposition}
Contrast this with the corresponding statement for $K[ [ X ] ]$:
a generating function $F \in K[ [ X ] ]$ is invertible iff $F(0) \neq 0$.

\begin{proof}
	If $x \equiv 0 \pmod p$ then $x_1 = 0$,
	so clearly not invertible.
	Otherwise, $x_e \not\equiv 0 \pmod p$ for all $e$,
	so we can take an inverse $y_e$ modulo $p^e$,
	with $x_e y_e \equiv 1 \pmod{p^e}$.
	As the $y_e$ are themselves compatible,
	the element $(y_1, y_2, \dots)$ is an inverse.
\end{proof}
\begin{example}
	[We have $-\half = \ol{\dots1111}_3 \in \ZZ_3$]
	We claim the earlier example is actually
	\begin{align*}
	-\half = \left( 1 \bmod 3, \; 4 \bmod 9, \;
	13 \bmod{27}, \; 40 \bmod{81}, \; \dots \right)
	&= 1 + 3 + 3^2 + \dots \\
	&= \ol{\dots1111}_3.
	\end{align*}
	Indeed, multiplying it by $-2$ gives
	\[ \left( -2 \bmod 3, \; -8 \bmod 9, \;
		-26 \bmod{27}, \; -80 \bmod{81}, \; \dots \right)
		= 1. \]
	(Compare this with the ``geometric series''
	$1 + 3 + 3^2 + \dots = \frac{1}{1-3}$.
	We'll actually be able to formalize this later, but not yet.)
\end{example}
\begin{remark}
	[$\half$ is an integer for $p > 2$]
	The earlier proposition implies that $\half \in \ZZ_3$
	(among other things);
	your intuition about what is an ``integer'' is different here!
	In olympiad terms, we already knew $\half \pmod 3$ made sense,
	which is why calling $\half$ an ``integer''
	in the $3$-adics is correct,
	even though it doesn't correspond to any element of $\ZZ$.
\end{remark}
\begin{exercise}
	[Unimportant but tricky]
	Rational numbers correspond exactly to
	eventually periodic base $p$ expansions.
\end{exercise}

With this observation, here is now the definition of $\QQ_p$.
\begin{definition}
	[Introducing $\QQ_p$]
	Since $\ZZ_p$ is an integral domain,
	we let $\QQ_p$ denote its field of fractions.
	These are the \vocab{$p$-adic numbers}.
\end{definition}
Continuing our generating functions analogy:
\[ \ZZ_p \text{ is to } \QQ_p
	\quad\text{as}\quad
	K[[X]] \text{ is to } K((X)). \]
This means
\begin{moral}
	$\QQ_p$ can be thought of as Laurent series with base $p$.
\end{moral}
and in particular according to the earlier proposition we deduce:
\begin{proposition}
	[$\QQ_p$ looks like formal Laurent series]
	Every nonzero element of $\QQ_p$ is uniquely of the form
	\[ p^k u \qquad \text{ where } k \in \ZZ, \; u \in \ZZ_p^\times. \]
\end{proposition}
Thus, continuing our base $p$ analogy,
elements of $\QQ_p$ are in bijection with ``Laurent series''
\[ \sum_{k \ge -n} a_k p^k
	= \ol{\dots a_2 a_1 a_0 . a_{-1} a_{-2} \dots a_{-n}}_p \]
for $a_k \in \left\{ 0, \dots, p-1 \right\}$.
So the base $p$ representations of elements of $\QQ_p$
can be thought of as the same as usual,
but extending infinitely far to the left
(rather than to the right).

\begin{remark}
	[Warning]
	The field $\QQ_p$ has characteristic \emph{zero}, not $p$.
\end{remark}
\begin{remark}
	[Warning on fraction field]
	This result implies that you shouldn't think about elements of $\QQ_p$
	as $x/y$ (for $x,y \in \ZZ_p$) in practice,
	even though this is the official definition
	(and what you'd expect from the name $\QQ_p$).
	The only denominators you need are powers of $p$.

	To keep pushing the formal Laurent series analogy,
	$K((X))$ is usually not thought of as quotient of generating functions
	but rather as ``formal series with some negative exponents''.
	You should apply the same intuition on $\QQ_p$.
\end{remark}

\begin{remark}
At this point I want to make a remark about the fact $1/p \in \QQ_p$,
connecting it to the wish-list of properties I had before.
In elementary number theory you can take equations modulo $p$,
but if you do the quantity $n/p \bmod{p}$ doesn't make sense
unless you know $n \bmod{p^2}$.
You can't fix this by just taking modulo $p^2$
since then you need $n \bmod{p^3}$ to get $n/p \bmod{p^2}$, ad infinitum.
You can work around issues like this,
but the nice feature of $\ZZ_p$ and $\QQ_p$
is that you have modulo $p^e$ information for ``all $e$ at once'':
the information of $x \in \QQ_p$ packages all the modulo $p^e$
information simultaneously.
So you can divide by $p$ with no repercussions.
\end{remark}

\section{Analytic perspective}
\subsection{Definition}
Up until now we've been thinking about things mostly algebraically,
but moving forward it will be helpful to start using the language of analysis.
Usually, two real numbers are considered ``close'' if
they are close on the number of line,
but for $p$-adic purposes we only care about modulo $p^e$ information.
So, we'll instead think of two elements of $\ZZ_p$ or $\QQ_p$
as ``close'' if they differ by a large multiple of $p^e$.

For this we'll borrow the familiar $\nu_p$ from elementary number theory.
\begin{definition}
	[$p$-adic valuation and absolute value]
	We define the \vocab{$p$-adic valuation} $\nu_p : \QQ_p^\times \to \ZZ$
	in the following two equivalent ways:
	\begin{itemize}
		\ii For $x = (x_1, x_2, \dots) \in \ZZ_p$ we let
		$\nu_p(x)$ be the largest $e$ such that
		$x_e \equiv 0 \pmod{p^e}$ (or $e=0$ if $x \in \ZZ_p^\times$).
		Then extend to all of $\QQ_p^\times$
		by $\nu_p(xy) = \nu_p(x) + \nu_p(y)$.
		\ii Each $x \in \QQ_p^\times$ can be written
		uniquely as $p^k u$ for $u \in \ZZ_p^\times$, $k \in \ZZ$.
		We let $\nu_p(x) = k$.
	\end{itemize}
	By convention we set $\nu_p(0) = +\infty$.
	Finally, define the \vocab{$p$-adic absolute value}
	$\left\lvert \bullet \right\rvert_p$ by
	\[ \left\lvert x \right\rvert_p = p^{-\nu_p(x)}. \]
	In particular $\left\lvert 0 \right\rvert_p = 0$.
\end{definition}
This fulfills the promise that $x$ and $y$ are close
if they look the same modulo $p^e$ for large $e$;
in that case $\nu_p(x-y)$ is large
and accordingly $\left\lvert x-y \right\rvert_p$ is small.

\subsection{Ultrametric space}
In this way, $\QQ_p$ and $\ZZ_p$ becomes a metric space
with metric given by $\left\lvert x-y \right\rvert_p$.

\begin{exercise}
	\label{exer:alternating}
	Suppose $f \colon \ZZ_p \to \QQ_p$ is continuous
	and $f(n) = (-1)^n$ for every $n \in \ZZ_{\ge 0}$.
	Prove that $p = 2$.
\end{exercise}

In fact, these spaces satisfy a stronger
form of the triangle inequality than you are used to from $\RR$.
\begin{proposition}
	[$\left\lvert \bullet \right\rvert_p$ is an ultrametric]
	For any $x,y \in \ZZ_p$, we have the \vocab{strong triangle inequality}
	\[ \left\lvert x+y \right\rvert_p
		\le \max \left\{ \left\lvert x \right\rvert_p,
		\left\lvert y \right\rvert_p \right\}.
	\]
	Equality holds if (but not only if)
	$\left\lvert x \right\rvert_p \neq \left\lvert y \right\rvert_p$.
\end{proposition}

However, $\QQ_p$ is more than just a metric space:
it is a field, with its own addition and multiplication.
This means we can do analysis just like in $\RR$ or $\CC$:
basically, any notion such as ``continuous function'',
``convergent series'', et cetera has a $p$-adic analog.
In particular, we can define what it means for an infinite sum to converge:
\begin{definition}
	[Convergence notions]
	Here are some examples of $p$-adic analogs of ``real-world'' notions.
	\begin{itemize}
		\ii A sequence $s_1$, \dots converges to a limit $L$
		if $\lim_{n \to \infty} \left\lvert s_n - L \right\rvert_p = 0$.
		\ii The infinite series $\sum_k x_k$ converges
		if the sequence of partial sums $s_1 = x_1$,
		$s_2 = x_1 + x_2$, \dots, converges to some limit.
		\ii \dots et cetera \dots
	\end{itemize}
\end{definition}
With this definition in place,
the ``base $p$'' discussion we had earlier is now true
in the analytic sense: if $x = \ol{\dots a_2 a_1 a_0}_p \in \ZZ_p$ then
\[ \sum_{k=0}^\infty a_k p^k \quad\text{converges to } x. \]
Indeed, the difference between $x$ and the $n$th partial sum is divisible by $p^n$,
hence the partial sums approach $x$ as $n \to \infty$.

While the definitions are all the same,
there are some changes in properties that should be true.
For example, in $\QQ_p$ convergence of partial sums is simpler:
\begin{proposition}
	[$|x_k|_p \to 0$ iff convergence of series]
	A series $\sum_{k=1}^\infty x_k$ in $\QQ_p$
	converges to some limit if and only if
	$\lim_{k \to \infty} |x_k|_p = 0$.
	\label{noharmonic}
\end{proposition}
Contrast this with $\sum \frac1n = \infty$ in $\RR$.
You can think of this as a consequence of strong triangle inequality.
\begin{proof}
	By multiplying by a large enough power of $p$,
	we may assume $x_k \in \ZZ_p$.
	(This isn't actually necessary, but makes the notation nicer.)

	Observe that $x_k \pmod p$ must eventually stabilize,
	since for large enough $n$ we have
	$\left\lvert x_n \right\rvert_p < 1 \iff \nu_p(x_n) \ge 1$.
	So let $a_1$ be the eventual residue modulo $p$
	of $\sum_{k=0}^N x_k \pmod p$ for large $N$.
	In the same way let $a_2$ be the eventual residue modulo $p^2$, and so on.
	Then one can check we approach the limit $a = (a_1, a_2, \dots)$.
\end{proof}

\subsection{More fun with geometric series}
Let's finally state the $p$-adic analog of the geometric series formula.

\begin{proposition}
	[Geometric series]
	Let $x \in \ZZ_p$ with $\left\lvert x \right\rvert_p < 1$.
	Then \[ \frac{1}{1-x} = 1 + x + x^2 + x^3 + \dots. \]
\end{proposition}
\begin{proof}
	Note that the partial sums satisfy
	$1 + x + x^2 + \dots + x^n = \frac{1-x^n}{1-x}$,
	and $x^n \to 0$ as $n \to \infty$ since
	$\left\lvert x \right\rvert_p < 1$.
\end{proof}

So, $1 + 3 + 3^2 + \dots = -\half$ is really a correct convergence
in $\ZZ_3$.
And so on.

If you buy the analogy that $\ZZ_p$ is generating functions
with base $p$, then all the olympiad generating functions
you might be used to have $p$-adic analogs.
For example, you can prove more generally that:
\begin{theorem}
	[Generalized binomial theorem]
	If $x \in \ZZ_p$ and $\left\lvert x \right\rvert_p < 1$,
	then for any $r \in \QQ$ we have the series convergence
	\[ \sum_{n \ge 0} \binom rn x^n = (1+x)^r. \]
\end{theorem}
(I haven't defined $(1+x)^r$, but it has the properties you expect.)

\subsection{Completeness}
Note that the definition of $\left\lvert \bullet \right\rvert_p$
could have been given for $\QQ$ as well;
we didn't need $\QQ_p$ to introduce it
(after all, we have $\nu_p$ in olympiads already).
The big important theorem I must state now is:
\begin{theorem}
	[$\QQ_p$ is complete]
	The space $\QQ_p$ is the completion of $\QQ$
	with respect to $\left\lvert \bullet \right\rvert_p$.
\end{theorem}
This is the definition of $\QQ_p$ you'll see more frequently;
one then defines $\ZZ_p$ in terms of $\QQ_p$
(rather than vice-versa) according to
\[ \ZZ_p = \left\{ x \in \QQ_p :
	\left\lvert x \right\rvert_p \le 1 \right\}. \]

\subsection{Philosophical notes}
Let me justify why this definition is philosophically nice.
Suppose you are an ancient Greek mathematician who is given:
\begin{quote}
	\textbf{Problem for Ancient Greeks.}
	Estimate the value of the sum
	\[ S = \frac{1}{1^2} + \frac{1}{2^2} + \dots + \frac{1}{10000^2} \]
	to within $0.001$.
\end{quote}
The sum $S$ consists entirely of rational numbers,
so the problem statement would be fair game for ancient Greece.
But it turns out that in order to get a good estimate,
it \emph{really helps} if you know about the real numbers:
because then you can construct the infinite series
$\sum_{n \ge 1} n^{-2} = \frac16 \pi^2$,
and deduce that $S \approx \frac{\pi^2}{6}$,
up to some small error term from the terms past $\frac{1}{10001^2}$,
which can be bounded.

Of course, in order to have access to enough theory
to prove that $S = \pi^2/6$, you need to have the real numbers;
it's impossible to do calculus in $\QQ$
(the sequence $1$, $1.4$, $1.41$, $1.414$,
is considered ``not convergent''!)

Now fast-forward to 2002, and suppose you are given
\begin{quote}
	\textbf{Problem from USA TST 2002.}
	Estimate the sum
	\[ f_p(x) = \sum_{k=1}^{p-1} \frac{1}{(px+k)^2} \]
	to within mod $p^3$.
\end{quote}
% (i.e.\ to within $p^{-3}$ in $\left\lvert \bullet \right\rvert_p$).
Even though $f_p(x)$ is a rational number,
it still helps to be able to do analysis with infinite sums,
and then bound the error term (i.e.\ take mod $p^3$).
But the space $\QQ$ is not complete with respect
to $\left\lvert \bullet \right\rvert_p$ either,
and thus it makes sense to work in the completion of $\QQ$
with respect to $\left\lvert \bullet \right\rvert_p$.
This is exactly $\QQ_p$.

In any case, let's finally solve \Cref{ex:token}.
\begin{example}
[USA TST 2002]
We will now compute
\[ f_p(x) = \sum_{k=1}^{p-1} \frac{1}{(px+k)^2} \pmod{p^3}. \]
Armed with the generalized binomial theorem,
this becomes straightforward.
\begin{align*}
	f_p(x) &= \sum_{k=1}^{p-1} \frac{1}{(px+k)^2}
	= \sum_{k=1}^{p-1} \frac{1}{k^2}
	\left( 1 + \frac{px}{k} \right)^{-2} \\
	&= \sum_{k=1}^{p-1} \frac{1}{k^2} \sum_{n \ge 0}
	\binom{-2}{n} \left( \frac{px}{k} \right)^{n} \\
	&= \sum_{n \ge 0} \binom{-2}{n}
	\sum_{k=1}^{p-1} \frac{1}{k^2} \left( \frac{x}{k} \right)^{n} p^n \\
	&\equiv \sum_{k=1}^{p-1} \frac{1}{k^2}
	 - 2x \left( \sum_{k=1}^{p-1} \frac{1}{k^3} \right) p
	 + 3x^2 \left( \sum_{k=1}^{p-1} \frac{1}{k^4} \right) p^2 \pmod{p^3}.
\end{align*}
Using the elementary facts that
$p^2 \mid \sum_k k^{-3}$ and $p \mid \sum_k k^{-4}$,
this solves the problem.
\end{example}

%\section{Digression on $\CC_p$}
%Before I go on, I want to mention that $\QQ_p$
%is not algebraically closed.
%So, we can take its algebraic closure $\ol{\QQ_p}$ --- but this
%field is now no longer complete (in the topological sense).
%However, we can then take the completion of this space
%to obtain $\CC_p$.
%In general, completing an algebraically closed field
%remains algebraically closed,
%and so there is a larger space $\CC_p$ which
%is algebraically closed \emph{and} complete.
%This space is called the $p$-adic complex numbers.
%
%We won't need $\CC_p$ at all in what follows,
%so you can forget everything you just read.

\section{Mahler coefficients}
One of the big surprises of $p$-adic analysis is that:
\begin{moral}
	We can basically describe all continuous functions $\ZZ_p \to \QQ_p$.
\end{moral}
They are given by a basis of functions
\[ \binom xn \defeq \frac{x(x-1) \dots (x-(n-1))}{n!} \]
in the following way.
\begin{theorem}
	[Mahler; see {\cite[Theorem 51.1, Exercise 51.b]{ref:schikof}}]
	Let $f \colon \ZZ_p \to \QQ_p$ be continuous, and define
	\begin{equation}
		a_n = \sum_{k=0}^n \binom nk (-1)^{n-k} f(k).
		\label{eq:mahler}
	\end{equation}
	Then $\lim_n a_n = 0$ and \[ f(x) = \sum_{n \ge 0} a_n \binom xn. \]

	Conversely, if $a_n$ is any sequence converging to zero,
	then $f(x) = \sum_{n \ge 0} a_n \binom xn$
	defines a continuous function satisfying \eqref{eq:mahler}.
	% true for C_p too
\end{theorem}
The $a_i$ are called the \emph{Mahler coefficients} of $f$.
\begin{exercise}
	Last post we proved that if $f \colon \ZZ_p \to \QQ_p$ is continuous
	and $f(n) = (-1)^n$ for every $n \in \ZZ_{\ge 0}$ then $p = 2$.
	Re-prove this using Mahler's theorem,
	and this time show conversely that a unique such $f$ exists when $p=2$.
\end{exercise}

You'll note that these are the same finite differences that one
uses on polynomials in high school math contests,
which is why they are also called ``Mahler differences''.
\begin{align*}
	a_0 &= f(0) \\
	a_1 &= f(1) - f(0) \\
	a_2 &= f(2) - 2f(1) + f(0) \\
	a_3 &= f(3) - 3f(2) + 3f(1) - f(0).
\end{align*}
Thus one can think of $a_n \to 0$ as saying that
the values of $f(0)$, $f(1)$, \dots behave like a polynomial modulo $p^e$
for every $e \ge 0$.

The notion ``analytic'' also has a Mahler interpretation.
First, the definition.
\begin{definition}
We say that a function $f \colon \ZZ_p \to \QQ_p$ is \vocab{analytic}
if it has a power series expansion
\[ \sum_{n \ge 0} c_n x^n \quad c_n \in \QQ_p
	\qquad\text{ converging for } x \in \ZZ_p. \]
\end{definition}
\begin{theorem}
	[{\cite[Theorem 54.4]{ref:schikof}}]
	The function $f(x) = \sum_{n \ge 0} a_n \binom xn$ is analytic
	if and only if
	\[ \lim_{n \to \infty} \frac{a_n}{n!} = 0. \]
	% true for Q_p too
\end{theorem}
Analytic functions also satisfy the following niceness result:
\begin{theorem}
	[Strassmann's theorem]
	Let $f \colon \ZZ_p \to \QQ_p$ be analytic.
	Then $f$ has finitely many zeros.
	% should be true for \CC_p? gdi
\end{theorem}

Te give an application of these results,
we will prove the following result,
which was interesting even before $p$-adics came along!
\begin{theorem}
	[Skolem-Mahler-Lech]
	Let $(x_i)_{i \ge 0}$ be an integral linear recurrence,
	meaning $(x_i)_{i \ge 0}$ is a sequence of integers
	\[ x_n = c_1 x_{n-1} + c_2 x_{n-2} + \dots + c_k x_{n-k}
		\qquad n = 1, 2, \dots \]
	holds for some choice of integers $c_1$, \dots, $c_k$.
	Then the set of indices $\{ i \mid x_i = 0 \}$
	is eventually periodic.
\end{theorem}

\begin{proof}
	According to the theory of linear recurrences,
	there exists a matrix $A$ such that we can write
	$x_i$ as a dot product
	\[ x_i = \left< A^i u, v \right>. \]
	Let $p$ be a prime not dividing $\det A$.
	Let $T$ be an integer such that $A^T \equiv \id \pmod p$
	(with $\id$ denoting the identity matrix).

	Fix any $0 \le r < N$.
	We will prove that either all the terms
	\[ f(n) = x_{nT+r} \qquad n = 0, 1, \dots \]
	are zero, or at most finitely many of them are.
	This will conclude the proof.

	Let $A^T = \id + pB$ for some integer matrix $B$.
	We have
	\begin{align*}
		f(n) &= \left< A^{nT+r} u, v \right>
		= \left< (\id + pB)^n A^r u, v \right> \\
		&= \sum_{k \ge 0} \binom nk \cdot p^n \left< B^n A^r u, v \right> \\
		&= \sum_{k \ge 0} a_n \binom nk \qquad \text{ where }
			a_n = p^n \left< B^n A^r u, v \right> \in p^n \ZZ.
	\end{align*}
	Thus we have written $f$ in Mahler form.
	Initially, we define $f \colon \ZZ_{\ge 0} \to \ZZ$,
	but by Mahler's theorem (since $\lim_n a_n = 0$)
	it follows that $f$ extends to a function $f \colon \ZZ_p \to \QQ_p$.
	Also, we can check that $\lim_n \frac{a_n}{n!} = 0$
	hence $f$ is even analytic.

	Thus by Strassman's theorem, $f$ is either identically zero,
	or else it has finitely many zeros, as desired.
\end{proof}

\section{\problemhead}
\begin{dproblem}
	[$\ZZ_p$ is compact]
	Show that $\QQ_p$ is not compact, but $\ZZ_p$ is.
	(For the latter, I recommend using sequential continuity.)
\end{dproblem}
\begin{dproblem}
	[Totally disconnected]
	Show that both $\ZZ_p$ and $\QQ_p$ are \emph{totally disconnected}:
	there are no connected sets other than the empty set and singleton sets.
\end{dproblem}

\begin{problem}
	[USA TST 2011]
	Let $p$ be a prime.
	We say that a sequence of integers $\{z_n\}_{n=0}^\infty$
	is a \emph{$p$-pod} if for each $e \geq 0$,
	there is an $N \geq 0$ such that whenever $m \geq N$,
	$p^e$ divides the sum
	\[ \sum_{k=0}^m (-1)^k \binom mk z_k. \]
	Prove that if both sequences $\{x_n\}_{n=0}^\infty$
	and $\{y_n\}_{n=0}^\infty$ are $p$-pods,
	then the sequence $\{x_n y_n\}_{n=0}^\infty$ is a $p$-pod.
\end{problem}


\section{Introduction}

Matrix factorization is listed as one of the most influential set
of techniques during the 20th century ~\cite{dongarra2000guest}, among
the Fast Fourier Transform, MCMC sampling methods and others.
As Stewart ~\cite{stewart2000decompositional} argues, the principle of the
decompositional approach aims to construct computational platforms from
which a variety of problems can be solved.

Although the decompositional approach to matrix computation remains
fundamental, nowadays in the era of big data, most of the classical algorithms
are inadequate to tackle most of the problems.

Data matrices are now incredibly
big, making most of the classical approaches too expensive in terms
of computation. Moreover, it is common in information sciences to have
data which is missing or innaccurate. This gives the opportunity to sacrify
some accuracy on the algorithm to gain on computation, which classical
algorithms are not able to do. Another important aspect is the role
of data transfer in the computational cost of a given algorithm, i.e,
techniques that may perform fewer passes over the data may be substantially
faster in practice.

In this review, we present, analyse and test \textit{randomized} algorithms
for matrix factorizations. This set of novel techniques addresses the issues
stated above, i.e, can trade-off computation and accuracy to an arbitrary
precision, gain on robustness, and reduce the number of passes on big datasets
when data transfer is expensive.

The purpose of approximated low-rank matrix factorization is to factorize
a given matrix $\mtx{A}\in\Rspace{m\times n}$ into a product of
two smaller matrices $\mtx{B}\in\Rspace{m\times k}$ and
$\mtx{C}\in\Rspace{k\times n}$.

\begin{equation}
\label{eq:lowrank}
\begin{array}{ccccccccccc}
\mtx{A} &\approx& \mtx{B} & \mtx{C},\\
m\times n && m \times k & k\times n.
\end{array}
\end{equation}

The matrix $\mtx{B}\times\mtx{C}$ in \ref{eq:lowrank} is called a rank-$k$ approximation of the
matrix $\mtx{A}$.

The inner dimension $k$ is called the \textit{numerical rank}
of the matrix. This
quantity differs from the \textit{algebraic rank}, which is defined as the
dimension of the image. The numerical rank is commonly defined as follows

\begin{equation}\label{eq:num-rank}
r(\mtx{A})\defeq\frac{\|\mtx{A}\|_F^2}{\|\mtx{A}\|^2}
= \sum_{j=1}^{\min(m,n)}\left(\frac{\sigma_j}{\sigma_1}\right)^2
\end{equation}

and it gives a better understanding of how accurate a rank-$k$ approximation
can be. Note that we always have $r(\mtx{A})\leq\text{rank}(\mtx{A})$. The notion
of numerical rank appears in ~\cite{vershynin2016high} and has been studied at the Theory Reading Group
course of the master in depth
\footnote{The comparison between both notions of rank can be better understood
through the following caracterization.
$\text{rank}(\mtx{A})=\dim(\textbf{A}B_2^n)$ and $r(\mtx{A})=d(\mtx{A}B_2^n)$
where $B_2^n$ is the euclidean ball, hence, $\mtx{A}B_2^n$ is the ellipsoid with
the axis of magnitude the singular values $\sigma_j$'s.
Here, $\dim(\cdot)$ denotes the algebraic dimension and
$d(\cdot)$ denotes the \textit{statistical dimension}.
The statistical dimension is defined as $d(T)\defeq \frac{h(T-T)^2}{\text{diam}(T)^2}
\sim \frac{w(T)^2}{\text{diam}(T)^2}$ where $h(T)^2=\Expect\sup_{t\in T}\la g, t\ra^2$
and $w(T)=\Expect\sup_{t\in T}\la g, t\ra$ is the gaussian width.
As discussed in ~\cite{vershynin2016high}, the statistical dimension is a more stable notion
of dimension, in the same way that the numerical rank is more stable than the
algebraic rank.
}.

The task of computing a low-rank approximation to a given matrix can be split into
two computational stages. The first is to construct a low dimensional subspace that
can capture the action of the matrix. The second is to restrict the matrix to the
low dimensional subspace and then compute a standard factorization (QR, SVD, etc)
of the reduced matrix.

\begin{itemize}
  \item\textbf{Stage 1:} Compute an approximate basis for the range of the input
  matrix $\mtx{A}$. We want to find a matrix $\mtx{Q}$ with a small
  number of orthonormal columns such that

  \begin{equation}\label{eq:app-basis}
    \mtx{A}\approx \mtx{Q}\mtx{Q}^\adj\mtx{A}
  \end{equation}

  The main idea is to approximate the range of the matrix via a randomized
  method. This can be accomplised by iteratively computing the image of
  random vectors from the input space and then orthogonalizing.
  All the randomness of these methods will belong to this first stage.

  \item\textbf{Stage 2:} Given a matrix $\mtx{Q}$ that satisfies \ref{eq:app-basis},
  compute a standard factorization (QR, SVD, etc.) of $\mtx{A}$.  Note that
  taking $\mtx{B}=\mtx{Q}$ and $\mtx{C}=\mtx{Q}^\adj\mtx{A}$ we already have
  a low rank approximation of the matrix.
   There is no randomness at this stage and only classical linear algebra computations are
  incolved.
\end{itemize}

During the rest of the introduction, we will review some basics about matrix approximation
and we will provide to the reader the basic aspects and insights of both stages,
which will be further studied in depth in the main body.

\subsection{Approximating the range of a matrix via randomness}

The problem of finding the best $\epsilon$-approximation of a given matrix $\mtx{A}$
 is called the \textit{fixed-precision approximation problem}. More concretely, we are given
 a tolerance $\epsilon$ and the goal is to find a matrix $\mtx{Q}$
 with $k=k(\epsilon)$ columns such that

\begin{equation}\label{eq:fixed-precision}
\|\mtx{A} - \mtx{Q}\mtx{Q}^\adj\mtx{A}\|\leq\epsilon
\end{equation}

The goal here is to find a $\mtx{Q}$ with the smaller number of columns
possible.

Another closely related problem is the so-called \textit{fixed-rank approximation problem},
which seeks to find the best rank-$k$ approximation of the matrix.

\begin{equation}
\label{eq:fixed-rank}
\min_{{\rm rank}(\mtx{X}) \leq j}\norm{ \mtx{A} - \mtx{X} }.
\end{equation}

The Singular Value Decomposition (SVD)
\footnote{The Singular Value Decomposition is a central tool for a data scientist.
It is the algorithmic tool used to perform \textit{Principal Component Analysis} (PCA) on a dataset
$\mtx{X}\in\Rspace{N\times d}$ of size $N$ and dimensionality $d$. PCA finds
linear hidden linear structure in the data by looking at the eigenvectors with largest
eigenvalues of the empirical covariance matrix of the data
$\hat{\mtx{\Sigma}}_N = \mtx{X}^\transp\mtx{X}=\frac{1}{N}\sum_{j=1}^N\mtx{x}_j\mtx{x}_j^\transp$.
These eigenvectors and eigenvalues can also be found by computing the SVD of the data matrix
$\mtx{X}=\mtx{U}\mtx{D}\mtx{V}^\transp$, because $\hat{\mtx{\Sigma}}_N = \mtx{V}\mtx{D}^2\mtx{V}^\transp$.
One important remark is that PCA does not suffer from the curse of dimensionality. For example,
if we assume sub-gaussianity on the random variable $X$, then we have
$\Expect\| \hat{\mtx{\Sigma}}_N - \mtx{\Sigma} \|\leq \varepsilon \|\mtx{\Sigma}\|$
if we take a sample size of $N\sim\varepsilon^{-2}d$, i.e, the number of samples
scales linearly with the dimensionality of the dataset. If instead of sub-gaussianity,
we relax it to $\|X\|_2\leq K(\Expect\|X\|_2^2)^{1/2}$ we only get an extra log-term.
This is one of the main reasons why PCA works!}
is key to analyze this problem. Recall that
the SVD of a matrix $\mtx{A}$ is the following decomposition

\begin{equation}\label{eq:svd}
\mtx{A} = \mtx{U}\mtx{\Sigma}\mtx{V}^\adj
= \sum_{j=1}^{\text{rank}(\mtx{A})}\sigma_j\mtx{u}_j\mtx{v}_j^\adj
\end{equation}

where $\{\mtx{u}_k\}_k, \{\mtx{v}_k\}_k$ are orthonormal basis on the output and input
space respectively, $\sigma_1\geq\sigma_2,\ldots,\sigma_{\text{rank}(\mtx{A})}\geq 0$
are the ordered singular values.

 The SVD provides an optimal answer to the
 \textit{fixed-rank approximation problem} ~\cite{mirsky1960symmetric} through the following
 important observation

\begin{equation}
\label{eqn:mirsky}
\min_{{\rm rank}(\mtx{X}) \leq k}\norm{ \mtx{A} - \mtx{X} } = \sigma_{k+1}.
\end{equation}

It is straightforward to check that the optimum is attained at
 $\mtx{X}_{*} = \sum_{j=1}^k\sigma_j\mtx{u}_j\mtx{v}_j^\adj$, namely, the $k$-truncated
 SVD (k-SVD) of the matrix $\mtx{A}$. More precisely, we have that
$\mtx{B}=\mtx{U}\mtx{\Sigma}_{[k]}^{1/2}$ and $\mtx{C} = \mtx{\Sigma}_{[k]}^{1/2}\mtx{V}^\adj$
are the best solutions \ref{eq:lowrank} when the rank is fixed.

Let's suppose now that we know the desired rank $k$ in advance. The goal is
to find a matrix $\mtx{Q}$ with $k+p$ orthonormal columns such that

\begin{equation}\label{eq:fixed-rank-app}
\|\mtx{A}-\mtx{Q}\mtx{Q}^\adj\mtx{A}\| \approx
\min_{{\rm rank}(\mtx{X}) \leq k}\norm{ \mtx{A} - \mtx{X} }
\end{equation}

where $p$ is called the \textit{oversampling parameter}.

% EXPLICACIO

\subsection{Intuition of the randomized method to find Q} \label{alg:proto-algorithm}

The key observation that leverage these methods is the fact that the matrix
$\mtx{Q}$ can be found by sampling, and now the reader will obtain intuition
on how this can be done.

Suppose we seek a basis for the range of a matrix $\mtx{A}$ with algebraic rank
$k$. Random elements $\mtx{y}^{(i)}$ from the range can be computed by computing the image
of random vectors $\mtx{w}^{(i)}$ from the input space. Let us repeat this
process $k$ times:

\begin{equation}\label{eq:iter-range}
\mtx{y}^{(i)} = \mtx{A}\mtx{w}^{(i)}, \hspace{0.5cm}i=1,\ldots,k
\end{equation}

Thanks to the randomness, the set $\{\mtx{w}^{(i)}\}_{i=1}^k$ is likely to be
in general linear position and no vector will fall in $\ker\mtx{A}$
if this is a set of measure zero under the probability measure we sampled from.
Therefore, an orthogonalization procedure gives the desired orthonormal basis.

What happens if the matrix $\mtx{A}$ has not exact algebraic rank equal to $k$
\footnote{In practice this is always the case.}?
Write $\mtx{A} = \mtx{B} + \mtx{E}$ where $\mtx{B}$ is a rank-$k$ matrix containing
the information we seek and $\mtx{E}$ a small perturbation.
We want a basis that covers the range of $\mtx{B}$, however, if we repeat
procedure \ref{eq:iter-range}, the vectors will be affected by the perturbation
and the $\{\mtx{y}^{(i)}\}_{i=1}^k$ will have small components that will make
them fall outside the desired space.

To overcome this issue, the idea is to take $p$ more samples:

\begin{equation}\label{eq:iter-range}
\mtx{y}^{(i)} = \mtx{A}\mtx{w}^{(i)} = \mtx{B}\mtx{w}^{(i)}
+ \mtx{E}\mtx{w}^{(i)}, \hspace{0.5cm}i=1,\ldots,k+p
\end{equation}

The enriched set $\{\mtx{y}^{(i)}\}_{i=1}^{k+p}$ has much more chance
of spanning the desired subspace, and this is grounded with some
theoretical results; see for example Theorem \ref{thm:avg-frob-error-gauss}.
The theory also shows that
$p$ can be quite small. In practice, $p=5$ is enough.

Here in the box below, we illustrate the basic prodecure to
find $\mtx{Q}$ that approximates the range of $\mtx{A}$ through randomization.
We call it Proto-Algorithm.
\begin{figure}[ht]
\begin{center}
\framebox{\begin{minipage}{.9\textwidth}
\begin{center}
\textsc{Proto-Algorithm: Solving the Fixed-Rank Problem}
\end{center}
\begin{tabbing}
\hspace{5mm} \= \hspace{5mm} \= \hspace{5mm} \= \hspace{5mm} \= \kill
\anum{1} \>Draw a random $n \times (k + p)$ test matrix $\mtx{\Omega}$.\\
\anum{2} \>Form the matrix product $\mtx{Y} = \mtx{A\Omega}$.\\
\anum{3} \>Construct a matrix $\mtx{Q}$ whose columns form an orthonormal basis for \\
         \>the range of $\mtx{Y}$.
\end{tabbing}
\end{minipage}}
\end{center}
% \caption{}
\end{figure}

\subsection{Construction of standard matrix factorizations from Q}

This corresponds to \textbf{Stage 2} of the algorithm. Once we have $\mtx{Q}$
such that $\mtx{A}\approx\mtx{Q}\mtx{Q}^\adj\mtx{A}$, taking
$\mtx{B}=\mtx{Q}$ and $\mtx{C}=\mtx{Q}^\adj\mtx{A}$ we directly produce a low rank
matrix decomposition $\mtx{A}\approx\mtx{B}\mtx{C}$. However,
there are more involved methods to produce a decomposition from $\mtx{Q}$ with
computational advantadges compared to the direct method.

One we have $\mtx{A}\approx\mtx{B}\mtx{C}$, standard matrix decompositions
(e.g. SVD, QR) can be easily derived as we show in \ref{itm:SVD-from-C}. \\

A lot of questions need to be addressed in order to turn these methods into
a technology or \textit{off-the-shelf} algorithms. The following
sections will we devoted to this.


% Full title as you would like it to appear on the page
\chapter{Interpolating The Global Wavefront with Least Squares}
\label{chap:interp}
\chaptermark{Interpolating The Global Wavefront}

\epigraph{In questions of science, the authority of a thousand is not worth the humble reasoning of a single individual.}{Galileo Galilei}
% \epigraph{The first principle is that you must not fool yourself and you are the easiest person to fool.}{Richard P. Feynman}

\section{The Interpolation Geometry}

In the second step, we interpolate the global wavefront from the local wavefront predictions from the first step. We interpolate the global wavefront coefficients $\beta_{jk}$ based on the predictions $\alpha$ from the previous step. More explicitly, we use ordinary least squares (LS) to solve

\begin{equation*}
\text{LS}(\alpha(x_1,y_1), \dots, \alpha(x_n,y_n)) \to \beta_{jk} \text{ such that } W(u,v,x,y) = \sum_{j=4}^{21}\sum_{k=1}^3\beta_{jk}Z_j(u,v)Z_k(x,y)
\end{equation*}

In the previous step, the 18 predictions $\{\alpha_4(x_i, y_i), \dots, \alpha_{21}(x_i,y_i)\}$ for a source image $D$ at position $(x_i,y_i)$. In this step, we start by aggregating the predictions for a single coefficient $j$ for all $n$ donuts in the field of view of the observation, or $\{\alpha_j(x_1, y_1), \dots, \alpha_{j}(x_n,y_n)\}$. Then the problem reduces to minimizing the distance between the $\alpha_j$ estimates and $\sum_{k=1}^3\beta_{jk}Z_k(x,y)$ for each index separately. Figure \ref{fig:rafts} shows the true and predicted $\alpha_4$ values on the four wavefront sensors.

\begin{figure}[!htbp]
\begin{center}
\begin{tabular}{c}
\includegraphics[width=\textwidth]{figs/interp/rafts.png}
\end{tabular}
\end{center}
\caption[Local Wavefront Predictions Across Four Wavefront Sensors]{The two rows correspond to the true ($\alpha_4^{\text{true}}$) and predicted ($\alpha_4^{\text{predicted}}$) $Z_4$ coefficients. The four columns correspond to the four corner wavefront sensors. The colormap is in units of waves.\label{fig:rafts}}
\end{figure}

The first three Zernike coefficients establish a plane. Thus we are effectively fitting a plane by the three coefficients along the $k$ index, for each local wavefront Zernike coefficient, or $j$ index. Figure \ref{fig:fieldols} shows how $\beta_{41}, \beta_{42}, \beta_{43}$ are fit from the local $\alpha_4$ predictions.

\begin{figure}[!htbp]
\begin{center}
\begin{tabular}{c}
\includegraphics[width=\textwidth]{figs/interp/fieldols.png}
\end{tabular}
\end{center}
\caption[Fitting Global $\beta_{41}, \beta_{42}, \beta_{43}$ From Local $\alpha_4$ Predictions]{\textit{Left:} the predicted coefficients $\alpha_4^{\text{pred}}$ from Figure \ref{fig:rafts} shown at their corresponding field locations within the field of view. \textit{Right:} the plane characterized by the three fit double Zernike coefficients $\beta^{\text{pred}}_{41},\beta^{\text{pred}}_{42}, \beta^{\text{pred}}_{43}$. The two circles represent the field of view of the Rubin Observatory. \label{fig:ls-intuition}}
\end{figure}

The explicit minimization is

\begin{equation*}
\label{eq:objective}
\text{min}_{\beta}\left|\left| \begin{bmatrix}\alpha_j(x_1,y_1) \\ \vdots \\ \alpha_j(x_n,y_n) \end{bmatrix} - \begin{bmatrix}Z_1(x_1,y_1) & Z_2(x_1,y_1) & Z_3(x_1,y_1) \\ \vdots & \vdots & \vdots\\  Z_1(x_n,y_n) & Z_2(x_n,y_n) & Z_3(x_n,y_n)\end{bmatrix} \begin{bmatrix}\beta_{j1} \\ \beta_{j2} \\ \beta_{j3} \end{bmatrix}\right|\right|_2
\end{equation*}

\noindent which we solve with ordinary least squares. Letting $\alpha_j, Z, \beta_j$ be the vectors and matrices from the equation above, we have the analytic solution $\beta_j = (Z^TZ)^{-1}Z^T\alpha_j$. After solving this equation for each $j$, we have $\beta_{jk}$ for $4 \leq j \leq 21$ and $1 \leq k \leq 3$. In the next section we describe a few variations to this basic framework.

\section{Evaluating Different Interpolation Strategies}

The interpolation is comprised of three steps that are reminiscent of a standard data query: select, reduce, and fit. The \textit{select} step decides which donuts and corresponding local estimates to use in the interpolation. The \textit{reduce} step, which is typically skipped, reduces these estimates across a wavefront sensor. The \textit{fit} step fits the local coefficients to a global Zernike basis based on the provided loss function.

We examine multiple variations in each of these steps to find which combination works best. We explore selecting donuts from all the sources (stars and blends), from only the non-blended stars (stars), the non-blended 10 brightest stars per chip (brightest stars), and using the true labels (labels). The results on the true labels provide a sanity check and bound the performance we can expect to achieve with alternatives.

We also analyze two variations in the reduce step. Either we make no changes to estimates and effectively skip this step, or we take the median of the estimates on each chip. In the median case, we would then fit against the four points corresponding to the four sensors in the fit step.

We explore three different fitting strategies. The $\ell_1$, or absolute loss, is convex and can be found with an iterative optimization algorithm. The $\ell_2$, or least squares loss, has an analytic solution. This has the added benefit of making error propagation analytic as well. Finally, the Huber loss $\ell_h$ is similar to the $\ell_2$ for samples with small error but scales like $\ell_1$ for large error. Thus it is similar to $\ell_2$ but less sensitive to outliers.

The results of these variations, applied to the local wavefront estimates from the neural network, applied to the full visit dataset, are shown in Table \ref{tab:variations}. We compare the true global wavefront and the residual global wavefront, where the residual is the true wavefront minus the interpolated optics wavefront. The residual wavefront is smaller than the original wavefront for all the samples in the majority of select-reduce-fit variations. The consistency of the improvements makes our method an attractive candidate for deployment.

\begin{table}
{
\begin{center}
\begin{tabular}{|c|c|c|c|r|}
\hline
 & Median &  & \multicolumn{1}{|c|}{\% Samples} & \multicolumn{1}{|c|}{Relative}  \\
Select & Reduce & Fit & \multicolumn{1}{|c|}{Improved} & \multicolumn{1}{|c|}{Residual} \\
\hline
 &  & $\ell_1$ & $ 99.6$ & $ 0.48 \pm  0.13$\\
Stars & & $\ell_2$ &  $ 99.8$ & $ 0.49 \pm  0.12$\\
and&  & $\ell_{\text{h}}$ &  $ 100.0$ & $ 0.48 \pm  0.12$\\
Blends& \checkmark & $\ell_{1}$ &  $ 97.8$ & $ 0.67 \pm  0.14$\\
& \checkmark & $\ell_{2}$ &  $ 100.0$ & $ 0.46 \pm  0.12$\\
\hline
&   & $\ell_1$ &  $ 99.8$ & $ 0.44 \pm  0.11$\\
&   & $\ell_2$ &  $ 100.0$ & $ 0.43 \pm  0.10$\\
Stars&  & $\ell_{\text{h}}$ &  $ 100.0$ & $ 0.43 \pm  0.10$\\
& \checkmark  & $\ell_{1}$ &  $ 97.2$ & $ 0.64 \pm  0.14$\\
& \checkmark  & $\ell_{2}$ &  $ 99.8$ & $ 0.41 \pm  0.11$\\
\hline
& & $\ell_1$ &  $ 99.6$ & $ 0.37 \pm  0.13$\\
Brightest& & \textcolor{blue}{$\ell_2$} & \textcolor{blue}{$ 100.0$} & \textcolor{blue}{$ 0.34 \pm  0.12$}\\
Stars& & $\ell_{\text{h}}$ &  $100.0$ & $ 0.34 \pm  0.12$\\
& \checkmark & $\ell_{1}$ &  $97.2$ & $ 0.60 \pm  0.16$\\
& \checkmark & $\ell_{2}$ &  $100.0$ & $ 0.35 \pm  0.12$\\
\hline
& & $\ell_1$ &  $ 100.0$ & $ 0.13 \pm  0.05$\\
Labels & & \textcolor{red}{$\ell_2$} & \textcolor{red}{$ 100.0$} & \textcolor{red}{$ 0.06 \pm  0.02$}\\
& & $\ell_{\text{h}}$ &  $ 100.0$ & $ 0.08 \pm  0.04$\\
\hline
\end{tabular}
\end{center}
}
\caption[Select-Reduce-Fit Results]{Each row contains the results for a different combination of select, reduce, and fit steps. The penultimate column contains the percentage of the number of samples where the residual improved. The final column contains the relative residual: the total magnitude of the residual divided by the total magnitude of the true wavefront. The best variation on neural network estimates is highlighted in \textcolor{blue}{blue}. The best variation on the true label estimates is highlighted in \textcolor{red}{red}.}
\label{tab:variations}
\end{table}

This experiment also taught us that more data is not always better. Ignoring the blended donuts leads to a clear improvement in performance. So does ignoring all but the brightest stars. This suggests that we should prioritize making accurate predictions on the best donuts, perhaps at the expense of making consistent estimates on all the donuts. It also may have consequences for wavefront sensing in crowded fields where almost all of the donuts are blended.

We can also draw conclusions about the variations. Taking the median and fitting with the $\ell_1$ norm appears to discard too much information. We also see that the benefit of using median with the $\ell_2$ norm goes away as the select becomes more selective. This is likely because the outliers, which the median reduce suppresses, get filtered and are no longer an issue. The $\ell_{h}$ norm also seems to do comparatively well on stars and blends, but loses this advantage on the more selective brightest stars selection. We conclude using the $\ell_2$ norm, with no median reduce, to fit the brightest stars, is the best variation.

We used this variation to explore how our framework performs in five scenarios. We used the relative residual $|\beta^{\text{res}}| / |\beta^{\text{true}}| = \sum_{jk}|\beta^{\text{true}}_{jk} - \beta^{\text{pred}}_{jk}| / \sum_{jk}|\beta^{\text{true}}_{jk}|$ to measure our algorithm's capability to reproduce the true global wavefront. A value close to zero means the predictions match the true global wavefront almost perfectly; a value close to one means the predictions capture almost none of the true global wavefront. Figure \ref{fig:beta-results} shows the results.

\begin{figure}[!htbp]
\begin{center}
\begin{tabular}{c}
\includegraphics[width=\textwidth]{figs/interp/dist_defense2.png}
\end{tabular}
\end{center}
\caption[Relative Residuals In Different Scenarios]{The distributions of relative residuals in four different scenarios. The distributions from performing the interpolation over the labels with and without noise are shown in black. The distributions from performing the interpolation over the local wavefront predictions from the neural network with and without noise are shown in blue. \label{fig:beta-results}}
\end{figure}

In the first scenario, we evaluated the second step of our algorithm on the true local wavefront labels from the first step. The relative residual is 6\%, which validates that our simulations are consistent and there are no sources of catastrophic numerical error. In the second scenario, we added uncorrelated Gaussian noise to each local wavefront coefficient. The variance of the noise for each coefficient was equal to the variance of the neural network error on that coefficient in the test data. Despite this noise, the relative residual remained very low at 10\%. This shows that our approach is not overly sensitive to uncorrelated noise.

In the third scenario, we fed the local wavefront predictions from the neural network to the second step. These have significant correlated error due to the atmosphere. The relative residual is 34\%, extremely promising, with a tight 12\% standard deviation. Furthermore, the right tail decays rapidly. Not a single relative residual is greater than one. Our framework improves the global wavefront on every observation in the full visit dataset!

In the fourth scenario we add noise as in the second scenario, effectively doubling the error. The relative residual increases slightly to 38\%. This suggests that the network is fairly robust to uncorrelated noise.

\section{Combined Runtime}

The run-time is another key advantage of our approach. It takes just under 4 seconds to process 811 donut images on a 2.4 GHz Intel Xeon CPU with a single Nvidia V100 GPU. It takes an additional 5 milliseconds to run the least squares optimization. The total processing time per donut image is 5 milliseconds; the total processing time per donut of the Rubin Observatory AOS under development is around 10 seconds\cite{2015Xin,2014Overiew}. Our scheme can process donuts 2,000 times faster, and is capable of processing 7,800 donuts in a single 39 second Rubin visit. The low latency allows it to process all the donuts in most observations.

\section{Image Quality Results}

The next step is to take this model and measure the repercussions of subtracting its estimate from the true wavefront on both the PSF FWHM and Strehl ratio \cite{strehl1895aplanatische}. We compute these by calculating the local wavefronts at the center and one corner of the focal plane. Then we take the fourier transform of the resulting pupil plane aberration to get the point spread function. The results for the original and corrected wavefronts are shown in Table \ref{tab:corrections}.

\begin{table}
{
    \begin{center}
    \begin{tabular}{|l|l|c|c|}
    \hline
    State & Position & FWHM & Strehl\\
    \hline
    Before & Center & $0.288 \pm 0.034$ & $0.093 \pm 0.39$ \\
    After & Center & $0.211 \pm 0.005$ &  $0.555 \pm 0.207$\\
    \hline
    Before & Corner & $0.314 \pm 0.045$ & $0.074 \pm 0.32$\\
    After & Corner & $0.215 \pm 0.009$ & $0.400 \pm 0.184$\\
    \hline
    \end{tabular}
    \end{center}
}
\caption[Improvements To The Optics PSF FWHM And Strehl Ratio]{We measure the optics PSF FWHM and Strehl ratio on the original optics wavefront from the observation (Before), and the residual wavefront resulting from subtracting our wavefront estimate from the original optics wavefront (After). The Center position is at the center of the Rubin focal plane; the Corner position is at the center of the $R00$ wavefront sensor in the corner of the focal plane.}
\label{tab:corrections}
\end{table}

The optics PSF FWHM decreases considerably, especially when compared to the standard deviation of the original. The Strehl ratio increases in an even more extreme manner. Figure \ref{fig:vision} provides an illustrative example of how the improvements to the optics PSF from our method can improve image quality. We apply the Rubin optics PSF, before and after subtracting the optics wavefront estimated by our framework, to six classic Hubble Telescope images in the absence of other significant PSF contributions.

\begin{figure}[!htbp]
\begin{center}
\begin{tabular}{c}
\includegraphics[width=\textwidth]{figs/interp/results.png}
\end{tabular}
\end{center}
\caption[Illustrative Hubble Telescope Images After Correction From Our Framework]{An illustration of the effect of wavefront correction using our techniques on image resolution.  We use actual Hubble Space Telescope images to show the effect.  The Before images are degraded by the wavefront aberrations we have simulated.  The After images show their reconstruction after wavefront estimation and correction using the technique we describe in this paper. On the bottom two rows, we show the effective PSF, both before and after wavefront correction. The images have an angular extent of 3 arcseconds and the PSFs are displayed on a $0.16 \times 0.16$ arcsecond grid. \label{fig:results}}
\end{figure}

For the Rubin Observatory however, the PSF is dominated by the atmosphere, which as indicated cannot be corrected. For the Rubin observatory the assumed PSF width is of order 0.71 arcseconds, of which 0.65 is contributed by the atmosphere. Therefore, the improvement in the Strehl and image quality is not as dramatic. Nevertheless, this improvement is still important for use on nights with unusually good atmospheric seeing and also for applications that are especially sensitive to the PSF width.