%%
%% Automatically generated file from DocOnce source
%% (https://github.com/hplgit/doconce/)
%%
%%

%-------------------- begin preamble ----------------------

\documentclass[%
oneside,                 % oneside: electronic viewing, twoside: printing
final,                   % draft: marks overfull hboxes, figures with paths
10pt]{article}

\listfiles               %  print all files needed to compile this document

\usepackage{relsize,makeidx,color,setspace,amsmath,amsfonts,amssymb}
\usepackage[table]{xcolor}
\usepackage{bm,ltablex,microtype}

\usepackage[pdftex]{graphicx}

\usepackage[T1]{fontenc}
%\usepackage[latin1]{inputenc}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}

\usepackage{lmodern}         % Latin Modern fonts derived from Computer Modern

% Hyperlinks in PDF:
\definecolor{linkcolor}{rgb}{0,0,0.4}
\usepackage{hyperref}
\hypersetup{
    breaklinks=true,
    colorlinks=true,
    linkcolor=linkcolor,
    urlcolor=linkcolor,
    citecolor=black,
    filecolor=black,
    %filecolor=blue,
    pdfmenubar=true,
    pdftoolbar=true,
    bookmarksdepth=3   % Uncomment (and tweak) for PDF bookmarks with more levels than the TOC
    }
%\hyperbaseurl{}   % hyperlinks are relative to this root

\setcounter{tocdepth}{2}  % levels in table of contents

% prevent orhpans and widows
\clubpenalty = 10000
\widowpenalty = 10000

% --- end of standard preamble for documents ---

% insert custom LaTeX commands...

\raggedbottom
\makeindex
\usepackage[totoc]{idxlayout}   % for index in the toc
\usepackage[nottoc]{tocbibind}  % for references/bibliography in the toc

%-------------------- end preamble ----------------------

\begin{document}

% matching end for #ifdef PREAMBLE

\newcommand{\exercisesection}[1]{\subsection*{#1}}

% ------------------- main content ----------------------

% ----------------- title -------------------------

\thispagestyle{empty}

\begin{center}
{\LARGE\bf
\begin{spacing}{1.25}
Project 5, deadline  December 9, 2016
\end{spacing}
}
\end{center}

% ----------------- author(s) -------------------------

\begin{center}
{\bf Partial Differential equations${}^{}$} \\ [0mm]
\end{center}

\begin{center}
% List of all institutions:
\end{center}

% ----------------- end author(s) -------------------------

% --- begin date ---
\begin{center}
Fall semester 2016
\end{center}
% --- end date ---

\vspace{1cm}

\subsection*{Theoretical background and description of the system}

This project is more numerically directed, with tests
of algorithms for solving partial differential equations using finite difference schemes.
Chapter 10 of the lecture notes provides the necessary theoretical background.

The physical problem can be that of the temperature gradient in a rod of length $L=1$ or that of channel flow
between two flat and infinite plates at $x=0$ and $x=1$, where the fluid is initially at rest
and the plate $x=1$ is given a sudden initial movement.  We are looking at a one-dimensional
problem

\begin{equation*}
 \frac{\partial^2 u(x,t)}{\partial x^2} =\frac{\partial u(x,t)}{\partial t}, t> 0, x\in [0,L]
\end{equation*}
or

\begin{equation*}
u_{xx} = u_t,
\end{equation*}
with initial conditions, i.e., the conditions at $t=0$,

\begin{equation*}
u(x,0)= 0 \hspace{0.5cm} 0 < x < L
\end{equation*}
with $L=1$ the length of the $x$-region of interest. The
boundary conditions are

\begin{equation*}
u(0,t)= 0 \hspace{0.5cm} t > 0,
\end{equation*}
and

\begin{equation*}
u(L,t)= 1 \hspace{0.5cm} t > 0.
\end{equation*}
The function $u(x,t)$  can be the temperature gradient of a  rod or represent the fluid velocity
in a direction parallel to the plates, that is normal to the $x$-axis. In the latter case,
for small $t$, only the part of the fluid
close to the moving plate is set in significant  motion, resulting in a thin boundary layer at $x=1$.
As time increases, the velocity approaches a linear variation with $x$. In this case, which can be derived
from the incompressible Navier-Stokes, the above equations constitute a model for
studying friction between moving surfaces separated by a thin fluid film.

In this project we want to study the numerical stability of three methods for partial differential equations
(PDEs).
These methods we will discuss are
the explicit forward Euler algorithm with discretized versions of time given by a forward formula and a centered difference in space resulting in
\begin{equation*}
u_t\approx \frac{u(x,t+\Delta t)-u(x,t)}{\Delta t}=\frac{u(x_i,t_j+\Delta t)-u(x_i,t_j)}{\Delta t}
\end{equation*}
and

\begin{equation*}
u_{xx}\approx \frac{u(x+\Delta x,t)-2u(x,t)+u(x-\Delta x,t)}{\Delta x^2},
\end{equation*}
or

\begin{equation*}
u_{xx}\approx \frac{u(x_i+\Delta x,t_j)-2u(x_i,t_j)+u(x_i-\Delta x,t_j)}{\Delta x^2}.
\end{equation*}
We will also discuss the implicit Backward Euler with
\begin{equation*}
u_t\approx \frac{u(x,t)-u(x,t-\Delta t)}{\Delta t}=\frac{u(x_i,t_j)-u(x_i,t_j-\Delta t)}{\Delta t}
\end{equation*}
and

\begin{equation*}
u_{xx}\approx \frac{u(x+\Delta x,t)-2u(x,t)+u(x-\Delta x,t)}{\Delta x^2},
\end{equation*}
or

\begin{equation*}
u_{xx}\approx \frac{u(x_i+\Delta x,t_j)-2u(x_i,t_j)+u(x_i-\Delta x,t_j)}{\Delta x^2},
\end{equation*}

Finally we use the implicit Crank-Nicolson scheme with  a time-centered scheme at $(x,t+\Delta t/2)$
\begin{equation*}
u_t\approx \frac{u(x,t+\Delta t)-u(x,t)}{\Delta t}=\frac{u(x_i,t_j+\Delta t)-u(x_i,t_j)}{\Delta t}.
\end{equation*}
The corresponding spatial second-order derivative reads

\begin{align*}
u_{xx}\approx &\frac{1}{2}\left(\frac{u(x_i+\Delta x,t_j)-2u(x_i,t_j)+u(x_i-\Delta x,t_j)}{\Delta x^2}\right. +\\
&\left. \frac{u(x_i+\Delta x,t_j+\Delta t)-2u(x_i,t_j+\Delta t)+u(x_i-\Delta x,t_j+\Delta t)}{\Delta x^2}\right).
\end{align*}
Note well that we are using a time-centered scheme wih $t+\Delta t/2$ as center.

\textbf{For this project you can collaborate with fellow students and you can  hand in a common report.}
This project (together with projects 3 and 4) counts 1/3 of the final mark.

\paragraph{Project 5a): Setting up the algorithms.}
Write down the algorithms for these three methods and the equations you need to implement.
For the implicit schemes show that the equations lead to a tridiagonal matrix system for the new values.

\paragraph{Project 5b): Truncation errors and analytic solutions.}
Find the truncation errors of these three schemes and investigate their stability properties.
Find also the analytic solution to the continuous problem.

\paragraph{Project 5c): Implementation.}
Implement the three algorithms in the same code and perform tests of the solution
for these three approaches
for $\Delta x=1/10$, $h=1/100$ using  $\Delta t$ as dictated by the stability limit of the explicit scheme.
Study the solutions at two time points $t_1$ and $t_2$ where $u(x,t_1)$ is smooth but still significantly curved
and $u(x,t_2)$ is almost linear, close to the stationary state.
Remember that forsolving the tridiagonal equations you can use your code from project 1.
\paragraph{Project 5d):}
Compare the solutions at $t_1$ and $t_2$ with the analytic result for the continuous problem.
Which of the schemes would you classify as the best?

\paragraph{Project 5e): Moving to two dimensions.}
Extend the code you have developed here to two
  dimensions.
It means that we deal with a $2+1$ dimensional problem. Our differential equation becomes
\[
 \frac{\partial^2 u(x,y,t)}{\partial x^2}+\frac{\partial^2 u(x,y,t)}{\partial y^2} =\frac{\partial u(x,y,t)}{\partial t}, t> 0, x,y\in [0,1],
\]
where we now have made a model with a square lattice for $x$ and $y$.
How would you extend the boundary conditions from one dimension to two dimensions? And can you
  find a closed form solution here as well?  It is left to you to decide upon what kind of boundary conditions
you deem appropriate.
\paragraph{Project 5f): Solving the two-dimensional equations numerically.}
Here you can choose between an explicit or an implicit scheme.
For the implicit scheme discussed in chapter 10 of the lecture notes, upi need to use for example an iterative method like Jacobi's.

Outline the algorithm for solving the two-dimensional diffusion equation and
implement the explicit or the implicit scheme as function of $\Delta x$ (assuming
$\Delta x = \Delta y$) and $\Delta t$. Solve the equations numerically and give a critical discussion of your results.
Compare your results with the closed-form answer. Discuss the stability
of the solution as function of different values of $\Delta x$  and $\Delta t$.

You should, if you have time, try to  parallelize the equations using MPI or OpenMP.

\paragraph{References.}
A very good reference is the textbook by \href{{http://www.springer.com/us/book/9783540225515}}{Winther and Tveito on partial differential equations}. It is available online \href{{https://vpn2.uio.no/+CSCO+0h756767633A2F2F797661782E66636576617472652E70627A++/book/10.1007/b138016/page/1}}{from the University library}.

\subsection*{Introduction to numerical projects}

Here follows a brief recipe and recommendation on how to write a report for each
project.

\begin{itemize}
  \item Give a short description of the nature of the problem and the eventual  numerical methods you have used.

  \item Describe the algorithm you have used and/or developed. Here you may find it convenient to use pseudocoding. In many cases you can describe the algorithm in the program itself.

  \item Include the source code of your program. Comment your program properly.

  \item If possible, try to find analytic solutions, or known limits in order to test your program when developing the code.

  \item Include your results either in figure form or in a table. Remember to        label your results. All tables and figures should have relevant captions        and labels on the axes.

  \item Try to evaluate the reliabilty and numerical stability/precision of your results. If possible, include a qualitative and/or quantitative discussion of the numerical stability, eventual loss of precision etc.

  \item Try to give an interpretation of you results in your answers to  the problems.

  \item Critique: if possible include your comments and reflections about the  exercise, whether you felt you learnt something, ideas for improvements and  other thoughts you've made when solving the exercise. We wish to keep this course at the interactive level and your comments can help us improve it.

  \item Try to establish a practice where you log your work at the  computerlab. You may find such a logbook very handy at later stages in your work, especially when you don't properly remember  what a previous test version  of your program did. Here you could also record  the time spent on solving the exercise, various algorithms you may have tested or other topics which you feel worthy of mentioning.
\end{itemize}

\noindent
\subsection*{Format for electronic delivery of report and programs}

The preferred format for the report is a PDF file. You can also use DOC or postscript formats or as an ipython notebook file.  As programming language we prefer that you choose between C/C++, Fortran2008 or Python. The following prescription should be followed when preparing the report:

\begin{itemize}
  \item Use Devilry to hand in your projects, log in  at  \href{{http://devilry.ifi.uio.no}}{\nolinkurl{http://devilry.ifi.uio.no}} with your normal UiO username and password and choose either 'fys3150' or 'fys4150'. There you can load up the files within the deadline.

  \item Upload \textbf{only} the report file!  For the source code file(s) you have developed please provide us with your link to your github domain.  The report file should include all of your discussions and a list of the codes you have developed.  Do not include library files which are available at the course homepage, unless you have made specific changes to them.

  \item In your git repository, please include a folder which contains selected results. These can be in the form of output from your code for a selected set of runs and input parametxers.

  \item In this and all later projects, you should include tests (for example unit tests) of your code(s).

  \item Comments  from us on your projects, approval or not, corrections to be made  etc can be found under your Devilry domain and are only visible to you and the teachers of the course.
\end{itemize}

\noindent
Finally,
we encourage you to work two and two together. Optimal working groups consist of
2-3 students. For this specific report you can  hand in a common report.

% ------------------- end of main content ---------------

\end{document}


\documentclass[usletter]{article}
\usepackage{graphicx}
\usepackage{amsfonts}
\usepackage{amsthm}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{hyperref}
\hypersetup{
    colorlinks=true,
    linkcolor=blue,
    filecolor=magenta,
    urlcolor=cyan,
}
\usepackage{scribe}
\usepackage[margin=1.5in]{geometry}

\begin{document}

\makeheader{Sanket Kanjalkar, Yunqi Li}                              % your name
           {October 09, 2019}                          % lecture date
           {11}                                       % lecture number
           {Fully Homomorphic Encryption I}  % lecture title

\newcommand{\floor}[1]{\left\lfloor #1 \right\rfloor}
\newcommand{\ceil}[1]{\left\lceil #1 \right\rceil}

Informally, a cryptosystem that supports arbitrary computation on ciphertexts is known as fully homomorphic encryption (FHE)\cite{wiki}. Throughout the course, we have studied encryption schemes for the purpose of information transfer from other party to the other. However, if we were to compute on data using our traditional encryption schemes, it would require that data must be decrypted before it can be analyzed or manipulated. It would be great if we can outsource the computation while keeping the data encrypted. That is, it should be possible to compute a known function on encrupted data without having access to the secret key.

Such a FHE scheme can be used for privacy-preserving outsourced storage and computation. In today's lecture, we will see what is fully homomorphic encryption (FHE) scheme and how to build a FHE scheme achieving homomorphism. We will also look at intuitive ways to construct FHE for addition and multiplication.

\begin{fact}
LWE is the only way we know to do an fully homomorphic encryption as of Oct 2019.
\end{fact}

\section{Recap}

Before stepping into how to build LWE-based FHE schemes, let's briefly recap how to build private and public encryption scheme with LWE problem.
Decisional LWE is the problem where given a matrix $A_{n,m}$, and a corresponding LWE vector result $b_m$, determining whether or not these
and results vector of some instance of an LWE problem OR whether the vector $b_m$ was simply
drawn from sampling values uniformly randomly from $Z^{m}_{q}$.

\begin{definition}
\textbf{\textit{Decisional $LWE{n,m,q,\mathcal{X}}$}} : For all non-uniform probabilistic polynomial time adversary $\mathcal{A}$
$$|\underset{\substack{
\pmb{s}\leftarrow \mathbb{Z}_q^{n\times1}\\
\pmb{A}\leftarrow\mathbb{Z}_q^{n\times m}\\
\pmb{e}\leftarrow \mathcal{X}^m}}{Pr}
[\mathcal{A}(\pmb{A},\pmb{s}^T\pmb{A}+\pmb{e}^T)=1]
-\underset{\substack{\pmb{A}\leftarrow\mathbb{Z}_q^{n\times m}\\
\pmb{b}\leftarrow\mathbb{Z}_q^m}}{Pr}
[\mathcal{A}(\pmb{A},\pmb{b})=1]|=negl(n)$$
where $q$ is a prime within $O(2^n)$, $m=O(n\log q)$ and norm $\parallel \pmb{e}\parallel=\omega(\log n)$. Where $\omega(f(n))$ means that
$e$ should be greater than $f(n)$ asymptotically.
\end{definition}

Next we present the secret key encryption (SKE) built with LWE which has $m=1$:

\begin{align*}
& KeyGen(1^n): \pmb{s}\leftarrow\mathbb{Z}_q^n\\
&\underset{\substack{
\pmb{s}\leftarrow \mathbb{Z}_q^{n\times1}\\
\pmb{a}\leftarrow\mathbb{Z}_q^{n\times 1}\\
\pmb{e}\leftarrow \mathcal{X}}}{Enc}(\pmb{s},\mu\in\{0,1\}): (\pmb{a}, b = (\pmb{s}^T\pmb{a}+e+\mu\lfloor\frac{q}{2}\rfloor)\mod q)\\
&Dec(\pmb{s},\pmb{a},b): b-\langle\pmb{s}^T,\pmb{a}\rangle=(e+\mu\lfloor\frac{q}{2}\rfloor) \mod q
\end{align*}

LWE can also be used to build public key encryption (PKE):
\begin{itemize}
\item[-]  $KeyGen(1^n): (sk=\pmb{s},pk=(\pmb{A}, \pmb{b}^T=\pmb{s}^T\pmb{A}+\pmb{e}^T))$
\begin{itemize}
\item[*]  $\pmb{s}\leftarrow\mathbb{Z}_q^n$
\item[*] $\pmb{A}\leftarrow\mathbb{Z}_q^{n\times m}$
\item[*] $\pmb{e}\leftarrow\mathcal{X}^m$
\end{itemize}

\item[-]  $Enc(pk,\mu\in\{0,1\}): (\pmb{c_1}=\pmb{A}\pmb{r}, c_2=(\pmb{b}^T\pmb{r}+\mu\lfloor\frac{q}{2}\rfloor)\mod q)$
\begin{itemize}
\item[*]  $\pmb{r}\longleftarrow\{0,1\}^m$
\end{itemize}

\item[-]  $Dec(sk,(\pmb{c_1},c_2)): c_2-\pmb{s}^T\pmb{c_1}=\pmb{e}^T\pmb{r}+\mu\lfloor\frac{q}{2}\rfloor$
\end{itemize}

\section{Overview of Fully Homomorphic Encryption (FHE)}

Let us consider the scenario shown in \ref{com}. A client has a secret value $x$. The client wants the server do some computation on $x$ without revealing what $x$ is. Firstly, a ciphertext $ct=Enc(x)$ is sent to the server along with the desired function $f$. Then the server could compute a new ciphertext $ct^*=Enc(f(x))$ by evaluating another function $g$ on $ct$ where $g$ is publicly computable from $f$. After receiving $ct^*$ from the server, the client can use its secret key $sk$ to get the desired result of $f(x)$.

\begin{figure}[!htbp]
\begin{center}
\includegraphics[width=0.9\textwidth]{client-server.pdf}
\end{center}
\caption{Outsourced Computation}
\label{com}
\end{figure}

A homomorphic encryption  can be used for privacy-preserving outsourced storage and computation. It allows operations and analysis on encrypted data without revealing the original one, which removes the privacy barriers in several real-life applications.

In the following sections, we will cover:
\begin{itemize}
\item The definition of FHE
\item Building additively homomorphic PKE based on LWE
\item Making the additively homomorphic PKE scheme also multiplicative homomorphic
\end{itemize}

\section{Definition of FHE}

\begin{definition} Let $\mathcal{C}$ be a class of circuits where for each $f\in\mathcal{C}$, $f:\{0,1\}^n \rightarrow \{0,1\}$. An encryption scheme $(KeyGen, Enc, Dec, Eval)$ is \textbf{$\mathcal{C}$-homomorphic} if $\forall f\in \mathcal{C}$, all ciphertexts $ct_1, \dots, ct_n$, $Eval(f, ct_1,\dots,ct_n)=ct^*$ such that if  $\forall i$, $\exists m_i, r_i$ s.t. $ct_i=Enc(m_i;r_i)$, then $Dec_{sk}(ct^*)=f(m_1,\dots,m_n)$ and the scheme is IND-CPA secure.
\end{definition}
At a high level, given ciphertexts $ct_1,\dots,ct_n$ that encrypt $m_1,\dots,m_n$, FHE should allow anyone to output a ciphertext  $ct^*$ that encrypts $f(m_1,\dots,m_n)$ for any desired function $f$ by evaluating another function $g$ which is publicly computable from $f$. Thus, the key holder could use the secret key $sk$ to decrypt $ct^*$ and get the result of $f(m_1,\dots,m_n)$.

Note that each function $f:\{0,1\}^n\rightarrow\{0,1\}^k$ can be split into $f_1,\dots,f_k$ where $\forall i$, $f_i:\{0,1\}^n\rightarrow\{0,1\}$ and also we can generalized the definition by regulating the input length of circuits in $\mathcal{C}$ from $n$ to $poly(n)$.

 In \cite{rothblum2011homomorphic}, they showed how to transform any additively homomorphic SKE into PKE. Since homomorphic SKE generically implies homomorphic PKE, without loss of generality, we only talk about how to build homomorphic PKE in this lecture.

\section{Construction of Fully Homomorphic Encryption:}
As described previously, FHE is an encryption scheme $(KeyGen, Enc, Dec)$
with an additional algorithm called $Eval$. In particular, we want to construct
such a $Eval$ namely for two operations, Addition and Multiplication. Constructing
such a FHE scheme which must work for \textbf{all} functions $f$ might seem like daunting task,
but can use the following fact to ease our task.
\begin{fact}
It turns out that all functions can be expressed by arithmetic circuits consisting
of only addition and multiplication gates. Therefore, we only implement our FHE operations for
addition and Multiplication. We can recursively compute every gate  in the arithmetic circuit
homomorphically to get the output of the function.
\end{fact}

\begin{definition}
An arithmetic circuit over a field $\mathbb{Z}_q$ is a directed
acyclic graph whose vertices are called gates. Gates of incoming degree $0$ are inputs to
the circuit. All other gates are labelled $+$ or $x$.
\end{definition}
We usually consider arithmetic circuits with fan-in $2$, in which
case all of the $+$ and $x$ gates have in-degree $2$.

\begin{remark}
Even though Fully Homomorphic encryption scheme is our actual goal, in practice we also consider
a simplification leveled fully homomorphic encryption scheme. Leveled FHE does not allow us to
compute arbitrary functions $f$ but only functions with a priori known depth $d$. Informally, when
we already know what is the most complex(in terms of depth of the arithmetic circuit) and use that
in the construction of our FHE scheme.
\end{remark}

Let us start with the simplest possible way to build a FHE for the addition operation.
For simplicity, let us consider that we want to single bit numbers
and output a single bit number. This is equivalent to implementing the XOR operation.

\section{FHE: Addition Operation}

Consider an encryption of message $\mu_1$ under the public key ($\pmb{s}^{T}\pmb{A} + \pmb{e}^{T}, \pmb{A}$). We call
$\pmb{s}^{T}\pmb{A} + \pmb{e}^{T}$ as $\pmb{b}$. One intuitive naive way to implementing addition might be addition of
ciphertexts

$$c_1 = ( \pmb{Ar_1}, \pmb{br_1} + \mu_1\floor{\frac{q}{2}} )$$
$$c_2 = ( \pmb{Ar_2}, \pmb{br_2} + \mu_2\floor{\frac{q}{2}} )$$
$$c_{add} = c_1 + c_2 = ( \pmb{A}(\pmb{r_1} + \pmb{r_2}), \pmb{b}(\pmb{r_1} +\pmb{r_2}) + (\mu_1 + \mu_2)\floor{\frac{q}{2}} )$$

It is possible to extend this to multi-bit XOR outputs by simply repeating
the circuit multiple times. However, it would only help in computing XOR for two $k$ bit numbers.
Let us try to decrypt the ciphertext $c_{add}$ and check what it decrypts to:

\begin{align*}
Dec(sk,c_{add})
&= \pmb{b}(\pmb{r_1} +\pmb{r_2}) + (\mu_1 + \mu_2)\floor{\frac{q}{2}}-\pmb{s}^T\pmb{A}(\pmb{r_1} + \pmb{r_2})\\
&=(\pmb{s}^{T}\pmb{A} + \pmb{e}^{T})(\pmb{r_1} +\pmb{r_2}) + (\mu_1 + \mu_2)\floor{\frac{q}{2}}-\pmb{s}^T\pmb{A}(\pmb{r_1} + \pmb{r_2})\\
&=\pmb{e}^{T}(\pmb{r_1} +\pmb{r_2}) + (\mu_1 + \mu_2)\floor{\frac{q}{2}}
\end{align*}

So applying the decryption algorithm we get $\mu_1$ + $\mu_2$ given the total error
is small $e_1 +e_2 \leq q/4$ where $e_i=\parallel\pmb{e}^{T}\pmb{r_i}\parallel$). The important observation to note here is to perform addition on
two ciphertexts we need to assume hardness of LWE for stronger security parameters.
Therefore, if we want to perform $l$ addition operations, we would have to keep our $max(e_i) \leq \floor{\frac{q}{2}}/l$.

\begin{figure}[!htbp]
\begin{center}
\includegraphics[width=0.9\textwidth]{error.pdf}
\end{center}
\caption{Decryption Error}
\label{error}
\end{figure}

\begin{corollary}
To compute addition of $k$ bit numbers $\mu_1$ and $\mu_2$, we must modify the encryption scheme by changing the factor which is multiplied with the plaintext from $\frac{q}{2}$ to $\frac{q}{2^{k+1}}$.

$$c_1 = ( \pmb{Ar_1}, \pmb{br_1} + \mu_1\floor{\frac{q}{2^{k+1}}} )$$
$$c_2 = ( \pmb{Ar_2}, \pmb{br_2} + \mu_2\floor{\frac{q}{2^{k+1}}} )$$
$$c_{add} = c_1 + c_2 = ( \pmb{A}(\pmb{r_1} + \pmb{r_2}), \pmb{b}(\pmb{r_1} +\pmb{r_2}) + (\mu_1 + \mu_2)\floor{\frac{q}{2^{k+1}}} )$$

On the basis of the similar argument described above, the error of the equations for 1 addition be
constrained by $e_i \leq \frac{q}{2^{k+1}}$.
\end{corollary}
\begin{remark}
It is also possible to implement a similar addition for the private key encryption  scheme
using LWE. That is, adding two ciphertexts $c_1$ and $c_2$ corresponding to $\mu_1$ and $\mu_2$
would also result in encryption of message $\mu_1 + \mu_2$.
\end{remark}

\begin{remark}
A natural question which arises from the above discussion is about the similarity between XOR and addition operations.
\end{remark}

\section{Towards FHE multiplication}

\begin{theorem}
Statement here
\end{theorem}

\begin{lemma}
Statement here
\end{lemma}

\begin{corollary}
Statement here
\end{corollary}

\begin{proposition}
Statement here
\end{proposition}

\begin{fact}
Statement here
\end{fact}

\begin{claim}
Statement here
\end{claim}

\begin{definition}
Statement here
\end{definition}

\begin{example}
Statement here
\end{example}

\begin{assumption}
Statement here
\end{assumption}

\begin{remark}
Statement here
\end{remark}

\begin{conjecture}
Statement here
\end{conjecture}

\begin{openproblem}
Statement here
\end{openproblem}

\begin{problem}
Statement here
\end{problem}

\noindent
\begin{align}
a = a_1+a_2+\cdots+a_n.
\end{align}
\noindent
For proofs, use the provided {\tt proof} environment,
illustrated below.

\begin{proof}
Proof goes here.
\end{proof}

\section*{Acknowledgement}
These scribe notes were prepared by editing a light modification of the template designed by Alexander Sherstov.
%retain this acknowledgement in all scribe notes.

\bibliographystyle{abbrv}
\bibliography{template}

\end{document}


\section{INSTANTON}
\label{INSTANTON}
\begin{ipifield}{}%
{A class for instanton calculations}%
{}%
{\ipiitem{mode}%
{Use the full or half of the ring polymer during the optimization}%
{default: `rate'; data type: string; options: `rate', `splitting'; }%
}
\begin{ipifield}{opt}%
{The geometry optimization algorithm to be used}%
{default: `None'; data type: string; options: `nichols', `NR', `lbfgs', `None'; }%
{}
\end{ipifield}
\begin{ipifield}{hessian\_asr}%
{Removes the zero frequency vibrational modes depending on the symmerty of the system.}%
{default: `none'; data type: string; options: `none', `poly', `crystal'; }%
{}
\end{ipifield}
\begin{ipifield}{alt\_out}%
{Alternative output:Prints different formatting of outputs for geometry, hessian and bead potential energies.
                                               All quantities are also accessible from typical i-pi output infrastructure.
                                               Default to 1, which prints every step. -1 will suppress the output (except the last one).
                                               Any other positive number will set the frequency (in steps) with which the quantities are
                                               written to file.
                                               The instanton geometry is printed in xyz format and the distances are in angrstroms
                                               The hessian is printed in one line with the following format:
                                               h1\_1,h2\_1,\ldots ,hN\_1,   h2\_2,h2\_2,hN\_2,   \ldots .   ,h1\_d,h2\_d,\ldots ,hN\_d.
                                               Where N represents the total number of replicas, d the number of dimension of each replica (3*n\_atoms) and
                                               hi\_j means the row j of the physical hessian corresponding to the replica i.
                                               The physical hessian uses a convention according to the positions convention used in  i-pi.
                                               Example of 2 particles, the first two rows of the physical hessian reads:
                                               'H\_x1\_x1, H\_x1\_y1, H\_x1\_z1, H\_x1\_x2, H\_x1\_y2,H\_x1\_z2'
                                               'H\_x2\_x1, H\_x2\_y1, H\_x2\_z1, H\_x2\_x2, H\_x2\_y2,H\_x2\_z2' }%
{default:  1 ; data type: integer; }%
{}
\end{ipifield}
\begin{ipifield}{prefix}%
{Prefix of the output files.}%
{default: `INSTANTON'; data type: string; }%
{}
\end{ipifield}
\begin{ipifield}{energy\_shift}%
{Set the zero of energy.}%
{dimension: energy; default:  0.0 ; data type: float; }%
{\ipiitem{units}%
{The units the input data is given in.}%
{default: `automatic'; data type: string; }%
}
\end{ipifield}
\begin{ipifield}{delta}%
{Initial stretch amplitude.}%
{default:  0.1 ; data type: float; }%
{}
\end{ipifield}
\begin{ipifield}{old\_pos}%
{The previous step positions during the optimization. }%
{dimension: length; default:  [ ] ; data type: float; }%
{\ipiitem{units}%
{The units the input data is given in.}%
{default: `automatic'; data type: string; }%
\ipiitem{shape}%
{The shape of the array.}%
{default:  (0,) ; data type: tuple; }%
\ipiitem{mode}%
{If 'mode' is 'manual', then the array is read from the content of 'cell' takes a 9-elements vector containing the cell matrix (row-major). If 'mode' is 'abcABC', then 'cell' takes an array of 6 floats, the first three being the length of the sides of the system parallelopiped, and the last three being the angles (in degrees) between those sides. Angle A corresponds to the angle between sides b and c, and so on for B and C. If mode is 'abc', then this is the same as for 'abcABC', but the cell is assumed to be orthorhombic. 'pdb' and 'chk' read the cell from a PDB or a checkpoint file, respectively.}%
{default: `manual'; data type: string; options: `manual', `file'; }%
}
\end{ipifield}
\begin{ipifield}{old\_pot}%
{The previous step potential energy during the optimization}%
{dimension: energy; default:  [ ] ; data type: float; }%
{\ipiitem{units}%
{The units the input data is given in.}%
{default: `automatic'; data type: string; }%
\ipiitem{shape}%
{The shape of the array.}%
{default:  (0,) ; data type: tuple; }%
\ipiitem{mode}%
{If 'mode' is 'manual', then the array is read from the content of 'cell' takes a 9-elements vector containing the cell matrix (row-major). If 'mode' is 'abcABC', then 'cell' takes an array of 6 floats, the first three being the length of the sides of the system parallelopiped, and the last three being the angles (in degrees) between those sides. Angle A corresponds to the angle between sides b and c, and so on for B and C. If mode is 'abc', then this is the same as for 'abcABC', but the cell is assumed to be orthorhombic. 'pdb' and 'chk' read the cell from a PDB or a checkpoint file, respectively.}%
{default: `manual'; data type: string; options: `manual', `file'; }%
}
\end{ipifield}
\begin{ipifield}{hessian}%
{(Approximate) Hessian.}%
{default:  [ ] ; data type: float; }%
{\ipiitem{shape}%
{The shape of the array.}%
{default:  (0, 0) ; data type: tuple; }%
\ipiitem{mode}%
{If 'mode' is 'manual', then the array is read from the content of 'cell' takes a 9-elements vector containing the cell matrix (row-major). If 'mode' is 'abcABC', then 'cell' takes an array of 6 floats, the first three being the length of the sides of the system parallelopiped, and the last three being the angles (in degrees) between those sides. Angle A corresponds to the angle between sides b and c, and so on for B and C. If mode is 'abc', then this is the same as for 'abcABC', but the cell is assumed to be orthorhombic. 'pdb' and 'chk' read the cell from a PDB or a checkpoint file, respectively.}%
{default: `manual'; data type: string; options: `manual', `file'; }%
}
\end{ipifield}
\begin{ipifield}{corrections\_lbfgs}%
{The number of past vectors to store for L-BFGS.}%
{default:  20 ; data type: integer; }%
{}
\end{ipifield}
\begin{ipifield}{tolerances}%
{Convergence criteria for optimization.}%
{}%
{}
\end{ipifield}
\begin{ipifield}{scale\_lbfgs}%
{Scale choice for the initial hessian.
                                                       0 identity.
                                                       1 Use first member of position/gradient list.
                                                       2 Use last  member of position/gradient list.}%
{default:  2 ; data type: integer; }%
{}
\end{ipifield}
\begin{ipifield}{glist\_lbfgs}%
{List of previous gradient differences for L-BFGS, if known.}%
{default:  [ ] ; data type: float; }%
{\ipiitem{shape}%
{The shape of the array.}%
{default:  (0,) ; data type: tuple; }%
\ipiitem{mode}%
{If 'mode' is 'manual', then the array is read from the content of 'cell' takes a 9-elements vector containing the cell matrix (row-major). If 'mode' is 'abcABC', then 'cell' takes an array of 6 floats, the first three being the length of the sides of the system parallelopiped, and the last three being the angles (in degrees) between those sides. Angle A corresponds to the angle between sides b and c, and so on for B and C. If mode is 'abc', then this is the same as for 'abcABC', but the cell is assumed to be orthorhombic. 'pdb' and 'chk' read the cell from a PDB or a checkpoint file, respectively.}%
{default: `manual'; data type: string; options: `manual', `file'; }%
}
\end{ipifield}
\begin{ipifield}{old\_direction}%
{The previous direction in a CG or SD optimization.}%
{default:  [ ] ; data type: float; }%
{\ipiitem{shape}%
{The shape of the array.}%
{default:  (0,) ; data type: tuple; }%
\ipiitem{mode}%
{If 'mode' is 'manual', then the array is read from the content of 'cell' takes a 9-elements vector containing the cell matrix (row-major). If 'mode' is 'abcABC', then 'cell' takes an array of 6 floats, the first three being the length of the sides of the system parallelopiped, and the last three being the angles (in degrees) between those sides. Angle A corresponds to the angle between sides b and c, and so on for B and C. If mode is 'abc', then this is the same as for 'abcABC', but the cell is assumed to be orthorhombic. 'pdb' and 'chk' read the cell from a PDB or a checkpoint file, respectively.}%
{default: `manual'; data type: string; options: `manual', `file'; }%
}
\end{ipifield}
\begin{ipifield}{old\_force}%
{The previous step force during the optimization}%
{dimension: force; default:  [ ] ; data type: float; }%
{\ipiitem{units}%
{The units the input data is given in.}%
{default: `automatic'; data type: string; }%
\ipiitem{shape}%
{The shape of the array.}%
{default:  (0,) ; data type: tuple; }%
\ipiitem{mode}%
{If 'mode' is 'manual', then the array is read from the content of 'cell' takes a 9-elements vector containing the cell matrix (row-major). If 'mode' is 'abcABC', then 'cell' takes an array of 6 floats, the first three being the length of the sides of the system parallelopiped, and the last three being the angles (in degrees) between those sides. Angle A corresponds to the angle between sides b and c, and so on for B and C. If mode is 'abc', then this is the same as for 'abcABC', but the cell is assumed to be orthorhombic. 'pdb' and 'chk' read the cell from a PDB or a checkpoint file, respectively.}%
{default: `manual'; data type: string; options: `manual', `file'; }%
}
\end{ipifield}
\begin{ipifield}{biggest\_step}%
{The maximum step size during the optimization.}%
{default:  0.4 ; data type: float; }%
{}
\end{ipifield}
\begin{ipifield}{hessian\_update}%
{How to update the hessian after each step.}%
{default: `powell'; data type: string; options: `powell', `recompute'; }%
{}
\end{ipifield}
\begin{ipifield}{hessian\_init}%
{How to initialize the hessian if it is not fully provided.}%
{default: `false'; data type: string; options: `true', `false'; }%
{}
\end{ipifield}
\begin{ipifield}{ls\_options}%
{"Options for line search methods. Includes:
                                  tolerance: stopping tolerance for the search,
                                  iter: the maximum number of iterations,
                                  step: initial step for bracketing,
                                  adaptive: whether to update initial step.
                                  }%
{}%
{}
\end{ipifield}
\begin{ipifield}{qlist\_lbfgs}%
{List of previous position differences for L-BFGS, if known.}%
{default:  [ ] ; data type: float; }%
{\ipiitem{shape}%
{The shape of the array.}%
{default:  (0,) ; data type: tuple; }%
\ipiitem{mode}%
{If 'mode' is 'manual', then the array is read from the content of 'cell' takes a 9-elements vector containing the cell matrix (row-major). If 'mode' is 'abcABC', then 'cell' takes an array of 6 floats, the first three being the length of the sides of the system parallelopiped, and the last three being the angles (in degrees) between those sides. Angle A corresponds to the angle between sides b and c, and so on for B and C. If mode is 'abc', then this is the same as for 'abcABC', but the cell is assumed to be orthorhombic. 'pdb' and 'chk' read the cell from a PDB or a checkpoint file, respectively.}%
{default: `manual'; data type: string; options: `manual', `file'; }%
}
\end{ipifield}
\begin{ipifield}{hessian\_final}%
{Decide if we are going to compute the final big-hessian by finite difference.}%
{default: `false'; data type: string; options: `false', `true'; }%
{}
\end{ipifield}
\end{ipifield}


\documentclass[12pt]{article}

\include{preamble}

\title{Math 621 Fall 2017 \\ Final Examination}
\author{Professor Adam Kapelner}

\date{December 19, 2017}

\begin{document}
\maketitle

\noindent Full Name \line(1,0){410}% ~~~ Section (A, B or C)~ \line(1,0){30}

\thispagestyle{empty}

\section*{Code of Academic Integrity}

\footnotesize
Since the college is an academic community, its fundamental purpose is the pursuit of knowledge. Essential to the success of this educational mission is a commitment to the principles of academic integrity. Every member of the college community is responsible for upholding the highest standards of honesty at all times. Students, as members of the community, are also responsible for adhering to the principles and spirit of the following Code of Academic Integrity.

Activities that have the effect or intention of interfering with education, pursuit of knowledge, or fair evaluation of a student's performance are prohibited. Examples of such activities include but are not limited to the following definitions:

\paragraph{Cheating} Using or attempting to use unauthorized assistance, material, or study aids in examinations or other academic work or preventing, or attempting to prevent, another from using authorized assistance, material, or study aids. Example: using an \emph{unauthorized} cheat sheet in a quiz or exam, altering a graded exam and resubmitting it for a better grade, etc.
\\

\noindent I acknowledge and agree to uphold this Code of Academic Integrity. \\

\begin{center}
\line(1,0){250} ~~~ \line(1,0){100}\\
~~~~~~~~~~~~~~~~~~~~~signature~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ date
\end{center}

\normalsize

\section*{Instructions}

This exam is 120 minutes and closed-book. You are allowed three 8.5'' $\times$ 11'' page (front and back) of a \qu{cheat sheet.} You may use a graphing calculator of your choice. Please read the questions carefully. If the question reads \qu{compute,} this means the solution will be a number otherwise you can leave the answer in choose, permutation, exponent, factorial or any other notation which could be resolved to a number with a computer. I advise you to skip problems marked \qu{[Extra Credit]} until you have finished the other questions on the exam, then loop back and plug in all the holes. I also advise you to use pencil. The exam is 100 points total plus extra credit. Partial credit will be granted for incomplete answers on most of the questions. \fbox{Box} in your final answers. Good luck!

\pagebreak

\problem Below are some theoretical questions.

\benum

\subquestionwithpoints{6} Let $U = F_X(X)$ where $F_X$ denotes the CDF of $X$, a continuous r.v. Show that $U \sim \stduniform$.  Hint: do not use the usual transformation of variables formula but begin with $F_U(u) = \prob{U \leq u}$ instead. \spc{6} %mid2

\subquestionwithpoints{10} If $X, Y \iid \stdnormnot$, show that $\prob{X < Y} = \half$. Get as far as you can. \spc{10} %mid1

\subquestionwithpoints{4} If $\X \sim \multinomial{6}{\bracks{\third ~ \half ~ \sixth}^\top}$, what is $\prob{\X = \bracks{3 ~3 ~3}^\top}$? \spc{6} %mid1

\subquestionwithpoints{6} If $\X \sim \multinomial{6}{\bracks{\third ~ \half ~ \sixth}^\top}$, what is $\corr{X_1}{X_2}$? Simplify but do not compute explicitly. \spc{3} %mid1

\subquestionwithpoints{5} If $X_1, ~X_2 \iid \stduniform$ and $T = X_1 + X_2$, fill in the square below with one of the following symbols: \qu{$=,~\geq,~\leq,~>,~<$} or write \qu{?} if it cannot be determined given the information provided.

\beqn
\prob{T \in \bracks{\half,~\frac{3}{2}}} ~~ \text{\Huge{$\square$}} ~~ \prob{T \notin \bracks{\half,~\frac{3}{2}}}
\eeqn~\spc{-1} %mid1

\subquestionwithpoints{10} If $\Xoneton \iid \stduniform$, prove that $X_{max} \convd 1$. \spc{10}

\subquestionwithpoints{5} If $\Xoneton \iid \stduniform$ and $X_{(2)}$ is the \emph{second smallest} $X$, what is the explicit PDF of $X_{(2)}$? \spc{1} %mid2

\subquestionwithpoints{10} [Extra credit] If $\Xoneton \iid \stduniform$ and $Y$ is the \emph{second smallest} $X$, show that $Y \convp 0$ without using the fact that $\forall r \geq 1~Y \convLp{r} 0 \Rightarrow Y \convp 0$. Leave this question for last. \spc{7}

\subquestionwithpoints{6} For discrete r.v. $X$, prove its ch.f. $\phi_X(t)$ exists for all $t$. \spc{5}

\subquestionwithpoints{4} Let $\Zoneton \iid \stdnormnot$ and let $\Z = \bracks{Z_1 ~ Z_2 ~ \ldots Z_n}^\top$. If

\beqn
X = \sqrt{2}\frac{Z_1}{\sqrt{Z_2^2 + Z_3^2}}\hspace{300px}
\eeqn

find the distribution of $X$. You do not need to provide the PDF, just the name of the distribution and its parameter(s). Write it above. \spc{0}

\subquestionwithpoints{4} Let $\Zoneton \iid \stdnormnot$ and let $\Z = \bracks{Z_1 ~ Z_2 ~ \ldots Z_n}^\top$. If

\beqn
X = 2\frac{Z_1^2}{{Z_2^2 + Z_3^2}}\hspace{300px}
\eeqn

find the distribution of $X$. You do not need to provide the PDF, just the name of the distribution and its parameter(s). Write it above. \spc{0}

\subquestionwithpoints{4} Under what circumstance(s) is the following true?

\beqn
\frac{\Xbar - \mu}{\frac{S}{\sqrt{n}}} \sim T_{n-1}
\eeqn

Note: $\Xbar$ is the estimator for the mean $\mu$ we discussed in class and $S$ is the biased estimator for the standard error $\sigma$ we discussed in class. This should be one sentence.\spc{1}

\subquestionwithpoints{6} Let $\Xoneton \iid \normnot{\mu}{\sigsq}$. Find the PDF of $Y = \sum_{i=1}^n \squared{X_i - \Xbar}$. \spc{3}

\subquestionwithpoints{4} Let $\Zoneton \iid \stdnormnot$ and let $\Z = \bracks{Z_1 ~ Z_2 ~ \ldots Z_n}^\top$. If $\X = A\Z$ where $A \in \reals^{n \times n}$ and $\rank{A} = n$, find the distribution of $\X$. You do not need to provide the PDF, just the name of the distribution and its parameter(s). If you are using vectors, be explicit in the dimensions. \spc{2}

\subquestionwithpoints{4} Let $\Zoneton \iid \stdnormnot$ and let $\Z = \bracks{Z_1 ~ Z_2 ~ \ldots Z_n}^\top$. If $\X = A^\top \Z - \c$ where $A \in \reals^{n \times n}$,  $\rank{A} = n$ and $\c \in \reals^n$, find the distribution  $\X$. You do not need to provide the PDF, just the name of the distribution and its parameter(s). If you are using vectors, be explicit in the dimensions. \spc{2}

\subquestionwithpoints{4} Let $\Zoneton \iid \stdnormnot$ and let $\Z = \bracks{Z_1 ~ Z_2 ~ \ldots Z_n}^\top$. If $\X = A\Z$ where $A \in \reals^{n \times n}$ and $\rank{A} = n$, find the distribution of $Y = \X^\top (A^{-1})^\top A^{-1} \X$. You do not need to provide the PDF, just the name of the distribution and its parameter(s). If you are using vectors, be explicit in the dimensions. \spc{2}

\subquestionwithpoints{8} Prove that for any r.v. $X$ and any constant $a$ that

\beqn
\prob{X \geq a} \leq \min_{t>0} \braces{ e^{-at} M_X(t)}
\eeqn

where $M_X(t)$ is the moment generating function of $X$. \spc{7}

\subquestionwithpoints{8} Let $X_n \sim \normnot{\oneover{n}}{\squared{\oneover{n}}}$. Prove that $X_n \convLp{2} 0$.  \spc{3}

\eenum

\end{document}


\documentclass[11pt]{article}
\usepackage[margin=1in]{geometry}
\usepackage{amsfonts,amsmath,amssymb}
\usepackage[none]{hyphenat}
\usepackage{fancyhdr}
\usepackage{graphicx}
\usepackage{float}
\usepackage{enumitem}
\usepackage{hyperref}
\usepackage[noline,boxed]{algorithm2e}
\usepackage[skins]{tcolorbox}
\usepackage[nottoc,notlot,notlof]{tocbibind}
\usepackage{xcolor}
\usepackage{mathtools}
\newcommand{\defeq}{\vcentcolon=}
\newcommand{\eqdef}{=\vcentcolon}

\pagestyle{fancy}
\fancyhead{}
\fancyfoot{}
\fancyhead[L]{\slshape \MakeUppercase{}}
\fancyhead[R]{\slshape}
\fancyfoot[C]{\thepage}
%\renewcommand{\headrulewidth}{0pt}
\renewcommand{\footrulewidth}{0pt}

%\parindent 0ex
\setlength{\parindent}{4em}
\setlength{\parskip}{0em}
\renewcommand{\baselinestretch}{1.5}
\graphicspath{images/}
\begin{document}

\begin{titlepage}
\begin{center}
\vspace*{0.5cm}
\Large{\textbf{Design and Analysis of Algorithms}}\\
\Large{\textbf{Theory Project}}\\
\vfill
\line(1,0){400}\\[1mm]
\huge{\textbf{Fractional Cascading}}\\[3mm]
\Large{\textbf{An Algorithmic Approach}}\\[1mm]
\line(1,0){400}\\
\vfill
By \\
IMT2019051 Mani Nandadeep Medicharla \\
IMT2019063 R Prasannavenkatesh \\
IMT2019525 Vijay Jaisankar \\
\end{center}
\end{titlepage}

\tableofcontents
\thispagestyle{empty}
\clearpage
\setcounter{page}{1}

\section{Abstract}
        In this paper, we investigate the \textit{Fractional Cascading} technique which is used in building range trees and for fast searching of an element in multiple arrays. In this venture, we introduce and examine \textit{Linear Search}, \textit{Binary Search}, \textit{Bridge Building} and \textit{Fractional Cascading}. We also look at some of the \textit{applications} of this technique, and give a video demonstration of its capabilities.

\section{Problem Statement}
\textit{Fractional cascading: }You are given an input of $k$ ordered lists of numbers, each of size $n$ as well as a query value $x$. The problem's output is
to return, for each list, \textit{True} if the query value appears in the list and
\textit{False} if it does not. For example, if the input is:
\begin{enumerate}[label=(\alph*)]
    \item List $L_1$: $[3,4,6]$
    \item List $L_2$: $[2,6,7]$
    \item List $L_3$: $[2,4,9]$
\end{enumerate}
and the query value is 4, then the expected output is [\textit{True},\textit{False},\textit{True}]. \\
\textbf{Give an algorithm to solve the fractional cascading problem.}

\section{Brute Force}
\subsection{Linear Search}

Linear search is the most basic search technique, wherein we sequentially compare each array element to the target element. In the worst case of the target element not coinciding with \textit{any} list element, the algorithm would reach the end of the list and we would report an unsuccessful search. \\
As each element is compared at most once, the time complexity is $O(n)$, where $n$ is the size of the list. \\ \\

This algorithm forms the basis of the simplest solution to our problem: We just run linear search on each of the $k$ lists. If we have $q$ queries, this takes $O(q \cdot k \cdot n)$ time, which is a lot and in real world situations, if $k$, $q$, and $n$ are even moderately large, the time taken would become astronomical. \\

\subsection{Pseudocode}
\begin{tcolorbox}[blanker,width=(\linewidth-3.5cm)]
\begin{algorithm}[H]
    \SetAlgoLined
    \KwData{k arrays of size n and a query element x}
    \KwResult{Boolean array regarding whether element is present in the indiced array or not}

    \SetKwFunction{LinearSearch}{LinearSearch}
    \SetKwFunction{FMain}{Main}

    \SetKwProg{Fn}{Function}{:}{\KwRet \textit{False}}
    \Fn{\LinearSearch{Array,x}}
    {
        \For{$i\leftarrow 0$ \KwTo $n$}
        {
            \If{\textit{Array}[i] == x}
                {\KwRet \textit{True}\;}
        }
    }

    \SetKwProg{Fn}{Function}{:}{\KwRet 0}
    \Fn{\FMain}{
        output = [] \;
        \For{$i\leftarrow 0$ \KwTo $k$}
        {
            output.append(linearSearch(input[i],x));
        }
    }

% \caption{Brute Force method by doing k linear searches}
\end{algorithm}
\end{tcolorbox}

Note that, however, this approach does not take into account any relevant information given to us in the question which can speed up this algorithm. It is given that the \textit{lists are sorted}, so we can exploit this property and employ a faster searching technique to solve this problem in a better way: \textit{Binary Search}

%------------------------------------------------------------------------------%

\section{Improved Brute Force}
\subsection{Binary Search}

Binary search is another searching algorithm that works correctly only on sorted arrays. \\
It begins by comparing the target element with the element at the middle of the list.
\begin{itemize}
    \item If they are equal, we have found the target in the list
    \item If the target is larger, and as the list is sorted, we must now turn our attention to the \textit{right half} of the list
    \item Similarly, if the target is smaller, we must focus on the \textit{left half} of the list.
\end{itemize}

In the worst case, Binary Search will take $O(\log n)$ comparisons, where n is the size of the list. \\ \\

To improve the performance of the Brute Force subroutine, we replace the Linear Search subroutine with the aforementioned Binary Search subroutine. If we have $q$ queries, this takes $O(q \cdot k \cdot \log n)$ time, which is certainly a lot better than the initial brute force algorithm, but can yet be improved further.

\subsection{Proof of correctness}
We prove the correctness of binary search by the \textit{method of strong induction}

Let $p(k) \defeq $ Binary search works on an array of size $k$
\begin{enumerate}
    \item \textbf{Base Case}
    \begin{itemize}
        \item If $k=1$, $p(1)$ is trivially true; as either $array[mid] = x$, or not
        \item Hence, $p(1)$ is true
    \end{itemize}
    \item \textbf{Induction Hypothesis}
    \begin{itemize}
        \item Let $p(1) \land p(2) \land \dots \land p(z) = $ True
        \item This means that binary search works for all arrays of size \textit{at most} $z$.
    \end{itemize}
    \item \textbf{Induction Step}
    \begin{itemize}
        \item Now, we look at an array of size $z+1$. Here, we have three cases as outlined in the algorithm:
        \begin{itemize}
            \item If $array[mid] = x$, we are done as we can return prematurely from the loop
            \item Else, our new search space will become roughly half(\textit{Note: It will be exactly $\frac{1}{2}$ of the original size, if the size of the array is a power of 2}). In any case, the size of the ``new'' array we should search in, is $\leq z$.
            \item Now, from our \textit{Strong Induction Hypothesis}, we are done as all of their previous cases are true.
            \item So, $p(1) \land p(2) \land \dots \land p(z) \rightarrow p(z+1)$
        \end{itemize}
    \end{itemize}
    \item \textbf{So, we have proved the correctness of the binary search algorithm by strong induction.}
\end{enumerate}

\subsection{Pseudocode}
\begin{tcolorbox}[blanker,width=(\linewidth-3.5cm)]
\begin{algorithm}[H]
    \SetAlgoLined
    \KwData{k arrays of size n and a query element x}
    \KwResult{Boolean array regarding whether element is present in the indiced array or not}

    \SetKwFunction{BinarySearch}{BinarySearch}
    \SetKwFunction{FMain}{Main}

    \SetKwProg{Fn}{Function}{:}{\textbf{end Function}}
    \Fn{\BinarySearch{Array,x,left,right}}
    {
        \If{right $\geq$ left}{
            mid=$\frac{(left+right)}{2}$\;
            \If{Array[mid] == x}{
                \KwRet \textit{True}\;
            }
            \If{Array[mid] $>$ x}{
                BinarySearch(Array,x,left,mid-1)\;
            }
            \Else{
                BinarySearch(Array,x,mid+1,right)\;
            }
        }
        \Else{
            \KwRet \textit{False}\;
        }

    }

    \SetKwProg{Fn}{Function}{:}{\KwRet 0}
    \Fn{\FMain}{
        output = [] \;
        \For{$i\leftarrow 0$ \KwTo $k$}
        {
            output.append(BinarySearch(input[i],x));
        }
    }

% \caption{Brute Force method by doing k linear searches}
\end{algorithm}
\end{tcolorbox}

%------------------------------------------------------------------------------%

\section{Bridge Building}
\subsection{Introduction to bridges}
A \textit{bridge} is a pointer from an element $a_i$ of $A_i$ to an element $a_j$ of $A_{i+1}$ where  $|a_i-a_j|$ is \textbf{small}, where $A$ is the list and $a$ represents an element of the array. By \textit{small}, we mean an element that is either of the same value, or with the smallest difference to the one considered as reference.$^{\cite{Fractional cascading notes by Prof Prof.Roberto Tamassia}}$ \\

Once we locate the position to a query in a array, we should be able to \textbf{follow a bridge} to a element that is close to the answer in the next array. \\
In the best case, we follow a bridge from the answer to the query in $A_i$ to the endpoint of the bridge in $A_{i+1}$ and then from there locate the answer in $A_{i+1}$, all in constant time. If we can do \textit{this},then once we have the answer in $A_1$ we can find the answer in the remaining $k-1$ sorted arrays in $O(k)$ time complexity. \\

From a technical standpoint, we \textit{implement} this method as follows: \\
For every element $e$ in the first array, give $e$ a pointer to the element with the \textit{same value} in the second array or if the value doesn't exist, the \textit{predecessor} (\textit{Note}: predecessor(x) = $v \in $ Search space where $x-v$ is minimum, and $x>v$.). This is called \textit{bridge building} between $A_i$ and $A_{i+1}$. Then, once we've found the item in the first array, we can just follow these pointers down in order to figure out where the item might be located in all the other arrays. To find the answer in $A_1$, we can just use a balanced binary search tree,thus making the overall time complexity of our algorithm $O(\log n + k)$ per query.

\begin{figure}[H]
    \centering
    \includegraphics[]{Images/Screenshot_2021-03-15 You could have invented fractional cascading Inside 245-5D.png}
    \caption{Process of bridge building for a given question.$^{\cite{Blog by Edward Z. Yang}}$}
    \label{fig:label}
\end{figure}

From the image, we can see the bridge building in action, where the lines between the arrays represent bridges. We can clearly see the predecessor linkages and how we can follow the pointers down and generate the output for all of the arrays. \\ \\
In this example, we are searching for 8 in all 3 arrays. We can clearly see the path of pointers we should traverse, as outlined in red.Note that, in the last array, the most plausible candidate element is 7 and not 8, so we would return \textit{False} for this array, and hence our overall output will be [\textit{True},\textit{True},\textit{False}] as the element 8 is only present in the first two arrays and not in the third one.

\subsection{Pseudocode}
\begin{tcolorbox}[blanker,width=(\linewidth-3.5cm)]
\begin{algorithm}[H]
    \SetAlgoLined
    \KwData{k arrays of size n and a query element x}
    \KwResult{Boolean array regarding whether element is present in the indiced array or not}

    \SetKwFunction{BuildBridges}{BuildBridges}
    \SetKwFunction{FMain}{Main}

    \SetKwProg{Fn}{Function}{:}{\textbf{end Function}}
    \Fn{\BuildBridges{Array,x}}
    {
        \For{$i\leftarrow 0$ \KwTo $k-1$}{
            \For{$j\leftarrow 0$ \KwTo $n$}{
                Build bridge from Array[i][j] to Array[i+1][x] where $|$Array[i+1][y] - Array[i][j]$|$ is small. In this approach, if both predecessor and successor exists, then we take predecessor first.
            }
        }
    }

    \SetKwProg{Fn}{Function}{:}{\KwRet 0}
    \Fn{\FMain}{
        output = [] \;
        BuildBridges(input,x) \;
        output.append(BinarySearch(Array[i],x)) \;
        Once the element is found in the first array, follow the bridge path till the final array and append it to the output. \;
    }

% \caption{Brute Force method by doing k linear searches}
\end{algorithm}
\end{tcolorbox}
% \subsection{Proof of correctness}
\subsection{Shortcomings}

This method seems like a very interesting and efficient alternative to solve this problem. However, there are some glaring weaknesses to this approach, the most important one being the fact that certain classes of inputs render this method useless. \\
In particular, if a later list is completely \textit{in between} two elements of the first list, we have to redo the entire search, as the pointer pre-processing gives us no information that we didn't already know.

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{Images/Screenshot_2021-03-15 fc dvi - notes08 pdf.png}
    \caption{An example where a later list is completely \textit{in between} two elements of the first list illustrated via bridge building.$^{\cite{Fractional cascading notes by Prof Prof.Roberto Tamassia}}$}
    \label{fig:label}
\end{figure}

\textbf{Let's consider a simple example to elucidate this statement} \\
Consider the case where $k = 2$. Everything would be better if only we could guarantee that the first list contained the \textit{right elements} to give you useful information about the second array. The bridges would be built with only the maximum and minimum of the lower array as endpoints leading us to either search the lower array again or we could just \textit{merge the arrays} naively, but if we did this, we'd end up with an array of size $k \cdot n$, which is not optimal at all, if $k$ is even moderately large.

Even if each time we find an answer in the sorted Array $A_i$, we follow the bridge pointer from it as well as from the key above it we are still left with the entire contents of array $A_{i+1}$ to search. This continues through the entire set of $k$ arrays. If we search each array by doing a linear scan from the point at which the bridge told us to begin,the total search time will be $O(n \cdot k)$ .Even if we build a balanced search tree over all elements in $A_{i+1}$ that appear between two consecutive bridge pointers from $A_i$ the query time will still be $O(k \cdot \log n)$ which is similar to performing $k$ binary searches to get the query element.

%------------------------------------------------------------------------------%
\section{Fractional Cascading}

\subsection{Intuition behind Fractional Cascading}
Because of the shortcomings of the \textit{Bridge Building} algorithm, i.e the case where every element of the ``below'' array is in between all the elements of the ``above'' array.(\textbf{Note}: We say that array 1 is “above” array 2 and array 2 is “below” array 1. Therefore, an Array $j$ is below Array $i$ if $j > i$). In this case, we either have to do $k$ binary searches, or we have to merge all $n$ elements of the below arrays recursively and maintain bridges, to get the output. This will warrant a sub-optimal, extra time complexity of $O(k \cdot n)$, as we are going through \textit{each} array element iteratively to merge it.

To avoid this problem of merging all elements and getting $O(k \cdot n)$ time complexity, we start with the \textit{lowest} list in the sequence and select every $i$th element, where i = $\frac{1}{\alpha}$ and insert it into the array above it while still maintaining sorted order. We then mark that element as \textit{promoted}$^{\cite{Fractional Cascading Demo}}$ and keep a pointer from it, to its original position in the bottom list. This operation of taking every $i$th element and \textit{promoting} it to the array above it is called \textit{cascading}, and since we are only promoting a fraction of the elements, the algorithm is called \textbf{fractional cascading}.

For selecting, $\alpha$, we can choose between a variety of fractions, however $\alpha$ = $\frac{1}{2}$ seems most appropriate because, we will have to compare only 2 elements while searching, after prepocessing. This will also ensure that our time complexity stays low, and easily computable.

\subsection{Algorithmic Approach to Fractional Cascading}

Let the input be specified by $k$ $n$-element arrays, $A_1,A_2,\dots,A_k$, Let the query element value be $x$.
Let $M_1,M_2,\dots,M_k$ be the new \textit{merged arrays} such that $M_k = A_k$ and $\forall i < k$, $M_i$ is defined as the result of merging $M_i$ with every $\frac{1}{\alpha}$th element of $M_{i+1}$. \\ \\
As we're taking $\alpha$ = $\frac{1}{2}$, $M_i$ will be the result of merging $M_i$ with every alternate element of $M_{i+1}$. For every \textbf{cascaded element} of $M_i$ $\forall$ i $<$ k, we keep two pointers from each element which are derived as follows:
\begin{itemize}
    %\item If the element came from the same array, i.e, $A_i$, we keep a pointer to the nearest neighbouring elements which are cascaded from $M_{i+1}$
    \item If the element is non cascaded,i.e, If the element is from the Array $A_i$, then the first pointer points to the smallest cascaded element greater than the element in $A_i$ and the second pointer points to the largest element lesser than the element in $A_i$.

    \item If the element has been cascaded, we keep a pointer to the non cascaded predecessor and non cascaded successor of the element in $M_i$ to be able to efficiently find the next and previous element which is a member of the current array; and also a keep a bridge between $M_i$ and $M_{i+1}$ at the position where it is present in both of the arrays.
\end{itemize}
Additionally, we add bridges between the pseudo-elements ,i.e,  -$\infty$ and $\infty$ in consecutive arrays.If there is no key of the appropriate type above or below the key,the pointer points to the pseudo-keys at $\pm \infty$, whichever is appropriate.$^{\cite{Fractional cascading notes by Prof Prof.Roberto Tamassia}}$
These pointers helps to find the position of the query element x in $A_i$ and also in the cascaded arrays below in $O(1)$ time. \\ \\
\textbf{Note}: Since we are merging every alternate element of the below list to the current list, we have
$|M_i| = |A_i| +\frac{1}{2}|M_{i+1}|$, which in turn ensures that $|A_i| \leq 2n = O(n).$ \\

After we perform the aforementioned pre-processing, querying $x$ in all $k$ lists is done as follows:
First, we make a query for x in $M_1$ using a binary search in $O(\log n)$. Once we have found the position of $x$ in $M_1$, we use the \textit{cascaded pointers} to find the position of x in $M_2$. Generalising this step, once we found the position of x in $M_i$ where i $<$ k, we use the cascaded pointers to find the position of x in $M_{i+1}$. \\ \\
To find the location in $M_{i+1}$, we find the \textit{two neighbouring elements} in $M_{i}$ that came from $M_{i+1}$ using the pointers we had assigned during the pre-processing phase. Now, these elements will have \textit{exactly one element} between them in $M_{i+1}$. \textbf{Therefore, to find the exact location in $M_{i+1}$, we just have to do a simple comparison with only the intermediate element}. This is the significance of taking $\alpha$ = $\frac{1}{2}$ as we just have to perform only one comparison, which takes $O(1)$ time, and hence we can retrieve the location of $x$ in $A_i$ from its location in $M_i$ again in $O(1)$ time. \\ \\

Hence, the time to perform the pre-processing for fractional cascading is O(nk), the total search time per query is $O(k+log n)$ and, the total time taken by the Fractional Cascading algorithm is $O(q(k+logn))$ for $q$ queries, which is an improvement over the previous algorithms.

\subsection{Pseudocode}
\begin{tcolorbox}[blanker,width=(\linewidth-3.5cm)]
\begin{algorithm}[H]
    \SetAlgoLined
    \KwData{k arrays of size n and a query element x}
    \KwResult{Boolean array regarding whether element is present in the indiced array or not}

    \SetKwFunction{FFractionalCascading}{Fractional\_Cascading}
    \SetKwProg{Fn}{Function}{:}{\KwRet output}
    \Fn{\FFractionalCascading}{
        output = [] \;
        MergedArrays = [] \;
        % // Insert all elements of the last array at position 0 of MergedArrays % \;
        MergedArrays = MergedArrays.insert(0,all elements of the last array) \;
        MergedArrays = MergedArrays.insert(0,merge the below array with the above array by only taking alternate elements of the below array) \;
        Generate the boundary case predecessors and successors; $- \infty$ and $+ \infty$ \;
        %For every element assign a two sized array to hold the pointers to the locations. \;
        For every element in the merged arrays, assign locations based on the presence of that particular element in $A_i$ and $M_{i+1}$.If the element came from the same array, i.e, $A_i$, we keep a pointer to the nearest neighbouring element on either side from $M_{i+1}$. If the element has been cascaded, we keep a pointer to the non cascaded predecessor and successor of the element in $M_i$ and also a bridge between $M_i$ and $M_{i+1}$ at the position where it is present in both of the arrays.\;
        We then check for the position of the target element in the merged array, then we follow the pointers down to get the positions of the predecessors of the said target element in all of the k arrays. Let's call this array $positions[k]$ \;
        The last step is to scan through $positions[k]$, and see if the respective predecessor is actually the given target, or not. This generates the $[True,False]$ format given in the question and append it to the output array.
    }

% \caption{Brute Force method by doing k linear searches}
\end{algorithm}
\end{tcolorbox}

\subsection{Example}
\begin{enumerate}[label=(\alph*)]
    \item List $L_1$: $[3,4,6]$
    \item List $L_2$: $[2,6,7]$
    \item List $L_3$: $[2,4,9]$
\end{enumerate}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{Images/WhatsApp Image 2021-03-17 at 8.19.27 PM.jpeg}
    \caption{Fractional cascading pre processing for the given question.}
    \label{fig:label}
\end{figure}

The figure gives the final MergedArrays and we can also see how the elements are cascaded. Every cascaded element has a pointer to it's predecessor and successor which was initally present in the same array $A_i$ as shown by the red curved pointers and also has a bridge between merged arrays $M_i$ and $M_{i+1}$ depending on the location of that particular element in both the arrays. The red lines denote the bridges. Also the elements that are present in both $A_i$ and $M_i$ has a pointer to the nearest cascaded elements as shown by the blue and green lines in the figure.

Here, the search query element, $x = 4$. So we start at $M_1$ and search for 4 using a binary search which will in turn return the first 4 in the array, i.e, the element present in index 1. Now we will follow this index's pointer to the nearest cascaded element which is 4 present at index 2. However, this is a cascaded element from the below arrays and hence will have a bridge to the below arrays. So the algorithm follows the bridge and goes to index 1 of $M_2$. However, this elements is also cascaded, so we check whether the previous element is equal to 4 or not. The previous element in this case is 2 present at index 0 of $M_2$ which is not equal to the query element. So we again go to the nearest cascaded element via 2's pointer and reach the next merged array which is $M_3$. Here the bridge's endpoint is the query element and it will return true and the algorithm is terminated as we have reached the lowest array. Hence the output will be [\textit{True},\textit{False},\textit{True}]. \\

We have made a short walkthrough of the above example which can be found on youtube in this \href{https://www.youtube.com/watch?v=eUQtziH6cDo}{Link(click here)}$^\cite{Youtube Walkthrough}$

\subsection{Proof of Correctness}
To prove that our Fractional Cascading algorithm is correct, we perform the following steps.
\begin{itemize}
    \item We look at another simpler algorithm and prove its correctness, let's call this algorithm X.
    \item We show that our algorithm and X produce the same output. This forms the proof of correctness of our algorithm.
\end{itemize}

Note that we already have a candidate for algorithm X; Our \textit{improved brute force algorithm} which uses binary search which we have already proved earlier.

%proof of correctness of binary search.

We will follow the concept of universal generalization here; lets consider the $r^{th}$ array of our input sequence consisting of k arrays of size $n$ where r $<$ k. Let's also consider a target element $x$. Clearly there are two cases
\begin{itemize}
    \item x $\in$ array[r] : let's call this situation 1.
     \item x $\notin$ array[r] : let's call this situation 2.
\end{itemize}

\textbf{Let's start with situation 1},

Now there are two possibilities here when our control reaches array[r]
\begin{itemize}
    \item We find element $x$ and it is not cascaded from below then we can say that it returns \textit{true} which is exactly what algorithm X returns in the same situation.
    \item We find element $x$ but it is cascaded from below,then, our control moves into the nearest non-cascaded neighbours of $x$. As x $\in$ array[r], \textit{native x} will be one of the neighbours of the cascaded $x$. So, the control gets transferred to the native $x$, and this case gets \textit{transformed} into the previous case. Hence, we return \textit{true} which is exactly what algorithm X returns in the same situation.
\end{itemize} \\ \\ \\
Now, \textbf{let's continue with situation 2}.
Now, there are two possibilities here when our control reaches array[r]

\begin{itemize}
    \item We find element \textit{x} but it is cascaded from below. So we look at its nearest non-cascaded neighbours which are guaranteed not to be $x$(as $x$ $\notin$ array[r] by supposition). So we can return \textit{false}, which is exactly what algorithm X returns in this situation.
    \item We find an element which is not equal to $x$. Now, regardless of whether it's cascaded or not when we traverse to the pointers, we are guaranteed not to find $x$ so we return $false$, which is exactly what algorithm X returns in this situation.
\end{itemize}

Hence, the output given by the fractional cascading algorithm and the one given by the binary search algorithm is equal. All the cases are exhausted and the algorithm terminates when $M_k$ = $A_k$. This proves that our \textbf{algorithm gives correct output}, and hence concludes the proof of correctness.

%------------------------------------------------------------------------------%

\section{Applications}
This technique has various applications$^\cite{Wikipedia article on fractional cascading}$ in numerous fields. \\
\begin{enumerate}
    \item Computational Geometry
    \begin{itemize}
        \item Half-Range Plane Reporting
        \item Explicit Searching
        \item Point Location
    \end{itemize}
    \item Networks
    \begin{itemize}
        \item Fast Packet filtering in internet routers.
        \item Data distribution and retrieval in sensor networks
    \end{itemize}
    \item Linear Range Queries
        \begin{itemize}
            \item As an accompaniment to \textit{Segment Trees}
        \end{itemize}
\end{enumerate}

%------------------------------------------------------------------------------%

\pagebreak
\begin{thebibliography}{}

\bibitem{Fractional cascading notes by Prof Prof.Roberto Tamassia}
C.S.252 ``Prof.RobertoTamassia'' ComputationalGeometry Sem.II, 1992-1993
\\
\texttt{http://cs.brown.edu/courses/cs252/misc/resources/lectures/pdf/notes08.pdf} \\

\bibitem{MIT OCW Notes on Fractional Cascading}
6.851:Advanced Data Structures Spring 2012 ``Prof.Erik Demaine'' \\
\href{https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-851-advanced-data-structures-spring-2012/calendar-and-notes/MIT6_851S12_L3.pdf}{Link to MIT Lecture notes by Prof Erik Demaine on the topic Advanced Data Structures} \\

\bibitem{Fractional Cascading Demo}
Fractional Cascading webpage created by Ravi Sinha alias ravix339-zz \\
\texttt{https://ravix339.github.io/FractionalCascading/index.html} \\
\texttt{https://ravix339.github.io/FractionalCascading/Demo.html} \\

\bibitem{Blog by Edward Z. Yang}
A blog about fractional cascading and bridge building by Edward Z. Yang \\
\texttt{http://blog.ezyang.com/2012/03/you-could-have-invented-fractional-cascading/} \\

\bibitem{Wikipedia article on fractional cascading}
Wikipedia article on fractional cascading \\
\texttt{https://en.wikipedia.org/wiki/Fractional\_cascading} \\

\bibitem{Youtube Walkthrough}
Our youtube walkthrough for fractional cascading \\
\texttt{https://www.youtube.com/watch?v=eUQtziH6cDo} \\

\end{thebibliography}
\end{document}