  A graph $X$ is pair $(X_0,X_1)$ of vertices and edges.
  The \emph{degree} of a vertex $x \in X_0$ is the number of edges meeting at $x$, where we count self-edges twice.
  We say that $X$ is \emph{connected} if any two vertices $x,y \in X_0$ are connected by a finite path of edges.
  \begin{mdframed}
    We will assume that all our graphs are connected and all the vertices have finite degrees.
    The set of vertices itself can be infinite.
  \end{mdframed}

  We want to think of graphs as 1-dimensional topological spaces.
  If there are no loops, we can define maps between graphs $f:X \rightarrow Y$ to be compatible pairs of maps $f_0:X_0 \rightarrow Y_0$ and $f_1:X_1 \rightarrow Y_1$, but when there are loops we need to specify directions for the loops. The map $f_1$ does not keep track of the "directions".
  More precisely, we want to be able to differentiate between the two maps
  \begin{align*}
    [0,1] &\rightarrow [0,1] &&& [0,1] &\rightarrow [0,1] \\
    x &\mapsto x &&& x&\mapsto1-x
  \end{align*}
  For this, we assign labels to each edge so that if $a$ is one direction then $a^{-1}$ is the reverse direction.
  We can then talk about the source $d_0 a$ and target $d_1 a$ vertex of each label, so that
  \begin{align*}
    d_0 a = d_1 a^{-1} && d_1 a = d_0 a^{-1}.
  \end{align*}
  Denote by $X_{\ell}$ the set of all labels (and their inverses).
  A map between the graphs $f:X \rightarrow Y$ is then a map between the label sets
  \begin{align*}
    f_\ell: X_\ell \rightarrow Y_\ell
  \end{align*}
  such that if $f_\ell(a^{-1}) = f_\ell(a)^{-1}$ and $d_i a = d_j b$ then $d_i f_\ell(a) = d_j f_\ell(b)$.
  It is an \emph{isomorphism} if it is bijective.
  \begin{align*}
    [0,1] &\rightarrow [0,1] &&& [0,1] &\rightarrow [0,1] \\
    x &\mapsto x &&& x&\mapsto1-x \\
    a &\mapsto a &&& a&\mapsto a^{-1} \\
    a^{-1} &\mapsto a^{-1} &&& a^{-1}&\mapsto a
  \end{align*}
  A map between graphs $f: X \rightarrow Y$ naturally induces a map between vertices $f_0 : X_0 \rightarrow Y_0$.
  %
  %
  %
  %
  %
  %
  %
  %
  %
  % \subsection{Covering spaces}
  % \begin{definition}
  %   We say that $\pi:X \rightarrow Y$ is a \emph{cover} or a \emph{covering map} if for every vertex $v$, every vertex $w \in \pi_0^{-1}(v)$ has the same degree as $v$.
  % \end{definition}
  % We call $\pi_0^{-1}(v)$ the fiber over $v$.
  % \begin{ex}
  %   \todo{example of covering space}
  % \end{ex}
  % We can construct covers using group actions.
  % A left group action of $G$ on a graph $X$, denoted $G \groupaction X$, is a collection of graph maps
  % \begin{align*}
  %   g \cdot - : X &\longrightarrow X\\
  %   x &\longmapsto gx
  % \end{align*}
  % satisfying $ex = x$, where $e$ is the identity element in $G$ and $g(hx) = (gh)x$ for all $g,h \in G$.
  % A left $G$-action on $X$ naturally defines left $G$-actions on $X_0$, $X_1$ and $X_\ell$.
  % The quotient graph $G \backslash X$ is then the graph with vertices, edges, and labels being $X_0/G$, $X_1/G$, and $X_\ell/G$ respectively.
  % \begin{definition}
  %   We say that $G \groupaction X$ is \emph{free} if the induced group action on the set of vertices $X_0$ is free.
  % \end{definition}
  % \begin{ex}
  %   \todo{add example of maps between $S^1$: rotation and reflection.}
  % \end{ex}
  % \begin{qbox}
  %   Show that if $G \groupaction X$ is free then $X \rightarrow G \backslash X$ is a cover.
  % \end{qbox}
  % \begin{definition}
  %   We say that a cover $\pi:X \rightarrow Y$ is a \emph{Galois cover} if there exists a left group action $G \groupaction X$ such that $Y \cong G \backslash X$.
  % \end{definition}
  % \begin{qbox}
  %   Come up with a cover that is not-Galois.
  % \end{qbox}
  %
  %
  %


\documentclass[a4paper]{article}

\usepackage{amssymb,amsmath}

\author{Edoardo Pasca}

\title{Iterative Sample Statistic}
\date{\today}

\newcommand{\avg}{\bar{v}}

\begin{document}
\maketitle
Some time ago I was sick with some fever. The only thing I could do was to lay in bed and measure my body temperature. The thermometer took so little time to measure that I tried again and again. Each time the temperature was different.

I thought to measure a few times and then to calculate the average, to get a more stable result.

Doing it without a piece of paper starts to be a bit difficult when you have to deal with more than 5 samples (at least for me). I started thinking that there must be a way to update the last calculated average with the next sample.

\section{Mean}
The average is defined as:
\begin{equation}
\langle v\rangle_N = \frac{1}{N} \sum_{i=1}^N v_i
\label{eq:avg_def}
\end{equation}

Adding one sample $v_{N+1}$, the next evaluation of the average is:
\begin{align}
\avg_{N+1}  & =  \frac{1}{N+1} \sum_{i=1}^{N+1} v_i \nonumber \\
 & =  \frac{1}{N+1} \left( \sum_{i=1}^{N} v_i + v_{N+1} \right) \nonumber\\
 & =  \frac{1}{N+1} \left( N \avg_{N} + v_{N+1} \right) \nonumber\\
 & =  \frac{N}{N+1} \avg_{N} + \frac{v_{N+1}}{N+1}
\label{eq:iterative-avg}
\end{align}

where $\avg_{N+1}$ is the average calculated with $N+1$ samples and $N$ is
the number of samples. This formula says that we can calculate the next
average by keeping in mind only 3 numbers: the previous average, the number
of samplings and the last sample. That's nice!

\section{Standard Deviation}
The variance is defined as the square of the standard deviation $\sigma$:
\begin{equation}
\sigma_N^2 = \frac{1}{N-1} \sum_{i=1}^N \left(v_i - \avg_N\right)^2
\label{eq:std_def}
\end{equation}

Again, adding the $N+1$-th sample the estimation of the variance can be
expressed as:

\begin{equation}
\sigma_{N+1}^2   =  \frac{1}{N-1} \sum_{i=1}^{N+1} \left(v_i -
\avg_{N+1}\right)^2
\end{equation}
we now use the eq. \ref{eq:std_def} to express $\avg_{N+1}$ in terms of
$\avg_N$

\begin{equation}
\sigma_{N+1}^2  = \frac{1}{N} \sum_{i=1}^{N+1} \left(v_i -
\left( \frac{N}{N+1} \avg_{N} + \frac{v_{N+1}}{N+1} \right)
 \right)^2
 \label{eq:var-n+1}
 %& = \frac{1}{N-1}\left( \sum_{i=1}^{N} \left(v_i -
 %\frac{N}{N+1} \avg_{N} + \frac{v_{N+1}}{N+1} \right)^2  +
 %\left(v_{N+1} -
 %\frac{N}{N+1} \avg_{N} + \frac{v_{N+1}}{N+1} \right)^2
 %\right) \nonumber
\end{equation}

Before going further, we better arrange the term in the sum with a nice trick:
\begin{align}
v_i - \left(\frac{N}{N+1} \avg_{N} + \frac{v_{N+1}}{N+1}\right)& =  \nonumber \\
& = v_i - \left( \avg_N - \avg_N + \frac{N}{N+1} \avg_{N}+
\frac{v_{N+1}}{N+1}\right) \nonumber \\
& = v_i - \avg_N - \left( - \avg_N + \frac{N}{N+1} \avg_{N}+
\frac{v_{N+1}}{N+1}\right) \nonumber \\
& = \left(v_i - \avg_N\right) - \left( \frac{N-(N+1)}{N+1} \avg_{N}+
\frac{v_{N+1}}{N+1}\right) \nonumber \\
& = \left(v_i - \avg_N\right) - \left(-\frac{1}{N+1} \avg_{N}+
\frac{v_{N+1}}{N+1}\right)
\label{eq:trick}
\end{align}

The term in the sum is actually expressed by the sum of two terms, the first
depending on the various samplings, $v_i$ and the previous evaluation of the
average, $\avg_N$,
and the second term depending only on the previous evaluation of the average,
$\avg_N$, the number of samplings, $N$, and the last sample, $v_{N+1}$.

\begin{align}
a_i = & v_i - \avg_N \label{eq:a_i} \\
b_{N,\avg_N} = b = & \frac{v_{N+1}-\avg_{N}}{N+1} \label{eq:b}
\end{align}

Now we can express the variance, eq. \ref{eq:var-n+1}, with the help of
these variables $a_i$ and b as,
\begin{align}
\sigma_{N+1}^2  = & \frac{1}{N} \left[\left(\sum_{i=1}^{N} \left(a_i -
b\right)^2 \right)
 + \left(a_{N+1} - b\right)^2\right] \\
 = & \frac{1}{N} \left[\left(\sum_{i=1}^{N} \left(a_i^2 +
b^2 - 2a_ib\right) \right)
 + \left(a_{N+1} - b\right)^2\right] \nonumber \\
 = & \frac{1}{N} \left[\left(\sum_{i=1}^{N} \left(a_i^2 +
b^2 \right) \right)
+ \left(a_{N+1} - b\right)^2\right] =
  \frac{1}{N} \left[\left(\sum_{i=1}^{N} a_i^2 \right) +
N b^2
 + \left(a_{N+1} - b\right)^2\right] \nonumber \\
 = & \frac{1}{N} \left[(N-1)\sigma_N^2  +
N b^2
 + \left(a_{N+1} - b\right)^2\right] \nonumber \\
 = & \frac{1}{N} \left[(N-1)\sigma_N^2  +
 \frac{N}{N+1} \left(v_{N+1} - \avg_N \right)^2
 \right] \nonumber \\
 \sigma_{N+1}^2 = & \frac{N-1}{N} \sigma_N^2  +
 \frac{1}{N+1} \left(v_{N+1} - \avg_N \right)^2
 \label{eq:iterative-var}
\end{align}

Eqs. (\ref{eq:iterative-avg}) and (\ref{eq:iterative-var}) allow to iterative
calculate the next estimation of average and variance using only the previous
estimation, the last sampling and the number of samples.

\section{Higher order moments}
As last algebraic remark,
\begin{align}
a_{N+1} - b = & (v_{N+1} - \avg_N) - \frac{1}{N+1} (v_{N+1} - \avg_N)
\nonumber \\
 = & \frac{N}{N+1} (v_{N+1} - \avg_N) \nonumber \\
 a_{N+1} - b= & N b
\end{align}
We can easily extend this treatment to higher standardized moments of the
distribution of the population:

\begin{equation}
x_k = \frac{\sum_{i=1}^N \left(v_i - \avg_N\right)^k}{\sigma^k_N} =
\frac{M_{N,k}}{\sigma^k_N}
\end{equation}
where $k$ is the moment's order. It is now very easy to calculate the

\begin{align}
M_{N+1,k} = & \sum_i^{N+1} (v_i - \avg_{N+1})^k \nonumber \\
 = & \left[\sum_i^N (a_i - b)^k\right] + (a_{N+1} - b) ^k \nonumber \\
 = & \left[\sum_i^N (a_i - b)^k \right]+ (Nb)^k
 \label{eq:kth-moment}
\end{align}

\subsection{Skewness}
The skewness is a third order moment, $k=3$.

\begin{align}
M_{N+1,3} = & \left[\sum_i^N (a_i - b)^3 \right]+ (Nb)^3 \nonumber \\
 = & \left[\sum_i^N (a_i^3 - b^3 +3a_ib^2 - 3a_i^2b) \right]+ (Nb)^3 \nonumber \\
 = & \left[\sum_i^N (a_i^3 - 3a_i^2b) \right] -Nb^3 + (Nb)^3 \nonumber \\
 = & \left[\sum_i^N (a_i^3 - 3a_i^2b) \right]+ Nb^3 (N^2-1)  \nonumber \\
 = & M_{N,3} - 3b \left[\sum_i^N a_i^2 \right]+ Nb^3 (N^2-1)  \nonumber \\
 = & M_{N,3} - 3 b (N-1)\sigma_N^2 + Nb^3 (N-1)(N+1)  \nonumber \\
\end{align}
\subsection{Kurtosis}
The kurtosis is a fourth order moment, $k=4$.

\begin{align}
M_{N+1,4} = & \left[\sum_i^N (a_i - b)^4 \right]+ (Nb)^4 \nonumber \\
 = & \left[\sum_i^N (a_i^4 +b^4-4a_i^3+5a_i^2) \right]+ (Nb)^4 \nonumber \\
 = & \left[\sum_i^N (a_i^3 - 3a_i^2b) \right] -Nb^3 + (Nb)^4 \nonumber \\
 = & \left[\sum_i^N (a_i^3 - 3a_i^2b) \right]+ Nb^3 (N^2-1)  \nonumber \\
 = & M_{N,3} - 3b \left[\sum_i^N a_i^2 \right]+ Nb^3 (N^2-1)  \nonumber \\
 = & M_{N,3} - 3 b (N-1)\sigma_N^2 + Nb^3 (N-1)(N+1)  \nonumber \\
\end{align}
\end{document}


\section{Proof Handling}

In the proof editing mode, the user can use the tactics to deal with logical reasoning,
and can also use some other specialized commands to deal with the proof environment.

In our implementation, the proof procedure is organized as a tree. Initially, the tree
consists only the theorem itself as the tree root and the only leaf.
Every time a tactic is applied, either
the current leaf is expanded to an internal node or the proof procedure of the subtree is completed, then the next leaf of the tree will be focused. The proof procedure is completed
when all of the tree leaves are proven, then the whole proof object will be built recursively.

To each subgoal is associated a number of hypotheses called the {\it local context} of the goal.
Initially the local context contains nothing, it is enriched by the use of certain tactics.

When a proof is completed, the message {\tt No more subgoals} is displayed. One can then register this
proof as a defined constant in the environment. Because there exists a correspondence between proofs and terms of
$\lambda\text{-calculus}$, known as the {\it Curry-Howard isomorphism}, the MiniProver stores proofs as terms of $\it CIC$.
Those terms are called {\it proof terms}.

\subsection{\tt Theorem}
The proof editing mode is entered by asserting a theorem.

\subsection{\tt Proof}
This command is a noop which is useful to delimit the sequence of tactic commands which start a proof, after a
{\tt Theorem} command.

{\tt Theorem \sl ident \{binder\} : term.}
\subsection{\tt Qed}
This command is available in interactive editing proof mode when the proof is completed.
Then {\tt Qed} extracts a proof term from the proof script, switches back to the top level and
attaches the extracted proof term to the declared name of the original goal.

\subsection{\tt Admitted}
This command is available in interactive editing proof mode to give up the current proof and declare the initial goal as an axiom.

\subsection{\tt Abort}
This command cancels the current proof development,
switching back to the top level.

\subsection{\tt Undo}
This command cancels the effect of the last command. Thus, it backtracks one step.

\subsection{\tt Restart}
This command restores the proof editing process to the original goal.


%!TEX root = uber-driver-strategy.tex
\section{Bisection Algorithm} \label{section:bisection}

The inner problem of the robust dynamic program is as follows:
\begin{eqnarray*}
\sigma^* &:=& \min_p p^Tv : p \in \Delta^n, \sum_j f(j)\log p(j) \geq \beta
\end{eqnarray*}
The Lagrangian $\textbf{L}: \textbf{R}^n \times \textbf{R}^n \times \textbf{R} \times \textbf{R} \rightarrow \textbf{R}$ associated with the inner optimization problem is
\begin{eqnarray*}
\textbf{L}(v, \zeta, \mu, \lambda) = p^Tv - \zeta^Tp + \mu(1 - p^T\textbf{1}) + \lambda(\beta - f^T\log p)
\end{eqnarray*}
where $\zeta, \mu,$ and $\lambda$ are Lagrange multipliers. The dual function $d: \textbf{R}^n \times \textbf{R} \times \textbf{R} \rightarrow \textbf{R}$ is the minimum value of the Lagrangian over $p$ for $\zeta \in \textbf{R}^n, \mu \in \textbf{R},$ and $\lambda \in \textbf{R}$,
\begin{eqnarray*}
d(\zeta, \mu, \lambda) &=& \inf_p \textbf{L}(v, \zeta, \mu, \lambda) \\
&=& \inf_p(p^Tv - \zeta^Tp + \mu(1 - p^T\textbf{1}) + \lambda(\beta - f^T\log p))
\end{eqnarray*}
The optimal $p^*$ minimizing the above dual function is obtained by solving $\frac{\partial \textbf{L}}{\partial p} = 0$, which gives us,
\begin{eqnarray*}
p^*(i) &=& \frac{\lambda f(i)}{v(i) - \zeta(i) - \mu}
\end{eqnarray*}
Plugging in the value of $p^*(i)$ into the dual function $d$, after some simplification, gives us the dual problem,
\begin{eqnarray*}
\bar{\sigma} &:=& \max_{\zeta, \mu, \lambda} \lambda(1 + \beta) + \mu - \lambda \sum_{j}f(j) \log\bigg(\frac{\lambda f(j)}{v(j) - \zeta(j) - \mu}\bigg) : \lambda \geq 0, \zeta \geq 0, v \geq \zeta + \mu\textbf{1}
\end{eqnarray*}
As the above problem is concave, has a feasible set with non-empty interior (checked by plotting), there is no duality gap, that is $\sigma^* = \bar{\sigma}$. \hc{Previous statement needs clarification. The problem is upper convex, i.e., concave, not convex. Not sure how to argue the no duality gap}. Moreover, by a monotonicity argument ($\zeta$ increasing causes the function value to decrease monotonically), we can conclude that $\zeta$ is zero. Thus, the last constraint of the dual problem can be expressed as $\mu \leq v_{min} := \min_j v(j)$. Hence, we get,
\begin{eqnarray}
\sigma^* &=& \max_{\lambda, \mu} h(\lambda, \mu) \nonumber \\
\textrm{where,} \nonumber \\
h(\lambda, \mu) &:=& \begin{cases}
					 \lambda(1+\beta) + \mu - \lambda \sum_{j}f(j) \log\bigg(\frac{\lambda f(j)}{v(j) - \mu}\bigg), &\text{if } \lambda > 0, \mu < v_{min}, \\
					 -\infty, &\text{otherwise}
				     \end{cases}
\end{eqnarray}
The gradient of $h$ is given by,
\begin{eqnarray}
\frac{\partial h(\lambda, \mu)}{\partial \lambda} &=& \beta - \sum_{j}f(j) \log\bigg(\frac{\lambda f(j)}{v(j) - \mu}\bigg) \\
\frac{\partial h(\lambda, \mu)}{\partial \mu} &=& 1 - \lambda \sum_{j} \bigg(\frac{f(j)}{v(j) - \mu}\bigg)
\end{eqnarray}
From (3), we obtain the optimal value of $\lambda$ for a fixed value of $\mu$, $\lambda(\mu)$ given by,
\begin{eqnarray*}
\lambda(\mu) &=& \bigg(\sum_{j}\frac{f(j)}{v(j) - \mu}\bigg)^{-1}
\end{eqnarray*}
which further reduces the problem to a 1-dimensional optimization problem,
\begin{eqnarray*}
\sigma^* &=& \max_{\mu < v_{min}} \sigma(\mu) \\
\textrm{where, } \\
\sigma(\mu) &=& h(\lambda(\mu), \mu)
\end{eqnarray*}
$\sigma(\mu)$ is a concave function. Now, we may use a bisection algorithm to maximise this function.

To initialize the bisection algorithm, we need upper and lower bounds $\mu_+$ and $\mu_-$ on a minimizer of $\sigma$. When $\mu \rightarrow v_{min}, \sigma(\mu) \rightarrow v_{min}$ and $\sigma^{'}(\mu) \rightarrow -\infty$ \hc{To be proved analytically, verified by graphing}. Thus, we may set upper bound $\mu_+ = v_{min}$. The lower bound $\mu_-$ must be chosen such that $\sigma^{'}(\mu_-) > 0$.
\begin{eqnarray*}
\sigma^{'}(\mu) &=& \frac{\partial h}{\partial \mu} (\lambda(\mu), \mu) + \frac{\partial h}{\partial \lambda} (\lambda(\mu), \mu) \frac{\partial \lambda(\mu)}{\partial \mu}
\end{eqnarray*}
By construction of $\lambda(\mu)$, the first term of the above derivative is zero. Furthermore, $\frac{\partial \lambda(\mu)}{\partial \mu} < 0$. Hence, we need $\mu$ such that $\frac{\partial h}{\partial \lambda} (\lambda(\mu), \mu) < 0$. Using the bounds on $\lambda(\mu)$, $v_{min} - \mu \leq \lambda(\mu) \leq v_{max} - \mu$, we can show that,
\begin{eqnarray*}
\frac{\partial h}{\partial \lambda} (\lambda(\mu), \mu) &=& \beta - \sum_{j}f(j) \log\bigg(\frac{\lambda(\mu) f(j)}{v(j) - \mu}\bigg) \\
&=& \beta - \sum_{j}f(j)\log f(j) - \sum_{j}f(j)\log\bigg(\frac{\lambda(\mu)}{v(j) - \mu}\bigg) \\
&=& \beta - \beta_{max} - \sum_{j}f(j)\log\bigg(\frac{\lambda(\mu)}{v(j) - \mu}\bigg) \textrm{(Need to verify this step)}\\
&\leq& \beta - \beta_{max} - \log\bigg(\sum_{j}\frac{f(j)\lambda(\mu)}{v(j) - \mu}\bigg) \\
&\leq& \beta - \beta_{max} - \log\bigg(\sum_{j}\frac{\lambda(\mu)}{v(j) - \mu}\bigg) \\
&\leq& \beta - \beta_{max} - \log\bigg(\sum_{j}\frac{v_{min} - \mu}{v(j) - \mu}\bigg) \\
&\leq& \beta - \beta_{max} - \log\bigg(\sum_{j}\frac{v_{min} - \mu}{v_{max} - \mu}\bigg) \\
&\leq& \beta - \beta_{max} - \log\bigg(n \times \frac{v_{min} - \mu}{v_{max} - \mu}\bigg) \\
&<& \beta - \beta_{max} - \log\bigg(\frac{v_{min} - \mu}{v_{max} - \mu}\bigg) \textrm{(n $>$ 1)}\\
\end{eqnarray*}
Therefore, the sufficient condition is,
\begin{eqnarray*}
0 &>& \beta - \beta_{max} - \log\bigg(\frac{v_{min} - \mu}{v_{max} - \mu}\bigg) \\
\log\bigg(\frac{v_{min} - \mu}{v_{max} - \mu}\bigg) &>& \beta - \beta_{max} \\
\bigg(\frac{v_{min} - \mu}{v_{max} - \mu}\bigg) &>& e^{\beta - \beta_{max}} \\
(v_{min} - \mu) &>& e^{\beta - \beta_{max}}(v_{max} - \mu) \\
v_{min} - e^{\beta - \beta_{max}}v_{max} &>& \mu(1 - e^{\beta - \beta_{max}}) \\
\end{eqnarray*}
Hence,
\begin{eqnarray*}
\mu_{-}^0 := \frac{v_{min} - e^{\beta - \beta_{max}}v_{max}}{1 - e^{\beta - \beta_{max}}} &>& \mu
\end{eqnarray*}
By construction, the interval [$\mu_-^0, v_{min}$) is guaranteed to contain the global maximiser of $\sigma$ over $(-\infty, v_{min})$. The bisection algorithm is as follows:
% \begin{enumerate}
% 	\item Set $\mu_+ = v_{min}$ and $\mu_- = \mu_-^0$. Let $\delta > 0$.
% 	\item While \hc{Fill termination condition}, repeat
% 	\begin{enumerate}
% 		\item Set $\mu = (\mu_+ + \mu_-)/2$.
% 		\item If $\sigma^{'}(\mu) > 0$, set $\mu_- = \mu$, otherwise $\mu_+ = \mu$.
% 		\item go to 2a.
% 	\end{enumerate}
% \end{enumerate}
\begin{enumerate}
	\item Set $\mu_+ = v_{min}$ and $\mu_- = \mu_-^0$. Let $k=1, \mu_1 = (\mu_+ + \mu_-)/2$.
	\item While $k \leq N$:
	\begin{enumerate}
		\item If $\sigma^{'}(\mu_1) > 0$, set $\mu_- = \mu_1$, if $\sigma^{'}(\mu_1) < 0$ then $\mu_+ = \mu_1$, else break
		\item $k = k + 1$,
		\item $\mu_{k} = (\mu_+ + \mu_-)/2$
	\end{enumerate}
	\item $\hat{\mu}^* = \arg \max_{i} \{\sigma(\mu_i)\}$
\end{enumerate}
\textbf{Lemma}: After $N \approx \log_2(V/\delta)$ where $V = \max\big(\sigma^* - \sigma(\mu_+), \sigma^* - \sigma(\mu_-)\big)$, the bisection algorithm provides optimal solution to the inner problem within an accuracy $\delta > 0$ i.e., $\sigma^* - \sigma(\hat{\mu}^*) \leq \delta$ which we call as the $\delta$-solution. \\ \\
\textbf{Proof}: Let the interval $[\mu_-,\mu_+]$ from step 1 of the bisection algorithm be denoted by $G$. At each iteration, the length of the interval that contains the global maximiser of $\sigma$ is exactly halved. Hence, let $[\mu_{N-}, \mu_{N+}]$ be the corresponding interval after $N$ iterations of the bisection algorithm. Length of the interval $G_N$ is $2^{-N}$ times the length of $G$. Using the bisection algorithm, we know that,
\begin{eqnarray*}
\forall \mu: \mu \in G\setminus G_N \rightarrow \sigma(\mu) \leq \sigma(\hat{\mu}^*)
\end{eqnarray*}
Let $2^{-N} < \alpha < 1$. $\alpha$-contraction of $G$ to $\mu^*$ is the segment given by points,
\begin{eqnarray*}
G^{\alpha} = (1 - \alpha)\mu^* + \alpha G = \big\{ (1-\alpha)\mu^* + \alpha z \big| z \in G \big\}
\end{eqnarray*}
Since, $\alpha > 2^{-N}$, we know that length of $G^{\alpha} > G_N$, in fact, length of $G^{\alpha}$ is $\alpha(\mu_+ - \mu_-)$. Hence, $\exists y \in G^{\alpha}$ such that $y \notin G_N$. Furthermore, $\exists z \in G$ such that $y = (1 - \alpha)\mu^* + \alpha z$. By the concavity of $\sigma(\mu)$ function,
\begin{eqnarray*}
\sigma(y) &\geq& (1 - \alpha)\sigma(\mu^*) + \alpha \sigma(z) \\
\sigma(\mu^*) - \sigma(y) &\leq& \alpha(\sigma(z) - \sigma(\mu^*)) \\
&\leq& \alpha V
\end{eqnarray*}
Hence, $y$ is an $\alpha V$-solution to our problem.
\begin{eqnarray*}
\sigma(\hat{\mu}^*) &\geq& \sigma(y) \\
\sigma(\mu^*) - \sigma(\hat{\mu}^*) &\leq& \sigma(\mu^*) - \sigma(y) \leq \alpha V
\end{eqnarray*}
Hence, sufficient condition for obtaining a $\delta$-solution is, $\alpha V \leq \delta$ i.e., $N \geq \log_2 (V / \delta)$. However, $V$ is unknown by itself. But, the objective function of the inner problem, $p^Tv$, is bounded from above by $v_{max}$. Therefore, the lemma still holds for $V = \max\big(v_{max} - \sigma(\mu_+), v_{max} - \sigma(\mu_-)\big)$.
\section{$\epsilon$-suboptimal Algorithm}

Each step of the robust dynamic programming algorithm from Section \ref{section:robust-dynamic-programming} involves finding the solution of an optimization problem, referred to as the `inner problem', of the form,
\begin{eqnarray*}
\sigma_{\mathcal{P}}(v) &:=& \min_{p \in \mathcal{P}} p^Tv,
\end{eqnarray*}
where $p$ corresponds to particular row of a transition matrix, $\mathcal{P} = \mathcal{P}^{a}_{i,t}$ is the set that describes the uncertainty of this row, and $v$ contains the elements of the value function at this step. Section \ref{section:likelihood} provides a criterion for the choice of uncertainty model that represents accurately the statistical uncertainty on the transition matrices. Section \ref{section:bisection} describes a bisection algorithm that provides an optimal solution to the inner problem within an accuracy $\delta > 0$. \hc{This should be ensured by the stopping criterion of the algorithm}. Thus, for a given $v \in \textbf{R}^n_+$ and $\delta > 0$, the bisection algorithm gives output of the form,
\begin{eqnarray*}
\hat{\sigma}_{\mathcal{P}}(v) &=& \sigma_{\mathcal{P}}(v) - \delta_{\mathcal{P}}(v) \textrm{ where } 0 \leq \delta_{\mathcal{P}}(v) \leq \delta
\end{eqnarray*}
An $\epsilon$-suboptimal policy, $\hat{\pi}$ is a policy such that the worst-case expected total reward under policy $\hat{\pi}$, namely, $\phi_N(\hat{\pi},\mathcal{T}) = \min_{\tau \in \mathcal{T}}R_N(\hat{\pi}, \tau)$, satisfies
\begin{eqnarray*}
\phi_N(\hat{\pi},\mathcal{T}) \leq \phi_N(\Pi,\mathcal{T}) \leq \phi_N(\hat{\pi},\mathcal{T}) + \epsilon
\end{eqnarray*}
Here, $\epsilon > 0$ is given. Using the uncertainty model from Section \ref{section:likelihood}, we solve the bisection algorithm with an accuracy $\delta := \epsilon /N$. This gives us the robust finite horizon dynamic programming algorithm as follows,

\begin{enumerate}
	\item Set $\epsilon > 0$. Initialize the value function to its terminal value $\hat{v}_N = r_N$
	\item Repeat until t=0:
	\begin{enumerate}
		\item For every state $i \in \mathcal{X}$ and action $a \in \mathcal{A}$, compute, using the bisection algorithm described in the previous section, a value $\hat{\sigma}^{a}_i$ such that,
		\begin{eqnarray*}
		\hat{\sigma}^{a}_i \leq \sigma_{\mathcal{P}^{a}_{i,t}}(v_{t}) \leq \hat{\sigma}^{a}_i + \frac{\epsilon}{N}
		\end{eqnarray*}
		\item Update the value function by,
		\begin{eqnarray*}
		v_{t-1}(i) &=& \max_{a \in \mathcal{A}} \bigg(r_{t-1}(i,a) + \hat{\sigma}^{a}_i\bigg), i \in \mathcal{X}
		\end{eqnarray*}
		\item t = t - 1
	\end{enumerate}
	\item For every $i \in \mathcal{X}$ and $t \in T$, set $\pi^{\epsilon} = (\textbf{a}^{\epsilon}_0, ..., \textbf{a}^{\epsilon}_{N-1})$, where
	\begin{eqnarray*}
	\textbf{a}^{\epsilon}_t &=& \arg \min_{a \in \mathcal{A}} \big\{r_{t-1}(i,a) + \hat{\sigma}^{a}_i\big\}, i \in \mathcal{X}, a \in \mathcal{A}
	\end{eqnarray*}
\end{enumerate}


% section3
\chapter {Determinants}

\section{Determinants; cofactor Expansion}

\begin{exer}
Compute the determinants of the matrix A:
\begin{equation*}
A=\begin{bmatrix}-4 & 1 & 1 & 1 & 1 & \\ 1 & -4 & 1 & 1 & 1 \\ 1 & 1 & -4 & 1 & 1 \\ 1 & 1 & 1 & -4 & 1 \\ 1 & 1 & 1 & 1 & -4\end{bmatrix}\\.
\end{equation*}
How can you construct $A$ brilliantly?
\end{exer}

\begin{sol}

\begin{verbatim}

A=ones(5)-5*eye(5);
disp('A is'); disp(A);
disp('Determinant of A is'); disp(det(A));
\end{verbatim}

\begin{outputs}
\begin{verbatim}

A is
    -4     1     1     1     1
     1    -4     1     1     1
     1     1    -4     1     1
     1     1     1    -4     1
     1     1     1     1    -4

Determinant of A is
  -5.5511e-14
\end{verbatim}
\end{outputs}
\end{sol}

\vspace{3mm}

\begin{exer}
Show that
$$\det\left(\begin{bmatrix} \displaystyle a & b & c & d \\ -b & a & d & -c \\ -c & -d & a & b \\ -d & c & -b & a \end{bmatrix}\right)=(a^2+b^2+c^2+d^2)^2.$$
\end{exer}

\begin{sol}

\begin{verbatim}

syms a b c d;

A=[a b c d; -b a d -c; -c -d a b; -d c -b a];

disp('Given matrix is'); disp(A);
disp('Determinant of the given matrix is');
disp(simplify(det(A)));
\end{verbatim}

\begin{outputs}
\begin{verbatim}

Given matrix is
[  a,  b,  c,  d]
[ -b,  a,  d, -c]
[ -c, -d,  a,  b]
[ -d,  c, -b,  a]

Determinant of the given matrix is
(a^2 + b^2 + c^2 + d^2)^2
\end{verbatim}
\end{outputs}
\end{sol}

\vspace{3mm}

\begin{exer}
The $n$th-order \textbf{Fibonacci matrix} [named for the Italian mathematician~(circa~1170~-~1250)] is the $n \times n$ matrix $F_{n}$ that has 1's on the main diagonal, 1's along the diagonal immediately above the main diagonal, -1's along the diagonal immediately below the main diagonal, and zeros everywhere else. Construct the sequence
$$\det(F_{1}), \,\det(F_{2}), \,\det(F_{3}), \,\cdots, \det(F_{7}).$$
Make a conjecture about the relationship between a term in the sequence and its two immediate predecessors, and then use your conjecture to make a guess at $\det(F_{8})$. Check your guess by calculating this number.
\end{exer}

\begin{sol}
\begin{verbatim}

% Construct the 10x10 Fibonacci matrix F.
N=10; nOnes=ones(N, 1);
F=diag(nOnes)+diag(nOnes(1:N-1),1)-diag(nOnes(1:N-1),-1);

for n=1:7 % n is from 1 to 7
    Fn=F(1:n,1:n); % nxn Fibonacci matrix is selected from F.
    disp(det(Fn));
end
\end{verbatim}

\begin{outputs}
\begin{verbatim}

     1
     2
     3
     5
     8
    13
    21
\end{verbatim}
\end{outputs}

\noindent The constructed sequence satisfies the relationship $\det(F_n)=\det(F_{n-1})+\det(F_{n-2}),$ for $\det(F_1)=1$ and $\det(F_2)=2$. From that, we may guess that $\det(F_8)=34$.
MATLAB gives us the same output value 34 as our guess.
\end{sol}

\vspace{3mm}

\begin{exer} Let $A_{n}$ be the $n \times n$ matrix that has 2's along the main diagonal, 1's along the diagonals immediately above and below the main diagonal, and zeros everywhere else. Make a conjecture about the relationship between $n$ and $\det(A_{n})$.
\end{exer}

\begin{sol}
\begin{verbatim}

format rat;
% Construct the 10x10 matrix A satisfying given conditions.
n=10; nOnes=ones(n, 1);
A=2*diag(nOnes)+diag(nOnes(1:n-1),1)+diag(nOnes(1:n-1),-1);

for i=1:10 % i is from 1 to 10
    Ai=A(1:i,1:i); % A_i matrix is selected from A.
    disp(det(Ai));
end
\end{verbatim}

\begin{outputs}
\begin{verbatim}

       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
\end{verbatim}
\end{outputs}

\noindent From the outputs, we make a conjecture about the relationship between $n$ and $\det(A_{n})$ as follows:
$$\det(A_{n})=n+1.$$

\end{sol}

\section{Properties of Determinants}

\begin{exer}(\textit{Determinants with LU-decomposition})
In this problem, we find the determinant of the matrix $A$ by using the $LU$-decomposition of $A$, where

\begin{displaymath}
A = \left[\begin{array}{rrrr} -2& \hspace{1mm} 2& \hspace{1mm}-4& \hspace{2mm} -6\\ -3 & 6 & 3 & -15 \\ 5 & -8 & -1 & 17 \\ 1 & 1 & 11 & 7 \end{array} \right].
\end{displaymath}
\vspace{1mm}
\begin{enumerate}
\item[(a)] Compute the determinant of $A$ directly by using the MATLAB command \textit{det} for $A$.
\vspace{1mm}
\item[(b)] Compute the determinant of $A$ by using the MATLAB command \textit{lu} for $A$. Confirm that you get the same results.
\end{enumerate}

\end{exer}

\begin{sol}

\begin{verbatim}

%(a)
A = [-2 2 -4 -6; -3 6 3 -15; 5 -8 -1 17; 1 1 11 7];

det_A = det(A); % Find the determinant of A by using the command det.

disp('The determinant of A by direct use of the command det is');
disp(det_A);

%(b)
[L U P] = lu(A); % We have a PLU-decomposition of A. (i.e., PA=LU ).

% Since the determinant of a triangular matrix is
% just a product of diagonal entries,

det_L = prod(diag(L)); % The product of diagonal entries of L.
% Or, you may use the command det for L, directly. (i.e., det_L = det(L)).

det_U = prod(diag(U)); % The product of diagonal entries of U.
% Or, you may use the command det for U, directly. (i.e., det_U = det(U)).

% If you observe the permutation matrix P, you can see that
% P is an odd permutation. Thus, we have det(P) = -1.
det_P = -1;
% Or, you may use the command det for P, directly. (i.e., det_P = det(P)).

% Since PA = LU, det(P)*det(A) = det(L)*det(U).
det_A = det_P * det_L * det_U;

disp('The determinant of A by using the LU-decomposition is'); disp(det_A);
\end{verbatim}

\begin{outputs}

\begin{verbatim}

The determinant of A by direct use of the command det is
   24.0000

The determinant of A by using the PLU-decomposition is
   24.0000
\end{verbatim}

\end{outputs}

\end{sol}

\vspace{3mm}

\begin{exer} (\textit{Effects of Elementary Row Operations on the Determinant})

Using the MATLAB command \textit{det}, confirm the formulas (a)-(c) in Theorem 4.2.2 of Section 4.2 for the matrix $A$ given in the problem 31 of Exercise set 4.1.
\end{exer}

\begin{sol}
\begin{verbatim}

A = [3 3 0 5; 2 2 0 -2; 4 1 -3 0; 2 10 3 2];

% (a). Multiply the second row of A by 2 and call it A2.
% Initialize the matrix A2 as A.
A2 = A;
% Multiply the second row of A by 2.
A2(2,:) = 2*A(2,:);
disp('The determinant of A2 is'); disp(det(A2));
disp('2*det(A) = '); disp(2*det(A));

% (b). Interchange the rows 2 and 4 of A and call it A24.
% Initialize the matrix A24 as A.
A24 = A;
% Interchange the rows 2 and 4 of A.
A24(2, :) = A(4, :) ; A24(4, :) = A(2, :);
disp('The determinant of A24 is'); disp(det(A24));
disp('-det(A) = '); disp(-det(A));

% (c). Add 2 times row 3 to row 4 of A and call it A234.
% Initialize the matrix A234 as A.
A234 = A;
% Add 2 times row 3 of A to row 4.
A234(4, :) = 2 * A(3, :) + A(4, :);
disp('The determinant of A234 is'); disp(det(A234));
disp('det(A) = '); disp(det(A));
\end{verbatim}

\begin{outputs}

\begin{verbatim}

The determinant of A2 is
  -480
2*det(A) =
 -480.0000

The determinant of A24 is
  240.0000
-det(A) =
  240.0000

The determinant of A234 is
 -240.0000
det(A) =
 -240.0000
\end{verbatim}
\end{outputs}
\end{sol}

\vspace{3mm}

\begin{exer} Use a determinant to show that if $a, b, c,$ and $d$ are not all zeros, then the vectors
\begin{eqnarray*}
\mathbf{v}_{1}&=&(a,\, b,\, c,\, d)\\
\mathbf{v}_{2}&=&(-b, a, d, -c)\\
\mathbf{v}_{3}&=&(-c, -d, a, b)\\
\mathbf{v}_{4}&=&(-d, c, -b, a)
\end{eqnarray*}
are linearly independent.
\end{exer}

\begin{sol}

\begin{verbatim}

syms a b c d;
v1=[a b c d];
v2=[-b a d -c];
v3=[-c -d a b];
v4=[-d c -b a];

V=[v1; v2; v3; v4];
disp('det(V) is'); disp(simplify(det(V)));
\end{verbatim}

\begin{outputs}
\begin{verbatim}

det(V) is
(a^2 + b^2 + c^2 + d^2)^2
\end{verbatim}

\end{outputs}

\end{sol}

\section{Cramer's Rule; Formula for $A^{-1}$; Applications}

No MATLAB problems in this section.

\newpage

\section{A First Look at Eigenvalues and Eigenvectors}

\begin{exer}(\textit{Eigenvalues and Eigenvectors})\\
Use the MATLAB command \textit{eig} to find the eigenvalues and the associated \mbox{eigenvectors} of the matrix $A$, where
$$A = \left[\begin{array}{rrrr} 2& \hspace{1mm}-3& \hspace{1mm} 1& \hspace{3mm} 0\\ 1 & 1 & 2 & 2\\ 3 & 0 & -1 & 4 \\ 1 & 6 & 5 & 6 \end{array} \right].$$
Display the results with long digits.
\end{exer}

\begin{sol}

\begin{verbatim}

% Construct the matrix A.
A=[2 -3 1 0; 1 1 2 2; 3 0 -1 4; 1 6 5 6];

% Find the eigenvalues and eigenvectors of A by using eig.
% This command gives AQ = QD.
[Q D] = eig(A);
lambda1 = D(1,1); lambda2 = D(2,2);
lambda3 = D(3,3); lambda4 = D(4,4);

% Extract each column vector as an eigenvector of A.
x1 = Q(:,1); x2 = Q(:,2); x3 = Q(:,3); x4 = Q(:,4);

% Display the result with long digits.
format long;
disp('lambda1 is'); disp(lambda1);
disp('The eigenvector corresponding to lambda1 is'); disp(x1');
disp('lambda2 is'); disp(lambda2);
disp('The eigenvector corresponding to lambda2 is'); disp(x2');
disp('lambda3 is'); disp(lambda3);
disp('The eigenvector corresponding to lambda3 is'); disp(x3');
disp('lambda4 is'); disp(lambda4);
disp('The eigenvector corresponding to lambda4 is'); disp(x4');
\end{verbatim}

\begin{outputs}
\begin{verbatim}

lambda1 is
  9.561855032395805
The eigenvector corresponding to lambda1 is
 -0.067716707308095  0.278176502030497  0.322465582156500  0.902246213399589
lambda2 is
 -3.364648937746373
The eigenvector corresponding to lambda2 is
  0.275562522991092  0.197508356444458 -0.885771126913498  0.316962546342283
lambda3 is
  1.802793905350564
The eigenvector corresponding to lambda3 is
 -0.833621905475750 -0.103812731179200 -0.147042873144503  0.522183711938150
lambda4 is
 -3.860931435448914e-16
The eigenvector corresponding to lambda4 is
 -0.705886578756789 -0.456750139195570  0.041522739926871  0.539795619049310
\end{verbatim}

\end{outputs}

\noindent\textit{Remark.} In fact, if we compute $\lambda_{4}$ by hand, we can obtain that $\lambda_{4}=0$. However, from the result, we see that the resulting value of $\lambda_{4}$ seems to be nonzero even though it is small enough. This is due to roundoff errors in arithmetic operations. Please refer to the help command of \textit{eps}, then you can see that $eps = 2.220446049250313e-016$ is floating-point relative accuracy, which means that \textit{eps} value is the allowable tolerance when we do numerical computations with rounding floating-point number off. ($i.e.$, $eps$ is an upper bound on the relative error due to rounding in floating point arithmetic.)
Therefore, we can regard the resulting value of $\lambda_{4}$ as zero.
\end{sol}

\vspace{3mm}

\begin{exer}(\textit{Eigenvalues and Eigenvectors})

Define an $n$th-order \textbf\textit{checkboard matrix} $C_{n}$ to be a matrix that has a 1 in the upper left corner and alternates between 1 and 0 along rows and columns (see the figure below). Find the eigenvalues of $C_{1}, C_{2}, \cdots$ to make a conjecture about the eigenvalues of $C_{n}$. What can you say about the eigenvalues of $C_{n}$?
\begin{figure}[h]\centering
\includegraphics[width=4cm]{figure.jpg}
\end{figure}
\end{exer}

\begin{sol}
\begin{verbatim}

format short;
n=10;   % Set the size of the large check board

% Construct your checkboard
CheckBoard=zeros(n);
CheckBoard(1:2:n, 1:2:n)=1;
CheckBoard(2:2:n, 2:2:n)=1;
for i=1:n
    Cn=CheckBoard(1:i, 1:i);
    [Qn Dn]=eig(Cn);    % Eigenvectors and eigenvalues
    fprintf('The size of the checkboard is %d \n',i);
    disp(diag(Dn)');
end
\end{verbatim}

\begin{outputs}

\begin{verbatim}

The size of the checkboard is 1
     1
The size of the checkboard is 2
     1 1
The size of the checkboard is 3
     0 1 2
The size of the checkboard is 4
     0 0 2 2
The size of the checkboard is 5
   -0.0000 -0.0000  0.0000  2.0000  3.0000
The size of the checkboard is 6
   -0.0000  -0.0000  -0.0000  -0.0000  3.0000  3.0000
The size of the checkboard is 7
   -0.0000  -0.0000   0.0000   0.0000  0.0000  3.0000  4.0000
The size of the checkboard is 8
   -0.0000  -0.0000  -0.0000  0.0000  0.0000  0.0000  4.0000  4.0000
The size of the checkboard is 9
   -0.0000  -0.0000  -0.0000 -0.0000   0   0.0000  0.0000  4.0000  5.0000
The size of the checkboard is 10
   -0.0000  -0.0000  -0.0000  0  0.0000  0.0000  0.0000  0.0000  5.0000  5.0000
\end{verbatim}
\end{outputs}

\noindent We may conclude that the eigenvalues of $C_{n}$ are given as follows:
$$\begin{cases} 1 &\text{if $n=1$,}\\
 k,\, k,\, \underbrace{0,\,0,\,\cdots,\,0}_{(n-2)}&\text{if $n=2k$,}\\
 k,\,k+1,\,\underbrace{0,\,0,\,\cdots,\,0}_{(n-2)}&\text{if $n=2k+1$,}
\end{cases}
$$
where $k$ is a positive integer.
\end{sol}

_codes/intro/Learning MATLAB with Linear Algebra (Jeon, Lee)/section4.tex

files/intro/Learning MATLAB with Linear Algebra (Jeon, Lee)/section4.tex


\documentclass{article}
\usepackage{amsmath}
\usepackage[margin=1.0in]{geometry}
\usepackage{xcolor}

\begin{document}

\noindent
Does $\displaystyle \sum_{n=1}^\infty \frac{n^2+1}{n^3+7}$
diverge, converge absolutely, or converge conditionally?

\subsection*{Solution 1}

$\displaystyle \sum_{n=1}^\infty \frac1n$ is a $p$-series with $p=1$. Since $p \leq 1$, the series $\displaystyle \sum_{n=1}^\infty \frac1n$ diverges by the $p$-series test. If we use $a_n = \frac{n^2+1}{n^3+7}$ and $b_n = \frac1n$, then
\begin{align*}
\lim_{n \to \infty} \frac{a_n}{b_n}
&= \lim_{n \to \infty} \frac{n^2+1}{n^3+7} \cdot \frac{n}{1}\\
&= \lim_{n \to \infty} \frac{n^3+n}{n^3+7} \\
&= \lim_{n \to \infty} \frac{3n^2+1}{3n^2} \text{ using L'hopital}\\
&= \lim_{n \to \infty} \frac{6n}{6n} \text{ using L'hopital}\\
&= \lim_{n \to \infty} 1\\
&= 1
\end{align*}
So by the Limit Comparison Test, the series $\displaystyle \sum_{n=1}^\infty \frac{n^2+1}{n^3+7}$ diverges.

\subsection*{Solution 2}

We are looking for a fixed $K > 0$ such that
\[ K \cdot  \frac{n^2+1}{n^3+7} \geq \frac1n\]
By multiplying both sides by $n(n^3+7)$, we have
\[ K(n^3 + n) \geq n^3+7\]
so
\[ Kn^3 + Kn \geq n^3+7\]
so
if we pick $K=8$, then we'd have
\[ 8n^3 + 8n \geq n^3+7\]
is likely true because
\[ n^3+ 7n^3 + 8n \geq n^3+7\]
with the $n^3$'s compared together, and $7n^3$ is greater than $7$. So we are ready to present starting from true inequalities.

With the scratch work above done, since $7n^3 \geq 7$ for $n \geq 1$, we have
\[ n^3+ 7n^3 + 8n \geq n^3+7\]
\[ 8n^3 + 8n \geq n^3+7\]
\[ 8(n^3 + n) \geq n^3+7\]
and dividing both sides by $8n(n^3+7)$ we get
\[\frac{n^2+1}{n^3+7} \geq \frac1{8n}\]
Since $\displaystyle \sum_{n=1}^\infty \frac1{8n} = \frac18\sum_{n=1}^\infty \frac1{n}$ diverges by the $p$-test, the series
$\displaystyle \sum_{n=1}^\infty \frac{n^2+1}{n^3+7}$ diverges by the Direct Comparison Test.

\end{document}%%%%%%%%%%%%%%%%%

\begin{align*}
L&=\lim_{n \to \infty} \sqrt[n]{|a_n|}\\
&= \lim_{n \to \infty} \sqrt[n]{\left| \right|}\\
\end{align*}

\begin{align*}
L&=\lim_{n \to \infty} \left|\frac{a_{n+1}}{a_n}\right|\\
&= \lim_{n \to \infty} \left| \right|\\
\end{align*}

\begin{align*}
\lim_{n \to \infty} a_n
&= \lim_{n \to \infty} \\
\end{align*}

Since $\sum |a_n| = \sum a_n$, the series $\displaystyle \sum_{n=1}^\infty AAAAAAAAAAAAAA$ converges absolutely.

Since $|r| < 1$, the series ...  converges by the Geometric Series Test.

Since $|r| \geq 1$, the series ...  diverges by the Geometric Series Test.

The function $f(x)=\frac{}{}$ is continuous, positive, and decreasing on $[1,\infty)$.

\subsection*{Solution}


\section{1-Dimensional Linear Convection}

One dimensional linear convection is the simplest model possible which can teach us something about CFD.
The equation for this is:

\begin{equation}
\frac{\delta{}u}{\delta{}t} + c = 0
\end{equation}


\documentclass[a4paper]{article}

% Nice way to display code
\usepackage{minted}

% Import images
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{multicol}

\title{Understanding the connection between
       ranking the internet and eigen vectors}
\author{Vince Knight}
\date{}

\begin{document}
\maketitle

\section{Introduction: The origins of the Page Rank algorithm}

In the early 1990s the internet was becoming popular and there was a need to
find an efficient way of searching \textbf{and finding} web pages for a specific
query. In~\cite{page1999pagerank} Larry Paige, Sergey Brin and Terry Winograd
described an approach they referred to as Page Rank. This became the basis of
the now successful company Google.

In this work, we will describe the problem of ranking web pages with an example,
then go on to describe the basis of the Page Rank algorithm before explaining
how it relates to the eigenvalue of a matrix.

\section{Ranking internet pages}\label{sec:ranking_pages}

The internet can be thought of as a mathematical object called a
graph~\cite{diestel2005graph}.

We can use numpy to generate a random adjacency matrix \(A\) where:

\[
    A_{ij}
    =
    \begin{cases}
        1,&\text{ if }(i,j) \text{ is an edge}\\
        0,&\text{ otherwise}
    \end{cases}
\]

\begin{minted}{python}
>>> size = 10
>>> np.random.seed(0)
>>> A = np.random.choice((0, 1), size=(size, size))
>>> A
array([[0, 1, 1, 0, 1, 1, 1, 1, 1, 1],
       [1, 0, 0, 1, 0, 0, 0, 0, 0, 1],
       [0, 1, 1, 0, 0, 1, 1, 1, 1, 0],
       [1, 0, 1, 0, 1, 1, 0, 1, 1, 0],
       [0, 1, 0, 1, 1, 1, 1, 1, 0, 1],
       [0, 1, 1, 1, 1, 0, 1, 0, 0, 1],
       [1, 0, 1, 0, 1, 0, 0, 0, 0, 0],
       [1, 1, 0, 0, 0, 1, 1, 0, 1, 0],
       [0, 1, 0, 1, 1, 1, 1, 1, 1, 0],
       [1, 1, 0, 0, 1, 0, 0, 1, 1, 0]])
\end{minted}

Each row and column of \(A\) corresponds to a web page (so in our example here
the internet only has 10 web pages). We see that the first web page (the first
row of \(A\) links to all the other webpages except the fourth.

Python has a library for studying networks called networkx. It can be used to
create visualisations:

\begin{minted}{python}
>>> G = nx.from_numpy_array(A, create_using=nx.DiGraph())
>>> plt.figure()
>>> nx.draw(G)
\end{minted}

Figure~\ref{fig:image_of_graph} shows the corresponding network.

\begin{figure}[!hbtp]
    \includegraphics[width=.8\textwidth]{graph.pdf}
    \label{fig:image_of_graph}
\end{figure}

The Page Rank algorithm assumes that users are going to browse the internet in a
random fashion where they are equally likely to go from a page to any of the
other pages that it links to. The score against which it is ranked correspond to
the likelihood of being on a given page:

\begin{minted}{python}
>>> nx.pagerank(G, alpha=1)
{0: 0.11988980099060496,
 1: 0.11337734639341283,
 2: 0.09229073511280955,
 3: 0.08488079369061019,
 4: 0.12554165518523688,
 5: 0.09448445809607922,
 6: 0.09608526108105925,
 7: 0.09314704994034195,
 8: 0.09384213164925923,
 9: 0.08646076786058658}
\end{minted}

\section{Using linear algebra}

We will normalise \(A\) to create a new matrix \(M\) such that the rows all add
to one (so each row give a probability of going to the next page):

\begin{minted}{python}
>>> row_sums = A.sum(axis=1)
>>> M = A / row_sums[:, np.newaxis]
>>> np.round(M, 2)
array([[0.  , 0.12, 0.12, 0.  , 0.12, 0.12, 0.12, 0.12, 0.12, 0.12],
       [0.33, 0.  , 0.  , 0.33, 0.  , 0.  , 0.  , 0.  , 0.  , 0.33],
       [0.  , 0.17, 0.17, 0.  , 0.  , 0.17, 0.17, 0.17, 0.17, 0.  ],
       [0.17, 0.  , 0.17, 0.  , 0.17, 0.17, 0.  , 0.17, 0.17, 0.  ],
       [0.  , 0.14, 0.  , 0.14, 0.14, 0.14, 0.14, 0.14, 0.  , 0.14],
       [0.  , 0.17, 0.17, 0.17, 0.17, 0.  , 0.17, 0.  , 0.  , 0.17],
       [0.33, 0.  , 0.33, 0.  , 0.33, 0.  , 0.  , 0.  , 0.  , 0.  ],
       [0.2 , 0.2 , 0.  , 0.  , 0.  , 0.2 , 0.2 , 0.  , 0.2 , 0.  ],
       [0.  , 0.14, 0.  , 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.  ],
       [0.2 , 0.2 , 0.  , 0.  , 0.2 , 0.  , 0.  , 0.2 , 0.2 , 0.  ]])
\end{minted}

Using this, we can raise \(M\) to a large power to see the long run probability
given starting in any given page.

\begin{minted}{python}
>>> np.round(np.linalg.matrix_power(M, 2000000), 2)
array([[0.12, 0.11, 0.09, 0.08, 0.13, 0.09, 0.1 , 0.09, 0.09, 0.09],
       [0.12, 0.11, 0.09, 0.08, 0.13, 0.09, 0.1 , 0.09, 0.09, 0.09],
       [0.12, 0.11, 0.09, 0.08, 0.13, 0.09, 0.1 , 0.09, 0.09, 0.09],
       [0.12, 0.11, 0.09, 0.08, 0.13, 0.09, 0.1 , 0.09, 0.09, 0.09],
       [0.12, 0.11, 0.09, 0.08, 0.13, 0.09, 0.1 , 0.09, 0.09, 0.09],
       [0.12, 0.11, 0.09, 0.08, 0.13, 0.09, 0.1 , 0.09, 0.09, 0.09],
       [0.12, 0.11, 0.09, 0.08, 0.13, 0.09, 0.1 , 0.09, 0.09, 0.09],
       [0.12, 0.11, 0.09, 0.08, 0.13, 0.09, 0.1 , 0.09, 0.09, 0.09],
       [0.12, 0.11, 0.09, 0.08, 0.13, 0.09, 0.1 , 0.09, 0.09, 0.09],
       [0.12, 0.11, 0.09, 0.08, 0.13, 0.09, 0.1 , 0.09, 0.09, 0.09]])
\end{minted}

We see that all the rows of \(M ^ {2000000}\) are equal and actually correspond to
the values computed using networkx's Page rank.

Interestingly, this set of probabilities can actually be obtained by finding the
right eigenvector of \(M^{T}\):

\begin{minted}{python}
>>> e_values, e_vectors = np.linalg.eig(M.T)
>>> np.real(np.round(e_vectors[:,0] / sum(e_vectors[:,0]), 2))
array([0.12, 0.11, 0.09, 0.08, 0.13, 0.09, 0.1 , 0.09, 0.09, 0.09])
\end{minted}

\section{Conclusion}

The first version of Google's search engine was in fact based on a building
block of linear algebra and graph theory. There are some modifications that
needed to be done to deal with the fact that the internet is not always
completely connected, this corresponds to the \texttt{alpha} argument in the
networkx function above.

\bibliographystyle{plain}
\bibliography{bibliography.bib}

\end{document}


\subsection{Algebraic dual spaces}\label{subsec:algebraic_dual_spaces}

\begin{definition}\label{def:dual_vector_space}
  Let \( V \) be a vector space over \( F \). The \hyperref[thm:functions_over_ring_form_algebra]{vector space} of all \hyperref[def:linear_operator]{linear functions} from \( V \) to \( F \) is called the \term{algebraic dual space} of \( V \) and is denoted by \( V^* \). The functions themselves are called \term{linear functionals}.
\end{definition}

\begin{definition}\label{def:canonical_duality_pairing}
  We denote by \( \inprod \cdot \cdot \) the \term{canonical \hyperref[def:duality_pairing]{duality pairing}}
  \begin{balign*}
     & \inprod \cdot \cdot: V^* \times X \to \BbbK \\
     & \inprod {x^*} x \mapsto x^*(x).
  \end{balign*}

  We are usually not interested in other duality pairings. See \fullref{def:locally_convex_duality_pairing}, however.
\end{definition}

\begin{definition}\label{def:double_dual_canonical_embedding}
  Fix a vector space \( V \). We define the \term{canonical embedding} into the double dual \( V^{**} \) of \( V \) by
  \begin{balign*}
     & \Phi: V \to V^{**}                              \\
     & \Phi(x) \coloneqq (\varphi \mapsto \varphi(x)),
  \end{balign*}
  where \( \varphi \in V^* \).
\end{definition}

\begin{proposition}\label{thm:finite_dimensional_dual_space_is_isomorphic}
  The dual vector space of a finite-dimensional vector space has the same dimension.
\end{proposition}
\begin{proof}
  Let \( V \) be an \( n \)-dimensional vector space over \( F \) and let \( B \) be a basis of \( V \). For each \( b \in B \), define its dual vector on \( V^* \) as the linear \hyperref[thm:linear_map_iff_function_on_basis]{extension} of the functions
  \begin{balign*}
     & \varphi: B \to F                               \\
     & \varphi(x) \coloneqq \begin{cases}
      1, & x = b    \\
      0, & x \neq b
    \end{cases}
  \end{balign*}
  from the basis to the whole space. Denote the dual basis vector of \( b \) by \( b^* \).

  We will now show that the set \( B^* \coloneqq \{ b^* \colon b \in B \} \) forms a basis of \( V^* \).

  Fix \( x^* \in V^* \). Define
  \begin{equation*}
    y^* \coloneqq \sum_{b \in B} x^*(b) b^*.
  \end{equation*}

  The linear functions \( x^* \) and \( y^* \) evidently agree on the basis \( B \). By \fullref{thm:linear_maps_agree_on_free_module_if_they_agree_on_basis}, they agree on the whole space.

  Hence, \( B^* \) is a basis of \( V^* \). Note that it has the same cardinality as the basis of \( B \).
\end{proof}

\begin{remark}\label{rem:finite_dimensional_dual_space_isomorphism}
  By \fullref{thm:finite_dimensional_spaces_are_isomorphic}, the vector space \( F^n \) is isomorphic to its dual \( {F^n}^* \).

  In practice, it is sometimes useful to distinguish between vectors and functionals. This is why we regard functionals as either
  \begin{itemize}
    \item functions
    \item column vectors
    \item row vectors
  \end{itemize}
  depending on what interpretation suits us best.

  This is consistent with \fullref{thm:finite_dimensional_operators_are_isomorphic_to_matrices}, where we regard linear operators as matrices that act on vectors by multiplication.

  For example, if we have the \hyperref[def:differentiability]{differentiable} function \( f(x, y) = xy \), we can regard its gradient at the point \( (\overline x, \overline y) \) as the row vector
  \begin{balign*}
    f'(\overline x, \overline y) =
    \begin{pmatrix}
      \overline y & \overline x
    \end{pmatrix}.
  \end{balign*}

  This is a linear functional that can acts on regular (column) vector by multiplying them from the left.
\end{remark}

\begin{definition}\label{def:dual_linear_operator}
  We define the \term{dual linear operator} of \( L: U \to V \) as
  \begin{balign*}
     & L^*: V^* \to U^*                \\
     & L^*(v^*) \coloneqq v^* \circ L.
  \end{balign*}
\end{definition}

\begin{definition}\label{def:vector_space_annihilator}\mcite[52]{Knapp2016BasicAlgebra}
  Annihilators in vector spaces are quite different than annihilators in modules (see \fullref{def:left_module_annihilator}).

  Fix a subset \( S \subseteq V \) of a vector space \( V \) over \( F \). We define the \term{annihilator} of \( S \) as the vector space of functionals
  \begin{equation*}
    \op{ann}(S) \coloneqq \{ x^* \in V^* \colon x^*(x) = 0_F \quad\forall x \in S \}.
  \end{equation*}
\end{definition}


\chapter{Constructions}
\section {Semidirect Product}
{\bf Definition 3:}
$G$ is an \emph{extension} of $K$ by $Q$ if $G \triangleright K$ and $G/K \cong Q$.  Equivalently,
there is a surjective homomorphism $\pi:G \rightarrow Q$.
\\
\\
{\bf Definition 1:}
Let $\pi: A \rightarrow Aut(K)$, $K \lhd G$, $H<G$ and $H \cap K = 1$ define the
\emph {semi-direct} product of $K$ by $H$ as the group who's elements
are $(a,b) \in H \times K$ with the product $(a, g) \cdot (b,h)= (ab, \pi(b)(g)h)$.
\\
\\
{\bf Definition 2:} An extension $G$ of $K$ by $H$ \emph {splits} if G
is a semi-direct product of $H$ and $K$.
\\
\\
{\bf Theorem 1:}
If $1 \rightarrow N \rightarrow_{i} G \rightarrow_{\varphi} Q \rightarrow 1$, the following
are equivalent
(1) $\exists H \subseteq G$ such that $\varphi: H \rightarrow Q$ is an isomorphism;
(2) $\exists s:Q \rightarrow G$ such that $\varphi \cdot s = id$;
(3) $G$ is a semi-direct product of $N$ by $H$ written $N \ltimes Q$; in this
case, we say $G$ is a split extension of $N$ by $H$.
\begin{quote}
\emph{Proof:}
\\
\\
$1 \rightarrow 2$:
Put $s= \varphi_{|H}^{-1}$, then (2) holds.
\\
\\
$2 \rightarrow 3$:
Let $H= s(Q)$, $N= ker(\varphi)$.  Suppose $x \in G$ and $q= \varphi(x)$.  Set
$h= s(q)$.  $\varphi(x h^{-1})= \varphi(x) \varphi(h)^{-1} = q q^{-1}= 1$ so
$x h^{-1} \in N$ and $x= nh$.
\\
\\
$3 \rightarrow 1$:
If $G= N \ltimes H$, the map $\varphi_{|H}$ is an isomorphism onto $Q$ and $H$ satisfies the
conditions of (1).
\end{quote}
\section {Presentations}
{\bf Theorem 2:}  Let $G=
\langle x_1, \ldots, x_n| R_1(x_1, \ldots, x_n), R_2(x_1, \ldots, x_n), \ldots R_m(x_1, \ldots, x_n)
\rangle
$. There is a one to one correspondance between homomorphisms
$\rho: G \rightarrow H$ and solutions to
$R_1(y_1, \ldots, y_n)=1, R_2(y_1, \ldots, y_n)=1, \ldots R_m(y_1, \ldots, y_n)=1, y_i \in H$.
\\
\\
{\bf Theorem 3:}  Two groups are isomorphic iff they admit the same presentation up to renaming.
\\
\\
{\bf Theorem 4:}  Let $G \cong F/R$, $F$, free.  Then for arbitrary abelian $A$, the transgression map
$Hom(R/[F,R],A) \rightarrow H^2(G,A)$ is associated with the natural exact sequence
$1 \rightarrow R/[F,R] \rightarrow F/[F, R] \rightarrow G \rightarrow 1$.
\\
\\
{\bf Projective representations and Schur:} Let $\rho: G \rightarrow GL_n({\mathbb C})/{\mathbb Z}$
so that $\rho(g)$ is a coset $\pmod {\mathbb Z}$.  Choose $A(g) \in \rho(g)$ then
$\rho(ab)= r_{a,b} \rho(a) \rho(b)$ and $r_{a,b} r_{ab,c}= r_{a, bc} r_{b,c}$ (cocycle identity).
Conversely, given an $r$ satisfying the cocycle identity, there is a projective representation
of degree $|G|=m$ giving rise to it.  If $B(g) \in \rho(g)$ is another, $A(g)= d_g B(g)$, and
if $B(ab)= s_{a,b}B(a)B(b)$, $r_{a,b}= {\frac {d_a d_b} {d_{ab}}} s_{a,b}$ (Relation 1).
If $r, s$ satisfy relation 1, they are called \emph{equivalent}.
\\
\\
{\bf Theorem 5:} Let $\rho^*: H \rightarrow GL_n({\mathbb C})$ be an ordinary irreducible
representation of $H$ with $A \le {\mathbb Z}(H)$ then $\rho^*(a)= j_a I_n$ and we can
define $\rho: H/A \rightarrow GL_n({\mathbb C})/{\mathbb Z}$ by
$\rho(hA)= \rho^*(h) {\mathbb Z}$.  Schur showed the converse:
$\exists H: A \le {\mathbb Z}(H)$ with $H/A \cong G$.
\begin{quote}
\emph{Proof:}
\end{quote}
Note that if $G= \langle F|R \rangle $ is of rank $k$,
$(F/[F,R])/(R/[F,R])= G$ and $R/[F,R]$ is central.
\section {Central Product}
{\bf Definition of Central Product:}
$G= \langle G_i \rangle , 1 \le i \le n$, $[G_i , G_j ]=1$ for $i \ne j$.  Equivalently,
$\rho: (x_1, x_2, \ldots , x_n) \mapsto x_1 x_2 \ldots x_n$ is a surjective homomorphism
from $D= (G_1 \times G_2 \times \ldots \times G_n)$ to $G$ with $\rho(D_i) = G_i$ with
$\pi_i (G_1, \ldots , G_n)= D_i$ and $ker(\rho) \cap D_i = 1$, $ker(\rho) \subseteq {\mathbb Z}(G)$.
\begin{quote}
\emph{Proof of equivalence:}
Let $\alpha_i: {\mathbb Z}(G_1) \rightarrow {\mathbb Z}(G_i)$.
$E= \langle z(\alpha_i(z^{-1})) \rangle $.  $E$ is a complement to ${\mathbb Z}(D_i )$ in
$Z= \langle {\mathbb Z}(D_i) \rangle $ so $D/E$ is a central product.  Suppose $G$ is a
central product of the $G_i$ with ${\mathbb Z}(G_i)={\mathbb Z}(G_1)$ and $\rho$ the surjective
homomorphism.  Let $\beta_i:{\mathbb Z}(D_1) \rightarrow {\mathbb Z}(G_i)$ be the composition of
$\rho_{|{\mathbb Z}(D_1)}$ and
$\rho_{|{\mathbb Z}(D_i)}^{-1}: {\mathbb Z}(G_1) \rightarrow {\mathbb Z}(D_i)$ then
$ker(\rho)= \langle x \beta_i(z^{-1}), z \in {\mathbb Z}(D_1) \rangle = A$ is a complement
to ${\mathbb Z}(D_i)$ in $Z$ and $G \cong D/A$.  Define $\gamma \in Aut(D)$ with $\gamma(D_1)=D_i$
and $\gamma(E)=A$.  This induces and isomorphism of $D/E$ with $D/A$.  Let
$\delta_i= \beta_i (\alpha_i)^{-1}$ then $\gamma_{i|{\mathbb Z}(D_i)}= \delta_i$ and
define $\gamma: D \rightarrow D$ by $(x_1, x_2 , \ldots , x_n) \mapsto (x_1, \gamma_1(x_2), \ldots,
\gamma_n(x_n))$.
\end{quote}
{\bf Example:}
Both $D_8$ and
$Q_8$ are central products of $Z_2$ by $Z_2 \times Z_2$.  Note that $D_8$ is also a direct product but $Q_8$
is not.
\\
\\
{\bf Theorem 6:}
Let $G_i, 1 \le i \le n$ be a family of groups with $Z(g_1)=Z(G_i)$ and
$Aut_{G_i}(Z(G_i))=Aut(Z(G_i))$.  The up to isomorphism there is a unique
central product with $Z(G_1)=Z(G_i)$.
\begin{quote}
\emph{Proof:}
By hypothesis, there are isomorphisms
$\alpha_i : {\mathbb Z}( D_1 ) \rightarrow {\mathbb Z}( D_i )$, $1 \le i \le n$.
Let $E$ be the subgroup of $D$ generates by $\{ \alpha_i (z^{-1})z, z \in {\mathbb Z}(D_1 )$.
$E$ is a complement to
${\mathbb Z}(D_i)$ in $Z= \langle {\mathbb Z}(D_i), 1 \le i \le n \rangle = {\mathbb Z}(D)$ so
$D/E$ is a central product of the $G_i$ with ${\mathbb Z}(G_1 ) = {\mathbb Z}(G_i )$ by
the foregoing equivalence.  Now suppose $G$ is a central product of the $G_i$ with
${\mathbb Z}(G_1 ) = {\mathbb Z}(G_i )$ and let $\pi: D \rightarrow G$ be the surjective
homomorphism in the equivalence proof.
Let $\beta_i : {\mathbb Z}(D_1 ) \rightarrow {\mathbb Z}(D_i )$ be the composition of
$\pi_{|{\mathbb Z}(D_1 )}$ and $\pi_{|{\mathbb Z}(D_i )}^{-1}$.
$ker(\pi ) = \langle z \beta_i(Z^{-1} ): z \in {\mathbb Z}(D_1 ), 1 \le i \le n \rangle
= A$ is a complement
to ${\mathbb Z}(D_i )$ in $Z$ for each $i$ and $G \cong D/A$.  Now let
$\delta_i = \beta_i \alpha_i^{-1}$ and $\delta_i \in Aut({\mathbb Z}(D_i ))$.  By
hypothesis, $\exists \gamma_i \in Aut( D_i )$ with $\gamma_{i | {\mathbb Z}(D_i )}= \delta_i$.
Define $\gamma : D \rightarrow D$ by
$(x_1 , x_2 , \ldots , x_n ) \mapsto
(x_1 , \gamma_2(x_2), \ldots , \gamma_n(x_n))$.  $\gamma \in Aut(D)$ and
$\alpha_i(\gamma (z^{-1})) z$ so $\gamma(E)=A$ and we're done.
\end{quote}
\section {Wreath Product}
{\bf Wreath Product:} $G^*= G^{X}$ - maps from $X$ to $G$.  $fg(x)= f(x) g(x)$.
Let $H$ act on $X$: $f^h(x)= f(xh^{-1})$.  Let $\phi$ be the natural action of
$H$ induced on $G^{|H|}$, then $G \wr H = H \rtimes_{\phi} G^*$. If $G_x =
\{f: f(y)= 1 \; if \; x \ne y \}$.  $G^* = \prod_X G_x$.
Put $g_x (y)= g (y)$ if $x=y$, 1 otherwise.  Note that
${g_x}^h= g_{xh}$.
\\
\\
{\bf Theorem 7:} If $D$ and $Q$ are groups with $Q$ finite then the regular wreath product
$D \wr_r Q$ contains an isomorphic copy of every extension of $D$ by $Q$.
\begin{quote}
\emph{Proof:}
If $G$ is an extension of $D$ by $Q$ then there is a surjective homomorphism
$G \rightarrow Q$ with kernel $D$ denoted by $a \mapsto {\overline a}$.
Choose a transversal $l: Q \rightarrow G$.  For $a \in G$, define
$\sigma_a(x) = l(x)^{-1} a l({\overline {a^{-1}}} x)$.
If $a, b \in G$ then
$
\sigma_a(x) \sigma_b^{\overline a}(x) =
\sigma_a(x) \sigma_b({\overline {a^{-1}}} x) =
l(x)^{-1} a l({\overline {a^{-1}}} x)
l(x)^{-1} a l({\overline {a^{-1}}} x)
l({\overline {a^{-1}}} x)^{-1} b l({\overline { b^{-1} a^{-1}}} x) =
l(x)^{-1} a b l({\overline { b^{-1} a^{-1}}} x) = \sigma_{ab}(x)$.
Define $\varphi : G \rightarrow D \wr_r Q$ by
$\varphi(a) = (\sigma_a, {\overline a}), \forall a \in G$.  A simple calculation shows
$\varphi$ is a injective homomorphism.
\end{quote}


%lecture 5

\subsection{Fixed point theory}
\label{sub:fixed_point_theory}
\begin{beispiel}
	Consider
	\[
		f(x)+ 5 \int_{0}^{1-x} \min(x,y)f(y) \,\mathrm{d}y = g(x), \qquad x \in [0,1] \qquad (*)
	\]
	where $g \in C([0,1])$. \\
	\textbf{Claim:} \text{    }     There exists an unique solution $f \in C([0,1])$ that $(*)$. \\
	Idea:
	\[
		f(x) = f(x) - 5 \int_{0}^{1-x} \min(x,y)f(y) \,\mathrm{d}y, \qquad x \in [0,1].
	\]
	Set for $x \in [0,1]$
	\[
			\tilde T(f)(x) = RHS(x).
	\]
	To find a solution to $(*)$ is the same finding $f \in C([0,1])$ such that
	\[
		f = \tilde T(f).
	\]
	Clearly $ \tilde T : C([0,1]) \to C([0,1])$. (continual later).
\end{beispiel}

\begin{theorem}[Banach's fixed point theorem]
	$(E, \norm{.})$ Banach space. $T: E \to E$ (no assumption on linearity) is a contraction on $E$, i.e. there exists $c<1$ such that
	\[
		\norm{T(x)-T(\tilde x)} \leq c \norm{x- \tilde x} \qquad \text{for all }x,\tilde x \in E.
	\]
	Then there exists a unique $ \bar{x} \in E$ such that
	\[
		\bar{x} = T( \bar{ x}).
	\]
	($\bar{x}$ is a fixed point)
\end{theorem}
\begin{beweis}
	\begin{description}
		\item[Uniqueness:]Assume $T( \bar{x}) = \bar{x}$ and $T ( \tilde x) = \tilde x$. Then
		\[
			\underset{\geq 0}{\underbrace{\norm{ \bar{ x} - \tilde x }}} = \norm{T( \bar{x})- T( \tilde x)} \leq \underset{< 1}{\underbrace{c}} \norm{ \bar{x}- \tilde x}.
		\]
		Thus $\norm{\bar{x}- \tilde x} = 0$, i.e. $\bar{x} = \tilde x$.
		\item[Existence:] Pick an arbitrary $x_0 \in E$. Set
		\[
			x_{n+1} = T(x_{n}), \qquad n=0,1,2,\dots.
		\]
		\textbf{Claim:} \text{    }     $(x_n)_{n=1}^{\infty}$ is a Cauchy sequence in $(E,\norm{.})$.
		Note:
		\begin{align*}
			\norm{x_{n+1}-x_n}  &= \norm{T(x_n)-T(x_{n-1})} \\
			&\leq  c \norm{x_n - x_{n-1}} \\
			&\leq \dots \\
			&\leq  c^n \norm{x_1-x_0}, \qquad n=1,2,\dots.
		\end{align*}
		For $n>m$
		\begin{align*}
			\norm{x_n-x_m} &= \norm{x_n - x_{n-1}+ x_{n-1}- \dots + x_{m+1}- x_m} \\
			&\leq \norm{x_n - x_{n-1}} + \norm{x_{n-1} - x_{n-2}} + \dots + \norm{x_{m+1}- x_m} \\
			&\leq (c^{n-1}+ c^{n-2} + \dots c^m) \norm{x_1-x_0} \\
			&\leq \frac{c^m}{1-c} \norm{x_1 - x_0} \to 0 \qquad \text{as }n,m \to \infty.
		\end{align*}
		Hence $(x_n)_{n=1}^{\infty}$ is a Cauchy sequence in $(E,\norm{.})$. $(E, \norm{.})$ is a Banach space. So $(x_n)_{n=1}^{\infty}$ converges in $(E,\norm{.})$. Call the limit $\bar{x}$. \\
		\textbf{Claim:} \text{    }     $\bar{x}$ is a fixed point for $T$.
		\begin{align*}
			\norm{\bar{ x}- T(\bar{x})} & = \norm{\bar{x}-x_{n+1}+ x_{n+1} - T(\bar{x})} \\
			&\leq \norm{\bar{x}-x_{n+1}} + \norm{\underset{T(x_n)}{\underbrace{x_{n+1}}} - T(\bar{x})} \\
			&\leq \underset{\to 0}{\underbrace{\norm{\bar{x}- x_{n+1}}}} + c \underset{\to 0}{\underbrace{\norm{x_n - \bar{x}}}} \to 0, \qquad n \to \infty
		\end{align*}
	\end{description}
\end{beweis}
\begin{bemerkung}
	\begin{enumerate}[(1)]
		\item $x_n \to \bar{x}$ for $n \to \infty$ independend of the choice of $x_0$
		\item Fix $z \in E$
		\begin{align*}
			\norm{\bar{x}-z} &= \norm{T(\bar{x})- T(z) + T(z) -z} \\
			&\leq \norm{T(\bar{x})-T(z)} + \norm{T(z) -z} \\
			&\leq c \norm{\bar{x}-z} + \norm{T(z) - z}.
		\end{align*}
		Hence
		\[
			\norm{\bar{x}-z} \leq \frac{1}{1-c}\norm{T(z)-z}.
		\]
	\end{enumerate}
\end{bemerkung}
\begin{beispiel}
	Consider now the example from above: $(C([0,1]), \norm{.})$ with $\norm{f} = \max_{x \in [0,1]}\abs{f(x)}$ is a Banach space! To apply Banach's fixed point theorem we need $\tilde T$ to be a contraction. \\
	Fix $f_1,f_2 \in C([0,1])$ and get for $x \in [0,1]$
	\begin{align*}
		\abs{(\tilde T(f_1)- \tilde T(f_2))(x)} & = \abs{5 \int_{0}^{1-x} \min(x,y)f_2(y) \,\mathrm{d}y - 5 \int_{0}^{1-x}\min(x,y)f_1(y) \,\mathrm{d}y} \\
		&= \abs{5 \int_{0}^{1-x} \min(x,y)(f_2(y)-f_1(y)) \,\mathrm{d}y} \\
		&\leq 5 \int_{0}^{1-x} \min(x,y) \underset{\leq \norm{f_2-f_1}}{\underbrace{\abs{f_2(y)-f_1(y)}}} \,\mathrm{d}y \\
		&\leq 5 \underset{0 \leq  \dots \leq \frac{1}{6}}{\underbrace{\int_{0}^{1-x} \min(x,y)\,\mathrm{d}y}} \norm{f_2-f_1} \\
		&\leq \frac{5}{6} \norm{f_2-f_1}.
	\end{align*}
	Hence \[
		\norm{\tilde T(f_1)- \tilde T(f_2)} \leq \frac{5}{6} \norm{f_1-f_2}.
	\]
	We conclude that $\tilde T$ is a contraction. We can take $c = \frac{5}{6}$. By Banach's fixed point theorem $\tilde T$ has a unique fixed point. Finally $(*)$
	has a unique solution $f \in C([0,1])$ which is the fixed point.
\end{beispiel}
\begin{theorem}[Banach's fixed point theorem (generalization)]
	$(E, \norm{.})$ Banach space. $T: F \to F$ where $F$ is a closed set in $E$. $N$ positive integer. Assume $T^N = \underset{N-\text{times}}{\underbrace{T \circ T \circ \dots \circ T}}$ is a contraction on $F$, i.e. there exists $c > 1$ such that
	\[
		\norm{T^N(x)- T^N(\tilde x)} \leq c \norm{x-\tilde x}, \qquad \text{for all }x, \tilde x \in F.
	\]
	Then $T$ has unique fixed point $\bar{ x}$, i.e.
	\[
		\bar{x} = T(\bar{x}) \in F.
	\]
\end{theorem}
\begin{beweis}
	\begin{description}
		\item[$N=1$:] Fix $x_0 \in F$ and consider $(x_n)_{n=1}^{\infty}$ where $x_{n+1} = T(x_n)$ for $n=0,1,2, \dots$. There $(x_n)_{n=1}^{\infty}$ is a Cauchy sequence and hence this converges in $E$ since this is a Banach space. Call the limit $\bar{x}$. Note
		\[
			\underset{\in F}{\underbrace{x_n}} \to \bar{x} \text{ in }E \text{ and $F$ is closed}
		\]
		implies $\bar{x} \in F$. The rest of the argument is the same as before.
		\item[$N>1$:] By previous result we know that $T^N$ has a unique fixed point $\bar{x} \in F$, i.e. $\bar{x} = T^N(\bar{x})$. \\
		\textbf{Claim:} \text{    }     $\bar{x}$ is a fixed point for $T$.
		\begin{align*}
			\norm{T(\bar{x})-\bar{x}} &= \norm{T(T^N(\bar{x}))- T^N(\bar{x})} \\
			&= \norm{T^N(T(\bar{x}))- T^N(\bar{x})} \\
			&\leq c \norm{T(\bar{x})-\bar{x}}.
 		\end{align*}
		This gives
		\[
			\norm{T(\bar{x}-\bar{x})} = 0, \qquad \text{i.e. }\bar{x} = T(\bar{x}).
		\]
		Existence of a fixed point for $T$ done. For the uniqueness assume $\bar{x} = T(\bar{x})$ and $\tilde x = T( \tilde x)$. Then
	\begin{align*}
		\bar{x} &= T( \bar{x}) = T^2(\bar{x}) = \dots = T^N(\bar{x}) \\
		\tilde x &= T(\tilde x) = T^2(\tilde x) = \dots = T^N(\tilde x).
	\end{align*}
	But $T^N$ has a unique fixed point so
	\[
		\bar{x} = \tilde x.
	\]
	\end{description}
\end{beweis}
\begin{bemerkung}
	\begin{enumerate}[(1)]
		\item $T: (0,1] \to (0,1]$ where $T(x) = \frac{x}{2}$. Clearly $T$ is a contraction on $(0,1]$ but has no fixed point. Note that $(0,1]$ is not a closed intervall.
		\item $T: [0,\infty) \to [0,\infty)$, where $T(x) = x + \frac{1}{x}$. Clearly $[0,\infty)$ is a closed intervall in $\mathbb{R}$ but $T$ has no fixed point. \\
		\textbf{Claim:} \text{    }     $T$ is not a contraction but 'close' to be a contraction. \\
		\begin{align*}
			\abs{T(x)-T(\tilde x)} < \abs{x- \tilde x} \qquad \text{for }x, \tilde x \in [1, \infty), x \neq \tilde x
		\end{align*}
		Note \[
			\abs{ T(x)- T( \tilde x)} = \abs{\underset{\substack{(1- \frac{1}{t})\leq 1 \\ \text{for }t \in [1,\infty)}}{\underbrace{T'(x)}}}\abs{x- \tilde x}
		\] for some $t$ betweeen $x$ and $\tilde x$.
	\end{enumerate}
\end{bemerkung}
\begin{beispiel}
	$(E,\norm{.})$ Banach space. $K$ compact set in $E$ and $T : K \to K$ where
	\[
		\norm{T(x)- T( \bar{x})} < \norm{x - \bar{x}} \qquad \text{for all }x, \bar{x} \in K, x \neq \bar{x}.
	\]
	Show: $T$ has a unique fixed point in $K$.
	\begin{description}
		\item[Uniqueness:] Assume $\bar{x} = T(\bar{x})$ and $\tilde x = T( \tilde x)$ and $\bar{x} \neq \tilde x$ for $ \bar{x}, \tilde x \in K$. Then
		\[
			\norm{\bar{x} - \tilde x} = \norm{ T( \bar{ x})- \tilde x} < \norm{ \bar{x}- \tilde x}.
		\]
		Contradiction because then $\bar{x} = \tilde x$.
		\item[Existence:] To show: There exists $x \in K$ such that $x = T(x)$, i.e.
		\[
			\norm{T(x)- x} = 0.
		\]
		Set $d := \inf_{x \in K} \norm{T(x)-x}$. Let $(x_n)_{n=1}^{\infty}$ be a sequence in $K$ such that
		\[
			\norm{T(x_n)-x_n} \to d, \qquad \text{as }n \to \infty.
		\]
		$K$ compact implies that there exists a subsequence $(\tilde x_n)_{n=1}^{\infty}$ of $(x_n)_{n=1}^{\infty}$ such that $(\tilde x_n)_{n=1}^{\infty}$ converges in $K$. Call the limit element $\bar{x} \in K$. We know
		\[
			\tilde x_n \to  \bar{x} \qquad \text{in }K
		\]
		and
		\[
			\norm{T( \tilde x_n)- \tilde x_n} \to d.
		\]
		Question: \[
			T(\tilde x_n) \to T(\bar{x}) \qquad \text{ in }K?
		\]
		But since
		\[
			\norm{T(x)- T( \tilde x)} \leq  \norm{x- \tilde x} \qquad \text{ for all }x, \tilde x \in K
		\]
		we have
		\[
			\tilde x_n \to \bar{x} \qquad \text{ in }K
		\]
		which implies
		\[
			T( \tilde x_n) \to T( \bar{ x}) \text{ in }K.
		\]
		Hence:
		\[
			\norm{T( \bar{ x})- \bar{x}} \leftarrow \norm{T(\tilde x_n)- \tilde x_n} \to d, \qquad  n \to  \infty.
		\]
		We obtain
		\[
			\norm{T(\bar{x})- \bar{x}} = d.
		\]
		Question: Is $d=0$? \\
		If $d>0$ then $\bar{x} \neq  T( \bar{x})$, $\bar{x}, T( \bar{x}) \in K$
		\[
			\norm{T(\bar{x})- T(T(\bar{x}))} < \norm{\bar{x}- T(\bar{x})} = d = \inf_{x \in K} \norm{x- T(x)}.
		\]
		This is a contradiction which gives $d=0$ and so $\bar{x} = T(\bar{x})$.
	\end{description}
\end{beispiel}
\begin{beispiel}
	Consider
	\[
		f(x) = \int_{0}^{x}k(x,y)h(y,f(y)) \,\mathrm{d}y + g(x), \qquad x \in [0,1] \qquad (*),
	\]
	where $g \in C([0,1])$, $k \in C([0,1] \times [0,1])$ and $h: [0,1] \times \mathbb{R} \to \mathbb{R}$ continuous and satisfies: \\
	There exists $M>0$ such that
	\[
		\abs{h(x,z_1)-h(x,z_2)} \leq M \abs{z_1- z_2} \qquad \text{for all }x \in [0,1],\,z_1,z_2 \in \mathbb{R}.
	\]
	\textbf{Claim:} \text{    }     $(*)$ has a unique solution $f \in C([0,1])$. \\ For $f \in C([0,1])$ set
	\[
		T(f)(x) = \int_{0}^{x}k(x,y)h(y,f(y)) \,\mathrm{d}y + g(x) \qquad x \in [0,1].
	\]
	Here $T(f)(x) \in C([0,1])$. \\ Want to show: $T: C([0,1]) \to C([0,1])$ has a unique fixed point. \\
	Start with the Banach space $(C([0,1]), \text{max-norm})$. Check if $T$ is a contraction in $C([0,1])$. Fix $f_1,f2 \in C([0,1])$
	\[
		T(f_1)(x)- T(f_2)(x) = \int_{0}^{x} k(x,y)(h(y,f_1(y))-h(y,f_2(y))) \,\mathrm{d}y.
	\]
	$k$ is continuous on the compact set $[0,1] \times [0,1]$ so
	\[
		\sup\limits_{(x,y) \in [0,1]\times[0,1]} \abs{k(x,y)} =: N < \infty.
	\]
	We obtain
	\begin{align*}
		\abs{(T(f_1)-T(f_2))(x)} &\leq \int_{0}^{x} \underset{\leq N}{\underbrace{\abs{k(x,y)}}}\underset{\leq M \underset{\leq \norm{f_1-f_2}}{\underbrace{f_1(y)-f_2(y)}}}{\underbrace{h(y,f_1(y))-h(y,f_2(y))}} \,\mathrm{d}y \\ & \leq \int_{0}^{x}NM \,\mathrm{d}y \norm{f_1-f_2} \\& \leq NM \norm{f_1-f_2}.
		\end{align*}
	This yields
	\[
		\norm{T(f_1)-T(f_2)} \leq NM \norm{f_1-f_2}.
	\]
	\begin{Large}
		\underline{IF:}
	\end{Large} \,$NM<1$ Then $T$ is a contaction. \\ Trick: For $a >0$ set
	\[
		\norm{f}_a = \max\limits_{x \in [0,1]} e^{-ax}\abs{f(x)}
	\]
	for $f \in C([0,1])$. \\
	\textbf{Claim:} \text{    }     $\norm{.}_a$ defines a norm on $C([0,1])$. This is easy to check. \\
	\textbf{Claim:} \text{    }     $\norm{.}$ and $\norm{.}_a$ are equivalent. \\
	This follows from
	\[
		e^{-a} \norm{f} \leq \norm{f}_a \leq \norm{f}
	\]
	for all $f \in C([0,1])$ (note that $\norm{.}$ is the max-norm).\\
	\textbf{Claim:} \text{    }     $(C([0,1]), \norm{.}_a)$ is a Banach space. \\
	This follows from the fact that $\norm{.}$ und $\norm{.}_a$ are equivalent and $(C([0,1]),\norm{.})$ is a Banach space. \\
	\textbf{Claim:} \text{    }     $T$ is a contraction on $(C([0,1]),\norm{.}_a)$ for $a >0$ large enough. \\
	For $f_1,f_2 \in C([0,1])$ and $x \in [0,1]$ we have
	\begin{align*}
		\abs{(T(f_1)-T(f_2))(x)} &\leq \int_{0}^{x} NM \abs{(f_1-f_2)(y)} \,\mathrm{d}y \\
		&= \int_{0}^{x}NM e^{ay} \cdot \underset{\leq \norm{f_1-f_2}_a}{\underbrace{e^{-ay} \abs{(f_1-f_2)(x)}}} \,\mathrm{d}y \\
		&\leq NM \underset{\frac{1}{a}(e^{ax}-1)}{\underbrace{\int_{0}^{x} e^{ay} \,\mathrm{d}y}} \norm{f_1-f_2}_a.
	\end{align*}
	So
	\[
		e^{-ax} \abs{(T(f_1)-T(f_2))(x)} \leq \frac{NM}{a}(1-e^{-ax})\norm{f_1-f_2}_a
	\]
	and
	\[
		\norm{T(f_1)-T(f_2)}_a \leq  \frac{NM}{a} \norm{f_1-f_2}_a
	\]
	For $a > NM$ is $T$ a contraction on $(C([0,1]),\norm{.}_a)$. Banach fixed point theorem implies that there is a unique $f \in C([0,1])$ that solves $(*)$.
\end{beispiel}


As a closing remark, let us briefly consider the question of \textit{what can and cannot be computed} with our logic language. On the surface our logic language looks very different from other programming languages such as, say, Java or F\#. Does this difference translate into higher or lower expressive power? In this chapter, we briefly and informally explore questions about the expressive power of different programming languages and programming paradigms.

We will do so to justify the importance of programming languages and appropriate formalisms, but not for the purpose of expressing constructs that cannot be expressed into the other formalisms: rather, different languages and formalisms allow \textit{human users} to better reason, and as such do offer a different perspective on the same object (the set of all programs) which lets some different properties emerge better.

We will begin by discussing intrinsically hard problems that cannot have an algorithmic answer. These problems make up the boundaries of computability, and as such if we could\footnote{Spoiler alert: we can noooooooooooooooooooot!} define a formalism that solves one of this problems algorithmically then we would have found something that is intrinsically more expressive than the formalisms and languages that we use now.

\subsection{Halting problem}
We will now explore the simple question: \textit{are there limits to what can be computed?} In order to effectively manipulate programs we build a way to translate an arbitrary formula within some formalism into a natural number, that is we show a way to encode arbitrarily complex data structures into simpler objects that are easier to manipulate.

This means that any statement within our formalism is now a natural number, and thus we have an isomorphism between ``properties of statements'' $\Leftrightarrow$ ``properties of numbers'', but ``properties of numbers'' are easier to determine algorithmically because we do not need to define data structures.

\paragraph{Gödel numbering}
There are many ways to define such an encoding. We present Gödel's original encoding from many decades ago, mostly for historical reasons\footnote{A nice side-effect is that this encoding still works, which in a world of short-lived, superficial technological innovation where stuff stops working after a couple of years might seem strange. ``Should we not use something new and shiny?'', I hear you say: ``no.''}

We assign a unique natural number to each basic symbol in our language, for example:
\begin{itemize}
\item \texttt{if} $=0$
\item \texttt{var} $=1$
\item \texttt{,} $=2$
\item ...
\end{itemize}

Given any formula $x_1,x_2,\dots,x_n$, $x_i$ where each $x_i$ is a symbol with associated a number from the list above, we define the encoding function:

$$enc(x_1,\dots,x_n)=2^{x_1} \cdot3^{x_2} \cdot \dots \cdot p_n^{x_n}$$

where $p_n$ is the $n$-th prime number. This means that we encode the $i$-th symbol of the sequence as the exponent of the $i$-th prime number. According to the fundamental theorem of arithmetic, we can always extract back the original sequence with a finite number of steps.

For example, in a specific Gödel numbering, the Gödel number for $0$ could be $6$, and the Gödel number for $=$ could be $5$. Thus, in such a system, the Gödel number of formula $0 = 0$ would be $2^6 \times 3^5 \times 5^6 = 243000000$.

\paragraph{Halting}
Let us now consider the halting problem itself. Consider the problem of determining whether a program \texttt{i} terminates on input \texttt{x}. We can formulate this problem as finding function $h(i,x)$ that returns:

\begin{itemize}
\item $1$ if program $i$ terminates on input $x$
\item $0$ otherwise
\end{itemize}

Since we wish to automate this process, we want to find a way to translate function $h(i,x)$ into some program. For this to be possible, then function $h$ needs to be \textit{total} and \textit{computable}. A function is total if, for every input, it gives back a result. Trivially, function $h$ is defined so as to always return either $0$ or $1$, therefore it is total by definition. A function is computable if it can be encoded into a Turing machine or any equivalent formalism such as the $\lambda$-calculus. We shall further discuss these formalisms later, but for the moment let us treat them as mathematically flavoured assembly languages.

We will now show\footnote{\textit{Show} in this case is meant in the informal sense.} a shocking result: no arbitrary total computable function $f$ can be equal to $h$, and we will do so by constructing a counter example. The outline of the proof is as follows:
\begin{itemize}
\item we consider an \textit{arbitrary} total computable function $f$
\item we construct a partial but computable function $g$ that is based on $f$
\item we feed $g$ into $h$ (the halting function) and show that $f \neq h$
\item since $f$ was an \textit{arbitrary} total computable function, then there exists no total computable function $f = h$
\end{itemize}

Let us now consider an arbitrary total computable function $f$ in two arguments. We then define partial computable function $g(i)$ so that it returns:
\begin{itemize}
\item $0$ if $f(i,i)=0$
\item $\uparrow$ otherwise
\end{itemize}

Note that in the above, $\uparrow$ denotes no or undefined return value. This is allowed, as $g$ was meant to be a partial function. Not returning a value in practice amounts to infinite looping or (tail-)recursion, which effectively prevents a function from returning anything.

Since we chose $f$ to be totally computable, then as a consequence $g$ is will also be computable. We could formulate the above as follows in the logic programming language defined so far:

\begin{lstlisting}
f i i => 0
-----------
g i => 0
\end{lstlisting}

As long as \texttt{f i i} is also defined within the same program and always returns a result, then the above is a valid formulation within our language.

Since $g$ is computable, albeit partial, we can give a program computes $g$ and encode it with Gödel's numbering into $e_g$.

Let us now consider what happens if we give $e_g$ \textit{twice} to function $h$. $h(e_g,e_g)$ will return:

\begin{itemize}
\item $f(e_g,e_g)=0 \rightarrow g(e_g)=0 \ \rightarrow h(e_g,e_g)=1$
\item $f(e_g,e_g)=1 \rightarrow g(e_g)=\uparrow \ \rightarrow h(e_g,e_g)=0$
\end{itemize}

The above means that whatever $f$ returns, then $h$ returns something else. This means that $h \neq f$, but since $f$ was an \textit{arbitrary} total computable function, then $h$ is different from it and thus $h$ is different from all total computable functions.

The consequence of this is that $h$ is not computable, that is we cannot encode it within a Turing machine, $\lambda$-calculus, etc.

Not all hope is lost, because even if we cannot give a function $h$ that always answers \texttt{yes} or \texttt{no}, nothing stops us from  building an \textit{approximated} version of it. For example, we might restrict ourselves to solutions such as:
\begin{inparaenum}[\itshape i\upshape)]
\item a partial function, that may sometimes loop forever;
\item a function that can also return no answer, such as \texttt{unknown};
\item a function that gives probabilistic answers;
\item ...
\end{inparaenum}

We will focus on approximation techniques (even though not applied to the halting problem itself but one of its cousins) in later chapters.

\subsection{Some desperate grasping}
A reader with an active imagination might, at this point, be trying to find reasons why this does not really apply to his or her experience with programming. After all, this might just be an isolated incident, and most interesting things hopefully remain easily doable, right?\footnote{Yeah, you wish: this chapter goes from bad to worse, so giving up hope now will save you from depression further down the road. Sorry.}

Unfortunately for us, the mere fact that we cannot solve the halting problem really means that we cannot solve many hardcore problems that would make the life of programmers much easier.\footnote{This is actually the only silver lining: insightful programmers cannot be automated away, and will never be out of a job. Up yours, robotics!}

Some examples of problems that cannot be solved automatically as a consequence of the halting problem are:
\begin{itemize}
\item \textbf{EQUIVALENCE PROBLEM} Given two programs, test to see if they are equivalent.
\item \textbf{SIZE OPTIMIZATION PROBLEM} - Given a program, find the shortest equivalent program.
\item \textbf{GRAMMAR PROBLEM} Given two grammars, find out whether they define the same language.
\item ...
\end{itemize}

All of the problems above cannot be solved, because solving them would make it possible to also solve the halting problem.\footnote{This proof strategy is known as \textit{reduction}, because it reduces one problem to another one, showing that solving the first implies solving the second or vice-versa.}

For example, consider the equivalence problem. Suppose (\textit{ad absurdum}\footnote{A proof based on \textit{reductio ad absurdum} assumes something that we suspect to be false and shows that from this assumption stems a clear contradiction against some previously established fact.}) we have a function \texttt{equivalent} that is capable of reliably testing whether two programs are equivalent on some input. We will now show how we could use this equivalence test to determine whether or not a program \texttt{P} halts on input \texttt{i}. To do so, we construct a new program \texttt{Q} that works precisely like \texttt{P}, but which returns \texttt{1} whenever \texttt{P} would return something. This would be equivalent to simply ignoring the result of \texttt{P}, for example:

\begin{lstlisting}
P => res
---------
Q => 1
\end{lstlisting}

We now construct a new, trivial program \texttt{T} that always returns \texttt{1}:

\begin{lstlisting}
-------
T => 1
\end{lstlisting}

We now use our function \texttt{equivalent} on \texttt{Q} and \texttt{T}. If \texttt{Q} and \texttt{T} are equivalent, then this means that \texttt{Q} halts, therefore \texttt{P} halts as well. Since we know that we cannot solve the halting problem, then our original hypothesis that we have a total, computable \texttt{equivalent} function was absurd.

Similar proofs can be set up for all problems that involve the automated, reliable determination of complex properties of programs. This generalization is known as \textit{Rice's Theorem}, which states that, for any non-trivial property of partial functions, there is no total computable function that decides whether an algorithm computes with that property. Here, non-trivial means that it is a property that holds for some but not all partial computable funcctions.

\textit{Rice's Theorem} is quite a dramatic result, because it ultimately guarantees that no fully reliable algorithm will ever be able to perfectly optimize a program for speed or size, no compiler will be able to verify the correctness of programs, etc.\footnote{This is the permanent guarantee of employment for at least some programmers that was mentioned in a previous footnote. Yay!}

\paragraph{Computers are really finite}
The last, desperate attempt at solving the halting problem could be done by a self-appointed smart person that observes how computers are not really correct implementations of Turing machines. In particular, it could be noticed how memory in a modern computer is actually finite, and thus a modern computer is really a large state machine. A non-terminating program will eventually run out of new memory states to explore, and revert back to an older memory state. Thus by tracking all memory states seen so far by the program we can determine if the program does not halt after \textit{enough steps} to explore all the possible memory states. But \textit{how large of a state machine are we talking about}, and thus how many is \textit{enough steps}? Consider a computer with just one gigabyte, which for practical purpose we will assume to be $10^9$ bits. This means that we would need to perform at most $2^{10^9}$ steps, in order to check all possible bit configurations. Now $2^{10^9}$ is indeed a finite number, if one is into this kind of distinction, but as far as finite numbers go it is sufficiently big to make any concrete implementation impossibly slow. It is safe to assume that the civilization of the creator of the program will likely be dead well before the program finishes running. Whether we choose to use a computer with less memory will still be pointless, since even for a computer with one kilobyte we would need to perform at most $2^{10^5}$ steps, which is still a prohibitively large number. Whoever tries to run such a program will not likely see the result within his or her lifetime.

Of course modern computers have far more memory, and programs are often connected to the Internet. This means that the actual program running is a large distributed program which state is far larger and distributed, further making any ``finite memory'' argument completely unusable in practice.

\subsection{Some equivalences and translations}
Someone might wonder if it is possible to define some formalism that solves the halting problem by simply shifting the perspective. Perhaps there exists some new way to formulate new programming languages that do not suffer from this issue?

The basic formalisms of computing are two well-known theoretical programming languages, which are \textit{Turing machines} on one hand and the $\lambda$\textit{-calculus} on the other. Already during the previous century\footnote{For the perspective of future readers, we are talking about the 1900's.} it was discovered that these languages are \textbf{isomorph}, that is they can both express exactly the same programs, and neither can do something that the other cannot. This shocking result was brought about by the presence of translation routines that allow to translate \textit{any} Turing-machine program in a semantically equivalent\footnote{That is which performs exactly the same computation.} $\lambda$-calculus program, and vice-versa.

Without diving into the details of these formalisms, it is possible to observe that we have not yet found a way to escape the circle. For example, we could notice that in our case, there exists a chain of translations that goes as follows:

\begin{lstlisting}[mathescape=true]
$\lambda$-calculus => F# => our logic language
\end{lstlisting}

Now if we can show that we can implement the $\lambda$-calculus into our logic language, then we have ``closed the circle''. The grammar of the language is made up of three simple terms:
\begin{inparaenum}[\itshape i\upshape)]
\item identifiers (intuitively similar to variables), which are simply strings wrapped by \texttt{\$};
\item lambda terms (intuitively similar to function declarations), which take the form \texttt{\textbackslash \$x -> TERM};
\item applications (intuitively similar to function calls), which take the form \texttt{TERM | TERM}.
\end{inparaenum}

This syntax is defined in our language as:

\begin{lstlisting}
Data [] [] "$" [<<string>>] Priority 10 Type Id
Data [] [] "\" [Id Arrow Term] Priority 9 Type Term
Data [] [Term] "|" [Term] Priority 8 Type Term
\end{lstlisting}

Of course an identifier is also a term, thus:

\begin{lstlisting}
Id is Term
\end{lstlisting}

The arrow is not strictly needed, but it is aesthetically pleasing in that it helps distinguish a function arguments from its body:

\begin{lstlisting}
Data [] [] "->" [] Priority 0 Type Arrow
\end{lstlisting}

Execution\footnote{This is just one of the many possible evaluation strategies for the $\lambda$-calculus, which is perhaps the most intuitive for a programmer with a traditional imperative background.} of a program within the calculus is quite simple: whenever we apply a lambda term (the function being called) to another term (the argument), then the result \textit{is the body of the function called where the function parameter has been replaced by the value of the argument}.

Execution of any term which is not the application of a lambda term to something else simply does nothing:

\begin{lstlisting}
---------
$x => $x

---------------
\$x -> t => \$x -> t

-----------------
$x | u => $x | u
\end{lstlisting}

Nested applications are solved from the right to the left:

\begin{lstlisting}
w => w'
u | v => uv
uv | w' => res
-------------------
(u | v) | w => res
\end{lstlisting}

Finally, applications of lambda terms to other, arbitrary terms instance \textit{substitutions}. Substitutions are of the form \texttt{TERM with \$x as TERM}, indicating that we are replacing variable \texttt{\$x} with some other term:

\begin{lstlisting}
Data [] [Term] "as" [Term] Priority 6 Type Where
Data [] [Term] "with" [Where] Priority 5 Type With
\end{lstlisting}

To evaluate such an application, which is of the form \texttt{\textbackslash\$x -> t | u}, we first evaluate \texttt{u} into \texttt{u'}, and then replace \texttt{\$x} with \texttt{u'} within \texttt{t}:

\begin{lstlisting}
u => u'
t with ($x as u') => t'
-----------------------------------
\$x -> t | u => t'
\end{lstlisting}

If we find the variable we needed to replace with term \texttt{u}, then instead of the variable we return \texttt{u} itself:

\begin{lstlisting}
  x == y
  ---------------------
  $y with $x as u => u
\end{lstlisting}

If we find a different variable than the one we needed to replace, then we return the variable unchanged because no replacement is possible:

\begin{lstlisting}
  x != y
  ----------------------
  $y with $x as u => $y
\end{lstlisting}

If we find a redefinition of the variable we were replacing, then we stop the replacement procedure as the variable we were replacement has been shadowed and a new one becomes active in the new scope created:

\begin{lstlisting}
  x == y
  ----------------------------------
  \$x -> t with $y as u => \$x -> t
\end{lstlisting}

If we find a variable definition for a new variable which is not the one we were replacing, then we proceed with replacement within the body of the function definition:

\begin{lstlisting}
  x != y
  t with $y as u => t'
  -----------------------------
  \$x -> t with $y as u => \$x -> t'
\end{lstlisting}

If we find an application, then we perform the replacement within both terms of the application and return the resulting application:

\begin{lstlisting}
  t with $x as v => t'
  u with $x as v => u'
  --------------------------------
  (t | u) with $x as v => t' | u'
\end{lstlisting}

Consider now the evaluation of a simple term such as:

\begin{lstlisting}
-----------------------------
(\$"x" -> $"x") | $"y" => ?
\end{lstlisting}

The first step of evaluation tries to evaluate term \texttt{\$"y"}, and then replace \texttt{\$"x"} with the result of this evaluation:

\begin{lstlisting}
$"y" => u'
$"x" with $"x" as u' => t'
--------------------------------------
(\$"x" -> $"x") | $"y" => t'
\end{lstlisting}

The evaluation of \texttt{\$"y"} simply returns \texttt{\$"y"} itself, because a variable always evaluates to itself:

\begin{lstlisting}
$"y" => $"y"
$"x" with $"x" as $"y" => t'
-----------------------------
(\$"x" -> $"x") | $"y" => t'
\end{lstlisting}

Evaluation then proceeds to the next premise, which is a replacement of the right variable

\begin{lstlisting}
$"x" == $"x"
t' := $"y"
----------------------------------------
$"x" with $"x" as $"y" => t'
--------------------------------------
(\$"x" -> $"x") | $"y" => t'
\end{lstlisting}

We can now begin the unwinding procedure, which after just one step returns:

\begin{lstlisting}
(\$"x" -> $"x") | $"y" => $"y"
\end{lstlisting}

Which is precisely the answer we expected.

By implementing the $\lambda$-calculus then we have shown how on one hand our language \textit{is not more powerful} than any of the well-known formalisms of programming, but on the other hand we have shown how our language \textit{is not less powerful} than any of the well-known formalisms of programming.

This being exactly as powerful as the $\lambda$-calculus or Turing machines is known as the \textbf{Church-Turing equivalence}, and is the best known way to define the expressive power of programming. The existance and apparent inescapability of the Church-Turing equivalence suggests that there is no preferred encoding which ``unlocks'' impossible problems such as the halting problem, therefore making them computable.

\section{Conclusion}
%One might be tempted to look outside of programming
%Maybe mathematical encodings offer a solution?

%Apparently we have the same issue
%Exemplified by Russel's paradox
%$R = \{ x . x \notin x \}$
%\begin{itemize}
%\item $R \in R$?
%\item $R \notin R$?
%\end{itemize}

%A devastating generalization of Russel's paradox
%First incompleteness theorem:
%\begin{itemize}
%\item no consistent system of axioms whose theorems can be listed by an ``effective procedure''
%\item capable of proving all truths about the relations of the natural numbers
%\item For any such system, there will always be statements about the natural numbers that are true, but unprovable
%\end{itemize}
%Proven by constructing self-referencing objects

%the direct relationship between computer programs and mathematical proofs
%observation that two families of formalisms that had seemed unrelated are in fact structurally the same kind of objects
%\textit{a proof is a program, the formula it proves is a type for the program}
%\textit{running a program is equivalent to ``simplifying'' a theorem}

%\includegraphics[width=11cm]{pics/curryhoward.png}

What we have seen so far leaves us with a new perspective on computation. No matter what language we choose, there seems to be a hard limit on what is possible to compute. This means that:

\begin{inparaenum}[\itshape i\upshape)]
\item programming languages all represent, at their core, the same set of objects (the set of all programs);
\item some problems cannot be solved with an automated solution, independently of the programming language;
\end{inparaenum}

Why is the search for new programming languages interesting thus, if they allow us no more expressive power? In reality, programming languages have more or less expressive power, but not because of what they can compute, but because they allow a clearer interaction with the \textit{humans} that use them. As A. Whitehead\footnote{A famous mathematician for as far as fame goes for mathematicians.} said in ``The Importance of Good Notation'', in his book \textit{An Introduction to Mathematics}:

\begin{displayquote}
[...] by the aid of symbolism, we can make transitions in reasoning almost mechanically by the eye, which otherwise would call into play the higher faculties of the brain.

[...]

One very important property for symbolism to possess is that it should be concise, so as to be visible at one glance of the eye and to be rapidly written.

[...]

It is interesting to note how important for the development of science a modest-looking symbol may be. It may stand for the emphatic presentation of an idea, often a very subtle idea, and by its existence make it easy to exhibit the relation of this idea to all the complex trains of ideas in which it occurs. For example, take the most modest of all symbols, namely, 0, which stands for the number zero. The Roman notation for numbers had no symbol for zero, and probably most mathematicians of the ancient world would have been horribly puzzled by the idea of the number zero. For, after all, it is a very subtle idea, not at all obvious. A great deal of discussion on the meaning of the zero of quantity will be found in philosophic works. Zero is not, in real truth, more difficult or subtle in idea than the other cardinal numbers.
\end{displayquote}

Thus programming languages make it easier to express some aspects of thoughts rather than some others, and by emphasizing concepts such as correctness or reliability we can dramatically change the impact of the language on the thought process of the programmer.

About the unsolvable problem, also here is some silver present. Even though we cannot provide a perfect solution to these problems, nothing forbids us from building partial solutions that allow for uncertainty. We can thus always build a program that can answer questions about termination, correctness, or equivalence between programs with answers taken from the set \texttt{yes}, \texttt{no}, and \texttt{unknown}. The ability to return \texttt{unknown} suddenly makes the program possible to build, therefore exchanging some precision with implementability.

The challenge that arises from this is thus that of reducing the number of programs for which our analyser returns \texttt{unknown} so that it is as small as possible. Not only do we wish to reduce the number of uncertain programs, we might also find it acceptable to have our analysis ``give up'' for programs where the flow of control is complex or hard to follow, in the assumptions that these programs, even when working, are not acceptable for being too confusing\footnote{If a program is confusing for a carefully built analyser, imagine what it does to the brain of Bob, a 20-something junior programmer who just came out of a coding bootcamp and sits two cubicles over.}. Theoretical frameworks such as dependent types, model checking, and abstract interpretation (just to name a few) are written with the goal in mind of standardizing the concepts of approximated analysis of computer programs, precisely with the goal of providing an incomplete, but still useful, solution to problems such as halting.


\subsection{??}

Calculating choices: Restricted choices

But what about where there is not clear maximum, like:

\(f=ln(x)+ln(y)\)

Here the agent would always prefer more of \(x\) and \(y\). In practice agents are often limited in their choices by budget constraints. That is, they cannot choose all combinations of inputs.

Here we can use a Lagrangian. This maximises the value of a function subject to constraints on inputs. This may not always be appropriate. The budget constraint for an agent is often an inequality, for example consumption is less than or equal to income, but the Lagrangian takes this to be binding.

Fortunately, this can be resolved. The value of \(\lambda \) in the Lagrangian corresponds to the marginal effect of weakening the constraint. This is positive where the constraint is binding on the agent, but not positive if it is not. Therefore if we find the constraint is not binding, we can remove it from the optimisation.

Under some conditions, constraints will always be binding. These are useful for specific cases of agents later.

In order for the constraint to be binding we make an additional assumption:

\subsubsection{Condition 1: Non-satiation}

The marginal utility of a good is always positive.

Note that we can “do economics” without this, but we want rely on Lagrangians.

\subsubsection{Condition 2: Decreasing marginal utility}

This ensures that we do not get corner solutions, for example consuming all apples.

These two assumptions allow the use of the Lagrangian.

We know that for the Lagrangian the following is true:

\(\dfrac{\dfrac{\delta f}{\delta x}}{\dfrac{\delta g}{\delta x}}=\dfrac{\dfrac{\delta f}{\delta y}}{\dfrac{\delta g}{\delta y}}\)

\(\dfrac{\dfrac{\delta f}{\delta x}}{\dfrac{\delta f}{\delta y}}=\dfrac{\dfrac{\delta g}{\delta x}}{\dfrac{\delta g}{\delta y}}\)

Where \(f\) is the utility function and \(g\) is the budget constraint.