\section{Two-Moment Model}\label{se:Two-MomentModel}

\subsection{Transport Equations}
\modified{We consider neutrino transport through a static background in a non-relativistic framework.
For simplicity, we only include neutrino--matter interactions due to emission, absorption, and isoenergetic scattering.
It is possible to include other interactions, such as neutrino--electron scattering and pair processes, in the two-moment model.
However, these interactions result in dense coupling across neutrino energies and across neutrino species, and we have not yet considered the realizability-preserving property of our scheme in this case.
We leave this for future work.}

After scaling to dimensionless units, the Boltzmann equation can be written as
\begin{equation}
  \pd{f}{t}+\vect{\ell}\cdot\nabla f
  =\f{1}{\tau}\,\cC(f),
  \label{eq:boltzmann}
\end{equation}
where the distribution function $f = f(\omega,\varepsilon,\vect{x},t)$ gives the number of neutrinos propagating in the direction $\omega\in\bbS^{2}$, with energy $\varepsilon\in\bbR^{+}$, at position $\vect{x}\in\bbR^{3}$ and time $t\in\bbR^{+}$.
$\vect{\ell}(\omega)\in\bbR^{3}$ is the unit vector parallel to the neutrino three-momentum: $\vect{p}=\varepsilon\,\vect{\ell}$.
On the right-hand side, $\tau$ is a collision time scale.
In opaque regions, where neutrinos have frequent interactions with the background, $\tau\ll1$.
In transparent regions, where neutrinos rarely interact and stream freely, $\tau\gg1$.
The collision term, $\cC(f)$, which models emission, absorption, and isoenergetic scattering is given by
\begin{equation}
  \cC(f)=\xi\,\big(\,f_{0}-f\,\big)
  +(1-\xi)\,\big(\,\f{1}{4\pi}\int_{\bbS^{2}}f\,d\omega-f\,\big),
  \label{eq:collisionTerm}
\end{equation}
where $\xi = \sigma_{\Ab} / (\sigma_{\Ab}  + \sigma_{\Scatt} )$ is the ratio of the absorption opacity $\sigma_{\Ab}$ to the total opacity.
The scattering opacity is $\sigma_{\Scatt}$.
The limit $\xi = 1$, when $\sigma_{\Scatt} = 0$, corresponds to pure absorption, while $\xi = 0$, when $\sigma_{\Ab} = 0$, corresponds to pure scattering.
The equilibrium distribution function for neutrinos is given by the Fermi-Dirac distribution:
\begin{equation}
  f_{0}(\vect{z})=\f{1}{e^{(\varepsilon-\mu(\vect{x}))/T(\vect{x})}+1},
  \label{eq:fermiDirac}
\end{equation}
where $\vect{z}:=\{\varepsilon,\vect{x}\}$, $T$ is the material temperature in energy units and $\mu$ is the neutrino chemical potential.
Both $T$ and $\mu$ depend on the spatial position $\vect{x}$.

\subsection{Two-Moment Model}

Approximate solutions to the Boltzmann equation, Eq.~\eqref{eq:boltzmann}, can be found by solving the two-moment model.
To this end, define the angular moments of the distribution function as follows:
\begin{equation}
  \big\{\,\cJ,\vect{\cH},\vect{\cK}\,\big\}(\vect{z},t)
  =\f{1}{4\pi}\int_{\bbS^{2}}f(\omega,\vect{z},t)\,\{\,1,\vect{\ell},\vect{\ell}\otimes\vect{\ell}\,\}\,d\omega.
  \label{eq:angularMoments}
\end{equation}
The zeroth moment, $\cJ$, is referred to as the particle density.
The first moment, $\bcH$, is the particle flux, and the second moment, $\bcK$, is proportional to the stress tensor.
By integrating Eq.~\eqref{eq:boltzmann} over the momentum-space angular dimension we obtain equations for the zeroth and the first moments:
\begin{equation}
  \pd{\vect{\cM}}{t}+\nabla\cdot\vect{\cF}=\f{1}{\tau}\,\vect{\cC}(\vect{\cM}),
  \label{eq:momentEquations}
\end{equation}
with $\vect{\cM}=(\cJ,\vect{\cH})^{T}$, $\vect{\cF}=(\vect{\cH},\vect{\cK})^{T}$, and
\begin{equation}
  \vect{\cC}(\vect{\cM})=\vect{\eta}-\vect{\cD}\,\vect{\cM},
  \label{eq:collisionTermMoments}
\end{equation}
where $\vect{\eta}=(\xi\,f_{0},\vect{0})^{T}$, $\vect{\cD}=\mbox{diag}(\xi,\vect{I})$, and
$\vect{I}$ is the identity matrix.
Hence, the process of solving the Boltzmann equation, Eq.~\eqref{eq:boltzmann}, for the neutrino distribution function $f(\omega,\vect{z},t)$, is replaced by solving the two-moment equations for the neutrino number density, $\cJ(\vect{z},t)$, and flux, $\bcH(\vect{z},t)$.

\subsection{Algebraic Closures}

The moment equation for $\bcH$ involves the higher moment $\bcK$ and the two-moment model is open.
To close the two-moment model, we consider algebraic closures.
For the two-moment model, algebraic closures give an approximation to $\bcK$ using the lower moments:
\begin{equation}
  \bcK = \vect{k} \cJ,
\end{equation}
where $\vect{k}$ is the Eddington tensor.
By assuming that the distribution function is symmetric about a preferred direction $\widehat{\vect{h}}=\vect{\cH}/|\vect{\cH}|$, Levermore~\cite{levermore_1984} proposed a simple form for the Eddington tensor:
\begin{equation}
  \vect{k}=\f{1}{2}\big[\,\big(1-\chi\big)\,\vect{I}+\big(3\,\chi-1\big)\,\widehat{\vect{h}}\otimes\widehat{\vect{h}}\,\big],
  \label{eq:eddingtonTensor}
\end{equation}
where $\chi=\chi(\cJ,|\vect{\cH}|)$ is the Eddington factor.
Thus, the two-moment model is closed by specifying the scalar $\chi$ in terms of $\cJ$ and $|\vect{\cH}|$.

\subsection{Constraints on the Moments}

Neutrinos are fermions and obey the Pauli exclusion principle.
Because of this, the neutrino distribution function is bounded; i.e.~$f \in [0,1]$.
As a result, the angular moments $\cJ$ and $\vect{\cH}$ and the Eddington factor $\chi$ satisfy the following bounds~\cite{levermore_1984,lareckiBanach_2011,kershaw_1976,shohatTamarkin_1943}:
\begin{align}
\cJ \in[0,1], \quad &(1-\cJ)\cJ-|\vect{\cH}| \geq 0, \label{eq:MomentsBounds} \\
  \chi_{\mbox{\tiny min}}
  =\max\big(1-\f{2}{3\cJ},h^{2}\big)
  \leq & \chi \leq \min\big(1,\f{1}{3\cJ}-\f{\cJ}{1-\cJ}h^{2}\big)=\chi_{\mbox{\tiny max}},
  \label{eq:eddingtonFactorBounds}
\end{align}
where $h = |\bcH|/\cJ$ is the flux factor.
\modified{The inequalities in Eqs.~\eqref{eq:MomentsBounds} and \eqref{eq:eddingtonFactorBounds} hold for moments as defined in Eq.~\eqref{eq:angularMoments} with $f\in[0,1]$.
In a relativistic model (e.g., \cite{shibata_etal_2011,cardall_etal_2013}), the evolved quantities --- functions of the moments in Eq.~\eqref{eq:angularMoments} --- satisfy different bounds.
We are currently exploring extensions of the realizability-preserving scheme presented here to the relativistic case.}

The constraints in Eq.~\eqref{eq:MomentsBounds} define realizable moments $\bcM$.
For fermions, realizable moments can only be constructed from a distribution satisfying the bounds $f \in [0,1]$.
Moreover, the set of realizable moments is convex: let $\cR$ be the realizability set and $\bcM_{1}, \bcM_{2} \in \cR$, then $\lambda \bcM_{1} + (1-\lambda)\bcM_{2} \in \cR$ for any $\lambda \in [0,1]$~\cite{chu_etal_2018}.
As we will see later in Section~\ref{se:SpatialDiscretization}, this convexity makes it possible to design a realizability-preserving discretization for solving the two-moment model numerically.

The inequalities in Eq.~\eqref{eq:eddingtonFactorBounds} deserve further attention.
They are as important as those in Eq.~\eqref{eq:MomentsBounds}, in maintaining consistency of the two-moment model with respect to Fermi-Dirac statistics.
When designing a numerical scheme for the two-moment model that maintains realizable $\bcM$, it is also necessary for the Eddington factor to satisfy the bounds in Eq.~\eqref{eq:eddingtonFactorBounds}.

However, recently reported CCSN simulations using two-moment neutrino transport with algebraic closures have employed Eddington factors that can violate the bounds in Eq.~\eqref{eq:eddingtonFactorBounds}.
As examples, we consider the Eddington factors discussed in~\cite{murchikova_etal_2017}, where the suitability of several algebraic closures for two-moment neutrino transport was evaluated.
In Fig.~\ref{fig:EddingtonFactorsWithDifferentClosure}, we plot the Eddington factor versus the flux factor for two occupancies: $\cJ = 0.1$ (low occupancy) and $\cJ = 0.9$ (high occupancy).
Of the algebraic closures plotted, few satisfy the bounds on the Eddington factor in Eq.~\eqref{eq:eddingtonFactorBounds}.
Kershaw~\cite{kershaw_1976}, Wilson~\cite{wilson_1975,leblancWilson_1970}, Levermore~\cite{levermore_1984}, Minerbo~\cite{minerbo_1978}, and Janka~2~\cite{janka_1992} closures may work fine when the occupancy is low.
When the occupancy is high, the Eddington factor due to these closures exceeds the upper bound for Fermi-Dirac statistics.
The Eddington factor of Janka~1~\cite{janka_1991} may violate both the upper and lower bound on the Eddington factor.
Only the closure due to Cernohorsky~\&~Bludman~\cite{cernohorskyBludman_1994} satisfies both the upper and lower bounds.
This is not surprising as this is the only closure based on Fermi-Dirac statistics.
Although the Levermore and Minerbo closures do not satisfy the bounds in Eq.~\eqref{eq:eddingtonFactorBounds}, they are widely used in simulations of neutrino transport in CCSNe and compact binary mergers; e.g., O'Connor \& Couch~\cite{oConnorCouch_2018}, Pan et al.~\cite{pan_etal_2018}, Glas et al.~\cite{glas_etal_2018}, Just et al.~\cite{just_etal_2018}, and Foucart et al.~\cite{foucart_etal_2015} use the Minerbo closure, while Vartanyan et al.~\cite{vartanyan_etal_2018}, Cabezon et al.~\cite{cabezon_etal_2018}, Kuroda et al.~\cite{kuroda_etal_2016}, and Fujibayashi et al.~\cite{fujibayashi_etal_2017} use the Levermore closure.
When employing these closures in conditions of high occupancy, a numerical scheme may evolve the moments outside the realizable domain of Fermi-Dirac statistics given in Eq.~\eqref{eq:MomentsBounds}.
If this were to happen, the update step may give $\cJ>1$.
Considering the fact that the collision term contains blocking factors --- i.e.~$(1-\cJ)\times$ something positive --- $\cJ > 1$ would change the blocking factors' sign, and it would be difficult to predict the impact of the subsequent induced errors on the simulation outcome.
Besides, even with the simplified collision term Eq~\ref{eq:collisionTermMoments}, $\cJ>1$ could result in information loss (such as loss to the neutrino distribution for the groups that exceed the bounds), closure failure, and other unpredictable consequences.
If treatments are developed to map the unrealizable moments into the realizable domain, they should be developed to conserve lepton number, energy, and momentum.

\begin{figure}[h]
  \centering
  \begin{tabular}{cc}
    \includegraphics[width=0.5\textwidth]{figures/Closures0_10}
    \includegraphics[width=0.5\textwidth]{figures/Closures0_90}
  \end{tabular}
   \caption{Plot of Eddington factors $\chi$ versus flux factor $h$ for different values of $\cJ$ for various algebraic closures: $\cJ=0.1$ (left panel, low occupancy) and $\cJ=0.9$ (right panel, high occupancy).  In each panel we plot the Eddington factors of two-moment closures due to Kershaw (red), Wilson (yellow), Levermore (green), Minerbo (light blue), Cernohorsky \& Bludman (blue), Janka~1 (purple), and Janka~2 (pink) .  We also plot $\chi_{\mbox{\tiny min}}$ and $\chi_{\mbox{\tiny max}}$, defined in Eq.~\eqref{eq:eddingtonFactorBounds} (lower and upper dashed black lines, respectively).}
  \label{fig:EddingtonFactorsWithDifferentClosure}
\end{figure}


\documentclass{article}

\include{stddefs}
\include{imodefs}

\chapterno{4}

\begin{document}

\chapter{What is optimization?}\label{whatisopt}

In this chapter we will denote the set of column vectors with $d$ rows by $\RR^d$.
The arithmetic
of $d\times 1$ matrices apply i.e., we may add vectors in $\RR^d$ and
multiply them by a number in $\RR$.

In the next chapter we will introduce them as \emph{euclidean} vector spaces. The
term euclidean refers to a norm: a function measuring the size of a vector. In this
chapter we only need the structure as column vectors.

\section{What is an optimization problem?}

An optimization problem consists of maximizing or minimizing a function subject
to constraints.

Below are two classical examples related to minimizing (non-linear) functions subject to
(non-linear) constraints. These are actually examples of
convex optimization problems. More about that later.

\begin{example}

  A cylindrical can is supposed to have a volume of $V$ $\text{m}^3$. The material used
  for the top and bottom costs $T$ DKK per $\text{m}^2$ and the material used for
  the side costs $S$ DKK per $\text{m}^2$. Give the dimensions $r$ and $h$ of the
  can minimizing the price of the materials.

\includegraphics{cylinder.svg}

The cost of the top and bottom pieces are $2 \pi r^2 T$. The cost of the
side material is $2\pi r h S$. The constraint is that the volume must be
$V$. This is expressed in the equation $\pi r^2 h = V$. All in all
the optimization problem is
\begin{align*}
  &\text{Minimize} &2\pi r^2 T + 2\pi r h S&\\
  &\text{with constraints}\\
  &&\pi r^2 h &= V\\
  &&r &\geq 0\\
  &&h &\geq 0,
\end{align*}
where $V, T$ and $S$ are constants.

\beginshex
Can you see a way of solving this optimization problem by eliminating $h$ in
the constraint $\pi r^2 h = V$?

\begin{hideinbutton}{Hint}
$$
\pi r^2 h = V\iff h = \frac{V}{\pi r^2}
$$
and $h$ can be inserted in $2\pi r^2 T + 2 \pi r h S$. Why is this helpful?
\end{hideinbutton}
\endshex
\end{example}

\begin{example}
  A person is in distress $D$ meters from the beach. The life guard
  spots the situation, but is $d$ meters from where he would naturally jump
  in the water as indicated below. The life guard runs $8$ m/s on the
  beach and swims $1$ m/s in the water. How far ($x$) should he run along the
  beach before jumping into the water in order to minimize the time needed
  to reach the person in distress?

  \includegraphics{lifeguard.png}

  The time spent moving with a speed of $v$ over a distance of $s$ is
  $$
  t = \frac{s}{v}.
  $$
  If the life guard jumps in the water at the point $x$ he will have
  to swim a distance of
  $$
  \sqrt{D^2 + (d-x)^2}
  $$
  using the Pythagorean theorem. Therefore the optimization problem becomes
\begin{align*}
  &\text{Minimize} &\frac{x}{8} + \sqrt{D^2 + (d-x)^2}&\\
  &\text{with constraints}\\
  &&x\geq 0\\
\end{align*}
Strictly speaking we do not need the constraint $x\geq 0$, as the life guard is free
to run in the other direction. So the optimization problem is simply to minimize
$$
\frac{x}{8} + \sqrt{D^2 + (d-x)^2}
$$
with no strings attached i.e., $x\in \RR$ is just assumed to be any number.

\begin{sage}
D = 200
d = 100
plot(x/8 + sqrt(D**2 + (d-x)**2), (x, 0, d))
\end{sage}

\end{example}

\beginshex
  You need to build a rectangular fence in front of your house for a herb garden.
  Your house will make up one side of the rectangle, so you only need to build three
  sides. Suppose you have 10 m of wire. What is the maximum area of the herb garden you can
  wall in?
\endshex

\section{General definition}

An optimization problem consists of a subset $D\subseteq \RR^d$ and
a function $f:D\rightarrow \RR$. We will consider optimization problems in the context
of minimization. Optimize in this situation means minimize.

\begin{definition}[emph]\label{defopt}
  In our most general setting an optimization problem looks like
\begin{align*}
  &\text{Minimize} &f(x)&\\
  &\text{with constraint}\\
  &&x\in C,
\end{align*}
     where $C$ and $D$ are subsets of $\RR^d$ with $C\subseteq D$ and $f: D\rightarrow \RR$ is a function.
A solution to the optimization problem is a vector $x_0\in C$, such that
$$
f(x_0) \leq f(x)
$$
for every $x\in C$. Here $x_0$ is called an \emph{optimum} and $f(x_0)$ is called the \emph{optimal value}.

We will often write the optimization problem defined above in short form as
\begin{align*}
  &\text{min}\, f(x)\\
  &x\in C.
\end{align*}
\end{definition}

  The complexity of the problem depends very much on the nature of $C$ and $f$. Also,
  we cannot even be certain that an optimization problem has a solution. Consider
  the problem

  \begin{align*}
    &\text{min}\,x\\
    &x\leq 0
  \end{align*}

  Here $x$ can be made arbitrarily small subject to the constraint $x\leq 0$ and
  the problem has no solution.

  \begin{remark}[emph]
    We have deliberately not included maximization problems in Definition \ref{defopt}. This is
because a maximization problem, such as
\begin{align}\label{maxprob}
  &\text{Maximize} &f(x)&\\
  &\text{with constraint}\\
  &&x\in C
\end{align}

 can be formulated as the minimization problem
\begin{align}
  &\text{Minimize} &-f(x)&\\
  &\text{with constraint}\\
  &&x\in C.
\end{align}

Again, we will use the short notation
\begin{align*}
  &\text{max}\,f(x)\\
  &x\in C
\end{align*}
for the maximization problem in \eqref{maxprob}.
A solution to \eqref{maxprob} is a vector $x_0\in C$, such that
$$
f(x_0) \geq f(x)
$$
for every $x\in C$. Again, $x_0$ is called an optimum and $f(x_0)$ the
optimal value.

\end{remark}

\beginshex
Suppose that the maximization problem
\begin{align}\label{maxprob1}
  &\text{max}\, f(x)\\
  &x\in C
\end{align}
is formulated as
the minimization problem
\begin{align}\label{minmaxprob}
  &\text{min}\, -f(x)\\
  &x\in C.
\end{align}
Show that $-m$ is the optimal value
and $x_0$ the optimum for \eqref{maxprob1} if
$m$ is the optimal value
and $x_0$ the optimum for \eqref{minmaxprob}.
\endshex

    \beginshex\label{mothercopt}
    Suppose that $a > 0$. Solve the optimization problem
\begin{align*}
    &\text{Minimize} &a x^2 + b x + c\\
    &\text{with constraint}\\
    &&x\in \RR.
  \end{align*}

    \endshex

    \section{Convex optimization}

    Particularly well behaved optimization problems are the convex ones. These are optimization
    problems, where $C\subseteq \RR^d$ is a convex subset and $f:C\rightarrow \RR$ a
    convex function in Definition \ref{defopt}. To define these concepts we first introduce
    the notion of
    a line in $\RR^d$.

    \begin{definition}[emph]\label{defline}
      A line $L\subseteq \RR^d$ is a subset of the form
      $$
      L = \{u + t v \mid t\in \RR\},
      $$
      where $u, v\in \RR^d$ with $v\neq 0$.
    \end{definition}

\begin{example}
A line $L$ in the plane $\mathbb{R}^2$ is (usually) given by its equation
\begin{equation}\label{lineeq}
y = a x + b.
\end{equation}
This means that it consists of points $(x, y)\in \mathbb{R}^2$ satisfying $y = a x + b$.
Here $a$ can be interpreted as the slope of the line and $b$ the intersection with the
$y$-axis.

What about all the points $(x, y)$ with $x = 0$? Certainly they also deserve to be called
a line. However, they do not satisfy an equation like \eqref{lineeq}. Informally, this line
has infinite slope.

Therefore we introduce the parametric representation of a line: a line is the
set of points of the form
$$
\begin{pmatrix} x_0 \\ y_0 \end{pmatrix} +
t \begin{pmatrix} u_0 \\ v_0 \end{pmatrix},
$$
where $t\in \mathbb{R}$,
$$
\begin{pmatrix} x_0 \\ y_0\end{pmatrix}
$$
is any point on the line and
$$
\begin{pmatrix} u_0 \\ v_0\end{pmatrix}
$$
is a non-zero (directional) vector.

\begin{figure}
  \includegraphics{line.png}
  \begin{center}
    Example of a line in $\RR^2$ with (directional) vector $v\in \RR^2$ through the point $u\in \RR^2$.
  \end{center}
\end{figure}

\end{example}

\beginshex
Compute the parametric representation of the line through the points $(1, 1)$ and $(2, 3)$.
\endshex

\beginshex
What is the parametric representation of the line with the equation
$y = a x + b$? What is the parametric representation of the
line consisting of the points $(x, y)$ with $x=0$?
\endshex

    \beginshex
    Show in Definition \ref{defline} that if $L$ is given by $u$ and $v$, then
    you might as well replace $v$ by $s v$, where $s$ is a real number and $s\neq 0$.  It gives
    the same line.
    \endshex

\begin{exercise}[emph]
  Show that there is a unique line passing through two distinct points
    $x, y\in \RR^d$ and that it is given by $u = x$ and $v = y - x$ in Definition \ref{defline}.
\end{exercise}

    \beginshex
    Do the points
    $$
    \begin{pmatrix}1\\ 2\\ 3\end{pmatrix}, \quad \begin{pmatrix} 4\\ 5\\ 6\end{pmatrix}
    \quad\text{and}\quad \begin{pmatrix} 7\\ 8\\ 9\end{pmatrix}
    $$
    lie on the same line in $\RR^3$?
    \endshex

    \beginshex
    Show that the line through two distinct points $x, y\in \RR^d$ is equal to the subset
    $$
    \{ (1-t) x + t y \mid t\in \RR\} \subseteq \RR^d.
    $$
    \endshex

    \begin{definition}[emph]\label{Def:convexsubset}
    A convex subset $C\subseteq \RR^d$ is a subset that contains the
    line segment between any two of its points $x, y\in C$ i.e.,
    $$
    (1 - t) x + t y\in C
    $$
    for every number $t$ with $0\leq t \leq 1$.
    \end{definition}

\begin{figure}
  \includegraphics{nonconvex.png}
  \begin{center}
    Example of non-convex subset of $\RR^2$.
  \end{center}
\end{figure}

\begin{quizexercise}[showhide]
  \begin{quiz}
    \question
    Which of the subsets below are convex?
    \answer{T}
    $$C=\{1\}\subseteq \RR$$
    \answer{F}
    $$C=\{1, 2\}\subseteq \RR$$
    \answer{T}
    The points $C$ on a line $y = a x + b$ in $\RR^2$.
    \answer{F}
    $$
    C=\{(x, y)\in \RR^2 \mid x y \geq 0\}.
    $$
  \end{quiz}
\end{quizexercise}

    \beginshex
      A closed interval in $\RR$ is a subset of the form
      $$
      [a, b] = \{x \mid a \leq x \leq b\}
      $$
      for $a \leq b$. Prove that $[a, b]$ is a convex subset of $\RR$.

\begin{hideinbutton}{Hint}

Keep cool and just apply the definitions! First of all, $x\in [a, b]$ if and only if
\begin{equation}\label{intdef}
a\leq x \land x \leq b.
\end{equation}
Now pick any $t\in [0, 1]$. We must show that if $x\in [a, b]$ and $y\in [a, b]$, then
$$
(1 - t)x + t y \in [a, b].
$$
You may also write this out as
$$
a\leq x \land x \leq b\qquad\land\qquad a\leq y \land y \leq b
$$
implies that
$$
a\leq (1 - t)x + t y \qquad \land \qquad (1 - t)x + t y \leq b.
$$

\begin{hideinbutton}{Hint}
$$
a\leq x\implies (1-t) a \leq (1-t) x\qquad\land\qquad a\leq y \implies t a \leq t y
$$
implies that
$$
(1 - t) a + t a \leq (1-t)x + t y.
$$
What is $(1-t)a + a$?
\end{hideinbutton}

\end{hideinbutton}

\endshex

    \beginshex
    Let $A$ and $B$ be convex subsets of $\RR^d$. Prove that $A\cap B$ is a
    convex subset of $\RR^d$. Generalize this to show that if $A_1, \dots, A_n$
    are any number of convex subsets of $\RR^d$, then their intersection
    $$
    A_1 \cap \cdots \cap A_n
    $$
    is a convex subset of $\RR^d$. Is the union of two convex subsets necessarily convex?
    \endshex

    \begin{definition}[emph]\label{Def:convexfunction}
    A convex function is a function $f: C\rightarrow \RR$
    defined on a convex subset $C\subseteq \RR^d$, such that
    $$
    f((1 - t) x + t y) \leq (1-t) f(x) + t f(y)
    $$
    for every number $t$ with $0\leq t \leq 1$.
    \end{definition}

\begin{figure}
  \includegraphics{convexfct.png}
  \begin{center}
    Graph of convex function. The line segment between $(x, f(x))$ and $(y, f(y))$
    lies above the graph.
  \end{center}
\end{figure}

    \beginshex\label{exconvfct}
    Let the function $f: \RR \rightarrow \RR$ be given by
    $f(x) = a x + b$, where $a, b\in \RR$. Show that $f$ is a convex
    function.

    \begin{hideinbutton}{Hint}
      Try the case $a = 0$ first.
    \end{hideinbutton}

    Can you at this point prove that $f(x) = x^2$ is a convex function?

    \begin{hideinbutton}{Hint}
      Simplify
      $$
      (1 - t) x^2 + t y^2 - ((1-t) x + t y)^2
      $$
      to an expression that has to be non-negative.

      \begin{hideinbutton}{Hint}
        \begin{sage}
from sympy.abc import x, y, t
f = (1-t)*x^2 + t*y^2 - ((1-t)*x + t*y)^2
factor(f)
        \end{sage}
    \end{hideinbutton}
  \end{hideinbutton}

  Using that $f(x) = x^2$ is a a convex function, prove that $g(x) = x^4$ is a convex function and in general, that $h(x) = x^m$ is a convex function, where
  $m = 2n$ is an even natural number.

\begin{hideinbutton}{Hint}
      $$g(x) = f(x)^2$$
    \end{hideinbutton}

    It is a fact that $f(x) = x^3$ is not a convex function, but can
    you explain this using the definition of a convex function?

    \begin{hideinbutton}{Hint}
      Try $x = -1, y = 0$ and $t =\frac{1}{2}$.
    \end{hideinbutton}
    \endshex

    \begin{lemma}[emph]
    Let $f: \RR^d\rightarrow \RR$ be a convex function. Then the subset
    $$
    C = \{x\in \RR^d \mid f(x) \leq a\}
    $$
    is a convex subset of $\RR^d$, where $a\in \RR$.
    \end{lemma}
    \begin{proof}
    Suppose that $u, v\in C$ and $t\in [0, 1]$. Looking at Definition \ref{Def:convexsubset}
we must prove that
$$
(1-t) u + t v \in C.
$$
By the definition of $f$ being convex (Definition \ref{Def:convexfunction}), it follows that
$$
f((1-t) u + t v) \leq (1- t) f(u) + t f(v).
$$
But, since $f(u)\leq a$ and $f(v)\leq a$ we have
\begin{align*}
(1- t) f(u) &\leq (1-t) a\\
t f(v) &\leq t a
\end{align*}
and therefore
$$
(1- t) f(u) + t f(v) \leq (1-t) a + t a = a.
$$
Therefore,
$$
f((1-t) u + t v)\leq a
$$
and $(1-t) u + t v\in C$.
\end{proof}

    We do not have the tools yet to prove the crucial
    result about convex optimization problems, but at least we can state it.

    \begin{remark}[emph]
    In hunting for optimal solutions to an optimization problem one is often stuck with a
    point $x_0\in \RR^d$, which is optimal locally. This means that $f(x_0)\leq f(x)$ for
    every $x$ that is sufficiently close to $x_0$ (we will explain what this means in the next
    chapter). The remarkable thing that happens in a convex optimization problem is
    that if $x_0$ is optimal locally, then it is a global optimum! It satisfies
    $f(x_0)\leq f(x)$ not only for $x$ close to $x_0$, but for every $x\in C$.
\end{remark}

    The optimization problem in Exercise \ref{mothercopt} is a very typical convex optimization
    problem.

    Below you see a plot of the function (press Compute)
    $$
    f(x) = x^3 + 2 x^2 + x + 1
    $$
    restricted
    to the interval $[-1.5, 0]$. You can see that it has a local minimum around $-0.3$ and
    also that this minimum is not a global minimum (certainly $f(-1.4)$ is smaller). So $f(x)$
    is not a convex function on this interval (but if you look at it more locally on the
    interval $[-0.6, 0]$ it is a convex function.

    \begin{sage}
      plot(x**3+2*x**2+x+1, (x, -1.5, 0))
    \end{sage}

    \beginshex
    Solve the optimization problem
    \begin{align*}
    &\text{Minimize} &x^3 + 2 x^2 + x + 1\\
    &\text{with constraint}\\
      &&x\in C
    \end{align*}
    for $C = [-0.6, 0]$ and $C = [-2, 0]$.
    \endshex

  \section{Linear optimization}

  The simplest convex optimization problems are the linear ones. Recall that a
  linear function $f: \RR^d\rightarrow \RR$ has the form
  $$
  f\begin{pmatrix} x_1 \\ \vdots \\ x_n \end{pmatrix} =
  c_1 x_1 + \cdots + c_n x_n
  $$
  for $c_1, \dots, c_n\in \RR$. Usually we write this with matrix notation as
  $$
  f(x) = c^T x,
  $$
  where
  $$
  c =
  \begin{pmatrix} c_1 \\ \vdots \\ c_n \end{pmatrix}\qquad \text{and} \qquad
  x = \begin{pmatrix} x_1 \\ \vdots \\ x_n \end{pmatrix}.
  $$

  \beginshex
  Show that a linear function is convex.
  \endshex

  A linear optimization problem is not about minimizing a linear function over
  an arbitrary convex subset. We choose the convex subset as an intersection of
  subsets of the form
  $$
  \{x\in \RR^d\mid a^T x \leq b\},
  $$
  where $a\in \RR^d$ is a non-zero vector and $b\in \RR$ a number i.e.,
  a linear optimization problem has the form
  \begin{align*}
    &\text{min}\,c^T x\\
    &x\in C,
  \end{align*}
  where
  \begin{align}\label{linsubs}
    C &= \{x\in \RR^d \mid a_1^T x \leq b_1, \dots, a_m^T x\leq b_m\}\\
    &= \{x\in \RR^d \mid a_1^T x \leq b_1\} \cap \cdots \cap \{x\in \RR^d \mid a_m^T x \leq b_m\}
  \end{align}
  and $c, a_1, \dots, a_m\in \RR^d$ and $b_1, \dots, b_m\in \RR$.

  \beginshex
  Use a selection of previous exercises to show that
  the subset $C$ defined in \eqref{linsubs} is a convex subset of $\RR^d$.
  \endshex

  Using matrix notation we write $C$ as
  $$
  C = \{x\in \RR^d \mid A x \leq b\},
  $$
  where $A$ is the $m\times d$ matrix with row vectors $a_1^T, \dots, a_m^T$ and
  $$
  b =
  \begin{pmatrix}
    b_1 \\ \vdots \\ b_m
  \end{pmatrix}.
  $$

  \begin{example}\label{exlinopt}

     Here is a concrete example for $d = 2$. The optimization problem
  \begin{align*}
    &\text{Maximize} &x + y&\\
    &\text{with constraints}\\
    &&2 x + y &\leq 1\\
    &&x + 2 y &\leq 1\\
    &&x &\geq 0\\
    &&y &\geq 0
  \end{align*}
  translates into matrix notation with the matrices
  \newcommand{\mi}{\hphantom{-}}
  $$
  c = \begin{pmatrix} 1 \\ 1 \end{pmatrix}, \qquad
  A = \begin{pmatrix} \mi 2 & \mi 1 \\ \mi 1 & \mi 2 \\ -1 & \mi 0 \\ \mi 0 & - 1\end{pmatrix}\qquad \text{and}\qquad
  b = \begin{pmatrix} 1 \\ 1 \\ 0 \\ 0 \end{pmatrix}.
  $$

  In this case it is helpful to draw the optimization problem in the plane $\RR^2$. This is done below.

  \includegraphics{LP.png}
  \begin{center}
    Constraints pictured as shaded area above. Optimum occurs in a vertex (corner).
    \end{center}

  \end{example}

  We will give a general (but rather slow) algorithm below for solving
  linear optimization problems. In fact it all boils down to solving
  systems of linear inequalities. Sometimes linear optimization is
  referred to as \url{linear
    programming}{https://en.wikipedia.org/wiki/Linear_programming}. The
  basic theory of linear programming was pioneered, among others, by
  one of the inventors of the modern computer, John von Neumann.

\includegraphics{vonNeumann.gif}
\begin{center}
  \url{John von Neumann}{https://en.wikipedia.org/wiki/John_von_Neumann} (1903-1957). Picture from \url{LANL}{https://en.wikipedia.org/wiki/Los_Alamos_National_Laboratory}.
\end{center}

  Sage has much more advanced algorithms built in for solving (integer) linear optimization
  problems. I have translated the linear optimization problem in Example \ref{exlinopt} into
  Sage below.

\begin{example}\label{sageLP}
\begin{sage}
LP = MixedIntegerLinearProgram(solver = "GLPK")

# generate variables (do not require integer solution)
x = LP.new_variable(integer=False)

# Linear function to be maximized (notice maximized!)

LP.set_objective(x[1] + x[2])

# Add constraints
LP.add_constraint( 2*x[1] + x[2] <= 1 );
LP.add_constraint( x[1] + 2*x[2] <= 1 );
LP.add_constraint( x[1] >= 0 );
LP.add_constraint( x[2] >= 0 );

LP.show()

oval = LP.solve()
xopt = LP.get_values(x);

print('Optimal Value:', oval)
print('occurs in')
for i, v in xopt.items():
    print('x_%s = %f' % (i, v))
\end{sage}
\end{example}

\section{Fourier-Motzkin elimination}

Fourier-Motzkin elimination is a classical method (dating back to 1826) for solving linear inequalities.
It is also a key ingredient in an algorithm for solving linear optimization problems.

I am convinced that
the best way to explain this method is by way of an extended example. For more formalities you may
consult \url{Chapter 1}{https://www.worldscientific.com/doi/suppl/10.1142/8527/suppl_file/8527_chap01.pdf} of my book \url{Undergraduate Convexity}{https://www.worldscientific.com/worldscibooks/10.1142/8527}.

Consider the linear optimization problem

\begin{equation}\label{optfmex}
  \begin{array}{llrrrr}
    \text{Maximize} &x + y\\
    \text{with constraints}\\
    &2 x &+ &y &\leq &6\\
    &x &+ &2 y &\leq &6\\
    &x &+ &2 y &\geq &2\\
    &x && &\geq &0\\
    &&&y &\geq &0.
  \end{array}
\end{equation}

We might as well write this as

$$
  \begin{array}{llrrrl}
    \text{Maximize} &z\\
    \text{with constraints}\\
    &&& z &= &x + y\\
    &2 x &+ &y &\leq &6\\
    &x &+ &2 y &\leq &6\\
    &x &+ &2 y &\geq &2\\
    &x &&&\geq &0\\
    &&&y &\geq &0
  \end{array}
$$

  by adding the extra variable $z$. This enables us to reformulate the problem as follows: Find
  the maximal value of $z$, such that there exists $(x, y)\in \RR^2$ with
  $$
  (x, y, z)\in P,
  $$
  where $P\subseteq \RR^3$ is the set of solutions to the system

\begin{equation}\label{Pset}

  \begin{array}{llrrrl}
    &&& z &= &x + y\\
    &2 x &+ &y &\leq &6\\
    &x &+ &2 y &\leq &6\\
    &x &+ &2 y &\geq &2\\
    &x &&&\geq &0\\
    &&&y &\geq &0
  \end{array}

\end{equation}

  of \footnote{inequalities}{An equality $a = b$ is logically equivalent to the two
    inequalities $a\leq b$ and $a \geq b$ in the sense that $(a\leq b) \wedge (a\geq b) \iff a = b$.}.

  We have the Gauss elimination method for solving systems of linear equations. How do we now solve
  \eqref{Pset}, where we also have inequalities?

  Well, at first we can actually do a Gauss elimination step by eliminating $x$ in the equation
  $z = x + y$ i.e., by putting $x = z - y$. This is then inserted into the inequalities in
  \eqref{Pset}:

$$
\begin{array}{llrrrl}
    &2 (z - y) &+ &y &\leq &6\\
    &(z - y) &+ &2 y &\leq &6\\
    &(z - y) &+ &2 y &\geq &2\\
    &(z - y) &  &    &\geq &0\\
    &          &  &  y &\geq &0
\end{array}
$$

and we get the system

$$
  \begin{array}{llrrrl}
    &2 z &- &y&\leq &6\\
    &z &+ &y &\leq &6\\
    &z &+ & y &\geq &2\\
    &z &- &y &\geq &0\\
    &  &  &y   &\geq &0
  \end{array}
$$
  of inequalities
  in the variables $z$ and $y$. Now we only have inequalities left and we have to invent a trick for
  eliminating $y$. Let us isolate $y$ on one side of the inequality signs $\leq$ and $\geq$:

  $$
    \begin{array}{llrrrl}
    &2 z &- &6 &\leq &y\\
    &6 &- &z &\geq &y\\
    &2 &- &z &\leq &y\\
    & & &z &\geq &y\\
    &&&0 &\leq &y
  \end{array}
$$

  Written a little differently this is the same as

\begin{equation}\label{fmex}
  \begin{array}{llllllllll}
    &2 z &- &6&\leq &{\color{red} y} \\
    &&&&&{\color{red} y} &\leq &6 &- &z\\
    &2&- &z&\leq &{\color{red} y} \\
    &&&&&{\color{red} y} &\leq &z\\
    &&&0&\leq &{\color{red} y}
  \end{array}
\end{equation}

Now the scene is set for elimination of $y$. Listen carefully. First the inequalities in
\eqref{fmex} can be boiled down to the following two inequalities

\begin{equation}\label{fmex1}
  \begin{array}{lllll}
\max(2 z - 6, 2 - z, 0) &\leq &{\color{red} y}\\
&& {\color{red} y} & \leq &\min(6 - z, z)
  \end{array}
\end{equation}
by using (repeatedly) that  $\max(a, b) \leq c \iff a\leq c \wedge b\leq c$ and
  $c\leq \min(a, b) \iff c\leq a \wedge c\leq b$ for three
numbers $a, b, c\in \RR$.

Then, finally comes the (Fourier-Motzkin) elimination step: The existence of a solution to \eqref{fmex1} is
equivalent to the single inequality

\begin{equation}\label{singleineq}
\max(2 z - 6, 2 - z, 0) \leq \min(6 - z, z).
\end{equation}

This single inequality can be exploded or expanded (see Exercise \ref{explex}) into the following $6 = 3\cdot 2$ inequalities

\begin{align*}
2 z - 6 &\leq 6 - z\\
2 z - 6 &\leq z\\
2 - z &\leq 6 - z\\
2 - z &\leq z\\
0 &\leq 6 - z\\
0 &\leq z.
\end{align*}

Similarly to \eqref{fmex} we now isolate $z$ from the above inequalities:

\begin{equation*}
  \begin{array}{lllll}
&&{\color{red} z} &\leq &4\\
&&{\color{red} z} &\leq &6\\
1&\leq &{\color{red} z}\\
&&{\color{red} z} &\leq &6\\
0&\leq &{\color{red} z}
\end{array}
\end{equation*}

and find that

\begin{equation*}
  \begin{array}{lllll}
\max(1, 0) = 1 &\leq &{\color{red} z}\\
&& {\color{red} z} & \leq &\min(4, 6) = 4.
  \end{array}
\end{equation*}

Therefore the maximum in the optimization problem \eqref{optfmex} is $z = x + y = 4$.
How do we now find numbers $x, y\in \RR$ satisfying the constraints
in the optimization problem \eqref{optfmex} with $z = x + y = 4$?

This is simply done inserting first $z = 4$ in \eqref{fmex1}. Here you get the two inequalities
$2 \leq y$ and $y\leq 2$. Therefore $y = 2$. Since we had $x = z - y$ from the very beginning
we therefore get $x = 2$ and we have the unique solution to the optimization problem.

\beginshex
What is the solution if we replace Maximize with Minimize in the optimization problem \eqref{optfmex}?
\endshex

\beginshex\label{explex}

Prove the following:

    Let $x_1, \dots, x_m, y_1, \dots, y_n\in \RR$ be $m + n$ numbers. Then
    $$
    \max(x_1, \dots, x_m) \leq \min(y_1, \dots, y_n)
    $$
    if and only if the $m n$ inequalities
    $$
    \begin{matrix}
      &x_1 \leq y_1 &x_1 \leq y_2 &\dots &x_1\leq y_n\\
      \\
      &\vdots & \vdots &\ddots &\vdots\\
      \\
      &x_m \leq y_1 &x_m \leq y_2 &\dots &x_m\leq y_n
    \end{matrix}
    $$
    are satisfied.
\endshex

\beginshex

The following is Exercise 1.8 from my book \url{Undergraduate Convexity}{https://www.worldscientific.com/worldscibooks/10.1142/8527}.

A vitamin pill $P$ is produced using two ingredients $M_1$
  and $M_2$. The pill needs to satisfy four constraints for the vital
  vitamins $V_1$ and $V_2$. It must contain at least $6$ milligrams and
  at most $15$ milligrams of $V_1$ and at least $5$ milligrams and at
  most $12$ milligrams of $V_2$. The ingredient $M_1$ contains $3$
  milligrams of $V_1$ and $2$ milligrams of $V_2$ per gram.  The
  ingredient $M_2$ contains $2$ milligrams of $V_1$ and $3$ milligrams
  of $V_2$ per gram:

$$
\def\arraystretch{1.5}
\begin{array}{c|cc}
& V_1 & V_2\\ \hline
M_1 & 3 & 2\\
M_2 & 2 & 3
\end{array}
$$

Let $x$ denote the amount of $M_1$ and $y$ the amount of $M_2$
  (measured in grams) in the production of a vitamin pill. Write down
  a system of linear inequalities in $x$ and $y$ describing the
  constraints above.

  We want a vitamin pill of minimal weight satisfying the
  constraints. How many grams of $M_1$ and $M_2$ should we mix?

Use Fourier-Motzkin elimination to solve this
  problem.

Check your solution by modifying the input to the Sage code in Example \ref{sageLP}.

\endshex

\section{Application in machine learning and data science}\label{sec:mlds}

To start with, consider a toy example of a machine learning problem: we wish
to tell the gender of a person based on a data point consisting of
the height and weight of the person.

To do this we train our model by measuring the height and weight of a
lot of people. Each of these measured data points are labeled
female or male according to the gender of the person.

\includegraphics{sephp.png}

Given a new data point, we wish to tell if the person is female or male. Here we
consider a very simple model for doing this. First we need to introduce some
new mathematical terms. We will introduce the terms generally for
data points in $\RR^d$ and not just in $\RR^2$ as above.

A hyperplane in $\RR^d$ is a generalization of a line $y = a x + b$ in the plane. In general
a hyperplane is defined as the set of points $x\in \RR^d$ satisfying
$$
a^T x = b,
$$
where $a\in \RR^d$ is a non-zero vector and $b$ is a number. A hyperplane divides $\RR^d$
into two subsets: the points above or on the hyperplane satisfying $a^T x \geq b$ and the ones
below the hyperplane satisfying $a^T x < b$.

Suppose we are given a data set as a finite set of points in $\RR^d$ and that each of
these points are labeled with either a blue or a red color. We wish to
find a hyperplane, such that the blue points are above and the red points
are below the hyperplane.

\includegraphics{sephpWithline.png}

We may then use the hyperplane to predict the label of a point. This could be
gender, if you win or lose money buying a stock, anything with
a binary classifier.

  \subsection{Formulation as a linear optimization problem}

  Suppose that the points labeled blue are $x_1, \dots, x_m\in \RR^d$ and
  the points labeled red are $y_1, \dots, y_n\in \RR^d$. Then we wish to
  find $a\in \RR^d$ and $b\in \RR$, such that
  $$
  a^T x_i > b
  $$
  for $i = 1, \dots, m$ and
  $$
  a^T y_j < b
  $$
  for $j = 1, \dots, n$. One can show that these strict inequalities may be solved
  for $a$ and $b$ if and only if the inequalities
  \begin{align*}
  a^T x_i &\geq b + 1\\
  a^T y_j &\leq b - 1
  \end{align*}
  are solvable for $a$ and $b$, where $i = 1, \dots, m$ and $j = 1, \dots, n$.

  It is, however, not realistic to expect data to behave this nicely. Instead one invents
  the rather ingenious linear optimization problem

  \begin{align}\label{linoptnice}
    &\text{Minimize} &\frac{1}{m}(u_1 + \cdots + u_m) + \frac{1}{n}(v_1 + \cdots + v_n)\\
    &\text{with constraints}\\
    &&a^T x_i + u_i&\geq b + 1\\
    &&a^T y_j - v_j &\leq b - 1\\
    &&u_i&\geq 0\\
    &&v_j&\geq 0
  \end{align}

  for $i = 1, \dots, m$ and $j = 1, \dots, n$. This linear optimization problem has optimal value zero
  if and only if data can be separated strictly. Otherwise, it finds a hyperplane minimizing the mean errors
  for the points involved.

  The linear optimization problem \eqref{linoptnice} may look untied to the real world, but it has
  been used very successfully in the diagnosis and prognosis of breast
  cancer. See \url{Mangasarian et al.}{https://www.jstor.org/stable/171686}

  In the sage window below we have implemented the solution of the
  linear optimization problem \eqref{linoptnice}, where the output is
  a graphical illustration of the optimal line, that separates the
  points in \texttt{xpts} and \texttt{ypts} with the smallest mean
  error as defined in the function to be minimized in \eqref{linoptnice}.

\begin{sage}
# Enter blue data points below
xpts = [(3, 1), (4, 2), (4, 3), (3, 5)]
# Enter red data points below
ypts = [(2, 2), (2.5, 4.5), (3, 2), (4, 1)]

LP = MixedIntegerLinearProgram(solver = "GLPK", maximization=False)

# generate variables (do not require integer solution)
a = LP.new_variable(integer=False)
b = LP.new_variable(integer=False)
u = LP.new_variable(integer=False, nonnegative=True)
v = LP.new_variable(integer=False, nonnegative=True)

# Linear function to be minimized

LP.set_objective(float(1)/len(xpts)*sum(u[i] for i in range(1, len(xpts)+1)) + float(1)/len(ypts)*sum(v[i] for i in range(1, len(ypts)+1)))

# Add constraints

for i in range(1, len(xpts)+1):
  LP.add_constraint(xpts[i-1][0]*a[1] + xpts[i-1][1]*a[2] + u[i] >= b[1] + 1)

for j in range(1, len(ypts)+1):
  LP.add_constraint(ypts[j-1][0]*a[1] + ypts[j-1][1]*a[2] - v[j] <= b[1] - 1)

# LP.show()

oval = LP.solve()
aopt = LP.get_values(a);
bopt = LP.get_values(b)

# Compute frame for showing separating line

pts = xpts + ypts
xvals = list(map(lambda x: x[0], pts))
yvals = list(map(lambda y: y[1], pts))

xmin = min(xvals)
xmax = max(xvals)
ymin = min(yvals)
ymax = max(yvals)

# Intersection of separating line with frame (cumbersome!):

if (aopt[2] == 0):
  x =  bopt[1]/aopt[1]
  p1 = (x, ymin)
  p2 = (x, ymax)
else:
  a = -aopt[1]/aopt[2]
  b = bopt[1]/aopt[2]
  y = a*xmin + b
  if (y >= ymin and y <= ymax):
    p1 = (xmin, y)
    x = (ymax - b)/a
    if (x > xmin and x <= xmax):
      p2 = (x, ymax)
    else:
      x = (ymin - b)/a
      if (x >= xmin and x <= xmax):
        p2 = (x, ymin)
      else:
        p2 = (xmax, a*xmax+b)
  else:
    x = (ymax - b)/a
    if (x >= xmin and x <= xmax):
      p1 = (x, ymax)
      y = a*xmax + b
      if (y >= ymin and y < ymax):
        p2 = (xmax, y)
      else:
        x = (ymin -b)/a
        p2 = (x, ymin)
    else:
      p1 = (xmax, a*xmax + b)
      x = (ymin - b)/a
      p2 = (x, ymin)

show(points(xpts, rgbcolor=(0, 0, 1), pointsize=30) + points(ypts, rgbcolor=(1,0,0), pointsize=30) + line([p1, p2], rgbcolor=(100, 100, 100), linestyle='dashed'))
\end{sage}

\end{document}


\subsection{Minimax}
\subsubsection{Games}

Games are different to search. In a search algorithm we are looking for a sequence of actions. In a game we are looking for a reaction function. Unlike in seach, there are other players.

We can use iterative deepening search.

\subsubsection{Heuristics}

The search space can be too big to look through all the nodes.

Rather than look for win states, we evaluate a non-terminal state using heuristics.

\subsubsection{Stochastic games}

Can use expectminimax

For max node, return highest expectminimax of children

For min node ---

For chance node, average of children weightted

Minimax: two players, max min

Max goes first, maximises results. min minimises results

A node's minimax value is the best outcome against best player.

To find optimal strategy, depth first search of game tree.

Propagate minimax values up tree as terminal nodes are discovered

If a state is terminal, its value is utility of state

If a state is max node, highest value of children

If a state is min node, lowest value of children

Minimax is optimal, complete (for finite trees)


\textit{Combinatorics. Logic. Programming.} This dissertation is an attempt to
explore how each entity relates to the others. Seen as a set of rules that
characterize our methodology, we apply and use them to study the topic of
\textit{infinite~sequences}.  Our approach is two-fold, it allows us, first, to
prove theoretical results about them; second, to built an orbit around them
composed of side-track techniques that helps our main goal, such as programming
techniques and logical reasoning.

In particular, the research field of interest concerns (i)~the manipulation of
a set of matrices that can be algebraically defined, which deserve interest
both as standalone objects and as tools to study other combinatorial
structures; (ii)~the \textit{practice of programming} that exposes our way of
thinking to its paradigms, functional and relational in particular, and
(iii)~the rigor and power of \textit{mechanized logic}.

In normal conditions, it is hard to tackle a problem in our context with both
entities present at the same time; however, we sacrifice a direct approach to
solve the given questions to get the most out of the process that uses the
three tenets together. In this philosophy the constant delay, needed to sharpen
our knowledge in each individual field, is balanced by the discovery of
relations among apparently unrelated subjects that when mixed together yield
nicer, more elegant and possibly unexpected solutions.

For this reason we spread our focus over many topics of mathematics and
computer science instead of composing a mono-theme discussion; pairwise,
we deepen into
\begin{description}

    \item[Combinatorics and Programming] the implementation of enumeration
    techniques for classes of combinatorial objects from both the algebraic
    and applicative points of view;

    \item[Programming and Logic] the study of a family of languages designed
    for relational programming, using a general purpose inference engine to
    perform deductions over domain specific objects;

    \item[Logic and Combinatorics] the formalization of proofs to which
    corresponds certified enumerations of classes of objects, using an extended
    theorem prover based on Higher Order Logic.

\end{description}

Even though abstract reasoning took place most of the time, we make
some room for practical stuff; in particular, we would like to provide a suite
of tools that helps interacting with the Online Encyclopedia of Integer
Sequences, to automate fetching, printing and graphing the networks they
compose. Moreover, some practice with bitmasking and backtracking techniques is
done to show some interesting tiling and enumeration problems.

Another force that drove our work is a pedagogical approach to problem solving,
hence we preferred to refine a first, naive solution over and over to cut away
unnecessary details and complexities; moreover, we don't seek for efficiency in
all cases in favor of simple and beauty definitions. This methodology allows us
to use many programming languages and environments to support theoretical
derivations and this dissertation collects this heterogeneous pool of techniques.

\section*{Structure}

This dissertation has 5 main chapters that rely on the first one, which
introduces basic definitions needed by the following ones; so, except for the
first, the others can be read in any order,
\begin{itemize}
\item Chapter \ref{ch:backgrounds} quickly recalls the theory characterizing
the Riordan group and provides a set of constructors to define those matrices
programmatically, after the introduction of the symbolic module Sympy
implemented on top of the Python programming language. Moreover, it shows our
programming style which is based on consecutive manipulation of symbolic
equations, as mush as possible close to paper and pencil derivations.
\item Chapter \ref{ch:Riordan-matrices-function} presents a theoretical and
practical framework that lifts a scalar function to a matrix function, toward
application to Riordan matrices. To the best of our knowledge, this is the
first implementation that allows the user to perform its computations fully
symbolically, postponing to the end the substitution of ground values to fill
matrices with numbers. Moreover, the Jordan Canonical Form of Riordan matrices
is also studied.
\item Chapter \ref{ch:algebraic-gfs-languages-avoiding-Riordan-patterns}
studies the enumeration of languages of binary words avoiding a given pattern,
provided that some contraints over the structure of each word are taken into
account.  In particular, when the pattern to avoid is a Riordan pattern then
the problem can be solved using Riordan arrays, deriving new series
developments about enumerations with respect to the number of $1$-bits and to
the length. Finally, some combinatorial interpretations are shown, at least for
simpler families of languages.
\item Chapter \ref{ch:OEIS:tools} implements a suite of tools that interact with
the Online Encyclopedia of Integer Sequences; in particular, a (i)~crawler fetches
sequences and their cross references recursively, using asynchronous primitive
to optimize network delays, a (ii)~(pretty) printer for the fetched sequences that
allows the user to filter the sections to be rendered both in the terminal and in
web interfaces and, finally, a (iii)~grapher that draw graphs where vertices
are sequences and edges are references among them.
\item Chapter \ref{ch:queens-tilings-polyominoes} is an exercise in
backtracking techniques to solve tiling and placement problems; for the sake of
efficiency, bit masking manipulation and encodings are used. Moreover, it
provides an implementation of a classic enumeration methodology that allows us
to clearly show generations of combinatorial objects starting from concise and
recursive symbolic definitions; finally, counting all of them is a check of
the correctness of our implementation.
\item Chapter \ref{ch:scilp} proposes an extension to the tactic mechanism
actually present in the HOL Light theorem prover inspired by the relational
paradigm. This prototype makes explicit use of meta-variables to support
substitutions and allows backtracking facilities during the interactive proof
process; nonetheless, the proposed generalization offers compositionality just
as the current system does. So this extension is stressed against an evaluator
for a subset of the Lisp language to find a \textit{quine} program.
\end{itemize}

Moreover, a final paragraph concludes with a quick summary of our main results
and the bibliography ends this dissertation.

\section*{Typographical and typesetting conventions}

This dissertation was typeset using the \LaTeX\,'s style \verb|tufte-book|,
which splits each page into two columns: the left-most is greater in width and
holds the main body, while the right-most is lesser and holds captions,
contextual data and code comments; in particular, code chunks are printed in
\verb|verbatim| and highlighted wherever possible, while outputs that denote
mathematical objects are printed in math style, as usual -- when their size
overflows over the right column we prefer to allow it to take the entire
page width for the sake of clarity.


\chapter{Method}\label{ch:method}

As stated in the introduction, this thesis examines a way to train a machine learning classifier to predict the nominal stock price change.
This chapter will now introduce the classifiers that were trained, as well as the approach to represent the business reports as feature vectors using \acs{BERT}.

\section{Overview of machine learning algorithms}
All these algorithms have in common, that they take a $n$-dimensional input vector to predict $m$ classes.
Mathematically this can be seen as a function $f(x)=y$ with $x\in\mathbb{R}^n$ and $y\in\{0, 1\}$, since it is a binary classification problem.
The algorithms are able to find patterns in the data and can autonomously improve their prediction performance.
This learning process requires a training phase.
During the research a Naive Bayes classifier, a \acl{KNN} classifier, a \acl{SVM} and a \acl{BLSTM} was trained.
\subsection{Naive Bayes classifiers}
The idea behind the Naives Bayes classifier is, that certain input features correlate with a certain class, which means that these input features result with a high probability from this certain class.
The Naive Bayes classifier makes use of the assumption, that each feature of the input vector is independent to one another.
Using the Bayes theorem (equation \ref{equation:bayes}), the Naive Bayes classifier can compute the probability of a class given the features of the input.
\begin{equation}
    P(Y|X) = \frac{P(X|Y)P(Y)}{P(X)}
    \label{equation:bayes}
\end{equation}
The probability $P(Y)$ is given by the proportion of a class in the whole training dataset.
For a balanced dataset with two classes this gives the probabilities $P(y_1, y_2)=\langle0.5,0.5\rangle$.
Determining $P(X|Y)$ is not as trivial:
Since $X$ can be a vector $(x_1, x_2, ..., x_n)$ of variables, it would be necessary to determine $P(x_1 \land x_2 \land ... \land x_n|Y)$ for all combinations of $x_i$, which would require $O(2^n)$ space to save the values and make it impossible to use the algorithm with high dimensional input \cite[p. 493]{Russel2016}.
The assumption that $x_1, ..., x_n$ are independent, however, simplifies this problem.
$P(x_1 \land x_2 \land ... \land x_n|Y)$ can now be decomposed into $P(x_1|Y)\cdot P(x_2|Y)\cdot...\cdot P(x_n|Y)$ which only requires $O(n)$ to save the conditional probabilities and allows for practical usage of the algorithm \cite[p. 499]{Russel2016}.
So during training the algorithm has to determine $P(Y)$ and $P(x_1|Y), P(x_2|Y) ... P(x_n|Y)$.
The probability $P(X)$ is a normalization factor which is chosen so that $P(y_1|X)+P(y_2|X)=1$ \cite[p. 493]{Russel2016}.
During prediction phase $P(Y|X)$ is computed for all classes and the class with the highest probability given $X$ will be the classification result:
\begin{equation}
    y_{pred} = arg max_i P(y_i|X)
\end{equation}
Even though the assumption, that the features are independent to one another is generally not true, the Naive Bayes classifier often performs equally or even better than more complex methods \cite[p. 211]{Hastie2009}.
For this research, the Naive Bayes implementation from scikit-learn was used.

\subsection{\acl{KNN} classifiers}
As the name of this method implies, a nearest neighbor classifier makes its predictions based on the surrounding data points.
Because every input to this classifier can be seen as a point in a n-dimensional space, the \ac{KNN} can compute the distances to other points.
The distance can be measured as Euclidean distance, however other distance metrics are also possible.
Scikit-learn's \texttt{KNeighborsClassifier} for example, measures the distance with the Minkowski distance, which is a generalisation of the Euclidean distance\footnote{\url{https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html}}:
\begin{equation}
    d(x_1, x_2) = \left(\sum_{i=1}^n \abs{x_{1,i} - x_{2,i}} ^p \right)^{\frac{1}{p}}
\end{equation}
The Minkowski distance collapses to the Euclidean distance for $p=2$ and to the Manhatten distance for $p=1$ \cite[p. 738]{Russel2016}.
The predicted class is then the most common class of all $k$ neighbors.
Because the \ac{KNN} classifier makes use of a geometric property of the input data, it is important to scale each feature to have a mean of zero and a variance of one \cite[p. 465]{Hastie2009}.

In this thesis the \ac{KNN} implementation from scikit-learn was used as well as the \texttt{StandardScaler} to scale the features.
$k$ was determined by creating a validation curve (section \ref{subsec:learning_validation_curve}), which shows how the accuracy changes for different values of $k$.
It is implemented with the code in listing \ref{py:knn_validation_curve}.
The results of this finding are presented in chapter \ref{ch:experiments}.
\lstinputlisting[language=Python,caption={Creates data for a validation curve},captionpos=b, label={py:knn_validation_curve}]{listings/knn_validation.py}
The \texttt{knn\_\_n\_neighbors} argument tells the \texttt{validation\_curve} function, which parameter it should test for the provided estimator.
In this case the estimator is a pipeline, which wraps the \texttt{StandardScaler} and the \texttt{KNeighborsClassifier} and executes them sequentially.
The \texttt{validation\_curve} also allows to use cross-validation, where the number of folds can be set with the \texttt{cv} argument.
Cross-validation is explained in detail in section \ref{subsec:cross_validation}.

\subsection{\acl{SVM}s}
A \ac{SVM} tries to fit a decision boundary between the points of two classes, so that the surrounding margin is maximised.
This decision boundary can mathematically be seen as a hyperplane in the n dimensional space of the data points.
For a two dimensional space that results in a line:
\begin{figure}[h]
    \centering
    \includegraphics[width=0.5\textwidth]{figures/svm_decision_boundary.png}
    \caption{A decision boundary with its support vectors.\protect\footnotemark}
    \label{figure:svm_decision_boundary}
\end{figure}
\footnotetext{A modified version of the example on scikit-learn: \url{https://scikit-learn.org/stable/auto_examples/svm/plot_separating_hyperplane.html}}
The line in figure \ref{figure:svm_decision_boundary} represents the decision boundary and the dashed lines mark the borders of the margin with maximum width, respecting some constraints which will be discussed in this section.
The circled points mark the \textit{support vectors} which are used to determine the hyperplane.
The decision boundary is mathematically a hyperplane defined by:
\begin{equation}
    f(x)=x^T\beta + \beta_0=0
    \label{equation:svm_hyperplane}
\end{equation}
where $\beta$ is a unit vector, $x_i \in \mathbb{R}^n$ are the sample points and $y_i \in \{-1, 1\}$ are the labels to these points.
Therefore the aim is to maximise the margin between the training points and the hyperplane, which can be re-expressed as
\begin{align}
    & min_{\beta, \beta_0}\frac{1}{2}\abs{\abs{\beta}}^2 + C\sum_{i=1}^N\xi_i \\
\text{subject to }
    & \xi_i \geq 0, y_i(x_i^T\beta + \beta_0)\geq 1 - \xi_i \forall i \notag
\end{align}
where $\xi_i$ is a penalty term, that increases with the distance a data point $x_i$ lies on the wrong side of its margin \cite[p. 7]{Fletcher2008}.
Since this penalty terms are weighted by the constant $C$, the width of the margin varies with choosing the value of $C$.
If $C$ is set really low, a broad margin with many support vectors is used.
If it is set high, only points near the decision boundary are used as support vectors \cite[p.421]{Hastie2009}.
This optimization problem can be solved using Lagrange multipliers and gives for $\beta$
\begin{equation}
    \beta = \sum_{i=1}^{N} \alpha_i y_i x_i
\end{equation}
and for $\beta_0$
\begin{equation}
    \beta_0 = \frac{1}{N_S} \sum_{i=1}^{N_S}(y_i - \sum_{j=1}^{N_S} \alpha_j y_J x_i^T x_j)
\end{equation}
$\alpha$ is the Lagrange multiplier which is $0 < \alpha \leq C$ for all \textit{support vectors} and $0$ for all other points \cite[p. 746]{Russel2016}.
Its exact value can be determined with the dual Lagrangian function of equation \ref{equation:svm_hyperplane} \cite[p. 9]{Fletcher2008}.

The \ac{SVM} as defined so far, has the drawback, that it only creates linear decision boundaries.
Often the classes are not linearly seperable, so a mathematical trick is used, that the \ac{SVM} can still perform on such data.
This is done with a kernel function, which maps the data points to a enlarged feature space.
% $\xi_i$ are the \textit{support vectors}.
% The support vectors are the points that are used to build up the decision boundary, so that it has a maximum distance to them.
% If the classes are linearly seperable, these are the closest points of one class to the other class.
% If the data points of both classes overlap, the support vectors are constructed from the points of one class that lie in the decision region of the other class.
% $C$ is the parameter which sets the width of the margin, if it is set really low, a broad margin with many support vectors is used.
% If it is set high, only points near the decision boundary are used as support vectors.
A way to find decision boundaries for data which is not linearly seperable is to enlarge the feature space with a kernel function.
Thus the data becomes linearly seperable in the enlarged feature spaces, which has the effect of nonlinear boundaries in the original space \cite[p.423]{Hastie2009}.
One typical kernel function is the \textit{Radial Basis Kernel} which has the following form:
\begin{equation}
    K(x_i, x_j) = e^{-\frac{\abs{\abs{x_i - x_j}}^2}{2\sigma^2}}
\end{equation}
The kernel function is used in the dual Lagrangian function of equation \ref{equation:svm_hyperplane}, which is then again optimised.
This has the implicit effect, that the datapoints are mapped to a higher dimensional feature space.
Figure \ref{figure:kernel_3d} shows this mapping to a three-dimensional space with the hyperplane, that seperates the classes, figure \ref{figure:kernel_decision_boundary} shows the base feature space with the resulting circular decision boundary.
\begin{figure}[h]
    % \centering
    \subcaptionbox{Datapoints with a circular decision boundary, resulting from the kernel trick.\label{figure:kernel_decision_boundary}}[0.5\linewidth]{\includegraphics[width=0.5\textwidth]{figures/svm_circular_decision_1.png}}
    \subcaptionbox{When mapped to a higher dimensional space, the datapoints become linearly seperable.\label{figure:kernel_3d}}[0.5\linewidth]{\includegraphics[width=0.5\textwidth]{figures/svm_circular_decision_2.png}}
    % \begin{subfigure}{0.5\textwidth}
    %     \centering
    %     \includegraphics[width=\textwidth]{figures/svm_circular_decision_1.png}
    %     \caption{Datapoints with a circular decision boundary, resulting from the kernel trick.}
    %     \label{figure:kernel_decision_boundary}
    % \end{subfigure}
    % \begin{subfigure}{0.5\textwidth}
    %     \centering
    %     \includegraphics[width=\textwidth]{figures/svm_circular_decision_2.png}
    %     \caption{When mapped to a higher dimensional space, the datapoints become linearly seperable.}
    %     \label{figure:kernel_3d}
    % \end{subfigure}
    \caption{The kernel trick as visualised in \cite[p. 747]{Russel2016}}
    \label{figure:svm_kernel_trick}
\end{figure}

\subsection{Artificial Neural Networks}
Artificial neural networks imitate the structure of a brain, where each neuron receives input from other neurons, processes this information and passes it on to other neurons \cite[p. 727]{Russel2016}.
In artificial neural networks these neurons are represented as \textit{units}, which perform a mathematical operation on their input data.
This consists of first computing a weighted sum of its inputs and then applying a activation function $g$ on that sum, which determines whether the unit "fires" \cite[pp. 727-728]{Russel2016}:
\begin{equation}
    a_j=g\left(\sum_{i=0}^{n}w_{i,j}a_{i}\right)
    \label{equation:nn_unit_operation}
\end{equation}
Where $w_{i,j}$ is the weight, assigned to the input $a_i$.
A typical activation function is the sigmoid function defined by $\sigma(x)=\frac{1}{1+e^{-x}}$.
\begin{figure}[h]
    \centering
    \includegraphics[width=0.5\textwidth]{figures/sigmoid.png}
    \caption{The sigmoid function.}
    \label{figure:sigmoid}
\end{figure}
As it can be seen in figure \ref{figure:sigmoid}, the sigmoid function results in values in the range from 0 to 1, which can be interpreted as the probability that the input belongs to one class.
The units are then structured into layers, with the simplest possible network having only two layers, one input and one output layer.
These simple networks are limited in their learning capabilities, that is why usually several hidden layers are added \cite[pp. 730-731]{Russel2016}.
% The input $a_j$ to a unit in such a hidden layer is itself the result of the computation with the function \ref{equation:nn_unit_operation}, therefore the whole network can mathematically be seen as a function applied to the input vector.
The network network predicts values by computing the output of each unit with function \ref{equation:nn_unit_operation} starting from the input layer.
The nodes in the subsequent layers use the same function to compute their outputs, however the input $a_i$ is the result of the computations on the previous layer.
The results of this computation on the output layer is the prediction of the network.
At training time, the units of the output layer then compute an error between the prediction it made and the actual value it should have predicted.
A common error metric is the squared-error loss $Loss_k$ on an output node $k$, which is defined in equation \ref{equation:squared_error_loss} \cite[p. 735]{Russel2016}.
\begin{equation}
    Loss_k=(y_k - a_k)^2
    \label{equation:squared_error_loss}
\end{equation}
With $a_k$ being the predicted value on output $k$ and $y_k$ being the actual value that should be predicted.
The aim is to minimise this loss with respect to all weights $w_{i,j}$ in the network:
\begin{equation}
    \frac{\partial Loss_k}{\partial w_{i,j}}
\end{equation}
This gives the gradient for the output node $k$.
The gradient can be computed with the \textit{back-propagation} algorithm.
The last layer computes an error term, based on the derivative of the loss function.
This error term is back-propagated to the previous layer, where each node again computes its own error, by summing the weighted errors of the subsequent nodes.
This is again repeated until the input layer is reached, then the weights between all nodes are updated based on the determined error terms \cite[p. 734]{Russel2016}.
% Auf nichtlinearität eingehen

A special class of neural networks are the \acp{RNN}, which have the ability to gather information from the order of the input items.
Therefore each input example consists of several \textit{timesteps}.
In case of report classification, like it is done in this thesis, the timesteps are the sentences of the report, which have a clear order to each other.
To process this sequential data, the units in a \acs{RNN} (figure \ref{figure:rnn_unfolded}) are different to the units of a traditional feed-forward neural network.
\begin{figure}[h]
    \centering
    \includegraphics[width=0.6\textwidth]{figures/rnn_unfolded.png}
    \caption{The flow of the activations between two timesteps in an RNN as depicted in \cite[p. 12]{Lipton2015}.}
    \label{figure:rnn_unfolded}
\end{figure}
They have one input from the previous layer and a second one from the previous timestep.
On the output side they have an output for the next timestep and optionally also an output to the next layer \cite[p. 373]{Goodfellow2016}.
The training is done by computing the loss in the last timestep and propagating the loss back to the previous timestep.
A problem that occurs in standard \acp{RNN} is that the gradient decayes or increases exponentially while propagated back in time \cite[p. 13]{Lipton2015}.
A concept to avoid this problem is the \acs{LSTM}, which was proposed by \cite{Hochreiter1997}.
Figure \ref{figure:lstm_cell} shows the structure of a LSTM cell.
\begin{figure}[h]
    \centering
    \includegraphics[width=0.6\textwidth]{figures/lstm_cell.png}
    \caption{A LSTM cell as depicted in \cite[p. 17]{Lipton2015}.}
    \label{figure:lstm_cell}
\end{figure}
Each of these units has an input gate which protects the unit against irrelevant input ($i_c$ in figure \ref{figure:lstm_cell}) and an output gate $o_c$ which protects other units from possibly irrelevant content in the respective unit \cite[p. 6]{Hochreiter1997}.
Both, the \ac{RNN} as well as the \ac{LSTM}, have forms that also process the input sequence in the reverse order; the \ac{LSTM} is then called \ac{BLSTM}.

In this research the \ac{LSTM} implementation from Keras was used.
Although different network architectures were used, the general implementation with all used layer types is shown in listing \ref{py:lstm_implementation}.
\lstinputlisting[language=Python,caption={Implementation of a \acs{LSTM} in Keras.},label={py:lstm_implementation}]{listings/lstm.py}
The \texttt{LSTM} layers in listing \ref{py:lstm_implementation} consist of the just introduced \ac{LSTM} cells.
With the \texttt{return\_sequences} parameter the hidden states of all \ac{LSTM} cells on the respective layer can be used and the \texttt{go\_backwards} parameter reverses the order in which the sequential data is processed\footnote{\url{https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/keras/layers/LSTM}}.
\texttt{Bidirectional} is a wrapper for a forward processing \ac{LSTM} layer and a backward processing \ac{LSTM} layer to create a \ac{BLSTM}\footnote{\url{https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/keras/layers/Bidirectional}}.
With the \texttt{GlobalAveragePooling1D} layer the hidden states from the previous \ac{BLSTM} layer are averaged across the timesteps\footnote{\url{https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/keras/layers/GlobalAveragePooling1D}}.
The \texttt{Dense} layer then is a layer with the regular feed forward units described at the beginning of this section \footnote{\url{https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/keras/layers/Dense}}.

A key aspect is the usage of \texttt{None} in the first dimension of the \texttt{input\_shape} on line 11 in listing \ref{py:lstm_implementation}.
This dimension defines the number of time steps each input sample has.
Since each report has a different length, this value was set to \texttt{None}.
Even though the \ac{LSTM} can handle input with a different number of time steps, on batch level it still requires the input to have a fixed number of time steps.
Therefore the reports need to be grouped by their lengths, which was accomplished with the \texttt{ReportSequence} class (Listing \ref{py:lstm_report_sequence}).
\lstinputlisting[language=Python,caption={The \texttt{ReportSequence} class creates batches with reports of same length.},label={py:lstm_report_sequence}]{listings/report_sequence.py}
This grouping is then used by calling the \texttt{fit\_generator} method on the model with an instance of the \texttt{ReportSequence} class in listing \ref{py:lstm_implementation} on line 26.
Similar to the previously described classifiers, a 5 fold cross-validation was used to select the best model architecture.

Now that the classification algorithms are introduced, one problem still remains:
The classifiers require the input to be a numerical vector.
Therefore the reports have to be converted to feature vectors.
This embedding step is described in the next section.

\section{Creating document embeddings}
\label{sec:creating_document_embeddings}
\subsection{Methods to create a document representation}
\label{subsec:methods_for_document_representation}
The task of the embedding step is to map a text of variable length to a vector of fixed length, which is necessary to train a classification algorithm.
The most basic approach for creating a text representation is the bag-of-words model.
For that, a vocabulary is created which contains each word of a text corpus.
Each word in this vocabulary is represented by a vector with the dimensionality of the vocabulary size, having a $1$ on the entry at which the word is stored in the vocabulary and zeros on all other positions.
To create the representation of a document, the vectors of the words in the document are summed \cite[p. 13]{Grzegorczyk2018}.
Since a single document typically does not contain all words of the vocabulary this vector is sparse and an inefficient representation of the text.
This method also does not take into account that some words occur more frequently than others, which can be solved by mutiplying the term frequency with the inverse document frequency.
The resulting metric is called \textit{TF-IDF}:
\begin{equation}
    TF-IDF(t,d) = f_{t,d} \cdot log \frac{N}{n(t)}
    \label{equation:tf_idf}
\end{equation}
Where $f_{t,d}$ is the number of occurences of a term $t$ in a document $d$, $N$ the total number of documents and $n(t)$ the number of documents that contain the term $t$ \cite[p. 13]{Grzegorczyk2018}.

Another drawback of the word vectors in the bag-of-words model is, that it is not possible to determine similarity between different words.
Since they are all orthogonal to each other, "powerful", "strong" and "Paris" share the same similarity with each other \cite[p. 1]{Mikolov2014}.
This problem can be solved by using distributed vector representations, which can be created with the popular \textit{Word2Vec} implementation.
Word2Vec is a simple neural network with one hidden layer \cite[p. 22]{Grzegorczyk2018}.
It offers two algorithms to learn the word representations.
With the \ac{CBOW} algorithm the network predicts a word based on its surrounding words and with the \textit{skip-gram} algorithm the network tries to predict context words based on a center word \cite[pp. 4-5]{Mikolov2013}.
The word embeddings are then a result of maximizing the probabilities of these predictions \cite[p. 23]{Grzegorczyk2018}.
A simple approach for creating a document representation of these word embeddings would be to average all word vectors of the document.
Similar to the bag-of-words model with the term frequencies, this approach also does not respect the order of the words.
An extension to Word2Vec that creates a document representation is the \textit{Paragraph Vector} model \cite[p. 1]{Mikolov2014}, which is better known as the implementation \textit{Doc2Vec}.
For this model an additional paragraph representation is trained in parallel to the word representations \cite[p. 3]{Mikolov2014}.
The paragraph vector is incorporated in all word prediction contexts, therefore it captures the meaning of the whole document \cite[p. 3]{Mikolov2014}.

At the time of publication, \textit{Paragraph Vector} created state-of-the-art results in different sentiment tasks \cite[pp. 4-7]{Mikolov2014}, this thesis however, will use the modern language representation model \ac{BERT} to create the document embeddings.
In comparison to \textit{Paragraph Vector}, \ac{BERT} can use distant context words to the same extend as close words to create an embedding, whereas for the \textit{Paragraph Vector} model, distant words contribute less to the embedding of a center word \cite[p. 4]{Mikolov2013}.
The ability to incorporate these long range relations is a result of the Transformer architecture, which was proposed in \cite{Vaswani2017} and is the basis of \ac{BERT}.
\begin{figure}[h]
    \centering
    \includegraphics[width=0.5\textwidth]{figures/transformer_architecture.png}
    \caption{The Transformer architecture as proposed in \cite[p. 3]{Vaswani2017}.}
    \label{figure:transformer_architecture}
\end{figure}
From the pure application area, Transformer is really similar to \acp{RNN} or \acp{LSTM}, as it is also optimised for handling sequential data.
It uses an encoder-decoder structure with a self-attention mechanism.
The encoder maps an input sequence to a vector representation, which the decoder uses to create an output sequence.
This can be used for example for translation tasks, where a sentence from the original language is passed to the encoder and the decoder returns the translated sentence \cite[p. 392]{Goodfellow2016}.
Traditional encoder-decoder models use \acp{RNN} or \acp{LSTM} for the encoder and decoder, with the vector representation being the final hidden state of the encoder \cite[p. 392]{Goodfellow2016}.
Enhancing these models with Attention, the hidden states of all encoder units can be used, with the focus on the most relevant states.
The Transformer architecture now relies purely on Attention and the sequential information is manually encoded in the data, which is labeled as \textit{Positional Encoding} in figure \ref{figure:transformer_architecture}.
Attention is accomplished by computing a weighted sum of the hidden states.
The weights are given by the similarity between the hidden states of the encoder, which are called the \textit{keys} and the most recent hidden state of the decoder, called the \textit{queries}.
Both, the \textit{keys} and the \textit{queries}, are matrices and the similarity between them is calculated as a multiplication, on which a softmax function is applied afterwards \cite[p. 4]{Vaswani2017}.
Between these weights and the \textit{values}, which are the hidden states of the encoder as well, another matrix multiplication is calculated, giving the weighted sum.
In the Transformer model, this Attention mechanism is combined to a \textit{multi-head attention}.
The multi-head attention consists of multiple attention layers, each performing the attention on different linear projections of the query, key and value inputs and concatenating the results in the end \cite[pp. 4-5]{Vaswani2017}.
The advantage of the Transformer model is a high parallelization of the computation and the already mentioned capability to capture long range relations \cite[p. 2]{Vaswani2017}.
For \ac{BERT} the Transformer blocks depicted in figure \ref{figure:transformer_architecture} are stacked in multiple layers, with 12 layers in case of BERT\textsubscript{BASE} and 24 layers in case of BERT\textsubscript{LARGE} \cite[p. 3]{Devlin2018}.
Because of resource limitations, the BERT\textsubscript{BASE} model is used in this research.

Another important property of \ac{BERT} that can't be ignored, is that it can only process input text with a maximum sequence length of 512 tokens.
\textit{Tokens} are the entries in the vocabulary of \ac{BERT}.
Before \ac{BERT} can process the input text, it has to be tokenized.
In this step, each word from the input sequence is replaced with the longest token in the \ac{BERT} vocabulary, that matches the word.
If no token matches the word exactly, the word is split in multiple tokens.
For example, the word \textit{discounted} is not in the vocab and the tokenizer therefore splits the word into the sub tokens \textit{discount} and \textit{\#\#ed}.
The restriction to a maximum sequence length of 512 tokens implies, that it is not possible to pass a whole business report to \ac{BERT}.
The segmentation into sentences with spaCy, that was described in section \ref{sec:data_preprocessing}, is part of solving this problem, however in some cases the resulting sentences are way shorter than the sequence length of 512 tokens.
Therefore the sentences are again concatenated until the maximum sequence length is reached.
For most applications working with \ac{BERT} involves two steps, a pre-training and a fine-tuning step.
Since \ac{BERT} is used to create the text embeddings, an additional feature extracting step was added which is described in section \ref{subsec:extracting_feature_vectors}.

\subsection{Pre-training}
In the \textit{pre-training} step, \ac{BERT} learns the general language structure which includes the positions of different words within a sentence and the relationship between successive sentences.
Since BERT is a language model with a deep architecture, bidirectional training of the word positions is not possible, because the network can trivially predict the target word \cite[p. 4]{Devlin2018}. %(<- Elaborate on here)
Therefore \cite{Devlin2018} decided to randomly mask out a number of words in the sentence and then predict the masked words.
\begin{figure}[h]
    \centering
    \includegraphics[width=1.0\textwidth]{figures/illustration_masked_lm.png}
    \caption{Illustration of the \textit{masked language model} pre-training task.}
    \label{figure:masked_lm}
\end{figure}
The masking of the words is also done by the already introduced tokenizer.
For representing the masked words, the special token \texttt{[MASK]} is used.
Another special token is the \texttt{[CLS]} token, which is prepended to each input sequence and holds the meaning of this sequence after the \textit{fine-tuning} procedure \cite[p. 4]{Devlin2018}.
The whole task of predicting masked words based on their contexts is called \textit{masked language model} \cite[p. 4]{Devlin2018}.
The second task during pre-training is the binary \textit{next sentence prediction}.
For that, \ac{BERT} has to predict whether a sentence \textit{B} is the actual successive sentence to a sentence \textit{A} or whether it s a randomly chosen sentence from the training corpus.

Because \ac{BERT} is using Tensorflow as a backend, it expects the input data to be in the TFRecord file format.
TFRecord is a binary file format, which contains the tokenized text and the tokens that are masked.
Because the creation of each TFRecord file takes a few seconds, the dataset is again split to optimise the processing with multiple cores, as it can be seen in Listing \ref{py:split_data_and_spawn_processes}.
\lstinputlisting[language=Python,caption={The dataset is split to optimise the TFRecord creation on multicore systems.},label={py:split_data_and_spawn_processes}]{listings/split_data_and_spawn_processes.py}
The \texttt{\_handlePretrainingDataCreation()} function, which is provided as as a callback to the \texttt{Process}, is mostly similar to the \texttt{main()} function of \texttt{create\_pretraining\_data.py} from the official \ac{BERT} repository\footnote{The official \ac{BERT} repository on GitHub: \url{https://github.com/google-research/bert}}.
It was modified, so it creates the TFRecord files for multiple input files at once.

The actual training of \ac{BERT} was done with the script in listing \ref{py:run_pretraining}.
\lstinputlisting[language=Python,caption={\ac{BERT} pre-training.},label={py:run_pretraining}]{listings/run_pretraining.py}
Again most of the source code is similar to the \texttt{main()} function of \texttt{run\_pretraining.py} from the official \ac{BERT} repository.
One change that was made, was to set the number of checkpoints, that are created during training, to 30 as opposed to 5 as default.
These checkpoints were later used to evaluate the model.
The validation step was also moved to a seperate script to evaluate the model for every training checkpoint.
This is accomplished by iterating over the created checkpoints and calling the evaluation code from \ac{BERT} for each of them.
In the original source code only the final model was evaluated.
The results are appended to a Pandas DataFrame, which is saved as a \ac{CSV} file in the end.

\subsection{Fine-Tuning}
\label{subsec:fine_tuning}
The \textit{fine-tuning} step is used to optimise \ac{BERT} for a specific task.
This can be a task like question answering or sentence classification \cite[pp. 5-7]{Devlin2018}.
Even though the final goal is to classify the business reports and not every sentence on its own, the fine-tuning is essential so that the \texttt{[CLS]} token holds a good representation of the sentence\footnote{This hint was mentioned in \texttt{extract\_features.py} on lines 237-239 \url{https://github.com/google-research/bert/blob/fe354751d7de010f60d362ae8d9343849ec39456/extract_features.py}}.
Since the final aim is to classify each business report, the most similar task was selected for fine-tuning, which is the sentence classification.
In difference to the pre-training, the sequences for fine-tuning need a label.
Since each input sequence to \ac{BERT} is a part of a larger report, the sequences "inherit" their label from this report.
Therefore every sentence of a \textit{positive} report is also labeled \textit{positive} and every sentence of a \textit{negative} report is also labeled as \textit{negative}.
Of course this approach seems wrong on the first glance, since a positive report can also contain negative sentences and vice versa.
However one could also re-interpret the classification task as follows:
Instead of saying that the label indicates whether a sentence is a \textit{positive} or \textit{negative} sentence, it can also be interpreted as indicating whether the sentence belongs to a \textit{positive} or \textit{negative} report.
The experimental results in section \ref{sec:fine_tuning_results} show how this approach affects fine-tuning.

As opposed to the pre-training, the creation of the TFRecord files was part of the training script for fine-tuning.
Therefore every training run was delayed, because the TFRecord files were generated first.
The call to the \texttt{file\_based\_convert\_examples\_to\_features} function, which creates the TFRecord files was therefore moved to another script (Listing \ref{py:convert_to_tfrecord}).
\lstinputlisting[language=Python,caption={Creates the TFRecord files for fine-tuning.},label={py:convert_to_tfrecord}]{listings/convert_to_tfrecord.py}
This allows to train the model without waiting for the creation of the TFRecord files.
Similar to the changes made for pre-training, the number of checkpoints to save was also increased and the validation of the model was moved to a second script.

\subsection{Extracting feature vectors}
\label{subsec:extracting_feature_vectors}
After the fine-tuning is accomplished, the \texttt{[CLS]} token at the start of each sequence holds information about the class of the sequence.
This information can be retrieved by accessing the hidden states of the Transformer blocks in the last layer of the model.
Listing \ref{py:extract_features} shows a snippet of the \texttt{extract\_features.py} script.
\lstinputlisting[language=Python,caption={Accesses the hidden states from the transformer layers.},label={py:extract_features}]{listings/extract_features.py}
The official implementation allows to access the hidden states from all layers of the model, however for this thesis only the last layer is used.
For that, the \texttt{layer\_indexes} variable which is passed to the \texttt{model\_fn\_builder} is set to -1.
Therefore, the hidden states of this layer can be accessed by \texttt{result["layer\_output\_0"]} on line 21.
In theory, the \texttt{result} dictionary also includes \texttt{layer\_output\_1, layer\_output\_2, ...}, but the \texttt{layer\_indexes} limits the results.
In the list comprehension on line 23 \texttt{layer\_output[0]} only the hidden states for the first token, the \texttt{[CLS]} token, is accessed.
The \texttt{for} loop is necessary because the \texttt{predict} method returns an iterator, where each element represents the hidden states for one line of the input file.
As a summary it can be said, that this implementation only takes the hidden states from the last layer for the first token, while the official implementation takes the hidden states for all tokens from possibly all layers.

Because of the limitation of \ac{BERT} that it can only process sequences with a maximum length of 512 tokens, many embedding vectors result for each report.
For the \ac{BLSTM} the single sequence embeddings give the timesteps for each sample.
The Naive Bayes, \ac{KNN} and \ac{SVM} however, need a single vector representation for each report, therefore the sequence embeddings are averaged per report.

\section{Other used techniques}
\subsection{Cross-Validation}
\label{subsec:cross_validation}
Cross-validation is a method to estimate the prediction accuracy of a model for a unknown dataset \cite[p. 241]{Hastie2009}.
For that a training dataset is split in $K$ equally sized parts.
$K-1$ of these parts are then used to train the model and the $k$th part is used to evaluate the model.
This is repeated for $k=1,2,...,K$ and the resulting accuracies are averaged \cite[p. 242]{Hastie2009}.
The averaged accuracy is used to optimise the model:
The hyperparameters of the model are changed so that the averaged accuracy is maximised \cite[p. 242]{Hastie2009}.
One drawback of the method is that the time to train and evaluate the model increases with a larger $K$ \cite[p. 242]{Hastie2009}.

For this research 5-fold cross-validation was used for the Naive Bayes, \ac{KNN}, \ac{SVM} and the \ac{BLSTM}.
The cross-validation dataset covered 80\% of the whole dataset, therefore for the test dataset remained 20\% of the data.
Both datasets were balanced so that they include the same number of positive and negative reports.
The hyperparameters for the \ac{BERT} pre-training and fine-tuning were determined without cross-validation, because \ac{BERT} itself is quite computational expensive.
Since for \ac{BERT} no cross-validation was used the training dataset covered 60\%, the validation and test dataset both covered 20\% of the data.
Again all datasets were balanced.

\subsection{Learning and validation curve}
\label{subsec:learning_validation_curve}
This thesis uses learning curves to visualise the learning progress of the machine learning models.
A learning curve is a line chart that shows the accuracy of a machine learning model depending on a second variable, like the training dataset size or the number of epochs a neural network trained \cite[pp. 702-703]{Russel2016}.
It can give evidence about whether the model would improve its performance if more data was available or the number of epochs was increased.
Figure \ref{figure:sample_learning_curve} shows a possible learning curve.
In this thesis, the accuracies of the Naive Bayes, \ac{KNN} and \ac{SVM} are indicated depending on the size of the training dataset and for the \ac{BLSTM} the accuracies are indicated depending on the number of epochs.
For \ac{BERT} the horizontal axis shows the number of training steps, which is a comparable measurement with the number of epochs (Chapter \ref{ch:experiments}).

Validation curves are used to find the best hyperparameters of a model.
They show the accuracy of the model depending on different values of this hyperparameter, that can be for example the number of $K$ in the case of the \ac{KNN} or the value of $C$ for a linear \ac{SVM}.
The best value is then selected by choosing the model that reached the highest validation accuracy \cite[p. 711]{Russel2016}.
Figure \ref{figure:sample_validation_curve} shows a possible validation curve for a \ac{KNN} classifier,

In this thesis both of the curves are created in combination with cross-validation.
\begin{figure}[h]
    % \begin{subfigure}{0.5\textwidth}
    %     \includegraphics[width=\textwidth]{figures/knn_sample_learning_curve.png}
    %     \caption{The learning curve shows the accuracy depending on the training dataset size.}
    %     \label{figure:sample_learning_curve}
    % \end{subfigure}
    % \begin{subfigure}{0.5\textwidth}
    %     \includegraphics[width=\textwidth]{figures/knn_sample_validation_curve.png}
    %     \caption{The validation curve shows how a change in the value of a hyperparameter affects the accuracy of the model.}
    %     \label{figure:sample_validation_curve}
    % \end{subfigure}
    \subcaptionbox{The learning curve shows the accuracy depending on the training dataset size.\label{figure:sample_learning_curve}}[0.5\linewidth]{\includegraphics[width=0.5\textwidth]{figures/knn_sample_learning_curve.png}}
    \subcaptionbox{The validation curve shows how a change in the value of a hyperparameter affects the accuracy of the model.\label{figure:sample_validation_curve}}[0.5\linewidth]{\includegraphics[width=0.5\textwidth]{figures/knn_sample_validation_curve.png}}
    \caption{Sample learning and validation curves of a \ac{KNN} classifier.}
    \label{figure:sample_learning_validation_curves}
\end{figure}

\FloatBarrier
\subsection{\acl{PCA}}
\acl{PCA} is a method to reduce the dimensionality of a dataset with loosing as little information as possible.
In this thesis \ac{PCA} is used to obtain a two dimensional representation of the data, which can then be visualised.
Mathematically the \ac{PCA} can be seen as the projection of a data matrix $X$ to a target matrix $Y$ with other basis vectors (equation \ref{equation:pca_projection}) \cite[p. 3]{Shlens2014}.
\begin{equation}
    Y=PX
    \label{equation:pca_projection}
\end{equation}
For the application of this thesis $X$ is a $m\times n$ matrix where each column contains the hidden states for the business report.
$Y$ is a $m\times n$ matrix as well, with the special property, that the rows are sorted by their variances, with the highest variance in the first row \cite[p. 5]{Shlens2014}.
To accomplish this, the covariance matrix $C_X$ of $X$ is computed.
If the row entries of $X$ have a zero mean, the covariance matrix is given by $C_X=\frac{1}{n}XX^T$ \cite[p. 5]{Shlens2014}.
If the mean is not zero, each value has to be subtracted by the corresponding row mean first.
Then the eigenvectors of $C_X$ are computed, which form the rows of the projection matrix $P$ \cite[p. 6]{Shlens2014}.
Another way of solving the \ac{PCA} is to use singular value decomposition, this is described in detail in \cite[p. 7]{Shlens2014}.

% This is done by fitting a hyperplane with dimension $q \leq p$ into the featurespace with dimension $p$ that minimises the squared error to the data points.
% For practical usage the dimension $q$ of the hyperplane is typically 2 or 3.
% For $q=2$, the hyperplane is a twodimensional plane on which the values are projected and can then be visualised.


\documentclass[letterpaper]{article}
\title{\textbf{Tokio university's entrance exam 2016}}
\author{Acosta Domínguez Jorge}
\date{}

\usepackage{amsmath}
\usepackage{tabto}
\usepackage{tasks}
\usepackage{geometry} %this reduces the amount of margin
\newcommand{\itab}[0]{\hspace{2em}}

\begin{document}
\pagenumbering{gobble}
\maketitle
\newpage
\pagenumbering{arabic}

\section{Mathmatics exam}
\subsection{Lineal algebra}
The tribonacci numbers ${T_n}$ are defined for non-negative integers n as follows.
\begin{displaymath}
\begin{cases}
T_0 &= T_1 = 0\\
T_2 &= 1\\
T_{n+3} &= T_{n+2} + T_{n+1} + T_{n} \itab(n\geq 0)
\end{cases}
\end{displaymath}
Answer the following questions.
\begin{tasks}
\task{
Find the matrix $A$ that satisfies Eq. (1) for all non-negative integers $n$.
\begin{equation}
\left(\begin{matrix}
T_{n+3}\\
T_{n+2}\\
T_{n+1}
\end{matrix}\right)
=A
\left(\begin{matrix}
T_{n+2}\\
T_{n+1}\\
T_{n}
\end{matrix}\right)
\end{equation}
}
\end{tasks}
\newpage
\subsubsection{Resolution}
\begin{tasks}
\task{
The resolution for this exercise would be the below matrix $A$, since it accomplishes the equation:
\begin{displaymath}
A = \left(\begin{matrix}
1 & 1 & 1\\
1 & 0 & 0\\
0 & 0 & 0
\end{matrix}\right)
\end{displaymath}
Because resolving $1$ we have:
\begin{displaymath}
\left(\begin{matrix}
T_{n+3}\\
T_{n+2}\\
T_{n+1}
\end{matrix}\right) =
\left(\begin{matrix}
T_{n+2} + T_{n+1} + T_{n}\\
T_{n+2}\\
0
\end{matrix}\right)
\end{displaymath}
}
\end{tasks}
\end{document}