\chapter{Training, validation and test data}
In a classification problem, we divide the (usually manually) labeled dataset $D=\{(x_i,y_i)\}$ into three disjoint sets, including a training dataset $X$, a validation dataset $V$ and a test dataset $T$.
% to give an expectation of the real generalization of NN.

	\begin{enumerate}
		\item The labeled dataset $D=\{(x_i,y_i)\}_{i=1}^N$: $x_i$ are samples and $y_i$ are corresponding labels. For example, Figure \ref{xd:lbd} shows the labeled dataset of five class points, i.e. $x_i\in\mathbb{R}^2,y_i=1:5$.

			\begin{figure}[!ht]
				\center{\includegraphics[width=10cm] {Homework1_5classes.png}}
				\caption{labeled dataset}
				\label{xd:lbd}
			\end{figure}

		\begin{enumerate}
			\item Training Data $X$: we use it to train the neural network $f_\theta(x)$, aiming at minimizing the loss function $\mathcal{L}(X;\theta)$ by adjusting $\theta$.

			\item Validation Data $V$:
			\begin{enumerate}
				\item Detecting whether the neural network is over-fitting, i.e. if $\mathcal{L}(X)$ decreases while $\mathcal{L}(V)$ increases.
				\item If it is the case, one may consider stopping training on $X$, because the prediction ability of $f_\theta$ on other samples (which are what we really want) may decrease.
				\item To make full use of the labeled data, one may continue to train $f_\theta$ with $V$, and make $X$ the validation data.
			\end{enumerate}
			\item Test Data $T$
			\begin{enumerate}
				\item Testing the real performance of $f_\theta$.
				\item We expect a similar accuracy on other unlabeled data as on the test data.
			\end{enumerate}
		\end{enumerate}

		\item Data augmentation: how to generate more data (to avoid over-fitting and make the training easier)
			\begin{enumerate}
				\item Shift: shift an image
				\item Rotation: rotate an image
				\item Crop: separate an image to small (intersected) images
				\item ...
			\end{enumerate}
	\end{enumerate}


\subsubsection{Combinations of Fungal Species}
The distribution of fungi in the nature is diverse and wide, and there are often hundreds of microorganisms in one square meter of natural land~\cite{land}. In order to simplify the study on the topic that \textbf{combinations of fungal species likely to persist in different environment conditions}, we only analyze and predict the growth and decomposition ability of two fungi in combination.
\par
Since the known information is not sufficient and the internal relationship of fungal species combinations in different environments has a degree of uncertainty, we use \textbf{Gray Prediction Model}~\cite{gray} and time series data based on the \textbf{Decomposition Models} in \textit{Model~4.1} and \textit{Model~4.2} to analyze and predict the living situation of the fungal species combination and the change trend of decomposition ability.
\par
Mark the known group of time series data as follows.
$$X^0 = (x^0(1),\ x^0(2),\ x^0(3),\ \ldots,\ x^0(n))$$
\par
To ensure the stability of the data, we perform \textbf{Accumulated Generating Operation} (AGO)~\cite{gray} on the data. The AGO process would be executed multiple times until the time series data is stable enough, and we get the following time series data.
$$X^1 = (\sum^1_{i=1}x^0(i),\ \sum^2_{i=1}x^0(i),\ \ldots,\ \sum^n_{i=1}x^0(i))$$
\par
The time series $X^1$ is then fitted by a first-order differential equation given by \textit{Eq.~(\ref{15eq})}
\begin{equation}
  \label{15eq}
  \frac{dx^1}{dt} = ax^1 = b
\end{equation}
where the parameters $a$ and $b$ are the development coefficient and control variable.
\par
According to the \textbf{whitening gray derivatives} of discrete data points, the following equation is obtained.
\begin{equation}
  \frac{dX^1(t)}{dt} = x^1(k) - x^1(k-1) = x^0(k)
\end{equation}
\par
A new variable $z^1(k)$, which is known as the whitening value of $x^1(k)$, is defined by the following equation.
\begin{equation}
  z^1(k) = 0.5x^1(k) + 0.5x^1(k-1)
\end{equation}
\par
Then we use \textbf{the least square method} to calculate the parameters.
\begin{equation}
  \hat{\delta} = (B^\top B)^{-1}B^\top Y
\end{equation}
where
$$\hat{\delta} = [\hat{a},\ \hat{b}]^\top,\ B=\left[ \begin{array}{cc}
      -z^1(2) & 1      \\
      -z^1(3) & 1      \\
      \vdots  & \vdots \\
      -z^1(n) & 1
    \end{array}
    \right],\ Y=\left[ \begin{array}{c}
      x^0(2) \\
      x^0(3) \\
      \vdots \\
      x^0(n)
    \end{array}
    \right]$$
\par
Finally, use \textbf{inverse AGO} and get the predicted value $x^2(k+1)$ as follows.
$$\hat{x}^2(k+1) = \hat{x}^1(k+1) - \hat{x}^1(k) = (1-e^a)(x^1(1) - \frac{b}{a})e^{-at},\ k = 1,\ 2,\ \ldots,\ n$$
\par
When new data are added, the parameters $a$ and $b$ in the Gray Prediction Model would be \textbf{updated} to ensure the accuracy of prediction.
\par
Due to the large differences in the suitable environment among the five fungi, we divide the five fungi into four combinations, including $F_A\ \&\ F_B$, $F_B\ \&\ F_C$, $F_C\ \&\ F_D$, and $F_D\ \&\ F_E$, based on the conclusion of \textit{Table~\ref{correspondingoptimalenvironment}} for analysis and prediction.
\par
Take combination $F_A\ \&\ F_B$ as an example. Calculate the time series data of the decomposition rate for the previous $122$ days using the data in \textit{Eq.~(\ref{sixtheq})}, \textit{Eq.~(\ref{ninetheq})}, and \textit{Table~\ref{decompositionrates}}, and mark it as follows.
$$X^0_{A\&B} = (x^0_{A\&B}(1),\ x^0_{A\&B}(2),\ \ldots,\ x^0_{a\&B}(122))$$
\par
Then the time series data is substituted into Gray Prediction Model, and the prediction data is calculated as follows.
$$X^2_{A\&B}=(x^2_{A\&B}(123),\ x^2_{A\&B}(124),\ \ldots,\ x^2_{A\&B}(200))$$
\par
Draw the decomposition rate of fungal species combination $F_A\ \&\ F_B$ in five climates, including arid, semi-arid, temperate, arboreal, and tropical rain forests, as shown in the figure below.
\begin{figure}[H]
  \centering
  \label{decompositioninfiveclimates}
  \includegraphics[width=\textwidth]{figures/A&B.png}
  \caption{Decomposition rate of fungal species combination $F_A\ \&\ F_B$ in five climates.}
\end{figure}
Combination $F_A\ \&\ F_B$ is most suitable for the decomposition of lignin or cellulose in semi-arid climate. From the prediction result, the cumulative decomposition rate after $200$ days can reach $34.7\%$.
\par
In the same way, the time series data of the other three fungal species combinations, $F_B\ \&\ F_C$, $F_C \&\ F_D$ and $F_D \&\ F_E$, are calculated as follows.
\begin{equation*}
  \begin{cases}
    X^0_{B\&C} = (x^0_{B\&C}(1),\ x^0_{B\&C}(2),\ \ldots,\ x^0_{B\&C}(122)) \\ \\
    X^0_{C\&D} = (x^0_{C\&D}(1),\ x^0_{C\&D}(2),\ \ldots,\ x^0_{C\&D}(122)) \\ \\
    X^0_{D\&E} = (x^0_{D\&E}(1),\ x^0_{D\&E}(2),\ \ldots,\ x^0_{D\&E}(122))
  \end{cases}
\end{equation*}
\par
Substituting Gray Prediction Model to obtain the following decomposition rate prediction curves in different climates in turn.
\par
\begin{figure}[H]
  \centering
  \subfigure[$F_B\ \&\ F_C$]{
    \includegraphics[width=0.95\textwidth]{figures/B&C.png}
  }
\end{figure}
\begin{figure}[H]
  \centering
  \subfigure[$F_C\ \&\ F_D$]{
    \includegraphics[width=0.95\textwidth]{figures/C&D.png}
  }
\end{figure}
\begin{figure}[H]
  \centering
  \subfigure[$F_D\ \&\ F_E$]{
    \includegraphics[width=0.95\textwidth]{figures/D&E.png}
  }
  \caption{Decomposition rate of fungal species combinations in five climates.}
\end{figure}
\par
The optimal environment and the predicted decomposition rate after $200$ days under different combinations of fungal species are summarized in the following table.
\begin{table}[H]
  \centering
  \caption{Summary of optimal environment and predicted decomposition rate.}
  \label{summaryofenvironmentandprediction}
  \begin{tabular*}{\hsize}{@{\extracolsep{\fill}}ccc}
    \toprule
    Combinations & Optimal Environment & Decomposition Rate Predicted Value \\
    \midrule
    $F_A\&F_B$ & Semi-arid & $34.7\%$ \\
    $F_B\&F_C$ & Temperate & $32.8\%$ \\
    $F_C\&F_D$ & Temperate / Arboreal & $32.7\%$ \\
    $F_D\&F_E$ & Tropical rain forests & $34.3\%$ \\
    \bottomrule
  \end{tabular*}
\end{table}
According to the above table, we find that the combination of $F_A\ \&\ F_B$ has the highest decomposition rate in semi-arid region, which is $34.7\%$, and the decomposition rates of other species combinations in their optimal environment are also relatively high, which are all above $32\%$.


\section{Preliminaries and exploratory data analysis}

We're going to revisit data we considered in the cluster analysis lab., so you need to load the \texttt{cluster} and the \texttt{flexclust} libraries to make the data available.

We'll start by considering the \texttt{milk} data; you should have done some interesting eda in the cluster lab.   We follow this up by carrying out multidimensional scaling using \texttt{cmdscale} from the \texttt{MASS} library (look at the helpfile):

\begin{verbatim}
library(cluster)
library(flexclust)
library(MASS)
data(milk)
milk.dist <- dist(milk)
milk.pco <- cmdscale(milk.dist)
par(xpd = NA, bty = "n") ## let the labels run past the
## plotting region, and remove the box
plot(milk.pco, type = "n", main = "PCO representation")
text(milk.pco, row.names(milk.pco), cex = 0.5)
\end{verbatim}

Note that we've produced a blank plot but then drawn text on in the positions of the data with the row names.   Can you make sense of the resulting plot?

If you're very interested, you could even plot the cluster solutions you find when doing cluster analysis against the 2-dimensional representation of the distance matrix:

\begin{verbatim}
milk.hclust <- hclust(milk.dist)
milk.cut <- cutree(milk.hclust, 3)
plot(milk.pco, type = "n", main = "PCO representation")
text(milk.pco, row.names(milk.pco), cex = 0.5, col = milk.cut, pch = milk.cut)
\end{verbatim}

More importantly this week is to consider some diagnostics.

Lets look at the $n-1$ solution, using the delightfully titled \texttt{zapsmall} function to round our eigenvalues to 8 digits (computer arithmetic means we have a lot of negative eigenvalues in the 15th decimal place which we get rid of).

\begin{verbatim}
milk.pco.24 <- cmdscale(milk.dist, eig = TRUE, k = 24)
evals <- zapsmall(milk.pco.24$eig, digits = 8)
evals
\end{verabtim}

Now, if we want to examine the fit of a $q$ dimensional approximation we use $2 \times n \times \sum_{j=q+1}^{n-1} \lambda_{j}$ (note we are summing all the discarded eigenvalues) to give us:

\begin{verbatim}
2 * dim(milk)[1] * sum(milk.pco.24$eig[3:24])
\end{verbatim}

You can adjust this last line to see the SS for a 3 dimensional approximation (use \texttt{4:24} in the square brackets) and so on.

And now try to calculate the same thing directly from the projection and from the data.   We already know $\boldsymbol{Delta}$ (although we have to tell \textbf{R} we want a matrix and not a distance matrix.   We can get an estimate of $d$ by taking the distance matrix of the points in the $q$ dimensional approximation that interests us.   Finally, we just want the sum of the differences in the squared distances:

\begin{verbatim}
milk.pco.2 <- cmdscale(milk.dist, eig = TRUE, k = 2)
delta <- as.matrix(milk.dist)
d <- as.matrix(dist(milk.pco.2$points))
sum(as.vector(delta)^2 - as.vector(d)^2)
\end{verbatim}

The other obvious method is just to use the percentage of variance explained / discarded.   Consider dividing the cumulative sum of the eigenvalues by the sum of the eigenvalues:

\begin{verbatim}
cumsum(evals) / sum(evals)
\end{verbatim}

Careful examination should convince you that a 2-dimensional approximation might well be adequate.

\section{Sammon Mapping}

You could try ``Sammon'' mapping on the mvmclass data.   Assuming you've used \texttt{daisy} to generate a distance matrix from the class data (we did this in an earlier lab.)

\begin{verbatim}
mvmclass <- read.csv("class06.csv", row.names = 1)
mvmclass.dist <- daisy(mvmclass)
mvmclass.sammon <- sammon(mvmclass.dist)
\end{verbatim}

If you want to plot the representation, the material you need is under \verb+mvmclass.sammon$points+.

\begin{itemize}
\item How do you determine whether your sammon fit is a good one or not?
\end{itemize}

\section{Further analyses}

In case you wish to follow this from Johnson and Wichern's perspective, two datasets have been placed in the portal are very interesting: \texttt{utilities.csv} and \texttt{USUni.csv}.   The former we saw last time when we were considering clustering.   The latter considers a number of measures on 25 US Universities.

Another interesting analysis involves the Painters data (see \texttt{?painters} in \texttt{MASS}) - in particular you may like to see whether there any evidence that the first dimension corresponds with a time axis?

\section{Summary}

\fbox{\parbox[c]{0.9\textwidth}{\color{blue}
In some ways, it might have been better for this topic to follow cluster analysis.   It is really a different way of examining the relationships between individuals in a dataset.   By the end of this week, we should be able to:

\begin{itemize}
\item Computer and plot a p.c.o projection of a distance matrix
\item Understand and justify methods for determining whether our low dimension projection is adequate
\item Explain and interpret results, especially if those results are placed relative to a cluster analysis
\item Be able to relate p.c.o. to p.c.a.
\end{itemize}
}}


\section{Proof Details}
\label{sec:ibhc-proof}

\treerefinement*
\begin{proof}

Suppose, first, that $\tree$ is a refinement of $\tree'$. Pick any triplet $(\{a,b\},c) \in \Delta(\tree')$. Then there is a node in $\tree'$ whose descendants include $a,b$ but not $c$. By the definition of refinement, $\tree$ contains a node with the same descendants. Hence the constraint $(\{a,b\},c)$ holds for $\tree$ as well.

Conversely, say $\Delta(\tree') \subseteq \Delta(\tree)$. Pick any cluster $S'$ of $\tree'$; it consists of the descendants of some node in $\tree'$. Consider the set of all triplet constraints consisting of two nodes of $S'$ and one node outside $S'$. Since these constraints also hold for $\tree$, it follows that the lowest common ancestor of $S'$ in $\tree$ must have exactly $S'$ as its set of descendants. Thus $S'$ is also a cluster of $\tree$.
\end{proof}

\irreducible*
\begin{proof}
To prove irreducibility, we show that there is a non-zero probability
of moving from state $\tree$ to $\tree'$, both of which satisfy $C$. We
accomplish this by first defining a \emph{canonical tree} $\tree_C$ given a triplet
set $C$ and showing that we can reach $\tree_C$ from $\tree$ using
constrained-SPR moves. We then show that for every constrained-SPR move,
there exists an equivalent reverse move that undoes it with non-zero
probability.
This proves that that from $\tree_C$ we can reach $\tree'$,
creating a path from $\tree$ to $\tree_C$ to $\tree'$.

A binary tree $\tree$ can be entirely defined by the bipartitions
made over the data at each node.
Let $G_n$ be the Aho graph for node $n$.
For a binary tree that satisfies a set of triplets,
the split over the data at each node $n$ must
be a bipartition of the connected components of $G_n$.
We define a particular node to be in \emph{canonical form}
if either a) it is a leaf, or b) the bipartition over $G_n$
at that node can be written as
$(l, r)$, where $l$ exactly matches a single, particular connected component of
$G_n$, and $r$ is the rest of the connected components.
The particular component
$l$ is the connected component in $G_n$ with the minimum data index
inside it.
Note that we treat the children of nodes as unordered.
A canonical tree $\tree_C$ is one such that every node in the tree is
in canonical form.
To convert an arbitrary tree $\tree$ that satisfies $C$ into $\tree_C$, we first
convert the root node of $\tree$ into canonical form
using constrained-SPR moves.

Let $s$ be the root of $\tree$ and let $l$
be the set of points that ought to be in their own partition according
to $G_s$. In order for
$s$ not to be in canonical form, $l$ must be in a partition with
data from other connected components in $G_s$, which we will call $o$.
The bipartition if $s$ were in canonical form would be $(l, r)$ and
the current non-canonical bipartition can thus be written as $(l + o, r - o)$.

We first examine $t$, the child of the root that contains $l + o$.
In general, the data from $l$ and the data from $o$ could
be split over the children of $t$, so the partition at $t$
can be written as $(l_1 + o_1, l_2 + o_2)$ where $l = l_1 + l_2$
and $o = o_1 + o_2$. This is visualized in the first
tree of \autoref{fig:canonical}.
We first group the data from $l$ into their own ``pure'' subtree
of $t$ as follows.
Let $u$ be the root of the
lowest non-pure subtree of $t$ that has
data from $l$ in both of its children.
There exist two subtrees that are descendants of $u$
that contain data from $l$ (one on the left and one on the right).
Those two subtrees
must be pure, and furthermore, they are both free to
move within $u$ via constrained-SPR moves because
they are in different connected components in $G_u$.
Thus, we can perform a constrained-SPR move to merge these
two pure subtrees together into a larger pure subtree.
We can repeat this process for $t$ until all nodes
from $l$ are in their own pure subtree of $t$.
The partition of $t$ can thus be written as
$(l + o_1, o_2)$, since the pure subtree may be several
levels down from $t$. This grouping process is visualized in
\autoref{fig:canonicalgrouping} and the results can be
seen in the second
tree in \autoref{fig:canonical}.

\begin{figure}
    \centering
    \includegraphics[width=0.5\textwidth]{img/ibhc/CanonicalTreeGrouping}
    \caption{The process of grouping the data in $u$
    that belong to $l$ into their own pure subtree. $u$ is the
    lowest node of $t$ (see \autoref{fig:canonical}) that has data from $l$
    in both of its children.}
    \label{fig:canonicalgrouping}
\end{figure}

We now perform a constrained-SPR move to detach
the pure subtree of $l$ and regraft it to
the edge between $s$ and $t$. This is a permissible
move since $l$ is its own connected component in
$G_s$. We now have the third tree in \autoref{fig:supcanonical}.
We now perform a final constrained-SPR
move, moving the subtree of $o$ to the opposite
side of $s$, creating the proper canonical partition
of $(l, r)$.
To entirely convert $\tree$ into $\tree_C$, we need to recurse
and convert every node in $\tree$ into canonical form.

\begin{figure*}
    \centering
    \includegraphics[width=\textwidth]{img/ibhc/CanonicalTree}
    \caption{The process of converting $s$ into canonical form.
    We first group nodes from $l$ into their own pure subtree, then perform
    two constrained-SPR moves to put $s$ into canonical form.}
    \label{fig:supcanonical}
\end{figure*}

Every constrained-SPR move has an associated reverse constrained-SPR move that
performs the opposite transition.
The reverse constrained-SPR move selects the same subtree as the forward one
and prunes it, and just regrafts the subtree to its original location
before the forward move.
We know that this regraft has non-zero probability because the original tree
did not violate constraints.
Thus, since any arbitrary $\tree$ can be converted into $\tree_C$ and since each move
has a non-zero probability reverse move,
$\tree_C$ can be converted into an arbitrary tree $\tree'$ and we have a non-zero probability
path to convert $\tree$ into $\tree'$.

\end{proof}

\section{Additional Results}

\begin{figure*}[h]
    \begin{subfigure}[b]{0.5\textwidth}
        \centering
        \includegraphics[width=\textwidth]{img/ibhc/Zoo-result.png}
        \caption{Zoo}
        \label{fig:zoo-result}
    \end{subfigure}
    \begin{subfigure}[b]{0.5\textwidth}
        \centering
        \includegraphics[width=\textwidth]{img/ibhc/20-Newsgroups-result.png}
        \caption{20 Newsgroups}
        \label{fig:20news-result}
    \end{subfigure}
    \caption{The average of four runs of constrained-SPR samplers
        for the Zoo dataset and the 20 Newsgroups dataset, using 5 different querying schemes.
        A query was made every 100 iterations.}
    \label{fig:main-results2}
\end{figure*}


\documentclass{article}

\usepackage{amsmath} % for \over version of fractions
\usepackage{amsfonts} %for number system sets
\usepackage{xfrac} %for slanted fractions with sfrac
\usepackage[ampersand]{easylist}
\usepackage{graphicx}

\begin{document}

\title{Introductory Programming Exercises}
\author{Hamik Mukelyan}
\maketitle

\section*{Variables and basic control}
\subsection*{Exercise 1: Fibonacci numbers the easy way}
The \textbf{fibonacci numbers} are $1, 1, 2, 3, 5, 8, 13, 21, 34, \cdots, F_{n}$, where $F_{0} = F_{1} = 1$. There is a \textbf{closed form} solution for $F_{n}$, which means we can find $F_{n}$ by just plugging $n$ into a formula that we only have to evaluate once. Find this formula online and write a program to compute $F_{5}, F_{10},$ and $F_{20}$. The fibonacci numbers are integers, so make sure that your program rounds or \textbf{truncates} your numbers correctly. After you've gotten your program to work correctly, \textbf{refactor} it - i.e., improve it - so that you store $\sqrt{5}$ in a variable. That way you don't have to perform the slow square root operation three times - you only have to perform it once at the beginning of the program then reuse that result over and over.

\subsection*{Exercise 2: Fibonacci numbers with a loop}
Write a program that displays the fibonacci numbers from $F_{0}$ to $F_{n}$. One way to do this is with a loop that keeps a \textbf{counter variable}, $i$, that is \textbf{initialized} to $0$ and is incremented by 1 at the end of each \textbf{iteration} of the loop until it reaches $n$. The variable names $i, j,$ and $k$ are traditionally used as loop counter variables, with $i$ being used in outermost loops and $j$ being used in loops nested within those outermost loops. For each iteration of the loop you plug $i$ into the formula from exercise 1 and display the result with a print statement. Try it for $n = 10$ and see if the correct numbers show up!

Now let's refactor. The closed form solution for $F_{n}$ uses divisions and exponentiations in addition to square roots. We mitigated the slowness of square root operations by storing $\sqrt{5}$ in a variable, but we still do divisions and exponentiations, which are super slow compared to simple additions. The \textbf{recursive formula} for $F_{n}$ elegantly (and simply) describes the sequence in terms of the numbers that came before it: $F_{n} = F_{n - 1} + F_{n - 2}$. Along with the \textbf{base cases} $F_{0} = F_{1} = 0$, this formula fully defines the sequence. Switch your program over from using the closed form solution to the recursive one. Recursive solutions - that is, solutions that are stated in terms of \textit{smaller or easier versions of the same problem}, show up often in computer science because closed form solutions are often difficult or impossible to find.

\subsection*{Exercise 3: The radix}
When we write a number like 987, we actually mean $9 \cdot 10^{2} + 8 \cdot 10^{1} + 7 \cdot 10^{0}$. The number 10 is called the \textbf{base} or \textbf{radix} of the number. Most civilizations have used the radix 10 for counting, but some have used weird ones like 60 - we'll see in a moment how to think about numbers in a non-standard base. \textbf{Binary} (base 2), \textbf{octal} (base 8), and \textbf{hexadecimal} (base 16) are the most commonly used bases in programming other than decimal (base 10). If a base $B$ is less than 10, we use only the digits 0 to $B - 1$. For example, in octal we never use the digits 8 or 9. The number $123_{10} = 1 \cdot 8^{2} + 7 \cdot 8^{1} + 3 \cdot 8^{0} = 173_{8}$, where the subscripts are reminders about the bases of the numbers. When the base is 10 we usually omit the subscript by writing e.g. $123$ instead of $123_{10}$.

In bases greater than 10 we use letters starting from A for digits greater than 9. For example, in hexadecimal the digit A is one greater than 9, B is two greater than 9, and F is the biggest digit, which represents $15_{10}$. So counting from 1 to 32 in base 16, or hexadecimal, looks like 1, 2, 3, $\cdots,$ 9, A, B, C, D, E, F, 10, 11, 12, 13, $\cdots,$ 1E, 1F, 20. In binary we have only the digits 0 and 1 at our disposal. Any number that can be represented in decimal can be represented in any other base. Computers store numbers and operate on numbers in binary for the simple reason that it's easy to represent them with hardware: it's much easier for a switch to be simply on or off - representing 1 or 0 - than it is for it to be in one of ten states.

Binary has a special relationship with octal and hexadecimal, which is why they're used frequently in programming. Binary numbers are annoyingly long, so we can represent them more compactly by using their relationship with hex. Write out the first 16 numbers in binary. E.g., 0, 1, 10, 11, 100, 101, $\cdots$. These numbers match up, respectively, with the hex numbers 0, 1, 2, 3, $\cdots$, E, F. If you want to write a long binary number like 111100101010$_{2}$ compactly, you break it up into 4-digit chunks and use the equivalent hex number for each chunk. E.g., 1111 0010 1010$_{2}$ = F2A$_{16} = 15 \cdot 16^{2} + 2 \cdot 16^{1} + 10 \cdot 16^{0} = 3882$.

In many programming languages, you can represent a binary, octal, or hexadecimal number by prefixing any numerical literal with 0b, 0, or 0x. For instance, if you want to double-check that F2A$_{16}$ is actually 3882, just type 0xF2A into a REPL and see the resulting decimal value. Similarly, you can double-check that 173$_{8} = 123$ by typing 0123. These prefixed numerical literals can used anywhere you'd use a typical decimal literal in a program.

Here's the exercise: prompt the user for $n_{10}$ then display it in base 3. Then prompt the user for $m_{3}$ and display it in base 10. Before you try this problem, make sure you can convert a base 10 number into base 3 by hand and vice versa. Next make sure you can clearly articulate the steps you took to do so: in other words, devise an \textbf{algorithm}, which is a step-by-step problem solving procedure.
\end{document}


\chapter{Memory Complexity}

\section{Introduction}
When discussing memory complexity, need to consider both
\begin{enumerate}
\item \textbf{Heap}: the declared variables' size.
\item \textbf{Stack}: the recursive functions' call stack.
\end{enumerate}}
\subsection{Memory for Data Type}
The memory usage is based on Java.\\

\begin{tabular}{ll}
\hline\noalign{\smallskip}
\textbf{Type} & \textbf{Bytes} \\
\noalign{\smallskip}\hline\noalign{\smallskip}

boolean & 1 \\
byte & 1 \\
char & 2 \\
int & 4 \\
float & 4 \\
long & 8 \\
double & 8\\

\noalign{\smallskip}\hline\noalign{\smallskip}
\caption {for primitive types}
\end{tabular}

\begin{tabular}{ll}
\hline\noalign{\smallskip}
\textbf{Type} & \textbf{Bytes} \\
\noalign{\smallskip}\hline\noalign{\smallskip}

char[] & 2N+24 \\
int[] & 4N+24 \\
double[] & 8N+24 \\
T[] & 8N+24 \\

\noalign{\smallskip}\hline\noalign{\smallskip}
\caption{for one-dimensional arrays}
\end{tabular}

\begin{tabular}{ll}
\hline\noalign{\smallskip}
\textbf{Type} & \textbf{Bytes} \\
\noalign{\smallskip}\hline\noalign{\smallskip}

char[][] & 2MN \\
int[][] & 4MN \\
double[][] & 8MN \\

\noalign{\smallskip}\hline\noalign{\smallskip}
\caption{for two-dimensional arrays}
\end{tabular}

\begin{tabular}{ll}
\hline\noalign{\smallskip}
\textbf{Type} & \textbf{Bytes} \\
\noalign{\smallskip}\hline\noalign{\smallskip}

Object overhead & 16 \\
Reference & 8 \\
Padding & 8x \\

\noalign{\smallskip}\hline\noalign{\smallskip}
\caption{for objects}
\end{tabular}
\\
Notice:
\begin{enumerate}
\item The reference takes memory of 8 bytes.
\item Reference includes object reference and innner class reference.
\item T[] only considers reference; if consider underlying data structure, the memory is 8N+24+xN, where $x$ is the underlying data structure memory for each element.
\item Padding is to make the object memory size as a 8's multiple.
\end{enumerate}

\subsection{Example}
The generics is passed as Boolean:
\begin{java}
public class Box<T> {   // 16 (object overhead)
    private in N;       // 4 (int)
    private T[] items;  // 8 (reference to array)
                        // 8N+24 (array of Boolean references)
                        // 24N (underlying Boolean objects)
                        // 4 (padding to round up to a multiple)
}
\end{java}

Notice the multiple levels of references.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% PROBLEM 1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Problem 1}

A reactor contains 5\% (by atom) enriched uranium dioxide, which is 15\% of the entire core by volume.
\begin{enumerate}[a)]
\item Calculate the macroscopic cross section for this core if we were to treat it as a homogeneous volume.
\item If the reactor were a cube with a side length of 4 m and a beam of $10^{15}$ thermal neutrons were incident on one face of the cube, how many neutrons would we expect to make it through to the other side?
\end{enumerate}

\begin{table}[htbp]
	\centering
	\begin{tabular}{|c|c|c|}
			\hline
			Nucleus		& Thermal $\sigma_{\text{t}}$ (b) & Mass (g/mol) \\
			\hline
			$^{1}$H		&  20.8						& 1.008			\\
			$^{16}$O	&  3.5 						& 15.995		\\
			$^{235}$U	&  607.5 					& 235.044		\\
			$^{238}$U	&  11.8						& 238.050		\\
			\hline
			Compound	& \multicolumn{2}{|c|}{$\rho \left(\text{g/cm}^3\right)$}          \\
			\hline
			H$_2$O		& \multicolumn{2}{|c|}{1.0}      \\
			UO$_2$		& \multicolumn{2}{|c|}{10.4}	\\
			\hline
	\end{tabular}
	\label{tab:design-specs}
\end{table}


\subsection{Relational Databases}\label{relational_databases}
The group last year chose to implement the WordCount and Fuseki databases using PostgreSQL.
PostgreSQL is a relational database system\cite{knox2020}.

In the coming sections, we will describe how one can model and describe a relational database design, both mathematically and with a more graphic design.
After doing this, we will examine how operations on these models can be described using SQL and relational algebra.

\subsubsection*{Relational Model}
Relational database systems can be mathematically described using relations and sets, mapping a unique key to a tuple of information\cite{DBSBook}.
The values of the tuples contained within the relation can be described by the attributes of the relation and their corresponding domains\cite{KatjaFirstPP}.
The relations are often described using a \textit{relational schema}, denoting the name and domains of the attributes.

Equation \ref{eq:relational_schema} shows an example of a relation describing books as tuples of three text fields ($ISBN$, $author\_name$, and $title$ ) and a positive integer ($number\_of\_pages$).
The relation also denotes a super key for the relation. A super key is one or more attributes that can uniquely identify a tuple in a given relation.
A super key is called a \textit{candidate key} if removal of a single attribute from the key results in it longer uniquely identifying tuples.
Attributes describing the primary key are underlined.

\begin{equation} \label{eq:relational_schema}
    book(\underline{ISBN: text}, author\_name:text, title: text, number\_of\_pages:\mathbb{Z}^+)
\end{equation}
Super keys can be explained by $t_1 \in r\neq t_2 \in r \implies t_1.K \neq t_2.K$ \cite{DBSBook}, where $t_1, t_2$ are tuples contained within the relation $r$.
That is, no two tuples $t_1, t_2$ from relation $r$ have the same values if they do not have the same superkey $K$.
If the super key does not contain extraneous attributes, it is said to be \textit{minimal} \cite{DBSBook}.
We will use the term \textit{primary key} to denote a chosen super key of a relation.
When describing a database, it is often necessary to specify how various data are connected.
To do this, one can use \textit{foreign keys} to denote that tuples in $r_1$ are related to the tuples in $r_2$.

One could model the relationship between a book owner and a book using the relational schemas seen in equation \ref{eq:relational_schema} and \ref{eq:bookOwnerExample}.
There, primary keys from other relations are used to reference unique tuples. The $owns$ relation describes how relations $book$ and $book\_owner$ are connected.
\begin{equation}\label{eq:bookOwnerExample}
    \begin{split}
        owns(\underline{owner\_id \rightarrow book\_owner}, \underline{ISBN \rightarrow book}), \\
        book\_owner(\underline{owner\_id:\mathbb{Z}^+}, name:text,)
    \end{split}
\end{equation}

\subsubsection*{Evaluating a Database Design}
When evaluating the design of relational database schemas, we want to avoid redundant data duplication, loss by decomposition and change of dependencies\cite{DBSBook}.
One approach to ensure this is to use normalization theory\cite{DBSBook}.
This approach examines the \textit{functional dependencies} of the relations and evaluates them based on their \textit{normal form}.
When a relation is in a normal form, it adheres to certain design criteria.
These criteria are defined using functional dependencies.

A functional dependency describes the relationship between two sets of attributes.
Functional dependency between $\alpha$ and $\beta$, written as $\alpha \rightarrow \beta$, defines that a value for $\alpha$ is sufficient to identify unique values for $\beta$ \cite{DBSBook}.
In this case, we say that $\alpha$ is the determinant and $\beta$ is the dependant.
We say that $\beta$ is functionally determined by $\alpha$ if $\alpha \rightarrow \beta$.
It is logically implied that if $\alpha \rightarrow \beta$ and $\beta \rightarrow \gamma$, then $\gamma$ is functionally determined by $\alpha$.
If all attributes are functionally determined by $\alpha$, it is a super key\cite{DBSBook}.
The closure of functional dependency $\alpha$, denoted $\alpha^+$, describes all attributes that can be logically implied by functional dependencies having $\alpha$ as determinant\cite{DBSBook}.

Boyce-Codd Normal Form (BCNF) eliminates all redundancy that can be discovered using functional dependencies\cite{DBSBook}.
A relation is BCNF with respect to functional dependencies $F$ if, for all functional dependencies in $F^+$ of the form $\alpha \rightarrow \beta$ if one of the following holds:
$\alpha \rightarrow \beta$ is trivial ($\alpha \rightarrow \alpha$) or $\alpha$ is a super key for the relation.

Thus, if a relation $R$ is not in BCNF, there must be a functional dependency $\alpha \rightarrow \beta$ where $\alpha$ is not a super key.
We can split such relation $R$ into two relations $R1(\alpha \cup \beta)$ and $R2(R-(\beta-\alpha))$ that are both in BCNF.

Third Normal Form (TNF) prevents partial and transitive dependencies\cite{MontayaNormalForms}.
That is, third normal form prevents functional dependencies of the form seen in equation \ref{eq:trans}
and functional dependencies where non-prime attributes are functionally dependent on only a part of the candidate key.
\begin{equation}\label{eq:trans}
    {\alpha} \rightarrow{\beta}, {\beta \rightarrow \gamma} \implies \alpha \rightarrow \gamma
\end{equation}

A relation is in TNF with respect to functional dependencies $F$ if, for all dependencies in $F^+$ of the form $\alpha \rightarrow \beta$, one of the following holds:

$\alpha \rightarrow \beta$ is trivial, $\alpha$ is a super key, or each attribute $A \in \beta-\alpha$ is contained in a candidate key for $R$ \cite{DBSBook}.

Having established how one can evaluate a relational database design schema, we can proceed to discuss how a different model that can be used to represent the logic of a relational schema can make it easier to ensure that relations are of at least TNF.

sections/Chapter 3 Sprint 1/Relational_databases.tex


\subsection{Equations of motion}\label{sec-1}
The formulation of SELFE is based on classic expressions of mass and momentum conservation within a shallow fluid,
as well as the transport equations for salt and heat. The three dimensional expressions of these laws of motion
involve much less reduction and averaging than one or two-dimensional models. However, 3D estuary-scale models
do invoke a standard set of assumptions that will be discussed in section \ref{sec-assump}.

The SELFE variant used in this equation is defined in a Cartesian (ie map projection) frame with
bathymetric depth $h$ and free surface $\eta$ defined relative to a fixed datum (NAVD 88 in our case) as shown
in Figure \ref{fig:depths}. The total depth $H$ is the sum of bathymetric depth and free surface elevation.

The 3D continuity, depth-integrated mass conservation, and horizontal momentum conservation equations are given by:
\begin{align}
  %& \text{3D Continuity}& \nonumber \\
  \nabla \cdot \bs{u} &= 0 \label{3Dcon}&\\
  %& \text{Depth-integrated Continuity}& \nonumber \\
  \eta_t+\nabla \cdot \int_{-h}^\eta \bs{u} & = 0 \label{cont1} &\\
  %& \text{Momentum}& \nonumber \\
  \frac{D\bs{u}}{Dt} &= \explain{ -f\bs{k} \times \bs{u} }{\text{Coriolis}}
	                      \; \explain{-\frac{1}{\rho_0} \nabla p_A}{\text{Atmos. pressure}}
												\; \explain{- \frac{g}{\rho_0} \int_z^\eta \nabla \rho d \xi}{\text{Baroclinic}}
	                      \; \; \; \explain{-g\nabla \eta}{\text{Gravity wave}} & \nonumber \\
	                      \; \; \; &\explain{+ \nabla \cdot (\mu \nabla \bs{u})}{\text{Horizontal diffusion}}
												\; \; \; \; \explain{+\frac{\pd}{\pd z}(\nu \frac{\pd \bs{u}}{\pd z})}{\text{Vertical diffusion}} \label{mom1}
	\end{align}
	with wind and bed stress boundary conditions at the surface and bottom of the water column:
	\begin{align}
	\nu \frac{\pd \bs{u}}{\pd z} &= \bs{\tau}_w \mbox{ at } z=\eta &\\
  \nu \frac{\pd \bs{u}}{\pd z} &= \chi \bs{u}_b \mbox{ at } z=-h &
\end{align}
where the variables and parameters are:
\begin{align*}
&\eta(x,y,t)  &\text{free surface elevation (m)} &\\
&\bs{u}(x,y,z,t)  &\text{horizontal velocity(m/s)}  &\\
&h(x,y)  &\text{bathymetric depth(m)} &\\
&w  & \text{vertical velocity (m/s)}  &\\
&f     & \text{Coriolis factor (s\textsuperscript{-1} )} & \\
&g     & \text{gravity (m/s\textsuperscript{2})} & \\
&\rho  & \text{water density (kg/m\textsuperscript{3})} & \\
&p_A(x,y, t) & \text{atmospheric pressure at the free surface (N m\textsuperscript{2})} & \\
&\nu    & \text{vertical eddy viscosity (m\textsuperscript{2}/s)} & \\
&\mu    & \text{horizontal eddy viscosity (m\textsuperscript{2}/s) }\\
&\kappa & \text{vertical eddy diffusivity, for salt and heat (m\textsuperscript{2}/s)} & \\
\end{align*}

and the labeled processes in the momentum equations include:
\begin{description}
    \item{Coriolis} an apparent force that results from writing equations on a rotating system (the earth) in a projected (x,y) coordinate system. The Coriolis force is most important in the ocean and near coast, less so in estuary or riverine systems.
	  \item{Atmospheric pressure} This is the horizontal variation of pressure above the water.
		\item{Gravity wave} Pressure differences due to slope in the water surface are the main force that causes tide and flood propagation.
	  \item{Baroclinic forcing} This is the driving mechanism for density-driven (exchange) flow, as
commonly found in estuaries, of which salinity intrusion is an example. Another example is gravity (dense) underflow. Due to this force, the horizontal gradient of the density field will initiate 3D flow that moves denser water under the lighter water and will lead to a two-layer flow structure commonly found in stratified estuaries.
\end{description}

\subsubsection{Assumptions}\label{sec-assump}
The Reynolds averaged shallow water equations include some simplifications of the raw (Navier-Stokes) equations in order
to eliminate small scale terms and make the equations more tractable.
\begin{itemize}
\item {\em Free surface} The free surface is incorporated into the equations and results from the
divergence of the horizontal flows. This simplification avoids a complex moving boundary problem at the water-atmosphere interface.
\item {\em Reynolds averaging} SELFE solves for velocity in a time-mean sense, averaged over small scale turbulence. Mixing induced by turbulent fluctuations
is accounted for by relating the fluctuations to mean flow properties using a set of relations called the {\em turbulence closure}.
\item {\em Boussinesq approximation} The Boussinesq approximation simplifies terms in the governing equations by considering the the effect of density differences due to soluble tracers (salt, temperature and sediment) only
in a single buoyancy term.
\item {\em Hydrostaticity} Most models in SELFE's class have both a hydrostatic and non-hydrostatic option. In hydrostatic mode, the model only considers pressure forces on a parcel of water arising only from the weight of the water and atmospheric pressure above, neglecting vertical momentum.
The assumption can be relaxed by enabling non-hydrostatic pressure; however, due to the required resolution and computation time,
this feature is not used at system scales in any model in the Bay-Delta.
\item In hydrostatic mode, volume conservation is first enforced over the water column using horizontal velocities into and out of the water column. Vertical velocity is implied in the equations and later inferred in a separate step that invokes the 3D continuity equation.
\end{itemize}

\subsubsection{Roughness and friction}
In SELFE, resistance is introduced into the water column through the combination of the bottom stress (drag)
boundary condition and vertical turbulent momentum diffusion. The mechanism is less direct than the direct
body force used in 1-D and 2-D models. [how does SELFE2D fit here?]

There are two options in the stipulation of the drag coefficient for each spatial location:
\begin{itemize}
	\item $C_d$ may be given directly
	\item $C_d$ may be inferred using an analytical description (logarithmic decay) of velocity in a viscous boundary layer. The parametrized form leads to a formula for $C_d$:
	$C_d=[\frac{1}{\kappa_0}log(\frac{\delta}{z_0})]^{-2}$, where $\kappa_0$ is von Karman's constant, and $\delta$ is the thickness
of the bottom layer. In this case the calibration parameter is roughness ($z_0$) rather than drag.
\end{itemize}

Once drag is characterized at the bed, it is mixed vertically up the water column through a turbulent eddy diffusion, which is
labeled "vertical viscosity" in equation (\ref{mom1}) and shown visually in figure ****. The mixing of slower water into
faster water has the effect of slowing the faster water down. Near the bed, this viscous process is the analog of
friction in a lower dimension model. Higher in the water column, the effect of eddy diffusion and location of the velocity
maximum are harder to predict.

In terms of calibration the above formulation leads to two sets of parameters that need to be estimated. The first is the
drag coefficient, which may take the form of $C_d$ or roughness $z_0$. In addition, an eddy diffusion coefficient
has to be supplied -- this is not stipulated directly but rather emerges from the choice of turbulence closure, described in Section \ref{sec-tur}.

\subsection{Turbulence closure}\label{sec-tur}
The final component of vertical mixing is the turbulence closure, which is used to obtain an eddy viscosity coefficient
for the momentum equations and a vertical eddy diffusivity for the transport equations.

We use the vertical component of Umlauf and Burchard's \cite{Umlauf2003}  generic length-scale model, a separate differential
equation which is integrated "off-line" of the other equations based on values from the previous time step.
The model is as follows:
\beqa
  \frac{D k}{D t}&=&\frac{\pd }{\pd z}\left( \nu_k^\Psi \frac{\pd k}{\pd z} \right)
  +\nu_t M^2+\nu_t^\theta N^2 -\epsilon \\
  \frac{D \Psi}{D t}&=& \frac{\pd }{\pd z}\left( \nu_\Psi \frac{\pd \Psi}{\pd z} \right)
    +\frac{\Psi}{k}(c_{\Psi 1}\nu_tM^2+c_{\Psi 3}\nu_t^\theta N^2-c_{\Psi 2}\epsilon F_{wall})
\eeqa
with natural b.c.:
\beq
   \left\{ \begin{array}{ll}
       \nu_k^\Psi \frac{\pd k}{\pd z} &=0, \mbox{ at } z=-h, \mbox{ or } \eta \\
       \nu_\Psi\frac{\pd \Psi}{\pd z} &= \kappa_0 n\nu_\Psi\frac{\Psi}{l} , \mbox{ at } z=-h \\
       \nu_\Psi\frac{\pd \Psi}{\pd z} &= -\kappa_0 n\nu_\Psi\frac{\Psi}{l} , \mbox{ at } z=\eta \\
           \end{array}
   \right.  \label{tur1}
\eeq
and essential b.c.:
\beq
   \left\{ \begin{array}{ll}
       k&=(c_\mu^0)^{-2} \nu|\frac{\pd \bs{u}}{\pd z}|, \mbox{ at } z=-h, \mbox{ or } \eta\\
       l&=\kappa_0 \D \\
       \Psi &= (c_\mu^0)^pk^m(\kappa_0 \D)^n
           \end{array}
   \right.  \label{tur2}
\eeq
where $k$ is the turbulent kinetic energy, $l$ is the mixing length, $c_{\Psi *}$ are some constants and $\Psi=(c_\mu^0)^pk^m l^n$ is a generic
length-scale variable, and $\D$ is the distance from 'walls' (i.e. surface and bottom).

The turbulence production and dissipation terms are:
\beqa
  M^2&=&\left( \frac{\pd u}{\pd z}\right)^2+\left( \frac{\pd v}{\pd z}\right)^2 \\
  N^2 &=&-\frac{g}{\rho_0}\frac{\pd \rho}{\pd z} \\
  \epsilon &=& (c_\mu^0)^3k^{1.5} l^{-1}
\eeqa

In the code, the natural b.c. is applied first (see the FEM formulation below), and the essential b.c. is then used to overwrite the
boundary values of the unknown, as in the GOTM code. SELFE has been directly coupled to the GOTM model to take advantage of the latter.

Once the equations for turbulent kinetic energy mixing length have been updated, the
values of eddy viscosity and eddy diffusivity used in the momentum and transport equations are obtained from the relations:
\beqa
  \nu &=& \sqrt{2k}s_m l \\
  \kappa &=& \sqrt{2k}s_h l
\eeqa
where $s_m$ and $s_h$ are stability functions, such as those given by \citet{Kantha94}

With this step, the specification of turbulent mixing (and indirectly the mechanism of friction) is complete.


% CReviewofCalculus.tex
% Fund Science! & Help Ernest finish his Physics Research! : quantum super-A-polynomials - a thesis by Ernest Yeung
%
% http://igg.me/at/ernestyalumni2014
%
% Facebook     : ernestyalumni
% github       : ernestyalumni
% gmail        : ernestyalumni
% google       : ernestyalumni
% linkedin     : ernestyalumni
% tumblr       : ernestyalumni
% twitter      : ernestyalumni
% youtube      : ernestyalumni
% indiegogo    : ernestyalumni
%
% Ernest Yeung was supported by Mr. and Mrs. C.W. Yeung, Prof. Robert A. Rosenstone, Michael Drown, Arvid Kingl, Mr. and Mrs. Valerie Cheng, and the Foundation for Polish Sciences, Warsaw University.

\section{Review of Calculus}

\subsection*{Total and Partial Derivatives}

\begin{proposition}[C.3] (The Chain Rule for Total Derivatives)

Suppose $V,W,X$ finite-dim. vector spaces

open $\begin{aligned}
  & \quad \\
  & U \subseteq V \\
  & \widetilde{U} \subseteq W \end{aligned}$

maps $\begin{aligned}
  & \quad \\
  & F : U \to \widetilde{U} \\
  & G: \widetilde{U} \to X \end{aligned}$

if $F$ diff. at $a\in U$, $G$ diff. at $F(a) \in \widetilde{U}$, \\
\phantom{ \quad } then $GF$ diff. at $a$
\[
D(GF)(a) = DG(F(a)) \circ DF(a)
\]
\end{proposition}

\begin{proposition}[C.4] Suppose $\begin{aligned}
    & \quad \\
    & U \subseteq \mathbb{R}^n \\
    & V\subseteq \mathbb{R}^m \end{aligned}$, \quad \, $F: U \to V$ diffeomorphism \\
Then $m=n$ and \\
\phantom{ \quad } $\forall \, a \in U$, $DF(a)$ invertible, with
\[
DF(a)^{-1} = D(F^{-1})(F(a))
\]
\end{proposition}

\begin{proof}
$F^{-1}F = 1_U$  \\
Chain rule implies $\forall \, a \in U$,

\begin{equation}
  1_{\mathbb{R}^n} = D(1_U)(a) = D(F^{-1}F)(a) = DF^{-1}(F(a))DF(a) \quad \quad \, (C.5)
\end{equation}

Similarly $FF^{-1}=1_V$ implies
\[
DF(a)DF^{-1}(F(a)) = 1_{\mathbb{R}^m}
\]
so thus
\[
DF(a)^{-1}=D(F^{-1})(F(a))
\]
\end{proof}


\subsection{Constraint Satisfaction Problem}

\subsubsection{Introduction}

For active learning, only need to update covariance matrix? just needed to select one with highest variance

Active learning is greedy algorithm to reduce entropy

As we get more info, our posterior becomes our new prior

If we can pick observations to use  to update model, can use those with biggest variance

Can be useful if getting y is expensive. requires experiement etc

4 steps:

\begin{itemize}
\item Form \(p(y_0|x_0,y,X)\) for all unmeasured \(x_0\).
\item Choose \(x_0\) with the largest \(\sigma_0^2\) and observe \(y_0\)
\item Updated the parameters with this.
\item repeat
\end{itemize}

\(\sigma_0^2 =\sigma^2+x_0^T\Sigma x_0\)

Updating Sigma and mu for bayesian linear:

\(\Sigma = (\lambda I + \sigma^{-2}(x_0x_o^T+\sum_{i=1}^nx_ix_i^T))^{-1}\)

\(\mu = (\lambda \sigma^2I+ x_0x_0^T+\sum_{i=1}^nx_ix_i^T)^{-1}(x_0y_0+\sum_{i=1}^nx_iy_i)\)

Once we have  an \(x_0\) we can easily get \(\mu_0\) by calculating \(x_0^T\mu\). Multiplying by the mean weights.

We can also get the variance of the estimate:

\(\sigma^2_0=\sigma^2+x_0^T\Sigma \)


\section{Introduction}
Fluid viscosity is one of the most important properties of fluids, determining
the fluid’s flow.
Motion of an object in a fluid is hindered by a drag force acting in the
direction opposite to the direction of motion, i.e. opposite to the object’s
velocity.

The magnitude of the drag force is related to the shape and speed of the object
as well as to the internal friction in the fluid.
The method used in this lab is known as Stokes’ method and it a common and
simple method for characterizing transparent or translucent fluids with high
viscosity.

% theoretical
Motion of an object in a fluid is hindered by a drag force acting in the
direction opposite to the direction of motion, i.e. opposite to the object’s
velocity.
The magnitude of the drag force is related to the shape and speed of the object
as well as to the internal friction in the fluid.

This internal friction can be quantified by a number known as the viscosity
coefficient $\mu$.
For a spherical object with radius R moving at speed v in an infinite volume of
a liquid, the magnitude of the drag force is usually modeled as linear in the
speed
$$  F_1 = 6 \pi \mu v R  $$

When a spherical object falls vertically downwards in a fluid, it is being acted
upon by the following three forces:
The viscous force \emph{$F_1$} and the buoyancy force \emph{$F_2$} both act
upwards, and the weight of the object \emph{$F_3$} is directed downwards.
The magnitude of the buoyancy force is
$$  F_2 = \frac{4}{3} \pi R^3 \rho_1 g $$
where $\rho_1$ is the density of the fluid and $g$ is the acceleration due to
gravity. The weight of the object
$$  F_3 = \frac{4}{3} \pi R^3 \rho_2 g $$
with $\rho_2$ being the density of the object. After some time, the three forces
will balance each other
$$  F_1 + F_2 = F_3  $$
so that the net force on the object will be zero and from that instant on, the
object will be moving with constant speed $v_t$, known as the terminal speed.
Applying the condition, we can find
$$  \mu = \frac{2}{9} g R^2 \frac{\rho_2 - \rho_1 }{v_t}  $$
Therefore, the fluid viscosity can be found by measuring the terminal speed.
Taking into account that the motion with terminal speed is a motion with
constant velocity, the equation can be rewritten as
$$  \mu = \frac{2}{9} g R^2 \frac{( \rho_2 - \rho_1 ) t  }{s}  $$
where $s$ is the distance traveled in time $t$ after reaching the terminal
speed.

Since the volume of the fluid used in the measurement is not infinite, the
results are affected by some boundary effects due to the presence of the
container.

Therefore, the equation should be modified, and the formula for the
corrected magnitude of the viscous force for a infinitely long cylindrical
container with radius $R_c$ is
$$  F_1 = 6 \pi \mu v R (1 + 2.4 \frac{R}{R_c})  $$
Consequently,
$$ \mu = \frac{2}{9} g R^2 \frac{( \rho_2 - \rho_1 ) t  }{s} (1 + 2.4
\frac{R}{R_c})  $$
Since the length $L$ of the container is limited, there may be further
corrections introduced, depending on the ratio on $\frac{R_c}{L}$.


\input{../header_class}

%---------- start document ---------- %
 \section{imaginary -- complex numbers and its functions}\linkedzero{imaginary}
The module {\tt imaginary} provides complex numbers. The functions provided are mainly corresponding to the \linklibrary{cmath} standard module.

 \begin{itemize}
   \item {\bf Classes}
   \begin{itemize}
     \item \linkingone{imaginary}{ComplexField}
     \item \linkingone{imaginary}{Complex}
     \item \negok \linkingone{imaginary}{ExponentialPowerSeries}
     \item \negok \linkingone{imaginary}{AbsoluteError}
     \item \negok \linkingone{imaginary}{RelativeError}
   \end{itemize}
   \item {\bf Functions}
     \begin{itemize}
       \item \linkingone{imaginary}{exp}
       \item \linkingone{imaginary}{expi}
       \item \linkingone{imaginary}{log}
       \item \linkingone{imaginary}{sin}
       \item \linkingone{imaginary}{cos}
       \item \linkingone{imaginary}{tan}
       \item \linkingone{imaginary}{sinh}
       \item \linkingone{imaginary}{cosh}
       \item \linkingone{imaginary}{tanh}
       \item \linkingone{imaginary}{atanh}
       \item \linkingone{imaginary}{sqrt}

     \end{itemize}
 \end{itemize}

This module also provides following constants:
\begin{description}
   \item[e]\linkedone{imaginary}{e}:\\
     This constant is obsolete (Ver 1.1.0).
   \item[pi]\linkedone{imaginary}{pi}:\\
     This constant is obsolete (Ver 1.1.0).
   \item[j]\linkedone{imaginary}{j}:\\
     \param{j} is the imaginary unit.
   \item[theComplexField]\linkedone{imaginary}{theComplexField}:\\
     \param{theComplexField} is the instance of \linkingone{imaginary}{ComplexField}.
 \end{description}

\C
 \subsection{ComplexField -- field of complex numbers}\linkedone{imaginary}{ComplexField}
 The class is for the field of complex numbers. The class has the single instance \linkingone{imaginary}{theComplexField}.

 This class is a subclass of \linkingone{ring}{Field}.

  \initialize
  \func{ComplexField}{}{\out{ComplexField}}\\
  \spacing
  % document of basic document
  \quad Create an instance of ComplexField.
  % added document
  You may not want to create an instance, since there is already \linkingone{imaginary}{theComplexField}.
  % \spacing
  % input, output document
  %See \linkingone{module}{point} for \param{point}.
  \begin{at}
    \item[zero]\linkedtwo{imaginary}{ComplexField}{zero}:\\ It expresses The additive unit 0. (read only)
    \item[one]\linkedtwo{imaginary}{ComplexField}{one}:\\ It expresses The multiplicative unit 1. (read only)
  \end{at}
  \begin{op}
  %  \verb|+| & Vector sum.\\
  %  \verb|-| & Vector subtraction.\\
  %  \verb|*| & Scalar multiplication.\\
  %  \verb|//| & Scalar division.\\
  %  \verb|-(unary)| & element negation.\\
  %  \verb|==| & equality or not.\\
  %  \verb|!=| & inequality or not.\\
  %  \verb+V[i]+ & Return the coefficient of i-th element of Vector.\\
  %  \verb+V[i] = c+ & Replace the coefficient of i-th element of Vector by c.\\
  %  \verb|len| & return length of \linkingtwo{vector}{Vector}{compo}.\\
    \verb|in| & membership test; return whether an element is in or not.\\
    \verb|repr| & return representation string.\\
    \verb|str| & return string.\\
  \end{op}
%\begin{ex}
%>>> A = vector.Vector([1,2])
%>>> A
%Vector([1, 2])
%>>> A.compo
%[1, 2]
%>>>
%\end{ex}%Don't indent!
  \method
   \subsubsection{createElement -- create Imaginary object}\linkedtwo{imaginary}{ComplexField}{createElement}
    \func{createElement}{\param{self},\ \hiki{seed}{integer}}{\out{Integer}}\\
    \spacing
    % document of basic document
    \quad Return a Complex object with \param{seed}.
    \spacing
    % added document
    %\quad \negok Note that this function returns integer only.\\
    \spacing
    % input, output document
    \quad \param{seed} must be complex or numbers having embedding to complex.\\
%
%
  \subsubsection{getCharacteristic -- get characteristic}\linkedtwo{imaginary}{ComplexField}{getCharacteristic}
   \func{getCharacteristic}{\param{self}}{\out{integer}}\\
   \spacing
   % document of basic document
   \quad Return the characteristic, zero.
   %\spacing
   % added document
   %\quad \negok Note that this function returns integer only.\\
   %\spacing
   % input, output document
   %\quad \param{a} must be int, long or rational.Integer.\\
%
  \subsubsection{issubring -- subring test}\linkedtwo{imaginary}{ComplexField}{issubring}
   \func{issubring}{\param{self},\ \hiki{aRing}{\linkingone{ring}{Ring}}}{\out{bool}}\\
   \spacing
   % document of basic document
   \quad Report whether another ring contains the complex field as subring.
   \spacing
   % added document
   %\quad \negok Note that this function returns integer only.\\
   %\spacing
   % input, output document
   %\quad if \param{as\_column} is True, try to create column matrix.\\
%
  \subsubsection{issuperring -- superring test}\linkedtwo{imaginary}{ComplexField}{issuperring}
   \func{issuperring}{\param{self},\ \hiki{aRing}{\linkingone{ring}{Ring}}}{\out{bool}}\\
   \spacing
   % document of basic document
   \quad Report whether the complex field contains another ring as subring.
   \spacing
   % added document
   %\quad \negok Note that this function returns integer only.\\
   %\spacing
   % input, output document
   %\quad if \param{as\_column} is True, try to create column matrix.\\
%\begin{ex}
%>>> A = module.HogeClass((1,2))
%>>> A.hogemethod1(2)
%(2, 4)
%>>>
%\end{ex}%Don't indent!

\C
 \subsection{Complex -- a complex number}\linkedone{imaginary}{Complex}
 Complex is a class of complex number. Each instance has a coupled numbers; real and imaginary part of the number.

 This class is a subclass of \linkingone{ring}{FieldElement}.

 All implemented operators in this class are delegated to complex type.
  \initialize
  \func{Complex}
       {\hiki{re}{number}
        \hikiopt{im}{number}{0}
       }
       {\out{Imaginary}}\\
  \spacing
  % document of basic document
  \quad Create a complex number.
  % added document
  \spacing
  % input, output document
  \param{re} can be either real or complex number. If \param{re} is real and \param{im} is not given, then its imaginary part is zero.
  \begin{at}
    \item[real]\linkedtwo{imaginary}{Complex}{real}:\\ It expresses the real part of complex number.
    \item[imag]\linkedtwo{imaginary}{Complex}{imag}:\\ It expresses the imaginary part of complex number.
  \end{at}
  %\begin{op}
  %  \verb|+| & Vector sum.\\
  %  \verb|-| & Vector subtraction.\\
  %  \verb|*| & Scalar multiplication.\\
  %  \verb|//| & Scalar division.\\
  %  \verb|-(unary)| & element negation.\\
  %  \verb|==| & equality or not.\\
  %  \verb|!=| & inequality or not.\\
  %  \verb+V[i]+ & Return the coefficient of i-th element of Vector.\\
  %  \verb+V[i] = c+ & Replace the coefficient of i-th element of Vector by c.\\
  %  \verb|len| & return length of \linkingtwo{vector}{Vector}{compo}.\\
  %  \verb|repr| & return representation string.\\
  %  \verb|str| & return string of \linkingtwo{vector}{Vector}{compo}.\\
  %\end{op}
%\begin{ex}
%>>> A = vector.Vector([1,2])
%>>> A
%Vector([1, 2])
%>>> A.compo
%[1, 2]
%>>>
%\end{ex}%Don't indent!
  \method
  \subsubsection{getRing -- get ring object}\linkedtwo{imaginary}{Complex}{getRing}
   \func{getRing}{\param{self}}{\out{ComplexField}}\\
   \spacing
   % document of basic document
   \quad Return the complex field instance.
   %\spacing
   % added document
   %\quad \negok Note that this function returns integer only.\\
   %\spacing
   % input, output document
   %\quad \param{a} must be int, long or rational.Integer.\\
%
  \subsubsection{arg -- argument of complex}\linkedtwo{imaginary}{Complex}{arg}
   \func{arg}{\param{self}}{\out{radian}}\\
   \spacing
   % document of basic document
   \quad Return the angle between the x-axis and the number in the Gaussian plane.
   %\spacing
   % added document
   %\quad \negok Note that this function returns integer only.\\
   %\spacing
   % input, output document
   \quad \out{radian} must be Float.\\
%
  \subsubsection{conjugate -- complex conjugate}\linkedtwo{imaginary}{Complex}{conjugate}
   \func{conjugate}{\param{self}}{\out{Complex}}\\
   \spacing
   % document of basic document
   \quad Return the complex conjugate of the number.
   %\spacing
   % added document
   %\quad \negok Note that this function returns integer only.\\
   %\spacing
   % input, output document
   %\quad \out{radian} must be Float.\\
%
  \subsubsection{copy -- copied number}\linkedtwo{imaginary}{Complex}{copy}
   \func{copy}{\param{self}}{\out{Complex}}\\
   \spacing
   % document of basic document
   \quad Return the copy of the number itself.
   %\spacing
   % added document
   %\quad \negok Note that this function returns integer only.\\
   %\spacing
   % input, output document
   %\quad \out{radian} must be Float.\\
%
  \subsubsection{inverse -- complex inverse}\linkedtwo{imaginary}{Complex}{inverse}
   \func{inverse}{\param{self}}{\out{Complex}}\\
   \spacing
   % document of basic document
   \quad Return the inverse of the number.
   \spacing
   % added document
   %\quad \negok Note that this function returns integer only.\\
   %\spacing
   % input, output document
   \quad If the number is zero, ZeroDivisionError is raised.
%
%\begin{ex}
%>>> A = module.HogeClass((1,2))
%>>> A.hogemethod1(2)
%(2, 4)
%>>>
%\end{ex}%Don't indent!
\C
 \subsection{ExponentialPowerSeries -- exponential power series}\linkedone{imaginary}{ExponentialPowerSeries}
  This class is obsolete (Ver 1.1.0).

 \subsection{AbsoluteError -- absolute error}\linkedone{imaginary}{AbsoluteError}
  This class is obsolete (Ver 1.1.0).

 \subsection{RelativeError -- relative error}\linkedone{imaginary}{RelativeError}
  This class is obsolete (Ver 1.1.0).

  \subsection{exp(function) -- exponential value}\linkedone{imaginary}{exp}
   This function is obsolete (Ver 1.1.0).

  \subsection{expi(function) -- imaginary exponential value}\linkedone{imaginary}{expi}
   This function is obsolete (Ver 1.1.0).

  \subsection{log(function) -- logarithm}\linkedone{imaginary}{log}
   This function is obsolete (Ver 1.1.0).

  \subsection{sin(function) -- sine function}\linkedone{imaginary}{sin}
   This function is obsolete (Ver 1.1.0).

  \subsection{cos(function) -- cosine function}\linkedone{imaginary}{cos}
   This function is obsolete (Ver 1.1.0).

  \subsection{tan(function) -- tangent function}\linkedone{imaginary}{tan}
   This function is obsolete (Ver 1.1.0).

  \subsection{sinh(function) -- hyperbolic sine function}\linkedone{imaginary}{sinh}
   This function is obsolete (Ver 1.1.0).

  \subsection{cosh(function) -- hyperbolic cosine function}\linkedone{imaginary}{cosh}
   This function is obsolete (Ver 1.1.0).

  \subsection{tanh(function) -- hyperbolic tangent function}\linkedone{imaginary}{tanh}
   This function is obsolete (Ver 1.1.0).

  \subsection{atanh(function) -- hyperbolic arc tangent function}\linkedone{imaginary}{atanh}
   This function is obsolete (Ver 1.1.0).

  \subsection{sqrt(function) -- square root}\linkedone{imaginary}{sqrt}
   This function is obsolete (Ver 1.1.0).

%
% \begin{ex}
% >>> A = vector.Vector([1,2,3])
% >>> B = vector.Vector([2,1,0])
% >>> vector.innerProduct(A,B)
% 4
% >>>
% \end{ex}%Don't indent!
\C

%---------- end document ---------- %

\input{../footer}


% !TeX root = constructions.tex

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{Lill's Method for Finding Roots}\label{c.lill}

\section{Magic}\label{s.magic}

Construct a path consisting of four line segments $\{a_3,a_2,a_1,a_0\}$ of lengths:
\[
\{a_3=1,a_2=6,a_1=11,a_0=6\}\,,
\]
starting from the origin, first along the positive direciton of the $x$-axis and turning $90^\circ$ counterclockwise between segments. Construct a second path as follows: draw a line from the origin at an angle of $63.4^\circ$ and mark its intersection with $a_2$ by $P$. Turn left $90^\circ$, draw a line and and mark its intersection with $a_1$ by $Q$. Turn left $90^\circ$ once again, draw a line and note that it intersects the end of the first path at $(-10,0)$. %See Figure~\ref{fig.magic}.

\begin{center}
\begin{tikzpicture}[scale=1.15]
% Draw help lines and axes
\draw[step=10mm,white!50!black] (-11,-1) grid (2,7);
\draw[thick] (-11,0) -- (2,0);
\draw[thick] (0,-1) -- (0,7);
\foreach \x in {-10,...,2}
  \node at (\x-.3,-.2) {\sm{\x}};
\foreach \y in {1,...,7}
  \node at (-.2,\y-.3) {\sm{\y}};

% Draw first path
\coordinate (A) at (0,0);
\coordinate (B) at (1,0);
\coordinate (C) at (1,6);
\coordinate (D) at (-10,6);
\coordinate (E) at (-10,0);
\foreach \x in {A,B,C,D,E}
  \fill (\x) circle(1.5pt);
\draw[very thick] (A) --
  node[below,xshift=1pt,yshift=-10pt] {$a_3=1$} (B);
\draw[very thick,name path=bc] (B) --
  node[right,yshift=6pt] {$a_2=6$} (C);
\draw[very thick,name path=cd] (C) --
  node[above,xshift=4pt] {$a_1=11$}(D);
\draw[very thick,name path=de] (D) --
  node[left,xshift=3pt,yshift=6pt] {$a_0=6$}(E);

% Draw first segment of second path
\path[name path=a2] (A) -- +(63.4:4);
\path [name intersections = {of = a2 and bc, by = {A2}}];
\fill (A2) circle(1.5pt) node[above right] {$P$};
\draw[very thick,dashed] (A) -- (A2);
\draw ($(A) + (14pt,0)$)
  arc [start angle=0, end angle = 63.4, radius=14pt];
\node[above right,xshift=35pt,yshift=2pt] at (A) {$63.4^\circ$};
\draw[->] ($(A)+(32pt,8pt)$) -- +(-18pt,0);
\draw[rotate=153.4] (A2) rectangle +(10pt,10pt);

% Draw second segment of second path
\path[name path=b2] (A2) -- +(153.4:10);
\path [name intersections = {of = b2 and cd, by = {B2}}];
\fill (B2) circle(1.5pt) node[above left] {$Q$};
\draw[very thick,dashed] (A2) -- (B2);
\draw[rotate=243.4] (B2) rectangle +(10pt,10pt);

% Draw third segment of second path%
\draw[very thick,dashed] (B2) -- (E);
\end{tikzpicture}
\end{center}
Let $p(x)=a_3x^3+a_2x^2+a_1x+a_0=x^3+6x^2+11x+6$. Compute $\tan 63.4^\circ=2$, the tangent of the angle at the start of the second path. Then:
\[
p(-\tan 63.4^\circ)=(-2)^3+6(-2)^2+11(-2)+6=0\,.
\]
Congratulations! You have found a root of the cubic polynomial $x^3+6x^2+11x+6$.

\newpage

\section{Introduction}
This example demonstrates a method discovered by Eduard Lill in 1867 for graphically finding (or more accurately, verifying) the real roots of any polynomial \cite{bradford, hull-beloch, riaz}. We limit the presentation to cubic polynomials. Lill's method has seen renewed interest because it can be implemented using origami, as we shall see in Chapter~\ref{c.beloch}.

In Sections~\ref{s.multiple}--\ref{s.noroots} we continue the initial example to find additional roots and to show that if an angle $\alpha$ is such that $(-\tan\alpha)$ is \emph{not} a root, then the construction doesn't work. Section~\ref{s.method} presents the full specification of Lill's method. Special cases of the method are demonstrated by the examples in Sections~\ref{s.negative}--\ref{s.noninteger}. Since Lill's method can find a real root of any cubic polynomial, it can be used to trisect an angle. By computing $\sqrt[3]{2}$ as a root of $x^3-2$, it can double a cube (Section~\ref{s.cube}). Section~\ref{s.proof} gives a proof that Lill's method can find the real roots of any cubic polynomial. The proof for arbitrary polynomials is similar.

%\newpage

\section{Multiple roots}\label{s.multiple}

Let us continue the example above. The polynomial $p(x)=x^3+6x^2+11x+6$ has three roots $-1,-2,-3$. Computing the arc tangent of the negation of the roots gives:
\[
\alpha=-\tan^{-1} (-1) = 45^\circ,\quad \beta=-\tan^{-1}(-2) = 63.4^\circ,\quad \gamma=-\tan^{-1} (-3)= 71.6^\circ\,.
\]
In the diagram below we see that for each of the three angles, the second path intersects the end of the first path.
\begin{center}
\begin{tikzpicture}[scale=1.15]
% Draw help lines and axes
\draw[step=10mm,white!50!black] (-11,-1) grid (2,7);
\draw[thick] (-11,0) -- (2,0);
\draw[thick] (0,-1) -- (0,7);
\foreach \x in {-10,...,2}
  \node at (\x-.3,-.2) {\sm{\x}};
\foreach \y in {1,...,7}
  \node at (-.2,\y-.3) {\sm{\y}};
\coordinate (A) at (0,0);
\coordinate (B) at (1,0);
\coordinate (C) at (1,6);
\coordinate (D) at (-10,6);
\coordinate (E) at (-10,0);
\foreach \x in {A,B,C,D,E}
  \fill (\x) circle(1.5pt);
\draw[very thick] (A) --
  node[below,yshift=-5pt] {$1$} (B);
\draw[very thick,name path=bc] (B) --
  node[right,yshift=24pt] {$6$} (C);
\draw[very thick,name path=cd] (C) --
  node[above] {$11$}(D);
\path[name path=de] (D) -- ($(E)+(0,-.8)$);
\draw[very thick] (D) --
  node[left,yshift=6pt] {$6$} (E);

% Draw first segment of first second path
\path[name path=a1] (A) -- +(45:3);
\path [name intersections = {of = a1 and bc, by = {A1}}];
\fill (A1) circle(1.5pt) node[above right] {$P_1$};
\draw[very thick,dashed] (A) -- (A1);
\draw[thick] ($(A) + (16pt,0)$)
  arc [start angle=0, end angle = 45, radius=16pt];
\node[above right,xshift=44pt,yshift=0pt] at (A) {$\alpha$};
\draw[rotate=135] (A1) rectangle +(10pt,10pt);
\draw[Stealth-,thick] ($(A) + (16pt,6pt)$) -- +(24pt,0);

% Draw second segment of first second path
\path[name path=b1] (A1) -- +(135:8);
\path [name intersections = {of = b1 and cd, by = {B1}}];
\fill (B1) circle(1.5pt) node[above right] {$Q_1$};
\draw[very thick,dashed] (A1) -- (B1);
\draw[rotate=225] (B1) rectangle +(10pt,10pt);

% Draw third segment of first second path
\draw[very thick,dashed] (B1) -- (E);

% Draw first segment of second second path
\path[name path=a2] (A) -- +(63.4:4);
\path [name intersections = {of = a2 and bc, by = {A2}}];
\fill (A2) circle(1.5pt) node[above right] {$P_2$};
\draw[very thick,dashed] (A) -- (A2);
\draw[thick] ($(A) + (24pt,0)$)
  arc [start angle=0, end angle = 63.4, radius=24pt];
\node[above right,xshift=44pt,yshift=8pt] at (A) {$\beta$};
\draw[rotate=153.4] (A2) rectangle +(10pt,10pt);
\draw[<-,thick] ($(A) + (22pt,14pt)$) -- +(18pt,0);

% Draw second segment of second second path%
\path[name path=b2] (A2) -- +(153.4:10);
\path [name intersections = {of = b2 and cd, by = {B2}}];
\fill (B2) circle(1.5pt) node[above right] {$Q_2$};
\draw[very thick,dashed] (A2) -- (B2);
\draw[rotate=243.4] (B2) rectangle +(10pt,10pt);

% Draw third segment of second second path%
\draw[very thick,dashed] (B2) -- (E);

% Draw first segment of second second path%
\path[name path=a3] (A) -- +(71.6:4);
\path [name intersections = {of = a3 and bc, by = {A3}}];
\fill (A3) circle(1.5pt) node[above right] {$P_3$};
\draw[very thick,dashed] (A) -- (A3);
\draw[thick] ($(A) + (38pt,0)$)
  arc [start angle=0, end angle = 71.6, radius=40pt];
\node[above right,xshift=44pt,yshift=22pt] at (A) {$\gamma$};
\draw[rotate=161.6] (A3) rectangle +(10pt,10pt);
\draw[<-,thick] ($(A) + (32pt,25pt)$) -- +(10pt,0);

% Draw second segment of second second path%
\path[name path=b3] (A3) -- +(161.6:10);
\path [name intersections = {of = b3 and cd, by = {B3}}];
\fill (B3) circle(1.5pt) node[above right] {$Q_3$};
\draw[very thick,dashed] (A3) -- (B3);
\draw[rotate=251.6] (B3) rectangle +(10pt,10pt);

% Draw third segment of second second path%
\draw[very thick,dashed] (B3) -- (E);
\end{tikzpicture}
\end{center}

\newpage

\section{Paths that do not lead to roots}\label{s.noroots}

Perhaps the second path intersects the end of the first path for \emph{any} initial angle, for example, $56.3^\circ$. In the following diagram, the second path intersects the extension of the line segment for the coefficient $a_0$, but not at $(-10,0)$, the end of the first path. We conclude that $-\tan 56.3^\circ =-1.5$ is \emph{not} a root of the equation.
\begin{center}
\begin{tikzpicture}[scale=1.15]
% Draw help lines and axes
\draw[step=10mm,white!50!black] (-11,-1) grid (2,7);
\draw[thick] (-11,0) -- (2,0);
\draw[thick] (0,-1) -- (0,7);
\foreach \x in {-10,...,2}
  \node at (\x-.3,-.2) {\sm{\x}};
\foreach \y in {1,...,7}
  \node at (-.2,\y-.3) {\sm{\y}};

% Draw first path
\coordinate (A) at (0,0);
\coordinate (B) at (1,0);
\coordinate (C) at (1,6);
\coordinate (D) at (-10,6);
\coordinate (E) at (-10,0);
\foreach \x in {A,B,C,D,E}
  \fill (\x) circle(1.5pt);
\draw[very thick] (A) --
  node[below,yshift=-5pt] {$1$} (B);
\draw[very thick,name path=bc] (B) --
  node[right,yshift=6pt] {$6$} (C);
\draw[very thick,name path=cd] (C) --
  node[above] {$11$}(D);
\draw[very thick] (D) --
  node[left,yshift=6pt] {$6$}(E);
\path[name path=de] (-10,-1) -- (-10,7);

% Draw first segment of second path
\path[name path=a2] (A) -- +(56.3:3);
\path [name intersections = {of = a2 and bc, by = {A2}}];
\fill (A2) circle(1.5pt) node[above right] {$P$};
\draw[very thick,dashed] (A) -- (A2);
\draw ($(A) + (14pt,0)$)
  arc [start angle=0, end angle = 56.3, radius=14pt];
\node[above right,xshift=10pt,yshift=6pt] at (A) {$56.3^\circ$};
\draw[rotate=146.3] (A2) rectangle +(10pt,10pt);

% Draw second segment of second path
\path[name path=b2] (A2) -- +(146.3:10);
\path [name intersections = {of = b2 and cd, by = {B2}}];
\fill (B2) circle(1.5pt) node[above right] {$Q$};
\draw[very thick,dashed] (A2) -- (B2);
\draw[rotate=236.3] (B2) rectangle +(10pt,10pt);

% Draw third segment of second path
\path[name path=c2] (B2) -- +(236.3:8.5);
\path [name intersections = {of = c2 and de, by = {C2}}];
\fill (C2) circle(1.5pt);
\draw[very thick,dashed] (B2) -- (C2);
\end{tikzpicture}
\end{center}

\newpage

\section{Specification of Lill's method}\label{s.method}

Examining the examples below will help understand the details.
\begin{itemize}
\item Start with an arbitrary cubic polynomial: $p(x)=a_3x^3+a_2x^2+a_1x+a_0$.
\item Construct the first path as follows:
\begin{itemize}
\item For each coefficient $a_3,a_2,a_1,a_0$ (in that order) draw a line segment starting at the origin $O=(0,0)$ in the positive direction of the $x$-axis. Turn $90^\circ$ counterclockwise between each segment.
\end{itemize}
\item Construct the second path as follows:
\begin{itemize}
\item We use the symbol for a coefficient $a_i$ to also denote the corresponding side of the first path.
\item Construct a line from $O$ at an angle of $\theta$ with the positive $x$-axis that intersects $a_2$ at point $P$.
\item Turn $\pm 90^\circ$ and construct a line from $P$ that intersects $a_1$ at $Q$.
\item Turn $\pm 90^\circ$ and construct a line from $Q$ that intersects $a_0$ at $R$.
\item If $R$ is the end point of the first path, then $-\tan\theta$ is a root of $p(x)$.
\end{itemize}
\item Special cases:
\begin{itemize}
\item When drawing the line segments of the first path, if a coefficient is negative, draw the line segment \emph{backwards}.
\item When drawing the line segments of the first path, if a coefficient is zero, do not draw a line segment but continue with the next $\pm 90^\circ$ turn.
\end{itemize}
\item Notes:
\begin{itemize}
\item ``Intersects $a_i$'' means ``intersects the line that contains the line segment $a_i$''.
\item When building the second path, choose to turn left or right by $90^\circ$ so that there is an intersection with the next segment of the first path.
\end{itemize}
\end{itemize}

\newpage

\section{Negative coefficients}\label{s.negative}

Section~\ref{s.ax6} gave an example of the use of Axiom~6 that resulted in the polynomial $p(x)=x^3-3x^2-3x+1$ with negative coefficients.

We start by drawing a segment of length $1$ to the right. Next we turn $90^\circ$ to face up, but the coefficient is negative, so we draw a segment of length $3$ \emph{down}. After turning $90^\circ$ to the left, the coefficient is again negative, so we draw a segment of length $3$ to the right. Finally, we turn down and draw a segment of length $1$.

We start the second path with a line angled $45^\circ$ with the $x$-axis. It intersects the \emph{extension} of the line segment for $a_2$ at $(1,1)$. Turning $-90^\circ$ (to the right), the line intersects the \emph{extension} of the line segment for $a_1$ at $(5,-3)$. Turning $-90^\circ$ again, the line intersects the end of the first path at $(4,-4)$.

Since $-\tan 45^\circ=-1$, a real root of the polynomial is $-1$:
\[
p(-1)=(-1)^3-3(-1)^2-3(-1)+6=0\,.
\]
The loosely dashed lines in the diagram will be discussed  in Section~\ref{s.noninteger}.

\begin{center}
\begin{tikzpicture}[scale=1.1]
% Draw help lines and axes
\draw[step=10mm,white!50!black] (-1,-6) grid (6,2);
\draw (0,-6) -- (0,2);
\foreach \x in {0,...,6}
  \node at (\x-.3,-.2) {\sm{\x}};
\foreach \y in {-5,...,-1}
  \node at (-.3,\y-.3) {\sm{\y}};
\foreach \y in {1,...,2}
  \node at (-.3,\y-.3) {\sm{\y}};

% Draw first path
\coordinate (A) at (0,0);
\coordinate (B) at (1,0);
\coordinate (C) at (1,-3);
\coordinate (D) at (4,-3);
\coordinate (E) at (4,-4);
\foreach \x in {A,B,C,D,E}
  \fill (\x) circle(1pt);
\draw[very thick,{Stealth[scale=1.4,inset=2pt,reversed]}-] (A) --
  node[below,yshift=-5pt] {$1$} (B);
\draw[very thick,{Stealth[scale=1.4,inset=2pt]}-,name path=bc] (B) --
  node[right,xshift=3pt] {$a_2=-3$} (C);
\draw[very thick,{Stealth[scale=1.4,inset=2pt]}-,name path=cd] (C) --
  node[above,xshift=11pt] {$a_1=-3$}(D);
\draw[very thick,{Stealth[scale=1.4,inset=2pt,reversed]}-,name path=de] (D) --
  node[right] {$1$}(E);

% Draw extensions of first path
\draw[very thick,loosely dotted,name path=a] (-1,0) -- (6,0);
\draw[very thick,loosely dotted,name path=b] (1,-6) -- (1,2);
\draw[very thick,loosely dotted,name path=c] (-1,-3) -- (6,-3);

% Draw first second path
\path[name path=a1] (A) -- +(-75:5);
\path [name intersections = {of = a1 and b, by = {B1}}];
\path[name path=b1] (B1) -- +(15:5);
\path [name intersections = {of = b1 and c, by = {C1}}];
\draw[thick,loosely dashed] (A) -- (B1) -- (C1) -- (E);

% Draw second second path
\draw[very thick,dashed] (4,-4) -- (5,-3) coordinate (A2);
\fill (5,-3) circle (1.5pt) node[above right] {$P$};
\draw[very thick,dashed] (5,-3) -- (1,1) coordinate (B2);
\fill (1,1) circle (1.5pt) node[above right] {$Q$};
\draw[very thick,dashed] (1,1) -- (0,0);

% Draw third second path
\path[name path=a3] (A) -- +(-15:5);
\path [name intersections = {of = a3 and b, by = {B3}}];
\path[name path=b3] (B3) -- +(-105:5);
\path [name intersections = {of = b3 and c, by = {C3}}];
\draw[thick,loosely dashed] (A) -- (B3) -- (C3) -- (E);
\end{tikzpicture}
\end{center}

\newpage

\section{Zero coefficients}\label{s.zero}

$a_2$, the coefficient of the $x^2$ term in the polynomial $x^3-7x-6=0$, is zero. For a zero coefficient, we ``draw'' a line segment of length $0$, that is, we do not draw a line, but we still make the $\pm 90^\circ$ turn before ``drawing'' it, as indicated by the arrow pointed up at point $(1,0)$ in the diagram. Next make an additional turn and draw a line of length $-7$, that is, of length $7$ backwards, to point $(8,0)$. Finally, turn again and draw a line of length $-6$ to point $(8,6)$.

There are three second paths that intersect the end of the first path. They start with angles of:
\[
\alpha=45^\circ,\quad \beta=63.4^\circ,\quad \gamma=-71.6^\circ\,.
\]
We conclude that there are three real roots:
\[
-\tan 45^\circ=-1,\quad -\tan 63.4^\circ =-2,\quad -\tan (-71.6^\circ)=3\,.
\]
Check:
\[
(x+1)(x+2)(x-3)=x^3-7x-6\,.
\]

\begin{center}
\begin{tikzpicture}[scale=1.1]
% Draw help lines and axes
\draw[step=10mm,white!50!black] (-1,-4) grid (11,7);
\foreach \x in {0,...,11}
  \node at (\x-.3,-.2) {\sm{\x}};
\foreach \y in {-3,...,-1}
  \node at (-.3,\y-.3) {\sm{\y}};
\foreach \y in {1,...,7}
  \node at (-.3,\y-.3) {\sm{\y}};

% Draw first path
\coordinate (A) at (0,0) node[above left] {$O$};
\coordinate (B) at (1,0);
\coordinate (C) at (8,0);
\coordinate (D) at (8,6);
\node[below right] at (D) {$A$};
\foreach \x in {A,B,C,D}
  \fill (\x) circle(1.5pt);
\draw[very thick,{Stealth[scale=1.4,inset=2pt,reversed]}-] (A) --
  node[below,yshift=-5pt] {$1$} (B);
\draw[{Stealth[scale=1.4,inset=2pt,reversed]}-,very thick] (B) --
  ($(B)+(0,.1)$);
\draw[very thick,{Stealth[scale=1.4,inset=2pt]}-,name path=bc] (B) --
  node[below,xshift=-6pt,yshift=-5pt] {$-7$} (C);
\draw[very thick,{Stealth[scale=1.4,inset=2pt]}-,name path=cd] (C) --
  node[right,yshift=4pt] {$-6$}(D);

% Draw extensions of first path
\draw[very thick,loosely dotted] (1,-3) -- (1,7);
\draw[very thick,loosely dotted] (-1,0) -- (11,0);

% Draw first second path
\draw[very thick,dashed,->] (0,0) -- (1,-3);
\fill (1,-3) circle (1.5pt) node[below left] {$P_1$};
\draw[very thick,dashed,->] (1,-3) coordinate (A1) -- (10,0);
\fill (10,0) circle (1.5pt) node[below right] {$Q_1$};
\draw[very thick,dashed,->] (10,0) coordinate (B1) -- (D);

% Draw second second path
\draw[very thick,dashed,->] (0,0) -- (1,1) coordinate (A2);
\fill (A2) circle (1.5pt) node[above right] at (A2) {$P_2$};
\draw[very thick,dashed,->] (A2) -- (2,0) coordinate (B2);
\fill (B2) circle (1.5pt) node[below right] at (B2) {$Q_2$};
\draw[very thick,dashed,->] (B2) -- (D);

% Draw third second path
\draw[very thick,dashed,->] (0,0) -- (1,2) coordinate (A3);
\fill (A3) circle (1.5pt) node[above left] {$P_3$};
\draw[very thick,dashed,->] (A3) -- (5,0) coordinate (B3);
\fill (B3) circle (1.5pt) node[below right] at (B3) {$Q_3$};
\draw[very thick,dashed,->] (B3) -- (D);
\end{tikzpicture}
\end{center}

\newpage

\section{Non-integer roots}\label{s.noninteger}

Consider the polynomial $p(x)=x^3-2x+1$. The first segment is from $(0,0)$ to $(1,0)$ and turns up. The coefficient of $x^2$ is zero so no segment is drawn and turns left. The next coefficient is negative so the segment it goes backwards from $(1,0)$ to $(3,0)$ and turns right. Finally, a segment is drawn from $(3,0)$ to $(3,-1)$. Clearly, $1$ is a root of $p(x)$ and since $-\tan^{-1} (-45^\circ)=1$, there is a path $\overline{OP_1Q_1A}$.

 If we divide $p(x)$ by $x-1$, we obtain the quadratic polynomial $x^2+x-1$ whose roots are $\frac{-1\pm\sqrt{5}}{2} \approx 0.62,\; -1.62$.
%\[
%\disfrac{-1\pm\sqrt{5}}{2} \approx 0.62,\; -1.62\,.
%\]
There are two additional second paths: one starting at $-31.8^\circ$ since $-\tan^{-1} 0.62=-31.8^\circ$, and one starting at $58.3^\circ$ since $-\tan^{-1}1.62=58.3^\circ$.

Similarly, the polynomial in Section~\ref{s.negative} has roots $ 2\pm\sqrt{3}\approx 3.73, 0.27$. The corresponding angles are $-75^\circ$ and $-15^\circ$, because $-\tan (-75^\circ)\approx 3.73$ and $-\tan (-15^\circ)\approx 0.27$.
\begin{center}
\begin{tikzpicture}[scale=1.6]
% Draw help lines and axes
\draw[step=10mm,white!70!black,] (-1,-2) grid (4,2);
\foreach \x in {0,...,4}
  \node at (\x-.2,-.1) {\sm{\x}};
\foreach \y in {-1}
  \node at (-.1,\y-.2) {\sm{\y}};
\foreach \y in {1,2}
  \node at (-.1,\y-.2) {\sm{\y}};

% Draw first path
\coordinate (A) at (0,0);
\node[above left] at (A) {$O$};
\coordinate (B) at (1,0);
\coordinate (C) at (3,0);
\coordinate (D) at (3,-1);
\node[below right] at (D) {$A$};
\foreach \x in {A,B,C,D}
  \fill (\x) circle(1pt);
\draw[very thick] (A) -- node[above,yshift=2pt] {$1$} (B);
\draw[{Stealth[scale=1.4,inset=2pt,reversed]}-,very thick] ($(A)+(.1,0)$) --
  ($(A)+(.15,0)$);
\draw[{Stealth[scale=1.4,inset=2pt,reversed]}-,very thick] ($(B)+(0,.05)$) --
  ($(B)+(0,.1)$);
\draw[very thick,name path=bc] (B) --
  node[above,xshift=-4pt,yshift=2pt] {$-2$} (C);
\draw[{Stealth[scale=1.4,inset=2pt,reversed]}-,very thick] ($(B)+(.22,0)$) --
  ($(B)+(.17,0)$);
\draw[very thick,name path=cd] (C) --
  node[right] {$1$}(D);
\draw[{Stealth[scale=1.4,inset=2pt,reversed]}-,very thick] ($(C)+(0,-.05)$) --
 ($(C)+(0,-.1)$);

% Draw extensions of first path
\draw[very thick,loosely dotted,name path=b] (1,-2) -- (1,2);
\draw[very thick,loosely dotted,name path=c] (-1,0) -- (4,0);
\draw[very thick,loosely dotted,name path=d] (3,-2) -- (3,2);

% Draw first second path
\coordinate (A1) at (1,-1);
\draw[very thick,dashed,->] (0,0) -- (A1);
\fill (A1) circle (1pt) node[below right] {$P_1$};
\coordinate (B1) at (2,0);
\draw[very thick,dashed,->] (A1) -- (B1);
\fill (B1) circle (1pt) node[above right,xshift=4pt] {$Q_1$};
\draw[very thick,dashed,->] (B1) -- (D);
\draw[rotate=45] (A1) rectangle +(6pt,6pt);
\draw[rotate=-135] (B1) rectangle +(6pt,6pt);

% Draw second second path
\path[name path=a2] (0,0) -- +(-31.7:4);
\path [name intersections = {of = a2 and b, by = {A2}}];
\draw[very thick,dashed,->] (0,0) -- (A2);
\fill (A2) circle (1pt) node[below right,yshift=2pt] {$P_2$};
\path[name path=b2] (A2) -- +(58.3:2.5);
\path [name intersections = {of = b2 and c, by = {B2}}];
\draw[very thick,dashed,->] (A2) -- (B2);
\fill (B2) circle (1pt) node[above] {$Q_2$};
\draw[very thick,dashed,->] (B2) -- (D);
\draw[rotate=58.3]   (A2) rectangle +(6pt,6pt);
\draw[rotate=-121.7] (B2) rectangle +(6pt,6pt);

% Draw third second path
\path[name path=a3] (0,0) -- +(58.3:2.5);
\path [name intersections = {of = a3 and b, by = {A3}}];
\draw[very thick,dashed,->] (0,0) -- (A3);
\fill (A3) circle (1pt) node[above left] {$P_3$};
\path[name path=b3] (A3) -- +(-31.7:4);
\path [name intersections = {of = b3 and c, by = {B3}}];
\draw[very thick,dashed,->] (A3) -- (B3);
\fill (B3) circle (1pt) node[above right] {$Q_3$};
\path[name path=c3] (B3) -- +(-121.7:4);
\draw[very thick,dashed,->] (B3) -- (D);
\draw[rotate=-121.7]   (A3) rectangle +(6pt,6pt);
\draw[rotate=-211.7]   (B3) rectangle +(6pt,6pt);
\end{tikzpicture}
\end{center}

%\newpage

\vspace{-13ex}

\section{The cube root of two}\label{s.cube}

$\sqrt[3]{2}$ is a root of the cubic polynomial $x^3-2$. In the first path, we turn left twice without drawing any line segments, because $a_2$ and $a_1$ are both zero. Then we turn left again (to face down) and draw backwards because $a_0=-2$ is negative. The first segment of the second path is drawn at an angle of $-51.6^\circ$ and $-\tan (-51.6^\circ)\approx 1.26\approx \sqrt[3]{2}$.
\begin{center}
\begin{tikzpicture}[scale=1.2]
% Draw help lines and axes
\draw[step=10mm,white!70!black,] (-1,-2) grid (3,2);
\foreach \x in {0,...,3}
  \node at (\x-.2,-.1) {\sm{\x}};
\foreach \y in {-1}
  \node at (-.2,\y-.2) {\sm{\y}};
\foreach \y in {1,2}
  \node at (-.2,\y-.2) {\sm{\y}};

% Draw first path
\coordinate (A) at (0,0);
\coordinate (B) at (1,0);
\coordinate (C) at (1,2);
\foreach \x in {A,B,C}
  \fill (\x) circle(1pt);
\draw[very thick] (A) -- node[above,yshift=2pt] {$1$} (B);

\draw[{Stealth[scale=1.4,inset=2pt,reversed]}-,very thick] ($(A)+(.05,0)$) --
  ($(A)+(.1,0)$);
\draw[{Stealth[scale=1.4,inset=2pt,reversed]}-,very thick] ($(B)+(0,.05)$) --
  ($(B)+(0,.1)$);
\draw[{Stealth[scale=1.4,inset=2pt,reversed]}-,very thick] ($(B)+(.1,.3)$) --
  ($(B)+(.08,.3)$);
\draw[{Stealth[scale=1.4,inset=2pt,reversed]}-,very thick] ($(B)+(0,.55)$) --
  ($(B)+(0,.5)$);

\draw[very thick] (B) --
  node[left,yshift=6pt] {$-2$} (C);

% Draw extensions of first path
\draw[very thick,loosely dotted,name path=a] (-1,0) -- (3,0);
\draw[very thick,loosely dotted,name path=b] (1,-2) -- (1,2);

% Draw first segment of second path
\path[name path=a1] (0,0) -- +(-51.6:2);
\path [name intersections = {of = a1 and b, by = {A1}}];
\draw[very thick,dashed,->] (A) -- (A1);
\fill (A1) circle (1pt) node[below left] {$P_1$};
\draw[rotate=38.4]   (A1) rectangle +(6pt,6pt);

% Draw second segment of second path
\path[name path=b1] (A1) -- +(38.4:2.5);
\path [name intersections = {of = b1 and a, by = {B1}}];
\draw[very thick,dashed,->] (A1) -- (B1);
\fill (B1) circle (1pt) node[above right] {$Q_1$};
\draw[rotate=128.4] (B1) rectangle +(7pt,7pt);

% Draw third segement of second path
\draw[very thick,dashed,->] (B1) -- (C);
\end{tikzpicture}
\end{center}

\newpage

\section{Proof of Lill's method}\label{s.proof}

We limit ourselves to monic cubic polynomials $p(x)=x^3+a_2x^2+a_1x+a_0$.\footnote{If the polynomial is not monic, divide it by $a_3$ and the resulting monic polynomial has the same roots.} In the diagram below, segments of the first path are labeled with coefficients and with $b_2,b_1,a_2-b_2,a_1-b_1$.

Since the sum of the angles of a triangle is $180^\circ$, in a right triangle if one acute angle is $\theta$, the other is $90^\circ-\theta$. Therefore, the angle above $P$ and the angle to the left of $Q$ are equal to $\theta$. We now derive a sequence of formulas for $\tan \theta$:
\vspace{-2ex}
\begin{form}{1.1}
\tan \theta &=& \disfrac{b_2}{1}=b_2\\
%a_2-b_2&=&a_2-\tan\theta\\
\tan \theta &=& \disfrac{b_1}{a_2-b_2}=\disfrac{b_1}{a_2-\tan\theta}\\
b_1&=&\tan\theta (a_2-\tan\theta)\\
\tan \theta &=& \disfrac{a_0}{a_1-b_1}=\disfrac{a_0}{a_1-\tan\theta(a_2-\tan\theta)}\,.
\end{form}
Simplifying the last equation gives:
\vspace{-2ex}
\begin{form}{1}
(\tan\theta)^3-a_2(\tan\theta)^2+a_1(\tan\theta)-a_0&=&0\\
-(\tan\theta)^3+a_2(\tan\theta)^2-a_1(\tan\theta)+a_0&=&0\\
(-\tan\theta)^3+a_2(-\tan\theta)^2+a_1(-\tan\theta)+a_0&=&0\,.
\end{form}
We conclude that $-\tan\theta$ is a real root of $p(x)=x^3+a_2x^2+a_1x+a_0$.

\begin{center}
\begin{tikzpicture}[scale=1.15]
% Draw grid and axes
\draw[step=10mm,white!50!black] (-11,-1) grid (2,7);
\draw[thick] (-11,0) -- (2,0);
\draw[thick] (0,-1) -- (0,7);
\foreach \x in {-10,...,2}
  \node at (\x-.3,-.2) {\sm{\x}};
\foreach \y in {1,...,7}
  \node at (-.2,\y-.3) {\sm{\y}};

% Draw the points of the first path
\coordinate (A) at (0,0);
\coordinate (B) at (1,0);
\coordinate (C) at (1,6);
\coordinate (D) at (-10,6);
\coordinate (E) at (-10,0);
\foreach \x in {A,B,C,D,E}
  \fill (\x) circle(2pt);
\draw[rotate=90] (B) rectangle +(10pt,10pt);

% Draw A -- B and arrow
\draw[very thick] (A) --(B);
\draw[thick,<->] ($(A)+(0,-16pt)$) --
  node[fill=white] {$1$} ($(B)+(0,-16pt)$);

% Draw B -- C and arrow
\draw[very thick,name path=bc] (B) -- (C);
\draw[thick,<->] ($(B)+(36pt,0)$) --
  node[fill=white] {$a_2$} ($(C)+(36pt,0)$);

% Draw C -- D and arrow
\draw[very thick,name path=cd] (C) --(D);
\draw[thick,<->] ($(C)+(0,24pt)$) --
  node[fill=white] {$a_1$} ($(D)+(0,24pt)$);

% Draw D -- E and arrow
\draw[very thick,name path=de] (D) -- (E);
\draw[thick,<->] ($(D)+(-16pt,0)$) --
  node[fill=white] {$a_0$} ($(E)+(-16pt,0)$);

% Draw first angled segment of the second path and intersection A2 with BC
\path[name path=a2] (A) -- +(63.4:4);
\path [name intersections = {of = a2 and bc, by = {A2}}];
\fill (A2) circle(2pt) node[above right] {$P$};
\draw[very thick,dashed] (A) -- (A2);
\path (B) -- node[right] {$b_2$} (A2);
\path (A2) -- node[right,yshift=8pt] {$a_2-b_2$} (C);
\draw[rotate=153.4] (A2) rectangle +(10pt,10pt);

% Draw second segment of the second path and intersection B2 with CD
\path[name path=b2] (A2) -- +(153.4:10);
\path [name intersections = {of = b2 and cd, by = {B2}}];
\fill (B2) circle(2pt) node[above right] {$Q$};
\draw[very thick,dashed] (A2) -- (B2);
\draw[rotate=243.4] (B2) rectangle +(10pt,10pt);
\path (D) -- node[above] {$a_1-b_1$} (B2);
\path (B2) -- node[above] {$b_1$} (C);

% Draw third segment of the second path to E
\draw[very thick,dashed] (B2)-- (E);

% Label A, A2, B2 with theta
\draw ($(A) + (14pt,0)$)
  arc [start angle=0, end angle = 63.4, radius=14pt];
\node[above right,xshift=10pt,yshift=8pt] at (A) {$\theta$};
\draw ($(A2) + (0,14pt)$)
  arc [start angle=90, end angle = 153.4, radius=14pt];
\node[above left,xshift=-4pt,yshift=14pt] at (A2) {$\theta$};
\draw ($(B2) + (-14pt,0)$)
  arc [start angle=180, end angle = 243.4, radius=14pt];
\node[below left,xshift=-14pt,yshift=-4pt] at (B2) {$\theta$};
\end{tikzpicture}
\end{center}


\section{Model and strategy} % (fold)
\label{sec:model_and_strategy}

%\subsection{Board description} % (fold)
%\label{sub:board_description}

%
% \paragraph{Change of square values and traps definition} % (fold)
% \label{par:change_of_square_values_and_traps_definition}
%
% % paragraph change_of_square_values_and_traps_definition (end)
%
% \paragraph{No consecutive traps} % (fold)
% \label{par:no_consecutive_traps}
% A first assumption in the way we defined our board concerns the non-existence
% of domino effect of traps. This means that the only way to activate a trap
% is to fall on its box immediately after one dice throw.
% For example, if one lands on box 8, which is a type 3 [??? type 2 if we can not
% modify the project rules] trap, it brings us back to box 5. In this case,
% even if box 5 is a trap, it won't affect us until the following dice throw
% (i.e. if a 0 is made at the following dice throw).
% % paragraph no_consecutive_traps (end)
%
% % subsection board_description (end)

\subsection{Markov model} % (fold)
\label{sub:markov_model}
Let us first recall the general idea behind Markov decision processes.
The objective is to determine the best set of actions $\{u(1), u(2), \dots
u(n)\}$ that minimizes $V_{\pi}(s_0)$, the total expected cost of the process.
The algorithm we use in order to determine this set is the one based on Bellman equation
\[
  V(k) \leftarrow \min_{a\in U(k)} \{c(a|k)
  + \sum_{k'=1}^n p(k'|k,a)V(k') \} \qquad k \neq d
\]
where $V(d)$ is set to 0 at each iteration, $d$ being the destination state.
In this \emph{value-iteration} algorithm, $c(a|k)$ and $p(k'|k,a)$
respectively represent the \emph{cost} of making action $a$
when being in state $k$ and the \emph{probability} of going
from state $k$ to $k'$ when action $a$ is triggered.

It is proven that if we find a set of values $\{V(1), \dots, V(n)\}$
that satisfies this set of equations, then it is \emph{optimal}
and it can be solved by \emph{recursion}.

Linking this process to our \textit{Snakes and Ladders} game,
a state $k$ corresponds to \emph{being on} the kth square,
thus taking values in the range 1 to 15.
We then define the action $a$ as the \emph{throw of a dice}
and since every one of the three dices can be thrown at each
iteration we have $U(1) = \dots = U(15) = \{a_{\text{secu}},
a_{\text{norm}}, a_{\text{risk}}\}$.
Finally, recalling that the goal of the game is to get to
the 15th square by \emph{minimizing the number of dice throws},
the cost $c(a|k)$ equals 1 as each turn will lead in one dice
throw, whatever the action $a$ or the state $k$.

At this point only one last ingredient is missing to start implementing
the above described algorithm~: the transition probabilities $p(k'|k,a)$.
Here lies the first main difficulty of the project as those depends
both on the type of square or trap from which one leaves $k$ and
on which one will arrive $k'$, as well as on the dice one
decides to use (i.e. $a$).
It is now important to \emph{find an efficient way of constructing
the 3 transition probability matrices} $P_{\text{secu}}$, $P_{\text{norm}}$
and $P_{\text{risk}}$ by minimizing the number of particular cases,
as one can not even imagine to compute the $3\times 15^2$ individual probabilities.

The precise implementation of the probability matrices won't be
discussed in this section but it appeared important to us
to still make a point on the mathematical idea of our method.
\begin{enumerate}
  \item First we fill the matrix by assuming that
  all squares are normal squares and thus filling the matrices
  with probabilities $\frac{1}{2}$, $\frac{1}{3}$ and $\frac{1}{4}$,
  making particular cases for the slow/fast \emph{lanes}
  and the \emph{circularity} of the board.
  \item Then we re-modify each matrix line by taking \emph{traps} into account
  and thus re-distributing the traps transition probabilities.
  For example, if there a \emph{type 2 trap} at square 8
  and the \emph{normal} dice is thrown, we make the following adjustments
  for each line $i$.
  \begin{align*}
    p(i,5) &\leftarrow p(i,5) + p(i,8)/2 \\
     p(i,8) &\leftarrow p(i,8)/2
  \end{align*}
\end{enumerate}

% subsection markov_model (end)

\subsection{Finding the optimal dice} % (fold)
\label{sub:choosing_the_optimal_dice}
Once the algorithm converged and we obtained the optimal
cost values $\{V(1), \dots, V(n)\}$, the optimal dices that
lead to these expected cost can be retrieved by applying
the following formula
\[
  u(k) \leftarrow \arg\min_{a\in U(k)} \{c(a|k)
  + \sum_{k'=1}^n p(k'|k,a)V(k') \} \qquad k \neq d
\]
This formula returns at each iteration the action
(i.e. the \emph{type of dice}) that enables to reach the minimum
expected cost contained in the values $\{V(1), \dots, V(n)\}$.

% subsection choosing_the_optimal_dice (end)

% section model_and_strategy (end)


% Created 2020-01-22 Wed 15:54
% Intended LaTeX compiler: pdflatex
\documentclass[11pt]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{grffile}
\usepackage{longtable}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{amssymb}
\usepackage{capt-of}
\usepackage{hyperref}
\author{Vaibhav Pujari}
\date{\today}
\title{Foundations of Haskell}
\hypersetup{
 pdfauthor={Vaibhav Pujari},
 pdftitle={Foundations of Haskell},
 pdfkeywords={},
 pdfsubject={},
 pdfcreator={Emacs 26.3 (Org mode 9.3)},
 pdflang={English}}
\begin{document}

\maketitle
\tableofcontents

\section{Lambda calculus and Category theory}
\label{sec:org3dd5759}
\begin{itemize}
\item Everything is a function
\item There is no error reporting - not that practical because there is no help from compiler
\item If we introduce error checking and types, we deviate from lambda calculus because not every expression is correct. But we also gain a lot of practical value
\item This then becomes a category - because in category composition is not arbitrary, it is constrained by morphisms
\end{itemize}
\section{The language}
\label{sec:org5892609}
\subsection{Data types}
\label{sec:org305db3e}
\begin{verbatim}
x :: Int
x = 5
\end{verbatim}
\subsection{Function types}
\label{sec:org3b97478}
Types are sets and in category of sets, a function is a member of Hom-set between two objects, so its a member of some other object in the same category (every set is an object)
\subsection{Partial functions}
\label{sec:org816f474}
\begin{itemize}
\item Recursion can create programs that never terminate
\item Well-founded recursion - compiler can prove that it terminates. But its a theoretical concept because for every algorithm that claims to find out if a recursion is well-founded or not, there always exists a particual recursion that breaks the algorithm
\item Therefore, in practice we need partial functions. A partial function gives result when it terminates, but there is no guarantee that it will terminate
\end{itemize}
\subsection{Composition}
\label{sec:org97e878e}
\begin{verbatim}
(.) :: (b->c) -> (a->b) -> a -> c
(g . f) x = g ( f x )
\end{verbatim}
Polymorphic types and higher order functions are foundational in haskell as opposed to a later added feature in most common languages
\subsection{Functors}
\label{sec:org7e9a78e}
\begin{itemize}
\item A structure-preserving mapping between two categories
\begin{itemize}
\item To preserve objects, we just need a type constructor
\end{itemize}

F: Ob(Hask) \(\to\) Ob(Hask)

\begin{itemize}
\item To preserve morphism structure, we need \texttt{fmap}. \texttt{fmap} maps morphism in first category to a morphism in second category such that it is still composable in the same way as original morphism was composable
\end{itemize}

fmap: Hom\textsubscript{Hask} \(\to\) Hom\textsubscript{Hask}

\item Shrinking the structure is a special case of preserving. When objects collapse, the morphisms must follow
\item There is a category \texttt{Cat}, in which objects are smaller categories and morphisms are functors
\end{itemize}
\end{document}


\subsection{Checking for heteroskedasticity using the White test}


\subsection{Convergence}

If \(x=c\) then the power series will be equal to \(a_0\).

For other values the power series may not converge.

\subsubsection{Cauchy-Hadamard theorem}

Radius of convergence:

\(\dfrac{1}{R}={\lim \sup}_{n_\rightarrow \infty} (|a_n|^{\dfrac{1}{n}})\)