\section{Finding Periodic Solutions}
\label{sec:initial}

A separate problem from evaluating, optimizing and continuing periodic solutions is finding an initial candidate solution.
Because continuation is a crucial part of this project, having just one single periodic solution might enable to find many others, through tracing and switching solution branches.
Because the systems considered in this work have stable periodic solutions, we focus on this case.
When this is not the case, as mentioned in the section about continuation methods (\autoref{sec:cont}), a homotopy between a trivial system and the target system, combined with continuation methods, might be a promising approach.

Starting from a point in the periodic solution's basin of attraction, one can simply forward integrate such a system.
There are several possible problems involved:
\begin{itemize}
	\item Forward integration can accumulate errors.
	\item Even if the starting point lies exactly on the periodic trajectory, the sampling interval would probably not be an integer fraction of the period of the periodic trajectory.
		Periodicity in the sequence of points are not directly related to periodicity in the continuous system.
	\item Given the nature of the project, it is very likely that period doubling bifurcations are encountered.
		These provoke situations where two periodic solutions exist which are difficult to distinguish.
\end{itemize}

Forward integration yields a sequence of points in phase space $(\textbf{s}_i)_{i \in \N}$.
To obtain more manageable data only intersections of the trajectory with a hyperplane $p(\textbf{x}) = \langle \textbf{n}, \textbf{x}-\textbf{x}_0 \rangle = 0$ in a single direction are considered (so called Poincaré sections).
To find these, $\textbf{s}_i, \textbf{s}_{i+1} = \textbf{s}_i + h_0 \cdot \textbf{f}(\textbf{s}_i)$ with $p(s_i) \le 0 < p(s_{i+1})$ are searched.
Because of continuity, there needs to exists $h \in [0,h_0]$ such that $p(\textbf{s}_i + h \cdot \textbf{f}(\textbf{s}_i)) = 0$, which is found via bisection.
Let $(\textbf{u}_i)_{i \in \N}$ be the sequence of intersections.

To find the number of intersections per period, the intersections are partitioned into $k \in \N$ disjoint clusters $V_{k,i} = \{ \textbf{u}_m\ |\ m \in k\N+i \}$ for $i \in \N$, $i \le k$.
The relative quality of the $i$-th cluster can then be assessed using the within-cluster variance $\sum_{v \in V_{k,i} } ||v - \E(V_{k,i})||^2$.
The correct number $k_{\text{min}} \in \N$ of intersections in one period is then taken to be the one minimizing the sum of within-cluster variances:
\[
	k_\text{min} \coloneqq \argmin_{k \in \N} \sum_{i = 1}^k \sum_{v \in V_{k,i} } ||v - \E(V_{k,i})||^2 \text.
\]
For further information about these measures see for example \cite{halkidi2001clustering}. %TODO measure...

In this form the criterion might at best work if the intersection sequence is infinite.
When dealing with finite sequences increasing the number of clusters inevitably leads to lower total within-cluster variances.
It is thus necessary to constrain the available values for $k$ and discourage the method of overestimating the number of clusters.
Trivially an upper limit for $k$ needs to be introduced.
Furthermore, the minimality criterion needs to be relaxed: Suppose there are $k_\text{min}$ intersections per period, then all multiples of $k_\text{min}$ yield equal or lower ratings.
One thus wants to choose the minimum $k$ which is in some sense still almost optimal.


\section{SCC}

%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Digraph as DAG}
  \begin{exampleblock}{Digraph as DAG (Problem 5.3)}
    Every digraph is a dag of its SCCs.
  \end{exampleblock}

  \begin{alertblock}{Remark}
    Two tiered structure of digraphs:
    \begin{itemize}
	  \item digraph $\equiv$ a dag of SCCs
      \item SCC: equivalence class over reachability
    \end{itemize}
  \end{alertblock}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{SCC}
  \begin{exampleblock}{Kosaraju SCC algorithm, 1978}
	\begin{quote}
	  ``SCCs can be topo-sorted in decreasing order of their highest finish time.''
	\end{quote}

	\pause
	\centerline{The vertice with the highest finish time is in a source SCC.}
  \end{exampleblock}

  \pause
  \begin{alertblock}{Remark}
    \begin{itemize}
      \item DFS on $G$; \; DFS/BFS on $G^{T}$
      \item DFS on $G^{T}$; DFS/BFS on $G$
    \end{itemize}
  \end{alertblock}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{SCC}
  \begin{exampleblock}{Kosaraju SCC algorithm, 1978 (Problem 5.4)}
    \begin{itemize}
      \item 1st DFS $\xLongrightarrow{?}$ BFS
      \item 2nd DFS $\xLongrightarrow{?}$ BFS
    \end{itemize}
  \end{exampleblock}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{One-to-all reachability}
  \begin{exampleblock}{One-to-all reachability (Problem 5.12)}
	Digraph $G = (V, E)$:
    \begin{itemize}
      \item given $v: v \leadsto^{?} \forall u$
      \item $\exists?\; v: v \leadsto \forall u$
    \end{itemize}
  \end{exampleblock}

  \pause
  \[
	\text{SCC; } \exists! \text{source vertex } v \iff v \leadsto \forall u
  \]

  \pause
  \begin{proof}
	\begin{itemize}
	  \item $\Longleftarrow$: (1) source (2) $\exists !$
		\pause
	  \item $\implies$: By contradiction.
		% \pause
		% \[
		%   \exists u: v \not\leadsto u \land \text{in}[u] > 0 \implies \exists u' \to u \land v \nrightarrow u' \implies \exists \text{ cycle}
		% \]
	\end{itemize}
  \end{proof}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Impacts of vertices}
  \begin{exampleblock}{Impacts of vertices (Problem 5.13)}
	Digraph $G$:
	\[
	  \text{impact}(v) = |\set{w: v \leadsto w}|
	\]

	\begin{itemize}
	  \item $\argmin_{v} \text{impact}(v)$
	  \item $\argmax_{v} \text{impact}(v)$
	\end{itemize}
  \end{exampleblock}

  \pause
  \vspace{0.50cm}
  \centerline{$\argmin_{v} \text{impact}(v) \in \text{SCC of smallest cardinality}$}

  \pause
  \begin{alertblock}{Question}
	$\forall v:$ computing $\text{impact}(v)$.
  \end{alertblock}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{One-way streets}
  \begin{exampleblock}{One-way streets (Problem 5.15)}
	Digraph $G$ for city:
    \begin{enumerate}
      \item $\forall u,v: u \leftrightsquigarrow v$
      \item $s: s \leadsto v \leadsto s$
    \end{enumerate}
  \end{exampleblock}

  \pause
  \[
	(2)\; \set{v \mid s \leadsto v} \text{ is an SCC}
  \]
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Connectivity}
  \begin{exampleblock}{Connectivity (Problem 5.7)}
	\begin{description}[Example:]
	  \item[Prove:] connected undirected graph $G$:
		\[
		  \exists v: G \setminus v \text{ is still connected}
		\]
	  \item[Example:] strongly connected digraph $G$:
		\[
		  \exists v: G \setminus v \text{ is not strongly connected}
		\]
	  \item[Example:] digraph $G$ with 2 SCCs:
		\[
		  (G + e) \text{ is not strongly connected}
		\]
	\end{description}
  \end{exampleblock}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{2SAT}
  \begin{exampleblock}{2SAT (Problem 5.17)}
	\[
	  I: (x_1 \lor \overline{x_2}) \land (\overline{x_1} \lor \overline{x_3}) \land (x_1 \lor x_2) \land (\overline{x_3} \lor x_4) \land (\overline{x_1} \lor x_4)
	\]
  \end{exampleblock}

  \pause
  \[
	\alpha \lor \beta \equiv \overline{\alpha} \to \beta \equiv \overline{\beta} \to \alpha
  \]

  \pause
  \begin{center}
	Implication graph $G_I$.
  \end{center}

  \pause
  \begin{theorem}
	\[
	  \exists \text{ SCC } \exists x: v_x \in \text{SCC} \land v_{\overline{x}} \in \text{SCC} \iff I \text{ is not satisfiable}.
	\]
  \end{theorem}

  \pause
  \begin{alertblock}{Reference}
	\begin{itemize}
	  \pause
	  \item ``A Linear-time Algorithm for Testing the Truth of Certain Quantified Boolean Formulas'' by Bengt Aspvall, Michael Plass, and Robert Tarjan, 1979.
	\end{itemize}
  \end{alertblock}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{2SAT}
  % \begin{gather*}
  %   (x_0\lor x_2)\land(x_0\lor\lnot x_3)\land(x_1\lor\lnot x_3)\land(x_1\lor\lnot x_4)\land\\
  %   (x_2\lor\lnot x_4)\land (x_0\lor\lnot x_5)\land (x_1\lor\lnot x_5)\land (x_2\lor\lnot x_5)\land\\
  %   (x_3\lor x_6)\land (x_4\lor x_6)\land (x_5\lor x_6)
  % \end{gather*}
  \begin{columns}
	\column{0.50\textwidth}
	  % \fignocaption{width = 0.80\textwidth}{figs/2sat-implication-graph-wiki.png}
	  \fignocaption{width = 0.80\textwidth}{figs/2sat-implication-graph.pdf}
	\column{0.50\textwidth}
	  \pause
	  \fignocaption{width = 0.80\textwidth}{figs/2sat-implication-graph.png}
  \end{columns}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
% \begin{frame}{Odd cycle in digraph}
%   \begin{exampleblock}{Odd cycle in digraph (Additional Problem)}
%     Find an odd cycle in a digraph $G$.
%   \end{exampleblock}
%
%   \pause
%   \begin{lemma}
%     A digraph $G$ has an odd directed cycle $\iff$ $\exists \text{scc}: $ scc is non-bipartite (when treated undirected).
%   \end{lemma}
%
%   \pause
%   \begin{alertblock}{Question}
% 	To prove the lemma and design an algorithm.
%   \end{alertblock}
%   % \begin{proof}
%   %   $\Longleftarrow$: undirected $C$; oriented
%   %     \begin{itemize}
%   %   	\item odd directed cycle
%   %   	\item choose a direction $\forall u \to v$: $\text{Len}(v \leadsto u)$
%   %     \end{itemize}
%   % \end{proof}
% \end{frame}
%%%%%%%%%%%%%%%%%%%%


\section{Determinants of \texorpdfstring{$2\times 2$}{2x2}- and \texorpdfstring{$3\times 3$}{3x3}-matrices}

\begin{outcome}
  \begin{enumerate}
    \item Calculate the determinant of $2\times 2$-matrices and
      $3\times 3$-matrices.
  \end{enumerate}
\end{outcome}

Let $A$ be an $n\times n$-matrix. The \textbf{determinant}%
\index{determinant} of $A$, denoted by $\det(A)$, is a very important
number which we will explore throughout this chapter.

The determinant of a$2\times 2$-matrix is given by the following
formula.

\begin{definition}{Determinant of a $2\times 2$-matrix}{two-by-two-determinant}
  Let $A=\begin{mymatrix}{rr}
    a & b \\
    c & d
  \end{mymatrix}$. Then
  \begin{equation*}
    \det(A) = ad-bc.
  \end{equation*}
\end{definition}

\begin{example}{A $2\times 2$ determinant}{two-by-two-determinant}
  Find $\det(A)$ for the matrix
  $A =  \begin{mymatrix}{rr}
    2 & 4 \\
    -1 & 6
  \end{mymatrix}$.
\end{example}

\begin{solution}
  We have $\det(A) = 2\cdot 6 - (-1)\cdot 4 = 12 + 4 = 16$.
\end{solution}

The determinant is also often denoted by enclosing the matrix with two
vertical lines. Thus
\begin{equation*}
  \det \begin{mymatrix}{cc}
    a & b \\
    c & d
  \end{mymatrix} =\begin{absmatrix}{cc}
    a & b \\
    c & d
  \end{absmatrix}
  = ad - bc.
\end{equation*}

\begin{definition}{Determinant of a $3\times 3$-matrix}{3-by-3-determinant}
  Let $A=\begin{mymatrix}{ccc}
    a_{11} & a_{12} & a_{13} \\
    a_{21} & a_{22} & a_{23} \\
    a_{31} & a_{32} & a_{33} \\
  \end{mymatrix}$. Then
  \begin{equation*}
    \det(A)
    = a_{11}a_{22}a_{33}
    + a_{12}a_{23}a_{31}
    + a_{13}a_{21}a_{32}
    - a_{31}a_{22}a_{13}
    - a_{32}a_{23}a_{11}
    - a_{33}a_{21}a_{12}.
  \end{equation*}
\end{definition}

The following picture may help in memorizing the formula:
\begin{equation*}
  \begin{tikzpicture}[yscale=0.6,xscale=0.9]
    \draw[ultra thick,red!30] (1,-3) -- (3,-1);
    \draw[ultra thick,red!30] (2,-3) -- (4,-1);
    \draw[ultra thick,red!30] (3,-3) -- (5,-1);
    \draw[ultra thick,blue!60] (1,-1) -- (3,-3);
    \draw[ultra thick,blue!60] (2,-1) -- (4,-3);
    \draw[ultra thick,blue!60] (3,-1) -- (5,-3);
    \foreach\i in {1,2,3} {
      \foreach\j in {1,2,3} {
        \path (\j,-\i) node {$a_{\i\j}$};
      }
    }
    \foreach\i in {1,2,3} {
      \foreach\j in {1,2} {
        \path (\j,-\i) + (3,0) node {$a_{\i\j}$};
      }
    }
    \draw(0.5,-2) node {$\left[\rule{0mm}{1.0cm}\right.$};
    \draw(3.5,-2) node {$\left.\rule{0mm}{1.0cm}\right]$};
  \end{tikzpicture}
\end{equation*}
Here, we have written down the matrix $A$, then repeated the first two
columns next to it. The blue lines correspond to the positive terms of
the determinant: $\color{blue}a_{11}a_{22}a_{33}$,
$\color{blue}a_{12}a_{23}a_{31}$, and
$\color{blue}a_{13}a_{21}a_{32}$. The pink lines correspond to the
negative terms: $\color{red}a_{31}a_{22}a_{13}$,
$\color{red}a_{32}a_{23}a_{11}$, and $\color{red}a_{33}a_{21}a_{12}$.

\begin{example}{A $3\times 3$ determinant}{3-by-3-determinant}
  Find $\det(A)$, where
  \begin{equation*}
    A = \begin{mymatrix}{rrr}
      0 & 1 & 2 \\
      3 & 1 & 0 \\
      1 & 1 & -1 \\
    \end{mymatrix}.
  \end{equation*}
\end{example}

\begin{solution}
  We have
  \begin{equation*}
    \det(A) ~=~
    \begin{absmatrix}{rrr}
      0 & 1 & 2 \\
      3 & 1 & 0 \\
      1 & 1 & -1 \\
    \end{absmatrix}
    ~=~
    0\cdot 1\cdot (-1)
    + 1\cdot 0 \cdot 1
    + 2 \cdot 3 \cdot 1
    - 1\cdot 1\cdot 2
    - 1\cdot 0\cdot 0
    - (-1)\cdot 3\cdot 1
    ~=~ 7.
  \end{equation*}
\end{solution}


\lab{Logistic Regression}{Logistic Regression}
\objective{Understand the basics of Logistic Regression, and apply to the Titanic problem.}

\subsection*{Binary Logistic Regression}
A \emph{Logistic Regression Model} is a probability model that can be used to predict outcomes for a set of data.  Usually ``Logistic Regression" refers to what is more appropriately called \emph{binary logistic regression}.  This is a model which can assign data points to one of two sets, and is used in many different fields.  One common medical example is predicting whether or not a patient has a particular disease.  Based upon several factors, which may be both continuous (age, height, weight) and categorical (gender, race), we can quantify the probability of infection.  This probability is computed by way of the \emph{logistic function}, which, given the contributing factors, will return a probability value between $0$ and $1$.  This probability will then be used to assign a label to our input data (`infected' or `not infected', for example) by using some cut-off value, and will depend on the need for accuracy in the specific application.  Success corresponds to a label of $1$, and failure to a label of $0$.

The logistic function takes in as input any real number and returns a value between $0$ and $1$, and is defined explicitly as
\begin{equation}
\phi(t) = \frac{1}{1 + e^{-s}},
\end{equation}
where $s$ is some combination of the input variables $x_1, \cdots, x_n$.  The graph of this function can be seen in Figure ???.   $\phi(x)$ can then be interpreted as the probability of success.  In some cases it is not possible to find a closed-form expression for the correct combination of the input variables.  However, in many cases we can acheive reasonable accuracy by using a linear combination of $x_i, \cdots, x_n$, i.e.
\begin{equation*}
s = c_0 + c_1 x_1 + \cdots + c_n x_n.
\end{equation*}
This can be written more compactly as
\begin{equation*}
s = \bf{c}^{T}\bf{x},
\end{equation*}
where $\bf{x}$ $= (1, x_1, \cdots , x_n)^{T}$ and $\bf{c}$ $\in \mathbb{R}^{n+1}$.

Given a training set of labeled data points, a Logistic Regression Model will find the optimal $\bf{c}$ for the data, and can then be used to predict labels for further data points.

\subsection*{The Titanic Problem}
The Titanic dataset is especially useful for Logistic Regression.  This dataset is composed of actual data obtained concerning the passengers on the ill-fated Titanic voyage, given in the bulleted list below.  Using logistic regression, we can predict whether or not a passenger survived based on this data.  We do so by training a model on a portion of the dataset, the training set, and predicting labels for the remaining data, the test set.

Before beginning our classification, however, we will first need to process our data.  The Titanic dataset contains much more information than is currently relevant for our purposes.  You can obtain this data in Excel Spreadsheet form at [Insert link here].  We recommend using pandas to read in and process the data.  The columns are as follows:
\begin{itemize}
\item \li{pclass}: An integer in $\{1, 2, 3\}$ which describes the class the passenger was in.
\item \li{survived}:  The dependent variable.  $1$ indicates survival, and $0$ death.
\item \li{name}: A string containing the passenger's name.
\item \li{sex}: A string, either `male' or `female'.
\item \li{age}: Either an integer or a float.
\item \li{sibsp}: An integer giving the number of siblings and/or spouse who embarked with the passenger.
\item \li{parch}: An integer giving the number of parents and/or children who embarked with the passenger.
\item \li{ticket}: A string containing the transaction code for the ticket(s) purchased.
\item \li{fare}: A float giving the cost of the ticket purchased.
\item \li{cabin}: A string giving the assigned sleeping cabin for the passenger (note that the majority of this column is blank).
\item \li{embarked}: A string in $\{S, C, Q\}$ corresponding to the location of the passenger's embarkment, Southampton (UK), Cherbourg (France), or Queenstown (Ireland), respectively.
\item \li{boat}: An int or string for those who survived giving which life boat they rode in.
\item \li{body}: An int giving the number of body for those who died who were found and identified.
\item \li{home.dest}: A string giving the home of or location to which the passenger was headed.
\end{itemize}

\begin{problem}
Create a function called \li{initialize} which will process the Titanic data set into useable format by doing the following:
\begin{enumerate}
\item Choose the coulmns that you believe will be relevant in predicting the survival of the passengers, and drop the other columns.  You may not use \li{boat} or \li{body}, as these are dependent on whether or not the passenger survived.  Be sure to include \li{survived}, which will be separated later as the independent variable, as well as \li{sex} and \li{pclass}.
\item Since \li{sex} is really a binary variable, make it one explicitly by changing ``female" and ``male" to be binary values.
\item Drop the rows that contain missing values.  Make sure you have a significant number of rows left.  If you have too few, you may need to choose fewer columns to keep before deleting the incomplete rows.
\item Because the \li{pclass} column is an integer in $\{1, 2, 3\}$, it will be treated as a ranked variable instead of simply a categorical variable.  It may be useful to rank this variable, or it may mess up our classification.  Include a keyword argument \li{pclass_change} with default \li{True}.  If it is set to \li{True}, eliminate this ranking by dividing \li{pclass} into two binary columns.  Make one column a boolean for being $1^{st}$ class and the other a boolean for being $2^{nd}$ class. (This means that a value of $1$ would correspond to $[1, 0]$, $2$ to $[0, 1]$, and $3$ to $[0, 0]$.)
\item Split the remaining rows into a training and a test set using a $60/40$ split.  Be sure and pick random rows for each group and not rows in any particular order.
\end{enumerate}
Have your function return the training set and the test set, in that order.
\end{problem}

\subsection*{Model Evaluation}
Now that we have our training set, we can train a model, which can be used to obtain the probability of success for each data point.  The label chosen depends heavily on the probability cut-off value mentioned previously, which we represent as $\tau$.  In the simplest manner, we can simply pick a value of $\tau = 0.5$, which will then assign the label with the highest likelihood.  However, it is often beneficial to choose a different value of $\tau$.  In regards to the probability of infection for serious diseases, it might be best to give a patient medicine if they have even a $10$ percent chance of infection.  So how can we find the ``best" cut-off value?

In order to determine this, we need to discuss how to measure the accuracy of the labels predicted.  Say that we have picked a cut-off value $\tau$, and have assigned labels to the test set.  Using the predicted labels and the actual labels, we can obtain four important values: the number of \emph{true positives}, \emph{false positives}, \emph{true negatives}, and \emph{false positives}, which are abbreviated TP, FP, TN, and FP, respectively.  These values are integers which together sum to the number of labels predicted.  You can see the definition of these in Table ??? (until the figure is up, true positives are the points with predicted label 1 and true label 1, false positives have predicted label 1 true label 0, true negatives predicted label 0 true label 0, and false negatives true label 1 predicted label 0).  We can now use these to report our accuracy in various metrics:
\begin{itemize}
\item \emph{Prediction accuracy} is defined as $\frac{TP+TN}{TP+FN+FP+TN}$, and is the percentage of correctly predicted cases.
\item \emph{Sensitivity}, also known as the \emph{true positive rate} or $TPR$, is given by the fraction of correctly predicted cases where the actual outcome is $1$, $\frac{TP}{TP+FN}$.
\item \emph{Specificity}, the \emph{true negative rate}, or $TNR$ is the proportion of correctly predicted cases where the true outcome is $0$, $\frac{TN}{FP+TN}$.
\item \emph{False Positive Rate}, or $FPR$, is the proportion of incorrectly predicted cases where the true outcome is $0$, and is given by $\frac{FP}{FP+TN}$.
\end{itemize}
All of these depend strongly on the value chosen for $\tau$.

A \emph{roc curve} is useful in measuring the accuracy of a model.  To make a roc curve, we pick many values for $\tau$, and obtain the False Positive Rate and the True Positive Rate for each.  Then we plot the data points  $(FPR_{\tau},TPR_{\tau})$ for each value of $\tau$ chosen and connect them into a curve.  A completely random label assignment would result in a nearly-linear roc curve, while a more accurate assignment would result in a more steeply-rising curve (see Figure ???).  A good choice for $\tau$ is the one that intersects the family of lines $y=x+b$ at only one point, which intuitively is the point closest to the vertex $(0, 1)$.  Mathematically, this is given by
\begin{equation}
\operatorname*{arg\,max}_{\tau} (TPR_{\tau} - FPR_{\tau}).
\end{equation}

\begin{problem}
Use the function declaration below to find the best value for $\tau$.  You should use evenly spaced values from $0$ to $1$, exclusive.
\begin{lstlisting}
def best_tau(predicted_labels, true_labels, n_tau=100, plot=True):
    """
    Parameters
    ----------
    predicted_labels : ndarray of shape (n,)
        The predicted labels for the data
    true_labels : ndarray of shape (n,)
        The actual labels for the data
    n_tau : int
        The number of values to try for tau
    plot : boolean
        Whether or not to plot the roc curve

    Returns
    -------
    best_tau : float
        The optimal value for tau for the data.
    """
    pass
\end{lstlisting}
\end{problem}

Now that we have a good value for $\tau$, we can quantify the accuracy of a model. We will do so for a few different types of models for the Titanic data you initialized previously.  For the first two, we will use the logistic classifier found in \li{sklearn.linear_model.LogisticRegression}.  The first model we use will be our ``Unchanged Logistic Classifier", which will use our Titanic data with \li{pclass} unchanged.  The second model is the ``Changed Logistic Classifier", which use the data with \li{pclass} changed.

When using this package to create a classifier, we need to input a keyword argument \li{C}.  This value represents the inverse of the regularization strength.  Different values will yield different results.  A better value for \li{C} will yield a more steeply-rising roc curve.  The model can find the coefficients (the vector $\bf{c}$), along with the probabilities of failure and success for each label.  With these in hand, we can use the function \li{sklearn.metrics.roc_curve} to obtain the False Positive Rates, the True Positive Rates, and the optimal value for $\tau$.  You will need to pass in the test data, the probability of success, and the keyword argument \li{pos_label} $= 1$.  The accuracy of the model with input value \li{C} can then be obtained using \li{sklearn.metrics.auc}, which will give the area under the curve.  The larger the area, the more steeply-rising curve we have, and the better the model.  Note that we create a single roc curve using one value for \li{C} and multiple values for $\tau$.

\begin{problem}
Use the following function declaration to return the auc score for the two Logistic Regression models described.
\begin{lstlisting}
def auc_scores(unchanged_logreg, changed_logreg):
    """
    Parameters
    ----------
    unchanged_logreg : float in (0,1)
        The value to use for C in the unchanged model
    changed_logreg : float in (0,1)
        The value to use for C in the changed model

    Returns
    -------
    unchanged_auc : float
        The auc for the unchanged model
    changed_auc : float
        The auc for the changed model
    """
    pass
\end{lstlisting}
\end{problem}

We can test a Naive Bayes model against our Logistic Regression model for both the unchanged and changed models to see the comparative accuracy.  Use the model \li{MultinomialNB}, found in \li{sklearn.naive_bayes}.  You can use it in the same manner as \li{LogisticRegression}, except instead of passing in the keyword argument \li{C}, you will pass in a keyword argument \li{alpha} corresponding to a smoothing parameter.
\begin{problem}
Add input variables \li{unchanged_bayes} and \li{changed_bayes} to your function from the previous problem to obtain the auc for each of these models.  Your function should return all four areas, unchanged logistic regression, changed logistic regression, unchanged Bayes, and changed Bayes, in that order.
\end{problem}

Different values for \li{C} and \li{alpha} will yield different results.  We seek to find those that will maximize the area under the curve.  One way to do so is to pick a number of evenly-spaced points between $0$ and $1$, exclusive, and try each one in turn, keeping the value that yields the greatest accuracy.
\begin{problem}
Use the function declaration below to find the optimal values for \li{C} and $alpha$ as described.
\begin{lstlisting}
def find_best_parameters(choices):
    """
    Parameters
    ----------
    choices : int
        The number of values to try for C and alpha

    Returns
    -------
    best : list of length 4
        The best values for C for the unchanged and changed logistic
         regression models, and the best values for alpha for the
         unchanged and changed Naive Bayes models, respectively.
    """
    pass
\end{lstlisting}
\end{problem}

Now that we have found the optimal inputs for these functions, we can test them against one another.
\begin{problem}
Create a function called \li{results} which will graph of the roc curves for each of the methods, and will print out the names of the models with their corresponding areas, in numerically descending order.
\end{problem}


\section{Result}
\subsection{Plot}
\begin{figure}[!hbtp]
    \centering
    \includegraphics[width=0.8\textwidth]{../src/lab_11_figure.pdf}
    \caption{Gauss Qudrature Error}
    \label{fig:my_label}
\end{figure}
\newpage
\subsection{Output file: \lstinline[style=Plain]{lab_11_output.txt}}
\lstinputlisting[style=Plain]{../src/lab_11_output.txt}
\newpage
\section{Script and Function files}
\subsection{Script file: \lstinline[style=Plain]{lab_11_script_01.m}}
\lstinputlisting[style=MATLAB]{../src/lab_11_script.m}
\newpage
\subsection{Function file: \lstinline[style=Plain]{gauss_quad.m}}
\lstinputlisting[style=MATLAB]{../src/gauss_quad.m}


\documentclass[12pt, a4paper]{report}

\usepackage{amsfonts}
\usepackage[usenames,dvipsnames,svgnames,table]{xcolor}
\usepackage{titlesec}
\usepackage{hyperref}
\hypersetup{
	colorlinks,
	citecolor=Fuchsia,
	filecolor=black,
	linkcolor=Sepia,
	urlcolor=blue
}

\titleformat{\chapter}{\normalfont\huge\bfseries}{\thechapter.}{10pt}{\huge\bfseries}

% Title Page
\title{Fourier-Motzkin extension to Multivariate Polynomial Integer Constraints}

\author{Diogo Sampaio}

\begin{document}

\maketitle

\begin{tabular}[c]{|c|c|p{0.5\linewidth}|}
	\hline
	2017 - 12 - 07 & Diogo Sampaio & First version\\
	\hline
\end{tabular}

\tableofcontents

\chapter{Introduction}
Quantifier elimination is the process of removing existential variables of a given first-order formula, obtaining one that is simpler in the number of variables, and that implies the original formula.

A very well known algorithm is the Fourier-Motzkin elimination process, that given a system (or formula) of inequalities removes one quantified variable by combining all of it's upper and lower bounds.

For example, let the system:
$$\exists x ~ | ~ 0 < y \land z < 1 \land y < x \land x < z$$
In such formula, $x$ is a existential quantified variable, and $y, z$ are parameters. Using FME to remove variable $x$ from this formula requires to find it's upper and lower bounds. In this case we have the upper bound $$x < z$$ and the lower bound $$y < x.$$ For last $x$ is eliminated by isolating $x$ of each formula and combining lower with upper bounds such as:
$$y < x < z \Rightarrow y < z$$
generating the new system:
$$0 < y \land z < 1 \land y < z.$$

This algorithm is designed for linear systems, where all coefficients of the variable being eliminated are numeric values, and the inequality can be classified as either a upper or lower bound.

When dealing with polynomials, variable coefficients might be symbolic expressions. In such case, all possible signs of the coefficient (positive, negative, or zero) must be explored.
e.g. imagine the system:
$$\exists x ~ | ~ a - 1 > 0 \land -b > 0 \land ax > 0 \land bx > 0$$
when eliminating the variable $x$ it is required to classify the terms $ax > 0$ and $bx > 0$ as either upper or lower bounds of $x$. Not knowing the sign of the coefficients ($a, b$) requires to evaluate nine different combinations:
$$a > 0 \land b > 0$$
$$a > 0 \land b = 0$${\color{red}$$a > 0 \land b < 0$$}$$a = 0 \land b > 0$$
$$a = 0 \land b = 0$$
$$a = 0 \land b < 0$$
$$a < 0 \land b > 0$$
$$a < 0 \land b = 0$$
$$a < 0 \land b < 0$$
To avoid this branching we use an implementation of a theorem used in the positiveness test algorithm, proposed by Markus Schweighofer (\url{https://doi.org/10.1016/S0022-4049(01)00041-X}), to retrieve symbolic coefficient signs. Using such algorithm it is possible to infer that only the red assumption holds correct.

The same positiveness test algorithm is of major importance when resolving system over integer variables, instead of reals. It is used in many other techniques required to preserve the precision of the simplified formula, such as extending the normalization (\url{https://doi.org/10.1145/125826.125848}) ~ technique to symbolic expressions, perform convex hull detection and remove redundant constraints.

Our C++ implementation uses GiNaC (\url{https://www.ginac.de/}) to manipulate symbolic expressions and GLPK to implement the positiveness test (\url{https://www.gnu.org/software/glpk}).

For further details please refer to \emph{Profile Guided Hybrid Compilation} \S2.7 and \S5. (\url{https://hal.archives-ouvertes.fr/tel-01428425})

\chapter{Access to it}
\section{Code access and description}
The code is publicly available for download at:

\url{https://gitlab.inria.fr/nunessam/pghc.git}

Source-code folders contents:
\begin{itemize}
	\item doc: this document.
	\item FM: the FME and a stand-alone positiveness tester.
	\item converters: distinct converters to generate input to other quantifier-elimination tools, such as
	\begin{itemize}
	\item QEPCAD: \url{https://github.com/PetterS/qepcad}
	\item reduce/redlog: \url{https://sourceforge.net/projects/reduce-algebra/}
	\end{itemize}
	as well as input to barvinok \url{http://barvinok.gforge.inria.fr/} where it tries to discover if it is possible to prove if a system of constraints define a empty / absurd space. For such task it computes maximum and minimum values of polynomials inside a convex polyhedron space (see \url{https://icps.u-strasbg.fr/upload/icps-2006-173.pdf}).
	\item packages: Dockerfile for a painless process of install.
\end{itemize}
\section{Compiling requirements}
\begin{itemize}
	\item C++11 compatible compiler
	\item GNU make, GiNaC, GLPK
\end{itemize}
{\color{red}OBS: These tools where tested solely in \textbf{Linux}.}

\chapter{How to use}
\section{Fourier-Motzkin elimination - Text IO}
\subsection{Inputs}
The application can either read a input text file or it can read from the standard input stream. Example files can be seen with extension \texttt{.in} inside the folder FM. The binary file it self is called {\color{red}\textbf{\texttt{Simplifier}}}.
\subsection{Input format}
The input formula must be given in a disjunction of systems, where each system is a conjunction of terms. In a bottom up manner we have that:
\begin{itemize}
	\item A \emph{term} is a INTEGER VALUED equality or inequality, such as:	\begin{verbatim}4*x^2 - 5*y < 0
x > 0
y <= s
N*h - 5*z == 0
	\end{verbatim}
	\item A system of terms declares all quantified variables, parameter such as the example in the file \texttt{FM/POL.in}:
	\begin{verbatim}[n]->[jj,j,k,kk,ii,i]: -kk^2+j+j^2-kk-2*ii+2*i = 0 &
		k >= 0 & -1-k+n >= 0 &
		i >= 0 & -1+k-i >= 0 &
		-1+j-k >= 0 & -1-j+n >= 0 &
		kk >= 0 & -1-kk+n >= 0 &
		ii >= 0 & -1+kk-ii >= 0 &
		-1+jj-kk >= 0 & -1-jj+n >= 0 &
		-1-k+kk >= 0 & -1-ii+i >= 0 &
		n >= 0;
		\end{verbatim}
		The distinct parts of the system are:
		\begin{itemize}
		\item A system starts with a list, comma separated, of parameters, e.g.: \texttt{[n]}, \texttt{[nn,n, z, b]}, \texttt{[]}.
		\item An arrow \texttt{->} describing "mapping from".
		\item A list of the quantified variables to be eliminated, comma separated e.g.: \texttt{[jj,j,k,kk,ii,i]}, \texttt{[a, bc,c]}, \texttt{[]}.
		\item A list of polynomial terms, initialized by a colon(\texttt{:}) and separated by the logic and symbol (\texttt{\&}) and terminated by a semi-colon (\texttt{;}).
	\end{itemize}

	\item Multiple systems can be represented in a single input file, being interpreted as an \texttt{logic or} of each one of the systems:
	\begin{verbatim}
	[]->[x]: x < 1 & x > 0;
	[x]->[a]: a < x & a > 3*x-4;
	\end{verbatim}
	is interpreted as $\{\exists x \in \mathbb{Z} | x < 1 \land x > 0\} \lor \{\exists a \in \mathbb{Z} | a < x \land 3x-4 < a\}$.
\end{itemize}

\section{Fourier-Motzlin elimination - Python Interface}

In the folder

\section{Schweighofer Tester}
In the same FM folder, the binary file \texttt{st} allows to interactively play with a single system of constraints. It can read from input files or iteratively add expressions to the system. It allows the user to:
\begin{itemize}
	\item ~[A]ppend or [D]elete a constraint.
	\item ~[T]est if an expression is implied by the system.
	\item ~Obtain the [R]oot expressions of the system.
	\item ~[S]how the generated expression of the Schweighofer Tester.
	\item ~Show the Schweighofer Tester [m]onomials generated.
	\item ~[C]hange the degree (number of multiplications) the Schweighofer Tester multiply all system constraints against all others.
\end{itemize}

\end{document}


% !TEX root = main.tex

%-------------------------------------------------
\section{Simple linear regression}\label{sec:regression}

Let $X$ and $Y$ be continuous random variables. We wish to investigate how $X$ influences the behaviour of $Y$.
\bit
\it $X$ is called the \emph{explanatory variable} or the \emph{independent variable}.
\it $Y$ is called the \emph{response variable} or the \emph{dependent variable}.
\eit

Suppose we observe that $X$ takes the value $x$. Unless $Y$ is completely determined by $X$, we cannot predict its value with certainty so instead we focus on the problem of estimating its conditional expectation $E(Y|X=x)$. This leads us to represent $Y$ as the sum of two random variables:
\[
Y = \mu(X) + \epsilon  \qquad\text{where $\epsilon\sim N(0,\sigma^2)$.}
\]
\bit
\it $\mu(x) = \expe(Y|X=x)$ is called the \emph{regression function};
\it $\epsilon = Y - \expe(Y|X)$ is called the \emph{error variable}.
\eit

%% lemma
%%By the law of total expectation, the expected value of the error variable is zero.
%\begin{lemma}
%The expected value of the error variable is zero.
%%$\expe(\epsilon) = 0$.
%\end{lemma}
%\begin{proof}
%By the law of total expectation,
%\[
%\expe(\epsilon) = \expe\big[Y - \expe(Y|X)\big] = \expe(Y) - \expe\big[\expe(Y|X)\big] = \expe(Y)-\expe(Y) = 0.
%\]
%\end{proof}

%% lemma
%%The law of total variance divides the variance of $Y$ into a component attributed to the explanatory variable $X$, and a component attributed to the error variable $\epsilon$.
%The following lemma shows that the variance of $Y$ can be divided into a component attributed to the explanatory variable $X$, and a component attributed to the error variable $\epsilon$.
%\begin{lemma}
%%$\var(Y) = \var(\mu) + \expe\big[\var(\epsilon|X)\big]$.
%If the error variable $\epsilon$ is independent of the explanatory variable $X$,
%\[\var(Y) = \var(\mu) + \var(\epsilon).
%\]
%\end{lemma}
%\begin{proof}
%$\epsilon = Y - \expe(Y|X)$, so (by the definition of conditional variance),
%\[
%\var(Y|X) = \expe\big(\big[Y-\expe(Y|X)\big]^2|X\big) = \expe(\epsilon^2|X) = \var(\epsilon|X) = \var(\epsilon).
%\]
%By the law of total variance,
%\begin{align*}
%\var(Y)
%	& = \var\big[\expe(Y|X)\big] + \expe\big[\var(Y|X)\big] \\
%	& = \var\big[\mu(X)\big] + \expe\big[\var(\epsilon)\big]
%	& = \var\big[\mu(X)\big] + \var(\epsilon).
%\end{align*}
%\end{proof}
%
%\begin{remark}
%\bit
%\it $\var(\mu) = \var\big[\expe(Y|X)\big]$ is the \emph{explained} variance.
%%\it $\expe\big[\var(\epsilon|X)\big] = \expe\big[\var(Y|X)\big]$ is the \emph{unexplained} variance.
%\it $\var(\epsilon) = \var(Y|X)$ is the \emph{unexplained} variance.
%\eit
%\end{remark}

% lemma
%The law of total variance divides the variance of $Y$ into a component attributed to the explanatory variable $X$, and a component attributed to the error variable $\epsilon$.
%The following lemma shows that the variance of $Y$ can be divided into a component attributed to the explanatory variable $X$, and a component attributed to the error variable $\epsilon$.
\begin{lemma}\label{lem:partition-of-variance}
%$\var(Y) = \var(\mu) + \expe\big[\var(\epsilon|X)\big]$.
$
\var(Y) = \var\big[\mu(X)\big] + \var(\epsilon).
$
\end{lemma}
\begin{proof}
$\epsilon = Y - \expe(Y|X)$, so by the definition of conditional variance,
\[
\var(Y|X) = \expe\big(\big[Y-\expe(Y|X)\big]^2|X\big) = \expe(\epsilon^2|X) = \var(\epsilon|X) = \var(\epsilon).
\]
By the law of total variance,
\begin{align*}
\var(Y)
	& = \var\big[\expe(Y|X)\big] + \expe\big[\var(Y|X)\big] \\
	& = \var\big[\mu(X)\big] + \expe\big[\var(\epsilon)\big] \\
	& = \var\big[\mu(X)\big] + \var(\epsilon).
\end{align*}
\end{proof}
Lemma~\ref{lem:partition-of-variance} expresses the variance of $Y$ as the sum of an \emph{explained variance}, attributed to the variance of the explanatory variable $X$, and an \emph{unexplained variance} attributed to the error variable $\epsilon$.
%\begin{remark}
%\bit
%\it $\var\big[\mu(X)\big] = \var\big[\expe(Y|X)\big]$ is the \emph{explained} variance.
%\it $\var(\epsilon) = \var(Y|X)$ is the \emph{unexplained} variance.
%\eit
%\end{remark}

%-----------------------------
\subsection{Linear models}

\begin{definition}
A \emph{linear model} is a model which is linear in its parameters.
\bit
\it $\mu(x) = \alpha + \beta x + \gamma x^2$ is a linear model.
\it $\mu(x) = \alpha e^{\beta x}$ is not a linear model.
\eit
A \emph{simple linear model} is a linear model of the form $\mu(x) = \alpha + \beta x$.
\end{definition}

The simple linear regression model is
\[
Y = \alpha + \beta X + \epsilon \quad\text{where}\quad \epsilon\sim N(0,\sigma^2).
\]
Given that $X=x$, we have that $Y\sim N(\alpha+\beta x, \sigma^2)$ and in particular,
\[
\expe(Y|X=x) = \alpha + \beta x
\quad\text{and}\quad
\var(Y|X=x) = \sigma^2.
\]

%-----------------------------
\subsection{Test statistics}

Let $(X_1,Y_1),(X_2,Y_2),\ldots,(X_n,Y_n)$ be a random sample from the joint distribution of $X$ and $Y$.
In Week~\ref{chap:likelihood} we saw that the maximum likelihood estimators of $\alpha$, $\beta$ and $\sigma^2$ are respectively
\[
\hat{\alpha} = \bar{Y}-\hat{\beta}\bar{X},\qquad
\hat{\beta} = \frac{\sum_{i=1}^n (X_i-\bar{X})(Y_i-\bar{Y})}{\sum_{i=1}^n (X_i-\bar{X})^2}
\qquad\text{and}\qquad
\hat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^n \hat{\epsilon}_i^2
\]
where $\hat{\epsilon}_i = Y_i-(\hat{\alpha}+\hat{\beta} X_i)$ is the so-called \emph{residual variable} at $X_i$.

%%-----------------------------
%\subsection{Residual variance}
%
%\begin{definition}
%%Let $(x_1,x_2,\ldots,x_n)$ be a realisation of the marginal sample $(X_1,X_2,\ldots,X_n)$.
%%Given $X=x$,
%
%\ben
%\it $\hat{y} = \hat{\alpha} + \hat{\beta}X$ is called the \emph{predicted value of $Y$} at $X$.
%\it $\hat{\epsilon} = Y - \hat{y}$ is called the \emph{residual variable} at $X$.
%\een
%\end{definition}

%% theorem: mle of alpha and beta
%\begin{theorem}
%The maximum likelihood estimator of the error variance $\sigma^2$ is the sample mean of the squared residuals,
%\[
%\hat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^n \hat{\epsilon}_i^2
%\text{\quad where\quad}
%\hat{\epsilon}_i = Y_i-(\hat{\alpha}+\hat{\beta} X_i).
%\]
%\end{theorem}
%
%% proof
%\begin{proof}
%Recall the log-likelihood function:
%\[
%\ell(\alpha,\beta,\sigma^2)
%	= \frac{n}{2}\log(2\pi\sigma^2) + \frac{1}{2\sigma^2}\sum_{i=1}^n \big[y_i-(\alpha+\beta x_i)\big]^2.
%\]
%The first partial derivative of $\ell(\alpha,\beta,\sigma^2)$ with respect to $\sigma^2$ is
%\[
%\frac{\partial\ell}{\partial(\sigma^2)}
%	= \frac{n}{2\sigma^2} - \frac{1}{2(\sigma^2)^2}\sum_{i=1}^n \big[y_i-(\alpha+\beta x_i)\big]^2.
%\]
%Setting this equal to zero,
%\[
%\sigma^2 = \frac{1}{n}\sum_{i=1}^n \big[y_i-(\alpha+\beta x_i)\big]^2.
%\]
%Substituting our estimates for $\alpha$ and $\beta$, we obtain the MLE
%\[
%\hat{\sigma^2} = \frac{1}{n}\sum_{i=1}^n \hat{\epsilon}_i^2 \text{\quad where\quad} \hat{\epsilon}_i = Y_i-(\hat{\alpha}+\hat{\beta} x_i)
%\]
%as required.
%
%\end{proof}
%
%\begin{remark}[Residual Analysis]
%Our model assumes that $\epsilon\sim N(0,\sigma^2)$ and that $\epsilon$ is independent of $X$. To test whether these assumptions hold, we plot the points $(x_i,\hat{\epsilon}_i)$ on a scatter diagram. If the assumptions do indeed hold, the points should be evenly spread about the horizontal axis, and the extent of their spread should not depend on the $x$-coordinate.
%This is an example of \emph{residual analysis}.
%\end{remark}
%
%% exercise
%\begin{exercise}
%Following a class test, 10 students were asked about the number of hours they had revised for the test. The data is shown in the table below.
%\begin{center}
%\begin{tabular}{|l|cccccccccc|} \hline
%Hours studied ($x$)	&  4	 &  9 & 10 & 14 &  4 &  7 & 12 & 22 &  1 & 17 \\
%Test score ($y$)		& 31 & 58 & 65 & 73 & 37 & 44 & 60 & 91 & 21 & 84 \\ \hline
%\end{tabular}
%\end{center}
%Perform a simple linear regression to estimate the relationship between the number of hours studied and the score achieved in the test.
%\begin{answer}
%%It is easy to show that
%%\begin{align*}
%%\sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})
%%		& = \sum_{i=1}^n x_iy_i - \frac{1}{n}\left(\sum_{i=1}^n x_i\right)\left(\sum_{i=1}^n y_i\right) \text{ and} \\
%%\sum_{i=1}^n (x_i - \bar{x})^2
%%		& = \sum_{i=1}^n x_i^2 - \frac{1}{n}\left(\sum_{i=1}^n x_i\right)^2.
%%\end{align*}
%%
%%From the table,
%%\bit
%%\it $n=10$,
%%\it $\sum_i x_i = 100$ and $\sum_i y_i = 564$,
%%\it $\sum_i x_i^2 = 1376$ and $\sum_i x_iy_i = 6945$.
%%\eit
%%This yields
%Tedious calculations yield
%\[
%\sum_{i=1}^n(x_i-\bar{x})^2  = 376 \text{\quad and\quad} \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y}) = 1305.
%\]
%Thus
%\begin{align*}
%\hat{\beta}
%	& = \displaystyle\frac{\sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^n(x_i-\bar{x})^2}
%	= \displaystyle\frac{1305}{376}	= 3.47,\\
%\intertext{and}
%\hat{\alpha}
%	& = \displaystyle\bar{y} - \hat{\beta}\bar{x}
%	= \displaystyle\frac{564}{10} - \left(\frac{1305}{376}\right)\left(\frac{100}{10}\right) = 21.69.
%\end{align*}
%The estimated relationship is therefore $\hat{y} = 21.69 + 3.471 x$.
%\end{answer}
%\end{exercise}

%-----------------------------
%\subsection{}
\bigskip
Let $x_1,x_2,\ldots,x_n$ be a fixed realisation of the marginal sample $X_1,X_2,\ldots,X_n$. Then $\hat{\alpha}$, $\hat{\beta}$ and $\hat{\epsilon}_i$ are all linear combinations of $Y_1,Y_2,\ldots,Y_n$. Because the $Y_i$ are independent normal variables it thus follows that $\hat{\alpha}$, $\hat{\beta}$ and $\hat{\epsilon}_i$ are also normal variables.

\begin{theorem}
The MLEs of $\alpha$, $\beta$ and $\sigma^2$ respectively satisfy
\[
\hat{\alpha} \sim N\left(\alpha,\frac{\sigma^2}{n}\right),
\quad
\hat{\beta} \sim N\left(\beta,\frac{\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right)
\quad\text{and}\quad
\frac{n\hat{\sigma}}{\sigma} \sim \chi^2_{n-2}.
\]
%respectively.
\end{theorem}

\begin{proof}
\ben
\it % alpha
The expected value of $\hat{\alpha}$ is
\begin{align*}
\expe(\hat{\alpha})
	= \expe(\bar{Y}-\beta\bar{x})
	& = \expe\left(\frac{1}{n}\sum_{i=1}^n Y_i - \frac{\beta}{n}\sum_{i=1}^n x_i\right) \\
	& = \frac{1}{n}\sum_{i=1}^n \expe(Y_i) - \frac{\beta}{n}\sum_{i=1}^n x_i \\
	& = \frac{1}{n}\sum_{i=1}^n (\alpha+\beta x_i) - \frac{\beta}{n}\sum_{i=1}^n x_i \\
	& = \alpha.
\end{align*}
Because $\var(Y_i)=\sigma^2$, the variance of $\hat{\alpha}$ is
\begin{align*}
\var(\hat{\alpha})
	& = \var\left(\frac{1}{n}\sum_{i=1}^n Y_i - \frac{\beta}{n}\sum_{i=1}^n x_i\right) \\
	& = \frac{1}{n^2}\sum_{i=1}^n \var(Y_i)
	= \frac{\sigma^2}{n}.
\end{align*}
Hence $\hat{\alpha} \sim N(\alpha,\sigma^2/n)$, as required.
\it % beta
Since $\expe(Y_i) = \alpha + \beta x_i$ and $\expe(\bar{Y}) = \alpha + \beta\bar{x}$, the expected value of $\hat{\beta}$ is therefore
\begin{align*}
\expe(\hat{\beta})
	& = \expe\left[\frac{\sum_{i=1}^n (x_i-\bar{x})(Y_i-\bar{Y})}{\sum_{i=1}^n (x_i-\bar{x})^2}\right] \\
	& = \frac{\sum_{i=1}^n (x_i-\bar{x})\expe(Y_i-\bar{Y})}{\sum_{i=1}^n (x_i-\bar{x})^2}
`	= \frac{\sum_{i=1}^n \beta(x_i-\bar{x})^2}{\sum_{i=1}^n (x_i-\bar{x})^2} = \beta.
\end{align*}
Using the fact that $\sum_{i=1}^n x_i = n\bar{x}$, it is easy to see that $\hat{\beta}$ can be rewritten as
\[
\hat{\beta} = \frac{\sum_{i=1}^n (x_i-\bar{x})Y_i}{\sum_{i=1}^n (x_i-\bar{x})^2}.
\]
Because $\var(Y_i)=\sigma^2$, the variance of $\hat{\beta}$ is
\begin{align*}
\var(\hat{\beta})
	= \var\left[\frac{\sum_{i=1}^n (x_i-\bar{x})Y_i}{\sum_{i=1}^n (x_i-\bar{x})^2}\right]
	& = \frac{1}{\left[\sum_{i=1}^n(x_i-\bar{x})^2\right]^2}\sum_{i=1}^n(x_i-\bar{x})^2\var(Y_i) \\
	& = \frac{\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}.
\end{align*}
Hence $\hat{\beta}\sim N\left(\beta,\frac{\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right)$, as required.
\it % sigma^2
Recall that
\[
\hat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^n \hat{\epsilon_i}^2
\quad\text{where}\quad
\hat{\epsilon}_i = Y_i-(\hat{\alpha}+\hat{\beta} x_i).
\]
Consider
\begin{align*}
\frac{1}{\sigma^2}\sum_{i=1}^n\big[Y_i-(\alpha+\beta x_i)\big]^2
	& = \frac{1}{\sigma^2}\sum_{i=1}^n\big[(\hat{\alpha}-\alpha) + (\hat{\beta}-\beta)x_i + (Y_i-(\hat{\alpha}+\hat{\beta}x_i))\big]^2 \\
	& = \frac{n(\hat{\alpha}-\alpha)^2}{\sigma^2} + \frac{(\hat{\beta}-\beta)^2}{\sigma^2}\sum_{i=1}^n x_i^2 + \frac{n\hat{\sigma}^2}{\sigma^2}.
\end{align*}
The first three terms in this expression all have chi-squared distribution.
\bit
\it
Because $Y_i\sim N(\alpha+\beta x_i,\sigma^2)$ it follows that $\big[Y_i-(\alpha+\beta x_i)\big]/\sigma\sim N(0,1)$, so
\[
\displaystyle\frac{1}{\sigma^2}\sum_{i=1}^n\big[Y_i-(\alpha+\beta x_i)\big]^2\sim\chi^2_{n}.
\]
\it
Because $\hat{\alpha}\sim N(\alpha,\sigma^2/n)$ it follows that $\sqrt{n}(\hat{\alpha}-\alpha)/\sigma\sim N(0,1)$, so
\[
\displaystyle\frac{n(\hat{\alpha}-\alpha)^2}{\sigma^2} \sim \chi^2_1.
\]
\it
Because $\hat{\beta}\sim N\left(\beta,\sigma^2/\sum_{i=1}^n(x_i-\bar{x})^2\right)$ it follows that $(\hat{\beta}-\beta)\sqrt{\sum_{i=1}^n x_i^2}/\sigma \sim N(0,1)$, so
\[
\frac{(\hat{\beta}-\beta)^2}{\sigma^2}\sum_{i=1}^n x_i^2 \sim \chi^2_1.
\]
\eit
It is easy to see that if $U\sim\chi^2_a$ and $V\sim\chi^2_b$ are independent, then $U+V\sim\chi^2_{a+b}$. It thus follows that $n\hat{\sigma}/\sigma\sim \chi^2_{n-2}$ as required.
\een
\end{proof}

We estimate the error variance using the following unbiased estimator (instead of the MLE),
\[
\hat{\sigma}^2 = \frac{1}{n-2}\sum_{i=1}^n \hat{\epsilon}_i^2
\quad\text{where}\quad
\hat{\epsilon}_i = Y_i-(\hat{\alpha}+\hat{\beta} x_i).
\]
This estimator for $\sigma^2$ yields the following test statistics:
\[
T_1 = \frac{\hat{\alpha}-\alpha}{\sqrt{\hat{\sigma}^2/(n-2)}}
\quad\text{and}\quad
T_2 = \frac{\hat{\beta}-\beta}{\sqrt{n\hat{\sigma}^2/[(n-2)\sum_{i=1}^n(x_i-\bar{x})^2]}}.
\]

\bit
\it Under the null hypothesis $H_0:\alpha=0$,
\[
T_1 = \frac{\hat{\alpha}-\alpha}{\sqrt{\hat{\sigma}^2/(n-2)}} \sim t_{n-2}.
\]
\it Uhder the null hypothesis $H_0:\beta=0$,
\[
T_2 = \frac{\hat{\beta}-\beta}{\sqrt{n\hat{\sigma}^2/[(n-2)\sum_{i=1}^n(x_i-\bar{x})^2]}} \sim t_{n-2}.
\]
\eit

$T_2$ can be used to test whether or not $Y$ depends (linearly) on $X$:
\begin{align*}
H_0: 	&\ Y = \alpha+\epsilon, \\
H_1:	&\ Y = \alpha+\beta X + \epsilon.
\end{align*}

%-----------------------------
\subsection{ANOVA for regression}

%For a fixed realisation $(x_1,x_2,\ldots,x_n)$ of the marginal sample $(X_1,X_2,\ldots,X_n)$,
The \emph{predicted value} of $Y_i$ is the random variable
\[
\hat{Y}_i = \hat{\alpha} + \hat{\beta}X_i.
\]
The total deviation of $Y_i$ from the overall mean $\bar{Y}$ can be divided into two components,
\[
Y_i - \bar{Y} = (Y_i-\hat{Y}_i) + (\hat{Y}_i - \bar{Y}),
\]
from which it follows that
\[
\sum_{i=1}^n(Y_i - \bar{Y})^2 = \sum_{i=1}^n(Y_i-\hat{Y}_i)^2 + \sum_{i=1}^n(\hat{Y}_i - \bar{Y})^2.
\]
As with ANOVA, we define the following sums-of-squares.
\[
\begin{array}{lll}
SST	& = \displaystyle\sum_{i=1}^n (Y_i-\bar{Y})^2
\qquad & \text{The \emph{total} sum-of-squares.} \\
SSR	& = \displaystyle\sum_{i=1}^n (\hat{Y}_i-\bar{Y})^2
\qquad & \text{The \emph{regression} sum-of-squares.} \\
SSE	& = \displaystyle\sum_{i=1}^n (Y_i-\hat{Y}_i)^2
\qquad & \text{The \emph{error} sum-of-squares.}
\end{array}
\]

The total sum-of-squares $SST$ is determined by the marginal sample $(Y_1,Y_2,\ldots,Y_n)$. Substituting for $\hat{Y}_i = \hat{\alpha} + \hat{\beta}X_i$ we see that the regression sum-of-squares satisfies
\[
SSR = \frac{\big[\sum_{i=1}^n (X_i-\bar{X})(Y_i-\bar{Y})\big]^2}{\sum_{i=1}^n (X_i-\bar{X})^2}
\]
The error sum-of-squares is then obtained via $SSE = SST - SSR$.

\bigskip
Under $H_0:\beta=0$,
\[
\frac{1}{\sigma^2}\sum_{i=1}^n (\hat{Y}_i-\bar{Y})^2	\sim \chi^2_1
\text{\quad and\quad}
\frac{1}{\sigma^2}\sum_{i=1}^n (Y_i-\hat{Y}_i)^2		\sim \chi^2_{n-2}.
\]
Thus we have the test statistic
\[
F
	= \frac{SSR}{(n-2)^{-1}SSE}
	= \frac{\sum_{i=1}^n (\hat{Y}_i-\bar{Y})^2}{(n-2)^{-1}\sum_{i=1}^n (Y_i-\hat{Y}_i)^2}
	\sim F_{1,n-2} \quad\text{under $H_0:\beta=0$.}
\]
which provides another means of testing whether or not $Y$ depends linearly on $X$.

%-----------------------------
\subsection{The coefficient of determination}

Recall that the \emph{correlation coefficient} of any pair of random variables $X$ and $Y$ is
\[
\rho(X,Y)
	= \frac{\cov(X,Y)}{\sqrt{\var(X)\var(Y)}}
	= \frac{\expe\big[(X-\expe X)(Y-\expe Y)\big]}
			{\sqrt{\expe\big[(X-\expe X)^2\big]\expe\big[(Y-\expe Y)^2\big]}}
\]

For a bivariate random sample $(X_1,Y_1),(X_2,Y_2),\ldots,(X_n,Y_n)$ the \emph{sample correlation coefficient}, also known as the \emph{Pearson correlation} is defined by
\[
R = \frac{\sum_{i=1}^n(X_i-\bar{X})(Y_i - \bar{Y})}{\sqrt{\sum_{i=1}^n(X_i-\bar{X})^2\sum_{i=1}^n(Y_i-\bar{Y})^2}}
\]

For the simple linear regression model $Y=\alpha+\beta X + \epsilon$, the MLE of $\beta$ can be written as,
\begin{align*}
\hat{\beta}
	& = \frac{\sum_{i=1}^n (X_i-\bar{X})(Y_i-\bar{Y})}{\sum_{i=1}^n (X_i-\bar{X})^2}
	= R\sqrt{\frac{\sum_{i=1}^n(Y_i-\bar{Y})^2}{\sum_{i=1}^n(X_i-\bar{X})^2}}.
\end{align*}

%The square of the empirical correlation coefficient is called the \emph{coefficient of determination}, denoted by $R^2$:
The \emph{coefficient of determination} is the square of the sample correlation, and denoted by $R^2$:
\[
R^2
	= \frac{\big[\sum_{i=1}^n(X_i-\bar{X})(Y_i - \bar{Y})\big]^2}{\sum_{i=1}^n(X_i-\bar{X})^2\sum_{i=1}^n(Y_i-\bar{Y})^2}
	= \frac{SSR}{SST}.
%	= 1 - \frac{SSE}{SST}.
\]

where $SSR$ and $SST$ are the regression sum-of-squares and total sum-of-squares respectively.

\bigskip
Thus $R^2$ is the proportion of the total variation explained by the regression model: it quantifies how well the regression line fits the data points, and as such is an example of a \emph{goodness-of-fit} statistic (the value $R^2=1$ indicates that the regression line perfectly fits the data). The corresponding quantity for one-way ANOVA is the so-called \emph{eta-squared} effect size:
\[
\eta^2 = \frac{SSG}{SST}.
\]

\begin{example}
The table below shows the deaths due to bronchitis ($x$) and corresponding daily temperatures ($y$), averaged over a long period.
\begin{center}
\begin{tabular}{lcccccccccc}\hline
$x$ & 253 & 232 & 210 & 200 & 191 & 187 & 134 & 102 & 81 & 25 \\
$y$ & 35 & 37 & 39 & 41 & 43 & 45 & 47 & 49 & 51 & 53 \\ \hline
\end{tabular}
\end{center}
\ben
\it Use the simple linear model $y = \alpha + \beta x + \epsilon$ to perform a least-squares regression of $y$ against $x$.
\it Test whether the slope of the regression line is significantly different from zero at the 5\% significance level.
\een
\begin{solution}
The various quantities of interest are computed here:
%\bit
%\it $n = 10$.
%\it $\sum x_{i} = 1615$.
%\it $\sum x_{i}^{2}	= 308929$.
%\it $\sum (x_i-\bar{x})^2	= 308929 - (1615)^{2}/10 = 48106.5$.
%\it $\sum y_{i} =  440$.
%\it $\sum y_{i}^{2}  = 19690$.
%\it $\sum (y_i-\bar{y})^2 = 19690 - (440)^{2}/10 = 330$.
%\it $\sum x_{i}y_{i}	= 67209$.
%\it $\sum (x_i-\bar{x})(y_i-\bar{y}) = 67209 - (1615){\times}(440)/10 = -3851$.
%\eit
\bit
\it $n = 10$.
\it $\bar{x} = 161.5$.
\it $\bar{y} = 44$.
\it $\sum (x_i-\bar{x})^2	= 48106.5$.
\it $\sum (y_i-\bar{y})^2 = 330$.
\it $\sum (x_i-\bar{x})(y_i-\bar{y}) = -3851$.
\eit

The MLEs of the regression coefficients are
\begin{align*}
\hat{\beta}		&\quad = \frac{\sum (x_i-\bar{x})(y_i-\bar{y})}{\sum (x_i-\bar{x})^2} = \frac{-3851}{48106.5} = -0.080052 \\
\hat{\alpha}	&\quad = \bar{y}-\hat{\beta}\bar{x} = 44 + 0.080052{\times}161.5 = 56.928326
\end{align*}

The least squares regression line is
\[
y = 56.928326 - 0.080052x.
\]
To test the null hypothesis $H_0:\beta=0$, we compute the regression sum-of-squares:
\[
SSR = \frac{\big[\sum (x_i-\bar{x})(y_i-\bar{y})\big]^{2}}{\sum (x_i-\bar{x})^2}
	=\frac{(-3851)^{2}}{48106.5}
	= 308.2785
\]
and the error sum-of-squares:
\[
SSE	= \sum (y_i-\bar{y})^2 - SSR = 330.0 - 308.2785 =  21.7215.
\]
The test statistic is
\[
F = \frac{SSR}{(n-2)^{-1}SSE} = \frac{308.2785}{21.7215/8} = 113.54.
\]

Critical values of the $F_{1,8}$-distribution are
\bit
\it $5.318$ at sig. level $0.05$
\it $7.570$ at sig. level $0.025$
\it $11.25$ at sig. level $0.001$
\it $14.68$ at sig. level $0.005$
\eit
Thus the null hypothesis $H_0:\beta = 0$ is strongly rejected at the $5\%$ significance level.

\bigskip
Here we have $R^2 = SSR/SST = 308.2785/330 = 0.9342$, which shows that $Y$ depends on $X$ to a very large extent.
\end{solution}
\end{example}


\chapter{Localization}
Before we proceed on to defining an affine scheme,
we will take the time to properly cover one more algebraic construction
that of a \emph{localization}.
This is mandatory because when we define a scheme,
we will find that all the sections and stalks
are actually obtained using this construction.

One silly slogan might be:
\begin{moral}
	Localization is the art of adding denominators.
\end{moral}
You may remember that when we were working with affine varieties,
there were constantly expressions of the form
$\left\{ \frac{f}{g} \mid g(p) \ne 0 \right\}$
and the like.
The point is that we introduced a lot of denominators.
Localization will give us a concise way of doing this in general.

\emph{Notational note}:
moving forward we'll prefer to denote rings by $A$, $B$, \dots,
rather than $R$, $S$, \dots.

\section{Spoilers}
Here is a preview of things to come,
so that you know what you are expecting.
Some things here won't make sense,
but that's okay, it is just foreshadowing.

Let $V \subseteq \Aff^n$, and for brevity let $R = \CC[V]$ be its coordinate ring.
We saw in previous sections how to compute $\OO_V(D(g))$
and $\OO_{V, p}$ for $p \in V$ a point.
For example, if we take $\Aff^1$ and consider a point $p$, then
$\OO_{\Aff^1}(D(x-p)) = \left\{ \frac{f(x)}{(x-p)^n} \right\}$
and $\OO_{\Aff^1, p} = \left\{ \frac{f(x)}{g(x)} \mid g(p) \ne 0 \right\}$.
More generally, we had
\begin{align*}
	\OO_{V}(D(g)) &= \left\{ \frac{f}{g^n} \mid f \in R \right\}
		\quad\text{by \Cref{thm:reg_func_distinguish_open}} \\
	\OO_{V,p} &= \left\{ \frac{f}{g} \mid f,g \in R, g(p) \ne 0 \right\}
		\quad\text{by \Cref{thm:stalks_affine_var}}.
\end{align*}

We will soon define something called a localization,
which will give us a nice way of expressing the above:
if $R = \CC[V]$ is the coordinate ring, then
the above will become abbreviated to just
\begin{align*}
	\OO_{V}(D(g)) &= R_{g} \\
	\OO_{V, p} &= R_{\km} \quad \text{where } \{p\} = \VV(\km).
\end{align*}
The former will be pronounced
``$R$ localized away from $g$''
while the latter will be pronounced
``$R$ localized at $\km$''.

Even more generally,
next chapter we will throw out the coordinate ring $R$
altogether and replace it with a general commutative ring $A$
(which are still viewed as functions).
We will construct a ringed space called $X = \Spec A$,
whose elements are \emph{prime ideals} of $A$
and is equipped with the Zariski topology and a sheaf $\OO_X$.
It will turn out that, in analogy to what we had before,
\begin{align*}
	\OO_X(D(f)) &= A[f\inv] \\
	\OO_{X,\kp} &= A_\kp
\end{align*}
for any element $f \in A$ and prime ideal $\kp \in \Spec A$.
Thus just as with complex affine varieties,
localizations will give us a way to more or less
describe the sheaf $\OO_X$ completely.

\section{The definition}
\begin{definition}
	A subset $S \subseteq A$ is a \vocab{multiplicative set}
	if $1 \in S$ and $S$ is closed under multiplication.
\end{definition}
\begin{definition}
	Let $A$ be a ring and $S \subset A$ a multiplicative set.
	Then the \vocab{localization of $A$ at $S$}, denoted $S\inv A$,
	is defined as the set of fractions
	\[ \left\{ a/s \mid a \in A, s \in S \right\} \]
	where we declare two fractions $a_1 / s_1 = a_2 / s_2$
	to be equal if $s(a_1s_2 - a_2s_1) = 0$ for some $s \in S$.
	Addition and multiplication in this ring
	are defined in the obvious way.
\end{definition}
In particular, if $0 \in S$ then $S\inv A$ is the zero ring.
So we usually only take situations where $0 \notin S$.

We give in brief now two examples which will be
motivating forces for the construction of the affine scheme.
\begin{example}
	[Localizations of {$\CC[x]$}]
	Let $A = \CC[x]$.
	\begin{enumerate}[(a)]
		\ii Suppose we let $S = \left\{ 1, x, x^2, x^3, \dots \right\}$
		be the powers of $x$.
		Then
		\[ S\inv A = \left\{ \frac{f(x)}{x^n}
			\mid f \in \CC[x], n \in \ZZ_{\ge 0} \right\}.  \]
		In other words, we get the Laurent polynomials in $x$.

		You might recognize this as
		\[ \OO_V(U) \text{ where } V = \Aff^1, \; U = V \setminus \{0\}. \]
		i.e.\ the sections of the punctured line.
		In line with the ``hyperbola effect'',
		this is also expressible as $\CC[x,y] / (xy-1)$.

		\ii Let $p \in \CC$.
		Suppose we let $S = \left\{ g(x) \mid g(p) \ne 0 \right\}$,
		i.e.\ we allow any denominators where $g(p) \ne 0$.
		Then
		\[ S\inv A = \left\{ \frac{f(x)}{g(x)}
			\mid f,g \in \CC[x], g(p) \ne 0 \right\}.  \]
		You might recognize this is as the stalk $\OO_{\Aff^1, p}$.
		This will be important later on.
	\end{enumerate}
\end{example}

\begin{remark}
	[Why the extra $s$?]
	We cannot use the simpler $a_1s_2 - a_2s_1 = 0$ since
	otherwise the equivalence relation may fail to be transitive.
	Here is a counterexample: take
	\[ A = \Zc{12} \qquad S = \{ 2, 4, 8 \}. \]
	Then we have for example $\frac12 = \frac24 = \frac64 = \frac32$.
	So we need to have $\frac12=\frac32$ which is only true
	with the first definition.
	Of course, if $A$ is an integral domain (and $0\notin S$)
	then this is a moot point.
\end{remark}

\begin{example}
	[Field of fractions]
	Let $A$ be an integral domain and $S = A \setminus \{0\}$.
	Then $S\inv A = \Frac(A)$.
\end{example}

\section{Localization away from an element}
\prototype{$\ZZ$ localized away from $6$ has fractions $\frac{m}{2^x 3^y}$.}
We now focus on the two special cases of localization we will need the most;
one in this section, the other in the next section.
\begin{definition}
	For $f \in A$, we define the \vocab{localization of $A$ away from $f$},
	denoted $A[1/f]$ or $A[f\inv]$,
	to be $\{1, f, f^2, f^3, \dots\}\inv A$.
	(Note that $\left\{ 1, f, f^2, \dots \right\}$ is multiplicative.)
\end{definition}
\begin{remark}
	In the literature it is more common to
	see the notation $A_f$ instead of $A[1/f]$.
	This is confusing, because in the next section
	we define $A_\kp$ which is almost the opposite.
	So I prefer this more suggestive (but longer) notation.
\end{remark}

\begin{example}
	[Some arithmetic examples of localizations]
	\listhack
	\begin{enumerate}[(a)]
		\ii We localize $\ZZ$ away from $6$:
		\[ \ZZ[1/6] = \left\{ \frac{m}{6^n} \mid m \in \ZZ,
			n \in \ZZ_{\ge 0} \right\}.  \]
		So $A[1/6]$ consist of those rational numbers whose
		denominators have only powers of $2$ and $3$.
		For example, it contains $\frac{5}{12} = \frac{15}{36}$.

		\ii Here is a more confusing example:
		if we localize $\Zc{60}$ away from the element $5$,
		we get $(\Zc{60})[1/5] \cong \Zc{12}$.
		You should try to think about why this is the case.
		We will see a ``geometric'' reason later.
	\end{enumerate}
\end{example}

\begin{example}
	[Localization at an element, algebraic geometry flavored]
	We saw that if $A$ is the coordinate ring of a variety,
	then $A[1/g]$ is interpreted geometrically as $\OO_V(D(g))$.
	Here are some special cases:
	\begin{enumerate}[(a)]
		\ii As we saw, if $A = \CC[x]$,
		then $A[1/x] = \left\{ \frac{f(x)}{x^n} \right\}$
		consists of Laurent polynomials.

		\ii Let $A = \CC[x,y,z]$.
		Then \[ A[1/x] = \left\{ \frac{f(x,y,z)}{x^n} \mid
			f \in \CC[x,y,z], \; n \ge 0 \right\} \]
		is rational functions whose denominators are powers of $x$.

		\ii Let $A = \CC[x,y]$.
		If we localize away from $y-x^2$ we get
		\[ A[(y-x^2)\inv] = \left\{ \frac{f(x,y)}{(y-x^2)^n} \mid
			f \in \CC[x,y], \; n \ge 0 \right\} \]
		By now you should recognize this as $\OO_{\Aff^2}(D(y-x^2))$.
	\end{enumerate}
\end{example}

\begin{example}
	[An example with zero-divisors]
	Let $A = \CC[x,y] / (xy)$
	(which intuitively is the coordinate ring of two axes).
	Suppose we localize at $x$:
	equivalently, allowing denominators of $x$.
	Since $xy = 0$ in $A$, we now have $0 = x\inv (xy) = y$,
	so $y = 0$ in $A$, and thus $y$ just goes away completely.
	From this we get a ring isomorphism
	\[ A[1/x] \cong \CC[x,1/x].\]
\end{example}

\section{Localization at a prime ideal}
\prototype{$\ZZ$ localized at $(5)$ has fractions $\frac{m}{n}$ with $5 \nmid n$.}
\label{sec:localize_prime_ideal}

\begin{definition}
	If $A$ is a ring and $\kp$ is a prime ideal, then we define
	\[ A_\kp \defeq \left( A \setminus \kp \right)^{-1} A. \]
	This is called the \vocab{localization at $\kp$}.
\end{definition}
\begin{ques}
	Why is $S = A \setminus \kp$ multiplicative
	in the above definition?
\end{ques}

%Warning: this notation sort of conflicts with the previous one.
%The ring $A_\kp$ is the localization with multiplicative set $A \setminus \kp$,
%while $A_f$ is the localization with multiplicative set $\{1,f,f^2,\dots\}$;
%these two are quite different beasts!
%In $A_\kp$ the subscript denotes the \emph{forbidden} denominators;
%in $A_f$ the subscript denotes the \emph{allowed} denominators.

This special case is important because we will see that
stalks of schemes will all be of this shape.
In fact, the same was true for affine varieties too.
\begin{example}
	[Relation to affine varieties]
	Let $V \subseteq \Aff^n$, let $A = \CC[V]$
	and let $p = (a_1, \dots, a_n)$ be a point.
	Consider the maximal (hence prime) ideal
	\[ \km = (x_1 - a_1, \dots, x_n - a_n). \]
	Observe that a function $f \in A$ vanishes at $p$
	if and only if $f \pmod{\km} = 0$, equivalently $f \in \km$.
	Thus, by \Cref{thm:stalks_affine_var} we can write
	\begin{align*}
		\OO_{V,p} &= \left\{ \frac{f}{g} \mid f,g \in A, g(p) \ne 0 \right\} \\
		&= \left\{ \frac{f}{g} \mid f \in A, g \in A \setminus \km \right\} \\
		&= \left( A \setminus \km \right)\inv A = A_\km.
	\end{align*}
	So, we can also express $\OO_{V,p}$ concisely as a localization.
\end{example}
Consequently, we give several examples in this vein.

\begin{example}
	[Geometric examples of localizing at a prime]
	\listhack
	\begin{enumerate}[(a)]
		\ii We let $\km$ be the maximal ideal $(x)$ of $A = \CC[x]$.
		Then \[ A_\km = \left\{ \frac{f(x)}{g(x)} \mid g(0) \ne 0 \right\} \]
		consists of the Laurent polynomials.

		\ii We let $\km$ be the maximal ideal $(x,y)$ of $A = \CC[x,y]$.
		Then \[ A_\km = \left\{ \frac{f(x,y)}{g(x,y)} \mid g(0,0) \ne 0 \right\}. \]

		\ii Let $\kp$ be the prime ideal $(y-x^2)$ of $A = \CC[x,y]$.
		Then
		\[ A_\kp = \left\{ \frac{f(x,y)}{g(x,y)} \mid g \notin (y-x^2) \right\}. \]
		This is a bit different from what we've seen before:
		the polynomials in the denominator are allowed to vanish
		at a point like $(1,1)$, as long as they don't vanish on
		\emph{every} point on the parabola.
		This doesn't correspond to any stalk we're familiar with right now,
		but it will later
		(it will be the ``stalk at the generic point of the parabola'').

		\ii Let $A = \CC[x]$ and localize at the prime ideal $(0)$.
		This gives \[ A_{(0)} = \left\{ \frac{f(x)}{g(x)} \mid g(x) \ne 0 \right\}. \]
		This is all rational functions, period.
	\end{enumerate}
\end{example}

\begin{example}
	[Arithmetic examples]
	We localize $\ZZ$ at a few different primes.
	\begin{enumerate}[(a)]
		\ii If we localize $\ZZ$ at $(0)$:
		\[ \ZZ_{(0)} = \left\{ \frac mn \mid n \ne 0 \right\}
			\cong \QQ. \]
		\ii If we localize $\ZZ$ at $(3)$, we get
		\[ \ZZ_{(3)} = \left\{ \frac mn \mid \gcd(n,3) = 1 \right\} \]
		which is the ring of rational numbers
		whose denominators are relatively prime to $3$.
	\end{enumerate}
\end{example}

\begin{example}
	[Field of fractions]
	If $A$ is an integral domain,
	the localization $A_{(0)}$
	is the field of fractions of $A$.
\end{example}

\section{Prime ideals of localizations}
\prototype{The examples with $A = \ZZ$.}
We take the time now to mention how you can
think about prime ideals of localized rings.
\begin{proposition}
	[The prime ideals of $S\inv A$]
	Let $A$ be a ring and $S \subseteq A$ a multiplicative set.
	Then there is a natural inclusion-preserving bijection between:
	\begin{itemize}
		\ii The set of prime ideals of $S\inv A$, and
		\ii The set of prime ideals of $A$ not intersecting $S$.
	\end{itemize}
\end{proposition}
\begin{proof}
	Consider the homomorphism $\iota \colon A \to S\inv A$.
	For any prime ideal $\kq \subseteq S\inv A$,
	its pre-image $\psi\pre(\kq)$ is a prime ideal of $A$
	(by \Cref{prob:prime_preimage}).
	Conversely, for any prime ideal $\kp \subseteq A$
	not meeting $S$,
	$S\inv \kp = \left\{ \frac{a}{s} \mid a \in \kp, s \in S \right\}$
	is a prime ideal of $S\inv A$.
	An annoying check shows that this produces the required bijection.
\end{proof}
In practice, we will almost always use the corollary
where $S$ is one of the two special cases we discussed at length:
\begin{corollary}
	[Spectrums of localizations]
	Let $A$ be a ring.
	\begin{enumerate}[(a)]
		\ii If $\kp$ is a prime ideal of $A$,
		then the prime ideals of $A[1/f]$ are naturally
		in bijection with prime ideals of $A$
		\textbf{do not contain the element} $f$.

		\ii If $\kp$ is a prime ideal of $A$,
		then the prime ideals of $A_\kp$ are naturally
		in bijection with prime ideals of $A$
		which are \textbf{subsets of} $\kp$.
	\end{enumerate}
\end{corollary}
\begin{proof}
	Part (b) is immediate; a prime ideal doesn't meet $A \setminus \kp$
	exactly if it is contained in $\kp$.
	For part (a), we want prime ideals of $A$ not containing
	any \emph{power} of $f$.
	But if the ideal is prime and contains $f^n$,
	then it should contain either $f$ or $f^{n-1}$,
	and so at least for prime ideals these are equivalent.
\end{proof}
Notice again how the notation is a bit of a nuisance.
Anyways, here are some examples, to help cement the picture.
\begin{example}
	[Prime ideals of {$\ZZ[1/6]$}]
	Suppose we localize $\ZZ$ away from the element $6$,
	i.e.\ consider $\ZZ[1/6]$.
	As we saw,
	\[ \ZZ[1/6] = \left\{ \frac{n}{2^x 3^y} \mid n \in \ZZ,
		x,y \in \ZZ_{\ge 0} \right\}.  \]
	consist of those rational numbers whose
	denominators have only powers of $2$ and $3$.
	Note that $(5) \subset \ZZ[1/6]$ is a prime ideal:
	those elements of $\ZZ[1/6]$ with $5$ dividing the numerator.
	Similarly, $(7)$, $(11)$, $(13)$, \dots\
	and even $(0)$ give prime ideals of $\ZZ[1/6]$.

	But $(2)$ and $(3)$ no longer correspond to
	prime ideals; in fact in $A_6$ we have $(2) = (3) = (1)$,
	the whole ring.
\end{example}

\begin{example}
	[Prime ideals of $A_{(5)}$]
	Suppose we localize $\ZZ$ at the prime $(5)$.
	As we saw,
	\[ \ZZ_{(5)} = \left\{ \frac{m}{n} \mid m,n \in \ZZ,
			5 \nmid n \right\}.  \]
	consist of those rational numbers whose
	denominators are not divisible by $5$.
	This is an integral domain, so $(0)$ is still a prime ideal.
	There is one other prime ideal: $(5)$,
	i.e.\ those elements whose numerators are divisible by $5$.

	There are no other prime ideals:
	if $p \ne 5$ is a rational prime,
	then $(p) = (1)$, the whole ring, again.
\end{example}

\section{Prime ideals of quotients}
While we are here, we mention that
the prime ideals of quotients $A/I$
can be interpreted in terms of those of $A$
(as in the previous section for localization).
You may remember this from \Cref{prob:inclusion_preserving}
a long time ago, if you did that problem;
but for our purposes we actually only care about the prime ideals.
\begin{proposition}
	[The prime ideals of $A/I$]
	\label{prop:prime_quotient}
	If $A$ is a ring and $I$ is any ideal (not necessarily prime)
	then the prime (resp.\ maximal) ideals of $A/I$
	are in bijection with prime (resp.\ maximal) ideals of $A$
	which are \textbf{supersets of} $I$.
	This bijection is inclusion-preserving.
\end{proposition}
\begin{proof}
	Consider the quotient homomorphism $\psi \colon A \surjto A/I$.
	For any prime ideal $\kq \subseteq A/I$,
	its pre-image $\iota\pre(\kq)$ is a prime ideal
	(by \Cref{prob:prime_preimage}).
	Conversely, for any prime ideal $\kp$
	with $I \subseteq \kp \subseteq A$,
	we get a prime ideal of $A/I$ by looking at $\kp \pmod I$.
	An annoying check shows that this produces the required bijection.
	It is also inclusion-preserving --- from which
	the same statement holds for maximal ideals.
\end{proof}
\begin{example}
	[Prime ideals of $\Zc{60}$]
	The ring $\Zc{60}$ has three prime ideals:
	\begin{align*}
		(2) &= \left\{ 0, 2, 4, \dots, 58 \right\} \\
		(3) &= \left\{ 0, 3, 6, \dots, 57 \right\} \\
		(5) &= \left\{ 0, 5, 10, \dots, 55 \right\}.
	\end{align*}
	Back in $\ZZ$, these correspond to the three prime ideals
	which are supersets of
	$60\ZZ = \left\{ \dots, -60, 0, 60, 120, \dots \right\}$.
\end{example}

\section{Localization commute with quotients}
\prototype{$(\CC[xy]/(xy))[1/x] \cong \CC[x,x\inv]$.}
While we are here, we mention a useful result from
commutative algebra which lets us compute localizations in quotient rings,
which are surprisingly unintuitive.
You will \emph{not} have a reason to care about this
until we reach \Cref{sec:localize_prime_ideal},
and so this is only placed earlier to emphasize that it's
a purely algebraic fact that we can (and do) state this early,
even though we will not need it anytime soon.

Let's say we have a quotient ring like
\[ A/I = \CC[x,y] / (xy) \]
and want to compute the localization of this ring
away from the element $x$.
(To be pedantic, we are actually localizing away from $x \pmod{xy}$,
the element of the quotient ring, but we will just call it $x$.)
You will quickly find that even the notation becomes clumsy: it is
\begin{equation}
	\left( \CC[x,y] / (xy) \right)[1/x]
	\label{eq:quotient_localization_before}
\end{equation}
which is hard to think about,
because the elements in play are part of the \emph{quotient}:
how are we supposed to think about
\[ \frac{1 \pmod{xy}}{x \pmod{xy}} \]
for example?
The zero-divisors in play may already make you feel uneasy.

However, it turns out that we can actually do the localization
\emph{first}, meaning the answer is just
\begin{equation}
	\CC[x,y,1/x] / (xy)
	\label{eq:quotient_localization_after}
\end{equation}
which then becomes $\CC[x, x\inv, y] / (y) \cong \CC[x,x\inv]$.

This might look like it should be trivial,
but it's not as obvious as you might expect.
There is a sleight of hand present here with the notation:
\begin{itemize}
	\ii In \eqref{eq:quotient_localization_before},
	the notation $(xy)$ stands for an ideal of $\CC[x,y]$
	--- that is, the set $xy \CC[x,y]$.

	\ii In \eqref{eq:quotient_localization_after}
	the notation $(xy)$ now stands for an ideal of $\CC[x,x\inv,y]$
	--- that is, the set $xy \CC[x,x\inv,y]$.
\end{itemize}
So even writing down the \emph{statement} of the theorem
is actually going to look terrible.

In general, what we want to say is that if we have our ring $A$
with ideal $I$ and $S$ is some multiplicative subset of $A$,
then \[ \text{Colloquially: ``}S\inv (A/I) = (S\inv A)/I\text{''}. \]
But there are two things wrong with this:
\begin{itemize}
\ii The main one is that $I$ is not an ideal of $S\inv A$, as we saw above.
This is remedied by instead using $S\inv I$,
which consists of those elements of those elements $\frac xs$
for $x \in I$ and $s \in S$.
As we saw this distinction is usually masked in practice,
because we will usually write $I = (a_1, \dots, a_n) \subseteq A$
in which case the new ideal $S\inv I \subseteq A$ can be denoted
in exactly the same way: $(a_1, \dots, a_n)$,
just regarded as a subset of $S\inv A$ now.

\ii The second is that $S$ is not, strictly speaking,
a subset of $A/I$, either.
But this is easily remedied by instead using the image of $S$
under the quotient map $A \surjto A/I$.

We actually already saw this in the previous example:
when trying to localize $\CC[x,y]/(xy)$,
we were really localizing at the element $x \pmod{xy}$,
but (as always) we just denoted it by $x$ anyways.
\end{itemize}

And so after all those words, words, words, we have the hideous:
\begin{theorem}
	[Localization commutes with quotients]
	\label{thm:localization_commute_quotient}
	Let $S$ be a multiplicative set of a ring $A$,
	and $I$ an ideal of $A$.
	Let $\ol S$ be the image of $S$
	under the projection map $A \surjto A/I$.
	Then
	\[ {\ol S}\inv (A/I) \cong S\inv A / S\inv I \]
	where $S\inv I = \left\{ \frac{x}{s} \mid x \in I, s \in S \right\}$.
\end{theorem}
\begin{proof}
	Do you actually care? No? Cool, I didn't either.
	(Atiyah-Macdonald is the right reference for these
	type of things in the event that you do care.)
\end{proof}
The notation is a hot mess.
But when we do calculations in practice, we instead write
\[ \left( \CC[x,y,z]/(x^2 + y^2 - z^2) \right)[1/x]
	\cong \CC[x,y,z,1/x] / (x^2 + y^2 - z^2) \]
or (for an example where we localize at a prime ideal)
\[ \left( \ZZ[x,y,z]/ (x^2 + yz) \right)_{(x,y)}
	\cong \ZZ[x,y,z]_{(x,y)} / (x^2 + yz) \]
and so on --- the pragmatism of our ``real-life'' notation
which hides some details actually guides our intuition
(rather than misleading us).
So maybe the moral of this section is that whenever
you compute the localization of the quotient ring,
if you just suspend belief for a bit,
then you will probably get the right answer.

We will later see geometric interpretations of these facts
when we work with $\Spec A/I$,
at which point they will become more natural.

\section{\problemhead}
\begin{problem}
	Let $A = \Zc{2016}$, and consider the element $60 \in A$.
	Compute $A[1/60]$, the localization of $A$ away from $60$.
	\begin{sol}
		One should get $A[1/60] = \Zc{7}$.
	\end{sol}
\end{problem}

\begin{problem}
	[Injectivity of localizations]
	Let $A$ be a ring and $S \subseteq A$ a multiplicative set.
	Find necessary and sufficient conditions
	for the map $A \to S\inv A$ to be injective.
	\begin{hint}
		Consider zero divisors.
	\end{hint}
	\begin{sol}
		If and only if $S$ has no zero divisors.
	\end{sol}
\end{problem}

\begin{sproblem}
	[Alluding to local rings]
	Let $A$ be a ring, and $\kp$ a prime ideal.
	How many maximal ideals does $A_\kp$ have?
	\begin{hint}
		Only one!
		A proof will be given a few chapters later.
	\end{hint}
\end{sproblem}

\begin{problem}
	Let $A$ be a ring such that $A_\kp$ is an integral domain
	for every prime ideal $\kp$ of $A$.
	Must $A$ be an integral domain?
	\begin{hint}
		No. Imagine two axes.
	\end{hint}
	\begin{sol}
		Take $A = \CC[x,y] / (xy)$.
	\end{sol}
\end{problem}


{\bfseries\slshape\sffamily\color{ChapterTitleColor} \chapter{Array Support}} \label{chap:arrys}

\section{Introduction}

In most programming languages, the word array usually refers to a data structure that stores identically sized elements in a sequential and continuous block of memory. Lists on the other hand are used to stored non-identically sized elements that might be located anywhere in memory. Arrays are therefore referred to as homogenous collections whereas lists are heterogeneous collections. The advantage of arrays is that they can be used to process information very efficiently. Arrays are often used to store numerical data which require efficient numerical processing. The simplest array is a one dimensional collection of numbers, such as:

\begin{lstlisting}
[1.3, 5.6, 7.8, 4.5, 2.3, 1.2]
\end{lstlisting}

This array has six elements. Since the data in an array is guaranteed to be laid out sequentially in memory, accessing individual elements can be efficiently done by using indexing to the appropriate element. Object Pascal has a variety of ways for declaring arrays. For example an array with a fixed number of elements (a static array) can be declared using:

\begin{lstlisting}
TMyArray = array[1..100] of double;
\end{lstlisting}

It is also possible to declare so-called dynamic arrays which are allocated a size at runtime, and makes them more flexible:

\begin{lstlisting}
var
   myArray : array of double;

setLength (myArray, 100);
\end{lstlisting}

Arrays need not be only one-dimensional but in principle can be any dimension. A common array structure used in mathematics is the two-dimensional matrix:

$$
\setlength{\delimitershortfall}{0pt}
\begin{bmatrix}
3.4 & 6.7 &  8.9\\
1.2 & 4.5 & 3.1 \\
5.6 & 7.7 & 2.5
\end{bmatrix}
$$

Such matrices have very widespread applications in science, engineering, statistics and machine learning. Matrices can easily represented using 2D arrays.

\subsection*{Rhodus Array Syntax}

We've already encountered the basic syntax used with arrays. Rhodus follows to some degree the model used by Python. In particular we repurpose the list syntax to define literal arrays and use a global method, {\tt array} to convert lists into the array data model. For example:

\begin{lstlisting}
>> a = array([[1,2],[3,4]])
>> println (a)
[ 1, 4;
  9, 16]
>> type (a)
array
\end{lstlisting}

The {\tt array} method also can be used to specify the size of a new array, for example:

\begin{lstlisting}
>> a = array(3,4)
>> b = array(5,5,5)
\end{lstlisting}

The first array, {\tt a} is a 3 by 4 array while the second array, {\tt b}, is a three-dimensional array of size 5 in each dimension. By default all elements in a new array are initialized to zero.
For now arrays will only be able to contain floating point values.

Like lists, arrays can be in indexed using the usual indexing syntax, for example:

\begin{lstlisting}
>> a = array(5,5,5)
>> a[1,1,2]
0.0
>> a[1,1,2] = 3.14
>> a[1,1,2]
3.14
\end{lstlisting}

Like lists, indexing starts at zero.

Rhodus has two variants of arrays. The first is an n-dimensional array and the second, derived from arrays, is the 2-dimensional matrix. The two types only differ is what operations can be applied to them. These operations are governed by two built-in libraries, {\tt arrays} and {\tt mat}. Arithmetic operations on arrays are element-wise whereas the operations provided by the matrix library correspond to the classic matrix operations found in linear algebra. For example, multiplying two arrays together is done by multiplying each corresponding element to form a new array of products, for example:

\begin{lstlisting}
>> import arrays
>> a = array([[1,2],[3,4]])
>> println (arrays.mult (a, a))
[ 1, 4;
  9, 16]
\end{lstlisting}

In contrast, multiplication using the {\tt mat} library yields a different result:

\begin{lstlisting}
>> import mat
>> a = array([[1,2],[3,4]])
>> println (mat.mult (a, a))
[   7, 10;
   15  22]
\end{lstlisting}

We will cover more of this topic shortly.

\section{Implementing Arrays}

How do we implement array support? In considering this question, the main point to keep in mind is that access to arrays are meant to be fast and operations on arrays should be as efficient as possible. This requirement will dictate how an array, and in particular a multidimensional arrays is stored. To keep thing simple, let's first consider arrays up to two dimensions. Since we don't know how big arrays will be we need to use dynamic arrays at the Object Pascal level. Secondly, our arrays will be objects that can be garbage collected, this means an array must be derived from the same parent as strings and lists. We begin with a simple array object class:

\begin{lstlisting}
TArrayObject = class (TRhodusObject)
end;
\end{lstlisting}

The parent {\tt TRhodusObject} class includes a number of fields:

\begin{lstlisting}
   blockType : TBlockType;
   objectType : TSymbolElementType;
   methodList : TMethodList;
\end{lstlisting}

The {\tt blockType} is for the garbage collector to know what to do with the object during garbage collection. The {\tt objectType} just tells us what kind of object it is, string, list, user function or array. Finally, the {\tt methodlist} is a new field for version 3 of Rhodus and points to a list of methods that can be applied to the object. This is what lets us do things like:

\begin{lstlisting}
a = "String"
l = a.len()

println ("How long am I".len())
\end{lstlisting}

The first thing to add to the {\tt TArrayObject} is the field that will hold the array data. For now, we will only support arrays that hold floating point numbers.

\begin{lstlisting}
T1DArray = array of double;
TArrayObject = class (TRhodusObject)
   data : T1DArray;
end;
\end{lstlisting}

You may be thinking, ok this will store a 1D array, but what about a 2D array? The best way to handle n-dimensional arrays is to map then into a 1-dimensional array. If we didn't do this we'd have to have special cases for all the possible dimensions, e.g {\tt T1DArray, T2DArray, T3DArray}, etc. I don't think we want to do that.

Figure~\ref{fig:rowMajor} shows how we can map a one-dimensional array to an dimensions. All we do is slice the 1D-array into segments represents the rows, in this case a 2D-array. We can slice at many times as we like to model n-dimensional arrays.

\begin{figure}[htpb]
\centering
\includegraphics[scale=0.55]{rowMajor.pdf}
\caption{Using a 1D-array to represent a 2D-array.}
\label{fig:rowMajor}
\end{figure}

With a bit of simple arithmetic we can convert any 2D-index, such as $(i, j)$, into a single index along the 1D-array. To do this we must know the intended column width of the 2D-array. In the example the column width is 4. By convention, in a 2D index such as $(i, j)$, the $i$ represents the row number and $j$ the column number. Let's say we had a coordinate of $(2,3)$, that represents the second row and third column. One thing we have to be careful of is what is the index of the first element in the array? As with lists, we will always index from zero. this means that the coordinate $(2,3)$ actually means the 3rd row and 4th column. Given a coordinate $(i,j)$, the index along the 1D-array that corresponds to this coordinate will be:
%
$$ \mbox{index} = j + (i \times \mbox{width}) $$
%
If we plug $(2,3)$ into this formula we get: $3 + 2 \cdot 4 = 11$. Hence we would pull the value out of index 11 in the 1D-array. If you look at Figure~\ref{fig:rowMajor} you can confirm this by eye. We can rewrite the above formula is a more general way:
%
$$ \mbox{index} = i \cdot d_j + j $$
%
where $d_j$ is the width or dimension (number of elements) in the $j$th direction. This approach can be extended to any number of dimensions. For example, for a 3D array, with coordinate, $i, j, k$, the formula for computing the index in a linear array can be extended to give:
%
$$ \mbox{index} = i \cdot d_j \cdot d_k + j \cdot d_k + k $$
%
where $d_j$ is the dimension of the $j$th coordinate, and $d_k$ the dimension of the $k$th coordinate. By induction the formula for a 4D array with correlates $i, j, k, l$, would be:
%
$$ \mbox{index} = i \cdot d_j \cdot d_k \cdot d_l + j \cdot d_k \cdot d_l + k \cdot d_l + l $$
%
This can be turned into a function for computing the index of any array with any dimension as follows\footnote{Modified from Heffernan's answer in \url{https://stackoverflow.com/questions/28569850/}}:
%

\begin{lstlisting}
// Given the dimension of the array in the array dimensions
// and the coordinates in the array index, this routine
// will return the index assuming the array is stored
// in a 1D block of memory.
function getIndex (const dimensions, index: array of Integer): integer;
var i: Integer;
begin
  result := idx[0];
  for i := 1 to high(dimensions) do
    result := result*dimensions[i] + index[i];
end;
\end{lstlisting}

For example, if we have a 6 dimensional array of dimensions $5,5,5,5,5,5$, which is 15,625 elements, the index at coordinate $1,2,3,3,0,4$ will be position 4829 in the 1D storage array.

We can create two helper methods for getting and setting a value in an n-dimensional array. The methods will take a dynamic array containing the indices to use.

\begin{lstlisting}
function TArrayObject.getValue (idx : array of integer) : double;
var index : integer;
begin
  for var i := 0 to length (dim) - 1 do
      if idx[i] >= dim[i] then
         raise ERuntimeException.Create(outOfRangeMsg);

  index := getIndex (dim, idx);
  result := data[index];
end;
\end{lstlisting}

For example in the {\tt getValue} method we can use the new open array syntax introduced in XE7.

\begin{lstlisting}
  value := getValue ([1,3,2])
\end{lstlisting}

Likewise we have a {\tt setValue} method that also takes an dynamic array argument for specifying the indices.

\begin{lstlisting}
procedure TArrayObject.setValue (idx : array of integer; value : double);
var index : integer;
    i : integer;
begin
  for i := 0 to length (dim) - 1 do
      if idx[i] >= dim[i] then
         raise ERuntimeException.Create(outOfRangeMsg);

  index := getIndex (dim, idx);
  data[index] := value;
end;
\end{lstlisting}

For example:

\begin{lstlisting}
  setValue ([6,2,7], value)
\end{lstlisting}

In both methods we need to check for index out of bounds errors. The bounds will be stored in a object variable called dim. We can expand the {\tt TArrayObject} to:

\begin{lstlisting}
  TArrayObject = class (TRhodusObject)
    private
    public
     data : T1DArray;
     dim  : TIndexArray;  // Of type array of integer;

     function     getValue (idx : array of integer) : double;
     procedure    setValue (idx : array of integer; value : double);

     function     clone : TArrayObject;

     constructor  Create; overload;
     constructor  Create (dim : TIndexArray); overload;
     destructor   Destroy; override;
\end{lstlisting}

We'll add a bunch of extra methods. One in particular is the ability to clone an array. This is implemented in the clone method:

\begin{lstlisting}
function TArrayObject.clone : TArrayObject;
begin
  result := TArrayObject.Create (dim);
  result.data := copy (self.data);
end;
\end{lstlisting}

There are also a bunch of other methods to make like easier, as well as some arithmetic functions such as {\tt add}, {\tt subtract}, and {\tt multiply}. These are all pair-wise operations.
That is addition of two arrays is archived by summing up pair-wise values in each array.  For this work, both arrays must have the same dimensions. For a 2 by 2 array addition is defined as:

$$
\begin{bmatrix}
  a_1 & a_2 \\
  a_3 & a_4
\end{bmatrix}
+
\begin{bmatrix}
  b_1 & b_2 \\
  b_3 & b_4
\end{bmatrix}
=
\begin{bmatrix}
  a_1+b_1 & a_2+b_2 \\
  a_3 + b_3 & a_4 + b_4
\end{bmatrix}
$$

This idea can be extended to n-dimensional arrays. Subtraction is defined in a similar way except we take the difference between each pair. When we come to multiplication we are confronted with two common multiplication approaches. This includes the simple pair-wise multiplication and, arguably, the more common matrix multiplication. For arrays we will provide pair-wise multiplication. This will be specified using the usual multiply operator {\tt `*'}.

For 2D-arrays, i.e matrices, we need a way to specify matrix multiplication. Python uses {\tt `@'}, so we might as well use that too. We'll have to update the lexical scanner, syntax analysis and AST constructions but that is simple to do. For example, let's compare pair-wise and matrix multiplication:

\begin{lstlisting}
>> a = array([[1,2],[3,4]])
>> println (a*a)
[[   1.0000,     4.0000]],
[    9.0000,    16.0000]]
>> println (a@a)
[[   7.0000,    10.0000]],
[   15.0000,    22.0000]]
\end{lstlisting}

Multiply arrays

[[0]*4]*3
[[0,0,0,0],[0,0,0,0],[0,0,0,0]]

or

\begin{lstlisting}
>>println (a.sqr())
\end{lstlisting}

This squares each element in the array {\tt a}. $n$-dimensional arrays can be created using the {\tt array} method:

There are at least two classes of operations we can apply to arrays, these include manipulating the contents of an array, for example extracting rows and columns, or adding rows and columns, and secondly carrying out arithmetic on arrays. Let's first focus on arithmetic operations.

\section{Matrices}

There isn't much to say about matrices other than they are strictly 2-dimensional arrays. There is nothing intrinsically different between an array and a matrix other than its dimensions. The key difference is that matrix operations are different from the equivalent array operations. This is particularly the case for multiplication which is dot-product based for matrices. This is handled by the {\tt mat} library which also offers other classical matrix operations such as computing the inverse and determinant.

All operations on arrays can also be applied to matrices, but not vice versa, since matrices are strictly 2-dimensional.

As mention in the last section, matrix multiplication comes in two forms, pair-wise and dot product. Pair-wise multiplication can be easily extended to arrays of any dimension. In Rhodus there are two ways to specify pair-wise multiplication, using the normal multiplication operator, {\tt *}:

\begin{lstlisting}
>> a = array([[1,2],[3,4]])
>> b = array([[5,6],[7,8]])
>> x = a*b
>>println (x)
[[   5.0000,    12.0000]],
[   21.0000,    32.0000]]
\end{lstlisting}

Matrix multiplication, as defined in linear algebra, is the generated by taking the dot-product of corresponding rows and columns.  Unlike array multiplication, where the two arrays must be the same size, in matrix multiplication this need not be the case. Moreover, matrix multiplication generally applies to 2-dimensional matrices. When applying matrix multiplication, the number of columns of the first matrix must equal the number of rows in the second matrix. The outer rows and columns can be of any size. In Rhodus there are two ways to specify matrix multiplication, either using the {\tt mult} method from the matrix library or using he special matrix multiplication operator symbol {\tt @}|. For example:

\begin{lstlisting}
>> a = array([[1,2],[3,4]])
>> b = array([[5,6],[7,8]])
>> x = a@b
>> x = mat.mult (a, b)
>>println (x)
[[  19.0000,    22.0000]],
[   43.0000,    50.0000]]
>>
\end{lstlisting}

Application

[m1, m2]

combine (m1, m2)

[m1; m2]

mat.appendRow ([m1, m2])
mat.appendCol ([m1, m2])

m1.appendRow (m2)
m1.appendCol (m2)

m1.hstack (m2)
m1.vstack (m2)

%
%\section{Useful Reading}
%
%\subsection{Introductory Books}
%
%{\bf 1.} Ball, Thorsten. Writing A Compiler In Go. Thorsten Ball, 2018.
%
%{\bf 2.} Kernighan, Brian W.; Pike, Rob (1984). The Unix Programming Environment. Prentice-Hall. ISBN 0-13-937681-X.
%
%{\bf 3.} Nisan, Noam, and Shimon Schocken. The elements of computing systems: building a modern computer from first principles. MIT press, 2005.
%
%{\bf 4.} Parr, Terence. Language implementation patterns: create your own domain-specific and general programming languages. Pragmatic Bookshelf, 2009.
%
%\subsection{More Advanced Books}
%
%{\bf 1.} Jim Smith, Ravi Nair, Virtual Machines: Versatile Platforms for Systems and Processes, Morgan Kauffmann, June 2005
%
%{\bf 2.} Aho, Alfred V., Ravi Sethi, and Jeffrey D. Ullman. Compilers: Principles, Techniques and Tools (also known as The Red Dragon Book), 1986.
%
%\subsection{Source Code}
%
%{\bf 1.} Mak, Ronald. Writing compilers and interpreters: an applied approach/by Ronald Mark. 1991
%
%Note, this is the first edition, 1991. The code is in C, which I found to be understandable. The later editions that use C++ are not as clear. The issue I found is that the object orientated approach that's used tends to obscure the design principles of the interpreter and requires much study to decipher, The C version is much more straightforward.
%
%{\bf 2.} Wren: \url{https://github.com/wren-lang/wren}.\index{wren}
%
%Of the open source interpreters on GitHub, I found this to be the easiest to read. It's written by Bob Nystrom in C, the same person who is writing the web book: Crafting Interpreters \url{https://craftinginterpreters.com/}.
%
%{\bf 3.} Gravity: \index{Gravity} Another open source interpreter worth looking at is Gravity (\url{https://github.com/marcobambini/gravity}). Gravity, like Wren, is also written in C.
%
%{\bf 4.}  If you prefer Go,\index{Go} then the source code to look at is the interpreter written by Thorsten Ball (see book reference above).
%
%There are umpteen BASIC interpreters\index{BASIC} and other languages that can be studied.
%
%\bigskip\medskip

\begin{center}
\pgfornament[width = 8cm, color = cardinal]{83}
\end{center}


\subsubsection*{Matrix and Vector Operations\hfill \hyperlink{linearAlgebraFunctions}{(up)}\hypertarget{matrixOperations}{}} \addcontentsline{toc}{subsubsection}{Matrix and Vector Operations}
\begin{table}[H]
\caption{Matrix and Vector Operations.}
\label{tab:matrixOperations}
\begin{center}
\begin{tabular}{|l|l|}\hline
\hlnkFunc{herm} & Matrix Hermitian\\
\hlnkFunc{jdot} & Complex Vector Conjugate Dot Product\\
\hlnkFunc{gemp} & General Matrix Product\\
\hlnkFunc{gems} & General Matrix Sum \\
\hlnkFunc{kron} & Kronecker Product \\
\hlnkFunc{prod3} & 3 by 3 Matrix Product\\
\hlnkFunc{prod4} & 4 by 4 Matrix Product\\
\hlnkFunc{prod} & Matrix product \\
\hlnkFunc{prodh} & Matrix Hermitian Product\\
\hlnkFunc{prodj} & Matrix Conjugate Product\\
\hlnkFunc{prodt} & Matrix Transpose Product\\
\hlnkFunc{trans} & Matrix Transpose\\
\hlnkFunc{dot} & Vector Dot Product\\
\hlnkFunc{outer} & Vector Outer Product\\
\hline\end{tabular}
\end{center}
%\label{default}
\end{table}
%