\section[The Real and Complex Number Systems]{\hyperlink{toc}{The Real and Complex Number Systems}}
\subsection{The Naturals, Integers, and Rationals}
We begin by a review of number systems which are already familiar.

\begin{ndef}{: The Natural Numbers}
    The \textbf{Naturals}, denoted by $\NN$, is the set $\set{1, 2, 3, \ldots}$.
\end{ndef}

\noindent For $x, y \in \NN$, we have that $x + y \in \NN$ and $xy \in \NN$, so the naturals are closed under addition and multiplication. However, we note that it is not closed under subtraction; take for example $2 - 4 = -2 \notin \NN$.

\begin{ndef}{: The Integers}
    The \textbf{Integers}, denoted by $\ZZ$, is the set $\set{\ldots, -3, -2, -1, 0, 1, 2, 3, \ldots}$.
\end{ndef}

\noindent The integers are closed under addition, multiplication, and subtraction. However, it is not closed under division; for example, $1/2 \notin \ZZ$.

\begin{ndef}{: The Rationals (informal)}
    The \textbf{Rationals}, denoted by $\QQ$, can be defined as $\set{\frac{m}{n}: m \in \ZZ, n \in \NN}$, where $\frac{m_1}{n_1}$ and $\frac{m_2}{n_2}$ are identified if $m_1n_2 = m_2n_1$.
\end{ndef}

\noindent We note that unlike the naturals/integers, the rationals do not have as obvious of a denumeration. This above is a good definition if we already have the same rigorous idea of what a rational number is in our mind; i.e. it works because we have a shared preconceived understanding of a rational number.

If this is not the case, it may help to define the rational numbers more rigorously/formally (even if the definition may be slightly harder to parse). As a second attempt at a definition, we can say that $\QQ$ is the set of ordered pairs $\set{(m, n): m \in \ZZ, n \in \NN}$. However, this is not quite enough as we need a notion of equivalence between two rational numbers (e.g. $(1, 2) = (2, 4)$). Hence, a complete and rigorous definition would be:

\begin{ndef}{: The Rationals (formal)}
    The \textbf{Rationals}, denoted by $\QQ$, is the set $\set{(m, n): m \in \ZZ, n \in \NN}/\sim$ where $(m_1, n_1) \sim (m_2, n_2)$ if $m_1n_2 = m_2n_1$.
\end{ndef}
\noindent Under the formal definition, the rationals are a set of equivalence classes of ordered pairs, under the equivalence relation $\sim$. We note that the rationals are closed under addition, subtraction, multiplication, and division.

This formal definition might be slightly harder to parse, so it might be useful to consider an example with a similar flavour. Consider the set $X = \set{m \in \ZZ}/\sim$ such that $m_1 \sim m_2$ if $m_1 - m_2$ is divisible by 12. This is "clock arithmetic", with equivalence classes $[0], [1], [2], \ldots$ for each hour on an analog clock. A fun side note: If instead of 12 we picked a prime number, we would get a field (we will discuss what this is in a later lecture)!

Note that under this definition, $(1, 2)$ and $(2, 4)$ are different representations of the same rational number. With this definition, we would define addition such that $(m_1, n_1) + (m_2, n_2) = (m_1n_2 + m_2n_1, n_1n_2)$. Note that $(2m_1, 2n_2) + (m_2, n_2) = (2m_1n_2 + 2m_2n_1, 2n_1n_2)$ and we can identify $(m_1n_2 + m_2n_1, n_1n_2)$ with $(2m_1n_2 + 2m_2n_1, 2n_1n_2)$. If we choose different representations when we do addition, we might get a different representation in our result, but it will represent the same rational number regardless of the choice of representations we originally chose to do the addition.

A natural question then becomes if the rationals are sufficient for doing all of real analysis. Certainly, it seems as we have a number system that is closed under all our basic arithmetic operations; but is this enough? For example, are we able to take limits just using the rationals? The answer turns out to be no (they are insufficient!) and the following example will serve as one illustration of this fact.

\begin{example}{Incompleteness of the Rationals}{1.1a}
    There exists no $p \in \QQ$ such that $p^2 = 2$.
\end{example}
\noindent We proceed via proof by contradiction. Recall in that these types of proof, we start with a certain wrong assumption, follow a correct/true line of reasoning, reach an eventual absurdity, and therefore conclude that the original assumption was mistaken.
\begin{nproof}
    Let us then suppose for the contradiction that there exists $p = \frac{m}{n}$ with $p^2 = 2$. We then have that not both $m, n$ are even, and hence at least one is odd. Then, we have that $2 = p^2 = \frac{m^2}{n^2}$ and hence $m^2 = 2n^2$, so $m^2$ is even, implying $m$ is even. So, let us write $m = 2k$ for $k \in \ZZ$. Then, $(2k)^2 = 4k^2 = 2n^2$, and hence $2k^2 = n^2$. Therefore, $n^2$ is even and hence $n$ is even. $m$ and $n$ are therefore both even, a contradiction. We conclude that no such $p$ exists. \qed
\end{nproof}
\noindent Why can we conclude that not both $m, n$ are even in the above proof? This is the case as if $m, n$ we both even, then we could write $m = 2m'$, $n = 2n'$ for some $m', n'$, and then $p = \frac{m}{n} = \frac{2m'}{2n'} = \frac{m'}{n'}$ which we can continue until either the numerator or denominator is odd. A natural question to consider is how to prove that this process of reducing fractions will eventually conclude. The resolution is to invoke the fundamental theorem of arithmetic, and write $m, n$ in terms of their unique prime factorization. We are then able to cancel out factors of 2 from the numerator/denominator until at least one is odd.

We note that this example leads us to conclude that the rationals have certain ``holes'' in them. This is concerning, as there are sequences of rational numbers that tend to $\sqrt{2}$. Conversely, its not as concerning that there is no rational number $x$ such that $x^2 = -1$, as there is no such sequence of rational numbers that is "close to" $i$ (note that both $\sqrt{2}$ and $i$ have not yet been defined, but this will come shortly).

\setcounter{rudin}{0}

\begin{example}{Incompleteness of the Rationals}{1.1b}
    Let $A = \set{p \in \QQ: p > 0, p^2 < 2}$, and $B = \set{p \in \QQ: p > 0, p^2 > 2}$. Then, $\forall p \in A, \exists q \in A$ such that $p < q$, and $\forall p \in B, \exists q \in B$ such that $q < p$.
\end{example}
\begin{figure}[htbp]
    \centering
    \begin{tikzpicture}
        \draw[latex-latex, very thick] (-6, 0) -- (6,0) node[anchor=south] {$\QQ$};
        \draw[] (0, 0) -- (0, -0.25) node[anchor=north] {0};
        \foreach \i in {-5.7,-5.6,...,5.7}{
        \draw[] (\i,0) -- (\i,-0.1);
        }
        \draw[] (1,0.1) node[anchor=south] {$\sqrt{2}$};
        \draw[] (0.96,0) node[] {$)$};
        \draw[] (1.04,0) node[] {$($};
        \draw[] (-2.5, 0) node[anchor=south] {$A$};
        \draw[] (3.5, 0) node[anchor=south] {$B$};
    \end{tikzpicture}
    \caption{Visualization of sets $A$ and $B$. We note that $\sqrt{2}$ has not been defined in our formalism yet, but from our prior mathematical intuition it would be what goes in the "hole" of the rationals.}
    \label{fig1}
\end{figure}

\noindent For the proof of this statement, we consider playing a 2 person game. One person is $\forall$, one person is $\exists$, and we consider if one person has a winning strategy. $\forall$ goes first, and then $\exists$ goes next, having seen the choice that $\forall$ has made. Then, we check if indeed $p < q$. If $p < q$, then $\exists$ wins. If $p \not< q$, then $\forall$ wins.

\begin{nproof}
    Let $p \in A$. Then, let $q = \frac{2p + 2}{2 + p}$. Since $p \in \QQ$, it follows that $2p + 2 \in \QQ$ and $2 + p \in \QQ$ so $q \in \QQ$. Furthermore, we have that $2p + 2 > 0$ and $2 + p > 0$, so $q > 0$. We also have that:
    \[q^2 = \frac{(2p+2)^2}{(2+p)^2} = 2 + \frac{2(p^2 - 2)}{(p+2)^2} < 2\]
    Where the inequality follows from the fact that $p^2 < 2$ and hence $(p^2 - 2) < 0$. It therefore follows that $q \in A$. Finally, we have that:
    \[q = p + \frac{2-p^2}{2+p} > p\]
    so $q > p$, completing the proof of the first part of the claim. The second part is left as an exercise (we note that the same $q$ can be used). \qed
\end{nproof}

\noindent The number $q = \frac{2p+ 2}{2 + p}$ seems to be pulled out of a hat, but actually comes from a fairly geometric picture (the secant method of approximating roots). Discussion on this topic can be found here: \url{https://math.stackexchange.com/questions/141774/choice-of-q-in-baby-rudins-example-1-1}.

\subsection{Ordered Sets}
Over the next couple sections, we will be discussing certain properties of sets that will give us a better understanding of the real numbers, and allow us to construct them.

\setcounter{rudin}{4}

\begin{definition}{Order}{1.5}
    An \textbf{order} $<$ on a set $S$ is a relation with the following properties:
    \begin{enumerate}[(i)]
        \item For every pair $x, y \in S$, exactly one of $x < y$, $x = y$, or $y < x$ is true.
        \item For $x, y, z \in S$, if $x < y$ and $y < z$, then $x < z$.
    \end{enumerate}
    A point on notation; We note that $x > y$ means $y < x$, and $x \leq y$ means $x < y$ or $x = y$.
\end{definition}

\begin{definition}{Ordered Sets}{1.6}
    An \textbf{ordered set} is a pair $(S, <)$. We may write just $S$ if the order can be inferred by the context.
\end{definition}
\noindent A familiar (and useful) set of examples is $S = \NN$ or $S = \ZZ$ or $S = \QQ$. For these three sets, we have that $x < y$ if $y-x$ is positive. For another example, consider the set $S$ of english words; then the order $<$ can be the dictionary/lexographic order.

\begin{definition}{Upper \& Lower Bounds}{1.7}
    Let $S$ be an ordered set and $E \subset S$ (for the duration of these notes, we will follow Rudin's notation, with $E \subset S$ as a non-strict subset, and $E \subsetneq S$ as a strict subset). $E$ is \textbf{bounded above} if there exists an element $\beta \in S$ such that $\forall x \in E$, $x \leq \beta$. Any such $\beta$ is an \textbf{upper bound} of $E$. Similarly, we say that $E$ is \textbf{bounded below} if there exists an element $\alpha \in S$ such that $\forall x \in E$, $\alpha \leq x$. In this case, $\alpha$ is a \textbf{lower bound} of $E$.
\end{definition}
\noindent As an example, one can take $S = \QQ$, $E = A = \set{p \in \QQ: p > 0, p^2 > 2}$ (as in Example \ref{exam:1.1b}(b)). Here, $E$ is bounded above, with $\beta = 2$ as one possible upper bound. to see this is the case, consider that if $p \in E$:
\[2 - p = \frac{4 - p^2}{2+p} > \frac{4-2}{2+p} > 0\]
\noindent However, if we take $S = A$, $E = A$, then $E$ is not bounded above as we saw in the example. There is no upper bound of $A$ in $A$. In general, this example reveals the subtle point that "the upper bound of a set" is ill-defined; we need to specify $E \subset S$.

\subsection{The Least Upper Bound Property}
\begin{definition}{Least Upper Bound \& Greatest Lower Bound}{1.8}
    Let $S$ be an ordered set, and let $E \subset S$ with $E$ bounded above. If $\exists \alpha \in S$ such that:
    \begin{enumerate}[(i)]
        \item $\alpha$ is an upper bound for $E$
        \item If $\gamma < \alpha$, then $\gamma$ is not an upper bound for $E$
    \end{enumerate}
    The $\alpha$ is the \textbf{least upper bound}, or \textbf{supermum} of $E$. This can be denoted as $\alpha = \sup(E)$. Analogously, the \textbf{greatest lower bound}, or \textbf{infimum} of E (denoted $\alpha = \inf(E)$) is an element $\alpha \in S$ (if it exists) such that:
    \begin{enumerate}[(i)]
        \item $\alpha$ is a lower bound for $E$
        \item If $\gamma > \alpha$, then $\gamma$ is not an upper bound of $E$.
    \end{enumerate}
\end{definition}

\begin{ntheorem}{}
    If the supremum/infimum of $E \subset S$ exist, they are unique.
\end{ntheorem}
\begin{nproof}
        Let $E \subset S$. Suppose that there exist $\alpha_1, \alpha_2$ such that $\alpha_1 = \sup(E)$ and $\alpha_2 = \sup(E)$. If $\alpha_1 < \alpha_2$, as $\alpha_1$ is an upper bound of $E$, this contradicts the fact that $\alpha_2$ is the least upper bound of $E$. We reach an identical contradiction if $\alpha_2 < \alpha_1$. Therefore we conclude that $\alpha_1 = \alpha_2$ and the supremum of $E$ is unique (if it exists). The proof for the infimum is analogous. \qed
\end{nproof}

\begin{ntheorem}{}
    If $E \subset S$ has a maximum element $\alpha$ (that is, an element such that $x < \alpha$ for all $x \in E$) then $\alpha = \sup(E)$. Similarly, if $E$ has a minimum element $\alpha$, then $\alpha = \inf(E)$.
\end{ntheorem}

\begin{nproof}
    Let $E \subset S$ and $\alpha = \max(E)$. By definition $\alpha$ is an upper bound of $E$, and if $x < \alpha$ for some $x \in E$ then $x$ is not an upper bound of $E$ as it is not greater than $\alpha \in E$. The claim follows (with an identical proof for the minimum). \qed
\end{nproof}

\begin{example}{}{1.9}
    \begin{enumerate}
        \item Consider again the sets $A, B \subset \QQ$ from example \ref{exam:1.1b}. $A$ is bounded above by any element in $B$, and the upper bounds of $A$ are exactly the elements of $B$. Since $B$ has no smallest member, $A$ does not have a least upper bound in $\QQ$.
        \item Let $E_1, E_2 \subset \QQ$ such that $E_1 = \set{r: \QQ, r < 0}$ and $E_2 = \set{r: \QQ, r \leq 0}$. Then $\sup(E_1) = \sup(E_2) = 0$. Note that this example shows that the supremum can either be contained or not contained in the set; $0 \notin E_1$ but $0 \in E_2$.
        \item Let $E \subset \QQ$ such that $E = \set{\frac{1}{n}: n \in \NN}$. Then $\sup(E) = 1$ and $\inf(E) = 0$. This is proven below.
    \end{enumerate}
\end{example}
\begin{nproof}
    $\sup(E) = 1$ immediately follows from the equivalence of the maximum and supremum as proven above. To see that $\inf(E) = 0$, first note that $0$ is a lower bound for $E$ as all of the elements of $E$ are positive. To see that it is the lower bound, take any $x > 0$. Then, we have that for any $n > \frac{1}{x}$, $\frac{1}{n} < x$ and hence $x$ is not an upper bound of $E$. This proves the claim. \qed
\end{nproof}

\begin{definition}{The LUB/GUB Property}{1.10}
    An ordered set $S$ has the \textbf{least upper bound property} if for every $E \subset S$, if $E \neq \emptyset$ and $E$ is bounded above, then $E$ has a least upper bound (that is, $\sup(E)$ exists in $S$). Similarly, an ordered set $S$ has the \textbf{greatest lower bound property} if for every $E \subset S$, if $E \neq \emptyset$ and $E$ is bounded below, then $E$ has a greatest lower bound.
\end{definition}
\noindent We will show in the next theorem that these properties are actually equivalent; before then, we briefly consider two examples.
\begin{nexample}{}
    $\ZZ$ has the least upper bound property, while $\QQ$ does not.
\end{nexample}
\begin{nproof}
    For the first claim, consider any nonempty $E \subset \ZZ$ that is bounded above. Choose any $x \in E$. Since $\ZZ$ is bounded above, there exist finitely many elements that are greater than $x$. Take the maximum of these finitely many elements. This maximum is also the maximum of $E$, so it is the supremum of $E$. Therefore $\ZZ$ has the LUB property as claimed.

    The second claim immediately follows from Example \ref{exam:1.9}(a). \qed
\end{nproof}

\begin{theorem}{}{1.11}
    Let $S$ be an ordered set. Then $S$ has the LUB property if and only if it has the GUB property.
\end{theorem}
\begin{nproof}
    $\boxed{\implies}$ Let $S$ be an ordered set with the LUB property. Let $E \subset S$ with $E \neq \emptyset$, with $E$ bounded below. Let $L = \set{x \in S: x\text{ is a lower bound of $E$.}}$. $L \neq \emptyset$ as $E$ is bounded below (and hence has at least one lower bound). If $y \in E$, then $y$ is an upper bound for $L$. Since $E$ is nonempty, $L$ is therefore bounded above. Since $S$ has the LUB property, then $\sup(L)$ must exist. Let us call this $\alpha$. Then, $\alpha \leq x\ \forall x \in E$ (as if $\gamma < \alpha$, then $\gamma$ is not an upper bound of $L$ and hence $\gamma \neq E$). Hence, $\alpha$ is a lower bound for $E$ and hence $\alpha \in L$. Since $\alpha = \sup(L)$ and $\alpha$ is an upper bound for $L$, we have that $\alpha \geq \gamma\ \forall \gamma \in L$. Thus, $\alpha = \inf(E)$.

    $\boxed{\impliedby}$ Left as an exercise. \qed
\end{nproof}

\subsection{Fields and Ordered Fields}
\begin{definition}{Fields}{1.12}
    A \textbf{field} $F$ is a set with two binary operations, $+$ and $\cdot$ (addition and multiplication) such that the following axioms are satisfied:
    \begin{enumerate}[start=1, label={(A\arabic*):}]
    \item If $x, y \in F$, then $x + y \in F$. (Closure under addition)
    \item $x + y = y + x$ for all $x, y \in F$. (Commutativity of addition)
    \item $(x+y) + z = x + (y + z)$ for all $x, y, z \in F$. (Associativity of addition)
    \item $\exists 0 \in F$ such that $\forall x \in F$, $0 + x = x$. (Additive identity)
    \item $\forall x \in F$, $\exists y$ such that $x + y = 0$. We can denote $y = -x$. (Additive inverse)
    \end{enumerate}
    \begin{enumerate}[start=1, label={(M\arabic*):}]
        \item If $x, y \in F$, then $x\cdot y\in F$. (Closure under multiplication)
        \item $x \cdot y = y \cdot x$ for all $x, y \in F$.
        \item $(x\cdot y)\cdot z = x \cdot (y \cdot z)$ for all $x, y, z \in F$. (Associativity under multiplication)
        \item $\exists 1 \in F$ such that $1 \neq 0$ and $\forall x \in F$, $1 \cdot x = x$. (Multiplicative identity)
        \item $\forall x \in F$, exists $y \in F$ such that $x \cdot y = 1$. We can denote $y = \frac{1}{x}$. (Multiplicative inverse)
    \end{enumerate}
    (D): $x \cdot (y + z) = x \cdot y + x \cdot z$, $\forall x, y, z \in F$. (Distributive law)
\end{definition}
\noindent Note that A3/M3 show that $x + y + z$ and $x\cdot y\cdot z$ are well defined in a mathematical sense; however, associativity may not hold for computers that do math with finite precision!
\begin{ntheorem}{}
    The additive/multiplicative identities given by (A4)/(M4) and the additive/multiplicative inverses given by (A5)/(M5) are unique.
\end{ntheorem}
\begin{nproof}
    Let $F$ be an ordered field. Suppose that there exist $0_1, 0_2 \in F$ such that $0_1 + x= x$ and $0_2 + x = x$ for all $x \in F$. We then have that:
    \begin{align*}
        0_1 + 0_2 &= 0_1 + 0_2
        \\ 0_1 + 0_2 &= 0_2 + 0_1 & \text{(A2)}
        \\ 0_2 &= 0_1 & \text{(Property of additive identity)}
    \end{align*}
    Which shows that the additive identity is unique. The remaining proofs are left as an exercise. \qed
\end{nproof}
\noindent Some easy (and familiar) consequences of the field axioms can be found in Rudin 1.14-1.16. Instead of repeating those here, we will discuss some examples.

The rationals form a field (under the usual notions of addition/multiplication), but the integers do not, as there are no multiplicative inverses (e.g. there exists no integer $x \in \ZZ$ such that $2\cdot x = 1$). The simplest example of a field is $F = \set{0, 1}$, with the relations:
\begin{align*}
    0 + 0 = 0\quad 0\cdot0 = 1
    \\ 0 + 1 = 0 \quad 0 \cdot 1 = 0
    \\ 1 + 1 = 0 \quad 1 \cdot 1 = 1
\end{align*}
This field is often called $\mathbb{F}_2$ or $F_2$, and is useful in computer science (where bits can take on two states, 0 or 1). As a slight tangent, a byte (8 bits) can be considered an element of an 8-dimensional vector space over the field $\mathbb{F}_2$, where $+$ would be the XOR operator and $\cdot$ would be the AND operation.

A generalization of the above example is $\mathbb{F}_p$ or $F_p$, for a prime number $p$. This field would consist of the elements $0, 1, \ldots, p-1$. The addition and multiplication are carried out mod $p$. An interesting result is that in general, finite fields must have cardinality of some prime power.

Note that a field cannot have a single element; the field axioms (A4) and (M4) require the existence of distinct additive and multiplicative identities, which a singleton set cannot satisfy.

Although algebra is not the focus of this course, it may be interesting to briefly think about sets with less structure than a field. We start by considering a group.

\phantom{i}

\noindent A \textbf{group} $G$ is a set with a binary operation $(a,b) \mapsto a\cdot b$ such that the following axioms are satisfied:
\begin{enumerate}[start=1, label={(M\arabic*):}]
    \item If $a, b \in G$, then $a\cdot b \in G$ (Closure)
    \stepcounter{enumi}
    \item For $a, b, c \in G$, $(a\cdot b)\cdot c = a\cdot(b\cdot c)$ (Associativity)
    \item There exists $1 \in G$ such that $\forall x \in G$, $1 \cdot x = x$. (Identity)
    \item $\forall x \in G$, there exists $y \in G$ such that $x \cdot y = 1$. (Inverse)
\end{enumerate}

We note that $\ZZ$ is a group under addition, but not under multiplication (due to lack of multiplicative inverses). We can also consider the set of 2x2 matrices with integer entries:
\[G = \set{\m{a & b \\ c & d}: a, b, c, d \in \ZZ}\]
$G$ is again a group under matrix addition, but not under matrix multiplication (as not every matrix in $G$ is invertible). If we restricted $G$ to be the set of $2\times 2$ invertible matrices, in this case it could form a group under matrix multiplication. A set with slightly more structure than a group (though not quite as structured as a field) is a ring:
\newpage
\noindent A \textbf{ring} $R$ is a set with two binary operations $(a,b) \mapsto a + b$ and $(a, b) \mapsto a \cdot b$ such that the following axioms are satisfied:
\begin{enumerate}[start=1, label={(A\arabic*):}]
    \item If $x, y \in R$, then $x + y \in R$. (Closure under addition)
    \item $x + y = y + x$ for all $x, y \in R$. (Commutativity of addition)
    \item $(x+y) + z = x + (y + z)$ for all $x, y, z \in R$. (Associativity of addition)
    \item $\exists 0 \in R$ such that $\forall x \in R$, $0 + x = x$. (Additive identity)
    \item $\forall x \in R$, $\exists y$ such that $x + y = 0$. We can denote $y = -x$. (Additive inverse)
    \end{enumerate}
    \begin{enumerate}[start=1, label={(M\arabic*):}]
        \item If $x, y \in R$, then $x\cdot y\in R$. (Closure under multiplication)
        \stepcounter{enumi}
        \item $(x\cdot y)\cdot z = x \cdot (y \cdot z)$ for all $x, y, z \in R$. (Associativity under multiplication)
        \item $\exists 1 \in R$ such that $1 \neq 0$ and $\forall x \in R$, $1 \cdot x = x$. (Multiplicative identity)
    \end{enumerate}
    \begin{enumerate}[start=1, label={(D\arabic*):}]
        \item $x \cdot (y + z) = x \cdot y + x \cdot z$, $\forall x, y, z \in R$. (Left distributivity)
        \item $(y + z) \cdot x = y \cdot x + z \cdot x$, $\forall x, y, z \in R$. (Right distributivity)
    \end{enumerate}

\noindent Rings have the same axioms as fields under addition, but multiplication is not necessarily commutative (this is why an additional distributivity axiom is added), and multiplicative inverses are not required. We note that $\ZZ$ and $G$ are both rings under their respective operations of addition and multiplication.

For the remainder of this course, we will really only be discussing fields; however, they will be the objects of interest in abstract algebra courses!

\setcounter{rudin}{16}
\begin{definition}{Ordered Field}{1.13}
    An \textbf{Ordered field} is a field $F$ that is also an ordered set, such that the following axioms are satisfied:
    \begin{enumerate}[(i)]
        \item If $x, y, z \in F$ and $y < z$, then $x + y < x + z$.
        \item If $x, y \in F$ and $x > 0, y > 0$, then $x\cdot y > 0$.
    \end{enumerate}
\end{definition}
\noindent Some properties of ordered fields are discussed in Rudin 1.18. We will again refer the reader to the discussion in the textbook for these properties, and here consider some examples.

$\QQ$ is an ordered field, with the familiar order of $a > b$ if $a - b > 0$. A question may arise if $\mathbb{F}_2$ is an ordered field. A priori fields do not have order, but is it possible to impose an order on this set such that it is an ordered field? The answer turns out to be no.

\begin{proof}
    It suffices to show that both possible orderings leads to a contradiction. Suppose $0 < 1$. Then, $1 = 0 + 1 < 1 + 1 = 0$ which is a contradiction. Suppose instead that $1 < 0$. Then, $0 = 1 + 1 < 1 + 0 = 1$ which again is a contradiction.
\end{proof}

\stepcounter{rudin}

\begin{theorem}{Existence of $\RR$}{1.19}
    There exists an ordered field $\RR$ which has the LUB property and contains $\QQ$ as a subfield.
\end{theorem}
\noindent What does it mean for $\QQ$ to be a subfield? It means that there exists an injective function $\QQ \mapsto \RR$ that respects the properties of an ordered field.

This field $\RR$ happens to be exactly the set of real numbers we are familiar with. However, a natural question is ``what does it mean that there exsits a field?" It turns out that we can define the reals based on the definitions we have made already. One further question might be that could there not exists several fields with the above property; however, taking the appropriate view, we will find that there is a unqiue such field.

\subsection{Consequences of the LUB Property}
We will use the least upper bound property and the fact that $\RR$ has $\QQ$ as a subfield to derive its properties.
\begin{theorem}{Archimedian Property, Density of Rationals/Irrationals in $\RR$}{1.20}
    \begin{enumerate}
        \item If $x, y \in \RR$ and $x > 0$, then $\exists n \in \NN$ such that $nx > y$.
        \item If $x, y \in \RR$, and $x < y$, then $\exists p \in \QQ$ such that $x < p < y$. ($\QQ$ is dense in $\RR$)
        \item If $x, y \in \RR$, and $x < y$, then $\exists \alpha \in \RR \setminus \QQ$ such that $x < \alpha < y$. ($\RR\setminus\QQ$ is dense in $\RR$)
    \end{enumerate}
\end{theorem}

\begin{nproof}
    (a) Let $A = \set{nx: n \in \NN}$. Suppose for the sake of contradiction that the conclusion was false; then $y$ is an upper bound of $A$. Then, $\alpha = \sup(A)$ exists by the LUB property of $\RR$. Since $x > 0$, we then have that $\alpha - x < \alpha$ by the property of an ordered field. Hence, $\alpha - x$ is not an upper bound for $A$. Therefore, there exists some $m \in \NN$ such that $mx > \alpha - x$. It then follows that $(m+1)x > \alpha$. We therefore have found $m+1 = k \in \NN$ such that $kx > \alpha$, contradicting $\alpha$ being the least upper bound of $A$. \qed
\end{nproof}

\noindent In order to prove (b) and (c), we first prove a stronger version of 1.20(a):

\begin{nlemma}{}
    If $x, y \in \RR$ and $x > 0$, then there exists $n \in \ZZ$ such that $(n-1)x \leq y < nx$.
\end{nlemma}
\begin{nproof}
    Suppose $y \geq 0$. Let $A = \set{m \in \NN: y < mx} \subset \NN$. By Theorem \ref{thm:1.20} (a), we have that $A \neq \emptyset$. Every non-empty subset of $\NN$ has a smallest element (to see this, let $x \in A$, and define $A' = \set{y \in A: y \leq x}$. This is finite and nonempty and so has a smallest element, and the minimum element of this set will also be a lower bound and hence the minimum element of all of $A$), so let $n = \min(A)$. The claim holds for this $n$.
    The case for $y < 0$ is left as an exercise. \qed
\end{nproof}
\begin{nproof}
    (b) Since $y - x > 0$, by (a), $\exists n \in \NN$ such that $1 < n(y-x)$. Furthermore, by the Lemma we have that $\exists m \in \ZZ$ such that $m - 1 \leq nx < m$ and hence $m \leq nx + 1$. From these inequalities we obtain that $nx < m \leq nx + 1 < ny$, and therefore $x < \frac{m}{n} < y$ for some $m \in \ZZ$, $n \in \NN$. \qed
\end{nproof}
\noindent For the proof of part (c), we will use the result of Theorem \ref{thm:1.21} from the next section, specifically that there exists $s \in \RR \setminus \QQ$ such that $s > 0$ and $s^2 = 2$. We will call this $\sqrt{2}$.
\begin{nproof}
    (c) First, we have that $\sqrt{2} < 2$ as if $\sqrt{2} = 2$ then $(\sqrt{2})^2 = 2 = 2^2 = 4$ which is a contradiction, and if $\sqrt{2} > 2$ then $2 = \sqrt{2}\cdot \sqrt{2} > 2\cdot 2 = 4$ by Rudin 1.18 which is yet again a contradiction. Thus, $\frac{\sqrt{2}}{2} < 1$.

    Let $x, y \in \RR$ such that $x < y$. By Theorem \ref{thm:1.20}(b), there exists $p, q \in \QQ$ such that $x < p < q < y$. Let $\alpha = p + \frac{\sqrt{2}}{2}(q - p)$. Then, we have that $p  <\alpha < p + 1(q-p) < q$ and hence $x < p < \alpha < q < y$.

    If $\alpha \in \QQ$, then $\sqrt{2} = 2\left(\frac{\alpha-p}{q-p}\right) \in \QQ$, which is a contradiction, so it follows that $\alpha \in \RR \setminus \QQ$. \qed
\end{nproof}

\subsection{Integer Roots of the Reals}
In this section, we will prove that $\sqrt{2}$ exists and is an irrational number, but we will not use the fact that $\RR \setminus \QQ$ is dense in $\RR$; this would of course be circular reasoning. The more general idea will be to prove that for any $n \in \NN$, there exists $y \in \RR$ such that $y = x^{1/n}$. Before this, we prove a lemma.
\begin{nlemma}{}
    If $0 < a < b$ and $n \in \NN$, then $0 < b^n - a^n \leq nb^{n-1}(b-a)$
\end{nlemma}
\noindent Note that a ``Calculus proof'' of this Lemma would be to let $f(x) = x^n$, and then
\[f(b) - f(a) = f'(c)(b-a) = nc^{n-1}(b-a) \leq nb^{n-1}(b-a)\]
Where we invoke the mean value theorem. But this obviously doesn't work as we have neither defined a derivative nor proven the mean value theorem. A proper proof would be:
\begin{nproof}
    Let $0 < a < b$. Then, we may factor $b^n - a^n$ such that:
    \[b^n - a^n = (b-a)(b^{n-1} + ab^{n-2} + a^2b^{n-3} + \ldots + a^{n-2}b + a^{n-1})\]
    The second factor is a sum of $n$ terms, each positive, and in between $0$ and $b^{n-1}$. ThereforE:
    \[b^n - a^n \leq nb^{n-1}(b-a)\]
    which proves the claim. \qed
\end{nproof}
\noindent We will now state the theorem formally:
\begin{theorem}{Integer Roots of the Reals}{1.21}
    Let $x \in \RR$, $x > 0$, and $n \in \NN$. Then, there exists a unique $y \in \RR$ such that $y > 0$ and $y^n = x$.
\end{theorem}
\noindent Note that somewhere in the proof, we will use the fact that $y \in \RR$; this statement doesn't hold for rationals (see Example \ref{exam:1.1a}) so some property of the reals must come into play somewhere.
\begin{nproof}
    If $n = 1$, then the unique solution is $y = x$; we may therefore assume that $n \geq 2$.
    \\ \textbf{Uniqueness:} Suppose there exist two distinct numbers $y_1, y_2$ with $y_1 > 0, y_2 > 0$, and $y_1^n = y_2^n = x$. WLOG, suppose $0 < y_1 < y_2$. We then have that $0 < y_1^n < y_2^n$ which is a contradiction.
    \\ \textbf{Existence:} We prove existence in three steps.
    \begin{enumerate}[1.]
        \item We show that $E \neq \emptyset$. Let $E = \set{t \in \RR: t > 0, t^n < x}$. If $x < 1$, then $x^n < x$, so $x \in E$. If $x \geq 1$, then $\left(\frac{1}{2}\right)^n < \frac{1}{2} < x$, so $\frac{1}{2} \in E$. Therefore, $E \neq \emptyset$.
        \item We show that $E$ is bounded above and has a supremum in $\RR$. If $t > 1 + x$, then it follows that $t^n > t > x$, so $t \neq E$. Hence, $1 + x$ is an upper bound of $E$. By Theorem \ref{thm:1.19} (the LUB property of $\RR$), it follows that $\sup(E) \in \RR$ exists.
        \item We show that $y = \sup(E)$ satisfies $y^n = x$. As $\RR$ is an ordered field, one of $y^n < x$, $y^n = x$, or $y^n > x$ must be true; we show that the first and third are impossible.
        \begin{enumerate}
            \item Suppose $y^n < x$. We will obtain a contradiction by finding $h > 0$ such that $(y+h)^n < x$. (Why is this a contradiction? $y+ h > y$, so if $(y+h)^n < x$, then $y + h \in E$, contradicting the fact that $y + h$ would be an upper bound of $E$). WLOG, suppose that $h < 1$. By the above Lemma, we have that:
            \[(y+h)^n - y^n \leq n(y+h)^{n-1}h \leq n(y+1)^{n-1}h\]
            By choosing $h$ sufficiently small, that is:
            \[h < \min\set{1, \frac{x-y^n}{n(y+1)^{n-1}}}\]
            Then $n(y+1)^{n-1}h < x^n - y^n$ from which it follows that $(y+h)^n - y^n < x^n - y^n$ and so $y+h < x$, which is the desired contradiction.
            \item Suppose $y^n > x$. We will obtain a contradivction by finding $h > 0$ such that $(y-h)^n > x$. If this is true, then $y-h$ is an upper bound for $E$, contradicting the fact that $y$ is the least upper bound for $E$. WLOG suppose that $h < 0$. Again applying the Lemma, we have that:
            \[y^n - (y-h)^n \leq ny^{n-1}h\]
            By choosing $h$ sufficiently small, that is:
            \[h < \min\set{1, \frac{y^n-x}{ny^{n-1}}}\]
            It then follows that:
            \[y^n - (y-h)^n \leq ny^{n-1}h < y^n - x\]
            and hence $(y-h)^n > x$, which is the desired contradiction. \qed
        \end{enumerate}
    \end{enumerate}
\end{nproof}
\subsection{Construction of the Reals}
Theorem \ref{thm:1.19} says that there exists an ordered field that contains $\QQ$ as a subfield. We now go about proving this statement. The construction is fairly technical and hence will be carried out in multiple steps. Some of the steps are left as exercises (one can refer to Rudin for the fully complete construction).

\begin{nblank}{Step 1: Defining the elements of $\RR$}
    The members of $\RR$ will be proper subsets of $\QQ$, called cuts. $\RR = \set{\text{all cuts}}$.
    \begin{ndef}{: Cuts}
        A \textbf{cut} is a proper subset $\alpha \subsetneq \QQ$ with the three properties:
        \begin{enumerate}[(I)]
            \item $\alpha \neq \emptyset$
            \item If $p \in \alpha$, then $q \in \alpha \; \forall q < p$.
            \item If $p \in \alpha$, then $\exists r \in \alpha$ such that $p < r$.
        \end{enumerate}
    \end{ndef}
\end{nblank}
\begin{figure}[htbp]
    \centering
    \begin{tikzpicture}
        \draw[latex-latex, very thick] (-6, 0) -- (6,0) node[anchor=south] {$\QQ$};
        \foreach \i in {-5.7,-5.6,...,0.9}{
        \draw[] (\i,0) -- (\i,-0.1);
        }
        \draw[] (1,0.1) node[anchor=south] {$\downarrow$};
        \draw[] (0.96,0) node[] {$)$};
        \draw[] (-2.5, 0) node[anchor=south] {$\alpha$};
    \end{tikzpicture}
    \caption{Visualization of a cut $\alpha$. The real number being described of this cut can be thought of as the number at the right boundary (the arrow).}
    \label{fig2}
\end{figure}
\noindent In a sense, a cut gives us a way of discussing the real numbers (in the way we are familiar with them already) without referring to them directly; much like we could formally define/refer to rationals as equivalence classes of ordered pairs.

\noindent As a note, we could very well define cuts to be bounded below rather than above, and the following construction would still work out.

\begin{nblank}{Step 2: $\RR$ is an ordered set}
    We define $\alpha < \beta$ to mean $\alpha \subsetneq \beta$. We show that this makes $\RR$ into an ordered set. First checking transitivity, we have that if $\alpha < \beta$ and $\beta < \gamma$ then $\alpha < \gamma$ by the fact that set inclusion is transitive. Furthermore, at most one of $\alpha < \beta$, $\alpha = \beta$, and $\beta < \alpha$ hold; to see this is the case, suppose the first two fail. Then, $\alpha \nsubseteq \beta$. Hence, $\exists p \in \alpha$ with $p \notin \beta$. If $q \in \beta$, $q < p$  and hence $q \in \alpha$ by (II), so $\beta \subset \alpha$, and since $\beta \neq \alpha$ it follows that $\beta \subsetneq \alpha$.
\end{nblank}

\begin{nblank}{Step 3: $\RR$ has the LUB property}
    We show that $\RR$ has the LUB property. To see this is the case, let $A \subset \RR$ with $A \neq \emptyset$, and suppose that there exists $\beta \in \RR$ that is an upper bound for $A$. We will now define $\gamma = \bigcup_{\alpha \in A}\alpha$ and prove that $\gamma \in \RR$ and $\gamma = \sup A$ (hence $A$ has a supremum and $\RR$ has the LUB property).

    Since $A \neq \emptyset$, $\exists \alpha_0 \in A$, and since $\alpha_0 \neq \emptyset$ (as it is a cut) and $\alpha \subset \gamma$, it follows that $\gamma \neq \emptyset$. Next, we have that $\gamma \subset \beta$, since $\alpha \subset \beta$ for every $\alpha \in A$, and hence $\gamma \neq \QQ$, that is, $\gamma \subsetneq \QQ$. Hence $\gamma$ satisfies property (I) of a cut.

    Take $p \in \gamma$. Then $p \in \alpha_1$ for some $\alpha_1 \in A$. If $q < p$, then $q \in \alpha$ (as $\alpha$ is a cut) so $q \in \gamma$, satisfying property (II).

    Next, choose $r \in \alpha_1$ such that $r > p$, then $r \in \gamma$ (as $\alpha_1 \subset \gamma$) and hence $\gamma$ satisfies property (III). Hence $\gamma$ is a cut, and $\gamma \in \RR$.

    Finally, we show that $\gamma = \sup A$. Clearly, $\alpha \leq \gamma$ for all $\alpha \in A$, as $\gamma = \bigcup_{\alpha \in A}\alpha$, so $\gamma$ is an upper boun dof $A$. To show that it is the least upper bound, let $\delta < \gamma$ be a cut. Then, $\exists s \in \gamma$ such that $s \notin \delta$. Therefore, $\exists \alpha_2 \in A$ such that $s \in \alpha_2$; hence $\delta < \alpha_2$, so $\delta$ is not an upper bound for $A$, giving the desired result.
\end{nblank}

\begin{nblank}{Step 4: Addition on $\RR$}
    \begin{ndef}{: Addition}
        If $\alpha, \beta \in \RR$, we define $\alpha + \beta = \set{s + t: s \in \alpha, t \in \beta}$. Showing that this is a cut is left as an exercise.
    \end{ndef}
    \begin{ndef}{: Zero}
        $0^* = \set{s \in \QQ}$. Showing that this is a cut is left as an exercise.
    \end{ndef}

    We leave it as an exercise to show that the addition axioms (A1)-(A5) of a field are satisfied under this definition of addition on $\RR$, with the 0 element as $0^*$ defined above.
\end{nblank}

\begin{nblank}{Step 5: $\RR$ satisfies the Ordered Field Property (i)}
    We verify that if $\alpha, \beta, \gamma \in \RR$ and $\beta < \gamma$, then $\alpha + \beta < \alpha + \gamma$.

    For every $s \in \alpha, t \in \beta$, we have that $t \in \gamma$ as $\beta$ is a subset of $\gamma$ by the definition of order on $\RR$. Hence, $s + t \in \alpha + \beta$ implies $s + t \in \alpha + \gamma$. Therefore, $\alpha + \beta \subset \alpha + \gamma$ and hence $\alpha + \beta \leq \alpha + \gamma$.

    We are then left to check that $\alpha + \beta \neq \alpha + \gamma$. To see that this is the case, if $\alpha + \beta = \alpha + \gamma$, then $\beta = \alpha + \beta - \alpha = \alpha + \gamma - \alpha = \gamma$ by the field axioms for addition. Therefore we obtain that $\beta = \gamma$, contradicting that $\beta < \gamma$. Hence the claim is proven.

    As a remark, note that $0^* < \alpha \iff -\alpha < 0^*$.
\end{nblank}
\noindent Next we will define multiplication on $\RR$. A first attempt would be $\alpha \cdot \beta = \set{s \cdot t: s\in \alpha, t \in \beta}$. However, this definition is incosistent with negative numbers from what we require multiplication to accomplish. $-1 \cdot -1$ would fail to be a cut (it would not contain any negative numbers and hence fail criteria (II)) and $-1 \cdot 1$ would yield the entirety of the rationals (again not a cut!)

\begin{nblank}{Step 6: Positive Multiplication on $\RR$}
    \begin{ndef}{: Positive Reals}
        We define $\RR^+ = \set{\alpha \in \RR: \alpha > 0^*}$
    \end{ndef}
    \begin{ndef}{Multiplication of Positive Reals}
        If $\alpha, \beta \in \RR^+$, we define $\alpha \cdot \beta = \set{r \cdot s: r \in \alpha, r> 0, s \in \beta, s > 0} \cup \set{t \in \QQ, t \leq 0}$. Equivalently, $\alpha \cdot \beta = \set{p \in \QQ:  \leq r \cdot s: r \in \alpha, r > 0, s \in \beta, s > 0}$. We leave it as an exercise to show that $\alpha \cdot \beta \in \RR$, and moreover, $\alpha \cdot \beta \in \RR^+$. Showing this second fact proves ordered field property (ii).
    \end{ndef}
    \begin{ndef}{: One}
        $1^* = \set{r \in \QQ: r < 1}$. We again leave showing $1^* \in \RR^+$ as an exercise.
    \end{ndef}
\end{nblank}

\begin{nblank}{Step 7: Multiplication on all of $\RR$}
    \begin{ndef}{: Multiplication by zero}
        $\alpha \cdot 0^* = 0^* = 0^* \cdot \alpha$
    \end{ndef}
    \begin{ndef}{: Multiplication}
        We define general multiplication as below, where the $\cdot$ on the RHS represents the multiplication of positive reals as outlined in Step 5.
        \begin{align*}
            \alpha \cdot \beta =
            \begin{cases}
            (-\alpha)\cdot(-\beta) & \text{if $\alpha < 0^*$ and $\beta < 0^*$}
            \\ -\left((-\alpha)\cdot\beta\right) & \text{if $\alpha < 0^*$ and $\beta > 0^*$}
            \\ -\left(\alpha \cdot (-\beta)\right) & \text{if $\alpha > 0^*$ and $\beta < 0^*$}
            \end{cases}
        \end{align*}
    \end{ndef}
    We leave it as an exercise to show that the multiplicative axioms (M1)-(M5), as well as the distributive law (D) of a field are satisfied under this definition of multiplication on $\RR$.
\end{nblank}
\noindent Up until this point, we have shown $\RR$ is an ordered field with the LUB property; we last check that it contains $\QQ$ as a subfield. Note that we do have to be a bit careful with what we mean here; $\RR$ does not literally contain $\QQ$; $\RR$ is indeed a set of proper subsets of $\QQ$. What we really mean is to associate every element of $\QQ$ to an element of $\RR$ such that the field structure is preserved.
\begin{nblank}{Step 8: $\RR$ contains $\QQ$ as a subfield}
    For each $r \in \QQ$, associate the cut $r^* = \set{p \in \QQ, p < r^*}$. We then leave as an easy exercise to verify that $r^* < s^* \iff r < s$, $r^* + s^* = r + s$, and $r^*\cdot s^* = r\cdot s$. This concludes the construction of the reals. \qed
\end{nblank}
\noindent Note that later on in the course, we will construct the real numbers in a different fashion; by considering Cauchy sequences modulo an equivalence relation. Also note that from here on out, it will suffice to have the standard/traditional picture of a "real number" in mind (i.e. infinite decimal expansions) and we will not have to really think about the real numbers as cuts; this was just necessary for the formal construction.

\subsection{The Complex Field}
\setcounter{rudin}{23}
\begin{definition}{The Complex Numbers}{1.24}
    We define the set of \textbf{complex numbers} to be $\set{(a, b): a, b \in \RR}$, denoted by $\CC$. For $x = (a, b) \in \CC$ and $y = (c, d) \in \CC$, we write $x = y$ if and only if $a = c$ and $b = d$ (note that this is a very different notion of equality compared to the rationals). We define the zero element to be $(0, 0)$ and the one element to be $(1, 0)$. We define addition of complex numbers such that:
    \begin{align*}
        x + y = (a, b) + (c, d) = (a + c, b + d)
    \end{align*}
    And multiplication of complex numbers such that:
    \begin{align*}
        x\cdot y = (a, b)\cdot (c, d) = (ac - ba, ad + bc)
    \end{align*}
\end{definition}
\begin{theorem}{}{1.25}
    The operations of $+$ and $\cdot$, as well as the zero/one elements defined above turn $\CC$ into a field.
\end{theorem}
\begin{nproof}
    It suffices to verify the field axioms (A1)-(A5), (M1)-(M5), and (D) as discussed in \ref{def:1.12}. We will here show (M3), (M4), and (M5) and leave the rest as exercises.
    \begin{enumerate}[start=3, label={(M\arabic*):}]
    \item Let $x, y, z \in \CC$. We show that $(x\cdot y)\cdot z = x \cdot (y \cdot z)$. Let $x = (a, b), y = (c, d)$, and $z = (e, f)$. We then have that:
    \begin{align*}
        (x\cdot y) \cdot z &= (ac - bd, ad + bc) \cdot (e, f)
        \\ &= ((ac-bd)e - (ad+bc)f, (ac-bd)f + (ad+bc)e)
    \end{align*}
    We also have that:
    \begin{align*}
        x \cdot(y\cdot z) &= (a, b)\cdot(ce - df, cf + de)
        \\ &= (a(ce-df) - b(cf+de), a(cf+de) + b(ce-df))
        \\ &= (ace - adf - bcf - bde, acf + ade + bce - bdf)
        \\ &= ((ac-bd)e - (ad+bc)f, (ac-bd)f + (ad+bc)e)
    \end{align*}
    So the claim is proven.
    \item $(a, b)(1, 0) = (a \cdot 1 - b \cdot 0, a \cdot 0 + b \cdot 1) = (a, b)$
    \item Let $x \in \CC$ such that $x \neq 0$. Then, $x = (a, b)$ where either $a \neq 0$ or $b \neq 0$ or both. Hence, $a^2 + b^2 > 0$. Then, let $\frac{1}{x} = (\frac{a}{a^2 + b^2}, -\frac{b}{a^2+b^2})$. We then have that:
    \begin{align*}
        x\frac{1}{x} &= (a, b)\left(\frac{a}{a^2 + b^2}, -\frac{b}{a^2+b^2}\right)
        \\ &= \left(a\frac{a}{a^2 + b^2} - b\left(-\frac{b}{a^2+b^2}\right), a\left(-\frac{b}{a^2+b^2}\right) + b\left(\frac{a}{a^2+b^2}\right)\right)
        \\ &= \left(\frac{a^2 +b^2}{a^2 + b^2}, -\frac{ab}{a^2+b^2} + \frac{ab}{a^2+b^2}\right)
        \\ &= (1, 0)
    \end{align*}
    Which proves the claim. \qed
    \end{enumerate}
\end{nproof}
\noindent Much like $\QQ$ was a subfield of $\RR$, $\RR$ is a subfield of $\CC$, and there exists a map $\phi$ from $\RR$ to $\CC$ that respects the field axioms, namely:
\begin{align*}
    \fullfunction{\phi}{\RR}{\CC}{x}{(x, 0)}
\end{align*}
The theorem below shows that $\phi$ preserves the field structure:
\begin{theorem}{}{1.26}
    For $a, b \in \RR$ we have that $(a, 0) + (b, 0) = (a + b, 0)$ and $(a, 0)(b, 0) = (ab, 0)$.
\end{theorem}
\begin{definition}{i}{1.27}
    $i = (0, 1)$.
\end{definition}
\begin{theorem}{}{1.28}
    $i^2 = -1$.
\end{theorem}
\begin{theorem}{}{1.29}
    If $a, b \in \RR$, then $(a, b) = a + bi$.
\end{theorem}
\begin{nproof}
    Below are the trivial proofs for the above three theorems.
    \begin{align*}
        (a, 0) + (b, 0) = (a + b, 0 + 0) = (a + b, 0)
        \\ (a, 0)\cdot(b, 0) = (a\cdot b - 0 \cdot 0, a \cdot 0 + 0 \cdot b) = (ab, 0)
        \\ i^2 = i\cdot i = (0, 1) \cdot (0, 1) = (-1, 0) = -1
        \\ a + bi = (a, 0) + b(0, 1) = (a, 0) + (0, b) = (a, b)
    \end{align*}
\end{nproof}
\noindent A slightly odd question may be to ask whether $\CC$ is a subfield of $\RR$, i.e. does there exist $\psi: \CC \mapsto \RR$ such that $\psi(a + b) = \psi(a) + \psi(b)$ and $\psi(a\cdot b) = \psi(a) \cdot \psi(b)$. As we will prove in Chapter 2, we do have that $\abs{\CC} = \abs{\RR^2} = \abs{\RR}$ (where $\abs{}$ denotes cardinality of the set, to be defined shortly), so there does exist a bijection (i.e. a function that is both injective/one-to-one and surjective/onto; we will define these terms precisely in the next chapter) between the two sets.

As a Lemma, we have that the only injective function $f: \QQ \mapsto \RR$ that satisfies $f(a+b) = f(a) + f(b)$ and $f(a\cdot b) = f(a)\cdot f(b)$ is $f(x) = x$. The proof of this is left as a homework problem (HW2). Therefore, it follows that the only injective function $g: \QQ \times \set{0} \mapsto \RR$ (where $\times$ denotes the Cartesian product) is given by $g((x, 0)) = x$. We now give a proof that $\CC$ is not a subfield of $\RR$.

\begin{proof}
    Suppose then for the sake of contradiction that there exists an injective function $\psi: \QQ \times \set{0, 1} \mapsto \RR$. Such a function then must satisfy$\psi(i \cdot i) = \psi(-1) = -1$, and $\psi(i \cdot i) = \psi(i) \cdot \psi(i) = \psi((0, 1))\cdot \psi((0, 1)) = 0 \cdot 0 = 0$ which is a contradiction. Hence, no such injection exists from $\QQ \times \set{0, 1}$ to $\RR$ and hence no such injection could exist from $\CC$ ($\RR^2$) to $\RR$. Hence $\CC$ is not a subfield of $\RR$.
\end{proof}

\begin{definition}{Real/Imaginary Parts and Complex Conjugates}{1.30}
Let $z = a + bi \in \CC$. Then, $\Re(z) = a$ is the \textbf{real part} of $z$ and $\Im(z) = b$ is the \textbf{imaginary part} of $z$. The \textbf{complex conjugate} of $z$, denoted by $\bar{z}$, is defined as $\bar{z} = a - bi$.
\end{definition}

\begin{theorem}{}{1.31}
    Let $z, w \in \CC$. It then follows that:
    \begin{enumerate}
        \item $\overline{z + w} = \bar{z} + \bar{w}$.
        \item $\overline{zw} = \bar{z} \cdot \bar{w}$.
        \item $z + \bar{z} = 2\Re(z)$, $z - \bar{z} = 2i\Im(z)$.
        \item $z\bar{z}$ is real and positive (except when $z = 0$).
    \end{enumerate}
\end{theorem}

\begin{nproof}
    We prove (d). We have that:
    \begin{align*}
        z\bar{z} = (a + bi)(a-bi) = a^2 + b^2
    \end{align*}
    $a^2 + b^2 \geq 0$, and $a^2 + b^2 = 0 \iff a = 0, b = 0$ which proves the claim. \qed
\end{nproof}

\begin{definition}{Absolute Value}{1.32}
    We define the \textbf{absolute value} $\abs{z}$ of a complex number $z$ as $\abs{z} = \sqrt{z\bar{z}}$. Note that if $a \in \RR$ and $z = (a, 0)$, then
    \begin{align*}
        \abs{z} = \sqrt{a^2} =
        \begin{cases}
            a & \text{if $a \geq 0$}
            \\ -a & \text{if $a < 0$}
        \end{cases}
    \end{align*}
    Hence if $a \in \RR$, we can define $\abs{a} = \abs{(a, 0)}$.
\end{definition}

\begin{theorem}{}{1.33}
    Let $z, w \in \CC$.
    \begin{enumerate}
        \item $\abs{z} \geq 0$, $\abs{z} = 0 \iff z = 0$.
        \item $\abs{\bar{z}} = \abs{z}$.
        \item $\abs{z}\abs{w} = \abs{zw}$.
        \item $\abs{\Re(z)} \leq \abs{z}$, $\abs{\Im(z)} \leq \abs{z}$.
        \item $\abs{z + w} \leq \abs{z} + \abs{w}$.
    \end{enumerate}
\end{theorem}
\begin{nproof}
    We prove (d) and (e). Let $z, w \in \CC$, with $z = a + bi$. For (d) we have that $\Re(z) = a$, so \begin{align*}
        \abs{\Re(a)} = \sqrt{a^2} \leq \sqrt{a^2 + b^2} = \abs{z}
    \end{align*}
    And an equivalent proof follows for $\Im(z)$. For (e), we have that:
    \begin{align*}
        \abs{z + w}^2 &= (z + w)(\overline{z + w})
        \\ &= z\bar{z} + z\bar{w} + w\bar{z} + w\bar{w}
        \\ &= \abs{z}^2 + 2\Re(z\bar{w}) + \abs{w}^2
        \\ &\leq \abs{z}^2 + 2\abs{\Re(z\bar{w})} + \abs{y}^2 & \text{($\abs{x} > x$)}
        \\ &= \abs{z}^2 + 2\abs{z\bar{w}} + \abs{w}^2 & \text{(1.33(d))}
        \\ &= \abs{z}^2 + 2\abs{z}\abs{\bar{w}} + \abs{w}^2 & \text{(1.33(c))}
        \\ &= \abs{z}^2 + 2\abs{z}\abs{w} + \abs{w}^2 & \text{(1.33(b))}
        \\ &= (\abs{z} + \abs{w})^2
    \end{align*}
The claim follows by taking square roots on both sides. \qed
\end{nproof}

\subsection{The Cauchy-Shwartz Inequality}
Recall the summation notation:
\begin{align*}
    x_1 + x_2 + \ldots + x_n = \sum_{j=1}^{n}x_i
\end{align*}
\stepcounter{rudin}
\begin{theorem}{Cauchy-Shwartz Inequality}{1.35}
    Let $a_1, \ldots, a_n, b_1, \ldots, b_n \in \CC$. We then have that:
    \begin{align*}
        \abs{\sum_{j=1}^na_j\bar{b}_j}^2 \leq \left(\sum_{j=1}^n\abs{a_j}^2\right)\left(\sum_{j=1}^n\abs{b_j}^2\right)
    \end{align*}
\end{theorem}
\noindent Note that in the above theorem, both the RHS and the LHS are real numbers (check!) so the equality makes sense (recall that there is no order on $\CC$; in fact, it is impossible to define one).

A geometric interpretation of the above inequality is as follows. Let $\v{a}, \v{b}$ be vectors in $\CC^n$. Then, $\avg{\v{a}, \v{b}} = \sum_{j=1}^na_j\bar{b}_j$ is the inner product of $\v{a}$ and $\v{b}$. Then, the inequality says that $\abs{\avg{\v{a}, \v{b}}}^2 \leq \avg{\v{a}, \v{a}}\cdot\avg{\v{b}, \v{b}}$.
\begin{nproof}
    Define $A = \sum_{j=1}^n\abs{a_j}^2$, $B = \sum_{j=1}^n\abs{b_j}^2$, and $C = \sum_{j=1}^na_j\bar{b}_j$. If $B = 0$ (that is, all of the $b_j$s are zero) then the LHS/RHS are both zero and we are done. So, let us assume that $B > 0$. Let $\lambda \in \CC$, and we then have that:
    \begin{align*}
        0 &\leq \sum_{j=1}^n\abs{a_j + \lambda b_j}^2 \\ &= \sum_{j=1}^n(a_j + \lambda b_j)(\bar{a}_j + \bar{\lambda}\bar{b}_j)
        \\ &= \sum_{j=1}^n\abs{a_j}^2 + \bar{\lambda}\sum_{j=1}^na_j\bar{b}_j + \lambda\sum_{j=1}^n\bar{a}_jb_j + \abs{\lambda}^2\sum_{j=1}^n\abs{b_j}^2
        \\ &= A + \bar{\lambda}C + \lambda\bar{C} + \abs{\lambda}^2B
    \end{align*}
    This inequality holds for any $\lambda$; it therefore holds for $\lambda = -\frac{C}{B}$, so:
    \begin{align*}
        0 &\leq A - \frac{\bar{C}}{B}C - \frac{C}{B}\bar{C} + \frac{C\bar{C}}{B^2}B
        \\ &= A - \frac{\abs{C}^2}{B}
    \end{align*}
    So we therefore obtain that $\abs{C}^2 \leq AB$ which is the desired inequality. \qed
\end{nproof}
\noindent A natural question given any inequality is when does equality hold; the answer turns out to be if the vectors are linearly independent, that is, at least one of $\v{a} = \alpha \v{b}$ and $\v{b} = \beta \v{a}$ ($\alpha, \beta \in \CC$) hold. Note that we only require one of the two relations to hold; in the case that one of $\v{a}, \v{b}$ are $\v{0}$ (the vector of all zeros) both equalities cannot be true. It is left as a homework problem to verify equality in the Cauchy-Shwartz inequality if and only if at least one of the two conditions holds (HW3).

\subsection{Euclidean Space}
\begin{definition}{Euclidean k-space}{1.36}
    If $k \in \NN$, define $\RR^k$ as the set of $k$-tuples of real numbers:
    \begin{align*}
        \RR^k = \set{\v{x} = (x_1, x_2, \ldots, x_k): x_1, x_2, \ldots, x_k \in \RR}
    \end{align*}
    We can then define vector addition as:
    \begin{align*}
        \v{x} + \v{y} = (x_1 + y_1, x_2 + y_2, \ldots, x_k + y_k)
    \end{align*}
    And scalar multiplication (for $\alpha \in \RR$) to be:
    \begin{align*}
        \alpha\v{x} = (\alpha x_1, \alpha x_2, \ldots, \alpha x_k)
    \end{align*}
    These operations make $\RR^k$ into a vector space over the real field. We can define the inner product over $\RR^k$ to be:
    \begin{align*}
        (\v{x}, \v{y}) = \v{x} \cdot \v{y} = \sum_{j=1}^k x_jy_j
    \end{align*}
    This allows us to define the norm of $\v{x}$ to be:
    \begin{align*}
        \abs{\v{x}} = \sqrt{\v{x} \cdot \v{x}} = \left(\sum_{j=1}^n x_j^2\right)^{1/2}
    \end{align*}
    $\RR^k$ with the above inner product and norm is called \textbf{Euclidean k-space}.
\end{definition}
\noindent We briefly remark that the above inner product we defined agrees with the inner product we defined over $\CC^k$; we can identify $r \in \RR$ with $(r, 0) \in \CC$, and hence recognize that $\RR^k \subset \CC^k$ where the imaginary part of each coordinate is zero. Then, for the inner product we get the exact same result, as $\bar{b}_j = b_j$ for any complex numbers with imaginary part zero. From this we can conclude that the Cauchy-Shwartz inequality also holds in $\RR^k$.

Note that although the field $\CC$ is $\RR^2$ with multiplication defined as in Definition \ref{def:1.24}, in general vector multiplication on $\RR^n$ is not well defined. That is, we cannot make $\RR^n$ into a field in general; though we can make it into a vector space, which has slightly less structure.

One possibly familiar notion of vector multiplication in $\RR^3$ is the cross product. For $\v{x} = (x_1, x_2, x_3)$ and $\v{y} = (y_1, y_2, y_3)$, the cross product is defined as:
\begin{align*}
    \v{x} \times \v{y} = (x_2y_3 - x_3y_2, x_3y_1 - x_1y_3, x_1y_2 - x_2y_1)
\end{align*}
However, the cross product does not satisfy properties that would be necessary to make $\RR^3$ a field. For one, it is not commutative, but anticommutative; $\v{x} \times \v{y} = -\v{y} \times \v{x}$. One might ask whether vectors in $\RR^3$ have well-defind inverses, but even before that, there does not exist an identity vector in $\RR^3$ under the cross product! In fact, $\RR^3$ under vector addition and cross product multiplication can be viewed as a noncommutative ring without an identity.

Note that there is a more general notion of a ``wedge product'' between vectors in $\RR^n$. We are in a sense very ``lucky'' that in $\RR^3$, the wedge product of two vectors returns another vector in $\RR^3$.

\begin{theorem}{}{1.37}
    Let $\v{x}, \v{y}, \v{z} \in \RR^k$, and $\alpha \in \RR$. Then:
    \begin{enumerate}
        \item $\abs{\v{x}} \geq 0$
        \item $\abs{\v{x}} = 0 \iff \v{x} = (0, \ldots, 0)$. This is often denoted as $\v{0}$, the ``zero vector''.
        \item $\abs{\alpha\v{x}} = \abs{\alpha}\abs{\v{x}}$
        \item $\abs{\v{x} \cdot \v{y}} \leq \abs{\v{x}}\abs{\v{y}}$
        \item $\abs{\v{x} + \v{y}} \leq \abs{\v{x}} + \abs{\v{y}}$
        \item $\abs{\v{x} - \v{z}} \leq \abs{\v{x} - \v{y}} + \abs{\v{y} - \v{z}}$
    \end{enumerate}
\end{theorem}
\noindent (e) and (f) are often called ``triangle inequalities''; a visual intuition for these inequalities is given in the following figure:
\begin{figure}[htbp]
    \centering
    \begin{tikzpicture}
    \draw[black, thick] (0, 0) node[anchor=north] {$\v{z}$} -- (2, 2) node[anchor=south] {$\v{y}$}  -- (-2, 2) node[anchor=south] {$\v{x}$} -- (0, 0);
    \draw[] (1.6, 1) node[anchor=north] {$\abs{\v{y} - \v{z}}$};
    \draw[] (-1.6, 1) node[anchor=north] {$\abs{\v{x} - \v{z}}$};
    \draw[] (0, 2) node[anchor=south]  {$\abs{\v{x} - \v{y}}$};
    \end{tikzpicture}
    \caption{Visual picture for Theorem \ref{thm:1.37}(f), drawn in $\RR^2$. Suppose we started at $\v{x}$ and wanted the shortest path to $\v{z}$; we could try walking directly to $\v{z}$, or we could try walking somewhere else first ($\v{y}$) and then to $\v{z}$. However, the theorem tells us that the direct path will always be shorter in Euclidean space.}
    \label{fig3}
\end{figure}

\noindent Note that equality in part (f) arises if and only if $\v{y}$ lies on the line segment between $\v{x}$ and $\v{z}$.

\begin{nproof}
    (a)-(c) are immediate, and (d) immediately follows from Theorem \ref{thm:1.35} (Cauchy-Shwartz). For (e), we have that:
    \begin{align*}
        \abs{\v{x} + \v{y}}^2 &= (\v{x} + \v{y})(\v{x} + \v{y})
        \\ &= \abs{\v{x}}^2 + 2\v{x}\cdot\v{y} + \abs{\v{y}}^2
        \\ &\leq \abs{\v{x}}^2 + \abs{2\v{x}\cdot\v{y}} + \abs{\v{y}}^2
        \\ &\leq \abs{\v{x}}^2 + 2\abs{\v{x}}\abs{\v{y}} + \abs{\v{y}}^2 \quad \text{(1.37(d))}
        \\ &= (\abs{\v{x}} + \abs{\v{y}})^2
    \end{align*}
    And the claim follows by taking square roots on both sides. For (f), substitute $\v{x} \mapsto \v{x} - \v{y}$ and $\v{y} \mapsto \v{y} - \v{z}$ into (e). \qed
\end{nproof}

Though we discuss the Euclidean norm here, it may also be of interest to consider/discuss other norms. One example is the $L_1$ norm (c.f. the norm discussed in Definition \ref{def:1.36}, which is the $L_2$ norm), which is the sum of the absolute values of each of the components. For $\v{x} = (x_1, x_2, \ldots, x_n)$ and $\v{y} = (y_1, y_2, \ldots, y_n)$ we have that:
\begin{align*}
    \abs{\v{x}}_1 = \abs{x_1} + \abs{x_2} + \ldots + \abs{x_n}, \quad \abs{\v{x} - \v{y}}_{1} = \abs{x_1 - y_1} + \abs{x_2 - y_2} + \ldots + \abs{x_n - y_n}
\end{align*}
The $L_1$ norm is often called the ``Taxicab norm'' or the ``Manhattan norm'' as the way it quantifies distance is akin to walking in discrete NSEW chunks; much like a taxi running through a grid-like New York City!
\begin{figure}[htbp]
    \centering
    \begin{tikzpicture}
    \draw[black, thick] (-3, 0) node[anchor=north] {$(0, 0)$} -- (1, 0) node[anchor=north] {$(x_1, 0)$}  -- (1, 2) node[anchor=south] {$\v{x} = (x_1, x_2)$} -- (-3, 0);
    \draw[] (-1, 0) node[anchor=north] {$\abs{x_1}$};
    \draw[] (1, 1) node[anchor=west] {$\abs{x_2}$};
    \draw[] (-2, 1.2) node[anchor=south]  {$\abs{\v{x}}_2 = \sqrt{x_1^2 + x_2^2}$};
    \draw[] (3.5, 1) node {$\abs{\v{x}}_{1} = \abs{x_1} + \abs{x_2}$};
    \end{tikzpicture}
    \caption{Visual comparison of the $L_1$ and $L_2$ norms in $\RR^2$.}
    \label{fig4}
\end{figure}

\noindent We are free to generalize this notion to the $L_n$ norm, and we may also define the $L_{\infty}$ norm, which for $\v{x} \in \RR^n$ is defined as:
\begin{align*}
    \abs{\v{x}}_\infty = \max_i \abs{x_i}
\end{align*}
In general for any $\v{x} \in \RR^n$, we have that $\abs{x}_1 \geq \abs{x}_2 \geq \abs{x}_3 \geq \ldots \geq \abs{x}_\infty$. We note that we that we can generalize these norms to the cases where we have infinite components:
\begin{align*}
    \norm{\v{x}}_p = \left(\sum_{i = 1}^\infty \abs{x_i}^p\right)^{1/p} < \infty \quad
    \|f\|_{p} \equiv\left(\int_{S}|f|^{p} \mathrm{~d} \mu\right)^{1 / p}<\infty
\end{align*}
Which allow us to define norms for function spaces. However, a detailed discussion of these are beyond the scope of this course (to be covered in a later course in functional analysis!) Moreover, we haven't even defined what an infinite sum or integral are yet, which we will get to in later chapters.


\section{Group actions on topological spaces}
  Let us start with reviewing some topology.

  \subsection{Topological prelimaniraries}
  A topological spaces $\calx$ is a set with a collection of open sets.
  The only facts we need from topology are the following.
  \begin{enumerate}
    \item For every point $x \in \calx$, there is an open set (called a \emph{neighborhood} of $x$) $U$ containing $x$.
    \item For every continuous function $f: \calx \rightarrow \caly$ the inverse $f^{-1}(U)$ of an open set $U$ is open.
  \end{enumerate}

  The examples of importance to us come from graphs.
  From a graph we can form a topological space by taking a point for each vertex $v$ and gluing a segment $[0,1]$ appropriately for each edge.

  \begin{ex}[Examples of graphs]
    \begin{enumerate}
      \item $S^1$
      \item $[0,n]$
      \item
      \item
    \end{enumerate}
  \end{ex}

  A \emph{space over $\calx$} is a space $\caly$ with a continuous map $\pi_\caly: \caly \rightarrow \calx$.
  Denote by $\cat{Spaces}_\calx$ the collection of all spaces over $\calx$.
  A map lying over $\calx$ between $\caly, \calz \in \cat{Spaces}_\calx$ is a map $f: \caly \rightarrow \calz$ such that the following diagram commutes
  \begin{equation*}
    \begin{tikzcd}
      \caly \arrow[rr,"f"] \arrow[rd,"\pi_\caly"']
      &&
      \calz \arrow[ld,"\pi_\calz"]\\
      & \calx
    \end{tikzcd}.
  \end{equation*}
  For $\caly \in \cat{Spaces}_\calx$, the \emph{fiber} over a vertex $v \in \calx_0$ is the set $\pi_\caly^{-1}(v)$.

  We now come to our first non-trivial definition.
  \begin{definition}
    $\caly \in \cat{Spaces}_\calx$ is called a \emph{cover} (or a \emph{covering space}) of $\calx$ if every point $y \in \caly$ has an open neighborhood $U$ satisfying
    \begin{enumerate}
      \item $\pi_\caly^{-1}(U)$ is homeomorphic to a disjoint union of spaces $\bigsqcup_{i \in I} U_i $,
      \item $\pi_\caly$ restricted to each $U_i$, for $i \in I$, is a homeomorphism onto $U$.
    \end{enumerate}
    A cover $\caly$ is a \emph{finite} cover if the set $I$ is finite for each such $U$.
  \end{definition}

    \begin{figure}[H]
    \centering
      \includegraphics[width=5cm]{example-image}
      % \includegraphics[width=]{}
      \caption{pancake picture of covering space}
    \end{figure}

  \subsection{Group actions}
  Whenever a group $G$ acts on a set $X$ we can form the set of orbits $\calG \backslash X$, and there is a natural projection map
  \begin{align*}
    \pi: X \rightarrow G \backslash X.
  \end{align*}
  We'll use this to produce covering  maps.

  A left group action of $G$ on a space $\calx$, denoted $G \groupaction \calx$, is a collection of continuous maps
  \begin{align*}
    g \cdot - : X &\longrightarrow X\\
    x &\longmapsto gx
  \end{align*}
  satisfying $ex = x$, where $e$ is the identity element in $G$ and $g(hx) = (gh)x$ for all $g,h \in G$.

  We say that the action of $G$ is \emph{properly discontinous} if every $x \in \calx$ has an open neighborhood $U_x$ such that
  \begin{align*}
    gU_x \cap hU_x = \varnothing
  \end{align*}
  for all $g \neq h \in G$.
  We'll call such a neighborhood \emph{good}. Note that if $U_x$ is a good neighborhood of $x$ and $V \subseteq U_x$ then $V$ is also good.

  If $G \groupaction \calx$ is properly discontinous, then $\calG \backslash X$ is a topological space whose underlying set is the set of orbits $\calG \backslash X$ and the neighborhood of each point $x$ is the image of a good neighborhood $U_x$ under the quotienting map

  The covering maps which can be obtained in this manner are called \emph{Galois coverings} i.e. a covering $\caly \rightarrow \calx$ is called a \emph{Galois cover} if there is a group $G$ such that $\calx \cong \caly / G$.
  The \emph{Galois group}, of a cover $p:Y \rightarrow X$ is the group of self-maps $q: Y \rightarrow Y$ lying over $X$
  \begin{align*}
    \Gal(\caly|\calx) = \{ q: Y \rightarrow Y : p = q \circ p \}.
  \end{align*}

  The group $\Gal(\caly|\calx)$ naturally acts on the space $\caly$, and it fixes the fiber over each $v \in \calx_0 $.
  Because of this $\Gal(\caly|\calx)$ is called the group of \emph{deck transformations}, as one can think of the action of $\Gal(\caly|\calx)$ on the fibers over $x$ as shuffling a deck of cards.

	Thus there is a homomorphism $\Gal(\caly|\calx) \rightarrow \Gal(\pi^{-1}(v))$, where $\Gal(\pi^{-1}(v))$ is the permutation group of the set $\pi^{-1}(v)$. Turns out, this homomorphism is injective, in fact, a much stronger statement is true.

	\begin{theorem}
		If $g \in \Gal(\caly|\calx)$ fixes a point $y \in Y$ then $g = \id_{\caly}$.
	\end{theorem}

	\begin{theorem}
		If $G$ acts on a graph $\caly$ and $\calx = \caly / G $ then $\Gal(\caly|\calx) \cong G$.
	\end{theorem}
	Note that this determines $G$ completely in terms of the spaces $X$ and $Y$.
	\begin{proof}
		There is a natural map $G \rightarrow \Gal(\caly|\calx)$.
	\end{proof}

  \begin{theorem}
    A finite cover $p:Y \rightarrow X$ is Galois if and only if the Galois group $\Gal(\caly|\calx)$ acts transitively on the fibers $\pi^{-1}(x)$ for any $x \in X$.
  \end{theorem}

  \begin{corollary}
    A finite cover $p: Y \rightarrow X$ is Galois if and only if $\abs{\Gal(\caly|\calx)} = \abs{{p}^{-1}(x)}$.
  \end{corollary}

  For a cover $p:Y \rightarrow X$, we say that a cover $q:Z \rightarrow X$ lies between $X$ and $Y$ if the map $p$ factors as
  \begin{equation*}
    \begin{tikzcd}
      Y \ar[rd, "p"'] \ar[r,"p'"]& Z \ar[d, "q"]
      \\
      &X
    \end{tikzcd}
  \end{equation*}

  \begin{proposition}
    With the notation as above, $p':Y \rightarrow Z$ is also a covering map. Further if $p$ is Galois then so is $p'$.
  \end{proposition}


% tex file for clustering
\par The heart of our analysis consists of different implementations of
procedures for multiple comparisons between the 24 subjects. More
specifically, we focused on an implementation of the Benjamini-Hochberg
procedure, the grouping of t-statistics, and the grouping of $\hat{\beta}$
values. For each of the three cases, we found appropriate parameters to be
used for all subjects by observing the images from a few subjects at different
points of input. The parameters that we played around with include the false
discovery rate, the cutoff value for significant t-values and $\hat{\beta}$-
values, and the number of neighbors to use when smoothing data. We also
considered hierarchical clustering. For hierarchical clustering using the
ward metric, we compared different parameter selections for the number of
clusters.

\subsubsection{Benjamini-Hochberg Correction}

\par When conducting multiple comparisons, it is important to have an
idea of the quantity of Type I errors that may be prevalent in the analysis.
In our analysis of the voxel time course data, we decided that limiting and
controlling the number of Type I errors is important to the process of
identifying active regions. The processes of limiting the number of Type I
errors are called FDR-controlling procedures. In the grand scheme of things,
FDR-controlling procedures give greater statistical power at the cost of
producing more Type I errors.

\par Once we coded the hypothesis function that will return t-test
values and ``p-values'', we implemented the Benjamini-Hochberg procedure to
control the proportion of rejected null hypotheses in the data. The Benjamini-
Hochberg procedure works by multiplying each of the ``p-values'' to a ratio of
the number of tests times the rank of the ``p-value'' in the ordered set,
and the chosen false discovery rate --- from these adjusted ``p-values'', only
the values that are less than the chosen false discovery rate will be chosen
to be returned. This way, we are able to adjust the proportion of null
hypotheses that will be rejected and the proportion that will return the
desired proportion of significant tests. This will reduce the
number of false positives returned in the data and extend greater statistical
power in later analysis performed on the voxel dataset.

\par The Benjamini-Hochberg procedure was not the only analysis we performed
over multiple subjects. After completing grouping of the t-statistics and
$\hat{\beta}$ values. we noticed that the variability between the subjects
of the study is different between the Benjamini-Hochberg procedure outputs,
but that the t-statistics and $\hat{\beta}$ value grouping procedure
outputs. Moreover, when performing the Benjamini-Hochberg procedure, along with
the t-statistics grouping procedures, we did play around with smoothing over
neighboring voxels. Ultimately, however, we decided to forgo the neighbor
smoothing to prevent loss of information in the plots that we generated in the
end, and neighbor smoothing didn't visually appear very needed.

\subsubsection{t-Statistics Grouping}

\par We use the t-statistics that we found earlier for another type of cutoff
analysis. The t-statistics measure the size of the difference relative to the
variation in the data. Thus, a small ``p-value'' corresponds directly to large
absolute values of t-statistics. In fact, the ``p-value'' and the t-statistics
are related by the following statement: More extreme t-statistics will return
lower ``p-values'', which increases the chances of the null hypothesis being
indicated as false. Although the t-statistics go hand in hand with the
Benjamini-Hochberg analysis on the ``p-values'', we ultimately decided that an
additional process of selecting t-statistics based on a threshold was
necessary because the ``p-values'' assume a normal distribution while the
t-statistics do not assume this from the data. It should be noted that the
t-statistics grouping analysis performed as well, if not better, than the
``p-value'' analysis using the Benjamini-Hochberg procedure.

\par Since the magnitude from zero represents how significant the test is for
t-statistics, we collected the absolute value t-statistics that were above a
certain threshold. This was the opposite of the procedure for finding the
significant tests in the Benjamini-Hochberg process. The threshold, also called
the cutoff in the scripts that were written when developing this process, is
calculated in a very similar fashion to the Benjamini-Hochberg false discovery
rate. This is because the t-values and the ``p-values'' are strongly linked, in
fact there is a 1-to-1 relation between the absolute value of t - values to
p-values.

\par Implementing the t-value grouping function was almost trivial because it
is the flip side of the Benjamini-Hochberg function that was implemented
previously. However, since we do not know if there's a function that relates
the absolute values of t-statistics to ``p-values'', there was a lot of
experimenting with different values to see which values of cutoff points versus
false discovery rates (the Q value in the Benjamini-Hochberg function) would
deliver similarly focused results.

\subsubsection{$\hat{\beta}$ Grouping}

\par Using the same function that we created to group a certain subset of the
t-statistics, we also looked over the $\hat{\beta}$ values as a part of our
analysis. Much of what we know in statistics tells us that observing
$\hat{\beta}$ values on top of looking at t-statistics would not tell us much
since the two variables are not explicitly connected; however, the main reason
why we decided it was of utmost importance to look at both the t-statistics
and the $\hat{\beta}$ values is that there might be a relationship between the
trends in each respective variables' output to the cutoff point, that is lost
when going from $\hat{beta}$ to t-statistic (dividing by variance).

\par Granted, it is pretty clear that the t-statistics grouping is very
theoretically similar to the Benjamini-Hochberg procedure. This is why the
$\hat{\beta}$ grouping portion of the analysis was added on. Since $\hat{\beta}$
values do not directly use normality assumptions, we thought it would offer
either a different angle on how to approach analysis, or confirm the results
found in the analysis that depended on the assumptions of normality.

\subsubsection{Hierarchical Clustering}

\par Using the across-subject average t-statistics for every voxel in the
brain, we are left with a 3-d array of t-statistics that contain both negative
and positive values. Instead of manually observing patterns in these images, we
instead implemented a clustering algorithm to split the entire 3-d images into
clusters based on the voxels' relative location to each other as well as the
values of their t-statistics.

\par In order to find a proper clustering algorithm, we decided to treat this
problem like a grayscale image segmentation problem and implemented a
agglomerative hierarchical cluster using Ward's method. Agglomerative means
that the clusters are built bottom up with each observation starting as its
own cluster and pairs being moved up the hierarchy. Ward's method creates
clusters based on a minimum variance criterion that minimizes the total
within-cluster variances. An example of this implementation for a 2-d image is
seen here:
\url{http://scikit-learn.org/stable/auto_examples/cluster/plot_lena_ward_segmen
tation.html}.

In our implementation, we defined a structure to our data using a connectivity
graph in order to ensure that each cluster is spatially constrained. Also,
since our scenario uses a 3-d image, the connectivity graph will also have to
take into account this extra dimension.

\par Ultimately, we did not use this clustering algorithm in our main paper
because it did not add any useful information on top of our other clustering
methods we used. See figure \ref{fig:cluster_comparison} to see similarities.
Furthermore, we faced a few
problems when trying to implement the algorithm. Most significantly, creating
the 3-d connectivity graph was a problem, and we were not able to properly
implement this in z-direction. Furthermore, runtime was an issue when compared
to our other options. Ultimately, the idea around using a hierarchical
clustering method was intriguing; however, the same results could be achieved
using simpler and faster methods.

\subsubsection{Comparison of Clustering Techniques}
Generally the clustering techniques for the multiple comparison and the
hierarchical clustering produces similar results, which can be seen if a
side by side comparison of the hierarchical clustering method using Ward
against the quantile-based clustering algorithm for t-statistics (we've
included the actual t-values as well in the center for comparisons)
[Figure \ref{fig:cluster_comparison}]. Depending upon the subject, the clustering
from Benjamini Hochberg, t grouping and $\hat{\beta}$ grouping can be
very different or very similar.

\begin{figure}[ht]

	\centering
	\includegraphics[width=.8\linewidth]{../images/cluster_comparison.png}
	\caption{Clustering Comparison between WARD, t-statistics, and t-grouping}
	\label{fig:cluster_comparison}

\end{figure}


\section{Background and objectives of the thesis}

\section{Notations}
\begin{flushleft}
\begin{tabular}{l l}

$t \in \mathbb{Q}$ & Discrete or continuos time\\
$D \in \mathbb{N}$ & Space dimension\\
$P \in \mathbb{N}$ & Number of variables: univariate, multivariate\\
$S \in \mathbb{N}$ & Number of sensors\\
$x_t \in \mathbb{R}^{D\times P}$ & Matrix representing state $x$ at time $t$\\
$z_t \in \mathbb{R}^S$  & Vector of measurement $z$ at time $t$\\
$u_t \in \mathbb{R}$ & Action $u$ at time $t$\\
$\hat z: \mathbb{R}^{D\times P}  \rightarrow \mathbb{R}^S $ & Function that predicts sensor measurements given a state\\
$\hat x_t \in \mathbb{R}^{D\times P}$ & Matrix representing predicted state $\hat x$ at time $t$\\
$\hat x_t^* \in \mathbb{R}^{D\times P}$ & Matrix representing best predicted state $\hat x$ at time $t$\\
$\theta_t \in \mathbb{R}$ & Angle of orientation at time $t$\\
$bel(x_t)$ & Belief of state $x$ at time $t$\\
$\overline{bel}(x_t)$ & Prediction belief of state $x$ at time $t$\\
$p(x)$ & Probability of continuous or discrete random variable $x$\\
$p(x | y)$ & Conditional probability of $x$ given $y$\\
$x_{1:t}$ & Sequence containing $\{x_1, x_2, ..., x_t\}$\\
$M$ & Number of Particles used in the Particles Filter\\
$x_t^{[m]}$ & State $x$ at time $t$ of particle $m$\\
$a \approx b$ & $a$ is approximately equal to $b$\\
$a \sim b$ & $a$ is similar to $b$\\
$a \propto b$ & $a$ is proportional to $b$\\
\end{tabular}
\end{flushleft}

\section{Abbreviations}
\begin{flushleft}
\begin{tabular}{l l}
EPFL & Swiss Federal Institute of Technology in Lausanne\\
ROS & Robot Operative System\\
3D & Three-Dimensional\\
DOF & Degrees Of Freedom\\
pdf & Probability Density Function\\
KF & Kalman Filter\\
PF & Particles Filter\\
EKF & Extended Kalman Filter\\
MCKF & Maximum Correntropy Kalman Filter\\
MMSE & Minimum Mean Square Error\\
MCC & Maximum Correntropy Criterion\\
IEKF & Invariant Extended Kalman Filter\\
PCC & Pearson Correlation Coefficient\\
PPF & Pearson Particles Filter\\
GPU & Grafical Processing Unit\\
GPGPU & General Purpose Graphical Processing Unit\\
MCL & Monte Carlo localization\\
SLAM & Simultaneous Localization And Mapping Problem\\
DP & Differentiable Programming\\
DPFs & Differentiable Particle Filters\\
PF-net & Particle Filter Network\\
HF & Histogram Filter\\
DMN & Differentiable Mapping Network\\
DPFRL & Discriminative Particle Filter Reinforcement Learning\\
DVRL & Deep Variational Reinforcement Learning \\
GRU & Gated Recurrent Unit \\
RNN & Recurrent Neural Networks\\
PF-RNN & Particles Filter Recurrent Neural Networks\\

\end{tabular}
\end{flushleft}


\documentclass[openany,11pt]{homework}

\coursename{ELEN 4903 Machine Learning (Spring 2018)} % DON'T CHANGE THIS

\studname{Pratyus Pati}    % YOUR NAME GOES HERE
\studmail{pp2636@columbia.edu}% YOUR UNI GOES HERE
\hwNo{1}                   % THE HOMEWORK NUMBER GOES HERE

% Uncomment the next line if you want to use \includegraphics.
\usepackage{graphicx}

\begin{document}
\maketitle

\section*{Problem 1(a)}

Given:
\begin{equation}
x_i \in \{0, 1\}
\end{equation}
Every random variable is taken from a Bernoulli's distribution
\begin{equation}
p(x_i\mid\pi) = \pi^{x_i}(1-\pi)^{1-x_i}, \pi \in [0, 1]
\end{equation}
To find:\\
Joint distribution of the data:
$$
p(x_1, x_2, ..., x_n \mid \pi)
$$
Since each random variable is I.I.D,
\begin{align}
	p(x_1, x_2, ..., x_n \mid \pi) & = \prod_{i = 1}^{n} p(x_i \mid \pi) \\
								   & = \prod_{i = 1}^{n} \pi^{x_i}(1-\pi)^{1-x_i} \\
								   & = \pi^{\sum_{i=1}^{n} x_i}(1-\pi)^{(n-\sum_{i=1}^{n} x_i)}
\end{align}
Let $\mu = (\sum_{i=1}^{n} x_i)/ n$ In which case, \\
$$
p(x_1, x_2, ..., x_n \mid \pi) = \pi^{n\mu}(1-\pi)^{n-n\mu}
$$

\section*{Problem 1(b)}

\begin{align}
\hat{\pi}_{ML} & = \operatornamewithlimits{arg\,max}_{\pi} p(x_1, x_2, ..., x_n \mid \pi) \\
			   & = \operatornamewithlimits{arg\,max}_{\pi} \pi^{n\mu}(1-\pi)^{n-n\mu} \\
			   & = \operatornamewithlimits{arg\,max}_{\pi} \ln(\pi^{n\mu}(1-\pi)^{n-n\mu}) \\
			   & = \operatornamewithlimits{arg\,max}_{\pi} [(n\mu) (\ln \pi)] + [(n - n\mu)(\ln (1-\pi))]
\end{align}

On taking the derivative w.r.t. $\pi$\\
\begin{align}
\frac{\partial }{\partial \pi} [(n\mu) (\ln \pi)] + [(n - n\mu)(\ln (1-\pi))]
& = \left[n\mu \frac{\partial \ln \pi}{\partial \pi}\right] + \left[(n - n\mu)\left(\frac{\partial \ln(1-\pi)}{\partial \pi}\right)\right] \\
& = \frac{n\mu}{\pi} -\frac{n-n\mu}{1-\pi}
\end{align}

On setting the partial derivative w.r.t $\pi$ as 0 to get $\hat{\pi}_{ML}$,
\begin{align}
\frac{n\mu}{\hat{\pi}_{ML}} -\frac{n-n\mu}{1-\hat{\pi}_{ML}} & = 0 \\
\Rightarrow n\mu(1-\hat{\pi}_{ML}) - (n-n\mu)(\hat{\pi}_{ML}) & = 0 \\
\Rightarrow n\mu - n\mu\hat{\pi}_{ML} - n\hat{\pi}_{ML} + n\mu\hat{\pi}_{ML} & = 0 \\
\Rightarrow n\mu - n\hat{\pi}_{ML} & = 0 \\
\Rightarrow \hat{\pi}_{ML} & = \mu = \frac{\sum_{i=1}^{n} x_i}{n}
\end{align}

\section*{Problem 1(c)}

\[
p(\pi) = beta(a, b) = \frac{\pi^{a-1}(1-\pi)^{b-1}}{B(a, b)}
\]

\begin{align}
p(\pi \mid x_1, x_2, ..., x_n) & = \frac{p(x_1, x_2, ..., x_n \mid \pi)p(\pi)}{p(x_1, x_2, ..., x_n)} \\
							   & = \frac{\pi^{n\mu}(1-\pi)^{n-n\mu}\pi^{a-1}(1-\pi)^{b-1}}{B(a, b)p(x_1, x_2, ..., x_n)} \\
							   & = \frac{\pi^{n\mu+a-1}(1-\pi)^{n-n\mu+b-1}}{B(a, b)p(x_1, x_2, ..., x_n)}
\end{align}

\begin{align}
\hat{\pi}_{MAP} & = \operatornamewithlimits{arg\,max}_{\pi} p(\pi \mid x_1, x_2, ..., x_n) \\
& = \operatornamewithlimits{arg\,max}_{\pi} \frac{\pi^{n\mu+a-1}(1-\pi)^{n-n\mu+b-1}}{B(a, b)p(x_1, x_2, ..., x_n)} \\
& = \operatornamewithlimits{arg\,max}_{\pi} \left[\pi^{n\mu+a-1}(1-\pi)^{n-n\mu+b-1}\right] \\
& = \operatornamewithlimits{arg\,max}_{\pi} \left[\ln(\pi^{n\mu+a-1}(1-\pi)^{n-n\mu+b-1})\right]	\\
& = \operatornamewithlimits{arg\,max}_{\pi} \left[(n\mu+a-1)\ln\pi + (n-n\mu+b-1)\ln(1-\pi)\right] \\
\end{align}

On taking the derivative w.r.t. $\pi$
\begin{align}
& \frac{\partial }{\partial \pi} \left[(n\mu+a-1)\ln\pi + (n-n\mu+b-1)\ln(1-\pi)\right] \\
= & \left[(n\mu+a-1) \frac{\partial \ln \pi}{\partial \pi}\right] + \left[(n-n\mu+b-1)\left(\frac{\partial \ln(1-\pi)}{\partial \pi}\right)\right] \\
& = \frac{n\mu+a-1}{\pi} -\frac{n-n\mu+b-1}{1-\pi}
\end{align}

On setting the partial derivative w.r.t $\pi$ as 0 to get $\hat{\pi}_{MAP}$,
\begin{align}
\frac{n\mu+a-1}{\hat{\pi}_{MAP}} -\frac{n-n\mu+b-1}{1-\hat{\pi}_{MAP}} & = 0 \\
\Rightarrow (n\mu+a-1)(1-\hat{\pi}_{MAP}) - (n-n\mu+b-1)(\hat{\pi}_{MAP}) & = 0 \\
\Rightarrow (n\mu + a - 1) - (n\mu + a - 1 + n - n\mu + b - 1)\hat{\pi}_{MAP} & = 0 \\
\Rightarrow (n\mu + a - 1) - (n + a + b - 2)\hat{\pi}_{MAP} & = 0 \\
\Rightarrow \hat{\pi}_{MAP} & = \frac{n\mu + a - 1}{n + a + b - 2} \\
							& = \frac{n\sum_{i=1}^{n} x_i + a - 1}{n + a + b - 2}
\end{align}

\section*{Problem 1(d)}

\begin{align}
p(\pi \mid x_1, x_2, ..., x_n) & = \frac{p(x_1, x_2, ..., x_n \mid \pi)p(\pi)}{p(x_1, x_2, ..., x_n)} \\
& = \frac{\pi^{n\mu}(1-\pi)^{n-n\mu}\pi^{a-1}(1-\pi)^{b-1}}{B(a, b)p(x_1, x_2, ..., x_n)} \\
& = \frac{\pi^{n\mu+a-1}(1-\pi)^{n-n\mu+b-1}}{B(a, b)p(x_1, x_2, ..., x_n)} \\
& \propto \pi^{n\mu+a-1}(1-\pi)^{n-n\mu+b-1}				\\
\end{align}

Therefore, the posterior probability is proportional to the $beta$ distribution. We can convert the proportionality to an equality by adding the correct co-efficient to form the $beta$ distribution

\begin{align}
p(\pi \mid x_1, x_2, ..., x_n) & \propto \pi^{n\mu+a-1}(1-\pi)^{n-n\mu+b-1}\\
& = \frac{\pi^{n\mu+a-1}(1-\pi)^{n-n\mu+b-1}}{B(a + n\mu, b + n-n\mu)} \\
& = beta(a + n\mu, b + n-n\mu)
\end{align}

\section*{Problem 1(e)}
The mean of the distribution $beta(a, b)$ is given by $\frac{a}{a+b}$ \\
Therefore, \\
\begin{align}
\mathbb{E}[\pi_{post}] & = \frac{a + n\mu}{b + n - n\mu + a + n\mu} \\
				& = \frac{a + n\mu}{a + b + n} \\
				& = \frac{a}{a+b+n} + \mu\frac{n}{a+b+n} \\
				& = \frac{a}{a+b}\frac{a+b}{a+b+n} + \mu\frac{n}{a+b+n} \\
\end{align}
This can be represented as:
\begin{align}
\mathbb{E}[\pi_{post}] & = \alpha\mathbb{E}[\pi_{prior}] + (1-\alpha)\hat{\pi}_{ML}
\end{align}
This proves that $\mathbb{E}[\pi_{post}]$ is a weighted average of $\mathbb{E}[\pi_{prior}]$ and $\hat{\pi}_{ML}$,where $\alpha = \frac{a+b}{a+b+n}$ that tends to 0 as n increases. Therefore, we can say:
\begin{align}
\lim_{n\to\infty}\mathbb{E}[\pi_{post}] & = \hat{\pi}_{ML}
\end{align}

Also, for relating $\mathbb{E}[\pi_{post}]$ and $\hat{\pi}_{MAP}$,
\begin{align}
	\mathbb{E}[\pi_{post}] & = \frac{a + n\mu}{a+b+n} \\
												 & = \frac{a + n\mu -1+1}{a+b+n} \\
												 & = \frac{a+n\mu-1}{a+b+n} + \frac{1}{a+b+n} \\
												 & = \frac{a+n\mu-1}{a+b+n-2}\frac{a+b+n-2}{a+b+n} + \frac{1}{2}\left(1-\frac{a+b+n-2}{a+b+n}\right)
\end{align}

This can be represented as:
\begin{align}
\mathbb{E}[\pi_{post}] & = \alpha\hat{\pi}_{MAP} + (1-\alpha)(0.5)
\end{align}

This proves that $\mathbb{E}[\pi_{post}]$ lies between $\hat{\pi}_{MAP}$ and $0.5$, and approaches $\hat{\pi}_{MAP}$ as $n \to \infty$.

The variance of the distribution $beta(a, b)$ is given by $\frac{ab}{(a+b)^2(a+b+1)}$ \\
Therefore, \\
\begin{align}
var[\pi_{post}] & = \frac{(a + n\mu)(b + n - n\mu)}{(b + n - n\mu + a + n\mu)^2(b + n - n\mu + a + n\mu +1)} \\
& = \frac{(a + n\mu)(b + n - n\mu)}{(a + b + n)^2(a + b + n+1)}
\end{align}
For relating it with the mean, the variance can be further represented as:
\begin{align}
	var[\pi_{post}] & = \left(\frac{1}{a+b+n+1}\right)\left(\frac{a+n\mu}{a+b+n}\right)\left(1-\frac{a+n\mu}{a+b+n}\right) \\
	& = \left(\frac{1}{a+b+n+1}\right)\left(\mathbb{E}[\pi_{post}]\right)\left(1-\mathbb{E}[\pi_{post}]\right)
\end{align}
From this relation, we can see that $var[\hat{\pi}_{MAP}]$ is a quadratic funtion of $\mathbb{E}[\pi_{post}]$ and exists for the range $\mathbb{E}[\pi_{post}] \in [0, 1]$, 0 at $\mathbb{E}[\pi_{post}] \in \{0, 1\}$ and achieves its maximum at $\mathbb{E}[\pi_{post}] = 0.5$
\\
\\
From the above results, we can see how $var[\pi_{post}]$ varies with respect $\mathbb{E}[\pi_{post}]$ due to changes in $\hat{\pi}_{ML}$ and $\hat{\pi}_{MAP}$. Also, as $n \to \infty$, $\mathbb{E}[\pi_{post}]$, $\hat{\pi}_{MAP}$ and $\hat{\pi}_{ML}$ converge.

\section*{Problem 2(a)}
\begin{figure}[h]
	\includegraphics[width=\textwidth]{wrr_vs_dfl}
	\caption{Ridge Regression Weights vs. Degrees of Freedom $df(\lambda)$}
\end{figure}

\section*{Problem 2(b)}
From the figure plotting the relationship between $w_i$ and $df(\lambda)$, it can be seen that there are two coefficients $w_3$ and $w_5$ that exhibit large magnitudes for normal least square condition. On decreasing the degrees of freedom ($df(\lambda)$) by increasing $\lambda$, these coefficients gradually converge towards 0.
\\
\\
These coefficients can either be uncorrelated values whose magnitudes signify their respective importance in calculating the predicted value. $w_5$ is positively correlated with the value of $\hat{y}$, and therefore, $\hat{y}$ increases as the $6^{th}$ feature increases. $w_3$ is negatively correlated with $\hat{y}$, and therefore $\hat{y}$ decreases as $4^{th}$ feature decreases.
\\
\\
Alternatively, $4^{th}$ and $6^{th}$ features are correlated and as a result, $w_{LS}$ is poorely determined, with high variance. Ridge regression helps in decreasing the variance of such weights by restraining it around 0.

\section*{Problem 2(c)}
\begin{figure}[h]
	\includegraphics[width=\textwidth]{MSE_vs_lambda}
	\caption{Mean Square Error vs. $\lambda$}
\end{figure}

From the figure, it can be seen that as $\lambda$ increases, the Mean Square Error increases accordingly. In this case, trying to decrease the variance by increasing $\lambda$, introduces too much bias into the model. In such a case, we would like to keep $\lambda$ as low as possible to reduce the bias.

\section*{Problem 2(d)}
\begin{figure}[h]
	\includegraphics[width=\textwidth]{RMSE_vs_lambda_p}
	\caption{Root Mean Square Error vs. $\lambda$ for differernt values of $p$}
\end{figure}

We can see that as we add more $p^{th}$ order terms, the RMSE decrease. Also for p = 2 and 3, there is a clear local minima of the RMSE with respect to the $\lambda$. Hence, we can set $\lambda$ to the minumum value, and use the Occam's Razor heuristic to select p as 2 to create the simplest possible model that decreases the RMSE.

\end{document}


\section{Number Theory}
\begin{code}{Extended Euclid}{$\mathcal{O}(\log(a + b))$}{math/extendedEuclid.cc}
  Calculates $a$ and $b$ such that $\gcd(x, y) = ax + by$
\end{code}

\lst{Modular Exponentiation}{$\mathcal{O}(\log b)$}{math/modpow.cc}

\begin{code}{Eulers Totoid Function}{$\mathcal{O}(\sqrt{n})$}{math/totoid.cc}
%TODO(Alex): Improve this -> better(more) details/ what is important, applications
$\phi(n)$: Number of Integers in $[1, n]$ that are coprime to $n$.\\
$p \text{ prime}, k \in \mathbb{N} \Rightarrow \phi(p^k) = p^k - p^{k-1}$ \\
$a, b \text{ coprime} \Rightarrow \phi(a \cdot b) = \phi(a) \cdot \phi(b)$ \\
$a, b \in \mathbb{N} \Rightarrow \phi(a \cdot b) = \phi(a) \cdot \phi(b) \cdot \frac{gcd(a, b)}{\phi(gcd(a,b))}$ \\
$P \text{ prime factors of } n \Rightarrow \phi(n) = n \cdot \prod_{p \in P} (1 - \frac{1}{p})$
\end{code}