\documentclass[journal,hidelinks]{IEEEtran}
\usepackage[utf8]{inputenc}
\usepackage[
  pdftitle={Assignment \#1},
  pdfauthor={Andrei Purcarus},
  pdfsubject={ECSE-543 -- Numerical Methods in EE}
]{hyperref}
\usepackage{graphicx}
\usepackage[all]{hypcap}
\usepackage{cleveref}
\usepackage{indentfirst}
\usepackage[per-mode=symbol]{siunitx}
\usepackage{listings}
\lstset{showstringspaces=false}
%\usepackage[title]{appendix}

\title{ECSE-543 \\ Numerical Methods in EE \\ Assignment \#1}
\author{Andrei~Purcarus,~260631911,~\IEEEmembership{McGill~University}}

\begin{document}
\sloppy

\maketitle

% \begin{abstract}

% \end{abstract}

% \section{Introduction}

\section*{Code Listings and Unit Testing}

The source code used for this assignment is listed in the appendices. In order to save space, we did not include the unit tests. For the full code, see the \href{https://github.com/Gripnook/ECSE543-F17-A1}{GitHub repository}.

\Cref{sec:main} contains the main function. \Cref{sec:matrix,sec:matrix-util} define a matrix library and helper functions. \Cref{sec:cholesky} defines functions that perform Cholesky decomposition using banded and non-banded methods. \Cref{sec:solver} defines a generic solver for systems of equations that have a positive-definite coefficient matrix. \Cref{sec:mesh-h,sec:mesh-cpp} define a circuit description file generator for an $N * 2N$ resistor mesh. \Cref{sec:circuit-solver-h,sec:circuit-solver-cpp} define functions that solve a circuit as given by a circuit description file. Finally, \Cref{sec:finite-differences-h,sec:finite-differences-cpp} define finite difference problem generators and solvers with iterative methods.

Whenever we had to test some functionality, we used unit tests that we will reference in the text. These tests all pass, as shown in \Cref{fig:test-output}.

\begin{figure}[!htb]
  \centering
  \includegraphics[width=\columnwidth]{test-output.png}
  \caption{Output after running the unit test suite.}
  \label{fig:test-output}
\end{figure}

\section*{Question 1}

We first wrote and tested a program that solves the matrix equation $A x = b$ using Cholesky decomposition. The code listings are shown in \Cref{sec:matrix,sec:matrix-util,sec:cholesky,sec:solver}. To test the solver, we generated several non-singular lower triangular matrices $L$ with positive entries on the main diagonal, then took $A = L L^T$, which guarantees that $A$ has a Cholesky decomposition and hence is positive-definite. For each such $n * n$ matrix, we invented an $n * 1$ vector $x$, multiplied it by $A$ to get an $n * 1$ vector $b$, then tested the solver with $A$ and $b$ and compared the result to $x$. The section of the unit tests which performs this function is shown in \Cref{sec:solver-test-cpp}. The matrices $L$ used for each of the $2 * 2$, $3 * 3$, $4 * 4$, and $5 * 5$ systems of equations, as well as the vectors $x$, should be clear from the unit test code.

Then, we wrote a program that reads a list of network branches from an input stream and solves for the node voltages. The code is shown in \Cref{sec:circuit-solver-h,sec:circuit-solver-cpp}. The network branches are expected to be lines of the form
\[ N_+ \  N_- \  J_k \  R_k \  E_k \]
where $N_+$ and $N_-$ are numeric node labels, $J_k$ is the short circuit current coming out of node $N_+$ and going into node $N_-$, $R_k$ is the equivalent resistance between nodes $N_+$ and $N_-$, and $E_k$ is the open circuit voltage between $N_+$ and $N_-$. This program can read from a file, as shown in \Cref{sec:main}, but to test it we used our unit testing framework to generate the input stream. An extract of the unit test is shown in \Cref{sec:circuit-solver-test-cpp}, which solves 5 circuits correctly. These circuits, along with their expected results, are shown in \Cref{fig:q1-circuit-1,fig:q1-circuit-2,fig:q1-circuit-3,fig:q1-circuit-4,fig:q1-circuit-5}.

\begin{figure}[!htb]
  \centering
  \includegraphics[width=\columnwidth,height=0.4\columnwidth,keepaspectratio]{question-1/circuit-1.pdf}
  \caption{Test circuit 1. $V_1 = \SI{5}{\volt}$.}
  \label{fig:q1-circuit-1}
\end{figure}

\begin{figure}[!htb]
  \centering
  \includegraphics[width=\columnwidth,height=0.35\columnwidth,keepaspectratio]{question-1/circuit-2.pdf}
  \caption{Test circuit 2. $V_1 = \SI{50}{\volt}$}
  \label{fig:q1-circuit-2}
\end{figure}

\begin{figure}[!htb]
  \centering
  \includegraphics[width=\columnwidth,height=0.4\columnwidth,keepaspectratio]{question-1/circuit-3.pdf}
  \caption{Test circuit 3. $V_1 = \SI{55}{\volt}$.}
  \label{fig:q1-circuit-3}
\end{figure}

\begin{figure}[!htb]
  \centering
  \includegraphics[width=\columnwidth,height=0.4\columnwidth,keepaspectratio]{question-1/circuit-4.pdf}
  \caption{Test circuit 4. $V_1 = \SI{20}{\volt}$, $V_2 = \SI{35}{\volt}$.}
  \label{fig:q1-circuit-4}
\end{figure}

\begin{figure}[!htb]
  \centering
  \includegraphics[width=\columnwidth,height=0.6\columnwidth,keepaspectratio]{question-1/circuit-5.pdf}
  \caption{Test circuit 5. $V_1 = \SI{5}{\volt}$, $V_2 = \SI{3.75}{\volt}$, $V_3 = \SI{3.75}{\volt}$.}
  \label{fig:q1-circuit-5}
\end{figure}

\section*{Question 2}

We then wrote a program that generates the network branches for an $N * 2N$ mesh of $\SI{1}{\ohm}$ resistors. We chose $\SI{1}{\ohm}$ instead of $\SI{1}{\kilo\ohm}$ since $\SI{1}{\ohm}$ is simpler to work with and the result will just scale linearly. The code that generates this network is given in \Cref{sec:mesh-h,sec:mesh-cpp}. Note that the program generates an extra branch, consisting of a $\SI{1}{\ampere}$ current source in parallel with another $\SI{1}{\ohm}$ resistor between the top right corner and the bottom left corner of the mesh. The equivalent resistance can then be found by using the node voltage at the top right corner, as $R_{eq} = V / (1 - V)$. The code that does this is shown in \Cref{sec:main}. The scaled results are shown in \Cref{tab:q2-resistance}.

\begin{table}[!htb]
  \centering
  \caption{$R$ vs. $N$ for the $N * 2N$ mesh of $\SI{1}{\kilo\ohm}$ resistors.}
  \label{tab:q2-resistance}
  \resizebox{0.3\columnwidth}{!}{\begin{tabular}{|l|l|}
    \hline
    $N$ & $R(\SI{}{\kilo\ohm})$ \\ \hline
    2 & 2.05742 \\ \hline
    3 & 2.49772 \\ \hline
    4 & 2.82749 \\ \hline
    5 & 3.09057 \\ \hline
    6 & 3.30919 \\ \hline
    7 & 3.49608 \\ \hline
    8 & 3.65925 \\ \hline
    9 & 3.80401 \\ \hline
    10 & 3.93407 \\ \hline
  \end{tabular}}
\end{table}

In theory, the time taken to solve the system of equations increases as $O(n^3)$, where $n$ is the size of the square matrix $A$ in the system $A x = b$. In this case, since there are $2N^2 + 3N$ free nodes, $n = O(N^2)$, and so the time taken should increase as $O(N^6)$. We measured the time it took for the program to solve the circuit for different values of $N$. However, since the initial matrix multiplication $A Y A^T$ is $O(N^6)$, we excluded the time needed to form the initial matrices from our measurements. The results are shown in \Cref{fig:q2-std-time}. We fitted a polynomial of the form $a * N ^ b$ to the data and found that it grows approximately as $O(N^{5.25})$. This agrees with the expectation. The reason that it is slightly lower is that for low values of $N$, lower order terms also contribute a significant amount. We would expect the effective exponent to increase to $6$ with higher values of $N$.

\begin{figure}[!htb]
  \centering
  \includegraphics[width=0.6\columnwidth]{question-2/standard_time.eps}
  \caption{Time vs. $N$ for solving the $N * 2N$ mesh of $\SI{1}{\kilo\ohm}$ resistors with the standard circuit solver.}
  \label{fig:q2-std-time}
\end{figure}

We then modified the program to compute and use the half-bandwidth of the matrix $A$ in solving for the Cholesky decomposition. These changes are shown in \Cref{sec:cholesky,sec:solver,sec:circuit-solver-h,sec:circuit-solver-cpp}. Since a node connects only to nodes a distance $N$ away using our numbering system, the half-bandwidth is $b = O(N)$. Hence, the time taken should increase as $O(n b^2)  = O(N^4)$. However, since the initial matrix multiplication $A Y A^T$ is $O(N^6)$, the effects will not be noticeable unless we exclude the time taken to form the initial matrices. To get a better measurement, we measured for $N$ up to $32$. The results are shown in \Cref{fig:q2-band-time}. We fitted a polynomial of the form $a * N ^ b$ to the data and found that it grows approximately as $O(N^4)$. This agrees with the expectation.

\begin{figure}[!htb]
  \centering
  \includegraphics[width=0.6\columnwidth]{question-2/banded_time.eps}
  \caption{Time vs. $N$ for solving the $N * 2N$ mesh of $\SI{1}{\kilo\ohm}$ resistors with the banded circuit solver.}
  \label{fig:q2-band-time}
\end{figure}

We then measured the values of $R$ for $N$ up to $32$ and plotted the results in \Cref{fig:q2-resistance}. After trying different functions to fit the data, we found that the resistance is best approximated by a function of the form $R(N) = a * log(N + b) + c$. The best such function is included in \Cref{fig:q2-resistance}. We can see that this function is an almost perfect fit over the entire range. Thus, we can conclude that, asymptotically, we have $R(N) = O(log(N))$.

\begin{figure}[!htb]
  \centering
  \includegraphics[width=0.6\columnwidth]{question-2/resistance.eps}
  \caption{$R$ vs. $N$ for the $N * 2N$ mesh of $\SI{1}{\kilo\ohm}$ resistors.}
  \label{fig:q2-resistance}
\end{figure}

\section*{Question 3}

We then wrote a program to use a SOR finite difference method to solve for the potential between two conductors, as given in the assignment specifications. We exploited both the vertical and horizontal symmetries to work with only a quarter of the grid. The code is shown in \Cref{sec:main,sec:finite-differences-h,sec:finite-differences-cpp}.

We then set $h = 0.02$ and varied $w$ over the range $[1.0, 2.0)$ in $0.1$ increments. The results are shown in \Cref{tab:q3-w-sor}. The number of iterations taken for each value of $w$ is also plotted in \Cref{fig:q3-w}. We omit the point at $w = 1.9$ to better visualize the data. From this we can see that the value of $w = 1.4$ results in the fastest convergence.

\begin{table}[!htb]
  \centering
  \caption{Results for SOR with $h = 0.02$ for different values of $w$.}
  \label{tab:q3-w-sor}
  \resizebox{0.7\columnwidth}{!}{\begin{tabular}{|l|l|l|}
    \hline
    $w$ & Iterations & $V_{(0.06,0.04)} (\SI{}{\volt})$ \\ \hline
    1.0 & 38 & 5.52627 \\ \hline
    1.1 & 31 & 5.52627 \\ \hline
    1.2 & 26 & 5.52630 \\ \hline
    1.3 & 21 & 5.52631 \\ \hline
    1.4 & 20 & 5.52634 \\ \hline
    1.5 & 25 & 5.52635 \\ \hline
    1.6 & 37 & 5.52634 \\ \hline
    1.7 & 54 & 5.52634 \\ \hline
    1.8 & 92 & 5.52635 \\ \hline
    1.9 & 903 & 5.52632 \\ \hline
  \end{tabular}}
\end{table}

\begin{figure}[!htb]
  \centering
  \includegraphics[width=0.6\columnwidth]{question-3/w_sor.eps}
  \caption{Iterations vs. $w$ for SOR with $h = 0.02$.}
  \label{fig:q3-w}
\end{figure}

We then set $w = 1.4$ and varied $h$. The results are shown in \Cref{tab:q3-h-sor} and plotted in \Cref{fig:q3-h-v-sor,fig:q3-h-iter-sor}. From this data, it is clear that the potential at the point $(0.06, 0.04)$ is dropping as we decrease $h$. However, it is hard to predict what the value is without trying lower values of $h$, which would take too much time. We can estimate that it is between $\SI{5.10}{\volt}$ and $\SI{5.20}{\volt}$. To three significant figures, we could therefore estimate it to be around $\SI{5.14}{\volt}$, but this estimate is very likely to be wrong. We can also observe from \Cref{fig:q3-h-iter-sor} that the number of iterations increases at supra-linear rates. That is, the iterations needed increase faster than $1/h$. This has the consequence that decreasing $h$ to find better approximations rapidly becomes impractical.

\begin{table}[!htb]
  \centering
  \caption{Results for SOR with $w = 1.4$ for different values of $h$.}
  \label{tab:q3-h-sor}
  \resizebox{0.7\columnwidth}{!}{\begin{tabular}{|l|l|l|}
    \hline
    $1/h$ & iterations & $V_{(0.06,0.04)} (\SI{}{\volt})$ \\ \hline
    50 & 20 & 5.52634 \\ \hline
    100 & 62 & 5.35057 \\ \hline
    200 & 209 & 5.28872 \\ \hline
    250 & 308 & 5.27838 \\ \hline
    500 & 1009 & 5.25778 \\ \hline
    1000 & 3215 & 5.23656 \\ \hline
  \end{tabular}}
\end{table}

\begin{figure}[!htb]
  \centering
  \includegraphics[width=0.6\columnwidth]{question-3/h_v_sor.eps}
  \caption{$V_{(0.06,0.04)}$ vs. $1/h$ for SOR with $w = 1.4$.}
  \label{fig:q3-h-v-sor}
\end{figure}

\begin{figure}[!htb]
  \centering
  \includegraphics[width=0.6\columnwidth]{question-3/h_iter_sor.eps}
  \caption{Iterations vs. $1/h$ for SOR with $w = 1.4$.}
  \label{fig:q3-h-iter-sor}
\end{figure}

Then, we used the Jacobi method to solve the same problem. The results are shown in \Cref{tab:q3-h-jacobi} and plotted in \Cref{fig:q3-h-v-jacobi,fig:q3-h-iter-jacobi}. From this data, we can see that the Jacobi method requires more iterations than SOR for the same values of $h$. However, the rate of increase appears to be similar. Thus, we conclude that the SOR approach has an initial speed advantage, but as $h$ gets smaller its performance drops at the same rate as the Jacobi method. It is possible that varying the value of $w$ can improve the performance of SOR at these smaller spacings. In addition, we notice that the Jacobi method is more accurate than SOR for the same values of $h$, since the potential it computes is slightly lower.

\begin{table}[!htb]
  \centering
  \caption{Results for the Jacobi method for different values of $h$.}
  \label{tab:q3-h-jacobi}
  \resizebox{0.7\columnwidth}{!}{\begin{tabular}{|l|l|l|}
    \hline
    $1/h$ & iterations & $V_{(0.06,0.04)} (\SI{}{\volt})$ \\ \hline
    50 & 63 & 5.52614 \\ \hline
    100 & 227 & 5.34994 \\ \hline
    200 & 781 & 5.28654 \\ \hline
    250 & 1153 & 5.27505 \\ \hline
    500 & 3764 & 5.24523 \\ \hline
    1000 & 11692 & 5.18932 \\ \hline
  \end{tabular}}
\end{table}

\begin{figure}[!htb]
  \centering
  \includegraphics[width=0.6\columnwidth]{question-3/h_v_jacobi.eps}
  \caption{$V_{(0.06,0.04)}$ vs. $1/h$ for the Jacobi method.}
  \label{fig:q3-h-v-jacobi}
\end{figure}

\begin{figure}[!htb]
  \centering
  \includegraphics[width=0.6\columnwidth]{question-3/h_iter_jacobi.eps}
  \caption{Iterations vs. $1/h$ for the Jacobi method.}
  \label{fig:q3-h-iter-jacobi}
\end{figure}

We then modified the program to allow uneven spacing. An interface was provided where the user could input a vector of $x$ positions and a vector of $y$ positions indicating where grid lines would be located. The code is shown in \Cref{sec:finite-differences-h,sec:finite-differences-cpp}. Using this setup, we separated the grid using the same number of nodes as for $h = 0.01$. We approached this by starting with the grid for $h = 0.02$ and adding lines close to the point $(0.06, 0.04)$ until we got a better result. The resulting $x$ vector was (0.0, 0.02, 0.04, 0.05, 0.055, 0.06, 0.065, 0.07, 0.08, 0.1) and the resulting $y$ vector was (0.0, 0.02, 0.03, 0.04, 0.041, 0.042, 0.045, 0.05, 0.06, 0.07, 0.08, 0.1). Using this setup, we were able to obtain a value of $V_{(0.06,0.04)} = \SI{5.33519}{\volt}$, which is closer to the real value than the value obtained with $h = 0.01$.

% \section{Conclusions}

\newpage
\onecolumn

\begin{appendices}

\section{Main.cpp}
\label{sec:main}
\lstinputlisting[language=C++]{src/main.cpp}
\newpage

\section{Matrix.h}
\label{sec:matrix}
\lstinputlisting[language=C++]{src/matrix.h}
\newpage

\section{Matrix-util.h}
\label{sec:matrix-util}
\lstinputlisting[language=C++]{src/matrix-util.h}
\newpage

\section{Cholesky.h}
\label{sec:cholesky}
\lstinputlisting[language=C++]{src/cholesky.h}
\newpage

\section{Solver.h}
\label{sec:solver}
\lstinputlisting[language=C++]{src/solver.h}
\newpage

\section{Mesh.h}
\label{sec:mesh-h}
\lstinputlisting[language=C++]{src/mesh.h}
\newpage

\section{Mesh.cpp}
\label{sec:mesh-cpp}
\lstinputlisting[language=C++]{src/mesh.cpp}
\newpage

\section{Circuit-solver.h}
\label{sec:circuit-solver-h}
\lstinputlisting[language=C++]{src/circuit-solver.h}
\newpage

\section{Circuit-solver.cpp}
\label{sec:circuit-solver-cpp}
\lstinputlisting[language=C++]{src/circuit-solver.cpp}
\newpage

\section{Finite-differences.h}
\label{sec:finite-differences-h}
\lstinputlisting[language=C++]{src/finite-differences.h}
\newpage

\section{Finite-differences.cpp}
\label{sec:finite-differences-cpp}
\lstinputlisting[language=C++]{src/finite-differences.cpp}
\newpage

\section{Extract from Solver-test.cpp}
\label{sec:solver-test-cpp}
\begin{lstlisting}[language=C++]
TEST_CASE("solve succeeds for a 2x2 system of equations")
{
    Matrix<double> lower = "["
                           "1,0;"
                           "2,1"
                           "]";
    Matrix<double> m = lower * transpose(lower);
    Matrix<double> x = "[4;3]";
    Matrix<double> b = m * x;
    auto result = solve(m, b);
    REQUIRE(result == x);
}

TEST_CASE("solve succeeds for a 3x3 system of equations")
{
    Matrix<double> lower = "["
                           "4, 0,0;"
                           "5, 1,0;"
                           "9,-1,2"
                           "]";
    Matrix<double> m = lower * transpose(lower);
    Matrix<double> x = "[1.0;2.5;-2.0]";
    Matrix<double> b = m * x;
    auto result = solve(m, b);
    REQUIRE(result == x);
}

TEST_CASE("solve succeeds for a 4x4 system of equations")
{
    Matrix<double> lower = "["
                           " 1, 0, 0, 0;"
                           " 2, 3, 0, 0;"
                           " 5, 7,11, 0;"
                           "13,17,19,23"
                           "]";
    Matrix<double> m = lower * transpose(lower);
    Matrix<double> x = "[1;2;4;8]";
    Matrix<double> b = m * x;
    auto result = solve(m, b);
    REQUIRE(result == x);
}

TEST_CASE("solve succeeds for a 5x5 system of equations")
{
    Matrix<double> lower = "["
                           " 1,0,0,0,0;"
                           " 2,1,0,0,0;"
                           " 4,2,1,0,0;"
                           " 8,4,2,1,0;"
                           "16,8,4,2,1"
                           "]";
    Matrix<double> m = lower * transpose(lower);
    Matrix<double> x = "[13;-7;19;-11;3]";
    Matrix<double> b = m * x;
    auto result = solve(m, b);
    REQUIRE(result == x);
}
\end{lstlisting}
\newpage

\section{Extract from Circuit-solver-test.cpp}
\label{sec:circuit-solver-test-cpp}
\begin{lstlisting}[language=C++]
// Truncates the least significant bits of the value for a better floating point
// comparison that ignores some round-off error.
float fround(double value)
{
    return static_cast<float>(value);
}

TEST_CASE("csolve succeeds for circuit 1")
{
    std::stringstream in{"1 0 0 10 0\n"
                         "1 0 0 10 10\n"};
    auto v = csolve(in);
    CHECK(fround(v(0)) == 5.0f);
}

TEST_CASE("csolve succeeds for circuit 2")
{
    std::stringstream in{"1 0 10 10 0\n"
                         "1 0 0 10 0\n"};
    auto v = csolve(in);
    CHECK(fround(v(0)) == 50.0f);
}

TEST_CASE("csolve succeeds for circuit 3")
{
    std::stringstream in{"1 0 0 10 10\n"
                         "1 0 10 10 0\n"};
    auto v = csolve(in);
    CHECK(fround(v(0)) == 55.0f);
}

TEST_CASE("csolve succeeds for circuit 4")
{
    std::stringstream in{"1 0 0 10 10\n"
                         "1 0 0 10 0\n"
                         "1 2 0 5 0\n"
                         "2 0 10 5 0\n"};
    auto v = csolve(in);
    CHECK(fround(v(0)) == 20.0f);
    CHECK(fround(v(1)) == 35.0f);
}

TEST_CASE("csolve succeeds for circuit 5")
{
    std::stringstream in{"1 0 0 20 10\n"
                         "1 2 0 10 0\n"
                         "1 3 0 10 0\n"
                         "2 3 0 30 0\n"
                         "2 0 0 30 0\n"
                         "3 0 0 30 0\n"};
    auto v = csolve(in);
    CHECK(fround(v(0)) == 5.0f);
    CHECK(fround(v(1)) == 3.75f);
    CHECK(fround(v(2)) == 3.75f);
}
\end{lstlisting}

\end{appendices}

\end{document}


\documentclass[12pt]{article}

\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{float}
\usepackage{ulem}
\usepackage{bm}

\usepackage{anysize}

\marginsize{2cm}{2cm}{0.9cm}{1.8cm}
\usepackage[framed,numbered,autolinebreaks,useliterate]{mcode}
\usepackage{listings}
\lstset{language=Matlab}
\lstset{breaklines}
\lstset{extendedchars=false}

\title{Machine Learning/Pattern Recognition\\  \begin{Large} Homework \#2 \end{Large} }
\author{Jiyu Tian}
\date{}

\begin{document}

\maketitle
%%---------------------------------------------------------------
%% Problem 1
%%---------------------------------------------------------------
\section{Problem 2.12}
\large{\textbf{Solution}}: \\
(a) From normalization condition we have:\\
\begin{equation*}
1 = \sum_{i=1}^{c} P(w_i|\bm{x}) \leq \sum_{i=1}^{c} P(w_{max}|\bm{x}) = c\times P(w_{max}|\bm{x})
\end{equation*}
And therefore we must have:
\begin{equation*}
P(w_{max}|\bm{x})\leq \frac{1}{c}
\end{equation*}
(b) The probability of error is 1 minus the probability of being correct, that is:
\begin{equation*}
\begin{aligned}
P(error) &= 1 - P(correct)\\ &= 1-\int P(correct|\bm{x})P(\bm{x}) \text{ d}\bm{x}\\ & = 1-\int P(w_{max}|\bm{x})P(\bm{x}) \text{ d}\bm{x}
\end{aligned}
\end{equation*}
(c) With two results above we have:
\begin{equation*}
\begin{aligned}
P(error)&=1-\int P(w_{max}|\bm{x})P(\bm{x})\text{ d}\bm{x}\\ &\leq 1- \frac{1}{c}\int P(\bm{x})\text{ d}\bm{x}\\ &= 1-\frac{1}{c}
\end{aligned}
\end{equation*}
(d) Equality occurs if and only if all posteriors are equal:
\begin{equation*}
P(w_i|\bm{x})=P(w_j|\bm{x}) \text{ for all }i,j=1,...,c
\end{equation*}

\vfill
\clearpage
%%---------------------------------------------------------------
%% Problem 2
%%---------------------------------------------------------------
\section{Problem 3.1}
\large{\textbf{Solution}}: \\
(a) Figure 1 can be plotted with MATLAB:
\begin{figure}[H]
\centering
\includegraphics[width = 1\textwidth]{PR31.png}
\caption{\label{fig2}$P(x|\theta)\ versus\ x\ and\ \theta$}
\end{figure}
\noindent(b) \\
\begin{equation*}
\begin{aligned}
\hat{\theta} &=\arg\max_{\theta}\ P(D|\theta) = \arg\max_{\theta}\ \prod_{i=1}^{n} P(x_i|\theta) =\arg\max_{\theta}\ \prod_{i=1}^{n} \theta e^{-\theta x_i}\\ &=\arg\max_{\theta}\ \theta^n e^{-\theta \sum^{n}_{i=1}x_i} =\arg\max_{\theta}\ n\text{ln}\theta-\theta \sum^{n}_{i=1}x_i
\end{aligned}
\end{equation*}
Taking the derivative and equating to 0, we have:
\begin{equation*}
\begin{aligned}
\frac{n}{\hat{\theta}}-\sum^{n}_{i=1}x_i = 0
\end{aligned}
\end{equation*}
Then the maximum likelihood estimator $\hat{\theta}$ is given by
\begin{equation*}
\hat{\theta}=\frac{1}{\frac{1}{n}\sum^{n}_{i=1}x_i}.
\end{equation*}

\vfill
\clearpage
%%---------------------------------------------------------------
%% Problem 3
%%---------------------------------------------------------------
\section{Problem 3.2}
\large{\textbf{Solution}}: \\
(a) Here we introduce an indicator function $\bm{1}\{\cdot\}$ which equals to 1 for condition within and 0 otherwise.
\begin{equation*}
\begin{aligned}
\hat{\theta} &=\arg\max_{\theta}\ P(D|\theta) = \arg\max_{\theta}\ \prod_{i=1}^{n} P(x_i|\theta) =\arg\max_{\theta}\ \prod_{i=1}^{n} \frac{1}{\theta}\ \bm{1}\{0\leq x_i\leq \theta\}\\
&=\arg\max_{\theta}\  \frac{1}{\theta^n} \prod_{i=1}^{n}\bm{1}\{0\leq x_i\leq \theta\}\\
&= \arg\max_{\theta}\  \frac{1}{\theta^n}\ \bm{1}\{0\leq\min_i x_i\}\ \bm{1}\{\max_i x_i\leq\theta\}
\end{aligned}
\end{equation*}
The function equals to zero for $\theta\leq \underset{i}{\max}\ x_i$ and decreases as $\theta$ increases for $\theta\geq \underset{i}{\max}\ x_i$.  Therefore, the likelihood function is maximized at $\hat{\theta} =  \underset{i}{\max}\ x_i$.
%%---------------------------------------------------------------
%% Problem 4
%%---------------------------------------------------------------
\section{Problem 3.3}
\large{\textbf{Solution}}: \\
(a) From statement we have $P\{z_{ik}=1|P(w_i)\}=P(w_i)$ and $P\{z_{ik}=0|P(w_i)\}=1-P(w_i)$, which can be combined as:
\begin{equation*}
\begin{aligned}
P(z_{ik}|P(w_i))=P(w_i)^{z_{ik}}(1-P(w_i))^{1-z_{ik}}
\end{aligned}
\end{equation*}
Then for the independent states, we have:
\begin{equation*}
\begin{aligned}
P(z_{i1},...,z_{ik}|P(w_i))=\prod^n_{k=1}P(z_{ik}|P(w_i))=\prod^n_{k=1}P(w_i)^{z_{ik}}(1-P(w_i))^{1-z_{ik}}
\end{aligned}
\end{equation*}
(b) With $\theta = P(w_i)$, the ML estimate for $P(w_i)$ can be expressed as:
\begin{equation*}
\begin{aligned}
\hat{\theta} &= \arg\max_{\theta}\ \prod^n_{k=1}\theta^{z_{ik}}(1-\theta)^{1-z_{ik}} =\arg\max_{\theta}\ \sum_{k=1}^{n} [z_{ik}\ln\theta+(1-z_{ik})\ln(1-\theta)]
\end{aligned}
\end{equation*}
Taking the derivative and equating to 0, we have:
\begin{equation*}
\begin{aligned}
\frac{1}{\hat{\theta}}\ \sum_{k=1}^{n} z_{ik} - \frac{1}{1-\hat{\theta}}\ \sum_{k=1}^{n} (1-z_{ik}) = 0\\
\end{aligned}
\end{equation*}
\begin{equation*}
\begin{aligned}
\hat{P}(w_i)=\hat{\theta} = \frac{1}{n}\sum_{k=1}^{n} z_{ik}
\end{aligned}
\end{equation*}
The ML estimate of prior probability is merely the number of this category divided by total number of samples.
\vfill
\clearpage
%%---------------------------------------------------------------
%% Problem 5
%%---------------------------------------------------------------
\section{Problem 3.4}
\large{\textbf{Solution}}: \\
\begin{equation*}
\begin{aligned}
\hat{\bm{\theta}} &= \arg\max_{\bm{\theta}} \ln P(\bm{D}|\bm{\theta}) =  \arg\max_{\bm{\theta}} \ln \prod^n_{k=1}\prod^d_{i=1}\theta_i^{x_{ki}}(1-\theta_i)^{1-x_{ki}}\\
&=\arg\max_{\bm{\theta}} \sum^n_{k=1}\sum^d_{i=1}x_{ki}\ln\theta_i + (1-x_{ki})\ln(1-\theta_i)
\end{aligned}
\end{equation*}
Taking the partial derivative with respect to $\theta_i$ and equating to 0, we have:
\begin{equation*}
\begin{aligned}
\frac{1}{\hat{\theta_i}}\sum_{k=1}^n x_{ki} - \frac{1}{1-\theta_i} \sum_{k=1}^n (1-x_{ki}) =0
\end{aligned}
\end{equation*}

\begin{equation*}
\begin{aligned}
\hat{\theta_i} = \frac{1}{n} \sum_{k=1}^n x_{ki}
\end{aligned}
\end{equation*}
Therefore the ML estimate for $\bm{\theta}$ is
\begin{equation*}
\begin{aligned}
\hat{\bm{\theta}} = \frac{1}{n} \sum_{k=1}^n x_{k}
\end{aligned}
\end{equation*}

%%---------------------------------------------------------------
%% Problem 6
%%---------------------------------------------------------------
\section{Problem 3.17}
\large{\textbf{Solution}}: \\
(a) For n $i.i.d$ samples $D=\{\bm{x}_1,...,\bm{x}_n\}$, we have:

\begin{equation*}
\begin{aligned}
P(D|\bm{\theta})&=P(\bm{x_1},...,\bm{x_n}|\bm{\theta}) = \prod_{k=1}^n P(x_k|\bm{\theta})\\ &= \prod_{k=1}^n \prod^d_{i=1} \theta_i^{x_{ki}}(1-\theta_i)^{1-x_{ki}} \\ &= \prod^d_{i=1} \theta_i^{\sum_{k=1}^n x_{ki}}(1-\theta_i)^{\sum_{k=1}^n (1-x_{ki})}\\ &= \prod^d_{i=1} \theta_i^{s_i}(1-\theta_i)^{n-s_i}
\end{aligned}
\end{equation*}
\noindent(b) With assumption that $\bm{\theta}$ is uniformly distributed, we have $P(\bm{\theta})=1$ for $0\leq \theta_i \leq1,\ i=1,...,d$. From Bayes' theorem we have:
\begin{equation*}
\begin{aligned}
P(\bm{\theta}|D)&=\frac{P(D|\bm{\theta})P(\bm{\theta})}{P(D)} =\frac{P(D|\bm{\theta})P(\bm{\theta})}{\int P(D|\bm{\theta})P(\bm{\theta})\text{ d}\bm{\theta}}= \frac{P(D|\bm{\theta})}{\int P(D|\bm{\theta})\text{ d}\bm{\theta}}\\
&= \frac{\prod^d_{i=1} \theta_i^{s_i}(1-\theta_i)^{n-s_i}}{\int \prod^d_{i=1} \theta_i^{s_i}(1-\theta_i)^{n-s_i}\text{ d}\bm{\theta}}\\
&=\frac{\prod^d_{i=1} \theta_i^{s_i}(1-\theta_i)^{n-s_i}}{\int_0^1 ... \int_0^1 \prod^d_{i=1} \theta_i^{s_i}(1-\theta_i)^{n-s_i}\text{ d}\theta_1...\text{d}\theta_d}\\
&=\prod_{i=1}^{d} \frac{\theta_i^{s_i}(1-\theta_i)^{n-s_i}}{\int_0^1 \theta_i^{s_i}(1-\theta_i)^{n-s_i}\text{ d}\theta_i}\\
&=\prod_{i=1}^{d} \frac{\theta_i^{s_i}(1-\theta_i)^{n-s_i}}{\frac{s_i!(n-s_i)!}{(n+1)!}}\\
&=\prod_{i=1}^{d} \frac{(n+1)!}{s_i!(n-s_i)!}\theta_i^{s_i}(1-\theta_i)^{n-s_i}\\
\end{aligned}
\end{equation*}
(c) Given $d=1,\ n=1$, we have:
\begin{equation*}
\begin{aligned}
P(\theta|D)= \frac{2!}{s_1!(1-s_1)!}\theta^{s_1}(1-\theta)^{1-s_1}\\
\end{aligned}
\end{equation*}
Note that $s_1$ has two values 0 and 1, and the density for each of them:
\begin{equation*}
\begin{aligned}
&s_1 = 0\ :\ P(\theta|D)= 2(1-\theta)\\
&s_1 = 1\ :\ P(\theta|D)= 2\theta\\
\end{aligned}
\end{equation*}
The densities can be plotted with MATLAB as Figure 2.
\begin{figure}[H]
\centering
\includegraphics[width = 1\textwidth]{pb317.png}
\caption{\label{fig3}$P(x|\theta)\ versus\ x\ and\ \theta$}
\end{figure}
%------------------------------------------------------------------------
\noindent(d) Since in Bernoulli case $x_i$ can be either 0 or 1, we have:
\begin{equation*}
\begin{aligned}
P(\bm{x}|D) &= \int P(\bm{x}|\bm{\theta}) P(\bm{\theta}|D)\text{ d}\bm{\theta}\\
&=\int \prod_{i=1}^d\theta_i^{x_i}(1-\theta_i)^{1-x_i}\prod_{i=1}^{d} \frac{(n+1)!}{s_i!(n-s_i)!}\theta_i^{s_i}(1-\theta_i)^{n-s_i} \text{ d}\bm{\theta}\\
&=\prod_{i=1}^d  \frac{(n+1)!}{s_i!(n-s_i)!} \int \prod_{i=1}^d\theta_i^{x_i}(1-\theta_i)^{1-x_i} \theta_i^{s_i}(1-\theta_i)^{n-s_i} \text{ d}\bm{\theta}\\
&=\prod_{i=1}^d  \frac{(n+1)!}{s_i!(n-s_i)!} \prod_{i=1}^d \int \theta_i^{x_i+s_i}(1-\theta_i)^{n+1-x_i-s_i} \\
&=\prod_{i=1}^d  \frac{(n+1)!}{s_i!(n-s_i)!} \prod_{i=1}^d  \frac{(x_i+s_i)!(n+1-x_i-s_i)!}{(n+2)!} \\
&=\prod_{i=1}^d  \frac{(n+1)!(x_i+s_i)!(n+1-x_i-s_i)!}{s_i!(n-s_i)!(n+2)!} \\
&=\prod_{i=1}^d  \frac{(x_i+s_i)!(n+1-x_i-s_i)!}{s_i!(n-s_i)!(n+2)} \ \ \ \ \text{($x_i$ can be 0 or 1)}\\
&=\prod_{i=1}^d  (\frac{(s_i+1)!(n-s_i)!}{s_i!(n-s_i)!(n+2)})^{x_i} (\frac{s_i!(n+1-s_i)!}{s_i!(n-s_i)!(n+2)})^{1-x_i} \\
&=\prod_{i=1}^d  (\frac{s_i+1}{n+2})^{x_i} (\frac{n+1-s_i}{n+2})^{1-x_i} \\
&=\prod_{i=1}^d  (\frac{s_i+1}{n+2})^{x_i} (1-\frac{s_i+1}{n+2})^{1-x_i}
\end{aligned}
\end{equation*}

%-----------------------------------------------------------------------
\noindent(e) Comparing the following two equations:
\begin{equation*}
\begin{aligned}
P(\bm{x}|D) = \prod_{i=1}^d  (\frac{s_i+1}{n+2})^{x_i} (1-\frac{s_i+1}{n+2})^{1-x_i}
\end{aligned}
\end{equation*}
\begin{equation*}
\begin{aligned}
P(\bm{x}|\bm{\theta}) = \prod_{i=1}^d\theta_i^{x_i}(1-\theta_i)^{1-x_i}
\end{aligned}
\end{equation*}
we have the effective Bayesian estimate for $\bm{\theta}$:
\begin{equation*}
\begin{aligned}
\hat{\theta_i} = \frac{s_i+1}{n+2}
\end{aligned}
\end{equation*}

\end{document}


\documentclass[t]{beamer}
\usetheme{Copenhagen}
\setbeamertemplate{headline}{} % remove toc from headers
\beamertemplatenavigationsymbolsempty

\usepackage{amsmath, bm, tcolorbox}

\title{Systems of Equations}
\author{}
\date{}

\AtBeginSection[]
{
  \begin{frame}
    \frametitle{Objectives}
    \tableofcontents[currentsection]
  \end{frame}
}

\begin{document}

\begin{frame}
\maketitle
\end{frame}

\section{Write a system of equations using matrices}

\begin{frame}{Matrix Multiplication}
Previously, we looked at the method for multiplying matrices:
\[
\begin{bmatrix}
7	&	2	&	-1	\\
0	&	5	&	4	\\
-3	&	6	&	2	\\
\end{bmatrix}
\cdot
\begin{bmatrix}
2	\\ 1 \\ 3 \\
\end{bmatrix}
\onslide<2->{= \begin{bmatrix}
13 \\ 17 \\ 6 \\
\end{bmatrix}}
\]
\onslide<3->{In this section, we will use matrix multiplication to solve a system of equations.}
\end{frame}

\begin{frame}
\begin{tcolorbox}[colframe=green!20!black, colback = green!30!white,title=\textbf{System of Equations}]
A \textbf{system of equations} consists of 2 or more equations with 2 or more variables.
\end{tcolorbox}
\vspace{6pt}
\onslide<2->{\textsc{Examples}}
\onslide<3->{
\begin{align*}
2x + 5y &= 8 \\
-7x - 3y &= 9 \\
\end{align*}
\begin{align*}
4x-9y+2z &= 10 \\
-x + z &= 15	\\
3x+10y-12z &= 0 \\
\end{align*}
}
\end{frame}

\begin{frame}{Writing a System of Equations Using Matrices}
We can use matrix multiplication with the coefficients and variables to write a system of equations using matrices.		\newline\\		\pause

So
\begin{align*}
2x + 5y &= 8 \\
-7x - 3y &= 9 \\
\end{align*}
\onslide<3->{
becomes
\[
\begin{bmatrix}
2	&	5	\\
-7	&	-3	\\
\end{bmatrix}
\begin{bmatrix}
x	\\
y	\\
\end{bmatrix}
=
\begin{bmatrix}
8	\\
9	\\
\end{bmatrix}
\]}
\end{frame}

\begin{frame}{Writing a System of Equations Using Matrices}
And
\begin{align*}
4x-9y+2z &= 10 \\
-x + z &= 15	\\
3x+10y-12z &= 0 \\
\end{align*}
\onslide<2->{
becomes
\[
\begin{bmatrix}
4	&	-9	&	2	\\
-1	&	\alert{0}	&	1	\\
3	&	10	&	-12	\\
\end{bmatrix}
\begin{bmatrix}
x	\\
y	\\
z	\\
\end{bmatrix}
=
\begin{bmatrix}
10	\\
15	\\
0		\\
\end{bmatrix}
\]
}
\end{frame}

\begin{frame}{Example 1}
Write each of the following systems of equations using matrices.	\newline\\
(a)	\quad
\begin{align*}
4y	+ z &= -1 \\
-4x - 5y + 5z &= 0 \\
6x + 5y &= 25 \\
\end{align*}
\onslide<2->{
\[
\begin{bmatrix}
0	&	4	&	1	\\
-4	&	-5	&	5	\\
6	&	5	&	0	\\
\end{bmatrix}
\onslide<3->{
\begin{bmatrix}
x \\ y \\ z \\
\end{bmatrix}
}
\onslide<4->{
= \begin{bmatrix}
-1 \\ 0 \\ 25 \\
\end{bmatrix}
}
\]
}
\end{frame}

\begin{frame}{Example 1}
(b) \quad
\begin{align*}
x - y &= -9 \\
-3x - 4y &= -8 \\
\end{align*}
\onslide<2->{
\[
\begin{bmatrix}
1 & -1 \\
-3 & -4 \\
\end{bmatrix}
\onslide<3->{
\begin{bmatrix}
x \\ y \\
\end{bmatrix}
}
\onslide<4->{
= \begin{bmatrix}
-9 \\ -8 \\
\end{bmatrix}
}
\]
}
\end{frame}

\section{Solve a system of equations using inverse matrices}

\begin{frame}{Solving Equations Using Inverse Operations}
Long ago, you learned that to solve something like
\[ 5x = 10 \]
you divide (the inverse operation of multiplication) both sides by 5.	\newline\\	\pause

We don't ``divide" matrices in this sense, but we do need to use an inverse operation to solve the previous examples.
\end{frame}

\begin{frame}{Identity Matrices}
\begin{tcolorbox}[colframe=green!20!black, colback = green!30!white,title=\textbf{Identity Matrix}]
An \textbf{identity matrix} is a square matrix with 1s along the diagonal from top left to bottom right, and 0s elsewhere.
\end{tcolorbox}
\vspace{10pt}	\pause
\[
I_2 = \begin{bmatrix}
1 & 0 \\
0 & 1 \\
\end{bmatrix}
\quad
I_3 = \begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1 \\
\end{bmatrix}
\quad
\text{etc.}
\]
\end{frame}

\begin{frame}{Inverse Matrix}
\begin{tcolorbox}[colframe=green!20!black, colback = green!30!white,title=\textbf{Inverse Matrix}]
The \textbf{inverse matrix} for square matrix $A$ is $A^{-1}$.
\end{tcolorbox}
\vspace{10pt}	\pause
Inverse matrices are such that
\[
A \cdot A^{-1} = I
\]	\pause
and that if $AX = B$ where $A$ is the matrix of coefficients and $B$ are the constants on the right side, then
\[
X = A^{-1} \cdot B
\]
\pause
\emph{Note:} We will not discuss the techniques of how to actually find the inverse of a matrix without a calculator.
\end{frame}

\begin{frame}{Example 2}
Solve each of the following using matrices.	\newline\\
(a) \quad
\begin{align*}
4y	+ z &= -1 \\
-4x - 5y + 5z &= 0 \\
6x + 5y &= 25 \\
\end{align*}
\onslide<2->{
\[
\begin{bmatrix}
0	&	4	&	1	\\
-4	&	-5	&	5	\\
6	&	5	&	0	\\
\end{bmatrix}
\begin{bmatrix}
x \\ y \\ z \\
\end{bmatrix}
= \begin{bmatrix}
-1 \\ 0 \\ 25 \\
\end{bmatrix}
\]
}
\end{frame}

\begin{frame}{Example 2}
\[
\begin{bmatrix}
x \\ y \\ z \\
\end{bmatrix}
= A^{-1}B
\]
\pause \vspace{6pt}
\[
\begin{bmatrix}
x \\ y \\ z \\
\end{bmatrix}
= \begin{bmatrix}
5 \\ -1 \\ 3 \\
\end{bmatrix}
\]
\pause \vspace{6pt}
\[x = 5, \quad y = -1, \quad z = 3\]
\end{frame}

\begin{frame}{Example 2}
(b) \quad
\begin{align*}
x - y &= -9 \\
-3x - 4y &= -8 \\
\end{align*}
\onslide<2->{
\[
\begin{bmatrix}
1 & -1 \\
-3 & -4 \\
\end{bmatrix}
\begin{bmatrix}
x \\ y \\
\end{bmatrix}
= \begin{bmatrix}
-9 \\ -8 \\
\end{bmatrix}
\]
}
\end{frame}

\begin{frame}{Example 2}
\[
\begin{bmatrix}
x \\ y \\
\end{bmatrix}
= A^{-1}B
\]
\pause \vspace{6pt}
\[
\begin{bmatrix}
x \\ y
\end{bmatrix}
= \begin{bmatrix}
-4 \\ 5
\end{bmatrix}
\]
\pause \vspace{6pt}
\[x = -4, \quad y = 5\]
\end{frame}

\begin{frame}{Example 2}
(c) \quad
\begin{align*}
x - 3y - 5z &= -18 \\
-2x + 3y + 5z &= 23 \\
-x + 3y + 6z &= 17 \\
\end{align*}
\onslide<2->{
\[
\begin{bmatrix}
1 & -3 & - 5 \\
-2 & 3 & 5 \\
-1 & 3 & 6 \\
\end{bmatrix}
\cdot
\begin{bmatrix}
x \\ y \\ z \\
\end{bmatrix}
= \begin{bmatrix}
-18 \\ 23 \\ 17 \\
\end{bmatrix}
\]
}
\end{frame}

\begin{frame}{Example 2}
\[
\begin{bmatrix}
x \\ y \\
\end{bmatrix}
= A^{-1}B
\]
\pause \vspace{6pt}
\[
\begin{bmatrix}
x \\ y \\ z
\end{bmatrix}
= \begin{bmatrix}
-5 \\ 6 \\ -1 \\
\end{bmatrix}
\]
\pause \vspace{6pt}
\[x = -5, \quad y = 6, \quad z = -1\]
\end{frame}

\section{Solve applications of systems of equations}

\begin{frame}{Applied Systems of Equations}
Setting up many applied systems of equations problems boils down to
\begin{center}
unit rate $\times$ amount = total amount
\end{center}
For instance, \$2.00 per gallon times 5 gallons of gas costs a total amount of \$10.
\end{frame}

\begin{frame}{Example 3}
(a) \quad How many mL of a solution containing 10\% pure hydrochloric acid must be mixed with a solution containing 15\% hydrochloric acid to produce 30 mL of a solution that is 11\% hydrochloric acid?	\pause	\newline\\
$x$ = mL of the 10\% acid solution \\
$y$ = mL of the 15\% acid solution \pause
\begin{align*}
x + y &= 30 &\text{volume of total liquid} \\
0.10x + 0.15y &= 30(0.11) &\text{volume of total hydrocholic acid} \\
\end{align*}
\end{frame}

\begin{frame}{Example 3}
\[
\begin{bmatrix}
1	&	1	\\
0.1	&	0.15	\\
\end{bmatrix}
\cdot
\begin{bmatrix}
x \\ y
\end{bmatrix}
= \begin{bmatrix}
30 \\ 3.3 \\
\end{bmatrix}
\]
\vspace{6pt}
\onslide<2->{
\[
\begin{bmatrix}
x \\ y \\
\end{bmatrix}
= \begin{bmatrix}
24 \\ 6 \\
\end{bmatrix}
\]
}
\newline\\ \pause
We need to mix 24 mL of a 10\% hydrochloric acid solution with 6 mL of a 15\% hydrochloric acid solution.
\end{frame}

\begin{frame}{Example 3}
(b) \quad A coffee shop wants to sell 8 pound bags of a coffee blend for \$28.64. They will do this by blending coffee that costs \$2.25 per pound with coffee that costs \$4.00 per pound. How much of each type of coffee should they use in the blend?	\newline\\	\pause
$x$ = number of pounds of cheaper coffee (\$2.25 per pound) \\
$y$ = number of pounds of more expensive coffee (\$4.00 per pound) \pause
\begin{align*}
x + y &= 8  &\text{weight of each bag} \\
2.25x + 4.00y &= 28.64 &\text{total cost} \\
\end{align*}
\end{frame}

\begin{frame}{Example 3}
\[
\begin{bmatrix}
1	&	1	\\
2.25 & 4	\\
\end{bmatrix}
\cdot
\begin{bmatrix}
x \\ y
\end{bmatrix}
= \begin{bmatrix}
8 \\ 28.64 \\
\end{bmatrix}
\]
\vspace{6pt}
\onslide<2->{
\[
\begin{bmatrix}
x \\ y \\
\end{bmatrix}
= \begin{bmatrix}
1.92 \\ 6.08 \\
\end{bmatrix}
\]
}
\newline\\ \pause
They will need 1.92 pounds of the cheaper coffee and 6.08 pounds of the more expensive coffee.
\end{frame}

\end{document}


\section{$H^\ast(BU(n))$, splitting principle}\label{homology-bun}
%I wanted to share some ideas about the question on the first nonzero homology group of an Eilenberg-MacLane space.
%One idea was to use this extension $0\to \Z\xar{p^k}\Z\to \Z/p^k\to 0$.
%This gives rise to a fiber sequence $K(\Z,n)\to K(\Z,n)\to K(\Z/p^k,n)\to 0$.
%You could maybe use the Serre spectral sequence here?
%Another idea that came up was that this is embedded in a long sequence of fibrations, so that you have $K(\Z,n)\to K(\Z,n)\to K(\Z/p^k, n) \to K(\Z,n+1)\to\cdots$.
%
%I'm really excited today -- it's more characteristic classes!
%Last week, I gave you Grothendieck's construction of Chern classes:
%if you have a complex $n$-plane bundle $\xi\downarrow X$, I described $\PP(\xi)$, which has a canonical tautologous line bundle $\lambda_\xi\downarrow \PP(\xi)$.
%This has an Euler class $e=e(\lambda_\xi)\in H^2(\PP(\xi))$, and this satisfies the following unique relation:
%$$
%e^n+c_1e^{n-1} + \cdots + c_n = 0
%$$
%Then $c_k\in H^{2k}(X)$ are called the Chern classes.
%
%This doesn't give you much insight as to how to compute them.
Theorem \ref{chern-classes} claimed that the Chern classes, which we
constructed in the previous section, generate the cohomology of $BU$ as a
polynomial algebra. Our goal in this section is to prove this result.
\subsection{The cohomology of $BU(n)$}
Recall that $BU(n)$ supports the universal principal $U(n)$-bundle $EU(n)\to
BU(n)$. Given any left action of $U(n)$ on some space, we can form the
associated fiber bundle. For instance, the associated bundle of the
$U(n)$-action on $\cC^n$ yields the universal line bundle $\xi_n$.

Likewise, the associated bundle of the action of $U(n)$ on $S^{2n-1}\subseteq
\cc^n$ is the unit sphere bundle $S(\xi_n)$, the unit sphere bundle. By
construction, the fiber of the map $EU(n)\times_{U(n)}S^{2n-1}\to BU(n)$ is
$S^{2n-1}$. Since
$$S^{2n-1} = U(n)/(1\times U(n-1)),$$
we can write
$$EU(n)\times_{U(n)}S^{2n-1} \simeq EU(n)\times_{U(n)} (U(n)/U(n-1)) \simeq
EU(n)/U(n-1) = BU(n-1).$$
In other words, $BU(n-1)$ is the unit sphere bundle of the tautologous line
bundle over $BU(n)$. This begets a fiber bundle:
$$S^{2n-1}\to BU(n-1)\to BU(n),$$
which provides an inductive tool (via the Serre spectral sequence) for
computing the homology of $BU(n)$. In \S \ref{gysin-sequence}, we observed that
the Serre spectral sequence for a spherical fibration was completely described
bythe Gysin sequence.

Recall that if $B$ is oriented and $S^{2n-1}\to E\xar{\pi} B$ is a spherical
bundle over $B$, then the Gysin sequence was a long exact sequence
$$
\cdots\to H^{q-1}(E) \xar{\pi_\ast} H^{q-2n}(B) \xar{e\cdot} H^q(B) \xar{\pi^\ast} H^q(E) \xar{\pi_\ast} \cdots
$$
Let us assume that the cohomology ring of $E$ is polynomial and concentrated in
even dimensions. For the base case of the induction, both these assumptions are
satisfied (since $BU(0) = \ast$ and $BU(1) = \CP^\infty$).

These assumptions imply that if $q$ is even, then the map $\pi_\ast$ is zero.
In particular, multiplication by $e|_{H^\mathrm{even}(B)}$ (which we will also
denote by $e$) is injective, i.e., $e$ is a nonzero divisor.  Similarly, if $q$
is odd, then $e\cdot H^{q-2n}(B) = H^q(B)$. But if $q=1$, then $H^{q-2n}(B) =
0$; by induction on $q$, we find that $H^\mathrm{odd}(B) = 0$. Therefore, if
$q$ is even, then $H^{q-2n+1}(B) = 0$. This implies that there is a short exact
sequence
\begin{equation}\label{inductive-step-cohomology}
    0\to H^\ast(B) \xar{e\cdot} H^\ast(B) \to H^\ast(E) \to 0.
\end{equation}
In particular, the cohomology of $E$ is the cohomology of $B$ quotiented by the
ideal generated by the nonzero divisor $e$.

For instance, when $n=1$, then $B=\CP^\infty$ and $E\simeq \ast$. We have the
canonical generator $e\in H^2(\CP^\infty)$; these deductions tell us the
well-known fact that $H^\ast(\CP^\infty) \simeq \Z[e]$.

Consider the surjection $H^\ast(B) \xar{\pi^\ast} H^\ast(E)$. Since $H^\ast(E)$
is polynomial, we can lift the generators of $H^\ast(E)$ to elements of
$H^\ast(B)$. This begets a splitting $s:H^\ast(E) \to H^\ast(B)$. The existence
of the Euler class $e\in H^\ast(B)$ therefore gives a map $H^\ast(E)[e]
\xar{\overline{s}} H^\ast(B)$. We claim that this map is an isomorphism.

This is a standard algebraic argument. Filter both sides by powers of $e$,
i.e., take the $e$-adic filtration on $H^\ast(E)[e]$ and $H^\ast(B)$. Clearly,
the associated graded of $H^\ast(E)[e]$ just consists of an infinite direct sum
of the cohomology of $E$. The associated graded of $H^\ast(B)$ is the same,
thanks to the short exact sequence \eqref{inductive-step-cohomology}. Thus the
induced map on the associated graded $\gr^\ast(\overline{s})$ is an
isomorphism. In this particular case (but not in general), we can conclude that
$\overline{s}$ is an isomorphism: in any single dimension, the filtration is
finite. Thus, using the five lemma over and over again, we see that the map
$\overline{s}$ an isomorphism on each filtered piece. This implies that
$\overline{s}$ itself is an isomorphism, as desired.

This argument proves that
$$H^\ast(BU(n-1)) = \Z[c_1,\cdots,c_{n-1}].$$
In particular, there is a map $\pi^\ast:H^\ast(BU(n)) \to H^\ast(BU(n-1))$
which an isomorphism in dimensions at most $2n$. Thus, the generators $c_i$
have \emph{unique} lifts to $H^\ast(BU(n))$. We therefore get:
\begin{theorem}
    There exist classes $c_i\in H^{2i}(BU(n))$ for $1\leq i\leq n$ such that:
    \begin{itemize}
	\item the canonical map $H^\ast(BU(n)) \xar{\pi_\ast} H^\ast(BU(n-1))$
	    sends
	    $$
	    c_i \mapsto \begin{cases}
		c_i & i<n\\
		0 & i=n,\text{ and }
	    \end{cases}
	    $$
	\item $c_n := (-1)^n e\in H^{2n}(BU(n))$.
    \end{itemize}
    Moreover,
    $$\boxed{H^\ast(BU(n)) \simeq \Z[c_1,\cdots,c_n]}.$$
\end{theorem}
\subsection{The splitting principle}
\begin{theorem}\label{splitting-principle}
    Let $\xi\downarrow X$ be an $n$-plane bundle. Then there exists a space
    $\Fl(\xi) \xar{\pi} X$ such that:
    \begin{enumerate}
	\item $\pi^\ast \xi = \lambda_1\oplus\cdots\lambda_n$, where the
	    $\lambda_i$ are line bundles on $Y$, and
	\item the map $\pi^\ast: H^\ast(X) \to H^\ast(\Fl(\xi))$ is monic.
    \end{enumerate}
\end{theorem}
\begin{proof}
    We have already (somewhat) studied this space. Recall that there is a
    vector bundle $\pi:\PP(\xi)\to X$ such that
    $$H^\ast(\PP(\xi)) = H^\ast(X)\langle 1,e,\cdots,e^{n-1}\rangle.$$
    Moreover, in \S \ref{grothendieck-chern}, we proved that there is a complex
    line bundle over $\PP(\xi)$ which is a subbundle of $\pi^\ast\xi$. In other
    words, $\pi^\ast\xi$ splits as a sum of a line bundle and some other bundle
    (by Corollary \ref{split}). Iterating this construction proves the
    existence of $\Fl(\xi)$.
\end{proof}
This proof does not give much insight into the structure of $\Fl(\xi)$.
Remember that the \emph{frame bundle} $\Fr(\xi)$ of $\xi$: an element of
$\Fr(\xi)$ is a linear, inner-product preserving map $\cC^n\to E(\xi)$.
This satisfies various properties; for instance:
$$E(\xi) = \Fr(\xi)\times_{U(n)}\cC^n.$$
Moreover,
$$\PP(\xi) = \Fr(\xi)\times_{U(n)} U(n)/(1\times U(n-1)).$$

The \emph{flag bundle} $\Fl(\xi)$ is defined to be
$$\Fl(\xi) = \Fr(\xi)\times_{U(n)} U(n)/(U(1)\times\cdots\times U(1)).$$
The product $U(1)\times\cdots\times U(1)$ is usually denoted $T^n$, since it is
the maximal torus in $U(n)$. For the universal bundle $\xi_n\downarrow BU(n)$,
the frame bundle is exactly $EU(n)$; therefore, $\Fl(\xi_n)$ is just the bundle
given by $BT^n\to BU(n)$. By construction, the fiber of this bundle is
$U(n)/T^n$. In particular, there is a monomorphism $H^\ast(BU(n))
\hookrightarrow H^\ast(BT^n)$. The cohomology of $BT^n$ is extremely simple ---
it is the cohomology of a product of $\CP^\infty$'s, so
$$H^\ast(BT^n) \simeq \Z[t_1,\cdots,t_n],$$
where $|t_k| = 2$. The $t_i$ are the Euler classes of $\pi_i^\ast\lambda_i$,
under the projection map $\pi_i:BT^n\to\CP^\infty$.


\section{Linear algebra}

Like in many methods which solve the Schr\"odinger equation, linear algebra plays a critical role in QMC algorithms and thus is crucial to the performance of QMCPACK.
There are a few components in QMCPACK use BLAS/LAPACK with their own characteristics.

\subsection{Real space QMC}
\subsubsection{Single particle orbitals}
Spline evaluation as commonly used in solid-state simulations does not use any dense linear algebra library calls.
LCAO evaluation as commonly used in molecular calculations relies on BLAS2 GEMV to compute SPOs from a basis set.

\subsubsection{Slater determinants}
Slater determinants are calculated on $N \times N$ Slater matrices. $N$ is the number of electrons for a given spin.
In the actually implementation, operations on the inverse matrix of Slater matrix for each walker dominate the computation.
To initialize it, DGETRF and DGETRI from LAPACK are called. The inverse matrix can be stored out of place.
During random walking, inverse matrices are updated by either Sherman-Morrison rank-1 update or delayed update.
Update algorithms heavily relies on BLAS. All the BLAS operations require S,C,D,Z cases.

Sherman-Morrison rank-1 update uses BLAS2 GEMV and GER on $N \times N$ matrices.

Delayed rank-K update uses
\begin{itemize}
  \item BLAS1 SCOPY on $N$ array.
  \item BLAS2 GEMV, GER on $k \times N$ and $k \times k$ matrices. $k$ ranges from 1 to $K$ when updates are delayed and accumulated.
  \item BLAS3 GEMM at the final update.
    \begin{itemize}
       \item 'T', 'N', K, N, N
       \item 'N', 'N', N, K, K
       \item 'N', 'N', N, N, K
    \end{itemize}
\end{itemize}
The optimal K depends on the hardware but it usually ranges from 32 to 256.

QMCPACK solves systems with a few to thousands of electrons. To make all the BLAS/LAPACK operation efficient on accelerators.
Batching is needed and optimized for $N < 2000$. Non-batched functions needs to be optimized for $N > 500$.
Note: 2000 and 500 are only rough estimates.

\subsubsection{Wavefunction optimizer}
to be added.

\subsection{Auxiliary field QMC}

The AFQMC implementation in QMCPACK relies heavily on linear algebra operations from BLAS/LAPACK. The performance of the code is netirely dependent on the performance of these libraries. See below for a detailed list of the main routines used from BLAS/LAPACK. Since the AFQMC code can work with both single and double precision builds, all 4 versions of these routines (S,C,D,Z) are generally needed, for this reason we omit the data type label.

\begin{itemize}
 \item BLAS1: SCAL, COPY, DOT, AXPY
 \item BLAS2: GEMV, GER
 \item BLAS3: GEMM
 \item LAPACK: GETRF, GETRI, GELQF, UNGLQ, ORGLQ, GESVD, HEEVR, HEGVX
\end{itemize}

While the dimensions of the matrix operations will depend entirely on the details of the calculation, typical matrix dimensions range from the 100s, for small system sizes, to over 20000 for the largest calculations attempted so far.
For builds with GPU accelerators, we make use of batched and strided implementations of these routines. Batched implementations of GEMM, GETRF, GETRI, GELQF and UNGLQ are particularly important for the performance of the GPU build on small to medium size problems. Batched implementations of DOT, AXPY and GEMV would also be quite useful, but they are not yet generally available.
On GPU builds, the code uses batched implementations of these routines when available by default.


This chapter contains little more advanced but still short example programs. Wonder how these programs work? Go figure!

\section{Quick Sort}

Prerequisites: Array, Function, Recursion, Closure

\begin{lstlisting}
10 QSORT = [XS] ~> IF LEN(XS) < 1 THEN NIL ELSE
    QSORT(FILTER([X] ~> X <  HEAD XS, TAIL XS)) # HEAD(XS)!NIL #
    QSORT(FILTER([X] ~> X >=HEAD XS, TAIL XS))
100 L={7,9,4,5,2,3,1,8,6}
110 PRINT L
120 PRINT QSORT(L)
\end{lstlisting}

\section{Fast Fibonacci Sequence}

Prerequisites: Array, Function, Recursion, Closure, Monad

\begin{lstlisting}
10 FIB_=[N,M] ~> IF LEN(MJOIN(M)) >= N THEN HEAD(MJOIN(M)) ELSE
    FIB_(N,M >>= ([XS] ~> MRET((XS(0) + XS(1)) ! XS)))
11 FIB = [N] ~> FIB_(N, MRET({1,1}))
100 FOR K = 1 TO 10
110 PRINT FIB(K);" ";
120 NEXT
\end{lstlisting}

\section{Count The Length of Chunks}

Prerequisites: Array, Function, Recursion

\begin{lstlisting}
10 DEFUN F(STR,XS) = IF (XS == UNDEFINED) THEN F(STR, {})
   ELSE IF (LEN(STR) == 0) THEN XS
   ELSE F(RIGHT(STR, LEN(STR) - 1),
      IF (LEN(XS) == 0) THEN XS ~ {LEFT(STR, 1), 1}
      ELSE IF (XS(LEN(XS) - 1, 0) >< LEFT(STR, 1)) THEN XS ~ {LEFT(STR, 1), 1}
      ELSE INIT(XS) ~ {LEFT(STR, 1), 1 + XS(LEN(XS) - 1, 1)}
   )
100 PAIRS = F("aaaabbbccaxyzzy")
110 PRINT PAIRS
\end{lstlisting}


\documentclass[./Thesis.tex]{subfiles}
\begin{document}

\chapter{Primality}
\label{chap:primality}

\epigraph{
  3 is prime, 5 is prime, 7 is prime. By induction, all the odd integers are
  prime.
}{\cite{leroy}}
In this chapter we will explore various proof relevant definitions of primality.
Then we will construct a decision procedure for these definitions via the
standard brute force primality algorithm.
\begin{code}[hide]
  module Primality where
  open import AKS.Nat using (ℕ; zero; suc; _+_; _∸_; _*_; _≤_; _<_; lte; _<?_; _≟_)
  open import AKS.Nat using (auto-≤; <⇒≱; n≮n; ≤-<-trans; m≤m+n; ≢⇒¬≟; suc-injective-≤; ≤-refl) renaming (n≤m⇒n<m⊎n≡m to n≤m⇒n<m∨n≡m)
  open import AKS.Nat using (<-irrefl; <-cmp; <-trans; n<1+n; 0<1+n)
  open import AKS.Nat using (Acc; acc; <-well-founded; Interval; [_,_∣_]; ⊂-well-founded; _⊂_; upward)
  open import Data.Nat.Properties using (*-identityˡ; *-comm; *-assoc; *-zeroʳ; *-distribʳ-∸; m+n∸n≡m)
  open import AKS.Nat.Divisibility using (_/_; Euclidean✓; _div_)
  open import Relation.Nullary using (¬_)
  open import Relation.Nullary.Decidable using (False; True; from-yes; from-no)
  open import Relation.Nullary.Negation using (contradiction)
  open import Relation.Binary using (Reflexive; Transitive; Tri)
  open Tri
  open import Relation.Binary.PropositionalEquality
    using (_≡_; _≢_; module ≡-Reasoning)
    renaming (refl to ≡-refl; sym to ≡-sym; cong to ≡-cong; cong₂ to ≡-cong₂)
  open import Data.Empty using (⊥; ⊥-elim)
  open import Data.Unit using (tt)
  open import Data.Sum using (inj₁; inj₂)
  open import Function using (_$_) -- $
  open ≡-Reasoning
\end{code}
\section{Divisibility}
\label{sec:divisibility}
Before we can define primality we require a definition of divisibility. We
follow a standard definition of divisibility, namely a natural number $d$
divides into another natural $a$ iff there exists some $q$ such that $a = q d$.
\begin{code}[hide]
  infix 3 _∣_
\end{code}
\begin{code}
  record _∣_ (d : ℕ) (a : ℕ) : Set where
    constructor divides
    field
      quotient : ℕ
      equality : a ≡ quotient * d

  _∤_ : ℕ → ℕ → Set
  d ∤ a = ¬ (d ∣ a)
\end{code}
In order to make a decision procedure for primality we first require a decision
procedure for divisibility. This would have to be a proof relevant algorithm. An
algorithm that would take two natural numbers and produce either a proof of
divisibility or a proof of non-divisibility. This is no small task, one which
will become easier with the correct lemmas. We first prove that the divisibility
relation is reflexive, that is that for any $n$ the expression
$n \, \AgdaFunction{∣} \, n$ is true. After which we will prove that the
divisibility relation is transitive.
\begin{code}
  ∣-refl : Reflexive _∣_
  ∣-refl {n} = divides 1 $ begin n ≡⟨ ≡-sym (*-identityˡ n) ⟩ 1 * n ∎
\end{code} % $
\begin{code}
  ∣-trans : Transitive _∣_
  ∣-trans {x} {y} {z} (divides q₁ y≡q₁*x) (divides q₂ z≡q₂*y) = divides (q₂ * q₁) $
    z ≡⟨ z≡q₂*y ⟩
    q₂ * y ≡⟨ ≡-cong (λ t → q₂ * t) y≡q₁*x ⟩
    q₂ * (q₁ * x) ≡⟨ ≡-sym (*-assoc q₂ q₁ x) ⟩
    (q₂ * q₁) * x ∎
\end{code} % $
As \Agda{} is a proof relevant logic, every proof of divisibility materializes
similarly. The user simply specifies what number to multiply the divisor by to
get the dividend. Then they supply a proof that the two numbers do indeed multiply
to the dividend. It the following proof we assume a divisibility relation is
true. We ``consume'' the divisibility relation.
Specifically we consider when $0$ divides another number. The only number $0$
divides is $0$ itself.
\begin{code}
  0∣n⇒n≡0 : ∀ {n} → 0 ∣ n → n ≡ 0
  0∣n⇒n≡0 {n} (divides q n≡q*0) = begin
    n ≡⟨ n≡q*0 ⟩
    q * 0 ≡⟨ *-zeroʳ q ⟩
    0 ∎
\end{code}
The following two lemmas have simple proofs, so we leave the proofs to the
reader. The first lemma shows that the divisibility relation respects equality
in its second argument and the second lemma shows that some number $m$ will
divide a number if it has $m$ as a factor.
\begin{code}
  ∣-respʳ : ∀ {i n m} → n ≡ m → i ∣ n → i ∣ m

  m∣m*n : ∀ {n m} → m ∣ m * n
\end{code}
\begin{code}[hide]
  ∣-respʳ {i} {n} {m} n≡m (divides q n≡q*i) =
    divides q $ begin
      m ≡⟨ ≡-sym n≡m ⟩
      n ≡⟨ n≡q*i ⟩
      q * i ∎

  m∣m*n {n} {m} = divides n $ begin m * n ≡⟨ *-comm m n ⟩ n * m ∎
\end{code}
This proof shows is the ``reverse'' of the idea that if a number $i$ divides two
numbers $m$ and $n$ then it must divide their sum. This because $i$ is a common
factor of the two numbers. If $n + m = q_1 i$ and $m = q_2 i$ then it follows
that $n = (q_1 - q_2) * i$. Note that $q_1 \geq q_2$ as $q_1$ contains $q_2$. This
is important in the following \Agda{} proof as the type $\AgdaDatatype{ℕ}$ does
not contain negative numbers. There is a subtraction like function commonly
called ``monus'' that returns $0$ when the difference is negative. This is
denoted as $\AgdaFunction{∸}$ in \Agda{} and it behaves similarly to subtraction in many
proofs.
\begin{code}
  ∣n+m∣m⇒∣n : ∀ {i n m} → i ∣ n + m → i ∣ m → i ∣ n
  ∣n+m∣m⇒∣n {i} {n} {m} (divides q₁ n+m≡q₁*i) (divides q₂ m≡q₂*i) =
    divides (q₁ ∸ q₂) $ begin
      n               ≡⟨ ≡-sym (m+n∸n≡m n m) ⟩
      (n + m) ∸ m     ≡⟨ ≡-cong₂ (λ x y → x ∸ y) n+m≡q₁*i m≡q₂*i ⟩
      q₁ * i ∸ q₂ * i ≡⟨ ≡-sym (*-distribʳ-∸ i q₁ q₂) ⟩
      (q₁ ∸ q₂) * i   ∎
\end{code} % $
This final lemma reduces many proofs about divisibility into proofs about
inequalities. This lemma occurs multiple times in the following sections and
chapters. The key idea is that if $n$ divides $m$ then it must be less than or
equal to $m$ in order to be a factor. Technically this is false when $m = 0$ as
every number divides $0$. So we just assume $m \neq 0$. This proof illustrates
the importance of choosing definitions correctly. Using the definition
$\exists k. \, n + k = m$ of $n \leq m$ makes this proof almost trivial. An
inductive definition of $n \leq m$ would complicate this proof by a large measure.
\begin{code}
  ∣⇒≤ : ∀ {n m} → m ≢ 0 → n ∣ m → n ≤ m
  ∣⇒≤ {n} {zero} m≢0 (divides q m≡q*n) = contradiction ≡-refl m≢0
  ∣⇒≤ {n} {suc m} m≢0 (divides (suc q) m≡[1+q]*n) =
    lte (q * n) $ begin
      n + q * n ≡⟨⟩
      suc q * n ≡⟨ ≡-sym m≡[1+q]*n ⟩
      suc m     ∎
\end{code} % $
With the lemmas out of the way we have one last bookkeeping task. What is the
correct type of decision procedures? A proof relevant decision procedure should
return not just the truth value of the predicate the user is testing. It should
also return a proof that the predicate is true or a proof that the predicate is
false. These cases are mutually exclusive so we can represent this idea with the
following datatype. Note this type can be used generically in decision producers
over any proposition $P$.
\begin{code}[hide]
  module Decidable where
\end{code}
\begin{code}
    data Dec (P : Set) : Set where
      yes :   P → Dec P
      no  : ¬ P → Dec P
\end{code}
\begin{code}[hide]
  open import Relation.Nullary using (Dec; yes; no)
\end{code}
The full decision procedure begins by checking if the divisor is $0$. If the
divisor is $0$, then we ensure the dividend is $0$ as $0$ only divides $0$.
\begin{code}
  _∣?_ : ∀ (d a : ℕ) → Dec (d ∣ a)
  d ∣? a with d ≟ 0
  d ∣? a | yes ≡-refl with a ≟ 0
  d ∣? a | yes ≡-refl | yes ≡-refl = yes ∣-refl
  d ∣? a | yes ≡-refl | no a≢0 = no λ 0∣a → contradiction (0∣n⇒n≡0 0∣a) a≢0
\end{code}
If the divisor is not $0$, then we can divide the dividend by the divisor. If
the integer remainder of that division is $0$, then the divisor divides the
dividend. Otherwise the divisor can not divide the dividend as the remainder is
strictly less than the divisor.
\begin{code}
  d ∣? a | no d≢0 with (a div d) {≢⇒¬≟ d≢0}
  d ∣? a | no d≢0 | Euclidean✓ q r pf r<d with r ≟ 0
  d ∣? a | no d≢0 | Euclidean✓ q r pf r<d | yes ≡-refl =
    yes $ divides q $ begin
      a ≡⟨ pf ⟩
      0 + d * q ≡⟨⟩
      d * q ≡⟨ *-comm d q ⟩
      q * d ∎
\end{code}
\begin{code}
  d ∣? a | no d≢0 | Euclidean✓ q r pf r<d | no r≢0 = no ¬d∣a
\end{code}
\begin{code}[hide]
    where
\end{code}
In this $\AgdaInductiveConstructor{no}$ case we must prove that the divisor
dividing the dividend leads to a contradiction. All proofs to a contradiction
start with assuming truth, so we assume that the divisor does divide the
dividend $d \, \AgdaFunction{∣} \, a$. We show this contradiction by producing a
proof that $d \leq r$.
\begin{code}
    ¬d∣a : d ∣ a → ⊥
    ¬d∣a d∣a = contradiction d≤r (<⇒≱ r<d)
\end{code}
\begin{code}[hide]
      where
\end{code}
Our division algorithm returns a proof of correctness namely that $a = r + dq$.
We substitute $a$ with $r + dq$ in our assumption of $d \mid a$. Then we remove
the $d * q$ term from the sum as $d$ trivially divides it. Lastly we transform
our divisibility relation into an inequality as the remainder is not zero.
This illustrates the contradiction and the decision procedure is complete.
\begin{code}
      d∣r+d*q : d ∣ r + d * q
      d∣r+d*q = ∣-respʳ pf d∣a

      d∣r : d ∣ r
      d∣r = ∣n+m∣m⇒∣n d∣r+d*q m∣m*n

      d≤r : d ≤ r
      d≤r = ∣⇒≤ r≢0 d∣r
\end{code}
\section{Investigating Definitions}
\label{sec:investigating-definitions}
There are many isomorphic definitions of primality. Unfortunately not all
definitions are equal in the eyes of \Agda{}. For instance, consider the
following definition, ``A prime number is a natural number greater than $1$ that
has no prime divisors smaller then itself.''. This definition translates
directly into \Agda{} as shown below.
\begin{code}[hide]
  module Primality₁ where
    {-# NO_POSITIVITY_CHECK #-}
\end{code}
\begin{code}
    record IsPrime (p : ℕ) : Set where
      inductive
      constructor IsPrime✓
      field
        1<p : 1 < p
        ∀i[i∤p] : ∀ {i} → i < p → IsPrime i → i ∤ p
\end{code}
This definition has one critical flaw, it is refused by \Agda{}. The type is
recursively defined but the recursion is in \textit{negative} position
\cite{harper}. This means the recursion occurs in the domain of a function type.
Allowing recursive calls in negative position makes your logic inconsistent.
Consider the following code and note that $\AgdaDatatype{Bad}$ has a recursive
call in negative position.
\begin{code}[hide]
  {-# NO_POSITIVITY_CHECK #-}
\end{code}
\begin{code}
  data Bad : Set where
    Bad✓ : (Bad → ⊥) → Bad

  Bad-false : Bad → ⊥
  Bad-false (Bad✓ b) = b (Bad✓ b)

  Bad-true : Bad
  Bad-true = Bad✓ Bad-false

  false : ⊥
  false = Bad-false Bad-true
\end{code}
Note how similar $\AgdaFunction{false}$ is to the non-terminating
\textit{omega combinator} shown in \ref{eqn:omega}. The only difference is the required
wrapping and unwrapping of $\AgdaDatatype{Bad}$ values. Unlike the type
$\AgdaDatatype{Bad}$, the type $\AgdaDatatype{IsPrime}$ given above is actually a safe
use of negative recursion. Convincing \Agda{} of this fact is quite hard and
outside the scope of this thesis.
\begin{align}
  \label{eqn:omega}
  \Omega = (\lambda x. \, x \, x) (\lambda x. \, x \, x)
\end{align}
We can remove the recursion by considering when a number can divide a prime. If
some number divides a prime then that number is either $1$ or the prime itself.
This leads to a non-recursive definition of primality. A natural number is prime
when it is greater than $1$ and its only divisors are $1$ and itself.
\begin{code}[hide]
  module Primality₂ where
    infixr 3 _or_
    data _or_ (A : Set) (B : Set) : Set where
\end{code}
\begin{code}
    record IsPrime (p : ℕ) : Set where
      constructor IsPrime✓
      field
        1<p : 1 < p
        ∀i∣p[i≡p] : ∀ {i} → i ∣ p → i ≡ 1 or i ≡ p
\end{code}
The definition above is usable, but in the following sections most of the proofs
will know $1 < i$ for relevant instantiations of $i$. Thus the definition
generates many superfluous cases where the user must show $i = 1$ is impossible.
So we bake the fact that $i$ must be greater than $1$ into the definition. This
yields our final definition.
\begin{code}
  record IsPrime (p : ℕ) : Set where
    constructor IsPrime✓
    field
      1<p : 1 < p
      ∀i∣p[i≡p] : ∀ {i} → 1 < i → i ∣ p → i ≡ p
\end{code}
Using this definition we can prove specific natural numbers are prime by hand.
\begin{code}
  3-isPrime : IsPrime 3
  3-isPrime = IsPrime✓ (from-yes (1 <? 3)) 3-primality
\end{code}
\begin{code}[hide]
    where
\end{code}
In order to prove $3$ is a prime we need to show that any divisor $i$ of $3$ is
equal to $3$. To accomplish this we perform a case analysis on $i$. \Agda{} can
infer that $i = 0, 1$ are impossible cases so we do not have to fill them out.
The case $i = 2$ is impossible as $2 \nmid 3$ and we can illustrate this fact by
running the decision procedure for divisibility on $2$ and $3$. Finally any
input of the form $4 + i$ will not divide $3$ as it will be too large.
\begin{code}
    3-primality : ∀ {i} → 1 < i → i ∣ 3 → i ≡ 3
    3-primality {suc (suc zero)} _ 2∣3 = contradiction 2∣3 (from-no (2 ∣? 3))
    3-primality {suc (suc (suc zero))} _ i∣3 = ≡-refl
    3-primality {suc (suc (suc (suc i)))} _ i∣3 = contradiction 3<3 n≮n
\end{code}
\begin{code}[hide]
      where
\end{code}
\begin{code}
      3+i<3 : 3 + i < 3
      3+i<3 = ∣⇒≤ (λ ()) i∣3

      3<3 : 3 < 3
      3<3 = ≤-<-trans m≤m+n 3+i<3
\end{code}
This rote work does outline a decision procedure for primality of a number $n$. Test every
number between $2$ and $n - 1$ for divisibility with $n$. If any of the numbers
divide $n$ return false, but what does falsehood represent in this instance?
In this case it implies that the number is a
composite. So we define another predicate $\AgdaDatatype{IsComposite}$. A number
is composite if it has a smaller prime factor. Note every composite number is
guaranteed by the fundamental theorem of arithmetic to have a a prime factor.
\begin{code}
  record IsComposite (c : ℕ) : Set where
    constructor IsComposite✓
    field
      p : ℕ
      p<c : p < c
      p-isPrime : IsPrime p
      p∣c : p ∣ c
\end{code}
Notice how proving compositionality reduces to proving primality.
\begin{code}
  6-isComposite : IsComposite 6
  6-isComposite = IsComposite✓ 3 (from-yes (3 <? 6)) 3-isPrime (from-yes (3 ∣? 6))
\end{code}
\section{A Decision Procedure}
\label{sec:a-decision-procedure}
As always we require a couple lemmas before we can get into the heart of the
decision procedure. First there is no prime less than $2$, this fact is almost
baked into the definition. Second we prove that a number $n$ can not be both prime and
composite. As the number is composite it has some prime factor, but as the
number is prime that divisor must be equal to itself. The factor is defined to
be less than itself and we reach a contradiction.
\begin{code}
  ¬prime<2 : ∀ p → p < 2 → ¬ (IsPrime p)
  ¬prime<2 p p<2 (IsPrime✓ 2≤p _) = contradiction 2≤p (<⇒≱ p<2)
\end{code}
\begin{code}
  exclusive : ∀ {n} → IsPrime n → IsComposite n → ⊥
  exclusive {n}
    (IsPrime✓ 1<n ∀i∣n⇒i≡n)
    (IsComposite✓ p p<n (IsPrime✓ 1<p _) p∣n)
    = contradiction p<n p≮n
\end{code}
\begin{code}[hide]
    where
\end{code}
\begin{code}
    p≮n : ¬ (p < n)
    p≮n = <-irrefl (∀i∣n⇒i≡n 1<p p∣n)
\end{code}
As we are working in a proof relevant setting, the types
$\AgdaDatatype{IsComposite} \, n$ and
$\AgdaFunction{¬} (\AgdaDatatype{IsPrime}\, n)$ do not carry the same amount of
information. The first type knows exactly one factor of $n$ while the
second type knows no factors. Therefore instead of writing a decision procedure
for prime or not prime, we write a decision procedure for prime or composite. The
later can be easily transformed into the former as will be shown at the end of the
chapter. Therefore we construct a type that can return a proof of primality or
compositionality.
\begin{code}
  data Primality (n : ℕ) : Set where
    Composite✓ : IsComposite n → Primality n
    Prime✓ : IsPrime n → Primality n
\end{code}
Unfortunately our final $\AgdaDatatype{IsPrime}$ type is difficult to construct.
This is because the type has ``forgotten'' that every number less than itself
can not divide itself. Luckily the original inconsistent definition of primality
remembers the divisibility of all numbers less than itself. We can break the
negative recursion by using the current definition of primality in the negative
position. We create a new return type that has this definition inlined into
itself. Thus the algorithm proceeds as follows. We test all prime divisors of
$n$ and store them in $\AgdaDatatype{Compositionality}$ then we convert a
$\AgdaDatatype{Compositionality}$ into a $\AgdaDatatype{Primality}$.
\begin{code}
  data Compositionality (n : ℕ) : Set where
    Composite✓ : IsComposite n → Compositionality n
    Prime✓ : (∀ {p} → p < n → IsPrime p → p ∤ n) → Compositionality n
\end{code}
In practice this algorithm is actually mutually recursive as in order to generate
a $\AgdaDatatype{Compositionality}$ you need to generate smaller
$\AgdaDatatype{IsPrime}$ proofs.
\begin{code}
  compositionality : ∀ n → 1 < n → Acc _<_ n → Compositionality n
  primality : ∀ n → 1 < n → Acc _<_ n → Primality n
\end{code}
\begin{code}
  compositionality n 1<n (acc downward)
    = loop 2 (from-yes (1 <? 2)) 1<n ⊂-well-founded ¬p<2[p∤n]
\end{code}
\begin{code}[hide]
    where
\end{code}
We store these smaller primes in a list encoded with continuation passing style.
The following code is an analog to the nil list case. As there are no primes
less than $2$, this makes a good empty list.
\begin{code}
    ¬p<2[p∤n] : ∀ {p} → p < 2 → IsPrime p → p ∤ n
    ¬p<2[p∤n] {p} p<2 p-isPrime _ = contradiction p-isPrime (¬prime<2 p p<2)
\end{code}
Next we need to construct an analog to append. If we have a list of prime
divisibility up to but not including $x$ and a proof that $x$ does not
divide $n$. Then we can construct a list of prime divisibility including $x$. This
proof proceeds by comparing $p$ and $x$ using a trichotomous comparison.
\begin{code}
    cons
      : ∀ {x}
      → (IsPrime x → x ∤ n)
      → (∀ {p} → p < x → IsPrime p → p ∤ n)
      → (∀ {p} → p < 1 + x → IsPrime p → p ∤ n)
    cons {x} x-isPrime⇒x∤n ∀p<x⇒p∤n {p} p<1+x p-isPrime p∣n with <-cmp p x
    ... | tri< p<x _ _ = contradiction p∣n (∀p<x⇒p∤n p<x p-isPrime)
    ... | tri≈ _ ≡-refl _ = contradiction p∣n (x-isPrime⇒x∤n p-isPrime)
    ... | tri> _ _ x<p = contradiction (suc-injective-≤ p<1+x) (<⇒≱ x<p)
\end{code}
Following this proof, we construct the main divisibility testing loop. We count upwards from
$x = 2$
until we reach $n$ using the accessibility predicate defined at the end of chapter
\ref{chap:termination}. We recursively test the primality of $x$. If $x$ is
composite then there is no reason to test divisibility as the prime factor of
$x$ will have been tested previously in the loop. If $x$ is prime then we test if
$x$ divides $n$ using the divisibility decision procedure defined earlier in the
chapter. If $x$ divides $n$ then we immediately return with a proof that $n$ has
a prime divisor. Otherwise we add $x$ into the list and recurse.
\begin{code}
    loop
      : ∀ x → 1 < x → (x≤n : x ≤ n) → Acc _⊂_ [ x , n ∣ x≤n ]
      → (∀ {p} → p < x → IsPrime p → p ∤ n)
      → Compositionality n
    loop x 1<x x≤n (acc next) ∀p<x⇒p∤n with n≤m⇒n<m∨n≡m x≤n
    ... | inj₂ ≡-refl = Prime✓ ∀p<x⇒p∤n
    ... | inj₁ x<n with primality x 1<x (downward x<n)
    ...   | Composite✓ x-isComposite
          = loop (1 + x) (<-trans 1<x n<1+n) x<n (next [1+x,n]⊂[x,n]) ∀p<1+x⇒p∤n
            where
            [1+x,n]⊂[x,n] : [ 1 + x , n ∣ x<n ] ⊂ [ x , n ∣ x≤n ]
            [1+x,n]⊂[x,n] = upward n<1+n ≤-refl
            ∀p<1+x⇒p∤n = cons (λ x-isPrime _ → exclusive x-isPrime x-isComposite) ∀p<x⇒p∤n
    ...   | Prime✓ x-isPrime with x ∣? n
    ...     | yes x∣n = Composite✓ (IsComposite✓ x x<n x-isPrime x∣n)
    ...     | no ¬x∣n = loop (1 + x) (<-trans 1<x n<1+n) x<n (next [1+x,n]⊂[x,n]) ∀p<1+x[p∤n]
            where
            [1+x,n]⊂[x,n] : [ 1 + x , n ∣ x<n ] ⊂ [ x , n ∣ x≤n ]
            [1+x,n]⊂[x,n] = upward n<1+n ≤-refl
            ∀p<1+x[p∤n] = cons (λ _ x∣n → ¬x∣n x∣n) ∀p<x⇒p∤n
\end{code}
Now we transform compositionalities into primalities. If the number is
composite then we can simply return the same proof. Otherwise we need to
transform our continuation passing style list into our final primality condition.
\begin{code}
  primality n 1<n wf@(acc downward) with compositionality n 1<n wf
  ... | Composite✓ isComposite = Composite✓ isComposite
  ... | Prime✓ ∀p<n⇒p∤n = Prime✓ (IsPrime✓ 1<n ∀i∣n⇒i≡n)
\end{code}
\begin{code}[hide]
    where
    n≢0 : n ≢ 0
    n≢0 n≡0 = <-irrefl (≡-sym n≡0) (<-trans 0<1+n 1<n)
\end{code}
To do this we must handle any possible divisor $i$ of $n$. First we
trichotomously compare $i$ and $n$. If $i$ is larger than $n$ we immediately run
into a familiar contradiction. If $i$ is equal to $n$ then we are done and we
return the proof that they are equal, and if $i$ is less than $n$ we
recursively call our decision procedure on $i$. If $i$ is prime then we have
reached a contradiction as we have a list of all the primes less than $n$ and
they do not divide $n$. If $i$ is composite then it has a prime divisor that
transitively divides $n$ and we have reached a contradiction.
\begin{code}
    ∀i∣n⇒i≡n : ∀ {i} → 1 < i → i ∣ n → i ≡ n
    ∀i∣n⇒i≡n {i} 1<i i∣n with <-cmp i n
    ... | tri> _ _ n<i = contradiction (∣⇒≤ n≢0 i∣n) (<⇒≱ n<i)
    ... | tri≈ _ i≡n _ = i≡n
    ... | tri< i<n _ _ with primality i 1<i (downward i<n)
    ...   | Prime✓ i-isPrime
          = contradiction i∣n (∀p<n⇒p∤n i<n i-isPrime)
    ...   | Composite✓ (IsComposite✓ p p<i p-isPrime p∣i)
          = contradiction (∣-trans p∣i i∣n) (∀p<n⇒p∤n (<-trans p<i i<n) p-isPrime)
\end{code}
After ensuring that our number is greater than $1$, we can invoke
$\AgdaFunction{primality}$ and finish our primality decision procedure.
\begin{code}
  prime? : ∀ n → Dec (IsPrime n)
  prime? n with 1 <? n
  ... | no ¬1<n = no λ { (IsPrime✓ 1<n _) → ¬1<n 1<n }
  ... | yes 1<n with primality n 1<n <-well-founded
  ... | Prime✓ isPrime = yes isPrime
  ... | Composite✓ isComposite = no λ { isPrime → exclusive isPrime isComposite }
\end{code}

\end{document}


\section{``Euler's theorem'', and ``homology approximation'' - CTC Wall. I) Singular homology, II) CW complexes, III) ``homological algebra''}
\begin{theorem}[``Euler'']
Let $X$ be a space which admits the structure of a finite CW complex. The sum $\sum_{h=0}^\infty (-1)^k\#(k\text{-cells})$ (generalizes $V-E+F$) is independent of that structure.
\end{theorem}
\begin{proof}
Pick a CW-structure. We have $0\to C_n\to\cdots\to C_2\to C_1\to C_0\to 0$. We also have a sexseq $0\to Z_k\to C_k\to B_{k-1}\to 0$, and another one $0\to B_k\to Z_k\to H_k\to 0$. Let's use them and facts about rank that I talked about on Monday to compute what this alternating sum is. The Euler sum is the same as:
\begin{align*}
\sum_{h=0}^\infty (-1)^k\#(k\text{-cells}) & = \sum^\infty_{k=0}(-1)^k\rank(C_k)\\
& = \sum^\infty_{k=0}(-1)^k\rank(Z_k)+\sum^\infty_{k=0}(-1)^k\rank(B_{k-1})\\
& = \sum^\infty_{k=0}(-1)^k(\rank( H_k)+\rank(B_k)+\rank(B_{k-1}))
\end{align*}
The terms $\rank B_k+\rank B_{k+1}$ telescope because it's an alternating sum, and hence vanish. The sum is $\sum^\infty_{k=0}(-1)^k\rank( H_k)$. But $ H_k(X)= H_k^\text{sing}(X)$ is an invariant of the space, independent of the CW-structure.
\end{proof}
Given $ H_k(X)$, $X$ a finite type CW-complex, what's a lower bound on the number of $k$-cells? Let's see. $ H_k(X)$ is finitely generated because $C_k(X)\subseteq Z_k(X)$ is, and it surjects onto $ H_k(X)$. Thus $ H_k(X)=\bigoplus^{t(k)}_{i=1}\Z/n_i(k)\Z\oplus \Z^{r(k)}$ where the $n_1(k)|\cdots|n_{t(k)}(k)$ are the torsion indices.

The minimal chain complex with $ H_k=\Z$ and $ H_q=0$ for $q\neq k$ is just the chain complex with $0$ everywhere else except for $\Z$ in the $k$th degree. The minimal chain complex with $ H_k=\Z/n\Z$ and $ H_q=0$ for $q\neq k$ is just the chain complex with $0$ everywhere else except for $\Z\xrightarrow{n}\Z$ in dimension $k+1$ to $k$. These things are called elementary chain complexes.

A lower bound on the minimal number of $k$-cells is $r(k)+t(k)+t(k-1)$ where the last term comes for the ``torsion generator in dimension $k-1$'' (didn't catch that).
\begin{theorem}[Wall]
Let $X$ be a simply connected CW-complex of finite type. Then there exists a CW complex $Y$ with $r(k)+t(k)+t(k-1)$ $k$-cells, for all $k$, and a homotopy equivalence $Y\to X$.
\end{theorem}
I'm not going to prove this theorem. You can read Wall's theorem. You really can't ask for more. Oh, also here's a theorem.
\begin{theorem}
Let $X$ be connected and pointed $\ast\in X$. Then $\pi_1(X,\ast)\to H_1(X,\ast)$ exists, called the Hurewicz homomorphism, and it factors as $\pi_1(X,\ast)\to \pi_1(X,\ast)^{ab}\to H_1(X,\ast)$. The last map is an isomorphism.
\end{theorem}
Some examples of Wall's theorem:
\begin{example}
We know that $S^k$ has $\widetilde{ H}_q(X)=\Z$ when $q=k$ and $0$ else. Can you construct a space with $\widetilde{ H}_q(X)=\Z/n\Z$ when $q=k$ and $0$ else? We need to construct a space with the elementary chain complex with $0$ everywhere else except for $\Z\xrightarrow{n}\Z$ in dimension $k+1$ to $k$. You need to have one $0$-cell, do nothing until you get to dimension $k$, which is when you add a $k$-cell, and then use the attaching map $S^k\to S^k$ of degree $n$, i.e.:
\begin{equation*}
\xymatrix{S^k\ar[r]^{\text{degree }n}\ar[d] & S^k\ar[d]\\
D^{k+1}\ar[r] & X}
\end{equation*}
For example, when $k=1$ and $n=2$, you have $\RP^2$. This is called a ``Moore space''.
\end{example}
I brought up doing this in more generality with generators and relations, and Professor Miller built up on that:
\begin{example}
For more general abelian groups, you have a free abelian group $F_0$ sitting in a sexseq $0\to F_1\to F_0\to M\to 0$ (this is an example of a \emph{resolution of $M$}, which is what I'm going to start talking about). Then $F_1$ is also free. Pick some $k>0$. You get a space whose homology is $F_0$, namely $\bigvee_\alpha S^k$, and a space whose homology is $F_1$, namely $\coprod S^k$. You can construct a map $\coprod S^k\to \bigvee_\alpha S^k$ such that the map $\alpha:F_1\to F_0$ is what's induced on homology. Then you get:
\begin{equation*}
\xymatrix{\coprod S^k\ar[r]^{\text{gives }\alpha}\ar[d] & \bigvee_\alpha S^k\ar[d]\\
\coprod D^{k+1}\ar[r] & X}
\end{equation*}
Such an $X$ is called a Moore space, and has homology $M$ in dimension $k$ and zero everywhere else. You can't make this into a functor, i.e., this can't be made into a functor $\mathbf{Ab}\to\mathbf{Top}$.
\end{example}
\subsection{Homological algebra}
You can put coefficients into homology. Let $M$ an abelian group. You can talk about homology with coefficients in $M$. For example, $M=\Z,\QQ,\Z/n\Z,\cdots$. The $\Z/n\Z$ case when $n$ is prime is pretty important because it's then a field.

Given $X$, you get a singular simplicial set $\Sin_\ast(X)$. Then we took the free abelian group $S_\ast$ generated by $\Sin_\ast(X)$. I.e., $S_n=\Z[\Sin_n(X)]=\bigoplus_{\Sin_n(X)}\Z$. But I could replace $\Z$ with anything I wanted, and do the \emph{exact} same construction. I can just as well as put any abelian group here.

Define the ``singular chain complex with coefficients in $M$'' as $S_n(X;M)=\bigoplus_{\Sin_n(X)}M$. There's a boundary map $d:S_n(X;M)\to S_{n-1}(X;M)$. Then the homology $ H(S_\ast(X;M))=: H_\ast(X;M)$. You can verify all the Eilenberg-Steenrod axioms yourself, except for one, namely the dimension axiom - $ H_k(\ast;M)=\begin{cases}M & k=0 \\ 0 & k\neq 0\end{cases}$.

If you think about it, you'll realize that this whole unit in CW-complexes didn't use anything except for the Eilenberg-Steenrod axioms. This shows, by the way, that if you get some weird homology theory satisfying the Eilenberg-Steenrod axioms you get all the same results as if you used what we constructed before.

As an experiment, let's compute $ H_\ast(\RP^n;\Z/2\Z)$. The cellular chain complex is $0\to \Z/2\Z\to\cdots\to\Z/2\Z\to\Z/2\Z\to 0$ where the maps are alternately multiplying by $2$ and $0$. But in this case, all the maps are $0$ because $2=0$! So $ H_k(\RP^n;\Z/2\Z)=\begin{cases}\Z/2\Z & 0\leq k\leq n \\ 0 & \text{else}\end{cases}$. How about $ H_\ast(\RP^n;\QQ)$? Or $ H_\ast(\RP^n;\Z[\frac{1}{p}])$ where $\Z[\frac{1}{p}]\subseteq \QQ$? What about $\Z_{(p)}\subseteq \QQ$ where you've localized at $p$?

Anyway, if I consider $ H_\ast(\RP^n;\Z[\frac{1}{2}])$, then the cellular chain complex simplifies, but in a different way. You have $0\to \Z[\frac{1}{2}] \to\cdots\to \Z[\frac{1}{2}] \to \Z[\frac{1}{2}] \to 0$. Multiplication by $2$, however, is an isomorphism. So, $ H_k(\RP^n;\Z[\frac{1}{2}])=\begin{cases}\Z[\frac{1}{2}] & q=0,n,\, q \text{ odd} \\ 0 & \text{else}\end{cases}$. You get a much simpler result. From this point of view, even projective spaces look like a point, and odd projective spaces look like a sphere!

It's a little awkward to go through this thing. I'd like to understand:
\begin{question}
How is $ H_\ast(X;M)$ related to $ H_\ast(X)= H_\ast(X;\Z)$? This is a reasonable question.
\end{question}
The answer is called the ``universal coefficient theorem''. I'll spend a few days developing what we need to talk about this.

I want to talk about tensor products. Let me take a poll. Do you know tensor products? Working with a commutative ring instead of $\Z$? Actually, all of these examples $M=\Z,\QQ,\Z/n\Z,\Z[\frac{1}{p}]\subseteq\QQ\supseteq\Z_{(p)},\cdots$ are rings. The boundary map $d:S_n(X;M)\to S_{n-1}(X;M)$ is a module homomorphism if $M=R$ is a (\emph{always commutative}) ring.

This means that if $R$ is a commutative ring, then $ H_\ast(X;R)$ is an $R$-module. If $R$ is a ring and $M$ is an $R$-module, then $ H_\ast(X;M)$ is an $R$-module. Just look at what you have here. The $\bigoplus_{\Sin_n(X)}M$ is an $R$-module, and $d$ is an $R$-module homomorphism.

I'll admit, this is a little bit scary, because commutative rings are pretty complicated in general. I won't talk about some weirdo rings, though. I'll develop this more on Friday. Let me pass out homework.


\documentclass[main.tex]{subfiles}
\begin{document}

\section{December exercises}

\subsection{Exercise 10}

We have a time series of \(N\) data points, \(D = \qty{d_i}\), corresponding to the times \(t_i\), which are separated by the constant spacing \(\Delta \).

We model them as
%
\begin{align}
d_i = \underbrace{B_1  \cos(\omega t_i) + B_2 \sin(\omega t_i)}_{f(t_i)} + n_i
\,,
\end{align}
%
where \(f(t)\) the signal we want to characterize, which depends on the unknown amplitudes \(B_1 \) and \(B_2 \) and the unknown frequency \(\omega \); while \(n_i \) is the noise: each \(n_i\) is i.i.d.\ as a zero-mean Gaussian with known variance \(\sigma^2\).

\begin{figure}[ht]
\centering
\includegraphics[width=\textwidth]{figures/signal.pdf}
\caption{Example of a portion of the signal the model assumes; the true signal should be longer than a few periods, and it is taken to be 20 times as long as this in the numerical examples which follow. The parameters are \(B_1 = 1/2\), \(B_2 = \sqrt{3} /2\), \(\sigma = 0.2\), \(\Delta = 1\), \(\omega = 0.015\).}
\label{fig:signal}
\end{figure}

\subsubsection{The full likelihood}

The likelihood of a single datum of index \(i\) attaining the value \(d_i\) is given\footnote{Omitting the dependence on previous information for simplicity.} by
%
\begin{align}
\mathscr{L} (d_i | \omega , B_1 , B_2 ) = \frac{1}{\sqrt{2 \pi } \sigma }\exp(- \frac{1}{2 \sigma^2}\qty(d_i - f(t_i))^2)
\,.
\end{align}

Now, since the noise at each point is independent, the full likelihood is the product of the likelihoods of each datum:
%
\begin{align}
\mathscr{L}(D | \omega , B_1 , B_2 ) &= \frac{1}{(\sqrt{2 \pi } \sigma )^{N}} \prod_{i=1}^{N} \exp(- \frac{1}{2 \sigma^2} \qty(d_i - f(t_i))^2)  \\
&= \frac{1}{(\sqrt{2 \pi } \sigma )^{N}}
\exp(- \frac{1}{2 \sigma^2} \sum_{i=1}^{N} \qty(d_i - f(t_i))^2) \\
&= \frac{1}{(\sqrt{2 \pi } \sigma )^{N}}
\exp(- \frac{1}{2 \sigma^2} \underbrace{\sum_{i=1}^{N} \qty(d_i - B_1  \cos(\omega t_i) - B_2 \sin(\omega t_i))^2}_{Q})
\,.
\end{align}

Let us manipulate the sum in the exponent, which we denote as \(Q\):
%
\begin{align}
Q &= \sum _{i} d_i^2 -2 \sum _i d_i \qty(B_1  \cos(\omega t_i) + B_2 \sin(\omega t_i))
+ \sum _{i} \qty(B_1  \cos(\omega t_i) + B_2 \sin(\omega t_i))^2  \\
\begin{split}
&= N \overline{d}^2
-2 B_1 \underbrace{\sum _{i} d_i \cos(\omega t_i)}_{R_1(\omega )}
-2 B_2 \underbrace{\sum _{i} d_i \sin(\omega t_i)}_{R_2(\omega )} + \\
&\phantom{=}\
+ B_1^2 \underbrace{\sum _{i} \cos^2( \omega t_i)}_{c}
+ B_2^2 \underbrace{\sum _{i} \sin^2( \omega t_i)}_{s}
+ 2 B_1 B_2 \sum _{i} \cos(\omega t_i) \sin(\omega t_i)
\end{split}  \\
&= N \overline{d}^2 - 2 B_1 R_1(\omega ) -2 B_2 R_2(\omega ) + B_1^2 c + B_2 s + B_1 B_2 \underbrace{\sum _{i} \sin(2 \omega t_i)}_{h}
\,.
\end{align}

\subsubsection{Large pulsation limit}

The condition we ask of our signal is to have many data points for each period (\(\Delta \ll \omega^{-1}\)) and many sampled periods (\(N \Delta  \gg \omega^{-1}\)), which is equivalent to \((N \Delta )^{-1 } \ll \omega \ll \Delta^{-1} \).

The values of \(c\), \(s\) and \(h\) for various values of \(\omega \) are shown in figure \ref{fig:large_pulsation}.

\begin{figure}[ht]
\centering
\includegraphics[width=\textwidth]{figures/large_pulsation}
\caption{Values of \(c\), \(s\) and \(h\) for different orders of magnitude of \(\omega \). The sampling frequency is fixed at \(\Delta^{-1} = 1\), and \(N=500\) points are always sampled. The conditions we ask, in the context of this plot, would mean that we put ourselves roughly in the middle of the two marked vertical lines, in the region \(\omega \sim \num{e-1}\).}
\label{fig:large_pulsation}
\end{figure}

% However, as we can see in figure \ref{fig:large_pulsation}, the three functions do not really \emph{converge} to those values, and stating something like ``\(\lim_{\omega \to \infty } c = N /2\)'' would be incorrect mathematically.
% This is due to the presence of \emph{resonance}: if the ratio \(\omega \Delta \) is a rational multiple of \(\pi \), especially with a small denominator, there will be a bias in the points sampled, resulting in values which may range all the way from 0 to \(N\) for \(c\) and \(s\), and from \(-N\) to \(N\) for \(h\).
% This should not really be an issue in realistic cases, as the set of points for which happens has measure zero.

% Really, working in the \(\omega \gg \Delta^{-1}\) regime is not wise, since we will necessarily have aliasing in the measured signal, as we are trying to measure a signal well above the Nyquist frequency of our sampler.

% Fortunately, there is a regime in the region \(\omega \lesssim \Delta^{-1}\) where the approximation we are discussing works well, and there are no aliasing issues.

Let us then assume that we are working in that region. Then, denoting the argument of the functions as \(x_i = \omega t_i \mod 2 \pi \), we will have:
%
\begin{align}
c = \sum _{i=1}^{N} \cos^2 x_i &\approx N \expval{\cos^2 x}_{\text{period}} = \frac{N}{2} \\
s = \sum _{i=1}^{N} \sin^2 x_i &\approx N \expval{\sin^2 x}_{\text{period}} = \frac{N}{2} \\
h = \sum _{i=1}^{N} \sin (2 x_i) &\approx N \expval{\sin (2x)}_{\text{period}} = 0
\,,
\end{align}
%
since \(x\) will be approximately uniformly distributed in the \([0, 2\pi )\) interval.
The figure also shows this; the small deviations from these values are due to non-integer amounts of periods being included in the sample, but this is an edge effect, which becomes negligible as \(N\) becomes very large.

\subsubsection{Marginalization}

With these simplifications, the likelihood looks like
%
\begin{align}
\mathscr{L}(D | \omega , B_1 , B_2 ) &= \frac{1}{(\sqrt{2 \pi } \sigma )^{N}}
\exp(- \frac{Q}{2 \sigma^2})  \\
Q &=
N \overline{d}^2 - 2 B_1 R_1(\omega ) -2 B_2 R_2(\omega ) + B_1^2 \frac{N}{2} + B_2 \frac{N}{2}  \\
&= N \qty(\overline{d}^2 + \frac{B_1^2 + B_2^2}{2}) - 2 B_1 R_1(\omega ) - 2 B_2 R_2(\omega )
\,.
\end{align}

The posterior is proportional to the likelihood, since we are assuming the priors on \(\omega \) and \(B_i\) are uniform.
We wish to marginalize it over the parameters \(B_i \in \mathbb{R}\), for \(i = 1, 2\).
This amounts to solving the integral
%
\begin{align}
P (\omega | D) &\propto \int_{\mathbb{R}^2} \dd{B_1 } \dd{B_2 } P (\omega , B_1 , B_2  | D)   \\
&\propto \int_{\mathbb{R}^2} \dd{B_1 } \dd{B_2 }
\exp(- \frac{N}{2 \sigma^2} \qty(\underbrace{\overline{d}^2}_{\text{constant}} + \frac{B_1^2 + B_2^2}{2} - 2 B_1 R_1(\omega ) -2  B_2 R_2(\omega )))  \\
&\propto \int_{\mathbb{R}^2} \dd{B_1} \dd{B_2 } \exp(- \frac{1}{2 \sigma^2} \qty( \sum _{i} \frac{N B_i^2}{2} - 2 B_i R_i)) \\
&\propto \prod_i \int_{\mathbb{R}} \dd{B_i}
\exp(- \frac{N B_i^2}{4 \sigma^2} + \frac{B_i R_i}{\sigma^2})  \\
&\propto \prod_i \sqrt{ \frac{\pi}{N / (4 \sigma^2)}}
\exp( \frac{R_i^2}{\sigma^{4}} \frac{1}{4} \frac{4 \sigma^2 }{N})  \\
&\propto N^{-1} \prod_i \exp( \frac{R_i^2}{\sigma^2 N})  \\
&\propto N^{-1} \exp(\frac{R_1^2(\omega ) + R_2^2(\omega )}{\sigma^2 N})
\,.
\end{align}

In the last step we have used the usual expression for a univariate Gaussian integral \eqref{eq:single-variable-gaussian-integral}.

Since the exponential is monotonic and we are keeping \(\sigma \) and \(N \) constant, the Maximum A-Posteriori (MAP) estimate is given by the maximum of \(R_1^2 (\omega ) + R_2^2 (\omega )\).

\subsubsection{The periodogram}

The periodogram \(C\) is defined as
%
\begin{align}
C(\omega ) = \frac{2}{N} \abs{\sum _{k=1}^{N} d_k \exp(- i \omega t_k)}^2
\,,
\end{align}
%
and while this definition could be applied for an arbitrary set of times \(t_k\), we will only consider it for evenly spaced times \(t_k = k \Delta  + t_0 \) for some \(t_0 \): a discrete-time Fourier transform.

We can rewrite the periodogram as
%
\begin{align}
C(\omega ) &= \frac{2}{N} \abs{\sum _{k=1}^{N} d_k \qty(\cos(\omega t_k) - i \sin(\omega t_k))}^2  \\
&= \frac{2}{N} \qty[ \qty(\sum _{k=1}^{N} d_k \cos(\omega t_k))^2 + \qty(\sum _{k=1}^{N} d_k \sin(\omega t_k))^2]  \\
&= \frac{2}{N} \qty[R_1^2(\omega ) + R_2^2 (\omega )]
\,.
\end{align}

Therefore, the value of \(\omega \) which maximizes \(C(\omega )\) is the same which maximizes \(R_1^2 (\omega ) + R_2^2 (\omega )\), which is the MAP estimate.

\subsubsection{Least-squares fitting}

Least-squares fitting the sinusoid with the same model means we minimize \(\chi^2 =  Q / \sigma^2\).
This is precisely equivalent to the MAP estimate for the full likelihood, which under the aforementioned conditions can be estimated through the maximum of \(R_1^2(\omega ) + R_2^2(\omega )\).

\begin{figure}[ht]
\centering
\includegraphics[width=\textwidth]{figures/chisquare_omega.pdf}
\caption{\(N\) data points \(d_i\) are generated with the same distribution as the theoretical model (see figure \ref{fig:signal}). The value of \(Q\) is computed by fixing \(B_1 \) and \(B_2 \) to their true values, which is why its extremal point is sharper: it assumes more knowledge than the alternative, which is what we must maximize after we marginalize over all possible amplitudes \(B_{1, 2}\). }
\label{fig:chisquare_omega}
\end{figure}

As is shown in figure \ref{fig:chisquare_omega}, the maximum of \(R_1^2 + R_2^2\) and the minimum of \(Q\) do indeed coincide. In fact, \(Q\) can also be computed for different values of \(B_1 \) and \(B_2 \), and it attains its global minimum near the true values of the whole triple \((\omega , B_1 , B_2 )\).

This procedure would yield a Gaussian likelihood for \(\omega \) under the following (sufficient) conditions:
\begin{enumerate}
    \item i.i.d.\ Gaussian noise on each data point;
    \item linear dependence of the model \(f(t)\) on its parameter \(\omega \).
\end{enumerate}

The first condition is satisfied under our hypotheses, the second is not unless the entire data range lies in a very small region: \(N \Delta  \ll \omega^{-1}\), in which case the model can be approximated to linear order; for example, if the region is near the origin we can approximate it as \(f(t) \sim B_1 + B_2 \omega t\).

In the general (and typical) case, instead, the likelihood \(\mathscr{L} \propto \exp(- Q / \sigma^2)\) is not Gaussian, and the model is not linear; however the minimum of \(Q\) is still rather peaked, and as long as we start near it we can numerically find the MAP, or equivalently least-squares, estimate of \(\omega \).

\end{document}


\chapter{Further Directions} \label{ch-1}

There are a number of avenues of research we seek to pursue and include within the scope of the thesis. The most natural one being to continue our work on computing the minimal polynomial of the Frobenius endomorphism by extending other approaches, such as the use of the Hasse Invariant

\section{Outcomes}

There are 3 key outcomes we wish to achieve:

\begin{enumerate}
    \item Devising new algorithms for elementary operations with various classes of skew polynomials. This is builds on both our previous efforts to provide a comprehensive analysis of operations related to skew polynomials, including multiplication, interpolation, and minimal subspace polynomial, in a bit complexity model.
    \item Develop new algorithms for computations related to Drinfeld modules, and in particular computing the minimal polynomial of Drinfeld modules of arbitrary rank $r > 2$. A significant amount of work in this regard has already been conducted and presented in chapter 4.
    \item Create implementations for existing and new algorithms that achieve optimal theoretical bounds where this is feasible.
\end{enumerate}

With regards to item (1) there are a number of different directions that could be pursued. For example, skew rings where the commutation rule takes alternate forms such as $\tau a = (\sigma(a) + \delta)\tau$ for $a \in \L$ or which obey a Leibniz rule appear to be less well studied. However, inspired by the work on computing approximant bases of Bartz et al. in \cite{rosen2021}, we've identified multiplication of matrices with coefficients in a skew polynomial ring as a primary focus. In particular, a fast algorithm for computing minimal approximant basis is achieved by transporting the classical algorithm from the commutative case using a suitable bijective mapping. We wonder if a similar approach can be leveraged to construct a suitably fast algorithm for matrix multiplication over skew polynomial rings.

As noted, a significant amount of work concerning the computation of minimal polynomials for higher rank Drinfeld modules has already carried out. However, we have identified two additional approaches that may yield further algorithms. The first is to generalize the Hasse invariant approach given by Gekeler in \cite{frobdist}. In the rank-two case when $\L = \F_{\frakp}$, computing the Frobenius trace reduces to computing the degree $m$ coefficient of $\phi_{\frakp}$, which is done using a recurrence based on Eisenstein series. analogs of this recurrence exist in higher ranks \cite{Gekeler1988}, and it may be possible to speed up the algorithm of Garai and Papikian using this approach. The second approach would be to attempt to generalize the algorithm of Kedlaya using $p$-adic cohmology to compute the zeta function for classical elliptic curves \cite{kedlaya01}. The initial goal with this approach would be to try to derive an algorithm that functions in the rank-two case at minimum, with a view towards extending this to all ranks.

With respect to item 3, some preliminary work including a number of basic operations and previous algorithms for computing the minimal polynomial of rank-two Drinfeld modules already exists and is publicly available at \url{https://github.com/ymusleh/Skew}. Our intention is to continue adding both existing algorithms and all algorithms developed as part of this thesis to the repository.