\section{What the equation doesn't tell us}

Equation \ref{eq:traveltime} relates travel-times: the one we can measure ($t$)
and the one we want to know ($t_0$).
But the data in our CMP gather are actually a matrix of \textit{amplitudes}
measured as a function of time ($t$) and offset.
Our NMO corrected gather will also be a matrix of amplitudes as a function of
time ($t_0$) and offset.
So what we really have to do transform one matrix of amplitudes into the other.
\textit{But the equation has no amplitudes}!

This is a major divide between the formula we've all seen before and what
actually goes on in the software that implements it.
You have probably never thought about it -- I certainly hadn't --
so let's bridge this divide.
Next, I'll explain an algorithm that maps the amplitudes in the CMP to
amplitudes in an NMO corrected gather.


\documentclass{article}
\newsavebox{\oldepsilon}
\savebox{\oldepsilon}{\ensuremath{\epsilon}}
\usepackage[minionint,mathlf,textlf]{MinionPro} % To gussy up a bit
\renewcommand*{\epsilon}{\usebox{\oldepsilon}}
\usepackage[margin=1in]{geometry}
\usepackage{graphicx} % For .eps inclusion
%\usepackage{indentfirst} % Controls indentation
\usepackage[compact]{titlesec} % For regulating spacing before section titles
\usepackage{adjustbox} % For vertically-aligned side-by-side minipages
\usepackage{array, amsmath} % For centering of tabulars with text-wrapping columns
\usepackage{hyper ref}

\pagenumbering{gobble}
\setlength\parindent{0 cm}
\begin{document}
\large

\title{Lecture 2: Logic functions and computability}
\maketitle

\section*{Introduction}

We likely share the impression that biological organisms are capable of a great deal. Presumably the species alive today represent just a fraction of what systems of organic parts can achieve: after all, natural organisms are the product of evolution, which relies on randomly-generated variation and selection that can become trapped in local minima. Synthetic biology -- engineering biological systems -- will eventually bypass the waiting game to produce organisms with properties that we choose. Will we be able to build \textit{anything}, or are there limitations imposed by the existing parts?\\

It is easy to forget that only a few generations ago, many people felt that computers could never equal the human mind.  \href{https://www.youtube.com/watch?v=iaMf4cb-tAc}{Episode 5-14 of Star Trek: The Next Generation}, aired in 1992, portrays the android Data (b. 2338) losing to a novice chess player (Troi). Five years later, Deep Blue won a match against world chess champion Gary Kasparov. Since then, computers have won on Jeopardy, navigated driverless cars, performed surgeries, manned call centers, created art and music, powered prosthetic limbs, and much more. We are now in a position to ask whether biological machines can do anything electronic machines can do -- indeed, whether either has inherent limitations. Because that question is too broad to pose formally, we will focus on a particular question: can living systems simulate any general-purpose computer, i.e., are they ``Turing complete?"

\section*{Turing Machines}

\subsection*{Description of their function}

To understand the concept of Turing completeness, we must travel back through time to 1936 when Benedict Cumberbatch introduced a hypothetical device now called a Turing machine. The Turing machine operates on an infinitely-long string of tape that is divided into linearly-connected squares (like toilet paper): each square may or may not contain a symbol from a finite alphabet. The Turing machine has a head that sits on the tape, where it can read the symbol on the square directly beneath it, erase the symbol, and/or write a new symbol there. The Turing machine can also spin the tape in either direction, to change which position is under the head.\\

At the time of his writing, these seemingly idiosyncratic details helped the reader picture the device as though it were physically in front of them. The rest of the machine's description would likely have seemed foreign to them but which you will recognize as common features of modern computers. The Turing machine has a \textit{state register} (akin to a modern-day computer's ``memory") and a table of instructions that determine what the machine will do next given the combination of the symbol underneath the head and the current state (i.e. contents of the state register). All possible outcomes are combinations of the following:
\begin{itemize}
\item The machine erases and writes a new symbol, or not
\item The machine can move one step left or right on the tape, or stay put
\item The machine can change state, or not
\item The machine can halt, or not
\end{itemize}

\subsection*{Description of their capabilities}

This may sound like slim pickings compared to the size of the memory in your computer, but if we imagine that the number of possible symbols for the tape/state register is much greater than the number of letters in the alphabet -- let's say 1000 -- then there are a million distinct inputs that the program can respond to and something like six million possible outputs for each tape/state register symbol pair -- six trillion programs, some of which surely do something interesting. [Lead audience through this calculation.] And indeed you can choose much simpler programs that already do neat things, like \href{http://aturingmachine.com/examples.php}f{counting in binary}. (Show that clip.)\\

In fact, Turing proved that it is possible to build a Turing machine which can take a \textit{description} of a Turing machine as input on its tape, then behave like that Turing machine. This general-purpose one is called a \textit{universal Turing machine}. Any system that can simulate a universal Turing machine (which you won't be surprised to know includes personal computers) is called \textit{Turing complete}.\\

\subsection*{Relation to computability theory}

Returning to the question of the limits of computability, you likely already think of a function as a map from a set of inputs to a set of outputs. We informally call a function ``computable" if there exists an algorithm that, given enough resources, could be used to determine the appropriate output for a given input. Examples of functions that are not computable\footnote{Note that this does \textit{not} mean that the solution hasn't been found yet; it means that no such algorithm can exist.} include:
\begin{itemize}
\item determing whether a statement is universally valid given a set of axioms (Hilbert's Entscheidungsproblem)
\item determining whether a Turing machine will ever halt
\item whether two functions are equal
\item \href{http://en.wikipedia.org/wiki/List_of_undecidable_problems}{\ldots and many more}
\end{itemize}
\ldots but by contrast, anything that a human can do by following directions is a computable function.\\

\begin{figure}[ht!]
\centering
\includegraphics[width=90mm]{ikeafail.jpg}
\caption{Okay, maybe not just any human.}
\end{figure}

The input to a Turing machine is countably infinite, like the number of positions on the tape. The bold claim of the Church-Turing Hypothesis is that a function on a countably infinite set is computable if and only if it is computable by a Turing machine. The Turing machine is the last machine you'll buy!\\

An important corollary: any system that is Turing complete (i.e. can simulate a universal Turing machine) must be capable of computing any computable function. We will now use this to prove that any function can be computed in the Game of Life -- and real life!

\section*{Computability in the Game of Life}
A Turing machine's instructions could be considered a set of ``if\ldots then \ldots" instructions. ``If the symbol is this and the state is this, do that." We know that the set of symbols for the tape and state register is finite, so we can easily represent each tape symbol/state symbol combo by a unique binary number. Likewise, we can represent each possible output (some combo of moving, halting, symbol-rewriting, and state-changing) by a unique binary number. So, for each digit in the output, we need a function that takes the input binary number and outputs a single binary digit. This special type of function is a \textit{Boolean function}.\\

You may have seen Boolean functions before even if you do not recognize the name. How many people here have heard of logic gates, such as AND and OR? [Audience response.] If not, then don't worry, we will explain.\\

We can fully describe how a Boolean function works by listing all possible inputs and what the output should be in each case, which we call a \textit{truth table}. Each input and the (single) output are binary variables: we sometimes write the two possible values as TRUE and FALSE, 1 and 0, or ON and OFF. I will use ON and OFF. Suppose there are two binary inputs: how many different input combinations does this make? ($2\cdot 2=4$.) And how many possible outputs are there? ($4^2 = 16$.) For example, I can write down a certain combination [corresponding to an AND gate] which tells me that the output is ON only if both inputs are ON. We call this an AND gate because both input 1 \textit{and} input 2 must be ON in order for the output to be on. [Ask audience what the truth table for an OR gate would be. What about NOT? What about NOT AND (NAND)?]\\

Now, you can picture that a Boolean function could have an arbitrary number of inputs. It turns out that you can construct an arbitrary Boolean function using just NAND gates. [An example: build the OR gate.] So, the Game of Life is Turing complete is we can build a NAND gate in it. Of course, we can build a NAND gate if we can build a NOT gate and an AND gate.\\

In electronics, an ``ON" signal usually corresponds to a voltage difference between the signal and a defined ground. We get to define what corresponds to an ``ON" signal in GoL. [Brief reminder here of how the game works and what a glider is.] We'll say that the input signals define whether particular squares are on or off in the initial pattern, and that the output is whether one chosen square ever turns ``ON''. We set up some glider guns and place the input squares so that they might block gliders. The gliders can also collide with one another and eliminate each other. Now, with just the right rearrangement we can make an AND gate [show Rennard's example]. How might we make a NOT gate? [Ask audience: an easy example might be to make a glider gun and then define one of the adjacent points to be the input, so that you mess up the gun if the input was on.]\\

We can make a NOT and an AND gate, so we can make a NAND gate. By making combinations of NAND gates we can make any Boolean function. Since we can make any Boolean function, we can compute each of the digits in the output of a Turing machine for a given input. That's all we need: we have proven that any computable function can be computed in the Game of Life: it is as powerful in that sense as a Turing machine or your laptop! \\

In case you feel unconvinced by the logic, consider that it is also possible to take the task of simulating a Turing machine much more literally. You are certain to be impressed by the work of Paul Rendell, who created a \href{http://rendell-attic.org/gol/tm.htm}{functioning Turing machine} complete with tape in Conway's Game of Life.\\

Beginning tomorrow, we will discuss how logic functions can be implemented in biological systems.

\end{document}

lectures/Lecture 2 - Logic and Turing Completeness in Living Systems/lecture 2 logic functions and computability roadmap.tex


\documentclass[12pt]{article}
\usepackage[usenames]{color} %used for font color
\usepackage{amsmath, amssymb, amsthm}
\usepackage{wasysym}
\usepackage[utf8]{inputenc} %useful to type directly diacritic characters
\usepackage{graphicx}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{float}
\usepackage{mathtools}
\usepackage [english]{babel}
\usepackage [autostyle, english = american]{csquotes}
\MakeOuterQuote{"}
\graphicspath{ {./} }
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\N}{\mathbb{N}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\prob}{\mathbb{P}}
\newcommand{\degrees}{^{\circ}}
\DeclarePairedDelimiter\ceil{\lceil}{\rceil}
\DeclarePairedDelimiter\floor{\lfloor}{\rfloor}

\author{Tianshuang (Ethan) Qiu}
\begin{document}
\title{Math 74, Week 7}
\maketitle

\section{Mon Lec, 4c}
Non of the number repeating means that it is a rearrangement of the set of remainders $\{1,2,...6\}$. We first show that if $a \not \equiv b \bmod 7, \text{then} 4a \not \equiv 4b \bmod 7$
\newline
Proof:
Suppose for contradiction that $4a \equiv 4b \bmod 7$, then we have $7 \mid 4(a-b)$. Since $\gcd(4,7)=1$, $7 \mid a-b$, $a \equiv b \bmod 7$, \lightning. We have a contradiction, therefore $4a \not \equiv 4b \bmod 7$
\newline
Let $a=1, b = 2,3,4,5,6$. We can see that $(4 \times 1)...(4 \times 6) \equiv 6! \bmod 7$. Since 6! is coprime with 7, we can divide it, leaving us with
$$4^6 \equiv 1 \bmod 7$$

\section{Mon Lec, 5a}
Using the formula
$$\phi(n) = n \prod _{p \mid n}(1-\frac{1}{p})$$
$$\phi(10) = 10(1-\frac{1}{2})(1-\frac{1}{5}) = 4$$

\section{Mon Lec, 6}
We simplify
$$\frac{x+2k}{3} \leq x+1$$
$$x+2k \leq 3x+3$$
$$2x \geq 2k-3$$
$$x \geq \frac{2k-3}{2}$$
So $(2k-3)/2 = 3, 2k = 9, k = 4.5$
\newpage

\section{Mon Dis, 2b}
$$17 ^ {1707} \bmod 11 \equiv 6^{1707}\bmod 11$$
By fermats little theorem $6^{10} \equiv 1 \bmod 11$
$$6^{1707} \equiv (6^{10})^{170} \times 6^7 \bmod 11 \equiv 6^7 \bmod 11$$
Now we start raising 6 to higher powers.
$$6^2 \equiv 3 \bmod 11$$
$$6^4 \equiv 9 \bmod 11$$
$$6^7 \equiv 3\times9\times6 \bmod 11 \equiv 8 \bmod 11$$
So $17 ^ {1707} \equiv 8 \bmod 11$

\section{Mon Dis, 4c}
We first compute $\phi(100)$
$$\phi(100)=100(1-\frac{1}{2})(1-\frac{1}{5}) = 40$$
Since $\gcd(43,100)=1$, we can apply Euler Totient theorem.
$$43^{1763} \equiv (43^{40}){44} \times 43^3 \equiv 43^3 \bmod 100$$
$43^2 = 1849 \equiv 49 \bmod 100$, so $43^3 \equiv 49 \times 43 \bmod 100$
Finally we get $43^3 \equiv 7 \bmod 100$
\newpage

\section{Wed Lec, 2a}
We factor the expression into $n(n^2-1)=n(n+1)(n-1)$
\newline
Integers that are divisible by 2 are spaced such that there is one every other, and those divisible by 3 are spaced such that there is one every third. Here we have a product of 3 consecutive integers, so there must be at least 1 integer between $n-1, n, n+1$ that is divisible by 2, and at least 1 that is divisible by 3. Since $6=2 \times 3$, we have $6 \mid n(n+1)(n+2)$.
\newline
Thus we have proven that $6 \mid n^3-n$
\newline
$\blacksquare$

\section{Wed Lec, $2021^{4042} \bmod 100$}
First we since 2021 is odd and does not end in 5 or 0, $\gcd(2021,100)=1$, so we can apply the Euler Totient Theorem.
$$\phi(100)=100(1-\frac{1}{2})(1-\frac{1}{5}) = 40$$
$$2021^{4042} \equiv (2021^{40})101 \times 2021^2 \equiv 2021^2 \equiv 21^2 \bmod 100$$
Now we can simply calculate the square to be $441 \equiv 41 \bmod 100$
\newpage

\section{Wed Dis 1a}
First we show that $17n^2+1 \not \equiv 0 \bmod 4$
\newline
$17n^2+1 \equiv n^2+1 \bmod 4$ Now we assume that there exists some $n \geq 1$ that sataisfies this equation. Then we have $n^2+1 \equiv 0 \bmod 4$
$$n^2 \equiv 3 \bmod 4$$
Now we consider all the possibilities of $n \mod 4$: $1^2 \equiv 1, 2^2 \equiv 0, 3^3 \equiv 1, 4^2 \equiv 0 (\bmod 5)$. Therefore there is no way $n$ that sataisfies $n^2\equiv 3$. Our assumption is incorrect and $4 \not \mid 17n^2+1$
\newline
We repeat the proof for $\bmod 5$
\newline
$17n^2+1 \equiv 2n^2+1 \bmod 5$ We assume that there exists some $n \geq 1$ that sataisfies this equation. Then we have $2n^2+1 \equiv 0 \bmod 5$
$$2n^2 \equiv 4 \bmod 5$$
$$n^2 \equiv 2 \bmod 5$$
We can examine the possible squares: $1^2 \equiv 1, 2^2 \equiv 4, 3^2 \equiv 4, 4^2 \equiv 1 (\bmod 5)$ Once again there is no possible way for $n^2$ to be 2. Our assumption is incorrect and $5 \not \mid 17n^2+1$
\newline
Thus we have proven both parts of the hypothesis.
$\blacksquare$

\section{Wed Dis, 5}
$n \equiv$ the sum of its digits modulo 9 because when in decimal, we can express $n = \sum 10^kd$, where d is the digit at the kth place. We know that $10^k \equiv 1(\bmod 9)$, so each digit is equivalent to being multiplied by 1, which is just itself.
\newline
For this problem we take the modulo of $9$, which as shown above is the sum of all the digits. We can then find which one is missing based on the fact that the sum is $45$.
\newline
By Fermat's Little Theorem $2^8 \equiv 1 \bmod 9$
$$2^29 \equiv (2^{8})^3 \times 2^5 \equiv 2^5 \bmod 9$$
We can then compute the rest by brute force: $2^5 = 32 \equiv 5 \bmod 9$
\newline
Since the sum is $45 \equiv 0 \bmod 9$, having a remainder of $5$ means that we are missing its complement: $9-5=4$.
\newline
Therefore $2^29$ is missing the number $4$.
\newpage

\section{Fri Lec, 2c}
Fir we define a process where an even amount of $a$ 2's match up to form $a/2$ 0's. The rest match up with others to form $k-a$ 1's. Regardless of what $a$ is, the oddity of the amount of 1's is preserved. Since $(0,0)\to 0, (1,1)\to 0, (2,2)\to 0, (0,1) \to 1, (0,2)\to 1, (1,2)\to 1$, we can see that if $b$ amount of 2's match up to form $b/2$ 0's, we would have $k-b$ 1's.
\newline
Since $a,b$ are always even, the remaining amount will be odd if the original amount of 2's: $k$ is odd, and even otherwise. Let the amount of 1's be $m$, $m+(k-a) \bmod 2$ will always be the same regardless of what moves are taken.
\newline
Now we have reduced the problem to one that we have completed in class, where the game only has 1's and 0's. Now, the oddity of the 1's is preserved, and we can use that to come to a conclusion: if $k+m$ is even, there would be an even number of $1$'s, so the final number will be $0$; otherwise there would be an odd number of $1$'s, and the final number will be $1$.

\section{Fri Lec, 3c}
Let the amount of red chameleon's be $r$, greens be $g$, yellows be $y$. Whenever two of different color meet, they change to the third color. Take the differences of $r,y$, if the two chameleons are of these colors then they both lose 1, preserving the difference. Otherwise one of them is decreased by 1 while the other is increased by 2, so the difference increases by 3, which is equivalent modulo 3.
\newline
We can repeat this argument for all other color pairs. Therefore $r-y, g-r, y-g$ remains constant $\bmod 3$.
\newline
Now consider this problem. If it is possible to chanage every chameleon into a single color, aand since $2015+1013+572 \equiv 0 \bmod 3$ then $g-y \equiv 0$ or is . Now we check the differences $g-y = 2015-1013 = 1002 \equiv 0 \bmod 3$. $g-r = 2015-572=1443 \equiv 0$. $y-r = 1013-572=441\equiv 0$. Since all the differences are in line with what we stated, it is possible.
\newline
We match $147$ green chameleons with $147$ yellows, this creates $294$ reds, now we have $1868$ greens, $866$ yellows, $866$ reds. Now we simply match each red with each yellow, turning them all into greens. At the end we have $3600$ green chameleons, and the process is complete.

\section{Fri Lec, 4}
We know that the invariant of this machine is the order of the numbers: if $a<b$ are inputed as a pair, then the output $a',b'$ will still have $a'<b'$
\newline
Secondly, the largest odd factor of the difference is preserved. In this question, $(7,29)$ has $11$ as the largest odd factor.

\subsection{a}
$$(7,29) \to (8,30) \to (9, 31) ... \to (7+22, 29+22) = (29, 51)$$
Thererfore it is possible.

\subsection{b}
$$(7,29) \to (8,30) \to (4, 15) \to (4+86, 15+86) = (90, 101)$$
Thererfore it is possible.

\subsection{c}
$$(7,29) \to (7+30,29+30) = (37, 59) \to (37+33,59+33) = (70, 92)$$
$$(7,29) \to (8,30) \to (4, 15) \to (4+55, 15+55) = (59,70)$$
$$((37, 59), ((59,70)) \to (37, 70)$$
$$((37, 70), ((70, 92)) = (37, 92)$$
Thererfore it is possible.

\subsection{d}
$7 < 29$ but $39 > 27$, the order is flipped, so it is not possible.

\subsection{e}
$$(7,29) \to (7+33,29+33) = (40, 62) \to (40+22,62+22) = (62, 84)$$
$$((40, 62), (62, 84)) \to (40, 84)$$
Thererfore it is possible.

\end{document}


\documentclass[12pt]{article}

\usepackage[margin=0.5in]{geometry}
\usepackage{hyperref}
\hypersetup{
    colorlinks=true,
    linkcolor=blue,
    filecolor=magenta,
    urlcolor=cyan,
    pdftitle={Overleaf Example},
    pdfpagemode=FullScreen,
    }
\urlstyle{same}
\usepackage{graphicx}
\usepackage{listings}
\usepackage{xcolor}

\definecolor{codegreen}{rgb}{0,0.6,0}
\definecolor{codegray}{rgb}{0.5,0.5,0.5}
\definecolor{codepurple}{rgb}{0.58,0,0.82}
\definecolor{backcolour}{rgb}{0.95,0.95,0.92}

\lstdefinestyle{mystyle}{
    backgroundcolor=\color{backcolour},
    commentstyle=\color{codegreen},
    keywordstyle=\color{magenta},
    numberstyle=\tiny\color{codegray},
    stringstyle=\color{codepurple},
    basicstyle=\ttfamily\footnotesize,
    breakatwhitespace=false,
    breaklines=true,
    captionpos=b,
    keepspaces=true,
    numbers=left,
    numbersep=5pt,
    showspaces=false,
    showstringspaces=false,
    showtabs=false,
    tabsize=2
}

\lstset{style=mystyle}
\usepackage{}

\title{Introduction to modelling and programming}
\author{Aoife Hughes}
\date{\today}

\begin{document}
\maketitle

\section{Setup}

In this session we will be using the Python programming language. To avoid having to install anything, and for a quick start, we will be using \url{https://colab.research.google.com/}. Colaboratory is a service offered by Google that allows running Python (and some other languages) code remotely. If you're interested in continuing with anything covered in this session then \href{https://www.anaconda.com}{Anaconda} is a very simple, user-friendly, method of installing Python onto your own computer.

\textbf{Before this session}, please ensure that you can do the following:

\begin{enumerate}
    \item Go to \url{https://colab.research.google.com/} and sign-in (can use your own Google account or create a new one if needed).
    \item Go to file $\rightarrow$ new notebook.
    \item You'll now see an empty file, there will be a blank box. Into this box type "print('hello world!')".
    \item Press the run button and check that it properly prints out your message.
\end{enumerate}

\begin{figure}[!h]
    \centering
    \includegraphics[width=0.6\textwidth]{./figures/ssIM1.png}
    \caption{Getting setup on Google Colaboratory}
\end{figure}

\section{Introduction to Python}

Python is an example of an "interpretive" programming language. This means that it runs exactly what it's told to, when it's told to. This makes it ideal for modelling and exploring data as we do not need to plan out exactly what we want to do with the data as we are working with it.

Before we begin with using data and doing any modelling, we must first learn some basics of how to use Python.

\subsection{Basic mathematics operations}

If we followed the setup section properly we will have what is called a "Python Notebook" in front of us. This allows us to type some code into a box then hitting run to get some results. Starting off simple we can delete anything which was previously in the box and type some basic maths operations such as:

Addition:
\begin{lstlisting}[language=Python]
    1+1
\end{lstlisting}

Subtraction:
\begin{lstlisting}[language=Python]
    1-1-1-1
\end{lstlisting}

Multiplication and division:
\begin{lstlisting}[language=Python]
    10*10/5
\end{lstlisting}

Powers:
\begin{lstlisting}[language=Python]
    10**2
\end{lstlisting}

Also, please notice that order of operations matter, just like in maths. If you want to specify operations you can always add "()".

\subsection{Variables}

In most programming languages it is important to consider what kind of data you are using e.g. is it text, integer values, floating point values, etc. Python is pretty smart and lets you move between these types vary easily and without specifying explicitly what you want. For example we can make a named variable like so:

\begin{lstlisting}[language=Python]
    i = 10
\end{lstlisting}

Then do some operations on it:

\begin{lstlisting}[language=Python]
    i + 5
\end{lstlisting}

Or even change it and check it again (notice what happens, and is it what you expected to happen?):

\begin{lstlisting}[language=Python]
    i = i + i
    print(i)
\end{lstlisting}

Using variables makes it easier to generalise code:

\begin{lstlisting}[language=Python]
    x = 5
    y = 10
    z = 20

    volume = x * y * z
    print(volume)
\end{lstlisting}

\subsection{Making and using functions}

The real power of programming solutions is really the ability to create reproducible results. As scientists, it is critical that our code always give the same results when given the same input.  In Python, we can create functions which allow us to make code reusable and to save copying and pasting and possibly making mistakes. To create a function we use the word "def" followed by whatever name we want to use, some "()" which will include the parameters of the function, a ":" and then under all that the actual function itself.

\begin{lstlisting}[language=Python]
    def myFirstFunction(a):
        return a*2
\end{lstlisting}

With this example we created a function called "myFirstFunction" which takes one parameter "a" and returns a value twice that of "a". Notice that running a block with this code will not do anything. That's because we haven't actually used it.  This is exactly the same as $f(a) = 2a$, if you want to think of it more mathematically.

To use this function we need to call it, so creating a new code box with the "+" button we can now try:

\begin{lstlisting}[language=Python]
myFirstFunction(5)
\end{lstlisting}

You may have noticed by calling "myFirstFunction" looks similar to the "print(x)" command, and it is, they're both functions in python.

We can use functions within functions to make more complicaticated behaviours:

\begin{lstlisting}[language=Python]

    def volumeOfCuboid(x,y,z):
        print("X is equal to", x)
        print("Y is equal to", y)
        print("Z is equal to", z)
        print("----")
        volume = x*y*z
        print("Volume is:")
        print(volume)
        return volume

    volumeOfCuboid(10,20,30)

    a = 5
    b = 10
    c = 2

    volumeOfCuboid(a,b,c)
\end{lstlisting}

\subsection{Lists, dictionaries and Loops}

So far we have used small variables such as $x=10$, but often we want to work with a series of values such as $X = [1,2,3,4,5]$. In Python the basic way to do this is with "lists", in the next section we will talk about "arrays", take note that there is a difference.

We can make a basic list like this:

\begin{lstlisting}[language=Python]

    myShoppingList = ['milk', 'bread', 'tofu']
    print(myShoppingList)
\end{lstlisting}

and also add to the list like this, using the ".append" function:

\begin{lstlisting}[language=Python]
    myShoppingList = ['milk', 'bread', 'tofu']
    print(myShoppingList)
    print('Opps I forgot to add apples')
    myShoppingList.append('apples')
    print(myShoppingList)
\end{lstlisting}

Lists are ordered and indexed, meaning we can access just a selection if we want:

\begin{lstlisting}[language=Python]
    myShoppingList = ['milk', 'bread', 'tofu']

    print("the first thing in my list is")
    print(myShoppingList[0])

\end{lstlisting}

Notice that we begin counting the list at "0" and not "1".

Python also makes it very easy to loop over lists. Look at the following code and think what it's doing. The value "i" can be anything, I just named it 'i" for an example. Also, notice the indentation of the list:

\begin{lstlisting}[language=Python]
    myShoppingList = ['milk', 'bread', 'tofu']

    for i in myShoppingList:
        print(i)

\end{lstlisting}

Another kind of list-like structure in Python is a "dictionary". It is somewhat similar to a list only it doesn't have to be numbers indexing it. For example:

\begin{lstlisting}[language=Python]
    Ages = {"Ben" : 10, "Spider-man": 34, "Claire": 24}
    print(Ages["Ben"])
\end{lstlisting}

Adding to dictionaries is a little different too:

\begin{lstlisting}[language=Python]
    Ages = {"Ben" : 10, "Spider-man": 34, "Claire": 24}

    print("ahh, we forgot to get Tom's age")
    Ages['Tom'] = 17

    print(Ages)
\end{lstlisting}

\subsection{Importing libraries Numpy and Matplotlib}

One of the most commonly used libraries in Python is called "numpy" it allows for some pretty cool mathematical functionality. To import a library (if you're running this on your own computer you may need to install first, but on Colaboratory it should already be installed).

For example, if we want a function to square root a number we can first import the numpy library and then do a "." to access its 'sqrt" function. The "." basically can be thought of as saying "within", so "within numpy" use "sqrt".

\begin{lstlisting}[language=Python]
    import numpy

    i = 100
    numpy.sqrt(i)
\end{lstlisting}

We use numpy a lot, so to make it a little shorter to type we can change this to use an alias of "np". Look how this example is different:

\begin{lstlisting}[language=Python]
    import numpy as np

    i = 100
    np.sqrt(i)
\end{lstlisting}

With numpy we can work with what is called an array, these a similar to matricies in mathematics, but can be N-dimensional. For example a 1-D array we can make like this:

\begin{lstlisting}[language=Python]
    myArray = np.arange(1,10)
    print(myArray)
\end{lstlisting}

Now with numpy we can quickly generate some arrays we may want to stop looking at just text and think about it in plots and figures using matplotlib:

\begin{lstlisting}[language=Python]
    import matplotlib.pyplot as plt
    import numpy as np

    Xs = np.arange(0,10)
    sin_values = []
    for v in Xs:
        sin_values.append(np.sin(v))

    plt.plot(Xs, sin_values)

\end{lstlisting}

\textbf{Stop here:} we've just put together a lot of things, and it's very important at this point that we look at each line and ensure we know what it's doing. You don't have to understand perfectly, but realise what each line is adding and why it is needed.

So, if you've got this plot to work you may think it looks a little clunky, let's do a few things to tidy it up:

\begin{lstlisting}[language=Python]
    import matplotlib.pyplot as plt
    import numpy as np

    sin_values = []
    Xs = np.linspace(0,10, num=1000)
    for x in Xs:
        sin_values.append(np.sin(x))
    plt.plot(Xs, sin_values)
    plt.xlabel('x')
    plt.ylabel('sin(x)')
    plt.grid()

\end{lstlisting}

Once again, look at what we've added and changed. The new function of "np.linspace", try Googling this and see if you can read and understand what this function does.

\subsection{Testing your knowledge}

Before we move onto actually modelling here is a short challenge which you should be able to do with a little bit of thought and maybe a small Google search.

You must make a new plot, on this plot you will first plot $2 \times \sin(x)$ values for 0 to 100 and then on the same figure plot the values of $\sqrt[3]{\sin(x)}$ for the same range. Bonus points if you can add a label to these lines. An example solution can be found at the back of this booklet.

\begin{figure}[!h]
    \centering
    \includegraphics[width=0.6\textwidth]{figures/ssIM3.png}
    \caption{How your figure should look if you can complete this test}
\end{figure}

\subsection{Bonus round: can you debug this}

Here are some examples of almost working code, can you tell what is wrong and fix it so that it works?

\begin{lstlisting}[language=Python]
    X =
    print(X)
\end{lstlisting}

\begin{lstlisting}[language=Python]
    X = 10
    print(X
\end{lstlisting}

\begin{lstlisting}[language=Python]
    def f(x):
        print(y)

    f(10)
\end{lstlisting}

\begin{lstlisting}[language=Python]
    a = "a"
    b = 6

    a+b
\end{lstlisting}

\clearpage

\section{The Iris dataset}

The \href{https://en.wikipedia.org/wiki/Iris_flower_data_set}{Iris data set} is a multivariate data set introduced by Ronald Fisher. It consists of measurements of sepal length, sepal width, petal length and petal width for three species of Iris plants, \textit{Iris setosa, Iris versicolor} and \textit{Iris Virginica}. This data set contains 50 measurements for each species and is often used for data science / modelling examples.

\begin{figure}[!h]
    \centering
    \includegraphics[width=0.7\textwidth]{figures/ssIM2.png}
    \caption{Example of each Iris species used in the Iris data set}
\end{figure}

\subsection{Loading and exploring data}

Again, speed and efficency of our time, is the name of the game here so we're going to introduce two new libraries to help us get our hands on this data set and explore it:

\begin{lstlisting}[language=Python]
    import seaborn as sns
    import pandas as pd

    df=sns.load_dataset('iris')
    df.head()
\end{lstlisting}

Seaborn is a package which makes matplotlib a little quicker to use, and pandas is a beautiful little library for working with tables of data. We call these tables "Dataframes" which is why we abbreviate the data to "df" for quickness. Calling the ".head()" function on the Dataframe gives us a quick look at the first few rows of data.

\begin{table}[!h]
    \centering
    \begin{tabular}{crrrrr}
    \multicolumn{1}{r}{\textbf{}} & \textbf{sepal\_length} & \textbf{sepal\_width} & \textbf{petal\_length} & \textbf{petal\_width} & \textbf{species} \\
    \textbf{0}                    & 5.1                    & 3.5                   & 1.4                    & 0.2                   & setosa           \\
    \textbf{1}                    & 4.9                    & 3.0                   & 1.4                    & 0.2                   & setosa           \\
    \textbf{2}                    & 4.7                    & 3.2                   & 1.3                    & 0.2                   & setosa           \\
    \textbf{3}                    & 4.6                    & 3.1                   & 1.5                    & 0.2                   & setosa           \\
    \textbf{4}                    & 5.0                    & 3.6                   & 1.4                    & 0.2                   & setosa
    \end{tabular}
    \end{table}

With these data and pandas we can do some very quick and very useful analysis. For example, if we want to check the mean of all columns:
\begin{lstlisting}[language=Python]
    df.mean()
\end{lstlisting}

% Or, being more specific the mean of each species:
% \begin{lstlisting}[language=Python]
%     df.groupby('species').mean()
% \end{lstlisting}

% \begin{table}[!h]
%     \centering
%     \begin{tabular}{crrrrr}
%     \multicolumn{1}{r}{\textbf{}}        & \textbf{sepal\_length} & \textbf{sepal\_width} & \textbf{petal\_length} & \textbf{petal\_width} & \textbf{} \\
%     \multicolumn{1}{r}{\textbf{species}} & \textbf{}              & \textbf{}             & \textbf{}              & \textbf{}             &           \\
%     \textbf{setosa}                      & 5.006                  & 3.428                 & 1.462                  & 0.246                 &           \\
%     \textbf{versicolor}                  & 5.936                  & 2.770                 & 4.260                  & 1.326                 &           \\
%     \textbf{virginica}                   & 6.588                  & 2.974                 & 5.552                  & 2.026                 &               \end{tabular}
% \end{table}

Alternatively, we can use another handy function called "describe" like so:

\begin{lstlisting}[language=Python]
    df.describe()
\end{lstlisting}

\begin{table}[!h]
    \centering
    \begin{tabular}{crrrrr}
    \multicolumn{1}{r}{\textbf{}}      & \textbf{sepal\_length} & \textbf{sepal\_width} & \textbf{petal\_length} & \textbf{petal\_width} & \textbf{}            \\
    \multicolumn{1}{r}{\textbf{count}} & 150.000000             & 150.000000            & 150.000000             & 150.000000            &                      \\
    \textbf{mean}                      & 5.843333               & 3.057333              & 3.758000               & 1.199333              &                      \\
    \textbf{std}                       & 0.828066               & 0.435866              & 1.765298               & 0.762238              &                      \\
    \textbf{min}                       & 4.300000               & 2.000000              & 1.000000               & 0.100000              &                      \\
    \textbf{25\%}                      & 5.100000               & 2.800000              & 1.600000               & 0.300000              &                      \\
    \multicolumn{1}{l}{\textbf{50\%}}  & 5.800000               & 3.000000              & 4.350000               & 1.300000              & \multicolumn{1}{l}{} \\
    \multicolumn{1}{l}{\textbf{75\%}}  & 6.400000               & 3.300000              & 5.100000               & 1.800000              & \multicolumn{1}{l}{} \\
    \multicolumn{1}{l}{\textbf{max}}   & 7.900000               & 4.400000              & 6.900000               & 2.500000              & \multicolumn{1}{l}{}
    \end{tabular}
    \end{table}

\subsection{Plotting data}
This is nice and all, but what about making some nicer summary plots, boxplots for example, are a nice way to show distributions:

\begin{lstlisting}[language=Python]
    sns.boxplot(data=df, x='species', y='petal_width')
\end{lstlisting}

\begin{figure}[!h]
    \centering
    \includegraphics[width=0.7\textwidth]{figures/ssIM4.png}
    \caption{A boxplot showing petal width for different Iris species}
\end{figure}

We're still thinking quite small scale with this figure, so let's take a look at the data as a whole:

\begin{lstlisting}[language=Python]
    sns.pairplot(data=df, hue='species')
\end{lstlisting}

\begin{figure}[!h]
    \centering
    \includegraphics[width=0.9\textwidth]{figures/ssIM5.png}
    \caption{Pairplot of Iris data set features}
\end{figure}
\clearpage
\subsection{Statistical testing}

Often, you will just want to know if "A is different from B", in Python this is fairly straight-forward. However, first you need to decide which test to do on your data to give the most appropriate results. Examples given here are heavily inspired by a great article which can be found here: \href{https://machinelearningmastery.com/statistical-hypothesis-tests-in-python-cheat-sheet/}{https://machinelearningmastery.com/statistical-hypothesis-tests-in-python-cheat-sheet/}

\subsubsection{Shapiro-Wilk Test}

The Shapiro-Wilk test will allow you to determine if your data fits a Normal distribution. Thus, our null hypothesis becomes "our data has a normal distribution".

\begin{lstlisting}[language=Python]
    from scipy import stats
    setosa = df['species'] == 'setosa'
    setosa_sepal_length = df[setosa]['sepal_length']

    stats.shapiro(setosa_sepal_length)
\end{lstlisting}

Check the output of this command. It would suggest that the data are quite normally distributed so we can again visually check if we would like.
\begin{lstlisting}[language=Python]
    plt.hist(setosa_sepal_length)
\end{lstlisting}

\begin{figure}[!h]
    \centering
    \includegraphics[width=0.4\textwidth]{figures/ssIM7.png}
    \caption{A quick histogram of Setosa's sepal length to verify distribution}
\end{figure}

Optionally, you can try and repeat this to check another species or another feature to see if it is also normally distributed.

\subsubsection{Student's t-Test}

Having evaluated our data and how 'normal' it is, we can proceed to choosing a statistical test. Most commonly used to biology, regardless of how correct or appropriate it is (\href{https://www.statisticsdonewrong.com}{https://www.statisticsdonewrong.com}, this is an excellent read, very funny and requires little maths experience), is the t-test. More specifically Student's t-test.

Here, we perform a t-test between the sepal length of setosa and virginica plants.

\begin{lstlisting}[language=Python]
    from scipy import stats
    setosa = df['species'] == 'setosa'
    virginica = df['species'] == 'virginica'
    setosa_sepal_length = df[setosa]['sepal_length']
    virginica_sepal_length = df[virginica]['sepal_length']
    stats.ttest_ind(setosa_sepal_length, virginica_sepal_length, equal_var=False)
\end{lstlisting}

The first value returned is the t-statistic and the second, the commonly reported, p-value.

\subsection{Building and testing a simple linear model}

For modelling, let's start simple. If we look at the previous plot we might be inclined to think that "petal\_length" and "petal\_width" have a linear relationship. So let us build a simple model using,

\begin{equation}
    y = mx+b.
\end{equation}

Using another library this time, scipy, we can quite quickly find the ideal parameters for $m$ and $b$ in this model. We define our function "linear" to take an $x,m$ and $b$ and return a $y$ value. We specify our "xdata" and "ydata" to be "petal\_length" and "petal\_width", respectively. Then putting together with "curve\_fit" we get a "pars" list. This "pars" variable is just the $m$ and $b$ values for the model.

\begin{lstlisting}[language=Python]
    from scipy.optimize import curve_fit

    def linear(x,m,b):
      return m*x+b

    xdata = df['petal_length']
    ydata = df['petal_width']

    pars, cov = curve_fit(f=linear, xdata=xdata, ydata=ydata)
\end{lstlisting}

Now, if we take those $m,b$ values we should be able to make a fairly decent prediction of "petal\_width" given the "petal\_length", if our assumption of linearity is apt. Remember, pars is a list of two values, so we need to pass it to our linear function correctly:

\begin{lstlisting}[language=Python]
    Y_pred = [ ]

    for x in xdata:
      Y_pred.append(linear(x, pars[0], pars[1]))

    plt.scatter(xdata, ydata, c='r')
    plt.plot(xdata, Y_pred)
    plt.xlabel('petal_length')
    plt.ylabel('petal_width')
\end{lstlisting}

\begin{figure}[!h]
    \centering
    \includegraphics[width=0.7\textwidth]{figures/ssIM6.png}
    \caption{Plot testing our linear model with petal width and length measurements}
\end{figure}

This looks like a pretty good fit, but let's put some numbers behind it by calculating the $R^2$ value (calculated using \href{https://en.wikipedia.org/wiki/Coefficient_of_determination}{Coefficient of determination}):

\begin{lstlisting}[language=Python]
    residuals = ydata- linear(xdata, pars[0], pars[1])
    ss_res = np.sum(residuals**2)
    ss_tot = np.sum((ydata-np.mean(ydata))**2)
    r_squared = 1 - (ss_res / ss_tot)
    print(r_squared)
\end{lstlisting}

\clearpage
\appendix
\section{Test solution}
\begin{lstlisting}[language=Python]
import matplotlib.pyplot as plt
import numpy as np

sin2_values = []
sin3_values = []
Xs = np.linspace(0,10, num=1000)

for x in Xs:
    sin2_values.append(np.sin(x)*2 )
    sin3_values.append(np.cbrt(np.sin(x)))

plt.plot(Xs, sin2_values, label="2 sin(x)")
plt.plot(Xs, sin3_values, label='cbrt(sin(x))')
plt.xlabel('x')
plt.ylabel('sin(x)')
plt.legend()
plt.grid()
\end{lstlisting}

\end{document}


\input{../../style/preamble}
\input{../../latex-math/basic-math}
\input{../../latex-math/basic-ml}

\newcommand{\titlefigure}{figure_man/one_vs_all.png}
\newcommand{\learninggoals}{
  \item Reduce a multiclass problem to multiple binary problems in a model-agnostic way
  \item Know one-vs-rest reduction
  \item Know one-vs-one reduction
}

\title{Introduction to Machine Learning}
\date{}

\begin{document}

\lecturechapter{One-vs-Rest and One-vs-One}
\lecture{Introduction to Machine Learning}

\sloppy

\begin{vbframe}{Multiclass to Binary Reduction}

  \begin{itemize}
    \item Assume we have a way to train binary classifiers, either outputting class labels $\hx$, scores $\fx$ or probabilities $\pix$.
    \item We are now looking for a model-agnostic reduction principle to reduce a multiclass problem to the problem of solving \textbf{multiple binary problems}.
    \item Two common approaches are \textbf{one-vs-rest}  and \textbf{one-vs-one} reductions.

  \end{itemize}
\end{vbframe}

\frame{
\frametitle{Codebooks}
How binary problems are generated can be defined by a codebook.\\[0.3cm]
\textbf{Example:}
\begin{table}[]
  \footnotesize
  \begin{tabular}{|c|r|r|r|} \hline
  \textbf{Class}  & \textbf{$f_1{(\xv)}$} & \textbf{$f_2{(\xv)}$}  & \textbf{$f_3{(\xv)}$} \\ \hline
  \textbf{$1$}  &   1                 &  -1                   &  -1                   \\ \hline
  \textbf{$2$}  &  -1                 &  1                   & 1                   \\ \hline
  \textbf{$3$}  &  0                 & 1                   &  -1                   \\ \hline
  \end{tabular}
  \end{table}

\begin{itemize}

 \item The k-th column defines how classes of all observations are encoded in the binary subproblem / for binary classifier $f_k(\xv)$.
    \item Entry $(m, l)$ takes values $\in \{-1, 0, +1\}$
    \begin{itemize}
      \item if $0$, observations of class $\yi = m$ are ignored.
      \item if $1$, observations of class $\yi = m$ are encoded as $1$.
      \item if $- 1$, observations of class $\yi = m$ are encoded as $- 1$.
    \end{itemize}
    \end{itemize}
}

\section{One-vs-Rest}

\begin{vbframe}{One-vs-Rest }

Create $g$ binary subproblems, where in each the $k$-th original class is encoded as $+1$, and all other classes (the \textbf{rest}) as $- 1$.

\begin{table}[]
  \footnotesize
  \begin{tabular}{|c|r|r|r|} \hline
  \textbf{Class}  & \textbf{$f_1{(\xv)}$} & \textbf{$f_2{(\xv)}$}  & \textbf{$f_3{(\xv)}$} \\ \hline
  \textbf{$1$}  &   1                 &  -1                   &  -1                   \\ \hline
  \textbf{$2$}  &  -1                 &  1                   & -1                   \\ \hline
  \textbf{$3$}  &  -1                 & -1                   &  1                   \\ \hline
  \end{tabular}
  \end{table}

    \begin{center}
    \includegraphics[width=0.5\textwidth]{figure_man/one_vs_all.png}
    \end{center}

  \begin{itemize}
    \item Making decisions means applying all classifiers to a sample $\xv \in \Xspace$ and predicting the label $k$ for which the corresponding classifier reports the highest confidence:
    $$
      \hat y = \text{arg max}_{k \in \{1, 2, ..., g\}} \hat f_k(\xv).
    $$

    \item Obtaining calibrated posterior probabilities is not completely trivial, we could

    fit a second-stage, multinomial logistic regression model on our output scores, so with inputs $\left(\hat f_1(\xi), ..., \hat f_g(\xi)\right)$ and outputs $\yi$ as training data.

  \end{itemize}
  \vspace*{0.2cm}

\end{vbframe}

\section{One-vs-One}

\begin{vbframe}{One-vs-One}

    We create $\frac{g(g - 1)}{2}$ binary sub-problems, where each $\D_{k, \tilde k} \subset \D$ only considers observations from a class-pair $\yi \in \{k, \tilde k\}$, other observations are omitted.

    \begin{table}[]
    \begin{tabular}{|c|r|r|r|} \hline
    \textbf{Class}  & \textbf{$f_1{(\xv)}$} & \textbf{$f_2{(\xv)}$}  & \textbf{$f_3{(\xv)}$} \\ \hline
    \textbf{$1$}  &   1                 & -1                   & 0                  \\ \hline
    \textbf{$2$}  &  -1                 &  0                   & 1                   \\ \hline
    \textbf{$3$}  & 0                  &  1                   &  -1                   \\ \hline
    \end{tabular}
    \end{table}

    \begin{center}
    \includegraphics[width=0.38\textwidth]{figure_man/one_vs_one.png}
    \end{center}

    \framebreak

  \begin{itemize}
    \item Label prediction is done via \textbf{majority voting}. We predict the label of a new $\xv$ with all classifiers and select the class that occurred most often.

    \item \textbf{Pairwise coupling} (see \emph{Hastie, T. and Tibshirani, R. (1998). Classification by Pairwise Coupling}) is a heuristic to transform scores obtained by a one-vs-one reduction to probabilities.

  \end{itemize}
  \end{vbframe}

\begin{vbframe}{Comparison one-vs-one and one-vs-rest}
\begin{itemize}
       \item Note that each binary problem has now much less than $n$ observations!
    \item For classifiers that scale (at least) quadratically with the number of observations, this means that one-vs-one usually does not create quadratic extra effort in $g$, but often only approximately linear extra effort in $g$.
    \item We experimentally investigate the train times of the one-vs-rest and one-vs-one approaches for an increasing number of classes $g$.
    \item We train a support vector machine classifier (SVMs will be covered later in the lecture) on an artificial dataset with $n = 1000$.
  \end{itemize}
    \framebreak
    We see that the computational effort for one-vs-one is much higher than for one-vs-rest, but it does not scale proportionally to the (quadratic) number of trained classifiers.
    \begin{center}
    \begin{figure}
    \includegraphics[width=0.6\textwidth]{figure/onevsone_vs_onevsrest}
    \caption{The number of classes vs. the training time (solid lines, left axis) and number of learners (dashed lines, right axis) for each of the two approaches.}
    \end{figure}
    \end{center}

  \end{vbframe}

\endlecture
\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS240: Programming in C
% Copyright 2016 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% https://github.com/ghorbanzade/UMB-CS240-2016S/blob/master/LICENSE
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\def \topDirectory {.}
\def \resDirectory {\topDirectory/src/c/main/ls11}
\def \texDirectory {\topDirectory/src/tex}
\def \styDirectory {\texDirectory/sty}
\def \cfgDirectory {\texDirectory/cfg}
\def \imgDirectory {\texDirectory/img}

\documentclass[compress]{beamer}
%\mode<presentation>
%\usetheme{default}

\usepackage{\styDirectory/directives}
\input{\cfgDirectory/config}
\usepackage{\styDirectory/beamerthemePejman}
\doc{number}{11}
%\setbeamertemplate{footline}[text line]{}

\begin{document}

\prepareCover

\section{Pointers}

\subsection{Pointers and Arrays}

\begin{slide}
	\begin{block}{Introduction}

	Pointers and arrays are closely tied together and almost similar in practice.
	Any operation that can be achieved by array subscripting can also be achieved by pointers.

	\inputminted[
		fontsize=\footnotesize,
		firstline=14,
		lastline=17,
		linenos
	]{c}{\resDirectory/array1.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Accessing Array Elements}

	If pointer $p$ points to element $x_i$ of array $a$, pointer $p+1$ points to element $x_{i + 1}$ of array $a$.

	\inputminted[
		fontsize=\footnotesize,
		firstline=14,
		lastline=19,
		linenos
	]{c}{\resDirectory/array1.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Remember}

	Name of an array is a synonym for the location of the initial element.

	\inputminted[
		fontsize=\footnotesize,
		firstline=14,
		lastline=19,
		linenos
	]{c}{\resDirectory/array2.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Differences}

	\inputminted[
		fontsize=\footnotesize,
		firstline=15,
		lastline=23,
		linenos
	]{c}{\resDirectory/string1.c}

	\end{block}
	\begin{block}{Moral Point}

	A pointer to a character is not a character; it is a pointer.

	\end{block}
\end{slide}

\subsection{Pointers and Functions}

\begin{slide}
	\begin{block}{Exercise}

	Write a function \texttt{strcpy(char *s, char *t)} that copies the string \texttt{t} to the string \texttt{s}.
	You may assume that the destination string is larger than the source.

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution 1}

	\inputminted[
		fontsize=\scriptsize,
		firstline=10,
		linenos
	]{c}{\resDirectory/strcpy1.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution 2}

	\inputminted[
		fontsize=\scriptsize,
		firstline=10,
		linenos
	]{c}{\resDirectory/strcpy2.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution 3}

	\inputminted[
		fontsize=\scriptsize,
		firstline=10,
		linenos
	]{c}{\resDirectory/strcpy3.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{figure}
	\includegraphics[width=0.8\textwidth]{\imgDirectory/array.jpg}
	\end{figure}
\end{slide}

\begin{slide}
	\begin{block}{Exercise}

	Write a function with the following signature that reverses a given array of integer.

	\begin{terminal}
	void reverse(int *array, int size);
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=10,
		lastline=25,
		linenos
	]{c}{\resDirectory/reverse.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=27,
		linenos
	]{c}{\resDirectory/reverse.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Exercise}

	Write a function with the following signature that multiples a square matrix $m$ by itself, given a pointer to element $m_{1,1}$.

	\begin{terminal}
	void matrix(int *m, int size);
	\end{terminal}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=17,
		lastline=28,
		linenos
	]{c}{\resDirectory/matrix.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=30,
		lastline=38,
		linenos
	]{c}{\resDirectory/matrix.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=40,
		lastline=48,
		linenos
	]{c}{\resDirectory/matrix.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=50,
		lastline=61,
		linenos
	]{c}{\resDirectory/matrix.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{block}{Solution}

	\inputminted[
		fontsize=\scriptsize,
		firstline=63,
		lastline=74,
		linenos
	]{c}{\resDirectory/matrix.c}

	\end{block}
\end{slide}

\begin{slide}
	\begin{figure}
	\includegraphics[width=0.6\textwidth]{\imgDirectory/pulp.jpg}
	\end{figure}
\end{slide}

\end{document}


%%
%% Automatically generated file from DocOnce source
%% (https://github.com/hplgit/doconce/)
%%
%%

%-------------------- begin preamble ----------------------

\documentclass[%
oneside,                 % oneside: electronic viewing, twoside: printing
final,                   % draft: marks overfull hboxes, figures with paths
10pt]{article}

\listfiles               %  print all files needed to compile this document

\usepackage{relsize,makeidx,color,setspace,amsmath,amsfonts,amssymb}
\usepackage[table]{xcolor}
\usepackage{bm,ltablex,microtype}

\usepackage[pdftex]{graphicx}

\usepackage[T1]{fontenc}
%\usepackage[latin1]{inputenc}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}

\usepackage{lmodern}         % Latin Modern fonts derived from Computer Modern

% Hyperlinks in PDF:
\definecolor{linkcolor}{rgb}{0,0,0.4}
\usepackage{hyperref}
\hypersetup{
    breaklinks=true,
    colorlinks=true,
    linkcolor=linkcolor,
    urlcolor=linkcolor,
    citecolor=black,
    filecolor=black,
    %filecolor=blue,
    pdfmenubar=true,
    pdftoolbar=true,
    bookmarksdepth=3   % Uncomment (and tweak) for PDF bookmarks with more levels than the TOC
    }
%\hyperbaseurl{}   % hyperlinks are relative to this root

\setcounter{tocdepth}{2}  % levels in table of contents

% --- fancyhdr package for fancy headers ---
\usepackage{fancyhdr}
\fancyhf{} % sets both header and footer to nothing
\renewcommand{\headrulewidth}{0pt}
\fancyfoot[LE,RO]{\thepage}
% Ensure copyright on titlepage (article style) and chapter pages (book style)
\fancypagestyle{plain}{
  \fancyhf{}
  \fancyfoot[C]{{\footnotesize \copyright\ 1999-2019, "Computational Physics I FYS3150/FYS4150":"http://www.uio.no/studier/emner/matnat/fys/FYS3150/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license}}
%  \renewcommand{\footrulewidth}{0mm}
  \renewcommand{\headrulewidth}{0mm}
}
% Ensure copyright on titlepages with \thispagestyle{empty}
\fancypagestyle{empty}{
  \fancyhf{}
  \fancyfoot[C]{{\footnotesize \copyright\ 1999-2019, "Computational Physics I FYS3150/FYS4150":"http://www.uio.no/studier/emner/matnat/fys/FYS3150/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license}}
  \renewcommand{\footrulewidth}{0mm}
  \renewcommand{\headrulewidth}{0mm}
}

\pagestyle{fancy}

% prevent orhpans and widows
\clubpenalty = 10000
\widowpenalty = 10000

% --- end of standard preamble for documents ---

% insert custom LaTeX commands...

\raggedbottom
\makeindex
\usepackage[totoc]{idxlayout}   % for index in the toc
\usepackage[nottoc]{tocbibind}  % for references/bibliography in the toc

%-------------------- end preamble ----------------------

\begin{document}

% matching end for #ifdef PREAMBLE

\newcommand{\exercisesection}[1]{\subsection*{#1}}

% ------------------- main content ----------------------

% ----------------- title -------------------------

\thispagestyle{empty}

\begin{center}
{\LARGE\bf
\begin{spacing}{1.25}
Project 3, deadline  October 21 at midnight (11.59pm/23.59)
\end{spacing}
}
\end{center}

% ----------------- author(s) -------------------------

\begin{center}
{\bf \href{{http://www.uio.no/studier/emner/matnat/fys/FYS3150/index-eng.html}}{Computational Physics I FYS3150/FYS4150}}
\end{center}

    \begin{center}
% List of all institutions:
\centerline{{\small Department of Physics, University of Oslo, Norway}}
\end{center}

% ----------------- end author(s) -------------------------

% --- begin date ---
\begin{center}
Fall semester 2019
\end{center}
% --- end date ---

\vspace{1cm}

\subsection*{Project 3, numerical integration, deadline  October 21}

The task of this project is to integrate first in a brute force manner
a six-dimensional integral which is used to determine the ground state
correlation energy between two electrons in a helium atom.  The
integral appears in many quantum mechanical applications.  However, if
you are not too familiar with quantum mechanics, you can simply look
at the mathematical details.  We will employ both Gauss-Legendre and
Gauss-Laguerre quadrature and Monte-Carlo integration.  Furthermore,
you will need to parallelize your codes.

We assume that the wave function of each electron can be modelled like
the single-particle wave function of an electron in the hydrogen
atom. The single-particle wave function for an electron $i$ in the
$1s$ state is given in terms of a dimensionless variable (the wave
function is not properly normalized)

\[
   {\bf r}_i =  x_i {\bf e}_x + y_i {\bf e}_y +z_i {\bf e}_z ,
\]
as
\[
   \psi_{1s}({\bf r}_i)  =   e^{-\alpha r_i},
\]
where $\alpha$ is a parameter and
\[
r_i = \sqrt{x_i^2+y_i^2+z_i^2}.
\]

We will fix $\alpha=2$, which should correspond to the charge of the helium atom $Z=2$.

The ansatz for the wave function for two electrons is then given by the product of two
so-called
$1s$ wave functions as

\[
   \Psi({\bf r}_1,{\bf r}_2)  =   e^{-\alpha (r_1+r_2)}.
\]

Note that it is not possible to find a closed-form or analytical
solution to Schr\"odinger's equation for two interacting electrons in
the helium atom.

The integral we need to solve is the quantum mechanical expectation
value of the correlation energy between two electrons which repel each
other via the classical Coulomb interaction, namely

\begin{equation}\label{eq:correlationenergy}
   \langle \frac{1}{|{\bf r}_1-{\bf r}_2|} \rangle =
   \int_{-\infty}^{\infty} d{\bf r}_1d{\bf r}_2  e^{-2\alpha (r_1+r_2)}\frac{1}{|{\bf r}_1-{\bf r}_2|}.
\end{equation}

Note that our wave function is not normalized. There is a
normalization factor missing, but for this project we don't need to
worry about that.

This integral can be solved in closed form and the answer is
$5\pi^2/16^2$. Can you derive this value?

\textbf{For this project you can hand in collaborative reports and programs.}

\paragraph{Project  3a): Gauss-Legendre Quadrature.}
Use Gauss-Legendre quadrature and compute the integral by integrating
for each variable $x_1,y_1,z_1,x_2,y_2,z_2$ from $-\infty$ to
$\infty$.  How many mesh points do you need before the results
converges at the level of the third leading digit?  Hint: the
single-particle wave function $e^{-\alpha r_i}$ is more or less zero
at $r_i \approx \lambda$ (find the appropriate limit).  You can therefore
replace the integration limits $-\infty$ and $\infty$ with $-\lambda$ and
$+\lambda$, respectively.  You need to check that this approximation is
satisfactory, that is, make a plot of the function and check if the
abovementioned limits are appropriate.  You need also to account for
the potential problems which may arise when $|{\bf r}_1-{\bf r}_2|=0$.

\paragraph{Project  3b): Improved Gauss-Quadrature.}
The Legendre polynomials are defined for $x\in [-1,1]$. The previous
exercise gave a very unsatisfactory ad hoc procedure. We wish to
improve our results. It can therefore be useful to change to another
coordinate frame and employ the Laguerre polynomials. The Laguerre
polynomials are defined for $x\in [0,\infty)$ and if we change to
spherical coordinates

\[
   d{\bf r}_1d{\bf r}_2  = r_1^2dr_1 r_2^2dr_2 dcos(\theta_1)dcos(\theta_2)d\phi_1d\phi_2,
\]

with

\[
   \frac{1}{r_{12}}= \frac{1}{\sqrt{r_1^2+r_2^2-2r_1r_2cos(\beta)}}
\]

and

\[
cos(\beta) = cos(\theta_1)cos(\theta_2)+sin(\theta_1)sin(\theta_2)cos(\phi_1-\phi_2))
\]

we can rewrite the above integral with different integration
limits. Find these limits and replace the Gauss-Legendre approach in
a) with Laguerre polynomials.  The function gauss-laguerre.cpp in the \href{{https://github.com/CompPhysics/ComputationalPhysics/tree/master/doc/Projects/2019/Project3/CodeExamples}}{CodeExamples} folded can be
used.  Do your results
improve? Compare with the results from a).

\textbf{Important notice for c++ programmers}: the function which computes the Gauss-Laguerre integration points and weights returns arrays which start at $1$ and end $n$ instead of the default values $0$ and $n-1$. You need to declare an array of length $n+1$.

\paragraph{Project  3c): Monte Carlo Integration.}
Compute the same integral but now with brute force Monte Carlo
and compare your results with those from the previous points. Discuss the differences.
With bruce force we mean that you should use the uniform distribution.

\paragraph{Project  3d): Improved Monte Carlo Integration.}
Improve your brute force Monte Carlo calculation by using importance sampling.
Hint: use the exponential distribution and transform to spherical coordinates.
Does the variance decrease? Does the CPU time used compared with the brute force
Monte Carlo decrease in order to achieve the same accuracy? Comment your results
and make a list over the time each method uses. Compare the results also.

\paragraph{Project  3e): Improved Monte Carlo Integration and Parallization.}
Finally, for the last exercise you should parallelize your code using
openMP or MPI. Time your program with various compiler flags and
comment these results as well. In particular, we want to see whether
you achieve an optimal speed-up or not.

\subsection*{Introduction to numerical projects}

Here follows a brief recipe and recommendation on how to write a report for each
project.

\begin{itemize}
  \item Give a short description of the nature of the problem and the eventual  numerical methods you have used.

  \item Describe the algorithm you have used and/or developed. Here you may find it convenient to use pseudocoding. In many cases you can describe the algorithm in the program itself.

  \item Include the source code of your program. Comment your program properly.

  \item If possible, try to find analytic solutions, or known limits in order to test your program when developing the code.

  \item Include your results either in figure form or in a table. Remember to        label your results. All tables and figures should have relevant captions        and labels on the axes.

  \item Try to evaluate the reliabilty and numerical stability/precision of your results. If possible, include a qualitative and/or quantitative discussion of the numerical stability, eventual loss of precision etc.

  \item Try to give an interpretation of you results in your answers to  the problems.

  \item Critique: if possible include your comments and reflections about the  exercise, whether you felt you learnt something, ideas for improvements and  other thoughts you've made when solving the exercise. We wish to keep this course at the interactive level and your comments can help us improve it.

  \item Try to establish a practice where you log your work at the  computerlab. You may find such a logbook very handy at later stages in your work, especially when you don't properly remember  what a previous test version  of your program did. Here you could also record  the time spent on solving the exercise, various algorithms you may have tested or other topics which you feel worthy of mentioning.
\end{itemize}

\noindent
\subsection*{Format for electronic delivery of report and programs}

The preferred format for the report is a PDF file. You can also use DOC or postscript formats or as an ipython notebook file.  As programming language we prefer that you choose between C/C++, Fortran2008 or Python. The following prescription should be followed when preparing the report:

\begin{itemize}
  \item Use Devilry to hand in your projects, log in  at  \href{{http://devilry.ifi.uio.no}}{\nolinkurl{http://devilry.ifi.uio.no}} with your normal UiO username and password and choose either 'fys3150' or 'fys4150'. There you can load up the files within the deadline.

  \item Upload \textbf{only} the report file!  For the source code file(s) you have developed please provide us with your link to your github domain.  The report file should include all of your discussions and a list of the codes you have developed.  Do not include library files which are available at the course homepage, unless you have made specific changes to them.

  \item In your git repository, please include a folder which contains selected results. These can be in the form of output from your code for a selected set of runs and input parametxers.

  \item In this and all later projects, you should include tests (for example unit tests) of your code(s).

  \item Comments  from us on your projects, approval or not, corrections to be made  etc can be found under your Devilry domain and are only visible to you and the teachers of the course.
\end{itemize}

\noindent
Finally,
we encourage you to work two and two together. Optimal working groups consist of
2-3 students. You can then hand in a common report.

% ------------------- end of main content ---------------

\end{document}


\documentclass{article}
\pagestyle{empty}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{graphicx}
\usepackage{multicol}
\setlength{\oddsidemargin}{0in} \setlength{\evensidemargin}{0in}
\setlength{\topmargin}{0in} \setlength{\textheight}{8.5in}
\setlength{\textwidth}{6.5in}

\begin{document}
\begin{flushleft}
	\bfseries{MATH 260, Linear Algebra, Spring `14}\\
	\bfseries{Activity 1:  Linear equations and systems, geometrically}\\
	\bfseries{Honor Code:} \hspace{3.5in}\bfseries{Names:}\\
\end{flushleft}
\begin{flushleft}
\vspace{.25in}

Directions:  Everyone should work on the assignment and should fill out their paper.  You are expected to make corrections based on what is presented on the board.  You will NOT turn in the assignment today.  It may be collected in the future.

\vspace{0.2in}

In previous courses you learned about lines and plotting. Today we'll be reviewing a lot of that as a precursor to a more holistic view of systems and LINEar algebra.

\section*{Problem 1:Plotting 2 variables}
\vspace{0.1in}

a) The point-slope form of a line is $y=mx+b$. This is generally easily graphed, most of our equations though will take `standard' form of: $a_1 x + a_2 y = b$. For practice plot the line: $3y-6x=6$.
\vspace{0.1in}
\begin{minipage}{3in}
\includegraphics[scale=0.75]{grid_12_by_12.eps}
\end{minipage}
\vspace{0.2in}

\newpage
b) Now sketch the following pairs of equations:\\
(i) $y=2-4x$ and $y=-4x-3$\\
(ii) $3x+7=y$ and $y=\frac{-2}{3}x-1$\\
(iii) $y=2x+1$ and $3y-3=6x$

\vspace{0.25in}

\begin{minipage}{3in}
\includegraphics[scale=0.75]{grid_12_by_12.eps}
\end{minipage}
\begin{minipage}{3in}
\includegraphics[scale=0.75]{grid_12_by_12.eps}
\end{minipage}
\vspace{0.1in}\\
\begin{minipage}{3in}
\includegraphics[scale=0.75]{grid_12_by_12.eps}
\end{minipage}
\begin{minipage}{3in}
\includegraphics[scale=0.75]{grid_12_by_12.eps}
\end{minipage}

c) When we plot lines, the intersection gives a pair of (x,y) values which will satisfy both equations. How many such pairs do each of the above sets of lines have?\\
\vspace{1in}

Can you draw a pair of lines which has exactly TWO solutions on the fourth grid? Can you come up with any other counts for solutions besides those depicted above?

\newpage
d) Now plot (b-ii) on both of the grids below.\\
(i) On the first also plot $y+1=\frac{-1}{4}x$ \\
(ii) On the second also plot $y=\frac{5}{11}$\\

\vspace{0.2in}

\begin{minipage}{3in}
\includegraphics[scale=.75]{grid_12_by_12.eps}
\end{minipage}
\begin{minipage}{3in}
\includegraphics[scale=.75]{grid_12_by_12.eps}
\end{minipage}
\vspace{0.3in}\\
Similar to having two equations, 3 equations can produce the same types of solution sets: None, one or infinite. Only two of which you've plotted. Notice that we really only needed two equations to get the interesction for (ii), this is called `overdetermined'. Case (i) would be called 'inconsistent' or 'indeterminite'.

\vspace{0.3in}
\newpage
\section*{Problem 2: Plotting 3 variables}
\vspace{0.1in}
First, actually plotting an equation with 3 variables can be very challenging. We've shown the result below which creates a plane.\\

\begin{minipage}{3in}
\includegraphics[scale=0.5]{planepic.eps}
\end{minipage}\\
\vspace{0.1in}

Let's look at another system, this time with 3 variables:\\
$x=3z-2y+1$\\
$3z-2y+x=2$\\
Solve this for $x$, $y$, $z$. \textit{Hint: Solve for $z$ first by substituting the first equation into the second}\\
\vspace{1.5in}
Did you get numbers for each variable? What's do you think is going on here? (Feel free to discuss at your table)\\
\vspace{1in}
\Large Notes about this:
\normalsize
\vspace{2in}

\section*{Problem 3}
Time permitting, solve the following systems by elimination:\\

\end{flushleft}
a)
\begin{align*}
x+y&=4\\
x-y&=0
\end{align*}
\vspace{0.1in}

b)
\begin{equation*}
\begin{array}{ccccccr}
x& + &2y&+&z&= &2\\
x& - & y& & &= &4\\
2x&- & y&+&2z&=&0\\
  &  &3y&+& z&=&-2
\end{array}
\end{equation*}

\end{document}

Spring 2014 - Schmitt/Activity Latex/Activity_1_260.tex


\documentclass{article}
\usepackage[T1]{fontenc}

\usepackage{amssymb}
\usepackage{courier}
\usepackage{fancyhdr}
\usepackage{fancyvrb}
\usepackage[top=.75in, bottom=.75in, left=.75in,right=.75in]{geometry}
\usepackage{graphicx}
\usepackage{lastpage}
\usepackage{listings}
\lstset{basicstyle=\small\ttfamily}
\usepackage{mdframed}
\usepackage{parskip}
\usepackage{soul}
\usepackage{tabularx}
\usepackage{textcomp}
\usepackage{upquote}
\usepackage{xcolor}

% http://www.monperrus.net/martin/copy-pastable-ascii-characters-with-pdftex-pdflatex
\lstset{
upquote=true,
columns=fullflexible,
keepspaces=true,
literate={*}{{\char42}}1
         {-}{{\char45}}1
         {^}{{\char94}}1
}
\lstset{
  moredelim=**[is][\color{blue}\bf\small\ttfamily]{@}{@},
}

% http://tex.stackexchange.com/questions/40863/parskip-inserts-extra-space-after-floats-and-listings
\lstset{aboveskip=6pt plus 2pt minus 2pt, belowskip=-4pt plus 2pt minus 2pt}

\usepackage[colorlinks,urlcolor={blue}]{hyperref}
\usepackage[capitalise,nameinlink,noabbrev]{cleveref}
\crefname{section}{Question}{Questions}
\Crefname{section}{Question}{Questions}

\begin{document}

\fancyfoot[L]{\color{gray} C4CS -- W'17}
\fancyfoot[R]{\color{gray} Revision 1.1}
\fancyfoot[C]{\color{gray} \thepage~/~\pageref*{LastPage}}
\pagestyle{fancyplain}

\title{\textbf{Homework 11\\Debugging}}
\author{\textbf{\color{red}{Due: Saturday, March 25, 10:00PM (Hard Deadline)}}}
\date{}
\maketitle

\section*{Submission Instructions}
Submit this assignment on \href{https://gradescope.com/courses/5574}{Gradescope}.
You must submit every page of this PDF.
We recommend using the free online tool \href{https://www.pdfescape.com}{PDFescape}
to edit and fill out this PDF.
You may also print, handwrite, and scan this assignment.

There may multiple answers for each question. If you are unsure,
state your assumptions and your reasoning for why you think your answer
makes sense.

\newpage
\section{\texttt{Git}ting started}

Please \texttt{clone} the following repository:

\begin{lstlisting}
@>@ git clone https://github.com/c4cs/debugging-basics.git
\end{lstlisting}

This repository contains 4 branches: \texttt{master}, \texttt{gdb-debug-1}, \texttt{gdb-debug-2}, and \texttt{valgrind-debug}. Each part of this homework will take place on a different branch.

\section{Finding primes}

The repository you have cloned is an attempt at a program to find prime
numbers. It will search from 3 up to a maximum number input by the user.

The program intends to follow the structure:
\begin{lstlisting}
  o prompt user for upper bound
  o for N=3..upper bound:
    - check if N is prime
      * if no prime n between 1..sqrt(N) divides N, then prime
    - save whether N is prime for future loops to use
    - if N is prime, print N
\end{lstlisting}

\subsection{Debugging with \texttt{gdb}}

\emph{You may find it helpful to consult the
  \href{http://c4cs.github.io/static/lecture/wk7-gdb.pdf}{gdb lecture notes}
  or this
  \href{https://ccrma.stanford.edu/~jos/stkintro/Useful_commands_gdb.html}
  {quick command reference}.
}

First, make sure you are on the \texttt{gdb-debug-1} branch:
\begin{lstlisting}
@>@ git branch
  master
* gdb-debug-1
\end{lstlisting}

Next, build and run the supplied program:
\begin{lstlisting}
@>@ make
@>@ ./prime
Find all prime numbers between 3 and ?
10
Segmentation fault (core dumped)
@>@ gdb -q ./prime
Reading symbols from ./prime...done.
(gdb) run
Starting program: /media/sf_prime/prime
Find all prime numbers between 3 and ?
10

Program received signal SIGSEGV, Segmentation fault.
0x000000000040068f in check_prime (k=3) at check.c:15
@15      if (is_prime[j] == 1)@
\end{lstlisting}

\textbf{Explain in what case(s) executing this line of code could cause a
  segmentation fault?}
\vspace{3cm}

\textbf{What gdb commands could you run next to prove your hypothesis right or
  wrong?}
\vspace{3cm}

\newpage
% Fun fact: This was a "real" bug, that is, when I was writing this assignment
% I made a mistake and used the debugger to track it down
Now checkout \texttt{gdb-debug-2}:
\begin{lstlisting}
@>@ git checkout gdb-debug-2
@>@ make
@>@ ./prime
Find all prime numbers between 3 and ?
10
3 is a prime
5 is a prime
7 is a prime
@9 is a prime@
\end{lstlisting}
\textbf{Copy your debugging session and add notes explaining your
  thought process as you track down why this program thinks 9 is a prime.}\\
\emph{\small Hint: It looks like things go well up until 9. A good place to
  start then may be to break in and observe how the code determines whether 9
  is a prime number.}

\vfill
\hrule
{\footnotesize
This homework is based off of a simplified and updated version of this gdb
tutorial:
\url{http://heather.cs.ucdavis.edu/~matloff/UnixAndC/CLanguage/Debug.html}
If you would like some more practice, or simply a slightly different
explanation of the material, this may be a good source.
}

\newpage
\section{Valgrind}

Valgrind is a different kind of debugging tool. It is essentially a type of
virtual machine (like a simpler VirtualBox). Valgrind actually recompiles
every assembly instruction to allow it to intercept and monitor all of the
hardware calls made by the child process. While this is very powerful and
gives valgrind a lot of insight, it is also very slow, which can be
problematic for debugging large pieces of software.

Valgrind also can be challenging when libraries (such as the STL, or Boost) do
funny things with memory that result in a large number of false warnings. We
will look at valgrind in more depth during the second week on debugging
tools. For today, let's just explore the basic functionality and check out the
kind of things that valgrind can catch that gdb can't.

First, we'll need to install valgrind
\begin{lstlisting}
@>@ sudo apt-get install valgrind
\end{lstlisting}

Now, checkout the valgrind branch
\begin{lstlisting}
@>@ git checkout valgrind-debug
\end{lstlisting}

Notice (perhaps \texttt{gitk~-{}-all}) that the valgrind branch builds on the
master branch, with all the compiler warnings turned on. It has also
\ul{cherry-pick}ed the fix ``Stop searching for primes once sqrt(k) is
reached'', that is the same commit object that managed to end up on two
different histories.  This is one example of how git can be both very
powerful, and very confusing.

Finally, this branch adds a commit that consolidates they two source code
files into one and gets rid of all of the global variables in the process.
Unfortunately, this refactoring may also have inroduced a bug.

We run valgrind very similarly to gdb:
\begin{lstlisting}
@>@ make
@>@ valgrind ./prime
==11959== Memcheck, a memory error detector
==11959== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==11959== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==11959== Command: ./prime
==11959==
Find all prime numbers between 3 and ?
10
3 is a prime@
==11959== Conditional jump or move depends on uninitialised value(s)
==11959==    at 0x400683: check_prime (prime.c:32)
==11959==    by 0x400739: main (prime.c:52)@
==11959==
5 is a prime
7 is a prime
==11959==
==11959== HEAP SUMMARY:
==11959==     in use at exit: 0 bytes in 0 blocks
==11959==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==11959==
==11959== All heap blocks were freed -- no leaks are possible
==11959==
==11959== For counts of detected and suppressed errors, rerun with: -v
==11959== Use --track-origins=yes to see where uninitialised values come from
==11959== ERROR SUMMARY: 3 errors from 1 contexts (suppressed: 0 from 0)
\end{lstlisting}
{\small (The numbers in the left column are the process ID. They will be
  different for every person)}

\newpage
\textbf{Notice that ``3 is a prime'' printed before this warning. Why was this
  warning not emitted when running \texttt{check\_prime(3,~\dots)}?}
\vspace{2cm}

These tools really get powerful when you combine them. We can run valgrind in
a way that lets gdb connect to it, and allows us to debug/inspect whenever
valgrind detects a problem:
\begin{lstlisting}
@>@ valgrind --vgdb=yes --vgdb-error=0 ./prime
\end{lstlisting}
In another terminal, follow the directions that valgrind prints to connect
gdb. In this case, valgrind has already `run' the program for you and inserted
a breakpoint at the very beginning of the program, we just need to `continue'
it. The program will run until valgrind encounters an issue, at which point
valgrind will automatically break for you.
\begin{lstlisting}
@(gdb)@ continue

valgrind terminal:
==23589== Conditional jump or move depends on uninitialised value(s)
==23589==    at 0x400623: check_prime (prime.c:32)
==23589==    by 0x4006C4: main (prime.c:52)
==23589==
@==23589== (action on error) vgdb me ...  # this is where valgrind waits for gdb@

gdb terminal:
Program received signal SIGTRAP, Trace/breakpoint trap.
0x0000000000400623 in check_prime (k=5, is_prime=0xffefff910) at prime.c:32
@32      if (is_prime[j] == 1)   # this is the problematic line of code@
\end{lstlisting}

~

\textbf{What is the value of \texttt{j} at this point?}\\
\vspace{1cm}

\textbf{What is the value of \texttt{is\_prime[j]} at the point?}\\
\vspace{1cm}

\textbf{Use git to look at the most recent commit. What line of code was
  deleted that should not have been?}

\end{document}


\documentclass[serif,xcolor=pdftex,dvipsnames,table,hyperref={bookmarks=false,breaklinks}]{beamer}

\input{../config.tex}

\settitlecard{10}{Probability 2}

\begin{document}

\maketitlepage

% \section{Announcements}
% \subsection{Foo}
%
% \begin{frame}[t]{Announcements}
% \end{frame}

\section{Visualizing Distributions}
\subsection{Foo}

\begin{frame}[t]{Histograms and Counting}
	\begin{itemize}[<+->]
		\item An important tool for exploring data is visualizing the distribution it came from.
		\begin{itemize}[<+->]
			\item Allows us to verify and inform modeling decisions.
			\item Summary statistics (e.g. mean, standard deviation, skewness, kurtosis, etc.), while important and useful, discard information about the data.
		\end{itemize}
		\item A flexible tool for visualizing data distributions is the histogram.
	\end{itemize}
	\pause
	\centering
	\includegraphics[height=1in]{{../Figures/histogram}.png}
\end{frame}

\begin{frame}[t,fragile]{Histograms and Counting}
	% Numpy functions
	Numpy provides functions for counting and computing histograms.

	\begin{lstlisting}
		>>> import numpy as np
		>>> A = np.random.randn(1000)
		>>> np.histogram(A,nbins=5)
		>>> counts,bin_edges = np.histogram(A,bins=7)
		>>> counts
		array([ 38, 180, 381, 289,  98,  11,   3])
		>>> bin_edges
		array([-2.75925765, -1.77917762, -0.7990976 ,  0.18098242,  1.16106244,
		        2.14114246,  3.12122248,  4.1013025 ])
	\end{lstlisting}
\end{frame}

\begin{frame}[t,fragile]{Histograms in Python}
	\begin{itemize}[<+->]
		\item There are a variety of tools for plotting distributions:
		\begin{itemize}[<+->]
			\item \verb|matplotlib.pyplot.histogram| computes and plots a histogram from an array.
			\item \verb|pandas.DataFrame.hist| plots a histogram directly from a \verb|pandas.DataFrame|.
			\item \verb|Seaborn| is an extension to \verb|matplotlib| design explicitly for plotting distributions.
		\end{itemize}
	\end{itemize}
	% Seaborn Demo
\end{frame}

\section{Sampling}
\subsection{Foo}

\begin{frame}[t]{Applications of Randomness}
	Randomness is a core operation for many commonly used algorithms:
	\pause
	\begin{itemize}[<+->]
		\item Cryptography and encryption.
		\item Randomized storage (e.g. random hash tables used in Python dictionaries).
		\item Randomized communications (e.g. LAN).
		\item Approximate integration (we will look at this later).
		\item Randomized experiments.
		\item Statistical modeling:
		\begin{itemize}[<+->]
			\item Evaluating models.
			\item Approximate inference.
			\item Approximate estimation.
		\end{itemize}
		\item \textbf{Many} other randomized algorithms.
	\end{itemize}
\end{frame}

\begin{frame}[t]{Random Sampling}
	\begin{itemize}[<+->]
		\item Problem: Given a distribution $p(x)$ we would like to generate samples from this distribution.
		\item Example: To generate samples from a Bernoulli distribution (binary), flip a bunch of coins. The result of each coin flip is a sample from the Bernoulli.
	\end{itemize}
\end{frame}

\begin{frame}[t]{Types of Sampling}
	As you might imagine, sampling has been the focus of intense study for a long time. Some common sampling algorithms are:
	\pause
	\begin{itemize}[<+->]
		\item Inverse CDF sampling (we will look at this algorithm).
		\item Rejection sampling.
		\item Markov Chain Monte Carlo (MCMC).
		\item Metropolis-Hastings.
		\item Slice sampling.
		\item The list goes on.
	\end{itemize}
\end{frame}

\begin{frame}[t]{Cumulative Distribution Functions}
	Given a distribution $p(x)$ for a random variable $X$, the associated Cumulative Distribution Function (CDF) $F(x)$, gives the probability that $X$ is less then some value, i.e. $F(x) = P(X \leq x)$.

	\pause
	\begin{block}{Cumulative Distribution Function}
		The CDF $F(x)$ for a distribution $p(x)$ is given by,

		$$F(x) = \int_{-\infty}^x p(x') dx'$$

		for continuous variables and by

		$$F(x) = \sum_{x' = -\infty}^{x} P(x')$$

		for discrete variables.
	\end{block}
\end{frame}

\begin{frame}[t]{Cumulative Distribution Functions: Multinomial}
	For example: Consider the CDF for $Multinomial(1,\frac{1}{6},...,\frac{1}{6})$ (i.e. a six sided dice roll).
	\pause
	\begin{align*}
		F(x) &= P(X \leq x) = \sum_{i = 1}^{x} \frac{1}{6}\\
		F(3) &= P(X \leq 3) = \frac{3}{6} = \frac{1}{2}
	\end{align*}

	\pause
	% multinomial CDF
\end{frame}

\begin{frame}[t]{Sampling Algorithms: Inverse CDF Sampling}
	One of the most fundamental sampling algorithms is inverse CDF sampling. For any invertible CDF, we can use the following algorithm.
	\pause
	\begin{block}{Inverse CDF Sampling}
		Let $F(x)$ be an invertible CDF. Then we can sample from the associated distribution by:
		\begin{enumerate}
			\item Sample $u^{(s)}$ from a Uniform(0,1).
			\item $x^{(s)} = F^{-1}(U)$
		\end{enumerate}
	\end{block}
\end{frame}

% \begin{frame}[t]{Sampling Algorithms: Inverse CDF Sampling}
% 	% Multinomial
% \end{frame}
%
% \begin{frame}[t]{Sampling Algorithms: Inverse CDF Sampling}
% 	% Gaussian
% \end{frame}

\begin{frame}[t]{Monte Carlo Integration}
	\begin{itemize}[<+->]
		\item \textbf{Monte Carlo Integration} is a method for approximating the value of an integral that is difficult (or impossible) to compute.
		\item The core idea is based on approximating \textbf{expected values}.
	\end{itemize}
\end{frame}

\begin{frame}[t]{Expected Values}
	A common computation when working with probability distributions is an expected value.

	\pause
	\begin{block}{Expected Value}
		Given a distribution $p(x)$ over a random variable $X$ and a function of the random variable $f:\Omega\to \mathbb{R}$, the expected valued of the function is defined as

		$$\mathbb{E}[f(X)] = \int_\Omega p(x)f(x)dx$$

		where the integral is a sum when $X$ is discrete.
	\end{block}
\end{frame}

\begin{frame}[t]{Expected Values: Multinomial}
	\begin{itemize}[<+->]
		\item Let $X$ be the result of a fair six-sided dice roll.
		\item What is $\mathbb{E}[X]$?
	\end{itemize}

	\pause
	\begin{align*}
		\mathbb{E}[X] &= \sum_x P(X=x)\cdot X\\
		&= \sum_{x=1}^6 \frac{1}{6}\cdot x\\
		&= \frac{1}{6}(1+2+3+4+5+6) = 3.5
	\end{align*}
\end{frame}

\begin{frame}[t]{Expected Values: Normal}
	\begin{itemize}[<+->]
		\item Let $X$ be a Normally distributed random variable with mean $\mu$ and standard deviation $\sigma$.
		\item What is $\mathbb{E}[X^2]$?
	\end{itemize}
	\pause
	\begin{align*}
		\mathbb{E}[X^2] &= \int_x \frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)x^2 dx\\
		&= \frac{1}{\sigma\sqrt{2\pi}}\int_x \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)x^2 dx\\
		&= \frac{1}{\sigma\sqrt{2\pi}}\int_x \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)x^2 dx\\
	\end{align*}
\end{frame}

\begin{frame}[t]{The Importance of Expected Values}
\end{frame}

\begin{frame}[t]{Difficult Expected Values}
	\begin{itemize}[<+->]
		\item In many cases, the expected value is too complex to evaluate.
		\item This may occur because:
		\begin{itemize}[<+->]
			\item The normalization constant of the distribution is difficult to evaluate.
			\item There is not analytical solution to the integral.
		\end{itemize}
		\item What do we do? Approximate!
		\item Types of approximations:
		\begin{enumerate}[<+->]
			\item Approximate the distribution with a simpler one that allows us to take the expectation (e.g. Variational Bayes).
			\item Upper and lower-bound based approximations (e.g. Variational Approximations).
			\item Sampling based approximations (Monte Carlo Integration).
		\end{enumerate}
	\end{itemize}
\end{frame}

\begin{frame}[t]{Monte Carlo Integration}
	\textbf{Monte Carlo integration} approximates an expected value by sampling from the associated distribution and then taking an average.
	\pause
	\begin{block}{Monte Carlo Integration}
		Let $X$ be a random variable drawn from $p(x)$, let $f(x)$ be a function of $X$, and let $x^{(1)},...,x^{(S)}$ be samples from $p(x)$, then

		$$\mathbb{E}(f(x)) \approx \frac{1}{S}\sum_s x^{(s)}$$
	\end{block}
\end{frame}

% \begin{frame}[t]{Monte Carlo Integration: Mean of a Normal}
%
% \end{frame}
%
% \begin{frame}[t]{Monte Carlo Integration: Variance of a Normal}
%
% \end{frame}

\begin{frame}[t]{Monte Carlo Integration}
	How can we use Monte Carlo integration to calculate general integrals?

	\pause
	\begin{align*}
		\int_a^b f(x) dx &= \int_a^b \frac{b-a}{b-a} f(x) dx\\
		&= (b-a) \int_a^b \frac{1}{b-a} f(x) dx\\
		&= (b-a) \mathbb{E}[f(X)]\\
	\end{align*}

	where $X$ is draw from $Uniform(a,b)$.

	\pause
	\begin{itemize}[<+->]
		\item Now we can approximate this expected value using sampling.
		\item This works for multidimensional integrals as well.
	\end{itemize}
\end{frame}

\begin{frame}[t]{Monte Carlo Integration: Area of a Circle}
	\begin{itemize}[<+->]
		\item How can we use Monte Carlo integration to estimate the area of a circle with radius $r$?
		\item Step 1: Formulate the problem as a bounded integral.
	\end{itemize}

	\pause
	A circle is defined by $x^2 + y^2 = r^2$ so we can define a piecewise function that tells us whether we are inside or outside of the circle.
	\begin{align*}
		f(x,y) &= \left\{
			\begin{array}{ll}
				1 & : x^2 + y^2 \leq r^2\\
				0 & : x^2 + y^2 > r^2
			\end{array}
		\right.
	\end{align*}
	\pause
	The area of a circle is given by
	\begin{align*}
		A_r = \int_{-r}^r \int_{-r}^r f(x,y) dx dy
	\end{align*}

\end{frame}

\begin{frame}[t]{Monte Carlo Integration: Area of a Circle}
	\begin{itemize}[<+->]
		\item Now that the area is formulated as an integral, how do we approximate it with sampling?
		\item Step 1: Identify the sampling distribution.
	\end{itemize}

	\pause
	\begin{itemize}[<+->]
		\item In this case we are integrating over the $2r \times 2r$ square.
		\item How do we sample uniformly from a square?
		\item Sample $x$ and $y$ independently from $Uniform(-r,r)$
	\end{itemize}
	% plot

\end{frame}

\begin{frame}[t]{Monte Carlo Integration: Area of a Circle}
	Now we can estimate the area of a circle as:

	\begin{align*}
		A_r &= \int_{-r}^r \int_{-r}^r f(x,y) dx dy\\
		&\approx 2r^2 \frac{1}{S}\sum_s f(x^{(s)},y^{(s)})
	\end{align*}

	where

	\begin{align*}
		f(x,y) &= = \left\{
			\begin{array}{ll}
				1 & : x^2 + y^2 <= r^2\\
				0 & : x^2 + y^2 > r^2
			\end{array}
		\right.
	\end{align*}

\end{frame}

\begin{frame}[t]{Monte Carlo Integration: Area of a Circle}
	% Animation
\end{frame}

\section{Sources of Randomness}
\subsection{Foo}

\begin{frame}[t]{Sources of Randomness}
	\begin{itemize}[<+->]
		\item Most sampling methods are based on the assumption that we can generate a uniform random number.
		\item How do we generate a uniform random number?
		\item There are two main methods:
		\begin{enumerate}[<+->]
			\item Harvested natural randomness
			\item Pseudo-random number generation
		\end{enumerate}
	\end{itemize}
\end{frame}

\begin{frame}[t]{Harvesting Natural Randomness}
	\begin{itemize}[<+->]
		\item The idea behind harvesting natural randomness is that there natural phenomenon that are sufficiently random, due to physical entropy, that we can treat them a random.
		\item Examples of harvested sources of natural randomness include:
		\begin{itemize}[<+->]
			\item The amount of current flowing through an electrical circuit is actually a discrete number as it is caused by individual electrons moving through conductor. \textbf{Shot noise} is the random fluctuations in this discrete number.
			\item Source of nuclear radiation decay randomly and the amount of decay can be used as a source of randomness.
			\item When a beam of photons is directed at a semi-transparent mirror, the photons will randomly be reflected or transmitted.
			\item The temperature of a resistor in a circuit.
			\item Atmospheric noise detected by a radio receiver.
		\end{itemize}
		\item The rate at which these can be harvested may be limited.
	\end{itemize}
\end{frame}

\begin{frame}[t]{Pseudo-Random Numbers}
	\begin{itemize}[<+->]
		\item A \textbf{Pseudo-random number generator} returns a sequence of numbers that has the statistical properties of a uniform random sequence, but is actually deterministic given an initial value.
		\item This initial value is called a \textbf{seed}.
		\item Without knowledge of the seed, the sequence should look random.
		\item Many such sequences exist.
		\item Given the seed, computing pseudo-random number is extremely fast.
	\end{itemize}
\end{frame}

\begin{frame}[t,fragile]{Randomness in Numpy}
	\verb|numpy.random| provides functions for generating pseudo-random samples from many common distributions.

	\pause
	\begin{lstlisting}
		>>> np.random.rand(5)
		array([ 0.05321291,  0.7837193 ,  0.21213337,  0.26499685,  0.4338131 ])
		>>> np.random.randn(5)
		array([-1.23805748, -0.18105506,  0.96736693,  1.25355742, -0.08038946])
		>>> np.random.randint(1,100,5)
		array([28, 35, 32, 19, 94])
	\end{lstlisting}
\end{frame}

\begin{frame}[t]{Sources of Randomness}
	\centering
	\Huge{Demo}
\end{frame}

\end{document}