\documentclass{article}

% Docoment-wide setting go in here %
\usepackage{listings}
\usepackage{xcolor}
\usepackage{algxpar}

\definecolor{backgroundcolor}{rgb}{0.9,0.9,0.8}
\definecolor{commentcolor}{rgb}{0,0.6,0}
\definecolor{keywordscolor}{rgb}{0.58,0,0.82}

\lstdefinestyle{codeblock}{
    backgroundcolor=\color{backgroundcolor},
    commentstyle=\color{commentcolor},
    keywordstyle=\color{keywordscolor},
    numberstyle=\tiny,
    stringstyle=\color{magenta},
    basicstyle=\ttfamily\footnotesize,
    breakatwhitespace=false,
    breaklines=true,
    captionpos=b,
    keepspaces=true,
    numbers=left,
    numbersep=5pt,
    showspaces=false,
    showstringspaces=false,
    showtabs=false,
    tabsize=2
}

\newcommand{\upperbound}[1]{
    \textbf{O(}#1\textbf{)}
}

\newcommand{\lowerbound}[1]{
    \textbf{$\Omega$(}#1\textbf{)}
}

\newcommand{\implementation}[3]{
    \textbf{#1:}
    #2
    \textit{#3}
}

\newcommand{\implementationfromfile}[3]{
    \lstset{style=codeblock}
    \implementation{#1}{\lstinputlisting[language=#1]{#2}}{#3}
}

\newcommand{\variable}[1]{
    \textit{\textbf{#1}}
}
%----------------------------------%

\begin{document}
    % Title page %
    \title{\textbf{Countinsort}}
    \author{Florian Rehm}
    \maketitle
    \newpage
    % The actual document starts here %
    \begin{abstract}
        \textit{
            When it comes to sorting elements - in this case numerical values - most of the time we
            lack a lot of information to reach a good overall performance. When we know the maximum key value in
            our list of elements as well as the number of elements we can employ an implementation of Countinsort
            to get them sorted. It basically counts the occurences of each key value and exploits these counts
            as pointers to their final position. Deeper analysis shows that in general the runtime is \textbf{linear}
            upon the number of elements but we can easily produce edge-cases where the overall runtime explodes.
        }
    \end{abstract}
    \newpage
    \tableofcontents
    \newpage
    \section{Problem and Motivation}
        Sorting elements topologically is a problem that we are facing in many different situations. A popular example would be sorting contestants in a leaderboard. Depending on the application we want to - and sometimes even must - aim for the best runtime complexity. We do not want to wait another season to determine the winner of a football-match, do we? Since we have to touch every element at least once we can already tell that we have \lowerbound{n} as our lower boundary in sorting. One way to reach that runtime is an implementation of \textbf{Countingsort}.
    \newpage
    \section{Solution}
        \subsection{Overview}
        In order for \textbf{Countingsort} to work we assume that our input only consists of numbers; more specifically
        positive integer values. The reason for this comes clear in \textit{\textbf{Phase 2}}. In total the algorithm
        consists of three phases.

        \subsection{Phase 1 \textit{(Counting-phase)}}
        This phase is pretty straight forward. We simply go over all our elements and count their occurence. For that we introduce a counting array\variable{Counts} which employs a length of \textbf{k} where \textbf{k} is the
        maximum value a key employed in our input. \textbf{Side note:} Even if unknown finding the maximum key only takes at most \textbf{n} $\in$\upperbound{n} steps.
        \begin{algorithmic}
            \Input{\variable{A} = [$a_1,a_2,...,a_n$]}, \variable{n} = Number of elements in\variable{A}
            \Output{\variable{A'} = [$a'_1,a'_2,...,a'_n$]} with $\forall a'_i,a'_j; 1 \leq i < j\leq n:a'_i\leq a'_j$
            \Function{Countingsort}{A, n}
                \State$k$ = \textit{Maximum(A)}
                \State$Counts$ = New Array of size k\Comment{Filled with zeroes}
                \State
                    \For{$i$ = 1...$n$}
                        \State$value$ = A[i]
                        \State$Counts[value]$ += 1
                    \EndFor
                \State[...]\Comment{End of Counting-Phase}
            \EndFunction
        \end{algorithmic}

        \subsection{Phase 2 \textit{(Pointing Phase)}}
        Now that we know how many times each key exists in our array we need to determine the final position of each
        key. The idea here is going to be that our\variable{Counts} array already holds the repsecive keys in a sorted
        order. The only thing missing is the respective offsets in the sorted array.

        Let $k_1, k_2,...,k_n=\textbf{k}$ be the possible keys going by\variable{Counts} then \variable{Counts($k_i$)}
        represents the remaining numbers of elements of value $k_i$ with $1 \leq i \leq \textbf{n}$. After \textit{\textbf{Phase 1}} these represent the exact counts for each key. For simplicity lets assume that
        every key $k_i$ exists at least once in our input. The lowest key $k_1$ obviously sits at the very beginning
        of the final array. The first element representing the the second lowest key $k_2$ sits at the next position
        after the last element of $k_1$. This pattern applies for every pair of key $k_i,k_j$ with $k_i < k_j$.

        To retreive a list of final positions after \textit{\textbf{Phase 1}} we need to produce...
        \begin{center}
            \variable{Counts($k_i$)}= $\sum_{i=1}^{i-1}\variable{Counts(i)}$
        \end{center}

        The fact that we potentially have no occurences for some of these keys does nothing to the overall idea
        of this pointers adjustment because they will have an initial value of zero and therefore do nothing in
        the sums portraied above. Recursively this whole setup boils down to...
        \begin{center}
            \variable{Counts(i)}= $\variable{Counts(i)} + \variable{Counts(i-1)}$
        \end{center}
        ... beginning from the lowest key going through to the highest key.
        \begin{algorithmic}
            \Function{Countingsort}{A, n}
                \State[...]\Comment{End of Counting-Phase}
                \State
                    \For{$i$ = 2...$k$}
                        \State$Counts[i]$ += $Counts[i - 1]$
                    \EndFor
                \State[...]\Comment{End of Pointing-Phase}
            \EndFunction
        \end{algorithmic}

        \subsection{Phase 3 \textit{(Sorting Phase)}}
        The final part of the algorithms is copying the elements to the respective positions that we now hold in our
        \variable{Counts} array. For that part we go over our input elements another time and for each element we retreive their final position from\variable{Counts}. Finally we copy the element over to the retreived postion
        in a array we call\variable{Result}of length \textbf{n}. This yeilds the following pattern...
        \begin{center}
            \variable{Result[Counts(x)]}=\variable{x}\goodbreak
            \variable{Counts(x)} =\variable{Counts(x)} - 1
        \end{center}
        \begin{algorithmic}
            \Function{Countingsort}{A, n}
                \State[...]\Comment{End of Pointing-Phase}
                \State$A'$ = New Array of size $n$
                \State
                    \For{$i$ = $n$...1}
                        \State$value = A[i]$
                        \State$position = Counts[value]$
                        \State$A'[position] = value$
                        \State$Counts[value]$ -= 1
                    \EndFor
                \Statep{\Return{Test}}
            \EndFunction
        \end{algorithmic}
        \newpage
        \subsection{Pseudocode(s)}
        \begin{algorithmic}
            \Input{\variable{A} = [$a_1,a_2,...,a_n$]}, \variable{n} = Number of elements in\variable{A}
            \Output{\variable{A'} = [$a'_1,a'_2,...,a'_n$]} with $\forall a'_i,a'_j; 1 \leq i < j\leq n:a'_i\leq a'_j$
            \Function{Countingsort}{A, n}
                \State$k$ = \textit{Maximum(A)}
                \State$Counts$ = New Array of size k\Comment{Filled with zeroes}
                \State
                    \For{$i$ = 1...$n$}
                        \State$value$ = A[i]
                        \State$Counts[value]$ += 1
                    \EndFor
                \State
                    \For{$i$ = 2...$k$}
                        \State$Counts[i]$ += $Counts[i - 1]$
                    \EndFor
                \State$A'$ = New Array of size $n$
                \State
                    \For{$i$ = $n$...1}
                        \State$value = A[i]$
                        \State$position = Counts[value]$
                        \State$A'[position] = value$
                        \State$Counts[value]$ -= 1
                    \EndFor
                \Statep{\Return{Test}}
            \EndFunction
        \end{algorithmic}

    \newpage
    \section{Implementations}
        \implementationfromfile{Python}{Python/code.py}{This code represents and implementation in Python. It is
        called \textit{'advanced Countingsort'} because it can already deal with negative values. The basic concept
        however is the same since we only need to normalize the values of our input data to zero which is a linear
        projection that persists topology.}
\end{document}


\chapter{Variables, expressions and statements}

One of the most powerful features of a programming language
is the ability to manipulate {\bf variables}.  Broadly
speaking, a variable is a name that refers to a value. It
might be more accurate to say that a variable is a container
that has a name and holds a value.
\index{variable}

\section{Assignment statements}
\label{variables}
\index{assignment!statement}
\index{statement!assignment}
\index{$=$ assignment operator}
\index{operator!assignment}

An {\bf assignment statement} uses the equal {\tt =} sign
and  gives a value to a variable, but, before you can assign
a value to a variable, you first need to
create the variable by declaring it if does not already
exist:

\begin{verbatim}
> my $message;          # variable declaration, no value yet
> $message = 'And now for something completely different';
And now for something completely different
> my $number = 42;      # variable declaration and assignment
42
> $number = 17;         # new assignment
17
> my $phi = 1.618033988;
1.618033988
>
\end{verbatim}
%
\index{assignment}
\index{statement!assignment}
\index{declaration!variable}
\index{variable!declaration}

This example makes four assignment statements.  The first
assigns a string to a new variable named {\tt \$message};
the second assigns the integer {\tt 42} to {\tt \$number}; the
third reassigns the integer 17 to {\tt \$number}; the fourth
assigns the (approximate) value of the golden ratio to {\tt \$phi}.

There are two important syntax features to understand here.

First, in Perl, variable names start with a so-called
\emph{sigil}, i.e. a special non-alphanumeric character such
as \verb'$', \verb'@', \verb'%', \verb'&' and some others.
This special character
tells us and the Perl compiler (the program that reads
the code of our program and transforms it into computer
instructions) which kind of variable it is. For example, the
\$ character indicates that the variables above all are \emph{scalar variables}, which means that they can contain only one value at any given time. We'll see later other types of variables that may contain more than one value.
\index{sigil}
\index{scalar}
\index{variable!scalar}
\index{scalar}

Second, notice that all three variables above are first introduced by the keyword {\tt my}, which is a way of declaring a new variable. Whenever you create a new variable in Perl, you need to \emph{declare} it, i.e. to tell Perl that you're going to use that new variable, and this is most commonly done  with the {\tt my} keyword, which declares a \emph{lexical} variable: we will explain later what a lexical variable is, let's just say for the time being that this enables you to make your variable local to a limited part of your code. One of the good consequences of the requirement to declare variables before you use them is that, if you accidentally make a typo when writing a variable name, the compiler will usually be able to tell you that you are using a variable which has not been declared previously and thus help you find your error. This has other far-reaching implications which we will examine later.
\index{my}
\index{declaring variables}
\index{variable!declaration}
\index{lexical}
\index{variable!lexical}

When we wrote at the beginning of this section that a variable has
to be declared before it is used (or just when it is used), it plainly
means that the declaration has to be before (or at the point of) the
variable(s first use in the text file containing the program. We will see later that
programs don't necessarily run from top to bottom in the order in which
the lines or code appear in the program file; still, the variable
declaration must be before its use in the text file containing the program.

If you neglect to declare a variable, you get a syntax error:
\index{syntax error}

\begin{verbatim}
> $number = 5;
===SORRY!=== Error while compiling <unknown file>
Variable '$number' is not declared
at <unknown file>:1
------> <BOL><HERE>$number = 5;
>
\end{verbatim}
%
Please remember that you may obtain slightly different
error messages depending on the version of Rakudo you
run. The above message was obtained in February 2016;
with a newer version (October 2016), the same error
is now displayed somewhat more cleanly as:
\begin{verbatim}
>
> $number = 5;
===SORRY!=== Error while compiling:
Variable '$number' is not declared
at line 2
------> <BOL><HERE>$number = 5;
>
\end{verbatim}

\index{state diagram}
\index{diagram!state}

A common way to represent variables on paper is to write the name with
an arrow pointing to its value.  This kind of figure is
called a {\bf state diagram} because it shows what state each of the
variables is in (think of it as the variable's state of mind).
Figure~\ref{fig.state2} shows the result of the previous example.

\begin{figure}
\centerline
{\includegraphics[scale=0.6]{figs/test_5.png}}
\caption{State diagram.}
\label{fig.state2}
\end{figure}

\section{Variable names}
\index{variable}

Programmers generally choose names for their variables that
are meaningful---they document what the variable is used for.

Variable names can be as long as you like.  They can contain
both letters and numbers, but user-defined variable names
can't begin with a number. Variable names are case-sensitive,
i.e. {\tt \$message} is not the same variable as {\tt \$Message}
or {\tt \$MESSAGE}. It is legal to use uppercase letters, but
it is conventional to use only lower case for most variables
names. Some people nonetheless like to use {\tt \$TitleCase}
for their variables or even pure {\tt \$UPPERCASE} for
some special variables.
\index{lower case}
\index{upper case}
\index{title case}
\index{case!lower}
\index{case!upper}
\index{case!title}

\index{Unicode}
\index{ASCII}
Unlike most other programming languages, Perl~6 does not require letters
and digits used in variable names to be plain ASCII.
You can use all kinds of Unicode letters, i.e.
letters from almost any language in the world, so that, for example,
{\tt \$brücke}, {\tt \$payé} or {\tt \$niño} are
valid variable names, which can be useful for non-English
programmers (provided that these Unicode characters are
handled correctly by your text editor and your
screen configuration). Similarly, instead of using
\verb"$phi" for the name of the golden ratio variable,
we might have used the \emph{Greek small letter phi}, \verb'φ'
(Unicode code point U+03C6), just as we could have used
the \emph{Greek small letter pi}, \verb'π',  for the well-known
circle circumference to diameter ratio:
\index{golden ratio}
\index{Unicode}
\index{phi}
\index{pi}

\begin{verbatim}
> my $φ = (5 ** .5 + 1)/2;       # golden ratio
1.61803398874989
> say 'Variable $φ = ', $φ;
variable $φ = 1.61803398874989
> my $π = 4 * atan 1;
3.14159265358979
> # you could also use the pi or π built-in constant:
> pi
3.14159265358979
\end{verbatim}

The underscore character, \verb"_", can appear anywhere in a variable name.
It is often used in names with multiple words, such as
\verb"$your_name" or \verb"$airspeed_of_unladen_swallow".
\index{underscore character}

You may even use dashes to create so-called
``kebab case''\footnote{Because the
words appear to be skewered like pieces of food prepared for
a barbecue.} and name those variables
\verb"$your-name" or \verb"$airspeed-of-unladen-swallow", and this
might make them slightly easier to read: a dash \- is
valid in variable names provided it is immediately followed by an
alphabetical character and preceded by an alphanumerical character.
For example, \verb"$double-click" or \verb"$la-niña" are legitimate
variable names. Similarly, you can use an apostrophe \' (or single
quote) between letters, so \verb"$isn't" or \verb"$o'brien's-age"
are valid identifiers.
\index{dash}
\index{apostrophe}
\index{single quote}
\index{quote!single}
\index{kebab case}
\index{case!kebab}

If you give a variable an illegal name, you get a syntax error:
\index{syntax error}

\begin{verbatim}
> my $76trombones = 'big parade'
===SORRY!=== Error while compiling <unknown file>
Cannot declare a numeric variable
at <unknown file>:1
------> my $76<HERE>trombones = "big parade";
>
> my $more§ = 100000;
===SORRY!=== Error while compiling <unknown file>
Bogus postfix
at <unknown file>:1
------> my $more<HERE>§ = 100000;
(...)
\end{verbatim}
%
{\tt \$76trombones} is illegal because it begins with a number.
{\tt more§} is illegal because it contains an illegal character, {\tt
§}.

If you've ever used another programming language and stumbled across a terse message such as {\tt"SyntaxError: invalid syntax"}, you will notice that the Perl designers have made quite a bit of effort to provide detailed, useful and meaningful error messages.
\index{error message}

Many programming languages have \emph{keywords} or \emph{reserved words} that are part of the syntax, such as {\tt if}, {\tt while} or {\tt for}, and thus cannot be used for identifying variables because this would create an ambiguity. There is no such problem in Perl: since variable names start with a sigil, the compiler is always able to tell the difference between a keyword and a variable. Names such as {\tt \$if} or {\tt \$while} are syntactically valid variable identifiers in Perl (whether such names make sense is a different matter).
\index{sigil}
\index{keyword}
\index{reserved word}

\section{Expressions and statements}
\label{expr_and_statements}

An {\bf expression} is a combination of terms and operators.
Terms may be variables or literals, i.e. constant values such as a number or a string. A value all by itself is considered an expression, and so is
a variable, so the following are all legal expressions:
\index{expression}
\index{term}
\index{literal}

\begin{verbatim}
> 42
42
> my $n = 17;
17
> $n;
17
> $n + 25;
42
>
\end{verbatim}
%
When you type an expression at the prompt, the interpreter
{\bf evaluates} it, which means that it finds the value of
the expression.
In this example, {\tt \$n} has the value 17 and
{\tt \$n + 25} has the value 42.
\index{evaluate}

A {\bf statement} is a unit of code that has an effect, like
creating a variable or displaying a value, and usually needs to end
with a semi-colon {\tt ;} (but the semi-colon can sometimes be omitted
as we will see later).
\index{statement}
\index{semi-colon}

\begin{verbatim}
> my $n = 17;
17
> say $n;
17
\end{verbatim}
%

The first line is an assignment statement that gives a value to
{\tt \$n}.  The second line is a print statement that displays the
value of {\tt \$n}.

When you type a statement and then press {\tt Enter}, the
interpreter {\bf executes} it, which means that it does
whatever the statement says.
\index{execute}

An assignment can be combined with expressions using arithmetic
operators. For example, you might write:
\index{operator}

\begin{verbatim}
> my $answer = 17 + 25;
42
> say $answer;
42
\end{verbatim}
%

The \verb'+' symbol is obviously the addition operator
and, after the assignment statement, the \verb'$answer'
variable contains the result
of the addition. The terms on each side of the operator
(here 17 and 25) are sometimes called the \emph{operands}
of the operation (an addition in this case).
\index{operand}
\index{addition operator}
\index{assignment}

Note that the REPL actually displays the result of the
assignment (the first line with ``42''), so that the
print statement was not really necessary in this
example \emph{under the REPL}; from now on, for the sake of
brevity, we will generally omit the print statements in the
examples where the REPL displays the result.
\index{REPL}

In some cases, you want to add something to a variable
and assign the result to that same variable. This could
be written:

\begin{verbatim}
> my $answer = 17;
17
> $answer = $answer + 25;
42
\end{verbatim}
%

Here, \verb"$answer" is first declared with a value of 17. The next
statement assigns to \verb"$answer" the current value of
\verb"$answer" (i.e. 17) + 25. This is such a common operation
that Perl, like many other programming languages, has a
shortcut for this:

\begin{verbatim}
> my $answer = 17;
17
> $answer += 25;
42
\end{verbatim}
%

\index{$+=$ augmented assignment operator}
The \verb"+=" operator combines the arithmetic addition operator
and the assignment operator to modify a value and apply the result
to a variable in one go, so that \verb"$n += 2" means: take
the current value of \verb"$n", add 2 and assign the result to
\verb"$n". This syntax works with all other arithmetic operators.
For example, \verb"-=" performs similarly a subtraction and an
assignment, \verb"*=" a multiplication and an assignment, etc. It
can even be used with operators other than arithmetic operators,
such as the string concatenation operator, that we will see later.

Adding 1 to a variable is a very common version of this, so that
there is a shortcut to the shortcut, the \emph{increment} operator,
which increments its argument by one, and returns the incremented value:
\index{increment operator}
\index{++ increment operator}
\index{operator!++ (increment)}

\begin{verbatim}
> my $n = 17;
17
> ++$n;
18
> say $n;
18
\end{verbatim}
%
This is called the prefix increment operator, because the \verb"++"
operator is placed before the variable to be incremented. There is
also a postfix version, \verb"$n++", which first returns the current
value and then increments the variable by one. It would not make
a difference in the code snippet above, but the result can be very different
in slightly more complex expressions.

There is also a \verb"--"
decrement operator, which decrements its argument by one and also
exists in a prefix and a postfix form.
\index{decrement operator}
\index{\verb'--' decrement operator}
\index{operator!$--$ (decrement)}

\section{Script mode}

So far we have run Perl in {\bf interactive mode}, which
means that you interact directly with the interpreter (the
REPL). Interactive mode is a good way to get started,
but if you are working with more than a few lines of code,
it can be clumsy and even tedious.
\index{interactive mode}

The alternative is to use a text editor and save code in a file
called a {\bf script} and then run the interpreter in {\bf script mode}
to execute the script.  By convention, Perl~6 scripts have names that
end with {\tt .pl}, {\tt .p6} or {\tt .pl6}.
\index{script}
\index{script mode}

Please make sure that you're really using a \emph{text editor}
and not a \emph{word processing program} (such as MS Word,
OpenOffice or LibreOffice Writer). There is a very large
number of text editors available for free. On Linux, you might use
\emph{vi} (or \emph{vim}), \emph{emacs}, \emph{gEdit} or
\emph{nano}. On Windows, you may use \emph{notepad} (very limited)
or \emph{notepad++}. There are also many cross-platform editors
or integrated development environments (IDEs) providing a
text editor functionality, including \emph{padre}, \emph{eclipse}
or \emph{atom}. Many of these provide various syntax highlighting
capabilities, which might help you use correct syntax (and
find some syntax errors).
\index{syntax!highlighting}
\index{text editor!emacs}
\index{text editor!vi}
\index{text editor!vim}
\index{text editor!gEdit}
\index{text editor!padre}
\index{text editor!eclipse}
\index{text editor!nano}
\index{text editor!notepad++}
\index{text editor!atom}

Once you've saved your code into a file (say, for example,
\verb'my_script.pl6'), you can run the program by issuing
the following command at the operating system prompt (for example
in a Linux console or in a \verb'cmd' window under Windows):
\begin{verbatim}
perl my_script.pl6
\end{verbatim}

Because Perl provides both modes,
you can test bits of code in interactive mode before you put them
in a script.  But there are differences between interactive mode
and script mode that can be confusing.
\index{interactive mode}
\index{script mode}

For example, if you are using Perl~6 as a calculator, you might type

\begin{verbatim}
> my $miles = 26.2;
26.2
> $miles * 1.61;
42.182
\end{verbatim}

The first line assigns a value to {\tt \$miles} and displays that value.
The second line is an expression, so the
interpreter evaluates it and displays the result.  It turns out that a
marathon is about 42~kilometers.

But if you type the same code into a script and run it, you get no
output at all.  In script mode, an expression, all by itself, has no
visible effect.  Perl actually evaluates the expression, but it doesn't
display the value unless you tell it to:

\begin{verbatim}
my $miles = 26.2;
say $miles * 1.61;
\end{verbatim}

This behavior can be confusing at first. Let's examine why.

A script usually contains a sequence of statements.  If there
is more than one statement, the results appear one at a time
as the print statements execute.

For example, the script

\begin{verbatim}
say 1;
my $x = 2;
say $x;
\end{verbatim}
%
produces the output

\begin{verbatim}
1
2
\end{verbatim}
%
The assignment statement produces no output.

To check your understanding, type the following statements in the
Perl interpreter and see what they do:

\begin{verbatim}
5;
my $x = 5;
$x + 1;
\end{verbatim}

Now put the same statements in a script and run it.  What
is the output?  Modify the script by transforming each
expression into a print statement and then run it again.

\section{One-liner mode}

Perl also has a one-liner mode, which enables you to type directly a very
short script at the operating system prompt. Under Windows, it might
look like this:
\index{one-liner mode}
\label{one-liner mode}

\begin{verbatim}
C:\Users\Laurent>perl6 -e "my $value = 42; say 'The answer is ', $value;"
The answer is 42

\end{verbatim}

The {\tt -e} option tells the compiler that the script to be run is not
saved in a file but instead typed at the prompt between quote marks immediately after
this option.

Under Unix and Linux, you would replace double quote marks with apostrophes (or single quotes)
and apostrophes with double quote marks:
\index{apostrophe}
\index{quote mark}

\begin{verbatim}
$  perl6 -e 'my $value = 42; say "The answer is $value";'
The answer is 42

\end{verbatim}

The one-liner above may not seem to be very useful, but
throw-away one-liners can be very practical to perform
simple one-off operations, such as quickly modifying
a file not properly formatted, without having to save a script
into a separate file before running it.

We will not give any additional details about the one-liner
mode here, but will give some more useful examples
later in this book; for example,
Subsection~\ref{one-liner-example},
Subsection~\ref{rot13_oneliner} (solving the ``rot-13'' exercise), or
Subsection~\ref{sol_cartalk} (solving the exercise on
consecutive double letters).

\section{Order of operations}
\index{order of operations}
\index{PEMDAS}
\index{operator precedence}
\index{precedence!operator}

When an expression contains more than one operator, the order of
evaluation depends on the {\bf order of operations} or \emph{operator precedence}.
For mathematical operators, Perl follows mathematical convention.
The acronym {\bf PEMDAS}\footnote{US students are sometimes taught
to use the "Please Excuse My Dear Aunt Sally" mnemonics to remember
the right order of the letters in the acronym} is a useful
way to remember the rules:

\begin{itemize}

\item {\bf P}arentheses have the highest (or tightest) precedence and can be used
to force an expression to evaluate in the order you want. Since
expressions in parentheses are evaluated first, {\tt 2 * (3-1)} is 4,
and {\tt (1+1)**(5-2)} is 8. You can also use parentheses to make an
expression easier to read, as in {\tt (\$minute * 100) / 60}, even
if it doesn't change the result.
\index{$()$ parenthesis operator}

\item {\bf E}xponentiation has the next highest precedence, so
{\tt 1 + 2**3} is 9 (1 + 8), not 27, and {\tt 2 * 3**2} is 18, not 36.
\index{$**$ exponentiation operator}
\index{operator!$**$ (exponentiation)}

\item {\bf M}ultiplication and {\bf D}ivision have higher precedence
  than {\bf A}ddition and {\bf S}ubtraction.  So {\tt 2*3-1} is 5, not
  4, and {\tt 6+4/2} is 8, not 5.
\index{$*$ multiplication operator}
\index{operator!$*$ (multiplication)}
\index{$/$ division operator}
\index{operator!$/$ (division)}

\item Operators with the same precedence are usually evaluated
from left to right (except exponentiation).  So in the expression
{\tt \$degrees / 2 * pi}, the division happens first and the
result is multiplied by {\tt pi}, which is not the expected
result. (Note that {\tt pi} is not a variable, but a predefined
constant in Perl~6, and therefore does not require a sigil).  To
divide by $2 \pi$, you can use parentheses:
\index{sigil}
\index{pi}

\begin{verbatim}
my $result = $degrees / (2 * pi);
\end{verbatim}

or write
  {\tt \$degrees / 2 / pi} or {\tt \$degrees / } $2 / \pi$, which
  will divide \verb'$degrees' by 2, and then divide the result of
  that operation by $\pi$ (which is equivalent \verb'$degrees' by
   $2 \pi$.

\end{itemize}

We don't work very hard to remember the precedence of
operators.  If we can't tell by looking at the expression, we use
parentheses to make it obvious. If we don't know for sure which of two operators
has the higher precedence, then the next person reading or maintaining
our code may also not know.
\index{precedence}
\index{parentheses}

\section{String operations}
\label{string_operations}
\index{string!operation}
\index{operator!string}
\index{coercion}
\index{type!coercion}

In general, you can't perform mathematical operations on strings, unless
the strings look so much like numbers that Perl can transform or \emph{coerce} them into numbers and still make sense, so the
following are illegal:

\begin{verbatim}
'2'-'1a'    'eggs'/'easy'    'third'*'a charm'
\end{verbatim}
%

For example, this produces an error:

\begin{verbatim}
> '2'-'1a'
Cannot convert string to number: trailing characters after number
in '1?a' (indicated by ?)
  in block <unit> at <unknown file>:1
\end{verbatim}
%

But the following expressions are valid because these strings
can be coerced to numbers without any ambiguity:
\begin{verbatim}
> '2'-'1'
1
> '3'/'4'
0.75
\end{verbatim}
%

The \verb'~' operator performs {\bf string concatenation}, which means
it joins the strings by linking them end-to-end.  For example:
\index{string concatenation}
\index{string!concatenation}

\begin{verbatim}
> my $first = 'throat'
throat
> my $second = 'warbler'
warbler
> $first ~ $second
throatwarbler
\end{verbatim}
%
The {\tt x} operator also works on strings; it performs repetition.
For example:
\index{string repetition}

\begin{verbatim}
> 'ab' x 3;
ababab
> 42 x 3
424242
> 3 x 42
333333333333333333333333333333333333333333
\end{verbatim}

Notice that, although the {\tt x} operator somewhat looks like
the multiplication operator when we write it by hand, {\tt x}
is obviously not commutative, contrary to the {\tt *}
multiplication operator. The first operator is a string or is
\emph{coerced} to a string (i.e. transformed into a string:
{\tt 42} is coerced to {\tt '42'}), and the second operator
has to be a number or something that can be transformed
into a number.
\index{commutativity}
\index{coercion}

\section{Comments}
\index{comment}

As programs get bigger and more complicated, they get more difficult
to read.  Formal languages are dense, and it is often difficult to
look at a piece of code and figure out what it is doing, or why.

For this reason, it is a good idea to add notes to your programs to explain
in natural language what the program is doing.  These notes are called
{\bf comments}, and they start with the \verb"#" symbol:

\begin{verbatim}
# compute the percentage of the hour that has elapsed
my $percentage = ($minute * 100) / 60;
\end{verbatim}
%
In this case, the comment appears on a line by itself.  You can also put
comments at the end of a line:

\begin{verbatim}
$percentage = ($minute * 100) / 60;     # percentage of an hour
\end{verbatim}
%
Everything from the {\tt \#} to the end of the line is ignored---it
has no effect on the execution of the program.

Comments are most useful when they document non-obvious features of
the code.  It is reasonable to assume that the reader can figure out
{\em what} the code does; it is more useful to explain {\em why}.

This comment is redundant with the code and useless:

\begin{verbatim}
my $value = 5;        # assign 5 to $value
\end{verbatim}
%
This comment, by contrast, contains useful information that
is not in the code:

\begin{verbatim}
my $velocity = 5;     # velocity in meters/second.
\end{verbatim}
%
Good variable names can reduce the need for comments, but
long names can make complex expressions hard to read, so there is
a tradeoff.
\index{variable name}

\section{Debugging}
\index{debugging}
\index{bug}

Three kinds of errors can occur in a program: syntax errors, runtime
errors, and semantic errors.  It is useful
to distinguish between them in order to track them down more quickly.

\begin{description}

\item[Syntax error:] ``Syntax'' refers to the structure of a program
  and the rules about that structure.  For example, parentheses have
  to come in matching pairs, so {\tt (1 + 2)} is legal, but
{\tt 8)} is a \emph{syntax error}. \footnote{We are using here
``syntax error'' as a quasi synonym for ``compile-time error'';
they are not exactly the same thing (you may in theory have
syntax errors that are not compile-time errors and the other way
around), but they can be deemed to be the same for our
practical purpose here. In Perl~6, compile-time error
have the ``===SORRY!==='' string at the beginning of
the error message.}

\index{error!syntax}
\index{error message}
\index{syntax}
\index{syntax!error}

If there is a syntax error
anywhere in your program, Perl displays an error message and quits
without even starting to run your program, and you will
obviously not be able to run the program.  During the first few
weeks of your programming career, you might spend a lot of
time tracking down syntax errors.  As you gain experience, you will
make fewer errors and find them faster.

\item[Runtime error:] The second type of error is a runtime error, so
  called because the error does not appear until after the program has
  started running.  These errors are also called \emph{exceptions}
  because they usually indicate that something exceptional (and bad)
  has happened.  \index{runtime error} \index{error!runtime}
  \index{exception} \index{safe language} \index{language!safe}

Runtime errors are rare in the simple programs you will see in the
first few chapters, so it might be a while before you encounter one.
We have seen one example of such errors, though, at the beginning
of Section~\ref{string_operations} (p.~\pageref{string_operations}),
when we tried to subtract \verb"'2'-'1a'".

\item[Semantic error:] The third type of error is ``semantic'', which
  means related to meaning.  If there is a semantic error in your
  program, it will run without generating error messages, but it will
  not do the right thing.  It will do something else.  Specifically,
  it will do what you \emph{told} it to do, but not what you
  \emph{intended} it to do.
  \index{semantic error}
  \index{error!semantic} \index{error message}

Identifying semantic errors can be tricky because it requires you to work
backward by looking at the output of the program and trying to figure
out what it is doing.

\end{description}

\section{Glossary}

\begin{description}

\item[variable:]  Informally, a name that refers to a value. More
accurately, a variable is a container that has a name and holds a value.
\index{variable}

\item[assignment:]  A statement that assigns a value to a variable.
\index{assignment}

\item[state diagram:]  A graphical representation of a set of variables and the
values they refer to.
\index{state diagram}

\item[keyword:]  A reserved word that is used to parse a
program; in many languages, you cannot use keywords like
{\tt if}, {\tt  for}, and {\tt while} as variable names.
This problem usually does not occur in Perl because variable
names begin with a \emph{sigil}.
\index{keyword}
\index{sigil}

\item[operand:]  One of the values on which an operator operates.
\index{operand}

\item[term:]  A variable or a literal value.
\index{term}

\item[expression:]  A combination of operators and terms that
represents a single result.
\index{expression}

\item[evaluate:]  To simplify an expression by performing the operations
in order to yield a single value.
\index{evaluate}

\item[statement:]  A section of code that represents a command or action.  So
far, the statements we have seen are assignments and print statements. Statements usually end with a semi-colon.
\index{statement}

\item[execute:]  To run a statement and do what it says.
\index{execute}

\item[interactive mode (or interpreter mode):] A way of using the Perl
interpreter by typing code at the prompt.
\index{interactive mode}

\item[script mode:] A way of using the Perl interpreter to read
code from a script and run it.
\index{script mode}

\item[one-liner mode:] A way of using the Perl interpreter to read
code passed at the operating system prompt and run it.
\index{one-liner mode}

\item[script:] A program stored in a file.
\index{script}

\item[order of operations:]  Rules governing the order in which
expressions involving multiple operators and operands are evaluated.
It is also called operator precedence.
\index{order of operations}
\index{operator precedence}
\index{precedence!operator}

\item[concatenate:]  To join two operands end-to-end.
\index{concatenation}

\item[comment:]  Information in a program that is meant for other
programmers (or anyone reading the source code) and has no effect on the
execution of the program.
\index{comment}

\item[syntax error:]  An error in a program that makes it impossible
to parse (and therefore impossible to compile and to run).
\index{syntax!error}

\item[exception:]  An error that is detected while the program is running.
\index{exception}

\item[semantics:]  The meaning of a program.
\index{semantics}

\item[semantic error:]   An error in a program that makes it do something
other than what the programmer intended.
\index{semantic error}

\end{description}

\section{Exercises}

\begin{exercise}

Repeating our advice from the previous chapter, whenever you learn
a new feature, you should try it out in interactive mode (under
the REPL) and make errors on purpose to see what goes wrong.
\index{interactive mode}
\index{REPL}

\begin{itemize}

\item We've seen that {\tt \$n = 42} is legal.  What about {\tt 42 = \$n}?

\item How about {\tt \$x = \$y = 1}? (Hint: note that you will
have to declare both variables, for example with a statement
such as {\tt my \$x; my \$y;} or possibly {\tt my (\$x, \$y);},
before you can run the above.)

\item In some languages, statements don't have to end with a semi-colon,
{\tt ;}. What happens in script mode if you omit a semi-colon at the end
of a Perl statement?
\index{semi-colon}

\item What if you put a period at the end of a statement?

\item In math notation you can multiply $x$ and $y$ like this: $x y$.
What happens if you try that in Perl?

\end{itemize}

\end{exercise}

\begin{exercise}

Practice using the Perl interpreter as a calculator:
\index{calculator}

\begin{enumerate}

\item The volume of a sphere with radius $r$ is $\frac{4}{3} \pi r^3$.
  What is the volume of a sphere with radius 5?

\item Suppose the cover price of a book is \$24.95, but bookstores get a
  40\% discount.  Shipping costs \$3 for the first copy and 75 cents
  for each additional copy.  What is the total wholesale cost for
  60 copies?

\item If I leave my house at 6:52 am and run 1 mile at an easy pace
  (8:15 per mile), then 3 miles at tempo (7:12 per mile) and 1 mile at
  easy pace again, what time do I get home for breakfast?
\index{running pace}

\end{enumerate}
\end{exercise}


\documentclass{article}

\usepackage[T1]{fontenc}
\usepackage[osf]{libertine}
\usepackage[scaled=0.8]{beramono}
\usepackage[margin=1.5in]{geometry}
\usepackage{url}
\usepackage{booktabs}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{nicefrac}
\usepackage{microtype}
\usepackage{bm}

\usepackage{sectsty}
\sectionfont{\large}
\subsectionfont{\normalsize}

\usepackage{titlesec}
\titlespacing{\section}{0pt}{10pt plus 2pt minus 2pt}{0pt plus 2pt minus 0pt}
\titlespacing{\subsection}{0pt}{5pt plus 2pt minus 2pt}{0pt plus 2pt minus 0pt}

\usepackage{pgfplots}
\pgfplotsset{
  compat=newest,
  plot coordinates/math parser=false,
  tick label style={font=\footnotesize, /pgf/number format/fixed},
  label style={font=\small},
  legend style={font=\small},
  every axis/.append style={
    tick align=outside,
    clip mode=individual,
    scaled ticks=false,
    thick,
    tick style={semithick, black}
  }
}

\pgfkeys{/pgf/number format/.cd, set thousands separator={\,}}

\usepgfplotslibrary{external}
\tikzexternalize[prefix=tikz/]

\newlength\figurewidth
\newlength\figureheight

\setlength{\figurewidth}{8cm}
\setlength{\figureheight}{6cm}

\setlength{\parindent}{0pt}
\setlength{\parskip}{1ex}

\newcommand{\acro}[1]{\textsc{\MakeLowercase{#1}}}
\newcommand{\given}{\mid}
\newcommand{\mc}[1]{\mathcal{#1}}
\newcommand{\data}{\mc{D}}
\newcommand{\intd}[1]{\,\mathrm{d}{#1}}

\begin{document}

{\large \textbf{CSE 515T (Spring 2015) Assignment 1 Solutions}} \\

\begin{enumerate}
\item
  (Barber.)
  Suppose that a study shows that 90\% of people who have contracted
  Creutzfeldt--Jakob disease (``mad cow disease'') ate hamburgers
  prior to contracting the disease.  Creutzfeldt--Jakob disease is
  incredibly rare; suppose only one in a million people have the
  disease.

  If you eat hamburgers, should you be worried?  Does this depend on
  how many other people eat hamburgers?
\end{enumerate}

\subsection*{Solution}
Let CJ be the random variaable ``has Creutzfeldt--Jakob disease,'' and
let H be the random variable ``eats hamburgers.''  From the problem,
we know $\Pr(\mathrm{H} \given \mathrm{CJ}) = 0.9$ and
$\Pr(\mathrm{CJ}) = 10^{-6}$.  From Bayes' theorem, we may compute the
posterior probability of having Creutzfeldt--Jakob disease given that
you eat hamburgers:
\begin{equation*}
  \Pr(\mathrm{CJ} \given \mathrm{H})
  =
  \frac
      {\Pr(\mathrm{H} \given \mathrm{CJ}) \Pr(\mathrm{CJ})}
      {\Pr(\mathrm{H})}.
\end{equation*}
The denominator, $\Pr(\mathrm{H})$, is the probability that a random
person eats hamburgers, so whether you should be worried does depend
on this value.  I estimate $\Pr(\mathrm{H})$ might be around
\nicefrac{1}{2}; plugging in this value, the posterior probability
of having Creutzfeldt--Jakob disease is
\begin{equation*}
  \frac{0.9 \times 10^{-6}}{\nicefrac{1}{2}}
  =
  1.8 \times 10^{-6},
\end{equation*}
so your probability of having the disease has only increased from
$10^{-6}$ to $1.8 \times 10^{-6} = 0.0000018$ as a result of eating
hamburgers.  You can sleep safe.

If hamburger eating were rare, say $\Pr(\mathrm{H}) = 10^{-5}$, then
we would instead have $\Pr(\mathrm{CJ} \given \mathrm{H}) = 9\%$, and
maybe you should be worried!

\clearpage

\begin{enumerate}
\setcounter{enumi}{1}
\item
  (O'Hagan and Forster.)
  Suppose $x$ has a Poisson distribution with unknown mean
  $\theta$:
  \begin{equation*}
    p(x \given \theta) = \frac{\theta^x}{x!} \exp(-\theta),
    \qquad
    x = 0, 1, \dotsc
  \end{equation*}
  Let the prior for $\theta$ be a gamma distribution:
  \begin{equation*}
    p(\theta \given \alpha, \beta)
    =
    \frac{\beta^\alpha \theta^{\alpha - 1}}{\Gamma(\alpha)} \exp(-\beta\theta),
    \qquad \theta > 0
  \end{equation*}
  where $\Gamma$ is the gamma function.  Show that, given an
  observation $x$, the posterior $p(\theta \given x, \alpha, \beta)$
  is a gamma distribution with updated parameters $(\alpha', \beta') =
  (\alpha + x, \beta + 1)$.
\end{enumerate}

\subsection*{Solution}
From Bayes' theorem, we have:
\begin{align*}
  p(\theta \given x)
  &\propto
  p(x \given \theta)
  p(\theta)
  \\
  &\propto
  \bigl(\theta^{x} \exp(-\theta)\bigr)
  \bigl(\theta^{\alpha - 1} \exp(-\beta\theta)\bigr)
  \\
  &=
  \theta^{x + \alpha - 1}
  \exp\bigl(-(\beta + 1)\theta\bigr)
  \\
  &\propto
  \mc{G}(\alpha + x, \beta + 1).
\end{align*}

Here we exploit a common trick: we manipulate the numerator, ignoring
constants independent of $\theta$.  If we can recognize the functional
form as belonging to a distribution family we know, we can simply
identify the parameters and trust that the distribution normalizes!

\clearpage

\begin{enumerate}
\setcounter{enumi}{2}
\item
  (Optimal \emph{Price is Right} bidding.)
  Suppose you have a standard normal belief about an unknown parameter
  $\theta$, $p(\theta) = \mc{N}(\theta; 0, 1^2)$.  You are asked to
  give a point estimate $\hat{\theta}$ of $\theta$, but are told that
  there is a heavy penalty for guessing too high.  The loss function is
  \begin{equation*}
    \ell(\hat{\theta}, \theta; c)
    =
    \begin{cases}
      (\theta - \hat{\theta})^2 & \hat{\theta}  <   \theta; \\
      c                         & \hat{\theta} \geq \theta
    \end{cases},
  \end{equation*}
  where $c > 0$ is a constant cost for overestimating.  What is the
  Bayesian estimator in this case?  How does it change as a function
  of $c$?
\end{enumerate}

\subsection*{Solution}
Let $\phi(x) = \mc{N}(x; 0, 1^2)$ be the standard normal \acro{PDF}
evaluated at $x$, and let $\Phi(x) = \int_{-\infty}^x \phi(x)
\intd{x}$ be the standard normal \acro{CDF} evaluated at $x$.  If we
fix a point $\hat{\theta}$, the expected loss is:
\begin{align*}
  \mathbb{E}\bigl[\ell(\hat{\theta}, \theta; c)\bigr]
  &=
  \int
  \ell(\hat{\theta}, \theta; c)
  p(\theta)
  \intd{\theta}
  \\
  &=
  \int_{-\infty}^{\hat{\theta}}
  c
  \phi(\theta)
  \intd{\theta}
  +
  \int_{\hat{\theta}}^{\infty}
  (\theta - \hat{\theta})^2
  \phi(\theta)
  \intd{\theta}.
\end{align*}
The first term is proportional to the standard normal \acro{CDF}
$\Phi$ evaluated at $\hat{\theta}$:
\begin{equation*}
  \int_{-\infty}^{\hat{\theta}}
  c
  \phi(\theta)
  \intd{\theta}
  =
  c\,\Phi(\hat{\theta}).
\end{equation*}

We may also compute the second integral using the following
antiderivatives:\footnote{\url{http://en.wikipedia.org/wiki/List_of_integrals_of_Gaussian_functions}
  is useful here! (Wolfram alpha also works.)}
\begin{equation*}
  \int \theta \phi(\theta) \intd{\theta} = -\phi(\theta) + C;
  \qquad
  \int \theta^2 \phi(\theta) \intd{\theta} = \Phi(\theta) - \theta \phi(\theta) + C.
\end{equation*}
Using the fundamental theorem of calculus, we may use these to calculate
\begin{equation*}
  \int_{\hat{\theta}}^{\infty}
  (\theta - \hat{\theta})^2
  \phi(\theta)
  \intd{\theta}
  =
  (\hat{\theta}^2 + 1) (1 - \Phi(\hat{\theta}))
  - \hat{\theta}\phi(\hat{\theta}).
\end{equation*}
Finally, the entire expected loss is
\begin{equation*}
  \mathbb{E}\bigl[\ell(\hat{\theta}, \theta; c)\bigr]
  =
  (c - \hat{\theta}^2 - 1)\Phi(\hat{\theta})
  - \hat{\theta}\phi(\hat{\theta})
  + \hat{\theta}^2 + 1.
\end{equation*}
The derivative with respect to $\hat{\theta}$ is
\begin{equation*}
  \frac{\partial \mathbb{E}\bigl[\ell(\hat{\theta}, \theta; c)\bigr]}
       {\partial \hat{\theta}}
  =
  2 \hat{\theta} \bigl(1 - \Phi(\hat{\theta})\bigr)
  + (c - 2) \phi(\hat{\theta}).
\end{equation*}
Unfortunately, I do not believe we may find a root explicitly, so we
would have to rely on numerical root finding.  For $c = 1$, the
minimial expected loss is achieved at $\hat{\theta} = 0.612$; for $c =
10$, the Bayes action is $\hat{\theta} = -1.0615$; for $c = 100$, the
Bayes action is $\hat{\theta} = -2.1167$.  In general, the larger $c$,
the smaller your estimate should be, due to the potentially high cost
of overestimation.  For $c = 0$, there is no Bayes action, because we
may continue to decrease the expected loss when taking $\hat{\theta}
\to \infty$ (there is no reason not to!).

\clearpage

\begin{enumerate}
\setcounter{enumi}{3}
\item
  (Maximum-likelihood estimation.)
  Suppose you flip a coin with unknown bias $\theta$, $\Pr(x =
  \text{H}) = \theta$, three times and observe the outcome HHH.  What
  is the maximum likelihood estimator for $\theta$?  Do you think this
  is a good estimator?  Would you want to use it to make predictions?

  Consider a Bayesian analysis of $\theta$ with a beta prior $p(\theta
  \given \alpha, \beta) = \mc{B}(\theta; \alpha, \beta)$.  What is the
  posterior mean of $\theta$?  What is the posterior mode?  Consider
  $(\alpha, \beta) = (\nicefrac{1}{5}, \nicefrac{1}{5})$.  Plot the
  posterior density in this case.  Is the posterior mean a good
  summary of the distribution?
\end{enumerate}

\subsection*{Solution}
The likelihood of HHH is proportional to $\theta^3$, which over the
domain $\theta \in [0, 1]$ is maximized at $\theta = 1$.  Whether you
think this is a good estimator is subjective; however, I certainly
wouldn't use it for prediction, because it completely discounts the
(in my opinion, still rather likely) possiblity of a tails event.

With a $\mc{B}(\alpha, \beta)$ prior, the posterior is $\mc{B}(\alpha
+ 3, \beta)$.  We may calculate the mean of an arbitrary beta
distribution:
\begin{equation*}
  \mathbb{E}[\theta \given \alpha, \beta]
  =
  \int
  \theta
  \mc{B}(\theta \given \alpha, \beta)
  \intd{\theta}
  =
  \frac{1}{B(\alpha, \beta)}
  \int_{0}^{1}
  \theta
  \bigl(
  \theta^{\alpha - 1}
  (1 - \theta)^{\beta - 1}
  \bigr)
  \intd{\theta}
  =
  \frac{B(\alpha + 1, \beta)}{B(\alpha, \beta)}
  =
  \frac{\alpha}{\alpha + \beta}.
\end{equation*}

Therefore the posterior mean is $\frac{\alpha + 3}{\alpha + \beta +
  3}$.  The posterior mode is $\frac{\alpha + 2}{\alpha + \beta +
  1}$.\footnote{\url{http://goo.gl/zigQER} -- this is how I would
  answer this type of question!} Note that the posterior mode only
exists for $\alpha, \beta > 1$.

The prior and posterior distributions are plotted below.  An
interesting point about the prior is that its mean and median are both
$\nicefrac{1}{2}$, but this is simultaneously the anti-mode!  It's an
unusual estimator.

\begin{figure}[h]
  \centering
  \input{figures/problem_4.tex}
  \caption{The prior and posterior distributions over $\theta$ for the
    coin-flipping problem.}
\end{figure}

\clearpage

\begin{enumerate}
\setcounter{enumi}{4}
\item
  (Gaussian with unknown mean.)
  Let $\bm{x} = \{ x_i \}_{i = 1}^N$ be independent, identically
  distributed real-valued random variables with distribution $p(x_i
  \given \theta) = \mc{N}(x_i; \theta, \sigma^2)$.  Suppose the
  variance $\sigma^2$ is known but the mean $\theta$ is unknown with
  prior distribution $p(\theta) = \mc{N}(\theta; 0, 1^2)$.
  \begin{itemize}
  \item
    What is the likelihood of the full observation vector $p(\bm{x}
    \given \theta)$?
  \item
    After observing $\bm{x}$, what is the posterior distribution of
    $\theta$, $p(\theta \given \bm{x}, \sigma^2)$?  (Note: you might
    find it more convenient in this case to work with the
    \emph{precision} $\tau = \sigma^{-2}$.)
  \item
    Interpret how the posterior changes as a function of $N$.  What
    happens if $N = 0$?  What happens if $N \to \infty$?  Does this
    agree with your intuition?
  \end{itemize}
\end{enumerate}

\subsection*{Solution}
In the first part, we use the definition of independence:
\begin{equation*}
  p(\bm{x} \given \theta)
  =
  \prod_{i = 1}^N
  p(x_i \given \theta)
  =
  \prod_{i = 1}^N
  \mc{N}(x_i; \theta, \sigma^2).
\end{equation*}

We will consider the slightly more general case with an arbitrary
Gaussian prior on $\theta$: $p(\theta \given m, s^2) = \mc{N}(\theta;
m, s^2)$.  For convenience, we will parameterize the Gaussians on the
$\{x_i\}$ and $\theta$ with the \emph{precision} parameters $\tau =
\sigma^{-2}; t = s^{-2}$.

By Bayes' theorem, we have
\begin{align*}
  p(\theta \given \bm{x})
  &\propto
  p(\bm{x} \given \theta)
  p(\theta)
  \\
  &=
  \prod \mc{N}(x_i; \theta, \tau)
  \,
  \mc{N}(\theta; m, t)
  \\
  &\propto
  \exp\Biggl(
  -\frac{1}{2}
  \biggl(
  \tau
  \sum_{i = 1}^N (x_i - \theta)^2
  +
  t
  (\theta - m)^2
  \biggr)
  \Biggr).
\end{align*}
This is an exponentiated, negative quadratic function of $\theta$ and
is therefore proportional to a Gaussian distribution over $\theta$.
Our goal is to identify the mean and precision of this distribution.

We expand:
\begin{equation*}
  \tau
  \sum_{i = 1}^N (x_i - \theta)^2
  +
  t
  (\theta - m)^2
  =
  (\tau N + t)\theta^2
  - 2(\tau \textstyle\sum x_i + tm) \theta
  + (\tau \sum x_i^2 + t m^2).
\end{equation*}
Notice that $\sum x_i = N \bar{x}$, where $\bar{x}$ is the sample mean
of the measurements.  Notice also that the last term is a constant
independent of $\theta$, which will be absorbed into the normalizing
constant.  To arrive at a more-familiar form, we ``complete the
square:''
\begin{equation*}
  (\tau N + t)\theta^2
  - 2(\tau N \bar{x} + tm) \theta
  =
  (\tau N + t) \biggl(
  \theta -
  \frac{\tau N \bar{x} + tm}{\tau N + t}
  \biggr)^2
  +
  c,
\end{equation*}
where $c$ is another constant independent of $\theta$.  Therefore:
\begin{align*}
  p(\theta \given \bm{x})
  &\propto
  \exp\bigl(
  \textstyle -\frac{t'}{2}
  (\theta - m')^2
  \bigr),
\end{align*}
which is a Gaussian distribution with mean and precision
\begin{equation*}
  m' = \frac{\tau N \bar{x} + t m}{\tau N + t};
  \qquad
  t' = \tau N + t,
\end{equation*}
respectively.  We recognize that the new precision is the sum of the
precisions of each measurement (the $N$ independent measurements
$\{x_i\}$ with precision $\tau$ aplus one additional prior
``measurement.'' with precision $t$).  The posterior mean can be
recognized as a precision-weighted average of the measurements,
including the ``measurement'' at the prior mean $m$.

As $N \to \infty$, the sample mean $\bar{x}$ dominates, and the
influence of the prior diminishes to zero.  If $N = 0$, we rely only
on our prior knowledge.

\clearpage

\begin{enumerate}
\setcounter{enumi}{5}
\item
  (Spike and slab priors.)
  Suppose $\theta$ is a real-valued random variable that is expected
  to either be near zero (with probability $\pi$) or to have a wide
  range of potential values (with probability $(1 - \pi)$).  Such
  scenarios happen a lot in practice: for example, $\theta$ could be
  the coefficient of a feature in a regression model.  We either
  expect the feature to be useless for predicting the output (and have
  a value close to zero) or to be useful, in which case we expect a
  value with larger magnitude but can't say much else.

  A common approach in this case is to use a so-called \emph{spike and
    slab prior.} Let $f \in \{0, 1\}$ be a discrete random variable
  serving as a flag.  We define the following conditional prior:
  \begin{equation*}
    p(\theta \given f, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2)
    =
    \begin{cases}
      \mc{N}(\theta; 0, \sigma_{\text{spike}}^2) & f = 0 \\
      \mc{N}(\theta; 0, \sigma_{\text{slab}}^2)  & f = 1,
    \end{cases}
  \end{equation*}
  where $\sigma_{\text{spike}}$ is the width of a narrow ``spike'' at
  zero, and $\sigma_{\text{slab}} > \sigma_{\text{spike}}$ is the
  width of a ``slab'' supporting values with larger magnitude.

  In practice, we will never observe the flag variable $f$; instead,
  we must infer it or marginalize it, as required.
  \begin{itemize}
  \item
    Suppose we choose a prior $\Pr(f = 1) = \pi = \nicefrac{1}{2}$,
    expressing no \emph{a priori} preference for the spike or the
    slab.  What is the marginal prior $p(\theta \given
    \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2$)?  Plot the
    marginal prior distribution for $(\sigma_{\text{spike}}^2,
    \sigma_{\text{slab}}^2) = (1^2, 10^2)$.
  \item
    Suppose that we can make a noisy observation $x$ of $\theta$, with
    distribution $p(x \given \theta, \omega^2) = \mc{N}(x; \theta,
    \omega^2)$, with known variance $\omega^2$.  Given $x$, what is
    the posterior distribution of the flag parameter, $\Pr(f = 1
    \given x, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2,
    \omega^2)$?  Plot this distribution as a function of $x$.  What
    observation would teach us the most about $f$?  What teaches us
    the least?
  \item
    Given an observation $x$ as in the last part, what is the
    posterior distribution of $\theta$, $p(\theta \given x,
    \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)$?
    (Hint: use the sum rule to eliminate $f$ and use the result
    above.)
  \item
    Suppose the noise variance is $\omega^2 = 0.1^2$ and we make an
    observation $x = 3$.  Plot the posterior distribution of $\theta$,
    using the parameters from the first part.
  \end{itemize}
\end{enumerate}

\subsection*{Solution}
For the first part, we use the sum rule:
\begin{equation*}
  p(\theta \given \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2)
  =
  \sum_f
  \Pr(f)p(\theta \given f, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2)
  =
  \frac{1}{2}\mc{N}(\theta; 0, \sigma_{\text{spike}}^2) +
  \frac{1}{2}\mc{N}(\theta; 0, \sigma_{\text{slab}}^2).
\end{equation*}
The prior density is plotted below.

\begin{figure}[h]
  \centering
  \input{figures/problem_6_prior.tex}
  \caption{The prior distribution over $\theta$ for the spike-and-slab
    prior problem.}
\end{figure}

For the second part, we have from Bayes' theorem that:
\begin{equation*}
  \Pr(f = 1 \given x, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)
  =
  \frac{p(x \given f = 1, \sigma_{\text{slab}}^2, \omega^2)\Pr(f = 1)}
       {\sum_f p(x \given f, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)\Pr(f)}.
\end{equation*}

We must derive the likelihood $p(x \given f, \sigma_{\text{spike}}^2,
\sigma_{\text{slab}}^2)$.  We apply the sum rule:
\begin{equation*}
  p(x \given f, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)
  =
  \int
  p(x \given \theta, \omega^2)
  p(\theta \given f, \sigma_{\text{slab}}^2, \omega^2)
  \intd{\theta}.
\end{equation*}
For $f = 1$, this becomes:
\begin{align*}
  p(x \given f = 1, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)
  &=
  \int
  p(x \given \theta, \omega^2)
  p(\theta \given f = 1, \sigma_{\text{slab}}^2, \omega^2)
  \intd{\theta}
  \\
  &=
  \int
  \mc{N}(x; \theta, \omega^2)
  \mc{N}(\theta; 0, \sigma_{\text{slab}}^2)
  \intd{\theta}
  \\
  &=
  \mc{N}(x; 0, \sigma_{\text{slab}}^2 + \omega^2).
\end{align*}
A similar expression holds for $f = 0$.  The final result is
\begin{equation*}
  \Pr(f = 1 \given x, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)
  =
  \frac{\mc{N}(x; 0, \sigma_{\text{slab}}^2 + \omega^2)}
       {\mc{N}(x; 0, \sigma_{\text{spike}}^2 + \omega^2) + \mc{N}(x; 0, \sigma_{\text{slab}}^2 + \omega^2)}.
\end{equation*}
This value is plotted as a function of $x$ on the range $x \in [-10,
  10]$ below for $\omega^2 = 0.1^2$.  Extreme values of $x$ teach us
the most, because we can conclude with near certainty that the
observation came from the slab.  The observations that would teach us
the least are $x \approx \pm 2.165$, either of which result in a
posterior slab probability of $\nicefrac{1}{2}$ -- the same as our
prior!

\begin{figure}[h]
  \centering
  \input{figures/problem_6_f_posterior.tex}
  \caption{The posterior probability for the slab flag $(f = 1)$ for
    the spike-and-slab prior problem as a function of the observation
    $x$.}
\end{figure}

For the third part, we use the sum rule:
\begin{equation*}
  p(\theta \given x, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)
  =
  \sum_f
  p(\theta \given f, x, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2)
  \Pr(f \given x, \sigma_{\text{spike}}^2, \sigma_{\text{slab}}^2, \omega^2).
\end{equation*}
We need to compute the posterior on $\theta$ given $f$ and $x$.  Here
we may use the result from the last problem:
\begin{align*}
  p(\theta \given f = 0, x, \sigma_{\text{spike}}^2, \omega^2)
  &=
  \mc{N}(\theta;
  \tau_{\text{spike}}^{-1} \omega^{-2} x,
  \tau_{\text{spike}}^{-1}) \\
  p(\theta \given f = 1, x, \sigma_{\text{slab}}^2, \omega^2)
  &=
  \mc{N}(\theta;
  \tau_{\text{slab}}^{-1} \omega^{-2} x,
  \tau_{\text{slab}}^{-1}),
\end{align*}
where $\tau_{\text{spike}} = (\omega^{-2} +
\sigma_{\text{spike}}^{-2})$, and $\tau_{\text{slab}}$ is defined
similarly.

Finally, the posterior for $\theta$ for the observation $x = 3$ with
$\omega^2 = 0.1^2$ is plotted below.

\begin{figure}[h]
  \centering
  \input{figures/problem_6_theta_posterior.tex}
  \caption{The posterior density for the parameter $\theta$ given
    the example observation $x = 3$, $\omega^2 = 0.1^2$.}
\end{figure}

\end{document}


\documentclass{beamer}

\mode<presentation> { \usetheme{gruvbox} }
\setbeamerfont{frametitle}{size=\huge}

\usepackage{graphicx} % Allows including images
\usepackage{booktabs} % Allows the use of \toprule, \midrule and \bottomrule in tables
%\usepackage{listings}             % Include the listings-package
\usepackage{minted}
\usepackage{tikz}
\usepackage{drawstack}
\usetikzlibrary{calc,shapes.callouts,shapes.arrows,chains,positioning,fit,shapes, arrows.meta, arrows}
\usepackage{verbatimbox}
\usepackage{tcolorbox}

\usemintedstyle{paraiso-dark}

\newcommand{\pointthis}[2]{
    \tikz[remember picture,baseline]{\node[anchor=base,inner sep=0,outer sep=0]%
    (#1) {\underline{#1}};\node[overlay,rectangle callout,%
    callout relative pointer={(0.2cm,0.7cm)},fill=green!50] at ($(#1.north)+(-.5cm,-1.4cm)$) {#2};}%
}%

%----------------------------------------------------------------------------------------
%	TITLE PAGE
%----------------------------------------------------------------------------------------

\title[Cryptography Day 2]{\huge \textbf{Cryptography Day 2}} % The short title appears at the bottom of every slide, the full title is only on the title page

\author{Brandon Hernandez} % Your name
\date{\today} % Date, can be changed to a custom date

\begin{document}

\begin{frame}
    \titlepage % Print the title page as the first slide
\end{frame}

\begin{frame}
    \frametitle{Overview} % Table of contents slide, comment this block out to remove it
    \tableofcontents % Throughout your presentation, if you choose to use \section{} and \subsection{} commands, these will automatically be printed on this slide as an overview of your presentation
\end{frame}

\section{Brief Review}
\begin{frame}
    \frametitle{Brief Review}
    In crypto, we utilize hard problems in mathematics to ensure that breaking the cryptosystem is not trivial.
\end{frame}

\section{Diffie-Hellman}
\begin{frame}
    \frametitle{Diffie-Hellman}
\end{frame}

\section{RSA}
\begin{frame}
    \frametitle{The RSA Cryptosystem}
    Public-key cryptosystem based around the difficulty in factoring a composite integer into primes.

\end{frame}
\subsection{The RSA Problem}
\begin{frame}
    \frametitle{The RSA Problem}
    \begin{itemize}
        \item Consider two large primes, $p$ and $q$
        \item $N = pq$
        \item Consider $e$, $m$, and $c$, where $e,C, m \in \mathbb{Z}$
        \item $C \equiv m^{e} \textrm{(mod N)}$
    \end{itemize}
\end{frame}
\subsection{Construction}
\begin{frame}
    \frametitle{Construction}
    \begin{itemize}
        \item Consider two large primes, p and q
        \item Let our modulus, $N = pq$ ($\mathbb{Z}/N\mathbb{Z}$)
        \item Our public key, $e$, where $gcd(\phi{N},e) = 1$ and $1 < e < \phi{N}$
        \item The private key, $d$, where $d*e \equiv 1 (\textrm{mod} \phi{N})$
    \end{itemize}
\end{frame}

\subsection{Encryption}
\begin{frame}
    \frametitle{Encryption}
    \begin{itemize}
        \item Given $N$, $m$, and $e$
        \item $C \equiv m^{e} (mod N)$
    \end{itemize}
\end{frame}

\subsection{Decryption}
\begin{frame}
    \frametitle{Decryption}
    \begin{itemize}
        \item Given $N$, $C$, and $d$
        \item $m \equiv C^{d} \equiv m^{e*d} (mod N)$
        \item \textbf{Remeber:} $e*d \equiv 1 (mod \phi{N})$
    \end{itemize}
\end{frame}

\subsection{Construction Example}
\begin{frame}
  \frametitle{Construction Example}
\end{frame}
\subsection{}
\begin{frame}
  \frametitle{Basic Exploit Example 1}
  \emph{Refer to Day2/Examples/cube}
\end{frame}

\begin{frame}
  \frametitle{Basic Exploit Example 2 (Factoring)}
  \emph{Refer to Day2/Examples/multiPrime}
\end{frame}

\begin{frame}
  \frametitle{Basic Exploit Example 3 (Hastad)}
  \emph{Refer to Day2/Examples/hastad}
\end{frame}

\section{Closing Thoughts}
\begin{frame}
    \frametitle{What's left?}
    \begin{itemize}
    	\item Elliptic Curve Cryptography
       	\item Post Quantum Cryptography
    	\begin{itemize}
    		\item Lattice-Based Cryptography
    		\item LWE
    		\item Multivariate Cryptograhy
    	\end{itemize}
    \end{itemize}
\end{frame}

\end{document}


%
% CMPT 473: Software Quality Assurance - A Course Overview
% Section: Graph Coverage
%
% Author: Jeffrey Leung
%

\section{Graph Coverage}
	\label{sec:graph-coverage}
\begin{easylist}

& \textbf{Control flow graph (CFG):} Graph where nodes represent code and edges represent paths taken
	&& Types of nodes: Entry, decision/branch, join, exit
	&& For a while loop, see figure~\ref{img:flow-diagram-while}
	&& For a for loop, see figure~\ref{img:flow-diagram-for}
	&& For a switch statement, see figure~\ref{img:flow-diagram-switch}
	&& For a short-circuited if statement, see figure~\ref{img:flow-diagram-if-shortcircuit}

\end{easylist}

\begin{figure}[!htb]
	\centering
	\includegraphics[width=.5\linewidth]{flow-diagram-while}
	\caption{Control Flow Graph of a while loop}
	\label{img:flow-diagram-while}
\end{figure}

\begin{figure}[!htb]
	\centering
	\includegraphics[width=.5\linewidth]{flow-diagram-for}
	\caption{Control Flow Graph of a for loop}
	\label{img:flow-diagram-for}
\end{figure}

\begin{figure}[!htb]
	\centering
	\includegraphics[width=.5\linewidth]{flow-diagram-switch}
	\caption{Control Flow Graph of a switch statement}
	\label{img:flow-diagram-switch}
\end{figure}

\begin{figure}[!htb]
	\centering
	\includegraphics[width=.5\linewidth]{flow-diagram-if-shortcircuit}
	\caption{Control Flow Graph of a if statement short-circuited}
	\label{img:flow-diagram-if-shortcircuit}
\end{figure}

\begin{easylist}

& Edge coverage (i.e. all branches used) is a superset of node coverage

& \textbf{Complete path coverage:} Coverage of all possible paths through the graph
	&& Reason: Permutations/combinations of multiple possible paths can affect results
	&& Infeasible/intractible because it would entail combinatorial explosion and inability to efficiently test looping paths

& \textbf{Edge pair coverage:} Coverage which includes every path of length $<= 2$
& \textbf{Specified path coverage:} Coverage which tests $k$ paths for a given $k$

& \textbf{Reachability}: Property of a piece of code which may or may not be executable based on states
	&& \textbf{Syntactic reachability:} Analysis of reachability based on the structure of the code
	&& \textbf{Semantic reachability:} Analysis of reachability based on the meaning of the code (cannot be checked by an automated program)

& Testing loops is relevant when each iteration may affect the next
	&& \textbf{Simple path:} Acyclic path between nodes where no node appears more than once (except first/last)
	&& \textbf{Prime path:} Simple path which is not a subpath of any other simple path
		&&& I.e. A simple path which cannot be extended
		&&& E.g. Simple path which starts and ends at the same node
	&& \textbf{Tour:} Path which is a subpath of another
		&&& Mathematical definition: A path $p$ tours path $q$ if $q$ is a subpath of $p$
		&&& Tour with sidetrips: Tour where every edge of the superpath appears in the same order in the subpath
		&&& Tour with detours: Tour where every node of the superpath appears in the same order in the subpath

& \textbf{def-use pair:} Definition statement and the next relevant uses of the variable (before reassignment)
	&& Used to test along data flow
	&& Possible test approaches:
		&&& All defs coverage: Every def must be covered by at least one test of its uses
		&&& All uses coverage: Every use must be tested with least one definition
		&&& All def-use pairs coverage
		&&& All def-use paths coverage: All simple paths between def-use pairs are covered

& Path/branch coverage is insufficient due to scalability and complex conditions (e.g. non-short-circuiting) which do not have the notion of a path

\end{easylist}
\clearpage


\subsection{Ordinary goods}

As price rises, demand goes down.

\subsection{Giffen goods}

As price rises, demand goes up. Not because of the slope of the demand curve, but because the income effect and inferior effect are strong.

Substitution means you buy less.

Income means you buy less generally, but move towards inferior goods.


\documentclass[11pt, letterpaper]{article}

\usepackage{anysize}
\usepackage[colorlinks=true,urlcolor=blue,citecolor=blue]{hyperref}

\marginsize{.5in}{.5in}{.5in}{.5in}

\begin{document}
% Force pdflatex to use correct paper size.
\special{papersize=8.5in,11in}
\setlength{\pdfpageheight}{\paperheight}
\setlength{\pdfpagewidth}{\paperwidth}

\begin{center}
  {\LARGE \textbf{Fortran 90: A Brief Introduction}\\}
  {\large D. T. Welling, \today}
\end{center}

\vspace{0.5cm}
\begin{center}
  \emph{“I don't know what the language of the year 2000 will look like, but I know it will be called Fortran.”}

    —Tony Hoare, winner of the 1980 Turing Award, in 1982.
\end{center}
\vspace{0.5cm}

Fortran is a powerful and \emph{fast} programming language for doing
scientific computations.
Though Fortran tends to elicit memories of the archaic Fortran77 standard,
modern versions (Fortran90, 95, 2008 and 2018) are modern languages
that embrace modern paradigms including modularization, object-oriented
approaches, and concurrent programming.  Fortran remains popular in scientific
programming because of its speed, its inherent arrays and mathematical
capabilities.

This tutorial will get you started with the basics
of the language syntax.  To do the tasks listed below, you will need a
text editor (I recommend \href{https://www.gnu.org/software/emacs/}{Emacs}
and a compiler.  Many factors can go into choice of compiler; for this
tutorial, the biggest factor is \emph{cost}.  The
\href{https://gcc.gnu.org/fortran/}{GNU Fortran compiler} (a.k.a. gfortran)
is free and widely available via package managers and software repositories.
We'll be using that for the remainder of this exercise.  Additionally,
while there are \href{https://www.google.com/search?q=fortran+ide&oq=fortran+ide&aqs=chrome..69i57j0l6j0i20i263.1596j0j4&sourceid=chrome&ie=UTF-8}
  {several Interactive Development Enviroments (IDEs) available
    for Fortran}, we'll be using the usual Linux command line interface.

\section{Hello World \& Compilation}

Let us begin at the common starting point: \emph{Hello World}.  Type the
following into a raw text file.  Name this file \texttt{hello\_world.f90}.
For Fortran 90 source code, use the file extension \texttt{.f90}.

\begin{verbatim}
program hello_world
  ! Basic hello world program.
  implicit none

  integer :: i=2

  write(*,*) "Hello World"
  write(*,*) "Here is an integer:", i

  i = i + 3.14
  write(*,*) "Is this correct? i + pi = ", i

  stop
end program hello_world
\end{verbatim}

Right away we see the basic elements of a Fortran program.  We start by
defining the name of the program and end with an {\tt end} statement.  This
``closes'' the program section of the file.  {\tt end} is a fundamental
statement in Fortran; it closes loops, {\tt if} statements, subroutines, etc.
We also see that comments are denoted with a leading exclamation point.
Be sure to comment your code well.  Next, there is \texttt{implicit none}.
\textbf{ALWAYS include \texttt{implicit none} at the beginning of your code.}
We'll talk more about that shortly.  Next, we declare a variable.  Fortran
is \emph{statically typed}, which means that once you declare a variable, it
cannot change its type.  An integer will remain an integer, even if you add
$\pi$ to it.  We then see some \texttt{write} statements that print information
to screen.  Finally, we see the \texttt{stop} statement, which tells the
computer to stop running the program.

A quick note concerning the \texttt{stop} statement: In situations where it is the
final statement, as in the above program, it is optional.  It is required in
other situations where you are stopping the program before reaching the end
of the file.  In real world situations, you rarely need the \texttt{stop}
statement.  Except in error-handling contexts, it is more often a sign of
sloppy programming rather than a situation of requirement.  In some contexts,
especially that of parallel programming, you want to avoid it.  Therefore,
if you are considering reaching for a \texttt{stop} statement, think twice, as it
is most often not necessary.

It is notable that our code is nicely spaced out.  All commands
between the start and end of our program are tabbed over slightly to make it
easier to see what is ``within'' our program.  This is not a requirement of
Fortran90; it is a \emph{free-form} language, in contrast to Python
(with its required tab-delimiting of code blocks) or earlier versions of
Fortran.  However, maintaining tabs for code blocks is strongly recommended.
It keeps your code easy to read and follow.  Doing so is widely accepted to
the point that many text editors will do this spacing for you.

We want to run this program, but we can't.  Fortran is a compiled language,
not an interpreted language.  This means that there is an extra step between
writing our code and executing the commands.  To compile with gfortran,
navigate to the same folder as your source code and type the following:
\begin{verbatim}
gfortran hello_world.f90
\end{verbatim}

If you look in the working directory, you'll now find an executable file
named \texttt{a.out}.  This is our program!  Run it by typing its name at the
command prompt and pressing enter.  You should see the following:

\begin{verbatim}
 Hello World
 Here is an integer:           2
 Is this correct? i + pi =           5
\end{verbatim}

At this point, a more friendly author would offer a tepid congratulations on
writing your first Fortran program.  Such patronizing is avoided here.

Some quick notes on this output: the formatting seems quite arbitrary, with our
integers hanging out in limbo away from our sentences.  Also, $2+\pi \neq 5$!
What gives with the math?  Remember that Fortran is \emph{statically typed}!  When
we did our addition, $2+\pi$ was converted into an integer before the result was
stored into a variable that is explicitly an integer.  Be careful with
\emph{mixed mode arithmatic} in statically-typed languages.

\section{The \texttt{read} \texttt{write} Statements}

Let's dive further into Fortran by exploring how to interact with the user.
This is absolutely critical because, unlike interpreted languages, there is
no easy way to explore our code and variables in a real-time, interactive
manner.

Start by examing the \texttt{write} statement from our above program.
It is immediately followed by two stars {\tt *}, which indicate the \emph{output}
method and the \emph{format code}, which specifies how to organize the
data.  The stars ({\tt *}) tell Fortran to use the defaults.  For the {\tt write}
statement, the default output is \emph{standard out} ({\tt stdout}), which is
the terminal window in a Linux environment.  For the {\tt read} statement, the
default input is \emph{standard in} ({\tt stdin}), or the keyboard.
\textbf{The default behavior of Fortran {\tt read} and {\tt write} is to interact
  with the user via the command line interface}.
It is possible to also read/write to and from files (more on this below) or
directly to character variables (which is very useful for building strings!).
Let's look at standard in/out behavior in action:

\begin{verbatim}
program readwrite
  ! Basic hello world program.
  implicit none

  integer :: i=0

  write(*,*) "Hello!  Please enter an integer:"
  read(*,*) i

  write(*,*) "The number you entered is", i

end program readwrite
\end{verbatim}

Compile and run.  At the {\tt read} statement, our program prompts you to type
in something.  Do so and press enter.  Neat!  The issue, of course, is what
happens if we \emph{don't} type a number?  Our program crashes, and it's not
pretty.  There are ways of handling this, but that's for another day.

Our input and output commands above all use the default formatting approach
(i.e., the {\tt *} as the second argument to {\tt read} and {\tt write}).  This
begs the question: \emph{what is the default behavior?}  This is a complicated
question, because it depends on several things: the compiler you're using, the
number of characters required, the type of values, etc.  Suffice to say, the
default behavior is usually not pretty.  Let's learn how to change that now.

Replace the last {\tt write} line in the above program with this:

\begin{verbatim}
write(*,'(a, 2x, i5.5)') "The number you entered is", i
\end{verbatim}

Run this code twice, once typing in a short number (less than 100,000) and again
with a long number (more than 100,000).
Our {\tt write} behavior is very different!  Under gfortran, the leading space
before writing to screen is now gone.  Our integer has preceding zeros and takes
up exactly 5 characters worth of space.  There are exactly two blank spaces
between our string and our number.  If our number exceeds 5 characters in
length, Fortran spits out asterisks instead of numbers.
We control this behavior with \emph{format descriptors}
(a.k.a. \emph{format codes}), as we do in other languages.
In this example, we're telling Fortran to write out a string of arbitrary
length ({\tt a}), then add two spaces ({\tt 2x}), then print out an
integer using a maximum of five characters and filling empty characters with
zeros ({\tt i5.5}).  You can chain many format codes together to
get tightly formated lines.  You can even insert strings into the format
code proper: \verb_'("I have ", i3, " dogs")'_ would print out
``I have  10 dogs'' (if the integer printed off is the number 10).
Fortran takes these codes as law, not as suggestions- hence the asterisks if
our format code cannot accomodate the values or characters to be written.

Table \ref{tab:codes} summarizes some of the most important format code specifiers
in Fortran.  Many of these are quite similar to those used in other languages, so
there shouldn't be any surprises to seasoned programmers.  The bottom half
provides a symbol key to describe the $r$, $w$, $m$, and $d$ values that
surround each code.  Each of these modifiers are optional.  If you just use
{\tt f}, you'll get a real number using the compiler's default options.
The $r$ specifier allows you to have one code repeated many times.  For
example, if you want to print off ten real numbers, you can just use
{\tt 10f14.4} instead of repeated {\tt f14.4} ten times.

\begin{table}[!h]
  \begin{center}
  \begin{tabular}{|c|c|r|c|}
    \hline
    Code & Use & Example &  Result \\
    \hline
    $r$\textbf{I}$w.m$ & Integers             & \verb_I10.6_ & \verb_|    001234|_\\
    $r$\textbf{F}$w.d$ & Real Numbers         & \verb_F10.4_ & \verb_| 1234.5679|_\\
    $r$\textbf{E}$w.d$ & Engineering Notation & \verb_E10.4_ & \verb_|0.1235E+04|_\\
    $r$\textbf{ES}$w.d$& Scientific Notation  & \verb_ES10.4_& \verb_|1.2346E+03|_\\
    $r$\textbf{G}$w.d$ & Best-Fit Notation    & \verb_G10.4_ & \verb_| 1235.    |_\\
    &                                         & \verb_G10.3_ & \verb_| 0.123E+04|_\\
    $w$\textbf{X} & Add $w$ spaces & \verb_'100',4X,'200'_   & \verb_|100    200|_\\
    $r$\textbf{A}$w$ & Character (string) data& \verb_a10_   & \verb_|abcdefghij|_\\
    \hline
    \multicolumn{4}{|l|}{\textbf{Symbol Key}}\\
    \multicolumn{1}{|c}{$r$} & \multicolumn{3}{l|}{Number of times to repeat code}    \\
    \multicolumn{1}{|c}{$w$} & \multicolumn{3}{l|}{Total width (number of characters)}\\
    \multicolumn{1}{|c}{$d$} & \multicolumn{3}{l|}{Number of digits right of decimal} \\
    \multicolumn{1}{|c}{$m$} & \multicolumn{3}{l|}{Minimum number of characters}      \\
    \hline

  \end{tabular}
  \end{center}
  \caption{Important format specifiers for the Fortran {\tt write} statement.}
  \label{tab:codes}
\end{table}

groups of codes, strings in format codes, repeating when running out, etc.

There are several ways to emply format codes.
You can place format codes directly in the {\tt write} statements, as we did above.
You can also

\section{File I/O}

\section{Declaring Variables}

The generic form for declaring variables is as follows:
\begin{verbatim}
type(type specifier) attributes :: variable_name=value
\end{verbatim}
Of the parts above, the only required portions are the \texttt{type} and
\texttt{variable\_name}.  Let's see some examples:

\begin{verbatim}
  integer, parameter :: iCount=3

  real         :: x1, x2=5
  real(kind=8) :: xDoublePrecision=3.14159265358979323846264338

  complex :: c1 = (3.141, -3.141)

  logical :: DoPrint = .true.

  character :: letter = 'a'
  character(len=10) :: NameAuthor='Dan'
  character(len=*), parameter :: NameProgram='test program'
\end{verbatim}

Above, we see the five (and only five!) basic variable types of Fortran90:
\texttt{integers} (numbers with no decimal values), \texttt{reals} (floating-point
values), `\texttt{complex} numbers (with a real and imaginary portion),
\texttt{logicals} (can either be .true. or .false.), and \texttt{characters}.  There
are also \texttt{derived types}, which are Fortran data structures whose
behavior is declared by the user.

Following some of the \texttt{real} statements, we see \emph{kind specifiers}.
These allow us to set the precision of our reals.  A \texttt{kind=4} is,
nominally, a single-precision real, while a \texttt{kind=8} is double precision.
While your mind may snap to the idea of 32- and 64-bit reals, this is not
a well defined standard.  Further, the number of bits dedicated to the
significand and the exponent varies from machine to machine and from
compiler to compiler.  Therefore, while it is good programming practice to
set the \texttt{kind}, it is not machine portable.  A better way is to explicitly
set the precision:
\begin{verbatim}
integer, parameter :: Real8_ = SELECTED_REAL_KIND(p=14, r=200)
real(kind=Real8_)  :: x=10
\end{verbatim}
\texttt{SELECTED\_REAL\_KIND} obtains a kind number corresponding to a floating
point value with precision, \texttt{p}, of 14 (14 decimal digits required) and
range, \texttt{r}, of 200 (200 powers of ten are required in the exponent).

Character len

Following the types and kinds, , we sometimes see attributes.  Attributes set
additional characteristics of the variables.  The \texttt{parameter} attribute
makes a variable unchangeable.  This is very useful when declaring physical
constants.  Other behavior can be set as well, such as if a variable is a input
or output to a subroutine, whether it is an array or allocatable, and others.

Some of our declared values have default values set.  Others do not.  What
are the default values of variables that we did not set explicitly?

implicitly typed variables are BAD!!!

Arrays: indexing defaults to 1-based, but can be anything.

\section{Math \& Logical Operations}

When two numbers of different type or precision are used together in a single
operation, a \emph{mixed-mode arithmatic} situation arises.  What happens when
you multiply an integer and a real number?  What about a single-precision
real and a double precision number?  The answer to these questions are
\emph{machine and compiler dependent}, but there are some ground rules that
generally hold true.  First, when combining an integer and a real, Fortran
will most often convert the integer and avoid loss of precision.  When
combining two reals of different precision, Fortran will typically convert the
value of lesser precision to the kind of higher precision.  Always be
careful when encountering mixed-mode arithmatic.

Here's another tricky situation:
\begin{verbatim}
  real(kind=8) :: x = 5
  write(*,*) x*3.1415
\end{verbatim}
What is the precision of the value written to screen?  What is true above
will hold here as well beholden to the usual caveats concerning different
machines and compiliers.  It is reasonable to expect Fortran to increase the
precision of the unnamed constant value.  Some compilers will complain about
this.  There are a few ways we can remove all doubt, however.
\begin{verbatim}
  integer, parameter :: _DBL=SELECTED_REAL_KIND(p=14,200)
  real(kind=8) :: x = 5
  write(*,*) x*3.1415_8  ! Works only if "8" is a valid kind.
  write(*,*) x*3.1415D0  ! As apposed to 3.1415E0 exponent notation.
  write(*,*) x*3.1415_DBL! ...where _DBL was defined above.
\end{verbatim}

\section{Code Structures}

if-then-else
case statements
do
do while

\section{Subprograms}

\section{Debugging}

\end{document}


\subsection{Working with Strings}

With some advanced macro programming, you might need to manipulate Strings (texts) from your code. For example, let's think about a title of an image ``exp13\_C0\_Z10\_T3.tif''. Such naming occurs often to indicate that this image is from the third time point (T3), at the 11th slice (Z10, imagine that the Z slice numbering starts from 0) and its the first channel (C0).

We might be lucky enough to read out its dimensional information from header, but quite often such information is only available in the file name (the title of the image). To extract dimensional information from file name, we need to know how to deal with strings in macro to decompose those strings and extract information that we need. Build-in macro functions which are related to such tasks with strings are the following.

\begin{shaded}\begin{indentCom}
\item lengthOf(str)
\item substring(string, index1, index2)
\item indexOf(string, substring)
\item indexOf(string, substring, fromIndex)
\item lastIndexOf(string, substring)
\item startsWith(string, prefix)
\item endsWith(string, suffix)
\item matches(string, regex)
\item replace(string, old, new)
\end{indentCom}\end{shaded}

Let's go back to the example file name ``exp13\_C0\_Z10\_T3.tif'' again. If we need to get the file name without file extension, what should we do? Several ways are there, but lets start with the simplest way.

We already know that all the file names are in the TIFF format, so all file names end with ``.tif''.  We could remove this suffix by replacing the ``.tif'' with a string with length 0. We could do this by using \ilcom{replace}.

\begin{lstlisting}
name = "exp13_C0_Z10_T3.tif";
newname = replace(name, ".tif", "");
print(newname);
\end{lstlisting}

This will print out "exp13\_C0\_Z10\_T3" in the log window. In the second line, the function \ilcom{replace} is used. The old string ".tif" is replaced by a new 0 length string "". So it works!

But what if our lucky assumption that all files end with ".tif" is not true and it could be anything? To work with this, we now need to use different strategy to know the file extension.

By definition, file extension and the file name is separated by a dot. Length of the extension could be different, as some extension such as a python file is ".py" and a C code is ".c". Thus, we cannot assume that the length of the file extension is constant, but we know that there is a dot.

For such cases with variable length of file extension being expected, we first need to know about the \textbf{index} of the dot within file name. Each character within file name is positioned at certain index from the beginning of the name. In the example we are now dealing with, the index 0 is ``e''. The index 1 is ``x''. Since the index starts from 0, the last index will be total length of the file name minus one. You could modify the code above like below to try getting the length of the file name.

\begin{lstlisting}
name = "exp13_C0_Z10_T3.tif";
tlength = lengthOf(name);
print(tlength);
\end{lstlisting}

You should see ``19'' in the log window. That is the length of this file name. So in this example string, index starts from 0 and the last index is 18.

Next, we use the function \ilcom{substring(string, index1, index2)}. With this function, you could extract part of the \ilcom{string} by giving the start index (index1) and the end index (index2) as arguments. We could just try this by again modifying the code above.

\begin{lstlisting}
name = "exp13_C0_Z10_T3.tif";
subname = substring(name, 0, 3);
print(subname);
\end{lstlisting}

The output after running this code is ``exp'' printed in the log window. The second argument of the function \ilcom{substring} is 0, and the third is 3. This tells the function \ilcom{substring} to extract characters from the index 0 to the index 2 (so the third argument will be the index just after the last index that would be included in the substring).

\begin{indentexercise}{1}
Test changing the second and the third argument so that different part of the file name is extracted.
\end{indentexercise}

How could we know the index of the dot? For this we use the \ilcom{indexOf(string, substring)}. Try the following code.

\begin{lstlisting}
name = "exp13_C0_Z10_T3.tif";
dotindex = indexOf(name, ".");
print(dotindex);
\end{lstlisting}

Now you know that the index of dot is ``15''. We could then combine the knowledge we have now to compose a single macro that extracts the filename without file extension.

\begin{lstlisting}
name = "exp13_C0_Z10_T3.tif";
dotindex = indexOf(name, ".");
filename = substring(name, 0, dotindex);
print(filename);
\end{lstlisting}

Let's make the problem a bit more complicated. If the file name contains multiple dots, what should we do? In the example below, I added two more dots.

\begin{lstlisting}
name = "exp13._C0._Z10_T3.tif";
dotindex = indexOf(name, ".");
filename = substring(name, 0, dotindex);
print(filename);
\end{lstlisting}

Output is now ``exp13''. Far from what we need. To treat such case, we use \ilcom{lastIndexOf}, which returns the index of the last appearance of the given character. Let's slightly modify the code.

\begin{lstlisting}
name = "exp13._C0._Z10_T3.tif";
dotindex = lastIndexOf(name, ".");
filename = substring(name, 0, dotindex);
print(filename);
\end{lstlisting}

It should then working again as we want.

Let's change our task: We now want to know the time point that this image was taken. How should we do that? Examining the file name again, we realize that the time point number appears after ``T''. The number could be any length of digits, but currently is 0. Then the dot comes right after the number. We then just need to know the index of ``T'' \ldots but wait, we might have ``T'' anywhere, as this is a single character alphabet that could easily be a file name. Therefore we find the index of ``\_T'' that looks like more specific.

\begin{lstlisting}
name = "exp13._C0._Z10_T3.tif";
timeindex = indexOf(name, "_T");
print(timeindex);
\end{lstlisting}

Now we know that ``\_T'' is at index 14, so the number should start from the index 16 (because index 15 will be ``T''). Taken this into account, we could extract the time point.

\begin{lstlisting}
name = "exp13._C0._Z10_T3.tif";
timeindex = indexOf(name, "_T");
dotindex = lastIndexOf(name, ".");
timepoint = substring(name, timeindex + 2, dotindex);
print(timepoint);
\end{lstlisting}

The time point that you have just now captured is a string. You can not pass this to mathematical assignments. To do so, you need to convert this to a number. For doing so, you could use \ilcom{parseInt(string)}.

\begin{lstlisting}
name = "exp13._C0._Z10_T3.tif";
timeindex = indexOf(name, "_T");
dotindex = lastIndexOf(name, ".");
timepoint = substring(name, timeindex + 2, dotindex);
timepoint = parseInt(timepoint);
print(timepoint * 2);
\end{lstlisting}

An example case where conversion of string to number (in this case an integer) required would be when you need to compare such file names and get the maximum time point from all the file names. Usage is diverse, but at some point you need to use this. If you need a Float number (numbers with decimal point), use \ilcom{parseFloat(string)}.