% LaTeX source for ``Python for Informatics: Exploring Information''
% Copyright (c)  2010-  Charles R. Severance, All Rights Reserved

\chapter{Strings}
\label{strings}

\section{A string is a sequence}
\index{sequence}
\index{character}
\index{bracket operator}
\index{operator!bracket}

A string is a {\bf sequence} of characters.
You can access the characters one at a time with the
bracket operator:

\beforeverb
\begin{verbatim}
>>> fruit = 'banana'
>>> letter = fruit[1]
\end{verbatim}
\afterverb
%
\index{index}
The second statement extracts the character at index position 1 from the
{\tt fruit} variable and assigns it to the {\tt letter} variable.

The expression in brackets is called an {\bf index}.
The index indicates which character in the sequence you
want (hence the name).

But you might not get what you expect:

\beforeverb
\begin{verbatim}
>>> print letter
a
\end{verbatim}
\afterverb
%
For most people, the first letter of \verb"'banana'" is {\tt b}, not
{\tt a}.  But in Python, the index is an offset from the
beginning of the string, and the offset of the first letter is zero.

\beforeverb
\begin{verbatim}
>>> letter = fruit[0]
>>> print letter
b
\end{verbatim}
\afterverb
%
So {\tt b} is the 0th letter (``zero-eth'') of \verb"'banana'", {\tt a}
is the 1th letter (``one-eth''), and {\tt n} is the 2th (``two-eth'')
letter.

\beforefig
\centerline{\includegraphics[height=0.50in]{figs2/string.eps}}
\afterfig

\index{index!starting at zero}
\index{zero, index starting at}

You can use any expression, including variables and operators, as an
index, but the value of the index has to be an integer.  Otherwise you
get:

\index{index}
\index{exception!TypeError}
\index{TypeError}

\beforeverb
\begin{verbatim}
>>> letter = fruit[1.5]
TypeError: string indices must be integers
\end{verbatim}
\afterverb
%

\section{Getting the length of a string using {\tt len}}

\index{len function}
\index{function!len}

{\tt len} is a built-in function that returns the number of characters
in a string:

\beforeverb
\begin{verbatim}
>>> fruit = 'banana'
>>> len(fruit)
6
\end{verbatim}
\afterverb
%
To get the last letter of a string, you might be tempted to try something
like this:

\index{exception!IndexError}
\index{IndexError}

\beforeverb
\begin{verbatim}
>>> length = len(fruit)
>>> last = fruit[length]
IndexError: string index out of range
\end{verbatim}
\afterverb
%
The reason for the {\tt IndexError} is that there is no letter in {\tt
'banana'} with the index 6.  Since we started counting at zero, the
six letters are numbered 0 to 5.  To get the last character, you have
to subtract 1 from {\tt length}:

\beforeverb
\begin{verbatim}
>>> last = fruit[length-1]
>>> print last
a
\end{verbatim}
\afterverb
%
Alternatively, you can use negative indices, which count backward from
the end of the string.  The expression {\tt fruit[-1]} yields the last
letter, {\tt fruit[-2]} yields the second to last, and so on.

\index{index!negative}
\index{negative index}

\section{Traversal through a string with a loop}
\label{for}

\index{traversal}
\index{loop!traversal}
\index{for loop}
\index{loop!for}
\index{statement!for}
\index{traversal}

A lot of computations involve processing a string one character at a
time.  Often they start at the beginning, select each character in
turn, do something to it, and continue until the end.  This pattern of
processing is called a {\bf traversal}.  One way to write a traversal
is with a {\tt while} loop:

\beforeverb
\begin{verbatim}
index = 0
while index < len(fruit):
    letter = fruit[index]
    print letter
    index = index + 1
\end{verbatim}
\afterverb
%
This loop traverses the string and displays each letter on a line by
itself.  The loop condition is {\tt index < len(fruit)}, so
when {\tt index} is equal to the length of the string, the
condition is false, and the body of the loop is not executed.  The
last character accessed is the one with the index {\tt len(fruit)-1},
which is the last character in the string.

\begin{ex}
Write a {\tt while} loop that starts at the last character in the string
and works its way backwards to the first character in the string,
printing each letter on a separate line, except backwards.
\end{ex}

Another way to write a traversal is with a {\tt for} loop:

\beforeverb
\begin{verbatim}
for char in fruit:
    print char
\end{verbatim}
\afterverb
%
Each time through the loop, the next character in the string is assigned
to the variable {\tt char}.  The loop continues until no characters are
left.

\section{String slices}
\label{slice}

\index{slice operator}
\index{operator!slice}
\index{index!slice}
\index{string!slice}
\index{slice!string}

A segment of a string is called a {\bf slice}.  Selecting a slice is
similar to selecting a character:

\beforeverb
\begin{verbatim}
>>> s = 'Monty Python'
>>> print s[0:5]
Monty
>>> print s[6:12]
Python
\end{verbatim}
\afterverb
%
The operator {\tt [n:m]} returns the part of the string from the
``n-eth'' character to the ``m-eth'' character, including the first but
excluding the last.

If you omit the first index (before the colon), the slice starts at
the beginning of the string.  If you omit the second index, the slice
goes to the end of the string:

\beforeverb
\begin{verbatim}
>>> fruit = 'banana'
>>> fruit[:3]
'ban'
>>> fruit[3:]
'ana'
\end{verbatim}
\afterverb
%
If the first index is greater than or equal to the second the result
is an {\bf empty string}, represented by two quotation marks:

\index{quotation mark}

\beforeverb
\begin{verbatim}
>>> fruit = 'banana'
>>> fruit[3:3]
''
\end{verbatim}
\afterverb
%
An empty string contains no characters and has length 0, but other
than that, it is the same as any other string.

\begin{ex}
Given that {\tt fruit} is a string, what does
{\tt fruit[:]} mean?

\index{copy!slice}
\index{slice!copy}

\end{ex}

\section{Strings are immutable}
\index{mutability}
\index{immutability}
\index{string!immutable}

It is tempting to use the {\tt []} operator on the left side of an
assignment, with the intention of changing a character in a string.
For example:

\index{TypeError}
\index{exception!TypeError}

\beforeverb
\begin{verbatim}
>>> greeting = 'Hello, world!'
>>> greeting[0] = 'J'
TypeError: object does not support item assignment
\end{verbatim}
\afterverb
%
The ``object'' in this case is the string and the ``item'' is
the character you tried to assign.  For now, an {\bf object} is
the same thing as a value, but we will refine that definition
later.  An {\bf item} is one of the values in a sequence.

\index{object}
\index{item assignment}
\index{assignment!item}
\index{immutability}

The reason for the error is that
strings are {\bf immutable}, which means you can't change an
existing string.  The best you can do is create a new string
that is a variation on the original:

\beforeverb
\begin{verbatim}
>>> greeting = 'Hello, world!'
>>> new_greeting = 'J' + greeting[1:]
>>> print new_greeting
Jello, world!
\end{verbatim}
\afterverb
%
This example concatenates a new first letter onto
a slice of {\tt greeting}.  It has no effect on
the original string.

\index{concatenation}

\section{Looping and counting}
\label{counter}

\index{counter}
\index{counting and looping}
\index{looping and counting}
\index{looping!with strings}

The following program counts the number of times the letter {\tt a}
appears in a string:

\beforeverb
\begin{verbatim}
word = 'banana'
count = 0
for letter in word:
    if letter == 'a':
        count = count + 1
print count
\end{verbatim}
\afterverb
%
This program demonstrates another pattern of computation called a {\bf
counter}.  The variable {\tt count} is initialized to 0 and then
incremented each time an {\tt a} is found.
When the loop exits, {\tt count}
contains the result---the total number of {\tt a}'s.

\begin{ex}
\index{encapsulation}

Encapsulate this code in a function named {\tt
count}, and generalize it so that it accepts the string and the
letter as arguments.
\end{ex}

\section{The {\tt in} operator}
\label{inboth}

\index{in operator}
\index{operator!in}
\index{boolean operator}
\index{operator!boolean}

The word {\tt in} is a boolean operator that takes two strings and
returns {\tt True} if the first appears as a substring in the second:

\beforeverb
\begin{verbatim}
>>> 'a' in 'banana'
True
>>> 'seed' in 'banana'
False
\end{verbatim}
\afterverb
%

\section{String comparison}

\index{string!comparison}
\index{comparison!string}

The comparison operators work on strings.  To see if two strings are equal:

\beforeverb
\begin{verbatim}
if word == 'banana':
    print  'All right, bananas.'
\end{verbatim}
\afterverb
%
Other comparison operations are useful for putting words in alphabetical
order:

\beforeverb
\begin{verbatim}
if word < 'banana':
    print 'Your word,' + word + ', comes before banana.'
elif word > 'banana':
    print 'Your word,' + word + ', comes after banana.'
else:
    print 'All right, bananas.'
\end{verbatim}
\afterverb
%
Python does not handle uppercase and lowercase letters the same way
that people do.  All the uppercase letters come before all the
lowercase letters, so:

\beforeverb
\begin{verbatim}
Your word, Pineapple, comes before banana.
\end{verbatim}
\afterverb
%
A common way to address this problem is to convert strings to a
standard format, such as all lowercase, before performing the
comparison.  Keep that in mind in case you have to defend yourself
against a man armed with a Pineapple.

\section{{\tt string} methods}

Strings are an example of Python {\bf objects}.  An object contains
both data (the actual string itself) and {\bf methods}, which
are effectively functions that are built into the object and
are available to any {\bf instance} of the object.

Python has a function called {\tt dir} which lists the methods available
for an object.  The {\tt type} function shows the type of an object
and the {\tt dir} function shows the available methods.

\beforeverb
\begin{verbatim}
>>> stuff = 'Hello world'
>>> type(stuff)
<type 'str'>
>>> dir(stuff)
['capitalize', 'center', 'count', 'decode', 'encode',
'endswith', 'expandtabs', 'find', 'format', 'index',
'isalnum', 'isalpha', 'isdigit', 'islower', 'isspace',
'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip',
'partition', 'replace', 'rfind', 'rindex', 'rjust',
'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines',
'startswith', 'strip', 'swapcase', 'title', 'translate',
'upper', 'zfill']
>>> help(str.capitalize)
Help on method_descriptor:

capitalize(...)
    S.capitalize() -> string

    Return a copy of the string S with only its first character
    capitalized.
>>>
\end{verbatim}
\afterverb
%

While the {\tt dir} function lists the methods, and you
can use {\tt help} to get some simple documentation on a method,
a better source of documentation for string methods would be
\url{https://docs.python.org/2/library/stdtypes.html#string-methods}.

Calling a {\bf method} is similar to calling a function---it
takes arguments and returns a value---but the syntax is different.
We call a method by appending the method name to the variable name
using the period as a delimiter.

For example, the
method {\tt upper} takes a string and returns a new string with
all uppercase letters:

\index{method}
\index{string!method}

Instead of the function syntax {\tt upper(word)}, it uses
the method syntax {\tt word.upper()}.

\index{dot notation}

\beforeverb
\begin{verbatim}
>>> word = 'banana'
>>> new_word = word.upper()
>>> print new_word
BANANA
\end{verbatim}
\afterverb
%
This form of dot notation specifies the name of the method, {\tt
upper}, and the name of the string to apply the method to, {\tt
word}.  The empty parentheses indicate that this method takes no
argument.

\index{parentheses!empty}

A method call is called an {\bf invocation}; in this case, we would
say that we are invoking {\tt upper} on the {\tt word}.

\index{invocation}

For example, there is a string method named {\tt find} that
searches for the position of one string within another:

\beforeverb
\begin{verbatim}
>>> word = 'banana'
>>> index = word.find('a')
>>> print index
1
\end{verbatim}
\afterverb
%
In this example, we invoke {\tt find} on {\tt word} and pass
the letter we are looking for as a parameter.

The {\tt find} method can find substrings as well as characters:

\beforeverb
\begin{verbatim}
>>> word.find('na')
2
\end{verbatim}
\afterverb
%
It can take as a second argument the index where it should start:

\index{optional argument}
\index{argument!optional}

\beforeverb
\begin{verbatim}
>>> word.find('na', 3)
4
\end{verbatim}
\afterverb
%
One common task is to remove white space (spaces, tabs, or newlines) from
the beginning and end of a string using the {\tt strip} method:

\beforeverb
\begin{verbatim}
>>> line = '  Here we go  '
>>> line.strip()
'Here we go'
\end{verbatim}
\afterverb
%
Some methods such as {\bf startswith} return boolean values.

\beforeverb
\begin{verbatim}
>>> line = 'Please have a nice day'
>>> line.startswith('Please')
True
>>> line.startswith('p')
False
\end{verbatim}
\afterverb
%
You will note that {\tt startswith} requires case to match, so sometimes
we take a line and map it all to lowercase before we do any checking
using the {\tt lower} method.

\beforeverb
\begin{verbatim}
>>> line = 'Please have a nice day'
>>> line.startswith('p')
False
>>> line.lower()
'please have a nice day'
>>> line.lower().startswith('p')
True
\end{verbatim}
\afterverb
%
In the last example, the method {\tt lower} is called
and then we use {\tt startswith}
to see if the resulting lowercase string
starts with the letter ``p''.  As long as we are careful
with the order, we can make multiple method calls in a
single expression.

\begin{ex}
\index{count method}
\index{method!count}

There is a string method called {\tt count} that is similar
to the function in the previous exercise.  Read the documentation
of this method at
\url{https://docs.python.org/2/library/stdtypes.html#string-methods}
and write an invocation that counts the number of times the
letter a  occurs
in \verb"'banana'".
\end{ex}

\section{Parsing strings}

Often, we want to look into a string and find a substring.  For example
if we were presented a series of lines formatted as follows:

\beforeverb
\begin{alltt}
From stephen.marquard@{\bf uct.ac.za} Sat Jan  5 09:14:16 2008
\end{alltt}
\afterverb

and we wanted to pull out only the second half of the address (i.e.,
{\tt uct.ac.za}) from each line, we can do this by using the {\tt find}
method and string slicing.

First, we will find the position of the at-sign in the string.  Then we will
find the position of the first space \emph{after} the at-sign.  And then we
will use string slicing to extract the portion of the string which we
are looking for.

\beforeverb
\begin{verbatim}
>>> data = 'From stephen.marquard@uct.ac.za Sat Jan  5 09:14:16 2008'
>>> atpos = data.find('@')
>>> print atpos
21
>>> sppos = data.find(' ',atpos)
>>> print sppos
31
>>> host = data[atpos+1:sppos]
>>> print host
uct.ac.za
>>>
\end{verbatim}
\afterverb
%
We use a version of the {\tt find} method which allows us to specify
a position in the string where we want {\tt find} to start looking.
When we slice, we extract the characters
from ``one beyond the at-sign through up to \emph{but not including} the
space character''.

The documentation for the {\tt find} method is available at
\url{https://docs.python.org/2/library/stdtypes.html#string-methods}.

\section{Format operator}

\index{format operator}
\index{operator!format}

The {\bf format operator}, {\tt \%}
allows us to construct strings, replacing parts of the strings
with the data stored in variables.
When applied to integers, {\tt \%} is the modulus operator.  But
when the first operand is a string, {\tt \%} is the format operator.

\index{format string}

The first operand is the {\bf format string}, which contains
one or more {\bf format sequences} that specify how
the second operand is formatted.  The result is a string.

\index{format sequence}

For example, the format sequence \verb"'%d'" means that
the second operand should be formatted as an
integer ({\tt d} stands for ``decimal''):

\beforeverb
\begin{verbatim}
>>> camels = 42
>>> '%d' % camels
'42'
\end{verbatim}
\afterverb
%
The result is the string \verb"'42'", which is not to be confused
with the integer value {\tt 42}.

A format sequence can appear anywhere in the string,
so you can embed a value in a sentence:

\beforeverb
\begin{verbatim}
>>> camels = 42
>>> 'I have spotted %d camels.' % camels
'I have spotted 42 camels.'
\end{verbatim}
\afterverb
%
If there is more than one format sequence in the string,
the second argument has to be a tuple\footnote{A tuple is a
sequence of comma-separated values inside a pair of parenthesis.
We will cover tuples in Chapter 10}.  Each format sequence is
matched with an element of the tuple, in order.

The following example uses \verb"'%d'" to format an integer,
\verb"'%g'" to format
a floating-point number (don't ask why), and \verb"'%s'" to format
a string:

\beforeverb
\begin{verbatim}
>>> 'In %d years I have spotted %g %s.' % (3, 0.1, 'camels')
'In 3 years I have spotted 0.1 camels.'
\end{verbatim}
\afterverb
%
The number of elements in the tuple must match the number
of format sequences in the string.  The types of the
elements also must match the format sequences:

\index{exception!TypeError}
\index{TypeError}

\beforeverb
\begin{verbatim}
>>> '%d %d %d' % (1, 2)
TypeError: not enough arguments for format string
>>> '%d' % 'dollars'
TypeError: illegal argument type for built-in operation
\end{verbatim}
\afterverb
%
In the first example, there aren't enough elements; in the
second, the element is the wrong type.

The format operator is powerful, but it can be difficult to use.  You
can read more about it at
\url{https://docs.python.org/2/library/stdtypes.html#string-formatting}.

% You can specify the number of digits as part of the format sequence.
% For example, the sequence \verb"'%8.2f'"
% formats a floating-point number to be 8 characters long, with
% 2 digits after the decimal point:

% \beforeverb
% \begin{verbatim}
% >>> '%8.2f' % 3.14159
% '    3.14'
% \end{verbatim}
% \afterverb
% %
% The result takes up eight spaces with two
% digits after the decimal point.

\section{Debugging}
\index{debugging}

A skill that you should cultivate as you program is always
asking yourself, ``What could go wrong here?'' or alternatively,
``What crazy thing might our user do to crash our (seemingly)
perfect program?''

For example, look at the program which we used to demonstrate
the {\tt while} loop in the chapter on iteration:

\beforeverb
\begin{verbatim}
while True:
    line = raw_input('> ')
    if line[0] == '#' :
        continue
    if line == 'done':
        break
    print line

print 'Done!'
\end{verbatim}
\afterverb
%
Look what happens when the user enters an empty line of input:

\beforeverb
\begin{verbatim}
> hello there
hello there
> # don't print this
> print this!
print this!
>
Traceback (most recent call last):
  File "copytildone.py", line 3, in <module>
    if line[0] == '#' :
\end{verbatim}
\afterverb
%
The code works fine until it is presented an empty line.  Then
there is no zero-th character, so we get a traceback.  There are two
solutions to this to make line three ``safe'' even if the line is
empty.

One possibility is to simply use the {\tt startswith} method
which returns {\tt False} if the string is empty.

\beforeverb
\begin{verbatim}
    if line.startswith('#') :
\end{verbatim}
\afterverb
%
\index{guardian pattern}
\index{pattern!guardian}
Another way is to safely write the {\tt if} statement using the {\bf guardian}
pattern and make sure the second logical expression is evaluated
only where there is at least one character in the string.:

\beforeverb
\begin{verbatim}
    if len(line) > 0 and line[0] == '#' :
\end{verbatim}
\afterverb
%

\section{Glossary}

\begin{description}

\item[counter:] A variable used to count something, usually initialized
to zero and then incremented.
\index{counter}

\item[empty string:] A string with no characters and length 0, represented
by two quotation marks.
\index{empty string}

\item[format operator:] An operator, {\tt \%}, that takes a format
string and a tuple and generates a string that includes
the elements of the tuple formatted as specified by the format string.
\index{format operator}
\index{operator!format}

\item[format sequence:] A sequence of characters in a format string,
like {\tt \%d}, that specifies how a value should be formatted.
\index{format sequence}

\item[format string:] A string, used with the format operator, that
contains format sequences.
\index{format string}

\item[flag:] A boolean variable used to indicate whether a condition
is true.
\index{flag}

\item[invocation:] A statement that calls a method.
\index{invocation}

\item[immutable:] The property of a sequence whose items cannot
be assigned.
\index{immutability}

\item[index:] An integer value used to select an item in
a sequence, such as a character in a string.
\index{index}

\item[item:] One of the values in a sequence.
\index{item}

\item[method:] A function that is associated with an object and called
using dot notation.
\index{method}

\item[object:] Something a variable can refer to.  For now,
you can use ``object'' and ``value'' interchangeably.
\index{object}

\item[search:] A pattern of traversal that stops
when it finds what it is looking for.
\index{search pattern}
\index{pattern!search}

\item[sequence:] An ordered set; that is, a set of
values where each value is identified by an integer index.
\index{sequence}

\item[slice:] A part of a string specified by a range of indices.
\index{slice}

\item[traverse:] To iterate through the items in a sequence,
performing a similar operation on each.
\index{traversal}

\end{description}

\section{Exercises}

\begin{ex}
Take the following Python code that stores a string:`

\beforeverb
\begin{alltt}
str = 'X-DSPAM-Confidence: {\bf 0.8475}'
\end{alltt}
\afterverb

Use {\tt find} and string slicing to extract the portion
of the string after the colon character and then use the
{\tt float} function to convert the extracted string
into a floating point number.

\end{ex}

\begin{ex}
\index{string method}
\index{method!string}

Read the documentation of the string methods at
\url{https://docs.python.org/2/library/stdtypes.html#string-methods}.
You might want to experiment with some of them to make sure
you understand how they work.  {\tt strip} and
{\tt replace} are particularly useful.

The documentation uses a syntax that might be confusing.
For example, in \verb"find(sub[, start[, end]])", the brackets
indicate optional arguments.  So {\tt sub} is required, but
{\tt start} is optional, and if you include {\tt start},
then {\tt end} is optional.
\end{ex}


\section{Hacker's Delight}\label{sec:book}
\epigraph{
``There is nothing here about hacking
in the sense of breaking into computers.''
}{
\emph{Hacker's Delight} Website
\cite{Warren:HD:Website}
}

\subsection*{The Book}
Henry Warren's book \emph{Hacker's Delight}
is described by himself as
``a collection of small programming tricks''
\cite{Warren:2012:HD:2462741} (p. \texttt{xv}).
In fact, the table of contents reads like
an encyclopedia of formulas for bit level manipulation:

It starts out with detecting, counting and changing
specific bits in a bitword.
From there on, everything imaginable seems to be part of this collection.
This also includes bit-level manipulations that result
in a mathematical meaning (interpreting the bitword as a number).

The bypassing of common algebraic steps
to achieve performance gains is the typical example
of what should be considered a ``hack''.
Concretely, like the book this paper exploits
bit-level parallelism of instructions
(e.g.~a~logical function is calculated for all bits of a bitword
at the same time).

\subsection*{The Book's Title}
As addressed in the epigraph, ``hacker'' refers to
someone enjoying clever ways of coding,
not necessarily being a professional
or creating useful work.
The ``delight'' is developed when faced with the elegance of some solutions.

\subsection*{This Paper}
Inspired by and named after \emph{Hacker's Delight},
this paper complements it with explanations
that have been left out by the author (intentionally):

``The presentation is informal.
Proofs are given only when the algorithm is not obvious,
and sometimes not even then.'' \cite{Warren:2012:HD:2462741}

Instead of simply explaining given formulas,
this essay assembles them from a few elementary operations
in a stepwise manner.
Besides a fundamental understanding of the covered topics,
the aim is to provide the reader with methods
that can be adapted and thereby used to create new formulas.


%% Copyright (C) 2009-2012, Gostai S.A.S.
%%
%% This software is provided "as is" without warranty of any kind,
%% either expressed or implied, including but not limited to the
%% implied warranties of fitness for a particular purpose.
%%
%% See the LICENSE file for more information.

\chapter{First Steps}
\label{sec:tut:first}

This section expects that you already know how to run \command{urbi}.  If
not, please first see \autoref{sec:tut:started}.

This section introduces the most basic notions to write \us code. Some
aspects are presented only minimally.  The goal of this section is to
bootstrap yourself with the \us language, to be able to study more in-depth
examples afterward.

\section{Comments}

Commenting your code is crucial, so let's start by learning how to do this
in \us. \dfn{Comments} are ignored by the interpreter, and can be left as
documentation, reminder, \ldots \us supports \langC and \Cxx style comments:

\begin{itemize}
\item \langC style comments start with \textcmt{/*} and end with \textcmt{*/}.
  Contrary to \langC/\Cxx, this type of comments does nest.
\item \Cxx style comments start with \textcmt{//} and last until the end of
  the line.
\end{itemize}

\begin{urbiscript}[firstnumber=1]
1; // This is a C++ style comment.
[00000000] 1

2 + /* This is a C-style comment. */ 2;
[00000000] 4

"foo" /* You /* can /* nest */ */ comments. */ "bar";
[00000000] "foobar"
\end{urbiscript}

\autoref{sec:tut:started} introduced some of the conventions used in this
document: frames such as the previous one denote ``\us sessions'', i.e.,
dialogs between \urbi and you.  The output is prefixed by a number between
square brackets: this is the date (in milliseconds since the server was
launched) at which that line was sent by the server. This is useful at
occasions, since \urbi is meant to run many parallel commands.  Since these
timestamps are irrelevant in documentation, they will often be filled with
zeroes.  More details about the typesetting of this document (and the other
kinds of frames) can be found in \autoref{sec:notations}.

\section{Literal values}

Several special kinds of ``values'' can be entered directly with a specific
syntax.  They are called \dfn{literals}, or sometimes \dfn{manifest values}.
We just met a first kind of literals: integers.  There are several others,
such as:

\begin{itemize}
\item \refObject[Float]{floats}: floating point numbers.
\begin{urbiscript}
42; // Integer literal.
[00000000] 42

3.14; // Floating point number literal.
[00000000] 3.14
\end{urbiscript}

\item \refObject[String]{strings}: character strings.
\begin{urbiscript}
"string";
[00000000] "string"
\end{urbiscript}

\item \refObject[List]{lists}: ordered collection of values.
\begin{urbiscript}
[1, 2, "a", "b"];
[00000000] [1, 2, "a", "b"]
\end{urbiscript}

\item \refObject[Dictionary]{dictionaries}: unordered collection of
  associations.
\begin{urbiscript}
["a" => 1, "b" => 2, 3 => "three"];
[00000000] [3 => "three", "a" => 1, "b" => 2]
\end{urbiscript}

\item \refObject{nil}: neutral value, or value placeholder. Think of it as
  the value that fits anywhere.
\begin{urbiscript}
nil;
\end{urbiscript}

\item \refObject{void}: absence of value. Think of it as the value that fits
  nowhere.
\begin{urbiscript}
void;
\end{urbiscript}
\end{itemize}

These examples highlight some points:
\begin{itemize}
\item \refObject[List]{Lists} and \refObject[Dictionary]{dictionaries} in
  \us are heterogeneous. That is, they can hold values of different types.
\item The printing of \refObject{nil} and \refObject{void} is empty.
\item There are many hyperlinks in this document: clicking on names such as
  \refObject{Dictionary} will drive you immediately to its specifications.
  This is also true for slots, such as \refSlot[String]{size}.
\end{itemize}

\section{Function calls}

You can call functions with the classical, mathematical notation.

\begin{urbiscript}
Math.cos(0); // Compute cosine
[00000000] 1
Math.max(1, 3); // Get the maximum of the arguments.
[00000000] 3
Math.max(1, 3, 4, 2);
[00000000] 4
\end{urbiscript}

Again, the result of the evaluation are printed out. You can see here
that function in \us can be variadic, that is, take different number
of arguments, such as the \lstinline{max} function. Let's now try the
\lstindex{echo} function, that prints out its argument.

\begin{urbiscript}
echo("Hello world!");
[00000000] *** Hello world!
\end{urbiscript}

The server prints out \lstinline{Hello world!}, as expected. Note that this
output is still prepended with the time stamp. Since \lstinline{echo}
returns \lstinline{void}, no evaluation result is printed.

\section{Variables}\index{variable}
Variables can be introduced with the \lstinline{var} keyword, given a
name and an initial value. They can be assigned new values with the
\lstinline{=} operator.

\begin{urbiscript}
var x = 42;
[00000000] 42
echo(x);
[00000000] *** 42
x = 51;
[00000000] 51
x;
[00000000] 51
\end{urbiscript}

Note that, just as in \Cxx, assignments return the (right-hand side) value, so
you can write code like ``\lstinline|x = y = 0|''. The rule for valid
identifiers is also the same as in \Cxx: they may contain alphanumeric
characters and underscores, but they may not start with a digit.

You may omit the initialization value, in which case it defaults to
\lstinline|void|.

\begin{urbiscript}
var y;
y;
// Remember, the interpreter remains silent because void is printed out
// as nothing.  You can convince yourself that y is actually void with
// the following methods.
y.asString;
[00000000] "void"
y.isVoid;
[00000000] true
\end{urbiscript}

\section{Scopes}
\dfn{Scopes} are introduced with curly brackets (\lstinline|{}|).  They can
contain any number of statements. Variables declared in a scope only exist
within this scope.

\begin{urbicomment}
removeSlots("x");
\end{urbicomment}
\begin{urbiscript}
{
  var x = "test";
  echo(x);
};
[00000000] *** test
// x is no longer defined here
x;
[00000073:error] !!! lookup failed: x
\end{urbiscript}

Note that the interpreter waits for the whole scope to be input to evaluate
it. Also note the mandatory terminating semicolon after the closing curly
bracket.

\section{Method calls}

Methods are called on objects with the dot (\lstinline{.}) notation as in
\Cxx.  Method calls can be chained.  Methods with no arguments don't require
the parentheses.

\begin{urbiscript}
0.cos();
[00000000] 1
"a-b-c".split("-");
[00000000] ["a", "b", "c"]
"foo".length();
[00000000] 3
// Method call can be chained
"".length().cos();
[00000000] 1
\end{urbiscript}

In \lstinline|obj.method|, we say that \lstinline{obj} is the
\dfn{target}, and that we are sending him the \lstinline{method}
\dfn{message}.

\section{Function definition}

You know how to call routines, let's learn how to write
some. Functions can be declared thanks to the \lstinline{function}
keyword, followed by the comma separated, parentheses surrounded list
of formal arguments, and the body between curly brackets.

\begin{urbiscript}
// Define myFunction
function myFunction()
{
  echo("Hello world");
  echo("from my function!");
};
[00000000] function () {
[:]  echo("Hello world");
[:]  echo("from my function!");
[:]}

// Invoke it
myFunction();
[00000000] *** Hello world
[00000000] *** from my function!
\end{urbiscript}

Note the strange output after you defined the function. \us seems to
be printing the function you just typed in again. This is because
a function definition evaluates to the freshly created function.

Functions are first class citizen: they are values, just as
\lstinline{0} or \lstinline{"foobar"}.  The evaluation of a function
definition yields the new function, and as always, the interpreter
prints out the evaluation result, thus showing you the function
again:

\begin{urbiscript}
// Work in a scope.
{
  // Define f
  function f()
  {
    echo("f")
  };
  // This does not invoke f, it returns its value.
  f;
};
[00000000] function () { echo("f") }
{
  // Define f
  function f()
  {
    echo("Hello World");
  };
  // This actually calls f
  f();
};
[00000000] *** Hello World
\end{urbiscript}

Here you can see that \lstinline{f} is actually a simple value. You can just
evaluate it to see its value, that is, its body. By adding the
parentheses, you can actually call the function. This is a difference
with methods calling, where empty parentheses are optional: method are
always evaluated, you cannot retrieve their functional value --- of
course, you can with a different construct, but that's not the point
here.

Since this output is often irrelevant, most of the time it is hidden in this
documentation using the \lstinline'|;' trick.  When a statement is
``missing'', an empty statement (\lstinline|{}|) is inserted.  So
\lstinline'\var{code}|;' is actually equivalent to
\lstinline'\var{code} | {};', which means ``run \var{code}, then run
\lstinline'{}' and return its value''.  Since the value of \lstinline'{}' is
\lstinline'void', which is not displayed, this is a means to discard the
result of a computation, and avoid that something is printed.  Contrast the
two following function definitions.

\begin{urbiscript}
function sum(a, b, c)
{
  return a + b + c;
};
[00003553] function (var a, var b, var c) { return a.'+'(b).'+'(c) }
function sum2(a, b, c)
{
  return a + b + c;
}|;
sum(20, 2, 20);
[00003556] 42
\end{urbiscript}

The \lstinline{return} keyword breaks the control flow of a function
(similarly to the way \lstinline{break} interrupts a loop) and returns the
control flow to the caller.  It accepts an optional argument, the value to
return to the caller.

In \us, if no \lstinline{return} statement is executed, the value of the
last expression is returned.  Actually, refrained from using
\lstinline{return} when you don't need it, it is both less readable (once
you get used to this programming style), and less efficient
(\autoref{sec:guideline:return}).

\begin{urbiscript}
function succ(i) { i + 1 }|;
succ(50);
[00000000] 51
\end{urbiscript}

\section{Conclusion}

You're now up and running with basic \us code, and we can dive in
details into advanced \us code.

%%% Local Variables:
%%% coding: utf-8
%%% mode: latex
%%% TeX-master: "../urbi-sdk"
%%% ispell-dictionary: "american"
%%% ispell-personal-dictionary: "../urbi.dict"
%%% fill-column: 76
%%% End:


% Document Type: LaTeX
% Master File: ps1-intro.tex
% this has been revised by GJS (14-jan) and LDB (16-jan, 27-jan)

\input ../6001mac

 \def\fbox#1{%
  \vtop{\vbox{\hrule%
     \hbox{\vrule\kern3pt%
 \vtop{\vbox{\kern3pt#1}\kern3pt}%
 \kern3pt\vrule}}%
 \hrule}}

\begin{document}

\psetheader{Spring Semester, 1992-93}{Problem Set 2}

\medskip

\begin{flushleft}
Issued:  February 9, 1992 \\
\smallskip
Tutorial preparation for: Week of February 15 \\
\smallskip

Written solutions are due in Recitation:
\begin{itemize}
\item Even Numbered Recitation Sections: Wednesday February 17;
\item Odd Numbered Recitation Sections: Friday, February 19.
\end{itemize}

\smallskip

Reading:
\begin{tightlist}

\item Text: Complete Chapter 1.

\end{tightlist}
\end{flushleft}

\section{1. Tutorial Exercises}

\paragraph{Exercise 2.1}
Do exercise 1.23 of the text. You should make use of the
{\tt sum} procedure defined on p. 47 of the notes.

\paragraph{Exercise 2.2}
Do exercise 1.24 of the text.

\paragraph{Exercise 2.3}
Determine the orders of growth for the {\tt my-sine}
and {\tt recurrent-sine} procedures in Problem Set 1.
Consider both the growth
of space and number of operations and their dependence on the
size of the argument and the value of the global variable {\tt epsilon},
using the number of
applications of the {\tt my-sine} and {\tt recurrent-sine}
procedures to measure the number of operations.

How would the orders of growth for the {\tt my-sine} procedure
be changed if it had been defined as

\beginlisp
(define (my-sine x)
  (if (small-enuf? x)
      x
      (* (my-sine (/ x 3))
         (- 3
            (* 4 (my-sine (/ x 3))
                 (my-sine (/ x 3)))))))
\endlisp

\paragraph{ Exercise 2.4 }
Procedure {\tt repeated} is defined as

\beginlisp
(define (repeated p n)
  (cond ((= n 0) (lambda (x) x))
        ((= n 1) p)
        (else (lambda (x) (p ((repeated p (-1+ n)) x))))))
\endlisp

Determine the values of the expressions

\beginlisp
((repeated square 2) 5)
\endlisp

and

\beginlisp
((repeated square 5) 2).
\endlisp

In tutorial you may be asked how the interpreter evaluates expressions
similar to these.

\section{2. Continued Fractions}

Continued fractions play an important role in number theory and in
approximation theory.  In this programming
assignment, you will write some short procedures that evaluate
continued fractions.  There is no predefined code for you to load.

An infinite continued fraction is an expression of the form

\begin{displaymath}
f={{N_1} \over
  {\displaystyle D_1 + {{\strut N_2} \over
                {\displaystyle D_2 + \ddots }}}}
\end{displaymath}

One way to approximate an irrational number is to expand as a
continued fraction, and truncate the expansion after a sufficient
number of terms.  Such a truncation---a so-called {\em $k$-term finite
continued fraction}---has the form
\begin{displaymath}
{{N_1} \over {\displaystyle D_1+
        {\strut {N_2} \over {\displaystyle \ddots +
        {\strut {N_K} \over {\displaystyle D_K+0}}}}}}
\end{displaymath}

For example, if the $N_i$ and the $D_i$ are all 1, it is not
hard to show that the infinite continued fraction expansion
\begin{displaymath}
{1 \over {\displaystyle 1 +
        {\strut 1 \over {\displaystyle 1 +\ddots } }}}
\end{displaymath}
converges to $1/\phi\approx .618$ where $\phi$ is the {\em golden ratio}
\begin{displaymath}
\frac{1+ \sqrt{5}}{2}
\end{displaymath}
The first few finite continued fraction approximations (also called
{\em convergents})  are:
\begin{displaymath}
1, \frac{1}{2}=.5, \frac{2}{3}\approx .667, \frac{3}{5}=.6,
\frac{5}{8}=.625, \cdots
\end{displaymath}

\paragraph{Pre-Lab Exercise 2.1}
Suppose that {\tt n} and {\tt d} are procedures
of one argument (the term index) that return the
$n_i$ and $d_i$ of the terms of the continued fraction.

Define procedures {\tt cont-frac-r} and {\tt
cont-frac-i} such that evaluating {\tt (cont-frac-r n d k)} and {\tt
(cont-frac-i n d k)} each compute the value of the $k$-term finite
continued fraction.  The process that results from applying {\tt
cont-frac-r} should be recursive, and the process that results from
applying {\tt cont-frac-i} should be iterative.  Check your procedures
by approximating $1/\phi$ using

\beginlisp
(cont-frac (lambda (i) 1)
           (lambda (i) 1)
           k)
\endlisp

for successive values of {\tt k}, where {\tt cont-frac} is {\tt
cont-frac-r} or {\tt cont-frac-i}.  How large must you make {\tt k}
in order to get an approximation that is accurate to 4 decimal places?
Turn in a listing of your procedures.

\paragraph{Lab Exercise 2.2}
One of the first formulas for $\pi$ was given around 1658 by the
English mathematician Lord Brouncker:

\begin{displaymath}
4/\pi -1  =  1/(2 + 9/(2 + 25/ (2 + 49/ (2 + 81 /(2 + \cdots
\end{displaymath}

This leads to the following procedure for approximating $\pi$:

\beginlisp
(define (estimate-pi k)
  (/ 4 (+ (brouncker k) 1)))
\endlisp

\beginlisp
(define (square x) (* x x))
\null
(define (brouncker k)
  (cont-frac (lambda (i) (square (- (* 2 i) 1)))
             (lambda (i) 2)
             k))
\endlisp

where {\tt cont-frac} is one of your procedures {\tt cont-frac-r} or
{\tt cont-frac-i}.  Use this method to generate approximations to $\pi$.
About how large must you take $k$ in order to get 2 decimal places
accuracy?  (Don't try for more places unless you are very patient.)

\subsection{Computing inverse tangents}

Continued fractions are frequently encountered when approximating
functions.  In this case, the $N_i$ and $D_i$ can themselves be
constants or functions of one or more variables.  For example, a
continued fraction representation of the inverse tangent function
was published in 1770 by the German mathematician J.H. Lambert:

\begin{displaymath}
{\tan^{-1}x} ={x \over
{\displaystyle 1+{\strut {\left( {1x} \right)^2} \over
{\displaystyle 3+{\strut {\left( {2x} \right)^2} \over
{\displaystyle 5+{\strut {\left( {3x} \right)^2} \over {7+\ddots }}}}}}}}
\end{displaymath}

\paragraph{Pre-Lab Exercise 2.3}
Define a procedure {\tt (atan-cf k x)} that computes an approximation
to the inverse tangent function based on a $k$-term continued fraction
representation of $\tan^{-1} x$ given above. Provided you use the
correct arguments, it should be possible to
define this procedure directly in terms of the {\tt cont-frac}
procedure you have already defined.

\paragraph{Lab Exercise 2.4}
Verify that your
procedure works (and check its accuracy) by using it to compute the
tangent of a number of arguments, such as $0$,
$1$, $3$, $10$, $30$, $100$.
Compare your results with those computed using
Scheme's {\tt atan} procedure\footnote{Evaluating {\tt (atan x)}
returns the same value as {\tt (atan x 1)}, an angle in the range
$(-\pi/2, +\pi/2)$.}.

How many terms are required to get
reasonable accuracy?  Turn in your procedure definition
together with some sample results.

The idea of a continued fraction can be generalized to include
arbitrary nested expressions such as (1) and (2) below:

\begin{equation}
%MathType!ZZhx46!633xZh9VgeZpaeVeH9Kppzqiq9taFeGa2-Y8WaXhsyprZBeecuFmWh
%bif9Y+se2t+8KbHa1pc8racy-lZddeFiH9enVrqiq9YaFeGu0lRu03
%Nu0l7lryp5ZtgeVeH92cFeGa2-Y8WaXhsyprZBeeFiH9RbFeGu0lRa
%1paaaaaaa8!2AB6!
\left( {N_1/\left( {D_1+N_2/\left( {D_2+\cdots +N_k/\left( {D_k+0}
\right)} \right)} \right)} \right)
\end{equation}
\begin{equation}
%MathType!ZZhx46!633xZh9VgeZpaeVeH9Kppzqiq9taFeGu0VYhsyprZBeecuFmWhbif9
%ln-aq8se2t+8KbHa1pc8rasr-kFiH9enVrqiq9YaFeGu0VKu03Nu0V
%08daXlryp5ZtgeVeH92cFeGu0VYhsyprZBeeFiH9RbFeGu0VKa1pba
%aaaap!2B00!
\left( {N_1+D_1\times \left( {N_2+D_2\times \cdots \times \left(
{N_k+D_k\times 1} \right)} \right)} \right)
\end{equation}
or in general
\begin{displaymath}
\left( T_1 {\cal O}_1 \left( T_2 {\cal O}_2 \cdots
            \left( T_k {\cal O}_k R \right) \right) \right)
\end{displaymath}
where the $T_i$ are the terms of the expression,
the ${\cal O}_i$ are the arithmetic operators, and $R$ is the
residual term (perhaps representing the effects of
terms beyond the $k$th in an approximation). For example,
in (2) the ${\cal O}_i$ are alternately the addition and the
multiplication operators, and $R=1$.

\paragraph{Pre-Lab Exercise 2.5}
Define a procedure {\tt nested-acc} such that evaluating {\tt
(nested-acc op r term k)} computes the value of a nested expression.
Argument {\tt op} is a procedure of one argument, that when applied to
the term index $i$ returns the $i$th arithmetic operation ${\cal O}_i$
(which is a procedure of two arguments).  The {\tt r} argument
represents the effect of the $R$ term.  Turn in a listing of your
procedure.

How would you use your {\tt nested-acc} procedure to
compute a $k$-term approximation to the function
\begin{displaymath}
%MathType!ZZhx46!633xZh9VgeZtbecu-esrVSZtbecu-esrVSZtbecu-esrVSsrFFWhba
%Fea8raaa!0C5C!
f(x)=\sqrt {x+\sqrt {x+\sqrt {x+\cdots }}}
\end{displaymath}

\paragraph{Lab Exercise 2.6}

Verify that your {\tt nested-acc} procedure works properly
by computing an approximation to $f(1)$, whose value is
the golden ratio. How large must you make {\tt k}
in order to get an approximation that is accurate to 4 decimal places?
Turn in a listing of your procedures.

\paragraph{Lab Exercise 2.7}
Verify that your {\tt nested-acc} procedure works
properly by using it to compute the value of the Taylor series for the
sine function discussed in Problem Set 1.

In certain continued fractions all $N_i$ and $D_i$ terms are equal
and it is convenient to regard the fraction as
being ``built-up'' from a base by a repetitive operation. For
example, given the base function $B$ and the augmentors
$N$ and $D$, one can build $N/(D+B)$. This can be computed in a
straightforward fashion by the procedure {\tt build}

\beginlisp
(define (build n d b)
  (/ n (+ d b)))
\endlisp

The value of the expression that is represented by the two-term
continued fraction

\begin{displaymath}
%MathType!ZZhx46!633xZh9VgeZtdeVeH9KdXlryprsrVSZtdeVeH9KdXlryprsrVSVeH9
%Kaaaa8!1036!
{N \over {D+{N \over {D+B}}}}
\end{displaymath}

can then be computed by evaluating {\tt (build n d (build n d b))}.
When further composition is of interest, it is possible to use the
{\tt repeated} procedure defined above.

\paragraph{ Pre-Lab Exercise 2.8 }
Write a procedure {\tt repeated-build} of four arguments {\tt k},
{\tt n}, {\tt d}, and {\tt b}, which returns a result
that is equivalent to applying the {\tt build} procedure $k$ times. In
particular {\tt (repeated-build 2 n d b)} should return the same result
as  {\tt  (build  n  d  (build  n  d  b))}.  Your  implementation  of  {\tt
repeated-build} should make use of the {\tt repeated} procedure given above
(in a non-trivial way).

\paragraph{ Lab Exercise 2.9 }
Verify that your {\tt repeated-build} procedure works properly by evaluating
the continued fraction for the reciprocal of the golden ratio:

\begin{displaymath}
{1 \over {\displaystyle 1 +
        {\strut 1 \over {\displaystyle 1 + \ddots } }}}
\end{displaymath}
converges to $1/\phi\approx .618$ where $\phi$ is the {\em golden ratio}

Now consider the problem of computing the rational functions
$r_1(x)=(1+0x)/(1+x)$, $r_2(x)=(1+x)/(2+x)$, $r_3(x)=(2+x)/(3+2x)$,
\begin{displaymath}
%MathType!ZZhx46!633xZh9VgeVeH9Nmpzq8se2Bl8raZddeFiH94bGu03Z8dbXhsyFyZB
%eeFiH9RbFeGu0l7lryVfZtgeVeH92kf9slq9taFeWhsypEq8He2h28
%gbXhsy-Asr-kcuFmWhbif9Y+se2BX8KbXlryVTWhb8He2JhasrFpZp
%eecuFmqiq9Xif9R8He2l38gbXhsy-AsrFlcuFmWhbm-aq8se2Fiaaa
%Wd!3C40!
r_k\left( x \right)={{a_k+a_{k-1}x} \over {a_{k+1}+a_kx}}={1 \over
{1+r_{k-1}\left( x \right)}}
\end{displaymath}
where $a_k$ is the $k$-th Fibonacci number ($a_0=0$).

\paragraph{ Lab Exercise 2.10 }
Making  use of your {\tt repeated-build} procedure, define a procedure
{\tt r} of one argument {\tt k} that evaluates to a procedure of
one argument {\tt x} that computes $r_k(x)$. For example,
evaluating {\tt ((r 2) 0)} should return $0.5$.

\subsection{Optional Problem}

One difficulty with using continued fractions to evaluate functions
is that there is generally no way to determine in advance how many
terms of an infinite continued fraction are required to achieve a
specified degree of accuracy. The result obtained by computing the
value of a continued fraction by working backwards from the $k$-th to
the first term cannot generally be used in the computation of the
$k+1$ term continued fraction.

Remarkably, there is a general technique for computing the value of a
continued fraction that overcomes this problem. This result is based
on the recurrence formulas
\begin{equation}
A_{-1}=1;B_{-1}=0;A_0=0;B_0=1
\end{equation}
\begin{equation}
%MathType!ZZhx46!633xZh9VgeVeH9umpzq8se2Bk8rasrFpFiH9enVrq8He2Rg8raVeH9
%umpzq8se2BQu0lTa1pb8rasr-kFiH9onVrq8He2Rg8raVeH9umpzq8
%se2BQu0lTa1pc8raaa!212C!
A_j=D_jA_{j-1}+N_jA_{j-2}
\end{equation}
\begin{equation}
%MathType!ZZhx46!633xZh9VgeVeH9Kmpzq8se2Bk8rasrFpFiH9enVrq8He2Rg8raVeH9
%Kmpzq8se2BQu0lTa1pb8rasr-kFiH9onVrq8He2Rg8raVeH9Kmpzq8
%se2BQu0lTa1pc8raaa!212F!
B_j=D_jB_{j-1}+N_jB_{j-2}.
\end{equation}
It is easy to show that the $k$-term continued fraction can be computed as
\begin{equation}
%MathType!ZZhx46!633xZh9VgeVeH9Mnpzq8se2Bl8rasrFpZpeeFiH9bnVrq8He2Vg8ra
%q8se2tY8KbXlryVTWhbaaa!1500!
f_k={{A_k} \over {B_k}}.
\end{equation}
By determining whether $f_j$ is close enough to $f_{j-1}$, it is easy
to determine whether to include more terms in the approximation.

Define a procedure that computes the value of a continued fraction
using this algorithm. The process that evolves when the procedure is
applied should be iterative. Test that your procedure works by
computing the tangent of various angles. Use the tracing feature of
SCHEME to determine how many terms are evaluated.

\vspace{10ex}

Turn in this sheet with your answers to the questions in the problem set.

\vspace{6ex}

How much time did you spend on this homework assignment? Report separately
time spent before going to lab and time spent in the 6.001 lab.

\vspace{6ex}

If you cooperated with other students working this problem set
please indicate their names in the space below. As you should know,
as long as the guidelines described in the
{\em 6.001 Policy on Cooperation}\ handout are followed,
such cooperation is allowed and encouraged.

\end{document}


 \documentclass [12pt]{article}

\usepackage {amsmath}
\usepackage {amsthm}
\usepackage {amssymb}
\usepackage {graphicx}
\usepackage {float}
\usepackage {multirow}
\usepackage {xcolor}
\usepackage {algorithmic}
\usepackage [ruled,vlined,commentsnumbered,titlenotnumbered]{algorithm2e} \usepackage {array}
\usepackage {booktabs}
\usepackage {url}
\usepackage {parskip}
\usepackage [margin=1in]{geometry}
\usepackage [T1]{fontenc}
\usepackage {cmbright}
\usepackage [many]{tcolorbox}
\usepackage [colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage {enumitem}
\usepackage {xparse}
\usepackage {verbatim}
\usepackage{listings}
\usepackage{xcolor}
\lstset { %
    language=C++,
    backgroundcolor=\color{black!5}, % set backgroundcolor
    basicstyle=\footnotesize,% basic font setting
}
\newtheorem{theorem}{Theorem}
\newtheorem{remark}{Remark}

\DeclareTColorBox {Solution}{}{breakable, title={Solution}} \DeclareTColorBox {Solution*}{}{breakable, title={Solution (provided)}} \DeclareTColorBox {Instruction}{}{boxrule=0pt, boxsep=0pt, left=0.5em, right=0.5em, top=0.5em, bottom=0.5em, arc=0pt, toprule=1pt, bottomrule=1pt} \DeclareDocumentCommand {\Expecting }{+m}{\textbf {[We are expecting:} #1\textbf {]}} \DeclareDocumentCommand {\Points }{m}{\textbf {(#1 pt.)}}

\begin {document}

\vspace {1em}
\begin {Instruction}
Adapted From Virginia Williams' lecture notes.
\end {Instruction}

{\LARGE \textbf {COMP 285 (NC A\&T, Spr `22)}\hfill \textbf {Lecture 10} }

\begin{centering}
\section*{QuickSort and Non-Comparison Sorts}
\end{centering}

Today we will complete our analysis of \texttt{Quicksort}, which is the last comparison-based sort we will cover in class.

Then, we will jump into learning more about non-comparison-based sorts, such as \texttt{CountingSort} and \texttt{RadixSort}.

\section{QuickSort Analysis}
\subsection{Worst-Case Analysis}
In this section we will derive a bound on the worst-case running time of \texttt{Quicksort}. If we
consider the worst random choice of pivot at each step, the running time will be $\Theta(n^2)$. This flavor of worst-case analysis (which gets an upper bound on the running time over all possible possible choices of pivots) is no different from the worst case analysis of the algorithm which picks an arbitrary pivot at every step. We are thus interested in what the running time of \texttt{Quicksort} is \textbf{on average over all possible choices of the pivots}. We should emphasize an important point: We still consider the running time for a \textbf{worst-case input}, and average only \textbf{over the random choices of the algorithm} (which is different from averaging over all possible inputs). Put differently, our analysis will guarantee that \textbf{for any input, the expected running time will be small}. We formalize the idea of averaging over the random choices of the algorithm by considering the running time of the algorithm on an input I as a random variable and bounding the expectation of that random variable.

\textbf{Proposition}. \textit{For every input array of size $n$, the expected running time of \texttt{QuickSort} is$O(n \log n)$}.

Recall that a random variable is a function that maps every element in the sample space to a real number. In the case of rolling a die, the real number (or value of the point in the sample space) would be the number on the top of the die. Here the sample space is the set of all possible choices of pivots, and an example for a random variable can be the running time of \texttt{Quicksort} on a specific input $I$.

Denote by $z_i$ the $i$-th element in the sorted array. For each $i, j$, we define a random variable $X_{i ,j} (\sigma)$ to be the number of times $z_i$ and $z_j$ are compared for a given series of pivot choices $\sigma$. What are the possible values for $X_{i ,j} (\sigma)$? It can be $0$ if $z_i$ and $z_j$ are not compared. Note that all comparisons are with the pivot, and that the pivot is not included in the elements of the arrays in the recursive calls. If $z_i$ and $z_j$ are compared, consider the first time that this happens: one of them must be the pivot at this stage and this pivot is excluded from the subarrays that recursive calls operate on. Thus, \textit{no two elements are compared twice}. Therefore, $X_{i ,j} (\sigma) \in \{0, 1\}$.

Our goal is to compute the expected number of comparisons that \texttt{Quicksort} makes. Recall the definition of expectation:

$$
\mathbb{E}[X] = \sum_{\sigma} \mathbb{P}[\sigma]X(\sigma) = \sum_{k} k\mathbb{P}[X = k]
$$

Unfortunately for us, this definition does not really give us any clue about how we can actually go about computing the expectation of the complicated random variable that we have at hand, i.e. the number of comparisons made by \texttt{Quicksort}. Can you imagine trying to figure out the probability that \texttt{Quicksort} actually performs exactly $k$ comparisons? We don’t have the foggiest idea how we might do that. If you do, please let us know!

Luckily for us, there is way around this. An important property of expectation is \textbf{linearity of expectation}. For any random variables $X_1, \cdots, X_n$:
$$
\mathbb{E}\left[\sum_{i=1}^n X_i \right] = \sum_{i=1}^n \mathbb{E}[X_i]
$$

This is a really simple to state, but amazingly useful property that makes the computation of expectations of seemingly complicated random variables much much easier. It is worth internalizing this technique that we are about to apply for analyzing the expected number of comparisons that \texttt{Quicksort} performs.

We start with computing the expected value of $X_{i ,j}$. These variables are indicator random variables, which take the value $1$ if some event happens, and $0$ otherwise. The expected value is:
\begin{align*}
\mathbb{E}[X_{i,j}] &= \mathbb{P}[X_{i,j} = 1] \cdot 1 + \mathbb{P}[X_{i,j} = 0] \cdot 0 \tag{definition of expectation} \\
&= \mathbb{P}[X_{i,j} = 1]
\end{align*}

Let $C(\sigma)$ be the total number of comparisons made by \texttt{Quicksort} for a given set of pivot choices $\sigma$:

$$
C(\sigma) = \sum_{i=1}^n \sum_{j=i+1}^n X_{i,j}(\sigma)
$$

We wish to compute $\mathbb{E}[C]$ to get the expected number of comparisons made by \texttt{Quicksort}
for an input array of size $n$.

\begin{align*}
  \mathbb{E}[C] &= \mathbb{E}\left[\sum_{i=1}^n \sum_{j=i+1}^n X_{i,j}(\sigma) \right] \tag{definition of $C$} \\
  &= \sum_{i=1}^n \mathbb{E}\left[\sum_{j=i+1}^n X_{i,j}(\sigma) \right] \tag{linearity of expectation on first sum} \\
  &= \sum_{i=1}^n \sum_{j=i+1}^n  \mathbb{E}[X_{i,j}(\sigma)] \tag{linearity of expectation on second sum} \\
  &= \sum_{i=1}^n \sum_{j=i+1}^n  \mathbb{P}[z_i, z_j \text{ are compared} ]  \tag{definition of expectation on indicator variable}\\
\end{align*}
Now we find $P[z_i, z_j \text{ are compared}]$. Apriori, this seems difficult to do, but it turns out that there is a really elegant way to analyze this. Note that each element in the array(except the pivot) is compared to the pivot at each level of the recurrence. To analyze $P[z_i, z_j \text{ are compared}]$, examine the portion of the array $[z_i, \cdots, z_j]$. After the array is split using a pivot from $[z_i, \cdots, z_j]$, $z_i$ and $z_j$ can no longer be compared. Hence, $z_i$ and $z_j$ are compared only when from the portion of the array $[z_i, \cdots, z_j]$, either $z_i$ or $z_j$ is the first one picked as the pivot. So,

\begin{align*}
\mathbb{P}[z_i, z_j \text{ are compared}] &= \mathbb{P}[z_i \text{ or } z_j \text{ is the first pivot picked from } [z_i, \cdots, z_j]] \tag{argued above} \\
&= \frac{1}{j-i+1} + \frac{1}{j - i + 1} \tag{picking from a set uniformly} \\
&= \frac{2}{j-i+1}
\end{align*}

Make sure you are able to explain the second line in the calculation above.

We return to the expected value of $C$:

\begin{align*}
\mathbb{E}[C] &= \sum_{i=1}^n \sum_{j = i + 1}^n \mathbb{P}[z_i, z_j \text{ are compared }] \\
&= \sum_{i=1}^n \sum_{j = i + 1}^n \frac{2}{j-i + 1}
\end{align*}
Note that for a fixed $i$,
\begin{align*}
  \sum_{j=i+1}^n \frac{1}{j-i+1} &= \frac{1}{2} + \frac{1}{3} + \cdots + \frac{1}{n - i + 1} \\
  &\leq \frac{1}{2} + \frac{1}{3} + \cdots + \frac{1}{n}
\end{align*}

And using $\sum_{k=2}^n \frac{1}{k} \leq \ln n$, we get that

\begin{align*}
\mathbb{E}[C] &= \mathbb{E}\left[\sum_{i=1}^n \sum_{j = i + 1}^n X_{i,j}(\sigma) \right] \\
&= \sum_{i=1}^n \sum_{j = i + 1}^n \frac{2}{j-i + 1}\\
&\leq 2n \ln n
\end{align*}

 Thus, the expected number of comparisons made by \texttt{Quicksort} is no greater than $2n \ln n = O(n \log n)$. To complete the proof, we have to show that the running time is dominated by the number of comparisons. Note that in each recursive call to \texttt{Quicksort} on an array of size $k$, the algorithm performs $k - 1$ comparisons in order to split the array, and the amount of work done is $O(k)$. In addition, \texttt{Quicksort} will be called on single-element arrays at most once for each element in the original array, so the total running time of \texttt{Quicksort} $is O(C + n)$. In conclusion, the expected running time of \texttt{Quicksort} on worst-case input is $O(n log n)$.

 \subsection{Alternative Proof}

Here we provide an alternative method for bounding the expected number of comparisons. Let $T(n)$ be the expected number of comparisons performed by \texttt{Quicksort} on an input of size $n$. In general, if the pivot is chosen to be the $i$-th order statistic of the input array (i.e. the $i$th largest element),

$$
T(n) = n - 1 + T(i - 1) + T(n - i)
$$

where we define $T(0) = 0$. Each of the $n$ possible choices of $i$ are equally likely. Thus, the
expected number of comparisons is:

\begin{align*}
T(n) &= n - 1 + \frac{1}{n} \sum_{i=1}^n \left(T(i - 1) + T(n - i)\right) \\
&= n - 1 + \frac{2}{n} \sum_{i=1}^{n-1} T(i) \tag{Each $T(i)$ shows up twice in the sum}
\end{align*}

Did you catch this minor detail: why does the second summation go from $1$ through $n - 1$, while on the previous line, the summation goes from $1$ through $n$?

Continuing the calculation, we will use two facts:

\begin{enumerate}
  \item $\sum_{i=1}^{n-1} f(i) \leq \int_1^n f(x) dx$ for an increasing function $f$.
  (How would you prove this? This gives us a way to upper-bound discrete sums by integrals of continuous functions – a nice trick to have in your toolkit that we will use below. If you understand this, what is the corresponding statement to lower-bound a discrete sum by an integral? What if $f$ was a decreasing function of $n$? It is good to get into the habit of asking yourself such questions to test your understanding.)
  \item $\int 2x \ln x dx = x^2 \ln x - \frac{x^2}{2} + C$
\end{enumerate}

Now we show that $T(n) \leq 2n \ln n$ by (strong) induction.

\textbf{Inductive Hypothesis.} $T(i) \leq 2i \ln i$.

\textbf{Base case (i = 1)} An array of size 1 requires no comparisons. Thus, $T(1) = 0$ and the inductive hypothis is true for $i = 1$.

\textbf{Inductive step}. We will show that if the inductive hypothesis is true for all $i \leq k - 1$ then
the inductive hypothesis is also true for $i = k$.

Let's bound $T(k)$

\begin{align*}
T(k) &= k - 1 + \frac{2}{k}\sum_{i=1}^{k-1}T(i) \tag{by definition} \\
&\leq k - 1 + \frac{2}{k}\sum_{i=1}^{k-1}2i \ln i \tag{using our strong inductive hypothesis} \\
&\leq k - 1 + \frac{2}{k}\int_{1}^k (2x \ln x) dx \tag{Using fact 1 above} \\
&= k - 1 + \frac{2}{k}\left[k^2 \ln k - \frac{k^2}{2} + \frac{1}{2} \right] \tag{Using fact 2 above} \\
&= 2k \ln k + k - 1 - k + \frac{1}{k} \tag{simplify} \\
&= 2k \ln k - 1 + \frac{1}{k} \\
&\leq 2k \ln k
\end{align*}

Thus the inductive hypothesis is true for $i = k$. This establishes the inductive step.

\textbf{Conclusion} By induction, we have proved that $T(i) \leq 2i \ln i$ for all $i$. Hence $T(n) \leq
2n \ln n$. This concludes the proof.

\section{Lower Bounds for Comparison-Based Sorting Algorithms}

This is not covered in this course, however, its an interesting exercise to follow. If you'd like to learn more as to why the fastest possible comparison-based sorting algorithm must be $\Omega(n \log n)$ (eg, must take at least $n \log n$ steps in the worst-case), see these \href{notes}{http://www.cs.cmu.edu/~avrim/451f11/lectures/lect0913.pdf}.

\section{Counting Sort}
Recall that our sorting lower bounds applied to the class of algorithms that can only evaluate the values being sorted via \textit{comparison queries}, namely via asking whether a given element is greater than, less than, or equal to some other element. For such algorithms, it's been proven that any correct algorithm (even a randomized one!) will require $\Omega(n \log n)$ such queries on some input. As with any lower-bound that we prove in a restricted model, it is fruitful to ask ``is it possible to have an algorithm that does not fall in this class, and hence is not subject to the lowerbound?'' For sorting, the answer is ``yes''.

We start by looking at a very simple non-comparison-based sorting algorithm called Counting Sort. Since it is not comparison-based, it is not restricted by the $\Omega(n \log n)$ lower bound for sorting. For a given input of $n$ objects, each with a corresponding key (or value) in the range $\{0, 1, \cdots, r - 1\}$, Counting Sort will sort the objects by their keys:

\begin{enumerate}
  \item Create an array $A$ of $r$ buckets where each bucket contains a linked list.
  \item For each element in the input array with key $k$, concatenate the element to the end of the linked list $A[k]$.
  \item Concatenate all the linked lists: $A[0], \cdots , A[r - 1]$.
\end{enumerate}

The algorithm correctly sorts the $n$ elements by their keys because the elements are placed into buckets by key where bucket $i$ (containing elements with key = $i$) will come before bucket $j$ (containing elements with key = $j$) in A for $i < j$. Therefore, when the algorithm concatenates the buckets, all elements with key = $i$ will come before elements with key = $j$. The worst case run time of Counting sort is $O(n+r)$ since it performs $O(1)$ passes over the $n$ input elements and $O(1)$ passes over the $r$ buckets of $A$. This is great if the size of the range $r$ is small, but we pay dearly in the running time (and space) if $r$ is large. The algorithm we discuss in the next section builds on Counting Sort and fixes this issue (that Counting Sort
behaves poorly if the range of values $r$ is very large).

An important property (which we will use in the next section) is that the algorithm described above is \textit{stable}: If two input elements $x, y$ have the same key, and $x$ appears before $y$ in the input array, $x$ will appear before $y$ in the output.

\end{document}


\documentclass[11pt, oneside]{article}

\usepackage{../../shared/preamble}
\addbibresource{../../shared/references.bib}

\usepackage{../sets/sets}
\usepackage{../topological-spaces/topological-spaces}
\usepackage{../real-numbers/real-numbers}
\usepackage{formal-specifications}

\title{The Role of Formalization and \Zed\ }
\author{Arthur Ryman, {\tt arthur.ryman@gmail.com}}
\date{\today}

\begin{document}

\maketitle

\begin{abstract}
The notion of {\it type} was introduced into set theory by Russel in an effort to establish a firm foundation for logic.
Types are now a standard feature of many computer programming languages.
\Zed\  is a formal language based on typed set theory for specifying computer programs.
Since \Zed\  is a formal language, a Z specification can itself be automatically checked for type errors.
Writing valid Z specifications requires that the author explicitly define all terms.
This encourages a style in which complex terms are gradually built up from simpler, previously defined terms.
This article proposes to use \Zed\  to formally specify mathematical structures, even if there is no intention of using
them in computer programs.
The hoped for benefits of doing so are that the exercise of formalization will lead the author to a better understanding of the subject
matter, resulting in a clearer, higher quality, result.
\end{abstract}

\section{Types}

Computer languages can be divided into two major groups, namely those that are strongly typed and those that are weakly typed.
Strongly-typed languages, such as Java, have complex type systems and require that all expressions and variable declarations
have well-defined types.
Weakly-typed languages, such as Javascript, have simpler type systems and allow greater flexibility in expressions and the use of variables.
Although no compiler can possibly tell you if your program is correct, a compiler for a strongly-typed language can at least detect type errors.
Correcting these errors at compile-time is often quicker than doing so at execution-time.

There are many similarities between mathematics and programming.
Both require the definition and use of named objects that are built up from simpler objects.
The concept of type applies to both.
Bertrand Russell discovered typed set theory in his efforts to solve some problems in the foundations of mathematics
long before computers were a reality.
Theoretical computer science now makes extensive use of types.
Mathematics can be used to specify the requirements for programs.
Specifications act as a bridge between mathematics and programs.
\Zed\  is a strongly-typed formal specification language.
However, \Zed\  can be used to formalize mathematics even if the goal is not to program a computer.

\section{Formalization}

\Zed\  can be used to precisely define mathematical objects and a Z specification can be type-checked by a program.
I have found that the exercise of expressing mathematics using \Zed\  is a great way make sure that concepts are well-defined.
This is especially helpful, although somewhat labour-intensive, when dealing with complex mathematical structures.
I propose to formalize all definitions using \Zed\  and to validate
the document using the \fuzz\ type checker.

I believe that the effort of precisely defining every needed concept pays off.
Consider how C\'{e}dric Villani \cite{villani-hbm} described the mathematical writing style of Alexander Grothendieck:
\begin{displayquote}
Let me quote a famous mathematical text by Alexander Grothendieck,
one of the most famous mathematicians of all times, who wrote long writings.
In one of them he talks about the metaphor \textit{de la noix}.
You know the parable, the metaphor, of the nut
to explain the difference between his style and
the style of his fellow mathematician Jean-Pierre Serre.
Both of them working in the same area of mathematics, but very different styles.
Grothendieck said,
\textquote{Imagine that we have a nut to open.
The style of Serre would be take a
hammer and, Bang!, smash the nut.
My style would be to take the nut and put
it in a sea of acid so that it would be
dissolved very slowly, the crust of the nut, without noticing anything.}
And yes, experts tell us that the style of Grothendieck is like everything is very
incremental from one step to another to
another by very tiny steps so that you
have the impression that nothing occurs
and we are really making no progress, and
at the end it's proven, there it is, the big theorem is is done.
\end{displayquote}
Similarly, Pierre Deligne  \cite{artin-ag1} said:
\begin{displayquote}
From Grothendieck I have also learned not to take glory in the difficulty of a proof:
difficulty means we have not understood. The ideal is to be able to paint a landscape in which the proof is obvious.
I admire how often he succeeded in reaching this ideal.
\end{displayquote}
The message is clear: a slow, methodical build-up of concepts is a good thing in mathematics.

\section{Given Sets and Signatures}

Although the integers are built-in to \Zed, the real numbers are not.
In principle, one could first build up the rational numbers from the integers,
and then the real numbers from the rational numbers, but that would take a lot of time and not really
accomplish much.
I'll assume that the real numbers are sufficiently well-understood and do not need a complete formalization.
Instead, I'll define real numbers as a given set and then declare the types and signatures of the usual constants and operations of real arithmetic.

\section{Names and Symbols}

Integers and reals are distinct types in the sense of typed set theory.
But distinct types are disjoint sets.
This implies that the $0$ element of the integers is not the same object as the $0$ element of the reals.
In fact, they aren't even comparable within a Z specification since doing so is a type error and the
specification would therefore not be valid.
The name \texttt{0} is built-in to \Zed\  as the name of the $0$ element of the integers.
A Z specification must therefore introduce a new name to refer to the $0$ element of the reals.
However, working mathematicians regard the integers as a subset of the reals and therefore have no need to deal with duplicate names.
As Henri Poincar\'{e} \cite{poincare-fm} said:
\begin{displayquote}
I think I have already said somewhere that mathematics is the art of giving the same name to different things.
It is enough that these things, though differing in matter, should be similar in form, to permit of their being, so to speak, run in the same mould.\end{displayquote}

How then can we make a Z specification valid while still honouring normal mathematical practice?
The way out of this difficulty is that if a mathematician would normally regard two distinct objects as the same
then we give then distinct \Zed\  names, but display them using the same typographic symbol.
To paraphrase Poincar\'{e}:
\begin{quote}
Formalizing mathematics is the art of giving different names, but the same symbol, to different things.
It is enough that these things, though differing in type, should be similar in typography,
to permit of their being, so to speak, run in the same mould.
\end{quote}

For example, Mike Spivey \cite{spivey-fm} describes three typographic symbols defined in the \fuzz\ package
that are each used to display two distinct objects:
\begin{displayquote}
A few symbols have two names, reflecting two different uses for the
symbol in Z:
\begin{itemize}
\item The symbol $\semi$ is called
\cmd{semi} when it is used as an operation on schemas, and
\cmd{comp} when it is used for composition of relations.
\item The symbol $\hide$ is called
\cmd{hide} as the hiding operator of the schema calculus, and
\cmd{setminus} as the set difference operator.
\item The symbol $\project$ is called
\cmd{project} as the schema projection operator, and
\cmd{filter} as the filtering operator on sequences.
\end{itemize}
Although the printed appearance of
each of these pairs of symbols is the same, the type checker
recognizes each member of the pair only in the appropriate
\hbox{context}.
\end{displayquote}
This approach keeps both the Z type checker and the mathematician reader happy.
For example, in my formalization of the reals:
\begin{itemize}
\item The symbol $0$ is called
\texttt{0} when it is used as an integer, and
\cmd{zeroR} as a real number.
\item The symbol $+$ is called
\texttt{+} as the addition operator on the integers, and
\cmd{addR} as the addition operator on the reals.
\end{itemize}
The mathematician reader will always be able to infer the actual type of an object from its context.

\section{Theorems, Examples, and Remarks}

In mathematical writing it is good practice to follow a definition with some examples.
When an example is given, there is a proof obligation associated with it,
namely that whatever the author is asserting about the example is in fact true.
Similarly, when the author makes a remark or states a theorem, there is also a proof obligation.
In each of these cases, the author makes a statement and asserts it to be true, but may not offer any proof.

\Zed\ allows the author to add constraints to a specification.
However, if the constraint is a logical consequence of the preceding definitions then it adds nothing to the specification.
Therefore, we can freely add examples, remarks, and theorems to a specification without changing its meaning.
These statements should be placed in a \cmd{zed} box that is enclosed in an appropriate \LaTeX\ theorem-like environment.

For example, the following statement is true and therefore does not change the meaning of
the specification but will be type-checked by \fuzz, .

\begin{example}

\begin{zed}
	1 + 1 = 2
\end{zed}

\end{example}

The benefit of placing theorem-like statements in \cmd{zed} boxes is that the \fuzz\ type checker will check their type-correctness.

\subsection{Generic Theorem-Like Statements}

It will often be the case that a theorem-like statement will involve generic constructs.
Although \Zed\ provides a mechanism for introducing formal generic parameters into the definition of
generic constants and schemas, it does not provide such a mechanism for constraints.
We can work around this limitation by declaring a set of global given sets which are understood to be used in theorem-like
statements where they are taken to represent arbitrary sets.
The article on topological spaces introduces the global given sets $\setX$, $\setY$, and $\setZ$ for this purpose.

For example, we can remark that every element $x$ of an arbitrary set $\setX$ is a member of that set.

\begin{remark}

\begin{zed}
	\forall x: \setX @ x \in \setX
\end{zed}

\end{remark}

\section{Simulating Standard Mathematical Notation}

The expression $(a,b)$ usually denotes an ordered pair in mathematical writing.
Sometimes an author asks us to interpret $(a,b)$ as an open interval of the real number line.
In this case we recognize that the symbols used in the expression $(a,b)$ do not have their usual meanings
and we interpret them accordingly.
The notation $(a,b)$ for denoting an open interval of the real number line is
arguably very compact, convenient, and easy to understand.
In contrast, the \Zed\  function application $\intervalR(a,b)$, defined in the article on real numbers,
may strike the reader as somewhat verbose and clumsy, albeit precise.

\subsection{Prefix, Infix, and Postfix Operators in \fuzz}

There is a way to achieve some of the compactness of standard mathematical notation in Z while
preserving its precision and type-correctness.
The approach is to take advantage of the \fuzz\ type checker's ability to define infix and postfix
operator symbols.
In the grammar used by \fuzz, postfix operators have higher precedence than prefix operators,
and prefix operators have higher precedence than infix operators.
Infix operators are assigned numerical priorities of 1 through 6 with higher priorities taking precedence
over lower ones.
We can use these relative precedences to define operators that allow us to construct expressions
that resemble usual mathematical writing.

\subsection{\zcmd{lowerBound}, \zcmd{upperBound}, and \zcmd{intersect}}

First, we use \LaTeX\ bold styling to provide a visual cue that helps the reader
distinguish the interval $\lowerBound a \intersect b \upperBound$ from the
usual ordered pair $(a,b)$.
Next, we interpret the symbols as prefix, infix, and postfix operators.
The symbol $\lowerBound$ is a prefix operator that maps $a$ to $\lowerBound a$,
the open interval bounded below by $a$.
Similarly, $\upperBound$ is a postfix operator that maps $b$ to $b \upperBound$,
the open interval bounded above by $b$.
Finally, we interpret the symbol $\intersect$ as an infix operator that forms the intersection of the two intervals.

\begin{axdef}
	\lowerBound: \R \fun \power \R \\
	\_ \upperBound: \R \fun \power \R \\
	\_ \intersect \_ : \power \R \cross \power \R \fun \power \R
\where
	\forall a: \R @ \lowerBound a = \{~ x: \R | a \ltR x ~\}
\also
	\forall b: \R @ b \upperBound = \{~ x: \R | x \ltR b ~\}
\also
	\forall a, b: \R @ \lowerBound a \intersect b \upperBound = \{~ x: \R | a \ltR x \ltR b ~\}
\end{axdef}

This design is a step in the right direction, but it doesn't prevent the writer from abusing the notation.
For example, the following paragraph looks odd but makes perfect sense to \fuzz.

\begin{zed}
	\forall a, b: \R @ \lowerBound a \intersect b \upperBound = b \upperBound \intersect \lowerBound a
\end{zed}

When we are asked to interpret $(a,b)$ as an interval, we are, in a sense, being asked to parse a sentence
in a new mathematical mini-language that defines intervals.
If we want to enforce the usual syntax for intervals,
we can introduce new types to represent fragments of the expression so that the operators can only be applied to
fragments in some prescribed order.

To illustrate this point, let's expand the example to include closed intervals $[a,b]$ as well as semi-closed intervals
$(a,b]$ and $[a,b)$.
Regard $(a$ and $[a$ as open and closed right half lines, and $b)$ and $b]$ as open and closed left half lines.
Only allow a right half line to be combined with a left half line.

\subsection{Right Half Lines: \zcmd{openLowerBound}, and \zcmd{closedLowerBound}}

Let $RightHalfLine$ denote the type of syntactic fragments that define open and closed right half-lines.

\begin{zed}
	RightHalfLine ::= \openLowerBound \ldata \R \rdata | \closedLowerBound \ldata \R \rdata
\end{zed}

\begin{example}

\begin{zed}
	\openLowerBound \zeroR \in RightHalfLine
\also
	\closedLowerBound \oneR \in RightHalfLine
\end{zed}

\end{example}

Let $rightSet(R)$ denote the set of real numbers that the right half-line syntactic fragment $R$ represents.

\begin{axdef}
	rightSet: RightHalfLine \fun \power \R
\where
	\forall a : \R @ rightSet(\openLowerBound a) = \{~ x: \R | a \ltR x ~\}
\also
	\forall a : \R @ rightSet(\closedLowerBound a) = \{~ x: \R | a \leR x ~\}
\end{axdef}

For example

\begin{zed}
	\zeroR \notin rightSet(\openLowerBound \zeroR)
\also
	\oneR \in rightSet(\closedLowerBound \oneR)
\end{zed}

\subsection{Left Half Lines: \zcmd{openUpperBound}, and \zcmd{closedUpperBound}}

Let $LeftHalfLine$ denote the type of syntactic fragments that define open and closed left half-lines.

\begin{zed}
	LeftHalfLine ::= (\_ \openUpperBound)  \ldata \R \rdata | (\_ \closedUpperBound)  \ldata \R \rdata
\end{zed}

For example

\begin{zed}
	\zeroR \openUpperBound \in LeftHalfLine
\also
	\oneR \closedUpperBound \in LeftHalfLine
\end{zed}

Let $leftSet(L)$ denote the set of real numbers that the left half-line syntactic fragment $L$ represents.

\begin{axdef}
	leftSet: LeftHalfLine \fun \power \R
\where
	\forall b : \R @ leftSet(b \openUpperBound) = \{~ x: \R | x \ltR b ~\}
\also
	\forall b : \R @ leftSet(b \closedUpperBound) = \{~ x: \R | x \leR b ~\}
\end{axdef}

For example

\begin{zed}
	\zeroR \notin leftSet(\zeroR \openUpperBound)
\also
	\oneR \in leftSet(\oneR \closedUpperBound)
\end{zed}

\subsection{\zcmd{intersectRightLeft}}

Let $R \intersectRightLeft L$ denote the set of real numbers in the intersection of the sets represented by
the half-line syntactic fragments $R$ and $L$.

\begin{axdef}
	\_ \intersectRightLeft \_ : RightHalfLine \cross LeftHalfLine \fun \power \R
\where
	\forall R: RightHalfLine; L: LeftHalfLine @ \\
	\t1	R \intersectRightLeft L = rightSet~R \cap leftSet~L
\end{axdef}

For example

\begin{zed}
	\zeroR \in \closedLowerBound \zeroR \intersectRightLeft \oneR \openUpperBound
\end{zed}

\subsection{Improvements}

After reading the above, my reaction is that it would be more readable to first define the relevant objects using meaningful names
and then define the symbols as abbreviations for them.

TODO: Fix up the above discussion.

TODO: Use the example environment for examples.

TODO: Improve the Summary. It should summarize the whole article, not just how to simulate mini-languages.

\subsection{Summary}

A lot of standard mathematical notation can be written directly using the built-in capabilities of \Zed\  and \fuzz.
However, mathematics contains many specialized notations including those for intervals, probability, and quantum mechanics (Dirac bra-ket notation).
One can view these specialized notations as mathematical mini-languages.
As demonstrated above for the case of intervals, it is possible to simulate these notations in Z by introducing new syntactic types along with infix and postfix operators that construct, combine, and reduce fragments of this syntax.
The general pattern is for the opening and closing symbols to correspond to prefix and postfix operators that
construct fragments, and for internal symbols to correspond to infix operators that combine and then finally
reduce fragments.
Reduction occurs when the final infix operator is applied and maps the completed mini-language sentence to a
some usual mathematical type.

\printbibliography

\end{document}


%%
%% Automatically generated file from DocOnce source
%% (https://github.com/hplgit/doconce/)
%%
%%
% #ifdef PTEX2TEX_EXPLANATION
%%
%% The file follows the ptex2tex extended LaTeX format, see
%% ptex2tex: http://code.google.com/p/ptex2tex/
%%
%% Run
%%      ptex2tex myfile
%% or
%%      doconce ptex2tex myfile
%%
%% to turn myfile.p.tex into an ordinary LaTeX file myfile.tex.
%% (The ptex2tex program: http://code.google.com/p/ptex2tex)
%% Many preprocess options can be added to ptex2tex or doconce ptex2tex
%%
%%      ptex2tex -DMINTED myfile
%%      doconce ptex2tex myfile envir=minted
%%
%% ptex2tex will typeset code environments according to a global or local
%% .ptex2tex.cfg configure file. doconce ptex2tex will typeset code
%% according to options on the command line (just type doconce ptex2tex to
%% see examples). If doconce ptex2tex has envir=minted, it enables the
%% minted style without needing -DMINTED.
% #endif

% #define PREAMBLE

% #ifdef PREAMBLE
%-------------------- begin preamble ----------------------

\documentclass[%
oneside,                 % oneside: electronic viewing, twoside: printing
final,                   % draft: marks overfull hboxes, figures with paths
10pt]{article}

\listfiles               %  print all files needed to compile this document

\usepackage{relsize,makeidx,color,setspace,amsmath,amsfonts,amssymb}
\usepackage[table]{xcolor}
\usepackage{bm,ltablex,microtype}

\usepackage[pdftex]{graphicx}

\usepackage{ptex2tex}
% #ifdef MINTED
\usepackage{minted}
\usemintedstyle{default}
% #endif

\usepackage[T1]{fontenc}
%\usepackage[latin1]{inputenc}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}

\usepackage{lmodern}         % Latin Modern fonts derived from Computer Modern

% Hyperlinks in PDF:
\definecolor{linkcolor}{rgb}{0,0,0.4}
\usepackage{hyperref}
\hypersetup{
    breaklinks=true,
    colorlinks=true,
    linkcolor=linkcolor,
    urlcolor=linkcolor,
    citecolor=black,
    filecolor=black,
    %filecolor=blue,
    pdfmenubar=true,
    pdftoolbar=true,
    bookmarksdepth=3   % Uncomment (and tweak) for PDF bookmarks with more levels than the TOC
    }
%\hyperbaseurl{}   % hyperlinks are relative to this root

\setcounter{tocdepth}{2}  % levels in table of contents

% prevent orhpans and widows
\clubpenalty = 10000
\widowpenalty = 10000

% --- end of standard preamble for documents ---

% insert custom LaTeX commands...

\raggedbottom
\makeindex
\usepackage[totoc]{idxlayout}   % for index in the toc
\usepackage[nottoc]{tocbibind}  % for references/bibliography in the toc

%-------------------- end preamble ----------------------

\begin{document}

% matching end for #ifdef PREAMBLE
% #endif

\newcommand{\exercisesection}[1]{\subsection*{#1}}

% ------------------- main content ----------------------

% ----------------- title -------------------------

\thispagestyle{empty}

\begin{center}
{\LARGE\bf
\begin{spacing}{1.25}
Fourth and fifth days: Homework set 3
\end{spacing}
}
\end{center}

% ----------------- author(s) -------------------------

\begin{center}
{\bf Data Analysis and Machine Learning${}^{}$} \\ [0mm]
\end{center}

\begin{center}
% List of all institutions:
\end{center}

% ----------------- end author(s) -------------------------

% --- begin date ---
\begin{center}
Jul 3, 2020
\end{center}
% --- end date ---

\vspace{1cm}

\subsection{Day four and five exercises}

The exercises here are somewhat longer and we expect to use at least two days on them.

\paragraph{Exercise 1, Bias-Variance tradeoff and Bootstrap.}
This exercise is a continuation of exercise 2 from the second homework set.
In that exercise we computed the MSE-score for the training
data and the test data as functions of the complexity of a polynomial,
that is the degree of a given polynomial.

One of the aims of that exercise was
to reproduce Figure 2.11 of \href{{https://github.com/CompPhysics/MLErasmus/blob/master/doc/Textbooks/elementsstat.pdf}}{Hastie et al}.

Our data is defined by $x\in [-3,3]$ with a total of for example $100$ data points.
\bpycod
np.random.seed()
n = 100
maxdegree = 14
# Make data set.
x = np.linspace(-3, 3, n).reshape(-1, 1)
y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
\epycod
where $y$ is the function we want to fit with a given polynomial.

\paragraph{Part (1a) Proving the bias-variance tradeoff.}
Consider a
dataset $\mathcal{L}$ consisting of the data
$\mathbf{X}_\mathcal{L}=\{(y_j, \boldsymbol{x}_j), j=0\ldots n-1\}$.

Let us assume that the true data is generated from a noisy model

\[
\bm{y}=f(\boldsymbol{x}) + \bm{\epsilon}.
\]

Here $\epsilon$ is normally distributed with mean zero and standard
deviation $\sigma^2$.

In our derivation of the ordinary least squares method we defined then
an approximation to the function $f$ in terms of the parameters
$\bm{\beta}$ and the design matrix $\bm{X}$ which embody our model,
that is $\bm{\tilde{y}}=\bm{X}\bm{\beta}$.

The parameters $\bm{\beta}$ are in turn found by optimizing the means
squared error via the so-called cost function

\[
C(\bm{X},\bm{\beta}) =\frac{1}{n}\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2=\mathbb{E}\left[(\bm{y}-\bm{\tilde{y}})^2\right].
\]

Show that you can rewrite  this as
\[
\mathbb{E}\left[(\bm{y}-\bm{\tilde{y}})^2\right]=\frac{1}{n}\sum_i(f_i-\mathbb{E}\left[\bm{\tilde{y}}\right])^2+\frac{1}{n}\sum_i(\tilde{y}_i-\mathbb{E}\left[\bm{\tilde{y}}\right])^2+\sigma^2.
\]

Explain what the terms mean, which one is the bias and which one is
the variance and discuss their interpretations.

\paragraph{Part (1b) Adding Bootstrap and Bias-Variance Tradeoff.}
Add now bootstrapping as discussed in the \href{{https://compphysics.github.io/MachineLearningECT/doc/pub/Day1/html/Day1.html}}{Regression lectures} (scroll down to the bias-variance code).
Add also the expressions for the bias and the variance as discussed above.

Discuss the bias and variance tradeoff as function
of your model complexity (the degree of the polynomial) and the number
of data points, and possibly also your training and test data.

Try to make a figure similar to Fig.~2.11 of Hastie et al.~You should include an analysis of the bias and variance for the test results. Figure 2.11 displays only the test and training MSEs while indicating regions of low/high bias and variance. You will most likely not get an
equally smooth curve! Note also that when you calculate the bias, in all applications you don't know the function values $f_i$. You would hence replace them with the actual data points $y_i$.

\paragraph{Exercise 2, Linear Regression for  a two-dimensional function.}
This is a longer  exercise and the aim is to study in more detail various
regression methods, including the Ordinary Least Squares (OLS) method,
Ridge regression and finally Lasso regression.
The methods are in turn combined with resampling techniques.

We will study how to fit polynomials to a specific
two-dimensional function called \href{{http://www.dtic.mil/dtic/tr/fulltext/u2/a081688.pdf}}{Franke's
function}.  This
is a function which has been widely used when testing various
interpolation and fitting algorithms. Furthermore, after having
established the model and the method, we will employ resamling
like the bootstrap from the previous exercise in order to perform a
proper assessment of our models. We will also study in detail the
so-called Bias-Variance trade off.

The Franke function, which is a weighted sum of four exponentials  reads as follows
\begin{align*}
f(x,y) &= \frac{3}{4}\exp{\left(-\frac{(9x-2)^2}{4} - \frac{(9y-2)^2}{4}\right)}+\frac{3}{4}\exp{\left(-\frac{(9x+1)^2}{49}- \frac{(9y+1)}{10}\right)} \\
&+\frac{1}{2}\exp{\left(-\frac{(9x-7)^2}{4} - \frac{(9y-3)^2}{4}\right)} -\frac{1}{5}\exp{\left(-(9x-4)^2 - (9y-7)^2\right) }.
\end{align*}

The function will be defined for $x,y\in [0,1]$.  Our first step will
be to perform an OLS regression analysis of this function, trying out
a polynomial fit with an $x$ and $y$ dependence of the form $[x, y,
x^2, y^2, xy, \dots]$. We will also include cross-validation (or bootstrap) as
resampling technique.  As in homeworks 1 and 2, we can use a uniform
distribution to set up the arrays of values for $x$ and $y$, or as in
the example below just a set of fixed
values for $x$ and $y$ with a given step
size.  We will fit a
function (for example a polynomial) of $x$ and $y$.  Thereafter we
will repeat much of the same procedure using the Ridge and Lasso
regression methods, introducing thus a dependence on the bias
(penalty) $\lambda$.

Finally we are going to use (real) digital terrain data and try to
reproduce these data using the same methods. We will also try to go
beyond the second-order polynomials metioned above and explore
which polynomial fits the data best.

The Python fucntion for the Franke function is included here (it performs also a three-dimensional plot of it)
\bpycod
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib import cm
from matplotlib.ticker import LinearLocator, FormatStrFormatter
import numpy as np
from random import random, seed

fig = plt.figure()
ax = fig.gca(projection='3d')

# Make data.
x = np.arange(0, 1, 0.05)
y = np.arange(0, 1, 0.05)
x, y = np.meshgrid(x,y)

def FrankeFunction(x,y):
    term1 = 0.75*np.exp(-(0.25*(9*x-2)**2) - 0.25*((9*y-2)**2))
    term2 = 0.75*np.exp(-((9*x+1)**2)/49.0 - 0.1*(9*y+1))
    term3 = 0.5*np.exp(-(9*x-7)**2/4.0 - 0.25*((9*y-3)**2))
    term4 = -0.2*np.exp(-(9*x-4)**2 - (9*y-7)**2)
    return term1 + term2 + term3 + term4

z = FrankeFunction(x, y)

# Plot the surface.
surf = ax.plot_surface(x, y, z, cmap=cm.coolwarm,
                       linewidth=0, antialiased=False)

# Customize the z axis.
ax.set_zlim(-0.10, 1.40)
ax.zaxis.set_major_locator(LinearLocator(10))
ax.zaxis.set_major_formatter(FormatStrFormatter('%.02f'))

# Add a color bar which maps values to colors.
fig.colorbar(surf, shrink=0.5, aspect=5)

plt.show()

\epycod

\paragraph{(2a) Ordinary Least Square on the Franke function  with resampling.}
We will generate our own dataset for a function
$\mathrm{FrankeFunction}(x,y)$ with $x,y \in [0,1]$. The function
$f(x,y)$ is the Franke function. You should explore also the addition
an added stochastic noise to this function using the normal
distribution $\cal{N}(0,1)$.

Write your own code (using either a matrix inversion or a singular
value decomposition from e.g., \textbf{numpy} ) or use your code from
homeworks 1 and 2 and perform a standard least square regression
analysis using polynomials in $x$ and $y$ up to fifth order. You can use \textbf{scikit-learn} as well.

Evaluate the Mean Squared error (MSE)

\[ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
\]

and the $R^2$ score function.  If $\tilde{\hat{y}}_i$ is the predicted
value of the $i-th$ sample and $y_i$ is the corresponding true value,
then the score $R^2$ is defined as

\[
R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
\]

where we have defined the mean value  of $\hat{y}$ as

\[
\bar{y} =  \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
\]

To set up the design matrix, the following code can be used
\bpycod
def FrankeFunction(x,y):
	term1 = 0.75*np.exp(-(0.25*(9*x-2)**2) - 0.25*((9*y-2)**2))
	term2 = 0.75*np.exp(-((9*x+1)**2)/49.0 - 0.1*(9*y+1))
	term3 = 0.5*np.exp(-(9*x-7)**2/4.0 - 0.25*((9*y-3)**2))
	term4 = -0.2*np.exp(-(9*x-4)**2 - (9*y-7)**2)
	return term1 + term2 + term3 + term4

def create_X(x, y, n ):
	if len(x.shape) > 1:
		x = np.ravel(x)
		y = np.ravel(y)

	N = len(x)
	l = int((n+1)*(n+2)/2)		# Number of elements in beta
	X = np.ones((N,l))

	for i in range(1,n+1):
		q = int((i)*(i+1)/2)
		for k in range(i+1):
			X[:,q+k] = (x**(i-k))*(y**k)

	return X

# Making meshgrid of datapoints and compute Franke's function
n = 5
N = 1000
x = np.sort(np.random.uniform(0, 1, N))
y = np.sort(np.random.uniform(0, 1, N))
z = FrankeFunction(x, y)
X = create_X(x, y, n=n)
\epycod

\paragraph{Part (2b) Resampling techniques, adding more complexity.}
Perform a resampling of the data where you split the data in training
data and test data. Here you can write your own function or use the
function for splitting training data provided by \textbf{Scikit-Learn}.
This function is called $train\_test\_split$.   You should also renormalize your data.

It is normal in essentially all Machine Learning studies to split the
data in a training set and a test set (sometimes also an additional
validation set).  There
is no explicit recipe for how much data should be included as training
data and say test data.  An accepted rule of thumb is to use
approximately $2/3$ to $4/5$ of the data as training data.

Use then the _bootstrap code you developed in the previous exercise to resample your data
and evaluate again the MSE function resulting
from the test data.

\paragraph{Part (2c): Bias-variance tradeoff.}
With a code which does OLS and includes bootstrap
we will now discuss the bias-variance tradeoff in the context of
continuous predictions such as regression. However, many of the
intuitions and ideas discussed here also carry over to classification
tasks and basically all Machine Learning algorithms.

Use the code from exercise 1 above and implement the bootstrap
resampling and perform a bias-variance tradeoff analysis like you did
in exercise 1.

\paragraph{Part (2d): Ridge Regression on the Franke function  with resampling.}
Write your own code for the Ridge method, either using matrix
inversion or the singular value decomposition ir use \textbf{scikit-learn}
Perform the same analysis as in the
previous three steps (for the same polynomials and include resampling
techniques) but now for different values of $\lambda$. Compare and
analyze your results with those obtained in parts 2a-2c). Study the
dependence on $\lambda$.

Study also the bias-variance tradeoff as function of various values of
the parameter $\lambda$. Comment your results.

\paragraph{Part (2e): Lasso Regression on the Franke function  with resampling.}
This part is essentially a repeat of the previous ones, but now
with Lasso regression. Write either your own code or
use the functionalities of \textbf{Scikit-Learn} (recommended).
Give a
critical discussion of the three methods and a judgement of which
model fits the data best.

% ------------------- end of main content ---------------

% #ifdef PREAMBLE
\end{document}
% #endif


\documentclass[12]{scrartcl}
\usepackage{amssymb,amsmath,gensymb,dsfont,calc,multicol,fullpage}
\makeatletter
\newcommand\Aboxed[1]{
   \@Aboxed#1\ENDDNE}
\def\@Aboxed#1&#2\ENDDNE{%
   &
   \settowidth\@tempdima{$\displaystyle#1{}$}
   \setlength\@tempdima{\@tempdima+\fboxsep+\fboxrule}
   \kern-\@tempdima
   \boxed{#1#2}
}
\makeatother

\begin{document}

\title{Homework 28, Section 4.5: 3, 9, 14, 20, 22, 25}
\author{Alex Gordon}
\date{\today}
\maketitle
\section*{Homework}
\subsection*{3.}
This subspace is $H = $Span $\{v_1, v_2, v_3\}, $ where $v_1 =  \begin{bmatrix} 0 \\ 1 \\ 0 \\ 1  \end{bmatrix}, v_2 =  \begin{bmatrix} 0 \\ -1 \\ 1 \\ 2  \end{bmatrix},$ and $v_3 =  \begin{bmatrix} 2 \\ 0 \\ -3 \\ 0  \end{bmatrix}$. Theorem 4 in 4.3 shows that the columns of this set are linearly independent. $v_1 \neq 0, v_2$ is not a multiple of $v_1$ and since its first entry is not zero, $v_3$ is not a linear combination of $v_1$ or $v_2$ Thus, this set is a basis for H and the dimension of the subspace is 3.
\subsection*{9.}
This subspace is $H = \begin{bmatrix} a \\ b \\ a  \end{bmatrix}: a, b $ in $R$ = Span $\{v_1, v_2 \}$ and where $v_1 = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}$ and $v_2 = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}$. Since $v_1$ and $v_2$ are not multiples of each other, they are linearly independent and are a basis for H. This means the dimension of H is 2.
\subsection*{14.}
The matrix is in echelon form. There are two pivot columns so the dimension of Col $A$ is 3. Since there are three columns without pivots, the equation $Ax = 0 $ has three free variables thus the dimension of Nul $A$ is 2.
\subsection*{20. A)}
False. the set $R^2$ is not even a subset of $R^3$.
\subsection*{20. B)}
False. The number of free variables is qual to the dimension of Null A.
\subsection*{20. C)}
False. A basis could still have only finitely many elements, which would make the vector space finite.
\subsection*{20. D)}
False. The set S must also have n elements.
\subsection*{20. E)}
True. The subspaces can only be subsets of themselves.

\subsection*{22.}
The matrix whose columns are the coordinate vectors of polynomials with the standard basis of $P_3$ is:
$\begin{bmatrix} 1 & 1 & 2 & 6 \\ 0 & -1 & -4 & -18 \\ 0 & 0 & 1 & 9 \\ 0 & 0 & 0 & -1  \end{bmatrix}$. This matrix has 4 pivots so it's columns are linearly independent. Since their coordinate vectors for a linearly independent set, the polynomials themselves are linearly independent in $p_3$. The Basis Theorem then states they form a basis for $P_3$.
\subsection*{25.}
Suppose that S Spans V and that S contains fewer than n vectors. This means that by the spanning set theorem, some subset of $S'$ of S is a basis for V. Since S contains fewer than n vectors, and $S'$  is a subset of S, $S'$ also contains fewer than n vectors. Thus there is a basis $S'$ for V with fewer than n vectors. However, this is inpossible because of theorem 10, because dimV = n. Thus S cannot span V.

\end{document}