\documentclass{enonce}
\selectlanguage{english}
\usepackage{palatino,mathpazo,color}
\usepackage{tutorial}
\setlength{\overfullrule}{3pt}
\course{IFM 2010 - TLAPS tutorial}
\title{Peterson algorithm}
\author{D. Cousineau, S. Merz}
%\institute{INRIA Lorraine - MSR INRIA Joint Lab}
\date{}

\begin{document}
\maketitle

\noindent In this tutorial, we shall prove that Peterson's algorithm satisfies the mutual exclusion property, using the TLA Proof System and the Toolbox IDE.

%\tableofcontents

\section{Toolbox and TLAPS install}
If you have not done it yet, you can download the Toolbox from
\begin{center}
\url{http://www.tlaplus.net/tools/tla-toolbox/}
\end{center}
and dowload the TLA Proof System from
\begin{center}
\url{http://www.msr-inria.inria.fr/~doligez/tlaps/}
\end{center}

\noindent Binaries are available for Windows, MacOS and Linux distributions. Notice that Cygwin (version 1.7 or higher) is required to install TLAPS on Windows.

\section{Peterson Algorithm}
This algorithm, formulated by Gary L. Peterson in 1981, is a concurrent programming algorithm for mutual exclusion that allows two processes to share a single-use resource without conflict, using only shared memory for communication.

\subsection{Description}
The algorithm allows two processes to share some resource with the three following properties:
\begin{itemize}
\item Mutual exclusion : the two processes cannot access the resource at the same time
\item Progress: a process cannot immediately re-access the resource if the other process wants also to access the resource.
\item Bounded waiting: there exists a limit on the number of times that the other process is allowed to access the resource after a process has made a request to access the resource.
\end{itemize}

\noindent We shall focus on the mutual exclusion property in the TLA+ specification we are to build.

\medskip

\noindent We call \textit{critical section} the moment when some process is accessing the shared resource. The Mutual Exclusion property ensures that the two processes are not both in the critical section at the same time.

\medskip

 The idea of the algorithm is the following: we use two variables, a function called {\tt flag}, which says for each of the processes, if it wants to access the resource or not. And a simple variable {\tt turn} which holds the process whose turn it is to enter the critical section.

\noindent At the beginning of the algorithm, {\tt flag[0]} and {\tt flag[1]} are equal to {\tt FALSE} (none of the processes wants to access the resource), and {\tt turn} is equal to {\tt 0} (process 0 has priority to access the resource).

\noindent Then, for making process {\tt 0} (for example) access the shared resource, we do the following:
\begin{enumerate}
	\item[({\tt a1})] turn the value of {\tt flag[0]} to {\tt TRUE} \\(process {\tt 0} does want to enter the critical section)
	\item[({\tt a2})] turn the value of {\tt turn} to {\tt 1} \\(give the priority to the other process)
	\item[({\tt a3})] wait for  {\tt flag[1]} to be equal to {\tt FALSE} and {\tt turn} to be equal to {\tt 1} \\(process {\tt 1} does not want to access the resource and process {\tt 0} has priority to access it)
	\item[({\tt cs})] enter the critical section \\(access the resource)
	\item[({\tt a4})] turn  the value of {\tt flag[0]} to {\tt FALSE} \\(process {\tt 0} does not want to access the resource anymore)
\end{enumerate}

\medskip

\subsection{PlusCal version}

Peterson's algorithm can be expressed in the PlusCal language as follows:
\smallskip

\begin{verbatim}
Not(i) == IF i = 0 THEN 1 ELSE 0

--algorithm Peterson {
   variables flag = [i \in {0, 1} |-> FALSE], turn = 0;
   process (proc \in {0,1}) {
     a0: while (TRUE) {
     a1:   flag[self] := TRUE;
     a2:   turn := Not(self);
     a3a:  if (flag[Not(self)]) {goto a3b} else {goto cs} ;
     a3b:  if (turn = Not(self)) {goto a3a} else {goto cs} ;
     cs:   skip;  \* critical section
     a4:   flag[self] := FALSE;
     } \* end while
    } \* end process
  }
\end{verbatim}

\noindent Notice that, for simplicity, the TLA+ specification we describe in the next section differs a little bit from the one that can be directly translated from the algorithm above, by the PlusCal translator.

\section{The specification}
Peterson's algorithm for two processes can already be verified by the Toolbox model-checker. But we use it as an exercise to make you learn ho to use TLA Proof Sytem.

\medskip

\noindent Peterson's TLA+ specification can be downloaded at
\begin{center}
\url{http://www.msr-inria.inria.fr/~doligez/tlaps/IFM2010/Peterson.tla}
\end{center}
Let us describe what you can find in this file.

\subsection{Variables}
We declare three variables, {\tt flag}, {\tt turn} and {\tt pc} (which gives the current step of the algorithm).
We define {\tt vars} as the tuple composed by those three variables, and {\tt ProcSet} as the set $\{0,1\}$.
\begin{verbatim}
VARIABLES flag, turn, pc
vars == << flag, turn, pc >>
ProcSet == ({0,1})
\end{verbatim}

\subsection{The {\tt Init} state}
We then express the initial state of the algorithm as follows:
\begin{verbatim}
Init == /\ flag = [i \in {0, 1} |-> FALSE]
        /\ turn = 0
        /\ pc = [self \in ProcSet |-> CASE self \in {0,1} -> "a0"]
\end{verbatim}
{\tt flag[0]} and {\tt flag[1]} are equal to {\tt FALSE}.
\\ {\tt turn} is equal to {\tt 0}.
\\ Both processes are entering the while spin.

\subsection{The {\tt Next} action}
In order to define the {\tt Next} action of Peterson's algorithm, we have to describe, for each process, every possible step of the algorithm. For example the {\tt a0} step describes the fact that the considered process ({\tt self}) is going from line {\tt a0} to line {\tt a1} in the algorithm (in that case, values of variables {\tt flag} and {\tt turn} don't change).

\begin{verbatim}
a0(self) == /\ pc[self] = "a0"
            /\ pc' = [pc EXCEPT ![self] = "a1"]
            /\ UNCHANGED << flag, turn >>
\end{verbatim}

\noindent The second line above expresses the fact that the next value of the function {\tt pc} is the same as the current one, except that its value on {\tt self} is now {\tt "a1"}.

\medskip

\noindent We then do the same for each possible step of the algorithm:
\begin{verbatim}
a1(self) == /\ pc[self] = "a1"
            /\ flag' = [flag EXCEPT ![self] = TRUE]
            /\ pc' = [pc EXCEPT ![self] = "a2"]
            /\ UNCHANGED turn

a2(self) == /\ pc[self] = "a2"
            /\ turn' = Not(self)
            /\ pc' = [pc EXCEPT ![self] = "a3a"]
            /\ UNCHANGED flag

a3a(self) == /\ pc[self] = "a3a"
             /\ IF flag[Not(self)]
                   THEN /\ pc' = [pc EXCEPT ![self] = "a3b"]
                   ELSE /\ pc' = [pc EXCEPT ![self] = "cs"]
             /\ UNCHANGED << flag, turn >>

\end{verbatim}
\newpage
\begin{verbatim}
a3b(self) == /\ pc[self] = "a3b"
             /\ IF turn = Not(self)
                   THEN /\ pc' = [pc EXCEPT ![self] = "a3a"]
                   ELSE /\ pc' = [pc EXCEPT ![self] = "cs"]
             /\ UNCHANGED << flag, turn >>

cs(self) == /\ pc[self] = "cs"
            /\ TRUE
            /\ pc' = [pc EXCEPT ![self] = "a4"]
            /\ UNCHANGED << flag, turn >>

a4(self) == /\ pc[self] = "a4"
            /\ flag' = [flag EXCEPT ![self] = FALSE]
            /\ pc' = [pc EXCEPT ![self] = "a0"]
            /\ UNCHANGED turn
\end{verbatim}

\noindent Now we can define {\tt proc(self)} as the fact that one of the previous actions is being accomplished:
\begin{verbatim}
proc(self) == \/ a0(self) \/ a1(self) \/ a2(self) \/ a3a(self)
              \/ a3b(self) \/ cs(self) \/ a4(self)
\end{verbatim}

\noindent Finally, we define the {\tt Next} action, as the fact that either {\tt proc} is accomplished for one of the processes, or the algorithm has finished (to prevent deadlock on termination).

\begin{verbatim}
Next == \E self \in {0,1}: proc(self)
\end{verbatim}

\medskip
\subsection{Specification}
The specification of the algorithm is given by the facts that the initial state is satisfied and that at every step either the action {\tt Next} is satisfied or the variables in {\tt vars} keep their values.

\begin{verbatim}
Spec == Init /\ [][Next]_vars
\end{verbatim}

\medskip
\subsection{Mutual Exclusion}
%\noindent The termination property is defined by the fact that there exists some state such that for that state and all the next ones, both processes are in step {\tt "Done"}.
%
%\begin{verbatim}
%Termination == <>(\A self \in ProcSet: pc[self] = "Done")
%\end{verbatim}
%
%\noindent The safeness property is defined by the fact that the two processes don't access the resource at the same time.
%
%\begin{verbatim}
%Safe == \E i \in {0, 1} : pc[i] # "cs"
%\end{verbatim}
%
%\noindent The liveness property, is defined by the fact that both processes access the resource at some time.
%
%\begin{verbatim}
%Live == \A i \in {0, 1} : []<>(pc[i] = "cs")
%\end{verbatim}

The property we want the algorithm to satisfy can be defined as the fact that {\tt pc[0]} and {\tt pc[1]} have not both value {\tt "cs"} at the same time.

\begin{verbatim}
MutualExclusion == ~ (pc[0] = "cs" /\ pc[1] = "cs")
\end{verbatim}

\medskip
\subsection{The invariant}
Let us first define the property that ensures well-typedness of the variables .
%\textcolor{red}{sm: if you remove the second disjunct of Next, also remove that remark.}

\begin{verbatim}
TypeOK == /\ pc \in [{0,1} -> {"a0","a1","a2","a3a","a3b","cs","a4"}]
          /\ turn \in {0,1}
          /\ flag \in [{0,1} -> BOOLEAN]
\end{verbatim}

\noindent Now we can define the invariant we want the algorithm to satisfy, in order to prove the \linebreak {\tt MutualExclusion} property. It is defined as the fact that for each process {\tt i},
\begin{itemize}
\item if it is in step {\tt "a2"}, {\tt "a3a"}, {\tt "a3b"}, {\tt "cs"} or {\tt "a4"}, then its flag is equal to {\tt TRUE} (i.e. it does want to access to the resource)
\item if it is in step {\tt "cs"} or {\tt "a4"}, then the other process is not in one of these steps, and if the other process is in step {\tt "a3a"} or {\tt "a3b"}  then process {\tt i} has priority to access the resource.
\end{itemize}

\smallskip

\begin{verbatim}
I == \A i \in {0, 1} :
       /\ (pc[i] \in {"a2", "a3a", "a3b", "cs", "a4"} => flag[i])
       /\ (pc[i] \in {"cs", "a4"})
            => /\ pc[Not(i)] \notin {"cs", "a4"}
               /\ (pc[Not(i)] \in {"a3a", "a3b"}) => (turn = i)
\end{verbatim}

\noindent Finally, we can define the actual invariant as the conjunction of the previous one and the fact that {\tt TypeOK} is satisfied.

\begin{verbatim}
FullInv == TypeOK /\ I
\end{verbatim}

\section{Checking a simple proof from the Toolbox}
\label{simple}
As usual when we want to prove that a property is always satisfied (here {\tt MutualExclusion}), we reason by induction and define a invariant property (here {\tt FullInv}) such that:
\begin{enumerate}
\item the initial state (here {\tt Init}) satisfies the invariant,
\item the invariant is preserved by the next-state relation (here {\tt [Next]\_vars}),
\item the invariant ensures the correctness property ({\tt MutualExclusion}).
\end{enumerate}

\noindent Proving the first fact is obvious. Let us check that by asking TLAPS to prove the following theorem (recall that we have to indicate explicitly which definitions we want to be usable in a proof):

\begin{verbatim}
THEOREM InitFullInv == Init => FullInv
  BY DEF Init, FullInv, TypeOK, I, ProcSet
\end{verbatim}

%\textcolor{red}{sm: this should be \texttt{FullInvariant}, not \texttt{MutualExclusion}}

\noindent Now, in the Toolbox, put the cursor on the first line of that theorem and then right-click on "Prove step or module". The Toolbox then colors that theorem in green, which means that it is proved (in this case, Zenon has succeeded in finding a proof).

\section{The main theorem}

\noindent Let us now focus fact number 2 above. For proving that fact it is easier to an intermediate lemma, which states that property {\tt TypeOK} is preserved by the next-state relation.

%consider two intermediate lemmas. The first one expresses that if {\tt TypeOK} is satisfied, then any of the process can be in step {\tt "Done"}. It comes directly from the definitions of {\tt TypeOK} and {\tt ProcSet}.
%\begin{verbatim}
%LEMMA NeverDone ==
%          TypeOK /\ (\A self \in ProcSet: pc[self] = "Done") => FALSE
%     BY DEFS TypeOK, ProcSet
%\end{verbatim}
%
%\textcolor{red}{sm: this lemma would disappear, and the proof of the subsequent one simplified}
%\noindent The second one states that property {\tt TypeOK} is preserved by the next-state relation.

\begin{verbatim}
LEMMA TypeCorrect == TypeOK /\ Next => TypeOK'
  <1>1. ASSUME NEW i \in {0,1}
        PROVE  TypeOK /\ proc(i) => TypeOK'
    BY DEFS TypeOK, proc, a0, a1, a2, a3a, a3b, cs, a4, Not
  <1>. QED
    BY <1>1, NeverDone DEF Next
\end{verbatim}

\noindent When asking TLAPS to prove the theorem above, you can notice that step {\tt<1>1} gets first colored black. That means that Zenon has been trying to prove it for more than three seconds. Then it gets colored red. That means that Zenon has failed to find a proof with the default timeout (ten seconds). Finally, it gets colored green since Isabelle succeeds in proving it.

\bigskip

\noindent Finally, let us prove the second item of section \ref{simple}.

\begin{verbatim}
THEOREM Inductive == TypeOK /\ I /\ Next => I'
\end{verbatim}

\noindent We first rephrase the sequent to be proved in a more perspicuous form:

\begin{verbatim}
<1>1. ASSUME TypeOK,
             I,
             NEW i \in {0,1},
             proc(i)
      PROVE  I'
\end{verbatim}

\noindent We then assert {\tt TypeOK'} (that will be useful in the following of the proof):

\begin{verbatim}
  <2>0. TypeOK'
\end{verbatim}

\noindent We then split the result we want to prove into three assertions:

\begin{verbatim}
  <2>1. ASSUME NEW j \in {0,1},
               pc'[j] \in {"a2", "a3a", "a3b", "cs", "a4"}
        PROVE  flag'[j]

  <2>2. ASSUME NEW j \in {0,1},
               pc'[j] \in {"cs", "a4"}
        PROVE pc'[Not(j)] \notin {"cs", "a4"}

  <2>3. ASSUME NEW j \in {0,1},
               pc'[j] \in {"cs", "a4"},
               pc'[Not(j)] \in {"a3a", "a3b"}
        PROVE turn' = j
\end{verbatim}

\noindent Those three assertions can be proved by case on the step in which process {\tt i} is.
If you ask TLAPS to prove that theorem, you can notice that all those proof-steps are colored yellow. That means that the proof is missing. It's up to you now to make them green...

\end{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{appendix}
\section{Prover options}

\section{Practical hints}

The TLA+ proof system is designed to check the validity of claims as
independently as possible of specific proof back-ends. We believe that users
should concentrate on writing proofs in terms of their particular applications,
not in terms of the capabilities of a particular proof system. In particular,
TLAPS invokes its back-ends with some default setup for automatic proof, and we
try to make it hard for users to change this default setup. Expert users of
back-end provers may be frustrated because they may have to develop proofs
somewhat further than what would be necessary with a fine-tuned tactic script.
The main payoff of limited access to the nitty-gritty details of provers is
greater clarity of the resulting proofs. They are also easier to maintain across
minor changes of the specification or new releases of the TLA prover.

On some occasions users will encounter situations where the prover cannot prove
an "obvious" proof obligation. Here are a few hints on what to try to make the
proof go through. Your additions to this list are welcome.

\subsection{Control the size of formulas and expressions}
Our provers are currently not good at making abstractions that humans understand
immediately. They are easily confused by moderately big proof obligations and
are just as likely to work on a top-level conjunction as on a set construction
buried deeply inside the formula. This can cause back-ends to become very slow
or even unable to complete seemingly trivial steps. While we intend to improve
the back-ends in this respect, you can help them by using local definitions in
proofs and hiding these definitions in order to keep expressions small. (Keep in
mind that definitions introduced in a proof are usable by default and must be
hidden explicitly, unlike definitions in specifications, which must be
explicitly $\USE$d.)

\medskip

\noindent Here is a contrived example:

\begin{verbatim}
LEMMA  /\ x \in SomeVeryBigExpression
       /\ y \in AnotherBigExpression
   <=>
       /\ y \in AnotherBigExpression
       /\ x \in SomeVeryBigExpression

<1> DEFINE S == SomeVeryBigExpression
    \** here and in the following, you may use positional names to
    \** avoid repeating the big expressions
<1> DEFINE T == AnotherBigExpression
<1>1. x \in S <=> x \in SomeVeryBigExpression
  OBVIOUS
    \** The provers should not have any trouble proving that
    \** a very large expression is equivalent to itself.  If it
    \** can't prove this, you've almost certainly made a mistake
    \** in the definition of S.
<1>2. y \in T <=> y \in AnotherBigExpression
  OBVIOUS
<1> HIDE DEF S, T
<1>3. /\ x \in S
      /\ y \in T
  <=>
      /\ y \in T
      /\ x \in S
  OBVIOUS
<1>4. QED
  BY <1>1, <1>2, <1>3
\end{verbatim}

\medskip

This kind of problem typically arises when reasoning about LET expressions,
which are \linebreak silently expanded by the proof manager. In a proof, introduce local
definitions corresponding to the LET (using copy and paste from the
specification or subexpression names), show that the overall expression equals the body of the LET
(trivial by reflexivity), establish the necessary facts about these locally
defined operators, and HIDE the definitions afterwards.

\medskip

Introducing definitions to hide irrelevant expressions is a useful
thing to do whenever you're having trouble proving something.  It
makes the obligations whose proofs fail easier to read, making
mistakes easier to find.

\subsection{Avoid "circular" (sets of) equations}

Rewriting is one effective way to reason about equations, and it underlies the
automatic proof methods used by the Isabelle back-end. The basic idea is to
orient equalities such that the expressions on the left-hand side are
systematically replaced by the right-hand sides. However, if the set of
equations contains cycles as in

\begin{verbatim}
s = f(t)
t = g(s)
\end{verbatim}

then rewriting may never terminate. Isabelle employs some (incomplete)
heuristics to detect such cycles and will refuse to rewrite equations that it
determines to be circular. This usually leads to its inability to infer anything
about these equations. If circularity is not detected, it may cause Isabelle to
enter an infinite loop. The suggested remedy is again to introduce local
definitions that are hidden to break the loops.

\medskip

\noindent As a concrete example consider the following proof snippet:

\begin{verbatim}
   <4>17. foo.name = "xyz"
     <5>1. foo = [name |-> "xyz", value = foo.value]
       BY <2>2
     <5>2. QED
       BY <5>1  \** may not work because <5>1 is a circular equation
\end{verbatim}

\noindent One possible workaround is as follows:

\begin{verbatim}
   <4>17. foo.name = "xyz"
     <5>   DEFINE fooval == foo.value
     <5>1. foo = [name |-> "xyz", value = fooval]
       BY <2>2
     <5>   HIDE DEF fooval
     <5>2. QED
       BY <5>1
\end{verbatim}
%\end{appendix}
%\end{document}
\subsection{Reasoning about CHOOSE expressions}

Consider a definition such as

\begin{verbatim}
  foo == CHOOSE x \in S : P(x)
\end{verbatim}

In order to prove a property Q(foo), you will typically prove the two following
assertions:

\begin{verbatim}
(a) \E x \in S : P(x)
(b) \A x \in S : P(x) => Q(x)
\end{verbatim}

In some cases, assertion (b) can be trivial and need not be shown explicitly.
Reasoning about an unbounded CHOOSE expression is analogous.

\medskip

Remember that CHOOSE always denotes some value, even if P(x) holds for no
$x \in S$ (in particular, if $S = \{\}$), in which case the CHOOSE expression is
fixed, but arbitrary. In practice, CHOOSE expressions usually arise when
condition (a) is satisfied. Should you have designed your property to work even
if the domain of the CHOOSE is empty, property Q must be trivial in that case,
and you can structure your proof as follows:

\begin{verbatim}
  <3>5. Q(foo)
    <4>1. CASE \E x \in S : P(x)
      <5>1. \A x \in S : P(x) => Q(x)
      <5>2. QED
        BY <4>1, <5>1
    <4>2. CASE ~ \E x \in S : P(x)
      <5>1. \A x : Q(x)
      <5>2. QED
        BY <5>1
    <4>3. QED
      BY <4>1, <4>2
\end{verbatim}

A frequent TLA+ idiom is to define a "null" value by writing

\begin{verbatim}
NoValue == CHOOSE x : x \notin Value
\end{verbatim}

The laws of set theory ensure that no set is universal, hence there exists an x
that is not an element of set Value, ensuring condition (a) above. The theorem
NoSetContainsEverything in the standard module TLAPS can be used to prove this
condition.

\subsection{Help the Prover When Reasoning About Records}

In one proof, we had

\begin{verbatim}
    mb == [type  |-> "1b", bal |-> b, acc |-> self,
          mCBal |-> maxCBal[self], mCVal |-> maxCVal[self]]
\end{verbatim}

and were trying to prove

\begin{verbatim}
    m1 # mb /\ m2 # mb
\end{verbatim}

from facts that included

\begin{verbatim}
    m1.type = "2av" /\ m2.type = "2av"
\end{verbatim}

Zenon failed on the proof and Isabelle proved it only after a long
time.  (In fact, we originally stopped the proof because it was taking
so long.)  However, Zenon proved it instantly when we added $mb.type =
"1b"$ to the BY statement's list of facts.  The provers are reluctant
to try finding relations of the form record.field = value; they often
need help.

\subsection{Divide and Conquer}

When the provers can't prove something that you think is obvious, it's
usually because it isn't true.  You can easily spend hours looking at
a proof obligation without noticing a tiny mistake.  The best way to
find a mistake is by breaking the proof into simpler steps.
Continuing to do this on the step or steps whose proof fails will
eventually lead you to discover the problem--usually a missing
hypothesis or a mistake in a formula.  When you correct the mistake in
the original proof step, the prover will usually be able to prove it.

\subsection{Don't Reinvent Mathematics}

We expect that most people who use TLAPS will do so because they want
to verify properties of an algorithm or system.  We have therefore not
devoted our limited resources to building libraries of mathematical
results.  If you want to create such libraries, we would welcome your
help.  However, if you are concerned with an algorithm or system, you
should not be spending your time proving basic mathematical facts.
Instead, you should assert the mathematical theorems you need as
assumptions or theorems.

\medskip

Asserting facts is dangerous, because it's easy to make a mistake and
assert something false, making your entire proof unsound.
Fortunately, you can use the TLC model checker to avoid such mistakes.
For example, our example correctness proof of Euclid's algorithm uses
this assumption

\begin{verbatim}
    ASSUME GCDProperty3 ==
           \A p, q \in Nat \ {0}: (p < q) => GCD(p, q) = GCD(p, q-p)
\end{verbatim}

TLC cannot check this assumption because it can't evaluate a
quantification over an infinite set.  However, you can tell TLC to
replace the definition of Nat with

\begin{verbatim}
    Nat == 0..50
\end{verbatim}

(In the Toolbox, use the Definition Override section of the model's
Advanced Options page.)  TLC quickly verifies this assumption.  (TLC
checks each ASSUME; to add an assumption that you don't want TLC to check,
make it an AXIOM.)

This kind of checking is almost certain to catch an error in
expressing a fundamentally correct mathematical result--except when
the only counterexamples are infinite.  Fortunately, this is rarely
the case when the result is needed for reasoning about an algorithm or
system.

\subsection{It's Easier to Prove Something if it's True}

Before trying to prove a property of an algorithm or system, try to
check it with TLC. Even if TLC cannot check a large enough model to
catch all errors, running it on a small model can still catch many
simple errors.  You will save a lot of time if you let TLC find these
errors instead of discovering them while writing the proof.

\end{appendix}

\end{document}


\hypertarget{haskell}{%
\section{Haskell - Declaring Types and Classes}\label{haskell}}

\hypertarget{type-declarations}{%
\subsection{Type Declarations}\label{type-declarations}}

In Haskell, a new name for an \textbf{existing} type can be defined
using a type declaration. (not new type)

\begin{lstlisting}[language=Haskell]
type String = [Char]
--or
type Pos = (Int,Int)
\end{lstlisting}

Like function definitions, type declarations can also have parameters.
For example, given

\begin{lstlisting}[language=Haskell]
type Pair a = (a,a)
\end{lstlisting}

Type declarations can be nested, but not recursive.

\begin{lstlisting}[language=Haskell]
--Allowed
type Pos = (Int,Int)
type Trans = Pos -> Pos

--Not allowed
type Tree = (Int,[Tree])
\end{lstlisting}

\hypertarget{data-declarations}{%
\subsection{Data Declarations}\label{data-declarations}}

A completely \textbf{new} type can be defined by specifying its values
using a data declaration.

\begin{lstlisting}[language=Haskell]
data Bool = False | True
\end{lstlisting}

\begin{itemize}
\tightlist
\item
  The two values False and True are called the constructors for the type
  Bool.
\item
  Type and constructor names must always begin with an upper-case
  letter.
\item
  Data declarations are similar to context free grammars. The former
  specifies the values of a type, the latter the sentences of a
  language.
\end{itemize}

\hypertarget{constructor}{%
\subsubsection{Constructor}\label{constructor}}

The constructors in a data declaration can also have parameters. For
example, given

\begin{lstlisting}[language=Haskell]
data Shape  = Circle Float
            | Rect Float Float
\end{lstlisting}

we can define

\begin{lstlisting}[language=Haskell]
square :: Float -> Shape
square n = Rect n n
area :: Shape -> Float
area (Circle r) = pi * r^2
area (Rect x y) = x * y
\end{lstlisting}

Not surprisingly, data declarations themselves can also have parameters.
For example, given

\begin{lstlisting}[language=Haskell]
data Maybe a = Nothing | Just a
\end{lstlisting}

we can define

\begin{lstlisting}[language=Haskell]
safediv :: Int -> Int -> Maybe Int
safediv _ 0 = Nothing
safediv m n = Just (m `div` n)
safehead :: [a] -> Maybe a
safehead [] = Nothing
safehead xs = Just (head xs)
\end{lstlisting}

\hypertarget{recursive-types}{%
\subsection{Recursive Types}\label{recursive-types}}

In Haskell, new types can be declared in terms of themselves. That is,
types can be recursive.

\begin{lstlisting}[language=Haskell]
data Nat = Zero | Succ Nat
--Nat is either zero or a successor of Nat
\end{lstlisting}

e.g.~Succ (Succ (Succ Zero)) = 3.

\hypertarget{binary-trees}{%
\subsection{Binary Trees}\label{binary-trees}}

A binary tree is a two-way branching structure. Using recursion, a
suitable new type to represent such binary trees can be declared by:

\begin{lstlisting}[language=Haskell]
data Tree a = Leaf a
            | Node (Tree a) a (Tree a)
\end{lstlisting}

\begin{figure}[H]
\centering
\includegraphics[width=0.5\textwidth]{figures/binaryTreeExample.png}
\caption{Tree Example}
\end{figure}

This tree could be represented with

\begin{lstlisting}[language=Haskell]
t :: Tree Int
t = Node (Node (Leaf 1) 3 (Leaf 4)) 5
(Node (Leaf 6) 7 (Leaf 9))
22
\end{lstlisting}

\clearpage


\documentclass[10pt]{article}
\usepackage[usenames]{color} %used for font color
\usepackage{amssymb} %maths
\usepackage{amsmath} %maths
\usepackage[utf8]{inputenc} %useful to type directly
\usepackage{tikz}
%diacritic characters
\begin{document}
\paragraph{Quotient Set Notation}
There are many ways denote the quotient set (check the \emph{Book of Proof} for one way).
A common method is to get a notation for an individual equivalence class first.
Say $[x] = \{ y \in C: x \sim y\}$ which is the set of all bundles in $C$ that are indifferent to $x$.
Then we might create a class of sets (set of sets) using the indexed collection of sets notation you covered in the prerequisite readings.
For example $\mathcal{I} = \{ [x] : x \in C\}$ where $[x]=\{ y \in C: x \sim y\}$.
Remember that sets only contain distinct elements so even though we ``loop'' through all values $x$ any two values $x,y$ such that $x \sim y$ will lead to $[x] = [y]$ and the distinct equivalence class will only show up once in the collection $\mathcal{I}$.

\paragraph{Problem 10}
Let $\succsim$ be rational preference relation and define $\sim$ as the indifference relation where $x\sim y$ iff $[x\succsim y \wedge y\succsim x]$.
Because $\succsim$ is rational we know that it is complete, reflexive and transitive.
To show $(x,x) \in \sim, \; \forall x$ we can refer to completeness to know that either $x\succsim x$ or $x\succsim x$ and by reflexiveness we know both statements are true so $x\sim x$.

Symmetry would imply that for any $x,y\in C$ if $(x,y)\in \sim$ then $(y,x) \in \sim$.
Again we now know that $\sim$ is reflexive and we still know $\succsim$ is complete, reflexive and transitive and we should use these facts to establish symmetry and transitivity of $\sim$.
Let $x,y\in C$ and suppose $x \sim y$.
Then we know $[x\succsim y \wedge y\succsim x]$ is true by definition of $\sim$.
Because the order of arguments in the conjunction don't alter its truth value we could say
$[y\succsim x \wedge x \succsim y]$ which is the definition of $y\sim x$.

For transitivity, suppose $x,y,z \in C$ and $[x\sim y \wedge y \sim z]$.
Then by definition $[x\succsim y \wedge y\succsim x]$ and $[y\succsim z \wedge z\succsim y]$.
Hence we have $x \succsim y \succsim z$ which by transitivity of $\succsim$ implies $x\succsim z$ and we also have $z\succsim y \succsim x$ which by transitivity of $\succsim$ implies $z\succsim x$.
Since we have $[x\succsim z \wedge z\succsim x]$ we see that $x \sim z$ and therefore $\sim$ is transitive.
\end{document}


\subsection{Simple algebraic structures}\label{subsec:simple_algebraic_structures}

We list here several basic algebraic structures, we will use mostly as building blocks for more complicated structures.

As discussed in \fullref{rem:first_order_model_notation}, denoting all operations explicitly is cumbersome, and we will usually avoid it.

\begin{definition}\label{def:pointed_set}\mcite[26]{MacLane1994}
  The simplest algebraic structure is a \term{pointed set}. It is simply a nonempty set \( \mscrX \) equipped with a distinguished element \( e \). It is an algebraic structure because \( e \) can be regarded as the sole value of a nullary function \( *: \mscrX^0 \to \mscrX \).

  We will call \( e \) the \term{origin} of \( \mscrX \) based on the terminology for \hyperref[def:euclidean_plane_coordinate_system/origin]{affine coordinate systems}.

  \begin{thmenum}
    \thmitem{def:pointed_set/theory} Pointed sets can also be viewed as \hyperref[def:first_order_semantics/satisfiability]{models} of an empty \hyperref[def:first_order_theory]{theory} for a \hyperref[def:first_order_language]{first-order logic language} with a constant symbol, i.e. a nullary \hyperref[def:first_order_language/func]{functional symbol}.

    \thmitem{def:pointed_set/homomorphism} A \hyperref[def:first_order_homomorphism]{homomorphism} between the pointed sets \( (\mscrX, e_{\mscrX}) \) and \( (\mscrY, e_{\mscrY}) \) is, explicitly, a function \( \varphi: \mscrX \to \mscrY \) that satisfies
    \begin{equation}\label{eq:def:pointed_set/homomorphism}
      \varphi(e_{\mscrX}) = e_{\mscrY}.
    \end{equation}

    \thmitem{def:pointed_set/submodel} The set \( S \subseteq \mscrX \) is a \hyperref[def:first_order_substructure]{substructure} if \( \mscrX \) if \( e \in S \).

    \thmitem{def:pointed_set/trivial} The \hyperref[thm:substructures_form_complete_lattice/bottom]{trivial} pointed set is, up to an isomorphism, the set \( \set{ e } \).

    It is a \hyperref[def:universal_objects/initial]{zero object} in \( \cat{Set_*} \) as discussed in \fullref{ex:def:universal_objects/grp}.

    \thmitem{def:pointed_set/category} We denote the \hyperref[def:category_of_small_first_order_models]{category of \( \mscrU \)-small models} for this theory by \( \ucat{Set}_* \).
  \end{thmenum}
\end{definition}

\begin{definition}\label{def:pointed_set_kernel}\mimprovised
  The \term{kernel} \( \ker(\varphi) \) of a \hyperref[def:pointed_set/homomorphism]{pointed set homomorphism} \( \varphi: \mscrX \to \mscrY \) is the \hyperref[def:zero_locus]{zero locus} of \( \varphi \). That is, \hyperref[thm:def:function/properties/preimage]{preimage} \( \varphi^{-1}(e_\mscrY) \).

  These are the \hyperref[def:zero_morphisms/kernel]{categorical kernels} in \hyperref[def:pointed_set/category]{\( \cat{Set_*} \)}. This is discussed in \fullref{ex:zero_morphisms_in_grp/kernel} for \hyperref[def:group]{groups}, where the argument only depends on groups being pointed sets.
\end{definition}

\begin{definition}\label{def:set_with_involution}\mimprovised
  A \term{set with an involution} is a \hyperref[def:set]{set} \( \mscrX \) with a unary operation \( (\anon)^{-1} \) such that
  \begin{equation*}
    (x^{-1})^{-1} = x
  \end{equation*}
  for every \( x \in \mscrX \).

  Such an operation is called, surprisingly, an \term{involution}.

  \begin{thmenum}
    \thmitem{def:set_with_involution/theory} We define the theory of sets with involution as a theory over the language consisting of a single unary functional symbol \( \anon^{-1} \) and the sole axiom
    \begin{equation}\label{eq:def:set_with_involution/theory/axiom}
      (\xi^{-1})^{-1} \doteq \xi.
    \end{equation}

    \thmitem{def:set_with_involution/homomorphism} A \hyperref[def:first_order_homomorphism]{homomorphism} between sets with involutions \( \mscrX \) and \( \mscrY \) is a function \( \varphi: \mscrX \to \mscrY \) satisfying
    \begin{equation}\label{eq:def:set_with_involution/homomorphism}
      \varphi(x^{-1})
      =
      \varphi(x)^{-1}.
    \end{equation}

    \thmitem{def:set_with_involution/submodel} Any subset of a set with involution is again a set with involution.

    \thmitem{def:set_with_involution/trivial} The \hyperref[thm:substructures_form_complete_lattice/bottom]{trivial} set with involution is the empty set.

    \thmitem{def:set_with_involution/category} We denote the \hyperref[def:category_of_small_first_order_models]{category of \( \mscrU \)-small models} for this theory by \( \ucat{Inv} \).
  \end{thmenum}
\end{definition}

\begin{definition}\label{def:magma}
  A \term{magma} is a set \( \mscrM \) equipped with a \hyperref[def:multi_valued_function/arguments]{binary function} \( \cdot: \mscrM \times \mscrM \to \mscrM \), called the \term{magma operation}. Unless specified otherwise, we denote this operation by juxtaposition as \( xy \) instead of \( x \cdot y \).

  We often call the operation \term{multiplication} or, in the case of \hyperref[def:endomorphism_monoid]{endomorphism monoids} --- \term{composition}. See also the notes in \fullref{rem:additive_magma} regarding additive magmas and in \fullref{def:monoid_delooping} regarding the order of operands.

  \begin{thmenum}[series=def:magma]
    \thmitem{def:magma/theory} In analogy to the \hyperref[def:pointed_set/theory]{theory of pointed sets}, we can define the theory of magmas as an empty theory over a language with a single \hyperref[rem:first_order_formula_conventions/infix]{infix} binary functional symbol.

    \thmitem{def:magma/homomorphism} A \hyperref[def:first_order_homomorphism]{homomorphism} between the magmas \( (\mscrM, \cdot_{\mscrM}) \) and \( (\mscrN, \cdot_{\mscrN}) \) is, explicitly, a function \( \varphi: \mscrM \to \mscrN \) such that
    \begin{equation}\label{eq:def:magma/homomorphism}
      \varphi(x \cdot_{\mscrM} y) = \varphi(x) \cdot_{\mscrN} \varphi(y)
    \end{equation}
    for all \( x, y \in \mscrM \).

    \thmitem{def:magma/submodel} The set \( S \subseteq \mscrM \) is a \hyperref[def:first_order_substructure]{first-order submodel} of \( \mscrM \) if it is closed under the magma operation. That is, if \( x, y \in S \) implies \( xy \in S \).

    We call \( S \) a \term{submagma} of \( \mscrM \).

    \thmitem{def:magma/category} We denote the \hyperref[def:category_of_small_first_order_models]{category of \( \mscrU \)-small models} for the theory of magmas by \( \ucat{Mag} \).

    \thmitem{def:magma/trivial} The \hyperref[thm:substructures_form_complete_lattice/bottom]{trivial magma} is the empty set with an empty operation. It is the unique \hyperref[def:universal_objects/initial]{zero object} in \( \cat{Mag} \).

    \thmitem{def:magma/opposite} The \term{opposite magma} of \( (\mscrM, \cdot) \), also called the \term{dual magma}, is the magma \( (\mscrM, \star) \) with multiplication reversed:
    \begin{equation*}
      x \star y \coloneqq y \cdot x.
    \end{equation*}

    We denote the opposite magma by \( \mscrM^{-1} \).

    \thmitem{def:magma/exponentiation} We define an additional \term{exponentiation} operation for positive integers \( n \) inductively as
    \begin{equation}\label{eq:def:magma/exponentiation}
      x^n \coloneqq \begin{cases}
        x,               & n = 1 \\
        x \cdot x^{n-1}, & n > 1
      \end{cases}
    \end{equation}

    \thmitem{def:magma/power_set} It is customary to perform magma operations with sets. That is, if \( A \) and \( B \) are sets in the magma \( \mscrM \), it is customary to write
    \begin{equation*}
      A \cdot B \coloneqq \set{ a \cdot b \colon a \in A, b \in B }.
    \end{equation*}

    This actually turns the power set \( \pow(\mscrM) \) into a magma, which we will call the \term{power set magma} of \( \mscrM \). This is especially useful with the convention \fullref{rem:singleton_sets} since it allows us to write \( aB \) for \( a \in M \) and \( B \subseteq M \).

    See \fullref{thm:power_set_magma_preservation}.
  \end{thmenum}

  We list some additional restrictions that are often imposed on magmas.
  \begin{thmenum}[resume=def:magma]
    \thmitem{def:magma/associative} We can add the (\hyperref[thm:implicit_universal_quantification]{universal closure} of) following axiom to the theory:
    \begin{equation}\label{eq:def:magma/associative}
      (x \cdot y) \cdot z = x \cdot (y \cdot z).
    \end{equation}

    If \eqref{eq:def:magma/associative} is satisfied, we say that the operation \( \cdot \) and, by extension, the magma itself, are \term{associative}. Associative magmas are usually called \term{semigroups}. Associativity imposes no additional restrictions on the homomorphisms, hence semigroups are a \hyperref[def:subcategory]{full subcategory} of \( \cat{Mag} \).

    \thmitem{def:magma/commutative} Another common axiom is \term{commutativity}:
    \begin{equation}\label{eq:def:magma/commutative}
      x \cdot y = y \cdot x.
    \end{equation}

    Commutative magmas also form a full subcategory. Obviously \( \mscrM = \mscrM^{-1} \) in a commutative magma.

    \thmitem{def:magma/idempotent} We say that the operation \( \cdot \) is \term{idempotent} if
    \begin{equation}\label{eq:def:magma/idempotent}
      x \cdot x = x.
    \end{equation}

    \thmitem{def:magma/cancellative} We say that \( \cdot \) is \term{left-cancellative} (resp. \term{right-cancellative}) if
    \begin{equation}\label{eq:def:magma/cancellative}
      z \cdot x = z \cdot y \T{for all} z \in \mscrM \T{implies that} x = y
      \quad
      (\T{resp.} x \cdot z = y \cdot z).
    \end{equation}

    The operation is \term{cancellative} if it is both left and right cancellative. Cancellative magmas also form a full subcategory.
  \end{thmenum}
\end{definition}

\begin{example}\label{ex:def:magma}
  \hfill
  \begin{thmenum}
    \thmitem{ex:def:magma/composition} The quintessential example of a non-\hyperref[def:magma/commutative]{commutative} operation is \hyperref[def:multi_valued_function/composition]{composition} in any set of functions or, more generally, \hyperref[def:category/composition]{morphism composition} in any \hyperref[def:category]{category}.

    Composition is \hyperref[def:magma/associative]{associative}. \hyperref[def:magma/cancellative]{Cancellation} with respect to composition is discussed in \fullref{def:morphism_invertibility} and, for function composition, in \fullref{thm:function_invertibility_categorical}.

    \thmitem{ex:def:magma/midpoint} The operation
    \begin{equation*}
      (x, y) \mapsto \dfrac {x + y} 2
    \end{equation*}
    makes \( \BbbR \) a commutative and cancellative magma, which is not associative.
  \end{thmenum}
\end{example}

\begin{remark}\label{rem:additive_magma}
  General groups often arise as \hyperref[def:automorphism_group]{automorphism groups}, which are, for the most part, non-commutative, while abelian groups are usually used as the main building block for \hyperref[def:semiring/ring]{rings} and \hyperref[def:left_module]{modules}.

  To make a further distinction, if the operation is denoted by \( \cdot \) or juxtaposition, we say that the group is a \term{multiplicative group}, and if the operation is denoted by \( + \), we say that the group is an \term{additive group}. This terminology usually, but not necessarily, coincides with the group (or, more generally, the \hyperref[def:magma]{magma}) being \hyperref[def:magma/commutative]{commutative}.

  To make things explicit, a \term{multiplicative magma} is any magma as defined in \fullref{def:magma}. Compare this to \term{additive magmas}, where
  \begin{thmenum}
    \thmitem{rem:additive_magma/addition} The magma operation is denoted by \( + \) and called \term{addition}.

    \thmitem{rem:additive_magma/multiplication} The magma \hyperref[def:magma/exponentiation]{exponentiation operation} is denoted by \( \cdot \) or juxtaposition and called \term{multiplication}. Thus, multiplication is not defined for two elements of the magma, but defined for a positive integer and an element of the magma. In the case of a \hyperref[def:magma/commutative]{commutative} \hyperref[def:unital_magma/monoid]{monoid}, if multiplication is extended to two elements of the monoid, we instead talk about \hyperref[def:semiring]{semirings}.

    \thmitem{rem:additive_magma/identity} The \hyperref[def:magma_identity]{identity} is usually denoted by \( 0 \).

    \thmitem{rem:additive_magma/inverse} If an \hyperref[def:unital_magma_inverse_element]{inverse} of \( x \) exists, it is denoted by \( -x \) rather than \( x^{-1} \).
  \end{thmenum}
\end{remark}

\begin{proposition}\label{thm:power_set_magma_preservation}
  \hyperref[def:magma/associative]{Associativity} and \hyperref[def:magma/commutative]{commutativity} from a magma \( \mscrM \) are preserved in \( \pow(\mscrM) \), unlike \hyperref[def:magma/cancellative]{cancellation}.
\end{proposition}
\begin{proof}
  Associativity and commutativity are obviously preserved.

  To show that cancellation is not, consider the group \hyperref[def:group_of_integers_modulo]{\( \BbbZ_2 \)}. It is a cancellative magma by \fullref{thm:def:group/properties/cancellative}. Define the sets \( A \coloneqq \{ 0, 1 \} \) and \( B \coloneqq \{ 0 \} \). Then
  \begin{equation*}
    A + A = A = A + B,
  \end{equation*}
  however we cannot cancel \( A \) from the left because \( A \neq B \).
\end{proof}

\begin{proposition}\label{thm:magma_exponentiation_properties}
  Fix a magma \( \mscrM \). \hyperref[def:magma/exponentiation]{Magma exponentiation} in \( \mscrM \) has the following basic properties:

  \begin{thmenum}
    \thmitem{thm:magma_exponentiation_properties/commutativity} We have the following \hyperref[def:magma/commutative]{commutativity}-like property: for \( x \in M \) and \( n = 1, 2, \ldots \),
    \begin{equation}\label{eq:thm:magma_exponentiation_properties/commutativity}
      x^n = x x^{n-1} = x^{n-1} x.
    \end{equation}

    \thmitem{thm:magma_exponentiation_properties/distributivity} Exponentiation distributes over multiplication: for any member \( x \in M \) and any two positive integers \( n \) and \( m \),
    \begin{equation}\label{eq:thm:magma_exponentiation_properties/multiplication}
      x^{n + m} = x^n x^m.
    \end{equation}

    \thmitem{thm:magma_exponentiation_properties/repeated} For any member \( x \in M \) and any two positive integers \( n \) and \( m \),
    \begin{equation}\label{eq:thm:magma_exponentiation_properties/repeated}
      (x^n)^m = x^{nm}.
    \end{equation}
  \end{thmenum}
\end{proposition}
\begin{proof}
  \SubProofOf{thm:magma_exponentiation_properties/commutativity} We use induction on \( n \). The cases \( n = 1 \) and \( n = 2 \) are obvious. For \( n > 2 \), we have
  \begin{equation*}
    x^n
    \reloset {\eqref{eq:def:magma/exponentiation}} =
    x x^{n-1}
    \reloset {\T{ind.}} =
    x x^{n-2} x
    \reloset {\eqref{eq:def:magma/exponentiation}} =
    x^{n-1} x.
  \end{equation*}

  \SubProofOf{thm:magma_exponentiation_properties/distributivity} We use induction on \( n \). The case \( n = 1 \) follows directly from \eqref{eq:def:magma/exponentiation}. The case \( n > 1 \) follows from
  \begin{equation*}
    x^{n + m}
    \reloset {\eqref{eq:def:magma/exponentiation}} =
    x x^{n + (m - 1)}
    \reloset {\T{ind.}} =
    x x^{n-1} x^m
    \reloset {\eqref{eq:def:magma/exponentiation}} =
    x^n x^m.
  \end{equation*}

  \SubProofOf{thm:magma_exponentiation_properties/repeated} We use induction on \( n \). The case \( n = 1 \) is obvious and the rest follows from
  \begin{equation*}
    (x^n)^m
    \reloset {\eqref{eq:def:magma/exponentiation}} =
    x^n (x^n)^{m-1}
    \reloset {\T{ind.}} =
    x^n x^{n (m - 1)}
    \reloset {\eqref{eq:thm:magma_exponentiation_properties/multiplication}} =
    =
    x^{nm}.
  \end{equation*}
\end{proof}

\begin{definition}\label{def:preordered_magma}
  A \term{preordered magma} is a magma \( \mscrM \) equipped with a \hyperref[def:preordered_set]{preorder} \( \leq \) such that \( x \leq y \) implies \( xz \leq yz \) and \( zx \leq zy \) for all \( z \in M \).

  The category of small preordered magmas is both \hyperref[def:magma/category]{\( \cat{Mag} \)}-\hyperref[def:concrete_category]{concrete} and \hyperref[def:preordered_set/category]{\( \cat{PreOrd} \)}-concrete.
\end{definition}

\begin{proposition}\label{thm:preordered_magma_max_distributivity}
  In a \hyperref[def:preordered_magma]{preordered magma} \( \mscrM \),
  \begin{equation}\label{eq:thm:preordered_magma_max_distributivity}
    \max \set{a b, c d} \leq \max \set{a, c} \cdot \max \set{b, d}.
  \end{equation}
\end{proposition}
\begin{proof}
  Since \( a \leq \max \set{a, c} \), then
  \begin{equation*}
    ab
    \leq
    \max \set{a, c} \cdot b
    \leq
    \max \set{a, c} \cdot \set{b, d}
  \end{equation*}

  Analogously, \( cd \leq \max \set{a, c} \cdot \set{b, d} \) and
  \begin{equation*}
    \max \set{a b, c d} \leq \max \set{a, c} \cdot \set{b, d}.
  \end{equation*}
\end{proof}

\begin{definition}\label{def:topological_magma}
  A \term{topological magma} is a magma equipped with a \hyperref[def:topological_space]{topology} such that the magma operation is continuous.

  The category of small topological magmas is both \hyperref[def:magma/category]{\( \cat{Mag} \)}-\hyperref[def:concrete_category]{concrete} and \hyperref[def:category_of_small_topological_spaces]{\( \cat{Top} \)}-concrete.
\end{definition}

\begin{definition}\label{def:magma_identity}
  An element \( e \) of a magma \( \mscrM \) is called a \term{left identity} (resp. \term{right identity}) if \( ex = x \) for all \( x \in M \) (resp. \( xe = x \) for all \( x \in M \)).

  If \( e \) is simultaneously a left and right identity, we call a \term{two-sided identity} or simply \term{identity} of \( \mscrM \).
\end{definition}

\begin{proposition}\label{thm:magma_identity_unique}
  A two-sided \hyperref[def:magma_identity]{magma identity} \( e \), if it exists, is unique.
\end{proposition}
\begin{proof}
  If \( f \) is another identity, then \( e = ef = f \).
\end{proof}

\begin{definition}\label{def:unital_magma}
  A \hyperref[def:magma]{magma} with an \hyperref[def:magma_identity]{identity} is called \term{unital}. This makes it a \hyperref[def:pointed_set]{pointed set}. We can consider it as a pair \( (\mscrM, \cdot) \) rather than a triple \( (\mscrM, \cdot, e) \) because, by \fullref{thm:magma_identity_unique}, a two-sided identity is uniquely determined by the magma operation.

  \begin{thmenum}
    \thmitem{def:unital_magma/theory} The theory of unital magmas consists of the single axiom
    \begin{equation}\label{eq:def:unital_magma/theory/identity}
      \qforall \xi (e \cdot \xi \doteq \xi \wedge \xi \cdot e \doteq \xi)
    \end{equation}
    over the combined language of \hyperref[def:pointed_set/theory]{pointed sets} and \hyperref[def:magma/theory]{magmas}.

    \thmitem{def:unital_magma/homomorphism} A \hyperref[def:first_order_homomorphism]{homomorphism} between unital magmas is a function that satisfies both \eqref{eq:def:pointed_set/homomorphism} and \eqref{eq:def:magma/homomorphism}.

    \thmitem{def:unital_magma/category} The \hyperref[def:category_of_small_first_order_models]{category of \( \mscrU \)-small models} \( \ucat{Mag}_* \) of unital magmas is \hyperref[def:concrete_category]{concrete} with respect to both \hyperref[def:pointed_set/category]{\( \ucat{Set}_* \)} and \hyperref[def:magma/category]{\( \ucat{Mag} \)}.

    \thmitem{def:unital_magma/submodel} The set \( S \subseteq X \) is a \hyperref[def:first_order_substructure]{substructure} if \( \mscrX \) if \( e \in S \). This is equivalent to \( S \) being a pointed subset.

    We say that \( S \) is a \term{unital submagma}.

    \thmitem{def:unital_magma/trivial} The \hyperref[thm:substructures_form_complete_lattice/bottom]{trivial} unital magma is the \hyperref[def:pointed_set/trivial]{trivial pointed set} \( \set{ e } \).

    \thmitem{def:unital_magma/monoid} An \hyperref[eq:def:magma/associative]{associative} unital magma is called a \term{monoid}. The category \( \cat{Mon} \) of monoids is a full subcategory of \( \cat{Mag}_* \).

    \thmitem{def:unital_magma/exponentiation} We extend \hyperref[def:magma/exponentiation]{magma exponentiation} to all nonnegative integers by defining
    \begin{equation*}
      x^0 \coloneqq e.
    \end{equation*}

    \thmitem{def:unital_magma/power_set} The \hyperref[def:magma/power_set]{power set magma} \( \pow(\mscrM) \) of a unital magma \( \mscrM \) with identity \( e \) is again a unital magma with identity \( \set{ e } \).
  \end{thmenum}
\end{definition}

\begin{example}\label{ex:monoid_cancellation_not_preserved_by_homomorphism}\mcite{MathSE:magma_cancellation_not_preserved}
  \hyperref[def:unital_magma/homomorphism]{Monoid homomorphisms} may not preserve the \hyperref[def:magma/cancellative]{cancellation property}. For example, the \hyperref[def:set_of_natural_numbers]{natural numbers} \( \BbbN \) are a cancellative monoid under addition, as shown in \fullref{thm:def:natural_number_addition/properties}), but the magma homomorphism
  \begin{equation*}
    \begin{aligned}
      &h: (\BbbN, +) \to (\hyperref[thm:galois_field_existence]{\BbbF_2}, \max) \\
      &h(n) \coloneqq \begin{cases}
        0, &n = 0 \\
        1, &n > 0
      \end{cases}
    \end{aligned}
  \end{equation*}
  does not preserve the cancellative property.

  Indeed, \( \max\set{ 0, 1} = \max\set{ 1, 1 } \), but \( 0 \neq 1 \).
\end{example}

\begin{proposition}\label{thm:unital_magma_kernel_is_submagma}
  The \hyperref[def:pointed_set_kernel]{kernel} of a unital magma homomorphism \( \varphi: \mscrM \to \mscrN \) is a \hyperref[def:first_order_substructure]{unital submagma} of \( \mscrM \).
\end{proposition}
\begin{proof}
  By \eqref{eq:def:pointed_set/homomorphism}, \( e_{\mscrM} \in \ker(\varphi) \), therefore \( \ker(\varphi) \) inherits its unital magma structure from \( \mscrM \). It remains only to show that it is closed under the magma operation. But this is trivial since, if \( x, y \in \ker(\varphi) \), then
  \begin{equation*}
    \varphi(xy) = \varphi(x) \varphi(y) = e_{\mscrN} e_{\mscrN} = e_{\mscrN}.
  \end{equation*}
\end{proof}


\chapter{Mathematical Background}
\section{One-Shot Games}
\subsection{Prisoner's Dilemma}
The Prisoner's Dilemma is the canonical game of game theory, finding its use in models for everything from share prices to the relationship between lions and small birds. Essentially, in any situation where there is the possibility of cooperation, the Prisoner's Dilemma can be a useful model in understanding the situation to a first approximation.
\subsubsection{The Story}
The motivating story goes that two accomplices are caught at a crime scene, arrested and placed in seperate cells with no contact. The police give them each a choice: they can confess their crimes to the authorities (defect) or they can stay silent (cooperate).\\
They learn that if they both cooperate, they only get one year in prison. If they both defect, they both get two years. However, if one of them cooperates and the other defects, the cooperator gets 3 years and the defector gets out of jail immediately. As they are in seperate cells, they cannot communicate.\\
The smallest combined time in prison for the couple is if they both cooperate and take 1 year in jail each. But even if they could communicate and agree to the appealing option of both cooperating, it would still make sense to defect from the `contract' and defect. But just 1 year in jail is clearly better than 2. How should they work through this? As with most things in life, they should get mathematical.
\subsection{Making this Mathematical}
\subsubsection{Key Concepts}
This story has all the features of a game-theoretic game. Let's go through it, pick out the key elements and name them.\\
The four essential elements of a game have a useful acronym: \textit{PAPI} \cite{rasmusen}. This stands for:\\
the \textbf{P}layers of the game,\\
the \textbf{A}ctions available to each player,\\
the \textbf{P}ayoffs for each outcome\footnote{An \textit{outcome} defines an action for each player. It can be written $(C,C)$ for two players both playing the action $(C)$. Note, actions are usually represented as a single letter i.e. $(C)$ for cooperate and $(D)$ for defect. } and\\
the \textbf{I}nformation available to each player.\\

In our story, there are two players: Lou and Avery. The actions are cooperate with the partner in crime (i.e. stay quiet) or defect (i.e. confess and betray partner). The payoffs are given by the years in jail, $-x$ for $x$ years in jail. As they cannot communicate, there is no information available about the other player's choice prior to choosing.
\subsubsection{Representing games}
\subsubsection{Payoff Matrix}
A payoff matrix is a way of defining two-player simultaneous games completely i.e. it tells you about every element of \textit{PAPI}. The payoff matrix for the Prisoner's Dilemma given in the story would be:\\
\setlength{\extrarowheight}{2pt}
\begin{tabular}{cc|c|c|}
	& \multicolumn{1}{c}{} & \multicolumn{2}{c}{Avery}\\
	& \multicolumn{1}{c}{} & \multicolumn{1}{c}{$C$}  & \multicolumn{1}{c}{$D$} \\\cline{3-4}
	\multirow{2}*{Lou}  & $C$ & $(-1,-1)$ & $(-3,0)$ \\\cline{3-4}
	& $D$ & $(0,-3)$ & $(-2,-2)$ \\\cline{3-4}
\end{tabular}
\\
\\
The players, actions and payoffs are defined. Also there is no information available about the other player's action. So this matrix completely describes the game. Note that the the ordered pair $(x,y)$ in each matrix entry gives the payoffs for each outcome. $x$ gives the payoff for the horizontal player and $y$ gives the vertical player's. So outcome $(D,C)$ has payoff $(0,-3)$ meaning a payoff of $0$ for Lou for defecting and a payoff of $-3$ for Avery for cooperating.\\
The payoff matrix is an extremely useful representation, allowing concise, complete descriptions of games. Consider, for example:\\
\setlength{\extrarowheight}{2pt}
\begin{tabular}{cc|c|c|c|}
	& \multicolumn{1}{c}{} & \multicolumn{3}{c}{Avery} \\
	& \multicolumn{1}{c}{} & \multicolumn{1}{c}{$R$}  & \multicolumn{1}{c}{$P$}  & \multicolumn{1}{c}{$S$} \\\cline{3-5}
	& $R$ & $(0,0)$ & $(-1,1)$ & $(1,-1)$ \\ \cline{3-5}
	Lou  & $P$ & $(1,-1)$ & $(0,0)$ & $(-1,1)$ \\\cline{3-5}
	& $S$ & $(-1,1)$ & $(1,-1)$ & $(0,0)$ \\\cline{3-5}
\end{tabular}
\\
\\
This game might initially look foreign, but it is simply Rock, Paper, Scissors. You can create stories to motivate them but the payoff matrix completely defines all the relevant mathematical aspects of the game. This allows us to abstract from the messy details of the human world and find `solutions' to the game.
\subsubsection{Solution Concepts}
A \textit{strategy} is a set of rules for which actions to play at each point of the game. It can respond to the other player's actions.
\subsubsection{Strategies}
\paragraph{Socially Optimal Strategy}Lou is able to recognise the \textit{socially optimal strategy} as the option of both cooperate so that they get only 1 year each. This is the outcome that leads to the highest joint payoff for all players.
\paragraph{Nash Equilibrium} However, the outcome with both defecting is the Nash Equilibrium. This is an outcome in which, even if all players knew what the rest of the players actions were, none of them would choose to change their action as it would not improve their payoff. In this game, both players defecting is the unique Nash Equilibrium.
\paragraph{Dominant Strategy}The outcome of $(D, D)$ actually satisfies a stronger condition as the action defect is a \textit{dominant strategy} for both players. This means that no matter what action the opponent plays, the player can always increase their payoff by defecting. This is the way (defect, defect) `pulls' players in, despite it having a worse payoff for both players than cooperate cooperate. Game theory assumes that players are \textit{rational} (i.e. the payoffs we have described actually match their desires and they are aware of how the game works). So if Lou and Avery are rational, they will always end up both defecting, betraying each other and being rewarded with a longer jail term. So it goes.

\section{Repeated Games}
\subsubsection{Introduction}
The previous games are called \textit{one-shot games} meaning, after the first game is played, no further games are played. It happens once and never again, with no possible repurcussions that are not already described in the payoff matrix.\\

Unlike one-shot games, repeated games have players play multiple games against each other. This means that it is harder or even impossible to `solve' these games to find the best actions and equilibria. However, this makes them more interesting, allowing for complex dynamics. For example, the prisoner's dilemma no longer has a best strategy when repeated.
\subsubsection{Repeated Prisoner's Dilemma Tournament}
We can create a tournament of different strategies all playing several hundred games against each other. The winner is the strategy with the highest payoff overall. This numerical experiment was famously first done by Robert Axelrod\cite{axelrod} in the 1980s. In Axelrod's first tournament, the winner was `Tit for Tat', a strategy that always cooperates on the first round and then copies the opponents previous move forever after.\\
(
My simulation of Prisoner's Dilemma tournament.\\
Include Tit for Tat, Grim Trigger, Random, always cooperate, always defect, tit for two tats etc.\\
brief analysis of my results- graph of results
)

\subsubsection{Repeated Games with Evolution}
[Include replicator equations at some point here.]\\
Incorporating repetitions, allows the possibility of 'evolutionary' behaviour. For example, one can create simulations where higher payoffs are more likely to produce 'offspring' i.e. players that use the same strategy. For example, below is a simulation of strategies playing Rock, Paper Scissors defined by the payoff matrix as above.\\
(
My simulation of Rock, Paper, Scissors
)\\
The effect is a self-balancing system. If the strategy \textit{rock} becomes more populous, \textit{paper} will start to get higher payoffs on average. This in turn brings the population back towards $\frac{1}{3}$ for each strategy.\\
The same scenario works with the admitedlly less well-known game Rock, Paper, Scissors, Spock\cite{for game}\cite{for code}.\\
(
My simulation of Rock, Paper Scissors, Spock
)\\
However, now with the different rules we can have the extinction of strategies.
\section{Games on Graphs}
In real life, games are usually not isolated situations that happen as if in a laboratory. Instead, we take part in many at the same time with several different players. Sometimes the different games effect each other. To model this, we can use graphs.
\subsubsection{Graphs}
A graph $G=(V,E)$ is a set of vertices $V$ and a set of edges $E$ which are pairs of vertices. For now we will consider only undirected ($(u,v)\in E\iff(v,u)\in E$) graphs with no loops ($\nexists v\textnormal{ s.t. } (v,v)\in E$). A graph looks something like this:\\
\begin{figure}
	\centering
	\begin{subfigure}{.3\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{directedGraph.png}
		\caption{Directed graph}
		\label{fig:dir}
	\end{subfigure}%
	\begin{subfigure}{.3\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{loopGraph.png}
		\caption{Graph with loops}
		\label{fig:loop}
	\end{subfigure}
	\begin{subfigure}{.3\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{graph.png}
		\caption{Undirected graph without loops}
		\label{fig:undirected}
	\end{subfigure}
	\caption{We will consider graphs of type \ref{fig:undirected} and ignore the others}
	\label{fig:complete graphs}
\end{figure}
Two vertices $u,v$ are \textit{adjacent} iff $(u,v)\in E$.
\subsubsection{Games on graphs we have already seen}
We can now consider games on graphs by allowing each vertex to represent a player and only allowing players to play a game against players they are connected to. In fact, we have sneakily been doing this the whole time but now we want to make this explicit.\\
The two person one-shot prisoner's dilemma in 1.1.2 was a game on a trivial graph $G=(\{Lou,Avery\},\{(Lou,Avery)\})$.\\
\begin{figure}[h]
	\centering
	\begin{tikzpicture}
	\draw
	(1,1) node[anchor=east,circle,draw]{Lou}--
	(3,1) node[anchor=west,circle,draw]{Avery};
	\end{tikzpicture}
	\caption{The trivial graph underlying the one-shot Prisoner's Dilemma}
\end{figure}
Similarly, for the tournament if we call the players $p_1,...,p_n$, the tournament was a repeated game on a \textit{complete graph}. This is the graph with each vertex connected to everyother vertex: $K_n=\{\{p_1,...,p_n\},\{(p_i,p_j):\forall i\neq j\}\}$\\

\begin{figure}
	\centering
	\begin{subfigure}{.5\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{K5.pdf}
		\caption{$K_5$}
		\label{fig:K5}
	\end{subfigure}%
	\begin{subfigure}{.5\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{K16.pdf}
		\caption{$K_{16}$}
		\label{fig:K16}
	\end{subfigure}
	\caption{The Complete Graphs $K_5$ and $K_{16}$}
	\label{fig:complete graphs}
\end{figure}

\subsubsection{Some new Graphs}
Now we have made the graphs we are playing games on explicit we can try some new ones.\\
A graph often used in models is the 2D lattice as it is both instructive and relatively easy to analyse.\cite{eq_of_life}\\
\begin{figure}
	\centering
	\includegraphics[width=.5\linewidth]{2d_lattice.pdf}
	\caption{A 2D lattice}
\end{figure}
The von Neumann neighbourhood of the 2D lattice (Figure \ref{fig:vonneumann}) makes each point a neighbour of the $4$ points vertically and horizontally next to it.\\
\begin{figure}
	\centering
	\begin{subfigure}{.5\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{vonNeumannNeighbourhood.pdf}
		\caption{The von Neumann Neighbourhood}
		\label{fig:vonneumann}
	\end{subfigure}%
	\begin{subfigure}{.5\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{mooreNeighbourhood.png}
		\caption{The Moore neighbourhood}
		\label{fig:moore}
	\end{subfigure}
	\caption{Different neighbourhoods on the 2D Lattice}
	\label{fig: lattice neighbourhoods}
\end{figure}
The Moore neighbourhood of the 2D lattice (Figure \ref{fig:vonneumann}) includes the nearest vertical and horizontal neighbours as well as the nearest diagonal points.\\
To avoid boundary effects when playing games on these lattices (i.e. the side players having fewer opponents), we can 'wrap' the 2D lattice around itself to create a torus. So each neighbour at the top of the lattice is joined to the neighbour in the same column displayed visually at the bottom of the lattice.
\subsection{Prisoner's Dilemma on a Torus}
\subsubsection{Setup}
We can extend the Prisoner's Dilemma to a repeated game on  2D Lattice.\\
Initialisation:\\
Make an $n \times n$ grid with wrapped ends to avoid boundary effects, creating a torus. Then let each vertex be a player of the Prisoner's Dilemma. In the first round, each player plays $C$ with probability $p$ and $D$ with probability $(1-p)$. They play their strategy simultaneously against every player in their Moore neighbourhood (Figure \ref{fig:moore}), using the same strategy against each of them.\\
Every stage after:\\
After playing the previous round, each player looks at their neighbours scores. They then adopt the strategy of the highest scoring neighbour as their strategy for the next round. Again they play the same strategy against every player in their neighbourhood.
The payoff matrix retains characteristics of the original given Prisoner's Dilemma and is given by:\\
\setlength{\extrarowheight}{2pt}
\begin{tabular}{cc|c|c|}
	& \multicolumn{1}{c}{} & \multicolumn{2}{c}{Lou}\\
	& \multicolumn{1}{c}{} & \multicolumn{1}{c}{$C$}  & \multicolumn{1}{c}{$D$} \\\cline{3-4}
	\multirow{2}*{Avery}  & $C$ & $(1,1)$ & $(\epsilon,b)$ \\\cline{3-4}
	& $D$ & $(b,\epsilon)$ & $(0,0)$ \\\cline{3-4}
\end{tabular}
\\
\\
where $\epsilon<1<b$.
\\

\subsubsection{Example Runs}
In this section we use values of a $100\times100$ grid, $p=0.5$ and $\epsilon=0$. We can create a wide variety of dynamic behaviour, including chaos and bifurcations by adjusting the value of $b$.
\subsubsection{Qualitative Analysis}
For $b>1.\bar{6}$, the board eventually tends to an equilibrium with mostly defectors. Clearly the rewards of non-cooperation are too high to a sustain a more socially optimal situation.
\begin{figure}
	\centering
	\begin{subfigure}{.3\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{/mooreLattice/1b=17.png}
		\caption{Early}
		\label{fig:dir}
	\end{subfigure}%
	\begin{subfigure}{.3\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{/mooreLattice/2b=17.png}
		\caption{Developing}
		\label{fig:loop}
	\end{subfigure}
	\begin{subfigure}{.3\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{/mooreLattice/3b=17.png}
		\caption{Equilibrium}
		\label{fig:undirected}
	\end{subfigure}
	\caption{The simulation running with $b=1.7$}
	\label{fig:complete graphs}
\end{figure}
Similarly, for $b<1.6$, the simulation tends towards a static equilibrium of mainly cooperators.
\begin{figure}
	\centering
	\begin{subfigure}{.3\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{/mooreLattice/1b=15.png}
		\caption{Early}
		\label{fig:dir}
	\end{subfigure}%
	\begin{subfigure}{.3\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{/mooreLattice/2b=15.png}
		\caption{Developing}
		\label{fig:loop}
	\end{subfigure}
	\begin{subfigure}{.3\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{/mooreLattice/3b=15.png}
		\caption{Equilibrium}
		\label{fig:undirected}
	\end{subfigure}
	\caption{The simulation running with $b=1.5$}
	\label{fig:complete graphs}
\end{figure}
Between these two parameter regions, exists the third which exhibits the most interesting behaviour with dynamics between cooperation and defection.
\begin{figure}
	\centering
	\begin{subfigure}{.3\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{/mooreLattice/1b=163.png}
		\caption{Early}
		\label{fig:dir}
	\end{subfigure}%
	\begin{subfigure}{.3\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{/mooreLattice/2b=163.png}
		\caption{Developing}
		\label{fig:loop}
	\end{subfigure}
	\begin{subfigure}{.3\textwidth}
		\centering
		\includegraphics[width=.9\linewidth]{/mooreLattice/3b=163.png}
		\caption{Dynamic equilibrium}
		\label{fig:undirected}
	\end{subfigure}
	\caption{The simulation running with $b=1.63$}
	\label{fig:complete graphs}
\end{figure}
\subsubsection{Quantative Analysis}
looking at individual squares
%\subsubsection{Prisoner's Dilemma on a Scale Free Graph}


\section{On-policy Prediction with Approximation}

\subsection{Exercise 9.1}
\subsubsection*{Q}
Show that tabular methods such as presented in Part I of this book are a special case of linear function approximation. What would the feature vectors be?
\subsubsection*{A}
Write $\hat{V}(s, \vec{w}) = w$ and we get that $\grad_{\vec{w}} \hat{V}(s, \vec{w}) = 1$ so we return to tabular TD learning. In this case the features are $x(s) = 1$ $\forall s \in \mathcal{S}$.

\subsection{Exercise 9.2}
\subsubsection*{Q}
Why does (9.17) define $(n+1)^k$ distinct features for dimension $k$?
\subsubsection*{A}
Each of the $k$ terms can independently have one of $n+1$ exponents, hence the total number of features is $(n+1)^k$.

\subsection{Exercise 9.3}
\subsubsection*{Q}
What $n$ and $c_{i,j}$ produce the feature vectors $\vec{x}(s) = (1, s_1, s_2, s_1s_2, s_1^2, s_2^2, s_1s_2^2, s_1^2 s_2, s_1^2 s_2^2)$?
\subsubsection*{A}
$n=2$ and $c_i, j = C_{ij}$ where
\[
C =
\begin{pmatrix}
    0 & 0 \\
    1 & 0 \\
    0 & 1 \\
    1 & 1 \\
    2 & 0 \\
    0 & 2 \\
    1 & 2 \\
    2 & 1 \\
    2 & 2
\end{pmatrix}
\]

\subsection{Exercise 9.4}
\subsubsection*{Q}
Suppose we believe that one of two state dimensions is more likely to have an effect on the value function than is the other, that generalization should be primarily across this dimension rather than along it. What kind of tilings could be used to take advantage of this prior knowledge?
\subsubsection*{A}
Tiles that are thin along the dimension of interest and long across it. Rectangles, for instance.

\subsection{Exercise 9.5}
\subsubsection*{Q}
Suppose you are using tile coding to transform a seven-dimensional continuous state space into binary feature vectors to estimate a state value function $\hat{v}(s, \vec{w}) \approx v_\pi(s)$. You believe that the dimensions do not interact strongly, so you decide to use eight tilings of each dimension separately (stripe tilings), for $7 \times 8 = 56$ tilings. In addition, in case there are some pairwise interactions between the dimensions, you also take all $\binom{7}{2} = 21$ pairs of dimensions and tile each pair conjunctively with rectangular tiles. You make two tilings for each pair of dimensions, making a grand total of $21 \times 2 + 56 = 98$ tilings. Given these feature vectors, you suspect that you still have to average out some noise, so you decide that you want learning to be gradual, taking about 10 presentations with the same feature vector before learning nears its asymptote. What step-size parameter $\alpha$ should you use? Why?

\subsubsection*{A}
Each tiling is a partition, so each tiling has exactly one tile activated per state. This means that in our case the number of features is 98. We consider each of these equally likely because we are uninformed. We therefore take
\[
\alpha = \frac{1}{10 \times 98} = \frac{1}{980}.
\]
So that on average we see each feature 10 times before asymptote. [Note that this assumes a constant target.]


\hypertarget{classnumpp_1_1differentiation_1_1symbolic_1_1sinus}{}\section{numpp\+:\+:differentiation\+:\+:symbolic\+:\+:sinus$<$ T $>$ Class Template Reference}
\label{classnumpp_1_1differentiation_1_1symbolic_1_1sinus}\index{numpp\+::differentiation\+::symbolic\+::sinus$<$ T $>$@{numpp\+::differentiation\+::symbolic\+::sinus$<$ T $>$}}
\subsection*{Public Types}
\begin{DoxyCompactItemize}
\item
\mbox{\Hypertarget{classnumpp_1_1differentiation_1_1symbolic_1_1sinus_a5bde5934631bcfb972ad6c15c81ccca5}\label{classnumpp_1_1differentiation_1_1symbolic_1_1sinus_a5bde5934631bcfb972ad6c15c81ccca5}}
{\footnotesize template$<$std\+::size\+\_\+t Active$>$ }\\using {\bfseries derivative} = simplify\+\_\+multiplication$<$ \hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1cosinus}{cosinus}$<$ T $>$, typename T\+::template derivative$<$ Active $>$ $>$
\end{DoxyCompactItemize}
\subsection*{Static Public Member Functions}
\begin{DoxyCompactItemize}
\item
\mbox{\Hypertarget{classnumpp_1_1differentiation_1_1symbolic_1_1sinus_a25872983c585075c533169136769caa7}\label{classnumpp_1_1differentiation_1_1symbolic_1_1sinus_a25872983c585075c533169136769caa7}}
static C\+O\+N\+S\+T\+E\+X\+PR auto {\bfseries calculate} (auto \&\&values)
\end{DoxyCompactItemize}

The documentation for this class was generated from the following file\+:\begin{DoxyCompactItemize}
\item
differentiation/symbolic/trigonometric.\+hpp\end{DoxyCompactItemize}


%%=========================================================================
%% Section 3.4: Boundedness of the Exponentially Weighted Hilbert Transform
%%=========================================================================

\documentclass[../dissertation.tex]{subfiles}

\begin{document}
\section{Boundedness of the Exponentially Weighted Hilbert Transform}\label{sec3:BndE}

Proving that the exponentially weighted Hilbert transform $E$ is bounded on
$L^p(\mathbb R)$ for $1< p < \infty$ is a multistep process in which we first compute
the Fourier multiplier of $E$ (Lemma \ref{lma:FMultE}), use the Fourier multiplier
of $E$ to show that $E$ is strong type $(2,2)$ (Theorem \ref{thm:Estrong}), prove $E$ is
weak type $(1, 1)$ (Theorem \ref{thm:Eweak}), and then use these results to ultimately
prove $E$ is strong type $(p, p)$ for $p>1$ (Theorem \ref{thm:EbndOp}). Note that an
operator is said to be weak type $(p,q)$ if it is a bounded operator from $L^p$ to
$weak$-$L^q$ (\textit{i.e.} $L^{q, \infty}$) and strong type $(p,q)$ if it
is a bounded operator from $L^p$ to $L^q$. For a defintion of the space
$L^{p,\infty}$, please see the \hyperref[app:HA]{``Harmonic Analysis Results''}
appendix.

\begin{lma}\label{lma:FMultE}
	The Exponentially weighted Hilbert transform $E$ has Fourier multiplier
	$$m_E(\xi) = \frac{1}{\pi} \arctan(\xi/\pi).$$\label{sym3:Emult}
\end{lma}
\begin{proof}
	In order to compute the Fourier multiplier of $E$, it suffices to compute the Fourier
	transform of the function $x^{-1} e^{-\pi|x|}$, which is $2\pi i$ times the
	convolution kernel of $E$. Letting $\theta$ denote the Heaviside function, we
	write $x^{-1} e^{-\pi|x|}$ as
	\begin{align}\label{eq:Ekern}
		\frac{e^{\pi |x|}}{x}
			=
			\frac{e^{\pi \, x}}{x} \theta(-x) + \frac{e^{-\pi \, x}}{x} \theta(x),
	\end{align}
	and compute the Fourier transform of two terms on the right-hand side of
	\eqref{eq:Ekern} separately. By further splitting $\frac{e^{\pi \, x}}{x} \theta(-x)$
	into even and odd parts, we obtain
	\begin{align}
		\mc F\left( \frac{e^{\pi (\dotarg)}}{(\dotarg)} \theta(-\dotarg) \right)
			&= \mc F^{(c)} \left(
					\frac{e^{\pi (\dotarg)}\theta(-\dotarg)-e^{-\pi(\dotarg)}\theta(\dotarg)}
						{2(\dotarg)}
				\right)(\xi)
				\\
			&\quad + i \, \mc F^{(s)} \left(
					\frac{e^{\pi(\dotarg)}\theta(-\dotarg)+e^{-\pi(\dotarg)}\theta(\dotarg)}
						{2(\dotarg)}
				\right)(\xi),
				\nonumber
	\end{align}
	where $\mc F^{(c)}$ and $\mc F^{(s)}$ respectively denote the Fourier cosine and
	Fourier sine transformations. Direct computation yields
	\begin{align*}
		\mc F^{(c)} \left(
			\frac{e^{\pi (\dotarg)}\theta(-\dotarg)-e^{-\pi(\dotarg)}\theta(\dotarg)}
				{2(\dotarg)}
		\right)(\xi)
			&= \frac{1}{\pi}
				\int_0^\infty
					\frac{e^{\pi \,x}\theta(-x)-e^{-\pi\,x}\theta(x)}{2x}
				\, \cos(\xi x) \, \mathrm{d}x
				\\
			&= \frac{1}{2} \big(
					\log(\xi^2 + \pi^2) + 2 \gamma
				\big)
				\nonumber
	\end{align*}
	and
	\begin{align*}
		\mc F^{(s)} \left(
			\frac{e^{\pi (\dotarg)}\theta(-\dotarg)+e^{-\pi(\dotarg)}\theta(\dotarg)}
				{2(\dotarg)}
		\right)(\xi)
			&= \frac{1}{2\pi}
				\int_0^\infty
					\frac{e^{\pi \,x}\theta(-x)+e^{-\pi\,x}\theta(x)}{2x}
				\, \sin(\xi x) \, \mathrm{d}x
				\\
			&= \frac{1}{2} \big(
					\arctan(\xi/\pi)
				\big),
				\nonumber
	\end{align*}
	where $\gamma$ denotes the Euler-Mascheroni constant, which implies
	\begin{align}
		\mc F \left(
			 \frac{e^{\pi (\dotarg)}}{(\dotarg)} \theta(-\dotarg)
		\right)(\xi)
			&=
				\gamma
				+ \frac{1}{2} \log(\xi^2 + \pi^2)
				+ i \, \arctan(\xi/\pi).
	\end{align}
	A similar computation also shows
	\begin{align}
		\mc F \left(
			 \frac{e^{-\pi (\dotarg)}}{(\dotarg)} \theta(\dotarg)
		\right)(\xi)
			&=
				- \gamma
				- \frac{1}{2} \log(\xi^2 + \pi^2)
				+ i \, \arctan(\xi/\pi),
	\end{align}
	from which the result follows.
\end{proof}

\begin{thm}\label{thm:Estrong}
	$E$ is strong type $(2, 2)$.
\end{thm}
\begin{proof}
	This result is an immediate consequence of Lemma \ref{lma:FMultE},
	Plancherel's Theorem, and the density of $\mathscr S(\mathbb R)$
	in $L^2(\mathbb R)$.
\end{proof}

\begin{thm}\label{thm:Eweak}
	$E$ is weak type $(1, 1)$. That is, $E$ is a bounded operator from
    $L^1(\mathbb R)$ into
    $L^{1, \infty}(\mathbb R)$, where  $L^{1, \infty}$ denotes weak $L^1$.
    \label{sym3:weakLp}
\end{thm}
\begin{proof}
    Fix $\Lambda > 0$, let $f$ be Schwartz class, and assume without loss of
    generality that $f \in \mathscr S(\mathbb R)$ is real-valued and nonnegative
    (otherwise, we can decompose $f$ in the appropriate pieces).
    Let $\{ I_j \}$ be the sequence sequence of dyadic intervals in the
    Calder\'on-Zygmund decomposition of
    $f$ at height $\Lambda$. For $\Omega := \bigcup_j I_j$, define
    \begin{align}\label{eq:C-Zdecomp}
        g(x) :=
            \begin{cases}
                \displaystyle f(x), & x \notin \Omega \\[0.5\baselineskip]
                \displaystyle \frac{1}{|I_j|} \int_{I_j} f, & x \in I_j
            \end{cases},
        \qquad \text{and} \qquad
        b(x) := \sum_j b_j(x),
    \end{align}
    where
    \[
        b_j(x) = \left( f(x) - \dfrac{1}{|I_j|} \int_{I_j} f \right) \chi_{I_j}(x).
    \]
    Note that $f = g+b$ and
    \begin{align} \label{eq:GoodPartBnd}
        g(x) \leq 2 \Lambda \quad \forall x \in \mathbb R.
    \end{align}
    To show that $Eg$ and $Eb$ are well defined, it suffices
    to bound $d_{Eg}(\Lambda)$ and $d_{Eb}(\Lambda)$ in terms of only $\Lambda$ and
    $\|f\|_1$, where
    the notation
    \[
        d_h (\Lambda) := \big| \big\{ x \in \mathbb R \, : \, |f(x)| > \Lambda \big\} \big|
    \]
    is used to denote the \textit{distributional function} of a function $h$. Using
    Chebyshev's inequality, the $L^2$  boundedness of $E$ found in Theorem \ref{thm:Eweak}
    and equations \eqref{eq:GoodPartBnd} and \eqref{eq:C-Zdecomp}, we find
    \begin{align}\label{eq:dEgBound}
        d_{Eg}(\Lambda)
            &\leq \frac{1}{\Lambda^2} \int_{\mathbb R} E g(x)^2 \, \mathrm{d}x \\
            &\leq \frac{C}{\Lambda^2} \int_{\mathbb R} g(x)^2 \, \mathrm{d}x \nonumber\\
            &\leq \frac{C}{\Lambda^2} \int_{\mathbb R} g(x) \, \mathrm{d}x \nonumber\\
            &= \frac{C}{\Lambda^2} \left( \int_{\Omega} g
                + \int_{\mathbb{R} \backslash \Omega} g  \right) \nonumber\\
            &\leq \frac{C}{\Lambda} \| f \|_1.  \nonumber
    \end{align}

    On the other hand, for $Eb$, let $\Omega^* = \bigcup_j 2 I_j$. Using the
    Calder\'on-Zygmund Covering Lemma and Chebyshev's inequality we find
    \begin{align} \label{eq:dEb}
        d_{Eb}(\Lambda)
            \leq |\Omega^*| + |\{ x \notin \Omega^* \, : \, |Eb(x)| > \Lambda \}|
            \leq \frac{2}{\Lambda} \|f\|_1 + \frac{1}{\Lambda}
                \int_{\mathbb R \backslash \Omega^*} |Eb(x)| \, \mathrm{d}x.
    \end{align}
    So, to show that $Eb$ is well defined, we need to bound the integral on the
    right hand side of the above inequality by $\|f\|_1$. To that end, note that
    if $x \notin \Omega^*$, then for each $j$, $x \notin 2 I_j$ and
    \begin{align*}
        E b_j(x)
            = \frac{1}{2\pi}
            	p.v. \int_{\mathbb R}
            		\frac{e^{-\pi|x - x'|}}{x - x'} b_j(x')
            	\, \mathrm{d}x'
            = \frac{1}{2\pi}
            	\int_{I_j}
            		\frac{e^{-\pi|x - x'|}}{x - x'} b_j(x')
            	\, \mathrm{d}x
            < \infty,
    \end{align*}
    as $\text{supp} \, b_j \subseteq I_j$. Since $E$ is a tempered distribution
    (Lemma \ref{lma:FMultE}) and
    $f \in \mathscr S(\mathbb R)$ means that $Ef \in L^2(\mathbb R)$ and, hence
    $\sum_j E b_j$ converges to $Eb$ in the $L^2$ norm, it follows that
    \[
        |Eb(x)| \leq \sum_j |Eb_j(x)| \quad a.e.
    \]
    As such, proving $Eb$ is well defined reduces to showing
    \begin{align}
        \int_{\mathbb R \backslash \Omega^*} \sum_{j} |E b_j(x)| \, \mathrm{d}x
            \leq C \| f \|_1.
    \end{align}

    If we let $c_j$ denote the center of $I_j$, then, for $x \notin \Omega^*$,
    since $b_j$ has zero average
    \begin{align*}
        \left|\int_{I_j} \frac{e^{-\pi|x - x'|}}{x - x'} b_j(x') \, \mathrm{d}x'\right|
            &= \left|\int_{I_j} e^{-\pi|x - x'|}
                \left(\frac{b_j(x')}{x - x'} - \frac{b_j(x')}{x - c_j}  \right)\, \mathrm{d}x'\right| \\
            &\leq \int_{I_j} e^{-\pi|x - x'|}
                \left|\frac{b_j(x')(x' - c_j)}{(x - x')(x-c_j)}\right| \, \mathrm{d}x' \\
            &\leq \int_{I_j} |b_j(x')| \frac{|I_j|}{(x - c_j)^2} \, \mathrm{d}x'
    \end{align*}
    as $|x - x'| \geq |x - c_j|/2$ and $|x' - c_j| \leq |I_j|/2$. Moreover,
    \begin{align} \label{eq:IjIntBnd}
        \int_{\mathbb R \backslash \Omega^*} \frac{|I_j|}{(x - c_j)^2} \, \mathrm{d}x'
            \leq \int_{\mathbb R \backslash I_j} \frac{|I_j|}{(x - c_j)^2} \, \mathrm{d}x'
            \leq 4,
    \end{align}
    and so, by Fubini's Theorem,
    \begin{align} \label{eq:sumEbj}
        \sum_j \int_{\mathbb R \backslash \Omega^*} \left| E b_j(x) \right| \, \mathrm{d}x
            &\leq \frac{1}{2\pi} \sum_j\int_{\mathbb R \backslash \Omega^*}
                \left| \int_{I_j} e^{-\pi|x-x'|} \frac{b_j(x')}{x - x'} \, \mathrm{d}x' \right| \, \mathrm{d}x
                \\
            &\leq \frac{1}{2\pi} \sum_j \int_{\mathbb R \backslash \Omega^*}  \int_{I_j}
                |b_j(x')| \frac{|I_j|}{(x - c_j)^2} \, \mathrm{d}x' \, \mathrm{d}x
                \nonumber \\
            &\leq \frac{2}{\pi} \sum_j \int_{I_j} |b_j(x')| \, \mathrm{d}x'
                \nonumber \\
            &\leq \frac{4}{\pi} \| f\|_1. \nonumber
    \end{align}
    Putting everything together, we find
    \begin{align} \label{eq:E11bnd}
        d_{Ef}(\Lambda)
            \leq d_{Eg}\left(\frac{\Lambda}{2}\right) + d_{Eb}\left(\frac{\Lambda}{2}\right)
            \leq \frac{C}{\Lambda} \| f\|_1,
    \end{align}
    where $C>0$ is independent of $\Lambda$ and $f$. Since $f$ is Schwartz class, we can
    therefore extend the inequality \eqref{eq:E11bnd} to $L^1$ \textit{via} density to
    conclude that $E$ is weak (1,1).
\end{proof}

\begin{thm}\label{thm:EbndOp}
	The operator $E$ is strong type $(p, p)$ for $p>1$.
\end{thm}
\begin{proof}
	Theorems \ref{thm:Estrong} and \ref{thm:Eweak} in conjunction with the
	Marcinkiewicz Interpolation Theorem \cite[Theorem 1.3.2]{Grafakos} immediately imply that $E$ is strong
	type $(p,p)$ for $p\in (1, 2]$. As such, it remains only to show that
	$E$ is strong type $(p, p)$ for $p > 2$.

    Let $E'$ denote the adjoint of $E$. By density, we need only consider
    $f \in \mathscr S(\mathbb R)$. Fix $p >2$ and let $q$
    denote its H\"older Conjugate. Note that the map
    \[
        g \mapsto \int_{\mathbb R} E f \bar{g} =: \left\langle Ef, g \right\rangle
    \]
    is a linear functional on $L^q$ with norm $\| E f \|_p$. As such,
    we see by H\"older's inequality that
    \begin{align*}
        \| E f \|_p
            &= \sup_{\|g\|_q = 1} |\langle Ef, g \rangle| \\
            &= \sup_{\|g\|_q = 1} |\langle f, E'g \rangle| \\
            &\leq  \|f\|_p \|E\|_q \\
            &\leq  C \|f\|_p,
    \end{align*}
    as $p>2$ implies $1 < q < 2$, which means $E$ is strong type $(q, q)$,
    and Theorem 5 from Chapter VII.3 of \cite{yosida} implies
    $\|E\|_q = \|E'\|_q$.
\end{proof}

\end{document}


% !TEX root = pythontex_subfile.tex

\chapter{Python\TeX{} integration}

\section{Console}

Use the \texttt{pyconsole} environment to interactively show results in a Python console.

\begin{pyconsole}
1+2
\end{pyconsole}
Other programming languages are supported, check the Python\TeX{} documentation.

\section{Automatic figure generation}

\Cref{fig:automatic-figure} is automatically built from \texttt{create\_figure.py} and rendered using \LaTeX.

\begin{pycode}
import matplotlib.pyplot as plt
import matplotlib as mpl

from scripts.example.create_figure import create_figure
pytex.add_dependencies("scripts/example/create_figure.py")

mpl.rcParams.update({
    "font.size":           pytex.context.normalsize,
    "figure.subplot.left": 0,
    "figure.subplot.right":0.86,
    "figure.figsize":      (pytex.pt_to_in(pytex.context['textwidth']),
                            0.37*pytex.pt_to_in(pytex.context['textwidth']))
})

info = create_figure("../data/winequality-red.csv")
pytex.add_dependencies("../data/winequality-red.csv")

plt.savefig("figs/test.pgf")
pytex.add_created("figs/test.pgf")
\end{pycode}

\begin{figure}[htbp]
    \centering
    \inputsafe{figs/test.pgf}
    \caption{Figure automatically generated from \texttt{create\_figure.py}}
    \label{fig:automatic-figure}
\end{figure}

Use the \texttt{pysub} environment to get values from Python to \LaTeX.
\begin{pysub}
Here $a= \num{ !{info["coeff"]} }$ and $b = \num{ !{info["intercept"]}}$.
\end{pysub}
Here is an equation that we would like to reference in the plot:
\begin{equation}
    y = a x + b.
    \label{eq:test}
\end{equation}

\section{Automatic table generation}

The following table is generated automatically from \texttt{create\_table.py} using Python\TeX{} integration.

\begin{pycode}
from scripts.example.create_table import create_table
pytex.add_dependencies("scripts/example/create_table.py")

info, _ = create_table(None,
                       column_format="RR",
                       width=r"0.5\\linewidth",
                       caption=r"Table automatically generated from \texttt{create\_table.py}",
                       label="tab:test-table",
                       buf="tables/test.tex",
                       show_index=False,
                       na_rep='na')
pytex.add_created("tables/test.tex")
\end{pycode}

\inputsafe{tables/test}


%---------------------------Shear and Size-----------------------------
\section{Shear and Size\label{s:quad-shear-and-size}}

Let $R$ be the relative size squared as defined in \S\ref{s:quad-rel-size-squared}
and $H$ be the shear as defined in \S\ref{s:quad-shear}.
The shear and size metric is the product of these two numbers:
\[
q = RH
\]

\quadmetrictable{shear and size}%
{$1$}%                                      Dimension
{$[0.2,1]$}%                                Acceptable range
{$[0,1]$}%                                  Normal range
{$[0,1]$}%                                  Full range
{Dependent on $\overline{A}$}%              Unit square
{\cite{knu:03}}%                            Citation
{v\_quad\_shear\_and\_size}%                Verdict function name


\sec{Cauchy Euler equations}
\subsection{The first challenge} \label{ssec:cauchyeulerhomo}
\begin{mdframed}[style=boxstyle, frametitle={The Setup}]
	An ODE of the form
	\begin{equation*}
		x^ny^{(n)} + a_{n-1}x^{n-1}y^{(n-1)} + \cdots + a_{1}xy' + a_0y = 0
	\end{equation*}
	is called a \defin{Cauchy Euler equation}. (Each $a_i$ is a constant.)
\end{mdframed}
We shall be interested in solving the above equation only on $(0, \infty).$
\begin{mdframed}[style=boxstyle, frametitle={The Solution}]
	We plug the trial solution $y = x^m.$ This gives us the polynomial equation:
	\begin{align}
		m(m - 1)\cdots(m-(n-1)) + a_{n-1}m(m-1)\cdots(m-(n-2)) + \nonumber \\
		 \cdots + a_1m + a_0 = 0. \label{eq:indic}
	\end{align}
	Note that if $m$ is a root of the above, then $x^m$ is a solution of the ODE.\\
	Thus, if the above equation has $n$ distinct roots $m_1, \ldots, m_n,$ then the original ODE has $n$ linearly independent solutions $x^{m_1}, \ldots, x^{m_n}$ and we are done.
\end{mdframed}
\begin{mdframed}[style=boxstyle, frametitle={Repeated roots}]
	Suppose $m_0$ is a repeated root of (\ref{eq:indic}). Suppose that it is repeated $k$ times. Then, one can show that the following $k$ functions are solutions of the original ODE:
	\begin{equation*}
		x^{m_0}, x^{m_0}\ln x, \ldots, x^{m_0}(\ln x)^{k-1}.
	\end{equation*}
	These are linearly independent as well.\\
	Thus, we are now done for any case as the total number of roots is always going to be $n$ when counted with multiplicity. (This is assuming that we work in $\mathbb{C},$ which we shall do.)
\end{mdframed}
\begin{mdframed}[style=boxstyle, frametitle={Getting real}]
	Suppose $m = a+ib$ is a root. Then, we have
	\begin{align*}
		x^m &:= \exp(m \ln x) & (\text{By definition})\\
		&= \exp(a \ln x + ib\ln x)\\
		&= \exp(a \ln x)\cdot\exp(ib \ln x)\\
		&= \exp(a \ln x)\cdot\big(\cos(b\ln x) + i\sin(b \ln x)\big)\\
		&=x^a(\cos(b\ln x) + i\sin(b \ln x))
	\end{align*}
	As before, if each $a_i$ is real, then the roots appear in the same multiplicity as their conjugates which will give us the real pair of solutions as:
	\begin{equation*}
		x^a\cos(b\ln x), x^a\sin(b\ln x).
	\end{equation*}
	In case of repetition twice, we get the four solutions:
	\begin{equation*}
		x^a\cos(b\ln x), x^a\sin(b\ln x),\quad x^a(\ln x)\cos(b\ln x), x^a(\ln x)\sin(b\ln x).
	\end{equation*}
	The general case is (hopefully) clear.
\end{mdframed}
\subsection{Welcome back, annihilators}
This time, we consider annihilators of polynomial (polylogmial?) functions.\\
As before, we shall see that it suffices to consider the case of just the monomials first. The annihilator table is particularly simple this time:
\begin{mdframed}[style=boxstyle, frametitle={Annihilators of special functions}]
	\begin{center}
	\bgroup
	\def\arraystretch{1.25}
	\begin{tabular}{|l|l|}
		\hline
		Function & Annihilator\\
		\hline
		$x^n$ & $xD - n$\\
		$x^n(\ln x)^k$ & $(xD - n)^{k+1}$\\
		\hline
	\end{tabular}
	\egroup
	\end{center}
\end{mdframed}
\begin{mdframed}[style=boxstyle, frametitle={A word on $xD$}]
	Note that $xD$ is the operator which acts on a function as:
	\begin{equation*}
		(xD)(f) = xf'.
	\end{equation*}
	On the other hand, $Dx$ is an operator which acts as:
	\begin{equation*}
		(Dx)(f) = D(xf) = xf' + f.
	\end{equation*}
	In particular, $xD \neq Dx.$ This means that $(xD)^2 \neq x^2D^2$ and so on.
\end{mdframed}
\emph{Note:} $xD$ is an operator but there is no $xP$ operator. \hfill (A joke.)
\begin{mdframed}[style=boxstyle, frametitle={Some arithmetic}]
	For ease of calculations, one may note the following useful identities:
	\begin{align*}
		x^2D^2 &= xD(xD - 1)\\
		x^3D^3 &= xD(xD - 1)(xD - 2)\\
		&\vdots
	\end{align*}
\end{mdframed}
\subsection{The main problem}
\begin{mdframed}[style=boxstyle, frametitle={The Setup}]
	We consider a linear ODE of the form
	\begin{equation}
		x^ny^{(n)} + a_{n-1}x^{n-1}y^{(n-1)} + \cdots + a_{1}xy' + a_0y = Q(x),
	\end{equation}
	where each $a_i$ is a constant and $Q(x)$ is a linear combination of functions of the form $x^k(\ln x)^m$.
\end{mdframed}
\begin{mdframed}[style=boxstyle, frametitle={The Solution}]
	The steps are now identical as the case of \nameref{sec:undcoeff}.\\
	We shall assume that $Q(x) = x^k(\ln x)^m$ and not a linear combination. The general case follows as before by considering the linear combination of solutions.
	\begin{enumerate}[leftmargin=*, label = \Roman*.]
		\item We first consider the equation
		\begin{equation*}
			x^ny^{(n)} + a_{n-1}x^{n-1}y^{(n-1)} + \cdots + a_{1}xy' + a_0y = 0.
		\end{equation*}
		This can be solved by the method we saw in \S\S\ref{ssec:cauchyeulerhomo}.
		\item We then apply the annihilator of $Q(x)$ from the table earlier.\\
		\textbf{\emph{This again gives us a Cauchy Euler equation.}}\\
		We solve this again using the method of \S\S\ref{ssec:cauchyeulerhomo}.
		\item We follow the step III as in the case of \nameref{sec:undcoeff} to determine the coefficients of the new functions we obtain in Step II.
	\end{enumerate}
\end{mdframed}
Note that $D^{11}$ is also an annihilator of $x^{10},$ however we do not use that.\\
The first reason is that the calculations would be a nightmare.\\
Secondly, applying this annihilator to both sides of the equation wouldn't technically give us a Cauchy Euler equation again. \\
Also, note that the method we have used doesn't require $k$ to be an integer. This would work even if we wish to solve something like $xy'' + y = x^{1/2}.$ Another reason to prefer the annihilator $xD - 1/2.$