 \documentclass [12pt]{article}

\usepackage {amsmath}
\usepackage {amsthm}
\usepackage {amssymb}
\usepackage {graphicx}
\usepackage {float}
\usepackage {multirow}
\usepackage {xcolor}
\usepackage {algorithmic}
\usepackage [ruled,vlined,commentsnumbered,titlenotnumbered]{algorithm2e} \usepackage {array}
\usepackage {booktabs}
\usepackage {url}
\usepackage {parskip}
\usepackage [margin=1in]{geometry}
\usepackage [T1]{fontenc}
\usepackage {cmbright}
\usepackage [many]{tcolorbox}
\usepackage [colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage {enumitem}
\usepackage {xparse}
\usepackage {verbatim}
\usepackage{listings}
\usepackage{xcolor}
\lstset { %
    language=C++,
    backgroundcolor=\color{black!5}, % set backgroundcolor
    basicstyle=\footnotesize,% basic font setting
}
\newtheorem{theorem}{Theorem}
\newtheorem{remark}{Remark}

\DeclareTColorBox {Solution}{}{breakable, title={Solution}} \DeclareTColorBox {Solution*}{}{breakable, title={Solution (provided)}} \DeclareTColorBox {Instruction}{}{boxrule=0pt, boxsep=0pt, left=0.5em, right=0.5em, top=0.5em, bottom=0.5em, arc=0pt, toprule=1pt, bottomrule=1pt} \DeclareDocumentCommand {\Expecting }{+m}{\textbf {[We are expecting:} #1\textbf {]}} \DeclareDocumentCommand {\Points }{m}{\textbf {(#1 pt.)}}

\begin {document}

\vspace {1em}
\begin {Instruction}
Adapted From Virginia Williams' lecture notes.
\end {Instruction}

{\LARGE \textbf {COMP 285 (NC A\&T, Spr `22)}\hfill \textbf {Lecture 12} }

\begin{centering}
\section*{Data Structures: Heaps and Red-Black Trees}
\end{centering}

\section{Data Structures Review}

Thus far in this course we have mainly discussed algorithm design, and have specified algorithms at a relatively high level. For example, in describing sorting algorithms, we often assumed that we could insert numbers into a list in constant time; we didn’t worry too much about the actual implementation of this, and took it as an assumption that this could actually be implemented (e.g. via a linked-list).

We will wrap-up this lecture by talking about the actual design and implementation of some useful datastructures. The purpose is mainly to give you a taste of data-structure design. We won’t discuss this area too much more in the course, and will switch back to discussing higher-level algorithms.

To motivate the data structures that we will discuss in this course, consider the following table that lists a bunch of basic operations that we would like to perform on a set/list of numbers, together with the worst-case runtime of performing those operation for two of the data structures that you are (hopefully) familiar with: linked lists of n (unsorted) numbers, and an array of n sorted numbers. In this course, we will assume that all the numbers we store are unique.

\begin{center}
\begin{tabular}{|c|p{2.4cm}|c|}
\hline \\
Operation & Runtime with Unsorted Linked Lists & Runtime with Sorted Array \\
\hline \\
Search (ie, find 17) & $\Theta(n)$ & $\Theta(\log n)$(via binary search) \\
Select (ie, find 12 smallest element) & $\Theta(n)$ & $\Theta(1)$ (just look at the 12th element) \\
Rank (ie, \# elt. less than 17) & $\Theta(n)$ & $\Theta(\log n)$\\
Predecessor/Successor & $\Theta(n)$ & $\Theta(1)$ \\
Insert element & $\Theta(1)$ & $\Theta(n)$ \\
Delete element & $\Theta(1)$ & $\Theta(n)$ \\
\hline
\end{tabular}
\end{center}

The above table makes Sorted Arrays look like a pretty decent data structure for \textit{static} data. In many applications, however, the data changes often, in which case the $\Theta(n)$ time it takes to insert or delete an element is prohibitive. This prompts the question: \textit{Is it possible to have one data structure that is the best of both worlds (logarithmic or constant time for all these operations)?} The answer is, essentially, ``Yes'', provided we don’t mind if some of the $\Theta(1)$’s turn into $\Theta(log n)$.

We will sketch out the Binary Search Tree. Before talking about binary search trees, we sketch another data structure called Heap. Heaps are worse than both unsorted linked lists and sorted arrays in just about any parameter, except that they can allow for efficient ($O(\log n)$ time) insertions - still worse than linked lists. But there is one operation that Heaps do exceptionally well (again, $\Theta(\log n)$): \texttt{extract-min}, which outputs the minimum element, and then deletes it from the heap. Heaps (in their simple form that we consider here) still don't beat the asymptotic runtime of binary search trees for any operation, but they have two other advantages: they are much simpler, and if you only want to run insert and \texttt{extract-min} they are also much faster in practice.

\section{Heaps} \textbf{Definition} (Complete binary tree). A complete binary tree is a rooted binary tree where each level is full except maybe the last level, and all nodes on the last level are as far left as they can be.

\textbf{Definition} (Binary min-heap).

A \textit{Binary min-heap} is a data structure that stores elements that have keys from a totally ordered universe (say, the integers). A binary min-heap supports the following operations:

\begin{itemize}
  \item \texttt{insert}(i): Inserts an element with key $i$ into the data structure
  \item \texttt{extract-min}: Returns the element with the minimum key and deletes it from the data structure.
\end{itemize}

A binary heap stores elements in a complete binary tree with a root $r$. Each node $x$ has key($x$) (the key of the element stored in $x$), p($x$) (the parent of $x$, where p($r$) = NIL), left($x$) (the left child of $x$), and right($x$) (the right child of $x$). The children of $x$ are either other nodes or NIL.

A binary heap should satisfy two key properties: The first key property is that for every node $x$, the keys of all nodes under $x$ are greater than key($x$). The second key property of a heap is that its tree is complete (see 1).

As explained above, binary min-heaps (which we call heaps for short) support two operations3 : \texttt{insert}(i) and \texttt{extract-min}. These are particularly useful if you want to have a priority queue where elements (e.g. jobs) arrive in an arbitrary order, but always leave in order of their key/priority (e.g. when the highest priority job is executed first).

\subsection{Basic Operations on Heaps}
In the heap data structure we always maintain a pointer to the last node in the heap (the right most node in the last level), as well as a pointer to the next node to be created (typically the one to the right of the last node; except when the last level is full, in which case this is the left-most node in the next level). Maintaining those is easy when we think of the
representation of a heap as an array. But to make things simpler, we will abstract these details and simply assume access to these pointers. (See CLRS for details.)

\subsubsection{\texttt{insert}(i)}
To insert an element, we create a new node with key $i$ at the bottom of the heap. Then recursively “propagate it up the heap” until the heap property is restored: That is compare i to the key of the new element's parent; if $i$ is smaller than the key of its parent, replace the keys, and continue to compare with the parent's parent, etc. Once the algorithm reaches a parent whose key is smaller, the insertion is complete.

\begin{algorithm}
\caption{Heap \texttt{insert}(i)}\label{alg:heap_insert}
\begin{algorithmic}
\STATE $x \gets $ first node without two children
\STATE $y \gets$ new node with key$(y) \gets i$, left$(y) \gets $ NIL, right$(y) \gets $ NIL, $p(y)\gets x$
\IF {$left(x) == NIL$}
  \STATE left$(x) \gets y$
\ELSE
  \STATE right$(x) \gets y$
\ENDIF
\WHILE {$p(y) \neq \text{NIL} \& \text{key}(p(y)) > \text{key}(y)$}
  \STATE $v \gets \text{key}(y)$
  \STATE $\text{key}(y) \gets \text{key}(p(y)) $
  \STATE $\text{key}(p(y)) \gets v$
  \STATE $y \gets p(y)$
\ENDWHILE
\end{algorithmic}
\end{algorithm}

\subsubsection{\texttt{extract-min}}
To extract-min, we save the key of the root, replace it with the key of the last node, and delete the last node. Then we recursively ``propagate the key copied from the last node down the tree''. That is compare to the children of the root; if the key of the root is larger than the keys of one of the children, swap the smaller key up, and recurse on the corresponding sub-tree. Once the algorithm reaches children whose keys are larger, the heap property is restored. Then we can return the value we saved at the beginning.

\begin{algorithm}
\caption{Heap \texttt{extract-min}(i)}
\label{alg:heap_extract}
\begin{algorithmic}
\STATE $x \gets $ last node
\STATE $r \gets$ root, $m \gets$ key($r$)
\STATE key($r) \gets $ key($x$)
\IF {right($p(x)$) == NIL}
  \STATE left($p(x)) \gets$ NIL
\ELSE
  \STATE right($p(x)) \gets$ NIL
\ENDIF
delete node($x$)
\WHILE {True}
  \IF {left($r$) == NIL}
    \RETURN m
  \ELSIF{key(left($r$)) $<$ key($r$) \& right($r$) == NIL}
    \STATE $v \gets \text{key}(r)$
    \STATE $\text{key}(r) \gets \text{key}(\text{left}(r))$
    \STATE $\text{key}(\text{left}(r)) \gets v$
    \STATE $r \gets \text{left}(r)$
  \ELSIF{key(left($r$)) $<$ key($r$), key(righ($r$))}
    \STATE $v \gets \text{key}(r)$
    \STATE $\text{key}(r) \gets \text{key}(\text{left}(r))$
    \STATE $\text{key}(\text{left}(r)) \gets v$
    \STATE $r \gets \text{left}(r)$
  \ELSIF{key(right($r$)) $<$ key($r$), key(left($r$))}
    \STATE $v \gets \text{key}(r)$
    \STATE $\text{key}(r) \gets \text{key}(\text{right}(r))$
    \STATE $\text{key}(\text{right}(r)) \gets v$
    \STATE $r \gets \text{right}(r)$
  \ELSE
    \RETURN m
  \ENDIF
\ENDWHILE
\RETURN m

\end{algorithmic}
\end{algorithm}

\section{Binary Search Trees}
\textbf{Definition} (Binary search tree).

A binary search tree (BST) is a data structure that stores elements that have keys from a totally ordered universe (say, the integers). In this lecture, we will assume that each element has a unique key. A BST supports the following operations:

\begin{itemize}
  \item search(i): Returns an element in the data structure associated with key i
  \item insert(i): Inserts an element with key i into the data structure
  \item delete(i): Deletes an element with key i from the data structure, if such an element exists
\end{itemize}

A BST stores the elements in a binary tree with a root $r$. Each node $x$ has key($x$) (the key of the element stored in $x$), $p(x)$ (the parent of $x$, where $p(r ) = NIL$), $left(x$) (the left child of $x$), and right($x$) (the right child of $x$). The children of $x$ are either other nodes or NIL.

The key BST property is that for every node $x$, the keys of all nodes under left($x$) are less than key($x$) and the keys of all nodes under right($x$) are greater than key($x$).

\textit{Example 4}. In the following example, the root node $r$ stores $3$ in it (key($r) = 3$), its left child left($x$) stores 1, its right child right($x$) stores 7, and all leaf nodes (storing 1, 4, and 8, respectively) have NIL as their two children.

\begin{center}
\begin{tikzpicture}[scale=0.15]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (25.7,-16.1) circle (3);
\draw (25.7,-16.1) node {$3$};
\draw [black] (25.7,-29.4) circle (3);
\draw (25.7,-29.4) node {$4$};
\draw [black] (31.4,-22.2) circle (3);
\draw (31.4,-22.2) node {$7$};
\draw [black] (19.9,-22.2) circle (3);
\draw (19.9,-22.2) node {$1$};
\draw [black] (36.7,-29.4) circle (3);
\draw (36.7,-29.4) node {$8$};
\draw [black] (23.63,-18.27) -- (21.97,-20.03);
\fill [black] (21.97,-20.03) -- (22.88,-19.79) -- (22.16,-19.1);
\draw [black] (27.75,-18.29) -- (29.35,-20.01);
\fill [black] (29.35,-20.01) -- (29.17,-19.08) -- (28.44,-19.76);
\draw [black] (29.3,-24.4) -- (27.45,-26.97);
\fill [black] (27.45,-26.97) -- (28.33,-26.61) -- (27.51,-26.02);
\draw [black] (33.18,-24.62) -- (34.92,-26.98);
\fill [black] (34.92,-26.98) -- (34.85,-26.04) -- (34.04,-26.64);
\end{tikzpicture}
\end{center}

Example 5. The following binary tree is not a BST since 2 > 3 and 2 is a child of 7, which is the right child of 3:

\begin{center}
\begin{tikzpicture}[scale=0.15]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (25.7,-16.1) circle (3);
\draw (25.7,-16.1) node {$3$};
\draw [black] (25.7,-29.4) circle (3);
\draw (25.7,-29.4) node {$2$};
\draw [black] (31.4,-22.2) circle (3);
\draw (31.4,-22.2) node {$7$};
\draw [black] (19.9,-22.2) circle (3);
\draw (19.9,-22.2) node {$1$};
\draw [black] (36.7,-29.4) circle (3);
\draw (36.7,-29.4) node {$8$};
\draw [black] (23.63,-18.27) -- (21.97,-20.03);
\fill [black] (21.97,-20.03) -- (22.88,-19.79) -- (22.16,-19.1);
\draw [black] (27.75,-18.29) -- (29.35,-20.01);
\fill [black] (29.35,-20.01) -- (29.17,-19.08) -- (28.44,-19.76);
\draw [black] (29.3,-24.4) -- (27.45,-26.97);
\fill [black] (27.45,-26.97) -- (28.33,-26.61) -- (27.51,-26.02);
\draw [black] (33.18,-24.62) -- (34.92,-26.98);
\fill [black] (34.92,-26.98) -- (34.85,-26.04) -- (34.04,-26.64);
\end{tikzpicture}
\end{center}

\textbf{Some properties.} \textit{Relationship to Quicksort: We can think of each node $x$ as a pivot for quicksort for the keys in its subtree. The left subtree contains $A_<$ for key$(x)$ and the right subtree contains $A_>$ for key($x$).}

\textit{Sorting the keys}: We can do an \textit{inorder} traversal of the tree to recover the nodes in sorted order from left to right (the smallest element is in the leftmost node and the largest element is in the rightmost node). The Inorder procedure takes a node $x$ and returns the keys in the subtree under $x$ in sorted order. We can recursively define Inorder($x$) as: (1) If left$(x) \neq$  NIL, then run Inorder(left$(x)$), then: (2) Output key$(x)$ and then: (3) If right$(x) \neq$ NIL, run Inorder(right$(x)$). With this approach, for every $x$, all keys in its left subtree will be output
before $x$, then $x$ will be output and then every element in its right subtree.

\textit{Subtree property}: If we have a subtree where $x$ has $y$ as a left child, $y$ has $z$ as a right child, and $z$ is the root for the subtree $T_z$ , then our BST property implies that all keys in $T_z$ are $> y $ and $< x$. Similarly, if we have a subtree where $x$ has $y$ as a right child, $y$ has $z$ as a left child, and $z$ is the root of the subtree $T_z$ , then our BST property implies that all keys in $T_z$ are between $x$ and $y$.

\subsection{Basic Operations of BSTs}
The three core operations on a BST are search, insert, and delete. For this lecture, we will assume that the BST stores distinct numbers, i.e. we will identify the objects with their names and we will have each name be represented by a number.

\subsubsection{search}
To search for an element, we start at the root and compare the key of the node we are looking at to the element we are searching for. If the node's key matches, then we are done. If not, we recursively search in the left or right subtree of our node depending on whether this node was too large or too small, respectively. If we ever reach NIL, we know the element does not exist in our BST. In the following algorithm in this case, we simply return the node that would be the parent of this node if we inserted it into our tree.

\begin{algorithm}
\caption{\texttt{search}(i)}\label{alg:bst_search}
\begin{algorithmic}
\RETURN search(root, i)
\end{algorithmic}
\end{algorithm}

\begin{algorithm}
\caption{\texttt{search}(x, i)}\label{alg:bst_search_substree}
\begin{algorithmic}
\IF {key(x) == i}
  \RETURN $x$
\ELSIF {key(x) < i}
  \IF {right(x) == NIL}
    \RETURN $x$
  \ENDIF
  \RETURN search(right(x), i)
\ELSIF {key(x) > i}
  \IF {left(x) == NIL}
    \RETURN $x$
  \ENDIF
  \RETURN search(left(x), i)
\ENDIF
\end{algorithmic}
\end{algorithm}

\textit{Example 6}. If we call \texttt{search}(5.5) on the following BST, we will return the node storing 5 (by taking the path $4 \to 6 \to 5$). This also corresponds to the path taken when calling \texttt{search}(4.5) or
\texttt{search}(5).

\begin{center}
\begin{tikzpicture}[scale=0.15]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (25.2,-16) circle (3);
\draw (25.2,-16) node {$4$};
\draw [black] (17.6,-25.2) circle (3);
\draw (17.6,-25.2) node {$3$};
\draw [black] (12.4,-34.4) circle (3);
\draw (12.4,-34.4) node {$1$};
\draw [black] (20.6,-34.4) circle (3);
\draw (20.6,-34.4) node {$2$};
\draw [black] (32.9,-25.2) circle (3);
\draw (32.9,-25.2) node {$6$};
\draw [black] (38.2,-34.4) circle (3);
\draw (38.2,-34.4) node {$8$};
\draw [black] (29.1,-34.4) circle (3);
\draw (29.1,-34.4) node {$5$};
\draw [black] (42.5,-43.4) circle (3);
\draw (42.5,-43.4) node {$7$};
\draw [black] (23.29,-18.31) -- (19.51,-22.89);
\fill [black] (19.51,-22.89) -- (20.41,-22.59) -- (19.63,-21.95);
\draw [black] (27.13,-18.3) -- (30.97,-22.9);
\fill [black] (30.97,-22.9) -- (30.84,-21.97) -- (30.08,-22.61);
\draw [black] (16.12,-27.81) -- (13.88,-31.79);
\fill [black] (13.88,-31.79) -- (14.71,-31.34) -- (13.83,-30.85);
\draw [black] (18.53,-28.05) -- (19.67,-31.55);
\fill [black] (19.67,-31.55) -- (19.9,-30.63) -- (18.95,-30.94);
\draw [black] (31.75,-27.97) -- (30.25,-31.63);
\fill [black] (30.25,-31.63) -- (31.01,-31.08) -- (30.09,-30.7);
\draw [black] (34.4,-27.8) -- (36.7,-31.8);
\fill [black] (36.7,-31.8) -- (36.74,-30.86) -- (35.87,-31.36);
\draw [black] (39.49,-37.11) -- (41.21,-40.69);
\fill [black] (41.21,-40.69) -- (41.31,-39.76) -- (40.41,-40.19);
\end{tikzpicture}
\end{center}

\textbf{Claim}. \texttt{search}(i) returns a node containing $i$ if $i \in$ BST, otherwise a node $x$ such that either key($x$) is the smallest in BST > $i$ or the largest in BST < $i$, where node $x$ happens to be the parent of the new node if it were to be inserted into BST. This follows directly from
the BST property. Try to formally prove this claim as an exercise.

\begin{center}
\begin{tikzpicture}[scale=0.15]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (25.2,-16) circle (3);
\draw (25.2,-16) node {$5$};
\draw [black] (18.5,-24.1) circle (3);
\draw (18.5,-24.1) node {$2$};
\draw [black] (23.29,-18.31) -- (20.41,-21.79);
\fill [black] (20.41,-21.79) -- (21.31,-21.49) -- (20.54,-20.85);
\end{tikzpicture}
\end{center}

\textit{Remark 8}. \texttt{search}(i) does not necessarily return the node in BST that is closest in value. Consider the tree above. If we \texttt{search}for an element with a key of 4, the node with key 2 is returned, whereas the element with key 5 is the closest element by value

\subsubsection{insert}
As before, we will assume that all keys are distinct. We will search(i) for a node $x$ to be the parent and create a new node y , placing it as a child of $x$ where it would logically go according to the BST property.

\begin{algorithm}
\caption{\texttt{insert}(i)}\label{alg:bst_inert}
\begin{algorithmic}
\STATE $x \gets $ search(i)
\STATE $y \gets$ new node with key$(y) \gets i$, left$(y) \gets$ NIL, right($y) \gets$ NIL, $p(y) \gets x$
\IF {i < key(x)}
  \STATE left$(x) \gets y$
\ELSE
  \STATE right$(x) \gets y$
\ENDIF
\end{algorithmic}
\end{algorithm}

\textit{Remark 9}. Notice that $x$ needed to have NIL as a child where we want to put $y$ by the properties of our search algorithm

\subsubsection{delete}
Deletion is a bit more complicated. To delete a node $x$ that exists in our tree, we consider several cases:
\begin{enumerate}
  \item If $x$ has no children, we simply remove it by modifying its parent to replace $x$ with NIL.
  \item If $x$ has only one child c, either left or right, then we elevate c to take $x$'s position in the tree by modifying the appropriate pointer of $x$'s parent to replace $x$ with c, and also fixing c's parent pointer to be $x$'s parent.
  \item  If $x$ has two children, a left child $c_1$ and right child $c_2$, then we find $x$'s immediate successor $z$ and have $z$ take $x$'s position in the tree. Notice that $z$ is in the subtree under $x$'s right child $c_2$ and we can find it by running $z \gets \texttt{search}(c_2, key(x))$. Note that since $z$ is $x$'s successor, it doesn't have a left child, but it might have a right child. If $z$ has a right child, then we make $z$'s parent point to that child instead of z (also fixing the child's parent pointer). Then we replace $x$ with z, fixing up all relevant pointers: the rest of $x$'s original right subtree becomes z's new right subtree, and $x$'s left subtree becomes z's new left subtree. (Note that alternatively, we could have used $x$'s immediate predecessor y and followed the same analysis in a mirrored fashion.)

\end{enumerate}
In the following algorithm, if $p$ is the parent of $x$, $child(p)$ refers to $left(p)$ if $x$ was the left child of $p$ and to $right(p)$ otherwise.

\begin{algorithm}
\caption{\texttt{delete}(i)}\label{alg:bst_delete}
\begin{algorithmic}
\STATE $x \gets $ search(i)
\IF {key(x) $\neq i$}
  \RETURN
\ENDIF
\IF {NIL = left(x) and NIL = right(x)}
  \STATE child(p(x)) $\gets$ NIL
  \STATE delete-node(x)
\ENDIF
\IF {NIL = left(x)}
  \STATE $y \gets $ right(x)
  \STATE $p(y) \gets p(x)$
  \STATE $child(p(y)) \gets y$
  \STATE delete-node(x)
\ELSIF {NIL = right(x)}
  \STATE $y \gets $ right(x)
  \STATE $p(y) \gets p(x)$
  \STATE $child(p(y)) \gets y$
  \STATE delete-node(x)
\ELSE
  \STATE $z \gets $ search(right(x), key(x))
  \STATE $z' \gets$ right(z)
  \STATE left($p(z)) \gets z'$
  \STATE $p(z') \gets p(z)$
  \STATE replace $x$ with $z$
  \STATE delete-node(x)
\ENDIF
\end{algorithmic}
\end{algorithm}

\subsubsection{Runtimes}
The worst-case runtime for search is $O(\text{height of tree})$. As both insert and delete call search a constant number of times (once or twice) and otherwise perform $O(1)$ work on top of that, their runtimes are also $O(\text{height of tree})$. In the best case, the height of the tree is $O(\log n)$, e.g., when the tree is completely balanced. However, in the worst case it can be $O(n)$ (a long rightward path, for example). This could happen because insert can increase the height of the tree by 1 every time it is called. Currently our operations do not guarantee logarithmic runtimes. To get $O(\log n)$ height we would need to rebalance our tree. There are many examples of self-balancing BSTs, including AVL trees, red-black trees, splay trees (somewhat different but super cool!), etc. Today, we will talk about red-black trees.

\section{Red-Black Trees}
One of the most popular balanced BST is the red-black tree developed by Guibas and Sedgewick in 1978. In a red-black tree, all leaves are assumed to have NILs as children.

\textbf{Definition}. A red-black tree is a BST with the following additional properties:

\begin{enumerate}
  \item Every node is red or black
  \item The root is black
  \item NILs are black
  \item The children of a red node are black
  \item For every node $x$, all $x$ to NIL paths have the same number of black nodes on them
\end{enumerate}

\textit{Example}  (B means a node is black, R means a node is red.) The following tree is not a red-black tree since property 5 is not satisfied:

\begin{center}
\begin{tikzpicture}[scale=0.1]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (25.2,-16) circle (3);
\draw (25.2,-16) node {$B$};
\draw [black] (18.5,-24.1) circle (3);
\draw (18.5,-24.1) node {$B$};
\draw [black] (30.5,-24.1) circle (3);
\draw (30.5,-24.1) node {$B$};
\draw [black] (25.2,-33.5) circle (3);
\draw (25.2,-33.5) node {$B$};
\draw [black] (23.29,-18.31) -- (20.41,-21.79);
\fill [black] (20.41,-21.79) -- (21.31,-21.49) -- (20.54,-20.85);
\draw [black] (26.84,-18.51) -- (28.86,-21.59);
\fill [black] (28.86,-21.59) -- (28.84,-20.65) -- (28,-21.19);
\draw [black] (29.03,-26.71) -- (26.67,-30.89);
\fill [black] (26.67,-30.89) -- (27.5,-30.44) -- (26.63,-29.94);
\end{tikzpicture}
\end{center}

\textit{Example}. (B means a node is black, R means a node is red.) The following tree is a
red-black tree since all the properties are satisfied:

\begin{center}
\begin{tikzpicture}[scale=0.1]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (25.2,-16) circle (3);
\draw (25.2,-16) node {$B$};
\draw [black] (18.5,-24.1) circle (3);
\draw (18.5,-24.1) node {$B$};
\draw [black] (30.5,-24.1) circle (3);
\draw (30.5,-24.1) node {$B$};
\draw [black] (25.2,-33.5) circle (3);
\draw (25.2,-33.5) node {$R$};
\draw [black] (23.29,-18.31) -- (20.41,-21.79);
\fill [black] (20.41,-21.79) -- (21.31,-21.49) -- (20.54,-20.85);
\draw [black] (26.84,-18.51) -- (28.86,-21.59);
\fill [black] (28.86,-21.59) -- (28.84,-20.65) -- (28,-21.19);
\draw [black] (29.03,-26.71) -- (26.67,-30.89);
\fill [black] (26.67,-30.89) -- (27.5,-30.44) -- (26.63,-29.94);
\end{tikzpicture}
\end{center}

\textit{Example}. (B means a node is black, R means a node is red.) The following tree is not a
red-black tree since property 4 is not satisfied:

\begin{center}
\begin{tikzpicture}[scale=0.1]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (28.3,-15.5) circle (3);
\draw (28.3,-15.5) node {$B$};
\draw [black] (19.8,-23) circle (3);
\draw (19.8,-23) node {$B$};
\draw [black] (37.7,-23) circle (3);
\draw (37.7,-23) node {$R$};
\draw [black] (33,-30.6) circle (3);
\draw (33,-30.6) node {$R$};
\draw [black] (28.3,-38.2) circle (3);
\draw (28.3,-38.2) node {$R$};
\draw [black] (26.05,-17.48) -- (22.05,-21.02);
\fill [black] (22.05,-21.02) -- (22.98,-20.86) -- (22.32,-20.11);
\draw [black] (30.65,-17.37) -- (35.35,-21.13);
\fill [black] (35.35,-21.13) -- (35.04,-20.24) -- (34.42,-21.02);
\draw [black] (36.12,-25.55) -- (34.58,-28.05);
\fill [black] (34.58,-28.05) -- (35.42,-27.63) -- (34.57,-27.11);
\draw [black] (31.42,-33.15) -- (29.88,-35.65);
\fill [black] (29.88,-35.65) -- (30.72,-35.23) -- (29.87,-34.71);
\end{tikzpicture}
\end{center}

\textit{Example}. (B means a node is black, R means a node is red.) The following tree is a
red-black tree:

\begin{center}
\begin{tikzpicture}[scale=0.1]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (28.3,-15.5) circle (3);
\draw (28.3,-15.5) node {$B$};
\draw [black] (19.8,-23) circle (3);
\draw (19.8,-23) node {$R$};
\draw [black] (37.7,-23) circle (3);
\draw (37.7,-23) node {$R$};
\draw [black] (33,-30.6) circle (3);
\draw (33,-30.6) node {$B$};
\draw [black] (11.6,-30.2) circle (3);
\draw (11.6,-30.2) node {$B$};
\draw [black] (23.9,-30.2) circle (3);
\draw (23.9,-30.2) node {$B$};
\draw [black] (43.4,-30.1) circle (3);
\draw (43.4,-30.1) node {$B$};
\draw [black] (48.5,-37.4) circle (3);
\draw (48.5,-37.4) node {$R$};
\draw [black] (26.05,-17.48) -- (22.05,-21.02);
\fill [black] (22.05,-21.02) -- (22.98,-20.86) -- (22.32,-20.11);
\draw [black] (30.65,-17.37) -- (35.35,-21.13);
\fill [black] (35.35,-21.13) -- (35.04,-20.24) -- (34.42,-21.02);
\draw [black] (36.12,-25.55) -- (34.58,-28.05);
\fill [black] (34.58,-28.05) -- (35.42,-27.63) -- (34.57,-27.11);
\draw [black] (17.55,-24.98) -- (13.85,-28.22);
\fill [black] (13.85,-28.22) -- (14.79,-28.07) -- (14.13,-27.32);
\draw [black] (21.28,-25.61) -- (22.42,-27.59);
\fill [black] (22.42,-27.59) -- (22.45,-26.65) -- (21.59,-27.15);
\draw [black] (39.58,-25.34) -- (41.52,-27.76);
\fill [black] (41.52,-27.76) -- (41.41,-26.82) -- (40.63,-27.45);
\draw [black] (45.12,-32.56) -- (46.78,-34.94);
\fill [black] (46.78,-34.94) -- (46.73,-34) -- (45.91,-34.57);
\end{tikzpicture}
\end{center}

\textbf{Claim} \textit{Any valid red-black tree on $n$ nodes (non-NIL) has height $\leq 2 \log_2(n + 1) = O(\log n)$.}

\textit{Proof} For some node $x$, let $b(x)$ be the ``black height'' of $x$, which is the number of black nodes on a $x \to$ NIL path excluding $x$. We first show that the number of non-NIL descendants of $x$ is at least $2^
{b(x)} - 1$ (including $x$) via induction on the height of $x$.

Base case: NIL node has $b(x) = 0$ and $2^{0}- 1 = 0$ non-NIL descendants. \checkmark

For our inductive step, let $d(x)$ be the number of non-NIL descendants of $x$. Then
\begin{align*}
  d(x) &= 1 + d(\text{left}(x)) + d(\text{right}(x)) \\
  &\geq 1 + (2^{b(x) - 1} - 1) + (2^{b(x) - 1} - 1) \tag{by induction} \\
  &= 2^{b(x)} - 1 \checkmark
\end{align*}

Notice that $b(x) \geq \frac{h(x)}{2}$ (where $h(x)$ is the height of $x$) since on any root to NIL path there are no two consecutive red nodes, so the number of black nodes is at least the number of red nodes, and hence the black height is at least half of the height. We apply this and the above inequality to the root $r$ (letting $h = h(r )$) to obtain $n \geq 2^{b(r)} - 1 \geq 2^{\frac{h}{2}} - 1$, and hence $h \leq 2 log(n + 1)$. Here is some intuition on why the tree is roughly balanced.

\textit{Intuition}: By Property (5) of a red-black tree, all $r \to$ NIL paths have $b(r)$ black nodes (excluding the root). Therefore, all these paths have length $\geq b(r)$. However, they also have length $\leq 2 \cdot b(r)$: by Property (4), the number of red nodes is limited to half of the path, since every red node must be followed by a black node, and hence the number of black nodes is at least half of the length of the path. Hence, the lengths of all paths from $r$ to a NIL are within a factor of $2$ of each other, and the tree must be reasonably balanced. Today we will take a brief look at how the red-black tree properties are maintained. Our coverage here is detailed, but not comprehensive, and meant as a case study. For complete coverage, please refer to Chapter 13 of CLRS.

\subsection{Rotations} Red-black trees, as do other balanced BSTs, use a concept called rotation. A tree rotation restructures the tree shape locally, usually for the purpose of balancing the tree better. A rotation preserves the BST property (as shown in the following two diagrams). Notably, tree rotations can be performed in $O(1)$ time.

\begin{center}
\begin{tikzpicture}[scale=0.15]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (28.3,-15.5) circle (3);
\draw (28.3,-15.5) node {$x$};
\draw [black] (19.8,-23) circle (3);
\draw (19.8,-23) node {$y$};
\draw [black] (37.7,-23) circle (3);
\draw (37.7,-23) node {$\gamma$};
\draw [black] (11.6,-30.2) circle (3);
\draw (11.6,-30.2) node {$\alpha$};
\draw [black] (23.9,-30.2) circle (3);
\draw (23.9,-30.2) node {$\beta$};
\draw [black] (26.05,-17.48) -- (22.05,-21.02);
\fill [black] (22.05,-21.02) -- (22.98,-20.86) -- (22.32,-20.11);
\draw [black] (30.65,-17.37) -- (35.35,-21.13);
\fill [black] (35.35,-21.13) -- (35.04,-20.24) -- (34.42,-21.02);
\draw [black] (17.55,-24.98) -- (13.85,-28.22);
\fill [black] (13.85,-28.22) -- (14.79,-28.07) -- (14.13,-27.32);
\draw [black] (21.28,-25.61) -- (22.42,-27.59);
\fill [black] (22.42,-27.59) -- (22.45,-26.65) -- (21.59,-27.15);
\end{tikzpicture}
\end{center}

\begin{center}
\begin{tikzpicture}[scale=0.15]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (28.3,-15.5) circle (3);
\draw (28.3,-15.5) node {$y$};
\draw [black] (19.8,-23) circle (3);
\draw (19.8,-23) node {$\alpha$};
\draw [black] (37.7,-23) circle (3);
\draw (37.7,-23) node {$x$};
\draw [black] (31.6,-31.9) circle (3);
\draw (31.6,-31.9) node {$\beta$};
\draw [black] (43.6,-31.9) circle (3);
\draw (43.6,-31.9) node {$\gamma$};
\draw [black] (26.05,-17.48) -- (22.05,-21.02);
\fill [black] (22.05,-21.02) -- (22.98,-20.86) -- (22.32,-20.11);
\draw [black] (30.65,-17.37) -- (35.35,-21.13);
\fill [black] (35.35,-21.13) -- (35.04,-20.24) -- (34.42,-21.02);
\draw [black] (36,-25.47) -- (33.3,-29.43);
\fill [black] (33.3,-29.43) -- (34.16,-29.05) -- (33.34,-28.48);
\draw [black] (39.36,-25.5) -- (41.94,-29.4);
\fill [black] (41.94,-29.4) -- (41.92,-28.46) -- (41.08,-29.01);
\end{tikzpicture}
\end{center}

Moving from the first tree to the second is known as a right rotation of $x$. The other direction (from the second tree to the first) is a left rotation of $y$. Notice that we only move the $\beta$ subtree, which is why we preserve the BST property.

\subsection{Insertion in a Red-Black Tree}
Let's see how we can perform Insert(i) on a red-black tree while still maintaining all of its properties. The process for inserting a new node is initially similar to that of insertion into any BST.

\begin{algorithm}
\caption{\texttt{insert\_rb}(i)}\label{alg:insert_rb}
\begin{algorithmic}
\STATE $x \gets $ search(i)
\STATE $y \gets$ new node with key$(y) \gets i$, left$(y) \gets$ NIL, right($y) \gets$ NIL, $p(y) \gets x$
\IF {i < key(x)}
  \STATE left$(x) \gets y$
\ELSE
  \STATE right$(x) \gets y$
\ENDIF
\STATE color(x) $\gets $ red
\STATE recolor if needed
\end{algorithmic}
\end{algorithm}

Note that when $x$ is inserted as a red node: Property (1) is satisfied, as we colored the new node red; Property (2) is satisfied, as we did not touch the root; Property (3) is satisfied, as we can color the new NILs black; and Property (5) is satisfied, as we did not change the number of black nodes in the tree. Thus, the only invariant we have to worry about is Property (4), that red nodes have black children.

The recoloring step is broken down into multiple cases. We consider each of them:

Case 1: p is black. In this case, Property (4) is also maintained. So, we simply add $x$ as a new red child of p, and the red-black tree properties are maintained.

\begin{center}
\begin{tikzpicture}[scale=0.2]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (28.3,-15.5) circle (3);
\draw (28.3,-15.5) node {$p\mbox{ }(B)$};
\draw [black] (19.7,-23) circle (3);
\draw (19.7,-23) node {$x\mbox{ }(R)$};
\draw [black] (26.04,-17.47) -- (21.96,-21.03);
\fill [black] (21.96,-21.03) -- (22.89,-20.88) -- (22.24,-20.13);
\end{tikzpicture}
\end{center}

Case 2: p is red, and $x$'s uncle $u$ is red. In this case, we insert a red $x$, change p and $u$ to black, and change p' to red. Because we switched the colors of two nodes on each of these paths (one red $\to$ black and one black $\to$ red), the number of black nodes on each path is unchanged, so Property (5) remains unchanged. If the parent of p' , p'', is black, then Property (4) is maintained. Otherwise, if p'' is red and breaks Property (4) by introducing a ``double-red'' pair of nodes (p' and p''), then we have to recolor recursively starting at p'.

\begin{center}
\begin{tikzpicture}[scale=0.2]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (28.3,-15.5) circle (3);
\draw (28.3,-15.5) node {$p'\mbox{ }(B)$};
\draw [black] (19.7,-23) circle (3);
\draw (19.7,-23) node {$p\mbox{ }(R)$};
\draw [black] (16,-31.6) circle (3);
\draw (16,-31.6) node {$x\mbox{ }(R)$};
\draw [black] (24.6,-31.2) circle (3);
\draw (24.6,-31.2) node {$...$};
\draw [black] (36.5,-22.3) circle (3);
\draw (36.5,-22.3) node {$u\mbox{ }(R)$};
\draw [black] (26.04,-17.47) -- (21.96,-21.03);
\fill [black] (21.96,-21.03) -- (22.89,-20.88) -- (22.24,-20.13);
\draw [black] (18.51,-25.76) -- (17.19,-28.84);
\fill [black] (17.19,-28.84) -- (17.96,-28.31) -- (17.04,-27.91);
\draw [black] (21.24,-25.58) -- (23.06,-28.62);
\fill [black] (23.06,-28.62) -- (23.08,-27.68) -- (22.22,-28.19);
\draw [black] (30.61,-17.42) -- (34.19,-20.38);
\fill [black] (34.19,-20.38) -- (33.89,-19.49) -- (33.26,-20.26);
\end{tikzpicture}
\end{center} becomes \begin{center}
\begin{tikzpicture}[scale=0.2]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (28.3,-15.5) circle (3);
\draw (28.3,-15.5) node {$p'\mbox{ }(R)$};
\draw [black] (19.7,-23) circle (3);
\draw (19.7,-23) node {$p\mbox{ }(B)$};
\draw [black] (16,-31.6) circle (3);
\draw (16,-31.6) node {$x\mbox{ }(R)$};
\draw [black] (24.6,-31.2) circle (3);
\draw (24.6,-31.2) node {$...$};
\draw [black] (36.5,-22.3) circle (3);
\draw (36.5,-22.3) node {$u\mbox{ }(B)$};
\draw [black] (26.04,-17.47) -- (21.96,-21.03);
\fill [black] (21.96,-21.03) -- (22.89,-20.88) -- (22.24,-20.13);
\draw [black] (18.51,-25.76) -- (17.19,-28.84);
\fill [black] (17.19,-28.84) -- (17.96,-28.31) -- (17.04,-27.91);
\draw [black] (21.24,-25.58) -- (23.06,-28.62);
\fill [black] (23.06,-28.62) -- (23.08,-27.68) -- (22.22,-28.19);
\draw [black] (30.61,-17.42) -- (34.19,-20.38);
\fill [black] (34.19,-20.38) -- (33.89,-19.49) -- (33.26,-20.26);
\end{tikzpicture}
\end{center}

Case 3: p is red, and $u$ is black. There are two possibilities here:

\begin{enumerate}
  \item We are inserting $x$ as a leaf node. Then, $u$ must be NIL for the red-black tree to have been valid before inserting $x$. We insert a red $x$.
  \item We are not inserting $x$; rather, we are recoloring the tree at $x$ from the recursive call in Case 2. We aim to recolor the tree and maintain the number of black nodes on each path from the root to NIL. Note that in this case, $x$ actually has nodes under it.
\end{enumerate}

In both cases, $x$ is red, so we make p black, make p' red, and do a right rotation at p' . We can see that this also maintains the same number of black nodes on each path from the root to NIL, and satisfies Property (4) below p because the original tree was a red-black tree.

\begin{center}
\begin{tikzpicture}[scale=0.2]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (28.3,-15.5) circle (3);
\draw (28.3,-15.5) node {$p'\mbox{ }(B)$};
\draw [black] (19.7,-23) circle (3);
\draw (19.7,-23) node {$p\mbox{ }(R)$};
\draw [black] (16,-31.6) circle (3);
\draw (16,-31.6) node {$x\mbox{ }(R)$};
\draw [black] (24.6,-31.2) circle (3);
\draw (24.6,-31.2) node {$...$};
\draw [black] (36.5,-22.3) circle (3);
\draw (36.5,-22.3) node {$u\mbox{ }(B)$};
\draw [black] (26.04,-17.47) -- (21.96,-21.03);
\fill [black] (21.96,-21.03) -- (22.89,-20.88) -- (22.24,-20.13);
\draw [black] (18.51,-25.76) -- (17.19,-28.84);
\fill [black] (17.19,-28.84) -- (17.96,-28.31) -- (17.04,-27.91);
\draw [black] (21.24,-25.58) -- (23.06,-28.62);
\fill [black] (23.06,-28.62) -- (23.08,-27.68) -- (22.22,-28.19);
\draw [black] (30.61,-17.42) -- (34.19,-20.38);
\fill [black] (34.19,-20.38) -- (33.89,-19.49) -- (33.26,-20.26);
\end{tikzpicture}
\end{center}

becomes

\begin{center}
\begin{tikzpicture}[scale=0.2]
\tikzstyle{every node}+=[inner sep=0pt]
\draw [black] (28.3,-15.5) circle (3);
\draw (28.3,-15.5) node {$p\mbox{ }(B)$};
\draw [black] (19.7,-23) circle (3);
\draw (19.7,-23) node {$x\mbox{ }(R)$};
\draw [black] (36.5,-22.3) circle (3);
\draw (36.5,-22.3) node {$p'(R)$};
\draw [black] (30.8,-29.4) circle (3);
\draw (30.8,-29.4) node {$...$};
\draw [black] (42.4,-29.4) circle (3);
\draw (42.4,-29.4) node {$u\mbox{ }(B)$};
\draw [black] (26.04,-17.47) -- (21.96,-21.03);
\fill [black] (21.96,-21.03) -- (22.89,-20.88) -- (22.24,-20.13);
\draw [black] (30.61,-17.42) -- (34.19,-20.38);
\fill [black] (34.19,-20.38) -- (33.89,-19.49) -- (33.26,-20.26);
\draw [black] (34.62,-24.64) -- (32.68,-27.06);
\fill [black] (32.68,-27.06) -- (33.57,-26.75) -- (32.79,-26.12);
\draw [black] (38.42,-24.61) -- (40.48,-27.09);
\fill [black] (40.48,-27.09) -- (40.36,-26.16) -- (39.59,-26.8);
\end{tikzpicture}
\end{center}

If we end up in Case 2 and recursively call recolor, then in the worst case the recursion will bottom out when we hit the root, with a constant number of relabelings and rotations ateach level. So, it will be an $O(h)$ operation overall, where $h$ is the height of the tree.

In the analysis above, we considered the cases where $x$ is a left child of $p$ and $u$ is a rightchild of its parent $p'$. These cases are representative, showing most of the machinery that we’ll need to insert an arbitrary element into an arbitrary red-black tree. (Within Case 2 and Case 3, there are actually a total of four cases each, where p's tree and p`'s children could each be swapped, but the recoloring procedure is similar. You are encouraged to read the text for details.)

To summarize, the following is the algorithm for recoloring, in the case where $x$ is a left childand $u$ is a right child.

\begin{algorithm}
\caption{\texttt{recolor}(x)}\label{alg:recolor_rb}
\begin{algorithmic}
\STATE $p \gets $ parent(x)
\IF {black = color(p))}
  \RETURN
\ENDIF

\STATE $p' \gets \text{parent}(p)$
\STATE $u \gets \text{right}(p')$
\IF {red = color(u) }
  \STATE color(p) $\gets $ black
  \STATE color(u) $\gets $ black
  \STATE color(p') $\gets $ red
  \STATE recolor(p')
\ELSIF {black = color(u)}
  \STATE color(p) $\gets $ black
  \STATE color(p') $\gets $ red
  \STATE \texttt{right\_rotate}(p')
\ENDIF
\end{algorithmic}
\end{algorithm}

Based on our analysis above, we can update our red-black trees in $O(h)$ time upon insertion, where $h$ is the height of the tree. The other operations are similar, and also give the guarantee of worst-case performance of $O(h)$ search, insertion, and deletion. Together with our claim that $h = O(\log n)$, we get:

\textbf{Claim} \textit{Red-Black trees support \texttt{insert}, \texttt{delete}, and \texttt{search} in $O(\log n)$ time}

As we have seen, BSTs are very nice – they allow us to maintain a set and report membership, insert, and delete in $O(\log n)$ time. In addition to these basic underlying operations, we can also support other types of queries efficiently. Because the elements are stored maintaining the binary search tree property, we can search for the next largest element or the elements on a range very efficiently. But what if we don’t care about these properties? What if we only need to support membership queries? Can we improve our performance of $O(\log n)$ time to nearly constant time? This question motivates our discussion of hash tables, which we will cover in the next lecture.

\end{document}


\section{Partial Redundancy Elimination}
\begin{flushright}
\textit{Notes by Akshin Singh}
\end{flushright}

A particularly good place to optimize is in loops. Since loops can run for a million iterations, moving any computation from
inside the loop to outside while maintaing correctness is a desirable optimization. Such kind of optimization is called loop invariant
code motion.

\begin{figure}[h]
\centering
\includegraphics[scale = 0.5]{images/mod_105_fig1.png}
\caption{Loop invariant Code Motion Example}
\label {fig:mod_105_01}
\end{figure}

Let us explain with the help of an example. See the original code present in Figure \ref{fig:mod_105_01}. Since i is the loop variable,
that gets incremented every iteration, both \textbf{A[i]} and \textbf{A[i+1]} are dependent on the loop's progress. Hence these variables
are not invariant. However, the expression \textbf{m+n} and \textbf{A[n-1]} are both loop invariant. This is because both n and m hold their
values across iterations. We can therefore transform the code as shown.

Notice that since we have removed an addition as well as an address calculation (which is a multiply and add instruction), we have reduced
the number of instructions in the loop by two. Now if the loop runs a million times, we have saved a lot of work (typically on the order of
two to three hunderd cycles).

What happens if the loop does not run even once? Our transformed code will then have two extra instructions and we have inadvertently slowed
ourseleves by optimizing. This is the tradeoff that an optimizing compiler decides to make or not.

\subsection{Partial Redundancy Elimination}

Let us take a deeper look into why we wanted to transform code in \ref{fig:mod_105_01}. Let us try to follow the control of the program.
We enter the program and then enter the loop. Inside the loop we calculate \textbf{m+n} and \textbf{A[n-1]} in the first iteration.
Then, before the values of m and n are changed we subsequently calculated \textbf{m+n} and \textbf{A[n-1]} again in the next iteration.

Keeping this in mind, let us define the problem of Partial Redundancy. If an expression is redundant (we will
define redundancy in a while) on any path leading to it, it is said to be partially redundant. This is not the case in Common subexpression
elimination, where we can remove only those expressions which are completely redundant, i.e., redundant on all paths.

\begin{figure}[h]
\centering
\includegraphics[scale = 0.5]{images/mod_105_fig2.png}
\caption{Partial Redundancy Elimination Example}
\label {fig:mod_105_02}
\end{figure}

A clearer example can be seen in Figure \ref{fig:mod_105_02}. Here \textbf{b+c} is redundant on the path from \textbf{a=b+c} and not on the
other path. The transformed code has no redundancy. Partial Redundancy Elimination (or PRE for short) is the problem of placing computations (like \textbf{a=b+c}) such that no path recomputes
the same expression.

\begin {itemize}
\item Since PRE is a general optimization that does not need to work on only loops, PRE subsumes Loop invariant code motion. This is because
any computation that is loop invariant is redundant on the path that goes from the loop body back to the loop body.

\begin{figure}[h]
\centering
\includegraphics[scale = 0.3]{images/mod_105_fig3.png}
\caption{Redundancy Example}
\label {fig:mod_105_03}
\end{figure}

\item PRE may require addition of basic blocks. To understand this we need to define redundancy. Let us first take an example. Figure
\ref{fig:mod_105_03}(a) has a redundancy on the green path. If the computation is moved as shown in \ref{fig:mod_105_03}(b), we have
a problem on the red path. The problem is that \textbf{a+b} is not needed anywhere on that path, and yet it is being computed. Naturally we
only want those computations to be present on a path that will be used eventually on that path. This leads us to define redundancy as
any computation on a path that is either happening more than once if needed or at least once if not needed. To solve the problem, we need
to add a basic block as shown in \ref{fig:mod_105_03}(c). Only then can we remove all partial redundancies.

Formally, the problem arised from the fact that we are hoisting computation across a \textbf{critical edge}. A critical edge is an edge
in the CFG where the source block has multiple successors and the desitination block has multiple predecessors. Hoisting across a critical
edge requires a new basic block since we may add redundancy on other paths. The pruple edge in \ref{fig:mod_105_03}(c) is the critical edge.

\item Even after adding basic blocks, there can be cases where we cannot elimate PRE until we decide to duplicate the code. See the
Figure \ref{fig:mod_105_04}(a).

\begin{figure}[h]
\centering
\includegraphics[scale = 0.3]{images/mod_105_fig4.png}
\caption{PRE Duplication Example}
\label {fig:mod_105_04}
\end{figure}

Here the expression \textbf{b+c} is redundant on the red path as shown. Here we need to duplicate the code as shown in
Figure \ref{fig:mod_105_04}(b). Code duplication has caching effects. A larger code-size means a jump can be a jump to
a far instruction which may incur a cache-miss which is not desirable, especially in today's out-of-order 6-8 wide issue processors.
The penalty for i-cache miss can be very high.
\end{itemize}


\section{Functions} % (fold)
\label{sec:functions}

\begin{frame}\frametitle{Simple example}
    \framesubtitle{}

    \emph{Example:} Suppose we want to find the circumference of a circle with radius
    2.5. We could write

    \codeblock{code/functions_radius0.py}

\end{frame}

\begin{frame}\frametitle{Functions}

    Functions are used to abstract components of a program.

    \vfill

    Much like a mathematical function, they take some input and then do something
    to find the result.

\end{frame}

\begin{frame}\frametitle{Functions: def}
    \framesubtitle{A closer look}

    Start a function definition with the keyword \texttt{def}

    \vfill

    Then comes the function name, with arguments in braces, and then a colon\pause

    \begin{center}
        \texttt{def func(arg1, arg2):}
    \end{center}

\end{frame}

\begin{frame}\frametitle{Functions: body}

    Then comes, indented, the body of the function

    \vfill\pause

    Use \texttt{return} to specify the output

    \begin{center}
        \texttt{return result}
    \end{center}

    \codeblock{code/functions_radius2.py}

\end{frame}

\begin{frame}\frametitle{Return}

    By default, Python returns None

    \vfill

    Once Python hits \texttt{return}, it will return the output and jump out of
    the function

    \pause

    \codeblock{code/functions_return.py}

    What does this function do?

\end{frame}

\begin{frame}\frametitle{How to call a function}
    \framesubtitle{}
    % to update

    Calling a function is simple (i.e. run/execute):

    \texttt{>>> func(2.3, 4)}

\end{frame}

\begin{frame}\frametitle{Quick question}

    What is the difference between \texttt{print} and \texttt{return}?

\end{frame}

\begin{frame}\frametitle{Exercise}
    \framesubtitle{Write your first function}

    \begin{enumerate}
         \item Write a function that prints `Hello, world!'
         \item Write a function that returns `Hello, \texttt{name}!', where \texttt{name}
         is a variable
     \end{enumerate}

\end{frame}

\begin{frame}\frametitle{Exercise solution}
    \framesubtitle{Write your first function}

    \codeblock{code/functions_ex_first.py}

\end{frame}

\begin{frame}\frametitle{Everything is an object}

    Everything is Python is an object, which means we can pass functions:

    \vfill

    \codeblock{code/functions_double.py}

\end{frame}

\begin{frame}\frametitle{Scope}
    \framesubtitle{}

    Variables defined within a function (local), are only accessible within the function.

    \vfill

    \codeblock{code/functions_scope.py}

\end{frame}

\begin{frame}\frametitle{Functions within functions}
    \framesubtitle{}

    It is also possible to define functions within functions, just as we
    can define variables within functions.

    \codeblock{code/functions_within.py}

\end{frame}

\begin{frame}\frametitle{Global keyword}
    \framesubtitle{}

    We could (but should not) change global variables within a function

    \codeblock{code/functions_global.py}

   Question: What is the difference between the last two functions?

\end{frame}

\begin{frame}\frametitle{Scope questions}

    \codeblock{code/functions_scope_q.py}

\end{frame}

\begin{frame}\frametitle{Default arguments}
    \framesubtitle{}

    It is sometimes convenient to have default arguments

    \codeblock{code/functions_default_args.py}

    The default value is used if the user doesn't supply a value.

\end{frame}

\begin{frame}\frametitle{More on default arguments}
    \framesubtitle{}

    Consider the function prototype: \texttt{func(x, a=1, b=2)}

    Suppose we want to use the default value for a, but change b:

    \codeblock{code/functions_default_args2.py}

\end{frame}

\begin{frame}\frametitle{Docstring}
    \framesubtitle{Documentation is important}

    It is important that others, including \emph{you-in-3-months-time} are able
    to understand what your code does.

    This can be easily done using a so called `docstring', as follows:

    \codeblock{code/functions_docstring.py}

    \pause

    We can then read the docstring from the interpreter using:

    \vfill

    \texttt{>>> help(nothing)}

    This function doesn't do anything.

\end{frame}

\begin{frame}\frametitle{Question}
    \codeblock{code/functions_docstring.py}
    \vfill
    Question: what does \texttt{nothing()} return?
\end{frame}

\begin{frame}\frametitle{Lambda functions}
    \framesubtitle{A quick way to define short functions}

    An alternative way to define short functions:

    \codeblock{code/functions_lambda.py}

    \vfill

    Pros:
    \begin{itemize}
        \item One line / in line
        \item No need to name a function
    \end{itemize}

    Try to use these for the homework if you can.

\end{frame}

% section functions (end)

% \section{Functions}
% \begin{frame}
% \frametitle{Functions}
% Functions are used to organize programs into coherent pieces

% \vspace{0.2in}

% In other words, functions are used to generalize or ``abstract" components of a program
% \end{frame}

% \begin{frame}
% \frametitle{Root finding}

% What is a limitation of this code?

% \codeblock{code/while2.py}

% \end{frame}

% \begin{frame}
% \frametitle{Root finding}
% We do not want a $\sqrt{7}$ method, we want a $\sqrt{x}$ method

% \vspace{0.2in}

% We still have the same capabilities (just let $x = 7$), but now our ``abstracted" root finder can be used for more cases
% \end{frame}

% \begin{frame}
% \frametitle{Root finding}

% With this function, we can call \texttt{root(7)}

% \codeblock{code/while3.py}

% \end{frame}

% \begin{frame}
% \frametitle{Root finding}
% More general, we can call \texttt{root(7, 0.1)}:

% \codeblock{code/while4.py}

% \end{frame}

% \begin{frame}
% \frametitle{Root finding}
% Python makes it easy to provide default argument values:

% \codeblock{code/while5.py}

% Can call \texttt{root(3)}, \texttt{root(113, 0.01)}
% \end{frame}

% \begin{frame}
% \frametitle{Root finding}
% A few things to think about:
% \vspace{0.1in}
% \begin{itemize}
% \setlength{\itemsep}{0.1in}
% \item{Are there cases where \texttt{root()} will have an error?}
% \item{Are there cases where \texttt{root()} will run forever? (fail to converge)}
% \item{How else can we generalize the function?}
% \end{itemize}
% \end{frame}

% \begin{frame}
% \frametitle{Basic functions}

% What will happen when this code runs?

% \codeblock{code/func1.py}

% \end{frame}

% \begin{frame}
% \frametitle{Basic functions}

% What about this code?

% \codeblock{code/func2.py}
% \end{frame}

% \begin{frame}
% \frametitle{More function examples}

% \codeblock{code/poly_eval1.py}
% \end{frame}

% \begin{frame}
% \frametitle{More function examples}

% \codeblock{code/poly_eval2.py}
% \end{frame}

% \begin{frame}
% \frametitle{Indentation / white space}

% Python uses indentation (``white space") to group statements

% \vspace{0.15in}

% Each code block is indented the same amount (for loop, while loop, function definition, etc.)

% \vspace{0.15in}

% Python will complain if your indentation is incorrect
% \end{frame}

% \begin{frame}
% \frametitle{Indentation}

% \codeblock{code/indent1.py}

% 2-space or 4-space indentation is standard.

% \end{frame}

% \begin{frame}
% \frametitle{Identation}

% \codeblock{code/indent2.py}

% \end{frame}

% \begin{frame}
% \frametitle{White space}

% Some people like this indentation structure and some people do not.

% \vspace{0.2in}

% ... Python isn't going to stop using it

% \end{frame}

% \begin{frame}
% \frametitle{End}
% \begin{itemize}
% \setlength{\itemsep}{0.05in}
% \item{Assignment 1 is posted on the course web site (due Tuesday 1/14)}
% \item{In-class exercises, python codes from slides, and readings also posted}
% \end{itemize}

% Next time:
% \begin{enumerate}
% \setlength{\itemsep}{0.05in}
% \item{More on Python functions}
% \item{Strings}
% \item{Lists}
% \item{Dictionaries}
% \item{Tuples}
% \end{enumerate}

% \end{frame}


\chapter{Examples}

\epigraph{\itshape "I've been working on a new electronic cash system that's fully peer-to-peer, with no trusted third party."}{---Satoshi Nakamoto, \textit{The Cryptography Mailing List} \cite{nakamotoFirstPost}}

We can \textbf{highlight} stuff in \underline{various} \textit{ways} in our text. We can also quote - either using a fancy quote as above or using regular in-line ones such as "I've been ..."\cite{nakamotoFirstPost}.
We can \enquote{ \enquote{specify a page and a chapter as well}, said Andrew}\cite[chapter, p.~215]{wattenhofer2016science}.
Note that for multi-level quotes it is best to use the \textbackslash{}enquote\{\} command which understands which quotation marks it needs to use.

This paragraph is separate from previous text as we used a blank line in LaTex. The spacing is automatic. If we want to force more whitespace, then we use \textbackslash{}\textbackslash{} (a newline).
\\
\\
\\
\\
\\
Each \textbackslash{}\textbackslash{} gives us one blank line. Note that \textbackslash{} is a special character and we need \textbackslash{}textbackslash command to put \textbackslash{} in the text. Many other special characters (e.g. \%, \$, \_, \#, \&, \{, =) can be used in text by putting \textbackslash{} in front of them in LaTex. Some of them, however, require special commands - e.g. for \textgreater we need the \textbackslash{}textgreater command \footnote{Most LaTex symbols can be found here: \url{http://mirror.hmc.edu/ctan/info/symbols/comprehensive/symbols-a4.pdf}}.

We won't need commands like \textbackslash{}textgreater  if  we use the in-line math mode by putting text in between \$ \$ (which exists for in-lining equations etc.) - e.g. $ 4 > 2, 5 < 9 $. Longer equations etc. are easier to write not in-line, e.g.:

\begin{align}
	\label{eq:der_1}
	\frac{\partial L}{\partial R} &= 2R - 2\sum_{i=1}^{n} m_i R = 2(1- \sum_{i=1}^{n} m_i)R = 0
	\implies (1- \sum_{i=1}^{n} m_i)R = 0\\
	%
	\label{eq:der_2}
	\frac{\partial L}{\partial \vec a} &=  \sum_{i=1}^{n} m_i 2(\vec x_i - \vec a)(-1) = -2 \sum_{i=1}^{n} m_i(\vec x_i - \vec a) = 0
	\implies \sum_{i=1}^{n} m_i(\vec x_i - \vec a) = 0\\
	%
	\frac{\partial L}{\partial \xi_i} &= C - m_i - n_i = 0
	\label{eq:der_3}
\end{align}

Note that equations are aligned on the = symbols we prepended in LaTex with \&. Each line/equation has to end in LaTex with \verb|\\|. In the example above they are all numbered, but numbers can be removed by using \verb|align*| environment instead of the usual \verb|align| or by putting \verb|\nonumber| right before \verb|\\| ending a line. As we also put \verb|\label| commands in LaTex next to the lines/equations, we can now refer e.g. to Equation (\ref{eq:der_2}).

\section{Verbatim and Code}

Another way to emphasise is to use \verb|verbatim style which is pretty funny| and (as you can notice in previous section) allows using many special characters as normal ones: \verb|\ _ % &&|.

\subsection{Code with Verbatim}

We often use verbatim when showing code samples:

\begin{verbatim}
import numpy as np

def foo(x):
	sum = x + 5
	return sum
\end{verbatim}

\subsection{Fancier Code}

We can also use a \verb|lstlisting| environment to get some syntax highlighting:

\begin{lstlisting}[language=Python]
import numpy as np

def foo(x):
	sum = x + 5
	return sum
\end{lstlisting}

\section{Theorems and Defs} \label{sec:theorems}

Examples:

\begin{defn}[Test definition] Here is a new definition.\end{defn}

\begin{thm}
	\label{the:test_theo}
	Here is a new theorem.
\end{thm}

Note that numbering of theorems and definitions starts with the current chapter number (this can be changed, we set it in our main.tex). As with equations, we can refer to them - e.g. Theorem \ref{the:test_theo}.

\section*{Section With No Number}

Well, it has no number because why not? :) Speaking of section/chapter numbers - as always, we can refer to them if we used \verb|\label| command right after the \verb|\section| command (and we did for Section \ref{sec:theorems} about Theorems).

\section{Lists etc.}

Listing things is useful so:

\begin{itemize}
	\item Simple list first entry.
	\item Second entry.
\end{itemize}

Or you can enumarate if you like:

\begin{enumerate}
	\item First level item
	\item First level item
	\begin{enumerate}
		\item Second level item
		\item Second level item
		\begin{enumerate}
			\item Third level item
			\item Third level item
		\end{enumerate}
	\end{enumerate}
	\item First level item
\end{enumerate}

We can also of course change the numberings/symbols used if we want \footnote{Read about it here: \url{https://www.sharelatex.com/learn/Lists}}.

Finally, the description 'list' is also nice:

\begin{description}
	\item[Dancing] \lipsum[1]
	\item[Walking] \lipsum[2]
\end{description}

\section{Tables and Figures}

A simple Table would be e.g. Table \ref{tab:classified_programs}. Btw, note how numbering changes when we refer to a copy of this table put in our Appendix - Table \ref{tab:classified_programs_appendix}.

\begin{table}[!h]
	\centering
	\begin{tabular}{|c|c|}
		\hline
		\textbf{Verification technique} & \textbf{Classified programs} \\ \hline
		Houdini algorithm & 42 \\\hline
		Sound BMC & 18 \\\hline
		Translation to C & 20 \\\hline
		Unsound BMC & 36 \\\hline
	\end{tabular}
	\caption{Number of programs classified as correct per technique.}
	\label{tab:classified_programs}
\end{table}

If you look at LaTex source, you can se we used [!h] where "h" means "place this Table \textbf{h}ere" (i.e. wherever we put it in LaTex source) and the "!" makes the compiler relax some restrictions and try harder to satisfy the request. We can also use b (bottom of page), t (top) or H. The last one puts the Table \textbf{exactly} where we put it in LaTex source - even if it causes empty space on a page/pages etc. Exactly the same placement options can be used with Figures:

\begin{figure}[!t]
	\begin{centering}
		\includegraphics[width=0.4\linewidth]{imperial.png}
		\caption{Test image.}
		\label{fig:test_imperial}
	\end{centering}
\end{figure}

Figure \ref{fig:test_imperial} has [!t] option used (so it is on the top of the page). The next ones (Figures \ref{fig:imperial_mini_page} and \ref{fig:imperial_mini_page_again}) have the [!h] option. Note that they are side-by-side (this is achieved with a "minipage" environment). Alternatively, we can have several images within one Figure such as in case of Figure \ref{fig:imperial_image_left} and Figure \ref{fig:imperial_image_right}.

\begin{figure}[!h]
	\begin{centering}
		\begin{minipage}{.60\textwidth}
			\begin{centering}
				\includegraphics[height=1cm]{imperial.png}
				\caption{Imperial image in a minipage.}
				\label{fig:imperial_mini_page}
			\end{centering}
		\end{minipage}
		\begin{minipage}{.40\textwidth}
			\begin{centering}
				\includegraphics[height=1cm]{imperial.png}
				\caption{Another Imperial image in a minipage.}
				\label{fig:imperial_mini_page_again}
			\end{centering}
		\end{minipage}
	\end{centering}
\end{figure}

\begin{figure*}[!h]
	\begin{subfigure}[b]{.49\textwidth}
		\centering
		\includegraphics[width=0.7\linewidth]{imperial.png}
		\caption{Imperial image - left.}
		\label{fig:imperial_image_left}
	\end{subfigure}
	\begin{subfigure}[b]{.49\textwidth}
		\centering
		\includegraphics[width=0.7\linewidth]{imperial.png}
		\caption{Imperial image - right.}
		\label{fig:imperial_image_right}
	\end{subfigure}

	\caption{Two imperial images.}
	\label{fig:two_imperial_images}
\end{figure*}

\textbf{NOTE:} When putting images within one Figure (i.e. as subfigures), do not use e.g. \verb|.50\textwidth| to occupy half of a line for each of your 2 images (or \verb|.25\textwidth| for each of 4 images etc.) - this seems to force one image on the next line of images. Use e.g. \verb|.49\textwidth| instead.


\documentclass{beamer}

\usepackage{beamerthemevictor,comment,verbatim,graphicx,rotating}

\input{tutmacs}
\input idxmacs

\begin{document}

\title{\LaTeX}
\author{Victor Eijkhout}
\date{Notes for CS 594 -- Fall 2004}

\frame{\titlepage}

\section{Introduction}
\subsection{Markup}

\frame[containsverbatim]{
  \frametitle{What is a macro language?}
\begin{itemize}
\item Macros are abbreviations for sequences of commands or text
\begin{quote}
\verb+\TeX+ gives `\TeX'
\end{quote}
\item Macro language can have the full power of a programming
  language; variables, conditionals, recursion
\item Typesetting macro languages like \TeX/\LaTeX\ have commands for
  document processing.
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Logical markup}
\begin{itemize}
\item Macros introduce an abstraction level
\begin{verbatim}
\section{Introduction}
\end{verbatim}
instead of (so to speak)
\begin{verbatim}
\MaybePageBreak \aLittleSpace \BiggerFont
Introduction
\end{verbatim}
\item Style no longer hardwired in the document: specified independently
\end{itemize}
}

\subsection{Absolute basics}

\frame[containsverbatim]{
  \frametitle{Characters}
\begin{itemize}
\item For most characters: type an~`a', get an~`a' on paper
\item Exceptions:
\begin{itemize}
\item Commands start with a backslash: \verb+\section+
\item Braces indicate macro arguments, as with section command, or
  grouping:
\begin{quote}\verb+not bold {\bfseries then bold} then not+ gives `not
  bold {\bfseries then bold} then not'
\end{quote}
\item Spaces are ignored at beginning/end of line and after control
  sequence (except\dots); multiple spaces equivalent to one. After one
  blank line, others are ignored.
\end{itemize}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{More special characters}
\begin{itemize}
\item Dollars indicate inline math; in math \verb+^+~is superscript
  and \verb+_+~is subscript:
\begin{quote} `\verb/and $x_{i+1}^2$ is positive/' looks like\\ `and
$x_{i+1}^2$ is positive'
\end{quote}
\item \verb+&+ is used in tables, \verb+#+~in macro definitions,
  \verb+~+~is non-breaking space
\item \verb+%+ is the comment character
\item To get these characters, use \verb+\%, \$+ et cetera.
\item Exception: \verb+$\backslash$+ for `$\backslash$'
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Character pitfalls}
\begin{itemize}
\item Characters not available in all styles:
\begin{quote}the \n{<>} characters come out `<>' in text font;
  however, \verb+$\langle$A$\rangle$+ is `$\langle$A$\rangle$'
\end{quote}
\item After control symbols -- backslash followed by anything but a
  letter -- spaces are not ignored; they are after \verb*+\ + control space.
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{A bit more about spaces}
\begin{itemize}
\item Multiple spaces count as one; line end inserts a space:
\begin{verbatim}
\section{
The Long Title}
\end{verbatim}
has an unwanted space. Prevent with
\begin{verbatim}
\section{%
\end{verbatim}
\item Spaces are ignored after control sequences:
\begin{quote}
`\verb*+\LaTeX is fun+' is `\LaTeX is fun'; but
`\verb*+\LaTeX\ is \LaTeX{} is fun+' is `\LaTeX\ is \LaTeX{} is fun'; and
`\verb+\LaTeX{}ing is fun+' is `\LaTeX{}ing is fun'
\end{quote}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Document structure}
\begin{itemize}
\item Minimum elements:
\begin{verbatim}
\documentclass[<options>]{<class name>}
  <preamble>
\begin{document}
  <text>
\end{document}
\end{verbatim}
\item Classes are \n{article}, \n{report}, \n{IEEEproc}, \n{beamer},
  et cetera. Class options: \n{twoside}, \n{a4paper}, \n{12pt}, et cetera.
\item Preamble has definitions and parameter settings, also
\begin{verbatim}
\usepackage[<options>]{<package>}
\end{verbatim}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Usual start of scientific prose}
\begin{itemize}
\item In preamble or text:
\begin{verbatim}
\title{My lecture notes}
\author{John Doe}
\date{August 2004} % leave out, get today's date
\end{verbatim}
and in the text \verb+\maketitle+.
\item Maybe also
\begin{verbatim}
\begin{abstract}....\end{abstract}
\end{verbatim}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Running \LaTeX}
\begin{itemize}
\item Traditionally, executable \n{latex} translates \n{.tex} file to
  \n{.dvi}, also generating \n{.log} and \n{.aux} (maybe more).
\item Preview \n{dvi} with \n{xdvi}
\item  Translate \n{dvi} to Postscript with \n{dvips}, then Pdf with
  \n{ps2pdf}.
\item Better: use \n{pdflatex} executable
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{External resources}
\begin{itemize}
\item Comprehensive \TeX\ Archive Network:
  \verb+\http://www.ctan.org/+
\item Newsgroup \n{comp.text.tex}; readable and archived through
  Google groups.
\item Lots more stuff on the web.
\end{itemize}
}

\section{Text elements}

\frame[containsverbatim]{
  \frametitle{Text element}
}

\frame[containsverbatim]{
  \frametitle{Conceptual model}
\begin{itemize}
\item Paragraph: one long strip, cut into lines when everything has
  been set.
\item Page: one long scroll, cut into pages when all material has been
  assembled
\item Upshot: asking `on what line/page is this text' is pretty hard
\end{itemize}
}

\subsection{Major stuff}

\frame[containsverbatim]{
  \frametitle{Sectioning}
\begin{itemize}
\item Write \verb+\section{Introduction}+, also \cs{subsection} and
  \cs{subsubsection}; \cs{chapter} and \cs{part} in \n{report} and
  \n{book} style only.
\item Use \cs{section*} for unnumbered.
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Input files}
\begin{itemize}
\item Use \verb+\input <file>+, \verb+\input <file>.tex+, \verb+\input{<file>}+
\item Input on new page: \cs{include}
\item On Unix, search locations are in \n{TEXINPUTS} environment variable.
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Environments}
\begin{itemize}
\item \verb+\begin{<name>} .... \end{<name>}+
\item Main use: group a section of text for different processing.
\item `Abstract' already mentioned
\item Different text treatment: \n{flushleft}, \n{flushright},
  \n{center}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Verbatim text}
\begin{itemize}
\item Problem: how to print all of \LaTeX's special characters?
\item Also: listing of code input (programming, HTML, \TeX\ itself)
\item Short snippets: \verb-\verb+&$^_\}+-
\item Longer:
\begin{quote}
\verb+\begin{verbatim}+\\
\verb+  TeX example+\\
\verb+\end{verbatim}+
\end{quote}
\item Whole files: \verb+\verbatiminput{<file>}+
\item \TeX{}nical stuff: verbatim command and environment can not be
  used in \verb+\SomeCommand{.. \verb ...}+ context; usually possible
  in environments.
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Lists}
\begin{itemize}
\item Bullet and numbered:
%
\begin{examplewithcode}
\begin{itemize}
\item One \item Two
\end{itemize}
\end{examplewithcode}
%
\begin{examplewithcode}
\begin{enumerate}
\item\label{first:item} One
\item Two comes after \ref{first:item}
\end{enumerate}
\end{examplewithcode}
%
\item Bullet and number style changes with level
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Description list}
\begin{itemize}
\item
\begin{examplewithcode}
\begin{description}
\item[Do] A deer
\item[\textbf{Re}] A beam
\end{description}
\end{examplewithcode}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Tabular stuff}
\begin{itemize}
\item Table data:
\begin{examplewithcode}
\begin{tabular}{|r|rl|}
\hline
instrument&\multicolumn{2}{|c|}{name}\\ \hline
drums: &"Philly" Joe & Jones\\
trumpet:& Dizzie & Gillespie\\
piano: &Art&Tatum\\ \hline
\end{tabular}
\end{examplewithcode}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Footnotes}
\begin{itemize}
\item Use \verb+\footnote{<fn text>}+ in the place where you want the
  marker or number to appear
\item Tinkering: set the \n{footnote} counter, or use
  \cs{footnotemark} and \cs{footnotetext} to place the mark and text
  seperately.
\item Latter option also for footnotes in tables and such
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Text boxes}
\begin{itemize}
\item
\begin{verbatim}
\parbox[pos]{width}{text}
\end{verbatim}
where \n{pos} is \n{t,b} for top, bottom; center~default
\begin{verbatim}
\begin{minipage}[pos]{width}  text  \end{minipage}
\end{verbatim}
\begin{examplewithcode}
This \parbox{2in}{text, text, more text
  and yet more more more text}
\end{examplewithcode}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Tabbing}
\begin{examplewithcode}
\begin{tabbing}
while \=\kill
do\>\{\\
\>$i_1\leftarrow{}$\=1\\
\>$\ldots$\>2\\
\>\}\\
while (1)
\end{tabbing}
\end{examplewithcode}
}

\subsection{Minor stuff}

\frame[containsverbatim]{
  \frametitle{Fonts and typefaces}
\begin{itemize}
\item Deprecated commands: \cs{bf}, \cs{it}, et cetera.
\item Changes for short amount of text:
\begin{examplewithcode}
\begin{upshape} %kindly ignore this
Text \textbf{in bold}, \textsl{slanted},
\textsc{Small Caps}.
\end{upshape}
\end{examplewithcode}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{}
\begin{itemize}
\item More general
\begin{description}
\item[family] roman \cs{rmfamily}, sans serif \cs{sffamily},
  typewriter \cs{ttfamily}
\item[series] bold \cs{bfseries}, medium weight \cs{mdfamily}
\item[shape] upright \cs{upshape}, italic \cs{itshape}, slanted
  \cs{slshape}, small caps \cs{scshape}
\end{description}
\begin{examplewithcode}
\begin{rmfamily}\begin{upshape}text \textbf{bold}
  \begin{slshape}slant
  \end{slshape}\end{upshape}\end{rmfamily}
\end{examplewithcode}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Comments}
\begin{itemize}
\item Inline comments, use \verb+line % comment+
\item Longer:
\begin{verbatim}
\usepackage{comment} or usepackage{verbatim}
\begin{comment}
 }}} uns&n_tact$ical
\end{comment}
\end{verbatim}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Hyphenation and line breaking}
\begin{itemize}
\item Prevent line breaking: \verb+\mbox{and the $1$}+. Also
  \verb+do~not~break+.
\item Repair: \verb+Eijk\-hout+ or
  \verb+\hyphenation{Eijk-hout}+. \emph{This is only as repair: using
    a different language takes more.}
\item Good habits: start sentences with \verb+A~further+, and end with
  \verb+to~1+.
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Tilde}
\begin{itemize}
\item Tilde is an active character: non-breaking space.
\item Tilde accent: `\verb+ma\~nana+' is `ma\~nana'
\item Tilde character: `\verb+\~{}+'~is~`\~{}';
  `\verb+$\sim$+'~is~`$\sim$'; `\verb+\char`\~+~is~`\char`\~'
\item In URLs: \cs{url} is defined in the \n{url} and \n{hyperref} packages
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Accents}
\begin{itemize}
\item Accents are backslashed, before the character:
  \verb+Sch\"on b\^et\'e+ for `Sch\"on b\^et\'e'.
\item Better: use package \n{inputenc} and select the proper code page.
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Lines and boxes}
\begin{itemize}
\item Do not underline!
\item Rule:
\begin{examplewithcode}
1\ \rule[.5ex]{2cm}{.5mm}\ The title
\end{examplewithcode}
\item \verb+\fbox{text}+ gives \fbox{text}.
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Line and page breaking}
\begin{itemize}
\item In general: leave it to \LaTeX.
\item \cs{linebreak} (\verb+\linebreak[1..4]+) is suggested
  location for normal line break, with filling out the margin
\item \cs{newline} breaks without adjusting margin
\item \cs{pagebreak} and \cs{newpage} similar
\item \cs{nolinebreak}, \cs{nopagebreak}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Spacing}
\begin{itemize}
\item \cs{hspace}, \cs{vspace}, \cs{hspace*}, \cs{vspace*}
\begin{examplewithcode}
This is a short bit\hspace{2cm}of text with a
manually inserted space. That space would\hspace{1in}
disappear in a line break. This space\hspace*{31.4mm}
does not.
\end{examplewithcode}
\item Also: \verb+\hspace{\fill}+ for `infinitely stretchable space':
  takes up whatever room there is.
\end{itemize}
}

\subsection{Technical stuff}

\frame[containsverbatim]{
  \frametitle{Horizontal and vertical mode}
\begin{itemize}
\item Horizontal mode: letters in paragraph
\item Vertical mode: paragraphs in page
\item Most of the time \LaTeX\ does the right thing.
\item Force vertical mode with \cs{par}
\item Force horizontal by putting them in an \cs{mbox} (or use
 \cs{leavevmode})
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Fragile commands}
\begin{itemize}
\item \verb+\section{Fragile commands\footnote{like this}}+
leads to strange error messages:
\begin{verbatim}
! Argument of \@sect has an extra }.
<inserted text>
                \par
l.691 ...gile commands\footnote{like this}}
\end{verbatim}
Do \verb+\section{...\protect\footnote{like this}}+
\end{itemize}
}

\section{Other \LaTeX\ elements}

\frame[containsverbatim]{
  \frametitle{More text elements}
}

\subsection{Floating material}

\frame[containsverbatim]{
  \frametitle{Floats}
\begin{itemize}
\item Big objects (tables, figures) may not fit at the current
  location
\item Declare as floating object, leave placement to \LaTeX
\begin{verbatim}
\begin{<table or figure>}[placement]
... table or figure material ...
\caption{Caption text}\label{tabfig:label}
\end{<table or figure>}
\end{verbatim}
\item placement example: \n{[htp]} means `place here, otherwise top of
  page, otherwise page of its own'
\item tables with \n{tabular} environment, figure see below
\item \cs{listoftables}, \cs{listoffigures}
\end{itemize}
}

\subsection{Math}

\frame[containsverbatim]{
  \frametitle{Getting into math mode}
\begin{itemize}
\item Inline math: \verb+$<formula$+, \verb+\(...\)+
\item Display math unnumbered: \verb+\[...\]+ or \n{displaymath}
  environment
\item Display math numbered: \n{equation} environment; use \cs{label}
  and \cs{ref} to refer to the equation number
\item Equations are centered, use \n{fleqn} (option, or use package)
  for flush left
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Display vs inline}
\begin{itemize}
\item Mostly visible in delimiter style:
\[ \mbox{text style: }\textstyle\sum_{n=1}^\infty 1/n^2 \]
\[ \mbox{display style: }\displaystyle\sum_{n=1}^\infty 1/n^2 \]
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Fonts in math}
\begin{itemize}
\item Variables are italic: `$x$'
\item Functions are roman: `$\sin(x)$'
\item Connecting text:
\begin{examplewithcode}
\[ \forall_x \quad \mbox{(sufficiently large)} \quad
   \colon \qquad x>5 \]
\end{examplewithcode}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Long formulas}
\begin{itemize}
\item Inline formulas can be broken, usually after operators ($+$) or
  relations~($=$). Prevent that with \verb+hence~$x=\nobreak1$+ or
  \verb+hence~\mbox{$x=1$}+
\item Display formulas are not broken, use
\begin{examplewithcode}
\begin{eqnarray}
x&=&3\\
y&>&2\sin y
\end{eqnarray}
\end{examplewithcode}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Sub and superscripts}
\begin{itemize}
\item \verb+$x_{i,j}^{n^2}$+ is $x_{i,j}^{n^2}$
\item also for delimiters: \verb+\sum_i^j 1/i+ is~$\sum_i^j 1/i$
\item complicated limits
\begin{examplewithcode}
\[ \sum_{\begin{array}{c}\scriptstyle i\geq0\\
            \scriptstyle j\geq0 \end{array}} i^j \]
\end{examplewithcode}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Matrices}
\begin{itemize}
\item \n{array} environment much like \n{tabular}, but in math
\begin{examplewithcode}
\[ A=\left( \begin{array}{cc} 1&2\\ 3&4
         \end{array}\right) \]
\end{examplewithcode}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Delimiters}
\begin{itemize}
\item Delimiters are \verb+()[]\{\}+
\item Use with matched \cs{left}, \cs{right}; omitted delimiter from~\n{.}
\begin{examplewithcode}
\[ \left( \frac{1}{1-x^2} \right)
\left\{ \begin{array}{ccc}
    \mathrm{(a)}&\Rightarrow&x>0\\
    \mathrm{(b)}&\Rightarrow&x=0\\
    \mathrm{(c)}&\Rightarrow&x<0
        \end{array} \right. \]
\end{examplewithcode}

\end{itemize}
}

\frame[containsverbatim]{
\begin{examplewithcode}
\[ A = \left( \begin{array}{cccccc}
       a_{11}&0&&\ldots&0&a_{1n}\\
       &a_{22}&0&\ldots&0&a_{2n}\\
       &&\ddots&\ddots&\vdots&\vdots\\
       &&&a_{n-2n-2}&0&a_{n-2n}\\
       &\emptyset&&&a_{n-1n-1}&a_{n-1n}\\
       &&&&&a_{nn}
     \end{array} \right) \]
\end{examplewithcode}
}

\subsection{Refering to things}

\frame[containsverbatim]{
  \frametitle{References}
\begin{itemize}
\item Make a label by \verb+\label{sec:intro}+ and such, then refer to
  \verb+\ref{sec:intro}+ and \verb+\pageref{sec:intro}+
\item Implemented through \n{.aux} file; two runs needed
\item Warnings on undefined references and duplicate labels
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Contents}
\begin{itemize}
\item Table of contents formed automatically
\item insert with \cs{tableofcontents}
\item star-commands (\cs{section*} and such) not in table of contents;
\item manual additions: \cs{addcontentsline} and more
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Index}
\begin{itemize}
\item \verb+\usepackage{makeidx}+
\item Create entries by \verb+\index{Keyword}+,
  \verb+\index{Key!subkey}+ and more.
\item \cs{printindex} to include index;
\item run external program \n{makeindex}
\item external file \n{.ind}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Bibliography}
\begin{itemize}
\item Refer to books, articles with \verb+\cite{Dongarra:1992a}+
\item
\begin{verbatim}
\bibliographystyle{plain}
\bibliography{cs}
\end{verbatim}
\item Run external program \verb+bibtex <mytexfile>+, giving \n{.bbl} file
\begin{verbatim}
@book{tbt,
author = {Victor Eijkhout},
title = {{\TeX} by Topic},
publisher = {Addison-Wesley UK},
year = {1991},
note = {out of print;
  available online at \url{http://www.eijkhout.net/tbt/}
}
\end{verbatim}
\end{itemize}
}

\section{Customizing \LaTeX}

\frame[containsverbatim]{
  \frametitle{Customizing \LaTeX}
}

\frame[containsverbatim]{
  \frametitle{Parameter changes}
\begin{itemize}
\item Layout parameters are `lengths'
\begin{verbatim}
\setlength{\textwidth}{10in}
\addtolength{\oddsidemargin}{-1cm}
\end{verbatim}
\item Some lengths are `rubber length'
\begin{verbatim}
\setlength{\parskip}{10pt plus 3pt minus 2pp}
\end{verbatim}
\end{itemize}
}

\subsection{Page layout}

\frame[containsverbatim]{
  \frametitle{Layout options and parameters}
\begin{itemize}
\item Document class choice, class options: \n{twoside}, \n{a4paper},
  \n{letterpaper}
\item Parameters: \cs{textheight}, \cs{textwidth}, \cs{topmargin}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Page styles}
\begin{itemize}
\item Use \verb+\pagestyle{plain}+ or \n{empty} or \n{headings}
\item More flexible: use package \n{fancyhdr}, page style \n{fancy}
\item Fancy: six ingredients \verb+{lcr}{head,foot}+
\begin{verbatim}
\lhead{<text>}  \chead{<text>}  \rhead{<text>}
\end{verbatim}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Fancy running heads}
\begin{itemize}
\item Use \verb+lhead{text}+ (also \n{c}, \n{r}; \n{foot})
\item Combined
\begin{verbatim}
\fancyhead[LE,OR]{\rightmark}
\fancyfoot[LE,OR]{\thepage}
\end{verbatim}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Automatic running heads}
\begin{itemize}
\item Commands \verb+\markright{head}+ and
  \verb+\markboth{left}{right}+
\item Fancy style uses \cs{rightmark} and \cs{leftmark} by default
\item \TeX\ searches back for last left and right mark
\item Marks are set automatically by sectioning commands
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Multicolumn text}
Load
\begin{verbatim}
\usepackage{multicol}
\end{verbatim}
and write
\begin{verbatim}
\begin{multicol}{3}
text in three column mode
\end{multicol}
\end{verbatim}
Can start/end anywhere.
}

\subsection{Graphics}

\frame[containsverbatim]{
  \frametitle{Just a bit about device drivers}
\begin{itemize}
\item \TeX\ does not support graphics
\item Extension mechanism: `specials'
\item Arbitrary text in the \n{dvi} output, interpreted by device
  driver
\item $\Rightarrow$ \TeX\ has to know about device driver
\item specific \LaTeX\ graphics extensions: \n{psfig}, \n{epsf}, \n{graphics}
\item \n{pdflatex} has device driver built in
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Included pictures}
\begin{itemize}
\item Use package \n{graphicx}
\begin{examplewithcode}
\includegraphics[scale=.3,angle=45]{spline-pic}
\end{examplewithcode}
\item good idea to put this in a float
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Text manipulation}
\begin{verbatim}
\usepackage{rotating}
\end{verbatim}
\begin{examplewithcode}
\begin{turn}{30}
Just a line of text
\end{turn}
\end{examplewithcode}
}

\subsection{\LaTeX\ programming}

\frame[containsverbatim]{
  \frametitle{Counters}
\begin{itemize}
\item Use existing counters \n{section}, \n{footnote}, \n{enumi}
\begin{verbatim}
\setcounter{subsection}{3}
\addtocounter{enumii}{-1}
\refstepcounter{footnote}
\arabic{section} \Roman{subsection}
\end{verbatim}
\item Define new counters:
\begin{verbatim}
\newcounter{exercise}[chapter]
\end{verbatim}
\end{itemize}
}

\frame[containsverbatim]{
  \frametitle{Commands}
\begin{examplewithcode}
\newcommand{\IncrByOne}{increased by~$1$}
\dots and $n$~\IncrByOne.
\newcommand{\IncrDecrBy}[2]{#1creased by~$#2$}
\dots and $n$~\IncrDecrBy{de}{5}.
\end{examplewithcode}
(\cs{renewcommand})
}

\frame[containsverbatim]{
  \frametitle{Commands, optional arguments}
\begin{examplewithcode}
\newcommand{\IncrDecrBy}[2][in]{#1creased by~$#2$}
\dots and $x$~\IncrDecrBy{1}.
\dots and $y$~\IncrDecrBy[de]{5}.
\end{examplewithcode}
}

\frame[containsverbatim]{
  \frametitle{New environments}
\begingroup
\begin{examplewithcode}
\renewenvironment{example}%
  {\begin{quote}\tiny\textbf{Example.}}%
  {\end{quote}}
\dots and text.
\begin{example} Blah blah. \end{example}
\end{examplewithcode}
\endgroup
Arguments and optional args as before.
}

\end{document}

\frame[containsverbatim]{
  \frametitle{}
\begin{itemize}
\item
\end{itemize}
}


%%% CREATED BY MATTHEW DU. COPYRIGHT 2021

% So this top part is known as the preamble. It contains all the formatting and shortcuts that you can possibly need. Nothing outside of the \begin{document} and \end{document} will show up in you document
% Also, the % symbol is the comment symbol. Feel free to remove them if you need anything

\documentclass[12pt]{article} % This is what your document is
% Change the 10pt to whatever font you need.
% Other document types are Beamer, report, thesis, etc. Beamer is special. I suggest you learn that last through google

\usepackage{amsmath} % These are your math symbols
\usepackage{amsfonts} % These are your math symbols
\usepackage{amssymb} % These are your math symbols

% These are like your formatting pages and extra fonts. I wouldn't worry about these
\usepackage{ifthen}
\usepackage{fancyhdr}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}

% These are specific packages I like using for CS documents
\usepackage{listings}
\usepackage{xcolor}
\usepackage{ulem}

% These are your page settings.
\usepackage{enumitem}
\usepackage{amsthm}
\usepackage[
top=2.54cm,
bottom=2.54cm,
left=2.54cm,
right=2.54cm,
]{geometry}

%%% Other packages you may want. Just use overleaf if you want to learn use these %%%
%\usepackage{pgfplots} % For graphs (https://www.overleaf.com/learn/latex/Pgfplots_package)
%\usepackage{graphicx} % For images (https://www.overleaf.com/learn/latex/Inserting_Images)
%\usepackage{tikz} % For drawings (see below)

% These are just personal prefrences on formatting. The remove the automatic indent and sets a better paragraph skip and such things like that
\parskip=.07in
\parindent=0in
\setlist{noitemsep}
\pagestyle{fancy}
\hyphenpenalty=10000

% Don't worry about this lol. Credits to Prof Jason Siefkan for this code :)
\newcommand{\setheader}[6]{
	\lhead{{\sc #1}\\{\sc #2} ({\small \it \today})}
	\rhead{
		{\bf #3}
		\ifthenelse{\equal{#4}{}}{}{(#4)}\\
		{\bf #5}
		\ifthenelse{\equal{#6}{}}{}{(#6)}%
	}
}

\lstdefinestyle{mystyle}{
	basicstyle=\ttfamily,
	backgroundcolor=\color{white},
	commentstyle=\color{green!50!black}\itshape,
	keywordstyle=\bfseries\color{blue},
	numberstyle=\small\color{black},
	stringstyle=\color{green!50!black},
	breakatwhitespace=false,
	breaklines=true,
	captionpos=b,
	keepspaces=true,
	numbers=left,
	numbersep=5pt,
	showspaces=false,
	showstringspaces=false,
	showtabs=false,
	tabsize=4,
}

\lstset{style=mystyle}

% These are just a few shortcuts. Again, I took most of these from the source document from MAT223, but I added a few useful ones at the bottom
\newcommand{\R}{\mathbb{R}}
\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\C}{\mathbb{C}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\Proj}{\mathrm{proj}}
\newcommand{\Perp}{\mathrm{perp}}
\newcommand{\proj}{\mathrm{proj}}
\newcommand{\Span}{\mathrm{span}}
\newcommand{\Null}{\mathrm{null}}
\newcommand{\Rank}{\mathrm{rank}}
\newcommand{\ihat}{\widehat{i}}
\newcommand{\jhat}{\widehat{j}}
\newcommand{\khat}{\widehat{k}}
\newcommand{\True}{\texttt{True}}
\newcommand{\False}{\texttt{False}}
\newcommand{\DIV}{\mathrel\mid}
\newcommand{\NDIV}{\mathrel\nmid}
\newcommand{\MOD}{\pmod}
\newcommand{\bigO}{\mathcal O}
\newcommand{\powset}{\mathcal P}
\newcommand{\floor}[1]{\left \lfloor #1 \right \rfloor}
\newcommand{\ceil}[1]{\left \lceil #1 \right \rceil}
\newcommand{\abs}[1]{\left | #1 \right |}
\DeclareMathOperator{\lcm}{lcm}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Seriously. Don't worry about these. Credits to Francois Pitt from CSC165 for this code.

\renewcommand{\ULdepth}{1.8pt}

\let\seiresfb\bfseries\def\bfseries{\boldmath\seiresfb}
\let\seiresdm\mdseries\def\mdseries{\unboldmath\seiresdm}

\let\strong\textbf

\let\bigimplies\implies
\renewcommand*\implies{\mathrel{\Rightarrow}}
\let\bigiff\iff
\renewcommand*\iff{\mathrel{\Leftrightarrow}}

\let\emptysetorig\emptyset
\renewcommand*\emptyset{\varnothing}

\let\subsetorig\subset
\renewcommand{\subset}{\subseteq}

\let\epsilonorig\epsilon
\renewcommand{\epsilon}{\varepsilon}

% (Theses are induction stuff that we used in CSC165)
\newcommand{\proofheader}[1]{\noindent \uline{\textbf{#1}}}
\newcommand{\base}{\proofheader{Base Case}: }
\newcommand{\istep}{\proofheader{Inductive Step}: }

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Commands to make your code look neater. The black square for proofs and the spacing vv symbol (credits to Alfonso Gracia Saz for the idea.)
\renewcommand\qedsymbol{$\blacksquare$}
\newcommand{\vv}{\vspace{.2cm}}

% Left and right footers
\fancyfoot[L]{{\sc Left Footer here}}
\fancyfoot[R]{{\sc Right footer here}}

% You can use the following. I don't like it so I made an alternate below.
% \title{Insert Title Here}
% \author{Insert Author Here}
% \date{} % Removes the date

\begin{document}
	% Here is a header for the document
	\setheader{course}{title}{name}{student number}{}{}

	% Here is my preferred title for things like homeworks. Replace with valid information
	\begin{center}
		{\large {\bf title}\\
			{\bf course} \\
			{name}}
	\end{center}

	%Another way to make titles. I don't like it but you might so here you go!
	%\maketitle %<- Makes a title...
	%\tableofcontents %<- Makes a table of contents... I WOULDN'T USE FOR ASSIGNMENTS!

	% So your document starts here. Nothing befor the \begin{document} will be printed in your document. Everything afterwords will be printed.

	%% Here are some helpful commands%%
	%\section{section name} %<- Beginning a section header. If you don't want numbers, add a "*" between "section" and "{" (so \section*{section title})
	%\chapter{chapter name} %<- depending on your document class, this one may look prettier than section. ** DOES NOT WORK IN ARTICLE. ONLY WORKS IN REPORT,THESIS, OR BOOK **

	%%%% Environments (Things that have a \begin{...} and \end{...} and everything in between will be formatted cleanly) %%%%
	%\begin{proof} %<- do I really need to explain this one?
	%\begin{center} %<- Again, do I really need to explain this one
	%\begin{align*} %<- For a system of equations. The '*' is to remove the automatic numbering which can be annoying. The alignment properties are the same as the tabular below

	%\begin{tabular}{c} %<- For tables. See https://www.overleaf.com/learn/latex/Tables for more info.
	% Basically, you the second bracket is to tell where to align things. For example, if I want a table like this:
	%|    Title 1    |Title 2      |       Title 3|
	%| Item Number 1 |Item Number 2| Item Number 3|
	% I would put {|c|l|r|}, where the c, l, r are center, left, and right respectively.
	% For each column in the table, you type in latex the following:
	% >>> Item Number 1 & Item Number 2 & Item Number 3 \\
	% Where the '&' symbol signifys a new block and the '\\' signifies a new line.

	% For numbered lists, use enumerate (Replace enumerate with itemize if you want unnumbered lists)%= math symbols in display style. See https://www.overleaf.com/learn/latex/Lists for examples
	%\begin{enumerate}
	%	\item <write items here>
	%\end{enumerate}

	%%%% Code %%%%
	% I would recommend using the lstlisting package that I have setup. It allows for syntax highlighting which is REALLY cool. The other option is the verbatim environment, which simply displays whatever you put in.
	% \begin{lstlisting}[language=language]
	% 	content...
	% \end{lstlisting}
	% For example, I can write:
	% \begin{lstlisting}[language=Java]
	% 	System.out.println("Wow_this_is_a_String!");
	% \end{lstlisting}
	% Verbatim is the same, but you don't need to specify language.
	% For inline code, you can use the \texttt{} font, which will produce a typewriter type font.

	% \begin{tikzpicture} %<- For diagrams. YOU NEED TO UNCOMMENT \usepackage{tikz} TO USE THIS! This is really powerful but tricky, so I suggest you read this to learn: https://www.overleaf.com/learn/latex/TikZ_package
	% In short, tikz allows you to draw diagrams on latex, but it works on a coordinate system, so you may need to use math to draw things. The basic command for a line is
	% >>> \draw (start (as a tuple)) -- (end (as a tuple));
	% You can format this line using optional commands like this
	% >>> \draw[optional commands] (start (as a tuple)) -- (end (as a tuple));
	% A node is an object that doesn't show up, but you can put text over. For example, if you want to write "Hello" as a "textbox" at (1, 2), you can do so using the following:
	% >>> \node at (1,2) {Hello};

	%%%% FONTS %%%%
	%\textbf{ <insert text here>} % <- for bold text. Replace 'textbf' with 'textit' (shortcut on TexStudio as ctrl i) for italicize. 'textsc' for small caps (shortcut ctrl shift c) 'texttt' for typewriter (mainly to signify computer code - shortcut ctrl shift t).

	%%%% Generally helpful things %%%%
	%\newpage %<- makes a new page
	%\pagebreak % <- makes a page break. I suggest you use \newpage but this also works
	%\footnote{<footnote>} % <- Place where you want the note to appear. It will put the note in the footer
	%% \\ %= line break (not integer division)
	%$\backslash$ %= backslash. Yes that is important.
	%\; %<- for space

	%%%% MATH %%%%
	% So to type math, you would need to be in a math environment. My preferred way to do this is using the $ symbol for inline math (for example $x^2 + 3$) or $$ for display style (For example $$\int_{a}^{b} x^2 \; dx$$). You can also use \[\] and \(\) respectively, but I don't like that
	% Everything else is basically the same as piazza or Perusall!

	% FOR SYSTEM OF EQUATION (The '&' is to tell latex where to align to)
	%\[
	%f(x) = \left\{
	%\begin{array}{ll}
	%	a + b &= 1 \\
	%	b - a &= 5
	%\end{array}
	%\right.
	%\]
	%

	%%%% FILE MANAGEMENT %%%%
	% If you have a large file, you can use the:
	% >>> \input{filename}
	% Command. This allows you to be more organized by writing code in another file! Preambles are not required in the new file. Simply think of this as you replacing the code with this line of text, which tells it where to get the code. I suggest you use this after you get used to using latex!
\end{document}