\subsubsection{Rapier}\label{weapon:rapier}
Weapon, Sword, One-Handed, Regular, Melee\\
Size: M\\
Cost: 250 Gold

\textbf{Draw/Sheath}\\
2AP draw, 4AP sheath

\textbf{Thrust}\\
4 AP, AG to hit, Critical on 18, 19 and 20, \passus{1} Reach\\
2d8 + \sfrac{1}{3}\texttimes AG Piercing Damage

\textbf{Slash}\\
3 AP, AG to hit, Critical on 18, 19 and 20, \passus{1} Reach\\
1d8 + \sfrac{1}{3}\texttimes AG Cutting Damage


%!TEX root = forallx-ubc.tex
\chapter{SL Trees}
\label{ch.sl.trees}

So far we have learned one way to evaluate SL argument forms for validity: an argument is valid just in case no interpretation satisfies the premises but falsifies the conclusion. We can check to see whether an argument is valid by constructing truth tables, representing all of the possible interpretations, and checking to see whether any of them do so. This method has two main advantages: one is that it can be done in a formal and formulaic way  --- it requires no particular rational insight, just a straightforward application of the rules of SL. Another is that, assuming you follow the rules correctly, it will always succeed in identifying whether the argument is valid or not.

The truth-table method also has a significant disadvantage, however: it rapidly becomes extremely cumbersome for evaluating arguments using more than two or three atomic sentence-letters. To evaluate an argument like this one, you'd need to consider four rows of a truth-table:

\begin{earg}
\item[] $P\eiff Q$
\item[] $\enot Q$
\item[] $\enot Q \eor (P \eif \enot Q)$
\item[\therefore] $\enot P \eif Q$
\end{earg}

But change just two of the atoms to new letters, and the required table will grow exponentially. For this argument, you'd need sixteen rows.

\begin{earg}
\item[] $P\eiff Q$
\item[] $\enot A$
\item[] $\enot Q \eor (A \eif \enot B)$
\item[\therefore] $\enot P \eif B$
\end{earg}

For this one, you'd need two hundred fifty-six!
\label{8letterargument}
\begin{earg}
\item[] $A\eiff B$
\item[] $\enot B \eif (C \eor D)$
\item[] $E \eif \enot C$
\item[] $(\enot D \eand F) \eor G$
\item[] $\enot A \eand E$
\item[\therefore] $H \eor G$
\end{earg}

So it is useful to have alternate ways of proving entailments. In this chapter we will introduce a \emph{proof system} for SL. The proofs we will construct in this chapter are called \define{analytic tableaux}. Another name for them is \define{trees}, because of the way they `branch' out to represent possibilities. In Ch.\ \ref{ch.ND.proofs} we will examine a different proof system.

\section{Satisfiability and entailment}

The method of trees is most directly a way to test for satisfiability of a set of SL sentences. Since validity is definable in terms of satisfiability, it also gives a way to test for validity.

Recall from \S\ref{sec.semanticsSL} the definition of entailment in SL: $\metaSetX{}\models\metaA{}$ means that there is no interpretation that satisfies every sentence in \metaSetX{} but falsifies \metaA{}. (Remember, `\metaSetX{}' can stand for any set of SL sentences.) This in turn is equivalent to the claim that there is no interpretation that satisfies $\{\metaSetX{},\enot\metaA{}\}$. Unsatisfiability can be represented with a double-turnstile with $\bot$ on the right-hand side.

\factoidbox{
In general, $\metaSetX{}\models\metaB{}$ is equivalent to $\metaSetX{},\enot\metaB{}\models\bot$.
}

So if you want to tell whether an argument in SL is valid, check to see whether the premises, along with the negation of the conclusion, are jointly satisfiable. If they are, the argument is invalid. If they're not, the argument is valid. This is the basic idea of the tree method. We'll write down the sentences we're attempting to satisfy, and then we'll work through their consequences, in order to see whether it's possible to complete the task.

Let's begin by working through an example that illustrates the general idea, then come back to give precise rules for the tree method.

\section{An example: proving validity}

Let's attempt to prove whether this is valid in SL:

\begin{earg}
\item[] $A\eand B$
\item[] $\enot(C\eor D)$
\item[] $(\enot B \eor C) \eor E$
\item[\therefore] $E$
\end{earg}

To evaluate this argument, we consider whether it's possible to satisfy the three premises along with the negation of the conclusion. We begin by writing down the sentences we're going to attempt to satisfy.

\begin{prooftree}
{ % begin tree preamble
single branches,
close with=\ensuremath{\times},
%to prove={\{A\eand B, \enot(C\eor D), (\enot B\eor C)\eor E\} \vdash{} E}
} % end tree preamble
[A\eand B
[\enot(C\eor D), grouped
[(\enot B \eor C) \eor E, grouped
[\enot E, grouped, name=negconc
%	[A
%	[B, grouped
%		[\enot C
%		[\enot D, grouped
%			[\enot B \eor C
%				[\enot B, close]
%				[C, close]
%			]
%			[E, close]
%		]
%		]
%	]
%	]
]
]
]
]
\end{prooftree}

Notice that we're putting the \emph{negation} of the conclusion here in line (4), since we're looking to see whether it's possible to satisfy the premises and \emph{falsify} the conclusion. The sentences we write down at the beginning of the tree, we call the \define{root}. Trees are designed to show whether the root is satisfiable, and if so, how.

With proof trees, the idea is to continue writing down that which our previous lines \emph{already commit us to}. Consider the first line, $A \eand B$. For this conjunction to be true, both conjuncts must be true. So any interpretation that satisfies the top four lines must also satisfy both $A$ and $B$; the truth of those sentences is a commitment of what we already have. Each follow from what is already written down. We represent this by continuing the tree with those sentences:

\begin{prooftree}
{ % begin tree preamble
single branches,
close with=\ensuremath{\times},
%to prove={\{A\eand B, \enot(C\eor D), (\enot B\eor C)\eor E\} \vdash{} E}
} % end tree preamble
[A\eand B, checked
[\enot(C\eor D), grouped
[(\enot B \eor C) \eor E, grouped
[\enot E, grouped
	[A
	[B, grouped
%		[\enot C
%		[\enot D, grouped
%			[\enot B \eor C
%				[\enot B, close]
%				[C, close]
%			]
%			[E, close]
%		]
%		]
	]
	]
]
]
]
]
\end{prooftree}

In addition to introducing lines (5) and (6), we also add a check mark on (1), to indicate that we've now considered it and represented its consequences. Think of the check mark as saying that we've extended the tree in a way that encodes the information from this line.

Now consider line (2). This is a negated disjunction. Disjunctions are true any time either disjunct is true, so this \emph{negated} disjunction can only be true if \emph{both} disjuncts are \emph{false}. So we include new lines for \enot $C$ and \enot $D$, adding a check mark on line (2):

\begin{prooftree}
{ % begin tree preamble
single branches,
close with=\ensuremath{\times},
%to prove={\{A\eand B, \enot(C\eor D), (\enot B\eor C)\eor E\} \vdash{} E}
} % end tree preamble
[A\eand B, checked
[\enot(C\eor D), grouped, checked
[(\enot B \eor C) \eor E, grouped
[\enot E, grouped
	[A
	[B, grouped
		[\enot C
		[\enot D, grouped
%			[\enot B \eor C
%				[\enot B, close]
%				[C, close]
%			]
%			[E, close]
		]
		]
	]
	]
]
]
]
]
\end{prooftree}

Line (3) is a disjunction. Unlike the previous two cases, it doesn't tell us what categorically \emph{must} be the case; it says that one of two possibilities must be met. We represent this in our tree by \emph{branching} into two different columns:

\begin{prooftree}
{ % begin tree preamble
single branches,
close with=\ensuremath{\times},
%to prove={\{A\eand B, \enot(C\eor D), (\enot B\eor C)\eor E\} \vdash{} E}
} % end tree preamble
[A\eand B, checked
[\enot(C\eor D), grouped, checked
[(\enot B \eor C) \eor E, grouped, checked
[\enot E, grouped
	[A
	[B, grouped
		[\enot C
		[\enot D, grouped
			[\enot B \eor C
%				[\enot B, close]
%				[C, close]
			]
			[E]
		]
		]
	]
	]
]
]
]
]
\end{prooftree}

Think of these two branches as representing the idea that the root implies that at least one of these ways of continuing the tree must be possible.

So now we have two branches to consider. Start by examining the right branch. It gives an atomic sentence, $E$. Notice, however, that line (4) was \enot $E$. So if we're looking for a way to satisfy (1)-(4), this right branch isn't a possible interpretation. It requires $E$ to be true, and it also requires \enot $E$ to be true. If a branch contains any sentence and also contains its negation, we know that branch doesn't correspond to a possible interpretation. We'll consider this branch \emph{closed}, and mark this with an `\ensuremath\times'.

The left branch is another disjunction. It too branches out into its two disjuncts:

\begin{prooftree}
{ % begin tree preamble
single branches,
close with=\ensuremath{\times},
%to prove={\{A\eand B, \enot(C\eor D), (\enot B\eor C)\eor E\} \vdash{} E}
} % end tree preamble
[A\eand B, checked
[\enot(C\eor D), grouped, checked
[(\enot B \eor C) \eor E, grouped, checked
[\enot E, grouped
	[A
	[B, grouped
		[\enot C
		[\enot D, grouped
			[\enot B \eor C, checked
				[\enot B, close]
				[C, close]
			]
			[E, close]
		]
		]
	]
	]
]
]
]
]
\end{prooftree}

Both of these disjuncts also result in closed branches. The left branch at (10) is the negation of (6), and the right branch is the negation of (7). Now every branch in this tree is closed. This corresponds to the idea that every possible way there could be to satisfy (1)-(4) ended up requiring a contradiction. There is no interpretation that satisfies (1)-(4). In other words, the argument we began with was valid.

In the previous chapter we used the double turnstile, `$\models$', to represent entailment. In this chapter we will use a different but related symbol, the \define{single turnstile}, which looks like this: `$\vdash$'. Think of this symbol as representing \emph{provability}. So in proving that $E$ follows from the premises above, we're showing that {\{$A\eand B$, \enot$(C\eor D)$, $(\enot B\eor C)\eor E\} \vdash{} E$}.

Unsurprisingly, provability and entailment will turn out to be closely connected; we'll consider the connection in detail in Chapter \ref{ch.SLsoundcomplete}.

\section{An example: proving invalidity}
\label{sec.SLinvalidtree}

Let's work through another example to further illustrate the idea. In \S\ref{sec.SLtreerules} we'll learn the formal rules for trees in SL. Consider this argument form:

\begin{earg}
\item[] $(D\eor A) \eand \enot N$
\item[] $N \eor \enot A$
\item[\therefore] $\enot N \eand A$
\end{earg}

As before, we'll construct a tree that includes the premises and the negation of the conclusion at the top, and we'll attempt to find an interpretation satisfying those three lines. By convention, we write the claim we're attempting to prove at the top of the tree. We begin by processing line (1), which is a conjunction; its conjuncts are given in (4) and (5). Line (6) processes the disjunction in line (2), and the left branch closes.

\begin{prooftree}
{
to prove={\{(D\eor A) \eand \enot N, N \eor \enot A\} \vdash{} \enot N \eand A},
single branches,
close with=\ensuremath{\times},
}
[(D\eor A) \eand \enot N, checked
[N \eor \enot A, grouped, checked
[\enot (\enot N \eand A), grouped%, checked
	[D \eor A%, checked
	[\enot N, grouped
		[N, close
		]
		[\enot A
%			[\enot\enot N, close
%			]
%			[\enot A
%				[D]
%				[A, close]
%			]
		]
	]
	]
]
]
]
\end{prooftree}

Line (3) is a negated conjunction. A conjunction is true if and only if both conjuncts are true, so it is false if at least one conjunct is false. So to resolve (3), line (7) branches into the negation of each conjunct, \enot\enot$N$ and \enot$A$. The former closes because it's the negation of line (5). The final sentence requiring resolution is the disjunction on line (4); it branches, and one branch closes.

\begin{prooftree}
{
to prove={\{(D\eor A) \eand \enot N, N \eor \enot A\} \vdash{} \enot N \eand A},
single branches,
close with=\ensuremath{\times},
}
[(D\eor A) \eand \enot N, checked
[N \eor \enot A, grouped, checked
[\enot (\enot N \eand A), grouped, checked
	[D \eor A, checked
	[\enot N, grouped
		[N, close
		]
		[\enot A
			[\enot \enot N, close
			]
			[\enot A
				[D, open]
				[A, close]
			]
		]
	]
	]
]
]
]
\end{prooftree}

The $\uparrow$ indicates that the open branch ending in $D$ is a \emph{completed} branch. (We'll precisely define `completion' in \S\ref{sec.SL.tree.completion}.) What this means is that this branch represents a way to satisfy all three sentences at the root of the tree. In other words, this argument form is \emph{not} valid in SL. Furthermore, examining the open branch demonstrates an interpretation that satisfies the root. The branch includes three wffs that are either atoms or negated atoms: \enot $A$, $D$, and \enot $N$. So it suggests this interpretation:

\begin{displaymath}
\script{I} =
\left\{
	\begin{array}{ll}
	A = 0\\
	D = 1\\
	N = 0
	\end{array}
\right.
\end{displaymath}

You can verify this result by evaluating the premises and the conclusion of the argument we began with on this interpretation. Since there is an interpretation that satisfies the premises and falsifies the conclusion, our tree system proves the argument invalid.

\section{Resolution rules for SL trees}
\label{sec.SLtreerules}

Hopefully the examples we've worked through have given you a decent intuitive sense of the tree method for SL. Now let's get more precise about it, by giving formal rules for trees. You should be able to recognize the following rules as a generalization of the steps of the proofs given above.

We begin with \define{resolution} rules. These rules identify ways that tree branches can be extended, given sentences that are already in that branch. The rules depend on the main connective of the sentence in question. (If the main connective is negation, then they also depend on the main connective of the negand.)
\label{SL.treerules.start}
\subsection{Conjunction}

The rule for conjunction is that if you have a conjunction in a branch, you may make a linear extension of the branch that includes each conjunct, adding a check mark next to the conjunction. Using our Greek symbols once again as variables to stand in for any sentence of SL, any time you have this,

\begin{center}
\begin{prooftree}
{not line numbering,
single branches}
[\metaA{}\eand\metaB{}%, checked
%	[\metaA{}
%	[\metaB{}, grouped
%	]
%	]
]
\end{prooftree}
\end{center}

you may extend each open branch of the tree to this:

\begin{center}
\begin{prooftree}
{not line numbering,
single branches}
[\metaA{}\eand\metaB{}, checked
	[\metaA{}
	[\metaB{}, grouped
	]
	]
]
\end{prooftree}
\end{center}

It is important to remember once again that \metaA{} and \metaB{} here can stand for \emph{any} sentences of SL, including complex ones.

If you have extended a tree branch using the conjunction rule, we say that you have \emph{resolved} the conjunction. When sentences are resolved, they are marked with a check. In stating the resolution rules, we'll typically omit the check mark; here is the conjunction rule:

\factoidbox{
\begin{center}
\begin{prooftree}
{not line numbering}
[\metaA{}\eand\metaB{}
	[\metaA{}
	[\metaB{}, grouped
	]
	]
]
\end{prooftree}
\end{center}
}

\subsection{Negated conjunction}

If you have a negated conjunction, you may resolve it by branching into the negation of each conjunct. This makes sense because there are two ways for a negated conjunction to be true --- either conjunct can be false.

\factoidbox{
\begin{center}
\begin{prooftree}
{not line numbering,
single branches}
[\enot(\metaA{}\eand\metaB{})
	[\enot\metaA{}]
	[\enot\metaB{}]
]
\end{prooftree}
\end{center}
}

\subsection{Disjunction}
\label{subsec.DisjunctionTreeRule}
\begin{groupitems}
Disjunctions branch into each disjunct.

\factoidbox{
\begin{center}
\begin{prooftree}
{not line numbering,
single branches}
[\metaA{}\eor\metaB{}
	[\metaA{}]
	[\metaB{}]
]
\end{prooftree}
\end{center}
}
\end{groupitems}

\subsection{Negated disjunction}

Since disjunctions are true any time either disjunct is true, they are only false if both disjuncts are false. So negated disjunctions are resolved with a linear development containing the negation of each disjunct.

\factoidbox{
\begin{center}
\begin{prooftree}
{not line numbering,
single branches}
[\enot(\metaA{}\eor\metaB{})
	[\enot\metaA{}
	[\enot\metaB{}, grouped
	]
	]
]
\end{prooftree}
\end{center}
}

\subsection{Conditional}

Recall the characteristic truth table for the conditional in SL:

\begin{center}
\begin{tabular}{c|c|c}
\metaA{} & \metaB{} & \metaA{}\eif\metaB{}\\
\hline
1 & 1 & 1\\
1 & 0 & 0\\
0 & 1 & 1\\
0 & 0 & 1
\end{tabular}
\end{center}

Conditionals are true any time the antecedent is false, and also any time the consequent is true. We can represent this with a branching tree development, similar to a disjunction.

\factoidbox{
\begin{center}
\begin{prooftree}
{not line numbering,
single branches}
[\metaA{}\eif\metaB{}
	[\enot\metaA{}]
	[\metaB{}]
]
\end{prooftree}
\end{center}
}

\subsection{Negated conditional}

As in the case of disjunction, the negation of a conditional is a relatively strong claim --- the only way for a conditional to be false is for its antecedent to be true \emph{and} for its consequent to be false. We represent this with a linear development for negated conditionals:

\factoidbox{
\begin{center}
\begin{prooftree}
{not line numbering,
single branches}
[\enot(\metaA{}\eif\metaB{})
	[\metaA{}
	[\enot\metaB{}, grouped
	]
	]
]
\end{prooftree}
\end{center}
}

\subsection{Biconditional}

Biconditionals require a slightly different structure. A biconditional says that two sentences have the same truth value: either both are true, or both are false. Since these represent two different ways a biconditional can be true, our tree rules will develop biconditionals into two new branches. But unlike in the case of our other rules, each new branch will contain two sentences. One branch will have both sides of the biconditional; the other will have both sides' \emph{negations}:

\factoidbox{
\begin{center}
\begin{prooftree}
{not line numbering,
single branches}
[\metaA{}\eiff\metaB{}
	[\metaA{}
		[\metaB{}, grouped]
	]
	[\enot\metaA{}
		[\enot\metaB{}, grouped]
	]
]
\end{prooftree}
\end{center}
}

\subsection{Negated biconditional}
\begin{groupitems}
Negated biconditionals yield the same structure, but instead of new branches where each subsentence has the same truth value, they yield branches in which the subsentences have opposite values.

\factoidbox{
\begin{center}
\begin{prooftree}
{not line numbering,
single branches}
[\enot(\metaA{}\eiff\metaB{})
	[\metaA{}
		[\enot\metaB{}, grouped]
	]
	[\enot\metaA{}
		[\metaB{}, grouped]
	]
]
\end{prooftree}
\end{center}
}
\end{groupitems}

\subsection{Double negation}

There is one more resolution rule, for double-negated sentences. One resolves a double negation by developing a linear branch with the negand of the negand --- i.e., the wff obtained from removing both negations from the left of the double-negation.

\factoidbox{
\begin{center}
\begin{prooftree}
{not line numbering, single branches}
[\enot\enot\metaA{}
	[\metaA{}]
]
\end{prooftree}
\end{center}
}
\label{SL.treerules.end}
\subsection{Resolution rules summary}

These nine resolution rules describe, in purely formal terms, how to resolve most sentences of SL. The only exceptions are atomic sentences, and atomic sentences with a single negation sign in front of them. In our formal proof system, atoms and negated atoms have a special role to play; they are not resolved. In developing a tree proof, you may apply any of these rules to any unresolved sentence (other than an atom or negated atom) in the branch. Mark resolved sentences with a check.

Note that it is \emph{not} a rule that sentences must be resolved in the order in which they appear in the tree; one can save sentences and come back later to resolve them. However, if you are resolving a sentence after the tree has already done some branching, you must perform the resolution rule under \emph{each} open branch that includes that sentence. Here is a tree illustrating the issue.

\begin{prooftree}
{
}
[(D \eand \enot R) \eor Q, name={p1}, checked
[\enot Q \eor R, grouped, checked, name={p2}
	[D \eand \enot R, just={\eor}:p1, name={and}, checked
		[\enot Q, just={\eor}:p2
			[D, just={\eand}:and
				[\enot R, grouped, open]
			]
		]
		[R, name={R}
			[D, just={\eand}:and
				[\enot R, grouped, close={:R,!c}]
			]
		]
	]
	[Q
		[\enot Q, close={:!u,!c}
		]
		[R, open
		]
	]
]
]
\end{prooftree}

This tree has two disjunctions in the root, so it will involve branching multiple times. We could start with either one, but this tree resolved line (1) first, branching into the two disjuncts at line (3). The `1 \eor' to the right at that line is the explanation for what is happening there: it indicates that the disjunction rule was applied to line (1). When the second disjunction is processed at line (4), it needs to be done on both branches, since both branches include line (2). That's why our two branches split into four at line (4). One branch closes at that point. At line (5), the conjunction at line (3) is processed. This needs to be done on both branches that include that conjunction, but not the right-most branch, which does not include it. Resolving a sentence affects everything below it in the tree in its own descendent branches, but it doesn't affect branches that are off to the side.

Besides the tree resolution rules, there are two more rules to our proof system.

\section{Branch closure rules}

In the tree examples above, we \emph{closed} branches when they contained sentences along with their negations. Here is our general rule for tree closure:

\factoidbox{
If any branch contains some formula \metaA{}, and also contains \enot\metaA{}, that branch is \define{closed}. Any branch that is not closed is \define{open}. We mark closed branches with the `\ensuremath{\times}' symbol.

A tree is \define{closed} if and only if every branch in that tree is closed. A tree is \define{open} if and only if it is not closed.
}

If every branch is closed, the method proves that the root is unsatisfiable.

Note that `\metaA{}' stands in for \emph{any} sentence of SL; it is not part of the closure rule that one must have an atomic sentence and its negation. Consider for example this tree:

\begin{prooftree}
{
}
[(\enot\enot S \eand T) \eif (\enot P \eiff (Q \eor R)), checked
[\enot\enot S \eand T, grouped, name={p2}
[\enot (\enot P \eiff (Q\eor R)), grouped, name={p3}
	[\enot(\enot\enot S \eand T), close={:p2,!c}
	]
	[(\enot P \eiff (Q \eor R)), close={:p3,!c}
	]
]
]
]
\end{prooftree}

In this tree, we only resolved the first sentence, using the conditional rule, and the tree immediately closed. The line numbers under the closure symbols specify exactly why the branches close: the left branch developed into the negation of (2), and the right branch developed into the negation of (3). Notice that we \emph{could} have developed lines (2) and (3), using the conjunction or negated biconditional rules, respectively, but this would have resulted in a more complicated tree that ultimately yielded the same result. Similarly, if we hadn't noticed that both branches close at line (4), we could have continued processing the tree, developing the left branch using the negated conjunction rule on (4), and developing the right branch using the biconditionals rule. But this too would have involved needless complication for no benefit. (Eventually, the tree would have closed with simpler sentences too.) You can save yourself a lot of work by noticing when branches are ready to mark as closed, and by thinking a bit strategically about which sentences to resolve first.

\section{Branch completion rules}
\label{sec.SL.tree.completion}
Here is the final piece of our formal proof system. In our examples above we have used `$\uparrow$' to indicate that an open branch is complete. We need a formal characterization of branch completion.

In SL, a \define{resolvable} wff is a wff for which we have a resolution rule. That is, anything other than an atom or a negated atom.

\factoidbox{
A branch is \define{complete} if and only if either it is closed, or every resolvable sentence in that branch has been resolved.

A tree is complete if and only if every branch in that tree is complete.}

If there is at least one open branch in a completed tree, the tree method indicates that that branch corresponds to an interpretation that satisfies the root.

We use the `$\vdash{}$' symbol to indicate tree closure. `$\metaSetX{}\vdash{} \bot$' means that a tree with root \metaSetX{} closes. We use `$\nvdash{}\bot$' to indicate that a completed tree remains open.

We will also define our single turnstile symbol so that $\metaSetX{}\vdash{} \metaA{}$ is equivalent to $\metaSetX{}, \enot\metaA{}\vdash{}\bot$. So the latter is a way of saying that the tree method shows that the argument from $\metaSetX{}$ to $\metaA{}$ is valid.

This completes the introduction of the formal rules for the SL tree proof system. In the remainder of this chapter, we'll work through some more examples and provide some advice for working through trees more efficiently. In Chapter \ref{ch.SLsoundcomplete} we'll explain why the tree method works, and prove why it's a good one.

\section{Resolution order}

The resolution rules do not specify the order in which you should resolve sentences in a tree; you are free to resolve in any order you like. But some ways of processing trees are more efficient than others. Here is an example to illustrate. Suppose we want to consider whether $\{\enot (C \eand A), D \eiff C, A \eor B, \enot B\}\vdash{}\bot$. So we put those sentences in the root of a tree. Here's what happens if we resolve the sentences in the order in which they're given:

\begin{prooftree}
{
}
[\enot (C \eand A), checked, name=p1
[D \eiff C, checked, grouped, name=p2
[A \eor B, grouped, checked, name=p3
[\enot B, grouped, name=p4
	[\enot C, just={\enot\eand}:p1
		[D, just={\eiff}:p2
			[C, grouped, close]
		]
		[\enot D
			[\enot C, grouped
				[A, just={\eor}:p3, open]
				[B, close]
			]
		]
	]
	[\enot A
		[D
			[C, grouped
				[A, close]
				[B, close]
			]
		]
		[\enot D
			[\enot C, grouped
				[A, close]
				[B, close]
			]
		]
	]
]
]
]
]
\end{prooftree}

Again, the justifications for particular steps are added to the right of the tree for clarity. For example, `1\enot\eand{}' indicates that line (5) was produced by performing the negated conjunction resolution rule on line (1). This tree remains open, offering this interpretation that satisfies the root:

\begin{displaymath}
\script{I} =
\left\{
	\begin{array}{ll}
	A = 1\\
	B = 0\\
	C = 0\\
	D = 0
	\end{array}
\right.
\end{displaymath}

This tree reminds us again that if a sentence in a tree has multiple open branches below it, then resolving that sentence requires that the resolution be performed in \emph{each} open branch below. That's why, for example, resolving line (3) at line (8) requires three different new branchings. (We do not resolve under the left-most column, because it is already closed.) So when there are more open branches, resolving sentences requires more work on the tree. Consequently, it is sometimes a good idea to think a bit strategically, and close off parts of the tree earlier, to save yourself some work.

The example above is a perfectly fine instance of a completed tree, and it does yield the correct answer. However, it's possible to get there with less work, by choosing to resolve sentences that will close off branches right away. Here is another way of developing a tree with the same root as in the previous example. This version begins by resolving line (3), because doing so will close off one of its new branches immediately. It then continues to resolve in the most efficient order:

\begin{prooftree}
{
}
[\enot (C \eand A), checked, name=p1
[D \eiff C, checked, grouped, name=p2
[A \eor B, grouped, checked, name=p3
[\enot B, grouped, name=p4
	[A, just={\eor:p3}
		[\enot C, just={\enot\eand:p1}
			[D, just={\eiff:p2}
				[C, grouped, close]]
			[\enot D
				[\enot C, grouped, open]]
		]
		[\enot A, close]
	]
	[B, close]
]
]
]
]
\end{prooftree}

This tree gets us to the same result much more quickly, pointing to the same interpretation that satisfies the root. As a general rule of thumb, it's good advice to look a step ahead, and resolve sentences that won't lead to new open branches, before resolving ones that will. For just the same reason, it is usually more efficient to resolve those sentences that have linear rules before those that have branching rules.

\section{Choosing the right root}
\label{sec.sl.treeroots}

Trees are used to indicate whether the root is satisfiable. We test arguments for validity with a tree by putting their premises along with the negation of their conclusions in the root; if the tree remains open, indicating that the set is satisfiable, that means it's possible to satisfy the premises while falsifying the conclusion, which means the argument is invalid. If the tree closes, that suggests the argument is valid.

But trees can be used to evaluate many other kinds of questions, aside from validity. Any question that can be converted into a question about satisfiability can be answered with a tree. For example, if you want to find out whether a set of sentences is consistent, put that set of sentences into the tree. A set of sentences is consistent if and only if there is an interpretation satisfying it, so you can use a tree to find out.

What if you want to find out whether a sentence is a tautology? A tautology is a sentence that is satisfied by \emph{every} interpretation. To test this claim with a tree, we'll attempt to find an interpretation that falsifies it, by putting its \emph{negation} in the root. If this tree remains open, it shows us how to satisfy the negation, which means that the sentence is \emph{not} a tautology. If it closes, then the negation is unsatisfiable, which means the sentence \emph{is} a tautology.

To find out whether a sentence is a contradiction, check and see whether it is possible to satisfy it, by putting it in the root of the tree. If the tree remains open, the sentence is satisfiable, which means it's not a contradiction. If the tree closes, then it is a contradiction.

To use a tree to answer a question, the first step is always to convert the question into a question about whether some sentence, or set of sentences, is satisfiable.

\practiceproblems
If you want additional practice, you can construct trees for any of the SL arguments and entailment claims in the exercises for the previous two chapters.

\solutions
\problempart
\label{pr.sl.treeroot}
To evaluate each of the following claims with a tree, (a) what would you put in the root of the tree?, and (b) if the tree closes, does that show that the claim is true or false?
\begin{earg}
\item $\{P, P \eif Q, Q \eif \enot P\} \vdash{}\bot$
%(a) $\{P, P \eif Q, Q \eif \enot P\}$, (b) true
\item $(P \eif Q) \eiff (Q \eif P)$ is a tautology.
%(a) $\enot((P \eif Q) \eiff (Q \eif P))$, (b) true
\item The following argument is valid:
	\begin{ekey}
		\item[] $P \eand Q$
		\item[] $\enot R \eif \enot Q$
		\item[\therefore] $P \eand R$
	\end{ekey}
%(a) $\{P \eand Q, \enot R \eif \enot Q, \enot (P \eand R)\}$, (b) true
\item There is no interpretation that satisfies $A \eor B$, $B \eif C$, and $A \eiff C$ without also satisfying C.
%$\{A \eor B, B \eif C, A \eiff C, \enot C\}, (b) true
\item $A \eiff \enot A$ is a contradiction.
%(a) $A \eiff \enot A$, (b) false
\item Every interpretation satisfying $P$, $P \eif Q$, and $\enot Q$ also satisfies $A$.
%(a) $\{P, P \eif Q, \enot Q, \enot A\}$
\item There is at least one interpretation that satisfies $P \eif Q$, $\enot P \eor \enot Q$, and $Q \eif P$.
%(a) $\{P \eif Q, \enot P \eor \enot Q, Q \eif P\}, (b) false.
 \end{earg}

\solutions
\problempart
\label{pr.sl.agtree}
Evaluate the argument given on p.\ \pageref{8letterargument} by constructing a tree. If it is invalid, give a model demonstrating it so.

\solutions
\problempart Evaluate each claim from Part \ref{pr.sl.treeroot} by constructing a tree. If applicable, give the interpretation that demonstrates the claim true or false.
\label{tree.examples}

\problempart Recall the discussion of the Sheffer stroke in Chapter \ref{ch.TruthTables}, page \pageref{pr.altConnectives}.  That connective, if added to SL, would have this characteristic truth table:

\begin{center}
\begin{tabular}{c|c|c}
\metaA{} & \metaB{} & \metaA{}$|$\metaB{}\\
\hline
1 & 1 & 0\\
1 & 0 & 1\\
0 & 1 & 1\\
0 & 0 & 1
\end{tabular}
\end{center}

What would be appropriate tree resolution rules for the Sheffer stroke, or the negated Sheffer stroke?


\chapter*{Appendices}
\addcontentsline{toc}{chapter}{Appendices}

\appendix

\chapter{Rotation curves}\label{app:rotcurves}
A rotation curve shows the circular velocity as a function of
radius. We discuss here three different types of rotation curves, and
show examples of them in Figure~\ref{rotcurve}.

\section{Solid-body rotation}
Think of a solid turntable, or a rotating DVD-disc. It rotates with a
constant angular speed $\Omega = \frac{V}{R} = {\rm constant}$.
The circular velocity is therefore proportional to the radius:
\begin{equation}
V \propto R.
\end{equation}

\section{Keplerian rotation: the Solar system}
\label{sect:kepler}
\begin{figure}[htbp]
\begin{center}
\includegraphics[width=8.2cm]{../figures/cdrot.pdf}
\includegraphics[width=8.2cm]{../figures/solsystrot.pdf}
\includegraphics[width=8.2cm]{../figures/mwrot.pdf}
\end{center}
\caption{\emph{Top left}: solid body rotation (e.g. a rotating cd
  disc). \emph{Top right}: Rotational velocities of the planets of the
  Solar system. The line corresponds to Kepler's law
  (eq.~(\ref{kepler})) to which the planets show excellent agreement.
  The distance scale shown is in so-called \emph{Astronomical Units}
  (AU) which is the distance between the Earth and the Sun. $1
  \text{AU}=150\cdot 10^6 \text{km}$. \emph{Bottom}: the actual
  rotation curve of the Milky Way (dots) and a model describing it
  (solid line). Because there mass in the Milky Way is distributed
  across the galaxy, the rotation curve does not follow the Keplarian
  case, as it would if all the mass was to be in the center.(Data from Sofue et al, 2009) }
\label{rotcurve}
\end{figure}

In the Solar system, the planets have a negligible mass compared to
the mass of the Sun. Therefore the center of mass of the Solar system
is very close to the center of the Sun. The centrifugal acceleration
from the planet's circular velocities counterbalances the
gravitational acceleration:
\begin{equation}
\frac{V^2}{R}= \frac{GM}{R^2}
\end{equation}
where $M$ is the mass and $G$ the gravitational constant.  The
rotation curve is said to be {\em Keplerian}, and the velocities
decrease with increasing radius:
\begin{equation}
\boxed{V_{\rm Keplerian}(R) = \sqrt{\frac{GM}{R}}.}
\label{kepler}
\end{equation}

\section{The rotation curve of a spiral galaxy}

Similarly, the rotation curve of a galaxy $V(R)$ shows the circular
velocity as a function of galacto-centric radius.  In contrast to the
rotation curve of systems like the Solar system with a large central
mass, most galaxies exhibit flat rotation curves, that is, $V(R)$
doesn't depend on $R$ beyond a certain radius:

\begin{equation}
\boxed{V_{\rm Galaxy}(R) \sim {\rm constant}}
\end{equation}

The angular velocity varies as $\Omega \propto 1/R$. Matter near the
center rotates with a larger angular speed than matter farther away.

At large radii, the velocities are {\it significantly larger} than in
the Keplerian case, and this is an indication of the existence of
additional matter at large radii. This is an indirect way to show the
existence of dark matter in galaxies.

\chapter{Early history of 21 cm line observations}\label{app-history}

The story of the discovery of the 21~cm line of hydrogen is a
fascinating one because it began during the Second World War, when
international scientific contacts were disrupted and some scientists
were struggling to carry out research.

In 1944, H.C. van de Hulst, a student in Holland, scientifically
isolated because of the Nazi occupation of his country, presented a
paper at a colloquium in Leiden in which he showed that the hyperfine
levels of the ground state of the hydrogen atom produce a spectral
line at a wavelength of about 21~cm, and that it could be detectable
in the Galaxy.  An article was published in a Dutch journal (Bakker
and van de Hulst 1945).

After the war, efforts were made in several countries to design and
construct equipment to detect the spectral line. It was first observed
in the United States by Ewen and Purcell on 21 March 1951; in May the
same year, it was observed by Muller and Oort (1951) in Holland. Both
papers were published in the same issue of the journal {\it Nature}.
Within two months Christiansen and Hindman (1952) in Australia had
detected the line.

\begin{figure}[h]
  \centering
  \includegraphics[width = 8 cm]{../figures/ev2.pdf}
  \caption{ H.I. (''Doc'') Ewen together with his antenna
	  during a visit to NRAO-Green Bank in May, 2001. He had not seen
	  his antenna since 1956!. Credit: NRAO / AUI / NSF. }
  \label{fig:evpu}
\end{figure}

The first systematic investigation of HI in the Galaxy was made in
Holland by van de Hulst, Muller, and Oort (1954).  The Dutch group
used a reflector from a German radar receiver of the ``Great
W\"urzburg'' type, 7.5~m in diameter.  The beamwidth at $\lambda$21~cm
was 1$\deg$.9 in the horizontal direction and 2$\deg$.7 in the
vertical direction.

Christiansen and Hindman used a section of a paraboidal reflector,
with a beamwidth of about~2$\deg$.


\subsubsection{\stid{4.14} ECP SZ: Fast, Effective, Parallel Error-bounded Exascale Lossy Compression for Scientific Data}

\paragraph{Overview}

Extreme scale simulations and experiments are generating more data than can be stored, communicated and analyzed. Error-bounded lossy compressor is critical because it can get a very high compression ratio while still respecting data fidelity based on user's requirement on compression errors.
%Current lossless compression methods suffer from low compression ratio and do not adequately address the limitations in storage bandwidth and storage space of projected exascale systems. Existing lossy compressors are not covering the needs of many ECP applications.

The VeloC-SZ project is extending and improving the SZ error-bounded lossy compressor for structured and unstructured scientific datasets. SZ offers an excellent compression ratio as well as very low distortion and compression time. Further work is essential, however, to improve our SZ lossy compressor for ECP scientific datasets, while ensuring that user-set error controls are respected. Specifically, we are: (i) optimizing SZ compression ratios, accuracy and speed based on end-user needs (ii) refactoring SZ in C++ to support a composable compression framework and all data types used in ECP applications, (iii) integrating SZ in ECP client applications, (iv) developing the GPU version of SZ which supports multiple supercomputers with different architectures (such as Aurora, Frontier and Summit), and (v) improving robustness and testability. %Our goal is to produce a high-quality lossy compressor responding to the needs of ECP exascale applications and experiment users. To this end,
We are working with multiple ECP application teams, including ExaSky cosmology teams (HACC), molecular dynamics simulations groups (EXAALT), x-ray laser imaging experimentalists (ExaFEL), and computational chemists (NWChem-X, GAMESS) to optimize SZ for their applications and to harden SZ.

\paragraph{Key Challenges}

SZ faces several key challenges:
\begin{itemize}
\item \textbf{Improving Prediction Accuracy} The core step in SZ is the point-wise data prediction for different datasets. The higher the prediction accuracy, the higher the compression quality in general. However, how to design an effective predictor that can adapt to diverse datasets is very challenging.
\item \textbf{Parameter tuning}: One challenge in optimizing lossy compression for scientific applications is the large
diversity of scientific data, dimensions, scales, and dynamic data changes in both space and time. Each application requires specific parameters tuning and in some cases, a specific compression pipeline, which is non-trivial to implement.
\item \textbf{Implementation \& optimization over GPU}: The third challenge is the sophisticated design in different stages of the SZ (such as data prediction, Huffman tree construction, Huffman encoding), which makes the development of efficient GPU kernels non-trivial.
\item \textbf{Diverse integration schemes}: The fourth challenge is the diversity of the integration schemes for the different ECP client applications: HACC integrates SZ in a proprietary I/O library (GIO), Exafel integrates SZ directly in the LCLS data processing pipeline. GAMESS integrates SZ in the application directly replacing some code sections. EXAALT integrates SZ for saving storage space.
\item \textbf{Portable support for GPU}: Optimization of SZ for Aurora and Frontier requires writing portable accelerator codes that are non trivial for complex compression pipeline.
\item \textbf{Improve development robustness and testability}: The SZ testing infrastructure (unit test, correctness test, performance test, regression test, continuous integration) will need to be adapted and its performance optimized for the new C++ implementation. A template based approach must be used to improve robustness, debugging and testability.
\end{itemize}

\paragraph{Solution Strategy}

As for the first two challenges, we develop a novel, adaptive predictor based on a dynamic spline interpolation which is combined with the classic Lorenzo predictor, such that the rate distortion (bit rate vs. data distoriton PSNR) can be improved significantly.

As for the third challenge, we keep exploring the new strategies to improve the GPU kernel performance especially for bottleneck stages of cuSZ -- GPU version of SZ. This requires in-depth understanding of the bottleneck of cuSZ and numerous performance tuning tests to explore new strategies.

As for the fourth challenge, we refactor SZ in C++, starting from the current C version. This refactoring is the perfect occasion to implement a new more modular design of SZ, capable of integrating more stages in the compression pipeline and of selecting compression stages based on specific application data features.

Concerning the fifth challenge, we are in contact with ALCF and OLCF as well as with vendors to access simulators and early systems. We also develop the portable GPU implementation of SZ for different GPUs by leveraging corresponding libraries/toolkits such as oneAPI and HIP.

We are continuously improving the robustness and testability. We integrated SZ into ORNL CI environment, and often discuss potential issues with application teams when needed.

\paragraph{Recent Progress}

We summarize all 10+ different versions of SZ in our official website (http://szcompressor.org), including CPU version, GPU version, customize version for applications such as EXAALT and GAMESS.
%Specifically, The flagship products include CPU version of SZ (the classic C version), composable version of SZ (a.k.a., SZ3 in C++), dynamic interpolation based SZ -- Interp-SZ (particularly effective for Seismic imaging data and turbulent flow data), CUDA version of SZ -- cuSZ, Kokkos version of SZ, ultrafast version of SZ -- SZx, specific version customized for GAMESS -- Pastri-SZ, specific version customized for ExaFEL -- Roibin-SZ, specific version customized for ExaLT -- MMD-SZ, FPGA version of SZ -- waveSZ, etc.

We significantly improved compression quality and performance for SZ. (1) We improved the rate distortion significantly by a dynamic spline interpolation method, which is particularly effective in compressing smooth datasets with wave patterns such as turbulent flow data and quantum chemistry data (QMCPack). (2) We customized an effiicent error-bounded lossy compressor - called MMD-SZ for molecular dynamics (MD) applications such as LAMMPS and ExaALT, by leveraging the potential data characteristics of MD simulation datasets. The compression ratio has been improved by about 100+\% over the generic version of SZ.

We have developed SZ3, which breaks down different stages of SZ to form a loosely-coupled model, such that the users can construct a new compressor by customizing each compression step conveniently. %By leveraging the SZ3 to compose new compression pipeline, we have successfully developed MMD-SZ for MD simulation datasets, which can improve the compressio ratio by 100+\% with the same distortion, and also improved the compression quality significantly for Advanced photon Source (APS) datasets.

We also significantly improved the GPU kernel performance of SZ's compression stage. We have released the cuda-based SZ (cuSZ) 0.2.9.1 and tested its performance on V100 GPU and A100 GPU. We identified that the bottleneck of cuSZ is its Huffman encoding stage and dictionary encidng stage. (1) We develop an efﬁcient parallel codebook construction on GPUs that scales effectively with the number of input symbols. (2) We propose a novel reduction based encoding scheme that can efﬁciently merge the codewords on GPUs. (3) We optimize the overall GPU performance by leveraging the state-of-the-art CUDA APIs such as Cooperative Groups. (4) We evaluate our Huffman encoder thoroughly using six real-world application datasets on two advanced GPUs and compare with our implemented multi-threaded Huffman encoder. Experiments show that our solution can improve the encoding throughput by up to 5.0$\times$ and 6.8$\times$ on NVIDIA RTX 5000 and V100, respectively. Figure \ref{fig:sz-huffman} illustrates the parallel design of the Huffman encoding in cuSZ.
\begin{wrapfigure}{l}{0.47\textwidth}
  \includegraphics[width=0.4\textwidth]{projects/2.3.4-DataViz/2.3.4.14-VeloC-SZ/SZ-Huffman}
  \caption{Parallelism implemented for Huffman coding’s subprocedures (kernels).
``sequential” denotes that only 1 thread is used due to data dependency.
``coarse-grained” denotes that data is explicitly chunked. ``fine-grained” denotes
that there is a data-thread mapping with little or no warp divergence.}%
  \label{fig:sz-huffman}%
\end{wrapfigure}

We also signficantly improved the GPU kernel performance for SZ's overall workflow and decompression stage in particular. (1) We propose an efficient compression workflow to adaptively perform run-length encoding and/or variable-length encoding. (2) We derive Lorenzo reconstruction in decompression as multidimensional partial-sum computation and propose a fine-grained Lorenzo reconstruction algorithm for GPU architectures. (3) We carefully optimize each kernel by leveraging state-of-the-art CUDA parallel primitives. (4) We evaluate the new version of cuSZ using seven real-world HPC application datasets on V100 and A100 GPUs. Experiments show the new implementation improves the compression throughputs and ratios by up to 18.4$\times$ and 5.3$\times$, respectively on the tested datasets.

Experiments on A100 GPU show that cuSZ's overall performance reaches up to 84$\sim$92GB/s based on both HACC and NYX datasets.

We ported cuSZ in active development onto the new backend: native DPC++, for the A21 platform.
DPC++ along with oneAPI SDK serves as the native programming language on A21 system. From DPC++ setup, we successfully executed our prototype kernels that can partially match the native CUDA kernel executed on NVIDIA V100.

We developed CI for SZ on ORNL CI environment and also added smoke test to Spack for SZ.

\paragraph{Next Steps} Our next efforts include: (1) Improve compression quality and performance for cuSZ, (2) Keep working on the compression quality improvement and integration of SZ in more ECP applications such as ExaFEL and EXAALT, and (3) evaluate the portable GPU version of SZ on ECP platforms.


\section{Ongoing Research}

My current research efforts are focus on the application of semantic-web related technologies to multidisciplinary data, especially biomedical data, which usually exhibits high degree of multidisciplinarity. I summarize my efforts in four topics.

\subsection{Multidisciplinary data-mining}

Application of information-mining technologies to multidisciplinary data, including text- and data-mining of biomedical related information, with a particular emphasis on the use of semantic similarity measures to that effect. Data sources include health records, metabolic pathway descriptions, scientific publications, digital representations of biomedical entities, \emph{etc}.

\subsection{Machine-learning in biomedical data}

Application of machine-learning algorithms (in particular neural networks and deep-learning algorithms) to biomedical data. I have not yet started to research this topic, but I will in the very near future.

\subsection{Semantic similarity}

Exploration of several ideas to improve semantic similarity measures in a multidisciplinary content. For the moment, this consists in a set of measures of relevance of concepts in an ontology.

\subsection{Data quality}

Assessment of data quality by measuring some characteristics of the associated metadata, which including measuring the specificity and coverage of the metadata.


 \documentclass [12pt]{article}

\usepackage {amsmath}
\usepackage {amsthm}
\usepackage {amssymb}
\usepackage {graphicx}
\usepackage {float}
\usepackage {multirow}
\usepackage {xcolor}
\usepackage {algorithmic}
\usepackage [ruled,vlined,commentsnumbered,titlenotnumbered]{algorithm2e} \usepackage {array}
\usepackage {booktabs}
\usepackage {url}
\usepackage {parskip}
\usepackage [margin=1in]{geometry}
\usepackage [T1]{fontenc}
\usepackage {cmbright}
\usepackage [many]{tcolorbox}
\usepackage [colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage {enumitem}
\usepackage {xparse}
\usepackage {verbatim}
\usepackage{listings}
\usepackage{xcolor}
\lstset { %
    language=C++,
    backgroundcolor=\color{black!5}, % set backgroundcolor
    basicstyle=\footnotesize,% basic font setting
}
\newtheorem{theorem}{Theorem}
\newtheorem{remark}{Remark}

\DeclareTColorBox {Solution}{}{breakable, title={Solution}} \DeclareTColorBox {Solution*}{}{breakable, title={Solution (provided)}} \DeclareTColorBox {Instruction}{}{boxrule=0pt, boxsep=0pt, left=0.5em, right=0.5em, top=0.5em, bottom=0.5em, arc=0pt, toprule=1pt, bottomrule=1pt} \DeclareDocumentCommand {\Expecting }{+m}{\textbf {[We are expecting:} #1\textbf {]}} \DeclareDocumentCommand {\Points }{m}{\textbf {(#1 pt.)}}

\begin {document}

\vspace {1em}
\begin {Instruction}
Adapted From Virginia Williams' lecture notes.
\end {Instruction}

{\LARGE \textbf {COMP 285 (NC A\&T, Spr `22)}\hfill \textbf {Lecture 13} }

\begin{centering}
\section*{Hashing}
\end{centering}

\section{Hash tables}
A hash table is a commonly used data structure to store an unordered set of items, allowing constant time inserts, lookups and deletes (in expectation). Every item consists of a unique identifier called a key and a piece of information. For example, the key might be a Social Security Number, a driver's license number, or an employee ID number. The way in which a hash table stores a item depends only on its key, so we will only focus on the key here, but keep in mind that each key is usually associated with additional information that is also stored in the hash table.

A hash table supports the following operations:
\begin{itemize}
  \item Insert($k$): Insert key $k$ into the hash table.
  \item Lookup($k$): Check if key $k$ is present in the table.
  \item Delete($k$): Delete the key $k$ from the table.
\end{itemize}
Each operation will take constant time (in expectation).

\subsection{Implementation}
Let $U$ be the universe of all keys. For example, $U$ could be the set of all 64 bit strings. In this case $|U| = 2^64$. This is a very large universe, but we do not need to store all of these $2^64$ keys, we only need to store a subset $S \subset U$. Suppose that we know that the size of the subset we will need to store is less than or equal to $n$, which is much less than the size of the universe $|U|$. In a hash table of size $n$, each key  $k \in U$ is mapped to one of n ``buckets'' by a hash function $h : U \to \{1, 2, \cdots , n\}$. Since the universe $U$ is much larger than $n$, multiple
keys could map to the same hash bucket. To accommodate this, each bucket contains a linked list of keys currently stored in that bucket.

\textbf{Example}

Suppose we have a hash table of size $n = 5$ with hash function $h(x) = 13x + 2 \mod 5$. After inserting the elements $\{1, 2, 4, 7, 8\}$ the hash table looks like this:

\begin{center}
\begin{tikzpicture}
\coordinate (0);
\foreach \t[count=\i from 0,evaluate=\i as\j using int(\i+1)] in {
  1  ,
  8  ,
  NIL  ,
  2 $\rightarrow$ 7,
  4
}
\node at(\i.south)[anchor=north,draw,minimum height=2em,minimum width=1.5em,outer sep=0pt](\j){}
    node at(\j.west)[align=right,left]{$B_{\i}$}
    node at(\j.east)[align=left,right,xshift=-.7em]{$\rightarrow$ \t};
\end{tikzpicture}
\end{center}

Where arrows denote pointers in the linked lists, and $B_2$ is empty. For example, $1$ is placed into bucket $B_0$ because $h(1) = 15 mod 5 = 0$.

\textbf{Time Complexity}
ith this setup, the time required to perform an Insert, Lookup, or Delete operation on key $k$ is linear in the length of the linked list for the bucket that key $k$ maps to. We just use the hash function to find the correct bucket for an input key k, and then search the corresponding linked list for the element, inserting or deleting if necessary. Note that an Insert could be performed in constant time by always inserting at the head of the list, but we first need to check if key $k$ is already present.

\textbf{Choice of size of hash table}
The hash table size is usually chosen so that the size of the hash table is at least as large as the maximum number of keys we will need to store at any point of time. If this condition is violated and the number of keys stored grows much larger than the size of the hash table, an implementation will usually increase the size of the table, and recompute the new table from scratch by mapping all keys to the bigger table. Our analysis ignores these complications and assumes that the number of keys is at most the hash table size.

\textbf{Potential problem with this implementation}
In order for the operations to be implemented efficiently, we would like the keys to be distributed uniformly amongst the buckets in the hash table. We might hope that all buckets have at most a constant number of keys mapped to them, so that all operations could be performed in constant time. But for any fixed choice hash function $h$, one can always produce a subset of keys $S$ such that all keys in $S$ are mapped to the same location in the hash table. In this case, the running times of all operations will be linear in the number of keys - far from the constant we were hoping for. Thus, for a fixed hash function $h$, it is impossible to give worst case guarantees for the running times of hash table operations.

\textbf{Possible Solutions}
There are two styles of analysis that we could use to circumvent this problem:

\begin{enumerate}
  \item Assume that the set of keys stored in the hash table is random, or
  \item Assume that the hash function $h$ is random.
\end{enumerate}

Both are plausible alternatives. The problem with the first alternative is that it is hard to justify that the set of keys stored in the hash table is truly random. It would be more satisfying to have an analysis that works for any subset of keys currently in the hash table. In these notes, we will explore the second alternative, i.e., assume that the hash function $h$ is random.

\end{document}


\subsection*{Previous semesters' grade distributions}

Below is a history of previous semesters' grade distributions with approximate cutoffs.

\begin{table}[h]
\centering
\begin{subtable}[h]{0.85\textwidth}
        \centering
\begin{tabular}{l|ccccccccccccc}
Grade & WU & F   & D      & D+   & C-    & C     & C+    & B-    & B     & B+   & A-    & A      & A+ \\ \hline
$n$   &  6 & 7 & 0      & 0            & 10   & 9      & 7       & 11   & 8     &4       & 11      & 5      & 1 \\
\%ile     &  ---       & --- & ---  &  ---    & 28.6 & 41.8 & 53.1  & 58.2 & 65.3 & 75.5 & 84.7 & 92.9 & 100 \\ \hline
\end{tabular}
\caption{Fall, 2017. Total enrollment out of two sections save no-shows was $n=79$.}
\end{subtable}\\

 \begin{subtable}[h]{0.85\textwidth}
        \centering
\begin{tabular}{l|ccccccccccccc}
Grade & WU & F   & D      & D+   & C-    & C     & C+    & B-    & B     & B+   & A-    & A      & A+ \\ \hline
$n$   &  14          & 13 & 0      & 0     & 14    & 11   & 5       & 7     & 10    & 9     & 7      & 8      & 0 \\
\%ile     &  ---       & --- & ---  &  ---    & 28.6 & 41.8 & 53.1  & 58.2 & 65.3 & 75.5 & 84.7 & 92.9 & --- \\ \hline
\end{tabular}
\caption{Fall, 2016. Total enrollment out of three sections save no-shows was $n=98$.}
\end{subtable}\\

\begin{subtable}[h]{0.85\textwidth}
        \centering
\begin{tabular}{l|cccccccccccc}
Grade & F   & D      & D+   & C-    & C     & C+    & B-    & B     & B+   & A-    & A      & A+ \\ \hline
$n$   & 11    & 0      & 0     & 6      & 6     & 6      & 5     & 5     & 4     & 5      & 6      & 0 \\
\%ile & --- & ---  &  ---        & 21    & 32   & 46    & 57   & 66   & 75 & 82 & 91 & --- \\ \hline
\end{tabular}
\caption{Fall, 2015. Total enrollment out of three sections save no-shows was $n=56$.}
\end{subtable}\\

\begin{subtable}[h]{0.85\textwidth}
        \centering
\begin{tabular}{l|cccccccccccc}
Grade & F   & D      & D+   & C-    & C     & C+    & B-    & B     & B+   & A-    & A      & A+ \\ \hline
$n$   & 3    & 3      & 4      & 4      & 6     & 4      & 3     & 6      & 3     & 4      & 3      & 0 \\
\%ile & --- & 7.1  &  16.7 & 23.8 & 33.3 & 47.6 & 57.1 & 64.3 & 76.2 &  81.0 & 92.9 & --- \\ \hline
\end{tabular}
\caption{Spring, 2015. Total enrollment out of two sections save no-shows was $n=42$.}
\end{subtable}\\

\begin{subtable}[h]{0.85\textwidth}
        \centering
\begin{tabular}{l|cccccccccccc}
Grade & F & D & D+ & C- & C & C+ & B- & B & B+ & A- & A & A+ \\ \hline
$n$ & 6 & 5 & 1 & 5 & 4 & 4 & 4 & 6 & 4 & 2 & 6 & 1 \\
\%ile & --- & 14.6 &  25.0 & 35.4 & 43.8 & 52.1 & 62.5 & 75.0 & 66.6 &  83.3 & 87.5 & 100 \\ \hline
\end{tabular}
\caption{Fall, 2014. Total enrollment out of two sections save no-shows was $n=48$.}
\end{subtable}

\caption{Grade distributions of previous Math 241 courses.}
\end{table}
\FloatBarrier

I am not obligated to mirror these grade distribution and cutoffs this semester. Grade distributions vary from semester-to-semester based on the relative difficulty of exams, student ability and randomness in raw score clustering. The above is meant for informational purposes only. Do not come to me expecting a negotiation of your grade based on a previous semester's cutoff.


\documentclass{article} % Sets up formatting rules for the document. Article is a basic style used everywhere

\usepackage{amsmath} % For math stuff
\usepackage{amssymb} % For math stuff
\usepackage{pifont} % For math stuff
\usepackage{graphicx} % For inputting pictures
\graphicspath{{./images/}} % Set the location for your images. You should have a subfolder called images to keep the project clean
\usepackage[letterpaper, portrait, margin=1in]{geometry} % Sets up some margins. The defaults aren't good imo
\usepackage{authblk} % Lets us use a cool trick for making multiline author blocks. Useful for when we have to write our names and IDs. Gets too much for 1 line

\begin{titlepage} % Lets you define a cover page. You could probably add more stuff here if you want
    \title{CLASS NAME ASSIGNMENT NAME}

    % Fill up the \author block with your names and ids, separated by \and commands (for it to be multiline)
    \author[1]{}

\end{titlepage}

\begin{document} % Starts the document and applies the article style to it

    \maketitle % Makes the title page based on what you put above

    \newpage % You should enforce a new page after the title page

    \section*{Problem 1}

    \paragraph{Can do math inline: $2 + 2 = 4$ }

    \section*{Problem 2} % We can have sections...

        \subsection*{1.} % We can have subsections...

        \paragraph{Can also do it in block form:} % It's good to write in paragraph (or other) environments for some formatting rules to be autmatically applied. Avoids warnings

        \begin{align*}
            1+1+1+1 & = 4 \\
            2+2 & = 4 \\
            4 & = 4 \\
        \end{align*}

        \subsection*{2.}

        \paragraph{or with a label to the side:}

        \begin{align}
            \begin{split}
                1 + 2 &= 3\\
                1 &= 3 - 2
            \end{split}
          \end{align}

        \subsection*{3.}

        \begin{equation} \label{eq1}
            \begin{split}
            A & = \frac{\pi r^2}{2} \\
             & = \frac{1}{2} \pi r^2
            \end{split}
            \end{equation}

    \section*{Problem 3}

    \section*{Problem 4}

        \subsection*{1.}

        \subsection*{2.}

            \subsubsection*{(a)} % We can have subsubsections...

            \paragraph{}

            \subsubsection*{(b)}

            \subsubsection*{(c)}

    \section*{Problem 5}

    % Here's a generic table. Put stuff between the & symbols. The & are for proper alignment btw
    \begin{center}
        \begin{tabular}{ |l|c|c|c|c|c| }
         \hline
          &  &  &  &  &  \\ % This should be the column headings. Row headings are before the first & of each row
         \hline
          &  &  &  &  & \\
         \hline
          &  &  &  &  &  \\
         \hline
            &  &  &  &  &  \\
         \hline
        \end{tabular}
    \end{center}

    \section*{Problem 6}
        % Here's a cheap, easy way to put in pictures. Make sure to properly include your graphics path at the top.
        % It's commented to avoid errors because we have no pictures now

        % \begin{center}
        %     \includegraphics[scale=0.5]{hasse}
        % \end{center}

    \section*{Problem 7}

    \section*{Problem 8}

    \section*{Problem 9}

\end{document} % Ends the document


\section{Invisibility}\label{spell:invisibility}
\textbf{Cost:} 500 CP\\
\textbf{Requirements:}~\nameref{rune:dark} I,~\nameref{rune:arcana} II\\
\textbf{Spell, Active, Source(200 Gold), Repeatable, Memory}\\
For 4 AP and 10 Mana, you become invisible.
This costs 1 Mana per minute to uphold.
If you perform any other action than moving -- opening a door, attacking a creature, consuming a potion etc.
While being invisible, you need to pay one Mana per AP of that action, otherwise the invisibility ends.\\
Invisibility improves your ability to sneak.
Whenever you try to move stealthily while being invisible, you can roll twice, taking the higher result.
Any attack roll against an invisible creature is reduced by 10.\\
\\
Level Progression:\\
II: 1,000 CP, You can affect other creatures.
You have to touch them to do this, but if they're not willing they can not resist you.\\

III: 5,000 CP, You can pay 25 Mana instead, and don't have pay mana for just keeping the spell up.
You still have to pay upkeep while performing actions other than moving.
If you're casting the spell in this way, you can only affect yourself with it.\\


\graphicspath{{figures--proofs/}}
\addtocontents{toc}{\protect\mbox{}\protect\hrulefill\par}

\chapter{Additional rules for TFL}\label{s:Further}

\section{Derived rules}
The rules of our natural deduction system are systematic. There is an introduction and an elimination rule for each logical operator. But why these basic rules rather than others? Some natural deduction systems have this rule:
\factoidbox{
\begin{proof}
	\have[m]{ab}{\meta{A}\eor\meta{B}}
	\have[n]{ac}{\meta{A}\eif\meta{C}}
	\have[o]{bc}{\meta{B}\eif\meta{C}}
	\have[\ ]{c}{\meta{C}} \by{DIL}{ab,ac,bc}
\end{proof}}

Let's call this rule \define{dilemma} (DIL) It might seem as if there will be some proofs that we cannot do without this rule. But that is not the case. Any proof that you can do using the Dilemma rule can be done with basic rules of our natural deduction system. It will just take more steps. Consider this proof:

\begin{proof}
	\hypo{ab}{A \eor B} \by{PR}{}
	\hypo{ac}{A\eif C} \by{PR}{}
	\hypo{bc}{B \eif C}\by{PR}{}
	\open
		\hypo{nc}{\enot C}\by{AS}{}
		\open
			\hypo{a1}{A} \by{AS}{}
			\have{c1}{C}\ce{ac, a1}
			\have{nc1}{\enot C}\by{R}{nc}
		\close
		\have{na}{\enot A}\ni{a1-nc1}
		\open
			\hypo{b2}{B}\by{AS}{}
			\have{c2}{C}\ce{bc, b2}
			\have{nc2}{\enot C}\by{R}{nc}
		\close
		\have{b}{B}\oe{ab, na}
		\have{nb}{\enot B}\ni{b2-nc2}
	\close
	\have{c}{C} \ne{nc-nb}
\end{proof}

This proof demonstrates that the dilemma rule is not really necessary. Adding it to the list of basic rules would not allow us to derive anything that we could not derive without it.
Nevertheless, the it would be convenient. It would allow us to do in one line what requires eleven lines with the basic rules (and subproofs within subproofs!). So we will add it to the proof system as a derived rule.

A \define{derived rule} is a rule of proof that does not make any new proofs possible. Anything that can be proven with a derived rule can be proven without it. You can think of a short proof using a derived rule as shorthand for a longer proof that uses only the basic rules. Anytime you use the dilemma rule, you could always take ten extra lines and prove the same thing without it.

For the sake of convenience, we will add several other derived rules, all of which can be used in Carnap. One is \define{modus tollens} (MT).
\factoidbox{
\begin{proof}
	\have[m]{ab}{\meta{A}\eif\meta{B}}
	\have[n]{a}{\enot\meta{B}}
	\have[\ ]{b}{\enot\meta{A}} \by{MT}{ab,a}
\end{proof}}
\noindent We leave the proof of this rule as an exercise. Note that if we had already proven the MT rule, then the proof of the DIL rule could have been done in only five lines.
We will also add the \define{hypothetical syllogism} (HS) as a derived rule. We have already given a proof of it on p.~\pageref{HSproof}.
\factoidbox{
\begin{proof}
	\have[m]{ab}{\meta{A}\eif\meta{B}}
	\have[n]{bc}{\meta{B}\eif\meta{C}}
	\have[\ ]{ac}{\meta{A}\eif\meta{C}}\by{HS}{ab,bc}
\end{proof}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Rules of replacement}

We will now introduce some derived rules that may be applied to part of a sentence. These are called \define{rules of replacement}, because they can be used to replace part of a sentence with a logically equivalent expression. One simple rule of replacement is \define{commutivity} (abbreviated Comm), which says that we can swap the order of conjuncts in a conjunction or the order of disjuncts in a disjunction. We define the rule this way:
\factoidbox{
\begin{center}
\begin{tabular}{rl}
$(\meta{A}\eand\meta{B}) \Longleftrightarrow (\meta{B}\eand\meta{A})$\\
$(\meta{A}\eor\meta{B}) \Longleftrightarrow (\meta{B}\eor\meta{A})$\\
$(\meta{A}\eiff\meta{B}) \Longleftrightarrow (\meta{B}\eiff\meta{A})$
& Comm
\end{tabular}
\end{center}}

The bold arrow means that you can take a subformula on one side of the arrow and replace it with the subformula on the other side. \textbf{The arrow is double-headed because rules of replacement work in both directions.} (In this case, and in the other cases below, the name `Comm' applies to each of the three versions, even though they use different logical operators.)

Consider this argument: $(M \eor P) \eif (P \eand M)$ \therefore\ $(P \eor M) \eif (M \eand P)$. Although it is obviously valid, a proof of it using only the basic rules would be quite long. With the Comm rule, however, we can easily provide a proof:

\begin{proof}
	\hypo{1}{(M \eor P) \eif (P \eand M)} \by{PR}{}
	\have{2}{(P \eor M) \eif (P \eand M)}\by{Comm}{1}
	\have{n}{(P \eor M) \eif (M \eand P)}\by{Comm}{2}
\end{proof}

\begin{comment} % I have DN with the basic rules
Another rule of replacement is \define{double negation} (DN). With the DN rule, you can remove or insert a pair of negations anywhere in a sentence. This is the rule:
\factoidbox{
\begin{center}
\begin{tabular}{rl}
$\enot\enot\meta{A} \Longleftrightarrow \meta{A}$ & DN
\end{tabular}
\end{center}}
\end{comment} % end comment

% \begin{comment} % This is taking out DeMorgan's and the material conditional rule
Two more replacement rules are called \define{De Morgan's Laws}, named for the 19th-century British logician August De Morgan. (Although De Morgan did discover these laws, he was not the first to do so.) The rules capture useful relations between negation, conjunction, and disjunction, and we demonstrated that the sentences in the first one are logically equivalent in \S\ref{equivalence--tt}. Here are the rules, which we abbreviate `DeM':
\factoidbox{
\begin{center}
\begin{tabular}{rl}
$\enot(\meta{A}\eor\meta{B}) \Longleftrightarrow (\enot\meta{A}\eand\enot\meta{B})$\\
$\enot(\meta{A}\eand\meta{B}) \Longleftrightarrow (\enot\meta{A}\eor\enot\meta{B})$
& DeM
\end{tabular}
\end{center}}

In \S\ref{s:unless}, we discussed different ways of translating `you will catch a cold unless you wear a jacket'. We found that multiple translations were plausible: `$\enot J \eif D$', `$\enot D \eif J$', and `$J \eor D$', and I said that these were all equivalent. The \define{material conditional} rule (MC) is used for this equivalence. It takes two forms:
\factoidbox{
\begin{center}
\begin{tabular}{rl}
$(\meta{A}\eif\meta{B}) \Longleftrightarrow (\enot\meta{A}\eor\meta{B})$ &\\
$(\meta{A}\eor\meta{B}) \Longleftrightarrow (\enot\meta{A}\eif\meta{B})$ & MC
\end{tabular}
\end{center}}

Now consider this argument: $\enot(P \eif Q)$ \therefore\ $P \eand \enot Q$. We could prove this argument using only the basic rules, but that would take 17 lines and five subproofs. With DN, DeM, and MC, the proof is much simpler:

\begin{proof}
	\hypo{1}{\enot(P \eif Q)} \by{PR}{}
	\have{2}{\enot(\enot P \eor Q)}\by{MC}{1}
	\have{3}{\enot\enot P \eand \enot Q}\by{DeM}{2}
	\have{4}{P \eand \enot Q}\by{DN}{3}
\end{proof}
% \end{comment}  % end comment

A final replacement rule captures the relation between conditionals and biconditionals. We will call this rule \define{biconditional exchange} and abbreviate it {\eiff}{ex}.
\factoidbox{
\begin{center}
\begin{tabular}{rl}
$[(\meta{A}\eif\meta{B})\eand(\meta{B}\eif\meta{A})] \Longleftrightarrow (\meta{A}\eiff\meta{B})$
& {\eiff}{ex}
\end{tabular}
\end{center}}


\newsection
\subsection{Training}
\label{sec:training}
\sectionauthors{Alex Tamkin}

Training objectives are mathematical functions describing how to transform a model architecture and large amount of broad data into a foundation model. For example, GPT-3 was trained with a language modeling objective, which rewards the model for predicting the next word correctly \citep{Shannon1948AMT}. We begin by laying out some goals of these training approaches, describe important design trade-offs in current approaches, and outline important goals for the path ahead.

\subsubsection{Goals of training objectives}

Here we outline some key goals for training algorithms in light of the recent rapid progress in these methods and models.\footnote{We use ``training" instead of pretraining to emphasize the primacy of the foundation model itself, and because some methods for adapting foundation models to downstream tasks do not involve any later stage of training.}

\paragraph{Leveraging broad data.}
The rise of \textit{self-supervised} learning algorithms has unlocked the power of internet-scale datasets which would be intractable to annotate by hand. This kind of broad data comes in many forms, including images, audio recordings, and video (\refsec{vision}); robotic and sensor data (\refsec{robotics}); and text, either in isolation or paired with other modalities like images (\refsec{language}). Because this data lacks external annotations, a major focus for researchers is designing bespoke self-supervised algorithms that leverage the unique structure within each kind of data to produce a training signal for a foundation model.

\paragraph{Domain completeness.} An important goal for foundation model training algorithms is to be \textit{domain complete}, in the sense that solving the training task requires capabilities that are broadly useful for downstream tasks in the domain (see \refsec{language}, \refsec{vision}, \refsec{robotics}). This property is crucial for the \textit{generality} of a foundation model. For example, language modeling may require models to acquire capabilities as wide-ranging as coreference, sentiment and translation as the model learns to predict the next word in a document. In contrast, a supervised learning task like sentiment classification may lead to a more narrow set of capabilities (see \refsec{language}). As important as this quality is, it is not obvious \textit{a priori} what tasks will result in a domain complete capabilities, or even how to evaluate the full breadth of a model's capabilities (see \refsec {evaluation} and \refsec{theory}).

\paragraph{Scaling and compute efficiency.}
Procedures for training foundation models must reliably convert data, a model architecture, and compute into a broadly capable model. To maximize the capability of a foundation model, we can identify the bottlenecks to this process and propose new training algorithms which remove them. The rise of self-supervised algorithms has made model size and compute resources increasingly salient bottlenecks \citep{kaplan2020, henighan2020}, leading to a shift where models are evaluated not solely on their capabilities but rather on the amount and kind of compute needed to reach those capabilities (\refsec{evaluation}). The efficiency of training objectives can vary tremendously,\footnote{\eg 4x for ELECTRA \citep{Clark2020ELECTRAPT} vs BERT \citep{devlin2019bert}, 12x for contrastive vs generative approaches to CLIP training \citep{radford2021learning}} laying in sharp relief how important the design of a training approach is to the emergence of powerful capabilities given a fixed compute budget. Thus, a major goal for training researchers is to design training objectives with a richer training signal, resulting in models which learn faster and attain stronger capabilities.\footnote{Of course, a key goal for computer systems designers is to alleviate compute as a bottleneck for training (see \refsec{systems}) And the choice of a training method is ultimately also constrained by the availability of diverse, high-quality data (\refsec{data}), which continues to be a major challenge for many domains, including robotics (\refsec{robotics}) and low-resource languages (\refsec{language})} One force aiding this development is the surprising predictability of how capabilities scale with different kinds of architectures, data sizes, and compute \citep{Hestness2017DeepLS, kaplan2020},  a striking phenomenon which enables model developers to make choices based on clearer trends instead of more costly random searches.

\subsubsection{Design trade-offs in current SSL methods}
Current self-supervised learning (SSL) methods for training foundation models are diverse, but what unites them is that they produce prediction problems from unlabeled data without the need for human annotators. SSL objectives manufacture a rich training signal from this data through carefully-designed constraints, either on the data itself (\eg redacting or noising) or on the way the model is able to represent or process the data (\eg latent bottlenecks). At some level, these constraints ``bake in" the kinds of capabilities desired when adapting models to downstream tasks (\refsec{adaptation}).\footnote{For example, the causal language modeling objective used to train GPT-3 \citep{brown2020gpt3} enabled conditioning it via prefixes. And the color jitter augmentations used during contrastive learning \citep{chen2020simclr} encourage invariance to properties not thought to be useful for downstream tasks. Better understanding how the particular choice and structure of these constraints influences the capabilities acquired by the model is an important area for future work (\refsec{theory}).}

Here, we describe three important design choices that current models explore, along with their respective tradeoffs in terms of their resulting capabilities.

\paragraph{At what level of abstraction should we model?} A fundamental question is what the input representation of a foundation model should be. One option is to model the input at the level of raw bytes. However, this high dimensionality may cause the model to focus on predicting less semantic aspects of the  input,\footnote{\eg blades of grass, audio compression artifacts, or spellings of words} slowing the rate at which it acquires more generally-useful capabilities. These approaches also become intractable when training models like transformers \citep{vaswani2017attention} whose compute costs grow quadratically with the input size.\footnote{See \refsec{vision} and \refsec{modeling} for discussions of training costs for high-dimensional sequences, such as images and video} Another option is to use domain knowledge to reduce the input space of a model\dash{}such strategies include patch embeddings \citep{visual_transformer} as well as fixed or learned tokenization \citep{Schuster2012JapaneseAK, Sennrich2016NeuralMT, Kudo2018SentencePieceAS, Oord2017NeuralDR, ramesh2021zeroshot}. These methods may alleviate some challenges facing generative approaches, but have the trade-off that they may jettison possibly-useful information in the input.\footnote{For example, tokenizing text may make it harder to learn rhymes, puns, or other tasks that benefit from character-level information \citep{branwen2020gpt}} The choice of a continuous vs discrete input also has trade-offs for adaptation (\refsec{adaptation}); more work is needed to capture the benefits of both approaches.

\paragraph{Generative vs discriminative models}
Generative training approaches are conceptually elegant yet powerful\dash{}they train models to learn joint or conditional distributions over training inputs. Two major families of generative foundation models include autoregressive foundation models \citep{Oord2016WaveNetAG, Radford2018ImprovingLU, chen2020imagegpt, Yang2019XLNetGA, ramesh2021zeroshot}, which generate inputs piece by piece, and denoising foundation models \citep{devlin2019bert, raffel2019exploring}  which corrupt and then recover the inputs. The specific kind of generation performed in the training process determines what kind of interactivity is available during adaptation\footnote{For example, autoregressive models like GPT-3 enable prefix-based conditioning, while denoising models like T5 or BERT facilitate the use of bidirectional context to replace arbitrary-length spans or fix typos.} (see \refsec{interaction} and \refsec{adaptation}), and future models may enable an even richer set  of interactions.\footnote{Other kinds of generative approaches less studied in a foundation modeling context include diffusion and score-based models \citep{SohlDickstein2015DeepUL, Song2019GenerativeMB, Ho2020DenoisingDP}, VAEs \citep{Kingma2014AutoEncodingVB}, flow models \citep{dinh2015nice, Kingma2018GlowGF}, and GANs \citep{goodfellow2014gan}\dash{}it remains to be seen whether these or other future approaches can also enable learning as diverse variety of capabilities as autoregressive or denoising approaches.}

While generative training approaches have their benefits, several discriminative approaches have also recently gained traction. These methods do not enable generation-based interaction, yet they may enable more efficient learning for classification- or regression-based tasks in high-dimensional continuous settings like images, audio, and video. Most of these methods output vectors for (parts of) inputs, which are trained to be similar for different ``views'' of an input \citep{Wu2018UnsupervisedFL, Oord2018RepresentationLW, chen2020simclr, He2020MomentumCF, Grill2020BootstrapYO, Caron2021EmergingPI, convirt, radford2021learning} or used to predict whether parts of inputs are real or fake \citep{Clark2020ELECTRAPT, Iida2021TABBIEPR}. Better understanding the trade-offs between generative and discriminative training, as well as capturing the best of both approaches, remain interesting avenues for future study.

\paragraph{Capturing multimodal relationships.} Another increasingly important research area is capturing the relationships between multiple kinds of data. What this means may differ based on the context and the goals of a modeler. For example, CLIP \citep{radford2021learning} and ViLBERT \citep{Lu2019ViLBERTPT} are both multimodal vision-language, but differ in the precise \textit{way} they are multimodal.\footnote{See \refsec{vision} and \refsec{language} for more discussion of multimodality in vision and language specifically} The former encodes images and text separately into vectors, enabling users who have examples from a single modality to retrieve, score, or classify examples from the other modality. The second processes images and text jointly at an early stage of the model, enabling downstream applications like visual question answering where reasoning over pairs of related images and text (\eg images and questions about them) are provided. Multimodal foundation models remain a nascent research area; much is still unexplored about the different ways a model can be multimodal as well as better understanding the capabilities these additional modalities bring.

\subsubsection{Paths forward}
We close with some important goals for the future of foundation model training.

\paragraph{Out-of-the-box SSL} Right now, SSL objectives are highly domain-specific: different methods currently prevail in natural language processing, computer vision, and speech processing. This has two major disadvantages: First, these different techniques make it challenging to grasp the common threads and scientific principles underlying \textit{why} each of these methods work. Second, this domain-specificity requires developing new foundation model training methods from scratch for each new field, including medical, scientific, and new multimodal settings. A more general objective for efficiently training foundation models on any kind of data would represent a significant milestone for the foundation model training community \citep{Tamkin2021DABS}.

\paragraph{Obtaining a rich training signal} It is clear that not all training objectives are made equal\dash{}some are radically more efficient than others, translating into far more capable foundation models for a given compute budget. Are there training methods orders of magnitude more efficient than those currently known? If so, how can we find them? These investigations will be shaped by many forces, including what future software and hardware advances (\refsec{systems}) make possible. We also need not view data (\refsec{data}) and training algorithms as independent factors: not only does the quality and availability of the data influence the training signal,\footnote{Including any undesirable or biased capabilities (\refsec{fairness})} but the training algorithm itself could adaptively seek out or construct richer training examples as the model improves to accelerate learning \citep{Tamkin2021ViewmakerNL}.

\paragraph{Goal-directed training of foundation models.} Adaptation methods such as prompting (\refsec{adaptation}) draw on emergent properties that result almost as an afterthought of training. Can we train foundation models where the ability to understand and reliably carry out goals in a complex world is part of the model's training objective? A focus on developing general capabilities distinguishes this direction from the goal of adapting an existing foundation model to a specific task via reinforcement learning (\eg \citet{Stiennon2020LearningTS}). Instead, one might imagine more sophisticated versions of current methods which acquire a diverse range of real-world capabilities from raw online \citep{Klyubin2005EmpowermentAU, singh2005intrinsically, Salge2013EmpowermentA, Mohamed2015VariationalIM, Florensa2017StochasticNN, Pathak2017CuriosityDrivenEB, Haber2018LearningTP} or offline  \citep{Precup2000EligibilityTF, lange2012batch, Ajay2021OPALOP, Yang2021RepresentationMO, Schwarzer2021PretrainingRF} interactions, without the need for human annotations or task construction. Such methods might use techniques quite similar to existing SSL algorithms: \eg training sequence models in goal-directed contexts where they can be directly \textit{asked} to carry out certain tasks via conditioning (\eg UDRL \citep{Schmidhuber2019ReinforcementLU, Srivastava2019TrainingAU} or Decision Transformer \citep{Chen2021DecisionTR}; also see \refsec{robotics}). The complex behaviors that have already emerged in simple interactive environments  \citep{Baker2020EmergentTU} suggest multitask, multiagent, and multimodal goal-directed training of foundation models as an interesting avenue for future study.


% This is a template for doing homework assignments in LaTeX, cribbed from M. Frenkel (NYU) and A. Hanhart (UW-Madison)

\documentclass{article} % This command is used to set the type of document you are working on such as an article, book, or presenation

\usepackage[margin=1in]{geometry} % This package allows the editing of the page layout. I've set the margins to be 1 inch.

\usepackage{amsmath, amsfonts}  % The first package allows the use of a large range of mathematical formula, commands, and symbols.  The second gives some useful mathematical fonts.

\usepackage{graphicx}  % This package allows the importing of images

\usepackage{datetime}

%This allows us to use the theorem and proof environment
\usepackage{amsthm}
\theoremstyle{plain}
\newtheorem*{theorem*}{Theorem}
\newtheorem{theorem}{Theorem}
\theoremstyle{definition}
\newtheorem*{definition*}{Definition}

%Custom commands.
\newcommand{\abs}[1]{\left\lvert #1 \right\rvert} %absolute value command

%Custom symbols
\newcommand{\Rb}{\mathbb{R}}

\begin{document}

\begin{center}
    \Large{
        \textbf{Assignment \#3}

        UW-Madison MATH 421
    }

    \vspace{5pt}

    \normalsize{
        GEOFF YOERGER

        \usdate
        \formatdate{16}{2}{2021}
    }

    \vspace{15pt}
\end{center}

\noindent\fbox{\textbf{Exercise \#1:}} Prove the following theorem by induction.

\begin{theorem*} If $n$ is a natural number, then $1^2+2^2 + \dots + n^2 = \frac{n(n+1)(2n+1)}{6}$.

\end{theorem*}

\begin{proof} We argue by induction.

    Let $P(n) = \sum_{i=1}^{n} i^2 = \frac{n(n+1)(2n+1)}{6}$

    \underline{Base case:} $P(1) = $
    \begin{align*}
        P(1) = & & \sum_{i=1}^{1} n^2 &= \frac{1(1+1)(2(1)+1)}{6} \\
        = & & 1^2 &= \frac{1*2*3}{6} \\
        = & & 1 &= \frac{6}{6} \\
        = & & 1 &= 1 \\
        = & & TRUE & \\
    \end{align*}

    \underline{Induction step:}
    Let $n \in \mathbb{N}$.  Suppose $P(n)$ is true.
    \begin{align*}
        P(n+1) &= \sum_{i=1}^{n+1} i^2 &= \frac{(n+1)((n+1)+1)(2(n+1)+1)}{6}
               &= \sum_{i=1}^{n} i^2 + (n+1)^2 = \frac{(n+1)(n+2)(2n+3)}{6}
               &=  i^2 + (n+1)^2 = \frac{(n+1)(n+2)(2n+3)}{6}
    \end{align*}

    \underline{Induction step:}

\end{proof}

\noindent\fbox{\textbf{Exercise \#2:}} Prove the following theorem by induction.

\begin{theorem*}[Bernoulli's inequality] Suppose $n$ is a natural number and $x$ is a real number. If $x > -1$, then
$$
(1+x)^n \geq 1 + nx.
$$
\end{theorem*}

\begin{proof} We argue by induction.

\underline{Base case:}

\underline{Induction step:}

\end{proof}

\noindent \fbox{\textbf{Exercise \#3:}} For this problem we need the following definition:

\begin{definition*}
An integer $n$ is \emph{divisible} by an integer $k$ if the ratio $n/k$ is an integer.
\end{definition*}

For example: -3, 0, 3, 6 are all divisible by 3 while 1, 2, 4, 5 are not divisible by 3.  Prove the following:

\begin{theorem*} Suppose $n$ is an integer. If $n^2$ is divisible by $3$, then $n$ is divisible by $3$.

\end{theorem*}

 \begin{proof} (Hint: if $n$ is not divisible by $3$, then $n=3k+1$ or $n=3k+2$ for some integer $k$.) \end{proof}

 \noindent \fbox{\textbf{Exercise \#4:}} Prove that $\sqrt{3}$ is irrational.

 \begin{proof} \end{proof}

\noindent \fbox{\textbf{Exercise \#5:}} Prove that $\sqrt{2} + \sqrt{3}$ and $\sqrt{2}-\sqrt{3}$ are both irrational.

 \begin{proof} (Hint: $(a+b)(a-b) = a^2-b^2$) \end{proof}

  \noindent \fbox{\textbf{Exercise \#6:}} Spivak, Chapter 3, Problem 14.

  \begin{proof}[Solution] (justify your answer)

  \end{proof}

  \noindent \fbox{\textbf{Exercise \#7:}} Spivak, Chapter 3, Problem 23.

  \begin{proof}[Proof of (a)]  \end{proof}

    \begin{proof}[Proof of (b)]  \end{proof}

     \noindent \fbox{\textbf{Exercise \#8:}} Spivak, Chapter 3, Problem 26.

     \begin{proof} \end{proof}

\section{Extra Credit Questions}

Each extra credit question is worth 1 extra point. \\

\noindent\fbox{\textbf{Exercise E.C.\#1:}} Spivak, Chapter 2, Problem 17

\smallskip

\noindent\fbox{\textbf{Exercise E.C.\#2:}} Spivak, Chapter 3, Problem 16

\smallskip

\noindent\fbox{\textbf{Exercise E.C.\#3:}} Spivak, Chapter 3, Problem 17

\smallskip

\noindent\fbox{\textbf{Exercise E.C.\#4:}} Spivak, Chapter 3, Problem 20 (b)

\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                                                                 %
%  GEANT manual in LaTeX form                              %
%                                                                 %
%  Michel Goossens (for translation into LaTeX)                   %
%  Version 1.00                                                   %
%  Last Mod. Jan 24 1991  1300   MG + IB                          %
%                                                                 %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Origin{H.C.Fesefeldt, F.Carminati}
\Documentation{F. Carminati}
\Version{Geant 3.10}\Routid{PHYS460}
\Submitted{20.12. 85}     \Revised{16.12.93}
\Makehead{Muon-nucleus interactions}
\section{Subroutines}
\Shubr{GMUNUI}{}
\Rind{GMUNUI} computes and stores in the appropriate bank the value of
the muon-nucleus cross-section for a given material.
It is called at initialisation time by \Rind{GPHYSI}.
\Shubr{GMUNU}{}
\Rind{GMUNU} is called by \Rind{GTMUON} every time a muon-nucleus interaction
has to happen. It generates the final state particles as well as the outgoing
muon. A call to \Rind{GUHADR} is performed if {\tt IMUNU} (which is the
variable set by the {\tt  MUNU} data record) is equal to 1.
An inelastic interaction is forced (which could also be a fission in case
of heavy materials).
The secondaries from the $\pi$-nucleus interaction are always generated
if {\tt IMUNU}
is equal to 1, irrespectively of the value of {\tt IHADR}.

{\bf Note:} {\tt IMUNU} should be set to 1 only when using the {\tt GHEISHA}
hadronic interactions package. Setting it to 1 when using {\tt FLUKA} can
give unpredictable results.

\Sfunc{GMUSIG}{VALUE = GMUSIG(E,E1,COSTET)}
\begin{DLtt}{MMMMMMMM}
\item[E] ({\tt REAL}) initial $\mu$ energy;
\item[E1] ({\tt REAL}) final $\mu$ energy;
\item[COSTET] ({\tt REAL}) $\mu$ scattering angle.
\end{DLtt}
This function returns the value of the differential cross-section in
millibarns for a muon of energy {\tt E} to generate
a nuclear interaction recoil with
energy {\tt E1} at an angle the cosine of which is
{\tt COSTET}.

\section{Method}
This set of routines generates the interactions of muons with the nuclei of
the tracking material. The code is a straight translation into the {\tt GEANT}
style
of the corresponding code from the {\tt GHEISHA} Monte Carlo Program.
The {\tt GHEISHA} routines {\tt CASMU} and {\tt CALIM} have been used
\cite{bib-GHEI}.

The information contained in this chapter is mainly taken from the {\tt GHEISHA}
manual to which the user is referred.
The muon-nucleus inelastic cross-section is taken as 0.0003 mb for
a nucleon when the energy of the incoming muon is $E<30$ GeV, and slowly
increases for $E>30$ GeV according to the law:
\begin{center}
\[
\sigma = 0.3 * (E/30)^{0.25}\quad [\mu \rm b]
\]
\end{center}
The energy and angle of final state muon is generated according to the ``free
quark parton model''. If $E$ is the energy of the incoming muon and
$\Omega$ and $W$ the
angle and the energy of the outgoing muon, the differential cross-section
can be written as:

\[
\frac{d \sigma}{d \Omega dW} =\gamma ( \sigma_T + \epsilon \sigma_S)
\]
where:
\begin{eqnarray*}
\Gamma   & = & \frac{k \alpha}{2 \pi}\frac{W}{E}\frac{1}{1- \epsilon } \\
\epsilon & = & \left [ 1+2 \frac{Q^2 + \nu^2}{Q^2}\tan^2 \theta^2 \right]
\end{eqnarray*}

$Q^2$ and $\nu$ are the normal scaling variables expressed by:

\[
Q^2 = -q^2 = 2(EW-|p||p'|\cos\theta-m^2_{-\mu})\mbox{\quad and\quad} \nu
= E-W
\]
 here $\sigma_T$ and $\sigma_S$ are the photo-absorption
cross-sections for transverse and
longitudinal photons respectively for which the relation used is:

\[
\sigma_S = 0.3 \left ( 1- \frac{1}{1.868}{Q^2}{\nu}\right)  \sigma_T
\]
and $\sigma_T$ is assumed to be constant $\sigma_T = 0.12 $ mb.
For the incident flux $K$ of
the photons Gillman's convention is used:
\[
K = \nu + Q^2/2\nu
\]
 A three-dimensional importance sampling in the variables $E$, $W$ and
$\theta$ is performed each time an interaction has to occur.

This set of routines only works if \Rind{GUPHAD} calls \Rind{GPGHEI} and
\Rind{GUHADR} calls \Rind{GHEISH}.
The hadrons are generated in an approximate way. The virtual photon is
replaced by a real pion of random charge with the same kinetic energy. Then the
\Rind{GUHADR} routine is called to generate a $\pi$-nucleus inelastic
scattering. While the final state generated this way gives a good
approximation for calorimetric purposes, the kinematics of the final state
may be a rather poor approximation of reality.
The muon-nucleus interactions are activated by the {\tt  MUNU} data record of
{\tt GEANT}. After a muon-nucleus interaction the muon will still be the current
particle. If {\tt MUNU} 1 has been specified, secondaries coming from the
interaction of the virtual photon with the nucleus will be in the {\tt GEANT}
temporary stack. If {\tt MUNU} 2 has been specified, then the secondary
particles will not be generated and the energy lost by the muon will be
added to {\tt DESTEP}.
For each material a table of muon-nucleus cross-sections is stored at
initialisation time. See material bank structure for details.