\chapter{Differential forms}
In this chapter, all vector spaces are finite-dimensional
real inner product spaces.
We first start by (non-rigorously) drawing pictures
of all the things that we will define in this chapter.
Then we re-do everything again in its proper algebraic context.

\section{Pictures of differential forms}
Before defining a differential form,
we first draw some pictures.
The key thing to keep in mind is
\begin{moral}
	``The definition of a differential form is:
	something you can integrate.'' \\ --- Joe Harris
\end{moral}

We'll assume that all functions are \vocab{smooth},
i.e.\ infinitely differentiable.

Let $U \subseteq V$ be an open set of a vector space $V$.
Suppose that we have a function $f : U \to \RR$, i.e.\
we assign a value to every point of $U$.
\begin{center}
	\begin{asy}
		bigblob("$U$");
		dot("$3$", (-2,1), red);
		dot("$\sqrt2$", (-1,-1), red);
		dot("$-1$", (2,2), red);
		dot("$0$", (-3,-3), red);
	\end{asy}
\end{center}
\begin{definition}
	A \vocab{$0$-form} $f$ on $U$ is just a smooth function $f : U \to \RR$.
\end{definition}
Thus, if we specify a finite set $S$ of points in $U$
we can ``integrate'' over $S$ by just adding up the values
of the points:
\[ 0 + \sqrt 2 + 3 + (-1) = 2 + \sqrt2. \]
So, \textbf{a $0$-form $f$ lets us integrate over $0$-dimensional ``cells''}.

But this is quite boring, because as we know we like
to integrate over things like curves, not single points.
So, by analogy, we want a $1$-form to let us integrate
over $1$-dimensional cells: i.e.\ over curves.
What information would we need to do that?
To answer this, let's draw a picture of a curve $c$,
which can be thought of as a function $c : [0,1] \to U$.
\begin{center}
	\begin{asy}
		bigblob("$U$");
		pair a = (-2,-2);
		pair b = (3,0);
		pair p = (0,1);
		pair q = (2,0);
		label("$c$", q, dir(45), heavygreen);
		dot(a, heavygreen);
		dot(b, heavygreen);
		draw(a..p..q..b, heavygreen);
		dot("$p$", p, dir(90), blue);
		pair v = p+1.5*dir(-10);
		label("$v$", v, dir(50), blue);
		draw(p--v, blue, EndArrow);
	\end{asy}
\end{center}
We might think that we could get away
with just specifying a number on every point of $U$
(i.e.\ a $0$-form $f$), and then somehow ``add up''
all the values of $f$ along the curve.
We'll use this idea in a moment, but we can in fact do something more general.
Notice how when we walk along a smooth curve, at every point $p$
we also have some extra information: a \emph{tangent vector} $v$.
So, we can define a $1$-form $\alpha$ as follows.
A $0$-form just took a point and gave a real number,
but \textbf{a $1$-form will take both a point \emph{and} a tangent
vector at that point, and spit out a real number.}
So a $1$-form $\alpha$ is a smooth function on pairs $(p,v)$,
where $v$ is a tangent vector at $p$, to $\RR$.  Hence
\[ \alpha : U \times V \to \RR. \]

Actually, for any point $p$, we will require that $\alpha(p,-)$
is a linear function in terms of the vectors:
i.e.\ we want for example that $\alpha(p,2v) = 2\alpha(p,v)$.
So it is more customary to think of $\alpha$ as:
\begin{definition}
	A \vocab{$1$-form} $\alpha$ is a smooth function
	\[ \alpha : U \to V^\vee. \]
\end{definition}
Like with $Df$, we'll use $\alpha_p$ instead of $\alpha(p)$.
So, at every point $p$, $\alpha_p$ is some linear functional
that eats tangent vectors at $p$, and spits out a real number.
Thus, we think of $\alpha_p$ as an element of $V^\vee$;
\[ \alpha_p \in V^\vee. \]

Next, we draw pictures of $2$-forms.
This should, for example, let us integrate over a blob
(a so-called $2$-cell) of the form
\[ c : [0,1] \times [0,1] \to U \]
i.e.\ for example, a square in $U$.
In the previous example with $1$-forms,
we looked at tangent vectors to the curve $c$.
This time, at points we will look at \emph{pairs} of tangent vectors
in $U$: in the same sense that lots of tangent vectors
approximate the entire curve, lots of tiny squares
will approximate the big square in $U$.
\begin{center}
	\begin{asy}
		bigblob("$U$");
		filldraw( (-2,-2)--(-2,2)--(2,2)--(2,-2)--cycle,
			opacity(0.4) + orange, red);
		label("$c$", (2,2), dir(45), red);
		for (real t = -1; t < 2; t += 1) {
			draw( (-2,t)--(2,t), red );
			draw( (t,-2)--(t,2), red );
		}
		pair p = (-1, -1);
		dot("$p$", p, dir(225), blue);
		pair v = p + dir(90);
		pair w = p + dir(0);
		draw(p--v, blue, EndArrow);
		draw(p--w, blue, EndArrow);
		label("$v$", v, dir(135), blue);
		label("$w$", w, dir(-45), blue);
	\end{asy}
\end{center}
So what should a $2$-form $\beta$ be?
As before, it should start by taking a point $p \in U$,
so $\beta_p$ is now a linear functional:
but this time, it should be a linear map on two vectors $v$ and $w$.
Here $v$ and $w$ are not tangent so much as their span cuts out
a small parallelogram. So, the right thing to do is in fact consider
\[ \beta_p \in V^\vee \wedge V^\vee. \]
That is, to use the wedge product to get a handle on
the idea that $v$ and $w$ span a parallelogram.
Another valid choice would have been $(V \wedge V)^\vee$;
in fact, the two are isomorphic, but it will be more convenient
to write it in the former.

\section{Pictures of exterior derivatives}
Next question:
\begin{moral}
	How can we build a $1$-form from a $0$-form?
\end{moral}
Let $f$ be a $0$-form on $U$; thus, we have a function $f : U \to \RR$.
Then in fact there is a very natural $1$-form on $U$ arising
from $f$, appropriately called $df$.
Namely, given a point $p$ and a tangent vector $v$,
the differential form $(df)_p$ returns the \emph{change in $f$ along $v$}.
In other words, it's just the total derivative $(Df)_p(v)$.
\begin{center}
	\begin{asy}
		bigblob("$U$");
		dot("$3$", (-2,1), red);
		dot("$\sqrt2$", (-1,-1), red);
		dot("$-1$", (2,2), red);
		dot("$0$", (-3,-3), red);
		draw((-1,-1)--(0,1), blue, EndArrow);
		label("$\sqrt2 + \varepsilon$", (0,1), dir(90), blue);
		label("$v$", (-0.5, 0), dir(0), blue);
	\end{asy}
\end{center}
Thus, $df$ measures ``the change in $f$''.

Now, even if I haven't defined integration yet,
given a curve $c$ from a point $a$ to $b$, what do you think
\[ \int_c df \]
should be equal to?
Remember that $df$ is the $1$-form that measures
``infinitesimal change in $f$''.
So if we add up all the change in $f$ along a path from $a$ to $b$,
then the answer we get should just be
\[ \int_c df = f(b) - f(a). \]
This is the first case of something we call Stokes' theorem.

Generalizing, how should we get from a $1$-form to a $2$-form?
At each point $p$, the $2$-form $\beta$ gives a $\beta_p$
which takes in a ``parallelogram'' and returns a real number.
Now suppose we have a $1$-form $\alpha$.
Then along each of the edges of a parallelogram,
with an appropriate sign convention the $1$-form $\alpha$ gives
us a real number.
So, given a $1$-form $\alpha$, we define $d\alpha$
to be the $2$-form that takes in a parallelogram
spanned by $v$ and $w$,
and returns \textbf{the measure of $\alpha$ along the boundary}.

Now, what happens if you integrate $df$ along the entire square $c$?
The right picture is that, if we think of each little square
as making up the big square, then the adjacent boundaries cancel out,
and all we are left is the main boundary.
This is again just a case of the so-called Stokes' theorem.

\begin{center}
	\begin{asy}
		bigblob("$U$");
		filldraw( (-2,-2)--(-2,2)--(2,2)--(2,-2)--cycle,
			opacity(0.4) + orange, red);
		label("$c$", (2,2), dir(45), red);
		for (real t = -1; t < 2; t += 1) {
			draw( (-2,t)--(2,t), red );
			draw( (t,-2)--(t,2), red );
		}
		pair p = (-1, -1);
		dot("$p$", p, dir(225), blue);
		pair v = p + dir(90);
		pair w = p + dir(0);
		pair x = w + v - p;
		draw(p--w, blue, EndArrow);
		draw(w--x, blue, EndArrow);
		draw(x--v, blue, EndArrow);
		draw(v--p, blue, EndArrow);
	\end{asy}
	\hspace{4em}
	\begin{minipage}[t]{6.2cm}
		\includegraphics[width=6cm]{media/stokes-patch.png}
		\\ \scriptsize Image from \cite{img:stokes}
	\end{minipage}
\end{center}

\section{Differential forms}
\prototype{Algebraically,
	something that looks like $f \ee_1^\vee \wedge \ee_2^\vee + \dots$,
	and geometrically, see the previous section.}

Let's now get a handle on what $dx$ means.
Fix a real vector space $V$ of dimension $n$,
and let $\ee_1$, \dots, $\ee_n$ be a standard basis.
Let $U$ be an open set.

\begin{definition}
	We define a \vocab{differential $k$-form} $\alpha$ on $U$
	to be a smooth (infinitely differentiable) map
	$\alpha : U \to \Lambda^k(V^\vee)$.
	(Here $\Lambda^k(V^\vee)$ is the wedge product.)
\end{definition}

Like with $Df$, we'll use $\alpha_p$ instead of $\alpha(p)$.

\begin{example}
	[$k$-forms for $k=0,1$]
	\listhack
	\begin{enumerate}[(a)]
		\item A $0$-form is just a function $U \to \RR$.
		\item A $1$-form is a function $U \to V^\vee$.
		For example,
		the total derivative $Df$ of a function $V \to \RR$ is a $1$-form.
		\item Let $V = \RR^3$ with standard basis $\ee_1$, $\ee_2$, $\ee_3$.
		Then a typical $2$-form is given by
		\[
			\alpha_p
			=
			f(p) \cdot \ee_1^\vee \wedge \ee_2^\vee
			+ g(p) \cdot \ee_1^\vee \wedge \ee_3^\vee
			+ h(p) \cdot \ee_2^\vee \wedge \ee_3^\vee
			\in \Lambda^2(V)
		\]
		where $f,g,h : V \to \RR$ are smooth functions.
	\end{enumerate}
\end{example}

Now, by the projection principle (\Cref{thm:project_principle}) we only have to specify
a function on each of $\binom nk$ basis elements of $\Lambda^k(V^\vee)$.
So, take any basis $\{e_i\}$ of $V$, and
take the usual basis for $\Lambda^k(V^\vee)$ of elements
\[ e_{i_1}^\vee \wedge e_{i_2}^\vee \wedge \dots \wedge e_{i_k}^\vee. \]
Thus, a general $k$-form takes the shape
\[ \alpha_p = \sum_{1 \le i_1 < \dots < i_k \le n}
	f_{i_1, \dots, i_k}(p) \cdot
	e_{i_1}^\vee \wedge e_{i_2}^\vee \wedge \dots \wedge e_{i_k}^\vee. \]
Since this is a huge nuisance to write, we will abbreviate this to just
\[ \alpha = \sum_I f_I \cdot de_I \]
where we understand the sum runs over $I = (i_1, \dots, i_k)$,
and $de_I$ represents $e_{i_1}^\vee \wedge \dots \wedge e_{i_k}^\vee$.

Now that we have an element $\Lambda^k(V^\vee)$, what can it do?
Well, first let me get the definition on the table, then tell you what it's doing.
\begin{definition}
	For linear functions $\xi_1, \dots, \xi_k \in V^\vee$
	and vectors $v_1, \dots, v_k \in V$, set
	\[
		(\xi_1 \wedge \dots \wedge \xi_k)(v_1, \dots, v_k)
		\defeq
		\det
		\begin{bmatrix}
			\xi_1(v_1) & \dots & \xi_1(v_k) \\
			\vdots & \ddots & \vdots \\
			\xi_k(v_1) & \dots & \xi_k(v_k)
		\end{bmatrix}.
	\]
	You can check that this is well-defined
	under e.g. $v \wedge w = -w \wedge v$ and so on.
\end{definition}

\begin{example}
	[Evaluation of a differential form]
	Set $V = \RR^3$.
	Suppose that at some point $p$, the $2$-form $\alpha$ returns
	\[ \alpha_p = 2 \ee_1^\vee \wedge \ee_2^\vee + \ee_1^\vee \wedge \ee_3^\vee. \]
	Let $v_1 = 3\ee_1 + \ee_2 + 4\ee_3$ and $v_2 = 8\ee_1 + 9\ee_2 + 5\ee_3$.
	Then
	\[
		\alpha_p(v_1, v_2)
		=
		2\det \begin{bmatrix}
			3 & 8 \\ 1 & 9 \end{bmatrix}
		+
		\det \begin{bmatrix}
			3 & 8 \\ 4 & 5 \end{bmatrix}
		= 21.
	\]
\end{example}

What does this definition mean?
One way to say it is that
\begin{moral}
	If I walk to a point $p \in U$,
	a $k$-form $\alpha$ will take in $k$ vectors $v_1, \dots, v_k$
	and spit out a number, which is to be interpreted as a (signed) volume.
\end{moral}

Picture:
\begin{center}
	\begin{asy}
		bigblob("$U$");
		pair p = (-2,-2);
		dot("$p$", p, dir(225), red);
		pair p1 = p + 1.4*dir(120);
		pair p2 = p + 1.7*dir(10);
		draw(p--p1, red, EndArrow);
		draw(p--p2, red, EndArrow);
		label("$v_1$", p1, dir(p1-p), red);
		label("$v_2$", p2, dir(p2-p), red);
		label("$\alpha_p(v_1, v_2) \in \mathbb R$", p+dir(45)*3);
	\end{asy}
\end{center}

In other words, at every point $p$, we get a function $\alpha_p$.
Then I can feed in $k$ vectors to $\alpha_p$ and get a number,
which I interpret as a signed volume of the parallelpiped spanned by the $\{v_i\}$'s
in some way (e.g.\ the flux of a force field).
That's why $\alpha_p$ as a ``function'' is contrived to lie in the wedge product:
this ensures that the notion of ``volume'' makes sense, so that for example,
the equality $\alpha_p(v_1, v_2) = -\alpha_p(v_2, v_1)$ holds.

This is what makes differential forms so fit for integration.

\section{Exterior derivatives}
\prototype{Possibly $dx_1 = \ee_1^\vee$.}
We now define the exterior derivative $df$ that we gave
pictures of at the beginning of the section.
It turns out that the exterior derivative is easy to compute
given explicit coordinates to work with.

First, given a function $f : U \to \RR$,
we define
\[ df \defeq Df = \sum_i \frac{\partial f_i}{\partial e_i} e_i^\vee \]
In particular, suppose $V = \RR^n$ and $f(x_1, \dots, x_n) = x_1$
(i.e.\ $f = \ee_1^\vee$). Then:
\begin{ques}
	Show that for any $p \in U$, \[ \left( d(\ee_1^\vee) \right)_p = \ee_1^\vee. \]
\end{ques}

\begin{abuse}
	Unfortunately, someone somewhere decided
	it would be a good idea to use ``$x_1$'' to denote $\ee_1^\vee$
	(because \emph{obviously}\footnote{Sarcasm.} $x_1$ means
	``the function that takes $(x_1, \dots, x_n) \in \RR^n$ to $x_1$'')
	and then decided that \[ dx_1 \defeq \ee_1^\vee. \]
	This notation is so entrenched that I have no choice
	but to grudgingly accept it.
	Note that it's not even right,
	since technically it's $(dx_1)_p = \ee_1^\vee$; $dx_1$ is a $1$-form.
	\label{abuse:dx}
\end{abuse}
\begin{remark}
	This is the reason why we use the notation $\frac{df}{dx}$ in calculus now:
	given, say, $f : \RR \to \RR$ by $f(x) = x^2$, it is indeed true that
	\[ df = 2x \cdot \ee_1^\vee = 2x \cdot dx \]
	and so by (more) abuse of notation we write $df/dx = 2x$.
\end{remark}

More generally, we can define the \vocab{exterior derivative}
in terms of our basis $e_1$, \dots, $e_n$ as follows:
if $\alpha = \sum_I f_I de_I$ then we set
\[ d\alpha \defeq \sum_I df_I \wedge de_I
	= \sum_I \sum_j \fpartial{f_I}{e_j} de_j \wedge de_I. \]
This doesn't depend on the choice of basis.

\begin{example}[Computing some exterior derivatives]
	Let $V = \RR^3$ with standard basis $\ee_1$, $\ee_2$, $\ee_3$.
	Let $f(x,y,z) = x^4 + y^3 + 2xz$.
	Then we compute
	\[ df = Df = (4x^3+2z) \; dx + 3y^2 \; dy + 2x \; dz. \]
	Next, we can evaluate $d(df)$ as prescribed: it is
	\begin{align*}
		d^2f &= (12x^2 \; dx + 2 dz) \wedge dx + (6y \; dy) \wedge dy
		+ 2(dx \wedge dz) \\
		&= 12x^2 (dx \wedge dx) + 2(dz \wedge dx) + 6y (dy \wedge dy) + 2(dx \wedge dz) \\
		&= 2(dz \wedge dx) + 2(dx \wedge dz) \\
		&= 0.
	\end{align*}
	So surprisingly, $d^2f$ is the zero map.
	Here, we have exploited \Cref{abuse:dx} for the first time,
	in writing $dx$, $dy$, $dz$.
\end{example}
And in fact, this is always true in general:
\begin{theorem}[Exterior derivative vanishes]
	\label{thm:dd_zero}
	Let $\alpha$ be any $k$-form.
	Then $d^2(\alpha) = 0$.
	Even more succinctly, \[ d^2 = 0. \]
\end{theorem}
The proof is left as \Cref{prob:dd_zero}.
\begin{exercise}
	Compare the statement $d^2 = 0$ to the geometric
	picture of a $2$-form given at the beginning of this chapter.
	Why does this intuitively make sense?
\end{exercise}

Here are some other properties of $d$:
\begin{itemize}
	\ii As we just saw, $d^2 = 0$.
	\ii For a $k$-form $\alpha$ and $\ell$-form $\beta$, one can show that
	\[ d(\alpha \wedge \beta) = d\alpha \wedge \beta + (-1)^k (\alpha \wedge d\beta). \]
	\ii If $f \colon U \to \RR$ is smooth, then $df = Df$.
\end{itemize}
In fact, one can show that $df$ as defined above is
the \emph{unique} map sending $k$-forms to $(k+1)$-forms
with these properties.
So, one way to define $df$ is to take as axioms
the bulleted properties above
and then declare $d$ to be the unique solution to this functional equation.
In any case, this tells us that our definition of $d$
does not depend on the basis chosen.

Recall that $df$ measures the change in boundary.
In that sense, $d^2 = 0$ is saying something like
``the boundary of the boundary is empty''.
We'll make this precise when we see Stokes' theorem in the next chapter.

\section{Closed and exact forms}
Let $\alpha$ be a $k$-form.
\begin{definition}
	We say $\alpha$ is \vocab{closed} if $d\alpha = 0$.
\end{definition}
\begin{definition}
	We say $\alpha$ is \vocab{exact} if for some $(k-1)$-form $\beta$,
	$d\beta = \alpha$.  If $k = 0$, $\alpha$ is exact only when $\alpha = 0$.
\end{definition}
\begin{ques}
	Show that exact forms are closed.
\end{ques}

A natural question arises: are there closed forms
which are not exact?
Surprisingly, the answer to this question is tied to topology.
Here is one important example.

\begin{example}
	[The angle form]
	\label{ex:angle_form}
	Let $U = \RR^2 \setminus \{0\}$,
	and let $\theta(p)$ be the angle formed by the $x$-axis
	and the line from the origin to $p$.

	The $1$-form $\alpha : U \to (\RR^2)^\vee$ defined by
	\[ \alpha = \frac{-y \; dx + x \; dy}{x^2+y^2} \]
	is called the \vocab{angle form}:
	given $p \in U$ it measures the change in angle $\theta(p)$
	along a tangent vector.
	So intuitively, ``$\alpha = d\theta$''.
	Indeed, one can check directly that the angle form is closed.

	However, $\alpha$ is not exact: there is no global smooth
	function $\theta : U \to \RR$ having $\alpha$ as a derivative.
	This reflects the fact that one can actually perform
	a full $2\pi$ rotation around the origin, i.e.\ $\theta$
	only makes sense mod $2\pi$.
	Thus existence of the angle form $\alpha$ reflects
	the possibility of ``winding'' around the origin.
\end{example}

So the key idea is that the failure of a closed form to be exact
corresponds quite well with ``holes'' in the space:
the same information that homotopy and homology groups are trying to capture.
To draw another analogy, in complex analysis Cauchy-Goursat
only works when $U$ is simply connected.
The ``hole'' in $U$ is being detected by the existence of a form $\alpha$.
The so-called de Rham cohomology will make this relation explicit.

\section\problemhead
\begin{problem}
	Show directly that the angle form
	\[ \alpha = \frac{-y \; dx + x \; dy}{x^2+y^2} \]
	is closed.
\end{problem}

\begin{problem}
	\label{prob:dd_zero}
	Establish \Cref{thm:dd_zero}, which states that $d^2 = 0$.
	\begin{hint}
		This is just a summation.
		You will need the fact that mixed partials are symmetric.
	\end{hint}
\end{problem}


\chapter{Introduction to Proof with HOL}\label{proof}

\newcommand\tacticline{\hline \hline}
\newenvironment{proofenumerate}{\begin{enumerate}}{\end{enumerate}}
% proofenumerate is distinguished from a normal enumeration so that
% h e v e a can spot these special cases and treat them better.

\paragraph{Preliminaries} This chapter discusses the nature of proof
in \HOL{} in more detail.  The previous chapter has provided a broad
overview of how proof is done in \HOL{}, while here the emphasis is on
attaining a more thorough grounding in the material.  As before, we
are using \texttt{<holdir>/bin/hol}.

\setcounter{sessioncount}{0}

For a logician, one definition of a formal proof is that it is a
sequence, each of whose elements is either an {\it axiom\/} or follows
from earlier members of the sequence by a {\it rule of inference\/}.
A theorem is the last element of a proof.

Theorems are represented in \HOL{} by values of an abstract type
{\small\verb|thm|}.  The only way to create theorems is by generating
such a proof.  In \HOL, following \LCF, this consists in applying
\ML{} functions representing {\it rules of inference\/} to axioms or
previously generated theorems.  The sequence of such applications
directly corresponds to a logician's proof.

There are five axioms of the \HOL{} logic and eight primitive
inference rules. The axioms are bound to ML names. For example, the Law of
Excluded Middle is bound to the \ML{} name {\small\verb|BOOL_CASES_AX|}:

\begin{session}
\begin{verbatim}
- BOOL_CASES_AX;
> val it = |- !t. (t = T) \/ (t = F) : thm
\end{verbatim}
\end{session}

Theorems are printed with a preceding turnstile {\small\verb+|-+} as
illustrated above; the symbol `{\small\verb|!|}' is the universal
quantifier `$\forall$'.  Rules of inference are \ML{} functions that
return values of type {\small\verb|thm|}.  An example of a rule of
inference is {\it specialization\/} (or $\forall$-elimination).  In
standard `natural deduction' notation this is:

\[ \Gamma\turn \uquant{x}t\over \Gamma\turn t[t'/x]\]

\begin{itemize}
\item $t[t'/x]$ denotes the result of substituting $t'$ for free
occurrences of $x$ in $t$, with the restriction that no free variables in $t'$
become bound after substitution.
\end{itemize}

\noindent This rule is represented in \ML\
by a function
{\small\verb|SPEC|},\footnote{{\tt SPEC} is not a
primitive rule of inference in the HOL logic, but is a derived rule. Derived rules
are described in Section~\ref{forward}.}
which takes as arguments a term
{\small\verb|``|}$a${\small\verb|``|} and a theorem
{\small\verb%|- !%}$x${\small\verb|.|}$t[x]$ and returns the theorem
{\small\verb%|- %}$t[a]$, the result of substituting $a$ for $x$ in $t[x]$.

\begin{session}
\begin{verbatim}
- val Th1 = BOOL_CASES_AX;
> val Th1 = |- !t. (t = T) \/ (t = F) : thm

- val Th2 = SPEC ``1 = 2`` Th1;
> val Th2 = |- ((1 = 2) = T) \/ ((1 = 2) = F) : thm
\end{verbatim}
\end{session}

This session consists of a proof of two steps: using an axiom and
applying the rule \ml{SPEC}; it interactively performs the following proof:

\begin{proofenumerate}
\item $ \turn \uquant{t} t=\top\ \disj\  t=\bot$ \hfill
[Axiom \ml{BOOL\_CASES\_AX}]
\item $ \turn (1{=}2)=\top\ \disj\ (1{=}2)=\bot$\hfill [Specializing line 1 to `$1{=}2$']
\end{proofenumerate}

If the argument to an \ML{} function representing a rule of inference
is of the wrong kind, or violates a condition of the rule, then the
application fails.  For example, $\ml{SPEC}\ t\ th$ will fail if $th$
is not of the form $\ml{|-\ !}x\ml{.}\cdots$ or if it is of this form
but the type of $t$ is not the same as the type of $x$, or if the free
variable restriction is not met.  When one of the standard
\ml{HOL\_ERR} exceptions is raised, more information about the failure
can often be gained by using the \ml{Raise} function.

\begin{session}
\begin{verbatim}
- SPEC ``1=2`` Th2;
! Uncaught exception:
! HOL_ERR <poly>

- SPEC ``1 = 2`` Th2 handle e => Raise e;

Exception raised at Thm.SPEC:

! Uncaught exception:
! HOL_ERR <poly>
\end{verbatim}
\end{session}
However, as this session illustrates, the failure token does not
always indicate the exact reason for failure. The failure conditions
for rules of inference are given in \REFERENCE.

A proof in the \HOL{} system is constructed by repeatedly applying
inference rules to axioms or to previously proved theorems.  Since
proofs may consist of millions of steps, it is necessary to provide
tools to make proof construction easier for the user.  The proof
generating tools in the \HOL{} system are just those of \LCF, and are
described later.

The general form of a theorem is $t_1,\ldots,t_n\ $\ml{|-}$\ t$, where
$t_1$, $\ldots$ , $t_n$ are boolean terms called the {\it assumptions}
and $t$ is a boolean term called the {\it conclusion\/}.  Such a
theorem asserts that if its assumptions are true then so is its
conclusion.  Its truth conditions are thus the same as those for the
single term
$(t_1${\small\verb|/\|}$\ldots${\small\verb|/\|}$t_n$)\ml{==>}$t$.
Theorems with no assumptions are printed out in the form \ml{|-}$\ t$.

The five axioms and eight primitive inference rules of the \HOL{} logic
are described in detail in the document \DESCRIPTION.  Every value of
type \ml{thm} in the \HOL{} system can be obtained by repeatedly
applying primitive inference rules to axioms.  When the \HOL{} system
is built, the eight primitive rules of inference are defined and the
five axioms are bound to their \ML{} names, all other predefined
theorems are proved using rules of inference as the system is
made.\footnote{This is a slight over-simplification.} This is one of
the reasons why building \ml{hol} takes so long.

In the rest of this chapter, the process of {\it forward proof\/},
which has just been sketched, is described in more detail.  In
Chapter~\ref{tactics} {\it goal directed proof\/} is described,
including the important notions of {\it tactics\/} and {\it
  tacticals\/}, due to Robin Milner.

\section{Forward proof}
\label{forward}

Three of the primitive inference rules of the \HOL{} logic are
\ml{ASSUME} (assumption introduction), \ml{DISCH} (discharging or
assumption elimination) and \ml{MP} (Modus Ponens).  These rules will
be used to illustrate forward proof and the writing of derived rules.

The inference rule \ml{ASSUME} generates theorems of the form \ml{$t$
  |- $t$}. Note, however, that the \ML{} printer prints each
assumption as a dot (but this default can be changed; see below).  The
function \ml{dest\_thm} decomposes a theorem into a pair consisting of
list of assumptions and the conclusion.

\begin{session}
\begin{verbatim}
- val Th3 = ASSUME ``t1==>t2``;;
> val Th3 =  [.] |- t1 ==> t2 : thm

- dest_thm Th3;
> val it = ([``t1 ==> t2``], ``t1 ==> t2``) : term list * term
\end{verbatim}
\end{session}

A sort of dual to \ml{ASSUME} is the primitive inference rule
\ml{DISCH} (discharging, assumption elimination) which infers from
a theorem of the form $\cdots t_1\cdots\ml{\ |-\ }t_2$ the new theorem
$\cdots\ \cdots\ \ml{|-}\ t_1\ml{==>}t_2$. \ml{DISCH} takes as arguments
the term to be discharged (\ie\ $t_1$) and the theorem from whose
assumptions it is to be discharged and returns the result of the discharging.
The following session illustrates this:

\begin{session}
\begin{verbatim}
- val Th4 = DISCH ``t1==>t2`` Th3;
> val Th4 = |- (t1 ==> t2) ==> t1 ==> t2 : thm
\end{verbatim}
\end{session}
Note that the term being discharged need not be in the assumptions; in
this case they will be unchanged.

\begin{session}\begin{verbatim}
- DISCH ``1=2`` Th3;
> val it =  [.] |- (1 = 2) ==> t1 ==> t2 : thm

- dest_thm it;
> val it = ([``t1 ==> t2``], ``(1 = 2) ==> t1 ==> t2``) : term list * term
\end{verbatim}\end{session}

    In \HOL\, the rule \ml{MP} of Modus Ponens is specified in
    conventional notation by:
\[
\Gamma_1 \turn t_1 \imp t_2 \qquad\qquad \Gamma_2\turn t_1\over
\Gamma_1 \cup \Gamma_2 \turn t_2
\]
The \ML{} function \ml{MP} takes argument theorems of the form
\ml{$\cdots\ $|-$\ t_1$\ ==>\ $t_2$} and \ml{$\cdots\ $|-$\ t_1$} and
returns \ml{$\cdots\ $|-$\ t_2$}. The next session illustrates the use
of \ml{MP} and also a common error, namely not supplying the \HOL\
logic type checker with enough information.

\begin{session}\begin{verbatim}
- val Th5 = ASSUME ``t1``;
<<HOL message: inventing new type variable names: 'a.>>
! Uncaught exception:
! HOL_ERR <poly>
- val Th5 = ASSUME ``t1`` handle e => Raise e;
<<HOL message: inventing new type variable names: 'a.>>

Exception raised at Thm.ASSUME:
not a proposition
! Uncaught exception:
! HOL_ERR <poly>

- val Th5 = ASSUME ``t1:bool``;
> val Th5 =  [.] |- t1 : thm

- val Th6 = MP Th3 Th5;
> val Th6 =  [..] |- t2 : thm
\end{verbatim}\end{session}

    The hypotheses of \ml{Th6} can be inspected with the \ML{} function
    \ml{hyp}, which returns the list of assumptions of a theorem (the
    conclusion is returned by \ml{concl}).

\begin{session}\begin{verbatim}
- hyp Th6;
> val it = [``t1 ==> t2``, ``t1``] : term list
\end{verbatim}\end{session}

    \HOL{} can be made to print out hypotheses of theorems explicitly
    by setting the global flag \ml{show\_assums} to true.

\begin{session}\begin{verbatim}
- show_assums := true;
> val it = () : unit

- Th5;
> val it =  [t1] |- t1 : thm

- Th6;
> val it =  [t1 ==> t2, t1] |- t2 : thm
\end{verbatim}\end{session}

\noindent Discharging \ml{Th6} twice establishes the theorem
\ml{|-\ t1 ==> (t1==>t2) ==> t2}.

\begin{session}\begin{verbatim}
- val Th7 = DISCH ``t1==>t2`` Th6;
> val Th7 = [t1] |- (t1 ==> t2) ==> t2 : thm

- val Th8 = DISCH ``t1:bool`` Th7;
> val Th8 = |- t1 ==> (t1 ==> t2) ==> t2 : thm
\end{verbatim}\end{session}

    The sequence of theorems: \ml{Th3}, \ml{Th5}, \ml{Th6}, \ml{Th7},
    \ml{Th8} constitutes a proof in \HOL{} of the theorem \ml{|-\ t1
      ==> (t1 ==> t2) ==> t2}. In standard logical notation this proof
    could be written:

\begin{proofenumerate}
\item $ t_1\imp t_2\turn t_1\imp t_2$ \hfill
[Assumption introduction]
\item $ t_1\turn t_1$ \hfill
[Assumption introduction]
\item $ t_1\imp t_2,\ t_1 \turn t_2 $ \hfill
[Modus Ponens applied to lines 1 and 2]
\item $ t_1 \turn (t_1\imp t_2)\imp t_2$ \hfill
[Discharging the first assumption of line 3]
\item $ \turn t_1 \imp (t_1 \imp t_2) \imp t_2$ \hfill
[Discharging the only assumption of line 4]
\end{proofenumerate}

\subsection{Derived rules}

A {\it proof from hypothesis $th_1, \ldots, th_n$} is a sequence each
of whose elements is either an axiom, or one of the hypotheses $th_i$,
or follows from earlier elements by a rule of inference.

For example, a proof of $\Gamma,\ t'\turn t$ from the hypothesis
$\Gamma\turn t$ is:

\begin{proofenumerate}
\item $ t'\turn t'$ \hfill [Assumption introduction]
\item $ \Gamma\turn t$ \hfill [Hypothesis]
\item $ \Gamma\turn t'\imp t$ \hfill [Discharge $t'$ from line 2]
\item $ \Gamma,\ t'\turn t$ \hfill [Modus Ponens applied to lines 3 and 1]
\end{proofenumerate}

\noindent This proof works for any hypothesis of the form $\Gamma\turn t$
and any boolean term $t'$ and shows that the result of adding an
arbitrary hypothesis to a theorem is another theorem (because the four
lines above can be added to any proof of $\Gamma\turn t$ to get a
proof of $\Gamma,\ t'\turn t$).\footnote{This property of the logic is
  called {\it monotonicity}.} For example, the next session uses this
proof to add the hypothesis \ml{``t3``} to \ml{Th6}.

\begin{session}\begin{verbatim}
- val Th9 = ASSUME ``t3:bool``;
> val Th9 = [t3] |- t3 : thm

- val Th10 = DISCH ``t3:bool`` Th6;
> val Th10 = [t1 ==> t2, t1] |- t3 ==> t2 : thm

- val Th11 = MP Th10 Th9;
> val Th11 = [t1 ==> t2, t1, t3] |- t2 : thm
\end{verbatim}\end{session}

    A {\it derived rule\/} is an \ML{} procedure that generates a proof
    from given hypotheses each time it is invoked. The hypotheses are
    the arguments of the rule.  To illustrate this, a rule, called
    \ml{ADD\_ASSUM}, will now be defined as an \ML{} procedure that
    carries out the proof above. In standard notation this would be
    described by:

\[ \Gamma\turn t\over \Gamma,\ t'\turn t \]

\noindent The \ML{} definition is:

\begin{session}\begin{verbatim}
- fun ADD_ASSUM t th = let
    val th9 = ASSUME t
    val th10 = DISCH t th
  in
    MP th10 th9
  end;
> val ADD_ASSUM = fn : term -> thm -> thm

- ADD_ASSUM ``t3:bool`` Th6;
> val it =  [t1, t1 ==> t2, t3] |- t2 : thm
\end{verbatim}\end{session}

\noindent The body of \ml{ADD\_ASSUM} has been coded  to mirror  the proof done
in session~10 above, so as to show how an interactive proof can be
generalized into a procedure.  But \ml{ADD\_ASSUM} can be written much
more concisely as:

\begin{session}\begin{verbatim}
- fun ADD_ASSUM t th = MP (DISCH t th) (ASSUME t);
> val ADD_ASSUM = fn : term -> thm -> thm

- ADD_ASSUM ``t3:bool`` Th6;
val it = [t1 ==> t2, t1, t3] |- t2 : thm
\end{verbatim}\end{session}

    Another example of a derived inference rule is \ml{UNDISCH}; this
    moves the antecedent of an implication to the assumptions.

\[ \Gamma\turn t_1\imp t_2 \over\Gamma,\ t_1\turn t_2 \]

\noindent An \ML{} derived rule that implements this is:

\begin{session}\begin{verbatim}
- fun UNDISCH th = MP th (ASSUME(#1(dest_imp(concl th))));
> val UNDISCH = fn : thm -> thm

- Th10;
> val it =  [t1 ==> t2, t1] |- t3 ==> t2 : thm

- UNDISCH Th10;
> val it =  [t1, t1 ==> t2, t3] |- t2 : thm
\end{verbatim}\end{session}

\noindent Each time \ml{UNDISCH\ $\Gamma\turn t_1\imp t_2$} is executed,
the following proof is performed:

\begin{proofenumerate}
\item $ t_1\turn t_1$ \hfill [Assumption introduction]
\item $ \Gamma\turn t_1\imp t_2$ \hfill [Hypothesis]
\item $ \Gamma,\ t_1\turn t_2$ \hfill [Modus Ponens applied to lines 2 and 1]
\end{proofenumerate}

The rules \ml{ADD\_ASSUM} and \ml{UNDISCH} are the first derived rules
defined when the \HOL{} system is built. For a description of the main
rules see the section on derived rules in \DESCRIPTION.

\subsection{Rewriting}

An important derived rule is \ml{REWRITE\_RULE}.  This takes a list of
conjunctions of equations, \ie\ a list of theorems of the form:

\[ \Gamma\turn (u_1 = v_1) \conj (u_2 = v_2) \conj \ldots\ \conj (u_n  = v_n)\]

\noindent  and a theorem
$\Delta\turn t$ and repeatedly replaces instances of $u_i$ in $t$ by
the corresponding instance of $v_i$ until no further change occurs.
The result is a theorem $\Gamma\cup\Delta\turn t'$ where $t'$ is the
result of rewriting $t$ in this way.  The session below illustrates
the use of \ml{REWRITE\_RULE}.  In it the list of equations is the
value \ml{rewrite\_list} containing the pre-proved theorems
\ml{ADD\_CLAUSES} and \ml{MULT\_CLAUSES}.  These theorems are from the
theory \ml{arithmetic}, so we must use a fully qualified name with the
name of the theory as the first component to refer to them.
(Alternatively, we could, as in the Euclid example of
chapter~\ref{chap:euclid}, use \ml{open} to bring declare all of the
values in the theory at the top level.)

\begin{session}\begin{verbatim}
- val rewrite_list = [arithmeticTheory.ADD_CLAUSES,
                      arithmeticTheory.MULT_CLAUSES];
> val rewrite_list =
    [ []
     |- (0 + m = m) /\ (m + 0 = m) /\ (SUC m + n = SUC (m + n)) /\
        (m + SUC n = SUC (m + n)),
      []
     |- !m n.
          (0 * m = 0) /\ (m * 0 = 0) /\ (1 * m = m) /\ (m * 1 = m) /\
          (SUC m * n = m * n + n) /\ (m * SUC n = m + m * n)]
    : Thm.thm list
\end{verbatim}\end{session}

\begin{session}\begin{verbatim}
- REWRITE_RULE rewrite_list (ASSUME ``(m+0)<(1*n)+(SUC 0)``);
> val it =  [m + 0 < 1 * n + SUC 0] |- m < SUC n : thm
\end{verbatim}\end{session}

\noindent
This can then be rewritten using another pre-proved theorem
\ml{LESS\_THM}, this one from the theory \ml{prim\_rec}:

\begin{session}\begin{verbatim}
- REWRITE_RULE [prim_recTheory.LESS_THM] it;
> val it =  [m + 0 < 1 * n + SUC 0] |- (m = n) \/ m < n : thm
\end{verbatim}\end{session}

    \ml{REWRITE\_RULE} is not a primitive in \HOL, but is a derived
    rule. It is inherited from Cambridge \LCF\ and was implemented by
    Larry Paulson (see his paper \cite{lcp_rewrite} for details). In
    addition to the supplied equations, \ml{REWRITE\_RULE} has some
    built in standard simplifications:

\begin{session}\begin{verbatim}
- REWRITE_RULE [] (ASSUME ``(T /\ x) \/ F ==> F``);
> val it = [T /\ x \/ F ==> F] |- ~x : thm
\end{verbatim}\end{session}

    There are elaborate facilities in \HOL{} for producing customized
    rewriting tools which scan through terms in user programmed
    orders; \ml{REWRITE\_RULE} is the tip of an iceberg, see
    \DESCRIPTION\ for more details.

\section{Goal Oriented Proof: Tactics and Tacticals}
\label{backward}\label{tactics}

The style of forward proof described in the previous chapter is
unnatural and too `low level' for many applications. An important
advance in proof generating methodology was made by Robin Milner in
the early 1970s when he invented the notion of {\it tactics\/}. A
tactic is a function that does two things.
\begin{myenumerate}
\item Splits a `goal' into `subgoals'.
\item Keeps track of the reason why solving the subgoals will solve the goal.
\end{myenumerate}

\noindent Consider, for example, the  rule of $\wedge$-introduction\footnote{In
  higher order logic this is a derived rule; in first order logic it
  is usually primitive.  In HOL the rule is called {\tt CONJ} and its
  derivation is given in \DESCRIPTION.}  shown below:

\[ \Gamma_1\turn
t_1\qquad\qquad\qquad\Gamma_2\turn t_2\over \Gamma_1\cup\Gamma_2 \turn t_1\conj
t_2 \]

\noindent In \HOL,  $\wedge$-introduction is  represented by  the \ML{} function
\ml{CONJ}:

\[\ml{CONJ}\ (\Gamma_1\turn t_1)\ (\Gamma_2\turn t_2) \ \ \leadsto\
\ (\Gamma_1\cup\Gamma_2\turn  t_1\conj  t_2)\]

\noindent  This  is   illustrated  in  the
following new session (note that the session number has been reset to
{\small\sl 1}:

\setcounter{sessioncount}{0}
\begin{session}\begin{verbatim}
- show_assums := true;
val it = () : unit

- val Th1 = ASSUME ``A:bool`` and Th2 = ASSUME ``B:bool``;
> val Th1 =  [A] |- A : thm
  val Th2 =  [B] |- B : thm

- val Th3 = CONJ Th1 Th2;
> val Th3 =  [A, B] |- A /\ B : thm
\end{verbatim}\end{session}

    Suppose the goal is to prove $A\conj B$, then this rule says that
    it is sufficient to prove the two subgoals $A$ and $B$, because
    from $\turn A$ and $\turn B$ the theorem $\turn A\conj B$ can be
    deduced. Thus:

\begin{myenumerate}
\item To prove $\turn A \conj B$ it is sufficient to
      prove $\turn A$ and $\turn B$.
\item The justification for the reduction of the
goal  $\turn A \conj B$  to the two  subgoals  $\turn A$
and $\turn B$ is the rule of $\wedge$-introduction.
\end{myenumerate}

A {\it goal\/} in \HOL{} is a pair \ml{([$t_1$;\ldots;$t_n$],$t$)} of
\ML{} type {\small\verb|term list * term|}. An {\it achievement\/} of
such a goal is a theorem \ml{$t_1$,$\ldots$,$t_n$\ |-\ $t$}.  A tactic
is an \ML{} function that when applied to a goal generates subgoals
together with a {\it justification function\/} or {\it validation\/},
which will be an \ML{} derived inference rule, that can be used to
infer an achievement of the original goal from achievements of the
subgoals.

If $T$ is a tactic (\ie\ an \ML{} function of type \ml{goal -> (goal
  list * (thm list -> thm))}) and $g$ is a goal, then applying $T$ to
$g$ (\ie\ evaluating the \ML{} expression $T\ g$) will result in an
object which is a pair whose first component is a list of goals and
whose second component is a justification function, \ie\ a value with
\ML{} type {\small\verb|thm list -> thm|}.

An example tactic is \ml{CONJ\_TAC} which implements (i) and (ii)
above.  For example, consider the utterly trivial goal of showing
{\small\verb|T /\ T|}, where \ml{T} is a constant that stands for
$true$:

\begin{session}\begin{verbatim}
- val goal1 =([]:term list, ``T /\ T``);
> val goal1 = ([], ``T /\ T``) : term list * term

- CONJ_TAC goal1;
> val it =
    ([([], ``T``), ([], ``T``)], fn)
    : (term list * term) list * (thm list -> thm)

- val (goal_list,just_fn) = it;
> val goal_list =
    [([], ``T``), ([], ``T``)]
    : (term list * term) list
  val just_fn = fn : thm list -> thm
\end{verbatim}\end{session}

\noindent \ml{CONJ\_TAC} has produced a goal  list consisting  of two identical
subgoals of just showing \ml{([],"T")}.  Now, there is a preproved
theorem in \HOL, called \ml{TRUTH}, that achieves this goal:

\begin{session}\begin{verbatim}
- TRUTH;
> val it = [] |- T : thm
\end{verbatim}\end{session}

\noindent Applying the justification function \ml{just\_fn} to a list
of theorems achieving the goals in \ml{goal\_list} results
in a theorem achieving the original goal:

\begin{session}\begin{verbatim}
- just_fn [TRUTH,TRUTH];
> val it =  [] |- T /\ T : thm
\end{verbatim}\end{session}

    Although this example is trivial, it does illustrate the essential
    idea of tactics.  Note that tactics are not special
    theorem-proving primitives; they are just \ML{} functions.  For
    example, the definition of \ml{CONJ\_TAC} is simply:

\begin{hol}\begin{verbatim}
   fun CONJ_TAC (asl,w) = let
     val (l,r) = dest_conj w
   in
     ([(asl,l), (asl,r)], fn [th1,th2] => CONJ th1 th2)
   end
\end{verbatim}\end{hol}

\noindent The \ML{} function \ml{dest\_conj} splits a conjunction into its
two conjuncts: If \ml{(asl,``$t_1$}{\small\verb|/\|}\ml{$t_2$``)} is a
goal, then \ml{CONJ\_TAC} splits it into the list of two subgoals
\ml{(asl,$t_1$)} and \ml{(asl,$t_2$)}. The justification function,
{\small\verb|fn [th1,th2] => CONJ th1 th2|} takes a list
\ml{[$th_1$,$th_2$]} of theorems and applies the rule \ml{CONJ} to
$th_1$ and $th_2$.

To summarize: if $T$ is a tactic and $g$ is a goal, then applying $T$
to $g$ will result in a pair whose first component is a list of goals
and whose second component is a justification function, with \ML{} type
{\small\verb|thm list -> thm|}.

Suppose
$T\ g${\small\verb| = ([|}$g_1${\small\verb|,|}$\ldots${\small\verb|,|}$g_n${\small\verb|],|}$p${\small\verb|)|}.
The idea is that $g_1$ , $\ldots$ , $g_n$ are subgoals and $p$ is a
`justification' of the reduction of goal $g$ to subgoals $g_1$ ,
$\ldots$ , $g_n$.  Suppose further that the subgoals $g_1$ , $\ldots$
, $g_n$ have been solved.  This would mean that theorems $th_1$ ,
$\ldots$ , $th_n$ had been proved such that each $th_i$ ($1\leq i\leq
n$) `achieves' the goal $g_i$.  The justification $p$ (produced by
applying $T$ to $g$) is an \ML{} function which when applied to the
list
{\small\verb|[|}$th_1${\small\verb|,|}$\ldots${\small\verb|,|}$th_n${\small\verb|]|}
returns a theorem, $th$, which `achieves' the original goal $g$.  Thus
$p$ is a function for converting a solution of the subgoals to a
solution of the original goal. If $p$ does this successfully, then the
tactic $T$ is called {\it valid\/}.  Invalid tactics cannot result in
the proof of invalid theorems; the worst they can do is result in
insolvable goals or unintended theorems being proved.  If $T$ were
invalid and were used to reduce goal $g$ to subgoals $g_1$ , $\ldots$
, $g_n$, then effort might be spent proving theorems $th_1$ , $\ldots$
, $th_n$ to achieve the subgoals $g_1$ , $\ldots$ , $g_n$, only to
find out after the work is done that this is a blind alley because
$p${\small\verb|[|}$th_1${\small\verb|,|}$\ldots${\small\verb|,|}$th_n${\small\verb|]|}
doesn't achieve $g$ (\ie\ it fails, or else it achieves some other
goal).

A theorem {\it achieves\/} a goal if the assumptions of the theorem are
included in the assumptions of the goal {\it and\/} if the conclusion of the
theorems is equal (up to the renaming of bound variables) to the conclusion of
the goal. More precisely, a theorem
\begin{center}
$t_1$, $\dots$, $t_m${\small\verb% |- %}$t$
\end{center}

\noindent  achieves a goal
\begin{center}
{\small\verb|([|}$u_1${\small\verb|,|}$\ldots${\small\verb|,|}$u_n${\small\verb|],|}$u${\small\verb|)|}
\end{center}

\noindent if and only if $\{t_1,\ldots,t_m\}$
is a subset of $\{u_1,\ldots,u_n\}$ and $t$ is equal to $u$ (up to
renaming of bound variables).  For example, the goal
{\small\verb|([``x=y``, ``y=z``, ``z=w``], ``x=z``)|} is achieved by
the theorem {\small\verb+[x=y, y=z] |- x=z+} (the assumption
{\small\verb|``z=w``|} is not needed).

A tactic {\it solves\/} a goal if it reduces the goal
to the empty list
of subgoals. Thus $T$ solves $g$ if
$T\ g${\small\verb| = ([],|}$p${\small\verb|)|}.
If this is the case and if $T$ is valid, then $p${\small\verb|[]|}
will evaluate to a theorem achieving $g$.
Thus if $T$ solves $g$ then the \ML{} expression
{\small\verb|snd(|}$T\ g${\small\verb|)[]|} evaluates to
a theorem achieving $g$.

Tactics are specified using the following notation:

\begin{center}
\begin{tabular}{c} \\
$goal$ \\ \tacticline
$goal_1\ \ \ goal_2 \ \ \ \cdots\ \ \ goal_n$ \\
\end{tabular}
\end{center}

\noindent For example, a tactic called {\small\verb|CONJ_TAC|} is described by

\newcommand\ttbs{\texttt{\symbol{"5C}}}
\newcommand\ttland{\texttt{/\ttbs}}

\begin{center}
\begin{tabular}{lr} \\
\multicolumn{2}{c}{$t_1$ \ttland{} $t_2$} \\ \tacticline
$t_1$ & $t_2$ \\
\end{tabular}
\end{center}

\noindent Thus {\small\verb|CONJ_TAC|} reduces a goal of the form
{\small\verb|(|}$\Gamma${\small\verb|,``|}$t_1${\small\verb|/\|}$t_2${\small\verb|``)|}
to subgoals
{\small\verb|(|}$\Gamma${\small\verb|,``|}$t_1${\small\verb|``)|} and {\small\verb|(|}$\Gamma${\small\verb|,``|}$t_2${\small\verb|``)|}.
The fact that the assumptions of the top-level goal
are propagated unchanged to the two subgoals is indicated by the absence
of assumptions in the notation.

Another example is {\small\verb|numLib.INDUCT_TAC|}, the tactic for
doing mathematical induction on the natural numbers:

\begin{center}
\begin{tabular}{lr} \\
\multicolumn{2}{c}{\texttt{!}$n$\texttt{.}$t[n]$} \\ \tacticline
$t[\texttt{0}]$ & $\quad\{t[n]\}\ t[\texttt{SUC}\;n]$
\end{tabular}
\end{center}

{\small\verb|INDUCT_TAC|} reduces a goal
{\small\verb|(|}$\Gamma${\small\verb|,``!|}$n${\small\verb|.|}$t[n]${\small\verb|``)|} to a basis subgoal
{\small\verb|(|}$\Gamma${\small\verb|,``|}$t[${\small\verb|0|}$]${\small\verb|``)|}
and an induction step subgoal
{\small\verb|(|}$\Gamma\cup\{${\small\verb|``|}$t[n]${\small\verb|``|}$\}${\small\verb|,``|}$t[${\small\verb|SUC |}$n]${\small\verb|``)|}.
The extra induction assumption {\small\verb|``|}$t[n]${\small\verb|``|}
is indicated in the tactic notation with set brackets.

\begin{session}\begin{verbatim}
- numLib.INDUCT_TAC([], ``!m n. m+n = n+m``);
> val it =
    ([([], ``!n. 0 + n = n + 0``),
      ([``!n. m + n = n + m``], ``!n. SUC m + n = n + SUC m``)], fn)
    : (term list * term) list * (thm list -> thm)
\end{verbatim}\end{session}

\noindent The first subgoal is the basis case and the second subgoal is
the step case.

Tactics generally fail (in the \ML{} sense, \ie\ raise an exception) if
they are applied to inappropriate goals. For example,
{\small\verb|CONJ_TAC|} will fail if it is applied to a goal whose
conclusion is not a conjunction. Some tactics never fail, for example
{\small\verb|ALL_TAC|}

\begin{center}
\begin{tabular}{c} \\
$t$ \\ \tacticline
$t$
\end{tabular}
\end{center}

\noindent is the `identity tactic'; it reduces a goal
{\small\verb|(|}$\Gamma${\small\verb|,|}$t${\small\verb|)|} to the
single subgoal
{\small\verb|(|}$\Gamma${\small\verb|,|}$t${\small\verb|)|}---\ie\ it
has no effect. {\small\verb|ALL_TAC|} is useful for writing complex
tactics using tacticals.

\subsection{Using tactics to prove theorems}
\label{using-tactics}

Suppose goal $g$ is to be solved. If $g$ is simple it might be
possible to immediately think up a tactic, $T$ say, which reduces it
to the empty list of subgoals. If this is the case then executing:

$\ ${\small\verb| val (|}$gl${\small\verb|,|}$p${\small\verb|) = |}$T\ g$

\noindent will bind $p$ to a function which when applied to the empty list
of theorems yields a theorem $th$ achieving $g$.  (The declaration
above will also bind $gl$ to the empty list of goals.) Thus a theorem
achieving $g$ can be computed by executing:

$\ ${\small\verb| val |}$th${\small\verb| = |}$p${\small\verb|[]|}

\noindent This will be illustrated using \ml{REWRITE\_TAC} which takes a list
of equations (empty in the example that follows) and tries to prove a goal
by rewriting with these equations together with
\ml{basic\_rewrites}:

\begin{session}\begin{verbatim}
- val goal2 = ([]:term list, ``T /\ x ==> x \/ (y /\ F)``);
> val goal2 = ([], ``T /\ x ==> x \/ y /\ F``) : (term list * term)

- REWRITE_TAC [] goal2;
> val it = ([], fn) : (term list * term) list * (thm list -> thm)

- #2 it [];
> val it =  [] |- T /\ x ==> x \/ y /\ F : thm
\end{verbatim}\end{session}

\noindent Proved theorems are usually stored in the current theory
so that they can be used in subsequent sessions.

The built-in function
 \ml{store\_thm} of
\ML{} type {\small\verb|(string * term * tactic) -> thm|} facilitates the use
of tactics:
{\small\verb|store_thm("foo",|}$t${\small\verb|,|}$T${\small\verb|)|} proves
the goal   {\small\verb|([],|}$t${\small\verb|)|}   (\ie\  the   goal  with  no
assumptions and  conclusion  $t$)  using  tactic  $T$  and  saves the resulting
theorem with name {\small\verb|foo|} on the current theory.

If the theorem is not to be saved, the function \ml{prove} of type
{\small\verb|(term * tactic) -> thm|} can be used.  Evaluating
{\small\verb|prove(|}$t${\small\verb|,|}$T${\small\verb|)|} proves   the   goal
{\small\verb|([],|}$t${\small\verb|)|} using $T$ and returns the result without
saving it.  In both cases  the evaluation  fails if  $T$ does  not solve the
goal {\small\verb|([],|}$t${\small\verb|)|}.

When conducting a proof that involves many subgoals and tactics, it is
necessary to keep track of all the justification functions and compose
them in the correct order.  While this is feasible even in large
proofs, it is tedious.  \HOL{} provides a package for building and
traversing the tree of subgoals, stacking the justification functions
and applying them properly; this package was originally implemented
for \LCF\ by Larry Paulson.

The subgoal package implements a simple framework for interactive
proof. A proof tree is created and traversed top-down.  The current
goal can be expanded into subgoals using a tactic; the subgoals are
pushed onto a goal stack and the justification function onto a proof
stack.  Subgoals can be considered in any order.  If the tactic solves
a subgoal (\ie\ returns an empty subgoal list), then the package
proceeds to the next subgoal in the tree.

The function \ml{set\_goal} of type \ml{goal -> proofs}\footnote{The
  \ml{proofs} type stores multiple proof attempts (multiple trees) at
  once.} initializes the subgoal package with a new goal. Usually
top-level goals have no assumptions; the function \ml{g} is useful in
this case.

To illustrate the subgoal package the trivial theorem $\vdash
\uquant{m}m+0=m$ will be proved from the definition of addition (we
first \ml{open} the theory of arithmetic to bring ML bindings for its
theorems to the top level):

\begin{session}\begin{verbatim}
- open arithmeticTheory;
> ...
- ADD;
> val it = |- (!n. 0 + n = n) /\ (!m n. (SUC m) + n = SUC(m + n)) : thm
\end{verbatim}\end{session}

\noindent Notice that \ml{ADD} specifies
$0+m=m$ but not $m+0=m$. Of course, $\uquant{m\ n}m+n = n+m$ is true,
but the first step of the proof is to show $\uquant{m}m+0=m$ from the
definition of addition.  Notice that the function \ml{g} does not take
a term as an argument, but rather a \emph{quotation}, with only one
set of back-quotes.

\begin{session}\begin{verbatim}
- g `!m. m+0=m`;
> val it =
    Proof manager status: 1 proof.
    1. Incomplete:
         Initial goal:
         !m. m + 0 = m
\end{verbatim}\end{session}

\noindent This sets up the goal. Next the goal is split into a basis and step case
with \ml{Induct}. To do this the function \ml{e} (or,
equivalently, \ml{expand}) is used. This applies a tactic to the top
goal on the stack, then pushes the resulting subgoals onto the goal
stack, then prints the resulting subgoals. If there are no subgoals,
the justification function is applied to the theorems solving the
subgoals that have been proved and the resulting theorems are printed.

\begin{session}\begin{verbatim}
- e Induct;
OK..
2 subgoals:
> val it =
    SUC m + 0 = SUC m
    ------------------------------------
      m + 0 = m

    0 + 0 = 0
\end{verbatim}\end{session}

\noindent The top of the goal stack is printed last. The basis case
is an instance of the definition of addition, so is solved by
rewriting with \ml{ADD}.

\begin{session}\begin{verbatim}
- e(REWRITE_TAC[ADD]);
OK..

Goal proved.
 [] |- 0 + 0 = 0

Remaining subgoals:
> val it =
    SUC m + 0 = SUC m
    ------------------------------------
      m + 0 = m
\end{verbatim}\end{session}

\noindent The basis is solved and the goal
stack popped so that its top is now the step case, namely showing that
{\small\verb|(SUC m) + 0 = SUC m|} under the assumption
{\small\verb|m + 0 = m|}. This goal can be solved by rewriting first with the
definition of addition:

\begin{session}\begin{verbatim}
- e(REWRITE_TAC[ADD]);
OK..
1 subgoal:
> val it =
    SUC (m + 0) = SUC m
    ------------------------------------
      m + 0 = m
\end{verbatim}\label{session:rewrite-add}\end{session}

\noindent and then with the assumption \ml{m+0=m}. The tactic
\ml{ASM\_REWRITE\_TAC} is used to rewrite with the assumptions of a
goal. It is just like \ml{REWRITE\_TAC} except that it adds the
assumptions to the list of equations used for rewriting. For the
example here no equations besides the assumptions are needed, so
\ml{ASM\_REWRITE\_TAC} is given the empty list of equations.

\begin{session}\begin{verbatim}
- e(ASM_REWRITE_TAC[]);
OK..

Goal proved.
 [m + 0 = m] |- SUC (m + 0) = SUC m

Goal proved.
 [m + 0 = m] |- SUC m + 0 = SUC m
> val it =
    Initial goal proved.
     [] |- !m. m + 0 = m
    : GoalstackPure.goalstack
\end{verbatim}\label{session:asm-rewrite-add}\end{session}

\noindent The top goal is solved, hence the preceding goal (the step case)
is solved too, and since the basis is already solved, the main goal is
solved.

The theorem achieving the goal can be extracted from the subgoal package with
\ml{top\_thm}:

\begin{session}\begin{verbatim}
- top_thm();
val it = [] |- !m. m + 0 = m : thm
\end{verbatim}\end{session}

    The proof just done can be `optimized'. For example, instead of
    first rewriting with \ml{ADD} (box \ref{session:rewrite-add}) and
    then with the assumptions (box \ref{session:asm-rewrite-add}), a
    single rewriting with \ml{ADD} and the assumptions would suffice.
    To illustrate, the last two steps of the proof will be `undone'
    using the function \ml{backup} (also, \ml{b}) which restores the
    previous state of the goal and theorem stacks.

\begin{session}\begin{verbatim}
- b();
> val it =
    SUC (m + 0) = SUC m
    ------------------------------------
      m + 0 = m

- b();
> val it =
    SUC m + 0 = SUC m
    ------------------------------------
      m + 0 = m

\end{verbatim}\end{session}

\noindent The proof can now be completed in one step instead of two:

\begin{session}\begin{verbatim}
- e(ASM_REWRITE_TAC[ADD]);
OK..

Goal proved.
 [m + 0 = m] |- SUC m + 0 = SUC m
> val it =
    Initial goal proved.
     [] |- !m. m + 0 = m
    : GoalstackPure.goalstack
\end{verbatim}\end{session}

    The order in which goals are attacked can be adjusted using
    \ml{rotate\ }$n$ (alternatively,~\ml{r}) which rotates the goal
    stack by $n$. For example:

\begin{session}\begin{verbatim}
- b(); b();
> ...

> val it =
    SUC m + 0 = SUC m
    ------------------------------------
      m + 0 = m

    0 + 0 = 0

- r 1;
> val it =
    0 + 0 = 0

    SUC m + 0 = SUC m
    ------------------------------------
      m + 0 = m
\end{verbatim}\end{session}

\noindent The top goal is now the step case not the basis case, so expanding
with a tactic will apply the tactic to the step case.

\begin{session}\begin{verbatim}
- e(ASM_REWRITE_TAC[ADD]);
OK..

Goal proved.
 [m + 0 = m] |- SUC m + 0 = SUC m

Remaining subgoals:
> val it =
    0 + 0 = 0
\end{verbatim}\end{session}

    It is possible to do the whole proof in one step, but this
    requires a compound tactic built using the {\it
      tactical\/}\footnote{This word was invented by Robin Milner:
      `tactical' is to `tactic` as `functional' is to `function'.}
    \ml{THENL}.  Tacticals are higher order operations for combining
    tactics.

\subsection{Tacticals}
\label{tacticals}

A {\it tactical\/} is an \ML{} function that takes one or more tactics
as arguments, possibly with other arguments as well, and returns a
tactic as its result.  The various parameters passed to tacticals are
reflected in the various \ML{} types that the built-in tacticals have.
Some important tacticals in the \HOL{} system are listed below.

\subsubsection{\tt THENL : tactic -> tactic list -> tactic}

If tactic $T$ produces $n$ subgoals and $T_1$, $\ldots$ , $T_n$ are
tactics then $T${\small\verb| THENL|}
{\small\verb|[|}$T_1${\small\verb|;|}$\ldots${\small\verb|;|}$T_n${\small\verb|]|}
is a tactic which first applies $T$ and then applies $T_i$ to the
$i$th subgoal produced by $T$.  The tactical {\small\verb|THENL|} is
useful if one wants to do different things to different subgoals.

\ml{THENL} can be illustrated by doing the proof of $\vdash \uquant{m}m+0=m$ in
one step.

\setcounter{sessioncount}{0}
\begin{session}\begin{verbatim}
- g `!m. m + 0 = m`;
> val it =
    Proof manager status: 1 proof.
    1. Incomplete:
         Initial goal:
         !m. m + 0 = m

- e(INDUCT_TAC THENL [REWRITE_TAC[ADD], ASM_REWRITE_TAC[ADD]]);
OK..
> val it =
    Initial goal proved.
     [] |- !m. m + 0 = m
\end{verbatim}\end{session}

\noindent The compound tactic
{\small\verb|INDUCT_TAC THENL [REWRITE_TAC[ADD];ASM_REWRITE_TAC[ADD]]|}
first applies \ml{INDUCT\_TAC} and then applies
\ml{REWRITE\_TAC[ADD]} to the first subgoal (the basis) and
\ml{ASM\_REWRITE\_TAC[ADD]} to the second subgoal (the step).

The tactical {\small\verb|THENL|} is useful for doing different things
to different subgoals. The tactical \ml{THEN} can be used to apply the
same tactic to all subgoals.

\subsubsection{\tt THEN : tactic -> tactic -> tactic}\label{THEN}

The tactical {\small\verb|THEN|} is an \ML{} infix. If $T_1$ and $T_2$
are tactics, then the \ML{} expression $T_1${\small\verb| THEN |}$T_2$
evaluates to a tactic which first applies $T_1$ and then applies $T_2$
to all the subgoals produced by $T_1$.

In fact, \ml{ASM\_REWRITE\_TAC[ADD]} will solve the basis as well as
the step case of the induction for $\uquant{m}m+0=m$, so there is an
even simpler one-step proof than the one above:
\setcounter{sessioncount}{0}
\begin{session}\begin{verbatim}
- g `!m. m+0 = m`;
> val it =
    Proof manager status: 1 proof.
    1. Incomplete:
         Initial goal:
         !m. m + 0 = m

- e(INDUCT_TAC THEN ASM_REWRITE_TAC[ADD]);
OK..
> val it =
    Initial goal proved.
     [] |- !m. m + 0 = m
\end{verbatim}\end{session}

\noindent This is typical: it is common to use a single tactic for several
goals. Here, for example, are the first four consequences of the
definition \ml{ADD} of addition that are pre-proved when the built-in
theory \ml{arithmetic} \HOL{} is made.

\begin{hol}\begin{verbatim}
   val ADD_0 = prove (
     ``!m. m + 0 = m``,
     INDUCT_TAC THEN ASM_REWRITE_TAC[ADD]);
\end{verbatim}\end{hol}

\begin{hol}\begin{verbatim}
   val ADD_SUC = prove (
     ``!m n. SUC(m + n) = m + SUC n``,
     INDUCT_TAC THEN ASM_REWRITE_TAC[ADD]);
\end{verbatim}\end{hol}

\begin{hol}\begin{verbatim}
   val ADD_CLAUSES = prove (
     ``(0 + m = m)              /\
       (m + 0 = m)              /\
       (SUC m + n = SUC(m + n)) /\
       (m + SUC n = SUC(m + n))``,
     REWRITE_TAC[ADD, ADD_0, ADD_SUC]);
\end{verbatim}\end{hol}

\begin{hol}\begin{verbatim}
   val ADD_COMM = prove (
     ``!m n. m + n = n + m``,
     INDUCT_TAC THEN ASM_REWRITE_TAC[ADD_0, ADD, ADD_SUC]);
\end{verbatim}\end{hol}

\noindent These proofs are performed when the \HOL{} system is made and the
theorems are saved in the theory \ml{arithmetic}. The complete list of
proofs for this built-in theory can be found in the file
\ml{src/num/arithmeticScript.sml}.

\subsubsection{\tt ORELSE : tactic -> tactic -> tactic}\label{ORELSE}

The tactical {\small\verb|ORELSE|} is an \ML{} infix. If $T_1$ and
$T_2$ are tactics,
%\index{tacticals!for alternation}
then $T_1${\small\verb| ORELSE |}$T_2$ evaluates to a tactic which
applies $T_1$ unless that fails; if it fails, it applies $T_2$.
\ml{ORELSE} is defined in \ML{} as a curried infix by\footnote{This is
  a minor simplification.}

\begin{hol}
   {\small\verb|(|}$T_1${\small\verb| ORELSE |}$T_2${\small\verb|)|} $g$
   {\small\verb|=|}  $T_1\; g$ {\small\verb|handle _ =>|} $T_2\; g$
\end{hol}
%\index{alternation!of tactics|)}

\subsubsection{\tt REPEAT : tactic -> tactic}

If $T$ is a tactic then {\small\verb|REPEAT |}$T$ is a tactic which
repeatedly applies $T$ until it fails. This can be illustrated in
conjunction with \ml{GEN\_TAC}, which is specified by:

\begin{center}
\begin{tabular}{c} \\
{\small\verb|!|}$x${\small\verb|.|}$t[x]$
\\ \tacticline
$t[x']$
\\
\end{tabular}
\end{center}

\begin{itemize}
\item Where $x'$ is a variant of $x$
not free in the goal or the assumptions.
\end{itemize}

\noindent \ml{GEN\_TAC} strips off one quantifier;
\ml{REPEAT\ GEN\_TAC} strips off all quantifiers:

\begin{session}\begin{verbatim}
- g `!x y z. x+(y+z) = (x+y)+z`;
> val it =
    Proof manager status: 1 proof.
    1. Incomplete:
         Initial goal:
         !x y z. x + (y + z) = x + y + z

- e GEN_TAC;
OK..
1 subgoal:
> val it =
    !y z. x + (y + z) = x + y + z

- e(REPEAT GEN_TAC);
OK..
1 subgoal:
> val it =
    x + (y + z) = x + y + z
\end{verbatim}\end{session}

\subsection{Some tactics built into HOL}

This section contains a summary of some of the tactics built into the
\HOL{} system (including those already discussed).  The tactics given
here are those that are used in the parity checking example.

Before beginning, note that the \ML{} type {\small\verb|thm_tactic|}
abbreviates {\small\verb|thm->tactic|}, and the type
{\small\verb|conv|}\footnote{The type {\small{\tt conv}} comes from
  Larry Paulson's theory of conversions \cite{lcp_rewrite}.}
abbreviates {\small\verb|term->thm|}.

\subsubsection{\tt REWRITE\_TAC : thm list -> tactic}
\label{rewrite}

\begin{itemize}
\item{\bf Summary:} {\small\verb|REWRITE_TAC[|}$th_1${\small\verb|,|}$\ldots${\small\verb|,|}$th_n${\small\verb|]|}
simplifies the goal by rewriting
it with the explicitly given theorems $th_1$, $\ldots$ , $th_n$,
and various built-in rewriting rules.

\begin{center}
\begin{tabular}{c} \\
$\{t_1, \ldots , t_m\}t$
\\ \tacticline
$\{t_1, \ldots , t_m\}t'$
\\
\end{tabular}
\end{center}

\noindent where $t'$ is obtained from $t$ by rewriting with
\begin{enumerate}
\item  $th_1$, $\ldots$ , $th_n$ and
\item  the standard rewrites held in the \ML{} variable {\small\verb|basic_rewrites|}.
\end{enumerate}

\item{\bf Uses:} Simplifying goals using previously proved theorems.

\item{\bf Other rewriting tactics}:
\begin{enumerate}
\item {\small\verb|ASM_REWRITE_TAC|} adds the assumptions of the goal
  to the list of theorems used for rewriting.
\item {\small\verb|PURE_REWRITE_TAC|} uses neither the assumptions nor
  the built-in rewrites.
\item {\small\verb|bossLib.RW_TAC|} of type \ml{simpLib.simpset -> thm
    list -> tactic}.  A \ml{simpset} is a special collection of
  rewriting theorems and other theorem-proving functionality.  Values
  defined by \HOL{} include \ml{bossLib.base\_ss}, which has basic
  knowledge of the boolean connectives, \ml{bossLib.arith\_ss} which
  ``knows'' all about arithmetic, and \ml{HOLSimps.hol\_ss}, which
  includes theorems appropriate for lists, pairs, and arithmetic.
  Additional theorems for rewriting can be added using the second
  argument of \ml{RW\_TAC}.
\end{enumerate}
\end{itemize}

\subsubsection{\tt CONJ\_TAC : tactic}\label{CONJTAC}

\begin{itemize}

\item{\bf Summary:} Splits a
goal {\small\verb|``|}$t_1${\small\verb|/\|}$t_2${\small\verb|``|} into two subgoals {\small\verb|``|}$t_1${\small\verb|``|}
and {\small\verb|``|}$t_2${\small\verb|``|}.

\begin{center}
\begin{tabular}{lr} \\
\multicolumn{2}{c}{$t_1$ \ttland{} $t_2$} \\ \tacticline
$t_1$ & $t_2$ \\
\end{tabular}
\end{center}

\item{\bf Uses:} Solving conjunctive goals.
{\small\verb|CONJ_TAC|} is invoked by {\small\verb|STRIP_TAC|} (see below).

\end{itemize}

\subsubsection{\tt EQ\_TAC : tactic}\label{EQTAC}

\begin{itemize}

\item{\bf Summary:}
{\small\verb|EQ_TAC|}
splits an equational goal into two implications (the `if-case' and
the `only-if' case):

\begin{center}

\begin{tabular}{lr} \\
\multicolumn{2}{c}{$u\; \ml{=}\; v$} \\ \tacticline
$u\; \ml{==>}\; v$ & $\quad v\; \ml{==>}\; u$ \\
\end{tabular}
\end{center}

\item{\bf Use:} Proving logical equivalences, \ie\ goals of the form
``$u$\ml{=}$v$'' where $u$ and $v$ are boolean terms.

\end{itemize}

\subsubsection{\tt DISCH\_TAC : tactic}\label{DISCHTAC}

\begin{itemize}

\item{\bf Summary:} Moves the antecedent
of an implicative goal into the assumptions.

\begin{center}
\begin{tabular}{c} \\
$u${\small\verb| ==> |}$v$
\\ \tacticline
$\{u\}v$
\\
\end{tabular}
\end{center}

\item{\bf Uses:} Solving goals of the form
{\small\verb|``|}$u${\small\verb| ==> |}$v${\small\verb|``|} by assuming {\small\verb|``|}$u${\small\verb|``|} and then solving
{\small\verb|``|}$v${\small\verb|``|}.
{\small\verb|STRIP_TAC|} (see below) will invoke {\small\verb|DISCH_TAC|} on implicative goals.
\end{itemize}

\subsubsection{\tt GEN\_TAC : tactic}

\begin{itemize}

\item{\bf  Summary:} Strips off one universal quantifier.

\begin{center}
\begin{tabular}{c} \\
{\small\verb|!|}$x${\small\verb|.|}$t[x]$
\\ \tacticline
$t[x']$
\\
\end{tabular}
\end{center}

\noindent Where $x'$ is a variant of $x$
not free in the goal or the assumptions.

\item{\bf   Uses:} Solving universally quantified goals.
{\small\verb|REPEAT GEN_TAC|} strips off all
universal quantifiers and is often the first thing one does in a proof.
{\small\verb|STRIP_TAC|} (see below) applies {\small\verb|GEN_TAC|} to universally quantified goals.
\end{itemize}

\subsubsection{\tt PROVE\_TAC : thm list -> tactic}

\begin{itemize}
\item {\bf Summary:} Used to do first order reasoning, solving the
  goal completely if successful, failing otherwise.  Using the
  provided theorems and the assumptions of the goal,
  {\small\verb|PROVE_TAC|} does a search for possible proofs of the
  goal.  Eventually fails if the search fails to find a proof shorter
  than a reasonable depth.
\item {\bf Uses:} To finish a goal off when it is clear that it is a
  consequence of the assumptions and the provided theorems.
\end{itemize}

\subsubsection{\tt STRIP\_TAC : tactic}

\begin{itemize}

\item{\bf Summary:} Breaks a goal apart.  {\small\verb|STRIP_TAC|}
  removes one outer connective from the goal, using
  {\small\verb|CONJ_TAC|}, {\small\verb|DISCH_TAC|},
  {\small\verb|GEN_TAC|}, \etc\ If the goal is
$t_1${\small\verb|/\|}$\cdots${\small\verb|/\|}$t_n${\small\verb| ==> |}$t$
then {\small\verb|STRIP_TAC|} makes each $t_i$ into a separate assumption.

\item{\bf Uses:} Useful for splitting a goal up into manageable pieces.
Often the best thing to do first is {\small\verb|REPEAT STRIP_TAC|}.
\end{itemize}

\subsubsection{\tt SUBST\_TAC : thm list -> tactic}

\begin{itemize}

\item{\bf Summary:}
  {\small\verb+SUBST_TAC[|-+}$u_1${\small\verb|=|}$v_1${\small\verb|,|}$\ldots${\small\verb+,|-+}$u_n${\small\verb|=|}$v_n${\small\verb|]|}
  converts a goal $t[u_1,\ldots ,u_n]$ to the subgoal form
  $t[v_1,\ldots ,v_n]$.

\item{\bf Uses:} To make replacements for terms in situations in which
  {\small\verb|REWRITE_TAC|} is too general or would loop.
\end{itemize}

\subsubsection{\tt ACCEPT\_TAC : thm -> tactic}\label{ACCEPTTAC}

\begin{itemize}

\item{\bf Summary:} {\small\verb|ACCEPT_TAC |}$th$
is a tactic that solves any goal that is
achieved by $th$.

\item{\bf Use:} Incorporating forward proofs, or theorems already
  proved, into goal directed proofs.  For example, one might reduce a
  goal $g$ to subgoals $g_1$, $\dots$, $g_n$ using a tactic $T$ and
  then prove theorems $th_1$ , $\dots$, $th_n$ respectively achieving
  these goals by forward proof. The tactic

\[\ml{  T THENL[ACCEPT\_TAC }th_1\ml{,}\ldots\ml{,ACCEPT\_TAC }th_n\ml{]}
\]

would then solve $g$, where \ml{THENL}
%\index{THENL@\ml{THENL}}
is the tactical that applies the respective elements of the tactic
list to the subgoals produced by \ml{T}.

\end{itemize}

\subsubsection{\tt ALL\_TAC : tactic}

\begin{itemize}
\item{\bf Summary:} Identity tactic for the tactical {\small\verb%THEN%}
(see \DESCRIPTION).

\item{\bf Uses:}
\begin{enumerate}
\item Writing tacticals (see description of {\small\verb|REPEAT|}
in \DESCRIPTION).
\item With {\small\verb%THENL%}; for example, if tactic $T$ produces two subgoals
and we want to apply $T_1$
to the first one but to do nothing to the second, then
the tactic to use is $T${\small\verb% THENL[%}$T_1${\small\verb|;ALL_TAC]|}.
\end{enumerate}
\end{itemize}

\subsubsection{\tt NO\_TAC : tactic}

\begin{itemize}
\item{\bf Summary:} Tactic that always fails.

\item{\bf Uses:} Writing tacticals.
\end{itemize}


\chapter{Isomorphism Theorems and Series}
\section {Isomorphism Theorems:}
{\bf Isomorphism Theorem:} (1) If $\varphi: G \rightarrow H$ is a homomorphism,
$G/ker(\varphi) \cong Im(\varphi)$,
(2)  If
$G \triangleright H$ and $G \triangleright N$ and $N \subseteq H \subseteq G$
then $G/H \cong (G/N)/(H/N)$,
(3) If $G=HN$, $G \triangleright N$ then
$HN/N \cong H/(H \cap N)$.
\begin{quote}
\emph{Proof of 1:}  Let $N= ker(\phi)$ and define $\psi(Nx)= \phi(x)$.  $\psi$ is
well defined since $Nx=Ny \rightarrow y=nx$ and $\phi(y)= \phi(nx)= \phi(n) \phi(x)= \phi(x)$
and is a homomorphism since $\phi$ is.
If $\psi(Nx)=\psi(Ny)$ then $\phi(xy^{-1}) \in N$ so $Nx=Ny$ and $\psi$ is 1-1.  The image
of $\psi$ is $Im(\phi)$. So $\psi$ is an isomorphism.
\\
\\
\emph{Proof of 2:}  Define $\psi(Kx)= Hx$.  $\psi$ is well defined and is a homomorphism.
$ker(\psi)= Kh, h \in H$ so $(G/K)/(H/K) \cong G/H$ by the previous result.
\\
\\
\emph{Proof of 3:}  Define $\psi(Nh)= (H \cap N) h$.  $\psi$ is well defined and is a homomorphism.
$ker(\psi)=  \{ h \in H: h \in H \cap N \}$ so $HN/N \cong H/(H \cap N)$ by (1).
\end{quote}
{\bf Definition 1:}
A \emph{derived series} is a sequence $G^{[0]}=G$, $G^{[i+1]}= [G^{[i]}, G^{[i]}]$.
$G$ is \emph{solvable} iff
derived series terminates at ${1}$.
\\
\\
{\bf Definition 2:} The sequence of homomorphisms $A \rightarrow_{\alpha} B \rightarrow_{\beta} C$ is
\emph{exact} if $im(\alpha )= ker( \beta)$.  The sequence
$1 \rightarrow N \rightarrow_{id} G \rightarrow_{\varphi} H \rightarrow_{\epsilon} 1 $ is a
\emph{short exact sequence} if the map $id$ is inclusion and $\epsilon$ is the trivial homomorphism and
each subsequence is exact; in this case,
$G/N \approx H$ by the above.
\\
\\
{\bf Definition 3:}
A \emph{subnormal series} is a sequence of groups $G_i$ such that
$G= G_0 \rhd G_1 \rhd \ldots \rhd G_k =H$, if this happens, we say
$G \rhd \rhd H$.
\\
\\
{\bf Definition 4:}
A \emph{normal series} is a subnormal series where $G \rhd G_i, \forall i$.
\\
\\
{\bf Definition 5:}
A \emph{chief series} is a normal series with no repeated terms and no normal subgroup properly
lying between two series elements.
\\
\\
{\bf Definition 6:}
A \emph{composition series} is a subnormal series
terminating at $1$ in which $G_i/G_{i-1}$ is simple.
\\
\\
{\bf Lemma 1:} If $K \lhd H <G$ and $N \lhd G$ then $NK \lhd NH$.
\begin{quote}
\emph{Proof:}
Suppose $a, b \in N, h \in H, k \in K$.
$[ak,bh]=
k^{-1} a^{-1}h^{-1}b^{-1} ak bh=
k^{-1} a^{-1} (h^{-1}b^{-1} a h)
(h^{-1}kh) (h^{-1} bh) \in K N N K N = NK$.  So $[NK,NH] \subseteq NK$ and $NK \lhd NH$.
\end{quote}
{\bf Lemma 2:}
$A(A^* \cap B) \cap A^* \cap B^*= B(B^* \cap A) \cap A^* \cap B^*$.
\begin{quote}
\emph{Proof:}
It suffices to show
$A(A^* \cap B) \cap B^*= B(B^* \cap A) \cap A^*$.
Suppose $x=at \in A(A^* \cap B) \cap B^* , a \in A, t \in (A^* \cap B) \subseteq B^*$.
$x= t (t^{-1}at) \in BA \cap B^*$; since $x \in B^*, t^{-1} x \in B^*$
and $t^{-1} a t \in B^* \cap A$. Thus
$A(A^* \cap B) \cap B^* \subseteq B(B^* \cap A) \cap A^*$.
Symmetrically, $A(A^* \cap B) \cap B^* \supseteq B(B^* \cap A) \cap A^*$.
\end{quote}
{\bf Zassenhaus Butterfly Lemma:}  If
$A \triangleleft A^{*}$ and $B \triangleleft B^{*}$ then
$A(A^{*} \cap B) \triangleleft A(A^{*} \cap B^{*})$ and
$B(B^{*} \cap A) \triangleleft B(B^{*} \cap A^{*})$; further,
${\frac {A(A^{*} \cap B^{*})} {A(A^{*} \cap B) }} \cong
{\frac {B(B^{*} \cap A^{*})} {B(B^{*} \cap A) }}$.
\begin{quote}
\emph{Proof:}
Apply lemma 1 with $N=A$, $K=A^* \cap B$, $H= A^* \cap B^*$, $G=A^*$ to get
$A(A^* \cap B) \lhd A(A^* \cap B^*)$.  Symmetrically,
$B(B^* \cap A) \lhd B(A^* \cap B^*)$.
Set $K_1= A(A^* \cap B)$ and $K_2= B(B^* \cap A)$.  Let
$t_1 \in {\frac {A(A^{*} \cap B^{*})} {A(A^{*} \cap B) }},
t_2 \in {\frac {B(A^{*} \cap B^{*})} {B(B^{*} \cap A) }}$, $t_1=ayK_1, t_2=bzK_2$ where
$a \in A, b \in B, y, z \in (A^* \cap B^*)$.
$ay= y (y^{-1}ay)= ya'$ so $t_1= yK_1$.  Similarly, $t_2= zK_2$.
Define $h:
{\frac {A(A^{*} \cap B^{*})} {A(A^{*} \cap B) }} \rightarrow
{\frac {B(B^{*} \cap A^{*})} {B(B^{*} \cap A) }}$ by $h(yK_1)= y K_2$.  The map is
well defined since $yK_1=zK_1$ iff $z^{-1}y \in A(A^* \cap B^*) \cap A^* \cap B^*$
iff
$z^{-1}y \in B(A^* \cap B^*) \cap A^* \cap B^*$ by lemma 2.
\end{quote}
{\bf Theorem 1:}
Let $G$ be a finite group.
The following are equivalent: (1) $G$ is solvable, (2) $G$ has a normal series
terminating at the identity whose factor groups are abelian,
(3) $G$ has a subnormal series with cyclic quotients.
\begin{quote}
\emph{Proof:} $1 \rightarrow 2 \rightarrow 3$ is trivial.
Suppose $S: 1=G_r \lhd G_{r-1} \lhd \ldots \lhd G_1 \lhd G_0=G$ has abelian factors.
Claim: $G^{(j)} \le G_j$.  Note that $G' \subseteq G_1$ since $G/G_1$ is abelian.
The claim follows
from by an easy induction.
\end{quote}
{\bf Definition:} Two normal series,
$G_0 \geq G_1 \geq G_2 \geq \ldots \geq G_n$ and
$H_0 \geq H_1 \geq H_2 \geq \ldots \geq H_m$, are \emph{equivalent} if $m = n$ and
$G_i/G_{i+1} \cong H_{\pi(i)}/H_{\pi(i)+1}$ for some permutation, $\pi$.
\\
\\
{\bf Schreier's Theorem:} Two normal series for $G$ have equivalent refinements.
Two compositions series for $G$ are equivalent.
\begin{quote}
\emph{Proof:}  By induction on length ($l$) of shortest such
series.  If $l=1$, $G$ is simple.
Suppose $G=G_0 \ge G_1 \ge \ldots \ge G_r = 1$ and
$H=H_0 \ge H_1 \ge \ldots \ge H_t = 1$ and assume $l=r>t$ and that the theorem
is true for all series of length less than $l$. If $H_1=G_1$ then we are done by induction
on the shortened series.  Assume $G_1 \ne H_1$, $H_1 \lhd G, G_1 \lhd G$ then
$G_1H_1 = G$ and $G/G_1 \cong H_1/K, K= G_1 \cap H_1$. Consider the two series
$G_1 \ge G_2 \ldots \ge G_r = 1$ and
$G_1 \ge K \ge K_1 \ldots \ge K_t = 1$.
By induction, $r-1=t+1$ and they are equivalent.
Thus,
$H_1 \ge H_2 \ldots \ge H_s = 1$ and
$H_1 \ge K \ge K_1 \ldots \ge K_{r-2} = 1$ so $r=s$ and they
are equivalent.
\end{quote}
{\bf Jordan-Holder Theorem:} If $G$ has a composition series, $S$,  then any
subnormal series $S^*$ can be refined to a composition series and any two composition series
are equivalent.
\begin{quote}
\emph{Proof:} $S$ and $S^*$ have equivalent refinements by Schreier.  Remove any repetitions to
produce new equivalent refinements.  Since a composition series does not have proper refinements,
the series are equivalent.
\end{quote}
{\bf Butterfly Lemma:} If $U, V  \subseteq G$, $u \lhd U$, and $v \lhd V$ then
$u(U \cap v) \lhd u(U \cap V)$,
$(u \cap V)v \lhd (U \cap V)v$ and
$u(U \cap V)/u(U \cap v) \cong (U \cap V)v/(u \cap V)v$.
\begin{quote}
\emph{Proof:}
Let $H= (U \cap V)$ [resp. $(U \cap V)v$] and $N= u(U \cap v)$ [resp. $(u \cap V)v$].
We show $N \lhd HN$.
Let $x= a_1 b_1 \in HN$ and $y = a_2 b_2 \in N$, $a_1, a_2 \in u, b_1 \in (U \cap V), b_2 \in (U \cap v)$.
$x^{-1} y x = b_1^{-1} a_1^{-1} a_2 b_2 a_1 b_1 = b_1^{-1} a_1^{-1} a_2 b_1 b_1^{-1} b_2 b_1 b_1^{-1} a_1 b_1 =$
$a_3 b_3 a_4 b_3^{-1} a_1 b_3 = a_3 b_3 a_4 $, $a_3, a_4 \in u, b_3 \in (U \cap v)$.  So $x^{-1} y x =
= a_3 b_3 a_4 b_3^{-1} b_3$.  Thus $x^{-1} y x \in u(U \cap v)$ and $N \lhd H$.
$HN =(U \cap V)u(U \cap v) = u(U \cap V)$.  $u(U \cap V)/u(U \cap v) = HN/N \cong H/(H \cap N) = (U \cap V)/(U \cap V \cap u(U \cap v)$.
Now we show, $U \cap V \cap u(U \cap v) = (U \cap v) (V \cap u)$.  Let $a_1 b_1 \in u(U \cap v), a_1 \in u, b_1 \in (U \cap v)$.
If $a_1 b_1 \in V$, $a_1 \in V$ (since $b_1 \in v \subseteq V$), thus $a_1 \in (u \cap V)$ and $a_1 b_2 \in (u \cap V) (U \cap v)$.
We've shown, $u(U \cap V)/u(U \cap v) \cong (U \cap V)/(u \cap V)(U \cap v)$.  A symmetric aargument shows
$(U \cap V)v/(u \cap V)v \cong (U \cap V)/(u \cap V)(U \cap v)$.  So $u(U \cap V)/u(U \cap v) \cong (U \cap V)v/(u \cap V)v$,
which is what we wanted.
\end{quote}
{\bf Another proof of Schrier:} Let
$1 \lhd G_r \lhd \ldots \lhd G_1 = G$ and
$1 \lhd H_s \lhd \ldots \lhd H_1 = H$ be normal towers, they have equivalent refinements.
\begin{quote}
\emph{Proof:}
Put $G_{ij} = G_{i+1}(H_j \cap G_i)$ and $H_{ji} = H_{j+1}(G_i \cap H_j)$.  Use the butterfly to show
$G_{ij}/G_{i,j+1} \cong H_{ji}/H_{j, i+1}$.
\end{quote}
\section {Indecomposible Groups and Krull Schmidt}
{\bf Definition 7:}
$\phi$ is a \emph{normal endomorphism} iff
$\phi(a^{-1}xa)=
a^{-1} \phi(x)a$, $\forall x,a \in G$.
A group, $G$, is \emph{indecomposible}, if $G \ne 1$, and if $G = H \times K$ then either
$H = 1$ or $K = 1$.
\\
\\
{\bf Lemma 1:}
If $\phi$ and $\psi$ are normal endomorphisms of a group $G$ then so is their composition
and so is $\phi^{-1}$..
If $\phi$ is a normal endomorphism of $G$ and if $H \lhd G$ then $\phi(H) \lhd G$.
\begin{quote}
\emph{Proof:}  Routine.
\end{quote}
{\bf Lemma 2:}  Let $G = H_1 \times \ldots \times H_m$ have projections $\pi_i : G \rightarrow H_i$
and inclusions $\lambda_j : H_i \rightarrow G$ then the sum of any $k$ distinct $\lambda_i \pi_i$
is a normal endomorphism of $G$.
\begin{quote}
\emph{Proof:}  Routine.
\end{quote}
{\bf Lemma 3:}
If $H \lhd G$ and both $H$ and $G/H$ have both chain conditions then $G$ has both chain conditions.
If $G= H \times K$ and $G$ has both chain conditions then so do $H$ and $K$.
\begin{quote}
\emph{Proof:}
If
$G_1 \ge G_2 \ge \ldots$ is a chain of normal subgroups of $G$ then
$H \cap G_1 \ge H \cap G_2 \ge \ldots$
is a chain of normal subgroups of $H$ and
$HG_1 /H \ge HG_2 /H \ge \ldots$
is a chain of normal subgroups of $G/H$.
$\exists t,s: H \cap G_t= H \cap G_{t+1} = \ldots$
$HG_s /H= HG_{t+1} /H = \ldots$; let $l= max(t,s)$.  $G_l=G_{l+1} = \ldots$ so
$G$ has ACC.  A similar argument holds for DCC.\\
\\
If $G = H \times K$ then every normal subgroup of $H$ is a normal subgroup of $G$ and
every chain of normal subgroups  of $H$ is a chain of normal subgroups of $G$.
\end{quote}
{\bf Theorem 2:}
If $G$ satisfies ACC or DCC then $G$ is the direct product of indecomposable groups.
\begin{quote}
\emph{Proof:}
Call $G$ ``good'' if it satisfies the conclusion of the theorem.  If $G$ is indecomposible,
$G$ is good and if $A$ and $B$ are good so is $A \times B$.  So if $G= U \times V$ is bad
either $U$ is bad or $V$ is bad.  Supposed $G$ is bad.  Define $H_0=G$ and by induction,
$\exists H_1 , H_2 , \ldots , H_n$ with $H_i$ a bad proper factor of $H_{i+1}$.
so, $G=H_0 > H_1 > \ldots$.  If $G$ has DCC, this must terminate at a bad indecomposible group
which is a contradiction.
\end{quote}
{\bf Lemma 3:}
If $G$ satisfies ACC (resp. DCC) on normal subgroups
and $f$ is a normal endomorphism
of $G$, then $f$ is an injection iff $f$ is an surjection.
\begin{quote}
\emph{Proof:}
Suppose $\phi$ is an injection and $g \in \phi(G)$.   We prove by induction
that $\phi^n(g) \notin \phi^{n+1}(G)$.   If not, $\exists h \in  G: \phi^n(g)= \phi^{n+1}(h)$
so $\phi(\phi^{n-1}(g)) = \phi(\phi^{n}(h))$.  Since
$\phi$ is injective, $\phi^{n-1}(g)= \phi^n(h)$ which contradicts the inductive hypothesis.
Thus $\exists$ a chain $G > \phi(G) > \phi^2(G) > \ldots$.  $\phi$ is normal so
$\phi^n$ is normal and by a previous lemma $\phi^n(G) \lhd G, \forall n$.  This violates
the DCC condition.
\\
\\
Now assume $\phi$ is surjective.  Define $K_n= ker(\phi^n(G))$ with each $K_n \lhd G$.
$1 = K_0 \le K_1 \le \ldots $.  This chain stops because of ACC.  Let $t$ be the smallest integer
such that $K_t= K_{t+1} = \ldots$.  We claim $t=0$.  If $t \ge 1$ then $\exists x \in K_t$ with
$x \notin K_{t-1}$ so
$\phi^{t}(x) \ne 1$ but $\phi^{t+1}(x)=1$.
Since $\phi$ is a surjection, $\exists g \in G$ with $x= \phi(g)$.  Hence
$1 = \phi^{t}(x) = \phi^{t+1}(g)$ so $g \in K_{t+1}= K_t$ and thus
$a= \phi(g)= \phi^{t-1}(g) ( \phi(g))= \phi^{t-1}(g)$ which is a contradiction so $\phi$
is injective.
\end{quote}
{\bf Definition 8:} An endomorphism $\phi$ of $G$ is nilpotent if $\exists k>0: \phi^j=0$;
note $0: g \mapsto 1$.
\\
\\
{\bf Fitting Lemma:}
Let $G$ satisfy both chain conditions.  If $\phi$ is a
normal endomorphism of $G$ with $\phi(H)=H$ and $\phi(K)=K$ then
$G= H \times K$ and
$\phi_{|K}$ is nilpotent and $\phi_{|H}$ is surjective.
\begin{quote}
\emph{Proof:}
Let $K_n= ker(\phi^n(G))$ and $H_n= im(\phi^n(G))$.
$G \ge H_1 \ge \ldots $ and
$1 \le K_1 \ge \ldots $.  Suppose $H_i$ stops at $t$ and $K_i$ stops at $s$, $l= max(r,s)$.
Let $x \in H \cap K$.  Since $x \in H, \exists g \in G$ with $x= \phi^l(g)$.
$\phi^l(x)=1$ so $\phi^{2l}(g)=1$ and $g \in K_{2l}= K_l$.  $x \in \phi^l(g)=1$ and
so $H \cap K=1$.
If $g \in G$ then $\phi^l(g) \in H_l = H_{2l}$ so
$\exists y \in G: \phi^l(g)= \phi^{2l}(y)= 1$ and
$\phi^l(g \phi^l(y^{-1}))=1$ so
$g \phi^l(y^{-1}) \in K_{l}=K_{2l}$ so $g=(g \phi^l(y^{-1}))\phi^l(y) \in KH$ and $G= K \times H$.
Now, $\phi(H)=\phi(H_l)= \phi(\phi^l(G))= \phi^{l+1}(G)= H_{l+1}=H_j=H$ so
$\phi$ is surjective.  Finally, if $x \in K$ then $\phi^l(x) \in K \cap H=1$ and so
$\phi_{|K}$ is nilpotent.
\end{quote}
{\bf Theorem 3:}
If $G$ is an indecomposable group satisfying ACC and DCC on normal subgroups
and if $f$ is a normal endomorphism then $f$ is nilpotent or an automorphism.
\begin{quote}
\emph{Proof:}
By Fitting Lemma, $G= K \times H$ with $\phi_{|K}$ and $\phi_{|H}$ surjective.
Since $G$ is indecomposible, then either $G=H$ or $G=K$.  In the first case,
$\phi$ is nilpotent.  In the second case, $\phi$ is surjective and so by the previous Lemma,
$\phi$ is an automorphism.
\end{quote}
{\bf Lemma:} Let $G$ be an indecomposible group with both chain conditions and
suppose $\phi, \psi$ are two normal, nilpotent endomorphisms of $G$.  If $\phi+\psi$ is
an endomorphism of $G$ then it is nilpotent.
\begin{quote}
\emph{Proof:}
By the previous result,
$\phi+\psi$ is either an automorphism or nilpotent.
If it is an automorphism, there is an inverse, $\gamma$, normal.
For each $x \in G$, $x= (\phi+\psi) \gamma(x))= \phi(\gamma(x)) + (\psi(\gamma(x))$.
If $\lambda= \phi \gamma$  and $\mu= \psi \lambda$,
$1_G= \lambda+\mu$.  In particular,
$x= \lambda(x) \mu(x)$ and so $\lambda+\mu= \mu+\lambda \rightarrow \lambda \mu = \mu \lambda$.
$End(G)$ is generated by $\mu, \lambda$ and
$(\lambda + \mu)^m = \sum {m \choose i} \phi^i \psi^{m-i}$.  Since both
$\phi$ and $\psi$ are nilpotent so are $\lambda$ and $\mu$ and they are not automorphisms.
So, $\exists r, s: \lambda^r=\mu^s=0$.  If $m=r+s-1$ then either
$i \ge r$ or $m-i \ge s$ so $1_G^m =0$ and $G=1$.
\end{quote}
{\bf Corollary:}  Let $G$ be an indecomposible group having both chain conditions.
If $ \varphi_1, \ldots, \varphi_n $
is a set of normal, nilpotent endomorphisms of $G$ such that every sum of distinct
$\varphi$'s is an endomorphism, then
$ \varphi_1 + \ldots + \varphi_n $ is nilpotent.
\begin{quote}
\emph{Proof:}  By induction on $n$.
\end{quote}
{\bf Krull-Schmidt Theorem:}  If $G$ has both chain conditions on normal subgroups and
$G= H_1 \times \ldots \times H_s = K_1 \times \ldots \times K_t$ are
two decompositions into indecomposable factors then $s=t$ and, after
reindexing, $H_i \cong K_i$ and for each $r<t$, $G= G_1 \times G_2 \times \ldots
\times G_r \times H_{r+1} \times H_t$.
\begin{quote}
\emph{Proof:}
Let $P(0)$ be the statement
$G= G_1 \times G_2 \times \ldots \times G_s$ and for $1 \le r \le min(s,t)$
let $P(i)$ be the statement
$G= G_1 \times G_2 \times \ldots \times G_r \times H_{r+1} \times \ldots H_t$.
$P(0)$ is true by assumption, assume $P(r-1)$.  Let $\pi_i$ (resp $\pi_i'$ be the
canonical epimorphisms from
$G_1 \times G_2 \times \ldots \times G_s$ (resp.
$G_1 \times G_2 \times \ldots \times G_r \times H_{r+1} \times H_t$ and $\lambda_i$
(resp $\lambda_i'$) be the inclusion maps,
$\varphi_i= \lambda_i \pi_i$
and
$\phi_i= \lambda_i' \pi_i'$.  $\varphi_r \phi_i= 0_{|G}$ for $i<r$ and
$\varphi_1 (1_{|G})= \varphi_r \phi_1 + \ldots + \varphi_r \phi_t=
\varphi_r \phi_r + \ldots + \varphi_r \phi_t$ so $(\varphi_r \phi_j)_{|G}$ is
an automorphism of $G_r$.  $\varphi_j \phi_r$ must be an automorphism of $H_j$ and
$\phi_j:G_r \rightarrow H_j$ is and isomorphism and so is $\varphi_r: H_j \rightarrow G_r$
reindexing we have the first half of $P(r)$.  Let
$g=g_1 g_2 \ldots g_{r-1} h_r h_{r+1} \ldots h_t$ define
$\theta(g)=g_1 g_2 \ldots g_{r-1} \varphi(h_r) h_{r+1} \ldots h_t$.
$G=Im(\theta)=G^*= G_1 \times G_2 \times \ldots \times G_r \times H_{r+1} \times H_t$
which completes the argument.
\end{quote}
\section {Inner Automorphisms}
{\bf Definitions 9:} $G$ is \emph{complete} if it is centerless and every automorphism is inner
in which case $G \cong Aut(G)$.
\\
\\
{\bf Theorem 4:} $S_n$ is complete if $n \ne 2,3$.
\begin{quote}
\emph{Proof:}
Let $T_k$ be the set of $k$ disjoint transpostions so $x \in T_k \rightarrow
x^2=1$; note that if $\theta \in Aut(S_n), \theta(T_1)= T_k$ for some $k$. Also
observe that $\theta$ preserves transpositions iff $\theta \in Inn(S_n)$.
Now we can show
$|T_1|= {\frac {n(n-1)} {2}}$ and
$|T_k|= {\frac {(n-2k+1)!} {(n-2k)! k! 2^k}}$.  Comparing the two $|T_1| = |T_k|$
is possible only if $k=2 ,3$ and in fact, only if $k=3$.   If
$\theta \in Out(S_6)$ and $\tau$ is a transposition, $\theta(\tau)$ must
be a product of three transpositions and such an automorphism exists.
\end{quote}
{\bf Theorem 5:}
If $G$ is a non-abelian simple group, then $Aut(G)$ is complete.  If
$K \lhd G$ and $K$ is complete, $G= K \times Q$.  $Hol(K) \subset S_K$
is $ \langle K^l, Aut(K) \rangle $, $K^l \lhd Hol(K)$, $Hol(K)/K^l \cong Aut(K)$ and
$C_{Hol(K)}(K^l)= K^r$.  If $K$ is a direct factor whenever $K$ is a normal
subgroup then $K$ is complete.
\begin{quote}
\emph{Proof:}
See, Scott.
\end{quote}


\subsection{Applications to combinatorial sums}

\citeauthor{sprugnoli:1991}, in \cite{sprugnoli:1991}, shows how the
\emph{Riordan group} and its elements \emph{Riordan arrays} provide an elegant
methodology to solve combinatorial sums. This method is quite different from
ones proposed in \cite{egorychev:1990}, \cite{wilf:zeilberger:1990} and
\cite{wilf:1989}, since it offer a \emph{constructive} proof for a sum identity
$\sum_{k}f_{k}=v$: \marginpar{not only a $certification$ for $\sum_{k}f_{k}=v$}
in addition to be a \emph{certification} for the identity, it shows how to find
such value $v$.

The following result is another way to state
\autoref{thm:riordan:group:fundamental:theorem} and it allows us to show in the
proof an interesting trick about coefficient extractor $[t^{n}]$.

\begin{theorem}
    Let $\mathcal{M}=(d(t),h(t))$ be a Riordan matrix and $\vect{\omega}$
    be a \emph{given} sequence $\lbrace\omega_{i}\rbrace_{i\in\mathbb{N}}$. Choose
    a row index $n\in\mathbb{N}$, then:
    \begin{displaymath}
        \sum_{k=0}^{n}{m_{nk}\,\omega_{k}}=[t^{n}]d(t)\Omega(h(t))
    \end{displaymath}
    where function $\Omega$ is a \ac{gf} of sequence $\vect{\omega}$.
    \label{thm:sprugnoli:riordan:combinatorial:sums}
\end{theorem}

\begin{proof}
    By relation among Riordan array $\mathcal{M}$ and matrix
    $\lbrace m_{nk}\rbrace_{n,k\in\mathbb{N}}$:
    \begin{displaymath}
        \sum_{k=0}^{n}{m_{nk}\omega_{k}}
            =\sum_{k=0}^{n}{[t^{n}]m_{k}(t)\,\omega_{k}}
            =d(t)\sum_{k=0}^{n}{[t^{n}]h(t)^{k}\,\omega_{k}}
    \end{displaymath}
    since summing $n+1$ coefficients extracted from $n+1$
    \marginpar{a trick on $[t^{n}]$ within a sum: moving it from inside to outside}
    convolutions of function $h$ with itself, is the same as
    do $n+1$ convolutions of function $h$ with itself, sum them
    and, lately, do \emph{one} coefficient extraction. Therefore:
    \begin{displaymath}
        =[t^{n}]d(t)\sum_{k=0}^{n}{\omega_{k}\,h(t)^{k}}
        =[t^{n}]d(t)\Omega(h(t))
    \end{displaymath}
    as required.
\end{proof}

For the sake of clarity, we would like to apply previous result to
Pascal array $\mathcal{P}$ in order to find diagonal sums. Define
a new array $\hat{\mathcal{P}}$ as:
\begin{equation}
    \hat{\mathcal{P}}=\left(\frac{1}{1-t},\frac{t^{2}}{1-t}\right)
    \label{eq:pascal:array:for:diagonals}
\end{equation}
in order to have coefficients lying on diagonals in \autoref{eq:pascal:array:derived:for:example}
to lie on rows. The following is the matrix denoted by $\hat{\mathcal{P}}$:
\marginpar{an example: computing diagonal sums of array $\mathcal{P}$}
\begin{displaymath}
    \left[
    \begin{array}{rrrrr}
    1 &  &  &  &   \\
    1 &  &  &  &   \\
    1 & 1 &  &  &   \\
    1 & 2 &  &  &   \\
    1 & 3 & 1 &  &   \\
    1 & 4 & 3 &  &   \\
    1 & 5 & 6 & 1 &   \\
    1 & 6 & 10 & 4 &  \\
    1 & 7 & 15 & 10 & 1 \\
    1 & 8 & 21 & 20 & 5
    \end{array}
    \right]
\end{displaymath}
where $d_{nk}\in\mathcal{P} \rightarrow d_{n-k,k}\in\hat{\mathcal{P}}$,
therefore array $\hat{\mathcal{P}}$ is defined over matrix
$\left\lbrace{{n-k}\choose{k}}\right\rbrace_{n,k\in\mathbb{N}}$.
Consider sequence $\vect{1}=\lbrace1\rbrace_{n\in\mathbb{N}}$,
\autoref{thm:sprugnoli:riordan:combinatorial:sums} allows us to write the sum
of coefficients lying on row $n$ as:
\marginpar{sum of the $n$-th $\mathcal{P}$'s diagonal yield the $n$-th Fibonacci number}
\begin{displaymath}
    \sum_{k=0}^{n}{{{n-k}\choose{k}}}=[t^{n}]
        \frac{d_{\hat{\mathcal{P}}}(t)}{1-h_{\hat{\mathcal{P}}}(t)}
        =[t^{n}]\frac{1}{1-t-t^{2}}
        =F_{n}
\end{displaymath}
where $F_{n}$ is the $n$-th \emph{Fibonacci number}.
\\\\
The following result is very interesting because it allows us to solve
more combinatorial sum using both a Riordan array $\mathcal{M}$
and its inverse $\mathcal{M}^{-1}$.

\begin{theorem}
    Let $\mathcal{M}=(d(t),h(t))$ be a Riordan array and $\vect{\omega}$
    be a sequence $\lbrace\omega_{i}\rbrace_{i\in\mathbb{N}}$ with an \emph{unknown}
    \ac{gf} $\Omega$. Choose a row index $n\in\mathbb{N}$, if $\vect{\theta}$ is a
    sequence $\lbrace\theta_{i}\rbrace_{i\in\mathbb{N}}$ with a \emph{given} \ac{gf} $\Theta$,
    then:
    \begin{displaymath}
        \sum_{k=0}^{n}{\tilde{m}_{nk}\,\theta_{k}}=\omega_{n}
        \quad\rightarrow\quad
        \sum_{k=0}^{n}{m_{nk}\,\omega_{k}}=\theta_{n}
    \end{displaymath}
    where $\tilde{m}_{nk}\in\mathcal{M}^{-1}$. Moreover,
    $\Omega(t)=\frac{1}{d(\hat{h}(t))}\,\Theta(\hat{h}(t))$,
    where function $\hat{h}$ is the compositional inverse of
    function $h$.
    \label{thm:sprugnoli:sequences:inversion:by:combinatorial:sums}
\end{theorem}

\begin{proof}
    By \autoref{thm:sprugnoli:riordan:combinatorial:sums} we can rewrite:
    \begin{displaymath}
        \sum_{k=0}^{n}{\tilde{m}_{nk}\,\theta_{k}}
            = [t^{n}]\frac{1}{d(\hat{h}(t))}\,\Theta(\hat{h}(t))
            = [t^{n}]\Omega(t) = \omega_{n}
    \end{displaymath}
    which sets unknown function $\Omega$, namely the \ac{gf} for
    sequence $\vect{\omega}$, such that $\Omega(t)=\frac{1}{d(\hat{h}(t))}\,\Theta(\hat{h}(t))$.
    Then:
    \begin{displaymath}
        \sum_{k=0}^{n}{m_{nk}\,\omega_{k}}
            = [u^{n}]d(u)\Omega(h(u))
            = [u^{n}]\Theta(u)
            = \theta_{n}
    \end{displaymath}
    as required.
\end{proof}

Why is \autoref{thm:sprugnoli:sequences:inversion:by:combinatorial:sums} important?
\marginpar{another way to find \ac{gf} for a row in a Riordan array}
Assume that you have a Riordan array $\mathcal{M}=(d(t),h(t))$ over a matrix of coefficients
$\lbrace m_{nk}\rbrace_{n,k\in\mathbb{N}}$. You are interested to find the
\ac{gf} $\Omega$ of a row $r\in\mathbb{N}$ over the sequence:
\begin{displaymath}
    \vect{\omega}
        =(w_{0},w_{1},w_{2},\ldots,w_{r},0,\ldots)
        =(m_{r0},m_{r1},m_{r2},\ldots,m_{rr},0,\ldots)
\end{displaymath}
when you know the \ac{gf} $\Theta$ over a sequence $\vect{\theta}$ such that:
\begin{displaymath}
    \sum_{k=0}^{n}{m_{nk}\,\omega_{k}}
        =\sum_{k=0}^{n}{m_{nk}\,m_{rk}}
        =\theta_{n}
\end{displaymath}
applying \autoref{thm:sprugnoli:sequences:inversion:by:combinatorial:sums},
it is necessary to find $\mathcal{M}^{-1}$ and compute function $\Omega$ as:
\begin{displaymath}
    \Omega(t)=\frac{1}{d(\hat{h}(t))}\,\Theta(\hat{h}(t))
\end{displaymath}
such function $\Omega$ is the desired one, namely the \ac{gf} over
sequence $\vect{\omega}$, itself over coefficients lying on row $r$, as required.


\documentclass[12pt]{article}

\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{float}
\usepackage{ulem}
\usepackage{bm}

\usepackage{anysize}

\marginsize{2cm}{2cm}{0.9cm}{1.8cm}
\usepackage[framed,numbered,autolinebreaks,useliterate]{mcode}
\usepackage{listings}
\lstset{language=Matlab}
\lstset{breaklines}
\lstset{extendedchars=false}

\title{Machine Learning/Pattern Recognition\\  \begin{Large} Homework \#3 \end{Large} }
\author{Jiyu Tian}
\date{}

\begin{document}

\maketitle
%%---------------------------------------------------------------
%% Problem 1
%%---------------------------------------------------------------
\section{Problem 3.1}
\large{\textbf{Solution:}}\\
(a) The maximum likelihood estimates for $\bm{\mu}$ and $\bm{\Sigma}$ are given by:
\begin{equation*}
\hat{\bm{\mu}} = \frac{1}{N} \sum^N_{i=1} \bm{x}_i
\end{equation*}
\begin{equation*}
\hat{\bm{\Sigma}} = \frac{1}{N} \sum^N_{i=1} (\bm{x}_i - \hat{\bm{\mu}}) (\bm{x}_i - \hat{\bm{\mu}})'
\end{equation*}
With MATLAB the maximum likelihood estimates for mean and covariances are:
\begin{equation*}
\begin{aligned}
\centering {\begin{matrix}
\hat{\bm{\mu_1}} = \begin{bmatrix}
 0.7969\\
 0.4314\\
 -1.4372
\end{bmatrix} &  \hat{\bm{\mu_2}} = \begin{bmatrix}
 1.5947\\
 4.1320\\
 -3.6648
\end{bmatrix} & \hat{\bm{\mu_3}} = \begin{bmatrix}
 0.2426\\
 -0.0509\\
 -0.0850
\end{bmatrix}
\end{matrix}}
\end{aligned}
\end{equation*}

\begin{equation*}
\begin{aligned}
\centering {\begin{matrix}
\hat{\bm{\Sigma}}_1 = \begin{bmatrix}
    2.8273  &  0.0296 &  -0.0082 \\
    0.0296  &  4.9341  & -0.0832 \\
   -0.0082  & -0.0832   & 1.9925
\end{bmatrix}
\end{matrix}}
\end{aligned}
\end{equation*}

\begin{equation*}
\begin{aligned}
\centering {\begin{matrix}
\hat{\bm{\Sigma}}_2  = \begin{bmatrix}
    1.0344  &  0.0419  &  0.0434 \\
    0.0419  &  3.8840  &  1.0384 \\
    0.0434  &  1.0384  &  5.9750
\end{bmatrix}
\end{matrix}}
\end{aligned}
\end{equation*}

\begin{equation*}
\begin{aligned}
\centering {\begin{matrix}
\hat{\bm{\Sigma}}_2  = \begin{bmatrix}
    9.7968  & -0.0261  &  0.1294\\
   -0.0261  &  9.1673  &  0.0610\\
    0.1294  &  0.0610  & 10.2171
\end{bmatrix}
\end{matrix}}
\end{aligned}
\end{equation*}

\vfill
\clearpage

\noindent(b) (c) As shown in Figure 1, the minimum error rate achieves at $\alpha=0.12$. The classifier goes from under-fitting to over-fitting as $\alpha$ goes from 1 to 0. The classifier does not perform well as $\alpha$ approaches 1, i.e, the individual covariances ``shrink'' toward a common one, since the model is getting further from the truth. \\

\begin{figure}[H]
\centering
\includegraphics[width = 1\textwidth]{1bc.png}
\caption{Training\ and\ testing\ error\ versus $\alpha$ from\ 0\ to\ 1}
\end{figure}

\noindent(d) With MATLAB the maximum likelihood estimates for mean and covariances are:
\begin{equation*}
\begin{aligned}
\centering {\begin{matrix}
\hat{\bm{\mu_1}} = \begin{bmatrix}
  -0.0110 \\
  0.0054 \\
  0.0144 \\
\end{bmatrix} &  \hat{\bm{\mu_2}} = \begin{bmatrix}
 0.9931 \\
 4.9504 \\
 -2.9815
\end{bmatrix} & \hat{\bm{\mu_3}} = \begin{bmatrix}
 0.0254 \\
 0.0571 \\
 -0.0778
\end{bmatrix}
\end{matrix}}
\end{aligned}
\end{equation*}

\begin{equation*}
\begin{aligned}
\centering {\begin{matrix}
\hat{\bm{\Sigma}}_1 = \begin{bmatrix}
    3.1963  & -0.7264  &  0.4860 \\
   -0.7264  &  2.7638  &  2.7435 \\
    0.4860  &  2.7435  &  3.7038
\end{bmatrix}
\end{matrix}}
\end{aligned}
\end{equation*}

\begin{equation*}
\begin{aligned}
\centering {\begin{matrix}
\hat{\bm{\Sigma}}_2  = \begin{bmatrix}
    1.0692  & -0.5619  &  0.1870  \\
   -0.5619  &  3.1287  & -0.5232  \\
    0.1870  & -0.5232  &  0.9506
\end{bmatrix}
\end{matrix}}
\end{aligned}
\end{equation*}

\begin{equation*}
\begin{aligned}
\centering {\begin{matrix}
\hat{\bm{\Sigma}}_2  = \begin{bmatrix}
    9.3002  & -5.2652  &  5.2015 \\
   -5.2652  & 10.2951  & -5.0706 \\
    5.2015  & -5.0706  & 12.1647
\end{bmatrix}
\end{matrix}}
\end{aligned}
\end{equation*}

\vfill
\clearpage
\noindent\\(e) (f) In this case, the training dataset is large enough for the model to learn the correct parameters. As we can see in Figure 2, the training error tracks the testing error. $\alpha \approx 0$ yields the best performance since the error rates reach minimum at $\alpha \approx 0$, where the estimated variance matrices are closest to the true ones. With shrinkage we could choose from numerous parameters for the variance matrix with best performance.\\
\begin{figure}[H]
\centering
\includegraphics[width = 1\textwidth]{1ef.png}
\caption{Training\ and\ testing\ error\ versus $\alpha$ from\ 0\ to\ 1}
\end{figure}
\text{ }\\
%%---------------------------------------------------------------
%% Problem 2
%%---------------------------------------------------------------
\section{Problem 3.2}
\large{\textbf{Solution:}}\\
(a) As described in the text, the basic version of the K-means algorithm is:\\
\\
\textbf{K-means clustering Algorithm}\\
1 Begin initialize $n,\ c,\ \bm{\mu}_1,\ \bm{\mu}_2,...,\ \bm{\mu}_c$\\
2 Do classify $n$ samples according to nearest $\bm{\mu}_i$\\
3 Recompute $\bm{\mu}_i$\\
4 Until no change in $\bm{\mu}_i$\\
5 Return $\bm{\mu}_1,\ \bm{\mu}_2,...,\ \bm{\mu}_c$\\
6 End\\
\vfill
\clearpage

(b) See Figure 3.
\begin{figure}[H]
\centering
\includegraphics[width = 1\textwidth]{ds1.png}
\caption{Result of clustering with k-means on dataset1.mat}
\end{figure}

\noindent(c) See Figure 4. Since k-means clustering is based on the assumption that all clusters have equal identity covariance matrices, it cannot classify the mixture ellipse dataset to their original shapes.
\\

\begin{figure}[H]
\centering
\includegraphics[width = 1\textwidth]{ds2.png}
\caption{Result of clustering with k-means on dataset2.mat}
\end{figure}

\vfill
\clearpage
\noindent(d) See Figure 5.
\begin{figure}[H]
\centering
\includegraphics[width = 1\textwidth]{ds3.png}
\caption{Result of clustering with k-means on dataset3.mat}
\end{figure}

\noindent(e) See Figure 6.

\begin{figure}[H]
\centering
\includegraphics[width = 1\textwidth]{ds4.png}
\caption{Result of clustering with k-means on dataset4.mat}
\end{figure}

\vfill
\clearpage
(f) Knee occurs around $K = 4$

\begin{figure}[H]
\centering
\includegraphics[width = 1\textwidth]{ds5.png}
\caption{MSE of dataset5.mat with k-means}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width = 1\textwidth]{ds52.png}
\caption{Result of clustering with k-means on dataset5.mat}
\end{figure}

\end{document}