\lstset{
    language=Matlab,%
    basicstyle=\fontsize{9}{11},
    breaklines=true,%
    morekeywords={matlab2tikz},
    keywordstyle=\color{blue},%
    morekeywords=[2]{1}, keywordstyle=[2]{\color{black}},
    identifierstyle=\color{black},%
    stringstyle=\color{mylilas},
    commentstyle=\color{mygreen},%
    showstringspaces=false,%without this there will be a symbol in the places where there is a space
    numbers=left,%
    numberstyle={\tiny \color{black}},% size of the numbers
    numbersep=0pt, % this defines how far the numbers are from the text
    linewidth=\columnwidth,
    emph=[1]{for,end,break},emphstyle=[1]\color{blue}, %some words to emphasize
}

\begin{appendices}
    \section{Matlab code}
    \subsection{Weights' Gradients}
    \label{sub:gradients}
    \begin{lstlisting}[language=Matlab]
        function [g1, g2] = gd(example, tau, w1, w2)
            %GD Compute the gradients of the error function for the given example
            %with respect to w1 and w1 (weights of the 2 hidden units).

            tanh_w1 = tanh(example * w1);
            tanh_w2 = tanh(example * w2);
            sigma = tanh_w1 + tanh_w2;
            g_comp = (sigma - tau) * example';
            g1 = g_comp * (1 - (tanh_w1 ^ 2));
            g2 = g_comp * (1 - (tanh_w2 ^ 2));
        end
    \end{lstlisting}
\end{appendices}


\chapter{Array}
\section{Two-pointer Algorithm}
\runinhead{Container With Most Water.} Given coordinate $(i, a_i)$, find two lines, which together with x-axis forms a container, such that the container contains the most water.
\begin{figure}[hbtp]
\centering
\subfloat{\includegraphics[scale=.50]{Container-With-Most-Water.jpg}}
\caption{Container with Most Water}
\label{fig:Container-With-Most-Water}
\end{figure}
Core clues:
\begin{enumerate}
\item \textbf{Two pointers}: $start$, $back$ at two ends. Calculate the current area
\item \textbf{Move one}: Move the shorter (lower height) pointer.
\end{enumerate}

\section{Circular Array}
This section describes common patterns for solving problems with circular arrays.

Normally, we should solve the linear problem and circular problem very differently.

\subsection{Circular max sum}
Linear problem can be solved linear with dp algorithm for maximum subarray sum - Section \ref{dpSequence}.

The circular sum should use dp.

Problem description: Given an integer array, find a continuous rotate subarray where the sum of numbers is the biggest. Return the index of the first number and the index of the last number.
\runinhead{Core clues:}
\begin{enumerate}
\item \textbf{State definitions}:

Construct left max sum $L_i$ for max sum over the $[0..i]$ with subarray starting at 0 (\textit{forward} starting from the left side).

Construct right max sum $R_i$ for max sum over the indexes $[i+1..n -1]$, with subarray ending at -1 (\textit{backward} starting from the right side).

Notice, for the two max sums, the index ends AT or BEFORE $i$.

\item \textbf{Transition functions:}
\begin{align*}
L_i = \max\Big(L_{i-1}, sum(A[:i])\Big) \\
R_i = \max\Big(R_{i+1}, sum(A[i:])\Big)
\end{align*}

\item \textbf{Global result}:
$$maxa = \max(R_i+L_{i-1}, \forall i)$$
\end{enumerate}

\subsection{Non-adjacent cell}
Maximum sum of non-adjacent cells in an array $A$. (House robbery problem)

To solve circular non-adjacent array problem in linear way, we should consider 2 cases:
\begin{enumerate}
\item Not consider the $A[1]$
\item Not consider the $A[-1]$
\end{enumerate}
and solve them using linear maximum sum of non-adjacent cells separately  - Section \ref{dpSequence}.
\subsection{Binary search}
Searching for an element in a circular sorted array. Half of the array is sorted while the other half is not.
\begin{enumerate}
\item If $A[0] < A[mid]$, then all values in the first half of the array are sorted.
\item If $A[mid] < A[-1]$, then all values in the second half of the array are sorted.
\item Then \textit{derive and decide} whether to got the \textbf{sorted half} or the \textbf{unsorted half}.
\end{enumerate}
\section{Voting Algorithm}
\subsection{Majority Number}
\subsubsection{$\frac{1}{2}$ of the Size}
Given an array of integers, the majority number is the number that occurs more than half of the size of the array.

Algorithm: Majority Vote Algorithm. Maintain a counter to count how many times the majority number appear more than any other elements before index $i$ and after re-initialization. Re-initialization happens when the counter drops to 0.

Proof: Find majority number $x$ in $A$. Mathematically, find $x$ in array $A$ with length $n$ s.t. $cnt_x > n -cnt_x$.

Find a \textbf{pair} $(a_i, a_j)$ in $A$, if $a_i \neq a_j$, delete both from $A$. The counter  still
holds that: $C^{A'}_x > |A'|-C^{A'}_x$. Proof, since $a_i\neq a_j$, at most 1 of them
equals $x$, then $C^{A'}_x$ decrements at most by 1, $|A'|$ decrements by 2.

To find such pair $(a_i, a_j), a_i\neq a_j$, linear time one-pass algorithm. That's
why \textit{Moore's voting algorithm} is correct.

At any time in the execution, let $A'$ be the prefix of $A$ that has been processed,
if $counter>0$, then keep track the candidate $x$'s counter, the $x$
is the majority number of $A'$.  If $counter =0$, then for $A'$ we can pair the elements
s.t. are all pairs has distinct element. Thus, it does not hold that $cnt_x>n-cnt_x$;
thus $x\in A'$. $\blacksquare$

Re-check: This algorithm needs to re-check the current number being counted is indeed the majority number.

\begin{python}
def majorityElement(self, nums):
    """
    Algorithm:
    O(n lgn) sort and take the middle one
    O(n) Moore's Voting Algorithm
    """
    mjr = nums[0]
    cnt = 0
    for i, v in enumerate(nums):
        if mjr == v:
            cnt += 1
        else:
            cnt -= 1

        if cnt < 0:
            mjr = v
            cnt = 1

    return mjr

\end{python}
\subsubsection{$\frac{1}{3}$ of the Size}
Given an array of integers, the majority number is the number that occurs more than $\frac{1}{3}$ of the size of the array. This question can be generalized to be solved by $\frac{1}{k}$ case.

\subsubsection{$\frac{1}{k}$ of the Size}
Given an array of integers and a number k, the majority number is the number that occurs more than $\frac{1}{k}$ of the size of the array. In this case, we need to generalize the solution to $\frac{1}{2}$ majority number problem.
\newpag
\begin{python}

def majorityNumber(self, nums, k):
    """
    Since majority elements appears more
    than ceil(n/k) times, there are at
    most k-1 majority number
    """
    cnt = defaultdict(int)
    for num in nums:
        if num in cnt:
            cnt[num] += 1
        else:
            if len(cnt) < k-1:
                cnt[num] += 1
            else:
                for key in cnt.keys():
                    cnt[key] -= 1
                    if cnt[key] == 0: del cnt[key]

    # filter, double-check
    for key in cnt.keys():
        if (len(filter(lambda x: x == key, nums))
            > len(nums)/k):
            return key

    raise Exception
\end{python}

\section{Two Pointers}
\subsection{Interleaving}
\runinhead{Interleaving positive and negative numbers.} Given an array with positive and negative integers. Re-range it to interleaving with positive and negative integers.
\begin{lstlisting}
Input:
[-33, -19, 30, 26, 21, -9]
Output:
[-33, 30, -19, 26, -9, 21]
\end{lstlisting}
Core clues:
\begin{enumerate}
\item In 1-pass.
\item What (positive or negative) is expected for the current position.
\item Where is the next positive and negative element.
\end{enumerate}
\begin{python}
def rerange(self, A):
    n = len(A)
    pos_cnt = len(filter(lambda x: x > 0, A))
    pos_expt = True if pos_cnt*2 > n else False

    neg = 0  # next negative
    pos = 0  # next positive
    for i in xrange(n):
        # search for the next
        while neg < n and A[neg] > 0: neg += 1
        while pos < n and A[pos] < 0: pos += 1

        if pos_expt:
            A[i], A[pos] = A[pos], A[i]
        else:
            A[i], A[neg] = A[neg], A[i]

        if i == neg: neg += 1
        if i == pos: pos += 1

        pos_expt = not pos_expt
\end{python}

\section{Index Remapping}
\subsection{Introduction}
\runinhead{Virtual Index.} Analogy to physical machine and virtual machine, the underlying indexing $i$ for array $A$ is the physical index. We can create virtual indexing $i'$ for the same array $A$ to map $A_{i'}$ to the physical entry $A_{i}$.
\subsection{Example}
\runinhead{Interleaving indexes} Given an array $A$ of length $n$, we want to mapping the virtual indexes to physical indexes such that $A_0$ maps to $A_1$, $A_1$ maps to $A_3$,..., $A_{\lfloor n/2\rfloor}$ maps to $A_0$, as followed:
\begin{figure}[hbtp]
\centering
\subfloat{\includegraphics[scale=.70]{virtual_indexes.png}}
\caption{Virtual Indexes. Remapping}
\label{fig:virtual_indexes}
\end{figure}
\begin{lstlisting}
0 -> 1
1 -> 3
2 -> 5
...
n/2-1 -> n-1 or n-2

n/2 -> 0
n/2+1 -> 2
...
n -> n-2 or n-1
\end{lstlisting}
If $n$ is even,
$$
(2*i+1)\%(n+1)
$$
If $n$ is odd,
$$
(2*i+1)\%(n)
$$
Thus, by combining two cases, we create the mapping relationship:
\begin{python}
def idx(i):
    return (2*i+1) % (n|1)
\end{python}


\documentclass[../PHYS306Notes.tex]{subfiles}

\begin{document}
\subsection{Worksheet - Review of Noninertial Frames}

\begin{center}
    \includegraphics[scale=0.5]{Lecture-14/w14-img1.png}
\end{center}

\begin{p}
A pendulum is inside of a railcar that is accelerating in the x-direction with acceleration $A$  Find the equilibrium value of the angle $\phi$.
\end{p}
\begin{s}
Apply Newton's law in the non-inertial frame:
\[m\ddot{\v{r}} = \v{F} - m\v{A}\]
We have that the forces in the inertial frame are given by $\v{F} = m\v{g} + \v{T}$, (gravity and tension) so:
\[m\ddot{\v{r}} = m\v{g} + \v{T} - m\v{A}\]
Grouping terms:
\[m\ddot{\v{r}} = \v{T} + m(\v{g} - \v{A})\]
Where we may call $\v{g} - \v{A}$ the effective acceleration, $\v{g}_{eff}$. By trigonometry, the equilibrium angle would be given by:
\[\tan\phi_{eq} = \frac{A}{g}\] As can be seen from the diagram below:
\begin{center}
    \includegraphics[scale=0.8]{Lecture-14/w14-img2.png}
\end{center}
\end{s}

\begin{center}
    \includegraphics[scale=0.75]{Lecture-14/w14-img3.png}
\end{center}
\begin{center}
    \includegraphics[scale=0.75]{Lecture-14/w14-img4.png}
\end{center}
\begin{p}
The earth and moon orbit each other, while both earth and moon frames of reference are accelerating. Find the acceleration in each frame of reference.
\end{p}
\begin{s}
In the inertial frame, the forces on the test mass on the surface of the Earth is given by:
\[\v{F} = m\v{g} - gM_{m}m\frac{\hat{\v{d}}}{d^2}\]
Where $M_m$ is the mass of the moon. The Earth is not an inertial system, so it experiences some acceleartion. The acceleration of the center of the mass of the earth is given by:
\[\v{A} = -GM_{m}\frac{\hat{\v{d}}_0}{d^2_0}\]
Hence we have:
\[m\ddot{\v{r}} = m\v{g} - GM_mm\left(\frac{\hat{\v{d}}}{d^2} - \frac{\hat{\v{d}}_0}{d^2_0}\right) = m\v{g} - \v{F}_{tidal}\]
The tidal force is the vector difference between if the mass is at the surface of earth vs. at the center of mass of the earth. This results in tidal effects at either side of the earth, with the same magnitude and in the opposite direction ($\hat{\v{d}}$ and $\hat{\v{d}}_0$ are parallel at these two points, so the effect is maximal). We end up with a bulge on both sides of the Earth. At the top and bottom, we have that the x components cancel by symmetry, so we only have the y component (weaker effect, inwards pointing). This is shown in the diagram below:
\begin{center}
    \includegraphics[scale=0.5]{Lecture-14/w14-img5.png}
\end{center}
\end{s}

\begin{p}
Find the equation of motion for a particle on the surface of the ocean of the earth, in the earth’s frame of reference.
\end{p}
\begin{s}

\end{s}
\end{document}


%
% 265
%
\chapter{The Zeta Function of Riemann}

\Section{13}{1}{Definition of the Zeta-function.}

Let s = a + it where a and t are real*; then, if S > 0, the series

= 1 n=i n is a uniformly convergent series of analytic functions
(\hardsubsectionref{2}{3}{3}, \hardsubsectionref{3}{3}{4}) %TODO:cite multiple
in any domain in which <r 1 + 8; and consequently the
series is an analytic function of s in such a domam. The function is
called the Zeta-function; although it was known to Eulerf, its most
remarkable properties were not discovered before RiemannJ who
discussed it in his memoir on prime numbers; it has since proved to
be of fundamental importance, not onl - in the Theory of Prime
Numbers, but also in the higher theory of the Gamma-function and
allied functions.

1311. The generalised Zeta-function .

Many of the properties possessed by the Zeta-function are particular
cases of properties possessed by a more general function defined, when
cr' 1 + 8, by the equation

where a is a constant. For simplicity, we shall suppose || that < \$
1, and then we take arg(a + n) = 0. It is evident that (s, 1) = (s).

1312. The expression of s, a) as an infinite integral.

Since (a-F ??)-' T (s) = / af- e' '' ''''' do-., when arg.r = and o- >
(and a fortiori when a \ - S), we have, when o- 1 + 8, V s) s,a)= \ \
m :L | .-r*-' e" < + '* c a;

.v xjio 1-6- .'o l-e- "

* The letters o", t will be used in this sense throughout the chapter.

t Commentatioiies Acad. ScL Imp. Petropolitunae, ix. (1737), pp.
160-188.

+ Berliner MonaUherichte, 1659, pp. 671-680. Ges. Werke (1876), pp.
136-141.

§ The definition of this function appears to be due to Hurwitz,
Zeitschrift fiir Math, ttnd Phys. xxvii. (1882), pp. 86-101.

II When a has this range of values, the properties of the function
are, in general, much simpler than the corresponding properties for
other values of a. The results of \hardsubsectionref{13}{1}{4} are true for all values of a
(negative integer values excepted); and the results of \hardsubsectionref{13}{1}{2},
\hardsubsectionref{13}{1}{3}, \hardsectionref{13}{2} are true when R (a) > 0.

%
% 266
%

Now, when ic' i), e l + x, and so the modulus of the second of these
integrals does not exceed

I "" A-'-2e-(- + ) rf = (N f aV- r (o- - 1),

-'0

which (when o- 1 + 8) tends to as JV x . Hence, when a l + 8 and arg x
= 0,

this formula corresponds in some respects to Euler's integral for the
Gamma- function.

\Subsection{13}{1}{3}{The expression* of (s, a) as a contour integral.}
When a 1 + 8,
consider

(0+) / y-i g-a2

' 1 - e

the contour of integration being of Hankel's t3rpe \hardsubsectionref{12}{2}{2}) and not
containing the points + 2n7ri(n = l, 2, 3,,..) which are poles of the
integrand; it is supposed (as in \hardsubsectionref{12}{2}{2}) that, arg(- z)\ ir.

It is legitimate to modify the contour, precisely as in \hardsubsectionref{12}{2}{2}, whenf
o- 1 + B; and we get

'(0+) - A -ip-az r° 8-ip-az

Therefore

27n . ' 1 - e

Now this last integral is a one-valued analytic function of s for all
values of s. Hence the only possible singularities of s, a) are at the
singularities of r (1 - s), i.e. at the points 1, 2, 3, ..., and, with
the exception of these points, the integral affords a representation
of s, a) valid over the whole plane. The result obtained corresponds
to Hankel's integral for the Gamma- function. Also, we have seen that
s, a) is analytic when o- 1 -H 8, and so the only singularity of s, a)
is at the point .s = 1. Writing 6- = 1 in the

integral, we get

1 r(o+) e-,

ZTTl J 1 - e ' which is the residue at £; = of the integrand, and this
residue is 1.

Hence lim f \ = -l.

* il (1 -s)

* Given by Riemann for the ordinary Zeta-function.

+ If (7 1, the integral taken along any straight line up to the origin
does not converge.

%
% 267
%

Since T (1 - s) has a single pole at s = 1 with residue - 1, it
follows that the only singularity of s, a) is a simple pole with
residue + 1 at 5 = 1.

Example 1. Shew that, when R (s) > 0,

(1 \ 21 -*) /-(s) = i: \ 1 + 1 \ i . 1 2* 3' 4s ~

1 / * x ~

Exuraple 2. -Shew that, when R s)> 1,

Example 3. Shew that

where the contour does not include any of the points +rr ±877 ±57ri',
....

1314. Values of s, a) for special values of s.

In the special case when s is an integer (positive or negative),

is a one- valued function of z. We may consequently apply Cauchy's
theorem, so that - . -r: dz is the residue of the intesfrand at = 0,
that

is to say, it is the coefficient of z'" in V .

1 - e~

To obtain this coefficient we differentiate the expansion \hardsectionref{7}{2})

. e-" - 1 \ \ I (-)n< (a)2 e- -l n=\ n'.

term-by-term with regard to a, where (f)n iO denotes the Bernoullian
poly- nomial.

(This is obviously legitimate, by \hardsectionref{4}{7}, when | s j < tt, since - \,
can be expanded into a power series in z imiformly convergent with
respect to a.)

Then i ll-r Ml".

Therefore f s is zero or a negative integer (= - m), we have

  - m, a) = - </)', +o(a)/[(m -f- 1) ( i -f- 2)|. In the special case
when a = 1, if s = - m, then (5) is the coefficient

/ y, yji I Z

of z-~ in the expansion of - ' ' .

%
% 268
%

Hence, by \hardsectionref{7}{2},

 (-2m) = 0, l-2m) = (-y-BJ 2m) (m = 1, 2, S, ...),

r(0)=-|.

These equations give the value of (s) ivhen s is a negative integer or
zero.

\Subsection{13}{1}{5}{TJie forniula* of Hurwitz for s, a) when cr< 0.}

Consider - - -; - r-; - dz taken round a contour C consisting of

27n J c I - e~ °

a (large) circle of radius (2iV+l)7r, (iV an integer), starting at the
point

(2iV"+ l)7r and encircling the origin in the positive direction, arg
(- z) being

zero at z = -(2N+l) ir.

In the region between C and the contour 2NTr +7r; +), of which the
contour of § 13"lo is the limiting form, (- zy~' e~' (1 - e~ )~ is
analytic and one-valued except at the simple poles + 2Tri, + 47, ...,
± 2N7ri.

Hence

27riJc l-e~' 27ri j n+i) n l-e'- n=i

where i2, Rn are the residues of the integrand at 2n7ri, - 2 7
respectively. At the point at which - z = 2mre~ ', the residue is

(2n7r)'-le- '''('-l)e-2 "''',

and hence Rn + Rn = (27?7r) ~ 2 sin i ' " 2'n-an j .

Hence

1 /(0+) (\ )s-ig-a

27riJ(2iv'+i) l-e-

\ 2 sin STT cos 2'rran) 2 cos sir sin (27ra? ) " "(27rr,=i /? - " ' \
2 ) = n':i n'-'

+ H-  T -, dz.

2in J c I - e

Now, since < a 1, it is easy to see that we can find a number K
independent of N' such that | e~" (1 - e~ )~ \ < K when z is on C.

Hence

1 r ( -z\ \ ~' e~"' I 1 f""

-  1 -- . dz\ < K\, (2.Y + 1) -rrYe'"' I rf

27r

<ir (2iV-|-l)7r|' e-l*l as i\" X if cr < 0.

* Zeitschrift fill- Math, und Phys. xsvii. (1882), p. 95,

%
% 269
%

Making N y:, we obtain the result of Hurwitz that, if o- < 0,

 u. r(l-,v) ( /I cos(27ra ) /i \ 4 sin(27ra/i))

each of these series being convergent.

13151. Riemairii's relation between (s) and (l - s).

If we write a = 1 in the formula of Hurwitz given in \hardsubsectionref{13}{1}{5}, and
employ \hardsubsectionref{12}{1}{4}, we get the remarkable result, due bo Riemann, that

2 -* r (S) (S) cos (I STTJ = (1 \ s).

Since both sides of this equation are analytic functions of s, save
for isolated values of s at which they have poles, this equation,
proved when a < 0, persists (by \hardsectionref{5}{5}) for all values of s save those
isolated values.

Example 1. If m be a positive integer, shew that

C (2>n) = S- "*- 1 7r2"' BJ 2m) ! .

Example 2. Shew that r is)n~ C (s) is unaltered by replacing s by 1 -
s.

\addexamplecitation{Riemann.}

Example 3. Deduce from Riemann's relation that the zeros of f (5) at -
2, - 4, - 6, ... are zeros of the tirst order.

\Section{13}{2}{Hennites* fornuda for TODO (s, a).}

Let us apply Plana's theorem (example 7, p. 145) to the function (p
(z) =(a + z)~\ where arg ii + z) has its principal value.

Define the function q x, y) by the equation

(/ < '' y) = 9- - K + + wT' - ( + A' - iy)~']

= - Ha + A')- + if] ~ * sin \ s arc tan - - .

  X + a)

Since + arc tan - - does not exceed the smaller of tt and ', we X + a
' x+ a

have

\ q x,ij)\ \ (a + x)'+f-] *< 1 y-'; sinh JItt, 5 j j,

' q (X, y) : [ a + xf + f]- -''\ jsinh | | | . Using the first result
when y > a and the second when y < a it is

* Annali di Matemalica, (3),fv. (1901), pp. 57-72.

t If t>0, arc tan = :,<. I 5; and arc tan < / dt.

J 1-rt- J 1-i-t- Jo

%
% 270
%

evident that, if o- > 0, q (.v, y) (e-" - 1) dy is convergent when x
and

Jo

tends to as ic - X; also (a + x)~ dx converges if a > 1.

.0

Hence, if o- > 1, it is legitimate to make 'o - oo in the result
contained in the example cited; and we have

 (s,a) = la- +\ \ a+ocy dx + 'lj (a2 + 2)-i jsin fs arc tan 0j- J .

So

  s, a) = la-s + f +2j\ a + f)-y jsin (. arc tan )| - .

This is Hermite's formula*; using the results that, if y 0,

arc tan y/a y/a f y < dT j, arc tan y/a < 2 tt (y>: a7r],

we see that the integral involved in the formula converges for all
values of s. Further, the integral defines an analytic function of s
for all values of s.

To prove this, it is sufficient \hardsubsectionref{5}{3}{1}) to shew that the integral
obtained by differentiating under the sign of integration converges
uniformly; that is to say we have to prove that

/ - 1 log a- + \ y2) (a' +y-) - 5* sin i s arc tan -

dij

o ' y

- I (( +y ) * arc tan - cos f s arc tan - j

di/

~' y -I

converges uniformly with respect to s in any domain of values of s.
Now when s ! A, where A is any positive number, we have

1 (a2+y-) ~ * arc tan '- cos (s arc tan j < a- + i/' ')i cosh (|n-A);

since a-+f)

  J

,2 iA y±y\

converges, the second integral converges uniformly by \hardsubsubsectionref{4}{4}{3}{1} (I).

By dividing the path of integration of the first integral into two
parts (0, rra), Una, X ) and using the results

sin'lsarctau- I <sinh -, sin (s arc tan -) l<sinhJr7rA

V / 1 V /,

in the respective parts, we can simihxrly shew that the first integral
converges uniformly.

Consequently Hermite's formula is valid \hardsectionref{5}{5}) for all values of s,
and it is legitimate to differentiate under the sign of integration,
and the differentiated integral is a continuous function of s.

* The corresponding formula when = 1 had been previously giveu by
Jensen.

%
% 271
%

\Subsection{13}{2}{1}{Deductions from Hermites formula.}
Writing s = in Hermite's
formula, we see that

Making s - 1, from the uniformity of convergence of the integral
involved in Hermite's formula we see that

li,, 1 (,, a)-- \ = lim + 1 + 2 r, -,, .

  i( ' *-lj,-*! s-\ 2a Jo a- + y-') e y-l)

Hence, by the example of \hardsubsectionref{12}{3}{2}, we have

hm|ri., )- - | = -j .

Further, differentiating* the formula for l s, a) and then making s -
0, we get

\ d, A y 1 \, a -*' log a a'-'

+ 2 - o log (a- + 2/-) . (a- + y-) ~ *' sin ( s arc tan -)

Jo i \ ci/

+ (a- + V-) ~ arc tan - cos ( s arc tan - ) [ - -

    a \ a)] e' y - 1

I i\, ' arc tan (Wa),

= ( a-,jlog - + 2J \ \ \ A rfy.

Hence, by § 1232,

These results had previously been obtained in a different manner by
Lerch -j*.

Corollary. lim k(s) 1 = 7> T (0) = -5 log(27r).

\Section{13}{3}{Euler's product for TODO(s).}

Let (T l+B; and let 2, 3, 5, .../>,... be the prime numbers in order.
Then, subtracting the series for 2~* (s) from the series for (s), we
get

 (.).(l-2-) = ~ +,+~ +, + ...,

* This was justified in \hardsectionref{13}{2}.

t The formula for f (s, a) from which Lerch derived these results is
given in a memoir published by the Academy of Sciences of Prague. A
summary of his memoir is contained in the Jahrbuch iiber die
Fortschritte der Math. 1893-1894, p. 484.

%
% 272
%

all the terms of Sn~ for which n is a multiple of 2 being omitted;
then in like manner

all the terms for which n is a miiltiple of 2 or 3 being omitted; and
so on; so that

 (s) . (1 - 2-0(1 - 3-0  (1 -p-') = 1 + 2' -

the ' denoting that only those values of n (greater than p) which are
prime to 2, 3, ... j:? occur in the summation.

Now* i t'n-' I \$ I'n-'- \$ 2 n' - as ja co .

Therefore if a I + 8, the product (s) Yl (1 -p~ ) converges to 1,
tuhere

p the number p assumes the prime values 2, 3, 5, ... only.

But the product 11 (1 -p~ ) converges when a I + 8, for it consists of
p

some of the factors of the absolutely convergent product 11 (1 - n~ ).

Consequently we infer that (s) has no zeros at which a 1 + 8; for if

it had any such zeros, IT (1 -p~ ) would not converge at them. p

Therefore, ii a 1 + 8,

This is Euler's result.

\Subsection{13}{3}{1}{Riemanns hypothesis concerning the zeros of (s).}

It has just been proved that (s) has no zeros at which a >1.

From the formula (| 13*1 51)

y s) = 2 -i r(.9)|-isec ( STT Ul-s)

it is now apparent that the only zeros of (.5) for which a < are the
zeros

of r(s) - sec (2 '''') ) i-e- the points s = - 2, - -4, ....

Hence all the zeros of f (s) except those at - 2, - ]>, ... lie in
that strip of the domain of the complex variable s ivhich is defined
by - a 1.

It was conjectured by Riemann, but it has not yet been proved, that
all

the zeros of (.s) in this strip lie on the line o" = 2 ' " ' ile it
has quite recently

been proved by Hardy -f- that an infinity of zeros of (s) actually lie
on cr = : .

It is highly probable that Riemann's conjecture is correct, and the
proof of it would have far-reaching consequences in the theory of
Prime Numbers.

* The first term of S' starts with the prime next greater than p. t
Comptes Rendiif!, clviii. (1914), p. 1012; see p. 280.

%
% 273
%

\Section{13}{4}{Riemanns integral for TODO(s).}
It is easy to see that, if cr > 0,

n' Hence, when a > 0,

   s) r ( ) TT - i = lim f 1 e- "''  x -'- dx.

X

Now, if OT x)= S e"""' '", since, by example 17 of Chapter vi (p.
124), 1 + 2ct x) = x~ ' 1 4- 2ts (I/*'), we have lim x -sr x) = 1;
and hence

 st(x)x ~ dx converges when a > .

Jo

Consequently, if a > 2, (s)r ('.7s')7r-'''"=lim ! (x) x -''' dx- i 1
e''''''' xi'-' dx'] .

V" / N o lJt) .0 M = .V+1 J

Now, as in \hardsubsectionref{13}{1}{2}, the modulus of the last integral does not exceed

Jo \ n = N+l j .'o l\ e-< V+l) x

.0

= 7r(iY+l)|-'|(iV H2;\ \ )7r l-i< rQ<r-l)

-*- as uV - - X, since a > 2. Hence, when cr > 2,

= ri-. + -i +x- -x;7(llx)\ x -'-' dx+ I t!r( )a:i -l(fa;

= - + 7 + f .rizTOr)a;-i'' + l(- )(;a;+[ nT(x)x '-Ux.

Consequently

r(5)r(L')7r-i*- - v -r,= I (x - -''> +x ')x-' (x)dx.

Now the integral on the right represents an analytic function of s for
all values of s, by \hardsubsectionref{5}{3}{2}, since on the path of integration

tn- x) < e-' * S e-"" e-"" (1 - e-"")-'.

;i=0

Consequently, by § o S, the above equation, proved when cr> 2,
persists for all values of 5.

w. M. A.  18

%
% 274
%

If now we put

s = l + it, ls(s- 1) (s) r ( s 7r- *- = 1(0, we have

  (t) = I - ff + j x-i (x) cos ( tlogx dx.

Since x~ -st x) log x\ cos U log x + - mr) dx

satisfies the test of \hardsubsectionref{4}{4}{4} corollary, we may differentiate any
number of times under the sign of integration, and then put = 0.
Hence, by Taylor's theorem, we have for all values* of

 (0= S a t''', =o

by considering the last integral ag i is obviously real. This result
is fundamental in Riemann's researches.

\Section{13}{5}{Inequalities satisfied by TODO(s, a) when TODO.}

We shall now investigate the behaviour of (s, a) as t - + oo, for
given values of cr.

When cr> 1, it is easy to see that, if N be any integer,

as, -)=U- + n)- - -,\ s)il aY- -LM' where

\ 1 J 1 \ 1 \ \ 1

/'"+! u-n,

fn

Now, when tr'> 0, l/ ( ) I ! I /

Jn (u + a)

" +] a-n

ai

fn+l J n (n

di

 n + af- ' = slin + a)-"-'. Therefore the series i" f (s) is a
uniformly convergent series of analytic functions

when cr >; so that 2 / (s) is an analytic function when <t> 0; and
consequently, bj'

;i=.V
\hardsectionref{5}{5}, the function ( (s, a) may be defined when (r>0 by the series

C (., a) = £ (a + )-.- (i\,)(; ).-. -j/. . Now let [t] be the
greatest integer in | < |; and take iV=[ ]. Then

|C )I 2 \ \ {a + nr \ + \ \ { l-sr'i[t] + ay- + 2 \ s\ \ {n + ar'' '

71=0 n=[t] [t] X

< 2 a + 7i)- + \ t mt] + ay ' + \ s\ 2 (n + a) ' \ n=0 i>=[t]

* In this particular piece of analysis it is convenieut to regard t as
a complex variable, defined by the equation s = + it; and then | (t)
is an integral function of t.

%
% 275
%

Using the Maclaurin-Cauchy sum formula \hardsubsectionref{4}{4}{3}), we get

r[t] r

Jo J[t]-l

Now when 8 a- 1 - S where S > 0, we have \ (s, a) \ <a- + l-a)- a +
[t]y-'' -a - + \ t [t] + af-'' + \ s\ < r-H[t]-l + a)-''. Hence f (s,
a) = 0 \ t p"* ), the constant implied in the symbol being independent
of s. But, when 1 - 8 cr l + S, we have

I C (, a) I = ( i i \'-n + / ( + A-)-'' dx

<0 \ t f-") + '-'+( + tf-"] I '' (a+x) - 1 dx,

since (a + x)~' a ~' a+x)-'>- when o- l, and (a+x)'" a+[t]) ~' (a +
x)- when o" 1, anoJ so

Cis, a) = 'Itr'' log \ t\ \ }. When 0- 1+8,

|C(, )| a~ + i (a + r'-* = 6'(l).

\Subsection{13}{5}{1}{Inequalities satisfied by f (s, a) wlien cr 0.}

We next obtain inequalities of a similar nature when <j h. In the case
of the function f (s) we use Riemaim's relation

C(s) = 2''7r -i r (1 -s) f (1 -s) sin (isTr). Now, when o- < 1 - 8, we
have, by § 1233,

r(l-s) = 0 e(*"*)'*' ( -*)-( -*' and ao

C (s) = [exp .V I | + ( -o--i01og|l-s|+iarctan /(l-o-) ]C(l-s).

Since arc tan i/(l - cr)= ±i7r4-0 ( ~ ), according as is positive or
negative, we see, from the results already obtained for f (s, a), that

i B) = 0 \ t\ \ -''\ i s).

In the case of the function (s, ), we have to use the formula of
Hurwitz \hardsubsectionref{13}{1}{5}) to obtain the generalisation of this result; we
have, when o- < 0,

i s,a): -i ±nY- V s)\ \ e ' ' Uiy- )- -''''" i-a s)\ where Ca (!-' )=
2

1 %'-

. - Hence (1 -e" ''"') f (l - ) = e2' + 2 e2 ' ' [/i'-i- (n- l) -i]

+ (S-1) i / T' /"" tt -2£;

since the series on the right is a uniformly convergent series of
analytic functions whenever o- l-S, this equation gives the
continuation of fa(l-*) over the range O o-: 1-S; so that, whenever cr
1 - S, we have

sin7raCa(l-s) 1 1+ 2 /i' ~' + (n-l)' -iH-|s-l I 2 /" n"-- dv..

1=2 re=iV+l ] n-X

18-2

%
% 276
%

And obviously

Taking V=[ ], we obtain, as in \hardsectionref{13}{5},

Ca l-s)=0 \ tr) 8 a l-8) .

= 0 \ tf\ og\ t ) -8 (T<8).

C s)=0 ) a<-8).

Consequently, whether a is unity or not, we have the results

C s,a) = 0 \ t\ \ -'') (a 8)

= 0 \ t\ \ ) (8 0- 1-8)

= Oi\ t\ \ \ og\ t ) -8 a 8).

We may combine these results and those of \hardsectionref{13}{5}, into the single
formula

C(s,a) = 0(i<r" 'log| |), where*

r(o-)-i-(r, (o- O); 7-(o-) = A, (O a i); r(a) = l-(r, (*-\$<t 1);
r(cr) = 0, ( r l);

and the log | t \ may be suppressed except when - 8 o- S or when 1 - S
cr l + S.

\Section{13}{6}{The asymptotic expansion of log TODO.}
From \hardsectionref{12}{1} example 3,
it follows that

\ aJ =i (.V a+nJ J T (z + a) Now, the principal values of the
logarithms being taken,

= 2

n=l

- az

  (-)"'"'

+ 2

(\ yn-i r

ji a + n)J 2 'ni (a - + w)"'J j,,' ! m a'' If I I < a, the double
series is absolutely convergent since

' ' - log 1 + -

 aA- n) ° V a+ n) a+ n

= 1 [\ /i(a + n)

converges.

Consequently

log

az

+

e-v r(a)

r ( + a) a =1 71 (a + n) ',,,=2 wi

1 1 772'

X / yn- 1

2 5 z'"' m,a).

Now consider;;; -; - -. tis, a) ds, the contour of integration being

ziri J c ssimrs

similar to that of \hardsubsectionref{12}{2}{2} enclosing the points 5 = 2, 3, 4, ... but
not the

points 1, 0, -1,-2, ...; the residue of the integrand at s = m(m 2) is

- z m, a); and since, as cr x (where s= a + it), s, a) = (1), the

integral converges if | 2 | < 1.

* It can be proved that t a) may be taken to be i (1 - a) when (t 1.
See Landau, Prim- zahlen, % 237.

%
% 277
%

Consequently

, e-y'r a) z az \ -rrz .. ..

I z + a) a =i n (a + m) Ztti q s sin its

Hence

- V a) V' a) 1 f -TTZ' .,,

  r ( + a) r (a) 27ri j c- 5 sin tts '

Now let D be a semicircle of (large) radius N with centre at s = f,
the semicircle lying on the right of the line <r = |. On this
semicircle (s, a)=0(l), !ir*| = |;<'e- ' '-g', and so the integrand
is* 5:; e-' i'f- ' s3l. Hence if | | < 1 and - tt + 8 arg tt - 8,
where S is positive, the integrand is \ zY e' *" ), and hence

t, s, a)ds 0

J j)S sm 7r5

as iV -* 00 . It follows at once that, if; arg z \ : 7r - 8 and j [ <
1,

, T(a) r'(a) i r + '' 'rrz',,,

log Y ~- -. =-z + r-. ~. (s, a) ds.

° 1 ( + a) 1 (a) 'Itti J s issimrs

But this integral defines an analytic function of for all values of 12
| if

j arg z] TT - 8.

Hence, by \hardsectionref{5}{5}, the above equation, proved when [ | < 1, persists for
all values of I I when ] arg zI' tt - 8.

Now consider I --. (s, a) ds, where n is a fixed integer and

-/ -n- ±iii sin TTS'

R is going to tend to infinity. By \hardsubsectionref{13}{5}{1}, the integrand is [z' e'
R"' ',"' where - n - - cr :\$ -; and hence if the upper signs be
taken, or if the lower signs be taken, the integral tends to zero as
i2 - x . Therefore, by Cauchy's theorem,

Via) T'ia) 1 f->'-h + i n

log \ \ = -z j -~- + -. - (s,a)ds+ X R,n,

where R is the residue of the integrand at s = - in. Now, on the new
path of integration

I s sin ITS \

where K is independent of z and t, and t (t) is the function defined
in \hardsubsectionref{13}{5}{1}.

* The constants implied in the symbol are independent of s and z
throughout.

%
% 278
%

Consequently, since j e~ - ',t]' - -i)dt converges, we have

when \ 2\ is large.

Now, when m is a positive integer, i?, = - - '- and so

- m

by \hardsubsectionref{13}{1}{4}, Rm = r '-,, where 6,/ (a) denotes the derivate of

m(m + l)(m + 2)

Bernoulli's polynomial.

Also Ro is the residue at s = of

and so i2 = f - - a j log + ' (0, a)

= (i - ) log + log r ( ) - log (27r), by \hardsubsectionref{13}{2}{1}.

And, using \hardsubsectionref{13}{2}{1}, R\, is the residue* at >Sf= of

\ ia\ s=\ ..,(i, >...),(i,s, g,+..,(l\ i:g,...).

XT 75 1 r' (a)

Hence R\ = - z\ oo'z+ z tt + z.

r(a)

Consequently, finally, if | arg z ir- S and | j is large, log r (z +
a) == (z + a -~ \ og z - z + l\ og 27r)

+ i (-)"'~ </>w+2( -), \ \ s

w=i w(77H- l)(m + 2) *

In the special case when a = 1, this reduces to the formula found
previously in \hardsubsectionref{12}{3}{3} for a more restricted range of values of arg 2.

The asymptotic expansion just obtained is valid when a is not
restricted by the inequality < a 1; but the investigation of it
involves the rather more elaborate methods which are necessary for
obtaining inequalities satisfied by (s, a) when a does not satisfy the
inequality 0<a%l. But if, in the formula just obtained, we write a=l
and then put z + a for z, it is easily seen that, when j arg ( + a) [
< tt - 8, we have

log r (2 + a + 1) = f + a + 2) log (z + a)-z-a + l ' + o l);

* Writings = 5+1.

%
% 279
%

subtracting log (z + a) from each side, we easily see that when both

I arg z + a) 7r- 8 and \ arg zI tt - 8, we have the asymptotic formula

logr( + a)=( + a- jlog - + . log(27r) + o(l),

where the expression which is o (1) tends to zero as \ Zi->X).

REFERENCES. G. F. B. Riemann, Ges. Werke, pp. 145-155. E. G. H.
Landau, Handbuch der Primzahlen. (Leipzig, 1909.) E. L. LiNDELOF, Le
Calcid des Residue, Ch. iv. (Paris, 1905.) E. W. Barnes, Messenger of
Mathematics, xxix. (1899), pp. 64-128. G. H. Hardy and J. E.
Littlewood, Acta Mathematical xli. (1917), pp. 119-196.

Miscellaneous Examples.

1. Shew that

(2 - 1 ) f (5) = - - + 2 / (i +y2)-* sin (s arc tan 2y)

(Jensen, D Intermediaire des Math. (1895), 'p. 346.)

2. Shew that

2 -i / * dv

C(s)= \ j-2 j (1+/)-** sin (5 arc tan y) - - .

\addexamplecitation{Jensen.}

3. Discuss the asymptotic expansion of \ ogG z + a), (Chapter xii
example 48) by aid of the generahsed Zeta-function. \addexamplecitation{Barnes.}

4. Shew that, if cr > 1,

p m=.l mp"

the summation extending over the prime numbers jd = 2, 3, 5,

(Dirichlet, Journal de Math. iv. (1839), p. 407.)

5. Shew that, if o-> 1,

where A (w) = when n is not a power of a prime, and A n) = \ ogp when
is a power of a prime p.

6. Prove that e~- -dx

log C (5) = 2 2

 \&

'(is) Jo

** r (is) 1

(Lerch, KraMw Rozprawy*, ll. See the Jahrbuch ilber die Fortschritte
der Math. 1893-1894, p. 482.

%
% 280
%

7. If 00

where | a; | < 1, and the real part of s is positive, shew that

and, if 5 < 1,

lim (1 - xy- (j) (s, A-) = r (1 - s).

\addexamplecitation{Appell, Comptes Rendus, lxxxvii.}

8. If X, a, and s be real, and < a < 1, and s > 1, and if

< (--' ' ).= ?,( : .' . .

shew that

and

(b (x, a, 1 - s) = tttVo

\addexamplecitation{Lerch, Acta Math, xi.}

9. By evaluating the residues at the poles on the left of the straight
line taken as contour, shew that, if k > 0, and | arg 3/ 1 < Att,

1 fk+cci

e-y= --.l rhi)y-''du,

and deduce that, if - > i,

9 - f ' ' (")  ('" )" " '') du = w x\

k - xi

and thence that, if a is an acute angle.

r 1 (0 = TT cos ia - W" 1 + 2 or (e -) . 1 t + t

\addexamplecitation{Hardy.}

10. By differentiating 2?i times under the integral sign in the last
result of example 9, and then making a - Jtt, deduce from example 17
on p. 124 that

r-*i- <*. (,)*= < -cos|

By taking n large, deduce that there is no number 0 such that | t) is
of fixed sign when t > to, and thence that f (s) has an infinity of
zeros on the line <r = .

\addexamplecitation{Hardy.}

[Hardy and Littlewood, P?'oc. London Math. Soc. xix. (1920), have
shewn that the number of zeros on the hue o- = i for which < < T' is
at least ( T) as - 00; if the

Kiemann hypothesis is true, the number is -- 7' log - " 7"+ (log 7;
see

Landau, Pnmzahlen, i. p. 370.]


Given a predictor $P$ and a test object $s \in \testset$,
the performance of the predictor for this conjecture can be computed.
The general performance of the predictor for a given $\testset$
can be computed by averaging the values of a metric for all elements in $\testset$.

\begin{definition}
  A metrics function takes a ranking and a known object and yields a rational number
  that represents how the ranking relates to the actual ranking for the known object.
  Such a function is of the type $\rankings \rightarrow \objs \rightarrow \rat$.
\end{definition}

\begin{definition} The set of dependencies required to solve conjecture $\type[s]$ is defined as
  \[ \required[s] = \deps{s} \]
\end{definition}

\begin{definition} The set of suggestions done in a ranking is defined as
  \[ \suggestions{r} = \{ d \in \depset ~|~ r(d) ~\text{is defined} \} \]
\end{definition}

\begin{definition} We define $\topn{n}{r}$ to be
  \[ \topn{n}{r} = \downset{\depset}{\nth{n}{(\suggestions{r}, \lambda x~y. r(x) > r(y))}} \]
\end{definition}

\begin{definition}
  Given a totally ordered set $(X, \leq)$, and an element $x \in X$, we define $\findindex{X}{x}$ to be the position of $x$ in totally ordered set $X$.
  \[ \findindex{X}{x} = \left\{
    \begin{array}{ll}
      0 & \text{if~} x = \infimum_X \\
      \findindex{X \cap \infimum_X}{x}+1 & \text{if~} x \neq \infimum_X \\
    \end{array}
    \right.
  \]
\end{definition}

In this thesis we use the following metrics:
\begin{definition}[\oocover]
  The recall of the set of proof dependencies by the first 100 suggestions is defined as
  \[ \oocoverf{r}{s} = \frac{ |\topn{100}{r} \bigcap \required[s]| } { |\required[s]| } \]
  This is our primary metric for evaluating the performance of our premise selection algorithms.
  An higher recall is better.
\end{definition}

\begin{definition}[\ooprecision]
  The precision of the set of proof dependencies by the first 100 suggestions is defined as
  \[ \ooprecisionf{r}{s} = \frac{ |\topn{100}{r} \bigcap \required[s]| } { |\topn{100}{r}| } \]
  An higher precision is better.
  Typically by making less suggestions a better precision is achieved.
  For the corpii we evaluated most of our algorithms yield more than 100 suggestions per proof goal.
  Thus this metric should often yield around the same value for differing algorithms.
\end{definition}

\begin{definition}[\recall]
  The \recall is defined to be the number of guesses that were made until the entire set of required dependencies is suggested.
  When the set of guesses does not include all required dependencies $\recallf{r}{s}$ yields $|\suggestions{r}|+1$,
  or the number of suggestions plus one.
  The function `$\firstsym$' is defined in Definition \ref{def:first}.
  \[ \recallf{r}{s} = \firstsym(\lambda i. (i \leq |\suggestions{r}|) \rightarrow \required[s] \subseteq \topn{i}{r}) \]
  A lower \recall is better.
\end{definition}

\begin{definition}[\rank]
  The \rank is defined to be the average position of an actual proof dependency in the sequence of suggestions ordered by predicted relevance.
  \[ \rankf{r}{s} = \frac{
      \sum\limits_{r \in \required[s]} \left\{
        \begin{array}{ll}
          \findindex{\suggestions{s}}{r} & \text{if~} r \in \suggestions{s} \\
          0 & \text{otherwise} \\
        \end{array}
        \right.
    }{
      |\suggestions{r} \bigcap \required[s]|
    }
  \]
  A lower \rank is better.
\end{definition}

\begin{definition}[Area Under Curve (AUC)]
  The area under the ROC Curve is defined to be the probability that a correctly suggested premise is ranked \emph{better} than an incorrectly suggested premise.
  Given $X$ to be the correctly suggested premises $X = \suggestions{r} \bigcap \required[s]$ and $Y$ to be the incorrectly suggested premises $Y = \suggestions{r} - \required[s]$ we define the \auc to be:
  \[
    \aucf{r}{s} = \frac{
      \sum_x^X \sum_y^Y {\findindex{\suggestions{s}}{x} < \findindex{\suggestions{s}}{y}}
    }{
      |X| \times |Y|
    }
  \]
  An higher AUC indicates a better performance.
\end{definition}

\begin{definition}[\volume]
  The volume is defined to be the number of suggestions made by a predictor.
  From a runtime performance standpoint it is better to make less suggestions, with the risk of missing relevant facts.
  When just the first 100 suggestions are looked at, this metric is of limited importance.
  \[
    \volumef{r}{s} = |\suggestions{s}|
  \]
\end{definition}

When reporting these metrics for the entire testset \testset~the average is taken for every computed prediction.
Technically we are then refering to the Mean \recall and Mean \rank, but for simplicity sake we will refer to
just '\recall' in the following sections when talking about the results for entire testsets.

\subsubsection{Varying terminology}
This thesis uses concepts from both the \ltr academic field (such as with \adarank)
as the premise selection / ATP with machine learning fields (as with the work by Kaliszyk).
However between these fields the terminology used is not always consistent.
This is especially the case for the metrics used, as some cross polination has occurred between the fields,
but where differing concepts have ended up with the same name.
To clarify this I've tabularized the terms used:

\begin{figure}[H]
  \centering
  \begin{tabular}{l|l}
  \ltr          & \emph{ATP with Machine Learning} \\\hline
  \oocover      & 100Cover \\
  \ooprecision  & 100Precision \\
  \recall       & Recall \\
  \rank         & Rank \\
  \auc          & \auc \\
  \end{tabular}
  \caption{The varying terminology used between academic fields.}
\end{figure}


\section{Relations}\label{Relations}

This library develops closure properties of relations.

\begin{itemize}
\item {\tt Relation\_Definitions.v} deals with the general notions
  about binary relations (orders, equivalences, ...)

\item {\tt Relation\_Operators.v} and {\tt Rstar.v} define various
  closures of relations (by symmetry, by transitivity, ...) and
  lexicographic orderings.

\item {\tt Operators\_Properties.v} states and proves facts on the
  various closures of a relation.

\item {\tt Relations.v} puts {\tt Relation\_Definitions.v}, {\tt
    Relation\_Operators.v} and \\
    {\tt Operators\_Properties.v} together.

\item {\tt Newman.v} proves Newman's lemma on noetherian and locally
  confluent relations.

\end{itemize}


% -*- TeX-master: "master.tex" -*-

\setcounter{section}{8}
\setcounter{subsection}{0}
\subsection{Function objects, Exponentials}

\subsubsection{Function objects}

Functions are separated from types so far: we learned in Hask that types are
objects and functions are morphisms. For category $\Set$, objects are all the
sets, and functions are mapping between the objects. Functions from object $a$
to object $b$ forms a set, called hom-set, denoted as $\Hom(a,b)$. And because
$\Set$ contains all sets, this morphisms representing hom-set $\Hom(a,b)$ is also
an object in $\Set$.

In an arbitrary category, we don't have the hom-sets as internal objects; but we
can view them as external things being in other categories. What we want is
something we can represent these hom-sets inside the category. It is indeed
possible to define these objects in many categories, although not in arbitrary
category. Just like we define product and coproduct, the same way we can apply
to define this function object. We can use univeral construction to define
function object.

\begin{remark} The 3 steps towards universal construction:
\begin{enumerate}
\item Define a pattern.
\item Define ranking between matches.
\item Find the best one.
\end{enumerate}
\end{remark}

It's clear to see that the pattern will involve three objects, the function $z$,
the argument type $a$ and the return type $b$. In a category, we can't directly
say $z$ takes argument $a$ returns $b$, so we think how can we work on this. For
example, in category $\Set$, we can say take $z$ and $a$ to form a pair $(z,a)$
such that $(z,a) \iso b$. This pairing in $\Set$ represent a cartesian product
of $z$ and $a$. This perfectly captures the idea of function application, and
this pattern can be generalize to any category.

This pattern implies that in order to define this construction in a category,
the category must contain products. This makes more sense when later we discuss
function objects as exponentials.

The pattern, drawn in communitive diagram, looks like:

\begin{center}
  \begin{tikzcd}[sep=large]
  z \rar[dash,dashed] & z\times a \rar[swap]{g} & b  \\
  & a \uar[dash, dashed] &
  \end{tikzcd}
\end{center}

Note that the morphism $g$ is called a \emph{eval} morphism. Next we need to
define a ranking to select the best function $z$ from $a$ to $b$.

\newcommand{\arb}{\ensuremath{}(a\Rightarrow{}b)}

With the pattern above, we say $\arb$ is the function object from $a$ to $b$ if
for other patterns defined on $z$ and $a$, we have a unique morphism $h:
\cd{z\rar[dashed] \& \arb}$, that means we have to be able to factorize
$g:\cd{z\times a\rar\& b}$ into $\textrm{eval}$ arrow and a unique arrow
$\cd{z\times a\rar[dashed] \&\arb\times a}$. $\textrm{eval}$ arrow is given by
the pattern, so we need to find the other arrow. By previously learned
functoriality of ADTs, we know that product is a bifunctor, thus it not only
maps objects, it also maps morphisms. By which we can say the unique arrow we
are looking for is $h\times \id_a$. The whole construct can be shown as the
diagram below:

\begin{center}
  \begin{tikzcd}[sep=large]
    z  \dar[dashed]{h} &
    z\times a \ar{d}[dashed]{h\times \id_a} \ar[rd, start anchor={east}, bend left=30, "g"] & \\

    (a\Rightarrow b)  &
    (a\Rightarrow b)\times a \ar[r,swap,start anchor={east},"\textrm{eval}"] & b  \\

    & a  &
  \end{tikzcd}
  \\
  (arrows representing $\pi$ maps in products are omitted for clarity)
\end{center}

In short, the function object $\arb$ only exists if for every possible $z$ and
$g$ in that pattern, there is a unique $h$ from $z$ to $\arb$ such that:

\[
  g = \textrm{eval} \circ (h \times \id)
\]

In most languages, functions with two arguments are basically functions with one
argument which is a pair. Think of $g$ as such a function, that takes $z$ and
$a$ as arguments. Our definition of function objects implies given $z$ and $g$
we actually have a unique arrow $h$, that takes a $z$ and returns a function
object $\arb$.

This captures the idea of partially applying function is
equivalent to the applying function with its arguments given in a pair; the idea
can be generalized to functions with multiple arguments. This gives rise to the
concept of currying.

\begin{lstlisting}
h :: z -> (a -> b)
g :: (z, a) -> b

curry :: ((a,b)->c) -> (a->b->c)
curry f = \lambdaa->(\lambdab->f (a,b))

uncurry :: (a->b->c) -> ((a,b)->c)
uncurry f = \lambda(a,b)->(f a) b
\end{lstlisting}

\subsubsection{Exponential}

Function objects in category theory are actually called exponential. A function
object $\arb$ will be called $b^a$. This makes sense if you think of the
cardinality of function types. In Haskell, $\Bool \to \Int$ is basically $(\Int,
\Int)$, or $\Int \times \Int$ in category notation, or simpler $\Int^2$.
Remember we learned the correspondence between nature numbers and Hask types:
$1$ corresponds to unit $()$, $2$ corresponds to $\Bool$, etc. Then $\Int^2$ is
basically $\Int^\Bool$.

The idea shows the connection between products and exponentials (\ie.
functions). A special kind of categories called \textbf{Cartesian Closed
  Category} (CCC) is useful in programming. Cartesian means there is a product
for every pair of objects. Closed means the products are \emph{inside} the
category, which means it has all the exponentials objects as well. Also, it must
have the terminal object. Why? Terminal object is like the $0$-th power of any
objects. Sometimes we also want co-products in the category, this kind of
categories is called Bi-Cartesian Closed Categories (BCCC), which also has the
initial object.

With products and co-products both being monoidal, we can combine them together
to form a semi-ring, and do algebra on them. Now we add exponential to the
system. With exponentials we can do more algebra, for example, $a^0~=~1$.

We can interpret $a^0 = 1$ as $\Void \to a \sim ()$, \ie. a function from void
type to $a$ is equivalent to the unit type. We need to show the function on the
left hand side exists and is unique. Turns out there is this function, called
\textbf{absurd}.

What about $1^a=1$? It is a function takes $a$ and returns unit $()$; we say it
is equivalent to just $()$. There is only one such function indeed, called
$\texttt{const}~()$.

Let's try another one, $a^1=a$, This is saying a function from $()$ to $a$ is
equivalent to $a$. These functions are just the selecting functions; there are
as many as the number of elements in $a$.


\subsection{Potential Matching Weight}\label{sec:matchingweight}

\Figure[bt](topskip=0pt, botskip=0pt, midskip=0pt){figures/tikz/build/main-figure0.pdf}{
    A cluster with vertices $\{a,b,c\}$ with potential matching weights $\{2\frac{1}{2}, 3\frac{1}{2}, 2\frac{1}{2}\}$. The line style and color of the colored edges correspond to the matching in the hypothetical union with an external vertex $v'$ of the same line style and color.\label{fig0}}

%In the following we give some intuition into the improvement of the Union-Find Balanced Bloom decoder upon the original Union-Find decoder.
% We compared the ratio of the matchings between the MWPM decoder and our own implementation of the UF decoder, averaged over many simulations, and found that UF matching weight has a constant prefactor of $\sim 1.043$ over the minimum weight for the toric code (\Cref{comp_weight}). From this, we suspected that a decreased matching weight is a heuristic for an increased threshold. Within the context of the UF decoder, the matching weight may be decreased by prioritizing the growth of vertices with low PWM's within the cluster.

\textcolor{cyan}{Let a \emph{matching} in an cluster be a pairing between its syndrome vertices, such that every syndrome pair is connected via uniquely occupied edges in the cluster. The \emph{matching weight} is the total number of occupied edges in the matching.} Consider a cluster containing the set of an odd number of non-trivial vertices $V=\{a,b,c\}$ and the set of edges $E=\{(a,b), (b, c)\}$ of \Cref{fig0}. Now let us investigate the weight of a matching if an additional non-trivial vertex $w$ is connected to the cluster. If $w$ is connected to $a$ or to $c$, then the resulting matching has a total weight of 2: $(w,a)$ and $(b,c)$, or $(a,b)$ and $(c,w)$. However, if $w$ is connected to vertex $b$, then the total weight is 3: $(w, b)$ and $(a, c)$. %If we want to minimize the weight of the

Inspired by this observation, we associate with each vertex $v$ of an odd-parity cluster a \textbf{Potential Matching Weight} $\text{PMW}(v)$. The PMW measures the matching weight, assuming a union occurs in the next growth iteration. More precisely,
%\begin{definition}
    % define cluster
    % define matching size or weight
let $v$ be a vertex in the boundary of an odd cluster and let $w$ be a hypothetical non-trivial vertex adjacent to $v$ but exterior to the cluster. We define $\text{PMW}(v)$ as the minimum-weight perfect-matching's weight of the even cluster that results from adding the $(v,w)$ vertex to the odd cluster. % is given by the
    %Let there be a hypothetical merger between odd cluster $\alpha$ of vertices $V_\alpha$ and edges $E_\alpha$, and odd cluster $\beta$ of $V_\beta$ and $E_\beta$, on the edge $(v_\alpha, v_\beta)$, where $v_\alpha \in V_\alpha$ and $v_\beta \in V_\beta$. In the merged even cluster with edges $E_{\gamma} = E_\alpha \cup E_\beta \cup (v_\alpha, v_\beta)$, there is a matching $\m{C}_{(v_\alpha,v_\beta)} \subseteq E_{\gamma}$  between the syndrome vertices internal to the cluster. The \textbf{Potential Matching Weight} (PMW) of vertex $v_\alpha$ is then defined as
    %\begin{equation}
      %\text{PMW}(v) = \abs{\m{C}_{(v_\alpha,v_\beta)} \cap E_\alpha} + 1.
     % \text{PMW}(v) = \abs{\m{C'}}.
    %\end{equation}
    %\end{definition}

Since per growth iteration half-edges are attached to the odd cluster's boundary, the addition of an edge $(v,w)$ to a cluster occurs in two steps. In the first step, half of $(v,w)$ is added to the cluster of $v$. If $w$ is part of another cluster, and if the other half of $(v,w)$ was already added to the cluster of $w$, $(v,w)$ becomes \emph{fully grown}. If this is not the case, another round of growth is required to fully grow $(v,w)$ and to merge the clusters of $v$ and $w$. The PWM of $v$ is identical in both cases. To distinguish between the two cases, we add $\nicefrac{1}{2}$ to the PMW's of vertices with half-edges attached. Using this definition, the vertices $\{a,b,c\}$ in the cluster shown in \Cref{fig0} have PMW's $\{2\nicefrac{1}{2}, 3\nicefrac{1}{2}, 2\nicefrac{1}{2}\}$. The PMW can be used to prioritize the growth of vertices with low PMW such that there is an increased probability of mergers between clusters on edges connected to these vertices, and there is an increased probability of a lower matching weight. However, this heuristic is only interesting if the calculation of the PMW's is lighter computationally than performing minimum-weight perfect-matching. %within a cluster is potentially , especially for clusters of increasingly larger size, as all edges of a cluster must be considered in its calculation. Furthermore, the PMWs within a cluster change due to cluster growth and mergers, both of which occur more frequently as the system size increases. For this reason, the scaling of the PMW computation is vital to the decoder.

\Figure[bt](topskip=0pt, botskip=0pt, midskip=0pt){figures/tikz/build/main-figure1.pdf}{
    The cluster of \Cref{fig0}, comprised of nodes $\{A, B, C\}$ with respective roots $\{a, b, c\}$, after two rounds of prioritized growth of $a$ and $c$. There are regions of vertices that are either interior elements or have equal potential matching weights, represented as nodes with different node radii (labelled below the node) in the node-tree $\nset$. \label{fig:pmw}}


\section{MacDonald's solution: transcritical flow with a shock}

This is a MacDonald's steady flow test involving a shock in a short channel. This test was used by Delestre et al.~\cite{Delestre-etal2012} in their \textsc{SWASHES} benchmark library of shallow water analytical solutions. The original derivation of the analytical solution was given by MacDonald et al.~\cite{MBNS1995, MBNS1997}.
MacDonald's analytical solution was derived using a backward framework, that is: given the water depth, we construct the topography which satisfies the shallow water equations.

When water is in a steady state, we have a fixed depth and velocity with respect to time. Consider a one dimensional domain. Suppose that we are given the depth $h(x)$. The steady state conditions make the shallow water equations to the single identity
\begin{equation}
z_x = \left(  \frac{q^2}{gh^3} -1 \right) h_x - S_f
\end{equation}
where $q=uh$ is the momentum or water discharge and $S_f$ is the symbol for the force of bottom friction involving Manning's coefficient $n$. We take
\begin{equation}
S_f = n^2 \frac{q|q|}{h^{10/3}}.
\end{equation}
The topography is then determined by
\begin{equation}
z(x) = -\int_{x}^{L} z_x~dx
\end{equation}
in which $L$ is the channel length.

\subsection{Results}

For our test, suppose that the channel length $L = 100$ and at steady state the discharge $q = 2$. The initial condition is $u=v=0$ and $w=2.87870797$. The boundary condition is enforced such that the upstream boundary has $q=2$ and the downstream boundary has $h=3.58431872$. When water is steady, suppose that a shock occurs at $x=66\frac23$. Following Delestre et al.~\cite{Delestre-etal2012}, we consider the water depth
\begin{equation}
h(x,y)= \left\{ \begin{array}{ll}
       \left( \frac{4}{g}\right)^{1/3}
       \left(\frac43 - \frac{x}{100}  \right) - \frac{9x}{1000} \left( \frac{x}{100} -\frac23 \right)
        & ~\textrm{if}\quad 0 \leq x < 66\frac23\\
        ~~ & ~~ \\
       \left( \frac{4}{g}\right)^{1/3}
       \left[ a_1\left( \frac{x}{100} -\frac23 \right)^4 +a_1\left(  \frac{x}{100} -\frac23 \right)^3 \right.\\
       \left. \quad -a_2 \left( \frac{x}{100} -\frac23 \right)^2 + a_3 \left( \frac{x}{100} -\frac23 \right) +a_4\right]
       & ~\textrm{if}\quad 66\frac23 \leq x \leq 100\\
\end{array} \right.
\end{equation}
where
$a_1 = 0.674202$,
$a_2 = 21.7112$,
$a_3 = 14.492$,
$a_4 = 1.4305$, and
$n  = 0.0328$.

The following three figures show the stage, $x$-momentum, and $x$-velocity when water is steady. We should see excellent agreement between the analytical and numerical solutions.

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{stage_plot.png}
\end{center}
\caption{Stage results}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{xmom_plot.png}
\end{center}
\caption{Xmomentum results}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{xvel_plot.png}
\end{center}
\caption{Xvelocity results}
\end{figure}

\endinput


\chapter{cstelm}

\section{Introduction}

An LPE may have parameters that do not change value throughout the state space of that process.
These parameters are essentially \emph{constant}.
Removing constant parameters results in a smaller state vector (which may improve performance), but otherwise the state space of an LPE remains the same.

Constants can be safely removed from a process as follows:
\begin{enumerate}
\item Substitute references to a constant parameter by the initial (explicit) value of that parameter;
\item Remove the constant from the process parameter list.
\end{enumerate}

\section{Algorithm}

The algorithm consists of the following steps \cite{groote2001computer}:

\begin{enumerate}

\item Make it so that all LPE parameters are \labeledas{constant}.

\item Let $X$ be the set of all LPE parameters that are \labeledas{constant}.
Define a substitution $V = [p \rightarrow v_I(p) \;|\; p \in X]$, where $v_I(p)$ gives the initial value of LPE parameter $p$.

\item Consider each summand $s$ of the LPE.
Construct an equation $g_s \rightarrow p = v_s(p)$ for all $p \in X$, where $g_s$ is the guard of $s$ and where $v_s(p)$ is the expression that defines the value of LPE parameter $p$ after the application of summand $s$, and apply the substitution $V$ to it.
(This gives $(g_s \rightarrow p = v_s(p))[V] \Leftrightarrow {g_s}[V] \rightarrow v_I(p) = v_s(p)[V]$.)

If the obtained equation is a tautology (that is, if its negation is unsatisfiable) for all $s$, $p$ remains \labeledas{constant}; otherwise, \removelabelfrom{constant} $p$.

\item Repeat the previous two steps until the fixpoint of $X$ is reached.
Then remove all LPE parameters that are still \labeledas{constant} from the LPE, substituting references to those parameters by their initial values.

\end{enumerate}

\section{Example}

Consider the following LPE:

\begin{lstlisting}
//Process definition:
PROCDEF example[A](x, y, z :: Int)
  = A [[z = 2]] >-> example[A](z-1, 1, 2)
  + A >-> example[A](y, x, x+y)
  + A >-> example[A](1, x, z+1)
  ;

//Initialization:
example[A](1, 1, 2);
\end{lstlisting}

First, $V = [ x \rightarrow 1, y \rightarrow 1, z \rightarrow 2 ]$.

We must check the following equations:

\begin{align*}
(z = 2 \rightarrow x = z-1)[V] &\Leftrightarrow (2 = 2 \rightarrow 1 = 2-1) \Leftrightarrow \textit{true} \\
(z = 2 \rightarrow y = 1)[V] &\Leftrightarrow (2 = 2 \rightarrow 1 = 1) \Leftrightarrow \textit{true} \\
(z = 2 \rightarrow z = 2)[V] &\Leftrightarrow (2 = 2 \rightarrow 2 = 2) \Leftrightarrow \textit{true} \\
(x = y)[V] &\Leftrightarrow (1 = 1) \Leftrightarrow \textit{true} \\
(y = x)[V] &\Leftrightarrow (1 = 1) \Leftrightarrow \textit{true} \\
(z = x+y)[V] &\Leftrightarrow (2 = 1+1) \Leftrightarrow \textit{true} \\
(x = 1)[V] &\Leftrightarrow (1 = 1) \Leftrightarrow \textit{true} \\
(y = x)[V] &\Leftrightarrow (1 = 1) \Leftrightarrow \textit{true} \\
(z = z+1)[V] &\Leftrightarrow (2 = 2+1) \Leftrightarrow \textit{false} \\
\end{align*}

The last equation is not a tautology, and the `constant' label is therefore removed from $z$.

The new value of $V$ is $[ x \rightarrow 1, y \rightarrow 1 ]$.

\clearpage
The equations are now the following:

\begin{align*}
(z = 2 \rightarrow x = z-1)[V] &\Leftrightarrow (z = 2 \rightarrow 1 = z-1) \Leftrightarrow \textit{true} \\
(z = 2 \rightarrow y = 1)[V] &\Leftrightarrow (z = 2 \rightarrow 1 = 1) \Leftrightarrow \textit{true} \\
(x = y)[V] &\Leftrightarrow (1 = 1) \Leftrightarrow \textit{true} \\
(y = x)[V] &\Leftrightarrow (1 = 1) \Leftrightarrow \textit{true} \\
(x = 1)[V] &\Leftrightarrow (1 = 1) \Leftrightarrow \textit{true} \\
(y = x)[V] &\Leftrightarrow (1 = 1) \Leftrightarrow \textit{true} \\
\end{align*}

All of the equations above are tautologies, and so $x$ and $y$ remain labeled with `constant'.
A fixpoint has been reached; removing $x$ and $y$ from the LPE gives

\begin{lstlisting}
//Process definition:
PROCDEF example[A](z :: Int)
  = A [[z==2]] >-> example[A](2)
  + A [[true]] >-> example[A](2)
  + A [[true]] >-> example[A](z+1)
  ;

//Initialization:
example[A](2);
\end{lstlisting}

Obviously, more simplification is possible.
Applying the \texttt{clean} command (see \ref{clean}) gives

\begin{lstlisting}
//Process definition:
PROCDEF example[A](z :: Int)
  = A [[true]] >-> example[A](2)
  + A [[true]] >-> example[A](z+1)
  ;

//Initialization:
example[A](2);
\end{lstlisting}

\section{Benchmark results}

The following durations were measured with a benchmark for several models:
\begin{itemize}
\item The average duration of \txs{} to make 500 steps in a model after it has been converted to LPE form;
\item The average duration of \txs{} to make 500 steps in a model after it has been converted to LPE form and after the \texttt{cstelm} operation has been applied.
\end{itemize}

When plotting the second series of measurements against the first (see Figure~\ref{cstelm-vs-lpe-only:fig}), it becomes clear that the effect of \texttt{cstelm} is tiny.

\begin{figure}[!ht]
\begin{center}
\includegraphics[width=0.7\linewidth]{charts/cstelm-vs-lpe-only}
\caption{Benchmark results: cstelm vs LPE transformation}
\label{cstelm-vs-lpe-only:fig}
\end{center}
\end{figure}


\subsection{Rogue Key Attack}

When Schnorr signatures are used to generate an aggregated signature of a transaction they are vulnerable to an attack known as Rogue Key attack. Rogue Key attacks performed by a malicious entity consists of generating an aggregated signature in such a way that they posses the public/private key pair for that signature. In the Schnorr signature scheme, the public key of participants are aggregated and the sum represent the public key associated to the signature. Assume that an honest participant use its public key $Q_a$ in the transaction and a malicious participant possesses $Q_b$. By sending the public key $Q_m = Q_b - Q_a$ to the honest participant, the malicious entity have access to the transaction as they will hold the private key for $Q_b$. This is because when the keys are aggregated i.e. $Q_m + Q_b$ the aggregated signature would be $Q_b$, for which the malicious user holds the private key (the honest user would not). For an aggregated public key, there should be no user that has a private key equivalent as it should be used to create a signature that can be verified that all users in the transaction participated. \\

The aggregation of public keys used in Catalyst which is based on Mu-Sig \cite{musig} signature scheme that is not vulnerable to this form of attack. Mu-Sig is protected from this form of attack as the scheme does not require a user to demonstrate each public key, only the sum of all the public keys. By not verifying individual public keys, a key rogue attack is not possible. Only one public key is needed for the verification (the aggregated key) for which there will not be an equivalent private key.

\subsection{Quantum Attack}

Quantum computers pose a very real threat to the encryption techniques used in blockchains in the medium to long term \cite{agarwal}. The threat is through the use of Shor's algorithm. A quantum attacker using Shor's algorithm on a quantum computer can gain an exponential speed-up in solving the discrete logarithmic problem. The assumption of security the discrete logarithmic functions the the primary basis as to which all elliptic curve cryptography is based. This means that even the schema demonstrated here will be vulnerable to attack. The use of aggregated signatures would provide some resistance, however this resistance would be negligible. \\

It must be impressed that this is not an issue for the near term and thereby, these schema are highly secure and efficient currently. The most efficient algorithm for classical computers to solve the discrete logarithm problem is the Pollard's rho\cite{pollard}, this does not run in polynomial time. While Catalyst is not currently resistant to quantum attack, this is a challenge that will be faced by all major distributed ledgers over time.


\BoSSSopen{shortTutorialMatlab/tutorialMatlab}
\graphicspath{{shortTutorialMatlab/tutorialMatlab.texbatch/}}

\BoSSScmd{
/// In this short tutorial we want to use common Matlab commands within the \BoSSS{} framework.
/// \section{Problem statement}
/// For our matrix analysis we use the following random matrix:
/// \begin{equation*}
/// A = \begin{bmatrix}
///      1 & 2 & 3\\
///      4 & 5 & 6\\
///      7 & 8 & 9
///     \end{bmatrix}
/// \end{equation*}
/// and the symmetric matrix:
/// \begin{equation*}
/// S = \begin{bmatrix}
///      1 & 2 & 3\\
///      2 & 3 & 2\\
///      3 & 2 & 1
///      \end{bmatrix}
/// \end{equation*}
/// We are going to evaluate some exemplary properties of the matrices and check if the matrices are symmetric, both in the \BoSSS{} framework and in Matlab.
/// \section{Solution within the \BoSSS{} framework}
/// First, we have to initialize the new project:
 }
\BoSSSexeSilent
\BoSSScmd{
restart;
 }
\BoSSSexeSilent
\BoSSScmd{
using ilPSP.LinSolvers;\newline
using ilPSP.Connectors.Matlab;
 }
\BoSSSexe
\BoSSScmd{
/// We want to implement the two 3x3 matrices in \BoSSSpad{}:
int Dim     = 3;\newline
MsrMatrix A = new MsrMatrix(Dim,Dim);\newline
MsrMatrix S = new MsrMatrix(Dim,Dim);\newline
double[] A\_firstRow = new double[]\{1,2,3\};\newline
double[] A\_secondRow = new double[]\{4,5,6\};\newline
double[] A\_thirdRow = new double[]\{7,8,9\};\newline
 \newline
double[] S\_firstRow = new double[]\{1,2,3\};\newline
double[] S\_secondRow = new double[]\{2,3,2\};\newline
double[] S\_thirdRow = new double[]\{3,2,1\};\newline
 \newline
for(int i=0; i<Dim; i++)\{\newline
\btab A[0, i] = A\_firstRow[i];\newline
\btab S[0, i] = S\_firstRow[i];\newline
\}\newline
 \newline
for(int i=0; i<Dim; i++)\{\newline
\btab A[1, i] = A\_secondRow[i];\newline
\btab S[1, i] = S\_secondRow[i];\newline
\}\newline
 \newline
for(int i=0; i<Dim; i++)\{\newline
\btab A[2, i] = A\_thirdRow[i];\newline
\btab S[2, i] = S\_thirdRow[i];\newline
\}
 }
\BoSSSexe
\BoSSScmd{
/// \paragraph{Test for symmetry in \BoSSS{}:}$~~$\\
/// To analyze if the matrices are symmetric, we need to compare the original matrix with the transpose:
MsrMatrix AT    = A.Transpose();\newline
MsrMatrix ST    = S.Transpose();\newline
bool SymmTest\_A;\newline
bool SymmTest\_S;\newline
for(int i = 0; i<Dim; i++)\{\newline
\btab for(int j = 0; j<Dim; j++)\{\newline
\btab \btab if(A[i,j] == AT[i,j])\{\newline
\btab \btab \btab SymmTest\_A = true;\newline
\btab \btab \btab \}\newline
\btab \btab else\{\newline
\btab \btab \btab SymmTest\_A = false;\newline
\btab \btab \btab break;\newline
\btab \btab \btab \}\newline
\btab \btab \}\newline
\btab \}\newline
for(int i = 0; i<Dim; i++)\{\newline
\btab for(int j = 0; j<Dim; j++)\{\newline
\btab \btab if(S[i,j] == ST[i,j])\{\newline
\btab \btab \btab SymmTest\_S = true;\newline
\btab \btab \btab \}\newline
\btab \btab else\{\newline
\btab \btab \btab SymmTest\_S = false;\newline
\btab \btab \btab break;\newline
\btab \btab \btab \}\newline
\btab \btab \}\newline
\btab \}\newline
if(SymmTest\_A == true)\{\newline
Console.WriteLine("Matrix A seems to be symmetric.");\newline
\}\newline
else\{\newline
Console.WriteLine("Matrix A seems NOT to be symmetric.");\newline
\}\newline
if(SymmTest\_S == true)\{\newline
Console.WriteLine("Matrix S seems to be symmetric.");\newline
\}\newline
else\{\newline
Console.WriteLine("Matrix S seems NOT to be symmetric.");\newline
\}
 }
\BoSSSexe
\BoSSScmd{
/// \paragraph{The interface to Matlab:}$~~$\\
/// The \code{BatchmodeConnector} initializes an interface to Matlab:
Console.WriteLine("Calling MATLAB/Octave...");\newline
BatchmodeConnector bmc = new BatchmodeConnector();\newline
/// We have to transfer out matrices to Matlab:
bmc.PutSparseMatrix(A, "Matrix\_A");\newline
bmc.PutSparseMatrix(S, "Matrix\_S");\newline
/// Now we can do calculations in Matlab within the \BoSSSpad{} using the \code{Cmd} command. It commits the Matlab commands as a string. We can calculate e.g. the rank of the matrix or the eigenvalues:
bmc.Cmd("Full\_A = full(Matrix\_A)");\newline
bmc.Cmd("Full\_S = full(Matrix\_S)");\newline
bmc.Cmd("Rank\_A = rank(Full\_A)");\newline
bmc.Cmd("Rank\_S = rank(Full\_S)");\newline
bmc.Cmd("EV\_A = eig(Full\_A)");\newline
bmc.Cmd("EV\_S = eig(Full\_S)");\newline
bmc.Cmd("Det\_A = det(Full\_A)");\newline
bmc.Cmd("Det\_S = det(Full\_S)");\newline
bmc.Cmd("Trace\_A = trace(Full\_A)");\newline
bmc.Cmd("Trace\_S = trace(Full\_S)");\newline
/// We can transfer matrices or arrays from Matlab to \BoSSSpad{} as well, here we want to have the results:
MultidimensionalArray Results = MultidimensionalArray.Create(2, 3);\newline
bmc.Cmd("Results = [Rank\_A, Det\_A, Trace\_A; Rank\_S,  Det\_S,  Trace\_S]");\newline
bmc.GetMatrix(Results, "Results");\newline
/// and the eigenvalues:
MultidimensionalArray EV\_A = MultidimensionalArray.Create(3, 1);\newline
bmc.GetMatrix(EV\_A, "EV\_A");\newline
MultidimensionalArray EV\_S = MultidimensionalArray.Create(3, 1);\newline
bmc.GetMatrix(EV\_S, "EV\_S");\newline
/// After finishing using Matlab we need to close the interface to Matlab:
bmc.Execute(false);\newline
Console.WriteLine("MATLAB/Octave closed, return to BoSSSPad");
 }
\BoSSSexe
\BoSSScmd{
/// And here are our results back in the \BoSSSpad{}:
double Rank\_A  = Results[0,0];\newline
double Rank\_S  = Results[1,0];\newline
double Det\_A   = Results[0,1];\newline
double Det\_S   = Results[1,1];\newline
double Trace\_A = Results[0,2];\newline
double Trace\_S = Results[1,2];\newline
Console.WriteLine("The results of matrix A are: rank: " + Rank\_A + ", trace: " + Trace\_A + ", dterminant: " + Det\_A);\newline
Console.WriteLine("The results of matrix S are: rank: " + Rank\_S + ", trace: " + Trace\_S + ", determinant: " + Det\_S);\newline
Console.WriteLine();\newline
Console.WriteLine("The eigenvalues of matrix A are: " + EV\_A[0,0] + ", " + EV\_A[1,0] + " and " + EV\_A[2,0]);\newline
Console.WriteLine("The eigenvalues of matrix S are: " + EV\_S[0,0] + ", " + EV\_S[1,0] + " and " + EV\_S[2,0]);
 }
\BoSSSexe
\BoSSScmd{
/// \paragraph{Test for symmetry within Matlab using the \code{BatchmodeConnector}:}$~~$\\
/// We do the same test for symmetry for both matrices. In Matlab we can use the convenient command \code{isequal}:
Console.WriteLine("Calling MATLAB/Octave...");\newline
BatchmodeConnector bmc = new BatchmodeConnector();\newline
bmc.PutSparseMatrix(A, "Matrix\_A");\newline
bmc.PutSparseMatrix(S, "Matrix\_S");\newline
bmc.Cmd("Full\_A = full(Matrix\_A)");\newline
bmc.Cmd("Full\_S = full(Matrix\_S)");\newline
bmc.Cmd("A\_Transpose = transpose(Full\_A)");\newline
bmc.Cmd("S\_Transpose = transpose(Full\_S)");\newline
bmc.Cmd("SymmTest\_A = isequal(Full\_A, A\_Transpose)");\newline
bmc.Cmd("SymmTest\_S = isequal(Full\_S, S\_Transpose)");\newline
 \newline
MultidimensionalArray SymmTest\_A = MultidimensionalArray.Create(1, 1);\newline
bmc.GetMatrix(SymmTest\_A, "SymmTest\_A");\newline
MultidimensionalArray SymmTest\_S = MultidimensionalArray.Create(1, 1);\newline
bmc.GetMatrix(SymmTest\_S, "SymmTest\_S");\newline
bmc.Execute(false);\newline
Console.WriteLine("MATLAB/Octave closed, return to BoSSSPad");
 }
\BoSSSexe
\BoSSScmd{
if(SymmTest\_A[0,0] == 1)\{\newline
Console.WriteLine("Matrix A seems to be symmetric.");\newline
\}\newline
else\{\newline
Console.WriteLine("Matrix A seems NOT to be symmetric.");\newline
\}    \newline
if(SymmTest\_S[0,0] == 1)\{\newline
Console.WriteLine("Matrix S seems to be symmetric.");\newline
\}\newline
else\{\newline
Console.WriteLine("Matrix S seems NOT to be symmetric.");\newline
\}
 }
\BoSSSexe


\chapter{Function}

\section{Fractional iterate}

Generalize \(f^p\) for real \(p\).
\begin{align*}
    f^2 &= f \circ f
    \\
    f^{1/2} \circ f^{1/2} &= f
\end{align*}

If \(f~x = x \uparrow a\) then \(f~(f~x) = (x \uparrow a) \uparrow a\).

What is a \(g\) that satisfies \(g~(g~x) = x^a\)?

What does \(d~(p \to f^p)\) even mean?
The \(d\) is differential operator.

% http://math.stackexchange.com/questions/676229/fractional-composite-of-functions

% https://en.wikipedia.org/wiki/Iterated_function#Fractional_iterates_and_flows.2C_and_negative_iterates


% This is part of the TFTB Reference Manual.
% Copyright (C) 1996 CNRS (France) and Rice University (US).
% See the file refguide.tex for copying conditions.

\markright{gdpower}
\section*{\hspace*{-1.6cm} gdpower}

\vspace*{-.4cm}
\hspace*{-1.6cm}\rule[0in]{16.5cm}{.02cm}
\vspace*{.2cm}

{\bf \large \sf Purpose}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
Signal with a power-law group delay.
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Synopsis}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
[x,gpd,f] = gdpower(N)
[x,gpd,f] = gdpower(N,k)
[x,gpd,f] = gdpower(N,k,c)
\end{verbatim}
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Description}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
        {\ty gdpower} generates a signal with a power-law group delay of
        the form \[t_x(f) = t_0 + c\ f^{k-1}.\] The output signal is of
        unit energy.\\

\hspace*{-.5cm}\begin{tabular*}{14cm}{p{1.5cm} p{8.5cm} c}
Name & Description & Default value\\
\hline
        {\ty N}   & number of points in time          (must be even)\\
        {\ty k}   & degree of the power-law           & {\ty 0}\\
        {\ty c}   & rate-coefficient of the power-law group delay.
              {\ty c} must be non-zero.               & {\ty 1} \\
  \hline {\ty x}   & time row vector containing the signal samples\\
        {\ty gpd} & output vector containing the group delay samples, of
	length {\ty round(N/2)}\\
        {\ty f}   & frequency bins\\
\hline
\end{tabular*}

\end{minipage}
\vspace*{1cm}

{\bf \large \sf Examples}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
Consider a hyperbolic group-delay law, and compute the Bertrand
distribution of it :
\begin{verbatim}
         sig=gdpower(128);
         tfrbert(sig,1:128,0.01,0.3,128,1);
\end{verbatim}
We note that the perfect localization property of the Bertrand distribution
on hyperbolic group-delay signals is checked in that case. \\
\end{minipage}
\newpage
\hspace*{1.5cm}\begin{minipage}[t]{13.5cm}
Plot the instantaneous frequency law on which the D-Flandrin distribution
is perfectly concentrated :
\begin{verbatim}
         [sig,gpd,f]=gdpower(128,1/2);
         plot(gpd,f);
         tfrdfla(sig,1:128,.01,.3,218,1);
\end{verbatim}
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf See Also}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
fmpower.
\end{verbatim}
\end{minipage}


\section{Background and Notation} \label{background_and_notation}
This section explains the basic sliding window technique as fundamental framework used in this thesis. Furthermore some
background on the well known time series similarity measure DTW is given. The sliding window filter is expounded at the
end of this section.

A time series $Q$ with the length $l$ over the domain set $\mathbb{U}$ is a sequence of data points
$Q = (q_1, q_2, \dots, q_i, \dots, q_l)$ with $q_i \in \mathbb{U}$. A distance measure function $d$ with
$d: \mathbb{U} \times \mathbb{U} \to \mathbb{R}$ exists on the set $\mathbb{U}$. Furthermore basic arithmetic operations
like a summing function $\mathbb{U} \times \mathbb{U} \to \mathbb{R}$ and a scalar multiplication
$\mathbb{R} \times \mathbb{U} \to \mathbb{U}$ should be defined on $\mathbb{U}$. A common assumption for time series is
that the containing data points are elements of the integer numbers $\mathbb{Z}$ or the real numbers $\mathbb{R}$. The
distance measure function on these common domains is just the absolute value of the difference of two elements. A
fundamental prerequisite for similarity measures on time series is often the distance measure function $d$ on the set
$\mathbb{U}$. Table~\ref{tab:notation} explains the basic notation that is used in this bachelor thesis.

\begin{table}
    \begin{center}
        \begin{tabularx}{\textwidth}{c X l}
            \hline
            \textbf{Symbol} \qquad & \textbf{Description} & \qquad \textbf{Section}\\
            \hline
            $\mathbb{U}$ & a set containing all items of a domain & \qquad\\
            $d$ & a distance measure function on $\mathbb{U}$ with $d: \mathbb{U} \times \mathbb{U} \to \mathbb{R}$
                & \qquad\\
            $Q$ & a time series over the set $\mathbb{U}$ of size $l$ with
                $Q = (q_1, q_2, \dots, q_i, \dots, q_l), q_i \in \mathbb{U}$ & \qquad\\
            $Q[i,j]$ & a subsequence time series of $Q$ over the set $\mathbb{U}$ with
                $Q[i,j] = (q_i, q_{i+1}, \dots, q_{j})$ & \qquad\\
            $t$ & the current time & \qquad\\
            $w$ & time series window size & \qquad \ref{sliding_window_technique}\\
            $s$ & time series step size & \qquad \ref{sliding_window_technique}\\
            $K_i$ & a class of time series & \qquad\\
            $\epsilon_i$ & the distance threshold of class $K_i$ & \qquad\\
            DTW & Dynamic Time Warping, a similarity measure for time series & \qquad \ref{dynamic_time_warping}\\
            $\bar{q}$ & the mean of a time series $Q$ over the set $\mathbb{U}$, $\bar{q} \in \mathbb{U}$ & \qquad
                \ref{time_series_normalization}\\
            $\sigma$ & the standard deviation of a time series $Q$ over the set $\mathbb{U}$, $\sigma \in \mathbb{R}$ &
                \qquad \ref{time_series_normalization}\\
            $\eta$, $\eta '$  & two different time series normalizations & \qquad \ref{time_series_normalization}\\
            CE & Complexity Estimate & \qquad \ref{complexity_estimate}\\
            LNCE & Length normalized Complexity Estimate & \qquad \ref{complexity_estimate}\\
            VAR & Sample Variance & \qquad \ref{sample_variance}\\
            \hline
        \end{tabularx}
    \end{center}
    \caption{Basic notation.}
	\label{tab:notation}
\end{table}

\input{background_and_notation/sliding_window_technique.tex}
\input{background_and_notation/dynamic_time_warping.tex}
\input{background_and_notation/sliding_window_filter.tex}
\input{background_and_notation/complexity_estimate.tex}
\input{background_and_notation/sample_variance.tex}


\documentclass{article}
\usepackage{color}
\usepackage{bigints}
\usepackage[italicdiff]{physics}
\color{white}
\definecolor{Green}{RGB}{0,66,37}
\begin{document}
\pagecolor{Green}
\title{Solution to Mathemaddict\textsc{\char13}s Problem}
\author{Jose Bedoya}
\maketitle
\section{Question}
{\LARGE
$$I = \bigintss_{1}^{\infty} \frac{x^2-1}{x^4 \ln x}\, dx$$
}
\section{Solution}
\vspace{5mm}
{\Large
Consider the substitution $x \rightarrow \frac{1}{u}$
$$ I = \bigintsss_{0}^{1} \frac{u^2-1}{\ln u}\,du$$

\vspace{5mm}
Let
$$ B\left(\alpha \right) = \bigintsss_{0}^{1} \frac{x^{\alpha} -1}{\ln x}\, dx$$

\vspace{3mm}
Notice our desire integral is $B\left(2\right)$.
\newpage
Differentiate both sides,
$$\dv{B}{\alpha} = \dv{\alpha}\bigintsss_{0}^{1} \frac{x^{\alpha}-1}{\ln x}\, dx$$

\vspace{3mm}
By the Leibniz rule for integrals
$$\dv{B}{\alpha}=\bigintsss_{0}^{1}\pdv{\alpha}\frac{x^{\alpha}-1}{\ln x}\, dx$$

\vspace{3mm}
Therefore,
$$\dv{B}{\alpha}=\bigintsss_{0}^{1} x^{\alpha}\,dx$$
$$=\frac{1}{\alpha+1}$$

\vspace{3mm}
Integrating both sides
$$B\left(\alpha\right)=\bigintsss \frac{1}{\alpha+1}\,d\alpha$$
$$=\ln (\alpha+1)+C$$

\vspace{2mm}
If you let $\alpha=0$, you will get $C=0$.

\vspace{7mm}
Therefore,
}

{\LARGE
$$I = \bigintss_{1}^{\infty} \frac{x^2-1}{x^4 \ln x}\, dx=\ln3$$
}

\end{document}

Calculus/Jose Bedoya's questions/Integral with Leibniz rule.tex


\chapter{Discriminant analysis}
\label{discriminant}

Discriminant analysis presupposes that we have a number of known groups of individuals, and a set of data which has been collected on individuals within these groups.   We wish to find a way of using that data to predict which group these individuals belong to, either to understand the differences or to be able to predict group membership.  \cite{Bumpus:1898} collected data on sparrows who survived and didn't survive a storm, these data are extensively analysed in this context by Manly, the primary aim of the analysis being to look for differences between the groups.   Are there different features which help us tell storm survivors from non-survivors?   More usually, we may be interested in predicting group membership.   Common examples can be found in finance; can banks tell good credit risks from bad based on data collected on customers who have subsequently defaulted on loans, see \cite{Johnson+Wichern:2002} for more details.   Another good account of discriminant analysis is given by \cite{Flury:1997} who suuggests it may be valuable when we have to carry out destructive procedures to determine group membership (such as in certain quality control investigations).  Finally a rather brief account is given in \cite{Venables+Ripley:2002}, which gives the example of disease diagnosis.   Consider a set of measurements of patient characterstics, and information determined on whether these patients have breast cancer or not.   We would be very interested in being able to make a determination of breast cancer based on the data, rather than having to wait for biopsy or other pathological information.

Discriminant analysis in one dimension seems straightforward enough.   We can examine the densities of the two groups and find an optimal cut-off point, which classifies the two groups as accurately as possible.   Some idea of the procedure is given in figure \ref{discrim}, which illustrates the idea behind discriminant function.

\begin{figure}
\begin{center}
\includegraphics[width = 0.5\textwidth]{images/discrim}
\caption{Idealised discrimant function}
\label{discrim}
\end{center}
\end{figure}

Note immediately that there is a measureable risk of misclassification, which depends on the variance within groups and the separation between groups.   All we need to do is extent this procedure to work in more than one dimension.   We are going to realise this by seeking a linear combination giving us the largest separation between groups.   In other words, we are going to find linear combinations based on the original variables:

\begin{equation}
z = a_{1} x_{1} + a_{2} x_{2} + \ldots + a_{p} x_{p}
\end{equation}

However our linear combination this time will be optimised to give us the greatest potential for distinguishing the two groups.   Having found a suitable linear combination, we select a cut-off point (denoted by the vertical dotted line in the figure above), and assign observations to group 1 or group 2 based on the value relative to the cut-off.   You can see from the stylised function shown that some observations will be misclassfied!   We check the performance of this aspect of our procedure by means of a confusion matrix.

Recall that when conducting the $T^{2}$ test we essentially looked for linear combination of variables which maximised the difference between groups.   Similar ideas apply in discriminant analysis.   We seek a transformation of the data which gives the maximum ratio of group means to group variance within the two groups, i.e. we are maximising the between group variation relative to the within group variance - this should sound vaguely like what goes on in ANOVA:

\begin{tabular}{llll}
Source & d.f. & Mean Square & F ratio \\
\hline
Between groups & $m-1$ & $M_{B}$ $M_{B} / M_{W}$\\
Within groups & $N-m$ & $M_{W}$ & \\
 & N-1 & & \\
\end{tabular}

We need to find a linear combination that yields as large an F ratio as possible, hence the coefficients $a_{1}, \ldots, a_{p}$ need to be chosen to maximise this value.   More than one discriminant function is available, there are

\begin{equation}
s = min(p, m-1)
\end{equation}
discriminant functions available, where $p$ is the number of variables, and $m$ is the number of groups.

Considering the case where we have $m > 2$ groups, and $p > 2$ variables, we are looking for the following discriminant functions:

\begin{eqnarray*}
z_{1} &=& a_{11}x_{1} + a_{12}x_{2} + \ldots + a_{1p}x_{p}\\
z_{2} &=& a_{21}x_{1} + a_{22}x_{2} + \ldots + a_{2p}x_{p}\\
\ldots\\
z_{s} &=& a_{s1}x_{1} + a_{s2}x_{2} + \ldots + a_{sp}x_{p}
\end{eqnarray*}

although hopefully only  a small number of these linear combinations will account for all important differences between groups.

\section{Fisher discimination}
%\section{Fisher, linear and quadratic discimination}
\label{fisherdisc}

Remember the $T^{2}$ statistic:

\begin{equation}
T^{2}(\boldsymbol{a}) = \frac{ \left( \boldsymbol{a} (\boldsymbol{\bar{x}}_{1} - \boldsymbol{\bar{x}}_{2} ) \right)^{2} n_{1}n_{2}/(n_{1}+n_{2})}{\boldsymbol{a}^{T}\boldsymbol{S}\boldsymbol{a}}
\end{equation}

This is equivalent to finding $\boldsymbol{a}$ which maximises $|\boldsymbol{a} (\boldsymbol{\bar{x}}_{1} - \boldsymbol{\bar{x}}_{2} ) |$ subject to $\boldsymbol{a}^{T}\boldsymbol{S}\boldsymbol{a}$ = 1.   This has a single solution:

\begin{displaymath}
\boldsymbol{a} = \boldsymbol{S}^{-1}(\boldsymbol{\bar{x}}_{1} - \boldsymbol{\bar{x}}_{2} )
\end{displaymath}

and so the linear discriminant function is given by:

\begin{displaymath}
z = (\boldsymbol{\bar{x}}_{1} - \boldsymbol{\bar{x}}_{2} )^{T}\boldsymbol{S}^{-1} \boldsymbol{x}
\end{displaymath}

In two dimensions, an obvious cut-off would be the midpoint between the mean value of $z$ for group 1 and 2.

Fisher's approach has been extended to cope with more than two groups.  Again, we wish to find a linear combination $z = \boldsymbol{a}^{T} \boldsymbol{x}$ which maximised the ratio of between group variance to within group variance.

If we calculate the within sample matrix of sum of squares and cross products $\boldsymbol{W}$, and the total sample matrix of sum of squares and cross products $\boldsymbol{T}$, we can easily find the between-groups sample matrix sum of squares and cross products:

\begin{equation}
\boldsymbol{B} = \boldsymbol{T} - \boldsymbol{W}
\end{equation}

In effect we wish to maximise:

\begin{equation}
\frac{ \boldsymbol{a}^{T} \boldsymbol{B} \boldsymbol{a}}{ \boldsymbol{a}^{T} \boldsymbol{W} \boldsymbol{a}}
\end{equation}
and usually do this subject to the condition that $\boldsymbol{a}^{T} \boldsymbol{W} \boldsymbol{a}$ = 1.   Fisher's method of discriminant analysis reduces to finding the eigenvalues and corresponding eigenvectors of $\boldsymbol{W}^{-1}\boldsymbol{B}$.   The ordered eigenvalues $\lambda_{1}, \ldots, \lambda_{s}$ are the ratio of between groups to within groups sum of squares and cross products for $z_{1}, \ldots, z_{s}$, the corresponding eigenvectors, $\boldsymbol{a}_{1}, \ldots, \boldsymbol{a}_{s}$, where $\boldsymbol{a}_{i} = \left( \begin{array}{c} a_{i1} \\ \vdots \\ a_{ip} \end{array} \right)$ are the coefficients of $z_{i}$.

We make a number of big assumptions in discriminant analysis: that observations are a random sample, that they are normally distributed and that the variance is the same for each group.   Discriminant analysis is relatively resistant to some departures from the normality assumption - it can cope with skewness but not with outliers.   %Some transformation of the data may be necessary in this situation.   It is also possible to use prior information to deal with unequal sample sizes.

%\begin{displaymath}
%W = \frac{ (\boldsymbol{X} - \boldsymbol{C} \boldsymbol{\bar{X}_{Class}})^{T} (\boldsymbol{X} - \boldsymbol{C} \boldsymbol{\bar{X}_{Class}})}{n-c}
%\end{displaymath}

%\begin{displaymath}
%B  = \frac{  (\boldsymbol{C \bar{X}_{Class}} - \boldsymbol{I \bar{X}})^{T}(\boldsymbol{C \bar{X}_{Class}} - \boldsymbol{I \bar{X}})}{c-1}
%\end{displaymath}

%where $\boldsymbol{\bar{X}_{Class}}$ are the mean values in a group denoted by $\boldsymbol{C}$, and $\boldsymbol{\bar{X}}$ are the overall means for data matrix $\boldsymbol{X}$.   $n$ denotes the number of individuals observed, $c$ the number of classes.

%\begin{displaymath}
%z = \alpha_{1} x_{1} + \alpha_{2} x_{2} + \ldots + \alpha_{p} x_{p}
%\end{displaymath}

%In principle, we need to carry out an eigen decomposition of the matrix $\boldsymbol{W^{-1}B}$ (although in modern computational practice a lot of the details are different, for example some rescaling goes on so that the within-group covariance is set to be $\boldsymbol{I}$).   The linear combination obtained are referred to by Manly as the canonical discriminant functions.   These linear combinations have within group variance $\boldsymbol{a^{T} W a}$ and between group variance  $\boldsymbol{a}^{T} \boldsymbol{B a}$, with the total variance given by:

%\begin{displaymath}
%\boldsymbol{a}^{T} \boldsymbol{S a}  = \frac{(n - g) \boldsymbol{W} + (g - 1) \boldsymbol{B}}{n - 1}
%\end{displaymath}

\section{Accuracy of discrimination}
\label{accuracy}

Clearly, one important measure of the success of our discriminant rule is the accuracy of group prediction: note that there are a number of ways of measuring this and that discriminant analysis is one technique among many used in \emph{supervised classification}.   There are many techniques in machine learning and other areas which are used for classification, for example you have already met the technique of logistic discrimination.

Model over-fitting is a known problem: we can fit a classifier really really well to our existing data but it doesn't work well next time we carry out a data collection exercise.   A key concept in this regard is the use of training and testing sets, where we split our data into two groups, and use one part to build a classifier, and the other to test it.   There are many other technques which can help in this regard, for example leave one out (loo) cross validation and some of the more recent multivariate texts should be consulted.

An important idea in terms of measuring the success of our classifier is the \emph{confusion matrix}.   This sounds rather grand, but is basically a matrix telling us how many times our discriminant function made a correct classification, and how many times it got it wrong.

\section{Importance of variables in discrimination}
\label{imporvar}

Some textbooks refer to questions surrounding selection of variables for use in a classifier.   It is important to consider whether variables are necessary for classification; often a tolerance test may be used prior to the analysis to remove multicollinear and singular variables.   It may even be desirable to carry out a dimension reducing technique.   The reason for carrying out these tests are related to over-fitting.

Some software provides ``standardised'' coefficients, the idea being that perhaps it is safe to remove variables with small standardised coefficients.   However, another approach could well be to consider classifiers with different numbers and combinations of variables and contrast the confusion matrix.   This might help identify variables which do the best job of distinguishing the groups, and those which are the least necessary.

%If $D^{2}$ is large if the discriminant function performs well.
%Wilks' $\Lambda$ is used to assess the importance of variables: the smaller it is the more important the variable is (there are associated F statistics and p values to help in this regard).

\section{Canonical discriminant functions}
\label{candisc}

As mentioned earlier, we can have more than one discriminant function.   It is usual to plot these on a scatterplot in an attempt to visualise the discriminant ability.   However, there are tests of significance.

For example, we wish to find discriminants with a small Wilk's lambda ($\frac{|\boldsymbol{W}|}{|\boldsymbol{T}|}$), in our case this can be derived as :

\begin{equation}
\Lambda^{2} = \left( \sum_{k=1}^{m} n_{k} - 1 - \frac{1}{2}(p + m) \right) \ln (1 + \lambda_{j}),
\end{equation}
which has a $\chi^{2}$ distribution with $p + m - 2j$ degrees of freedom.

%\section{Logistic discrimination}
%\label{logdisc}

%\section{Dimension reduction and discriminant analysis}
%\label{drdisc}

\section{Linear discrimination - a worked example}

In practice, we're going to consider a classification exercise on the Iris data.   This is rather well known data featuring three species of Iris, and four anatomical measures.   First of all we need to load the \texttt{MASS} to obtain the \texttt{lda()} function.   Then we are going to pull out a training set from the Iris data.

\singlespacing
\begin{verbatim}
> library(MASS)
>  data(iris3)
>  Iris <- data.frame(rbind(iris3[,,1], iris3[,,2], iris3[,,3]),
+                         Sp = rep(c("s","c","v"), rep(50,3)))
>      train <- sample(1:150, 75)
\end{verbatim}
\onehalfspacing

\texttt{train} is a set of index numbers which will allow us to extract a training set.   We use \texttt{lda()} to fit a discriminant analysis, setting all priors equal to 1 (i.e. group memberships the same), and \texttt{subset = train} to fit the analysis to the training set.   The squiggle dot indicates that we wish to use all other variables within Iris to predict Species (Sp).

\begin{verbatim}
> z <- lda(Sp ~ ., Iris, prior = c(1,1,1)/3, subset = train)
> z
\end{verbatim}

Having extracted a training set, we are going to classify the remaining Iris' and see how well the predicted and actual species line up.

\singlespacing
\begin{verbatim}
> actual <-  Iris[-train,]$Sp
> preds <- predict(z, Iris[-train, ])$class)
> xtabs(~actual + preds)
\end{verbatim}
\onehalfspacing

One little thing to watch when using software is that in practice, Fisher's approach tends not to be used.   An approach based on probability distributions and using Bayes rule is common.   All this does, is correct for the proportions in each group to start with.   Instead of finding a discriminant rule assuming a 50:50 split, we use information on more plausible group numbers.

%%% Local Variables: ***
%%% mode:latex ***
%%% TeX-master: "../book.tex"  ***
%%% End: ***