\section{Theories and Signatures}
\label{sec:theories-signatures}

\begin{figure*}[t]
\[
\footnotesize
\renewcommand{\arraystretch}{1.0}
\begin{array}[t]{l}
\mbox{\textbf{Theory Elements}}\\
\SET\ s\ [\EQUALS\ \mathit{set}\ ]\\
\CONST\ c\ [\COLON \mathit{set}\ ]\ [\EQUALS\ \mathit{term}\ ]\\
{}[\STABLE{}]\ \RELATION\ r\ [\COLON \mathit{set}\ ]\ [\EQUALS \mathit{prop}\ ]\\
\EQUIVALENCE \COLON \mathit{set}\\
\MODEL\ M\COLON\mathit{theory}\\
\AXIOM\ a\ [\ M\COLON\mathit{theory}\ ]^{*}\ [x\COLON\mathit{set}\ ]^*\EQUALS\mathit{prop}\\
\\
\mbox{\textbf{Propositions}}\\
\TRUE\\
\FALSE\\
\NOT\ \Prop\\
\Prop \AAND \Prop\\
\Prop \OOR \Prop\\
\Prop \IIMPLY \Prop\\
\Prop \IIFF \Prop\\
r [\ \Term\ ]^*\\
\Term\EQUALS\Term\\ %[\IN\ \Set]\\
\ALL\ [x\COLON \Set] \PERIOD \Prop\\
\SOME\ [x\COLON \Set] \PERIOD \Prop\\
\UNIQUE\ [x\COLON \Set] \PERIOD \Prop\\
\end{array}
\qquad
\begin{array}[t]{l}
\mbox{\textbf{Sets}}\\
\ZERO\\
\ONE\\
\BOOL\\
s\\
\mathit{Model}\PERIOD \mathit{name}\\
\Set \TIMES \cdots \TIMES \Set\\
\Setexp \ARROW \Setexp\\
\Label\ [\COLON \Setexp\ ]\ \PLUS \cdots \PLUS \Label\ [\COLON \Setexp\ ]\\
\LBRACE \Ident [\ \COLON \Setexp\ ]\ \BAR \Proposition \RBRACE\\
\Setexp \PERCENT \metav{relation}\\
%\RZ\ \Setexp\\
\\
\mbox{\textbf{Terms}}\\
x\\
\LPAREN \Term\COMMA \cdots \COMMA \Term \RPAREN\\
\Term\PERIOD \metav{n}\\
\Label\ [\ \Term\ ]\\
\MATCH\ \Term\ \WITH\ \mbox{\textit{pattern-matches}}\\
\LAMBDA\ x\COLON\Set\ \PERIOD\ \Term\\
\Term\ \Term\\
\Term \PERCENT \metav{relation}\\
\LET\ x \PERCENT \metav{relation}\ \IN\ \Term\EQUALS\Term\\
\Term \SUBIN \Set\\
\Term \SUBOUT \Set\\
\THE\ x\ [\COLON\Set\ ] \PERIOD \Prop\\
\LET\ x\ [\COLON\Set\ ] \EQUALS\Term\ \IN\ \Term
\end{array}
\]
\vspace{-0.6truecm}
\caption{Input Language Summary}
\label{fig:input}
\end{figure*}

In this section we describe first-order theories and signatures.
Our system translates the former into the later.

\subsection{Theories}
\label{sec:theories}

A \emph{theory} is a description of a mathematical structure, such as
a group, a vector space, a directed graph, etc. A theory consists of
%
\begin{itemize}
\item a list of \emph{basic sets},
\item a list of \emph{basic constants} belonging to specified sets,
\item a list of \emph{basic relations} on specified sets,
\item a list of axioms.
\end{itemize}
%
To take a simple example, consider the theory of a semigroup in which
every element has a (possibly non-unique) square root; recall that a
semigroup is a set with an associative binary operation and a neutral
element.\footnote{An example of a semigroup with square roots is the
  complex numbers with multiplication as the binary operation.} In our
system it could be written as follows:
\vspace{-0.2truecm}
{
\small
\VerbatimInput{semigroup.thy}
}
\goodbreak\goodbreak
\noindent The theory is enclosed by \Verb|thy|\ldots\Verb|end|. This theory
defines one basic set \Verb|s|, and two basic constants: an element
\Verb|e| of \Verb|s| and a (curried) binary infix operator \Verb|*| on
the set \Verb|s|. The \Verb|implicit| operator is not part of the
theory proper, but signals to the type checker that bound
variables named \Verb|x| or \Verb|y| or \Verb|z| should be assumed to
range over \Verb|s| unless otherwise specified. Finally,
we have three axioms. Axiom arguments, e.g., \Verb|x|, \Verb|y|, and
\Verb|z| in the associativity axiom, name the free variables occuring
in the axiom. It is not too big a mistake to think of them as being
universally quantified.

It is important to note that theories do not include proofs, but
rather just the statements of the axioms (and theorems) specified to
hold. Thus although axioms can be defined, one cannot actually refer
to them within the theory.

There are several features of theories that our system supports other
than those shown in this example above; the input language is
summarized in Figure~\ref{fig:input}, where brackets imply optional
elements.

Theories may declare or define relations. They may be \Verb|stable|,
i.e., their computational interpretation is trivial (see
Section~\ref{sec:implementation} for further discussion of this
point). Axioms can universally quantify over all models of a theory.
This is useful for describing universality properties, such as
initiality of an algebra or finality of a coalgebra.

The propositions are the familiar ones from first-order logic;
$\UNIQUE$ is unique existence ($\exists!$). In addition to the basic
empty ($\ZERO$) and unit ($\ONE$) sets, one can form cartesian
products, function spaces, tagged disjoint unions, subsets, and
quotients by stable equivalence relations. The corresponding
introduction and elimination forms appear in the language of terms.
For example, $\Term \PERCENT \metav{relation}$ is the equivalence
class under $\metav{relation}$ containing $\Term$, while $\LET\ x
\PERCENT \metav{relation} \mbox{\Verb| = |} \Term_1\ \IN\ \Term_2$
binds $x$ to a representative of the equivalence class $\Term_1$ to be
used in $\Term_2$. The expression $\Term \SUBIN \Set$ injects $\Term$
into a given subset (recording a proof obligation of the term actually
being a member of the subset), while $\Term \SUBOUT \Set$ projects
$\Term$ from a subset out into its superset $\Set$. The value of the
description operator $\THE\ x \,.\, \Prop$ is the unique $x$
satisfying $\Prop$; using it incurs the obligation of proving that
there is exactly one such~$x$.

\subsection{Signatures}
\label{sec:signatures}

On the logical side, we have models described by theories.  Thus on
the programming side we should have implementations being described by
specifications.  Our tool thus translates theories into
\emph{signatures}, which are ML's module interfaces.

Signatures allow us to require the existence of certain types, as well
as values of given type.  This allows decidable typechecking, but we
need more expressiveness in order to faithfully translate the content
of a theory.  We therefore generate signatures augmented by assertion
comments, which specify constraints on the values and functions an
implementation beyond their type.  It is the responsibility of the
programmer to check that the implementation satisfies these
assertions, as \RZ does not attempt to do any theorem proving.

Assertions are written in ordinary classical first-order logic. Since
programmers typically are not trained in constructive logic, this may
make it easier to verify the assertions.
\goodbreak
The output for the theory \Verb|SQGROUP| above is then:
{\small \VerbatimInput{semigroup.mli}}

At the ML level we have required a type \Verb|s|, and three values
\Verb|e|, \Verb|*|, and \Verb|sqrt|, of types \Verb|s|,
\Verb|s->s->s|, and \Verb|s->s|, respectively. The third value was
generated from the square root axiom, which has a non-trivial
computational content, cf.\ Subsection~\ref{subsec:real-transl}.

Comments contain other requirements, not expressible in ML, that
further contstrain the allowed implementations. The assertion
\Verb|PER(=s=)| abbreviates the requirement that \Verb|=s=| be a
partial equivalence relation on~\Verb|s|; its domain \Verb+||s||+ is
the subset of terms of type \Verb|s| that realize semigroup
elements, and the relation \Verb|=s=| identifies (possibly different)
terms realizing the same abstract semigroup element. These data
together determine a modest set. The assertion following the
declarations of \Verb|e| asserts that \Verb|e| realizes a valid
semigroup element, and the one following \Verb|*| asserts that
\Verb|*| must not be affected by the choice of realizers. Both
\Verb|e| and \Verb|*| must of course still satisfy the \Verb|unit| and
\Verb|assoc| axioms. Finally, the new function \Verb|sqrt| derived
from the logic must compute square roots. Since the theory requires
existence but not uniqueness of square roots, there is no requirement
that \Verb|sqrt| be invariant with respect to the partial equivalence
relation on \Verb|s|; different realizers of the same semigroup
element are allowed to produce (realizers of) different square roots.

\subsection{Parameterized Theories}
\label{sec:param-theor-funct}

A theory may be parameterized by one or more models of other theories.
For example, a theory \Verb|Real| of the reals may be parameterized in
terms of a model \Verb|N| of the naturals.  A theory of free groups may be
parameterized in terms of the generating set.

Parameterized theories serve two purposes.  A model of a
parameterized theory is a generic implementation that, given any
implementation of the parameters, returns an implementation of the
resulting theory.  At the level of ML, this would be a function from
modules to modules, a so-called \emph{functor}, and so a
parameterized theory can be translated into the signature of a
functor.

Alternatively, once we have described a parameterized theory
\Verb|Real|, we may wish to use it to describe a single specific
implementation of real numbers based on a specific model \Verb|N1|
(implementation) of the natural numbers; this can be described as
an implementation satisfying the theory \Verb|Real(N1)|.

The dual nature of parameterized theories as being both a description
of a parameterized model (a $\Pi$ type) and something which can be
applied to a model to produce a specialized theory (a $\lambda$) is
very reminiscent the type inclusion of Automath~\cite{automath}.  ML
does not permit applications of functor signatures, however, so we
beta-reduce all theory applications before generating signatures;
\Verb|Real(N1)| would produce a signature for a real-number
implementation that refers directly to \Verb|N1| rather than to a
generic parameter \Verb|N|.

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "case"
%%% End:


\section{Basic elements of  Lie groups and Lie algebras theory.}
Let us recall the definitions of the  Lie group Theory taken from \cite{Iserles.ea_AN2000} and \cite{Varadarajan_book1984}.

\subsection{Differential equation (evolving) on a manifold $\mathcal M$}
\begin{definition}
 A $d$-dimensional manifold $\mathcal M$ is a $d$-dimensional smooth surface $ M\subset \RR^n$ for some $n\geq d$.
\end{definition}
\begin{definition}
  Let $\mathcal M$ be a $d$-dimensional manifold and suppose that $\rho(t) \in\mathcal M$  is a smooth curve such that $\rho(0) = p$. A tangent vector at $p$ is defined as
  \begin{equation}
    \label{eq:12}
    a = \left. \frac{d}{dt} (\rho(t)) \right|_{t=0}.
  \end{equation}
The set of all tangents at $p$ is called the tangent space at $p$ and denoted by $T\mathcal M|_p$. It has the structure of a linear space.
\end{definition}
\begin{definition}
   A (tangent) vector field on $\mathcal M$ is a smooth function $F : \mathcal M \rightarrow T\mathcal M$ such that $F (p) \in T\mathcal M|_p$ for all $p \in \mathcal M$. The collection of all vector fields on $\mathcal M$ is denoted by $\mathcal X(\mathcal M)$.
 \end{definition}

 \begin{definition}[Differential equation (evolving) on $\mathcal M$]
   Let $F$ be a tangent vector field on $\mathcal M$. By a differential equation (evolving) on $\mathcal M$ we mean a differential equation of the form
   \begin{equation}
     \dot y =F(y), t\geq  0, y(0)\in \mathcal M\label{eq:13}
   \end{equation}
   where $F \in \mathcal X(\mathcal M)$. Whenever convenient, we allow $F$ in~\eqref{eq:13} to be a function of time, $F = F(t,y)$. The flow of $F$ is the solution operator $\Psi_{t,F} : \mathcal M \rightarrow  \mathcal M$ such that
   \begin{equation}
     y(t) = \Psi_{t,F} (y0).\label{eq:14}
   \end{equation}
 \end{definition}

 \subsection{Lie algebra and Lie group}
 \begin{definition}[commutator]
   Given two vector fields $F, G$ on $\RR^n$ , the commutator $H = [F, G]$ can
   be computed componentwise at a given point $y ∈ \RR^n$ as
   \begin{equation}
     H_i(y)= \sum_{j=1}^n  G_j(y)\frac{\partial F_i(y)}{\partial y_j}   −F_j(y) \frac{\partial G_i(y)}{\partial y_j} .\label{eq:15}
   \end{equation}
 \end{definition}

 \begin{lemma}\label{lemma:LieBracket}
The commutator of vector fields satisfies the identities
\begin{equation}
  \label{eq:16}
  \begin{array}[lclr]{lclr}
    \protect{[}F, G]&=& −\protect{[}G, F ] & (skew symmetry), \\
    \protect{[} \alpha F,G] &=& \alpha \protect{[}F,G], \alpha \in \RR &  \\
    \protect{[}F + G, H]&=& \protect{[}F, H] + \protect{[}G, H] & (bilinearity),\\
    0 &=&  \protect{[}F,\protect{[}G,H]]+\protect{[}G,\protect{[}H,F]]+\protect{[}H,\protect{[}F,G]] &(Jacobi’s identity).
  \end{array}
\end{equation}
\end{lemma}
\begin{definition}
  A Lie algebra of vector fields is a collection of vector fields which is closed under linear combination and commutation. In other words, letting $\mathfrak g$ denote the Lie algebra,

  \begin{equation}
    \begin{array}[lclr]{l}
    B \in \mathfrak g \implies \alpha B \in \mathfrak  g \text{ for all } \alpha ∈ R .\\
    B_1,B_2 \in\mathfrak g \implies B_1 +B_2, [B_1,B_2]\in\mathfrak g\label{eq:17}
    \end{array}
\end{equation}

Given a collection of vector fields $B = {B_1 , B_2 , \ldots}$, the least Lie algebra of vector fields containing $B$ is called the Lie algebra generated by $B$
\end{definition}

\begin{definition}
  A Lie algebra is a linear space $V$ equipped with a Lie bracket, a bilinear, skew-symmetric mapping
  \begin{equation}
    \label{eq:18}
    [ \cdot , \cdot ] : V \times V \rightarrow V
  \end{equation}
that obeys identities \eqref{eq:16} from Lemma~\ref{lemma:LieBracket}
\end{definition}

\begin{definition}[(General) Lie algebra]
  A Lie algebra homomorphism is a linear map between two Lie algebras, $\varphi : \mathfrak g \rightarrow \mathfrak h$, satisfying the identity
  \begin{equation}
\varphi ([v, w]_{\mathfrak g}) = [\varphi(v), \varphi(w)]_{\mathfrak h}, v, w in \mathfrak g\label{eq:19}.
\end{equation}
An invertible homomorphism is called an isomorphism.
\end{definition}

\begin{definition}
  A Lie group is a differential manifold $\mathcal G$ equipped with a product $\glaw : \mathcal G\times \mathcal G →\rightarrow \mathcal  G$ satisfying
  \begin{equation}
    \label{eq:20}
    \begin{array}[lclr]{lr}
      p \glaw(q \glaw r) = (p\glaw q)\glaw r, \forall  p, q, r ∈ \mathcal G &\text{(associativity)}\\
      \exists I \in \mathcal G \text{ such that } I\glaw p = p \glaw I = p,  \forall p \in \mathcal G&\text{(identity element)}\ \\
      \forall p \in \mathcal G, \exists  p^{-1}  \in \mathcal G \text{ such that }  p^{-1}\glaw p = I&\text{(inverse) }\ \\
      \text{ The maps}  (p, r)  \rightarrow p\glaw r \text{ and }  p  \rightarrow p^{-1} \text{are smooth functions }&\text{(smoothness)}\
    \end{array}
\end{equation}
\end{definition}

\begin{definition}[Lie algebra $\mathfrak g $ of a Lie group $\mathcal G$]
  The Lie algebra $\mathfrak g$ of a Lie group $\mathcal G$ is defined as the linear space of all tangents to $G$ at the identity $I$. The Lie bracket in $\mathfrak g$ is defined as
  \begin{equation}
    [a,b]= \left.\frac{\partial^2 }{\partial s\partial t} \rho(s)\sigma(t)\rho(-s)\right|_{s=t=0}\label{eq:21}
\end{equation}
where $\rho(s)$ and $\sigma(t)$ are two smooth curves on $\mathcal G$ such that $\rho(0) = \sigma(0) = I$, and
$\dot \rho(0) = a$ and $\dot \sigma(0) = b$.
\end{definition}

\subsection{Actions of a group $\mathcal G$ on  manifold $\mathcal M$}
\begin{definition}
   A left  action of Lie Group $\mathcal G$ on a manifold $\mathcal M$ is a smooth map $\Lambda^l: \mathcal G \times  \mathcal M \rightarrow \mathcal M$ satisfying
\begin{equation}
  \label{eq:22}
  \begin{array}[lcl]{rcl}
    \Lambda^l(I,y) &=& y, \quad \forall y \in \mathcal M \\
    \Lambda^l(p,\Lambda(r,y)) &=& \Lambda^l(p\glaw r, y) , \quad \forall p,r \in \mathcal G,\quad  \forall y \in \mathcal M .
  \end{array}
\end{equation}
\end{definition}

\begin{definition}
   A  right  action of Lie Group $\mathcal G$ on a manifold $\mathcal M$ is a smooth map $\Lambda^r: \mathcal M \times \mathcal G   \rightarrow \mathcal M$ satisfying
\begin{equation}
  \label{eq:23}
  \begin{array}[lcl]{rcl}
    \Lambda^r(y,I) &=& y, \quad \forall y \in \mathcal M \\
    \Lambda^r(\Lambda(y,r), p) &=& \Lambda^r(y,  r\glaw p) , \quad \forall p,r \in \mathcal G,\quad  \forall y \in \mathcal M .
  \end{array}
\end{equation}
\end{definition}

A given smooth curve  $S(\cdot) : t\in \RR \mapsto S(t)\in \mathcal G$ in $\mathcal G$ such that $S(0)= I$ produces a flow $\Lambda^l(S(t),\cdot)$ (resp. $\Lambda^r(\cdot, S(t))$) on $\mathcal M$ and by differentiation we find a tangent vector field
\begin{equation}
  \label{eq:24}
  F(y) = \left. \frac{d}{dt} (\Lambda^l(S(t),y) \right|_{t=0}\quad( \text{resp.  }  F(y) = \left. \frac{d}{dt} (\Lambda^r(y,S(t)) \right|_{t=0} )
\end{equation}
that defines a ordinary differential equation on a Lie Group
\begin{equation}
  \label{eq:25}
  \dot y(t) = F(y(t)) = \left. \frac{d}{dt} (\Lambda^l(S(t),y) \right|_{t=0}  \quad( \text{resp.  }\dot y(t) = F(y(t)) = \left. \frac{d}{dt} (\Lambda^r(y,S(t)) \right|_{t=0})
\end{equation}

\begin{lemma}
  Let $\lambda^l_{*} : \mathfrak g \rightarrow \mathcal X(\mathcal M) $ (resp. $\lambda^r_{*} : \mathfrak g \rightarrow \mathcal X(\mathcal M) $ be defined as
  \begin{equation}
  \lambda^l_{*}(a)(y) = \left.\frac{d}{ds}{ \Lambda^l (\rho(s), y)}\right|_{s=0} \quad (\text{ resp. }  \lambda^r_{*}(a)(y) = \left.\frac{d}{ds}{ \Lambda^r (y, \rho(s))}\right|_{s=0})\label{eq:26}
\end{equation}
 where $\rho(s)$ is a curve in $\mathcal G$ such that $\rho(0)=I$ and $\dot\rho (0)=a$. Then $\lambda^l_{8}$ is a linear
map between Lie algebras such that
\begin{equation}
  [a, b]_{\mathfrak g} = [\lambda^l_{*}(a), \lambda^l_{*}(b)]_{\mathcal X(\mathcal M)}.\label{eq:27}
\end{equation}
\end{lemma}

The following product between an element of an algebra $a \in \mathfrak g$ with an element of a group $\sigma  \in \mathcal G$
 can be defined. This will served as a basis for defining the exponential map.
\begin{definition}
  We define the left product $(\cdot, \cdot)^l : \mathfrak g \times \mathcal G \rightarrow  \mathcal G$ of an element of an algebra $a \in \mathfrak g$ with an element of a group $\sigma  \in \mathcal G$ as
  \begin{equation}
 (a, \sigma)^l = a \cdot \sigma = \left.\frac{d}{ds} \rho(s) \glaw \sigma \right|_{s=0}\label{eq:28}
\end{equation}
where $\rho(s)$ is a smooth curve such that $\dot\rho(0)=a$ and $\rho(0)=I$. In the same way, we can define the right product $(\cdot, \cdot)^r : \mathcal G \times \mathfrak g  \rightarrow   \mathcal G$
\begin{equation}
  \label{eq:29}
  (\sigma,a)^r = \sigma \cdot a  = \left.\frac{d}{ds} \sigma \glaw \rho(s)   \right|_{s=0}
\end{equation}
\end{definition}

\subsection{Exponential map}
\begin{definition}
  Let $\mathcal G$ be a Lie group and $\mathfrak g$ its Lie algebra. The exponential mapping $exp : \mathfrak g \rightarrow \mathcal G$ is defined as $\exp(a) = \sigma(1)$ where $\sigma (t)$ satisfies the  differential equation
\begin{equation}
\dot \sigma(t) = a \cdot \sigma(t), \quad \sigma (0) = I.\label{eq:30}
\end{equation}
\end{definition}

Let us define $a^k$ as
\begin{equation}
  \label{eq:31}
  \left\{\begin{array}[l]{l}
    a^k = \underbrace{a\glaw a \glaw \ldots a\glaw a}_{k \text{ times}} \text{ for } k \geq 1 \\
    a^0  = I
  \end{array}\right.
\end{equation}
The exponential map can be expressed as
\begin{equation}
  \label{eq:32}
  \exp(at) = \sum_{k=0}^\infty \frac{(ta)^k}{k!}
\end{equation}
since it is  a solution of \eqref{eq:30}. A simple computation allows to check this claim:
\begin{equation}
  \label{eq:33}
   \frac{d}{dt}\exp(at) = \sum_{k=1}^\infty  k t^{k-1} \frac{a^k}{k!} = a \glaw \sum_{k=0}^\infty  t^{k} \frac{a^k}{k!} = a \glaw \exp(at).
\end{equation}
A similar computation gives
\begin{equation}
  \label{eq:34}
  \frac{d}{dt}\exp(at)  = \sum_{k=0}^\infty  t^{k} \frac{a^k}{k!} \glaw a = \exp(at) \glaw a.
\end{equation}
The exponential mapping $exp : \mathfrak g \rightarrow \mathcal G$ can also be defined as $\exp(a) = \sigma(1)$ where $\sigma (t)$ satisfies the  differential equation
\begin{equation}
  \label{eq:35}
  \dot \sigma(t) = \sigma(t) \cdot a, \quad \sigma (0) = I.
\end{equation}

\begin{theorem}
  \label{Theorem:solutionofLieODE}
  Let $\Lambda^l:\mathcal G\times\mathcal M \rightarrow \mathcal M$ be a left  group action and $\lambda^l_{∗} : \mathfrak g\rightarrow \mathcal X(\mathcal M)$ the corresponding Lie algebra homomorphism. For any $a \in \mathfrak g$ the flow of the vector field $F = \lambda^l_{a}(a)$, i.e. the solution of the equation
  \begin{equation}
    \dot y(t) = F(y(t)) = \lambda^l_{*}(a)(y(t)),\quad  t \geq 0, y(0) = y_0 \in \mathcal M,\label{eq:36}
\end{equation}
  is given as
  \begin{equation}
y(t) = \Lambda^l(\exp(ta), y_0).\label{eq:37}
\end{equation}
Let $\Lambda^r:\mathcal M\times\mathcal G \rightarrow \mathcal M$ be a right group action and $\lambda^r_{∗} : \mathfrak g\rightarrow \mathcal X(\mathcal M)$ the corresponding Lie algebra homomorphism. For any $a \in \mathfrak g$ the flow of the vector field $F = \lambda^r_{*}(a)$, i.e. the solution of the equation
  \begin{equation}
    \dot y(t) = F(y(t)) = \lambda^r_{*}(a)(y(t)),\quad  t \geq 0, y(0) = y_0 \in \mathcal M,\label{eq:38}
\end{equation}
  is given as
  \begin{equation}
y(t) = \Lambda^r(y_0,\exp(ta)).\label{eq:39}
\end{equation}

\end{theorem}

\subsection{Translation (Trivialization) maps}
The left and right translation maps defined by
\begin{equation}
  \label{eq:148}
  \begin{array}{rcl}
    L_z  : \mathcal G \times \mathcal G &\rightarrow& \mathcal G \quad \text{ (left translation map )} \\
    y &\mapsto&  z \glaw y
  \end{array}
\end{equation}
and
\begin{equation}
  \label{eq:149}
  \begin{array}{rcl}
    R_z(y)  :  \mathcal G \times  \mathcal G  & \rightarrow& \mathcal G \quad \text{ (right translation map )} \\
    y  &\mapsto&  y \glaw z
  \end{array}
\end{equation}

If we identify the manifold $\mathcal M$ with the group $\mathcal G$, The left and right translations can be interpreted as the simplest example of group action on the manifold. Note that the left translation map can be viewed as a left or right action on the group.

If we consider $L_z(y)$ as a right group action $ L_z(y) = \Lambda^r( z, y) =z \glaw y $, by differentiation we get a $L'_z : T \mathfrak g \cong  \mathfrak g \rightarrow T_z\mathcal G$ with $\dot\rho (0)=a$ such that
\begin{equation}
  \label{eq:150}
  \lambda^r_{*}(a)(z) = L'_z(a) = \left.\frac{d}{ds}{ \Lambda^r (z, \rho(s))}\right|_{s=0} = z \glaw a
\end{equation}
The map
\begin{equation}
  \label{eq:152}
  \begin{array}{rcl}
  L'_z  : \mathfrak g &\rightarrow& T_z\mathcal G  \\
         a &\mapsto&  z \glaw a
  \end{array}
\end{equation}
determines an isomorphism of $\mathfrak g$ with the tangent space  $T_z\mathcal G$. In other words, the  tangent space can be identified to $\mathfrak g$ as
\begin{equation}
  \label{eq:153}
  T_z\mathcal G =\{L'_z(a) = z \glaw a \mid a \in \mathfrak g  \}
\end{equation}

Respectively, if we consider $R_z(y)$ as a left group action $ R_z(y) = \Lambda^l( y, z) =y \glaw z $, by differentiation we get a $R'_z : T \mathfrak g \cong  \mathfrak g \rightarrow T_z\mathcal G$ with $\dot\rho (0)=a$ such that
\begin{equation}
  \label{eq:150}
  \lambda^l_{*}(a)(z) = R'_z(a) = \left.\frac{d}{ds}{ \Lambda^l (\rho(s),z)}\right|_{s=0} = a \glaw z
\end{equation}
The map
\begin{equation}
  \label{eq:152}
  \begin{array}{rcl}
  R'_z  : \mathfrak g &\rightarrow& T_z\mathcal G  \\
         a &\mapsto&  a \glaw z
  \end{array}
\end{equation}
determines an isomorphism of $\mathfrak g$ with the tangent space  $T_z\mathcal G$. In other words, the  tangent space can be identified to $\mathfrak g$ as
\begin{equation}
  \label{eq:153}
  T_z\mathcal G =\{R'_z(a) = a \glaw z \mid a \in \mathfrak g  \}
\end{equation}
Any tangent vector $F : \mathcal G \rightarrow T_z\mathcal G$ can be written in either of the forms
\begin{equation}
  \label{eq:155}
  F(z) = L'_z(f(a)) = R'_z(g(z))
\end{equation}
where $f,g \mathcal G \rightarrow \mathfrak g$.
\subsection{Adjoint representation}
\begin{definition}
Let $p \in \mathcal G$ and let $\sigma (t)$ be a smooth curve on $\mathcal G$ such that $\sigma (0)$ = I and $\dot \sigma(0) = b \in \mathfrak g$. The adjoint representation is defined as
\begin{equation}
\Ad_p(b) =\left. \frac{d}{dt} p\sigma(t)p^{-1}\right|_{t=0}\label{eq:40}
\end{equation}
The derivative of $\Ad$ with respect to the first argument is denoted $\ad$. Let $\rho(s)$ be a smooth curve on $\mathcal G$ such that $\rho(0) = I$  and $\dot \rho(0) = a$, it  yields:
\begin{equation}
  \label{eq:41}
    \ad_a(b) = \left.\frac{d}{ds} \Ad_{\rho(s)}(b)\right|_{s=0}  = [a, b]
\end{equation}
\end{definition}
The adjoint representation can also be expressed with the map
\begin{equation}
  \label{eq:154}
  \Ad_p(b)  = (L_p \glaw R_{p^{-1}})' (b) = (L'_p \glaw R'_{p^{-1}}) (b) = p \glaw b \glaw p^{-1}
\end{equation}

For a tangent vector given in~\eqref{eq:155}, we have
\begin{equation}
  \label{eq:151}
  g(z) = Ad_z(f(z))
\end{equation}
Another important relation relating $\Ad$, $\ad$ and $\exp$ is
\begin{equation}
  \label{eq:164}
  \Ad_{\exp(a)} =\exp{\ad_a}
\end{equation}

\subsection{Differential of the exponential map} There are multiple ways to represent the differential of $\exp(\cdot)$ at a point $a\in \mathfrak g$. Let us start by the following definition of the differential map at $a\in\mathfrak g$
\begin{equation}
  \label{eq:147}
  \begin{array}{lcl}
    \exp_a' & : & \mathfrak g \rightarrow  T_{exp(a)}\mathcal G\\
            & &  v \mapsto \exp'_a(v)  = \left.\frac{d}{dt} \exp(a+tv)\right|_{t=0}
  \end{array}
\end{equation}
The definition is very similar to the definition of the directional derivative of $\exp$ in the direction $v \in \mathfrak g$ at a point $a\in\mathfrak g$. Using the expression \eqref{eq:153} of the tangent space at $\exp(a)$, we can defined another expression of the differential map denoted as $\dlexp_a : \mathfrak g  \rightarrow \mathfrak g$ such that
\begin{equation}
  \label{eq:156}
  \dlexp_a = L'_{\exp^{-1}(a)} \glaw \exp_a' = L'_{\exp(-a)} \glaw \exp_a'
\end{equation}
This expression appears as a trivialization of the differential map $\exp'_a$. Using the expression of $L'_z$ in \eqref{eq:152}.
In~\cite[Theorem 2.14.13]{Varadarajan_book1984}, an explicit formula relates $\dlexp_{a}$ to the iteration of the adjoint operator:
\begin{equation}
  \label{eq:43}
  \dlexp_a(b) = \sum_{k=0}^\infty \frac{(-1)^k}{(k+1)!} (\ad_a(b))^k \coloneqq \frac{e - \exp\glaw\ad_a}{\ad_a}(b)
\end{equation}
where $(\ad_a)^k$ is the kth iteration of the adjoint operator:
\begin{equation}
  \label{eq:44}
  \left\{\begin{array}[l]{l}
    (\ad_a)^k(b) = \underbrace{[a, [ a, [ \ldots, a, [ a, b]]]}_{k \text{ times}} \text{ for } k \geq 1 \\
    (\ad_a)^0(b)  = b
  \end{array}\right.
\end{equation}
It is also possible to define the right trivialized differential of the exponential map
\begin{equation}
  \label{eq:162}
  \drexp_a = R'_{\exp^{-1}(a)} \glaw \exp_a' = R'_{\exp(-a)} \glaw \exp_a'
\end{equation}
that is
\begin{equation}
  \label{eq:163}
  \drexp_a(b) = \exp'_a(b) \glaw \exp(-a)
\end{equation}
With these expression, we have equivalently for
\begin{equation}
  \label{eq:157}
   \exp_a'(b)  = \exp_a \glaw \dlexp_a(b)\quad \text{ and } \exp_a'(b)  = \drexp_a(b) \glaw   \exp(a)
\end{equation}

To avoid to burden to much the notation, we introduced the unified definition of the differential map  that corresponds to $\dexp=\drexp$
\begin{definition}
The differential of the exponential mapping, denoted by $\dexp_a : \mathfrak g \times \mathfrak g \rightarrow \mathfrak g$ is defined as the ``right trivialized'' tangent of the exponential map
\begin{equation}
  \label{eq:42}
  \frac{d}{dt} (\exp(a(t))) = \dexp_{a(t)}(a'(t)) \exp(a(t))
\end{equation}
\end{definition}
An explicit formula relates $\dexp_{a}$ to the iteration of the adjoint operator:
\begin{equation}
  \label{eq:43}
  \dexp_a(b) = \sum_{k=0}^\infty \frac{1}{(k+1)!} (\ad_a(b))^k \coloneqq \frac{\exp\glaw\ad_a-e}{\ad_a}(b)
\end{equation}

\begin{ndrva}
  Say what is not the Jacobian in $\RR^4$
\end{ndrva}

As for $\Ad_a$ and $\ad_a$, the mapping $\dexp_{a}(b)$ is a linear mapping in its second argument for a fixed $a$. Using the relation~\eqref{eq:164}, we can also relate the right and the lest trivialization tangent
\begin{equation}
  \label{eq:165}
\dlexp_a (b) =   (\Ad_{\exp(a)} \glaw \dexp(a))(b) = (\exp(\ad_{-a}) \glaw \frac{e - \exp\glaw\ad_a}{\ad_a})(b) = \frac{e - \exp\glaw\ad_{-a}}{\ad_a}(b) = \dexp_{-a}(b)
\end{equation}
It is also possible to define the  the ``left trivialized'' tangent of the exponential map
\begin{equation}
  \label{eq:46}
   \frac{d}{dt} (\exp(a(t))) =  \exp(a(t)) \dlexp_{a(t)}(a'(t)) = \exp(a(t)) \dexp_{-a(t)}(a'(t))
\end{equation}

\begin{ndrva}
  other notation and Lie derivative
  \begin{equation}
    \label{eq:178}
      Df \cdot \widehat \Omega (p) = (\widehat \Omega^r f )(p)
  \end{equation}
\end{ndrva}

\paragraph{Inverse of the exponential map}

The function $\dexp_{a}$ is an analytical function so it possible to invert it to get
\begin{equation}
  \label{eq:45}
  \dexp^{-1}_{a} = \sum_{k=0}^\infty \frac{B_k}{(k)!} (\ad_a)^k(b)
\end{equation}
where $B_k$ are the Bernouilli number.

\subsection{Differential of a map $f : \mathcal G \rightarrow \mathfrak g$}

We follow the notation developed in~\cite{Owren.Welfert_BIT2000}. Let us first define the differential of the map $f : \mathcal G \rightarrow \mathfrak g$ as
\begin{equation}
  \label{eq:166}
  \begin{array}[rcl]{rcl}
    f'_z : T_z\mathcal G &\rightarrow&T_{f(z)}\mathfrak g \cong  \mathfrak g\\
    b &\mapsto& \left.\frac{d}{dt} f(z\glaw \exp(t L'_{z^{-1}}(b))) \right|_{t=0}
  \end{array}
\end{equation}
The image of $b$ by $f'_z$   is obtained by first identifying $b$ with an element of $v \in \mathfrak g$ thanks to the left representation of $T_{f(z)}\mathfrak g$ view the left translation map $v= t L'_z(b)$. The exponential mapping transforms $v$ an element $y$ of the Lie Group $\mathcal G$. Then $f'_z$ is obtained by
\begin{equation}
  \label{eq:167}
  f'_z(b) = \lim_{t\rightarrow 0} \frac{f(z\glaw y) - f(z)}{t}
\end{equation}
As we have done for the exponential mapping, it is possible to get a left trivialization of
\begin{equation}
  \label{eq:169}
  \dd f_z = (f\glaw L_z)' = f'_z \glaw L'_z
\end{equation}
thus
\begin{equation}
  \label{eq:170}
  \dd f_z (a) =  f'_z \glaw L'_z(a) = f'_z(L'_z(a)) =  \left.\frac{d}{dt} f(z\glaw \exp(t a )) \right|_{t=0}
\end{equation}

\paragraph{Newton Method}
Let us imagine that we want to solve $f(y) = 0 $ for $y \in \mathcal G$. A newton method can be written as
%%% Local Variables:
%%% mode: latex
%%% TeX-master: "DevNotes"
%%% End:


\newpage
\section{Asymptotic convergence estimates for $H^m$ norm}
In this section, we will extend the previous asymptotic approximation analysis
to $H^k$ Sobolev norm.

\subsection{Asymptotic error estimates}
\begin{theorem}\label{approximation_rate_theorem}
 Let $\Omega\subset \mathbb{R}^d$ be a bounded domain. If the activation function $\sigma\in W^{m,\infty}(\mathbb{R})$ is non-zero and satisfies the polynomial decay condition
 \begin{equation}\label{growth_condition}
  |\sigma^{(k)}(t)| \leq C_p(1 + |t|)^{-p}
 \end{equation}
 for $0\leq k\leq m$ and some $p > 1$, we have
 \begin{equation}
  \inf_{f_n\in \dnn(\sigma,n)}\|f - f_n\|_{H^m(\Omega)} \leq |\Omega|^{\frac{1}{2}}C(p,m,\text{\normalfont diam}(\Omega),\sigma)n^{-\frac{1}{2}}\|f\|_{\mathcal{B}^{m+1}},
 \end{equation}
 for any $f\in \mathcal{B}^{m+1}$.
\end{theorem}
Before we proceed to the proof, we discuss how this bound depends on
the dimension $d$. We first note that $|\Omega|$ may in a sense depend
on the dimension, as the measure may be exponentially large in high
dimensions. However, bounding the $H^m$ error over a larger set is
also proportionally stronger. This can be seen by noting that dividing
by the $|\Omega|^\frac{1}{2}$ factor transforms the left hand side
from the total squared error to the average squared error.

The dimension dependence of this result is a consequence of how the
Barron norm behaves in high dimensions. This issue is discussed in
\cite{barron1993universal}, where the norm $\|\cdot\|_{\mathcal{B}^1}$
is analyzed for a number of different function classes. A particularly
representative result found there is that $H^{\frac{d}{2}+2}\subset
\mathcal{B}^1$. This shows that sufficiently smooth functions have
bounded Barron norm, where the required number of derivatives depends
upon the dimension. It is known that approximating functions with such
a dimension dependent level of smoothness can be done efficiently
\cite{petrushev1998approximation, kainen2007sobolev}. However, the
Barron space $\mathcal{B}^1$ is significantly larger that
$H^{\frac{d}{2}+2}$, in fact we only have $\mathcal{B}^1 \subset H^1$
by lemma \ref{smoothness-lemma}. The precise properties of the Barron
norm in high dimensions are an interesting research direction which
would help explain exactly how shallow neural networks help alleviate
the curse of dimensionality.

\begin{proof}
Given the integral representation given by Lemma~\ref{lem:sampleHk}, we follow
a similar line of reasoning as in \cite{barron1993universal}. Our
argument differs from previous arguments in how we write $f$ as convex
combination of shifts and dilations of $\sigma$. This is what allows
us to relax our assumptions on $\sigma$. In order to do this, we must
first find a way to normalize the above integral.

 The above integral is on an unbounded domain, but the decay
 assumption on the Fourier transform of $f$ allows us to normalize the
 integral in the $\omega$ direction.  To normalize the integral in the
 $b$ direction, we must use the assumption that $x$ is bounded and
 that $\sigma$ decays polynomially. Consider the $\sigma$ part of the
 above integral representation,
 \begin{equation}
  D_x^\alpha\sigma\left(\frac{\omega}{a}\cdot x+b\right).
 \end{equation}
 Note that by the triangle inequality and the boundedness of $x\in
 \Omega$, we can obtain a lower bound on the above argument uniformly
 in $x$. Specifically, we have
 \begin{equation}
  \left|\frac{\omega}{a}\cdot x+b\right| \geq \max\left(0,|b| - \frac{R\|\omega\|}{|a|}\right),
 \end{equation}
 where $R$ is the maximum norm of an element of $\Omega$. Note that
 without loss of generality, we can translate $\Omega$ so that it
 contains the origin and so $R \leq \text{\normalfont diam}(\Omega)$.

Combining this with the polynomial decay of $\omega$ \eqref{growth_condition} implies that
\begin{equation}\label{eq_779}
  \left|\sigma^{(k)}\left(\frac{\omega}{a}\cdot x+b\right)\right| \leq C_p\left(1 + \left|\frac{\omega}{a}\cdot x+b\right|\right)^{-p} \leq C_p\left(1 + \max\left(0,|b| - \frac{R\|\omega\|}{|a|}\right)\right)^{-p}.
\end{equation}
Thus the function $h$ defined by
 \begin{equation}\label{h_definition}
  h(b,\omega) = \left(1 + \max\left(0,|b| - \frac{R\|\omega\|}{|a|}\right)\right)^{-p}
 \end{equation}
 provides (up to a constant) an upper bound on
 $\sigma^{(k)}\left(\frac{\omega}{a}\cdot x+b\right)$ uniformly in
 $x$.  The decay rate of $h$ is fast enough to make it integrable in
 $b$. Moreover, its integral in $b$ grows at most linearly with
 $\omega$. Namely, we calculate
 \begin{equation}\label{eq_775}
 \begin{split}
  \int_\mathbb{R} h(\omega,b)db& = \int_{|b|\leq \frac{R\|\omega\|}{|a|}} db + 2\int_{b > \frac{R\|\omega\|}{|a|}} \left(1 + b - \frac{R\|\omega\|}{|a|}\right)^{-p}db \\
  & =~2R|a|^{-1}\|\omega\| + 2\left[(1-p)^{-1}\left(1 + b - \frac{R\|\omega\|}{|a|}\right)^{1-p}\right]_{\frac{R\|\omega\|}{|a|}}^\infty \\
  &=~2R|a|^{-1}\|\omega\| + \frac{2}{p-1}\leq C_1(p,\text{\normalfont diam}(\Omega),\sigma) (1 + \|\omega\|).
  \end{split}
 \end{equation}
\end{proof}

 Finally, we note that the approximation rate in this theorem holds as long as the growth condition \eqref{growth_condition}
 hold for some $f\in \dnn(\sigma)$, i.e. the condition \eqref{growth_condition} need not hold for $\sigma$ itself.
 We state this as a corollary below.
 \begin{corollary}\label{GeneralApproximation}
  Let $\sigma\in W^{m,\infty}_{loc}(\mathbb{R})$ be an activation function and suppose that there exists a $\nu\in \Sigma_1^{n_0}(\sigma)$ which satisfies the polynomial decay condition \eqref{growth_condition} in Theorem \ref{approximation_rate_theorem}. Then for any $f$ satisfying the assumptions of Theorem \ref{approximation_rate_theorem}, we have
  \begin{equation}
     \inf_{f_n\in \dnn(\sigma,n)}\|f - f_n\|_{H^m(\Omega)} \leq |\Omega|^{\frac{1}{2}}C(p,m,\text{\normalfont diam}(\Omega),\sigma)\sqrt{n_0}\|f\|_{\mathcal{B}^{m+1}}n^{-\frac{1}{2}}.
  \end{equation}

 \end{corollary}
 \begin{proof}
 The result follows immediately from Theorem \ref{approximation_rate_theorem} and the observation that $v\in \dnn(\sigma,n_0)(\sigma)$ implies that
 \begin{equation}
  \dnn(\nu,n) \subset \dnn(\sigma,nn_0)(\sigma).
 \end{equation}

 \end{proof}

Next, we consider the case of periodic activation functions. We show that neural networks with periodic activation functions achieve the same rate of approximation in Theorem \ref{approximation_rate_theorem}. The argument makes use of a modified integral representation and allows us to relax the smoothness condition on $f$, which now only has to be in $\mathcal{B}^m$.
\begin{theorem}\label{periodic-activation}
 Let $\Omega\subset \mathbb{R}^d$ be a bounded domain. If the activation function $\sigma\in W^{m,\infty}(\mathbb{R})$ is a non-constant periodic function, we have
 \begin{equation}
  \inf_{f_n\in \dnn(\sigma,n)}\|f - f_n\|_{H^m(\Omega)} \leq |\Omega|^{\frac{1}{2}}C(\sigma)n^{-\frac{1}{2}}\|f\|_{\mathcal{B}^m},
 \end{equation}
 for any $f\in \mathcal{B}^{m}$.
\end{theorem}
\begin{proof}
 Using the integrability condition on $\hat{f}$, we define the probability distribution on $\mathbb{R}^d \times [0,2\pi]$ as
 \begin{equation}
  d\lambda = \frac{1}{2\pi\|f\|_{\mathcal{B}^m}}(1 + |\omega|)^m|\hat{f}(x)|dxdb.
 \end{equation}
 Then $f(x)$ can be written
 \begin{equation}\label{periodic_representation}
 f(x) = \mathbb{E}_{d\lambda}\left(\|f\|_{\mathcal{B}^m}|a_i|^{-1}(1 + |\omega|)^{-m}\chi(\omega,b)\sigma\left(\omega\cdot x + b\right)\right).
 \end{equation}
 We have now written $f\in \mathcal{B}^m\subset H^m(\Omega)$ as a convex combination of functions $f_{\omega,b}\in H^m(\Omega)$. As in the proof of \ref{approximation_rate_theorem}, we now utilize Lemma 1 in \cite{barron1993universal} and proceed to bound $\|f_{\omega,b}\|_{H^m(\Omega)}$ using much the same argument.
\end{proof}


\section{Compactification}
\begin{figure}
  \centering
  \input{images/RamifiedCoverings1.tex}
  \caption{Not sure how to adjust width in this case.}
\end{figure}
We saw that the Riemann surface $\Gamma_{p(z)}$ is homeomorphic to $\bbc$.

We can generalize the previous constructions to studying graphs of
\begin{align*}
  p(z) &= w^p \\
  p(z,w) &= 0
\end{align*}
But before we do this, we need to make one more leap which is again inspired from topology.

The surface $\Gamma_{p(z)}$ costructed above is not compact.
Non-compact objects are harder\footnote{In most case, but of course not always.} to study than compact ones because the behaviour of limits is harder to control for non-compact surfaces and functions on non-compact .
\begin{qbox}(Optional exercise)
  Let $X$ be a subset of $\bbc$.
  \begin{enumerate}
    \item Show that if $X$ is compact then every function $f:X \rightarrow \bbr$ is bounded.
    \item If $X$ is open, find an unbounded continuous function $f:X \rightarrow \bbc$.
  \end{enumerate}
\end{qbox}
Furthermore, non-compact subsets come in all shapes and sizes (for example, every open subset of $\bbc$ is non-compact) but as we'll see in the next section there are various classification theorems for compact ones.

\subsection{Behaviour at $\infty$}


\section{Polymorphism}

\begin{frame}[fragile,fragile]
  \frametitle{Introduction}
  Take a language such as $\lggeEF$ with base types $\enum{}$ and
  $\estr{}$ and product types.

  Can we define \emph{one} function which takes a triple (e.g.,
  $\cpair{a}{\cpair{b}{c}}$) and returns the middle element (i.e., $b$)?

  % \bigskip

  \[
  \elam{\typrod{\enum{}}{\typrod{\enum{}}{\enum{}}}}{x}{ \eprl{\eprr{\var x}}}
  \]

  \begin{lstlisting}[language=Haskell]
-- Haskell
middle :: (t1,(t2,t3)) -> t2
middle (x,(y,z)) = y
\end{lstlisting}

\begin{lstlisting}[language=Java]
// Java
// (assume class Pair<T, V> exists)
public <S> S getMiddle(Pair<T,Pair<S,V>> triple){
  return triple.getRight().getLeft();
}
\end{lstlisting}

\bigskip

  More examples?

\end{frame}

\begin{frame}
  \frametitle{Polymorphicity}
  \begin{itemize}
  \item The term \emph{polymorphism} refers to a range of language mechanisms
    that allow a single part of a program to be used with different
    types in different contexts.
    %
  \item The languages we have considered so far are all
    \emph{monomorphic} in that every expression has at most \emph{one}
    type (given the types of its free variables).
    %
  \item \emph{Overloading} is a form of ``ad-hoc polymorphism''. It
    associates a single function symbol with many implementations.
    The compiler chooses an appropriate implementation for each
    application of the function, based on the types of the arguments.
  \end{itemize}
\end{frame}

\subsection{Syntax}

\begin{frame}
  \frametitle{Language $\lggeF$}
  \[
  \begin{array}{llclll}
    \TYPES & \tau & \Coloneqq & \tvar{t} & \tvar{t} & \text{type variable}
    \\
           &&& \tyarr{\tau_1}{\tau_2}  & \tau_1 \rightarrow \tau_2 & \text{function}
    \\
           &&& \tyall{t}{\tau} & \ctyall{t}{\tau} & \text{polymorphic}
    \\
    \\
    \EXPS & e & \Coloneqq & \var{x} & \var{x} & \text{variable}
    \\
           &&& \elam{\tau}{x}{e} & \clam{\tau}{x}{e} & \text{abstraction}
    \\
           &&& \eapp{e_1}{e_2} & \capp{e_1}{e_2} & \text{application}
    \\
           &&& \eLAM{t}{e} & \cLAM{t}{e} & \text{type abstraction}
    \\
           &&& \eAPP{\tau}{e} & \cAPP{\tau}{e} & \text{type application}
  \end{array}
  \]
\end{frame}

\begin{frame}
  \frametitle{Exercises}
  \begin{itemize}
  \item Define the identify function ($\texttt{id}(x) = x$)
  \item Define a polymorphic function to compose function together ($f \circ g = f(g(x))$)
  \item Any more ``standard'' polymorphic functions?
  \end{itemize}
\end{frame}

% L(T) . \x:t . x
% L(T1) . L(T2) . L(T2) \f:T2->T3 \g:T1->T2 = f(g(x))

\subsection{Statics}

\begin{frame}
  \frametitle{Judgements}

  Two judgement forms and two kinds of environments, the hypotheses in
  $\Delta$ have the form $\tvar{t} \ \typeok$, where $\tvar{t}$ is a
  variable of sort $\TYPES$ and the hypotheses in $\Gamma$ have the
  form $\var{x} : \tau$, where $\var{x}$ is a variable of sort $\EXPS$.

  \bigskip

  \begin{itemize}
  \item   {\Huge
      $
      {\typejudge{\Delta}{\tau}}
      $
    }

    says that the type $\tau$ is well-formed, under the hypotheses in $\Delta$

    \bigskip

  \item   {\Huge
      $
      {\tyFjudge{\Delta}{\Gamma}{e}{\tau}}
      $
    }

    says that the expression $e$ has type $\tau$, under  the hypotheses in $\Delta$ and $\Gamma$
  \end{itemize}
\end{frame}

\note{As usual, we write $\varnothing$ for the empty context $\Delta$ or $\Gamma$}

\begin{frame}
  \frametitle{Well-formed types}
 \[
  \tyrule
  {}
  {\,}
  {\typejudge{\Delta, \tvar{t} \ \typeok , \Delta_2}{\tvar{t}} }
  \]

  \[
  \tyrule
  {}
  {
    \typejudge{\Delta}{\tau_1}
    \and
    \typejudge{\Delta}{\tau_2}
  }
  {\typejudge{\Delta}{\tyarr{\tau_1}{\tau_2}} }
  \]

  \[
  \tyrule
  {}
  {
    \typejudge{\Delta, \tvar{t} \ \typeok }{\tau}
  }
  {\typejudge{\Delta}{\tyall{t}{\tau}} }
  \]

\end{frame}

\begin{frame}
  \frametitle{Examples}
  Are these types well-formed?
  \begin{itemize}
  \item $\tyall{t}{\tyarr{\tvar t}{\tvar t}}$
  \item $\tyall{t_1}{\tyall{t_2}{\tvar{t_1}}}$
  \item $\tyall{t_1}{\tyall{t_2}{\tyarr{\tvar{t_2}}{\tvar{t_1}}}}$
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Typing judgements (1)}
  \[
  \tyrule
  {\ftyrulename{var}}
  {\,}
  {\tyFjudge{\Delta}{\Gamma_1, \var{x}: \tau, \Gamma_2 }{\var{x}}{\tau} }
  \]

  \[
  \tyrule
  {\ftyrulename{lam}}
  {\tyFjudge{\Delta}{\Gamma, \var{x} : \tau_1}{e}{\tau_2}}
  {\tyFjudge{\Delta}{\Gamma}{\elam{\tau_1}{x}{e}}{\tyarr{\tau_1}{\tau_2}}}
  \]

  \[
  \tyrule
  {\ftyrulename{ap}}
  {
    \tyFjudge{\Delta}{\Gamma}{e_1}{\tyarr{\tau_1}{\tau_2}}
    \and
    \tyFjudge{\Delta}{\Gamma}{e_2}{\tau_2}
  }
  {
    \tyFjudge{\Delta}{\Gamma}{\eapp{e_1}{e_2}}{\tau_2}
  }
  \]

\end{frame}

\begin{frame}
  \frametitle{Typing judgements (2)}

  \[
  \tyrule
  {\ftyrulename{tlam}}
  {\tyFjudge{\Delta, \tvar{t} \ \typeok}{\Gamma}{e}{\tau}}
  {
    \tyFjudge{\Delta}{\Gamma}{\eLAM{t}{e}}{\tyall{t}{\tau}}
  }
  \]

  \[
  \tyrule
  {\ftyrulename{tap}}
  {
    \typejudge{\Delta}{\tau}
    \and
    \tyFjudge{\Delta}{\Gamma}{e}{\tyall{t}{\tau'}}
  }
  {
    \tyFjudge{\Delta}{\Gamma}{\eAPP{\tau}{e}}{\tsubs{\tau'}{\tau}{t}}
  }
  \]

\end{frame}

\begin{frame}
  \frametitle{Examples}
  Assuming we have product types, do these expression have a type?
  \begin{itemize}
  \item $\eLAM{t}{\elam{\tvar{t}}{x}{\var x}}$
  \item $\eLAM{t}{\elam{\typrod{\tvar{t}}{\typrod{\tvar{t}}{\tvar{t}}}}{x}{ \eprl{\eprr{\var x}}}}$
  \item Check for the solution of $f \circ g$
  \end{itemize}
\end{frame}

% let as lambda => (\id . <id "hello", id 123>) (\x . x)

\subsection{Semantics}

\begin{frame}
  \frametitle{Dynamics}
  \textbf{Call-by-name} semantics
  \[
  \semrule
  {\fsemrulename{lam}}
  {\,}
  {\valjudge{\elam{\tau}{x}{e}}}
  \]

  \[
  \semrule
  {\fsemrulename{tlam}}
  {\,}
  {\valjudge{\eLAM{t}{e}}}
  \]

\end{frame}

\begin{frame}
  \frametitle{Dynamics}
  \[
  \semrule
  {\fsemrulename{ap}}
  {\jtrans{e_1}{e'_1}}
  {
    \jtrans{\eapp{e_1}{e_2}}{\eapp{e'_1}{e_2}}
  }
  \]

  \[
  \semrule
  {\fsemrulename{lav}}
  {\,}
  {\jtrans{\eapp{\elam{\tau}{x}{e_1}}{e_2}}{\subs{e_1}{e_2}{x}}}
  \]
\end{frame}

\begin{frame}
  \frametitle{Dynamics}
  \[
  \semrule
  {\fsemrulename{ap}}
  {\,}
  {
    \jtrans
    {\eAPP{\tau}{\eLAM{t}{e}}}
    {\tsubs{e}{\tau}{t}}
  }
  \]

  \[
  \semrule
  {\fsemrulename{lav}}
  {\jtrans{e}{e'}}
  {
    \jtrans
    {\eAPP{\tau}{e}}
    {\eAPP{\tau}{e'}}
  }
  \]
\end{frame}

\begin{frame}
  \frametitle{Expressivity of $\lggeF$}
  Product and sum type can be expressed with the construct
  of $\lggeF$!
  %
  See Chapter 16.2 of PFPL.
\end{frame}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "main"
%%% End:


\section{Introduction}

Our aim is to provide a tool with which one can easily identify
the optimal quantum query complexity as well as corresponding
query optimal quantum algorithm for a given Boolean function. This
gives researchers an easy
way to identify problems for which a quantum computer
is better suited than a classical computer. It has been shown that
there are many functions for which this is the case, such as
search and factoring.

We say that a function is better suited to a quantum computer than a classical computer if a quantum computer could evaluate the function more efficiently. Generally, we would measure this efficiency by runtime, but the runtime of a quantum computer for a given function is difficult to calculate. Instead, we can approximate the runtime with the algorithm's query complexity. To evaluate a function the algorithm must query the inputs to gain information about the given problem and the number of queries required is the query complexity.

For example, if we take a look at the search function, which is a generalization of the single bit OR function, where there is a bit string input and we return a $1$ to indicate that there is a one somewhere in the input bitstring, and a $0$ if there is not. In a classical computer, in the worst case, we would need to examine all input bits to evaluate this function, which would take $n$ queries for a bitstring of length $n$. However, there exists a quantum implementation of the search function that only requires $\sqrt{n}$ queries of the input bitstring \cite{grover1996fast}--- this is the query optimal quantum algorithm. In this case, there is a large advantage in using a quantum computer as opposed to a classical computer.

While the optimal quantum query complexity of the search function has been shown to be $\sqrt{n}$, and the query optimal quantum algorithm is known, it is clearly of great interest to identify other functions like search where it would be substantially more efficient to use a quantum computer instead of a classical one. To aid in this search, we have implemented an algorithm that takes a Boolean function and returns the optimal quantum query complexity as well as the query optimal quantum algorithm in the form of a span program. We hope that these results can help build more efficient quantum algorithms and help us better understand the advantages of quantum computers.

\begin{comment}
Given

Why quantum computers
Why we care about query complexity

Quantum computers have many advantages over classical computers as
algorithms for a given function can be substantially less time
complex on quantum computers. More specifically, in many of these
examples, such as search and factoring, quantum computers require
significantly fewer queries of the function input. For an
example, a classic search algorithm for a bit string of length
$n$ could take up to $n$ queries to see if a $1$ is present.
However, quantum computers only require $\sqrt{n}$ queries of the
input.

It is then of great interest to identify these functions to
understand the advantages of quantum computers over classical
computers. We implement such a tool for Boolean functions. We
calculate the asymptotic quantum query complexity of a given
function which provides a lower bound of run time for a quantum
computer. Furthermore, for each Boolean function, we can also
provide a query optimal quantum algorithm such that this optimal
algorithm could be implemented to achieve the potential
improvement over classical computers.

\end{comment}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% PROBLEM 1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Problem 1}
The radioactive isotope $^{233}$Pa can be produced following neutron capture by $^{232}$Th when the resulting $^{233}$Th decays to $^{233}$Pa. In the neutron flux of a typical reactor, neutron capture in 1 g of $^{232}$Th produces $^{233}$Th at of a rate of $2.0 \times 10^{11}\text{ s}^{-1}$.
\begin{enumerate}[a)]
\item What are the activities (in Ci) of $^{233}$Th and $^{233}$Pa after this sample is irradiated for 1.5 hours?
\item The sample is then placed in storage with no further irradiation so that the $^{233}$Th can decay away. What are
the activities (in Ci) of $^{233}$Th and $^{233}$Pa after 48 hours of storage?
\item The decay of $^{233}$Pa results in $^{233}$U, which is also radioactive. After the above sample has been stored for 1 year what is the $^{233}$U activity in Ci? (Hint: it should not be necessary to set up an additional differential equation to find the $^{233}$U activity.)
\end{enumerate}

\begin{table}[htbp]
	\centering
	\begin{tabular}{|c|c|}
			\hline
			Nucleus		&	Half-life \\
			\hline
			$^{233}$Th	&  $22.3$ min\\
			$^{233}$Pa	&  $27.0$ days\\
			$^{233}$U	&  $1.592 \times 10^5$ yr\\
			\hline
	\end{tabular}
	\label{tab:design-specs}
\end{table}
\begin{center}$1\text{ Ci} = 3.7 \times 10^{10}\text{ s}^{-1}$\end{center}


\section{Transpose}
\label{transpose}

Matrix transpose seems an unlikely candidate for
SIMD optimization, but there are at least two cases
where it has been applied in APL.

Our implementation of transpose for SHARP APL performed
an optimization, similar to the one we used in rotate, on
all array types. A dyadic transpose that leaves one or
trailing axes in place moves those elements as a unit,
using rbemove or its equivalent, {\em e.g.}:

\medskip

{\apl T\qlarrow\02~2~3~3\qrho\qiota\036\\
~~0~~1~~2\\
~~3~~4~~5\\
~~6~~7~~8\\
\\
~~9~10~11\\
12~13~14\\
~15~16~17\\\
\\
~18~19~20\\
~21~22~23\\
~24~25~26\\
\\
~27~28~29\\
~30~31~32\\
~33~34~35\\}

{\apl 1~0~2~3\qtran\0T\\
~~0~~1~~2\\
~~3~~4~~5\\
~~6~~7~~8\\
~\\
~18~19~20\\
~21~22~23\\
~24~25~26\\
~\\
~\\
~~9~10~11\\
12~13~14\\
15~16~17\\\
\\
27~28~29\\
30~31~32\\
33~34~35\\}

\noindent
A more interesting transpose algorithm applies to
Boolean arrays whose shape is a multiple of eight
on its last two axes. Monadic transpose
({\em e.g.}, {\apl \qtran\010000 75000\qrho\00 1}), or a
dyadic transpose that interchanges only the last two argument axes
({\em e.g.}, {\apl 0 2 1\qtran\03 10000 75000\qrho\00 1}), can exploit
an algorithmic kernel for transposing Boolean arrays of shape {\apl 8~8}
that appears, at very least, in
{\em Hacker's Delight}.\cite{Warren:2012:HD:2462741}
The algorithm performs 16 logical and shift operations on
a 64-bit value that contains the ravel of the array, and
produces the transposed value as its result. The 8x8 results of
applying this transpose kernel are copied into the final result of the
larger array transpose. Jay Foad's implementation of this
approach for Dyalog APL produced a speedup of about ten times,
for 10000x75000 Boolean arrays.\cite{JFoad:pc2016}.
Section~\ref{compressexpand} describes two vector instructions,
{\tt PDEP} and {\tt PEXT},
that appear in some recent machine architectures; {\tt PDEP} can
be used to implement the above transpose kernel.
A {\em perfect shuffle}, as its name suggests, splits an array into
two halves, then selects elements from alternate halves.
Foad noted that two {\tt PDEP} instructions and an {\tt OR} produce a
highly efficient perfect shuffle verb, modeled in APL as {\apl S},
and that composing {\apl S} three times produces the
transpose kernel described above:

{\apl S\qlarrow\qlbrace\qomega\qlbr\qugrade\qlpar\qrho\qomega\qrpar\qrho\00~1\qrbr\qrbrace}\\

{\apl T\qlarrow\08~8\qrho\qiota\064\\

{\apl 8~8\qrho\0S~S~S~\qcomma\0T}\\
0~~8~16~24~32~40~48~56\\
1~~9~17~25~33~41~49~57\\
2~10~18~26~34~42~50~58\\
3~11~19~27~35~43~51~59\\
4~12~20~28~36~44~52~60\\
5~13~21~29~37~45~53~61\\
6~14~22~30~38~46~54~62\\
7~15~23~31~39~47~55~63\\
}

\noindent In general, any square array, {\apl T},
whose shape vector is all powers of two, can be transposed with this verb,
where {\apl POW} is the APL {\em power} conjunction.

{\apl TR\qlarrow\qlbrace\0S POW\qlpar\02\qlog\01\qrho\qrho\qomega\qrpar\qomega\qrbrace}

\noindent We plan to investigate SIMD algorithms
for efficient transposition of Boolean arrays of shapes other than
powers of two.

Foad pointed out that the IBM POWER architecture
includes the {\tt vgbbd} instruction, which can perform
two 8x8 Boolean transposes at once.
He notes that its performance is less than that of straightforward
C code, because execution time is dominated by
argument setup and result storing for the {\tt vgbbd} instruction.


%!TEX root = ../notes.tex
\section{February 11, 2022}
\subsection{Elgamal \emph{continued}}

\recall we perform Elgamal by starting with a prime $p$ and $g\in (\ZZ/p\ZZ)^\times$ which is \emph{public knowledge}.

Alice computes $a$ which is her \emph{private key}, and $A = g^{a}$ which is her \emph{public key}.

\textbf{Encryption:} Bob generates a random $k$ and sends Alice
\[c_0\equiv g^k\mod{p}\qquad c_1\equiv mA^k\mod{p}\]

\textbf{Decryption:} Alice computes
\[c_1\cdot (c_0^a)^{-1}\equiv m(g^a)^k \left((g^k)^a\right)^{-1}\]

We continue as we did from last time:
\begin{lstlisting}[language=Python]
import ext_gcd, pow_mod
from random import randrange
def e(A, m):
    k = randrange(p)
    return (pow_mod(g, k, p), m * pow_mod(A, k, p))

def d(a, c):
    return c[1] * ext_gcd(pow_mod(c[0], a, p), p)[0]
\end{lstlisting}
Which works as intended (try it out!).

We note a property of Elgamal that there is an expansion factor of 2. It takes \emph{twice} as much space to store $c$ as $m$. We note that the expansion factor is always at least $1$ (otherwise, we wouldn't be able to invert it).

\subsection{Midterm Details}
\emph{Feb 16 @ 2pm in class}. If remote, send email.

Topics will include: \emph{everything up to now} (literally right now).

Focus: More theoretical, less computational. (Both are fair game!)

Resources: Pen/pencil, paper. No notes and no book. Nothing else.

Weighting: 20\% Midterm 1 and 30\% on Final. 30\% Midterm 2, 20\% Homework. Half on written and half on in-class exams.

Problem set \#3 which is shorter than \#2. (Good practice!)

Midterm results/curve will be announced hopefully by Friday after the midterm.

\subsection{Introduction to Group Theory}
Groups are an algebraic structure... they're sets endowed with an operation.
\begin{example}
    We have that $(\ZZ/p\ZZ, +)$ and $((\ZZ/p\ZZ)^\times, \cdot)$ are both groups.
    \[\begin{array}{rcc}
            \toprule
                                & (\ZZ/p\ZZ, +)           & ((\ZZ/p\ZZ)^\times, \cdot )           \\ \midrule
            \text{Identity:}    & 0+a=a                   & 1\cdot a = a                          \\
            \text{Inverse:}     & a + (-a) = (-a) + a = 0 & a\cdot a^{-1} = a^{-1}\cdot a = 1     \\
            \text{Associative:} & a + (b + c) = (a+b)+c   & a\cdot (b\cdot c) = (a\cdot b)\cdot c \\
            \text{Commutative:} & a + b = b + a           & a\cdot b = b\cdot a                   \\ \bottomrule
        \end{array}\]
\end{example}
\begin{definition}[Group]
    A group $G$ is a set plus an operation
    \[\circ : G\times G \to G\]
    satisfying
    \begin{enumerate}
        \item \emph{Identity:} There is $e\in G$ with $e\circ a = a\circ e = a$.
        \item \emph{Inverse:} For any $a\in G$, there is $a^{-1}\in G$ with
              \[a\circ a^{-1} = a^{-1}\circ a = e\]
        \item \emph{Associativity:} $a\circ (b\circ c)=(a\circ b)\circ c$
    \end{enumerate}
    We additionally say $G$ is \ul{Abelian} if we have
    \[a\circ b = b\circ a\]
\end{definition}

\begin{definition}[Group Order]
    The \ul{order} of $G$ written $\#G$ is the number of elements in group $G$. If the order is finite, we say $G$ is \ul{finite}.
\end{definition}
\begin{example}
    $(\ZZ/p\ZZ, +)$ and $((\ZZ/p\ZZ)^\times, \cdot)$ are both Abelian and finite.
\end{example}


\section{Weierstrass' Elliptic Functions}
We shall explore the definition and properties of the Weierstrass $\wp$-functions, also known as Weierstrass' elliptic functions.
\subsection{The Definition}
We know that a non-constant elliptic function has degree at least $2$.
The Weierstrass $\wp$-function on a lattice is a elliptic function that behaves like $z\mapsto z^{-2}$ near any lattice point.
\begin{definition}
    Let $\Lambda$ be a lattice in $\mathbb C^2$.
    The associated Weierstrass $\wp$-function is defined by
    $$\wp(z)=\wp_\Lambda(z)=\frac{1}{z^2}+\sum_{\omega\in\Lambda\setminus\{0\}}\left(\frac{1}{(z-\omega)^2}-\frac{1}{\omega^2}\right)$$
\end{definition}
We have A LOT to check.
\begin{lemma}
    Let $\Lambda=\langle\omega_1,\omega_2\rangle$ be a lattice in $\mathbb C$ and $t\in\mathbb R$.
    Then the sum
    $$\sum_{\omega\in\Lambda\setminus\{0\}}\frac{1}{|\omega|^t}$$
    converges iff $t>2$.
\end{lemma}
\begin{proof}
    Consider the tilted square (or unit circle in $\ell^1$ metric) $Q=\{(t_1,t_2)\in\mathbb R^2:|t_1|+|t_2|=1\}$.
    By compactness, the continuous function $Q\to\mathbb R$ via $(t_1,t_2)\to|t_1\omega_1+t_2\omega_2|$ attains its maximum $M$ and minimum $m$ on $Q$.
    $m\neq 0$ since $\omega_1,\omega_2$ needs to be linearly independent over $\mathbb R$.
    So $0<m\le t_1\omega_1+t_2\omega_2\le M<\infty$ for any $t_1,t_2\in Q$.
    Consider $(k,l)\in\mathbb Z^2\setminus\{0\}$ and take
    $$t_1=\frac{k}{|k|+|l|},t_2=\frac{l}{|k|+|l|}$$
    Therefore $m(|k|+|l|)\le |k\omega_1+l\omega_2|\le M(|k|+|l|)$, hence the sum we wanted is bounded by positive multiples of
    $$\sum_{(k,l)\in\mathbb Z^2\setminus\{0\}}\frac{1}{(|k|+|l|)^t}$$
    So we only need to understand the convergence of this sum.
    Now for each $n\in\mathbb Z_{>0}$, the equation $n=|k|+|l|$ has exactly $4n$ solutions of $(k,l)\in\mathbb Z^2\setminus\{0\}$, therefore this sum converges iff
    $$\sum_{n=1}^\infty\frac{4n}{n^t}=4\sum_{n=1}^\infty\frac{1}{n^{t-1}}$$
    which converges iff $t>2$.
\end{proof}
\begin{theorem}
    $\wp_\Lambda$ is a well-defined elliptic function with $\Lambda$ its set of periods.
    Moreover, $\wp_\Lambda$ is even and has degree $2$.
\end{theorem}
\begin{proof}
    For convergence, we shall estimate the summands.
    \begin{align*}
        \left|\frac{1}{(z-\omega)^2}-\frac{1}{\omega^2} \right|&=\left|\frac{z(2\omega-z)}{\omega^2(z-\omega)^2} \right|\\
        &=\left|\frac{z}{\omega^2}\right|\left|\frac{2\omega-z}{(z-\omega)^2}\right|\\
        &\le \left|\frac{z}{\omega^2}\right|\left( \frac{2}{|z-\omega|}+\frac{|z|}{|z-\omega|^2} \right)
    \end{align*}
    Fix $R\ge|z|$.
    For all but finitely many $\omega$, we have $|\omega|\ge 2R$, so $|\omega-z|\ge |\omega|/2\ge R$.
    So after throwing away finitely many terms,
    $$\left|\frac{z}{\omega^2}\right|\left( \frac{2}{|z-\omega|}+\frac{|z|}{|z-\omega|^2} \right)\le\frac{R}{|\omega|^2}\left( \frac{2}{|\omega|/2}+\frac{R}{|\omega|R/2} \right)=\frac{6R}{|\omega|^3}$$
    So the sum converges by the preceding lemma since $3>2$, which means $\wp_\Lambda(z)$ is indeed well-defined and automatically meromorphic.
    It is clear that it is even.
    To see it is elliptic, choose $\omega_0\in\Lambda$, we need to show that $\omega_0$ is a period of $\wp_\Lambda$.
    Now it is clear that $\omega_0$ is a period of
    $$\wp_\Lambda^\prime(z)=\sum_{\omega\in\Lambda}\frac{-2}{(z-\omega)^3}$$
    So $f(z)=\wp_\Lambda(z+\omega_0)-\wp_\Lambda(z)$ has zero derivative, hence $f$ is constant.
    This means that $\wp_\Lambda(z+\omega_0)=\wp_\Lambda(z)+C$ for some constant $C$.
    But $\wp_\Lambda$ is even, so setting $z=-\omega_0/2$ gives $C=0$ and hence $\omega_0$ is a period, so anything in $\Lambda$ is a period of $\wp_\Lambda$.
    Also the poles of $\wp_\Lambda$ is exactly $\Lambda$, so the set of periods of $\wp_\lambda$ has to be exactly $\Lambda$.
    In particular, $\wp_\Lambda$ has a unique pole of order $2$ on $\mathbb C/\Lambda$, so $\deg\wp_\Lambda=2$
    This completes the proof.
\end{proof}
\begin{remark}
    We now know that:\\
    (i) $\wp_\Lambda$ is meromorphic with set of periods $\Lambda$.\\
    (ii) $\wp_\Lambda$ has poles only at $\Lambda$.\\
    (iii) $\wp_\Lambda(z)-z^{-2}\to 0$ as $z\to 0$.\\
    Furthermore, these properties uniquely characterised $\wp_\Lambda$ up to a constant.
\end{remark}


\subsection{Scheduling: Non-Preemptive, Size-Based}
\label{sec:Scheduling-Non-Preemptive-Size-Based}

The most common non-preemptive size-based scheduling policies are

\begin{description}

	\item [Shortest-Job-First (SJF)] the server chooses the job with the smallest size.

\end{description}

It is convenient to evaluate size-based policies as \textit{non-preemptive priority queueing}, where priority classes are job sizes and a job has higher priority if it has lower job size.

\subsection{Non-Preemptive Priority Queueing}
\label{sec:NP-Priority}

For non-preemptive priority queueing (NP-Priority), we have that

\begin{equation}
\label{eqn:NP-Priority-Queue-Time-Size}
\expected{T_{Q}(x)}^{\mathit{NP-Priority}}=
\frac{\varrho\expected{S^{2}}}{2\expected{S}}
\frac{1}{\Big(1-\sum_{i=1}^{x}\varrho_{i}\Big)\cdot\Big(1-\sum_{i=1}^{x-1}\varrho_{i}\Big)}
\end{equation}

The first factor of the denominator can be seen as the contribution due to waiting for jobs in queue with higher or equal priority.
The second factor of the denominator can be seen as the contribution due to those jobs arriving later with strictly higher priority.

Recall that $\expected{T_{Q}}^{\mathit{FCFS}}=\frac{\varrho\frac{\expected{S^{2}}}{\expected{S}}}{1-\varrho}$.

If $k$ is low, $\expected{T_{Q}(x)}^{\mathit{NP-Priority}}<\expected{T_{Q}^{\mathit{FCFS}}}$.
If $k$ is high and job size distribution is heavy-tailed, $\expected{T_{Q}(x)}^{\mathit{NP-Priority}}<\expected{T_{Q}}^{\mathit{FCFS}}$.

The above results hold because $\sum_{i=1}^{k}\varrho_{i}<<\varrho$.

We determine the mean queue time as follow

\begin{equation}
\label{eqn:NP-Priority-Queue-Time}
\expected{T_{Q}}^{\mathit{NP-Priority}}=\sum_{i=1}^{n}\expected{T_{Q}(i)}\cdot p_{i}=\sum_{i=1}^{n}\expected{T_{Q}(i)}\cdot\frac{\lambda_{i}}{\lambda}
\end{equation}

where $p_{i}$ is the fraction of jobs belonging to class $i$.

We will use these results to analyze SJF.

\subsection{Scheduling: SJF}
\label{sec:Scheduling-SJF}

We model SJF as NP-Priority with infinite priority classes, where the smaller the job, the higher the priority.

By applying \Cref{eqn:NP-Priority-Queue-Time-Size,eqn:NP-Priority-Queue-Time}, we obtain:

\begin{equation}
\label{eqn:SJF-Queue-Time-Size}
\expected{T_{Q}(x)}^{\mathit{SJF}}=
\frac{\varrho\expected{S^{2}}}{2\expected{S}}\cdot\frac{1}{\Big(1-\varrho_{x}\Big)^{2}}
\end{equation}

where $\varrho_{x}=\lambda F(x)\cdot\int_{t=0}^{x}t\frac{f(t)}{F(t)}\partial t$ is the arrival rate of jobs of size no more than $x$, namely $\lambda F(x)$, multiplied by the expected size of jobs of size no more than $x$, namely $\int_{t=0}^{x}t\frac{f(t)}{F(t)}\partial t$.

We also obtain:

\begin{equation}
\label{eqn:SJF-Queue-Time}
\expected{T_{Q}}^{\mathit{SJF}}=
\frac{\varrho\expected{S^{2}}}{2\expected{S}}\cdot\int_{x=0}^{x_{n}}\frac{f(x)\partial x}{\Big(1-\lambda\int_{t=0}^{x}tf(t)\partial t\Big)^{2}}
\end{equation}

By comparing $\expected{T_{Q}}^{\mathit{SJF}}$ with $\expected{T_{Q}}^{\mathit{FCFS}}$, we have that SJF is a poor choice for (i) very large jobs (see term $\varrho_{x}$) and (ii) high-variable job sizes \footnote{Since real workload are heavy-tailed, most jobs are small, hence we may think not to worry about very large jobs. This is a wrong idea. Heavy-tailed distribution have also high variability.} (see term $\expected{S^{2}}$).

When using SJF under high-variable job size, it is more efficient to kill long-running jobs, given that there are some other jobs in queue. In fact, if there are many jobs in queue, they are likely to include many short jobs, thus mean response time would improve. This is the idea behind TAGS policy in \cite{harchol2000task}.

The only way to get good performance out of SJF under real workload is to provide preemption.


\section{Divergence and Curl}\label{sec:DivergenceCurl}

Divergence and curl are two measurements of vector fields that are
very useful in a variety of applications. Both are most easily
understood by thinking of the vector field as representing a flow of a
liquid or gas;
that is, each vector in the vector field should be interpreted as a
velocity vector.
Roughly speaking, divergence
measures the tendency of
the fluid to collect or disperse at a point, and curl measures the
tendency of the fluid to swirl around the point. Divergence is a
scalar, that is, a single number, while curl is itself a vector. The
magnitude of the curl measures how much the fluid is swirling, the
direction indicates the axis around which it tends to swirl. These
ideas are somewhat subtle in practice, and are beyond the scope of
this course.
%You can find additional information on the web, for
%example at
%\texonly
%\url{http://mathinsight.org/curl_idea}%
%\vb|http://mathinsight.org/curl_idea|\endurl\
%and
%\url{http://mathinsight.org/divergence_idea}%
%\vb|http://mathinsight.org/divergence_idea|\endurl\
%\endtexonly
%\htmlonly
%
%http://mathinsight.org/curl_idea
%
%and
%
%http://mathinsight.org/divergence_idea
%
%\endhtmlonly
%and in
%many books including {\em
%Div, Grad, Curl, and All That: An Informal Text on Vector Calculus},
%by H. M. Schey.

Recall that if $f$ is a function, the gradient of $f$
is given by
$$\nabla f=\left\langle {\partial f\over\partial x},{\partial
  f\over\partial y},{\partial f\over\partial z}\right\rangle.$$
A useful mnemonic for this (and for the divergence and curl, as it
turns out) is to let
$$\nabla = \left\langle{\partial \over\partial x},{\partial
  \over\partial y},{\partial \over\partial z}\right\rangle,$$
that is, we pretend that $\nabla$ is a vector with rather odd looking
entries. Recalling that $\langle u,v,w\rangle a=\langle ua,va,wa\rangle$,
we can then think of the gradient as
$$\nabla f=\left\langle{\partial \over\partial x},{\partial
  \over\partial y},{\partial \over\partial z}\right\rangle f =
\left\langle {\partial f\over\partial x},{\partial
  f\over\partial y},{\partial f\over\partial z}\right\rangle,$$
that is, we simply multiply the $f$ into the vector.

The divergence and curl can now be defined in terms of this same
vector $\nabla$ by using the cross product and dot product.

\begin{formulabox}[Divergence]
The divergence of a vector field $\vect{f}=\langle f_1, f_2, f_3 \rangle$ is
$$\nabla \cdot \vect{f} =
\left\langle{\partial \over\partial x},{\partial
  \over\partial y},{\partial \over\partial z}\right\rangle\cdot
\langle f_1,f_2,f_3\rangle
= {\partial f_1 \over\partial x}+{\partial
  f_2 \over\partial y}+{\partial f_3\over\partial z}.$$\index{divergence}\index{vector field!divergence}
\end{formulabox}

\begin{formulabox}[Curl]
The curl of $\vect{f}$ is
$$\nabla\times\vect{f} = \left|
\begin{matrix}
\vect{i}	&	\vect{j}	&	\vect{k}	\\
{\partial \over\partial x}	&	{\partial \over\partial y}	&	{\partial \over\partial z}	\\
f_1	&	f_2	&	f_3
\end{matrix}
\right| =
\left\langle {\partial f_3\over\partial y}-{\partial f_2\over\partial z},
{\partial f_1\over\partial z}-{\partial f_3\over\partial x},
{\partial f_2\over\partial x}-{\partial f_1\over\partial y}\right\rangle.$$\index{curl}\index{vector field!curl}
\end{formulabox}

Here are two simple but useful facts about divergence and curl.

\begin{theorem}{Divergence of Curl is Zero}{div of curl is zero}
$\nabla\cdot(\nabla\times\vect{f})=0$.
\end{theorem}

In words, this says that the divergence of the curl is zero.

\begin{theorem}{Curl of Gradient is Zero}{curl of gradient is zero}
$\nabla\times(\nabla f) = \vect{0}$.
\end{theorem}

That is, the curl of a gradient is the zero vector. Recalling that
gradients are conservative vector fields, this says that the curl of a
conservative vector field is the zero vector. Under suitable
conditions, it is also true that if the curl of $\vect{f}$ is $\vect{0}$
then $\vect{f}$ is conservative. (Note that this is exactly the same test
that we discussed at the end of Section ~\ref{page:test for conservative vector field}.)

\begin{example}{}{conservative}
Let $\vect{f} = \langle 2e^x - y, -x ,e^z\rangle$. Show that $\vect{f}$ is conservative and find the function $f$ such that $\vect{f} = \langle f_x, f_y, f_z \rangle$.
\end{example}

\begin{solution}
If $\vect{f} = \langle 2e^x - y, -x ,e^z \rangle$ then $\nabla\times\vect{f} = \langle 0, 0, (-1)-(-1) \rangle = \vect{0}$.
Thus, $\vect{f}$ is conservative, and we can exhibit this directly by
finding the corresponding $f$.

Since $f_x=2e^x-y$, $f=2e^x-xy+g(y,z)$. Since $f_y=-x$, it must be that
$g_y=0$, so $g(y,z)= C + h(z)$. Thus $f=2e^x-xy+ C + h(z)$ and
$$f_z = h'(z) = e^z,$$
so $h(z)=e^z$. This leaves $f=2e^x -xy + e^z + C$.
\end{solution}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Opensolutionfile{solutions}[ex]
\section*{Exercises for \ref{sec:DivergenceCurl}}

\begin{enumialphparenastyle}

\begin{ex}
Let $\vect{f}=\langle xy,-xy\rangle$ and
let $D$ be given by $0\le x\le 1$, $0\le y\le 1$.
Compute $\ds\int_{\partial D} \vect{f}\cdot d\vect{r}$ and
$\ds\int_{\partial D} \vect{f}\cdot\vect{N}\,ds$.
\begin{sol}
	$-1$, $0$
\end{sol}
\end{ex}

\begin{ex}
Let $\vect{f}=\langle ax^2,by^2\rangle$ and
let $D$ be given by $0\le x\le 1$, $0\le y\le 1$.
Compute $\ds\int_{\partial D} \vect{f}\cdot d\vect{r}$ and
$\ds\int_{\partial D} \vect{f}\cdot\vect{N}\,ds$.
\begin{sol}
	$0$, $a+b$
\end{sol}
\end{ex}

\begin{ex}
Let $\vect{f}=\langle ay^2,bx^2\rangle$ and
let $D$ be given by $0\le x\le 1$, $0\le y\le x$.
Compute $\ds\int_{\partial D} \vect{f}\cdot d\vect{r}$ and
$\ds\int_{\partial D} \vect{f}\cdot\vect{N}\,ds$.
\begin{sol}
	$(2b-a)/3$, $0$
\end{sol}
\end{ex}

\begin{ex}
Let $\vect{f}=\langle \sin x\cos y,\cos x\sin y\rangle$ and
let $D$ be given by $0\le x\le \pi/2$, $0\le y\le x$.
Compute $\ds\int_{\partial D} \vect{f}\cdot d\vect{r}$ and
$\ds\int_{\partial D} \vect{f}\cdot\vect{N}\,ds$.
\begin{sol}
	$0$, $1$
\end{sol}
\end{ex}

\begin{ex}
Let $\vect{f}=\langle y,-x\rangle$ and
let $D$ be given by $x^2+y^2\le 1$.
Compute $\ds\int_{\partial D} \vect{f}\cdot d\vect{r}$ and
$\ds\int_{\partial D} \vect{f}\cdot\vect{N}\,ds$.
\begin{sol}
	$-2\pi$, $0$
\end{sol}
\end{ex}

\begin{ex}
Let $\vect{f}=\langle x,y\rangle$ and
let $D$ be given by $x^2+y^2\le 1$.
Compute $\ds\int_{\partial D} \vect{f}\cdot d\vect{r}$ and
$\ds\int_{\partial D} \vect{f}\cdot\vect{N}\,ds$.
\begin{sol}
	$0$, $2\pi$
\end{sol}
\end{ex}

\begin{ex}
Prove Theorem~\ref{thm:div of curl is zero}.
\end{ex}

\begin{ex}
Prove Theorem~\ref{thm:curl of gradient is zero}.
\end{ex}

\begin{ex}
If $\nabla \cdot \vect{f}=0$, $\vect{f}$ is said to be \dfont{incompressible}.  Show that any vector field
of the form $\vect{f}(x,y,z) = \langle f(y,z),g(x,z),h(x,y)\rangle$ is
incompressible.  Give a non-trivial example.
\end{ex}

\end{enumialphparenastyle}


\documentclass[
	superscriptaddress,
	twocolumn,
	aps, pre
]{revtex4-1}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{hyperref}
\usepackage{graphicx}

\input{../common.tex}
\newcommand{\dx}{\Delta x}
\newcommand{\dr}{\Delta r}
\renewcommand{\L}{_\mathrm{L}}
\newcommand{\R}{_\mathrm{R}}
\newcommand{\dom}{\Omega}
\newcommand{\bndry}{\partial\Omega}

\begin{document}

\title{Differential operators and boundary conditions in `py-pde`}
\author{David Zwicker}
\date{\today}

\begin{abstract}
We here document the differential operators and the associated boundary conditions that are used in the finite difference approximation.
We also derive special versions of the Laplace operator that conserves the total mass.
\end{abstract}

\maketitle
\tableofcontents

\section{General considerations}
The main objects in `py-pde` are fields that are expressed on discretized grids.
The package provides many functions to manipulate these fields.
Some functions (e.g., addition, multiplication, integrals) do only use the actual field values, while others (e.g., differential operators and interpolation) might require information on the behavior of the field at the boundary.
This documents specifies details for how boundary conditions are handled.

The simplest fields in `py-pde` are scalar fields, which associate a single value to each grid point.
In the continuum, a scalar field $c(\vect r)$ often has the following boundary conditions:
\begin{salign}
	\text{value:} &&	c &= A
\\
	\text{normal derivative:} && n_\alpha \partial_\alpha c &= B
\end{salign}
where $n_\alpha$ is again the outward normal of the boundary.
These two conditions are also often known as Dirichlet and Neumann boundary conditions, respectively.
These options are implemented as \texttt{value} and \texttt{derivative} in `py-pde`.

For a vector field $v_\alpha(\vect r)$, which associate a vector with each grid point, possible boundary conditions are
\begin{salign}
	\text{value:} && v_\alpha  &= A_\alpha
\\
	\text{normal component:} && n_\alpha v_\alpha  &= A
\\
	\text{normal derivative:} && n_\beta \partial_\beta v_\alpha &= B_\alpha
\\
	\text{normal comp. of normal deriv.:} && n_\alpha n_\beta \partial_\beta v_\alpha &= B
\end{salign}
There are now many more options, where the first two specify values (Dirichlet conditions) and the last two specify derivatives (Neumann conditions).
Additionally, the conditions differ in that they either specify the full vector (conditions 1 and 3) or just the normal component (conditions 2 and 4).
The latter choice obviously contains less information, but is sufficient for differential operators; see below.
Conversely, specifying the full vector using the first or third condition is necessary for interpolating vector fields correctly.
Note also that the Dirichlet conditionimposes the dot product between the normal vector and the vector field, which in a simple 1d system implies $v_1(x=x\L)=-A$ and $v_1(x=x\R)=A$ for the lower and upper boundary, respectively.
Similarly, the Neumann condition evaluates the directional derivative of the vector field in the normal direction~$n_\beta$ of the boundary.

\section{Boundary conditions for differential operators}
We start with general considerations that are independent of the grid type.
Here, we consider differential operators that act on fields defined in a domain~$\dom$.
In particular, we discuss the boundary conditions that can be enforced with each type of operator.
We here use a component notation with implicit Einstein summation to denote contractions explicitly.
For instance, $\partial_\alpha v_\alpha$ denotes the divergence of a vector field $v_\alpha(\vect r)$ and $t_{\alpha\alpha}(\vect r)$ is the trace of a tensor field $t_{\alpha\beta}(\vect r)$.

\subsection{Laplace operator}
The ordinary Laplace operator $\partial_\alpha^2 c$ is defined to act on scalar fields~$c(\vect r)$.
Possible boundary conditions are
\begin{salign}[eqn:bc_laplace]
	\text{value:} &&	c &= A
\\
	\text{normal derivative:} && n_\alpha \partial_\alpha c &= B
\end{salign}
applied at positions $\vect r \in \bndry$.
Consequently, either the value~$A$ of the field (Dirichlet condition $c(\vect r) = A$ at $\vect r \in \bndry$) or the normal derivative (Neumann condition $n_\alpha \partial_\alpha c = B$) is specified, where $n_\alpha$ is the outwards oriented normal vector at the boundary.

\subsection{Gradient operator}
The gradient operator $\partial_\alpha c$ acts on a scalar field $c(\vect r)$ yielding a vector field.
Possible boundary conditions are
\begin{salign}[eqn:bc_gradient]
	\text{value:} &&	c &= A
\\
	\text{derivative:} && n_\alpha \partial_\alpha c &= B
\end{salign}
where $n_\alpha$ is again the outward normal of the boundary.

\subsection{Divergence operator}
The divergence operator $\partial_\alpha v_\alpha$ acts on a vector field $v_\alpha(\vect r)$ with possible boundary conditions
\begin{salign}[eqn:bc_divergence]
	\text{normal component:} && n_\alpha v_\alpha  &= A
\\
	\text{derivative:} && n_\alpha n_\beta \partial_\beta v_\alpha &= B
\end{salign}
Note that the Dirichlet condition in this case imposes the dot product between the normal vector and the vector field, which in a simple 1d system implies $v_1(x=x\L)=-A$ and $v_1(x=x\R)=A$ for the lower and upper boundary, respectively.
Similarly, the Neumann condition evaluates the directional derivative of the vector field in the normal direction~$n_\beta$ of the boundary.
This implies that the tangential components of the vector field do not affect the boundary condition.

\subsection{Vector Laplacian}
The Vector Laplacian is a simple generalization of the ordinary Laplace operator to a vector field $v_\alpha(\vect r)$.
Possible boundary conditions are
\begin{salign}[eqn:bc_vector_laplace]
	\text{value:} &&	v_\alpha &= A_\alpha
\\
	\text{normal derivative:} && n_\beta \partial_\beta v_\alpha &= B_\alpha
\end{salign}
which are applied at positions $\vect r \in \bndry$.

\subsection{Vector gradient operator}
The vector gradient  operator $\partial_\beta v_\alpha$ acts on a vector field $v_\alpha(\vect r)$ and yields a tensorial field.
This operator can be though of as a gradient applied to each component of $v_\alpha$ and the boundary conditions thus are also applied to each component,
\begin{salign}
	\text{value:} &&	v_\alpha &= A_\alpha
\\
	\text{derivative:} && n_\beta \partial_\beta v_\alpha &= B_\alpha
\end{salign}

\subsection{Tensor divergence operator}
The vector gradient  operator $\partial_\beta t_{\alpha\beta}$ acts on a tensor field $t_{\alpha\beta}(\vect r)$ and yields a vector field.
This operator can be though of as a divergence applied along the $\beta$-direction to each $\alpha$-component.
The boundary conditions thus are applied to each component,
\begin{subequations}
\begin{align}
	\text{value:} &&	n_\beta t_{\alpha\beta} &= A_\alpha
\\
	\text{derivative:} &&\partial_\beta t_{\alpha\beta}&= B_\alpha
\end{align}
\end{subequations}

\subsection{Operators not implemented}
Currently not implemented are the following operators
\begin{subequations}
\begin{align}
	\text{Curl:} &&
		\epsilon_{\alpha\beta\gamma} \partial_\beta v_\gamma
\\
	\text{Material derivative (Advection):}&&
		w_\alpha \partial_\alpha v_\beta
\end{align}
\end{subequations}

%\subsection{Boundary conditions of compound operators}
%The Laplace operator can be written as a gradient operator followed by a divergence operator, $\partial_\alpha^2 c = \partial_\alpha(\partial_\alpha c)$.
%When implementing the involved operators in finite difference, one necessarily needs to determine boundary conditions for all three differential operators.
%Clearly, boundary conditions given in \Eqref{eqn:bc_laplace} for the Laplace operator can also be applied to the gradient operator, which employs the same conditions; see \Eqref{eqn:bc_gradient}.
%Conversely, the boundary conditions on the divergence operator specify

\section{Differential operators on Cartesian grids}
We consider a linear domain of length $L$ discretized by $N$ support points.
We place these points equidistantly at $x_n = (n+\frac12)\dx$ for $n=0,1,\ldots, N-1$ using the discretization $\dx=L/N$.
Any function $y=f(x)$ is then represented by discretized values $y_n = f(x_n)$.

In the finite difference schemes we discuss, we require differential operators of first and second order.
Targeting second order accuracy, we obtain these as
\begin{subequations}
\begin{align}
	f'(x_n) &\sim \frac{y_{n+1} - y_{n-1}}{2\dx}
\\
	f''(x_n) &\sim \frac{y_{n+1} - 2 y_n + y_{n-1}}{\dx^2}
\end{align}
\end{subequations}
where $\sim$ denotes the finite difference approximation.

These derivatives can only be evaluated directly for $n=1,2,\ldots, n-2$, while the boundary points $n=0$ and $n=N-1$ require knowledge of the value at the respective virtual points $x_{-1}$ and $x_N$ outside the domain.
We can derive expressions for the associated function values $y_{-1}$ and $y_N$  taking the boundary conditions in to account, which typically specify the value at the boundary or the (outward) derivative.
A particular simple case are periodic boundary conditions, where we have $y_{-1} = y_{N-1}$ and $y_N = y_0$, allowing the above formula to be used everywhere.
For more complicated boundary conditions, we need to treat both sides separately.

\subsection{Lower boundary}
Let us first consider conditions at the lower boundary, which we place at $x=0$.
We thus need to evaluate the virtual support point $y_{-1}$ from the boundary condition.
Using estimates for the value and the derivative at the boundary,
\begin{align}
	f(0) &\sim \frac{y_0 + y_{-1}}{2}
&
	f'(0) &\sim \frac{y_0 - y_{-1}}{\dx}
\end{align}
we can now handle several boundary conditions.

\paragraph{Dirichlet (Constant value):}
Assuming the boundary condition $f(0) = a$, we obtain $y_{-1} = 2a - y_0$.

\paragraph{Neumann (Constant derivative):}
Here, we consider the boundary condition $-f'(0) = b$, which specifies the outward derivative to equal the value $b$.
We obtain $y_{-1} = y_0 + b \dx$.

\paragraph{Robin (Mixed):}
Assuming the boundary condition $-f'(0) + c f(0) = b $, we obtain $y_{-1} = A - B y_0$
where
\begin{align}
	A &= \frac{2\dx}{c \dx + 2} b
&
	B &= \frac{c\dx - 2}{c \dx + 2}
\end{align}
%Hence,
%\begin{align}
%	\takenat{\pfrac{^2 f}{x^2}}{x_0} \sim \frac{A - (2 + B) y_0 + y_{1}}{\dx^2}
%\end{align}
As a sanity check, we obtain Dirichlet conditions in the limit $b \rightarrow \infty$ with $b/c = a$, where $A=2a, B=1$.
Likewise, we recover Neumann condition for $c=0$.

\paragraph{Summary:}
Taken together, we can express the value of the support point as
\begin{align}
	y_{-1} &= \alpha\L + \beta\L y_{k\L}
	\;,
\end{align}
where the coefficients $\alpha\L$, $\beta\L$, and $k\L$ are taken from \tabref{tab:cartesian_lower}.

\begin{table}[t]
\caption{\label{tab:cartesian_lower}%
Coefficients for lower cartesian boundary
}
\begin{ruledtabular}
	\begin{tabular}{cccc}
		Boundary condition & Offset $\alpha\L$ & Pre-factor $\beta\L$  & Index $k\L$\\
		\colrule
		Periodic & $0$ & $1$ & $N-1$ \\
		Dirichlet & $2a$ & $-1$ & $0$ \\
		Neumann & $b\dx$ & $1$  & $0$ \\
		Robin & $\frac{2b\dx}{2 + c \dx}$ &
				$\frac{2 - c\dx}{2 + c \dx}$ & $0$ \\
	\end{tabular}
\end{ruledtabular}
\end{table}

\subsection{Upper boundary}
The upper boundary at $x=L$ can be treated analogously to the lower one by introducing a virtual point at $x=x_N$.
Consequently, we find the following conditions for the virtual support point $y_N$:
\paragraph{Dirichlet (Constant value):}
Assuming the boundary condition $f(L) = a$, we obtain $y_N = 2a - y_{N-1}$.

\paragraph{Neumann (Constant derivative):}
Here, we consider the boundary condition $f'(L) = b$, which specifies the outward derivative to equal the value $b$.
We obtain $y_N = y_{N-1} + b \dx$.

Taken together, we express the value at the virtual support point as
\begin{align}
	y_N &= \alpha\R + \beta\R y_{k\R}
	\;,
\end{align}
where the coefficients $\alpha\R$, $\beta\R$, and $k\R$ are taken from \tabref{tab:cartesian_upper}.

\begin{table}[t]
\caption{\label{tab:cartesian_upper}%
Coefficients for upper cartesian boundary
}
\begin{ruledtabular}
	\begin{tabular}{cccc}
		Boundary condition & Offset $\alpha\R$ & Pre-factor $\beta\R$  & Index $k\R$\\
		\colrule
		Periodic & $0$ & $1$ & $0$ \\
		Dirichlet & $2a$ & $-1$ & $N-1$ \\
		Neumann & $b\dx$ & $1$  & $N-1$ \\
		Robin & $\frac{2b\dx}{2 + c \dx}$ &
				$\frac{2 - c\dx}{2 + c \dx}$ & $N-1$ \\
	\end{tabular}
\end{ruledtabular}
\end{table}

\section{Differential operators on cylindrical grids}
We here consider fields in $3$ dimensions that possess angular symmetry and thus only depend on the radial coordinate~$r$ and the axial coordinate~$z$.
Clearly, the axial coordinate behaves exactly as the Cartesian ones described above, so we here only have to deal with the polar symmetry.
Similar to Cartesian grids, we discretize the associated radial coordinate as $r_n = r_0 + (n + \frac12) \dr$, where $r_0$ is the inner radius, which often will be zero.

The radial part of the gradient operator in cylindrical coordinates is simply given by $\partial_r f(r)$ and thus obeys the same discretization as in a Cartesian grid.
The divergence of a vector field $v_\alpha(r) = \rho(r) \vect e_r$ reads
$\partial_\alpha v_\alpha(r) =  \rho'(r) + \frac{\rho(r)}{r}$, implying the discretized version
\begin{align}
	\partial_\alpha v_\alpha \sim
		\frac{y_{n+1} - y_{n-1}}{2\dr}
		+ \frac{y_n}{r_n}
\end{align}
Moreover, the Laplace operator in spherical coordinates reads $\nabla^2 f(r) = f''(r) + \frac{f'(r)}{r}$, which in the discretized version becomes
\begin{align}
	\partial_\alpha^2 f \sim
		\frac{y_{n+1} - 2 y_n + y_{n-1}}{\dr^2}
		+ \frac{y_{n+1} - y_{n-1}}{2 r_n \dr}
\end{align}

\subsection{Boundary condition at the origin}
Due to the symmetry, only vanishing derivatives are allowed at the inner boundary at the origin, implying the virtual support point $y_{-1} = y_0$.
Hence, the differential operators become
\begin{subequations}
\begin{align}
	\takenat{\partial_\alpha v_\alpha(r)}{r=0} &\sim
		\frac{y_1 - y_0}{2\dr}
		+ \frac{2y_0}{\dr}
\\
	\takenat{\partial_\alpha^2 f(r)}{r=0} &\sim
		2\frac{y_1 - y_0}{\dr^2}
%		\frac{y_1 - 2 y_0 + y_0}{\dr^2}
%		+ \frac{2(y_1 - y_0)}{\dr^2}
%		\frac{3y_1 -  3y_0}{\dr^2}
\end{align}
\end{subequations}

\subsection{Boundary condition at the outer side}
At the outer boundary, we can impose boundary conditions similar to the Cartesian grid.

\paragraph{Dirichlet (Constant value):}
Assuming the boundary condition $f(R) = a$, we obtain $y_N = 2a - y_{N-1}$.

\paragraph{Neumann (Constant derivative):}
Here, we consider the boundary condition $f'(L) = b$, which specifies the outward derivative to equal the value $b$.
We obtain $y_n = y_{n-1} + b \dx$.

\section{Differential operators on spherical grid}
We here consider fields that a spherically symmetric in $3$ dimensions and thus only depend on the radial coordinate~$r$.
Similar to Cartesian grids, we discretize the radial coordinate as $r_n = (n + \frac12) \dr$.

The radial part fo the gradient operator in spherical coordinates is simply given by $\partial_r f(r)$ and thus obeys the same discretization as in a Cartesian grid.
The divergence of a vector field $v_\alpha(r) = \rho(r) \vect e_r$ reads
$\partial_\alpha v_\alpha(r) =  \rho'(r) + \frac{2\rho(r)}{r}$, implying the discretized version
\begin{align}
	\partial_\alpha v_\alpha \sim
		\frac{y_{n+1} - y_{n-1}}{2\dr}
		+ \frac{2y_n}{r_n}
\end{align}
Moreover, the naive implementation of Laplace operator in spherical coordinates reads $\nabla^2 f(r) = f''(r) + \frac{2f'(r)}{r}$, which in the discretized version becomes
\begin{align}
	\partial_\alpha^2 f \sim
		\frac{y_{n+1} - 2 y_n + y_{n-1}}{\dr^2}
		+ \frac{y_{n+1} - y_{n-1}}{r_n \dr}
\end{align}
Note that this form of the Laplace operator is not conservative, \ie, the discretized version of the integral $\int \partial_\alpha^2 f \diff r$  does not vanish.

\subsection{Boundary condition at the origin}
Due to the symmetry, only vanishing derivatives are allowed at the inner boundary at the origin, implying the virtual support point $y_{-1} = y_0$.
Hence, the differential operators become
\begin{subequations}
\begin{align}
	\takenat{\partial_\alpha v_\alpha(r)}{r=0} &\sim
		\frac{y_1 - y_0}{2\dr}
		+ \frac{4y_0}{\dr}
\\
	\takenat{\partial_\alpha^2 f(r)}{r=0} &\sim
%		\frac{y_1 - 2 y_0 + y_0}{\dr^2}
%		+ \frac{2(y_1 - y_0)}{\dr^2}
		3\frac{y_1 -  y_0}{\dr^2}
\end{align}
\end{subequations}
Note that the latter expression applies to both the non-conservative and the conservative form of the Laplacian.

\subsection{Boundary condition at the outer side}
At the outer boundary, we can impose boundary conditions similar to the Cartesian grid.

\paragraph{Dirichlet (Constant value):}
Assuming the boundary condition $f(R) = a$, we obtain $y_N = 2a - y_{N-1}$.

\paragraph{Neumann (Constant derivative):}
Here, we consider the boundary condition $f'(L) = b$, which specifies the outward derivative to equal the value $b$.
We obtain $y_n = y_{n-1} + b \dx$.

\subsection{Conservative operator}

We call a discrete Laplace operator conservative when it preserves the desirable conservation equation
\begin{align}
	\int_\Omega \partial_\alpha^2 f \diff V
		= \oint_{\partial\Omega} \partial_\alpha f n_\alpha \diff S= 0
\end{align}
when Neumann boundary conditions are imposed on the function $f$.
A conservative operator can be derived by integrating the definition of the Laplace operator in spherical coordinates over one spherical shell from $r=r_n - \dr/2$ to $r=r_n  + \dr/2$,
\begin{align}
%	r^2 \partial_\alpha^2 f &= \partial_r\bigl(r^2 \partial_r f(r)\bigr)
%\\
	\int_{r_{n - \frac12}}^{r_{n + \frac12}} r^2 \partial_\alpha^2 f \diff r
		&= \int_{r_{n - \frac12}}^{r_{n + \frac12}} \partial_r\bigl(r^2 \partial_r f(r)\bigr) \diff r
	\;,
\end{align}
where we skipped the factor $4\pi$ stemming from the angle integration.
Assuming that the quantity $\partial_\alpha^2 f$ is constant across the discretization cell, we thus find
\begin{align}
	V_n \partial_\alpha^2 f %\left[ \frac{r^3}{3} \right]_{r_n - \dr/2}^{r_n + \dr/2}
		&\sim \Bigl[r^2 \partial_r f(r)\Bigr]_{r_n - \dr/2}^{r_n + \dr/2}
\end{align}
where we introduced the (scaled) shell volumes
\begin{align}
%	V_n &= \frac{\dr^3}{3}\bigl[(n + 1)^3 - n^3\bigr]
	V_n &= \frac{r_{n + \frac12}^3 - r_{n - \frac12}^3}{3}
	= \dr \left(	r_n^2 + \frac{\dr^2}{12} \right)
\;.
\end{align}
Consequently,
\begin{align}
	\partial_\alpha^2 f
%		&\sim \dr^2 \frac{(n + 1)^2 f'\left(r_n + \frac{\dr}{2}\right) - n^2 f'\left(r_n - \frac{\dr}{2}\right)}{V_n}
		&\sim \frac{r_{n+\frac12}^2 f'\left(r_{n+\frac12}\right) - r_{n-\frac12}^2 f'\left(r_{n-\frac12}\right)}{V_n}
\end{align}
where the derivatives $f'(r)$ are evaluated at the midpoints and can thus be represented as
\begin{subequations}
\begin{align}
	f'\left(r_{n + \frac12}\right) \sim \frac{y_{n+1} - y_n}{\dr}
\\
	f'\left(r_{n - \frac12}\right) \sim \frac{y_n - y_{n-1}}{\dr}
\end{align}
\end{subequations}
In the special case where the spherical grid does not have a hole ($r_0=0$), we then arrive at
\begin{align}
	\partial_\alpha^2 f
		&\sim \frac{3}{\dr^2} \, \frac{(n + 1)^2 (y_{n+1} - y_n) - n^2 (y_n - y_{n-1})}{(n + 1)^3 - n^3}
	\;,
\end{align}
which is a conservative discretization of the spherical Laplacian by construction.

Similarly, we can consider a spherical grid where fields depend on the polar angle~$\theta$ while still keeping azimuthal symmetry, i.e., assuming no dependence on $\phi$.
The condition for a conservative Laplace operator then reads
\begin{widetext}
\begin{multline}
	\iint_{\text{cell}(n, m)} r^2 \sin\theta \, \partial_\alpha^2 f  \diff r \diff \theta
=
	\iint_{\text{cell}(n, m)} \biggl(
		\sin\theta  \partial_r\bigl[r^2 \partial_r f(r, \theta)\bigr]
		+ \partial_\theta \bigl[ \sin\theta \partial_\theta  f(r, \theta)\bigr]
	\biggr)\diff r \diff \theta
\\=
	\Bigl(\cos\theta_{m-\frac12} - \cos\theta_{m + \frac12}\Bigr)
		\int_{r_{n - \frac12}}^{r_{n + \frac12}}\left(
			 \partial_r\bigl[r^2 \partial_r f(r, \theta)\bigr]
		\right)  \diff r
%\notag\\&\quad
	+ \Delta r
		\int_{\theta_{m - \frac12}}^{\theta_{m + \frac12}}
			 \partial_\theta \bigl[ \sin\theta \partial_\theta  f(r, \theta)\bigr]	 \diff\theta
\\=
	\Bigl(\cos\theta_{m-\frac12} - \cos\theta_{m + \frac12}\Bigr)
			\bigl[r^2 \partial_r f(r, \theta)\bigr]_{r_{n - \frac12}}^{r_{n + \frac12}}
	+  \Delta r
		\bigl[ \sin\theta \partial_\theta  f(r, \theta)\bigr]_{\theta_{m - \frac12}}^{\theta_{m + \frac12}}
%\\
%	\partial_\alpha^2 f \left[ \frac{r^3}{3} \right]_{r_n - \dr/2}^{r_n + \dr/2}
%		&\sim \Bigl[r^2 \partial_r f(r)\Bigr]_{r_n - \dr/2}^{r_n + \dr/2}
\end{multline}
\end{widetext}
Here we assumed that we can neglect angular dependencies in the radial integral and vice versa.
Using the (scaled) volume of a grid cell,
\begin{align}
	V_{n,m} =
		\left(\cos\theta_{m-\frac12} - \cos\theta_{m + \frac12}\right)
		\frac{r_{n + \frac12}^3 - r_{n - \frac12}^3}{3}
%\notag\\=
%	\left(\cos\theta_{m-\frac12} - \cos\theta_{m + \frac12}\right)
	\;,
\end{align}
we can write this as
\begin{multline}
	V_{n,m} \partial_\alpha^2 f
=\Bigl(\cos\theta_{m-\frac12} - \cos\theta_{m + \frac12}\Bigr)
			\bigl[r^2 \partial_r f(r, \theta)\bigr]_{r_{n - \frac12}}^{r_{n + \frac12}}
\\
	+  \Delta r
		\bigl[ \sin\theta \partial_\theta  f(r, \theta)\bigr]_{\theta_{m - \frac12}}^{\theta_{m + \frac12}}
\end{multline}

%\bibliographystyle{apsrev4-1}
%\bibliography{bibdesk.bib}

\end{document}