\section{A case study in clustering}\label{section:examples}

\subsection{\(k\)-means clustering}

The following examples act as a form of validation for EDO, and also highlight
some of the nuances in its use. The objective of these examples is to use the
proposed method to reproduce some known results about the clustering of data in
the absence of any external forces, and to examine how clustering algorithms are
typically evaluated. In particular, the focus will be on the well-known
\(k\)-means (Lloyd's) algorithm. Clustering was chosen as it is a
well-understood problem that is easily accessible \-- especially when restricted
to two dimensions. The \(k\)-means algorithm is an iterative, centroid-based
method that aims to minimise the `inertia' of the current partition, \(Z =
\left\{Z_1, \ldots, Z_k\right\}\), of some dataset \(X\):
\begin{equation}
    I(Z, X) := \frac{1}{|X|} \sum_{j=1}^{k} \sum_{x \in Z_j} {d(x, z_j)}^2
    \label{eq:inertia}
\end{equation}

A full statement of the algorithm to minimise~(\ref{eq:inertia}) is given
in~\ref{app:kmeans}.

This inertia function is taken as the objective of the \(k\)-means algorithm,
and is used for evaluating the final clustering. This is particularly true when
the algorithm is not being considered an unsupervised classifier where accuracy
may be used~\cite{Huang1998}. With that, the first example will use this inertia
as the fitness function in EDO.\ That is, to find datasets which minimise \(I\).

For the purposes of visualisation, EDO is restricted to the space containing
only two-dimensional datasets, i.e.\ \(C = \left((2, 2)\right)\). In addition to
this, all columns are formed from uniform distributions where the bounds are
sampled from the unit interval. Thus, the only family in \(\mathcal{P}\) is:
\begin{equation}
    \mathcal{U} := \left\{U(a, b)~|~a, b \in [0, 1]\right\}
\end{equation}

The remaining parameters are as follows: \(N~=~100\), \(R~=~(3, 100)\),
\(M~=~1000\), \(b~=~0.2\), \(l~=~0\), \(p_m~=~0.01\), and shrinkage is excluded.
Figure~\ref{fig:small-inertia-50} shows an example of the fitness (above) and
dimension (below) progression of the evolutionary algorithm under these
conditions up until the \(50^{th}\) epoch.

There is a steep learning curve here; within the first 50 generations an
individual is found with a fitness of roughly \(10^{-10}\) which could not be
improved on for a further 900 epochs. The same quick convergence is seen in the
number of rows. This behaviour is quickly recognised as preferable and was
dominant across all the trials conducted in this work. This preference for
datasets with fewer rows is expected given that \(I\) is the sum of the mean
error from each cluster centre. With that, when \(k\) is fixed \textit{a
priori}, reducing the number of points in each cluster (i.e.\ the terms of the
second summation) quickly reduces the mean error of that cluster and thus the
value of \(I\).

\addtocounter{figure}{1}
\begin{figure}[htbp]
    \ContinuedFloat%
    \centering
    \begin{tabular}{c}
        \includegraphics[width=\imgwidth]{Fig7a-1.pdf}
        \\
        \includegraphics[width=\imgwidth]{Fig7a-2.pdf}
    \end{tabular}
    \caption{%
        Progressions for final inertia and dimension across the first 50
        epochs with \(R~=~(3,100)\).
    }\label{fig:small-inertia-50}
\end{figure}

\begin{figure}[htbp]
    \ContinuedFloat%
    \centering
    \begin{tabular}{c}
        \includegraphics[width=\imgwidth]{Fig7b-1.pdf}
        \\
        \includegraphics[width=\imgwidth]{Fig7b-2.pdf}
    \end{tabular}
    \caption{%
        Progressions for final inertia and dimension across the first 50 epochs
        with \(R~=~(50,100)\).
    }\label{fig:large-inertia-50}
\end{figure}

However, something that may be seen as unwanted is a compaction of the cluster
centres. Referring to Figure~\ref{fig:small-inertia-inds}, the best and median
individuals show two clusters that are essentially the same point whereas the
worst is a random cloud across the whole of \(\mathcal{U}\) which was found in
the initial population. The kind of behaviour exhibited by the best performing
individuals here occurs in part because it is allowed. There are two immediate
ways in which this allowed: first, that a near-trivial case is included in \(R\)
and, secondly, that the fitness function does nothing to penalise the proximity
of the inter-cluster means, as well as aiming to reduce the intra-cluster means.
This kind of unwanted behaviour highlights a subtlety in how EDO should be used;
that experimentation and rigour are required to properly understand an
algorithm's quality.

\begin{figure}[htbp]
    \centering
    \subfloat[][]{%
        \label{fig:small-inertia-inds}
        \centering
        \includegraphics[width=\imgwidth]{Fig8a.pdf}
    }\\

    \subfloat[][]{%
        \label{fig:large-inertia-inds}
        \centering
        \includegraphics[width=\imgwidth]{Fig8b.pdf}
    }
    \caption[]{%
        Representative individuals based on inertia with:
        \subref{fig:small-inertia-inds} \(R~=~(3,100)\);
        \subref{fig:large-inertia-inds} \(R~=~(50,100)\). Centroids displayed as
        crosses.
    }\label{fig:inertia-inds}
\end{figure}

Hence, consider Figure~\ref{fig:large-inertia-inds} where the individuals have
been generated with the same parameters as previously except with adjusted row
limits, \(R = (50, 100)\), so as to exclude this trivial case. In these trials,
the results are equivalent: the worst performing individuals are without
structure whilst the best-performing individuals display clusters that are dense
about a single point despite the minimum number of rows being increased.
Supposing this was not already a known result, we can see mounting evidence in
favour of this compaction being `optimal' behaviour in a dataset for \(k\)-means
clustering.

However, the fitness function may be addressed still, and more extensive
studying may be done. Indeed, the final inertia could be considered a flawed or
fragile fitness function if it is supposed to evaluate the efficacy of the
\(k\)-means algorithm. Incorporating the inter-cluster spread to the fitness
of an individual dataset would reduce this observed compaction. For instance,
the silhouette coefficient is a metric used to evaluate the appropriateness of a
clustering to a dataset and does precisely that. The silhouette coefficient of a
clustering of a dataset is given by the mean of the silhouette value,
\(S(x)\), of each point \(x \in Z_j\) in each cluster:
\begin{equation}
    \begin{gathered}
        A(x) := \frac{1}{|Z_j| - 1} \sum_{y \in Z_j \setminus \{x\}} d(x, y),
        \\
        B(x) := \min_{k \neq j} \frac{1}{|Z_k|} \sum_{w \in Z_k} d(x, w),
        \\
        S(x) :=
            \begin{cases}
                \frac{B(x) - A(x)}{\max\left\{A(x), B(x)\right\}}
                &\quad \text{if } |Z_j| > 1\\
                0 &\quad \text{otherwise}
            \end{cases}
    \end{gathered}\label{eq:silhouette}
\end{equation}\\

The optimisation of the silhouette coefficient is analogous to finding a dataset
which increases both the intra-cluster cohesion (the inverse of \(A\)) and
inter-cluster separation (\(B\)). Hence, the objective of minimising inertia is
addressed by maximising cohesion. Meanwhile, the additional desire to spread out
the clusters is considered by maximising separation.

Repeating the trials with the same parameters as with inertia, the silhouette
fitness function yields the results summarised in
Figures~\ref{fig:small-silhouette}~and~\ref{fig:large-silhouette}. Irrespective
of row limits, the datasets produced show increased separation from one another
whilst maintaining low values in the final inertia of the clustering as shown in
Figure~\ref{fig:silhouette-inds}. Again, the form of the individual clusters is
much the same. The low values of inertia correspond to tight clusters, and the
tightest clusters are those with a minimal number of points, i.e.\ a single
point. As with the previous example, albeit at a much slower rate, the
preferable individuals are those leading toward this case. That this gradual
reduction in the dimension of the individuals occurs despite adjusting the
fitness function and considering the space which excludes the trivial case
bolsters the claim that the base case is also optimal.

At this point, it should be noted that, due to the nature of the implementation,
any individual from any generation may be retrieved and studied should the final
results be too concentrated on any given case. The summary provided here is one
particular way of studying the body of datasets generated with this method and
this transparency in the history and progression of the proposed method is
something that sets it apart from other methods such as GANs which have a
reputation of providing so-called `black box' solutions.

\addtocounter{figure}{1}
\begin{figure}[htbp]
    \ContinuedFloat%
    \centering
    \begin{tabular}{c}
        \includegraphics[width=\imgwidth]{Fig9a-1.pdf}
        \\
        \includegraphics[width=\imgwidth]{Fig9a-2.pdf}
    \end{tabular}
    \caption{%
        Progressions for silhouette and dimension across 1000 epochs at 100
        epoch intervals with \(R~=~(3, 100)\).
    }\label{fig:small-silhouette}
\end{figure}

\begin{figure}
    \ContinuedFloat%
    \centering
    \begin{tabular}{c}
        \includegraphics[width=\imgwidth]{Fig9b-1.pdf}
        \\
        \includegraphics[width=\imgwidth]{Fig9b-2.pdf}
    \end{tabular}
    \caption{%
        Progressions for silhouette and dimension across 1000 epochs at 100
        epoch intervals with \(R~=~(50,100)\).
    }\label{fig:large-silhouette}
\end{figure}

\begin{figure}[htbp]
    \centering
    \subfloat[][]{%
        \label{fig:small-silhouette-inds}
        \centering
        \includegraphics[width=\imgwidth]{Fig10a.pdf}
    }\\

    \subfloat[][]{%
        \label{fig:large-silhouette-inds}
        \centering
        \includegraphics[width=\imgwidth]{Fig10b.pdf}
    }
    \caption[]{%
        Representative individuals based on silhouette with:
        \subref{fig:small-silhouette-inds} \(R~=~(3,100)\);
        \subref{fig:large-silhouette-inds} \(R~=~(50,100)\). Centroids displayed
        as crosses.
    }\label{fig:silhouette-inds}
\end{figure}

\subsection{Comparison with DBSCAN}\label{subsec:dbscan}

The extent of the capabilities EDO holds as a tool to better understand an
algorithm are especially apparent when comparing an algorithm against another
(or set of others) simultaneously. This is done by utilising the freedom of
choice in a fitness function for EDO.\ Consider two algorithms, \(A\) and \(B\),
and some common metric between them, \(g\). Then their similarities and
contrasts can be explored by considering the differences in this metric on the
two algorithms. In terms of EDO, this means using \(f = g_A - g_B\), \(f = g_B -
g_A\) or \(f = \left| g_B - g_A \right|\) as the fitness function. By doing so,
pitfalls, edge cases or fundamental conditions for the method may be
highlighted. Overall, this process allows the researcher to more deeply learn
about the method of interest beyond the traditional method of literature
comparison on a particular example.

Consider the following use case with another clustering algorithm of a different
form, Density Based Spatial Clustering of Applications with Noise (DBSCAN). In
this particular case, the objective is to find datasets for which the method of
interest, \(k\)-means, outperforms its alternative, DBSCAN.\ Here there is no
concept of inertia as DBSCAN is density-based and is able to identify
outliers~\cite{Ester1996}. As such, a valid metric must be chosen. One such
metric is the silhouette score as defined in~(\ref{eq:silhouette}).

In this case, however, an adjustment to the fitness function must be made so as
to accommodate for the condition of the silhouette coefficient that there must
be more than one cluster present. Let \(S_k (X)\) and \(S_D (X)\) denote the
silhouette coefficients of the clustering found by \(k\)-means and DBSCAN
respectively. Then the fitness function is defined to be:
\begin{equation}
    f(X) =
        \begin{cases}
            S_D (X) - S_k (X), &\quad \text{%
                \begin{tabular}{l}%
                    if DBSCAN identifies two or
                    \\
                    more clusters (inc.\ noise)
                \end{tabular}
            }\\
            \infty &\quad \ \ \text{otherwise.}
        \end{cases}\label{eq:dbscan-fitness}
\end{equation}

There are several remarks to be made here. First, note the order of the
subtraction here as EDO minimises fitness functions by default. Also, \(f\)
takes values in the range \([-2, 2]\) where \(-2\) is the best, i.e.\ \(S_D(X) =
-1\) and \(S_k(X) = 1\). Likewise, 2 is the worst score. Finally, the silhouette
coefficient requires at least two clusters to be present and so if DBSCAN
identifies a single cluster then that individual will be penalised heavily under
this fitness function when, in fact, that clustering may be of high quality. As
such, this fitness function may require adjustment.

It must also be acknowledged that \(k\)-means and DBSCAN share no common
parameters and so direct comparison is more difficult. For the purposes of this
example, only one set of parameters is used but a thorough investigation should
include a parameter sweep in similar, real-world use cases. The parameters being
used are \(k~=~3\) for \(k\)-means, and \(\epsilon~=~0.1,\ MinPoints~=~5\) for
DBSCAN.\ This set was chosen following informal experimentation using the Python
library Scikit-learn~\cite{scikit} to find comparable parameters in the given
search space defined by the EDO parameters used previously with
\(R~=~(50,100)\).

\begin{figure}[htbp]
    \centering
    \begin{tabular}{c}
        \includegraphics[width=\imgwidth]{Fig11-1.pdf}
        \\
        \includegraphics[width=\imgwidth]{Fig11-2.pdf}
    \end{tabular}
    \caption{%
        Progressions for difference in silhouette (\(k\)-means-preferable) and
        dimension across 1000 epochs at 100 epoch intervals.
    }\label{fig:dbscan-silhouette}
\end{figure}

\begin{figure}[htbp]
    \centering
    \subfloat[][]{%
        \label{fig:dbscan-inds-k}
        \centering
        \includegraphics[width=\imgwidth]{Fig12a.pdf}
    }\\
    \subfloat[][]{%
        \label{fig:dbscan-inds-d}
        \centering
        \includegraphics[width=\imgwidth]{Fig12b.pdf}
    }
    \caption[]{%
        Representative individuals from a \(k\)-means-preferable run with
        clustering by: \subref{fig:dbscan-inds-k} \(k\)-means;
        \subref{fig:dbscan-inds-d} DBSCAN.\ Concave and convex hulls illustrated
        by shading and outline respectively.
    }\label{fig:dbscan-inds}
\end{figure}

Figure~\ref{fig:dbscan-silhouette} shows a summary of the progression of EDO
for this use case. As with the previous examples where \(R~=~(50, 100)\), the
variation in the population fitness is unstable but there is a clear trend of
improvement in the best individual over the course of the run. There is also a
convergence seen in the number of rows a dataset has. The resting dimension
varied across the trials conducted in this work but none exhibited a dramatic
shift toward the lower limit of 50 rows as with previous examples. This is
suggestive of a more competitive environment for individuals where slight
changes to an individual can drastically alter their fitness.

The effect of such changes can be seen in Figure~\ref{fig:dbscan-inds} where
representative individuals are shown for this example. Here, the best performing
individual, when clustered by \(k\)-means, shows three clear and nicely
separated clusters. Note that they are not so tightly packed; again, this
suggests that the route to an optimal individual is less clearly defined. In
contrast, when the same dataset is clustered by DBSCAN a single cluster is found
with a single noise point held within the convex hull of the cluster, i.e.\
there are overlapping clusters (since noise points form a single cluster).
Hence, along with the fact that the larger cluster is widely spread, it follows
that the clustering has a relatively small, negative silhouette coefficient.

Another point of interest here is the convexity of the clusters. A known
condition for the success of \(k\)-means is that the presented clusters are of
roughly equal size and are convex. This is due to the overall objective being to
approximate the centroidal Voronoi tessellation~\cite{Du2006}. Without this
condition, up to the correct choice of \(k\), the algorithm will fail to produce
adequate results for either inertia or silhouette. DBSCAN, on the other hand,
does not have this condition and is able to detect non-convex clusters so long
as they are dense enough. Figure~\ref{fig:dbscan-inds} shows the clustering
found by each method and the respective convex and concave hulls of the clusters
found. The `concave hull' of a cluster is taken to be the \(\alpha\)-shape of
the cluster's data points~\cite{Edelsbrunner1983} where \(\alpha\) is determined
to be the smallest value such that all the points in the cluster are contained
in a single polygon. The convexity of cluster \(Z_j\), denoted
\(\mathcal{C}_j\), is then determined to be the ratio of the area of its concave
hull, \(H_c\), to the area of its convex hull, \(H_v\)~\cite{Sonka1993}:
\begin{equation}
    \mathcal{C}_j := \frac{area(H_c)}{area(H_v)}
\end{equation}

With this definition, it should be clear that a perfectly convex cluster, such
as a single point or line, would have \(\mathcal{C}_j = 1\).

It can be seen that the convexity of the clustering found by \(k\)-means appears
to be higher than that by DBSCAN.\ This was apparent across all trials conducted
in this work and indicates that the condition for convex clusters is being
sought out during the optimisation process. Meanwhile, however, it is not clear
whether the performance of DBSCAN falls owing to its parameters or the method
itself. This is a point where parameter sweeping would prove most useful so as
to determine a crossing point for these two driving forces.

Now, to add to the discussion above, the inverse optimisation should be
considered. That is, using the same parameters, the datasets for which DBSCAN
outperforms \(k\)-means with respect to the silhouette coefficient are to be
investigated. This is equivalent to using \(-f\) as the fitness function
except with the same penalty of \(\infty\) for the case set out
in~(\ref{eq:dbscan-fitness}).

\begin{figure}[htbp]
    \centering
    \begin{tabular}{c}
        \includegraphics[width=\imgwidth]{Fig13-1.pdf}
        \\
        \includegraphics[width=\imgwidth]{Fig13-2.pdf}
    \end{tabular}
    \caption{%
        Progressions for difference in silhouette (DBSCAN-preferable) and
        dimension across 1000 epochs at 100 epoch intervals.
    }\label{fig:negative-prog}
\end{figure}

\begin{figure}[htbp]
    \centering
    \subfloat[][]{%
        \label{fig:neg-inds-k}
        \centering
        \includegraphics[width=\imgwidth]{Fig14a.pdf}
    }\\
    \subfloat[][]{%
        \label{fig:neg-inds-d}
        \centering
        \includegraphics[width=\imgwidth]{Fig14b.pdf}
    }
    \caption[]{%
        Representative individuals from a DBSCAN-preferable run with clustering
        by: \subref{fig:dbscan-inds-k} \(k\)-means; \subref{fig:dbscan-inds-d}
        DBSCAN.\ Concave and convex hulls illustrated by shading and outline
        respectively.
    }\label{fig:negative-inds}
\end{figure}

Figures~\ref{fig:negative-prog}~and~\ref{fig:negative-inds} show the same
summary as above with the revised fitness function. Inspecting the former, it is
seen that the best fitness found is worse than with the previous example. This,
in part, is due to the fact that \(k\)-means cannot find a clustering with
negative values as no clusters may overlap. It can, however, produce results
with small silhouette scores where the clusters are tightly packed. Hence, the
best fitness score is now \(-1\) whereas the worst is 2, still.

Note in the first two frames of Figure~\ref{fig:neg-inds-k} how \(k\)-means is
forced to split what is evidently a single cluster in two whereas DBSCAN is able
to identify the single cluster and the outlying noise
(Figure~\ref{fig:neg-inds-d}). The proximity of these clusters has then dragged
the silhouette score down for \(k\)-means. Referring to
Figure~\ref{fig:neg-inds-d}, this kind of behaviour is certainly preferable for
DBSCAN under these parameters: the beginning individuals are likely random
clouds (as seen in the rightmost two frames of the figure) and the simplest step
toward a fit dataset is one that maintains that vaguely dense body with minimal
noise points far from it.

As has already been stated, the software implementation of the EDO method
has been produced in line with the best practices of open source software
development and reproducible research. In aid of this, all of the source code
used in these examples (including to create the figures) has been archived
under the DOI
\href{https://doi.org/10.5281/zenodo.3492236}{10.5281/zenodo.3492236}.
Likewise, all of the data produced to support this case study have been archived
under the DOI
\href{https://doi.org/10.5281/zenodo.3492228}{10.5281/zenodo.3492228}.


\section{Exercise 01}
\subsection{}

\begin{frame}
\frametitleTC{Direct synthesis for set point tracking}
\framesubtitleTC{Some examples}
\myPause
 \begin{itemize}[<+-| alert@+>]
 \item Example 1 (OK):
       \begin{displaymath}
        \begin{array}{lcl}
         P(z)              = \frac{(z-0.5)}{(z-0.4)^2},\,
         G_{yw}^{\circ}(z) = \frac{0.2}{z-0.8} &
         \Rightarrow &
         C(z)              = \frac{0.2(z-0.4)^2}{(z-1)(z-0.5)}.
        \end{array}
       \end{displaymath}
 \item Example 2 (\red{NO}, process pole outside the circle $\Rightarrow$ unstable hidden part):
       \begin{displaymath}
        \begin{array}{lcl}
         P(z)              = \frac{1}{\red{z-2}},\,
         G_{yw}^{\circ}(z) = \frac{0.2}{z-0.8} &
         \Rightarrow &
         C(z)              = \frac{0.2\red{(z-2)}}{z-1}.
        \end{array}
       \end{displaymath}
 \item Example 3 (\red{NO}, infeasible relative degree):
       \begin{displaymath}
        \begin{array}{lcl}
         P(z)              = \frac{1}{(z-0.4)^2},\,
         G_{yw}^{\circ}(z) = \frac{0.2}{z-0.8} &
         \Rightarrow &
         C(z)              = \frac{0.2(z-0.4)^{\textcolor{red}{2}}}{z-1}.
        \end{array}
       \end{displaymath}
 \item Try one/two cases of your own, verify that everything is clear,\\
       ask questions if necessary.
 \end{itemize}
\end{frame}

\begin{frame}
\frametitleTC{Direct synthesis for set point tracking}
\framesubtitleTC{Dealing with the limitations}
\myPause
 \begin{itemize}[<+-| alert@+>]
 \item If the process has poles on or outside the circle, we must preserve them in the desired loop transfer
       function
       \begin{displaymath}
        L^{\circ}(z) = \frac{G_{yw}^{\circ}(z)}{1-G_{yw}^{\circ}(z)}
       \end{displaymath}
       so as to not have $C(z)$ cancel them and generate an unstable hidden part.
 \item If the process has zeroes on or outside the circle, we must include them in $G_{yw}^{\circ}(z)$\\
       -- hence in $L^{\circ}(z)$ -- so that $C(z)$ does not cancel them either; this may\\
       require to also add poles to have at least as many as there are zeroes.
 \item If the relative degree of $G_{yw}^{\circ}(z)$ is infeasible, we need to add ``fast''\\
       (near to the origin) poles and/or delay terms.
 \item\vfill Let us clarify with a few simulation examples.
 \end{itemize}
\end{frame}

\begin{frame}[fragile]
\frametitleTC{Direct synthesis for set point tracking}
\framesubtitleTC{Dealing with the limitations -- Scilab script (1/3, copy{\&}paste to try at home)}
{\tiny
\def\baselinestretch{0.3}
\begin{verbatim}
// Direct synthesis examples -- set point tracking
clear; clc; z = %z;
// Reference and time vectors
w  = ones(1,30);
k  = 0:length(w)-1;
// Process 1, asymptotically stable, no zeroes
P1 = syslin('d',2/(z-0.5)^2);
// Infeasible relative degree
To = syslin('d',0.2/(z-0.8));
R  = 1/P1*To/(1-To); disp(R); y1 = dsimul(tf2ss(To),w);
// Adding a fast pole for a realisable R
To = syslin('d',0.16/(z-0.8)/(z-0.2));
R  = 1/P1*To/(1-To); disp(R); y2 = dsimul(tf2ss(To),w);
// Adding a faster pole
To = syslin('d',0.19/(z-0.8)/(z-0.05));
R  = 1/P1*To/(1-To); disp(R); y3 = dsimul(tf2ss(To),w);
// Adding a one-step delay
To = syslin('d',0.2/(z-0.8)/z);
R  = 1/P1*To/(1-To); disp(R); y4 = dsimul(tf2ss(To),w);
// Using two poles, both faster than the design one
p1 = 0.75;
p2 = 0.45;
To = syslin('d',(1-p1)*(1-p2)/(z-p1)/(z-p2));
R  = 1/P1*To/(1-To); disp(R); y5 = dsimul(tf2ss(To),w);
\end{verbatim}
}
\end{frame}

\begin{frame}[fragile]
\frametitleTC{Direct synthesis for set point tracking}
\framesubtitleTC{Dealing with the limitations -- Scilab script (2/3)}
{\tiny
\def\baselinestretch{0.3}
\begin{verbatim}
// Process 2, asymptotically stable,, 1 zero outside the circle
P2 = syslin('d',(z-2)/(z-0.5)^2);
// Infeasible To
To = syslin('d',0.2/(z-0.8));
R  = 1/P2*To/(1-To); disp(R); y6 = dsimul(tf2ss(To),w);
// Adding the required zero to To
To = syslin('d',-0.18*(z-2)/(z-0.8)/(z-0.1));
R  = 1/P2*To/(1-To); disp(R); y7 = dsimul(tf2ss(To),w);
// Improving by acting on the To poles
p1 = 0.78;
p2 = 0.02;
To = syslin('d',-(1-p1)*(1-p2)*(z-2)/(z-p1)/(z-p2));
R  = 1/P2*To/(1-To); disp(R);
y8 = dsimul(tf2ss(To),w);
\end{verbatim}
}
\end{frame}

\begin{frame}[fragile]
\frametitleTC{Direct synthesis for set point tracking}
\framesubtitleTC{Dealing with the limitations -- Scilab script (3/3)}
{\tiny
\def\baselinestretch{0.3}
\begin{verbatim}
// Plot the results for process 1
h               = scf(0); clf;
h.figure_size   = [500,500];
title("${\Large \text{Cases 1--5}$");
xlabel("k");
plot(k,y1,'r',k,y1,'r.');
plot(k,y2,'b',k,y2,'b.');
plot(k,y3,'m',k,y3,'m.');
plot(k,y4,'g',k,y4,'g.');
plot(k,y5,'k',k,y5,'k.');
ax              = gca();
ax.data_bounds  = [0,0;max(k),1.1];
ax.tight_limits = "on";
// Plot the results for process 2
h               = scf(1); clf;
h.figure_size   = [500,500];
title("${\Large \text{Cases 6--8}$");
xlabel("k");
plot(k,y6,'r',k,y6,'r.');
plot(k,y7,'b',k,y7,'b.');
plot(k,y8,'k',k,y8,'k.');
ax              = gca();
ax.data_bounds  = [0,-0.3;max(k),1.1];
ax.tight_limits = "on";
\end{verbatim}
}
\end{frame}

\begin{frame}
\frametitleTC{Synthesis for set point tracking}
\framesubtitleTC{Dealing with the limitations -- summary}
\myPause
\begin{center}
 {\scriptsize
 \begin{tabular}{llll}
  \hline
Case & Process                   & Requirement                                                           & Controller  \\
\hline\hline
1& $P_1(z)=\frac{2}{(z-0.5)^2}$  & $G_{yw,11}^{\circ}(z)=\frac{0.2}{z-0.8}$                              & Infeasible  \\
2&                               & $G_{yw,12}^{\circ}(z)=\frac{0.16}{(z-0.8)(z-0.2)}$                    & $C_{12}(z)$ \\
3&                               & $G_{yw,13}^{\circ}(z)=\frac{0.19}{(z-0.8)(z-0.05)}$                   & $C_{13}(z)$ \\
4&                               & $G_{yw,14}^{\circ}(z)=\frac{0.2}{z(z-0.8)}$                           & $C_{14}(z)$ \\
5&                               & $G_{yw,15}^{\circ}(z)=\frac{(1-0.75)(1-0.45)}{(z-0.75)(z-0.45)}$      & $C_{15}(z)$ \\
\hline
6& $P_2(z)=\frac{z-2}{(z-0.5)^2}$& $G_{yw,21}^{\circ}(z)=\frac{0.2}{z-0.8}$                              & Infeasible  \\
7&                               & $G_{yw,22}^{\circ}(z)=-\frac{0.18(z-2)}{(z-0.8)(z-0.1)}$              & $C_{22}(z)$ \\
8&                               & $G_{yw,23}^{\circ}(z)=-\frac{(1-0.78)(1-0.02)(z-2)}{(z-0.78)(z-0.02)}$& $C_{23}(z)$ \\
  \hline
 \end{tabular}
 }
\end{center}
\end{frame}

\begin{frame}
\frametitleTC{Synthesis for set point tracking}
\framesubtitleTC{Dealing with the limitations -- results}
\myPause
\begin{center}
 {\scriptsize
 \begin{tabular}{ll}
  \includegraphics[width=0.35\textwidth]{./Unit-05/img/PS02-ex01-res-1to5.pdf} &
  \includegraphics[width=0.35\textwidth]{./Unit-05/img/PS02-ex01-res-6to8.pdf}      \\
   \red{$P_1,G_{yw,11}^{\circ}\Rightarrow$ infeasible} &  \red{$P_2,G_{yw,21}^{\circ}\Rightarrow$ infeasible} \\
  \blue{$P_1,G_{yw,12}^{\circ}\Rightarrow$ $C_{12}$}   & \blue{$P_2,G_{yw,22}^{\circ}\Rightarrow$ $C_{22}$}   \\
   \mgt{$P_1,G_{yw,13}^{\circ}\Rightarrow$ $C_{13}$}   &       $P_2,G_{yw,23}^{\circ}\Rightarrow$ $C_{23}$    \\
   \grn{$P_1,G_{yw,14}^{\circ}\Rightarrow$ $C_{14}$}   \\
        $P_1,G_{yw,15}^{\circ}\Rightarrow$ $C_{15}$    \\
 \end{tabular}
 }
\end{center}
\end{frame}

\begin{frame}
\frametitleTC{Synthesis for set point tracking}
\framesubtitleTC{Summary}
\myPause
 \begin{itemize}[<+-| alert@+>]
 \item The technique is powerful, however there are \TC{objective} limits.
 \item In many of the cases seen the controller has one or two zeroes and one or two poles,
       one of which in $z=1$.
 \item We shall soon call this a PI(D).
 \item \vfill You can try at home: open Scilab, launch the code editor SciNotes\\
       from the \texttt{Applications} menu, enter the code from the previous\\
       slides (copy{\&}paste) and run with the \texttt{Execute} command\\
       --- all IDEs look pretty much the same, in the end...
 \end{itemize}
\end{frame}


\documentclass[10pt,letterpaper]{article}

\usepackage{amsmath}
\usepackage{tikz}
\usepackage{hyperref}

\newcommand{\volume}{{\ooalign{\hfil$V$\hfil\cr\kern0.08em--\hfil\cr}}}

\author{Thaddeus Hughes \\ hughes.thad@gmail.com \\ thaddeus-maximus.github.io}
\date{\today}
\title{Adiabatic Model of a Pneumatic Cylinder}

\begin{document}
	\maketitle

	\begin{abstract}
		There are some basic models for pneumatic cylinders that result in modeling cylinders as constant-force devices, or nearly so. This is a rough first pass at creating a model that is a little more sophisticated. This model assumes that the air in the cylinder is adiabatic; that is, no heat transfer takes place. This model will be a differential equation intended to be used in conjunction with a numerical DE solver due to the non-linear nature of many conditions.

		Pneumatic pistons can be used for many scenarios- they can act as quick stops, can push or pull heavy loads, or can be configured to quickly launch objects. Depending on the nature of the system, they can behave quite differently, and this model may capture more of the underlying physics that affects these various use cases.
	\end{abstract}

	\section*{System Definition}

	We will analyze the open system that is the air contained within a pneumatic cylinder, plus the air in the hose leading up to it (inclusion of the hose will become of crucial importance and will be evident later on).
	\begin{center}
	\begin{tikzpicture}[x=1.0in,y=1.0in]
		\draw[->] (-0.5,0)--(-0.1,0) node[pos=0, left]{$P_{in}$, $\dot{\volume}_{in}$, $T_{in}$};

		\draw[] (0.3,-0.1)--(0,-0.1)--(0,0.1)--(0.3,0.1)--(0.3,0.5)--(1.2,0.5)--(1.2,-0.5)--(0.3,-0.5)--cycle;
		\node (state) [text width=0.4] at (0.65,0) {$m_s$ \\ $P_s$ \\ $\volume_s$ \\ $T_s$};
		\draw[->] (1.0, 0.6)--(1.4, 0.6) node[pos=0.5, above] {$v$, $x$};
		%\draw[->] (1.9,0)--(1.4,0) node[pos=0, above] {$F = P_{atm} A$};
	\end{tikzpicture}
	\end{center}

	Air at pressure $P_{in}$ and temperature $T_{in}$ enters the cylinder at rate $\dot{\volume}_{in}$.
	It mixes with air in the cylinder ($m_s$, $\volume_s$) and is assumed to become of uniform pressure and temperature $P_s$ and $T_s$.

	The piston ram is moving at velocity $v$ and with position $x$. $x$ is zero when the cylinder is bottomed out (no air in the cylinder).

	\begin{center}
	\begin{tikzpicture}[x=1.0in,y=1.0in]
		\draw[ultra thick] (0,-0.5)--(0,0.5);
		\draw[] (0,-0.15)--(0,0.15)--(1,0.15)--(1,-0.15)--cycle;

		\node (ctr) [] at (0.5,0) {$M$};

		\draw[->] (-0.3,0)--(-0.1,0) node[pos=0,left]{$P_s A$};
		\draw[->] (-0.3,0.2)--(-0.1,0.2); \draw[->] (-0.3,0.4)--(-0.1,0.4); \draw[->] (-0.3,-0.2)--(-0.1,-0.2); \draw[->] (-0.3,-0.4)--(-0.1,-0.4);

		\draw[->] (1.3,0)--(1.1,0) node[pos=0,right]{$F_{load}$};
		\draw[->] (0.3,0.3)--(0.1,0.3) node[pos=0,right]{$P_{atm} A$};
		\draw[->] (0.3,-0.3)--(0.1,-0.3);
	\end{tikzpicture}
	\end{center}

	The cylinder itself is considered to have a mass $M$ and sees the cylinder pressure, a resistive force $F_{load}$, and atmospheric pressure.

	\section*{Conservation Principles In The Cylinder}

	First, let's apply conservation of mass to the air.

	\begin{align}
		\frac{d}{dt} m_{sys} &= \sum_{+ in} \dot{m} \nonumber
	\end{align}\begin{align} \label{eq:ddt_m}
		\frac{d}{dt} m_s &= \dot{m}_{in}
	\end{align}

	We can use the ideal gas law $P \volume = m R T$ to write this in terms of knowns.
	\begin{align}
		\dot{m}_{in} = \frac{P_{in} \dot{\volume}_{in}}{R T_{in}} \nonumber
	\end{align}

	We'll assume that the primary flow restriction comes from the tube. We can use an equation derived from \href{https://www.engineeringtoolbox.com/pressure-drop-compressed-air-pipes-d_852.html}{this empirical pipe flow equation in The Engineering ToolBox}.

	\begin{align}
		\dot{\volume} = 0.0197862 (d_{tube}^5 [P_{in}-P_{atm}] [P_{in}-P_{s}]/L_{tube})^{20/37} \nonumber
	\end{align}
	(Assuming units of $Pa$, $m$, and $m^3/s$)

	\begin{align} \label{eq:sup_mdot}
		\dot{m}_{in} = \frac{0.0197862}{R T_{in}} (d_{tube}^5 [P_{in}-P_{atm}] [P_{in}-P_{s}]/L_{tube})^{20/37}
	\end{align}

	Now, let's apply conservation of energy.

	\begin{align}
		\frac{d}{dt} E_{sys} = \sum \dot{Q}_{in} + \sum \dot{W}_{in} + \sum_{+in} \dot{m} (v^2/2 + gz + h) \nonumber
	\end{align}

	We will assume:
	\begin{itemize}
		\item The energy of the system is entirely thermal; $E_sys$ = $m_s u(T)$
		\item No heat transfer; $\dot{Q}_{in} = 0$
		\item Work is out of the system, defined by the pressure of the system and velocity of the ram; $\dot{W}_{in} = - \vec{F} \cdot \vec{v} = - P_s A v$
		\item Mass transfer into the system has negligible velocity ($v$) and head height ($gz$), leaving only enthalpy $h(T)$.
	\end{itemize}

	\begin{align}
		\frac{d}{dt} [m_s u(T)] = - P_s A v + \dot{m}_{in} (h(T_{in} + (\frac{\dot{\volume}}{\pi / 4 \ d_{tube}})^2) \nonumber
	\end{align}

	The derivative here is quite ugly, since both the mass and temperature of the system are changing. However, it can be broken up with the product rule ($\frac{d}{dt}[xy] = y \frac{dx}{dt} + x \frac{dy}{dt}$).

	\begin{align}
		m_s \frac{d u(T_s)}{dt} + u(T_s) \frac{d m_s}{dt} = - P_s A v + \dot{m}_{in}  (\frac{\dot{\volume}}{\pi / 4 \ d_{tube}})^2 + \dot{m}_{in} h(T_{in}) \nonumber \\
		m_s \frac{d u(T_s)}{dt} = \dot{m}_{in} h(T_{in}) - P_s A v - u(T_s) \frac{d m_s}{dt} + \dot{m}_{in}  (\frac{\dot{\volume}}{\pi / 4 \ d_{tube}})^2\nonumber
	\end{align}

	Recognizing that $u(T)$ and $h(T)$ can be approximated as $u(T) = c_p T$ and $h(T) = c_v T$, we can then solve the equation for $\frac{dT_s}{dt}$.

	\begin{align} \label{eq:ddt_T}
		\frac{d T_s}{dt} = \frac{ \dot{m}_{in}[ c_v T_{in} - c_p T_s + (\frac{\dot{\volume}}{\pi / 4 \ d_{tube}})^2]     - P_s A v}{c_p m_s}
	\end{align}

	All of these are knowns or states aside from $P_s$, which can be determined with the ideal gas law, and the modeling of the system volume as based on cylinder extension, crossectional area, and initial (i.e. hose) dead volume.

	\begin{align}
		P_s = \frac{m_s R T_s}{\volume_s} \nonumber
	\end{align}\begin{align} \label{eq:sup_P}
		P_s = \frac{m_s R T_s}{\volume_{dead} + x A}
	\end{align}

	\section*{Conservation Principles on the Piston}

	We can simply apply conservation of linear momentum to the piston.

	\begin{align}
		\frac{d}{dt} P_{sys}= \sum F + \sum \dot{m} v \nonumber
	\end{align}

	We will assume:
	\begin{itemize}
		\item No mass transfer in this system, so $\frac{d}{dt} P_{sys} = M \frac{dv}{dt}$ and $\dot{m}=0$
		\item The forces acting on the piston are the cylinder pressure, atmospheric pressure, and the external load.
	\end{itemize}

	\begin{align}
		M \frac{dv}{dt} &= P_s A - P_{atm} A - F_{load} \nonumber
	\end{align}\begin{align}  \label{eq:ddt_v}
		\frac{dv}{dt} &= \frac{(P_s - P_{atm}) A - F_{load}}{M}
	\end{align}

	We are also interested in the position of the piston.

	\begin{align} \label{eq:ddt_x}
		\frac{dx}{dt} = v
	\end{align}

	\section*{Limiting the Model}
	In review we have:

	\begin{itemize}
		\item A model for rate of change of $m_s$
		\item A model for rate of change of $T_s$
		\item Supporting equation for cylinder pressure $P_s$
		\item Supporting equation for massflow into cylinder $\dot{m}$
		\item A model for velocity $v$
		\item A model for position $x$
	\end{itemize}

	We simply now need initial conditions, "bumper" conditions, and termination criteria.

	We will assume the initial temperature $T_s$ will be set to that of the input gas $T_{in}$. (This may seem like it would negate all the point of this model, but recall that when gases expand/contract, they change pressure)

	\begin{align} \label{eq:ic_T}
		T_s(t=0) = T_{in}
	\end{align}

	The initial mass of cylinder air can be found with the ideal gas law, assuming we start at atmospheric pressure (wholly unpressurized)

	\begin{align} \label{eq:ic_m}
		m_s(t=0) = \frac{P_{atm} L_{tube} \frac{\pi}{4} d_{tube}^2}{R T_s(t=0)}
	\end{align}

%	There's an interesting edge case that our model doesn't look at, and that's when we overfill the cylinder. Air can't keep coming into our cylinder. An orifice model may be more appropriate, but let's just say that if the cylinder pressure is greater than the input pressure, no more air can be added.

%	\begin{equation} \label{eq:bump_pres}
%		\mbox{if } P_s > P_{in} \mbox{ then } \dot{m}_{in} = 0
%	\end{equation}

	Assume the piston starts from rest.

	\begin{align} \label{eq:ic_x}
		x(t=0) &= 0
	\end{align}\begin{align}\label{eq:ic_v}
		v(t=0) &= 0
	\end{align}

	And that the piston cannot go past its endstops at $x=x_f$ or $x=0$.

	\begin{align} \label{eq:bump_end}
		\mbox{if } x \geq x_f &\mbox{ then } x = x_f, v \leq 0
	\end{align}\begin{align}\label{eq:bump_start}
		\mbox{if } x \leq 0   &\mbox{ then } x = 0, v \geq 0
	\end{align}

	We will terminate our simulation when we reach the endstop and fully pressurize.

	\begin{align} \label{eq:term}
		\mbox{terminate simulation if } x \geq x_f \mbox{ and } P_s \geq P_{in}
	\end{align}

	\section*{The Full Model}

	\begin{align}
		\dot{m}_{in} &= \frac{0.0197862}{R T_{in}} (d_{tube}^5 [P_{in}-P_{atm}] [P_{in}-P_{s}]/L_{tube})^{20/37} \tag{\ref{eq:sup_mdot}} \\
		P_s &= \frac{m_s R T_s}{L_{tube} \frac{\pi}{4} d_{tube}^2 + x A} \tag{\ref{eq:sup_P}} \\
		\mbox{if } x \geq x_f &\mbox{ then } x = x_f, v \leq 0 \tag{\ref{eq:bump_end}} \\
		\mbox{if } x \leq 0   &\mbox{ then } x = 0, v \geq 0 \tag{\ref{eq:bump_start}} \\
		\frac{d}{dt} m_s &= \dot{m}_{in} \tag{\ref{eq:ddt_m}} \\
		\frac{d T_s}{dt} &= \frac{ \dot{m}_{in} c_v T_{in} - P_s A v - c_p T_s \dot{m}_{in} }{c_p m_s} \tag{\ref{eq:ddt_T}} \\
		\frac{dv}{dt} &= \frac{(P_s - P_{atm}) A - F_{load}}{M} \tag{\ref{eq:ddt_v}} \\
		\frac{dx}{dt} &= v \tag{\ref{eq:ddt_x}} \\
		T_s(t=0)& = T_{in} \tag{\ref{eq:ic_T}} \\
		m_s(t=0) &= \frac{P_{atm} L_{tube} \frac{\pi}{4} d_{tube}^2}{R T_s(t=0)} \tag{\ref{eq:ic_m}} \\
		v(t=0) &= 0 \tag{\ref{eq:ic_v}} \\
		x(t=0) &= 0 \tag{\ref{eq:ic_x}} \\
		\mbox{terminate}& \mbox{ if } x \geq x_f \mbox{ and } P_s \geq P_{in} \tag{\ref{eq:term}}
	\end{align}

	There's a lot going on here. But let's do a sanity check and see if there's going to be any issues going forth.

	The big equation of interest is $dT_s/dt$.
	\begin{itemize}
		\item If the heat capacity or system mass is larger, then the rate of temperature change decreases (makes sense)
		\item If the cylinder expands ($v > 0$) then the gas expands, lowering the temperature.
		\item Let's focus on this term
			\begin{align}
				  & \dot{m}_{in} c_v T_{in} - c_p T_s \dot{m}_{in} \nonumber \\
				=\ & \dot{m} [c_v T_{in} - c_p T_s] \nonumber \\
				=\ & \dot{m} [h(T_{in}) - u(T_s)] \nonumber \\
				=\ & \dot{m} [u(T_{in}) + P_{in} / \rho_{in} - u(T_s)] \nonumber \\
				\approx \ & \dot{m} [u(T_{in}-T_s) + P_{in} / \rho_{in}] \nonumber
			\end{align}

			Recognizing that $T_in > T_s$ since the cylinder is expanding, we see that yes, the massflow into the system corresponds to higher thermal energy transfer, and higher pressure energy transfer.
		\item The mass of the system $m_s$ is in the denominator - if this is zero, the derivative would not exist, so it is imperative that we begin our simulation with some mass (hence, the dead mass $L_{tube} \frac{\pi}{4} d_{tube}^2$ term)
	\end{itemize}

	\section*{Implemented Simulation and Examples}

	This has been implemented into my \href{http://thaddeus-maximus.github.io/swissarmyengineer/}{Swiss Army Engineer} suite. Here are some example cases, starting with a base case.

	\includegraphics[width=\textwidth]{pneu_case_1.png}

	This case has mass, and a load much higher than the mass (simulating, say, a piston pushing on a spring). The results aren't too interesting - air inrushes into the cylinder rather quickly, producing the quadratic position profile you'd expect of a constant-force system.

	\includegraphics[width=\textwidth]{pneu_case_2.png}

	This case increases the mass by ten times. The timescale, accordingly, stretches out, taking much longer to reach its end stop.

	\includegraphics[width=\textwidth]{pneu_case_3.png}

	This case removes the load and returns to the lighter system mass, so that this is a purely inertial system. In this case, the piston accelerates like normal, but the cylinder pressure actually begins to decrease as the inertia of the system carries it further forwards. Eventually the hardstop is hit, and at this point the piston can fully pressurize with air.

	What happens if we choke it out even further, by shrinking the line diameter?

	\includegraphics[width=\textwidth]{pneu_case_4.png}

	The line diameter here has been shrunk to 2mm. We see this same decayed acceleration as in the previous example, but to a larger degree.

	\includegraphics[width=\textwidth]{pneu_case_5.png}

	If we add a large load, the position curve becomes linear, as the cylinder has to fight the heavy load and its air supply eventually becomes the limiting factor on speed.

	\includegraphics[width=\textwidth]{pneu_case_6.png}

	If we make the load an expression, we could model a piston pulling on a spring, giving us this very neat mass-spring system.

	\section*{Validation}
		No validation has been performed yet on these models, as I don't have access to the equipment to test them. If anyone wants to perform this testing, it would be very cool!

	\section*{Future Improvements (and when to say no)}
		The next obvious improvement to make would be to model the flow restriction provided by valves. Beyond that, looking at pressure drop in the tank as air is consumed, or flow restriction as a result of the pressure regulator seems to me to be splitting hairs, and chasing high-speed actuation which would require significant testing anyways.

\end{document}


%! Author = tstreule

\section{SPECT \textnormal{-- Single Photon Emission CT}}

$\textrm{SNR} \propto \sqrt{\textrm{total \#detected gamma-rays}}$
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Radioactivity in the body}
%
\highlight{$Q = -\frac{dN}{dt} = \lambda \cdot N \implies N(t) = N_0 \eu^{-\lambda t}$}\\
$[Q] = \unit{Curie} = \unit{Ci} = 3.7 \cdot 10^{10} \unit{Bq}$
\quad where \quad $1\unit{Bq} = 1\unitfrac{disintegration}{s}$

\textbf{Physical} $t\ped{1/2} = \frac{\ln 2}{\lambda}$, \quad \textbf{biological} $t\ped{1/2 bio} = \frac{\ln 2}{\lambda\ped{bio}}$

$N(t) = N_0 \eu^{-(\lambda + \lambda\ped{bio}) t} = N_0\eu^{-\lambda\ped{1/2 eff}t}$
\quad where \quad $t\ped{1/2 eff} = \frac{t\ped{1/2} \cdot t\ped{1/2 bio}}{t\ped{1/2} + t\ped{1/2 bio}}$

\textbf{$\gamma$ photons: Absorption} or \textbf{Scattering}: Change of direction by $\theta$. \highlight{$E_{\gamma}' = \frac{m_e \cdot c^2}{m_e c^2 / E_\gamma + 1 - \cos \theta}$}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Interaction of $\gamma$ photons with matter}
%
\textbf{Absorption} leads to ejection of orbital electron\\
\textbf{Scattering}: Change of direction by $\theta$. \highlight{$\displaystyle E_{\gamma}' = \frac{m_e \cdot c^2}{m_e c^2 / E_\gamma + 1 - \cos \theta}$}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Anger Camera}
\textbf{Collimation} necessary $\implies$ losses of 99.9\%

\includegraphics[width = \linewidth]{SPECT_NuclearCollimator}
Bad isolation of \textbf{septa} (lamellae) $\implies L\ped{eff} = L - \frac{2}{\mu\ped{septa}}$\\
%
\highlight{$\displaystyle R_0(x) = \frac{2 r_{col}}{L_{\text{eff}}}(x + \frac{L_{\text{eff}}}{2})$} \; \highlight{$\displaystyle \textrm{FWHM} = \Delta r_{col}(x) = \frac{2 r_{col}}{L_{\text{eff}}}(x + L_{\text{eff}})$}\\

\textbf{Signal Path}: $\gamma$-rays @140keV $\to$ Scintillation crystal $\to$ PMT $\to$ (Pos. network || pulse height analyzer) $\to$ digitizer
\begin{itemize}
    \item Scintillation crystal efficiency: \highlight{$\displaystyle \epsilon = 1 - \eu^{-\mu d}$}. $\mu$: attenuation coefficient of crystal, $d$: its thickness

    \item Usually many PMTs per crystal. Positioning network calculates where the event in the crystal was.

    \item $U_{PMT} \propto \gamma$ energy. Scattering $\implies$ $E\ped{scattered photons}\downarrow$. Pulse height analyzer has energy window $\Delta E$ around $E_0$.

    \item Poisson distributed $P_n(N)\vert \sigma^2 = \mu \implies \highlight{SNR = \sqrt{\mu}}$

    \item \textbf{Spatial resolution}: $R_{system}^2 = R\ped{gamma}^2 + R_{coll}^2$ \quad $R\ped{gamma}$ = uncertainty of positioning network. $R\ped{coll}$ see above.
\end{itemize}

\textbf{CT Version}: Res.: best at edge, worst in center. $\approx$ 7mm\\

For small objects: Use \textbf{pinhole} (cone-shaped, tungsten) camera: Problem: more dose needed, ``transparency'' around hole
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Generation of Technetium}
No cyclotron needed, only generator: Essentially \ce{^{99}_{42}Mo} ($\unit[67]{h} \;\widehat{=}\; \unit[2.9\E{-6}]{s^{-1}}$) decays into \ce{^{99}_{43}Tc} ($\unit[6]{h} \;\widehat{=}\; \unit[3.1\E{-5}]{s^{-1}}$)

\textbf{Kinetic equations}:\\
$\deriv{~}{t}N_{Mo} = -k_{Mo}N_{Mo}$ and $\deriv{~}{t}N_{Tc} = k_{Mo}N_{Mo} - k_{Tc} N_{Tc}$\\
\highlight{$\displaystyle \implies N_{Tc}(t) = N_{Mo}(0) \frac{k_{Mo}}{k_{Tc} - k_{Mo}} (e^{-k_{Mo}t} - e^{-k_{Tc} t})$} {\scriptsize exp. decay}


% This is part of the TFTB Reference Manual.
% Copyright (C) 1996 CNRS (France) and Rice University (US).
% See the file refguide.tex for copying conditions.

\markright{margtfr}
\section*{\hspace*{-1.6cm} margtfr}

\vspace*{-.4cm}
\hspace*{-1.6cm}\rule[0in]{16.5cm}{.02cm}
\vspace*{.2cm}

{\bf \large \sf Purpose}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
Marginals and energy of a time-frequency representation.
\end{minipage}
\vspace*{.2cm}

{\bf \large \sf Synopsis}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
[margt,margf,E] = margtfr(tfr)
[margt,margf,E] = margtfr(tfr,t)
[margt,margf,E] = margtfr(tfr,t,f)
\end{verbatim}
\end{minipage}
\vspace*{.3cm}

{\bf \large \sf Description}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
        {\ty margtfr} calculates the time and frequency marginals and the
        energy of a time-frequency representation. The definitions used for
        the computation are the following\,:
\begin{eqnarray*}
m_f(t)=\int_{-\infty}^{+\infty} \mbox{tfr}(t,f)\ df && \mbox{\it time
marginal}\\
m_t(f)=\int_{-\infty}^{+\infty} \mbox{tfr}(t,f)\ dt && \mbox{\it frequency
marginal}\\
E= \int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty} \mbox{tfr}(t,f)\ df\ dt && \mbox{\it energy}
\end{eqnarray*}

\hspace*{-.5cm}\begin{tabular*}{14cm}{p{1.5cm} p{8.5cm} c}
Name & Description & Default value\\
\hline
        {\ty tfr} & time-frequency representation {\ty (M,N)}\\
        {\ty t}   & vector containing the time samples in sec.
               & {\ty (1:N)}\\
        {\ty f}   & vector containing the frequency samples in Hz, not
               necessary uniformly sampled & {\ty (1:M)}\\
 \hline {\ty margt} & time marginal\\
        {\ty margf} & frequency marginal\\
        {\ty E}     & energy of {\ty tfr}\\

\hline
\end{tabular*}

\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Example}
\vspace*{-.1cm}
\begin{verbatim}
         S=amgauss(128).*fmlin(128);
         [tfr,t,f]=tfrscalo(S,1:128,8,.05,.45,128,1);
         [margt,margf,E] = margtfr(tfr);
         subplot(211); plot(t,margt);
         subplot(212); plot(f,margf);
\end{verbatim}
\vspace*{.3cm}

{\bf \large \sf See Also}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
momttfr, momftfr.
\end{verbatim}
\end{minipage}


\chapter[Additional Derivations and Formulae]{Additional Derivations and \\ Formulae}
\label{app:derivations}
\stoptocwriting
This appendix outlines further derivations and formulae used throughout this thesis.

\section{Ellipse Geometry}
\label{app:derivellipse}

Section \ref{s:trphysprop} considers the change in area when distorting a unit circle to an ellipse with the same circumference.
An ellipse can be defined in terms of the major and minor axis radii, denoted $a$ and $b$ respectively.
The distortion can then be described by the eccentricity, $\xi$, given by:
\begin{equation}
	\xi = \left(1-\frac{b^2}{a^2}\right)^{1/2}.
\end{equation}
The circumference of an ellipse of given eccentricity, $C\left(\xi\right)$, can then be calculated from the complete elliptic integral of the second kind,
\begin{equation}
	\label{eq:ellipsec}
	C\left(\xi\right) = 4a\int_0^{\pi/2} \left(1-\xi^2\sin^2\theta\right)^{1/2}\dd\theta,
\end{equation}
whilst the area is given more straightforwardly by
\begin{equation}
	A = \pi ab.
\end{equation}
The relative area between an ellipse and a unit circle for a given eccentricity is then
\begin{equation}
 	A/A^0=ab,
\end{equation}
where $a$,$b$ satisfy $C\left(\xi\right)=2\pi$.

\section{Aboav\--Weaire with aG}
\label{app:derivag}

Section \ref{s:toptmapconfigspace} considers the meaning of the \aw{} parameter for aG systems \ie{} those containing just 5\--, 6\-- and 7\--rings.
For these relatively constrained systems, $\alpha$ can be related specifically to the proportions of specific ring adjacencies.
To derive this relationship, results are used from sections \ref{s:theorynodeprob} and \ref{s:toptmapconfigspace}.

The aG system has the node joint degree distribution
\begin{align}
	\mathbf{e} &= \begin{blockarray}{*{3}{c} l}
	\begin{block}{*{3}{>{$\footnotesize}c<{$}} l}
	5 & 6 & 7 \\
	\end{block}
	\begin{block}{[*{3}{c}]>{$\footnotesize}l<{$}}
	e_{55} & e_{56} & e_{57} \: \bigstrut[t]& 5\\
	e_{65} & e_{66} & e_{67} & 6 \\
	e_{75} & e_{76} & e_{77} & 7\\
	\end{block}
	\end{blockarray} .
\end{align}
Taking the \aw{} law, equation \eqref{eq:aboavweaire}, and noting that $m_5=\sumk ke_{5k}/q_5$, leads to the relationship
\begin{equation}
	\frac{5}{q_5}\left(5e_{55}+6e_{56}+7e_{57}\right) = \ki^2+\mu_2+\ki\left(1-\alpha\right)\left(5-\ki\right),
\end{equation}
to which several simplifications can be made.
These arise from the constraints $\ki=6$ and $\sumk e_{5k}=q_5$, which on substitution and rearrangement yield:
\begin{align}
	\frac{5}{q_5}\left(5e_{55}+6\left(q_5-e_{55}-e_{57}\right)+7e_{57}\right) &= 36+\mu_2-6\left(1-\alpha\right) \nonumber \\
	\frac{5}{q_5}\left(e_{57}-e_{55}\right)&=\mu_2+6\alpha.
\end{align}
This can be further simplified by applying the relationships $q_5=5p_5/6$, $p_5=\left(1-p_6\right)/2$, $\mu_2=1-p_6$ and introducing the parameter $\chi_{75}^{5}=e_{57}-e_{55}$, to obtain:
\begin{equation}
	\alpha=\frac{12\chi_{75}^5-\left(1-p_6\right)^2}{6\left(1-p_6\right)}.
\end{equation}
This final relationship is the same as equation \eqref{eq:agalpha}, which expresses the \aw{} parameter in terms of the difference between the 5\--7 and 5\--5 ring adjacencies.

\clearpage
\section{Relating Aboav\--Weaire to Assortativity}
\label{app:derivalphaassort}

Section \ref{s:assortativity} provides a relationship between the assortativity  and the \aw{} parameter, the derivation for which is detailed here.
The assortativity is defined
\begin{equation}
	r = \frac{\sumjk jk\left(e_{jk}-q_jq_k\right)}{\sumk k^2q_k - \left(\sumk kq_k\right)^2}\,,
\end{equation}
which can be rewritten by noting that $q_k=kp_k/\ki$, in the form
\begin{equation}
	\label{appeq:assort}
	r = \frac{\ki^2\sumjk jke_{jk}-\kii^2}{\ki\kiii-\kii^2}\,.
\end{equation}
The mean node degree about a node of degree $j$ is given by $m_j=\frac{1}{q_j}\sumk e_{jk}$, which leads to the relationship
\begin{equation}
	\sumjk jk e_{jk} = \sumj jq_jm_j = \frac{1}{\ki}\sumj jp_j jm_j\,,
\end{equation}
that contains within it the left hand component of the \aw{} law, equation \eqref{eq:aboavweaire}.
Substituting and simplifying gives:
\begin{align}
	\sumjk jk e_{jk} &= \frac{1}{\ki}\sumj jp_j\left[\ki^2+\mu_2+\ki\left(1-\alpha\right)\left(j-\ki\right)\right] \\
	&= \frac{1}{\ki}\left[ \ki\left(1-\alpha\right)\sumj j^2p_j + \left(\alpha\ki^2+\mu_2\right)\sumj jp_j\right] \nonumber \\
	&= \kii\left(1-\alpha\right)+\alpha\ki^2+\mu_2 \nonumber \\
	&=-\alpha\mu_2+\mu_2+\kii\,.
\end{align}
This allows equation \eqref{appeq:assort} to be written
\begin{align}
	r &= \frac{-\alpha\mu_2+\mu_2\ki^2+\kii\ki^2-\kii^2}{\ki\kiii-\kii^2} \\[0.5em]
	r &= \frac{-\alpha\mu_2-\mu_2^2\ki^2}{\ki\kiii-\kii^2} \nonumber \\[0.5em]
	\alpha &= -\frac{r\left(\ki\kiii-\kii^2\right)}{\mu_2\ki^2}-\frac{\mu_2}{\ki^2}\,,
\end{align}
which is the final form given in equation \eqref{eq:rawlink}.

\clearpage
\section{Assortativity of Crystalline Lattices}
\label{app:crystals}

Crystalline systems can provide a useful comparison point to contrast with random networks, as is demonstrated for procrystalline lattices in chapter \ref{ch:procrystals}.
Figure \ref{appfig:crystals} shows two series of crystalline motifs taken from Altman \etal{} \cite{Malashevich2016}, and their corresponding assortativities.

An example calculation for these crystals is included for the $8 - 6^1 - 5^2$ lattice, figure \ref{appfig:86152}.
The edge joint degree distribution is given by:
\begin{align}
	\mathbf{e} &= \frac{1}{24}\, \begin{blockarray}{*{3}{c} l}
	\begin{block}{*{3}{>{$\footnotesize}c<{$}} l}
	5 & 6 & 8 \\
	\end{block}
	\begin{block}{[*{3}{c}]>{$\footnotesize}l<{$}}
	2 & 4 & 4 \: \bigstrut[t]& 5\\
	2 & 2 & 2 & 6 \\
	4 & 2 & 2 & 8\\
	\end{block}
	\end{blockarray}.
\end{align}
The rows are deduced using two considerations.
Firstly, the relative ring adjacencies for each ring size are accounted for \eg{} each 8\--ring has $4\times5$\--ring neighbours, $2\times6$\--ring neighbours and $2\times8$\--ring neighbours.
Secondly, the relative ring proportions are factored in to the row sums \ie{} $p_5=2p_6=2p_8$.
Once this matrix has been constructed, the assortativity can be calculated using equation \eqref{appeq:assort}.

\begin{figure}[h!]
     \centering

	\begin{subfigure}[b]{0.15\textwidth}
         \centering
         \includegraphics[width=\textwidth]{./appendices/figures/crystal_8_60_52.pdf}
         \caption{$8 - 6^0 - 5^2$, \\$r=-0.350$}
         \label{appfig:86052}
     \end{subfigure}
     \hfill
      \begin{subfigure}[b]{0.15\textwidth}
         \centering
         \includegraphics[width=\textwidth]{./appendices/figures/crystal_8_61_52.pdf}
         \caption{$8 - 6^1 - 5^2$, \\$r=-0.185$}
         \label{appfig:86152}
     \end{subfigure}
     \hfill
      \begin{subfigure}[b]{0.15\textwidth}
         \centering
         \includegraphics[width=\textwidth]{./appendices/figures/crystal_8_62_52.pdf}
         \caption{$8 - 6^2 - 5^2$, \\$r=-0.373$}
         \label{appfig:86252}
     \end{subfigure}
     \hfill
      \begin{subfigure}[b]{0.15\textwidth}
         \centering
         \includegraphics[width=\textwidth]{./appendices/figures/crystal_8_63_52.pdf}
         \caption{$8 - 6^3 - 5^2$, \\$r=-0.220$}
         \label{appfig:86352}
     \end{subfigure}
     \hfill

     \vspace{2mm}
      \begin{subfigure}[b]{0.15\textwidth}
         \centering
         \includegraphics[width=\textwidth]{./appendices/figures/crystal_8_60_4.pdf}
         \caption{$8 - 6^0 - 4$, \\$r=-0.500$}
         \label{appfig:8604}
     \end{subfigure}
     \hfill
      \begin{subfigure}[b]{0.15\textwidth}
         \centering
         \includegraphics[width=\textwidth]{./appendices/figures/crystal_8_61_4.pdf}
         \caption{$8 - 6^1 - 4$, \\$r=-0.260$}
         \label{appfig:8614}
     \end{subfigure}
     \hfill
      \begin{subfigure}[b]{0.15\textwidth}
         \centering
         \includegraphics[width=\textwidth]{./appendices/figures/crystal_8_62_4.pdf}
         \caption{$8 - 6^2 - 4$, \\$r=-0.412$}
         \label{appfig:8624}
     \end{subfigure}
     \hfill
      \begin{subfigure}[b]{0.15\textwidth}
         \centering
         \includegraphics[width=\textwidth]{./appendices/figures/crystal_8_63_4.pdf}
         \caption{$8 - 6^3 - 4$, \\$r=-0.395$}
         \label{appfig:8634}
     \end{subfigure}
     \hfill

     \caption{Repeating units from two series of crystalline motifs and their assortativities.
      Panels (a)\--(d) show the series $8-6^i-5^2$ with half the number of 8\-- as 5\--rings, interspersed with varying numbers of 6\--rings. Panels (e)\--(h) show the series $8-6^i-4$ with equal numbers of 8\-- and 4\--rings, interspersed with varying numbers of 6\--rings.}
     \label{appfig:crystals}
\end{figure}

\resumetocwriting


\subsubsection{TypicalHistoryFromHistorySet}
\label{TypicalHistoryFromHistorySet}

This Post-Processor performs a simplified procedure of \cite{wilcox2008users} to form a ``typical'' time series from multiple time series. The input should be a HistorySet, with each history in the HistorySet synchronized. For HistorySet that is not synchronized, use Post-Processor method \textbf{HistorySetSync}  to synchronize the data before running this method.

Each history in input HistorySet is first converted to multiple histories each has maximum time specified in \xmlNode{outputLen} (see below). Each converted history $H_i$ is divided into a set of subsequences $\{H_i^j\}$, and the division is guided by the \xmlNode{subseqLen} node specified in the input XML. The value of \xmlNode{subseqLen} should be a list of positive numbers that specify the length of each subsequence. If the number of subsequence for each history is more than the number of values given in \xmlNode{subseqLen}, the values in \xmlNode{subseqLen} would be reused.

For each variable $x$, the method first computes the empirical CDF (cumulative density function) by using all the data values of $x$ in the HistorySet. This CDF is termed as long-term CDF for $x$. Then for each subsequence $H_i^j$, the method computes the empirical CDF by using all the data values of $x$ in $H_i^j$. This CDF is termed as subsequential CDF. For the first interval window (i.e., $j=1$), the method computes the Finkelstein-Schafer (FS) statistics \cite{finkelstein1971improved} between the long term CDF and the subsequential CDF of $H_i^1$ for each $i$. The FS statistics is defined as following.
\begin{align*}
FS & = \sum_x FS_x\\
FS_x &= \frac{1}{N}\sum_{n=1}^N\delta_n
\end{align*}
where $N$ is the number of value reading in the empirical CDF and $\delta_n$ is the absolute difference between the long term CDF and the subsequential CDF at value $x_n$. The subsequence $H_i^1$ with minimal FS statistics will be selected as the typical subsequence for the interval window $j=1$. Such process repeats for $j=2,3,\dots$ until all subsequences have been processed. Then all the typical subsequences will be concatenated to form a complete history.

\ppType{TypicalHistoryFromHistorySet}{TypicalHistoryFromHistorySet}

In the \xmlNode{PostProcessor} input block, the following XML sub-nodes are required,
independent of the \xmlAttr{subType} specified:

\begin{itemize}
   \item \xmlNode{pivotParameter}, \xmlDesc{string, optional field}, ID of the temporal variable
   \default{Time}
   \item \xmlNode{subseqLen}, \xmlDesc{integers, required field}, length of the divided subsequence (see above)
   \item \xmlNode{outputLen}, \xmlDesc{integer, optional field}, maximum value of the temporal variable for the generated typical history
   \default{Maximum value of the variable with name of \xmlNode{pivotParameter}}
\end{itemize}

For example, consider history of data collected over three years in one-second increments,
where the user wants a single \emph{typical year} extracted from the data.
The user wants this data constructed by combining twelve equal \emph{typical month}
segments.  In this case, the parameter \xmlNode{outputLen} should be \texttt{31536000} (the number of seconds
in a year), while the parameter \xmlNode{subseqLen} should be \texttt{2592000} (the number of seconds in a
month).  Using a value for \xmlNode{subseqLen} that is either much, much smaller than \xmlNode{outputLen} or
of equal size to \xmlNode{outputLen} might have unexpected results.  In general, we recommend using a
\xmlNode{subseqLen} that is roughly an order of magnitude smaller than \xmlNode{outputLen}.


\subsubsection*{Unary Operations\hspace*{\fill}\hyperlink{ElementwiseOperations}{(up)}\hypertarget{unaryOperations}{}}\addcontentsline{toc}{subsubsection}{Unary Operations}
Unary operations involve calculations on a single \ttbf{view}. Functions which involve a calculation where the answer is a scalar, such as \ttbf{sumval} generally have a \ttbf{val} as part of the root name.
\begin{table}[H]
\caption{Unary Operations}
\label{tab:unaryOperations}
\begin{center}
\begin{tabular}{|l|l|}\hline
\hlnkFunc{arg} & Argument\\
\hlnkFunc{ceil} & Ceiling\\
\hlnkFunc{conj} & Conjugate\\
\hlnkFunc{cumsum} & Cumulative Sum\\
\hlnkFunc{euler} & Euler\\
\hlnkFunc{floor} & Floor\\
\hlnkFunc{mag} & Magnitude\\
\hlnkFunc{cmagsq} & Complex Magnitude Squared\\
\hlnkFunc{meanval} & Mean Value\\
\hlnkFunc{meansqval} & Mean Square Value\\
\hlnkFunc{modulate} & Modulate\\
\hlnkFunc{neg} & Negate\\
\hlnkFunc{recip} & Reciprocal\\
\hlnkFunc{round} & Round\\
\hlnkFunc{rsqrt} & reciprocal Square Root\\
\hlnkFunc{sq} & Square\\
\hlnkFunc{sumval} & Sum Value\\
\hlnkFunc{sumsqval} & Sum of Squares Value\\
\hline\end{tabular}
\end{center}
%\label{default}
\end{table}%


% Project Specifications

\chapter{Quantization}

%In such networks a large part of the computation is done with an
%operation called the \gls{gemm}. An efficient implementation of \gls{gemm} is crucial to the
%acceleration. However, \glspl{cnn} are normally implemented with floating-point numbers, which are usually less
%efficient to handle in hardware than fixed-point numbers. A class of techniques called quantization can
%be used to carry out the computation in fixed-point numbers and then convert the result back to
%floating-point numbers, realizing high performance in hardware. Chapter 4 provides a detailed discussion
%of the quantization scheme used in this project.

Machine learning programs normally represent numbers as floating-point. Although floating-point numbers
have limited precision and can sometimes lead to numerical instability when the numbers involved get too
small or too large, they are intuitive and easy to work with. But floating-point arithmetic can be rather
expensive to implement in hardware. Modern \glspl{fpga} contain dedicated \gls{dsp} units, some of which
natively
support floating-point, others require special \gls{ip} cores to implement the relevant
circuitry. \gls{fpga}'s processing power mainly shines on fixed-point. In order to run our model in fixed-point
on \gls{fpga}, it is necessary to convert the model into fixed-point representation, through a process known as
quantization. When the result is computed by \gls{fpga}, it is then converted back to floating-point, which is
naturally called dequantization.

Quantization brings additional benefits with the compression of model size. When 32-bit floating-point
weights are quantized to 8-bit, the model size can be reduced up to $75\%$. This cuts down memory bandwidth
requirement and improves memory efficiency. Consequently, the power consumption can be greatly reduced.

One could imagine that quantization would cause a tremendous loss to the precision of a model, for example, when
32-bit floating-point numbers
are converted to
8-bit unsigned integers, the range drops from $(1.175494351 \times 10^{-38}, 3.402823466 \times 10^{38})$ to $(0, 255)$. However, one of the peculiar
properties of neural networks is that they are very resilient to noise. Of course this is also one of the
reasons why they are so successful in real world applications. We could view the internal precision loss of
quantization as a form of internal noise. It turned out that the precision loss incurred is rather small
\cite{vincent2011}.

There exist several different quantization methods. This project adopted the quantization scheme
implemented in Google's \textit{gemmlowp} library \cite{github:gemmlowp}
as well as described in \cite{DBLP:journals/corr/abs-1712-05877}. \textit{gemmlowp} is
a low precision \gls{gemm} implementation for fixed-point numbers.
The original 32-bit single-precision weights represented in \mintinline{c}{float} are mapped to
\mintinline{c}{uint8_t}. \gls{gemm} is then
carried out on the \gls{fpga} board with these 8-bit integers. Intermediate \gls{mac} results are 32-bit
integers to accommodate the accumulation. Eventually the final output of the model is converted back to
\mintinline{c}{float}.

The quantization works by first identifying the range of the data. In neural networks, the values are usually
distributed within a relatively small range. For instance, for 8-bit quantization, if the minimum value of the
weights is $-10.0$ and maximum value is $20.0$, then $-10.0$ would be mapped to $0$ while $20.0$ would be mapped to $255$.
However, there is an additional requirement that can bring great benefits to subsequent computations:
the real value $0$ should be exactly representable. As will be shown, this can be achieved with a small
modification to the mapping equation.

Th quantization scheme can be derived with basic algebra. Define a real number $f$ as the result of an affine
mapping $f = S q + B$, where $q$ is the corresponding quantized integer, $S \in \mathbb{R}$ is a constant
scaling factor, and $B \in \mathbb{R}$ is a constant shift. Next, modify the mapping a bit so that the
real value $0$ is exactly representable:

\begin{equation} \label{eq:dequantization}
  f = S(q - Z),
\end{equation}

where $Z \in \mathbb{N}$ is a constant shift applied to $q$. Since convolution and transposed convolution
both can involve zero-paddings, having an exact representation of $0$ avoids the accumulation of errors
(essentially a form of bias). With this form of affine mapping, $0$ is trivially mapped to $q = Z$. $S$ and
$Z$ are called the quantization parameters of this mapping. From equation \ref{eq:dequantization},
it is clear that the quantization mapping is done with

\begin{equation} \label{eq:quantization}
  q = \frac{f}{S} + Z
\end{equation}

Let $A \in \mathbb{R}^{m \times k}$, $B \in \mathbb{R}^{k \times n}$, $A B = C \in \mathbb{R}^{m \times n}$,
let the corresponding quantization parameters for $A$, $B$ and $C$ be $S_A$ and $Z_A$, $S_B$ and $Z_B$,
$S_C$ and $Z_C$. These parameters can be predetermined, since in most cases the ranges of numbers in these matrices
are known. Let $p$, $q$, $r$ be the entries in the quantized version of $A$, $B$ and $C$.

For an entry $f_C$ in $C$, $f_C = \sum_{i}^{} A_i B_i$, where $i$ is the index of entries in a particular
row in $A$ and corresponding column in $B$,

\begin{equation}
\begin{split}
  f_C & = \sum_{i} A_i B_i \\
      & = \sum_{i} S_A (p_i - Z_A) S_B (q_i - Z_A) \\
      & = S_A S_B \sum_{i} (p_i - Z_A) (q_i - Z_B)
\end{split}
\end{equation}

Clearly the term $\sum_{i} (p_i - Z_A) (q_i - Z_B)$ is the computational core here, let it be
$\mathfrak{K}$. Then,

\begin{equation}
\begin{split}
  r_C & = \frac{f_C}{S_C} + Z_C \\
      & = \frac{S_A S_B}{S_C} \mathfrak{K} + Z_C
\end{split}
\end{equation}

Therefore, once $\mathfrak{K}$ is computed, it can be simply multiplied by a constant multiplier
$M = \frac{S_A S_B}{S_C}$ and added the zero offset $Z_C$ to obtain the quantized value $r_C$. The
multiplication by $M$ can be implemented by multiplying $\mathfrak{K}$ by a constant integer and right shifting the
result by a fixed number of bits. These can be predetermined as well. The computation of $\mathfrak{K}$ might seem problematic at first glance:
it contains a subtraction operation for each entry,
which would bring extra overhead to the \gls{gemm} computation. To solve this, expand the expression $\mathfrak{K}$:

\begin{equation}
\begin{split}
  \mathfrak{K} & = \sum_{i} (p_i - Z_A) (q_i - Z_B) \\
               & = \sum_{i} (p_i q_i - Z_B p_i - Z_A q_i + Z_A Z_B) \\
               & = \sum_{i} p_i q_i - \sum_{i} Z_B p_i - \sum_{i} Z_A q_i + \sum_{i} Z_A Z_B \\
               & = \sum_{i} p_i q_i - Z_B \sum_{i} p_i - Z_A \sum_{i} q_i + k Z_A Z_B
\end{split}
\end{equation}

Note that $\sum_{i} 1$ is k, which is the number of columns of $A$ and the number of rows of $B$. Four terms are
obtained here. The first term $\sum_{i} p_i q_i$ is the main term. The second term, $- Z_B \sum_{i} p_i$
is the product of $- Z_B$ and the sum of the current row, which only needs to be calculated once and added
to each result of the first term. The third term $- Z_A \sum_{i} q_i$ and the fourth term $k Z_A Z_B$ are
similar. The computation can be carried out by computing the first term as the core, and add the rest three
terms one by one.

\section{Handling of Biases}

Biases are added to the 32-bit accumulator result. Recall that $f_C = \sum_{i} A_i B_i = S_A S_B \mathfrak{K}$,
it is clear that biases should be quantized with $S = S_A S_B$ and $Z = 0$. When zero offset is $0$, negative
numbers will remain negative, consequently the type of bias values should be signed 32-bit integers.

\clearpage %force the next chapter to start on a new page. Keep that as the last line of your chapter!