\graphicspath{ {./img/TheFEM/} }
\chapter{Interpolation in the Finite Element Method}

\section*{Preliminary}
In a broad sense the finite element method is nothing else than an
approximation technique for solutions to boundary and initial value problems.
In this section we will discuss some fundamental and basic aspects related to
the approximation of functions through interpolation. In the finite element
method, such approximation takes place for the geometry of the computational
domain and for the field variable of the problem at hand. As it will be seen,
the concept of finite element it self corresponds to a local domain or
sub-domain in which the solution function is approximated via interpolation
techniques. We will cover this subject from a materialistic point of view, as
required on the implementation of a first finite element algorithm, with
mathematical rigor left to the excellent texts on the subject. The set of notes
starts with the problem definition and its solution in terms of the Lagrange
interpolation theorem. From that point the notes discuss practical applications
including its implementation in Python scripts built for finite element
analysis.

At the end of this chapter\footnote{This chapter, together with theoretical and
computational learning activities is complemented by Jupyter Notebooks 1
through 4 available at the course's repository.} the student should be able to:

\begin{itemize}
\item[•] Formulate appropriate interpolation schemes to generate approximate functions out of sets of discrete values representing the behavior of the unknown functions over one-dimensional and two-dimensional domains.
\item[•] Recognize the differences in terms of advantages and limitations  between global and local interpolation schemes. This includes the pathologies associated with each method.
\item[•] Recognize the concept {\bf finite element} like a family of locally defined interpolation schemes valid over canonical domains.
\item[•] Develop efficient and effective  Python implementations of interpolation schemes within the context of the finite element method for one-dimensional and two-dimensional applications.
\end{itemize}

\section{Statement of the problem}
Let $f(x)$ be an unknown function, whose values, however, are known at $n$ discrete points ${x_1, x_2,...,x_n}$. We want to know (interpolate) the value of $f(x)$ at an arbitrary point $x \in \left[ {{x_1},{x_n}} \right]$ and different to one of the $n$ points.

The problem of interpolation is precisely that of finding the unknown value of $f(x)$ using the known values $\left\{ {{f^1},{f^2},...,{f^n}} \right\}$. As schematically described in \cref{fig:interpol1} it involves two steps:

\begin{itemize}
\item[i]  Fitting an approximate function, known as the interpolating polynomial, to the known data points.
\item[ii] Evaluating the function at the point of interest where the function is unknown.
\end{itemize}

We can (i) follow a global approach using all the known $n$-data points and fit an $(n-1)$-th order polynomial to these data points\footnote{As this approach is problem-dependent it is also cumbersome and difficult to code and therefore not amenable to be used in finite elements.}(\cref{fig:interpol1})  or (ii) use a local approach where one splits the domain into sub-intervals and fits lower order polynomials to the data points within each sub-interval(\cref{fig:interpol2}).

Local interpolation uses a finite number of nearest-neighbors and generates interpolated versions of $f(x)$ that do not in general have continuous first or higher derivatives. The advantage of this local approach lies in the fact that independent of the function or the number of data points, the interpolation operation always uses the same polynomial. For instance in \cref{fig:interpol2} only first-order polynomials are being used in each local sub-domain: as a result a computer implementation of this scheme would only need to store the local polynomial once. In the jargon of the finite element method the interpolation functions or its resulting local polynomials are termed a finite element.

\begin{figure}[H]
\centering
\includegraphics[width=10cm]{img/interpol1.pdf}
\caption{The black points represent known values of an otherwise unknown function. The dashed line represents an approximation to the unknown function in terms of a polyonmial of order $n-1$.}
\label{fig:interpol1}
\end{figure}

\begin{figure}[H]
\centering
\includegraphics[width=10cm]{img/interpol2.pdf}
\caption{Interpolation takes place inside each sub-interval producing a piecewise interpolation approximation to the unknown function}
\label{fig:interpol2}
\end{figure}

\section{One-dimensional scalar functions}
\subsection{Lagrange interpolation theorem}
Given a set of n-points $\{ (x^1, y^1),\cdots,(x^n, y^n)\}$ where $y^n \equiv f({x^n})$ then: ``there exists a unique polynomial $p(x)$ of order at most $(n-1)$ such $p(x^I) = f(x^I)$ for $I=1,2,\cdots,n$". The polynomial is given by;

\begin{equation}\label{eq:inter_expl}
p(x) = {L^1}(x)f({x^1}) + {L^2}(x)f({x^2}) +  \ldots  + {L^n}(x)f({x^n})
\end{equation}

and the term ${L^I}(x)$ is computed as;

\begin{equation}\label{eq:coef}
  L^I(x) = \prod_{\substack{J = 1\\ I \ne J}}^n \frac{(x - x^J)}{(x^I - x^J)}.
\end{equation}

The approximate function $p(x)$ of order $n-1$ is termed {\bf the interpolating polynomial} such

\[f(x)\simeq p(x)\]

while each one of the terms ${L^I}(x)$, also of order $n-1$, are termed {\bf the interpolation functions}. In the context of the finite element method these are also called {\bf shape functions}.

The use of index notation, and particularly the summation convention, can be extended to represent the linear superposition of functions given by \cref{eq:inter_expl}. We use capital super-scripts to denote interpolation in such a way that a capital superscript denotes a data point in the interpolation scheme. Accordingly \cref{eq:inter_expl} can be equivalently written like:

\begin{equation}\label{eq:pol}
  p(x^I) = L^I(x) f(x^I)
\end{equation}

where the fact that $I=1,2,...,n$ is implicit in the notation.

Recall that the interpolating polynomial $p(x)$ is just an approximation to the actual function $f(x)$. However, the approximation should at least be such that $p(x^I) = f(x^I)$ at the $n$ nodal points. To satisfy this condition the interpolation polynomials must be such that:

\[L^I(x^J) = \delta^{IJ}\]

and where ${\delta ^{IJ}}$ is the delta function extended to the interpolation polynomials.

\begin{tcolorbox}
The approximate function $p(x)$ resulting from the interpolation process is called the {\bf interpolating polynomial} while the Lagrange polynomials $L^I(x)$ are called interpolation functions. In the language of the finite element methods these are simply called {\bf shape functions}.
\end{tcolorbox}

\begin{tcolorbox}
\paragraph*{A note regarding superscripts in indicial notation:} In this Class Notes superscripts associated to symbols like in the expression $x^4$ are frequently used to describe a variable associated to a nodal point: for instance, in this context the expression $x^4$ represents the $x$-coordinate of nodal point $4$. However, in some other cases this same expression might appear, for example, in the definition of a function like in $f(x) = {x^4} + 4{x^3}$. In both cases the specific meaning should be clear by the context in which it appears.

Python function {\bf LagrangPoly()} in NB-1 generates Lagrange interpolation polynomials of different order and over a varying range.
\end{tcolorbox}

\paragraph*{Example: Interpolation of a function using 3 data points}

Use a Lagrange interpolation scheme to find an interpolating polynomial that approximates the function

\[ f(x) = {x^3} + 4{x^2} - 10 \]

at the sampling (or also nodal) points ${x^1} =  - 1.0$, ${x^2} =  + 1.0$ and ${x^3} = 0.0$ over the interval $[-1,1]$ together with its first order derivative at $x = 0.7$.  Assume that the values of the first derivative at the nodal points are unknown.

\Cref{ejemplo2} contains the exact values for the function

\begin{center}
\begin{tabular}{ll}
  \hline
  $x$ & $f(x)$ \\
  \hline
  $-1.0$  & $-7.000$  \\
  $ 0.00$  & $-10.00$  \\
  $ 1.00$  & $-5.000$  \\
  \hline
\end{tabular}
\captionof{table}{Known values of the function $f(x) = {x^3} + 4{x^2} - 10$ over the interval $[-1,1]$}
\label{ejemplo2}
\end{center}

The formulation of the interpolation scheme consists in finding the interpolation polynomials $L^I(x)$ and the interpolating function $p(x)$. From \cref{eq:coef} the interpolation functions, shown in \cref{fig:pols}, are:
\begin{align*}
L^1(x) = \frac{(x - x^2)(x - x^3)}{(x^1 - x^2)(x^1-x^3)} & \equiv  - \frac{1}{2}(1 -  x)x\\
L^2(x) = \frac{(x - x^1)(x - x^3)}{(x^2 - x^1)(x^2 - x^3)} & \equiv +   \frac{1}{2}(1 + x)x\\
L^3(x) = \frac{(x - x^1)(x - x^2)}{(x^3 - x^1)(x^3 - x^2)} & \equiv + (1 - x^2).
\end{align*}

\begin{figure}[H]
  \centering
  \includegraphics[width=10cm]{func1.pdf}
  \caption{(top) Interpolation polynomials $L^1(x)$, $L^2(x)$ and $L^3(x)$ as per \cref{eq:coef} computed for a second order scheme with 3 nodal points.}
  \label{fig:pols}
\end{figure}

The interpolating polynomial approximating the function is obtained using \cref{eq:pol} which in this case is given by

\[p(x) = 10{x^2} + \frac{7}{2}(1 - x)x - \frac{5}{2}(1 + x)x - 10.\]

The approximate and actual function are compared in \cref{fig:functions}. Clearly $p(x)$, being a second order function differs with $f(x)$, which is a third order function. However, as stated in the interpolation theorem both functions coincide at the nodal points where the known values of the function are available.

\begin{figure}[H]
  \centering
  \includegraphics[width=10cm]{func2.pdf}
  \caption{Resulting interpolating function $p(x)$ as per \cref{eq:pol} compared with the exact function $f(x) = {x^3} + 4{x^2} - 10$. The Python code used for the interpolation is available in NB-1}
  \label{fig:functions}
\end{figure}

In finding an approximations to the first derivative recall that at the nodal points the values of these first derivatives are unknown. Thus the only available choice is to operate directly on $p(x)$ and use it to approximate also the first derivative like:

\begin{equation}\label{eq:der}
  \frac{ dp(x)}{dx} = \frac{dL^1(x)}{dx} f^1 + \frac{dL^2(x)}{dx}(x) f^2 + \frac{dL^3(x)}{dx} f^3
\end{equation}

It is evident that the approximation takes the form of a lineal superposition of products between interpolation functions (which in this case are derivatives of the $L^I$) and known values of the function. The first derivative obtained in this form differs from the actual derivative of $f(x)$ since this approach uses the values of $f$ and not those of $\frac{{df(x)}}{{dx}}$ and $\frac{dL^I(x)}{dx}$ instead of $L^I$ and these functions do not satisfy the condition $\frac{dL^I(x^J)}{dx} = \delta ^{IJ}$.

\begin{figure}[H]
  \centering
  \includegraphics[width=10cm]{deriv.pdf}
  \caption{Comparison between the first order derivative of $p(x)$ with the closed-form result of $\frac{{df(x)}}{{dx}}$}
  \label{fig:deriv}
\end{figure}

\Cref{fig:first der 1} shows the approximation to $f'(x)$ when $p(x)$ is computed using 4th-order polynomials.

\begin{figure}[H]
  \centering
  \includegraphics[width=10cm]{firstder.pdf}
  \caption{Comparison between the first order derivative of $p(x)$ computed using 4th-order polynomials with the closed-form result of $\frac{{df(x)}}{{dx}}$}
  \label{fig:first der 1}
\end{figure}

To identify the variation in the solution with different interpolation schemes \cref{fig:several interpol} compares the exact and interpolated solution for polynomials of order 1, 2 and 4 respectively. The left column displays the interpolation polynomials ${{L^I}(x)}$.

\begin{figure}[H]
\centering
	\begin{subfigure}[b]{0.450\textwidth}\qquad
		\includegraphics[width=\textwidth]{lineal.pdf}
		\caption{First order. }
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.450\textwidth}\qquad
		\includegraphics[width=\textwidth]{interlin.pdf}
		\caption{Actual and interpolated function.}
	\end{subfigure}\\
%
\centering
	\begin{subfigure}[b]{0.450\textwidth}\qquad
		\includegraphics[width=\textwidth]{quadra.pdf}
		\caption{Second order.}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.450\textwidth}\qquad
		\includegraphics[width=\textwidth]{interqua.pdf}
		\caption{Actual and interpolated function.}
	\end{subfigure}\\
%
\centering
	\begin{subfigure}[b]{0.450\textwidth}\qquad
		\includegraphics[width=\textwidth]{third.pdf}
		\caption{Fourth order.}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.450\textwidth}\qquad
		\includegraphics[width=\textwidth]{intertri.pdf}
		\caption{Actual and interpolated function.}
	\end{subfigure}
\caption{Interpolation of the function $f(x) = {x^3} + 4{x^2} - 10$ using Lagrange polynomials of increasing order.}
\label{fig:several interpol}
\end{figure}

\subsection{Local interpolation using a piece-wise continuous function}
An alternative to the non-uniform nodal distribution approach used in the previous section to improve the interpolation scheme is based on splitting the solution interval $[x_1, x_n]$ in smaller sub-domains where the interpolation is performed locally. For instance \cref{subdo} shows such a partition for the interval $[-1.0, 1.0]$ where each sub-domain is conformed by a pair of consecutive nodes. The table shows values of the function $f(x) = {x^3} + 4{x^2} - 10$ at the edges of the sub-domains. In this particular case, considering each sub-domain to be conformed by a pair of points, the lineal interpolation scheme reduces to finding the straight line (first order polynomial) that passes along the pair of nodes. Higher order local schemes are possible if additional points are added to the sub-domains.

\begin{center}
\begin{tabular}{ccc}
  \hline
  Subdomain & Range & Values for $f(x)$ \\
  \hline
   1  & $[-1.0, -0.5]$ & $[-7.000, -9.125]$  \\
   2  & $[-0.5,  0.00]$ & $[-9.125, -10.00]$  \\
   3  & $[+0.0,  +0.5]$ & $[-10.00, -8.875]$   \\
   4  & $[+0.5, +1.0]$ & $[-8.875, -5.000]$   \\
  \hline
\end{tabular}
\captionof{table}{Partition of the interval $[-1.0, 1.0]$ into subdomains}
\label{subdo}
\end{center}

\begin{tcolorbox}

Using piecewise interpolation over constant size intervals implies also the use of constant polynomials. In the finite element method each sub-domain is mapped into a constant size interval facilitating systematization of the interpolation process.
Notebook 2 in the REPO shows the Python implementation of piecewise interpolation.

\end{tcolorbox}

\begin{figure}[H]
  \centering
  \includegraphics[width=10cm]{localone.pdf}
  \caption{Local interpolating polynomials.}
  \label{fig:loc-pols}
\end{figure}

This approach results in an interpolating polynomial as shown in \cref{fig:fully local 1} and where the approximated function is piece-wise continuous. As a result the local based technique the first derivative of the function is now discontinuous at the boundaries of the subdomains. However this local schema is advantageous since the local polynomials are unique and the scheme can be used for an arbitrary number of nodal points facilitating computer implementation.

\begin{figure}[H]
\centering
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{localfun.pdf}
		\caption{Approximation to the function using first order interpolation polynomials.}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{localfirst.pdf}
		\caption{Variation of the first order derivative.}
	\end{subfigure}\\

\caption{Locally based interpolation scheme for the function $f(x) = {x^3} + 4{x^2} - 10$.}
\label{fig:fully local 1}
\end{figure}

\subsection{Distribution of the sampling (or nodal) points}
The simple problems considered so far have used a small number of sample points and a constant separation distance. This approach worked nicely considering the smooth functions involved. However if the function to be interpolated exhibits strong gradients, as might be the case in problems of wave propagation, the approximation with a small number of data points is very likely insufficient. The natural solution seems to be the addition of data points and thus an increase in the order of the interpolating polynomial. Unfortunately, as we will show here this approach does not always works in the desired direction.

Consider the function:

\[f(x) = \frac{1}{{1 + 25{x^2}}}\]

shown in \cref{fig:rungelag}.

\begin{figure}[H]
\centering
\includegraphics[width=8cm]{runge.pdf}
\caption{Runge function $f(x) = \frac{1}{1 + 25{x^2}}$.}
\label{fig:rungelag}
\end{figure}

This function shows a strong spatial variation requiring an interpolating polynomial of larger order and as a result a larger number of nodal points. The 11 black dots shown in the figure represent nodal points equally spaced at $\Delta x = 0.2$ and where the function is assumed to be known. We wish to approximate this function using these 11 points and an order 10 interpolating Lagrange polynomial.

\Cref{fig:rungeequi} shows the 11 order-10 Lagrange interpolation polynomials for the equidistant nodal distribution and the resulting interpolating polynomial $p(x)$. Clearly the approximation is highly inaccurate, specially near the edges of the interval where it exhibits strong oscillations. This spurious result along the edges is introduced by the equidistant separation of the sampling points. The interpolation scheme can be improved using a non-uniform nodal spacing. The resulting alternative scheme is shown in  \cref{fig:rungechevy} where there is a large concentration of nodal points along the edges.

\begin{figure}[H]
\centering
	\begin{subfigure}[b]{0.50\textwidth}\qquad
		\includegraphics[width=\textwidth]{lag11.pdf}
		\caption{}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{rungaprlag.pdf}
		\caption{}
	\end{subfigure}\\

\caption{(a) Lagrange interpolation polynomials of order 10 associated to the 11 sampling points for \cref{fig:rungelag}. (b) Interpolating polynomial to approximate the Runge function with the order-10 polynomials from part (a).}
\label{fig:rungeequi}
\end{figure}

\begin{figure}[H]
\centering
	\begin{subfigure}[b]{0.50\textwidth}\qquad
		\includegraphics[width=\textwidth]{lagchv.pdf}
		\caption{}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{rungaprchv.pdf}
		\caption{}
	\end{subfigure}\\

\caption{(a) Order 10 Lagrange interpolation polynomials associated to non-equidistant sampling points. (b) Interpolating polynomial to approximate a Runge function built with the polynomials derived in part (a).}
\label{fig:rungechevy}
\end{figure}

To understand this numerical pathology, related with the distribution of the nodal points, consider the interpolation polynomials corresponding to the central point and to the edge point of the equidistant distribution (\cref{fig:rungeequi}) as shown in part(a) of \cref{fig:compara}. The green line corresponds to the polynomial associated to the central node, while the blue line is that of the edge nodal point. Clearly, the central-point polynomial introduces a strong variation along the edges of the sampling interval, while the edge-point polynomial exhibits a rather smooth variation. Similarly, part(b) in the same figure shows once again the central-point and the edge-point polynomials associated to non-uniform nodal distribution. It can be observed how in this last case both polynomials exhibit a smooth variation over the interval eliminating the strong oscillation towards the edges.

\begin{figure}[H]
\centering
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{central.pdf}
		\caption{}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{extremo.pdf}
		\caption{}
	\end{subfigure}\\

\caption{(a) Equidistant nodal points (b) Non-uniform nodal points.}
\label{fig:compara}
\end{figure}

\section{Extension to two-dimensional domains}
Assume we are now interested in conducting interpolation of a function over a spatial 2-dimensional domain where every point is specified by a position vector of the form $\overrightarrow x = x \hat{\imath} + y\hat{\jmath}$. We want to know, via interpolation, the value of a function $f(\overrightarrow x)$ at an arbitrary point $\overrightarrow x$ provided we know the set of n-pairs of the form $\{(\overrightarrow x ^1, f^1),\cdots,(\overrightarrow x ^n, f^n)\}$. The domain and the visualization of the function are shown in \cref{fig:element}.

\begin{figure}[H]
\centering
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{element.pdf}
		\caption{Square two-dimensional domain}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{surface.pdf}
		\caption{Interpolated function $f(x,y)$.}
	\end{subfigure}\\

\caption{Function $f(x,y)$ over a square two-dimensional domain with nodal points labeled 1 , 2 , 3 , 4.}
\label{fig:element}
\end{figure}

Since now the function $f$ depends on the 2D space coordinates $(x , y)$ it is natural to expect that interpolation functions depend also on $(x , y)$. Using this condition on the function approximation we have:

\[f(x,y) = N^1(x,y)f^1 + N^2(x,y)f^2 + N^3(x,y)f^3 + N^4(x,y)f^4\]

where now $N^Q(x,y)$ is the 2D interpolation (or shape) function associated to the sampling point $Q$. Using indicial summation convention, we can write the interpolated function as:

\[f(x,y) = N^Q(x,y)f^Q\]

where now $Q = 1,...,N$.

The method to find the required 2D shape functions $N^Q(x,y)$ consists in the recursive (or iterated) application of the Lagrange one-dimensional scheme discussed previously in terms of interpolating polynomials $L^Q(\eta)$ and where now $\eta$ is a dummy variable that can assume the role of $x$ or $y$.

In the domain shown in  \cref{fig:element} assume that we wish to interpolate the value of the function along the 1-4 direction. Note that along this line x is constant and then the function depends only on $y$. Fixing $x = x^A$ it is possible to conduct 1-dimensional interpolation along the $y$ direction as shown in \cref{fig:onedimn}. In this case $\eta$ assumes the role of $y$ and we have:

\[f(x^A,y) = L^1(y)f^1 + L^4(y)f^4.\]

Since the interpolation scheme is taking place along the 1-4 direction in terms of the 2 nodal values $f^1$ and $f^4$, the functions $L^1$ and $L^4$ in this case are the first order Lagrange polynomials associated to the points 1 and 4 respectively, and obtained with the already known product formula given in \cref{eq:coef}.

Clearly, the above scheme provides the value of the function for an arbitrary point $A$ along the 1-4 line. Proceeding similarly along the 2-3 direction, that is setting $x = x^B$ and interpolating once again along the $y$ direction we have:
\[f(x^B,y) = L^2(y)f^2 + L^3(y)f^3.\]

\begin{figure}[H]
\centering
\includegraphics[width=10cm]{inter1D.pdf}
\caption{Interpolation along the $y$-direction}
\label{fig:onedimn}
\end{figure}

So far, we have captured only the dependence on $y$ since $x$ was assumed constant as indicated by $f(x^A,y)$ and $f(x^B,y)$. The dependence on $x$ is now captured proceeding similarly along the arbitrary line $A-B$ using the functions $f(x^A,y)$ and $f(x^B,y)$ respectively as follows;

\[f(x,y) = L^A(x) f(x^A,y) + L^B(x)f(x^B,y)\]

which after substituting with the found expressions for $f(x^A,y)$ and $f(x^B,y)$ becomes

\begin{align*}
  &f(x,y) = L^A(x)\{L^1(y)f^1 + L^4(y)f^4\} + L^B(x)\{L^2(y)f^2 + L^3(y)f^3\}\\
  &f(x,y) = L^A(x)L^1(y)f^1 + L^A(x)L^4(y)f^4 + L^B(x)L^2(y)f^2 + L^B(x)L^3(y)f^3 \enspace .
\end{align*}

Note that strictly speaking there are only 2 interpolation functions of the form $L^Q(\eta)$ since one-dimensional interpolation is taking place. Thus the interpolating polynomials satisfy the following equivalences:

where
\begin{align*}
L^A(x) & \equiv L^1(x)\\
L^B(x) & \equiv L^2(x)\\
L^1(y) & \equiv L^1(y)\\
L^2(y) & \equiv L^1(y)\\
L^3(y) & \equiv L^2(y)\\
L^4(y) & \equiv L^2(x) \enspace .
\end{align*}

The resulting two-variable shape functions $N^Q(x,y)$ follow from the product of one-dimensional interpolation functions like:

\begin{align*}
N^1(x,y) & = L^1(x)L^1(y)\\
N^2(x,y) & = L^2(x)L^1(y)\\
N^3(x,y) & = L^2(x)L^2(y)\\
N^4(x,y) & = L^1(x)L^2(y) \enspace .
\end{align*}

In the actual computer implementation of the discussed interpolation scheme it is desirable to have the actual functions embedded into the code instead of having the computer finding the corresponding Lagrange polynomials each time the size of the square domain changes. In practice, the functions $N^Q(x,y)$ are coded for a canonic square of general side $h$. This resulting canonic domain can be referred as a finite element.

\paragraph*{A 2D finite element}
From the geometric point of view a {\bf finite element} is a canonical interpolation domain together with a set of shape functions and its derivatives. \Cref{fig:four-nodes-interp} shows the shape functions for a so-called bi-linear element of side $h = 1.0$. The element is called bi-linear as linear (or first order) interpolation is used along the $x$ and $y$ directions. Elements of higher order result after adding nodal points and the required corrections to the 2D-shape functions $N^Q(x,y)$.

%The shape functions for a 9-noded element are displayed in \cref{fig:nine-nodes-interp}.

\begin{figure}[H]
\centering
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-4-nodes-1.pdf}
		\caption{Shape function ${N^1(x,y)=\frac{1}{4}(1-x)(1-y)}$. }
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-4-nodes-2.pdf}
		\caption{Shape function ${N^2(x,y)=\frac{1}{4}(1+x)(1-y)}$.}
	\end{subfigure}\\
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-4-nodes-3.pdf}
		\caption{Shape function ${N^3(x,y)=\frac{1}{4}(1+x)(1+y)}$.}
	\end{subfigure}\,
%
	\begin{subfigure}[b]{0.45\textwidth}\qquad
		\includegraphics[width=\textwidth]{shape_func-4-nodes-4.pdf}
		\caption{Shape function ${N^4(x,y)=\frac{1}{4}(1-x)(1+y)}$.}
	\end{subfigure}
\caption{Shape functions for a 4-nodes element.}
\label{fig:four-nodes-interp}
\end{figure}

\begin{tcolorbox}

See Notebooks 3 and 4 for an easy to follow Python implementation of interpolation resembling the finite element method.

\end{tcolorbox}

%\begin{figure}[H]
%\centering
%	\begin{subfigure}[b]{0.45\textwidth}\qquad
%		\includegraphics[width=\textwidth]{shape_func-9-nodes-1.pdf}
%		\caption{Shape function $N^1(x,y)$. }
%	\end{subfigure}\,
%%
%	\begin{subfigure}[b]{0.45\textwidth}\qquad
%		\includegraphics[width=\textwidth]{shape_func-9-nodes-2.pdf}
%		\caption{Shape function $N^2(x,y)$.}
%	\end{subfigure}\\
%%
%	\begin{subfigure}[b]{0.45\textwidth}\qquad
%		\includegraphics[width=\textwidth]{shape_func-9-nodes-3.pdf}
%		\caption{Shape function $N^3(x,y)$.}
%	\end{subfigure}\,
%%
%	\begin{subfigure}[b]{0.45\textwidth}\qquad
%		\includegraphics[width=\textwidth]{shape_func-9-nodes-4.pdf}
%		\caption{Shape function $N^4(x,y)$.}
%	\end{subfigure}\\
%	%
%	\begin{subfigure}[b]{0.45\textwidth}\qquad
%		\includegraphics[width=\textwidth]{shape_func-9-nodes-5.pdf}
%		\caption{Shape function $N^5(x,y)$.}
%	\end{subfigure}\,
%%
%	\begin{subfigure}[b]{0.45\textwidth}\qquad
%		\includegraphics[width=\textwidth]{shape_func-9-nodes-6.pdf}
%		\caption{Shape function $N^6(x,y)$.}
%	\end{subfigure}
%	\caption{Shape functions for a 9-nodes element.}
%	\label{fig:nine-nodes-interp}
%\end{figure}
%%
%\begin{figure} [H]
%	\ContinuedFloat
%	\centering
%	\begin{subfigure}[b]{0.45\textwidth}\qquad
%		\includegraphics[width=\textwidth]{shape_func-9-nodes-7.pdf}
%		\caption{Shape function $N^7(x,y)$.}
%	\end{subfigure}\,
%%
%	\begin{subfigure}[b]{0.45\textwidth}\qquad
%		\includegraphics[width=\textwidth]{shape_func-9-nodes-8.pdf}
%		\caption{Shape function $N^8(x,y)$.}
%	\end{subfigure}\\
%	%
%	\begin{subfigure}[b]{0.45\textwidth}\qquad
%		\includegraphics[width=\textwidth]{shape_func-9-nodes-9.pdf}
%		\caption{Shape function $N^9(x,y)$.}
%	\end{subfigure}
%\caption{Shape functions for a 9-nodes element. (Continued)}
%\end{figure}

\begin{tcolorbox}
\paragraph*{Finite element mesh}
Spatial-discretization of the computational domain is in the core of finite element analysis. This corresponds to the partition of the whole domain into {\bf finite elements}. The complete set of finite elements, and its defining attributes, corresponding to a particular domain is termed a {\bf mesh}. If the geometry is irregular the mesh would contain mostly distorted elements with respect to the canonical shape. In the finite element method this is nicely solved using space transformations between the distorted and the canonical shape.

\end{tcolorbox}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Interpolation over distorted domains}
So far all the 2D interpolation operations have taken place over perfectly squared domains of side $h$ where the interpolation functions are known at least in terms of the side parameter $h$. In many cases and particularly in finite element methods it is common to find distorted interpolation domains (see \cref{fig:physical}) which difficult the interpolation operation as the interpolation polynomials would be element-dependent and the problem would become impossible to code in a systematic way.

\begin{figure}[h]
\centering
\includegraphics[width=6cm]{rombo}
\caption{Distorted quadrilateral interpolation domain.}
\label{fig:physical}
\end{figure}

In these cases the solution approach is based upon the continuous mapping of the distorted domain and the functions defined in this space into a constant or canonical element where the interpolation functions are always the same. Moreover, the mapping between both spaces is conducted also using interpolation theory. This idea is explained next with reference to \cref{fig:natural domain}.

\begin{figure}[h]
\centering
\includegraphics[width=8cm]{figure1.pdf}
\caption{Schematic representation of the one-to-one mapping between points in the physical space (left) and the natural or canonical space (right).}
\label{fig:natural domain}
\end{figure}

In the figure the space of the distorted domain, and with position vectors $\overrightarrow x$, is termed the physical space as this corresponds to the space of interest in a particular problem. The figure also shows a perfectly squared domain where we denote space coordinates by a position vector $\overrightarrow r$: we refer to this perfectly squared element as the canonical element and to its mathematical space as the natural space. For reasons that will be explained later, it is convenient to  have canonical elements of side $h = 2.0$. Note also that since the canonical element is a constant square of side $h$ the corresponding Lagrange interpolation functions are always the same. This implies that if the interpolation process were to be conducted over the natural space it could be easily coded as it would amount to coding the interpolation functions per se.

In mathematical language the connection between both spaces is written in terms of general functional relations as:

\begin{equation}
\begin{aligned}
x_i &=x_i(\overrightarrow r)\\
r_I &=r_I(\overrightarrow x).
\end{aligned}
\label{eq:trans}
\end{equation}

The first of these functional relationships maps every point $\overrightarrow r$ from the canonical space into a point $\overrightarrow x$ of the physical space. In particular the relationship \ref{eq:trans}(a) can be written using:

\begin{equation}
x_i=N^Q(\overrightarrow r)x^Q.
\label{eq:trans_iso}
\end{equation}

where the physical space has been represented as an interpolated approximation using as the exact functions the coordinates of the nodal points of the physical space. In particular, the first expression provides the position vector $\overrightarrow x$ in the physical space for a point that in the canonical space occupies the position vector $\overrightarrow r$. Similarly, the inverse relation gives the position vector $\overrightarrow r$ in the canonical space for a point that occupies the position vector $\overrightarrow x$ in the physical space. Note that since in \ref{eq:trans}(a) we have used the interpolation functions formulated for the perfectly squared canonical space to approximate the geometry or actual physical space this same transformation can be used to transform functions as explained next.

Assume that $f=f(\overrightarrow x)$ is a function that describes the space variation of a quantity of interest over a given domain.  Using  the mapping \ref{eq:trans} it is also possible to obtain the variation of the quantity in the canonical space after writing:

\[ f=f(\overrightarrow x)\equiv f\lbrack x_i(\overrightarrow r)\rbrack\equiv F(\overrightarrow r). \]

In the above expression $F = F(\overrightarrow r)$ represents the same physical variable but expressed in terms of the position vector in the canonical space. Being able to represent functions in the physical and the canonical space is an important result since it is now possible to conduct interpolation of functions over constant domains as:

\begin{equation}
F(\overrightarrow r)=N^Q(\overrightarrow r)f^Q.
\label{eq:trans_inter}
\end{equation}

Note that as a result of the space transformation \ref{eq:trans}, finding the physical function at a point $r_I$ is equivalent to finding the function at an associated point $x_i$ in the physical domain.

\begin{tcolorbox}

Notebooks 3 and 4 in the REPO make use of Pyhton functions to perform interpolation over two-dimensional domains. In particular, NB4 applies interpolation over arbitrary domains to visualize closed-form solutions.

\end{tcolorbox}

%
\paragraph*{Proposed problems}
\begin{enumerate}

\item \label{punto01} For the computational domain $x\in\lbrack-1.0\;,\;1.0\rbrack$ and 3 nodal points corresponding to $x^1 = -1.0$, $x^2 = +1.0$ and $x^3 = 0.0$ find the Lagrange polynomials $L^1(x)$, $L^2(x)$ and $L^3(x)$.

\item \label{punto02} Verify that the polynomials $L^1(x)$, $L^2(x)$ and $L^3(x)$ satisfy the property $L^I(x^J)=\delta^{IJ}$.

\item \label{punto03} Implement a Python script that uses the vector of known values of a function $[f^1 , f^2 , f^3 ]$ and the polynomials from problem 1 and compute the interpolating polynomial $p(x)$.

\item \label{punto04} Using $p(x)$ from problem 3 compute and plot the first order derivative of $f(x)$ in the interval $[-1 , 1]$.

\item \label{punto 05} For the function  $f(x) = {x^3} + 4{x^2} - 10$ for $x$ in the range $[-1.0, 1.0]$ find values at nodal points that result from splitting the complete interval into 4 sub-domains each one with 3 nodal points. Using these values implement a local interpolation scheme using 2-nd order local interpolation polynomials. Plot the interpolation polynomial in each sub-domain and the corresponding interpolating function $p(x)$. In the same plot compare $p(x)$ and $f(x)$. Additionally, plot the first derivative of the function obtained from $p(x)$ and $f(x)$.

\item \label{punto 06} For the Runge function defined by:

\[f(x) = \frac{1}{{1 + 25{x^2}}}\]

implement an interpolation scheme using local 1st-order Lagrange polynomials. Use (i) sub-domains of constant size $\Delta x = 0.2$ and (ii) sub-domains whose size decreases towards the edges of the interval.

\item \label{punto 07} Using an independent script (or a notebook) implement a local interpolation scheme using a canonical element of size 2.0 and use it to approximate the Runge function.

\item \label{punto 08} Assume that at the 4 nodal points of a 2D square domain of side $2h$ we know the vector field

\[\overrightarrow u=u(x,y)\widehat i+v(x,y)\widehat j \]

and where $u(x,y)$ and $v(x,y)$ are the scalar rectangular components along the $x$ and $y$ direction of a cartesian coordinate system.

\begin{itemize}
\item[•] Implement an interpolation scheme to compute the vector field $\overrightarrow u\;=\;\overrightarrow u(x,y)$ at an arbitrary point $(x,y)$.
\item[•] Use the interpolation scheme to compute $\varepsilon_{xx}=\frac{\partial u}{\partial x}$ and $\varepsilon_{yy}=\frac{\partial v}{\partial y}$
\item[•] Implement a Python script to visualize the vector field and the scalars $\varepsilon_{xx}$ and $\varepsilon_{yy}$.
\end{itemize}

\item \label{punto 09}
Implement a Python script to visualize analytic (or numerical) solutions available at a set of nodes. Use the following steps;

\begin{itemize}
\item[•] Use external software\footnote{Gmsh is an open source software for pre and post processing of complex 1D, 2D and 3D domains. Meshio is a set of Python scripts to read and write Gmsh readable files using dictionaries.} to define and mesh and arbitrary solution domain.
\item[•] Evaluate the solution at the nodal points of the mesh and store the results into arrays.
\item[•] Use Python triangularization objects together with matplotlib routines to visualize the solution.
\end{itemize}

\end{enumerate}


\documentclass[12pt, letterpaper, oneside, notitlepage, onecolumn]{article}
\author{Buck Baskin}
\title{Vector-Rotation to Euler Angles}
\pagestyle{plain}

\usepackage{parskip}

\usepackage{textcomp}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{listings}
\usepackage{color}
\usepackage{verbatim}
\usepackage{soul}
\usepackage[margin=0.69in]{geometry}

% math
\usepackage{amsmath, amssymb, amsthm, gensymb}

\usepackage{graphicx}
\graphicspath{ {images/tc1/} }
% \includegraphics[height=6.75in,angle=270]{HW25}

\definecolor{dkgreen}{rgb}{0,0.6,0}
\definecolor{gray}{rgb}{0.5,0.5,0.5}
\definecolor{mauve}{rgb}{0.58,0,0.82}

\lstset{frame=tb,
  language=Matlab,
  aboveskip=3mm,
  belowskip=3mm,
  showstringspaces=false,
  columns=flexible,
  basicstyle={\small\ttfamily},
  numbers=none,
  numberstyle=\tiny\color{gray},
  keywordstyle=\color{blue},
  commentstyle=\color{dkgreen},
  stringstyle=\color{mauve},
  breaklines=true,
  breakatwhitespace=true,
  tabsize=3
}

\DeclareMathOperator*{\argmax}{arg\,max}

\DeclareMathOperator*{\argmin}{arg\,min}

\newcommand{\subsubsubsection}{\paragraph}
\newcommand{\bbs}[1]{\section{#1}}
\newcommand{\bbss}[1]{\subsection{#1}}
\newcommand{\bbsss}[1]{\subsubsection{#1}}
\newcommand{\bbssss}[1]{\subsubsubsection{#1}}

\newcommand{\norm}[1]{\left\lVert#1\right\rVert}

\begin{document}
\maketitle

Some calculations were verified using Python.

\section{Answer}

$R_{BA} = R_{y}(\alpha) R_{x}(\beta) R_{y}(\gamma)$

$R_{y} (\alpha) =
\left[\begin{matrix}\cos{\left (\alpha \right )} & 0 & \sin{\left (\alpha \right )}\\0 & 1 & 0\\- \sin{\left (\alpha \right )} & 0 & \cos{\left (\alpha \right )}\end{matrix}\right]$

$R_{x} (\beta) =
\left[\begin{matrix}1 & 0 & 0\\0 & \cos{\left (\beta \right )} & - \sin{\left (\beta \right )}\\0 & \sin{\left (\beta \right )} & \cos{\left (\beta \right )}\end{matrix}\right]$

$R_{y} (\gamma) =
\left[\begin{matrix}\cos{\left (\gamma \right )} & 0 & \sin{\left (\gamma \right )}\\0 & 1 & 0\\- \sin{\left (\gamma \right )} & 0 & \cos{\left (\gamma \right )}\end{matrix}\right]$

$R_{BA} =
\left[\begin{matrix}\cos{\left (\alpha \right )} & 0 & \sin{\left (\alpha \right )}\\0 & 1 & 0\\- \sin{\left (\alpha \right )} & 0 & \cos{\left (\alpha \right )}\end{matrix}\right]
\left[\begin{matrix}1 & 0 & 0\\0 & \cos{\left (\beta \right )} & - \sin{\left (\beta \right )}\\0 & \sin{\left (\beta \right )} & \cos{\left (\beta \right )}\end{matrix}\right]
\left[\begin{matrix}\cos{\left (\gamma \right )} & 0 & \sin{\left (\gamma \right )}\\0 & 1 & 0\\- \sin{\left (\gamma \right )} & 0 & \cos{\left (\gamma \right )}\end{matrix}\right]$

$R_{BA} =
\left[\begin{matrix}- \sin{\left (\alpha \right )} \sin{\left (\gamma \right )} \cos{\left (\beta \right )} + \cos{\left (\alpha \right )} \cos{\left (\gamma \right )} & \sin{\left (\alpha \right )} \sin{\left (\beta \right )} & \sin{\left (\alpha \right )} \cos{\left (\beta \right )} \cos{\left (\gamma \right )} + \sin{\left (\gamma \right )} \cos{\left (\alpha \right )}\\\sin{\left (\beta \right )} \sin{\left (\gamma \right )} & \cos{\left (\beta \right )} & - \sin{\left (\beta \right )} \cos{\left (\gamma \right )}\\- \sin{\left (\alpha \right )} \cos{\left (\gamma \right )} - \sin{\left (\gamma \right )} \cos{\left (\alpha \right )} \cos{\left (\beta \right )} & \sin{\left (\beta \right )} \cos{\left (\alpha \right )} & - \sin{\left (\alpha \right )} \sin{\left (\gamma \right )} + \cos{\left (\alpha \right )} \cos{\left (\beta \right )} \cos{\left (\gamma \right )}\end{matrix}\right]$

The equivalent rotation to $R(\begin{bmatrix}1/\sqrt{3} \\ 1/\sqrt{3} \\ 1/\sqrt{3}\end{bmatrix}, -90 \degree)$ using Rodriguez Formula:

$e^{\hat{\omega} \theta} = I + \hat{\omega} sin(\theta) + (1 - cos(\theta) )\hat{\omega}^{2}$

For $\theta = -90$:

$e^{\hat{\omega} \theta} = I - \hat{\omega} + \hat{\omega}^{2}$

% ---- %
$\hat{\omega} =
\left[\begin{matrix}0 & - \frac{\sqrt{3}}{3} & \frac{\sqrt{3}}{3}\\\frac{\sqrt{3}}{3} & 0 & - \frac{\sqrt{3}}{3}\\- \frac{\sqrt{3}}{3} & \frac{\sqrt{3}}{3} & 0\end{matrix}\right]$

$\hat{\omega}^{2} =
\left[\begin{matrix}- \frac{2}{3} & \frac{1}{3} & \frac{1}{3}\\\frac{1}{3} & - \frac{2}{3} & \frac{1}{3}\\\frac{1}{3} & \frac{1}{3} & - \frac{2}{3}\end{matrix}\right]$

$e^{\hat{\omega} \theta} = I -
\left[\begin{matrix}0 & - \frac{\sqrt{3}}{3} & \frac{\sqrt{3}}{3}\\\frac{\sqrt{3}}{3} & 0 & - \frac{\sqrt{3}}{3}\\- \frac{\sqrt{3}}{3} & \frac{\sqrt{3}}{3} & 0\end{matrix}\right]
 +
\left[\begin{matrix}- \frac{2}{3} & \frac{1}{3} & \frac{1}{3}\\\frac{1}{3} & - \frac{2}{3} & \frac{1}{3}\\\frac{1}{3} & \frac{1}{3} & - \frac{2}{3}\end{matrix}\right]$

$e^{\hat{\omega} \theta} = \left[\begin{matrix}\frac{1}{3} & \frac{1}{3} + \frac{\sqrt{3}}{3} & - \frac{\sqrt{3}}{3} + \frac{1}{3}\\- \frac{\sqrt{3}}{3} + \frac{1}{3} & \frac{1}{3} & \frac{1}{3} + \frac{\sqrt{3}}{3}\\\frac{1}{3} + \frac{\sqrt{3}}{3} & - \frac{\sqrt{3}}{3} + \frac{1}{3} & \frac{1}{3}\end{matrix}\right] = R_{BA} =$
$\begin{bmatrix}
r_{11} & r_{12} & r_{13} \\
r_{21} & r_{22} & r_{23} \\
r_{31} & r_{32} & r_{33} \\
\end{bmatrix}$

$\left[\begin{smallmatrix}- \sin{\left (\alpha \right )} \sin{\left (\gamma \right )} \cos{\left (\beta \right )} + \cos{\left (\alpha \right )} \cos{\left (\gamma \right )} & \sin{\left (\alpha \right )} \sin{\left (\beta \right )} & \sin{\left (\alpha \right )} \cos{\left (\beta \right )} \cos{\left (\gamma \right )} + \sin{\left (\gamma \right )} \cos{\left (\alpha \right )}\\\sin{\left (\beta \right )} \sin{\left (\gamma \right )} & \cos{\left (\beta \right )} & - \sin{\left (\beta \right )} \cos{\left (\gamma \right )}\\- \sin{\left (\alpha \right )} \cos{\left (\gamma \right )} - \sin{\left (\gamma \right )} \cos{\left (\alpha \right )} \cos{\left (\beta \right )} & \sin{\left (\beta \right )} \cos{\left (\alpha \right )} & - \sin{\left (\alpha \right )} \sin{\left (\gamma \right )} + \cos{\left (\alpha \right )} \cos{\left (\beta \right )} \cos{\left (\gamma \right )}\end{smallmatrix}\right] = $
$\begin{bmatrix}
r_{11} & r_{12} & r_{13} \\
r_{21} & r_{22} & r_{23} \\
r_{31} & r_{32} & r_{33} \\
\end{bmatrix}$

\subsection{Solving for $\alpha, \beta, \gamma$}

\subsubsection{$\beta$}

Using the atan method, one can solve for $\beta$ using the middle column of the matrix. The middle value, $r_{22} = cos(\beta)$. Using $r_{12}^{2} + r_{32}^{2} = sin^{2}(\beta)(sin^{2}(\gamma) + cos^{2}(\gamma)) = sin^{2}(\beta)$, $sin(\beta) = \sqrt{r_{12}^{2} + r_{32}^{2}}$. Therefore, $\beta = atan2(\sqrt{r_{12}^{2} + r_{32}^{2}}, r_{22})$.

\subsubsection{$\gamma$}

Looking at two elements in the middle row of the matrix, one can solve for $\gamma$ using the atan method. $r_{21} = sin(\beta) sin(\gamma)$, so $sin(\gamma) = r_{21} / sin(\beta)$. $r_{23} = -sin(\beta) cos(\gamma)$, so $cos(\gamma) = -r_{23} / sin(\beta)$. Therefore, $\gamma = atan2(r_{21} / sin(\beta), -r_{23} / sin(\beta))$.

\subsubsection{$\alpha$}

$r_{12} = sin(\alpha) sin(\beta)$, $sin(\alpha) = r_{12} / sin(\beta)$

$r_{32} = cos(\alpha) sin(\beta)$, $cos(\alpha) = r_{32} / sin(\beta)$

$\alpha = atan2(r_{12} / sin(\beta), r_{32} / sin(\beta))$

\subsection{Conclusion}

Substituting the correct values from $R_{BA}$ calculated by Rodriguez Formula into the spaces allocated by $r_{ij}$ that were solved into an $atan2$ formula using $R_{BA}$ calculated by the rotation matrices gives the solution for the equivalent rotations in radians (rounded to 3 decimal places).

$\alpha = 1.833, \beta = 1.231, \gamma = -2.880$

\end{document}


\chapter{Yao's Principle}
Using game theory we may develop a method for proving lower bounds on complexity
of randomized algorithms. To illustrate this method we will study the problem we
duscussed in \Cref{chapter:expectation,chapter:structural-induciton}: Alice
choose a number from $\range{1000}$, Bob can flip a coin and ask yes/no
questions; Bob would like to guess it using, on average, as few questions as
possible. We claim that Bob needs at least $9$ questions.
\begin{theorem}
\label{theorem:randomized-dts-lower-bound}
  Let $\Omega$ be a set of $B$-decision trees and $\Distribution{D}$ be a
  probability distribution on $\Omega$. If $\DTValue{T}{x} = x$ for all
  $x \in \range{1000}$ and $T \in \Omega$, then
  $\expectation[\Distribution{D}]{\DTValue{T}{x}} \ge 9$ for some $x \in
  \range{1000}$.
\end{theorem}

To prove this statement we would need to give formal notions of randmoized
$B$-decision tree complexity and heuristic $B$-decision tree cmplexity.
\begin{definition}
  Let $f : [1000] \to \Z$ be a function. We say that $\bar{T} =
  (\Omega, \Distribution{T})$ is a \emph{randomized}\footnote{%
    In fact, there are many notions of randomized computation; the algorithms
    discussed in this chapter are usually called zero-error randomized
    algorithms.
  }
  \emph{$B$-decision tree for $f$ of cost $C$}
  if $\Omega$ is a set of $B$-decision trees, $\Distribution{T}$ is a
  probability distribution on $\Omega$, $\DTValue{T}{x} = f(x)$ for all $x \in
  \range{1000}$, and
  $\expectation[T \gets \Distribution{T}]{\DTHeight[x]{T}} = C$.

  We also say that $T$ is a \emph{heuristic $B$-decision tree for $f$ of
  expected cost $C$ with respect to a distribution $\Distribution{X}$} if
  $\DTValue{T}{x} = f(x)$ for all $x \in \range{1000}$ and
  $\expectation[x \gets \Distribution{X}]{\DTHeight[x]{T}} = C$.

  The \emph{randmoized $B$-decision tree complexity $\RandCBDT{f}$ of $f$}
  is equal to the minimal $C$ such that there is a randomized $B$-decision tree
  for $f$ of cost $C$.

  The \emph{heuristic $B$-decision tree complexity
  $\HeurCBDT[\Distribution{X}]{f}$ of $f$ with respect to $\Distribution{X}$} is
  equal to the minimal $C$ such that there is a $B$-decision tree for $f$ of
  expected cost $C$ with respect to $\Distribution{X}$. Finally, \emph{the
  heuristic $B$-decision tree complexity $\HeurCBDT{f}$ of $f$} is equal to the
  maximum of $\HeurCBDT[\Distribution{X}]{f}$ over all distributions
  $\Distribution{X}$.
\end{definition}

Nonetheless that we have not had the notion of randomized complexity before, we
proved upper bounds in \Cref{chapter:expectation}. Unfortunately, proving lower
bounds directly for randomized complexity is not easy, but it is usually way
easier to prove lower bounds on heuristic complexity. Yao's principle (also
called Yao's minimax principle or Yao's lemma) gives a connection between these
measures.
\begin{theorem}[Yao's principle]
\label{theorem:yaos-principle}
  Let $f : [1000] \to \Z$ be a function. Then $\RandCBDT{f} =
  \HeurCBDT{f}$.
\end{theorem}

Using this principle is easy to prove the lower bound.
\begin{proof}[Proof of \Cref{theorem:randomized-dts-lower-bound}]
  Note that the theorem says that $\RandCBDT{\identity{\range{1000}}} \ge 9$;
  hence, it is enough to prove that $\HeurCBDT{\identity{\range{1000}}} \ge 9$.
  Let us assume the opposite; i.e., that for any distribution $\Distribution{X}$
  there is a $B$-decision tree $T$ such that $\DTValue{T}{x} = x$ for all $x \in
  \range{1000}$ and $\expectation[x \gets \Distribution{X}]{\DTHeight[x]{T}} <
  9$.
  This implies that there is a $B$-decision tree $T$ such that
  $\frac{1}{1000} \sum_{x \in \range{1000}} \DTHeight[x]{T} < 9$; we are going to prove that
  this is impossible. Let us fix such a $T$.

  We prove, using induction by $|S|$ that $\sum_{x \in S} \DTHeight[x]{T} \ge
  |S| \log_2 |S|$. The base case for $|S| = 1$ is clear since
  $\DTHeight[x]{T} \ge 1$ for all $x \in \range{1000}$.

  Let us prove the induction step. Let $T = \DTIf{f}{T_0}{T_1}$, and let
  $S_0 = \set[f(x) = 0]{x \in S}$ and $S_1 = \set[f(x) = 1]{x \in S}$.
  It is clear that $\sum_{x \in S} \DTHeight[x]{T} = |S| +
  \sum_{x \in S_0} \DTHeight[x]{T} + \sum_{x \in S_1} \DTHeight[x]{T}$. By the
  induction hypothesis, this implies that $\sum_{x \in S} \DTHeight[x]{T} = |S|
  + |S_0| \log_2 |S_0| + |S_1| \log_2 |S_1| \ge |S| + |S| \log_2 (|S| / 2) =
  |S| \log_2 |S|$.

  Therefore, $\frac{1}{1000} \sum_{x \in \range{1000}} \DTHeight[x]{T} \ge
  \log_2 1000 > 9$, which contradicts the assumption.
\end{proof}

Now we are ready to prove Yao's principle.
\begin{proof}[Proof of \Cref{theorem:yaos-principle}]
  We start from proving that $\RandCBDT{f} \ge \HeurCBDT{f}$. Assume the
  opposite; i.e., that $\RandCBDT{f} < \HeurCBDT{f}$.

  Let $\bar{T} = (\Omega, \Distribution{T})$ be a randomized $B$-decision tree
  for $f$ of cost $C < \HeurCBDT{f}$. Note that
  $\expectation[T \gets \Distribution{T}]{\DTHeight[x]{T}} \le C$ for all $x \in
  \range{1000}$. Hence,
  $\expectation[x \gets \Distribution{X}]{
    \expectation[T \gets \Distribution{T}]{\DTHeight[x]{T}}
  } \le C$. However,
  \begin{multline*}
    C \ge
    \expectation[x \gets \Distribution{X}]{
      \expectation[T \gets \Distribution{T}]{\DTHeight[x]{T}}
    } = \\
    \expectation[T \gets \Distribution{T}]{
      \expectation[x \gets \Distribution{X}]{\DTHeight[x]{T}}
    } \ge
    \expectation[T \gets \Distribution{T}]{
      \HeurCBDT[\Distribution{X}]{f}
    } = \HeurCBDT[\Distribution{X}]{f}.
  \end{multline*}
  As a result, $C \ge \HeurCBDT{f}$, which is a contradiction to the assumption.

  Let us prove that $\RandCBDT{f} \le \HeurCBDT{f}$. To prove this inequality we
  are going to use von Neumann's minimax theorem (\Cref{theorem:von-neumanns-minimax}).
  Let $X = \range{1000}$, $Y$ be a set of all $B$-decision trees $T$ such that
  $\DTHeight{T} \le 1000$ and $\DTValue{T}{x} = f(x)$ for all $x \in
  \range{1000}$, and $A(x, T) = - B(x, T) = \DTHeight[x]{T}$.

  \Cref{theorem:von-neumanns-minimax} implies that there are distributions
  $\Distribution{X}$ and $\Distribution{T}$ over $X$ and $Y$ respectively such
  that
  \[
    \expectation[x \gets \Distribution{X}]{\DTHeight[x]{T'}} \ge
    \expectation[x \gets \Distribution{X}]{
      \expectation[T \gets \Distribution{T}]{\DTHeight[x]{T}}
    } \ge
    \expectation[T \gets \Distribution{T}]{\DTHeight[x']{T}}
  \]
  for all $x' \in X$ and $T' \in Y$. Which implies that
  \[
    \HeurCBDT[\Distribution{X}]{f} \ge
    \expectation[x \gets \Distribution{X}]{
      \expectation[T \gets \Distribution{T}]{\DTHeight[x]{T}}
    } \ge \RandCBDT{f}.
  \]
\end{proof}


%
% CMPT 433: Embedded Systems - A Course Overview
% Section: Bitwise
%
% Author: Jeffrey Leung
%

\section{Bitwise}
	\label{sec:bitwise}
\begin{easylist}

& Bitwise operators:
	&& OR (|): Set selected bits
	&& AND (\&): Clear unselected bits; only show selected
	&& NOT (\~): Invert all bits
	&& XOR (\textasciicircum): Invert selected bits
	&& Bitshifting (\lstinline[columns=fixed]{<<} for left, \lstinline[columns=fixed]{>>} for right): Moving all bits one way or another and removing the displaced digit
	&& Mask: Selects a field in a bit-flag

& Bitshifting 1 by the number of a certain position creates a mask for that position, which can be ORed to toggle it (i.e. if \lstinline[columns=fixed]{LED0_BIT == 4}, then \lstinline[columns=fixed]{1 << LED0_BIT} creates a mask at the 4th bit from the right)

& Test to check whether data fields are listed in the correct order in the data value

& Applying a mask with \& and checking whether the remaining value is equal to 0 returns whether the bit is on or off

& Given the following predefined values where LEDs are active high and buttons are active low:

\begin{lstlisting}[language=C, columns=fixed]
VALUE

LED0_BIT
LED1_BIT
LED2_BIT
LED_MASK = (1 << LED0_BIT) | (1 << LED3_BIT) | (1 << LED2_BIT)

BTN0_BIT
BTN1_BIT
BTN_MASK = (1 << BTN0_BIT) | (1 << BTN1_BIT)

SPD_BIT_BEGIN
SPD_MASK
\end{lstlisting}

\vspace{1cm}

The calculations are as follows:

\begin{lstlisting}[language=C, columns=fixed]
_Bool isLed0On = (VALUE & (1 << LED0_BIT)) != 0;

_Bool isAnyLEDOn = ((VALUE & LED_MASK) != 0);

_Bool areAllLEDsOn = (VALUE & LED_MASK) == LED_MASK;

// If (VALUE & BTN_MASK) == BTN_MASK,
// then because buttons are active low, no buttons are pressed.
_Bool isAnyButtonPressed = (VALUE & BTN_MASK) != BTN_MASK;

_Bool areAllButtonsPressed = (VALUE & BTN_MASK) == 0;
\end{lstlisting}

\clearpage

\begin{lstlisting}[language=C, columns=fixed]
void turnOnLed0() {
	VALUE |= (1 << LED0_BIT);
}

void turnOnAllLeds() {
	VALUE |= LED_MASK;
}

void turnOffLed() {
	// ~(1 << LED_BIT) sets the LED0 bit to 0, and all
	// other bits to 1.
	// ANDing it changes the LED0 bit to 0, and does not
	// change other bits.
	VALUE &= ~(1 << LED_BIT);
}

void turnOffLeds1And2() {
	VALUE &= ~(1 << LED1_BIT | 1 << LED2_BIT);
}

void turnOffAllLeds() {
	VALUE &= ~LED_MASK;
}

void turnOffAllLedsExcept2() {
	// Remove the LED2 bit from the inverted LED mask
	VALUE &= (~LED_MASK | (1 << LED2_BIT));
}

void toggleLed0() {
	VALUE ^= (1 << LED0_BIT);
}

void toggleAllLeds() {
	VALUE ^= LED_MASK;
}

// Assume ints are in the correct format and do not need
// to be converted to/from binary.
int getSpeed() {
	// Mask the correct bits
	// Bitshift the value so that the speed is at the least
	// significant (rightmost) bit
	return (VALUE & SPD_MASK) >> SPD_BIT;
}

int setSpeed(int speed) {
	// Shift speed to the correct location
	// Remove excess bits outside the speed bits
	int newSpeedBits = (speed << SPD_BIT) & SPD_MASK;
	// Create the value with cleared speed bits
	// Add the new speed bit
	VALUE = (VALUE & ~SPD_MASK) | newSpeedBits;
}
\end{lstlisting}

\end{easylist}
\clearpage


\RequirePackage[l2tabu, orthodox]{nag}
\documentclass{article}

\usepackage[letterpaper]{geometry}
\usepackage{booktabs}
\usepackage{mathtools}
\usepackage[binary-units=true]{siunitx}
\usepackage{tikz}

\title{ECE 487 Assignment 6}
\author{Michael Kwok}
\begin{document}

\maketitle
\subsection*{CSMA/CD minimum frame size}
Propagation Time: \(T_p = \frac{1200}{3 \times 10^8} = 4\times 10 ^{-6}\) \\
Frame time: \(T_{fr} \geq 8 \times 10^{-6} = \SI{8}{\micro\second}\) \\
Minimum frame size: \( \SI{10}{\mega\bit\per\second} \cdot \SI{8}{\micro\second} = 80 \text{ bits}\)

\subsection*{Reservation Access Method}
Reservation phrase: \(\SI{10}{\micro\second} \cdot 5 = \SI{50}{\micro\second}\) \\
Data transmission time: \( \frac{5 \cdot 1000}{10 \times 10^6} = \SI{0.00055}{\second} \) \\
Throughput: \(\frac{0.001}{0.00055} = \SI{1.818}{\mega\bit\per\second}\)

\subsection*{WLAN}
\begin{enumerate}
    \item First transmission: \(\text{DIFS} + \SI{200}{\micro\second} = \SI{250}{\micro\second}\)
    \item \(\text{RTS}+ \text{SIFS} + \text{CTS} + \text{SIFS} + \text{DATA} + \text{SIFS} + \text{ACK} + (\text{Station 2 timer} - \text{Station 1 timer}) = \SI{1580}{\micro\second}\)
\end{enumerate}

\end{document}


\documentclass[axodraw]{amsart}
\usepackage{amsmath}
\usepackage{feyn}
\usepackage{feynmp}
\begin{document}

\section{feyn}

\begin{align*}
    \feyn{a} \\
    \feyn{b} \\
    \feyn{c} \\
    \feyn{d} \\
    \feyn{e} \\
    \feyn{f} \\
    \feyn{g} \\
    \feyn{h} \\
    \feyn{i} \\
    \feyn{j} \\
    \feyn{k} \\
    \feyn{l} \\
    \feyn{m} \\
    \feyn{n} \\
    \feyn{o} \\
    \feyn{p} \\
    \feyn{q} \\
    \feyn{r} \\
    \feyn{s} \\
    \feyn{t} \\
    \feyn{u} \\
    \feyn{v} \\
    \feyn{w} \\
    \feyn{x} \\
    \feyn{y} \\
    \feyn{z} \\
\end{align*}
\[
\Diagram{\vertexlabel^a \\
  fd \\
& g\vertexlabel_{\mu,c} \\
\vertexlabel_b fu \\
}
=
\Diagram{\vertexlabel^a \\
  fdA \\
& gA\vertexlabel_{\mu,c} \\
\vertexlabel_b fuV \\
}
=
\Diagram{\momentum{fdA}a\\
   & \vertexlabel_{\gamma_\mu} \momentum{gA}{\mu,c} \\
\momentum{fuV}b
}
=
ig\gamma_\mu (T^c)_{ab}
\]

Two-loop diagram:
\[
\feyn{ff gl ff}
\to
\feyn{ff\bigbosonloopA\smallbosonloopV ff}
=
\feyn{!{fA}p f!{glBV}k !{glSA}l a\vertexlabel_{p+k-l} ff}
\]

Bremsstrahlung:
\[
  \Diagram{        & x \\
                   & ![ulft]{gvV}{q=p'+k-p}   & !{guA}k \\
           ![bot]{fA}p & ![bot]{fA}{p'+k} & ![lrt]{fA}{p'} \\
        }
        +
  \Diagram{    & !{guA}k \\
           !{fA}p & fs ![bot]{fA}{p-k} & !{fA}{p'} \\
               &         & ![lrt]{gvA}{q=p'+k-p} \\
               &         & x\\
        }
\]

OPE:
\def\bracket#1{\langle #1 \rangle}
\begin{eqnarray*}
-i\Sigma_{\mathrm{ope}} &=&
     \left[\feyn{faf + fsfglffs
                 + \cdots} \right] 1 \\
     && \quad{}+
           \left[ \feyn{ fxfs0glfs0xf
                  + \cdots} \right]
         \bracket{\overline\psi M \psi} \\
     && \quad{}+ \left[
         \Diagram{   &x  &x  \\
                     &gv &gv \\
                  fs &fs &fs \\} + \cdots
         \right]
         \bracket{G_{\mu\nu}^a G_{\mu\nu}^a}\\
\end{eqnarray*}

Complete vertex:
\begin{eqnarray*}
\feyn{fcf} &=& \feyn{faf + fpf + fpfpf + \cdots} \\
    &=& \sum_{n=0}^\infty \feyn{fA ( pfA)}^n \\
    &=& \feyn{\frac{fA}{1-(pfA)}}.
\end{eqnarray*}

\section{feynmp}
\begin{fmffile}{simple} \begin{fmfgraph}(40,25)
% Note that the size is given in normal parentheses % instead of curly brackets.
% Define external vertices from bottom to top
\fmfleft{i1,i2}
\fmfright{o1,o2}
\fmf{fermion}{i1,v1,o1}
\fmf{fermion}{i2,v2,o2}
\fmf{photon}{v1,v2}
\end{fmfgraph}
\end{fmffile}

\begin{fmffile}{simple_tree} \begin{fmfgraph}(40,25)
\fmfleft{i} \fmfright{o1,o2,o3} \fmf{fermion}{i,v1,o1} \fmf{photon}{v1,v2} \fmf{fermion}{o2,v2,o3}
\end{fmfgraph} \end{fmffile}

\begin{fmffile}{simple_box} \begin{fmfgraph}(40,25)
\fmfleft{i1,i2} \fmfright{o1,o2} \fmf{fermion}{i1,v1,v2,o1} \fmf{fermion}{o2,v4,v3,i2} \fmf{photon}{v1,v3} \fmf{photon}{v2,v4}
\end{fmfgraph} \end{fmffile}

\begin{fmffile}{simple_labels} \begin{fmfgraph*}(40,25)
\fmfleft{i1,i2} \fmfright{o1,o2}
\fmflabel{$e^-$}{i1} \fmflabel{$e^+$}{i2} \fmflabel{$e^+,\mu^+$}{o1} \fmflabel{$e^-,\mu^-$}{o2}
\fmflabel{$i\sqrt{\alpha}$}{v1} \fmflabel{$i\sqrt{\alpha}$}{v2}
\fmf{fermion}{i1,v1,i2} \fmf{fermion}{o1,v2,o2}
\fmf{photon,label=$\gamma,,Z^0$}{v1,v2} \end{fmfgraph*}
\end{fmffile}

\begin{fmffile}{tree} \begin{fmfgraph}(40,30)
\fmfbottom{i1,d1,o1} \fmfright{o0,o2,o3} \fmf{fermion}{i1,v1,o1} \fmffreeze \fmf{fermion}{o2,v2,o3} \fmf{photon,tension=1.5}{v1,v2}
\end{fmfgraph} \end{fmffile}

\begin{fmffile}{simple_cross} % This is a bad sample. \begin{fmfgraph}(40,25)
\fmfleft{i1,i2} \fmfright{o1,o2} \fmf{fermion}{i1,v1,o2} \fmf{fermion}{i2,v2,o1} \fmf{photon}{v1,v2}
\end{fmfgraph} \end{fmffile}

\begin{fmffile}{penguin_both} \begin{fmfgraph}(40,25)
\fmfleft{i1,i2} \fmfright{o1,o2} \fmf{fermion}{i1,v1,o1} \fmf{fermion}{i2,v2,v3,v4,o2} \fmf{photon}{v1,v3} \fmf{photon,left=0.5,tension=0.2}{v2,v4} % W line
\end{fmfgraph} \end{fmffile}

\begin{fmffile}{penguin_oneside} \begin{fmfgraph*}(40,30)
\fmfstraight \fmfleft{g1,g2,i3} % g1 and g2 will be used later \fmfright{o1,o2,o3} \fmf{fermion}{i3,v2,v3,v4,o3} \fmf{fermion}{o1,v1,o2} \fmf{photon,label=$Z^0$}{v1,v3} \fmf{photon,left=0.5,tension=0.2}{v2,v4}
\fmf{phantom}{g1,v1,g2} % pull Z0 from the left \end{fmfgraph*}
\end{fmffile}

\begin{fmffile}{sunset} \begin{fmfgraph}(40,40)
\fmfleft{i} \fmfright{o} \fmf{plain,tension=5}{i,v1} \fmf{plain,tension=5}{v2,o} \fmf{plain,left,tension=0.4}{v1,v2,v1} \fmf{plain}{v1,v2} \fmfdot{v1,v2}
\end{fmfgraph} \end{fmffile}

\begin{fmffile}{ears} \begin{fmfgraph}(50,30)
\fmftop{t0,t1,t2,t3} \fmfbottom{b0,b1,b2,b3} \fmf{phantom}{t1,v1,b1} \fmf{phantom}{t2,v2,b2} \fmffreeze \fmf{plain,right}{v1,v2,v1} \fmf{plain,tension=0.8,right=270}{v1,v1} \fmf{plain,tension=0.8,right}{v2,v2} \fmfdot{v1,v2}
\end{fmfgraph} \end{fmffile}

\begin{fmffile}{eyeglasses} \begin{fmfgraph}(50,30)
\fmfleft{i} \fmfright{o} \fmf{phantom,tension=10}{i,i1} \fmf{phantom,tension=10}{o,o1} \fmf{fermion,left,tension=0.4}{i1,v1,i1} \fmf{fermion,right,tension=0.4}{o1,v2,o1} \fmf{dashes}{v1,v2}
\end{fmfgraph} \end{fmffile}

\end{document}


\batchmode
\documentclass[paper=a4, fontsize=12pt, prl, notitlepage]{revtex4-1}
\usepackage[ssp]{xetex_import}
\linespread{1.4}
\errorstopmode

\begin{document}

\title{Math behind Bayesian uncertainty propagation using normal approximations}
\author{Christopher Körber}
\affiliation{%
Department of Physics,
University of California,
Berkeley, CA 94720, USA\\
Nuclear Science Division, %
LBNL,
Berkeley, CA 94720, USA
}
\date{\today}
\begin{abstract}
This document addresses the math used to propagate errors and to infer posterior distributions (Bayesian statistics) used in \href{https://github.com/pennsignals/chime_sims/pull/49}{the performance boost through normal approximations PR in the \texttt{pennsignals/chime\_sims} repo}.
\end{abstract}

\maketitle

%==============================================================================
% Content
%==============================================================================

\section{Assumptions}
The general theme of this document is: \textit{casting everything to normal distribution simplifies computations}.

In general, the techniques in PR \#49 make two approximations
\begin{enumerate}
    \item All model and data \textit{input} distributions can be sufficiently approximated with multivariate normal distributions
    \item The posterior distributions can be sufficiently approximated by a multivariate normal distribution
\end{enumerate}

\section{Error propagation}
The \texttt{gvar} module is used to analytically propagate errors.
The eqution used to computed the standard deviation $\sigma_f$ of a function $f(\vec p)$ at a given point in the parameter space $\bar {\vec p}$ is
\begin{equation}
    \sigma_f^2(\bar {\vec p})
    =
    \left[
        \sum_{i,j=1}^N
        \frac{\partial f(\vec p)}{\partial p_i}
        \left(\Sigma_p\right)_{ij}
        \frac{\partial f(\vec p)}{\partial p_j}
    \right]_{\vec p = \bar {\vec p}}
    \, ,
\end{equation}
where $\Sigma_p$ is the covariance matrix of the normally distributed parameters $\vec p$.
In the uncorrelated case, this simplifies to
\begin{equation}
    \sigma_f^2(\bar {\vec p})
    =
    \left[
        \sum_{i=1}^N
        \left(\frac{\partial f(\vec p)}{\partial p_i}\right)^2
        \sigma_{p_i}^2
    \right]_{\vec p = \bar {\vec p}}
    \, .
\end{equation}

The module \texttt{gvar} is capable of tracing such derivatives since implemented functions are aware of their analytical derivatives.

\subsubsection{Example}
For example, suppose
\begin{align}
    f(x, y)
    &=
    x y^2
    && \Rightarrow &
    \sigma_f^2(\bar{\vec p})
    &=
    y^4 (\Sigma_p)_{xx}
    + 2 x y^3  \left[ (\Sigma_p)_{xy} + (\Sigma_p)_{yx} \right]
    + 4 x^2 y^2 (\Sigma_p)_{yy}
\end{align}
Thus, if $x, y$ follow a multivariate normal distribution of mean $\bar{\vec p}$ and covariance $\Sigma_p$, one finds
\begin{align}
    \bar{\vec p}
    &=
    \begin{pmatrix}
        1 \\ -2
    \end{pmatrix}
    \, , &
    \Sigma_p
    &=
    \begin{pmatrix}
        2 & 1 \\ 1 & 1
    \end{pmatrix}
    &
    \Rightarrow
    \sigma_f(\bar{\vec p})
    &=
    \sqrt{32 - 32 + 16} = 4
\end{align}
The corresponding \texttt{gvar} code returns
\begin{lstlisting}[style=python]
from gvar import gvar

mean = [1, -2]
cov = [[2, 1], [1, 1]]
x, y = gvar(mean, cov)
f = x * y ** 2
print(f)

> 4.0(4.0)
\end{lstlisting}

\section{Computation of the posterior}

This section explains how the \texttt{lsqfit} module approximates the posterior distribution $P(\vec p|D,M)$ given data $D$, an input model $M$ and it's corresponding priors $P(\vec p| M)$.

\subsection{Defintions}

The posterior distribution $P(\vec p|D, M)$ is proportional to the prior times the probability of the data given the model and parameters $P(D|\vec p, M)$ (the likelihood)
\begin{align}
    P(\vec p|D, M) &=
    \frac{P(D|\vec p, M)P(\vec p | M)}{P(D|M)}
    \propto
    P(D|\vec p, M)P(\vec p | M)
    \, .
\end{align}
The marginal likelihood of the data $D$ given the model $M$ is obtained by integrating over the whole parameter space
\begin{equation}
    P(D|M)
    =
    \int d \vec p P(D|\vec p, M)P(\vec p | M) \, .
\end{equation}
Because the posterior is normalized by the ratio of both distributions, one can neglect constant factors in the computation.

The likelihood of the data given the model and parameters is described by a $\chi^2$ distribution
\begin{equation}
    P(D|\vec p, M)
    \sim
    \exp\left\{
        - \frac{1}{2}
        \sum_{i,j=1}^N
        \left[y_i - \vec f_M(x_i, \vec p)\right]
        \left(\Sigma_y^{-1}\right)_{ij}
        \left[y_j - \vec f_M(x_j, \vec p)\right]
    \right\}
    =
    \exp\left\{
        - \frac{1}{2}
        \chi^2_D(\vec p)
    \right\}
    \, ,
\end{equation}
where $\Sigma_y$ is the covariance matrix of the data and $f_M(x_j, \vec p)$ the model function evaluated at point $x_j$ which aims to describe the data point $y_j$.

Maximizing the Likelihood as a function of $\vec p$ corresponds to minimizing the exponent--which is the standard $\chi^2$-minimization procedure.
Computing the posterior distribution function for a given prior $P(\vec p| M)$ is called Bayesian inference.
Normal approximations of the posterior distribution are somewhere in the middle of a full Bayesian treatment and regular $\chi^2$-minimization.

\subsection{Normal approximation of the posterior}
Including the multivariate normal prior distribution with mean $\vec p_0$ and covariance $\Sigma_{p_0}$, the posterior distribution is proportional to
\begin{align}
    P(\vec p|D, M)
    &\sim
    \exp\left\{
        - \frac{1}{2}
        \chi^2_D(\vec p)
        - \frac{1}{2}
        \chi^2_M(\vec p)
    \right\}
    =
    \exp\left\{
        - \frac{1}{2}
        \chi^2_{DM}(\vec p)
    \right\}
    \, , &
    \chi^2_M(\vec p)
    &=
    \left[\vec p - \vec p_0\right]^T \cdot
    \Sigma_{p_0}^{-1}
    \left[\vec p - \vec p_0\right]\, .
\end{align}
In short, the \texttt{lsqfit} module approximates the posterior by expressing the function $\chi^2_{DM}(\vec p)$ up to second order in $\vec p$ at the point $\bar{\vec p}$ where the first derivative vanishes (stationary or almost always minimal point)
\begin{align}
    \chi^2_{DM}(\vec p)
    & \approx
    \chi^2_{DM}(\bar{\vec p})
    +
    \left[\vec p - \bar{\vec p}\right]^T
    \Sigma_{DM}^{-1}(\bar{\vec p})
    \left[\vec p - \bar{\vec p}\right]^T
    \, , & &
    \left.\frac{\partial \chi^2_{DM}(\vec p)}{\partial p_\alpha}\right|_{\vec p = \bar{\vec p}} = 0 \, \quad \forall_\alpha \, .
\end{align}
In this approximation, the posterior is again a multivariate normal distribution of mean $\bar{\vec p}$ (same as maximal likelihood estimation) with covariance $\Sigma_{DM}(\bar{\vec p})$.
The \texttt{nonlinear\_fit} method numerically computes the vector which minimizes the posterior $\bar{\vec p}$ (fitting) and analytically computes and evaluates the covariance matrix $\Sigma_{DM}(\bar{\vec p})$ at this point.
The appendix A of \cite{Bouchard:2014ypa} describes how $\Sigma_{DM}(\bar{\vec p})$ is estimated using derivatives of the residuals with respect of prior parameters.
%
% \subsubsection{Example}
%
% Suppose the fit function is a linear function $f(x_i, \vec p) = p^{(0)} + p^{(1)} x_i$.
% In this case, the normal approximation of the posterior is exact
% \begin{align}
%     \chi^2_{DM}(\vec p)
%     & =
%     \sum_{i,j=1}^N
%     \left[y_i - p^{(0)} - p^{(1)} x_i \right]
%     \left(\Sigma_y^{-1}\right)_{ij}
%     \left[y_j  - p^{(0)} - p^{(1)} x_j\right]
%     +
%     \left[\begin{pmatrix} p^{(0)} \\ p^{(1)} \end{pmatrix} - \vec p_0 \right]^T
%     \left(\Sigma_p^{-1}\right)_{ij}
%     \left[\begin{pmatrix} p^{(0)} \\ p^{(1)} \end{pmatrix} - \vec p_0 \right]
%     \\
%     & =
%     \left[
%         \sum_{i,j=1}^N y_i \left(\Sigma_y^{-1}\right)_{ij} y_j
%         + \vec p_0^T
%         \left(\Sigma_p^{-1}\right)
%         \vec p_0
%     \right]
%     +
%     \left[\begin{pmatrix} p^{(0)} \\ p^{(1)} \end{pmatrix} - \bar{\vec p} \right]
%     \left(\Sigma_{DM}^{-1}\right)_{ij}
%     \left[\begin{pmatrix} p^{(0)} \\ p^{(1)} \end{pmatrix} - \bar{\vec p} \right]^T
%     \, ,
% \end{align}
% with
% \begin{align}
%     \bar{\vec p} &= \ldots
%     \\
%     \Sigma_{DM}^{-1} &= \ldots
% \end{align}

%==============================================================================
% End Content
%==============================================================================
\bibliography{notes.bib}{}
\bibliographystyle{plainurl}

\batchmode
\end{document}


\section{Synthesis}
\subsection{Resource Graph}
A resource graph is a graph $G_R = (V_R, E_R), V_R = V_S \cup V_T$, where
$V_S$ are the nodes from the sequence graph and $V_T$ are the different resource
types. It is bipartite, $E_R = V_S \times V_T, (v_s, v_t) \in E_R$ means $v_s$
can be executed on resource type $v_t$. Additionally there is a cost function
$c: V_T \to \mathbb{Z}$ and a execution time function $w: E_R \to \mathbb{Z}^{\geq 0}$

Furthermore $\alpha(v_t)$ denotes the number of available instances of resource
type $v_t$, $\beta(v_s)$ denotes on which resource type $v_s$ is running, and
$\gamma(v_s)$ denotes on which instance of this resource type it is running.

\subsection{Scheduling}
A schedule is a function $\tau: V_S \to \mathbb{Z}^{>0}$ that determines the
starting times of operations. It is feasible if
\begin{equation*}
	\forall (v_i, v_j) \in E_S . \tau(v_j) - \tau(v_i) \geq w(v_i)
\end{equation*}
where $w(v_i) = w(v_i, \beta(v_i))$ denotes the execution time of $v_i$.

The latency $L$ is $L = \tau(v_n) - \tau(v_0)$, the differnce between the
starting times.

\subsubsection{ASAP}
\begin{lstlisting}[escapeinside={(*}{*)}]
ASAP((*$V_S$*), (*$E_S$*), w) {
	\tau((*$v_0$*)) = 1
	do {
		(*$v_i \leftarrow \set{v \in V_S | \text{v's predecessors are planned}}$*);
		(*$\tau(v_i) = \max\set{\tau(v_j) + w(v_j) | (v_j, v_i) \in E_S}$*);
	} while (unplanned operations exist);
}
\end{lstlisting}

\subsection{ALAP}
\begin{lstlisting}[escapeinside={(*}{*)}]
ALAP((*$V_S$*), (*$E_S$*), w, (*$L_{max}$*)) {
	(*$\tau(v_n)$*) = (*$L_{max}$*) + 1;
	do {
		(*$v_i \leftarrow \set{v \in V_S | \text{v's successors are planned}}$*);
		(*$\tau(v_i) = \min\set{\tau(v_j) | (v_i, v_j) \in E_S} - w(v_i)$*);
	} while (unplanned operations exist);
}
\end{lstlisting}

\subsection{Bellman Ford}
One might add additional constraints as weighted edges in the sequence graph,
which can be resolved using Bellman Ford to find the single source longest path.
Set the weight of the normal edges to their execution times.

\begin{lstlisting}[escapeinside={(*}{*)}]
BELLMAN-FORD((*$V_S$*), (*$E_S$*), W) {
	(*$\tau_0^0$*) = 0;
	for (int i = 0; i < (*$\abs{V_S}$*); i++) (*$\tau_i^0 = w(v_0, v_i)$*);
	for (int j = 0; i < (*$\abs{V_S}$*); i++) {
		for (int i = 0; i < (*$\abs{V_S}$*); i++) {
			(*$\tau_i^{j+1}$*) = (*$\max\set{\tau_i^j, \tau_k^j + w(v_k, v_i) | k \neq i}$*);
		}
		if ((*$\forall i \,.\, \tau_i^{j+1} = \tau_i^j$*)) return true; // success
	}
	return false; // failed
}
\end{lstlisting}

\subsubsection{Example constraints}
\begin{tikzpicture} [node distance=1.5cm]
	\tikzset{nop/.append style={minimum width=5mm}}
	\tikzset{inner/.append style={minimum width=5mm}}
	\node[nop]		(0)						{0};
	\node[inner]	(1) [below left of=0]	{1};
	\node[inner]	(2) [below of=1]		{2};
	\node[inner]	(3) [below right of=0]	{3};
	\node[inner]	(4)	[below of=3]		{4};
	\node[nop]		(5) [below right of=2]	{5};

	\path
		(0) edge[dashed]		node[above]	{0}		(1)
		(0) edge[dashed]		node[above]	{0}		(3)
		(1) edge[->]			node[right]	{2}		(2)
		(1) edge[->]			node[above]	{2}		(4)
		(3) edge[->]			node[right]	{2}		(4)
		(2) edge[dashed]		node[above]	{1}		(5)
		(4) edge[dashed]		node[above]	{1}		(5)
		;
\end{tikzpicture}

We add the following constraints
\begin{itemize}
	\item Between 1 and 2 there are at most 3 time units
	\item Between 0 and 4 there are at least 4 time units
\end{itemize}

\begin{tikzpicture} [node distance=1.5cm]
	\tikzset{nop/.append style={minimum width=5mm}}
	\tikzset{inner/.append style={minimum width=5mm}}
	\node[nop]		(0)						{0};
	\node[inner]	(1) [below left of=0]	{1};
	\node[inner]	(2) [below of=1]		{2};
	\node[inner]	(3) [below right of=0]	{3};
	\node[inner]	(4)	[below of=3]		{4};
	\node[nop]		(5) [below right of=2]	{5};

	\path
		(0) edge[dashed]		node[above]	{0}		(1)
		(0) edge[dashed]		node[above]	{0}		(3)
		(1) edge[->]			node[right]	{2}		(2)
		(1) edge[->]			node[above]	{2}		(4)
		(3) edge[->]			node[right]	{2}		(4)
		(2) edge[dashed]		node[above]	{1}		(5)
		(4) edge[dashed]		node[above]	{1}		(5)
		(2) edge[bend left, ->]	node[left]	{-3}	(1)
		(0) edge[->]			node[left]	{4}		(4)
		;
\end{tikzpicture}

\subsection{List Scheduling}
\begin{lstlisting}[escapeinside={(*}{*)}]
LIST((*$V_S$*), (*$E_S$*), (*$V_R$*), (*$E_R$*), (*$\alpha$*), (*$\beta$*), priorities) {
	(*$V_T$*) = (*$V_R - V_S$*);
	t = 1;
	do {
		foreach((*$v_k \in V_T$*)) {
			(*$U_k$*) = candidates to be scheduled;
			(*$T_k$*) = running operations;
			(*$S_k$*) = subset of (*$U_k$*) with maximal priority and (*$\abs{S_k} + \abs{T_k} \leq \alpha(v_k)$*);
			foreach((*$v_i \in S_k$*)) { (*$\tau(v_i)$*) = t; }
		}
		t = t + 1;
	} while ((*$v_n$*) unplanned);
}
\end{lstlisting}

\subsection{Integer Linear Programming}
To get optimal results one can use ILP. First, for each $v_i \in V_S$ we have
to determine $l_i$ and $h_i$, the earliest and latest starting time
respectively, using ASAP and ALAP with a suitable $L_{max}$. $x_{i,t} = 1 \iff$
operation $v_i$ starts at time $t$.
\begin{align*}
	\min \tau(v_n) - \tau(v_0) &\qquad \text{subject to} \\
	\forall v_i \in V_S \forall l_i \leq t \leq h_i &\,.\,x_{i,t} \in \set{0, 1} \\
	\forall v_i \in V_S &\,.\, \sum_{t = l_i}^{h_i} x_{i,t} = 1 \\
	\forall v_i \in V_S &\,.\, \sum_{t = l_i}^{h_i} t \cdot x_{i,t} = \tau(v_i) \\
	\forall (v_i, v_j) \in E_S &\,.\, \tau(v_j) - \tau(v_i) \geq w(v_i) \\
	\forall v_k \in V_T \forall 1 \leq t \leq \max_i\set{h_i} &\,.\, \sum_{\forall i. (v_i, v_k) \in E_R} \sum_{p' = \max(0, t - h_i)}^{\min(w(v_i) - 1, t - l_i)} x_{i, t - p'} \leq \alpha(v_k)
\end{align*}

\subsubsection{Modifications}
To adapt the ILP to iterative algorithms (marked graphs, pipelining), replace
\begin{align*}
	\forall (v_i, v_j) \in E_S &\,.\, \tau(v_j) - \tau(v_i) \geq w(v_i) \\
	\forall v_k \in V_T \forall 1 \leq t \leq \max_i\set{h_i} &\,.\, \sum_{\forall i. (v_i, v_k) \in E_R} \sum_{p' = \max(0, t - h_i)}^{\min(w(v_i) - 1, t - l_i)} x_{i, t - p'} \leq \alpha(v_k)
\end{align*}
by
\begin{align*}
	\forall (v_i, v_j) \in E_S &\,.\, \tau(v_j) - \tau(v_i) \geq w(v_i) - d_{i,j} \cdot P\\
	\forall v_k \in V_T \forall 1 \leq t \leq \max_i\set{h_i} &\,.\, \sum_{\forall i. (v_i, v_k) \in E_R}
	\sum_{p' = 0}^{w(v_i) - 1} \sum_{\forall p \,.\, l_i \leq t - p' + p \cdot P \leq h_i} x_{i, t - p' + p \cdot P} \leq \alpha(v_k)
\end{align*}

and

where $d_{i,j}$ is the amount of tokens on edge $(i,j)$.

\subsection{DVS ILP}
\begin{align*}
	\min \sum_{k \in K} \sum_{v_i \in V_S} y_{ik} \cdot e_k(v_i) &\qquad\text{subject to} \\
	\forall v_i \in V_S, k \in K &\,.\, y_{ik} \in \set{0,1} \\
	\forall v_i \in V_S &\,.\, \sum_{k \in K} y_{ik} = 1 \\
	\forall (v_i, v_j) \in E_S &\,.\, \tau(v_j) - \tau(v_i) \geq \sum_{k \in K} y_{ik} \cdot w_k(v_i) \\
	\forall v_i \in V_S & \,.\, \tau(v_i) + \sum_{k \in K} y_{ik} \cdot w_k(v_i) \leq d(v_i)
\end{align*}
where $K$ is the set of voltage levels and there are no resource constraints.


\chapter{confcheck} \label{confcheck}

\section{Introduction}

The \texttt{confcheck} command perform a \emph{confluence analysis} on the LPE in order to determine which \istep{} summands are \emph{confluent}
A confluent \istep{} summand is an \istep{} summand with the property that the possible behavior of a system is the same up to branching bisimulation before and after the application of that summand.

The \texttt{confcheck} command will rename confluent \istep{}s to \cistep{}s.
Afterwards, the \texttt{confelm} command can be used next in order to prioritize \cistep{}s, which may reduce the state space.

\section{Algorithm}

Consider all possible summands pairs, referencing the elements of the summands conform \ref{summandelements}.
Select all pairs $(s_\alpha, s_\beta)$ of which $C_\alpha$ equals \istep{}.

Since it cannot be assumed that
\begin{align*}
\{ x_1(1), \cdots{} x_1(m_1) \} \cap \{ x_2(1), \cdots{} x_2(m_2) \} = \emptyset{}
\end{align*}

a substitution $X$ is introduced such that
\begin{align*}
X = [ x_2(j) \rightarrow q(x_2(j)) \;|\; 1 \leq j \leq m_2 ]
\end{align*}

where $q(x)$ is a surjective function that yields fresh variables.

We also define
\begin{align*}
V_{1} &= [p_j \rightarrow v_1(p_j) \;|\; 1 \leq j \leq k] \\
V_{2} &= [p_j \rightarrow v_2(p_j)[X] \;|\; 1 \leq j \leq k]
\end{align*}

Using the definitions above, a particular \istep{} summand $s_1$ is confluent if the following expression is a tautology for all pairs $(s_1, s_2)$ such that $s_2 \neq s_1$:
\begin{align*}
g_1 \land g_2[X] \rightarrow g_1[V_2] \land g_2[X][V_1] \land \bigwedge\limits_{j=1}^{k} p_j[V_2][V_1] = p_j[V_1][V_2]
\end{align*}

Note that this approach does \emph{not} yield all confluent \istep{} summands (it is an under-approximation).

\section{Example}

Consider the following example:

\begin{lstlisting}
//Process definition:
PROCDEF example[A :: Int](x, y :: Int)
  = A ? i [[x<=9 /\ x==i]] >-> example[A](x+1, y)
  + ISTEP [[y<=9]] >-> example[A](x, y+1)
  ;

//Initialization:
example[A](0, 0);
\end{lstlisting}

Let $s_1$ be the first and $s_2$ the second summand.
Is $s_2$ confluent?

First, since $\{ x_1(1), \cdots{} x_1(m_1) \} \cap \{ x_2(1), \cdots{} x_2(m_2) \} = \emptyset{}$, $X$ can be ignored.

Second, if
\begin{align*}
g_1 \land g_2 \Leftrightarrow (x \leq 9 \land x=i) \land (y \leq 9)
\end{align*}

holds, then
\begin{align*}
g_1[V_2] \land g_2[V_1] &\Leftrightarrow (x \leq 9 \land x=i)[ y \rightarrow y+1 ] \land (y \leq 9)[ x \rightarrow y+1 ] \\
&\Leftrightarrow (x \leq 9 \land x=i) \land (y \leq 9)
\end{align*}

holds as well.

Third, it is the case that
\begin{align*}
x[V_1][V_2] = x+1 = x[V_2][V_1] \\
y[V_1][V_2] = y+1 = y[V_2][V_1]
\end{align*}

Therefore the confluence condition holds, which means that $s_2$ is confluent.

To store the new information about the second summand, the channel is renamed to \cistep{}:

\begin{lstlisting}
//Process definition:
PROCDEF example[A :: Int](x, y :: Int)
  = A ? i [[x<=9 /\ x==i]] >-> example[A](x+1, y)
  + CISTEP [[y<=9]] >-> example[A](x, y+1)
  ;

//Initialization:
example[A](0, 0);
\end{lstlisting}