\documentclass{tufte-handout}
\usepackage{amsmath}
\usepackage[utf8]{inputenc}
\usepackage{mathpazo}
\usepackage{graphicx}
\usepackage{listings}

\usepackage{tikz}
\usetikzlibrary{chains}
\usetikzlibrary{decorations}
\input{vc.tex}

\title{Four-Sum}
\author{Thore Husfeldt and Riko Jacob}
\date{\GITAuthorDate, rev. \GITAbrHash}
\begin{document}
\maketitle
\begin{abstract}
  Implement a very simple algorithm in two different languages, run some experiments.
  Improve the algorithm.
\end{abstract}

 \section{Description}

 Given a list of $N$ (long) integers, detect if there are four of them that sum to $0$.

\begin{description}
  \item[Exhaustive search.]
The simplest solution is to use exhaustive search using four nested for loops.
Go ahead and implement that in the language that you’re comfortable in, Python or Java.
There is a code skeleton called \texttt{sol/Simple.java} or \texttt{sol/simple.py} that you must use.
The program expects a list of $N$ integers on standard input, following the number $N$ itself.
The directory \texttt{data} contains several of such lists for various $N$.

The program must write \texttt{True} to standard output if it finds a solution, and \texttt{False} otherwise;
your program may write other output to standard error (\texttt{stderr}).
This is the basis of the tests on codeJudge, and all your programs must at least pass all of \texttt{TestsTiny} and \texttt{TestsSmall}.

Make it work; and determine the running time on your machine on the input files.
There are a handful of inputs for each size.
Report the fastest observed time, the slowest, and the average.
(Try to be half-way serious about this--- \emph{e.g.}, restart the computer before the experiment, so as to make sure no spurious processes are running in the background, don’t have a World of Warcraft-server or Bitcoin mining programme running at the same time.)
You probably won’t be able to handle the larger inputs---don’t report experiments that take more than a few minutes.
Plot the result using both a standard and a log-log plot.

\item[Exhaustive search in a different language.]
  Do the same as above, but in the other language.
  This may involve installing compiler or a run-time environment, figuring out where the semicolons or brackets go (or don’t go) and other annoyances.
  Compare the running times of the two implementations of exhaustive search in the different languages.
  This is the only task you are to implement in both languages.

\item[Faster.]
  Use the idea of the fast three-sum algorithm in [SW, Chapter 1.4] \texttt{
  ThreeSumFast} to write a faster algorithm for the four-sum problem.
  Your running time must be close to cubic time, proportional to $N^3\operatorname{lg} N$.
  Call it \texttt{Faster.java}  or \texttt{faster.py}.
  Redo the experiments and report them.

  \emph{Optional} There is a cute, simple, and very satisfying way to solve the problem in time proportional to $N^2\operatorname{lg} N$ instead.
If you can figure that trick out, implement it and hand in that code instead.
\end{description}

\section{Tips and comments}

\begin{description}
  \item[Java types.] The integers in the input files are very large, and will not fit into the Java datatypes \texttt{int} or \texttt{Integer}.
    Use \texttt{Long} instead.
    Do not worry about overflow.
  \item[Python dialect.] You can use Python 2 (which may be already installed on your computer) or Python 3 (remember that the library of the course is written in python3).
  \item[Course libraries.]
    In this exercise (and this exercise \emph{only}) you are \emph{not} required to use the course libraries, say for input/output.
    This is because it is too much overhead to ask students in language $x$ to install the libraries for language $y$.
    Therefore, this particular exercise is very bare-bones.
    You are welcome to use the course library if you want to.
  \item[Measuring time.]
    On a unix system, the \texttt{time} command gives sufficiently good performance estimates, and is better than sitting with a stopwatch.
    Both Java and Python come with better solutions; the course libraries offer a \texttt{Stopwatch} class, and Python has a handy \texttt{time} module. You are free to use either.
    On some modern Java systems, the compiler performs very funky optimisations behind your back, giving wildy divergent measurements for the same code on the same machine.
    You can handle that by running time code a few times before measuring it.
    It’s a black art.
  \item[Experiment design.]
    If you like doing repetitive work, you  can run your experiments individually by starting dozens of processes from the command line (or, worse, an IDE) and writing the results into a spreadsheet to compute averages.
    However, such an approach quickly becomes both tedius and error-prone.

    Instead, construct another program that performs all the experiments for you, and writes the results, including a time stamp of the experiment, into another file.
    There are many schools of thought about how to write that program; you can make it part of your original solution, or you can use a separate programme in a dedicated scripting language.
    One useful program (or \texttt{builtin} in \texttt{bash}) is \texttt{time}, a simple usage would be
    \begin{lstlisting}
      time java Simple < ../data/ints-200-2.txt
    \end{lstlisting}
    Because of the setup on codeJudge, our programs have debugging output on \texttt{stderr}, and there is the need to ignore this when collecting experimental results into a file.
    The following is an example how this could look like (note that the trailing \verb+\+ continues the command on the next line):
    \lstinputlisting{../src/simpleExp.sh}

    More advanced solutions (maybe in other languages) could compute averages, or even draw the plots for you.
    One example (that turned out to be somewhat more complicated) can be found in \texttt{../src/experiment.py}.
\end{description}

\section{Learning outcomes}

Besides some important secondary skills (input/output, exposing yourself to an unfamiliar programming language, basic experimentation, drawing graphs, computing averages), the main points of this exercise are
\begin{itemize}
  \item It does \emph{not} make a big difference which language you use.
  \item It \emph{does} make a big difference which algorithm you use.
  \item The slowest observed performance is well characterised by the predicted worst-case running time.
  \item The fastest observed performance contains very little information about general behaviour.
\end{itemize}

\end{document}


\documentclass[11pt]{article}
\usepackage{enumitem}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{float}

\graphicspath{{images/}}

\begin{document}
\title{MAT1830 --- Assignment 7}
\author{Dylan Pinn --- 24160547}
\maketitle

\section*{Question 1}

Let $R$ be the relation on $\mathbb{Z} \times \mathbb{Z}$ defined by $(w, x)R(y,
z)$ if any only if $w + x \leq y + z$.

\noindent
Let $S$ be the relation on $\mathbb{Z} \times \mathbb{Z}$ defined by $(w, x)S(y,
z)$ if and only if $w \leq y$ and $x \leq z$

\begin{enumerate}[label= (\alph*)]
  \item Is $R$ antisymmetric?

  Yes. Because the relation is defined across integers, if we make $w + x$ some
    integer $a$ and $y + z$ soem integer $b$. Then we can show that for any $a,b
    \in \mathbb{Z}$ we have that $a \leq b$ or $b \leq a \Rightarrow a = b$.

  \item Is $S$ antisymmetric?

    No. Because we can show that for $(1, 4)\not S(2, 2)$ and $(2,2) \not
    S(1,4)$

\end{enumerate}
\break{}

\section*{Question 2}

You know the following facts about a relation $T$:

\begin{itemize}
  \item $T$ is defined on the set $\{a, b, c, d, e \}$,
  \item $aTc, bTc, cTd, eTb$ and $e\not{T}a$,
  \item $T$ is a partial order relation.
\end{itemize}

Draw a Hasse diagram for every possible relation that $T$ could be and, for
each, write down whether it is a total order relation.

\begin{center}
    \includegraphics{A7}
\end{center}

No, not total order relation.

\break{}
\section*{Question 3}

Find expressions for each of the following

\begin{enumerate}[label= (\alph*)]
  \item The number of strings of 7 lower case letters (a-z) that do not contain
    any letter twice or more.

    As we need to do unordered selections without repetition it will be:
  \[
    = \binom{n}{r} \\
    = \binom{26}{7}
  \]

  \item The number of binary strings of length 50 that contain at most three 1s.

    As we want to get all of these that contain at most three it will be:

    \[
      =\binom{50}{0} + \binom{50}{1} + \binom{50}{2} + \binom{50}{3}
    \]

  \item The number of ternary strings of length 10 (strings of 0s, 1s and 2s)
    containing exactly one 1 and exactly four 2s.

    We want to find the strings such that 1222200000. As order is not important
    it will be and we don't want repetition. It is:

    \[
      =\binom{10}{5}
    \]

\end{enumerate}

\break{}
\section*{Question 4}

A small school with 14 students has a basketball team with 5 players, a chess
team with 4 players, a netball team with 7 players, and a soccer team with 11
players.

\begin{itemize}
  \item Each student is on at least one of these teams and no student is on all
    four teams.
  \item Only one student, Hawa, is on three of the teams; she plays chess,
    netball and soccer.
  \item There is exactly 1 student who plays basketball and chess.
  \item There is exactly 1 student who plays basketball and netball.
  \item Hawa is the only student who plays chess and netball.
  \item There are exactly 3 students (including Hawa) who play chess and soccer.
  \item There are exactly 5 students (including Hawa) who play netball and
    soccer.
\end{itemize}

How many students play basketball and soccer?

\begin{align*}
  |B \cup C \cup N \cup S| &= |B| + |C| + |N| + |S| \\
  &- |B \cap C| - |B \cap N| - |B \cap S| - |C \cap N| - |C \cap S| - |N \cap
  S| \\
  &+|B \cap C \cap N| + |B \cap C \cap S| + |B \cap N \cap S| + |C \cap N \cap
  S| \\
  &- |B \cap C \cap N \cap S| \\
  14 &= 5 + 4 + 7 + 11 - 1 - 1 - x - 1 - 3 - 5 + 1 + 0 + 0 + 0 + 0 - 0 \\
  14 &= 17 - x \\
  -3 &= -x \\
  x &= 3
\end{align*}

Therefore there are three students how play soccer and basketball.

\end{document}


\documentclass{beamer}

\mode<presentation>
{
  \usetheme{Hawke}
  \setbeamercovered{transparent}
}

\usepackage[english]{babel}
\usepackage[latin1]{inputenc}
\usepackage{times}
\usepackage[T1]{fontenc}
\usepackage{multimedia}

%%%%%%
% My Commands
%%%%%%

\newcommand{\bb}{{\boldsymbol{b}}}
\newcommand{\bx}{{\boldsymbol{x}}}
\newcommand{\by}{{\boldsymbol{y}}}
\newcommand{\bfm}[1]{{\boldsymbol{#1}}}
\newcommand{\pda}[2]{\frac{\partial{#1}}{\partial{#2}}}
\newcommand{\pdb}[2]{\frac{\partial^2{#1}}{\partial{#2}^2}}
\newcommand{\pdc}[3]{\frac{\partial^2{#1}}{\partial{#2}\partial{#3}}}

%%%%

\title[Lecture 28] % (optional, use only with long paper titles)
{Lecture 28 - Von Neumann stability 2}

\author[I. Hawke] % (optional, use only with lots of authors)
{I.~Hawke}
\institute[University of Southampton] % (optional, but mostly needed)
{
  School of Mathematics, \\
  University of Southampton, UK
}

\date[Semester 1] % (optional, should be abbreviation of conference name)
{MATH3018/6141, Semester 1}

\subject{Numerical methods}

\pgfdeclareimage[height=0.5cm]{university-logo}{mathematics_7469}
\logo{\pgfuseimage{university-logo}}

\AtBeginSection[]
{
  \begin{frame}<beamer>
    \frametitle{Outline}
    \tableofcontents[currentsection]
  \end{frame}
}

\begin{document}

\begin{frame}
  \titlepage
\end{frame}

\section{Stability}

\subsection{Evidence}

\begin{frame}
  \frametitle{Partial differential equations}

  Partial differential equations (PDEs) involve derivatives of
  functions of more than one variable, say $u(x, y)$ or $y(t,
  x)$. Hence more complex behaviour and more interesting
  physics. \pause

  \vspace{1ex}

  Only look at linear problems.  Also only consider finite difference
  methods: simple to analyse but not always competitive. \pause

  \vspace{1ex}

  Previously looked at simple finite difference algorithms applied to
  heat and advection equations. At present have no explanation for
  which worked and which did not; for that we need stability theory.

\end{frame}

\begin{frame}
  \frametitle{The results}

  \begin{center}
    \begin{tabular}{c|c c}
      Algorithm & Heat equation & Advection equation \\ \hline
      FTCS & Stable $s < 1/2$ & Unstable \\
      BTCS & Stable $\forall s$ & ?? \\
      FTBS & ?? & Stable $c < 1$
    \end{tabular}
  \end{center}

  Here $s = k \delta / h^2$ for the heat equation
  \begin{equation*}
    \partial_t y - k \partial_{x x} y = 0,
  \end{equation*}
  and $c = v \delta / h$ for the advection equation
  \begin{equation*}
    \partial_t y + v \partial_x y = 0.
  \end{equation*}

\end{frame}

\section{Von Neumann stability theory}

\subsection{Von Neumann stability theory}

\begin{frame}
  \frametitle{Determining stability}

  Von Neumann stability method for linear algorithms and equations:
  \begin{enumerate}
  \item<2-> Make ansatz that solutions of the finite difference
    equation have form
    \begin{equation*}
      y_{\ell}^k =  e^{\alpha j \ell h} q^k, \qquad (\, j = \sqrt{-1}, \alpha \in {\mathbb R}, q \in {\mathbb C} ).
    \end{equation*}
  \item<3-> Substitute this solution into the finite difference
    equation.
  \item<4-> Determine $q$ as a function of $\alpha$, the grid spacings
    $h,\delta$, and known numbers (such as the advection velocity
    $v$).
  \item<5-> Determine under what conditions we have stability, i.e.\ when
    \begin{equation*}
      |q| < 1 \quad \forall \alpha.
    \end{equation*}
  \end{enumerate}

\end{frame}

\section{Von Neumann stability in practise}

\subsection{Von Neumann stability in practise}

\begin{frame}
  \frametitle{BTCS for the heat equation}

  We look at BTCS in the form
  \begin{equation*}
    (1 + 2 s) y_i^{n+1} = y_i^n + s \left( y_{i+1}^{n+1} +
      y_{i-1}^{n+1} \right).
  \end{equation*} \pause
  Use ansatz $y_{\ell}^k =  \exp (\alpha j \ell h) q^k$
  and removing common factors:
  \begin{equation*}
    (1 + 2 s) q = 1 + s q \left( e^{\alpha j h} +  e^{-\alpha j h}
    \right).
  \end{equation*} \pause
  Rewrite this as
  \begin{align*}
    && q & = 1 + 2 s \left( e^{\alpha j h} +  e^{-\alpha j h} - 2 \right)
    q \\
    && & = 1 - 4 s q \sin^2 \left( \tfrac{\alpha h}{2} \right) \\
    & \implies & q & = \left( 1 + 4 s \sin^2 \left( \tfrac{\alpha
          h}{2} \right) \right)^{-1},
  \end{align*} \pause
  which is \emph{always} less than 1. Hence BTCS is unconditionally
  stable.

\end{frame}

\begin{frame}
  \frametitle{FTCS for the advection equation}

  Ignore difference in \emph{differential} equation; only need
  analysis at level of finite difference equation:
  \begin{equation*}
    y_i^{n+1} = y_i^n - \frac{c}{2} \left( y_{i+1}^n - y_{i-1}^n
    \right).
  \end{equation*}
  Use ansatz $y_{\ell}^k = \exp (\alpha j \ell h) q^k$ and remove
  common factors:
  \begin{align*}
    && q & = 1 - \frac{c}{2} \left( e^{\alpha j h} - e^{-\alpha j h}
    \right) \\
    & \implies & q & = 1 - j c \sin ( \alpha h ).
  \end{align*} \pause
  As this is complex the modulus is
  \begin{equation*}
    |q|^2 = 1 + c^2 \sin^2 ( \alpha h )
  \end{equation*}
  which is always greater than 1: method is unconditionally unstable.

\end{frame}

\begin{frame}
  \frametitle{FTBS for the advection equation}

  Finally for the upwind method, FTBS, we had the difference equation
  \begin{equation*}
    y_i^{n+1} = y_i^n - c \left( y_{i}^n - y_{i-1}^n \right).
  \end{equation*}
  Use ansatz $y_{\ell}^k =  \exp (\alpha j \ell h) q^k$
  and remove common factors:
  \begin{equation*}
    q = 1 - c \left( 1 - e^{-\alpha j h} \right).
  \end{equation*} \pause

  Describes circle radius $c$ in the complex plane, centre at $1 -
  c$. \pause If $c > 1$ there are finite values of $\alpha$ for which
  $|q| > 1$, leading to instability; if $c < 1$ then $q$ is at most 1,
  leading to stability.

\end{frame}

\section{Summary}

\subsection{Summary}

\begin{frame}
  \frametitle{Summary}

  \begin{itemize}
  \item Von Neumann stability analysis is the key tool for
    understanding the results seen ``experimentally'' in the last
    lecture.
  \item By looking at the growth or decay of individual frequency
    modes we can check the stability of the method; Lax's theorem then
    proves convergence.
  \item For complex equations and / or algorithms, it is often very
    difficult or impossible to apply even Von Neumann stability
    analysis.
  \item If an algorithm does not work in the linear case then it is
    highly unlikely to work for more complex problems!
  \end{itemize}

\end{frame}

\end{document}


\lab{Python}{F2PY}{F2PY}
\label{lab:f2py}
\objective{Learn how to use F2PY, especially for wrapping Fortran code involving arrays.}

Here we will explore another simple way to access external code from Python.
Various utilities exist to do this, and many are specialized for specific tasks.
F2PY is a utility that can be used to automatically wrap Fortran subroutines so that they are callable from Python.
It is included with NumPy and in most cases does not usually require heavy configuration for use.
The primary requirement for using F2PY is a working Fortran compiler.
If you have access to GFortran, the Fortran equivalent of gcc, it should work well.

\section*{Examples}

F2PY is very simple to use.
Many simple Fortran files can be compiled \emph{automatically} without any hand-written interfaces between languages.
Windows 64 bit users, if you encounter errors, see the Windows 64 bit install guide.
We will start by wrapping the Fortran function we wrapped as a part of Lab \ref{lab:cythonwrap}.
Here we will wrap the subroutine in unmodified form.
In Lab \ref{lab:cythonwrap} we modified the Fortran file to make it callable from C.
F2PY will take care of this for us so that we can wrap the subroutine directly.
Here is the file in its unmodified form:
\lstinputlisting[style=fromfile, language=Fortran]{ftridiag.f90}
We can wrap this file by running the following in our terminal:
\begin{lstlisting}[style=ShellInput]
python f2py.py -c ftridiag.f90 -m ftridiag
\end{lstlisting}
We can also tell F2PY to pass an optimization flag to the compiler using the \li{--opt} flag, as in:
\begin{lstlisting}[style=ShellInput]
python f2py.py -c ftridiag.f90 -m ftridiag --opt="-O3"
\end{lstlisting}
Depending on your system configuration, you may be able to call F2PY without expressly calling python, or without including its file extension (or both).

Running these commands will create a file called \li{ftridiag.pyd} on a Windows computer or \li{ftridiag.so} on a Linux or Mac based computer.
This file is a Python module that is linked against the compiled Fortran code.
All the details about array type, etc. have been infered by F2PY.
We can test the module the same way we tested the module we built using Cython.
\lstinputlisting[style=fromfile]{ftridiag_test.py}
For our example here we compiled the subroutine by itself instead of compiling an entire module.
If the function were a part of a Fortran module \li{tridiag} F2PY would have made it available to us as \li{ftridiag.tridiag.ftridiag}.

% Show example of wrapping FORTRAN using Cython.

% Introduce f2py, teach how to wrap FORTRAN automatically.

% Wrap a C function using f2py to teach about signature files.

% Wrap a LAPACK function (dgtsv?), have them do it too.

% Discuss how memory layout affects performance.

% Discuss programmer time value.


\section{Parton string model algorithm.}

\hspace{1.0em}
The parton string model algorithm can be considered as a set of
steps should be performed:
\begin{enumerate}
\item Create two vectors (projectile and target) of particles: assign initial
projectile and target particle types, their coordinates and momenta.
In the case of hadron-nucleus (or nucleus-nucleus) interaction
one should perform
target nucleus (or projectile and target nuclei) initial state simulation
 and sample impact parameter;
\item Sample
collision participants and separated them into diffractive and non-diffractive.
Store the total interaction four momentum of participants;
\item For non-diffractive inelastic collisions sample the number
of the soft longitudinal and hard kinky string can be produced.
\item Excite and reexcite colliding particles in the case of diffractive collisions and
create diffractive longitudinal strings, if particles are not participate in
further soft or hard collisions;
\item Perform longitudinal and kinky string excitations;
\item Perform string decay simulation;
\item Correct energies and momenta of produced particles, if it is needed.
\end{enumerate}


% Generated by GrindEQ Word-to-LaTeX 2008
% ========== UNREGISTERED! ========== Please register! ==========
% LaTeX/AMS-LaTeX

\documentclass[a4paper]{article}
\usepackage{anysize}
\marginsize{1cm}{1cm}{1cm}{1cm}

\usepackage{amssymb}
\usepackage{amsmath}
\usepackage[dvips]{graphicx}
\usepackage{listings}
\lstset{language=haskell}
\lstset{commentstyle=\textit}
\lstset{mathescape=true}
%\lstset{labelstep=1}
%\lstset{backgroundcolor=,framerulecolor=}
\lstset{backgroundcolor=,rulecolor=}
\linespread{2.0}

\begin{document}

%%% remove comment delimiter ('%') and select language if required
%\selectlanguage{spanish}

\title{Customizable Objects}

\textit{Giuseppe Maggiore}
\textit{Michele Bugliesi}
\textit{Universitá Ca'  Foscari -- Venice}

\section{Introduction}

 In this paper we present a technique that aims towards supporting many different programming paradigms and techniques with a clear and intuitive syntax (and no particular knowledge requirements) for the developer. We also wish to make it possible to easily switch between one paradigm and the other without modifying the client code: at the ``flipping of a switch'' the actual execution of a developer code could radically change to support different features \textit{without any modification whatsoever}. The kind of techniques we wish to support as a ``runtime backend'' for client programs is quite a large set; among these various techniques we discuss some of our favorites: transactional systems, concurrent/distributed systems and reactive programs. We absolutely wish to avoid creating a new language or to extend the syntax of an existing one; instead we try to push the expressivity o the highly flexible Haskell programming language so that it supports our desired constructs. As a starting point we build a framework for working with mutable objects; this system must (of course) be type-safe and also as generic as possible: for this reason we will use meta-programming (type predicates and type functions) a lot. Our object system heavily uses heterogeneous lists as a basis for a customized type-safe heap. We define references to values inside our heap to offer support for mutable operations such as assignment and in-place modification; our heap requires some notion of subtyping to allow references that expect a smaller heap to work wherever a larger heap is active. Whenever a reference represents a value based on our heterogeneous lists then we give a selection operator that returns a reference to a field inside the references heterogeneous list; thanks to this selection operator we now have built a framework for handling mutable records. We extend our mutable records with support for methods and method selection and invocation; since our methods are not invoked directly but through our customized selection operator we can control not only the automated passing of the \textit{this} value to each method as its first parameter, but also the propagation of effects into \textit{this }after the method invocation. Some work is dedicated to overloading the same selection operator so as to not having to distinguish between selecting a field and a method. We add support for inheritance so that the fields and methods of the inherited object can be accessed from the inheriting object in a fully transparent way, and we also add an explicit casting operator to convert a reference to an object to a reference to the \textit{base} (inherited) object.

 We pay attention to working on two distinct levels. On one hand we define all our constructs as very generic type predicates and type functions, thereby giving the general shape of our operators. In the most decoupled way possible we instance these functions and predicates with heterogeneous lists, a concrete representation for references, record labels, methods, etc. At the interface level we also define a way to write our computations as statements in the state monad. This allows a developer to write code that uses very general objects, without caring about the plumbing involved whenever the various assignment, selection, method invocation etc. operations occur; by instancing our generic operators multiple times we get the bonus of executing client code with a completely different runtime behavior than simple stateful objects. We will show a few examples of how we can do so and what kind of constructs we may implement. As a first example we will show how we could add three simple operators (\textit{beginT}, \textit{commitT} and \textit{discardT}) to enable transactional code. Then we will show how we could support reactive programming by modifying the behavior of the binding and selection operators, storing and passing on appropriate callbacks that signal the need for propagating changes in the operators to the results of computations. Finally we will discuss how we could make our objects distributed or concurrent, either with shared memory or outright message-passing: by storing a process ID or machine ID inside each reference we could track to which process the current reference belongs to so that whenever we try and access the value represented by such reference we can check whether or not this reference is local and if it is not we will use some synchronization/message-passing scheme to access it.

 Our last step will compare our approach to various, more specific approaches for supporting mutable objects, reactive computations, concurrency, etc. on top of pure functional languages.

 Note on syntax: we will use an ambiguous set of operators with some conventions. The ambiguity will always be clearly resolved by the context, since some operators will both be present at the type level and at the value level, but there is never any confusion as to whether we are manipulating types or values. Also, we will use a Haskell-like syntax:

\begin{enumerate}
\item  $x\ =\ v$ for binding

\item  $let\ x\ =\ v_1\ in\ \ v_2$ for binding

\item  $T=t$ for defining types

\item  $F\ :{\left(*\ \to \ \right)}^n*$ for giving the kind of a type function

\item  $F\ T_1\dots \ \ T_n=T_{n+1}$ for specifying the returned value of a type function with certain input parameters

\item  $P\ v_1\dots \ v_n$ for defining a type predicate

\item  $P_1\ v_{11}\dots \ v_{1m}\Rightarrow P_1\ v_{21}\dots v_{2m}$ for defining a type predicate that requires another type predicate

\item  $P\ T_1\dots \ T_n$ for instancing a type predicate
\end{enumerate}

 Rather than use the full Haskell syntax, we will try and adhere to a more standard syntax. For example, the fact that $x$ has type $T$ we will write

\begin{lstlisting}
x:T
\end{lstlisting}

Rather than the peculiar

\begin{lstlisting}
x::T
\end{lstlisting}

Used in Haskell programs. Also, when defining a type predicate and the operators it enables we will just say:

\begin{lstlisting}
P $v_1 \dots v_n$
$f_1$ : $T_1$
   $\vdots$
$f_m$ : $T_m$
\end{lstlisting}

Rather than the Haskell

\begin{lstlisting}
class P $v_1 \dots v_n$  where
  $f_1$ : $T_1$
    $\vdots$
  $f_m$ : $T_m$
\end{lstlisting}

\section{State Monad}

 We build a system for performing stateful computation. This kind of computations can be expressed very cleanly with the state monad, which essentially allows us to express the denotational semantics of our statements and which automatically concatenates the semantics of those statements that must be executed sequentially. An ulterior incentive towards using monads is that as language constructs monads have seen an impressive support in many languages, starting from Haskell and ranging to F\# and C\#; this is probably related to the expressive power of monads and to the way they allow extending a language cleanly and seamlessly, especially if supported by proper syntactic sugar. A statement that evaluates to a value $\alpha $ with a state $s$ in the state monad has type

\begin{lstlisting}
ST s $\alpha$ = s -> ($\alpha \times$s)
\end{lstlisting}

A monad is a triplet of:

\begin{enumerate}
\item  a type constructor $M$ that denotes the type of our monads

\item  a $unit$ operator with type $\alpha \to M\ \alpha $ that generates a monad encapsulating a value

\item  a $bind$ operator with type $M\ \alpha \to \left(\alpha \to M\ \beta \right)\to M\ \beta $ that ``concatenates'' two monads into one
\end{enumerate}

For the state monad we have:

\begin{lstlisting}[frame=tb,mathescape]{somecode}
M = ST s
unit x = ST($\lambda$s.(x,s))
bind (ST m) k=ST($\lambda$s.
		let (x,s')=m s in
		let (ST m')=k x in
		m' s')
\end{lstlisting}

We also add an evaluation operation for values of the state monad:

\begin{lstlisting}[frame=tb]
runST : ST s $\alpha$ -> s -> $\alpha$
runST m s = fst (m s)
\end{lstlisting}

We will use Haskell style syntactic sugar, so that using monads may feel more intuitive than using the explicit syntax; the kind of sugarization we use is based on two (purely cosmetic) simplifications, one for the $unit$ operator and the other for the $bind\ $operator:

\begin{lstlisting}[frame=tb]
return $\equiv$ unit
bind m ($\lambda$v. $\dots$) $\equiv$ do v $\leftarrow$ m
		      $\dots$
\end{lstlisting}

\section{Heap}

 We start by giving a definition of our notion of heap. The heap will be the basis for all our computations, in that it will provide the main, shared storage to which all statements and references in our programs refer to. We do not define our heap as a concrete type, but rather we give an ``interface'' (a Haskell type class) that specifies the operations that must be available on some generic type for it to be used in whichever context a heap is expected. Such an interface may also contain functions from types to types that specify some type transformations that our heap will need to offer. We define this predicate on a type variable $h$ as:

\begin{lstlisting}
Heap h
\end{lstlisting}

We start with type-level functions. A heap $h$ must support a type function that, given $h$ and a type $\alpha $ (some new item), returns a new heap; we will expect the returned heap to contain not only all the elements that the original heap $h$ contained, but also a new slot for storing a value of type $\alpha $:

\begin{lstlisting}
New h : *$\to$*
\end{lstlisting}

Thanks to this definition we can now proceed to giving the two fundamental operators on a heap: $new$ and $delete$, which respectively allow us to add an element to our heap and remove the last added element to our heap. The allocation and deallocation operators have the following types:

\begin{lstlisting}
new : h$\to\alpha\to$New h $\alpha$
del : New h $\alpha$$\to$h
\end{lstlisting}

\section{References}

 Our notion of heap is too poor for our purposes. We immediately extend our heap predicate so that instead of having ``just a heap'' (which would be relatively pointless), whenever we have a heap $h$ we also get a type $ref$ for expressing references to within our heap $h$. This way we strongly couple the idea of some state (the heap itself) and the idea of doing mutable computations on it (the references we will define shortly). References are represented as pointing to some value $\alpha $ inside some heap $h$; because of this a reference will be represented with a functor $ref$ that takes in two type parameters.

Our heap predicate now takes two type variables as parameters: the heap $h$ and the functor $ref$; the signature now becomes:

\begin{lstlisting}
Heap h ref
\end{lstlisting}

Followed by the various functions and operators that both $h$ and $ref$ must support. We give a small ``calculus of references'' that allows us to manipulate references by creating references from other references (selection), invoking methods from references (method invocation), and generating stateful statements from references. Statements that evaluate to a value of type $\alpha $ in a context with a heap $h$ will have the form (known from the state monad):

\begin{lstlisting}
ST h $\alpha$
\end{lstlisting}

Some of the operators on references we will not introduce right away; these operators will be presented after some more background is covered. Two simple and basic operators on references are the evaluation and assignment operators:

\begin{lstlisting}
eval : ref h $\alpha\to$ST h $\alpha$
\end{lstlisting}

Which evaluates a reference to a statement that will return the same value contained inside the reference when passed a concrete heap, and:

\begin{lstlisting}
(:=) : ref h $\alpha\to\alpha\to$ST h Unit
\end{lstlisting}

Which assigns a reference of type $\alpha $ a new value of type $\alpha $ and returns a statement that when evaluated will return a value of type $Unit$ ($()$) and the new heap where the assignment has been made.

 Another kind of assignment operator is the in-place modification operator:

\begin{lstlisting}
(*=) : ref h $\alpha\to(\alpha\to\alpha)\to$ST h Unit
\end{lstlisting}

which takes a reference of type $\alpha $, a function of type $(\alpha \to \alpha )$ and returns a statement which, when passed a concrete value of the heap $h$, will evaluate the initial value of the reference, apply the function and return the result of assigning it to the original reference. This operator is quite handy for implementing common operators such as the in-place increment by one of an integer $(++)$ or the in-place increment by some integer value $(+=)$ that are widely used in stateful programming languages such as C, Java or C\#. The alternative to having such an operator is to evaluating and then assigning a reference; indeed, in light of this consideration we may realize that we already could define the body of this operator in terms of the state monad:

\begin{lstlisting}[frame=tb,mathescape]{somecode}
r *= f =
	do v $\leftarrow$ eval r
	   r := (f v)
\end{lstlisting}

It is interesting to realize that this definition is indeed extremely abstract, and implies absolutely nothing in terms of what is happening exactly inside the various operators: whether this operation simply writes a mutable state, synchronizes access to some shared memory location or even sends and receives some network messages with some other instance of the same distributed process is indeed unknown by just looking at this piece of code.

 A very important operation on some reference $ref$ and some heap $h$ is the ``introduction-and-elimination'' operator that:

\begin{enumerate}
\item  Allocates an initial value of type $\alpha $ on our heap $h$

\item  Builds a reference to this value

\item  Performs some arbitrary computation on this newly create reference, returning a result of type $\beta $

\item  Cleans up the heap deleting the new value of type $\alpha $ before returning the $\beta \ $result
\end{enumerate}

 This operator is called $\left(\gg +\right)$, in order to recall the (similar) concept of binding in a monad. In fact, this operator will bind a new value of type $\alpha $ to a corresponding reference. The type of this operator is:

\begin{lstlisting}
(>>+) : $\alpha\to$(ref (New h $\alpha$) $\alpha\to$ST (New h $\alpha$) $\beta$)$\to$ST h $\beta$
\end{lstlisting}

We use the same representation that the state monad uses for values where our heap $h$ is the mutable state of the computation. It is also interesting to notice that this operator works in a way that is very similar to a ``scoping'' operator. To realize this, notice that the value of type $\alpha $ is accessed through a reference that is exclusively available within the confines of the function that is passed as the second parameter to $(\gg +)$. As an example, let us consider the following pseudo-code:

\begin{lstlisting}[frame=tb,mathescape,language=C++]{somecode}
A a = new A(...);
{
  B b = new B(...);
  {
    // do something with a and b
  }
}
\end{lstlisting}

Would be written, with our $(>>+)$ operator:

\begin{lstlisting}[frame=tb,mathescape]{somecode}
A(...) $>>+$ ($\lambda$a.
B(...) $>>+$ ($\lambda$b.
  -- do something with a and b
))
\end{lstlisting}

\section{First Examples:}

 Thanks to the system we have put together, we may already start writing some simple examples. These examples will only use the common operators that are required to exist for every heap $h$ with references $ref$.

 For the first example, let us say that we have a reference $i$ to an integer within some heap $h$ such that:

\begin{lstlisting}
i : ref h Int
\end{lstlisting}

We can write:

\begin{lstlisting}[frame=tb,mathescape]{somecode}
$ex_1$ =
	do v $\leftarrow$ eval i
	   i := v+2
	   v' $\leftarrow$ eval i
	   return v
\end{lstlisting}

The type of the code above is:

\begin{lstlisting}
Heap h ref $\Rightarrow$ $ex_1$ : ST h Int
\end{lstlisting}

%Why $ex_1$ has this type is maybe more obvious if we consider the de-sugarized version of the same code:
%\[ex^'_1=\underbrace{bind\underbrace{\left(eval\ i\right)}_{ST\ h\ Int}\underbrace{\left(\lambda \underbrace{v}_{Int}.%\underbrace{bind\underbrace{\left(i?v+1\right)}_{ST\ h\ Int}\underbrace{\left(\lambda \underbrace{{\rm \_}}_{Unit}.%\underbrace{bind\underbrace{\left(eval\ i\right)}_{ST\ h\ Int}\ \left(\lambda \underbrace{v^'}_{Unit}.\underbrace{return\ %v^'}_{ST\ h\ Int}\right)}_{ST\ h\ Int}\right)}_{Unit\to \ ST\ h\ Int}}_{ST\ h\ Int}\right)}_{Int\to ST\ h\ Int}}_{ST\ h\ Int}\]

The example above makes a quite limited use of our operators, but most importantly it assumes that a reference has already been created outside the example and is given. Let us show now how we can generate our local references in a self-contained example. The following example is more interesting since it shows the most reasonable pattern of introduction for references: we expect that a reference will be allocated with the $\gg +$ operator, used in its scope and then simply discarded, exactly as we would do in a ``typical'' imperative language. The example is:

\begin{lstlisting}[frame=tb,mathescape]{somecode}
$ex_2$ =
	do 10 $>>=$ ($\lambda$i.
	do "hello " $>>=$ ($\lambda$s.
	do s *= ($\lambda$x.x++"world")
	   v$\leftarrow$eval s
	   x$\leftarrow$eval i
	   return v ++ show x))
\end{lstlisting}

In this example we start by allocating an integer on the heap, then we allocate a string on the heap, and finally we manipulate the two references to these values on the heap before returning the concatenation of their values. Sadly, the example above does not compile without some modifications. We will see very shortly how we can write the example above so that at least it compiles correctly.

 The last example we see is another term which does not type correctly. This time however this behavior is quite desired:

\begin{lstlisting}[frame=tb,mathescape]{somecode}
$ex_{wrong}$ =
	do $i'$ $\leftarrow$ 10 $>>=$ ($\lambda$i.return i)
	   $i'$:=20
\end{lstlisting}

In this example we try to access a location on the heap that is not available, but luckily the last line cannot be typed since ${{\rm i}}^{{\rm '}}{\rm :}{\rm ref}{\rm \ }\left({\rm New}{\rm \ }{{\rm h}}_0{\rm \ }{\rm Int}\right){\rm Int}$ and ${{\rm i}}^{{\rm '}}?{\rm 20\ :}{\rm ST}{\rm \ }\left({\rm New}{\rm \ }{{\rm h}}_0{\rm \ }{\rm Int}\right){\rm \ }{\rm ()}$ but the expected value for the first type parameter is ${{\rm h}}_0\ne {\rm (New\ }{{\rm h}}_0{\rm \ Int)}$.

\end{document}

% == UNREGISTERED! == GrindEQ Word-to-LaTeX 2008 ==


\documentclass{article}
\usepackage[latin9]{inputenc}
\usepackage{geometry}
\geometry{verbose,tmargin=2cm,bmargin=2cm,lmargin=2cm,rmargin=2cm}
\usepackage{float}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{esint}
\usepackage[unicode=true,pdfusetitle,
 bookmarks=true,bookmarksnumbered=false,bookmarksopen=false,
 breaklinks=false,pdfborder={0 0 1},backref=section,colorlinks=false]
 {hyperref}
\usepackage{breakurl}

\makeatletter

\providecommand{\tabularnewline}{\\}

\@ifundefined{date}{}{\date{}}
\usepackage{subfigure}
\usepackage{cancel}

\makeatother

\begin{document}

\title{CSE397: Group Assignment \#3}
\author{Ben Crestel, Nick Malaya, Prabhakar Marepalli, Ajay Vadakkepatt}

\maketitle

\section*{Problem Statement}

\section{Selection of calibration}

Select what you consider to be the best calibration of the ball-drop
model from group problem 2. You may either use one of the calibrations
from problem 2, or revise your calibration based on any feedback you
received (your choice). Be sure to document what calibration you are
using (the priors, error models and likelihood formulation) and your
justification for selecting this particular formulation.

\subsubsection*{Solution}

For this project, we choose the parameters inferred from the data
of both the ball drops together (inferring same alpha and sigma, but
different $C_{d}'s$). This selection is one of the realistic cases
as $\alpha$ and $\sigma$ only depend on the measurement error (motion
of the iPad while recording) whereas it is quite reasonable to infer
different $C_{d}'s$ for different balls as $C_{d}$ is a function
of the surface roughness and diameter of the ball. Also, in contrast
with our solution procedure for Project 2, the parameter $H$ is not
inferred here. This change is from the feedback received for that
report. We outline the calibration procedure below:

\subsubsection*{Prior for $\alpha$}

The coefficient $\alpha$ defined above is said to be ``of order
$0.5$~m/s with uncertainty of the same order''. Since we could
not exclude negative values of $\alpha$ we can choose $\alpha$ to
be Gaussian with mean $0.5$~m/s and $95\%$~confidence interval
given by the interval of uncertainty. Here the uncertainty is said
to be ``of the same order'' as the mean. In order to be conservative,
we'll pick our $67\%$~confidence interval to be $[-1,1]$~m/s.
\[
\alpha\sim\mathcal{N}(0.5,1).
\]

\subsubsection*{Prior for $\sigma$}

For $\sigma$, we know it has to be non-negative. We can therefore
assume that $\sigma$ is uniformly distributed over the interval $[0,0.2]$~m.
\[
\sigma\sim\mathcal{U}(0,0.2).
\]

\subsubsection*{Prior on $C_{d}$, the coefficient of drag}

The coefficient of drag is a non-dimensionalization of the force that
results from a scaling analysis.
\[
\frac{F_{D}}{\frac{1}{2}\rho V^{2}A}=C_{D}(Re).
\]
 Thus, the coefficient depends only on the Reynolds number of the
flow. Let's estimate the Reynolds number to give us a sense of the
range we believe may be possible in this particular experiment.

The Reynolds number is defined as,
\[
Re=\frac{VL}{\nu}
\]
 Let's estimate the characteristic velocity (V), and length (L) as
well as the viscosity, $\nu$. An upper bound for the velocity would
be from the calculating the velocity in the absence of drag. Here,
$V=gt_{\text{final}}$, and with $t_{\text{final}}\approx3$ seconds,
our V is $\approx29.4$ m/s. The characteristic length is the ball
diameter, which for the basketball is 23.33 cm. Finally, the kinematic
viscosity of air is $15.68*10^{-6}\text{m}^{2}/s$. Thus, our $\text{Re}\approx437,000$,
which means the flow is certainly turbulence. Furthermore, we expect
roughness effects to become significant around $Re\approx10^{5}$.
Given that we do not know, with confidence, if the balls are rough
or smooth, our prior must be sufficiently broad to capture the possibility
of either condition.

Looking at a NASA chart for the measured drag coefficient on a sphere,
at low reynolds numbers (when we release the ball) we expect that
our drag coefficient might be as large as 2. Should the sphere be
smooth, the drag coefficient could be as small as 0.1 at the end.
We also know that our prior should not be capable of negative values,
that would be aphysical. For these reasons, we decided to define $C_{d}$
as
\begin{align*}
C_{d}\sim\mathcal{U}(0,2).
\end{align*}

\subsubsection*{Likelihood}

We will infer (at most) for the following parameters: $\alpha,\sigma,C_{d}$.
The likelihood gives us the probability that the results from our
simulation~$h_{ode}$ match the data~$h_{d}$ we have.
\[
\pi\left(h_{ode}=h_{d}\,|\,\alpha,\sigma,C_{d}\right).
\]
 In the hypothetical case we'd know all parameters exactly ($\alpha,\sigma,C_{d}$),
the data would be connected to the results of our simulation through
the following relation
\begin{align*}
h_{d}=\left(h^{\text{ode}}+\varepsilon\right),
\end{align*}
We already detailed the distribution of the random variable $\varepsilon$.
It is Gaussian with mean and covariance given by the time measurements
of each experiment and random variables $\alpha$ and $\sigma$ (which
are assumed to be sampled when evaluating the likelihood). Calling
$\mu$ and $R$ the mean and covariance matrix corresponding to the
right format of $h$ (ie, stacking information from all 4 experiments
'on top of each other'), we get the following distribution for the
likelihood,
\begin{align*}
\pi(h_{ode}=h_{d}\,|\,\alpha,\sigma,C_{d})=\frac{1}{\sqrt{(2\pi)^{n}|R|}}\exp\left\{ -\frac{1}{2}\left(h_{d}-h^{\text{ode}}-\mu\right)^{T}\cdotp R^{-1}\cdotp\left(h_{d}-h^{\text{ode}}-\mu\right)\right\} ,
\end{align*}
 where $n=t_{f}^{1}+t_{f}^{2}+t_{f}^{3}+t_{f}^{4}$ is the number
of data (ie, the dimension of $h_{d}$) and $h_{d}$ is defined as
\begin{align*}
h_{d}=\begin{bmatrix}h_{1}(t_{1}^{1})\\
h_{1}(t_{2}^{1})\\
\vdots\\
h_{1}(t_{f}^{1})\\
h_{2}(t_{1}^{2})\\
\vdots\\
h_{2}(t_{f}^{2})\\
\vdots\\
h_{4}(t_{f}^{4})
\end{bmatrix}.
\end{align*}
 Note that the times used in $h_{d}$ (and hence $h_{ode},\mu,R$)
can be a subset of the times used in the actual experiments. For instance,
the last measurements look highly suspicious and should be probably
excluded from our inference. $\mu$ and $R$ are defined as follows
\begin{align*}
\mu=\alpha\begin{bmatrix}t_{1}^{1}\\
t_{2}^{1}\\
\vdots\\
t_{f}^{1}\\
t_{1}^{2}\\
\vdots\\
t_{f}^{4}
\end{bmatrix},\quad R=\begin{bmatrix}R_{1} & 0 & 0 & 0\\
0 & R_{2} & 0 & 0\\
0 & 0 & R_{3} & 0\\
0 & 0 & 0 & R_{4}
\end{bmatrix}
\end{align*}
 where
\begin{align*}
R_{i}=\sigma^{2}\begin{bmatrix}1 & e^{-|t_{1}^{i}-t_{2}^{i}|/\tau} & e^{-|t_{1}^{i}-t_{3}^{i}|/\tau} & \ldots & e^{-|t_{1}^{i}-t_{f-1}^{i}|/\tau} & e^{-|t_{1}^{1}-t_{f}^{i}|/\tau}\\
\vdots & 1 & e^{-|t_{2}^{i}-t_{3}^{i}|/\tau} & \ldots & e^{-|t_{2}^{i}-t_{f-1}^{i}|/\tau} & e^{-|t_{2}^{i}-t_{f}^{i}|/\tau}\\
\\
\vdots &  & \ddots &  &  & \vdots\\
\\
 & \text{symm} &  &  & 1 & e^{-|t_{f-1}^{i}-t_{f}^{i}|/\tau}\\
\\
 &  & \ldots &  &  & 1
\end{bmatrix}
\end{align*}
 Because of the very specific form of our covariance matrix (block-diagonal),
we can simplify the expression for the likelihood as follows
\begin{align*}
 & \pi(h_{ode}=h_{d}\,|\,\alpha,\sigma,C_{d})=\ldots\\
 & \frac{1}{\sqrt{(2\pi)^{n}\prod_{i=1}^{4}|R_{i}|}}\exp\left\{ -\frac{1}{2}\sum_{i=1}^{4}\left(h_{d}-h^{\text{ode}}-\mu\right)_{i}^{T}\cdotp R_{i}^{-1}\cdotp\left(h_{d}-h^{\text{ode}}-\mu\right)_{i}\right\} ,
\end{align*}
 where
\begin{align*}
\left(h_{d}-h^{\text{ode}}-\mu\right)_{i}
\end{align*}
 corresponds to the data, computed results and mean for the $i^{\text{th}}$
experiment. This formulation clarifies the fact that we work experiment
per experiment as each experiment is independent from the others.

\begin{figure}[H]
\begin{centering}
\includegraphics[width=8cm]{figs/a_post}
\par\end{centering}

\caption{Prior and posterior probability function of parameter $\alpha$. }

\label{fig:Uc}
\end{figure}

\begin{figure}[H]
\begin{centering}
\includegraphics[width=8cm]{figs/s_post}
\par\end{centering}

\caption{Prior and posterior probability distribution function of parameter
$\sigma$.}

\label{fig:Uc}
\end{figure}

\begin{figure}[H]
\begin{centering}
\includegraphics[width=8cm]{figs/C1_post}
\par\end{centering}

\caption{Prior and posterior probability function of drag coefficient of bowling
ball $C_{d}1$. }

%Cd1

\label{fig:Uc}
\end{figure}

\begin{figure}[H]
\begin{centering}
\includegraphics[width=8cm]{figs/C2_post}
\par\end{centering}

\caption{Prior and posterior probability function of drag coefficient of basket
ball $C_{d}2$. }

%Cd1

\label{fig:Uc}
\end{figure}

\section{Independent Samples}

Generate samples of the joint posterior distribution of the calibration
parameters for use in performing the validation calculations. Instead
of using every sample in the MCMC chain, you may want to take every
$n^{th}$ sample in the chain, so the samples will be closer to independent.
Again be sure to document what you do. You will need enough nearly
independent samples to accurately perform the Monte Carlo integrals
in (3).

\subsubsection*{Solution}

We created samples of the joint posterior distribution for $C_{d},\sigma$
and $\alpha$ using MCMC. The correlation between the samples were
measured using the program `Autoregressive process modeling tools'
(https://github.com/RhysU/ar). Table \ref{table:creatingindesamples}
shows the number of steps for each of the parameters required so that
the samples are independent (correlation fall below a specified value).
$C_{d}$ has by far the longest correlation between steps, more than
30 chain steps before decorrelation.

An example of the arsel output is:
\begin{verbatim}
[nick@magus data]$ arsel -s < cd.dat | grep '#'
# absrho    true
# criterion CIC
# eff_N     32.751365129344251
# eff_var   0.12904055664629624
# gain      5.0428322668791719
# maxorder  512
# minorder  0
# mu        0.67116199032442436
# mu_sigma  0.062769462607289453
# N         1000
# AR(p)     71
# sigma2eps 0.024807597117980005
# sigma2x   0.12510055121028832
# submean   true
# T0        30.533078424386947
# window_T0 1
\end{verbatim}
Where $T_0$ presents the estimated envelope window of time of
correlation between samples.

We generated a set of samples 300,000 long, and downsampled to 10,000
uncorrelated samples for the integrals. We then confirmed that these
samples were uncorrelated using arsel on the downsampled chains. Later,
we found that these 10k uncorrelated samples were complete overkill with
respect to the integration. Only a few hundred samples were needed to
compute the posterior distributions shown below, which are generally
quite smooth (and no further smoothing function or KDE was utilized afterwards).

\begin{table}
\begin{centering}
\begin{tabular}{|c|c|}
\hline
Parameter & Chain steps to decorrelation\tabularnewline
\hline
\hline
$C_{d}$ & 30.533078\tabularnewline
\hline
$\sigma$ & 12.089690\tabularnewline
\hline
$\alpha$ & 11.870957\tabularnewline
\hline
\end{tabular}
\par\end{centering}

\caption{Creating independent samples.}

\label{table:creatingindesamples}
\end{table}

\section{Posterior of the observations}

Recall that the posterior distribution of an observation $y$, including
the uncertainty in the observation measurement is given by:

\begin{equation}
p\left(y|D,X\right)=\int_{\tilde{y}}\int_{\theta}p\left(y|\tilde{y},\theta,X\right)p\left(\tilde{y}|\theta,X\right)p\left(\theta|D,X\right)d\theta d\tilde{y}
\end{equation}

where $\tilde{y}$ is the unknown true value of the observable, and
$\theta$ is the vector of parameters. The conditional probabilities
in the integrand are:
\begin{itemize}
\item $p\left(y|\tilde{y},\theta,X\right)$ -- the probabilistic representation
of measurement error. It is conditioned on because of the calibration
parameters in the error model.
\item $p\left(\tilde{y}|\theta,X\right)$ -- this is the physical model,
which in our case is deterministic. Therefore, $p\left(\tilde{y}|\theta,X\right)=\delta\left(\tilde{y}-Y\left(\theta\right)\right)$
, where $Y\left(\theta\right)$ is the model predicted value of $\tilde{y}$
for given parameters , and $\delta\left(\right)$ is the Dirac delta
function. You made use of this fact when formulating the likelihood.
\item $p\left(\theta|D,X\right)$ is the posterior distribution of the parameters
obtained from the Bayesian calibration.
\end{itemize}
Because the model is deterministic, the $\tilde{y}$ integral can
be done trivially, further the samples of the parameters obtained
from the MCMC algorithm are samples from the posterior distribution
$p\left(\theta|D,X\right)$, so the $\theta$ integral can be performed
using a Monte Carlo method to obtain the approximation:

\begin{equation}
p\left(y|D,X\right)=\int_{\theta}p\left(y|Y\left(\theta\right),\theta,X\right)p\left(\theta|D,X\right)d\theta\approx\frac{1}{N}\sum_{j=1}^{N}p\left(y|Y\left(\theta_{j}\right),\theta_{j},X\right)
\end{equation}

where $\theta_{j}$ is the $j^{th}$ sample of $\theta$. Use this
Monte Carlo approximation, the samples from (2) and the measurement
error model developed for problem 2 to compute the posterior distribution
$p\left(y|D,X\right)$ for each observation of both balls considered
in problem 2. \textbf{Hint}: You can define a grid in $y$, with grid
points $y_{i}$ and evaluate the sum above for each $y_{i}$ . Be
sure to choose the grid spacing in $y$ to be fine enough to accurately
represent the distribution.

\subsubsection*{Solution}

In the general case, the posterior distribution of the observations~$y$
can be marginalized over the parameters~$\theta$ and the true value
of the observed quantity~$\tilde{y}$,
\begin{align*}
p(y|D,X)=\int_{\tilde{y}}\int_{\theta}p(y,\tilde{y},\theta|D,X)\, d\theta d\tilde{y}.
\end{align*}
 We can then apply Bayes' theorem twice to get
\begin{align*}
p(y|D,X) & =\int_{\tilde{y}}\int_{\theta}p(y|\tilde{y},\theta,D,X)p(\tilde{y},\theta|D,X)\, d\theta d\tilde{y}\\
 & =\int_{\tilde{y}}\int_{\theta}p(y|\tilde{y},\theta,D,X)p(\tilde{y}|\theta,D,X)p(\theta|D,X)\, d\theta d\tilde{y}.
\end{align*}
 We can now simplify this expression a bit more by noticing that the
first two conditional probability density functions~(pdf) do not
depend on the observed quantities~$D$. We end up with the expression
found in the problem statement
\begin{equation}
p(y|D,X)=\int_{\tilde{y}}\int_{\theta}p(y|\tilde{y},\theta,X)p(\tilde{y}|\theta,X)p(\theta|D,X)\, d\theta d\tilde{y}.\label{eq:postobsfull}
\end{equation}
 In this exercise, we did not assume any model inadequacy. Therefore,
the second pdf inside the integral is simply given by the deterministic
physical model, i.e.
\[
p(\tilde{y}|\theta,X)=\delta(\tilde{y}-Y(\theta)),
\]
 where $Y$ is the deterministic physical model. Then expression~\ref{eq:postobsfull}
simplifies more to
\begin{equation}
p(y|D,X)=\int_{\theta}p(y|Y(\theta),\theta,X)p(\theta|D,X)\, d\theta,\label{eq:postobs}
\end{equation}
 where $p(y|Y(\theta),\theta,X)$ is the observational model and $p(\theta|D,X)$
is the posterior distribution coming from the calibration.

Applying a Monte-Carlo approximation we evaluate integral~(\ref{eq:postobs})
thanks to the samples drawn in question~2. This gives
\begin{equation}
p(y|D,X)\approx\frac{1}{N}\sum_{i=1}^{N}p(y|Y(\theta_{i}),\theta_{i},X).\label{eq:postobsmc}
\end{equation}
 The variable~$y$ represents the observation and could be as such
understood as a vector of all the observations made. However multi-dimensional
model validation is cumbersome and instead we'll treat each observation
separately. For each ball, we had the results from two experiments.
Let's number these experiments and call the observations $y^{1}(t),y^{2}(t),y^{3}(t),y^{4}(t)$.
In each experiment, the ball position was only recorded at a discrete
number of times, we in fact we are dealing with the data
\[
y^{1}(t_{1}^{1}),y^{1}(t_{2}^{1}),\ldots,y^{1}(t_{N_{1}}^{1}),y^{2}(t_{1}^{2}),\ldots,y^{2}(t_{N_{2}}^{2}),\ldots,y^{4}(t_{N_{4}}^{4}).
\]
 And therefore for any $i=1,\ldots,4$ and $j=1,\ldots,N_{i}$, let
$y^{i,j}:=y^{i}(t_{j}^{i})$. We then want to evaluate
\[
p(y^{i,j}|D,X)\approx\frac{1}{N}\sum_{s=1}^{N}p(y^{i,j}|Y(\theta_{s}),\theta_{s},X).
\]
 We are going to evaluate that sum on a grid. For storage efficiency,
each observation will have its own grid centered around a reasonable
value (to be defined later). The grid is then defined as
\[
y_{k}^{i,j}=y_{c}^{i,j}+k\Delta y,\quad\forall k\in\{-m^{i,j},m^{i,j}\}.
\]
 For a given $i=1,\ldots,4$ and $j=1,\ldots,N_{i}$, we are going
to store the values of
\[
\frac{1}{N}\sum_{s=1}^{N}p(y^{i}(t_{j}^{i})=y_{k}^{i,j}|Y(\theta_{s}),\theta_{s},X),\quad\forall k\in\{-m^{i,j},m^{i,j}\}.
\]
 We can now write down explicitly the formula for that quantity. In
the group project~\#2, we defined the measurement error as a multivariate
Gaussian with mean and covariance depending on the calibration parameters~$\alpha$
and $\sigma$. But here we only care about the marginalized distribution
of a single observation. Fortunately, marginalized distributions of
Gaussian are also Gaussian with mean and covariance being given by
the restriction of the multivariate mean and covariance to the variables
of interest. In our case, we end up with a Gaussian of mean $\alpha t_{j}^{i}$
and covariance $\sigma^{2}$. Let's call $y_{c}^{i,j}$ the computed
solution of the ball drop without noise. We can therefore write
\[
y^{i,j}=y_{c}^{i,j}+\varepsilon,
\]
 with $\varepsilon\sim\mathcal{N}(\alpha t_{j}^{i},\sigma^{2})$.
Then we store at each grid point
\[
\frac{1}{N}\frac{1}{\sqrt{2\pi}\sigma}\sum_{s=1}^{N}e^{-(y_{k}^{i,j}-y_{c}^{i,j}-\alpha t_{j}^{i})^{2}/2\sigma^{2}},\quad\forall k\in\{-m^{i,j},m^{i,j}\}.
\]

\section{Credibility}

For each observation, compute the the credibility $1-\beta$, where
the datum is the maximum or minimum value of a $\beta-$credibility
set. You may use either an HPD or quantile credibility set. Plot these
credibility values as a function of time for the four ball drops considered.
\textbf{Hint}: you can approximate the required integrals using the
grid of values obtained in 3) and a quadrature rule, such as trapezoidal
rule.

\subsubsection*{Solution}

The credibility of a given distribution is defined as alpha = 1 \textendash{}
beta, where beta = integral(P(y)dy). In this project, we made the
decision to
use the HPD method to compute the credibility. HPD was selected because
we did not know if the distributions would be skewed (in reality, they
did not appear to have been strongly skewed) and HPD would provide a
more robust estimate than the quartile analysis in this case.
In the figures below, we
plot the credibility as a function of time for the 4 sets of measurements
(2 basket ball and 2 bowling ball). As one can clearly observe from
the figures, the data seems to be credible (wrt the model) only for
the first fractions of a second and then becomes incredible later on. In other
words, this means the predictions of the model are quickly inconsistent with the
observations. This figure depicts the credibility interval for the first
data set with the basketball, but all data sets and balls were
qualititively similar.

\begin{figure}[H]
\begin{centering}
\includegraphics[width=8cm]{figs/credibility}
\par\end{centering}
\caption{Credibility as a function of time.}

\label{fig:Uc}
\end{figure}

For all the datasets, after 1.5 seconds, the HPD credibility is assessed
as exceeding low, at less than $1e-5$. Around 2.5 seconds, every data
set is completely implausible, with credibilities assessed at less than
$1e-10$. In summary, the credibility decreases as a function of time,
and after a few seconds, the predictions of our model are completely
inconsistent with the observations.

\section{Validation test}

Choose two or more data points for each drop (including those with
the highest and lowest credibility), and plot the predicted posterior
distribution of the observation $p\left(y|D,X\right)$ along with
the value of the actual observation.

\subsubsection*{Solution}

The plots below show the best and the worst credibility intervals
along with credibility at few other time steps. The cases where $\beta$
falls within distribution give credence that the model is plausible \textendash{}
i.e. the model prediction is not inconsistent with the data. For the
values far outside the distribution, the model prediction is
inconsistent with the data, which certainly implies that we cannot trust
the model predictions to match those of observations. This is evidence
for an invalidation of the model, but further investigations would be
necessary to actually determine what is the ultimate cause of the
inconsistency (such as overly optimistic observation error bounds,
inaccurate scenario measurements, etc.).

\subsection{Ball drop - 1 (Basket Ball) }

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_1_5}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_1_20}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_1_35}}
\par\end{centering}

\caption{Predicted posterior distribution at time (a) $t=5$ (b) $t=20$ and
(c) $t=35$ secs vs the actual observation}

\label{balldrop1t5}

\end{figure}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_1_maxbeta22}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_1_minbeta1}}
\par\end{centering}

\caption{Predicted posterior distribution at (a) maximum $\beta$ (b) minimum
$\beta$}

\label{balldrop1t5-3}
\end{figure}

\subsection{Ball drop - 2 (Basket Ball)}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_2_5}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_2_20}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_2_35}}
\par\end{centering}

\caption{Predicted posterior distribution at time (a) $t=5$ (b) $t=20$ and
(c) $t=35$ secs vs the actual observation}

\label{balldrop1t5-1}
\end{figure}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_2_maxbeta40}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Basket_Ball_2_minbeta10}}
\par\end{centering}

\caption{Predicted posterior distribution at (a) maximum $\beta$ (b) minimum
$\beta$}

\label{balldrop1t5-3-1}
\end{figure}

\subsection{Ball drop - 1 (Bowling Ball)}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_1_5}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_1_20}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_1_35}}
\par\end{centering}

\caption{Predicted posterior distribution at time (a) $t=5$ (b) $t=20$ and
(c) $t=35$ secs vs the actual observation}

\label{balldrop1t5-2}
\end{figure}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_1_maxbeta24}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_1_minbeta0}}
\par\end{centering}

\caption{Predicted posterior distribution at (a) maximum $\beta$ (b) minimum
$\beta$}

\label{balldrop1t5-3-2}
\end{figure}

\subsection{Ball drop - 2 (Bowling Ball)}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_2_5}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_2_20}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_2_35}}
\par\end{centering}

\caption{Predicted posterior distribution at time (a) $t=5$ (b) $t=20$ and
(c) $t=35$ secs vs the actual observation}

\label{balldrop1t5-4}
\end{figure}

\begin{figure}[H]
\begin{centering}
\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_2_maxbeta20}}\,\,\,\subfigure[]{\includegraphics[width=6cm]{figs/Bowling_Ball_2_minbeta0}}
\par\end{centering}
\caption{Predicted posterior distribution at (a) maximum $\beta$ (b) minimum
$\beta$}

\label{balldrop1t5-3-3}
\end{figure}

\section{Consistency of the model}

Given the results of the above validation calculations, is the model
consistent with the observations for all drops? If not, what might
be wrong?

\subsubsection*{Solution}

From the above results, the model is clearly not consistent with the
observations for all ball drops. Rather, the model is accurate for
a finite time horizon, after which the predictions are no longer accurate.
Possible reasons for the model failure:
\begin{enumerate}
\item The assumptions made in the model may not be accurate. For
      instance, the ball may not be starting at rest, but with a
      non-zero downward velocity.
\item The model may be inadequate, without rich enough physics to
      capture all the significant dynamics.
\item The measurement data may not be reliable
\item The scenario parameters may have been inaccurate: for instance,
      the height of the bridge, or the size of the basketball.
\end{enumerate}

In addition, we note that the model appears to be systematically
overestimating the balls drop rate. In other words, it appears as if our
predictions are consistently projecting the ball to fall farther than it
actually does. This hints that our drag model may be at fault, because
it is not reducing the rate at which the ball falls as much as will in
reality.

\end{document}