\setcounter{section}{7}
\setcounter{subsection}{0}
\subsection{Bifunctors}

We start by studying if ADTs form functors. Product type in haskell is
actually a type constructor of two argument. \ie. \code{(a,b)} can be written as
\code{(,) a b}.

It's not hard to see that \code{(,) a}, \ie. fix on one parameter \code{a}, is a
functor from \code{b} to \code{(a, b)}.

\begin{center}
\begin{tikzcd}[math mode=false, sep=large]
  \code{(e,x)} \rar{\code{fmap f}} & \code{(e,y)} \\
  \code{x} \uar{\code{(e,)}} \rar{\code{f}} & \code{y} \uar{\code{(e,)}}
\end{tikzcd}
\end{center}

Now think, why don't we make a functor type that has two arguments, so we can
make \code{(,)} a functor on both arguments. In math, we could try to define a
category that formalize product of two categories instead of inventing a
something new, like a new kind of functor, for that purpose. But what will this
category look like? Well it's basically just product of the hom sets. For
categories $C$ and $D$, we define $C \times D$ as follows:

\begin{itemize}
\item For each pair of objects $c\in C$ and $d\in D$, there is an object $(c,d)
  \in C\times D$;
\item For each pair of arrows $f=c\mapsto c'$ in $C$ and $g
  = d\mapsto d'$ in $D$, there is an arrow $(f, g)=(c,d)\mapsto (c',d')$ in $C\times D$;
\item Composition: $(f', g') \circ (f, g) = (f'\circ f, g'\circ g)$;
\item Identity: $\id_{(a,b)}=(\id_a, \id_b)$.
\end{itemize}

So the functor that takes two arguments is just a functor $C\times
D \to E$. This kind of functors is called a bifunctor.

In Haskell, bifunctor is defined as:

\begin{lstlisting}
class Bifunctor b where
  bimap :: (a -> a') -> (b -> b') -> f a b -> f a' b'
\end{lstlisting}

And this is how \code{(,)} (product) and \code{Either} (sum) implemented as
bifunctors:

\begin{lstlisting}
instance Bifunctor (,) where
  bimap f g (a,b) = (f a, g b)

instance Bifunctor Either where
  bimap f g (Left a)  = Left (f a)
  bimap f g (Right a) = Right (g a)
\end{lstlisting}

In a cartesian category $C$, in which there are products for every pair of
objects, then this ``product'' is a bifunctor $C\times C \to C$. This also
applies to ``co-product'' categories. Here's how it works:

\begin{center}
\begin{tikzcd}[sep=large]
  & a \times b \ar{ld}[below]{\pi_1} \ar{rd}[below]{\pi_2} & \\
  a \ar{dd}{f} & & b \ar{dd}{g} \\
  & a' \times b' \ar{ld}{\pi_1'} \ar{rd}[below]{\pi_2'} & \\
  a' & & b'
\end{tikzcd}
\end{center}

So we have this diagram. For each $a, b \in C$, by definition, we have its
product $a\times b$ which projects ($\pi_1, \pi_2$) to $a$ and $b$. And for the
morphisms, we have \cd{a \rar{f} \& a'} and \cd{b \rar{g} \& b'}. In order for
the product ($\times : C\times C \to C$), to be a bifunctor, we need to show it
is possible to lift the pair $f, g$ into something that \cd[normal]{a\times b
  \rar{f\times g} \& a' \times b'}. \ie there exists a morphism from $a\times b$
to $a'\times b'$ on the diagram above.

\begin{center}
\begin{tikzcd}[sep=large]
  & a \times b
  \ar{ld}[below]{\pi_1} \ar[red]{lddd}[left]{f \circ \pi_1}
  \ar{rd}[below]{\pi_2} \ar[red]{rddd}[right]{g \circ \pi_2}
  \ar[dashed]{dd}[below]{f\times g}
  & \\
  a \ar{dd}{f} & & b \ar{dd}{g} \\
  & a' \times b' \ar{ld}{\pi_1'} \ar{rd}[below]{\pi_2'} & \\
  a' & & b'
\end{tikzcd}
\end{center}

Here's how we show that. Since we have $\pi_1$ and $f$, they will compose into
$f \circ \pi_1$, same for $\pi_2$ and $g$. By definition of the univeral
construction of products, we can see that $a'\times b'$ is the product on $a'$
and $b'$. Therefore, there must exist a unique factorialization from any pair
$a\times b$ to $a'\times b'$ (shown as the dashed arrow). This morphism is what
we are looking for and therefore we have shown the existence of $\times$
bifunctor.

In haskell, this is not an issue. As the implementation above, we just apply $f$
to $a$ and $g$ to $b$, then we will form $(f~a, g~b)$. But this is not enough
general for Math, we need to think out of just the $\Hask$ category. So the
above diagram shows how it work generally in any category.


\section{Introduction}
\begin{frame}{\insertsec}
	Neural Networks are a type of supervised learning. So we have a set of labeled data:
	$$\{(x^{(1)}, y^{(1)}), (x^{(2)}, y^{(2)}), ..., (x^{(m)}, y^{(m)})\}$$

	We want to use it to predict new data $\hat{y} = f(x)$

	\textbf{Problem}: search the proper $f(x)$
\end{frame}


\documentclass[12pt]{article}

\usepackage{amsthm}
\usepackage{graphicx}

\begin{document}

\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]

\theoremstyle{remark}
\newtheorem*{remark}{Remark}
\newtheorem{theorem}{Theorem}[section]
\newtheorem{lemma}[theorem]{Lemma}

\section{On a special intersection graph of class 2}
Our interest here is to find a graph of class 2 with $\Delta = 3$ and $\omega = 2$. Moreover, we want to ask ourselves if such a graph can be an intersection graph
of squares, and an intersection graph of unit squares.\\
Let's take a look at the following graph:

\begin{figure}[h]
    \centering
    \caption{Graph $G$ used as a base.}
    \includegraphics[scale=0.5]{tex_images/8cube.png}

\end{figure}

\begin{lemma}
    A graph obtained by replacing an edge of $G$ by a vertex of degree 2 is not 3-edge colorable.
\end{lemma}

\begin{proof}
    Notice that the previous embedding of $G$ is consisting of two rectangles, one inside the other, with edges linking the cornes.
    Call $a, b, c, d$ the corners of the inner rectangle, and $A, B, C, D$ the corner of the outer rectangle.\\
    The edges we consider are the obvious $ab, bc, cd, da$ (same for capitals) and the $aA$ (same for other letters.).\\
    Then we obtain an other graph, $G^'$ by supressing $AB$, adding a vertex $O$, then adding two edges: $AO$ and $OB$.\\

    \begin{figure}[h]
        \centering
        \includegraphics[scale=0.4]{tex_images/Gprime_uncolored.png}
    \end{figure}

    We start by coloring the 5-cycle $AOBba$. Without loss of generality, consider that a 3-coloring of it uses two times colors 1 and 2, and one time color 3. Consider the three following cases:
    \begin{itemize}
        \item 1. color 3 is on the $AO$ edge.
        \item 2. color 3 is on the $ab$ edge.
        \item 3. color 3 is on the $Aa$ edge. testestest
    \end{itemize}

    \textbf{\underline{case 1:}} \\
    Suppose $Aa$ is in color 1 and $AO$ is in 3.  . Then, by trying to use only 3 colors you end up with, in the following order: \\
    $ad$ 3; $bc$ 3; $BC$ 3; $AD$ 2; $Dd$ 1. Then you are forced to color $DC$ in a 4th color. \\

    \begin{figure}[h]
        \centering
        \caption{Case 1: we end up needing a 4th color.}
        \includegraphics[scale=0.3]{tex_images/case_1_1.png}
        \includegraphics[scale=0.3]{tex_images/case_1_2.png}
    \end{figure}

    \textbf{\underline{case 2:}} \\
    Suppose $Aa$ is in color 1. Then you must color in the following order: \\
    $ad$ 2; $AD$ 3; $Dd$ 1; $dc$ 3; $bc$ 1; $cC$ 2. Then you have to use a 4th color for $Dd$.\\

    \textbf{\underline{case 3:}} \\
    Suppose $Aa$ is in 3 and $ab$ is in 1. The coloring order is:\\
    $ad$ 2; $bc$ 3; $dc$ 1; $Dd$ 3; $Cc$ 2; $DC$ 1. Then a 4th color is necessary for $BC$ and $AD$.

\end{proof}

\newpage

\subsection{Case of squares intersection graph}
\begin{lemma}
    $G^'$ can be the intersection graph of axis parralel squares.
\end{lemma}
Observe an example of such a graph:
\begin{figure}[h]
    \centering
    \includegraphics[scale=0.4]{tex_images/8cube+_as_squares.png}
\end{figure}

\subsection{Case of unit squares intersection graph}

\begin{definition}[direction of an intersection for unit squares]
    We say that a square $a$ intersect $b$ in the direction "UL" if the upper-left corner of $a$ lies in $b$.
    Consequently, it would means $b$ intersect $a$ in the "DR" direction. There is 4 different directions: "UL", "UR", "DR", "DL"
\end{definition}

\begin{remark}
    Right now, we will suppose that a square $a$ can't intersect a square $b$ in two adjacent directions (see definition 1.2) at the same time.
    Later we will prove that in the case of our problem, this is not possible: see lemma 1.5.
\end{remark}

\begin{lemma}
    If $a$ intersect $b$ in a direction and $a$ intersect $c$ in the same direction, then $b$ and $c$ must intersect.
\end{lemma}

\begin{remark}
    Thus, if $a$ intersect $b$ in a direction, and $a$ intersect $c$; such that $b$ and $c$ do not intersect, then the intersection of $a$ and $c$ must be in a different direction.
\end{remark}

\begin{lemma}
    Let $a$ and $b$ two unit squares with affixes respectively $z_a$ and $z_b$ and $p_x, p_y$ the projections on axis $x, y$. Then:
    \begin{center}
        $a \cap b = \emptyset \Leftrightarrow (|p_x(z_a) - p_x(z_b)| > 1)$ or $(|p_y(z_a) - p_y(z_b)| >1)$

    \end{center}
\end{lemma}

\begin{proof}
    Trivial.
\end{proof}

\begin{definition}[adjacent directions]
    $a$ intersect $b$ and $c$ in distinct adjacent directions if the corners of $a$ that are inside $b$ and $c$ lies on an edge of $a$.
    For instance, UR and DL are not adjacent directions.
\end{definition}

\begin{lemma}
    If $a$ intersect $b$ in direction UL and $c$ in directions UR and DL at the same time, then:
    \begin{center}
        $\forall d$ such that $d \cap b \ne \emptyset$ and $d \cap c \ne \emptyset \Rightarrow d \cap a \ne \emptyset$
    \end{center}
\end{lemma}

\begin{proof}
    As $a$ and $c$ share $a$'s upper and down right corners, the only possibility for $d$ and $c$ to intersect without $d$ intersection $a$
    would be that $c$ interest $d$ in direction UL or DL. Applying lemma 1.4 on $d$ and $c$ on axis $x$ and noticing that $p_x(z_c) < p_x(z_d)$ would give us
    that $p_x(z_b) < 1 + p_x(z_d)$, ie $d \cap b = \emptyset$.
\end{proof}

\begin{remark}
    This implies that in an intersection graph of unit squares that is a $C_4$, we can't find two squares with one having two of it's corners inside the other.
\end{remark}

\begin{lemma}
    Let $G$ be $C_4$, an intersection graph of unit squares. Then; $\forall v \in V(G)$, $v$ intersects it's neighbours in adjacent directions.
\end{lemma}

\begin{proof}
    Let $p_x$ and $p_y$ the projections on respectively $x$ and $y$ axis.\\
    Let $a, b, c$ and $d$ be unit squares such that their corresponding intersection graph is $C_4$. Without loss of generality, suppose
    that $a$ intersect $b$ in direction UR and $a$ intersect $d$ in direction DL. \\
    Call $z_a, z_b, z_c$ and $z_d$ the affixes of the center of the squares.
    \begin{figure}[h]
        \centering
        \includegraphics[scale=0.4]{tex_images/proof_1_4_1.png}
    \end{figure}
    \\
    If $c$ intersect $d$ in direction DL, then $a, c$ and $d$ would share a common point in the upper right corner of $d$, contradicting the fact we have $C_4$.\\
    Suppose $c$ intersect $d$ in a direction different than DL such that $a$ and $c$ share no common point; then $c \cap b = \emptyset$, and we shall prove this using projections.
    As $a$ and $c$ share no commont point, in at least one of the projections, we must have an empty intersection. Call it $p$.
    Thus, we have $p(z_c) < 1 + p(z_a)$.
    as $a$ intersect $b$ in UR direction, then $p_x(z_a) < p_x(z_b)$ and $p_y(z_a) < p_y(z_b)$.
    By transitivity; we must have $p(z_c) < 1+ p(z_b)$; or, in other words; $c \cap b = \emptyset$.
\end{proof}

\begin{theorem}
    $G^'$ can't be a intersection graph of unit squares.
\end{theorem}

\begin{proof}
    We will actually prove that a subgraph of $G^'$ can't be done with unit squares. Let $G^{''}:=G'[U]$ with $U := \{a, b, d, c, C, D, A\}$
    The proof is done by trying to construct the squares according to the edges of $G''$.\\
    First, place $d$. It belongs to three cycles of size 4 at the same time through it's three neighbours: $a, c$ and $D$. Suppose, without loss of generality,
    that $d$ intersect $a$ in direction UL; then $d$ must intersect $D$ in direction either DL or UR in order to complete the cycle $daAD$. For the same reason,
    $d$ must intersect $c$ in direction either DL or UR to complete $dcCD$. Thus, completing the cycle $abcd$ becomes impossible as $d$ is intersecting $a$ and $c$
    in non-adjacent directions.
\end{proof}

\end{document}


% !TeX root = ../propa-cheat-sheet.tex
\section{Haskell Prelude functions}
\subsection{Collections}
\begin{minted}[breaklines]{haskell}
elem :: Eq a => a -> t a -> Bool
maximum :: forall a. Ord a => t a -> a
minimum :: forall a. Ord a => t a -> a
sum :: Num a => t a -> a
\end{minted}
\subsubsection{Lists}
\begin{minted}[breaklines]{haskell}
map :: (a -> b) -> [a] -> [b]
(++) :: [a] -> [a] -> [a] -- Concat lists
filter :: (a -> Bool) -> [a] -> [a]
head :: [a] -> a -- first element
last :: [a] -> a -- last element
tail :: [a] -> [a] -- everything except the first element
init :: [a] -> [a] -- everything except the last element
null :: Foldable t => t a -> Bool -- Empty check
length :: Foldable t => t a -> Int
reverse :: [a] -> [a]
(!!) :: [a] -> Int -> a -- Index operator
any :: Foldable t => (a -> Bool) -> t a -> Bool -- Test wether any element satisfies a predicate
all :: Foldable t => (a -> Bool) -> t a -> Bool -- Test wether all elements satisfies a predicate
replicate :: Int -> a -> [a] -- replicate n x is a list of length n with x the value of every element.
\end{minted}
\subsubsection{Infinite lists}
\begin{minted}[breaklines]{haskell}
iterate :: (a -> a) -> a -> [a] -- iterate f x == [x, f x, f (f x), ...]
repeat :: a -> [a] -- infinite list from first element: repeat x = [x, x, ...]
cycle :: [a] -> [a] -- turn finite list into circular infinite list
\end{minted}
\subsubsection{Sub-Lists}
\begin{minted}[breaklines]{haskell}
take :: Int -> [a] -> [a]
drop :: Int -> [a] -> [a]
splitAt :: Int -> [a] -> ([a], [a])
takeWhile :: (a -> Bool) -> [a] -> [a]
dropWhile :: (a -> Bool) -> [a] -> [a]
span :: (a -> Bool) -> [a] -> ([a], [a]) -- Tuple longest list of elements satisfying p and second element remainder
break :: (a -> Bool) -> [a] -> ([a], [a]) -- inverse of span, applied to a predicate p and a list xs, returns a tuple where first element is longest prefix (possibly empty) of xs of elements that do not satisfy p and second element is the remainder of the list
zipWith :: (a -> b -> c) -> [a] -> [b] -> [c]
zip = zipWith (,)
unzip :: [(a, b)] -> ([a], [b])
\end{minted}
\subsubsection{Folding}

\textbf{Right to left:}
\begin{minted}[breaklines]{haskell}
foldr :: (a -> b -> b) -> b -> t a -> b
\end{minted}
Usage:
\begin{minted}[breaklines]{haskell}
folded = foldr (\val -> \acc -> newAcc) acc coll
\end{minted}
\textbf{Left to right:}
\begin{minted}[breaklines]{haskell}
foldl :: (b -> a -> b) -> b -> t a -> b
\end{minted}
Usage:
\begin{minted}[breaklines]{haskell}
folded = foldl (\acc -> \val -> newAcc) acc coll
\end{minted}
\textbf{Bsp.:}
\begin{minted}[breaklines]{haskell}
foldr (+) 0 [1,2,3,4]
\end{minted}
berechnet den Wert (1+(2+(3+(4+0)))) – rechts-geklammert
\begin{minted}[breaklines]{haskell}
foldl (+) 0 [1,2,3,4]
\end{minted}
den Wert ((((0+1)+2)+3)+4) – links-geklammert\\
\subsubsection{Tuples}
\begin{minted}[breaklines]{haskell}
fst :: (a,b) -> a
snd :: (a,b) -> b
curry :: ((a, b) -> c) -> a -> b -> c
uncurry :: (a -> b -> c) -> (a, b) -> c
\end{minted}
\subsection{Values and numbers}
\begin{minted}[breaklines]{haskell}
succ :: a -> a
pred :: a -> a
even :: Integral a => a -> Bool
odd :: Integral a => a -> Bool
gcd :: Integral a => a -> a -> a
lcm :: Integral a => a -> a -> a
(^) :: (Num a, Integral b) => a -> b -> a
\end{minted}
\subsubsection{Float}
\begin{minted}[breaklines]{haskell}
isNaN :: a -> Bool
isInfinite :: a -> Bool
\end{minted}
\subsection{Misc}
\subsubsection{IO}
\begin{minted}[breaklines]{haskell}
putStrLn :: String -> IO ()
print :: Show a => a -> IO ()
getLine :: IO String
getChar :: IO Char
\end{minted}
\subsubsection{Maybe}
\begin{minted}[breaklines]{haskell}
data Maybe = Nothing |  Just a
\end{minted}
\begin{minted}[breaklines]{haskell}
maybe :: b -> (a -> b) -> Maybe a -> b
\end{minted}
Usage:
\begin{minted}[breaklines]{haskell}
value = maybe orElse mappingAToB maybe instance
\end{minted}
\subsection{Type classes}
\begin{minted}[breaklines]{haskell}
class (Num a) where
  (+) :: a -> a -> a
  (*) :: a -> a -> a
  negate :: a -> a
  fromInteger :: Integer -> a

data PowerSeries = PS [Double]

instance (Num PowerSeries) where
  (PS p) + (PS q) = PS (addP p q)
  (PS p) * (PS q) = PS (multP p q)
  negate (PS p) = PS (map negate p)
  fromInteger i = PS (fromStep (fromInteger i) (const 0))
\end{minted}


\hypertarget{group__numpp__differentiation__symbolic}{}\section{Symbolic Differentiation}
\label{group__numpp__differentiation__symbolic}\index{Symbolic Differentiation@{Symbolic Differentiation}}

This module provides the means for performing symbolic differentiation.

\subsection*{Classes}
\begin{DoxyCompactItemize}
\item
class \hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1differentiate}{numpp\+::differentiation\+::symbolic\+::differentiate$<$ Function, Order $>$}
\begin{DoxyCompactList}\small\item\em Allows differentiation of N-\/th order. \end{DoxyCompactList}\item
class \hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1constant}{numpp\+::differentiation\+::symbolic\+::constant$<$ Value $>$}
\begin{DoxyCompactList}\small\item\em Class representing constant integer number. \end{DoxyCompactList}\item
class \hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1variable}{numpp\+::differentiation\+::symbolic\+::variable$<$ T, Number $>$}
\begin{DoxyCompactList}\small\item\em Class representing variable of any type. \end{DoxyCompactList}\end{DoxyCompactItemize}
\subsection*{Typedefs}
\begin{DoxyCompactItemize}
\item
\mbox{\Hypertarget{group__numpp__differentiation__symbolic_gac865497d2896f51d0cab2e9c64799a15}\label{group__numpp__differentiation__symbolic_gac865497d2896f51d0cab2e9c64799a15}}
{\footnotesize template$<$std\+::size\+\_\+t Number$>$ }\\using \hyperlink{group__numpp__differentiation__symbolic_gac865497d2896f51d0cab2e9c64799a15}{numpp\+::differentiation\+::symbolic\+::x} = \hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1variable}{variable}$<$ double, Number $>$
\begin{DoxyCompactList}\small\item\em Convenience alias for variable with specified variable type as double. \end{DoxyCompactList}\end{DoxyCompactItemize}
\subsection*{Functions}
\begin{DoxyCompactItemize}
\item
{\footnotesize template$<$typename T $>$ }\\constexpr auto \hyperlink{group__numpp__differentiation__symbolic_ga186ca63e28a816d1cb9c8175c7839468}{numpp\+::differentiation\+::symbolic\+::log} (const T \&)
\item
{\footnotesize template$<$typename T $>$ }\\constexpr auto \hyperlink{group__numpp__differentiation__symbolic_gad414fd158986bf7fff7e258a6ebf182c}{numpp\+::differentiation\+::symbolic\+::cos} (const T \&)
\end{DoxyCompactItemize}
\begin{DoxyCompactItemize}
\item
{\footnotesize template$<$typename Left , typename Right $>$ }\\constexpr auto \hyperlink{group__numpp__differentiation__symbolic_ga45bd7508b366431d72157d0f1ef6fac2}{numpp\+::differentiation\+::symbolic\+::operator+} (const Left \&, const Right \&)
\item
{\footnotesize template$<$typename T $>$ }\\constexpr auto \hyperlink{group__numpp__differentiation__symbolic_ga4aecf7125ae67f3c834ed8ab5e14248a}{numpp\+::differentiation\+::symbolic\+::operator-\/} (const T \&)
\item
{\footnotesize template$<$typename Left , typename Right $>$ }\\constexpr auto \hyperlink{group__numpp__differentiation__symbolic_ga0f6f3d22e3e712ce154f7f6c91e0cf46}{numpp\+::differentiation\+::symbolic\+::operator$\ast$} (const Left \&, const Right \&)
\item
{\footnotesize template$<$typename Left , typename Right $>$ }\\constexpr auto \hyperlink{group__numpp__differentiation__symbolic_gac882bc2dcbffae1107c46bc3eac93a44}{numpp\+::differentiation\+::symbolic\+::operator/} (const Left \&, const Right \&)
\item
{\footnotesize template$<$std\+::size\+\_\+t N, typename Left $>$ }\\constexpr auto \hyperlink{group__numpp__differentiation__symbolic_gaa52e3714ea3492f60c68cdb4974fe495}{numpp\+::differentiation\+::symbolic\+::pow} (const Left \&)
\end{DoxyCompactItemize}
\begin{DoxyCompactItemize}
\item
{\footnotesize template$<$typename T $>$ }\\constexpr auto \hyperlink{group__numpp__differentiation__symbolic_ga4bcf2dcdbc549b58c9fb7cd1b8c1cf70}{numpp\+::differentiation\+::symbolic\+::exp} (const T \&)
\end{DoxyCompactItemize}
\begin{DoxyCompactItemize}
\item
{\footnotesize template$<$typename T $>$ }\\constexpr auto \hyperlink{group__numpp__differentiation__symbolic_ga49ba3eed5ff463e79a88f4952f73f435}{numpp\+::differentiation\+::symbolic\+::sin} (const T \&)
\end{DoxyCompactItemize}

\subsection{Detailed Description}
This module provides the means for performing symbolic differentiation.

\begin{DoxyCode}
\hyperlink{namespacenumpp}{numpp}/differentiation/symbolic.hpp #includes every method in the module
\hyperlink{namespacenumpp}{numpp}/differentiation/symbolic/SUBCLASS.hpp #include specific SUBCLASS
\end{DoxyCode}

\begin{DoxyWarning}{Warning}
{\bfseries Second include option S\+H\+O\+U\+L\+DN\textquotesingle{}T be used}
\end{DoxyWarning}
Idea developed independently, but merged with the research paper \href{https://arxiv.org/abs/1705.01729}{\tt Compile Time Symbolical Differentiation 2017} and many ideas based on theirs. ~\newline
{\bfseries Thank you for this paper, most of the credits belong to them! Please refer to their paper for thourough speed comparison against automatic differentiation.}

For differences mine and theirs application of the type differentiation see this paper\+: \href{https://github.com}{\tt numpp dissertation } ~\newline
 For comparison against \href{https://github.com/symengine/symengine}{\tt symengine}, base of \href{http://www.sympy.org/en/index.html}{\tt Sym\+Py} see these tests\+: \href{https://github.com}{\tt sympy vs numpp }

{\bfseries Details and usage\+:}

Overloads for common mathemathical operations are provided (details in classes).~\newline
Approach taken by type differentiating doesn\textquotesingle{}t resemble well-\/known symbolic differentiation in C++, hence some examples are needed\+:

\begin{DoxyWarning}{Warning}
For convenience, in the following examples, namespace numpp\+::differentiation\+::symbolic is abbreviated as nds.
\end{DoxyWarning}
{\bfseries 1. Variables and constants }

Basing building blocks of the function. Can be defined as follows\+:
\begin{DoxyCode}
\hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1variable}{nds::x<0>} x0\{\};
\hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1variable}{nds::x<1>} x1\{\};
\end{DoxyCode}

Above x variables are implicilty of type double. To express type explicitly use\+:

\begin{DoxyCode}
\hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1variable}{nds::variable<int, 0>} x0\{\};
\end{DoxyCode}

Numeric parameter (of type std\+::size\+\_\+t) differentiates every variable from the others.

\begin{DoxyCode}
\hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1constant}{nds::constant<14>} cons\{\};
\end{DoxyCode}

Currently only integer constants are supported.

{\bfseries 2. Defining functions }

We only have to define Function type, there is no need to create the object.~\newline
Arithmetic operators and some other mathematical functions are overloaded for convenience in function defining.

You should use the
\begin{DoxyCode}
\textcolor{keyword}{using} and decltype()
\end{DoxyCode}
 when defining it.

{\bfseries Example of $ \sin(\cos(\frac{x_0}{x_1})*\log(x_0)) + x_2 $ creation} \+:

\begin{DoxyCode}
\textcolor{preprocessor}{#include"numpp/differentiation/symbolic.hpp"}

\textcolor{keyword}{using} Function = decltype(\hyperlink{group__numpp__differentiation__forward__automatic_ga6f6f07250087b18ada9ea1f84418f24f}{sin}(\hyperlink{group__numpp__differentiation__forward__automatic_gaa5a06320f93037d94fda441546d7d690}{cos}(\hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1variable}{nds::x<0>}\{\}/\hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1variable}{nds::x<1>}\{\})*
      \hyperlink{group__numpp__differentiation__forward__automatic_ga0f6011595edffe756a984de60358fe5f}{log}(\hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1variable}{nds::x<0>}\{\})) + \hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1variable}{nds::x<2>}\{\});
\end{DoxyCode}

{\bfseries 3. Differentiating the function\+:}

There are two ways to differentiate type, as shown below\+:

{\bfseries 3.\+1. Predicate with\+\_\+respect\+\_\+to\+:}
\begin{DoxyCode}
\textcolor{comment}{//Function as defined above}
\textcolor{keyword}{using} Derivative = \hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1differentiate}{nds::differentiate<Function, 3>::with\_respect\_to}
      <0,0,1>;
\end{DoxyCode}

Above code performs 3-\/rd order differentiation with respect to $x_0, x_0, x_1 $ in this exact order.

{\bfseries 3.\+2. With derivative type\+:}
\begin{DoxyCode}
\textcolor{comment}{//Function as defined above}
\textcolor{keyword}{using} Derivative = Function::derivative<0>
\end{DoxyCode}

Above performs differentiation with respect to $x_0$.

{\bfseries 4. Obtaining numerical values from expressions}

After obtaining Function type (or Derivative or similiar) you would probably like to obtain value at a given point. ~\newline
To do so, you have to create an array which size is equal to the number of independent variables~\newline
and pass it to the static constexpr function {\bfseries calculate}

{\bfseries Example}

\begin{DoxyCode}
\textcolor{comment}{//Derivative as defined above}
constexpr \textcolor{keyword}{auto} value = Derivative::calculate(std::array<double, 3>\{1.,5.,7.\});
\end{DoxyCode}

Any object providing operator\mbox{[}\mbox{]} may be used as an argument to calculate.

{\bfseries Technical Details\+:}

Approach of type differentiating allows this module to obtain near hand-\/coded efficiency for n-\/th order partial derivatives.

Cons -\/ compilation may take longer as order and complexity of formula increases after this initial phase, every derivative evaluation or next differentiation will take substantially shorter amount of time than current available approaches.

\subsection{Function Documentation}
\mbox{\Hypertarget{group__numpp__differentiation__symbolic_gad414fd158986bf7fff7e258a6ebf182c}\label{group__numpp__differentiation__symbolic_gad414fd158986bf7fff7e258a6ebf182c}}
\index{Symbolic Differentiation@{Symbolic Differentiation}!cos@{cos}}
\index{cos@{cos}!Symbolic Differentiation@{Symbolic Differentiation}}
\subsubsection{\texorpdfstring{cos()}{cos()}}
{\footnotesize\ttfamily template$<$typename T $>$ \\
constexpr auto numpp\+::differentiation\+::symbolic\+::cos (\begin{DoxyParamCaption}\item[{const T \&}]{ }\end{DoxyParamCaption})}

cos returns object representing $ \cos(T) $
\begin{DoxyTemplParams}{Template Parameters}
{\em T} & Parameter representing type of the argument (can be complex like log(x))\\
\hline
\end{DoxyTemplParams}
\begin{DoxyReturn}{Returns}
Type representing $ \cos(T) $
\end{DoxyReturn}
\mbox{\Hypertarget{group__numpp__differentiation__symbolic_ga4bcf2dcdbc549b58c9fb7cd1b8c1cf70}\label{group__numpp__differentiation__symbolic_ga4bcf2dcdbc549b58c9fb7cd1b8c1cf70}}
\index{Symbolic Differentiation@{Symbolic Differentiation}!exp@{exp}}
\index{exp@{exp}!Symbolic Differentiation@{Symbolic Differentiation}}
\subsubsection{\texorpdfstring{exp()}{exp()}}
{\footnotesize\ttfamily template$<$typename T $>$ \\
constexpr auto numpp\+::differentiation\+::symbolic\+::exp (\begin{DoxyParamCaption}\item[{const T \&}]{ }\end{DoxyParamCaption})}

exp returns object representing $ e^T $
\begin{DoxyTemplParams}{Template Parameters}
{\em T} & Parameter representing type of the exponent (can be complex like sin(x))\\
\hline
\end{DoxyTemplParams}
\begin{DoxyReturn}{Returns}
Type representing $ e^T $
\end{DoxyReturn}
\mbox{\Hypertarget{group__numpp__differentiation__symbolic_ga186ca63e28a816d1cb9c8175c7839468}\label{group__numpp__differentiation__symbolic_ga186ca63e28a816d1cb9c8175c7839468}}
\index{Symbolic Differentiation@{Symbolic Differentiation}!log@{log}}
\index{log@{log}!Symbolic Differentiation@{Symbolic Differentiation}}
\subsubsection{\texorpdfstring{log()}{log()}}
{\footnotesize\ttfamily template$<$typename T $>$ \\
constexpr auto numpp\+::differentiation\+::symbolic\+::log (\begin{DoxyParamCaption}\item[{const T \&}]{ }\end{DoxyParamCaption})}

log returns object representing $ \log(T) $
\begin{DoxyTemplParams}{Template Parameters}
{\em T} & Parameter representing type being logarithmized (can be complex like sin(x))\\
\hline
\end{DoxyTemplParams}
\begin{DoxyReturn}{Returns}
Type representing $ \log(T) $
\end{DoxyReturn}
\mbox{\Hypertarget{group__numpp__differentiation__symbolic_ga0f6f3d22e3e712ce154f7f6c91e0cf46}\label{group__numpp__differentiation__symbolic_ga0f6f3d22e3e712ce154f7f6c91e0cf46}}
\index{Symbolic Differentiation@{Symbolic Differentiation}!operator$\ast$@{operator$\ast$}}
\index{operator$\ast$@{operator$\ast$}!Symbolic Differentiation@{Symbolic Differentiation}}
\subsubsection{\texorpdfstring{operator$\ast$()}{operator*()}}
{\footnotesize\ttfamily template$<$typename Left , typename Right $>$ \\
constexpr auto numpp\+::differentiation\+::symbolic\+::operator$\ast$ (\begin{DoxyParamCaption}\item[{const Left \&}]{,  }\item[{const Right \&}]{ }\end{DoxyParamCaption})}

operator$\ast$ returns object representing multiplication of Left and Right hand-\/side
\begin{DoxyTemplParams}{Template Parameters}
{\em Left} & Parameter representing left hand-\/side (can be complex like sin(x)) \\
\hline
{\em Right} & Parameter representing Right hand-\/side (can be complex like sin(x))\\
\hline
\end{DoxyTemplParams}
\begin{DoxyReturn}{Returns}
Type representing multiplication of Left and Right
\end{DoxyReturn}
\mbox{\Hypertarget{group__numpp__differentiation__symbolic_ga45bd7508b366431d72157d0f1ef6fac2}\label{group__numpp__differentiation__symbolic_ga45bd7508b366431d72157d0f1ef6fac2}}
\index{Symbolic Differentiation@{Symbolic Differentiation}!operator+@{operator+}}
\index{operator+@{operator+}!Symbolic Differentiation@{Symbolic Differentiation}}
\subsubsection{\texorpdfstring{operator+()}{operator+()}}
{\footnotesize\ttfamily template$<$typename Left , typename Right $>$ \\
constexpr auto numpp\+::differentiation\+::symbolic\+::operator+ (\begin{DoxyParamCaption}\item[{const Left \&}]{,  }\item[{const Right \&}]{ }\end{DoxyParamCaption})}

operator+ returns object representing addition of Left and Right hand-\/side
\begin{DoxyTemplParams}{Template Parameters}
{\em Left} & Parameter representing left hand-\/side (can be complex like sin(x)) \\
\hline
{\em Right} & Parameter representing Right hand-\/side (can be complex like sin(x))\\
\hline
\end{DoxyTemplParams}
\begin{DoxyReturn}{Returns}
Type representing addition of Left and Right
\end{DoxyReturn}
\mbox{\Hypertarget{group__numpp__differentiation__symbolic_ga4aecf7125ae67f3c834ed8ab5e14248a}\label{group__numpp__differentiation__symbolic_ga4aecf7125ae67f3c834ed8ab5e14248a}}
\index{Symbolic Differentiation@{Symbolic Differentiation}!operator-\/@{operator-\/}}
\index{operator-\/@{operator-\/}!Symbolic Differentiation@{Symbolic Differentiation}}
\subsubsection{\texorpdfstring{operator-\/()}{operator-()}}
{\footnotesize\ttfamily template$<$typename T $>$ \\
constexpr auto numpp\+::differentiation\+::symbolic\+::operator-\/ (\begin{DoxyParamCaption}\item[{const T \&}]{ }\end{DoxyParamCaption})}

operator-\/ returns object representing the negative of it, e.\+g. sin(x)-\/$>$ -\/sin(x)
\begin{DoxyTemplParams}{Template Parameters}
{\em T} & Parameter representing any type (can be complex like sin(x))\\
\hline
\end{DoxyTemplParams}
\begin{DoxyReturn}{Returns}
Type representing negative of a given object
\end{DoxyReturn}
\mbox{\Hypertarget{group__numpp__differentiation__symbolic_gac882bc2dcbffae1107c46bc3eac93a44}\label{group__numpp__differentiation__symbolic_gac882bc2dcbffae1107c46bc3eac93a44}}
\index{Symbolic Differentiation@{Symbolic Differentiation}!operator/@{operator/}}
\index{operator/@{operator/}!Symbolic Differentiation@{Symbolic Differentiation}}
\subsubsection{\texorpdfstring{operator/()}{operator/()}}
{\footnotesize\ttfamily template$<$typename Left , typename Right $>$ \\
constexpr auto numpp\+::differentiation\+::symbolic\+::operator/ (\begin{DoxyParamCaption}\item[{const Left \&}]{,  }\item[{const Right \&}]{ }\end{DoxyParamCaption})}

operator/ returns object representing division of Left and Right hand-\/side
\begin{DoxyTemplParams}{Template Parameters}
{\em Left} & Parameter representing left hand-\/side (can be complex like sin(x)) \\
\hline
{\em Right} & Parameter representing Right hand-\/side (can be complex like sin(x))\\
\hline
\end{DoxyTemplParams}
\begin{DoxyReturn}{Returns}
Type representing division of Left and Right
\end{DoxyReturn}
\mbox{\Hypertarget{group__numpp__differentiation__symbolic_gaa52e3714ea3492f60c68cdb4974fe495}\label{group__numpp__differentiation__symbolic_gaa52e3714ea3492f60c68cdb4974fe495}}
\index{Symbolic Differentiation@{Symbolic Differentiation}!pow@{pow}}
\index{pow@{pow}!Symbolic Differentiation@{Symbolic Differentiation}}
\subsubsection{\texorpdfstring{pow()}{pow()}}
{\footnotesize\ttfamily template$<$std\+::size\+\_\+t N, typename Left $>$ \\
constexpr auto numpp\+::differentiation\+::symbolic\+::pow (\begin{DoxyParamCaption}\item[{const Left \&}]{ }\end{DoxyParamCaption})}

returns object representing Left hand-\/side to the n-\/th power

{\bfseries Example usage\+:}
\begin{DoxyCode}
\textcolor{keyword}{using} Function = decltype(pow<4>(\hyperlink{classnumpp_1_1differentiation_1_1symbolic_1_1variable}{nds::x<0>}\{\}));
\end{DoxyCode}

\begin{DoxyWarning}{Warning}
Unluckily there seems to be no workaround around passing exponent as a template parameter
\end{DoxyWarning}

\begin{DoxyTemplParams}{Template Parameters}
{\em N} & Power of the function, Left is taken to the power of N \\
\hline
{\em Right} & Parameter representing type being any valid symbolic expression (can be complex like sin(x))\\
\hline
\end{DoxyTemplParams}
\begin{DoxyReturn}{Returns}
Type representing $ Left^N $
\end{DoxyReturn}
\mbox{\Hypertarget{group__numpp__differentiation__symbolic_ga49ba3eed5ff463e79a88f4952f73f435}\label{group__numpp__differentiation__symbolic_ga49ba3eed5ff463e79a88f4952f73f435}}
\index{Symbolic Differentiation@{Symbolic Differentiation}!sin@{sin}}
\index{sin@{sin}!Symbolic Differentiation@{Symbolic Differentiation}}
\subsubsection{\texorpdfstring{sin()}{sin()}}
{\footnotesize\ttfamily template$<$typename T $>$ \\
constexpr auto numpp\+::differentiation\+::symbolic\+::sin (\begin{DoxyParamCaption}\item[{const T \&}]{ }\end{DoxyParamCaption})}

sin returns object representing $ \sin(T) $
\begin{DoxyTemplParams}{Template Parameters}
{\em T} & Parameter representing type of the argument (can be complex like log(x))\\
\hline
\end{DoxyTemplParams}
\begin{DoxyReturn}{Returns}
Type representing $ \sin(T) $
\end{DoxyReturn}


A plane wave state is considered in a semi-infinite medium of length $l=6m$ in direction $\vect{e}_1$, made of a hyperelastic Saint-Venant-Kirchhoff material:
\begin{align*}
  & \tens{F} = F \vect{e}_1 \otimes \vect{e}_1 + \vect{e}_2 \otimes \vect{e}_2 + \vect{e}_3 \otimes \vect{e}_3 \\
  & \tens{\Pi} = \Pi \vect{e}_1 \otimes \vect{e}_1 + \Pi_r \(\vect{e}_2 \otimes \vect{e}_2 + \vect{e}_3 \otimes \vect{e}_3\)
\end{align*}
where $\Pi = \frac{2\mu + \lambda}{2} F(F^2 - 1)$, $\Pi_r = \frac{\lambda}{2}(F^2 - 1)$ and $(\mu,\lambda)$ are the Lam\'e's coefficients. A traction force is enforced on the left boundary of the solid initially at rest: $\tens{\Pi}\cdot\(-\vect{e}_1\)=\Pi^d\vect{e}_1$. The exact solutions of the Picard problem thus formulated have been developed in section \ref{sec:SVK_solution}.
With characteristic speeds depending on the deformation gradient, a compressive (\textit{resp. tensile}) load leads to a rarefaction (\textit{resp. shock}) wave traveling in the medium.
Recall that this non-intuitive behavior is due to the concave nature of the SVK fluxes (see remark \ref{rq:charach_neoHook} in section \ref{sec:SVK_solution}).
Both cases are considered hereinafter before reflection on the right end.
%Recall that since the characteristic speeds of this non-linear problem depends on the deformation gradient, a compressive (\textit{resp. tensile}) load leads to a rarefaction (\textit{resp. shock}) wave travelling in the medium, both cases being considered hereinafter before reflection on the right end.
Moreover, it has been established that the problem is no longer hyperbolic if the deformation gradient is such that $F<\sqrt{\frac{1}{3}}$ (see remark \ref{rq:hyperbolicity_limit_SVK} in section \ref{sec:SVK_solution}).
Hence, we consider here loading conditions that do not yield a loss of hyperbolicity.

The one-dimensional medium is discretized by using either $100$ or $200$ material points lying in $100$ regular grid cells.
The 1ppc and 2ppc discretizations used here are the same as before as well as material parameters of table \ref{tab:material}.

\subsubsection{Compressive impact on a SVK medium}
To begin with, the body is submitted to a compressive load on its left end so that a rarefaction wave propagates rightward.
The total and updated Lagrangian formulations of the MPM are used along with DGMPM schemes.
First of all, the compressive load is set to $\Pi^d= 4\times 10^{8} \: Pa$ and numerical solutions are compared in figure \ref{fig:he_rarefaction_UL} at two different times.
Though the low load amplitude applied results in a solution which looks like a discontinuity, it avoids grid crossing.

\begin{figure}[h!]
  \centering
  {\input{chapter4/pgfFigures/he_stress_low_UL}}
  \caption{First Piola–Kirchhoff stress along a horizontal line of the one-dimensional hyperelastic medium at different times: solution of the compressive impact problem $\Pi^d= 4\times 10^{8} \: Pa$. Comparison between updated and total Lagrangian MPMs (CFL=$0.5$), DGMPM-Euler with 1ppc (CFL=$1$) or 2ppc (CFL=$0.5$), DGMPM-RK2 with 2ppc (CFL=$1$), and the exact solution.}
  \label{fig:he_rarefaction_UL}
\end{figure}
First, both solutions of updated and total Lagrangian MPM formulations oscillate.
It is noteworthy that these two stress profiles are superimposed except at the left end of the domain where an instability occurs in the updated Lagrangian solution due to the displacement of particles.
Second, DGMPM solutions still behave well and show good agreement with the exact solution. However, the enforcement of a higher load amplitude is more relevant in order to see the error made by the DGMPM.
The applied load is therefore raised to $\Pi^d= 2\times 10^{10} \: Pa$.

Significant differences between numerical methods and the exact solution are then visible in figure \ref{fig:he_rarefaction}.
\begin{figure}[h!]
  \centering
  {\phantomsubcaption{\label{subfig:he_rarefaction1}}}
  {\phantomsubcaption{\label{subfig:he_rarefaction2}}}
  {\input{chapter4/pgfFigures/he_stress_high_UL}}
  \caption{First Piola–Kirchhoff stress along a horizontal line of the one-dimensional hyperelastic medium at different times: solution of the compressive impact problem $\Pi^d= 2\times 10^{10} \: Pa$. Comparison between MPM (CFL=$0.5$), DGMPM-Euler with 1ppc (CFL=$1$) or 2ppc (CFL=$0.5$), DGMPM-RK2 with 2ppc (CFL=$1$), and the exact solution.}
  \label{fig:he_rarefaction}
\end{figure}
First, grid crossing now occurs so that the MPM results based on the updated Lagrangian formulation are no longer correct. Hence, this formulation is omitted from now on.
On the other hand, the total Lagrangian MPM scheme yields a solution which does not oscillate more than for linear cases.
%The total Lagrangian approach, on the other hand, provides a solution in which the wave travel at the correct speed, although spurious oscillations still arise.
Next, as expected from the use of an approximate-state Riemann solver for non-linear problems, DGMPM stresses, though close to the analytical solution, no longer fit it exactly (see figure \ref{fig:he_rarefaction}\subref{subfig:he_rarefaction2}).
Furthermore, the use of DGMPM-Euler with 2ppc leads to a smoother solution.

One can imagine the employment of an exact Riemann solver based on the exact solution in order to reduce the error made in the DGMPM.
% In order to reduce the error made in the DGMPM, one can imagine to use an exact Riemann solver based on the exact solution.
However, such an implementation requires the solution of a non-linear problem at each cell interface and is very costly.
Moreover, an exact solution is available for a SVK material but this is not the case for other constitutive models, which prevents the generic use of an exact solver.

% \begin{remark}
%   This problem can be solved in a two-dimensional setting by considering an infinite medium in direction $\vect{e}_3$, of dimension $l\times h$ in directions $\vect{e}_1$ and $\vect{e}_2$, which bottom and top boundaries are assumed vertically fixed.
%   %A two-dimensional setting may be used to solve this problem by considering an infinite medium in direction $\vect{e}_3$, of dimension $l\times h$ in directions $\vect{e}_1$ and $\vect{e}_2$, which bottom and top boundaries are assumed vertically fixed.
%   In that case, DGMPM results still show a good agreement with the exact solution of the problem \cite{DGMPM}.
% \end{remark}

\subsubsection{Tensile impact on a SVK medium}
A tensile load of magnitude $\Pi^d= -2\times 10^{10} \: Pa$ that gives rise to a shock wave in the medium is now studied. Recall that for this problem, the characteristic speeds are: $c=\pm \sqrt{\frac{\lambda+2\mu}{2\rho_0}\(3F^2-1\) }$ (see equation \eqref{eq:SVK_charac_fields} in section \ref{sec:SVK_solution}).
It then follows that the higher the deformation gradient, the faster waves propagate. Hence, the celerity of the fastest wave is given by the deformation gradient on the upwind side and must be used to adapt the time step and hence, satisfy the CFL condition for fixed Courant number.
This situation did not arise in the above case of a rarefaction wave since the highest deformation gradient value was (constant) on the downwind side.

\begin{figure}[h!]
  \centering
  {\phantomsubcaption \label{subfig:he_shock1}}
  {\phantomsubcaption \label{subfig:he_shock2}}
  {\input{chapter4/pgfFigures/he_stress_50shock}}
  \caption{First Piola–Kirchhoff stress solution of the tensile impact problem at different times: $\Pi^d=-2\times 10^{10} \: Pa$. Comparison between MPM (CFL=$0.5$), DGMPM-Euler with 1ppc (CFL=$1$) or 2ppc (CFL=$0.5$), DGMPM-RK2 with 2ppc (CFL=$1$), and the exact solution.}
  \label{fig:he_shock}
\end{figure}
Total Lagrangian MPM and DGMPM are used to solve this problem, and compared to the exact solution in figure \ref{fig:he_shock}.
%% MPM
The error made in the MPM solution grows during the computation, implying that the method is no longer stable in spite of the CFL number set to $0.5$, which ensured stability for the above problems.
Moreover, slight oscillations also appear after the shock in DGMPM solutions based on a unit Courant number (see figure \ref{fig:he_shock}\subref{subfig:he_shock1}) but not in the DGMPM-Euler using 2ppc due to the lower Courant number used.
%This numerical noise (figure \ref{fig:he_shock}\subref{subfig:he_shock1}) does not appear in the DGMPM-Euler using 2ppc due to the lower Courant number used.
Nevertheless, the oscillations do not increase with time as can be seen in figure \ref{fig:he_shock}\subref{subfig:he_shock2}, meaning that this noise is not an instability.
%% Capturing shocks
Next, despite a CFL number set to one, the DGMPM-Euler and DGMPM-RK2 do not perfectly capture the discontinuity, though a good behavior is shown.
Indeed, according to Lax entropy condition \eqref{eq:Lax_entropy}, the wave celerity used for the time step evaluation is an upper bound of the shock speed so that such a wave cannot be captured.
Nevertheless, the loading conditions are deliberately extreme for the purpose of visualization.
%% Lower shock
\begin{figure}[h!]
  \centering
  {\phantomsubcaption \label{subfig:he_low_shock1}}
  {\phantomsubcaption \label{subfig:he_low_shock2}}
  {\input{chapter4/pgfFigures/he_stress_5shock}}
  \caption{First Piola–Kirchhoff stress solution of the tensile impact problem at different times: $\Pi^d=-2\times 10^{9} \: Pa$. Comparison between MPM (CFL=$0.5$), DGMPM-Euler with 1ppc (CFL=$1$) or 2ppc (CFL=$0.5$), DGMPM-RK2 with 2ppc (CFL=$1$), and the exact solution.}
  \label{fig:he_low_shock}
\end{figure}

A lower tensile load leads to a numerical solution closer to analytical results as can be seen in figure \ref{fig:he_low_shock}, which shows the comparison for $\Pi^d=-2\times 10^{9} \: Pa$.
In addition, the oscillations appearing in DGMPM-Euler with 1ppc and DGMPM-RK2 with 2ppc solutions are almost eliminated in figure \ref{fig:he_low_shock}\subref{subfig:he_low_shock1}.
The MPM solution on the other hand, still exhibits an increasing error although a Courant number ensuring the stability in the scheme for linear problems is used.

% \subsubsection{Impact problem}

% \begin{figure}[h!]
%   \centering
%   %\input{chapter4/2dplaneWave}
%   \input{chapter4/2DplaneWave}
%   \caption{test}
%   \label{fig:2dplane_Wave}
% \end{figure}

% \begin{figure}[h!]
%   \centering
%   \input{chapter4/pgfFigures/HE_PW_energies}
%   \caption{Evolution of total energy}
%   \label{fig:pw_energy}
% \end{figure}

%%% Local Variables:
%%% mode: latex
%%% ispell-local-dictionary: "american"
%%% TeX-master: "../mainManuscript"
%%% End:


\input{PreambleCommon}
\input{WeekTitles}
\begin{document}

\newpage
\setfont
\pagestyle{fancy}
\renewcommand{\Week}{10}
\renewcommand{\WeekTitle}{\WeekTitleTen}

\fancyhead[LE,RO]{Week \Week}  % default, usually only for first page
\fancyfoot{}
\sectionbox{Week \#\Week: \WeekTitle}

\vspace{5mm}
\goals
\begin{itemize}
\item Express vectors, linear combinations, and compute dot products.
\item Write information in matrix form in the context of engineering applications.
\item Understand the definition of the transpose, and use MATLAB to compute it.
\item Use MATLAB to compute the inverse of a matrix.
\end{itemize}

\newpage
\topic{Linear Algebra - Introduction}
\subsection*{Introduction to Vectors}
In a physical context we often need to describe measurable quantities such as pressure, mass, and speed, and these objects can be completely described by a single number known as the \textbf{magnitude}. However there are other quantities which help us describe the world around us such as force, velocity and acceleration that require more than magnitude to describe them. They also need a \textbf{direction}. A \textbf{vector} is a magnitude (a number describing how much, how fast, etc) in combination with a direction. Vectors are denoted by boldface letters, such as $\overrightarrow{u}, \overrightarrow{v}, \overrightarrow{x}, \overrightarrow{y}, \overrightarrow{z}$.

\newpage

When discussing direction, we need to decide on a frame of reference. We will be using the Cartesian coordinate system. A \textit{point} in the two-dimensional Cartesian plane, denoted as $\mathbb{R}^2$ is denoted by an ordered pair (x,y) of real numbers, which we call \textit{coordinates}. So for example, (2,1) can be represented as a point on a grid like so:

\newpage

Consider the point (x,y) in $\mathbb{R}^2$. If we draw a directed line segment from the \textbf{origin} (the point (0,0)) to (x,y), we get the following picture:
``insert picture here"

Notice there is an arrow pointing at (x,y). We call this the \textbf{head} of the vector and the \textbf{tail} is at the origin, indicating a direction. The \textbf{magnitude} of the vector is its length from (0,0) to (x,y).

\newpage

To find the magnitude of a vector, we first find its \textbf{components}. If the head of the vector is at the point (x', y') and the tail of the vector is at the point (x,y), then the components of the vector are $x'-x$ and $y'-y$, and we represent them in the following way:

$$ \begin{bmatrix}x'-x\\y'-y\end{bmatrix} $$

Once you have found the components, the formula for finding the magnitude of the vector is:
$$\text{magnitude} = \sqrt{(x'-x)^2 + (y'-y)^2} $$

\newpage

Draw and find the magnitude of the following vectors:$ \overrightarrow{a}$ has the tail at (-1,1) and the head at (4,3), $\overrightarrow{b}$ has the tail at (0,4) and the head at (-2,-2), and $\overrightarrow{c}$ has the tail at (3,0) and the head at (0,3). We use the notation $||\overrightarrow{a}||$ to represent the magnitude of $\overrightarrow{a}$.

``Insert picture here"

\newpage

An interesting fact to note is that two vectors are \textbf{equal} if their components are equal. Consider the vector $\overrightarrow{A}$ going from (-1,4) to \\ \noindent (-3,1) (tail to head), $\overrightarrow{B}$ going from (2,3) to (0,0) and $\overrightarrow{C}$ going from (5,5) to (3,2). All of these vectors occupy the different regions of $\mathbb{R}^2$, yet we can define them as equal since they have the same components. A \textbf{position vector} is a vector whose tail starts at the origin, and if a vector is given with just its components, you may assume it is a position vector.
\newpage
\subsection*{Vector Addition and Scalar Multiplication}

If we have two vectors in $\mathbb{R}^2$ with components
$$ \overrightarrow{x} = \begin{bmatrix}x_1\\x_2\end{bmatrix} \text{ and } \overrightarrow{y} = \begin{bmatrix}y_1\\y_2\end{bmatrix}$$

then the \textbf{sum} of the vectors \textbf{x} and \textbf{y} is

$$ \overrightarrow{x} + \overrightarrow{y} = \begin{bmatrix}x_1+y_1\\x_2+y_2\end{bmatrix} $$

\newpage

So if we have vectors

$$ \overrightarrow{x} = \begin{bmatrix}1\\1\end{bmatrix}, \text{ } \overrightarrow{y} = \begin{bmatrix}-2\\5\end{bmatrix}, \text{ and } \overrightarrow{z} = \begin{bmatrix}4\\-1\end{bmatrix}$$

Then $$ \overrightarrow{x} + \overrightarrow{y} = \begin{bmatrix}1+(-2)\\1+5  \end{bmatrix} = \begin{bmatrix}
-1\\6
\end{bmatrix} $$

Or we could have

$$ \overrightarrow{y} + \overrightarrow{z} = \begin{bmatrix}-2+4\\5+(-1)  \end{bmatrix} = \begin{bmatrix}
2\\4
\end{bmatrix} $$

\newpage

We can interpret vector addition geometrically.

``Insert picture here''

\newpage

If $\overrightarrow{x} = \begin{bmatrix}
x_1\\x_2
\end{bmatrix}$ is a vector and $c$ is a scalar (meaning a real number), then the \textbf{scalar} multiple $c\overrightarrow{x}$, meaning every component of $\overrightarrow{x}$ is multiplied by $c$, is $c\overrightarrow{x} = \begin{bmatrix}
cx_1\\cx_2
\end{bmatrix}$. If $c>0$, then $c\overrightarrow{x}$ is in the same direction as $\overrightarrow{x}$. If $c<0$, then $c\overrightarrow{x}$ is in the opposite direction as $\overrightarrow{x}$.

``Insert picture here"

\newpage

For example, if $c = -1$, $d = 2$ and $\overrightarrow{y} = \begin{bmatrix}
4\\-1
\end{bmatrix}$ (Recall that $\overrightarrow{y}$ is a position vector, whose tail starts at the origin), then $c\overrightarrow{y} =\begin{bmatrix}(-1)\cdot(4)\\(-1)\cdot(-1)\end{bmatrix} = \begin{bmatrix}
-4\\1
\end{bmatrix}$ and $d\overrightarrow{y}= \begin{bmatrix}(2)\cdot(4)\\(2)\cdot(-1)\end{bmatrix} = \begin{bmatrix} 8\\-2
\end{bmatrix}$

``insert picture here"

\newpage
\subsection*{3-Dimensional Vectors}

3-dimensional (also known as 3-space) vectors exist in $\mathbb{R}^3$, meaning that we are now dealing another axis, the z-axis. Just like in $\mathbb{R}^2$, there is an origin where all of the axes meet, (0,0,0). Points in $\mathbb{R}^3$ are represented by an ordered triplet (x, y, z). The points (2,1,3), (4, -2, -2), and (1,0,5) would be drawn like so:

"Insert Picture Here"

\newpage

Components for 3-dimensional vectors are defined in the same way as 2-dimensional vectors, except that now there are three of them. So for a vector $\overrightarrow{u}$ whose tail starts at the point (x,y,z) and has its head at the point (x',y',z'), the components of $\overrightarrow{u}$ are:

$$\begin{bmatrix}
x'-x\\y'-y\\z'-z
\end{bmatrix} $$

A 3-dimensional vector whose components are given without information about the head or the tail is a position vector, and you can assume its tail starts at the origin.

\newpage

All of the rules for vector addition and scalar multiplication we presented for 2-dimensional vectors are the same for the 3-dimensional versions. If we have position vector $\overrightarrow{a} = \begin{bmatrix}
a_1\\a_2\\a_3
\end{bmatrix}$ then the magnitude of $\overrightarrow{a}$ is
$$||\overrightarrow{a}|| = \sqrt{a_1^2 + a_2^2 + a_3^2}$$

Keep in mind that we get this formula from the fact that the tail of $\overrightarrow{a}$ is at the origin (0,0,0).

\newpage

\end{document}


% !TEX root = main.tex

%------------------------------------------------
 \chapter{Second Order Equations}
A second order linear differential equation is an equation of the following form : $$\frac{d^2y}{dt^2} +a_1(t)\frac{dy}{dt} + a_0(t)y = f(t).$$ The functins $a_1(*)$, $a_0(*)$ and $f(*)$ are supposed to be known and we want to find all of the solutions to $y$.
\begin{example}
The Legendre equation is $$\frac{d^2y}{dt^2} - \frac{2t}{1-t^2}\frac{dy}{dt} + \frac{n(n+1)}{1-t^2}Y = 0.$$
The equation has a solution $P_n(t)$, the Legendre polynomial of degree $n$.
\begin{itemize}
\item For $n = 0$ we can take $P_0(t)$ for all $t$
\item For $n = 1$ we let $P_1(t) = at+b$. We get $$\frac{-2t}{1 - t^2}a + \frac{2}{1-t^2}(at+b) = 0$$ for all $t\in\mathbb{R}\backslash \{\pm 1\}$ This gives $b=0$ and we can take $a = 1$ to get $P_1(t) = t$
\end{itemize}
\end{example}
\begin{example}
Bessel's equation of order $n$ is : $$\frac{d^2y}{dt^2} + \frac{1}{t}\frac{dy}{dt}+(1 - \frac{n^2}{t^2})y = 0.$$
The solution are Bessel functions, $J_n(t)$ and $Y_n(t).$
\end{example}
\section{Equations with Constant Coefficients}
$$\frac{d^2y}{dt^2}+a_1\frac{dy}{dt}a_0y = f(t)$$
In order to solve the above equation we adopt a two step solution strategy: \begin{enumerate}
\item Find all the solution of the equation with $f = 0$. The equation when $f = 0$ is called the homogeneous equation
\item Develop a formula called the variation of parameters formula for the case $f \not = 0$, using the solutions of the homogeneous equation
\end{enumerate}
\subsubsection*{Solution Method for $\mathbf{f = 0}$}
We look for a solution of the form $y(t) = A\exp(\mu t)$, where $A$ and $\mu$ are constants. We have: $$\frac{dy}{dt} = \mu A\exp(\mu t) \quad \mbox{      and      } \quad \frac{d^2y}{dt^2} = \mu^2 A \exp(\mu t).$$
Substituting into the differential equation gives: $$(\mu^2 + a_1\mu + a_0) * A\exp(\mu t) = 0.$$
Since $\exp(\mu t ) \not = 0$ and we do not want the trivial solution which comes from choosing $ A = 0$, we have $\mu^2 + a_1\mu +a_0 = 0$ this is called the auxiliary quadratic. The possibilities are:
\begin{itemize}
\item Two distinct roots
\item One double root
\end{itemize}
If $a_1$ and $a_0$ are real there is an alternative dichotomy:
\begin{itemize}
\item Two real roots (which may or may not coincide)
\item A complex conjugate pair of distinct roots
\end{itemize}
\subsubsection*{Case 1 : Distinct Roots $\mathbf{\mu_1 \not = \mu_2}$}
$$\mu^2 +a_1\mu +a_0 = (\mu - \mu_1)(\mu-\mu_2).$$ We have the solutions $$y_j(t) = A_j \exp(\mu_j t) \quad j = 1, 2.$$ For completely arbitary constants $A_1$, $A_2$. Suppose $y(t) = y_1(t) + y_2(t)$ then $$\frac{dy}{dt} = \frac{dy_1}{dt} + \frac{dy_2}{dt} \quad \mbox{   and   } \quad  \frac{d^2y}{dt^2} = \frac{d^2y_1}{dt^2} + \frac{d^2y_2}{dt^2}.$$
So, $$\frac{d^2y}{dt^2} + a_1\frac{dy}{dt} + a_0y = [\frac{d^2y_1}{dt^2} + a_1\frac{dy_1}{dt} +a_0y_1] +  [\frac{d^2y_2}{dt^2} + a_1\frac{dy_1}{dt} + a_0y_2].$$
 \begin{theorem}

Suppose that the auxiliary quadratic has a distinct root $\mu_1 \not = \mu_2$. Then every solution of the homogeneous equation has the form $$y(t) = A_1 \exp(\mu_1t) + A_2 \exp(\mu_2t)$$ for some $A_1$, $A_2 \in \mathbb{C}.$
\end{theorem}
\begin{example}
$$\frac{d^2y}{dt^2} - (\mu_1+\mu_2)\frac{dy}{dt} + \mu_1\mu_2y = 0$$
such that $y(0) = y(0) \mbox{ and } y'(0) = v_0.$
\begin{solution}

We know $y(t) = A_1 \exp(\mu_1t) + A_2 \exp(\mu_2t)$ and we choose $A_1$, $A_2$ so that:
$$
\begin{cases}
y_0 = y(0) = A_1 + A_2 \\
v_0 = y'(0) = A_1\mu_1 +A_2\mu_2
\end{cases} $$
Hence $$\mu_2y_0 - v_0 = A_1(\mu_2 - \mu_1) \implies$$
$$
\begin{cases}
A_1 = \frac{\mu_2y_0 - v_0}{\mu_2 - \mu_1} \\
A_2 = y_0 - A_1 = \frac{v_0 - \mu_1y_0}{\mu_2 - \mu_1}
\end{cases} $$
Thus $$y_1(t) = \frac{\mu_2y_0 - v_0}{\mu_2 - \mu_1} \exp(\mu_1t) + \frac{v_0 - \mu_1y_0}{\mu_2 - \mu_1} \exp(\mu_2t) = y_0\bigg(\frac{\mu_2\exp(\mu_1t) - \mu_1\exp(\mu_2t)}{\mu_2-\mu1}\bigg) + v_0\bigg(\frac{\exp(\mu_2t) - \exp(\mu_1t)}{\mu_2-\mu_1}\bigg).$$
\end{solution}
\end{example}
\begin{example}
In the previous example let $\mu_1$ be fixed and find $\lim_{\mu_2 \to \mu_1} y(t)$.
\begin{solution}
First we compute $$\lim_{\mu_2 \to \mu_1}\bigg[\frac{\exp(\mu_2t) - \exp(\mu_1t)}{\mu_2 - \mu_1}\bigg] = t\exp(\mu t)\lim_{\mu_2 \to \mu_1}\bigg[\frac{\exp((\mu_2 - \mu_1)t - 1}{(\mu_2 - \mu_1)t}\bigg].$$
Now set $ h = \mu_2 - \mu_1$. Thus : $$t\exp(\mu_1t)\lim_{h \to 0}\bigg[\frac{\exp(h) - 1}{h}\bigg]$$ $$= t\exp(\mu_1)\lim_{h \to 0}\bigg[\frac{\exp(h) - \exp(0)}{h}\bigg] = t\exp(\mu_1t)\exp'(0).$$
Where $\exp'(0) = 0$. Hence $$\lim_{\mu_2 \to \mu_1}\bigg(\frac{\exp(\mu_2t) - \exp(\mu-1t)}{\mu_2 - \mu_1}\bigg) = t\exp(\mu_1t).$$
Next we compute:
$$\lim_{\mu_2 \to \mu_1}\bigg(\frac{\mu_2\exp(\mu_1t) - \mu_1\exp(\mu-2t)}{\mu_2 - \mu_1}\bigg).$$
We observe that:
$$\frac{\mu_2\exp(\mu_1t) - \mu_1\exp(\mu-2t)}{\mu_2 - \mu_1} = \bigg[\frac{\mu_1(\exp(\mu_2t) - \exp(\mu_1t)) + (\mu_2 - \mu_1) \exp(\mu_1t)}{\mu_2 - \mu_1}\bigg]$$ $$= \exp(\mu_1t) - \mu_1 \bigg[\frac{\exp(\mu_2t -\exp(\mu_1t)}{\mu_2 - \mu_1}\bigg].$$
Thus $$\lim_{\mu_2 \to \mu_1}\bigg(\frac{\mu_2\exp(\mu_1t) - \mu_1\exp(\mu-2t)}{\mu_2 - \mu_1}\bigg) = \exp(\mu_1t) - \mu_1t\exp(\mu_1t).$$
Hence $$\lim_{\mu_2 \to \mu_1} y(t) = y_0(1 - \mu_1 )\exp(\mu_1t) + v_ t \exp(\mu_1t).$$
Let $$z(t) = y_0(1 - \mu_1t )\exp(\mu_1t) + v_0 t \exp(\mu_1t).$$ Then
$$
\begin{cases}
z(0) = y_0 \\
z'(t) = y_0(-\mu_1 + \mu_1(1 - \mu_1t)) \exp(\mu_1t) + v_0(1+\mu_1t)\exp(\mu_1t)
\end{cases}
$$
giving $z'(0) = v_0.$
Let $z(t) = (At+B)\exp(\mu_1t)$, where $A$ and $B$ are constants. We want to check that $$\frac{d^2z}{dt^2}-2\mu_1\frac{dz}{dt}+{\mu^2}_1z = 0.$$
We have $$z(t) = (At+B)\exp(\mu_1t) \implies \frac{dz}{dt} = (A +\mu_1(At+B))\exp(\mu_1t) = (\mu_1At + (A + \mu_1 B))\exp(\mu_1t)$$ $$\implies \frac{d^2z}{dt2} = (\mu_1At + (\mu_1A +\mu_1(A+\mu_1B))\exp(\mu_1).$$
Hence
$$\frac{d^2z}{dt^2}-2\mu_1\frac{dz}{dt}+{\mu^2}_1z = (({\mu^2}_1A - 2\mu_1*\mu_1A +{\mu^2}_1A)t + 2\mu_1A +{\mu^2}_1B - 2\mu_1(A+\mu_1B) +{\mu^2}-1B)\exp(\mu_1t) = 0 \mbox{ as required.}$$
\end{solution}
\end{example}
\begin{theorem}
Every solution of the equation $$\frac{d^2y}{dt^2}-2y\frac{dy}{dt} +\mu^2y = 0$$ has the form $$y(t) = (At+B)\exp(\mu t)$$ for some appropriate constants $A$, $B$.
\end{theorem}
\begin{example}
Find the general solution of $$\frac{d^2y}{dt^2} + \omega^2y = 0$$ where $\omega>0$ is constant.
\begin{solution}
We look for a solution of the form $$y(t) = A\exp(\mu t)$$ and obtain the auxiliary quadratic $\mu^2 + \omega^2 = 0$, which has roots $\mu_1 = i\omega$, $\mu_2 = -i\omega$. Thus the general solution has the form $$y(t) = A\exp(i\omega t) + B\exp(-i\omega t)$$
$$ = A(\cos(\omega t ) +i\sin(\omega t)) + B(\cos(\omega t) - i\sin(\omega t))$$ $$= (A+B)\cos(\omega t) +i(A-B)\sin(\omega t).$$
Suppose $y(0) = y_0 \in \mathbb{R}$ and $y'(0) = v_0 \in \mathbb{R}.$ Then
$$
\begin{cases}
y(0) = A+B = y_0 \\
y'(0) = i\omega(A-B) = V_0
\end{cases} $$
Thus $y(t) = y_0\cos(\omega t) + v_)\frac{\sin(\omega t)}{\omega}.$
\end{solution}
\end{example}
\begin{remark}
Observe that $\lim_{\omega \to 0}\cos(\omega t) = \cos(0) = 1 \mbox{ and } \lim_{\omega \to 0}\frac{\sin(\omega t)}{\omega} = t$. When $\omega = 0$, we get $$y(t) = y_0 + v_0t$$ which does indeed solve $\frac{d^2y}{dt^2} = 0$.
\end{remark}
\begin{definition}
Let $y_1$, $y_2$ be solutions of $$\frac{d^2y}{dt^2} + a_1(t)\frac{dy}{dt} +a_0(t)y=0.$$ We say that $y_1$ and $y_2$ are linearly independent if and only if neither solution can be written as a constant multiple of the other.
\end{definition}
\begin{example}
In the previous example, $$y_1(t) = \cos(\omega t) \mbox{ and }  y_2 = \frac{\sin(\omega t)}{\omega}$$ are linearly independent: so are $$y_1(t) = \exp(i \omega t) \mbox{ and } y_2(t) = \exp(-i \omega t)$$  $$y_1(t) = \cos(\omega t) \mbox{ and } y_2(t) = \exp(i\omega t).$$ However if $y_1t = \cos(\omega t)$ and $y_2(t) = 17 \cos(\omega t)$, are not linearly independent.
\end{example}
\begin{definition}
Let $y_1$ and $y_2$ be two differentiable functions. The Wronskian (determinant) of $y_1$ and $y_2$ is the function $$W = \det \begin{bmatrix}
y_1 \hspace{4pt} y_2 \\
y'_1 \hspace{4pt} y'_2 \\
\end{bmatrix}
= y_1y'_2 - y'_1y_2$$
\end{definition}
\subsubsection*{Objective}
We want to find a formula for the general solution of $$\frac{d^2y}{dt^2} + a_1\frac{dy}{dt} + a_0y = f$$ where the function $f$ on the right hand side is non-trivial. We assume that we know two lineralry independent solutions $y_1$ and $y_2$ of the homogeneous equation $$\frac{d^2y}{dt^2}j + a_1\frac{dy}{dt}j + a_0yj = 0 \mbox{ for } j = 1,2.$$ We can look for a solution in the form $$y(t) = A_1(t) +A_2(t)y_2(t).$$ Differentiating, $$y'(t) = A_1(t)y'_1(x) + A_2(t)y'_2(t) + A'_1(t)y_1(t) + A'_2(t)y_2(t). $$
We impose the condition $A'_1y_1 +A'_2y_2 = 0$. Then we obtain by the second derivative $$y''(t) = A_1(t)y''_1(t) +A_2y''_2(t) + A'_1(t)y'_1(t) +A'_2(t)y'_2(t).$$ Combining the expressions for $y$, $y'$ and $y''$, we obtain $$y'' + a_1y +a_0y = A_1(y''_1 + a_1y'_1 + a_0y1) + A_2(y''_2 + a_1y'_2 + a_0y2) +A'_1y'_1+A'_2y'_2 = A'_1y'_1+A'_2y'_2 $$ because $y_1$ and $y_2$ solve the homogeneous equation. Using the original formula we see that our second equation for $A_1$ and $A_2$ is $$A'_1y'_1+A'_2y'_2 = f.$$ We write the equation for $A_1$ and $A_2$ as $$\begin{bmatrix}
y_1 \hspace{10pt} y_2 \\
y'_1 \hspace{10pt} y'_2 \\
\end{bmatrix}
\begin{bmatrix}
A'_1 \\
A'_2 \\
\end{bmatrix}
= \begin{bmatrix}
0 \\
f \\
\end{bmatrix}
$$
which yields
$$ \begin{bmatrix}
A'_1 \\
A'_2 \\
\end{bmatrix} = \frac{1}{W}
\begin{bmatrix}
y'_2 \hspace{10pt} -y_2 \\
-y'_1 \hspace{10pt} y_1 \\
\end{bmatrix}
\begin{bmatrix}
0 \\
f \\
\end{bmatrix}
= \frac{1}{W}
\begin{bmatrix}
-y_2f \\
y_1f \\
\end{bmatrix}
$$ This means $$ \begin{cases}
A'_1 = \frac{-y_2 f}{W}\\
A'_2 = \frac{y_1 f}{W} \end{cases}$$ giving
$$
\begin{cases}
A_1(t) = \int_{t_0}^{t} \frac{-y_2(x) f(x)}{W(x)} dx + \alpha_1 \\

\bigskip

A_2(t) = \int_{t_0}^{t} \frac{y-1(x) f(x)}{W(x)} dx +\alpha_2\\
\end{cases}$$
Where $\alpha_1$ and $\alpha_2$ are constants of integration and $t_0$ can be chosen. Recalling that $$y(t) = A_1(t)y_1(t) + A_2(t)y_2(t)$$ we obtain
$$ y(t) = \int_{t_0}^{t} \bigg( \frac{y_1(x) y_2(t) - y_1(t) y_2(x)}{W(x)} \bigg) f(x)dx +\alpha_1 y_1(t) + \alpha_2 y_2(t).$$
This formula is called the variation of parameters or variation of constants formula.
\begin{itemize}
\item $y$ is the general solution of $$y''+a_1y'+a_0y = f$$
\item $a_1$, $a_0$ and $f$ are given functions.
\item $y_1$, $y_2$ are the solutions of the homogeneous equation $$y''_j +a_1y'_j +a_0y_j = 0, \hspace{6pt} j = 1,2$$
\item $W$ is the Wronskian $$ W = \det \begin{bmatrix}
y_1 \hspace{5pt} y_2 \\
y'_1 \hspace{5pt} y'_2\\
\end{bmatrix} = y_1y'_2 - y'_1y_2$$
\end{itemize}
\begin{example}
Find the general solution of the equation $$y'' - y = 1.$$
\begin{solution}
The corresponding homogeneous equation is $y''-y = 0$, with auxiliary quadratic $\lambda^2 - 1 = 0$ having roots $\lambda = \pm 1$. Thus the functions $$\begin{cases}
y_1(x) = \exp(+1x) \\
y_2(x) = \exp(-1x) \\
\end{cases}$$
are two linearly independent solutions of the homogeneous equation.  Hence $$W = \det \begin{bmatrix}
y_1 \hspace{5pt} y_2 \\
y'_1 \hspace{5pt} y'_2\\
\end{bmatrix} = \det \begin{bmatrix}
\exp(x) \hspace{5pt} \exp(-x)\\
\exp(x) \hspace{5pt} -\exp(-x)\\
\end{bmatrix} = -2.$$ For this example $f(x) = 1$. The variation of parameters formula gives $$y(t) = \int_{t}^{t_0} \frac{\exp(x)\exp(-t)-\exp(t)exp(-x)}{(-2)}*1dx + \alpha_1\exp(t)+\alpha_2\exp(-t)$$ and we chose $t_0 = 0$ for convenience. Thus $$ y(t) = \frac{-1}{2}\exp(-t) \int_{0}^{t}exp(x)dx +\frac{1}{2}\exp(t)\int_{0}^{t}\exp(-x)dx +\alpha_1\exp(t) +\alpha_2\exp(-t)$$ which yields $$y(t) = \frac{-1}{2}\exp(-t)[exp(t)-1]+\frac{1}{2}\exp(t)[1-exp(-t)]+\alpha_1\exp(t)+\alpha_2\exp(-t)$$ $$ = -1+(\alpha_1 + \frac{1}{2})\exp(t) +(\alpha_2 +\frac{1}{2})\exp(-t).$$ Hence $$y(t) = -1 +A_1\exp(t) + A_2\exp(-t).$$ Where $A_1$ and $A_2$ are constants.
\end{solution}
\end{example}

\bigskip

\begin{example}
Find the general solution of $$\frac{d^2y}{dt^2} - 4\frac{dy}{dt} + 4y = t\exp(2t).$$
\begin{solution}
Our auxiliary quadratic is $$\lambda^2 -4\lambda + 4 = 0 \mbox{ or } (\lambda - 2)^2 = 0.$$ One solution of the homogeneous equation is $y_1(t) = \exp(2t)$ and the second is $y_2(t) = t\exp(2t).$ Thus $$ W = \begin{bmatrix}
\exp(2t) \hspace{44pt} t\exp(2t) \\
2\exp(2t) \hspace{12pt} (1+2t)\exp(2t)\\
\end{bmatrix} = \exp(4t)$$ Also $f(t) = t\exp(2t)$. The variation of parameters formula gives $$y(t) = \int_{0}^{t}\frac{\exp(2X)t\exp(2t)-\exp(2x)x\exp(2t)}{\exp(4(x)}x\exp(2x)dx + (\alpha_1+\alpha_2t)\exp(2t)$$ $$ =\int_{0}^{t}(tx\exp(2t) - x^2\exp(2t))dx + (\alpha_1+\alpha_2t)\exp(2t)$$ $$= t\exp(2t)\bigg[\frac{x^2}{2}\bigg]_{0}^{t} -\bigg[\frac{x^3}{3}\bigg]_{0}^{t}\exp(2t) +(\alpha_1+\alpha_2t)\exp(2t).$$ Finally $$y(t) = \frac{1}{6}t^3\exp(2t)+(\alpha_1+\alpha_2t)\exp(2t).$$
\end{solution}
\end{example}

\begin{theorem}
Let $y_1$ and $y_2$ be linearly independent solutions of $$\frac{d^2 y}{dt^2}+ a_1\frac{dy}{dt}+a_0y = 0 $$ and let $y_p$ be any solution of the equation $$\frac{d^2 y}{dt^2}+ a_1\frac{dy}{dt}+a_0y = f, $$
then every solution $y$ of this equation has the form $$y(t) =y_p (t) + A_1y_1(t) + A_2y_2(t)$$ for the appropriate constants $A_1$ and $A_2$, furthermore for every pair of constants $A_1$ $A_2$ the function $y$ defined above and solves the homogeneous equation.The Proof for this will be given in year 2.
\end{theorem}
\subsubsection*{Guidelines for Guessing $\mathbf{y_p}$}
\begin{enumerate}
\item If $f(t) = \alpha \exp (\beta t)$, where $\beta$ is not a root of the auxiliary quadratic, then there exists a particular solution $y_p$ of the form $$y_p(t) = \gamma \exp (\beta t).$$ This is proven below.
$$\frac{d^2y_p}{dt^t}=\beta^2 y_p$$ and so$$(\beta^2 + a_1\beta +a_0)y_p = \alpha \exp (\beta t)$$ by canceling the common factor $\exp (\beta t)$ we get $$(\beta^2 + a_1\beta +a_0)\gamma = \alpha \neq 0$$ since $\beta$ is not a root of the auxiliary quadratic. Hence,
$$\gamma = \alpha (\beta^2 + a_1\beta +a_0).$$
\item Suppose that the auxiliary quadratic has distinct root $\lambda_1 \neq \lambda_2$ and that $$f(t) = \alpha \exp (\lambda_1 t)$$ then there exists a solution of the form $$y_p(t) \gamma t \exp (\lambda_1 t).$$ This again is proven below. $$\frac{dy_p}{dt}=\gamma \exp (\lambda_1 t) [1 + \lambda_1 t]$$
$$\frac{d^2y_p}{dt^2}=\gamma \exp (\lambda_1 t) [2\lambda_1 + \lambda_1^2 t]$$ Hence, $$\frac{d^2 y_p}{dt^2}+ a_1\frac{dy_p}{dt}+a_0y_p $$ $$=\gamma \exp (\lambda_1 t) [2\lambda_1 + \lambda_1^2 t + a_1(\lambda_1 t +1) + a_0 t]$$ $$= \gamma \exp (\lambda_1 t) [t (\lambda_1^2 + a_1\lambda_1 + a_0) + 2\lambda+1 +a_1]$$
we want this to be equal to $$f(t) = \alpha \exp (\lambda_1 t)$$ which as $$(\lambda_1^2 + a_1\lambda_1 + a_0) = 0 $$ this holds precisely when $$\gamma(2 \lambda_1 + a_1)= \alpha $$ as $a_1 = -\lambda_1 -\lambda_2$ and as a result $$\gamma (\lambda_1-\lambda_2) = \alpha$$ so
$$\gamma = \frac{\alpha}{(\lambda_1-\lambda_2}.$$
\item This deals with the case when the auxiliary quadratic has roots $\lambda_1 = \lambda_2$ and $$f(t)= \alpha t + \mu) \exp (\lambda_1 t)$$ this particular solution has the form  $$y_p(t) = q(t) \exp (\lambda_1 t) $$ of the form $$q (t) = a t^3  + bt^2$$
\item If $$f(t) = \alpha_1 \sin (\beta_1 t) + \alpha_2 \cos (\beta_2 t)$$ where $\beta_1$ and $\beta_2$ are real numbers and $i\beta_1$ and $i\beta_2$ are not solutions of the auxiliary quadratic. Then there exists a solution $$y_p(t) = [a_1 \sin (\beta_1 t) + c_1 \cos (\beta_1 t ) + a_2 \sin (\beta_2 t) + c_2 \cos (\beta_2 t ) ]$$ where $a_1, c_1, a_2, c_2$ are to be found.
\end{enumerate}
\section{Coupled Linear Systems}
We are interested in equations of the form $$\frac{dx}{dt} = ax+by$$ $$\frac{dy}{dt} = cx+dy$$ in which $a$, $b$, $c$ and $d$ are constant.
\subsubsection*{Solution Method 1}
Reduce to a single, second order equation. Differentiating the second equation you get $$\frac{d^2y}{dt^2}= c\frac{dx}{dt}+d\frac{dy}{dt} = c(ax+by)+d\frac{dy}{dt}$$ $$ = acx+bcy +d\frac{dy}{dt}.$$ From our second equation, if $c\neq 0$ then $$x = \frac{1}{c}\frac{dy}{dt}-\frac{d}{c}y$$ and so $$\frac{d^2y}{dt^2}= a\frac{dy}{dt}-ady+bcy+d\frac{dy}{dt}$$ or equivalently
$$\frac{d^2y}{dt^2}-(a+d)\frac{dy}{dt}+(ad-bc)y=0.$$
If we write this system as $$\begin{bmatrix}
\frac{dx}{dt}\\
\vspace{10pt}
\frac{dy}{dt}\\
\end{bmatrix} = \begin{bmatrix}
a \hspace{10pt} b \\
c \hspace{10pt} d \\
\end{bmatrix} * \begin{bmatrix}
x \\
y \\
\end{bmatrix} =:A$$ Then $$\frac{d^2y}{dt^2}-trace(A)\frac{dy}{dt}+\det(A)y = 0.$$
If $c \neq 0$ then $x$ is obtained from $$x = \frac{1}{c}\frac{dy}{dt}-\frac{d}{c}y.$$ If $c = 0$ and $b \neq 0$ then elimiate $y$ and obtain $$\frac{d^2x}{dt^2}-trace(A)\frac{dx}{dt}+\det(A)x = 0.$$ Then if $c=0=b$ and $$\frac{dx}{dt} = ax$$ $$\frac{dy}{dt} =dy$$ therefore they are no longer coupled.
\subsubsection*{Solution Method 2}
Let $z = \begin{bmatrix}
x \\
y
\end{bmatrix}$ so that $$\frac{dz}{dt}= Az, \hspace{15pt} A = \begin{bmatrix}
a \hspace{10pt} b \\
c \hspace{10pt} d \\
\end{bmatrix}$$ We look for a solution $z(t) = v \exp(\lambda t)$ where $v$ is a constant vector and $\lambda \in \mathbb{C}.$ Then $$ \frac{dz}{dt} = v \lambda\exp(\lambda t) \implies Az = Av \exp(\lambda t)$$ which is equivalent to $$Av = \lambda v.$$
This means that $v$ must be an eigenvector of $A$ with eigenvalue $\lambda.$ The eigenvalues of a matrix $A$ satisfy $$\det(A - \lambda I ) = 0 $$ or  $$\lambda^2 -(a+d)\lambda +(ad-bc) = 0 $$ this is the same as the auxiliary quadratic of second order equation arising from method one.

\begin{example}
\smallskip
Solve the coupled linear equations;
$$\frac{dz}{dt} = Az, \hspace{15pt} A=\begin{bmatrix}
-2 &1 \\
1 & -2 \\
\end{bmatrix}$$
\begin{solution}
The eigenvalues of $A$ solve, $$\lambda^2 +4\lambda +3 = 0 $$ this factorises to give $\lambda = -3$ and $-1$ then we need the eigenvector for each, first $\lambda = -1$ this eigenvector satisfies $$\begin{bmatrix}
-2- (-1) & 1\\
1 & 2-(-1)\\
\end{bmatrix}v =0$$ so we take  $$v = \alpha \begin{bmatrix}
\frac{1}{\sqrt[]{2}}\\
\frac{1}{\sqrt[]{2}}\\
\end{bmatrix} \hspace{15pt} \alpha \in \mathbb{C}.$$
Some true for $\lambda -3$ that gives
$$v = \alpha \begin{bmatrix}
\frac{1}{\sqrt[]{2}}\\
\frac{1}{\sqrt[]{2}}\\
\end{bmatrix} \hspace{15pt} \alpha \in \mathbb{C}$$
\end{solution}
\end{example}

%------------------------------------------------
\endinput


\documentclass[11pt, oneside]{amsart}   	% use "amsart" instead of "article" for AMSLaTeX format
\usepackage[top=0.3cm, bottom=1.3cm, left=1cm, right=1cm]{geometry}                		% See geometry.pdf to learn the layout options. There are lots.
\geometry{letterpaper}                   		% ... or a4paper or a5paper or ...
%\geometry{landscape}                		% Activate for rotated page geometry
%\usepackage[parfill]{parskip}    		% Activate to begin paragraphs with an empty line rather than an indent
\usepackage{graphicx}				% Use pdf, png, jpg, or eps§ with pdflatex; use eps in DVI mode
\usepackage{bm}							% TeX will automatically convert eps --> pdf in pdflatex
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{subcaption}
\usepackage{float}
\pagenumbering{gobble}

\title{MVA-DM3 Probabilistic graphical models}
\author{Hugo Cisneros}
\date{}							% Activate to display a given date or no date

\begin{document}
\maketitle
\section{Implementation - HMM}
\textbf{1.2} For a set of parameters $\theta$ in the Gaussian Mixture Hidden Markov model (GM-HMM), the complete likelihood can be written:
\begin{align*}
	l_c(\theta) &= \log\left(p(q_0)\prod_{t=0}^{T-1} p(q_{t+1}|q_t) \prod_{t=0}^T p(\overline{y}_t|q_t)\right)\\
		&= \sum_{i=1}^K \delta(q_0 = i)\log(\pi_i) + \sum_{t=0}^{T-1}\sum_{i,j=1}^K\delta(q_{t+1}=i, q_t=j)\log(A_{i,j})+  \sum_{t=0}^{T}\sum_{i=1}^K \delta(q_t = i) \log \mathcal{N}(\overline{y}_t|\mu_t, \Sigma_t)
\end{align*}
 For the EM algorithm, we wish to compute the parameters of the HMM with the observations $(\overline{y}_t)$. A lower bound of the likelihood is found by using Jensen's inequality with the $\log$ function, which gives $ \log p(\overline{y}_0, ..., \overline{y}_T) \geq \mathbb{E}_q[l_c(\theta)]$.

 We use the notations $\gamma(q_t)_i = p(q_t = i | \overline{y}, \theta)$ and $\xi(q_{t+1}, q_t)_{ij} = p(q_{t+1}=i, q_t=j |  \overline{y}, \theta)$. The expression $ \mathbb{E}_q[l_c(\theta)]$ becomes
 \[
 \mathbb{E}_q[l_c(\theta)] = \sum_{i=1}^K\gamma(q_0)_i\log(\pi_i) + \sum_{t=0}^{T-1}\sum_{i,j=1}^K\xi(q_{t+1}, q_t)_{ij}\log(A_{i,j})+  \sum_{t=0}^{T}\sum_{i=1}^K\gamma(q_t)_i \log \mathcal{N}(\overline{y}_t|\mu_t, \Sigma_t)
 \]
 By maximizing w.r.t $\pi$ and $A$ we get $\boxed{\pi_i = \dfrac{\gamma(q_0)_i}{\sum_{j=1}^K\gamma(q_0)_j}}$ and $\boxed{A_{i,j} = \dfrac{\sum_{t=0}^{T-1}\xi(q_{t+1}, q_t)_{ij}}{\sum_{i=1}^K \sum_{t=0}^{T-1}\xi(q_{t+1}, q_t)_{ij}}}$. The other derivations are similar to the ones from the Gaussian Mixture model in Homework 2.
 \\

 We have : $\boxed{\mu_i = \dfrac{\sum_{t=0}^T \gamma(q_t)_i \overline{y}_t}{\sum_{t=0}^T \gamma(q_t)_i}}$ and $\boxed{\Sigma_i = \dfrac{\sum_{t=0}^{T}  \gamma(q_t)_i  (\overline{y}_t - \mu_i)(\overline{y}_t - \mu_i)^T}{\sum_{t=0}^T \gamma(q_t)_i}}$ weighted sums of the empirical means and covariances.
 \begin{figure}[!h]
 \centering
 \begin{subfigure}{.4\textwidth}
 \centering
  \includegraphics[width=.8\linewidth]{full.pdf}
  \caption{Gaussian mixture}
 \end{subfigure}
   \begin{subfigure}{.4\textwidth}
    \centering
 \includegraphics[width=.8\linewidth]{training_gmhmm.pdf}
 \caption{GM-Hidden markov model}
 \end{subfigure}
 \caption{Comparison of mixture of gaussian profile and cluster assignment (most probable sequence of hidden state for HMM) for the Gaussian mixture model and GM-HMM.}
 \end{figure}

 \textbf{1.5} The GM-HMM learns a slightly different mixture configuration than the GM model. The assumption that data is temporally distributed is strong and yields some artifacts such as outliers assigned to a very unlikely hidden state. However, the likelihood on both training data and test data is much higher than for the Gaussian mixtrure:  -2101.4 and -2201 for GM-HMM against -2340.2 for the training set and -2431 for the test set for the GM model.

 \end{document}


\subsection{Definition of Abstract Map}
For any concrete map $\CMap$,
if abstract key $\AKey$ and abstract value $\AVal$ is defined,
and there exists a sound relation $\wsf{isRelated}$
that abstracts \emph{may equal} of two abstract key valaues,
we can define a abstract map $\AMap$.\\

$\begin{array}{l}
\poset(\CMap) \xrightarrow{\alpha_{m}}
\AMap \xrightarrow{\gamma_{m}} \poset(\CMap) \vspace{1mm} \\
\poset(\CKey) \xrightarrow{\alpha_{k}}
\AKey \xrightarrow{\gamma_{k}} \poset(\CKey) \\
\poset(\CVal) \xrightarrow{\alpha_{v}}
\AVal \xrightarrow{\gamma_{v}} \poset(\CVal) \vspace{1mm} \\
\wsf{isRelated} \in \AKey \times \AKey \rightarrow \textbf{Boolean} \\
\end{array}$\\\\
Formal definition of $\alpha_m$ and $\gamma_m$ are at section \ref{sec:galois}.\\
Our definition of abstract map consist of 2 parts:
\begin{enumerate}
\item $\Map$ part which is a partial map from $\AKey$ to $\AVal$, and
\item $\Defset$ part which is a set of definitely existing key values.
The size of $\Defset$ may be infinite,
for example, bottom of $\AMap$ has the whole $\CKey$ as its $\Defset$.
\end{enumerate}
$\begin{array}{l}
\hat{m},\ \langle \emph{map},\ \emph{defset} \rangle
\in \AMap = (\AKey \rightarrowtail \AVal)
\times \poset(\CKey)\\
\quad \Map(\langle \emph{map},\ \emph{defset} \rangle) =
\emph{map} \in \AKey \rightarrowtail \AVal \\
\quad \Defset(\langle \emph{map},\ \emph{defset} \rangle) =
\emph{defset} \in \poset(\CKey) \\
\quad \Dom(\langle \emph{map},\ \emph{defset} \rangle) =
\{ \hat{k} \mid \emph{map}
\textrm{ has a mapped value for key } \hat{k} \} \in \poset(\AKey) \\
\end{array}$

\subsubsection{Partial Order and Join}
$\begin{array}{rl}
\hat{m}_1 \po_m \hat{m}_2 = &
\forall \hat{k}_1 \in \Dom(\hat{m}_1) \cdot
\exists \hat{k}_2 \in \Dom(\hat{m}_2) :
\hat{k}_1 \po_k \hat{k}_2
\land \Map(\hat{m}_1)(\hat{k}_1) \po_v \Map(\hat{m}_2)(\hat{k}_2) \\
& \land\ \Defset(\hat{m}_2) \subseteq \Defset(\hat{m}_1) \\
\end{array}$\\\\
$\begin{array}{l}
\hat{m}_1 \sqcup_m \hat{m}_2 =
\langle \emph{map}_1 \cup \emph{map}_2 \cup \emph{map}_{\cap},\
\Defset(\hat{m}_1) \cap \Defset(\hat{m}_2) \rangle \\
\quad \begin{array}{rl} \textrm{where}
& \emph{map}_1 = \left \{~ [ \hat{k} \mapsto \Map(\hat{m}_1)(\hat{k}) ] ~\mid~
\hat{k} \in \Dom(\hat{m}_1)
\setminus \Dom(\hat{m}_2) ~\right \} \\
& \emph{map}_2 = \left \{~ [ \hat{k} \mapsto \Map(\hat{m}_2)(\hat{k}) ] ~\mid~
\hat{k} \in \Dom(\hat{m}_2)
\setminus \Dom(\hat{m}_1) ~\right \} \\
& \emph{map}_{\cap} = \left \{~ [ \hat{k} \mapsto
\Map(\hat{m}_1)(\hat{k}) \sqcup_v \Map(\hat{m}_2)(\hat{k}) ] ~\mid~
\hat{k} \in \Dom(\hat{m}_1)
\cap \Dom(\hat{m}_2) ~\right \} \\
\end{array} \\
\end{array}$

\subsubsection{Abstraction and Concretization} \label{sec:galois}
The abstract map $\AMap$ has Golois connection with concrete map $\CMap$.
\begin{center}
\textbf{Theorem }
$\forall \emph{mset} \in \poset(\CMap), ~
\forall \hat{m} \in \AMap, ~
\alpha_m(\emph{mset}) \po_m \hat{m}
\Longleftrightarrow
\emph{mset} \subseteq \gamma_m(\hat{m})$
\end{center}
$\gamma_m$ and $\alpha_m$ are defined as follow: \\

$\begin{array}{l}
\forall \emph{mset} \in \poset(\CMap) \cdot \\
\left \{
\begin{array}{l}
\alpha_m(\varnothing) = \bot_m \\
\quad \textrm{where } \bot_m = \langle \varnothing,\ \textbf{String} \rangle \\
\alpha_m(\emph{mset}) =
\langle \emph{map},\ \bigcap \left \{ \Dom(m) \mid m \in \emph{mset} \right \} \rangle \\
\quad \begin{array}{ll}\textrm{where}
& \emph{map} = \left \{
[ \hat{k} \mapsto \alpha_{v}( \{ m(k') \mid m \in \emph{mset}
\land k' \in \Dom(m) \land \alpha_k(\{k'\}) = \hat{k} \} ) ]
\mid \hat{k} \in \emph{S}  \right \} \\
& \emph{S} = \left \{
\alpha_k(\{k\}) \mid m \in \emph{mset} \land k \in \Dom(m) \right \}\\
\end{array}\\
\end{array} \right. \\\\

\forall \hat{m} \in \AMap \cdot \\
\quad \gamma_{m}(\hat{m}) = \bigcup \left \{ \emph{mset} ~\mid~
\alpha_{m}(\emph{mset}) \po_m \hat{m} \right \} \\
\end{array}$\\\\\\
\textbf{Monotonicity of $\alpha_m$}
\begin{center}
$\forall \emph{mset}_1, \emph{mset}_2 \in \poset(\CMap), ~
\emph{mset}_1 \subseteq \emph{mset}_2
\Rightarrow
\alpha_m(\emph{mset}_1) \po_m \alpha_m(\emph{mset}_2)$
\end{center}
\textbf{Proof }
$\forall \emph{mset}_1, \emph{mset}_2 \in \poset{\CMap},~
\emph{mset}_1 \subseteq \emph{mset}_2, ~$
\begin{itemize}
\item \textbf{If} $\emph{mset}_1 = \varnothing$,
since $\alpha_m(\emph{mset}_1) = \bot_m$,
$\alpha_m(\emph{mset}_1) \po_m \alpha_m(\emph{mese}_2)$.
\item \textbf{If} $\emph{mset}_2 = \varnothing$,
$\emph{mset}_1$ must be $\varnothing$ by $\emph{mset}_1 \subseteq \emph{mset}_2$.
Then $\alpha_m(\emph{mset}_1) = \bot_m \po_m \alpha_m(\emph{mset}_2)$.
\item\textbf{If} $\emph{mset}_1 \neq \varnothing$, and $\emph{mset}_2 \neq \varnothing$, \\
By definition of $\alpha_m$ and $\po_m$, we need to prove that
\begin{enumerate}[label=({\arabic*})]
\item $\forall \hat{k}_1 \in \emph{S}_1 \cdot \exists \hat{k}_2 \in \emph{S}_2:
\hat{k}_1 \po_k \hat{k}_2 \land \emph{map}_1(\hat{k}_1) \po_v \emph{map}_2(\hat{k}_2) ~ $
where \\
$\begin{array}{l}
\quad \emph{map}_1 = \left \{[ \hat{k}_1 \mapsto \alpha_{v}( \{ m_1(k_1') \mid m_1 \in \emph{mset}_1
\land k_1' \in \Dom(m_1) \land \alpha_k(\{k_1'\}) = \hat{k}_1 \} ) ]
\mid \hat{k}_1 \in \emph{S}_1  \right \}\\
\land~ \emph{map}_2 = \left \{[ \hat{k}_2 \mapsto \alpha_{v}( \{ m_2(k_2') \mid m_2 \in \emph{mset}_2
\land k_2' \in \Dom(m_2) \land \alpha_k(\{k_2'\}) = \hat{k}_2 \} ) ]
\mid \hat{k}_2 \in \emph{S}_2  \right \}\\
\land~ \emph{S}_1 = \{ \alpha_k(\{ k_1 \}) \mid m_1 \in \emph{mset}_1 \land k_1 \in \Dom(m_1) \} \\
\land~ \emph{S}_2 = \{ \alpha_k(\{ k_2 \}) \mid m_2 \in \emph{mset}_2 \land k_2 \in \Dom(m_2) \} \\
\end{array}$
\item $\bigcap\{ \Dom(m_2) \mid m_2 \in \emph{mset}_2\} \subseteq
\bigcap\{ \Dom(m_1) \mid m_1 \in \emph{mset}_1\}$
\end{enumerate}
Prove (1).
\begin{enumerate}[label={\arabic*})]
\item For arbitrary $\hat{k} \in \AKey$, $\alpha_v(\emph{vset}_1) \po_v \alpha_v(\emph{vset}_2)$ where \\
$\begin{array}{l}
\quad \emph{vset}_1 = \{m_1(k_1') \mid
m_1 \in \emph{mset}_1 \land k_1' \in \Dom(m_1) \land \alpha_k(\{k_1'\}) = \hat{k} \} \\
\land ~ \emph{vset}_2 = \{m_2(k_2') \mid
m_2 \in \emph{mset}_2 \land k_2' \in \Dom(m_2) \land \alpha_k(\{k_2'\}) = \hat{k} \} \\
\end{array}$\\
If there is no $k_1' \in \Dom(m_1) \land m_1 \in \emph{mset}_1$ such that $\alpha_k(\{k_1'\}) = \hat{k}$,
then $\emph{vset}_1 = \varnothing \subseteq \emph{vset}_2$.\\
If $m_1 \in \emph{mset}_1$ then $m_1 \in \emph{mset}_2$ as $\emph{mset}_1 \subseteq \emph{mset}_2$.
Therefore if there exists $k_1' \in \Dom(m_1) \land m_1 \in \emph{mset}_1$
such that $\alpha_k(\{k_1'\}) = \hat{k}$,
then $\emph{vset}_1 \subseteq \emph{vset}_2$.\\
By monotonicity of $\alpha_v$, $\alpha_v(\emph{vset}_1) \po_v \alpha_v(\emph{vset}_2)$.
\item $\{ \alpha_k(\{ k_1 \}) \mid m_1 \in \emph{mset}_1 \land k_1 \in \Dom(m_1) \}
\subseteq \{ \alpha_k(\{ k_2 \}) \mid m_2 \in \emph{mset}_2 \land k_2 \in \Dom(m_2) \}$ \\
as $\emph{mset}_1 \subseteq \emph{mset}_2$.
\end{enumerate}
By 1) and 2), $\forall \hat{k} \in \emph{S}_1,
\exists \hat{k} \in \emph{S}_2: \hat{k} \po_k \hat{k}
\land \emph{map}_1(\hat{k}) \po_v \emph{map}_2(\hat{k})$ \vspace{1mm}\\
Prove (2). It is obvious since $\emph{mset}_1 \subseteq \emph{mset}_2$. \vspace{1mm}\\
Thus $\alpha_m(\emph{mset}_1) \po_m \alpha_m(\emph{mset}_2)$.
\end{itemize}
\textbf{Join morphism of $\alpha_m$}
\begin{center}
$\forall \emph{msetset} \in \poset(\poset(\CMap)), ~
\alpha_m(\bigcup\emph{msetset})
=\bigsqcup_m\{ \alpha_m(\emph{mset}) \mid \emph{mset} \in \emph{msetset}\}$
\end{center}
\textbf{Proof } $\forall \emph{msetset} \in \poset(\poset(\CMap))$
\begin{itemize}
\item \textbf{If } $\bigcup \emph{msetset} = \varnothing$,
Then
\begin{enumerate}[label={\arabic*)}]
\item $\alpha_m(\bigcup \emph{msetset}) = \bot_m$.
\item $\forall \emph{mset} \in \emph{msetset},~ \emph{mset} = \varnothing$.\\
Thus $\bigsqcup_m\{\alpha_m(\emph{mset}) \mid \emph{mset} \in \emph{msetset}\}
= \bigsqcup_m \{ \bot_m \mid \emph{mset} \in \emph{msetset} \}
= \bot_m$.
\end{enumerate}
Therefore,
$\alpha_m(\bigcup \emph{msetset}) = \bot_m
= \bigsqcup_m\{\alpha_m(\emph{mset}) \mid \emph{mset} \in \emph{msetset}\} $.
\item \textbf{If }
$\bigcup\emph{msetset} \neq \varnothing$,
let $\hat{m}_l = \alpha_m(\bigcup \emph{msetset})$
and $\hat{m}_r = \bigsqcup_m\{ \alpha_m(\emph{mset}) \mid \emph{mset} \in \emph{msetset} \}$.
Then we need to prove $\Map(\hat{m}_l) = \Map(\hat{m}_r)
\land \Defset(\hat{m}_l) = \Defset(\hat{m}_r)$. \vspace{1mm} \\
Prove $\Map(\hat{m}_l) = \Map(\hat{m}_r)$.
\begin{enumerate}[label={\arabic*)}]
\item $\Dom(\hat{m}_l) = \Dom(\hat{m}_r)$. \vspace{1mm} \\
$\Dom(\hat{m}_l) = \{ \alpha_k(\{k\}) \mid m \in \bigcup\emph{msetset} \land k \in \Dom(m) \}$
by definition of $\alpha_m$.\\
For arbitrary $\hat{m}_1, \hat{m}_2 \in \AMap$,
$\Dom(\hat{m}_1 \sqcup_m \hat{m}_2) = \Dom(\hat{m}_1) \cup \Dom(\hat{m}_2)$
by definition of $\sqcup_m$.
Thus $\Dom(\hat{m}_r) = \bigcup \{ \Dom(\alpha_m(\emph{mset})) \mid \emph{mset} \in \emph{msetset} \}$, and \\
$\Dom(\hat{m}_r ) = \{ \alpha_k(\{k\})
\mid k \in \Dom(m) \land m \in \emph{mset} \land \emph{mset} \in \emph{msetset} \}$
by definition of $\alpha_m$. \vspace{1mm} \\
Therefore $\Dom(\hat{m}_l) = \Dom(\hat{m}_r)$.
\item $\forall \hat{k} \in \Dom(\hat{m}_l) \cdot \Map(\hat{m}_l)(\hat{k}) = \Map(\hat{m}_r)(\hat{k})$.\\
$\begin{array}{ll}
& \Map(\hat{m}_l)(\hat{k}) \\
= & \alpha_v(\{ m(k') \mid m \in \bigcup \emph{msetset} \land k' \in \Dom(m) \land \alpha_k(\{k'\}) = \hat{k} \})
\textrm{ by definition of } \alpha_m.\\
= & \bigsqcup_v\{ \alpha_v(\{ m(k') \mid m \in \emph{mset} \land k' \in \Dom(m) \land \alpha_k(\{k'\}) = \hat{k} \})
\mid \emph{mset} \in \emph{msetset} \} \\
& \textrm{ by Join morphism of } \alpha_v.\\
\end{array}$ \vspace{1mm} \\
$\forall \hat{k} \in \Dom(\hat{m}_l) \cdot \hat{k} \in \Dom(\hat{m}_r)$ by 1). \vspace{1mm}\\
$\begin{array}{ll}
& \Map(\hat{m}_r)(\hat{k}) \\
= & \bigsqcup_v\{ \Map(\hat{m})(\hat{k}')
\mid \hat{k}' = \hat{k} \land \hat{k}' \in \Dom(\hat{m})
\land \hat{m} = \alpha_m(\emph{mset}) \land \emph{mset} \in \emph{msetset} \} \\
& \textrm{ by definition of } \sqcup_m.\\
= & \bigsqcup_v \left \{ \emph{map}(\hat{k})
\middle| \begin{array}{l}
\emph{map} = \left \{ \left[ \hat{k} \mapsto \alpha_v \left(
\left \{ m(k') \middle| \begin{array}{l}
m \in \emph{mset}\\
\land k' \in \Dom(m)\\
\land \alpha_k(\{k'\}) = \hat{k}\end{array} \right \}
\right) \right] \middle| \hat{k} \in \emph{S} \right \} \\
~ \land \hat{k}' = \hat{k} \land \hat{k}' \in \emph{S} \\
~ \land \emph{S} = \{ \alpha_k(\{k\}) \mid k \in \Dom(m) \land m \in \emph{mset} \}
\land \emph{mset} \in \emph{msetset}\\
\end{array} \right \} \\
& \textrm{ by definition of } \alpha_m.\\
= & \bigsqcup_v \{ \alpha_v(\{ m(k') \mid k' \in \Dom(m) \land \alpha_k(\{ k' \}) = \hat{k}
\land m \in \emph{mset} \}) \mid \emph{mset} \in \emph{msetset}\}\\
\end{array}$ \vspace{1mm}\\
Therefore $\forall \hat{k} \in \Dom(\hat{m}_l) = \Dom(\hat{m}_r) \cdot
\Map(\hat{m}_l)(\hat{k}) = \Map(\hat{m}_r)(\hat{k})$.
\end{enumerate}
By 1) and 2), $\Map(\hat{m}_l) = \Map(\hat{m}_r)$. \vspace{2mm} \\
Prove $\Defset(\hat{m}_l) = \Defset(\hat{m}_r)$. \vspace{1mm} \\
$\Defset(\hat{m}_l) = \bigcap \left \{ \Dom(m) \mid m \in
\bigcup \emph{msetset} \right \}$ and \\
$\Defset(\hat{m}_r) = \bigcap \left \{ \bigcap \{ \Dom(m) \mid m \in \emph{mset} \} \mid
\emph{mset} \in \emph{msetset} \right \}$ by definition of $\alpha_m$ and $\sqcup_m$. \vspace{1mm} \\
We can simply write it again as follow: \vspace{1mm} \\
$\bigcap \left \{ \Dom(m) \mid m \in \bigcup \emph{msetset} \right \} \\
= \bigcap \{ \Dom(m) \mid m \in \emph{mset} \land \emph{mset} \in \emph{msetset} \} \\
= \bigcap \left \{ \bigcap \{ \Dom(m) \mid m \in \emph{mset} \} \mid \emph{mset} \in \emph{msetset} \right \}$
\vspace{1mm} \\
Thus $\Defset(\hat{m}_l) = \Defset(\hat{m}_r)$. \vspace{1mm} \\
To sum up, $\hat{m}_l = \hat{m}_r$. $i.e.$,
$\alpha_m(\bigcup \emph{msetset} )
=\bigsqcup_m\{ \alpha_m(\emph{mset}) \mid \emph{mset} \in \emph{msetset}\}$.
\end{itemize}
\textbf{Proof of Theorem }
$\forall \emph{mset} \in \poset(\CMap), ~
\forall \hat{m} \in \AMap, ~$
\begin{enumerate}[label={(\arabic*)}]
\item
$\alpha_m(\emph{mset}) \po_m \hat{m}
\Rightarrow
\emph{mset} \subseteq \gamma_m(\hat{m})$ \\
It is obvious by the definition of $\gamma_m$. \hfill $\square$
\item
$\emph{mset} \subseteq \gamma_m(\hat{m})
\Rightarrow
\alpha_m(\emph{mset}) \po_m \hat{m}$ \\
$\emph{mset} \subseteq \gamma_m(\hat{m})
\Rightarrow \alpha_m(\emph{mset}) \po_m \alpha_m(\gamma_m(\hat{m}))$
by \textbf{Monotonicity of $\alpha_m$}.\\
$\alpha_m(\gamma_m(\hat{m})) =
\alpha_m(\bigcup\{ \emph{mset}' \mid \alpha_m(\emph{mset}') \po_m \hat{m}\}) =
\bigsqcup_m\{\alpha_m(\emph{mset}') \mid \alpha_m(\emph{mset}') \po_m \hat{m} \}$
by the definition of $\gamma_m$ and \textbf{Join morphism of $\alpha_m$}. \\
$\bigsqcup_m\{\alpha_m(\emph{mset}') \mid \alpha_m(\emph{mset}') \po_m \hat{m} \}
\po_m \hat{m}$, therefore $\alpha_m(\emph{mset}) \po_m \hat{m}$. \hfill $\square$
\end{enumerate}