\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{physics}
\usepackage{amsmath}
\usepackage{showlabels}
\usepackage{amssymb}
\title{Statistical Computing for Scientists and Engineers\\[1em] Homework 4}
\author{Jiale Shi}
\date{Oct/29/2018}

\usepackage{natbib}
\usepackage{graphicx}
\usepackage{array}
\begin{document}
\maketitle

\newpage
\section{Accept-Reject}
Generate samples of a standard normal distribution, $f(x) \sim N(0,1)$, using the accept-reject method with a double-exponential proposal distribution, $g(x|\alpha) = (\alpha/2)\exp(-\alpha |x|)$.

(a) Derive the upper bound for the likelihood ratio, $M=f(x)/g(x)$ and show that the ideal acceptance rate is obtained when $\alpha=1$

Solution:
a standard normal distribution, $f(x) \sim N(0,1)$

\begin{equation}
    f(x|0,1) = \frac{1}{\sqrt{2\pi}}\exp(-\frac{x^2}{2})
\end{equation}

\begin{equation}
    M = \frac{f(x)}{g(x)} = \frac{\frac{1}{\sqrt{2\pi}}\exp(-\frac{x^2}{2})}{(\alpha/2)\exp(-\alpha |x|)} = \frac{\sqrt{2}}{\alpha \sqrt{\pi}} \exp{-\frac{|x|^2}{2}+\alpha |x|}
\end{equation}

The ratio $M$ is max at $|x|=\alpha$.

\begin{equation}
    M =  \frac{\sqrt{2}}{\alpha \sqrt{\pi}} \exp{\frac{\alpha^2}{2}}
\end{equation}

\begin{equation}
\begin{aligned}
    &\pdv{M}{\alpha} = \frac{\sqrt{2}}{\sqrt{\pi}} \exp{\frac{\alpha^2}{2}} (1-\frac{1}{\alpha^2}) = 0 \\
    & \alpha = 1, M' = \frac{\sqrt{2}}{\sqrt{\pi}} \exp{\frac{1}{2}}
\end{aligned}
\end{equation}

(b) Implement the accept-reject method and plot the true PDF and the proposal distribution for $\alpha=1$ super-imposed on to the normalized histogram of your samples.

Solution:

$\alpha = 1$ and
$M' = \frac{\sqrt{2}}{\sqrt{\pi}} \exp{\frac{1}{2}}$

\begin{figure}[h!]
\centering
\includegraphics[scale=0.4]{h4p1b1.png}
\caption{$<E>$- iteration for $\alpha = 1$}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.4]{h4p1b2.png}
\caption{COV- iteration for $\alpha = 1$}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.4]{h4p1b3.png}
\caption{ the true pdf and the proposal distribution for $\alpha = 1$}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.4]{h4p1b4.png}
\caption{histogram of samples for $\alpha = 1$}
%\label{fig:universe}
\end{figure}

\newpage
(c) Repeat part (b) but now use a sub-optimal proposal distribution with $\alpha=2$, plot both distributions and your histogram. How do the acceptance rates compare?

Solution:

$\alpha = 2$ and
$M' = \frac{\sqrt{2}}{2\sqrt{\pi}} \exp{2}$

\begin{figure}[h!]
\centering
\includegraphics[scale=0.4]{h4p1c1.png}
\caption{$<E>$- iteration for $\alpha = 1$}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.4]{h4p1c2.png}
\caption{COV- iteration for $\alpha = 1$}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.4]{h4p1c3.png}
\caption{ the true pdf and the proposal distribution for $\alpha = 1$}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.4]{h4p1c4.png}
\caption{histogram of samples for $\alpha = 1$}
%\label{fig:universe}
\end{figure}

By comparing Figure 1 and Figure 3, it is easy to figure out that the acceptance rates ($\alpha =2$) is smaller than that acceptance rates ($\alpha =1$)

\newpage
\section{Independent Metropolis-Hastings}
Traditionally in the Metropolis-Hastings algorithm the arbitrary proposal distribution is conditioned on the current state of the chain. Namely, one draws samples from $x' \sim q(x'|x_{t})$ where $x_t$ indicates the state of the chain. Consider a proposal distribution that is independent of the chain's current state $q(x')$. When such a distribution is used, this is referred to as the \textit{Independent Metropolis-Hasting algorithm}.

Prove that the Independent Metropolis-Hastings accepts more than the Accept-Reject method when both have identical target ($f(x')$) and proposal ($g(x')$) distributions.

Solution:

for Accept-Reject method:
\begin{equation}
\begin{aligned}
    & prob_{AJ} = \frac{f(x')}{M \cdot g(x')} \\
    & M = sup \frac{f(x)}{g(x)} \geq \frac{f(x)}{g(x)}
\end{aligned}
\end{equation}
Since $M \cdot g(x') \geq f(x')$, $M \cdot g(x')$ curve is above $f(x')$ curve, $prob_{AJ} \leq 1$

for Metropolis-Hasting algorithm
\begin{equation}
\begin{aligned}
     & prob_{MH} = \min [1,\frac{\frac{f(x')}{g(x')}}{\frac{f(x_t)}{g(x_t)}}]  \\ %\geq \frac{f(x')}{M \cdot g(x')} \\
    & if  1 \leq \frac{\frac{f(x')}{g(x')}}{\frac{f(x_t)}{g(x_t)}} \to prob_{MH}=1 \geq prob_{AJ} \\
    & if  1 \geq \frac{\frac{f(x')}{g(x')}}{\frac{f(x_t)}{g(x_t)}} \to prob_{MH}=\frac{\frac{f(x')}{g(x')}}{\frac{f(x_t)}{g(x_t)}} \geq \frac{f(x')}{M \cdot g(x')} = prob_{AJ}
\end{aligned}
\end{equation}

The prob of Accept-Reject method is smaller than that of Metropolis-Hasting algorithm.
Therefore,the Independent Metropolis-Hastings accepts more than the Accept-Reject method when both have identical target ($f(x')$) and proposal ($g(x')$) distributions.

\newpage
\section{Accept-Reject $\&$ Metropolis-Hastings}
(a) Implement the accept-reject algorithm to calculate the mean of a gamma distribution $\mathcal{G}(4.3,6.2)$ using a $\mathcal{G}(4,7)$ candidate. Draw the true density function on top of the sample histogram and plot the convergence.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.35]{h4p3a4.png}
\caption{ the true pdf and histogram of samples for accept-reject}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.35]{h4p3a2.png}
\caption{ COV for accept-reject}
%\label{fig:universe}
\end{figure}

\newpage
(b) Implement the Metropolis-Hastings algorithm to calculate the mean of a gamma distribution $\mathcal{G}(4.3,6.2)$ using the following candidate densities:

A gamma $\mathcal{G}(4,7)$ candidate distribution.

A gamma $\mathcal{G}(5,6)$ candidate distribution.

For both candidate distributions draw the true and candidate density functions on top of
the sampled histogram. Plot the convergence using each candidate distribution on the same axis . How do the means compare?

\begin{figure}[h!]
\centering
\includegraphics[scale=0.3]{h4p3b13.png}
\caption{ $\mathcal{G}(4,7)$}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.3]{h4p3b23.png}
\caption{ $\mathcal{G}(5,6)$}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.45]{h4p3bEcompare.png}
\caption{ $<E>$ comparation for different proposal functions}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.45]{h4p3bCOVcompare.png}
\caption{ COV comparation for different proposal functions}
%\label{fig:universe}
\end{figure}

$\mathcal{G}(5,6)$ convergences much quickly than $\mathcal{G}(4,7)$. And we can see it from COV-iteration and $<E>$-iteration figures.
\\
\\
\\
\\
\\
\\
\\
\\
\\
\\
\\
\\
\\
\\
\\

\section{Gibbs $\&$ Metropolis-Hastings}
Consider sampling from a 2D Gaussian. Suppose $x \sim \mathcal{N}(\mu, \Sigma) $ where $\mu = (1,1)$ and $\Sigma = (1,-0.5;-0.5,1)$.

(a) Derive the full conditional $p(x_1 | x_2)$ and $p(x_2 | x_1)$. Implement the Gibbs algorithm for this case and plot the 1D marginals $p(x_1)$ and $p(x_2)$ as well as (superimposed) the computed histograms.

Derive the full conditional distribution from Bishop-Pattern Recognition and Machine Learning(2.81 and 2.82)
\begin{equation}
\begin{aligned}
\mu_{a|b} = \mu_{a} + \Sigma_{ab}\Sigma_{bb}^{-1}(x_{b}-\mu_{b})
\end{aligned}
\end{equation}

\begin{equation}
\begin{aligned}
\Sigma_{a|b} = \Sigma_{aa} -\Sigma_{ab}\Sigma_{bb}^{-1}\Sigma_{ba}
\end{aligned}
\end{equation}

Therefore,

\begin{equation}
\begin{aligned}
& \mu_{x_1|x_2} = 1 -0.5(x_{2}-1) \\
& \Sigma_{x_1|x_2} = 0.75 \\
& x_1|x_2 \sim \mathcal{N}(1 -0.5(x_{2}-1), 0.75) \\
& \mu_{x_2|x_1} = 1 -0.5(x_{1}-1) \\
& \Sigma_{x_2|x_1} = 0.75 \\
& x_2|x_1 \sim \mathcal{N}(1 -0.5(x_{1}-1), 0.75) \\
\end{aligned}
\end{equation}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.45]{HW4P41.png}
\caption{ }
%\label{fig:universe}
\end{figure}

(b) Let us now consider block-wise Metropolis Hastings. For our proposal distribution, $q(x)$ let us  use a normal centered at the previous state/sample of the Markov chain/sampler, i.e: $q(x|x^{(t-1)}) \sim N(x^{(t-1)}, I)$, where I is a 2D identity matrix. Show the 2D target distribution and its sampled approximation.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.45]{HW4P42.png}
\caption{ }
%\label{fig:universe}
\end{figure}

(c) We now consider component-wise Metropolis Hastings approximation of the same problem. The proposal distribution $q(x)$ is now a univariate Normal distribution with unit variance in the direction of the i-th dimension to be sampled. Show the sampled and exact target distribution.
Show your results and compare the convergence with that obtained with the block-wise, component-wise Metropolis-Hastings and Gibbs implementation.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.45]{HW4P43.png}
\caption{}
%\label{fig:universe}
\end{figure}

Compare the convergence with that with that obtained with the block-wise, component-wise Metropolis-Hastings and Gibbs implementation.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.45]{HW4P44.png}
\caption{$<E>$ convergence}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.45]{HW4P45.png}
\caption{COV convergence}
%\label{fig:universe}
\end{figure}

From $<E>$ convergence and COV convergence, it is easy to find that
the convergence rate: Gibbs is the fastest, Block-wise MH is the second, component-wise is the slowest.

\newpage
\section{Metropolis-Hastings}
Consider the braking data of Tukey. It corresponds to breaking distances $y_{i,j}$ of cars driving at speeds $x_{i}$. It is thought that a good model for this dataset is quadratic model:
\begin{equation}
    y_{i,j} = \beta_{0} +\beta_{1}x_{i} + \beta_{2}x_{i}^{2}+\epsilon_{i,j}
\end{equation}
where $\epsilon_{i,j} \sim  N(0,\sigma^2), i=1, ..., k$ and $j=1,...,n_{i}$
If we assume that $\epsilon_{i,j} \sim  N(0,\sigma^2)$ are independent, then the likelihood function is
\begin{equation}
    (\frac{1}{\sigma^2})^{N/2} e^{-\frac{1}{2\sigma^2}\sum_{i,j}(y_{i,j}-\beta_{0}-\beta_{1}x_{i} - \beta_{2}x_{i}^{2})}
\end{equation}
We can view this likelihood as a posterior distribution of $\beta_{0},\beta_{1},\beta_{2},\sigma^2$ and we can sample from it with a Metropolis-Hasting algorithm.

(a) Obtain maximum likelihood estimate for $\beta_{0}$, $\beta_{1}$, $\beta_{2}$, $\sigma^{2}$
\begin{equation}
    I(\beta_{0},\beta_{1},\beta_{2},\sigma^2 | y,X ) =  (\frac{1}{\sigma^2})^{N/2} e^{-\frac{1}{2\sigma^2}\sum_{i,j}(y_{i,j}-\beta_{0}-\beta_{1}x_{i} - \beta_{2}x_{i}^{2})}
\end{equation}
take the log of $I$
\begin{equation}
    \log I(\beta_{0},\beta_{1},\beta_{2},\sigma^2 | y,X) = (N/2) \log(\frac{1}{\sigma^2}) -\frac{1}{2\sigma^2}\sum_{i,j}(y_{i,j}-\beta_{0}-\beta_{1}x_{i} - \beta_{2}x_{i}^{2})
\end{equation}

for $\beta_{0}$, $\beta_{1}$, $\beta_{2}$:
\begin{equation}
\begin{aligned}
    & \frac{\partial I}{\partial \beta_{0}} = 0;\\
    & \frac{\partial I}{\partial \beta_{1}} = 0;\\
    & \frac{\partial I}{\partial \beta_{2}} = 0;\\
    & \frac{\partial I}{\partial \sigma} = 0;
\end{aligned}
\end{equation}
then
\begin{equation}
\begin{aligned}
    & \sum_{i,j} y_{i,j} = \sum_{i} n_{i}(\beta_{0}+\beta_{1}x_{i}+\beta_{2}x_{i}^{2});\\
    & \sum_{i,j} y_{i,j} x_{i} = \sum_{i} n_{i}(\beta_{0}+\beta_{1}x_{i}+\beta_{2}x_{i}^{2}) x_{i};\\
    & \sum_{i,j} y_{i,j} x_{i}^{2} = \sum_{i} n_{i}(\beta_{0}+\beta_{1}x_{i}+\beta_{2}x_{i}^{2})x_{i}^{2};\\
    & \sigma^2 = \frac{1}{N}\sum_{i,j}(y_{i,j}-\beta_{0}-\beta_{1}x_{i} - \beta_{2}x_{i}^{2})
\end{aligned}
\end{equation}

We call
\begin{equation}
\begin{aligned}
& Y = [y_{1,1},...,y_{1,n_{1}},y_{2,1},...,y_{k,1},...,y_{k,n_{k}}] \\
& X = [x_{1}I_{(n_{1} \times 1)},...,x_{k}I_{(n_{k}\times 1)}]
\end{aligned}
\end{equation}

The solution of $\beta$ satisfying those equation has a closed form:
\begin{equation}
    \hat{\beta} = ([I,X,X^2]^{T}[I,X,X^2])^{-1}[I,X,X^2]^{T} Y
\end{equation}

for $\sigma^2$, put estimated $\hat{\beta}$ into the likelihood expression of $\sigma^{2}$.
\begin{equation}
     \hat{\sigma}^2 = \frac{1}{N}\sum_{i,j}(y_{i,j}-\hat{\beta}_{0}-\hat{\beta}_{1}x_{i} - \hat{\beta}_{2}x_{i}^{2})
\end{equation}

Use the braking data of Tukey,
MLE of beta is $\hat{\beta} = [2.47 0.91 0.1]^{T}$; MLE of sigma is $\sigma^2 = 216.5$.

(b) Use the estimates to select a candidate distribution. Take normal for $\beta_{0}$, $\beta_{1}$, $\beta_{2}$, and inverted Gamma for $\sigma^{2}$.

The MLE estimated $\hat{\beta}$ can be used as the mean parameter of normal proposal density for $\beta$ because it is unbiased estimator. As for the variance parameter in proposal density, we can rely on its covariance matrix approximation.

\begin{equation}
    \mathbb{V} (\beta) | X,\sigma^2) = ([I,X,X^2]^{T}[I,X,X^2])^{-1} \hat{\sigma}^{2}
\end{equation}

The proposal density for $\beta$ is then
\begin{equation}
    \beta \sim N(\hat{\beta}, \mathbb{V} (\beta|X,\sigma^2))
\end{equation}

For the proposal density of parameters $\sigma^2$, according to Cochran's theorem:
\begin{equation}
    \frac{N \hat{\sigma}^2}{\sigma^2} \sim \mathcal{X}_{N-3}^2 = \mathcal{G}(\frac{N-3}{2},2) \to \frac{1}{\sigma^2} \sim  \mathcal{G}(\frac{N-3}{2},\frac{2}{N\hat{\sigma}^2})
\end{equation}

Therefore, the final proposal density for $\beta, \sigma$ is then
\begin{equation}
\begin{aligned}
    p(\beta, \sigma^2) &= \mathcal{N}(\beta| \hat{\beta}, \mathbb{V}(\beta|X,\sigma^2)) \mathcal{IG}(\sigma^2|\frac{N-3}{2},\frac{2}{N\hat{\sigma}^2}) \\
    & = \mathcal{N}([2.47,0.91,0.1],\left[ \begin{array}{ccc}
206.37 & -27.22 & 0.821 \\
-27.22 & 3.89 & -0.124 \\
0.821 & -0.124 & 0.0041
\end{array} \right]) \mathcal{IG}(23.5,5405)
\end{aligned}
\end{equation}

For student T distribution
\begin{equation}
    \mathbb{V} (\beta) | X,\sigma^2) = ([I,X,X^2]^{T}[I,X,X^2])^{-1} \hat{\sigma}^{2} \frac{v}{v-2}
\end{equation}
This covariance is bigger than the previous one.

\begin{equation}
\begin{aligned}
    p(\beta, \sigma^2) &= \mathcal{N}(\beta| \hat{\beta}, \mathbb{V}(\beta|X,\sigma^2)) \mathcal{IG}(\sigma^2|\frac{N-3}{2},\frac{2}{N\hat{\sigma}^2}) \\
    & = \mathcal{N}([2.47,0.91,0.1],\left[ \begin{array}{ccc}
412.74 & -54.44 & 1.642 \\
-54.44 & 7.78 & -0.248 \\
1.642 & -0.248 & 0.0082
\end{array} \right]) \mathcal{IG}(23.5,5405)
\end{aligned}
\end{equation}

(c) Make histogram of the posterior distributions of the parameters. Monitor convergence.

Robustness considerations could lead to using an error distribution with heavier tails. If we assume that $\epsilon_{i,j} \sim Gamma(0,\sigma^{2})$ independent, then the likelihood function is

\begin{equation}
    (\frac{1}{\sigma^2})^{N/2} \prod_{i,j} (1+\frac{1}{v} \frac{(y_{i,j}-\beta_{0}-\beta_{1}x_{i}-\beta_{2}x_{i}^{2})^2}{\sigma^2})^{(v+1)/2}
\end{equation}

where $v$ is the degrees of freedom. For $v=4$, use Metropolis-Hastings to sample  $\beta_{0}$, $\beta_{1}$, $\beta_{2}$, $\sigma^{2}$ from the posterior distribution. Use either normal or $\Gamma$ candidates for $\beta_{0}$, $\beta_{1}$, $\beta_{2}$ and inverted Gamma or half-$\Gamma$ for $\sigma^2$.

\begin{figure}[h!]
\centering
\includegraphics[scale=0.45]{HW4P51.png}
\caption{$\beta$ Normal distribution}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.45]{HW4P52.png}
\caption{$\sigma$ Normal distribution}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.45]{HW4P53.png}
\caption{Normal distribution}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.45]{HW4P54.png}
\caption{$\beta$ Student T distribution}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.45]{HW4P55.png}
\caption{$\sigma$ Student T distribution}
%\label{fig:universe}
\end{figure}

\begin{figure}[h!]
\centering
\includegraphics[scale=0.45]{HW4P56.png}
\caption{Student T distribution}
%\label{fig:universe}
\end{figure}

Normal distribution candidate is better than student T distribution candidate.
%\bibliographystyle{plain}
%\bibliography{references}
\end{document}


%! Author = tstreule

\section*{General}

\subsection*{Formeln}
%
\textbf{Circle}: $U\ped{circle} = 2\pi r$, $A\ped{circle} = \pi r^2$

%	\textbf{Laplace}:\\
%	\begin{tabular}{ c@{$\;\laplace\;$}c c c@{$\;\laplace\;$}c }
%		$f(at)$			& $\frac{1}{\abs{a}}F(s/a)$	&& $f(t-a)$		& $\eu^{-as}F(s)$\\
%		$f(t)\eu^{at}$	& $F(s-a)$					&& $f'(t)$		& $sF(s) - f(0^+)$\\
%		$t^n$			& $n!/s^{n+1}$				&& $t^n f(t)$	& $(-1)^n F^{(n)}(s)$\\
%		$\sin(at)$		& $\frac{a}{s^2 + a^2}$		&& $\cos(at)$	& $\frac{s}{s^2 + a^2}$\\
%		$\eu^{at}$		& $\frac{1}{s - a}$			&& $t^n \eu^{at}$& $\frac{n!}{(s-a)^{n+1}}$
%	\end{tabular}

\textbf{Constants}:\\
\begin{tabular}{r@{$\;=\;$}l}
    $h$			& $\unit[6.626 {\scriptstyle\mathrm{E}-34}]{Js} = \unit[4.135 {\scriptstyle\mathrm{E}-15}]{eV\,s}$,
    \quad $\hbar = \frac{h}{2\pi}$,
    \quad $hc = \unit[1.986 {\scriptstyle\mathrm{E}-25}]{Jm}$\\
    $\epsilon_0$& $\unitfrac[8.85 {\scriptstyle\mathrm{E}-5}]{As}{Vm}$\\
    $\mu_0$		& $\unitfrac[4\pi {\scriptstyle\mathrm{E}-7}]{N}{A^2}$\\
    $k\ped{B}$	& $\unitfrac[1.38 {\scriptstyle\mathrm{E}-23}]{J}{K} = \unitfrac[8.617 {\scriptstyle\mathrm{E}-5}]{eV}{K}$\\
    $q$			& $\unit[1.602 {\scriptstyle\mathrm{E}-19}]{C}$, \quad $m_e = \unit[9.109 {\scriptstyle\mathrm{E}-31}]{kg}$, \quad $m_p = \unit[1.672 {\scriptstyle\mathrm{E}-27}]{kg}$\\
    $F$			& $\unitfrac[96485]{C}{mol}$ (Faraday)\\ % charge of 1 mole of electrons
    $R$         & $N\ped{A} k\ped{B} = \unitfrac[8.314]{J}{mol\;K}$ (Ideal gas constant)\\
    $N\ped{A}$	& $\unitfrac[6.022 {\scriptstyle\mathrm{E}23}]{particles}{mol}$
    \qquad $\SI{0}{\degreeCelsius} = \unit[273.15]{K}$
\end{tabular}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection*{Einheiten}
%
\formula{Druck}{\unit[1]{Pa} = \unitfrac[1]{N}{m^2} = \unitfrac[1]{J}{m^3} = \unitfrac[10]{g}{cm\cdot s^2}}\\
\formula{Induktivität}{\unit[1]{H} = \unitfrac[1]{Vs}{A} = \unit[1]{\Omega\,s}}\\
\formula{Power}{\unit[1]{W} = \unitfrac[1]{J}{s} = \unit[1]{VA}}\\
\formula{electron volt}{\unit[1]{eV} = \unit[1.602 {\scriptstyle\mathrm{E}-19}]{J} = \unitfrac[23.06]{kcal}{mol}}\\
\formula{Charge}{\SI{1}{\coulomb} = \SI{1}{\ampere\second}}\\
\formula{Energy}{\SI{1}{\joule} = \unitfrac[1]{kg\;m^2}{s^2} = \unit[1]{Nm} = \unit[1]{VAs} = \unit[1]{CV} = \unit[1]{Ws}}\\
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection*{Good to know}
%
\formula{Power in \unit{dB}}{10\,\log_{10}\frac{I}{I_0}}\\
\formula[\unit{\frac{\!W}{m^2}}]{Intensity}{ I = \frac{\textrm{avg. Power }(P)}{\textrm{area }(A)} }\\
\formula[\unit{W} = \unit{\frac{kg\,m^2}{s^3}}]{avg. Power}{P = \frac{\textrm{avg. Work in cycle }(\overline{W})}{\textrm{cycle }(T)}}\\
\formula[\unit{J}=\unit{Ws}]{(avg.) Work}{\overline{W} = \int_{\textrm{cycle}} \frac{1}{\textrm{cycle} (T)} \cdot \vec{F}\cdot\vec{x} \diff t }\\

mass $m$ vibrates with an amplitude $a$ along $x$-axis:\\
$\vec{x}(t) = a\sin(\omega t)\cdot \vec{e}_x, \quad \omega = 2\pi f = 2\pi/T, \quad \vec{F}(t) = m\,\ddot{\vec{x}}(t)$
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection*{NuS}
%
\formula{Induktivität}{u(t) = L \deriv{i_L}{t} \enskip\laplace\enskip U(s) = sL\,I_L(s)}\\
\formula{Konduktivität}{i(t) = C \deriv{u_C}{t} \enskip\laplace\enskip U(s) = \frac{1}{sC}\,U_C(s)}\\

\formula{Transformator}{u_1 = L_1 \deriv{i_1}{t} - M \deriv{i_2}{t} \textnormal{,\enskip $M$: mutual inductance}}\\
\formula{LCR-Schwingkreis}{\omega_0 = 2\pi f_0 = 1/\sqrt{LC}}\\
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection*{Laplace}
%	%Laplace Transformation: \;
%	$\phantom{\laplace} \dis{0.4} u(t)=\mathcal{L}^{-1}\{U(s)\} = \frac{1}{2\pi\iu} \int_{\bar{s}}^s U(s)\eu^{st} \diff s$\\
%	$\laplace \dis{0.4} U(s)=\mathcal{L}\{u(t)\} = \int_0^\infty u(t)\eu^{-st} \diff t$
%
\begin{tabular}{ rcl }
    \toprule
%		$u(t)$							& \laplace	& $U(s)$								\\%\addlinespace
%		\midrule
    $\lambda\;u(t) + \mu\;v(t)$		& \laplace	& $\lambda\;U(s) + \mu\;V(s)$			\\
    ${u(at),\; \scriptstyle a>0}$	& \laplace	& $\frac{1}{a}$ $U(\frac{s}{a})$		\\
    { \color{red} $u(t-t_0)$ }		& \laplace	& $\eu^{-st_0}$ $U(s)$					\\
    $\eu^{-at}$ $u(t)$				& \laplace	& { \color{red} $U(s+a)$ }				\\
    ${ (-t)^{n} u(t) }$				& \laplace	& $U^{(n)}(s)$							\\
    $u^{(n)}(t)$					& \laplace	& $ {s^nU(s) - \ldots - u^{(n-1)}(0)} $	\\
    \midrule
%		$t^n$							& \laplace	& $\frac{n!}{s^{n+1}}$					\\\addlinespace
%		$\eu^{-at}$						& \laplace	& $\frac{1}{s+a}$						\\\addlinespace
%		$t\eu^{-at}$					& \laplace	& $\frac{1}{(s+a)^2}$					\\\addlinespace
%		$t^n\eu^{-at}$					& \laplace	& $\frac{n!}{(s+a)^{n+1}}$				\\\addlinespace
%		$\sin (at)$						& \laplace	& $\frac{a}{s^2+a^2}$					\\\addlinespace
%		$\cos(at)$						& \laplace	& $\frac{s}{s^2+a^2}$					\\\addlinespace
    \multicolumn{3}{c}{
        \begin{minipage}{.45\columnwidth} \centering
        \begin{tabular}{ ccc }
%					$u(t)$							& \laplace	& $U(s)$								\\%\addlinespace
%					\midrule
            $t^n$							& \laplace	& $\frac{n!}{s^{n+1}}$					\\
            $\eu^{-at}$						& \laplace	& $\frac{1}{s+a}$						\\
            $t\eu^{-at}$					& \laplace	& $\frac{1}{(s+a)^2}$					\\
        \end{tabular}
        \end{minipage}%
        \begin{minipage}{.45\columnwidth} \centering
        \begin{tabular}{ ccc }
%					$u(t)$							& \laplace	& $U(s)$								\\%\addlinespace
%					\midrule
            $t^n\eu^{-at}$					& \laplace	& $\frac{n!}{(s+a)^{n+1}}$				\\
            $\sin (at)$						& \laplace	& $\frac{a}{s^2+a^2}$					\\
            $\cos(at)$						& \laplace	& $\frac{s}{s^2+a^2}$					\\
        \end{tabular}
        \end{minipage}
    }\\
    \bottomrule
\end{tabular}


\subsection{Matrix Element Methods}
\label{subsec:MEM}

The ME method is based on \emph{ab initio} calculations of the probability density function $\mathcal{P}$ of an event with observed final-state particle momenta ${\bf x}$ to be due to a physics process $\xi$ with theory parameters $\boldsymbol\alpha$.
One can compute $\mathcal{P}_{\xi}({\bf x}|{\boldsymbol\alpha})$ by means of the factorization theorem from the corresponding partonic cross-sections of the hard-scattering process involving parton momenta ${\bf y}$ and is given by
\begin{equation}
 \mathcal{P}_{\xi}({\bf x}|{\boldsymbol\alpha}) = \frac{1}{\sigma^{\rm fiducial}_{\xi}(\boldsymbol\alpha)} \int d\Phi ({\bf y}_{\rm final}) \; dx_1 \; dx_2~\frac{f(x_1)f(x_2)}{2s x_1 x_2} \; |\mathcal{M}_{\xi}({\bf y}|\boldsymbol\alpha)|^2 \; \delta^{4}({\bf y}_{\rm initial}-{\bf y}_{\rm final}) \; W({\bf x}, {\bf y})
 \label{eqn:MEProb}
\end{equation}
where and $x_i$ and ${\bf y}_{{\rm initial}}$ are related by $y_{{\rm initial},i}\equiv \frac{\sqrt{s}}{2}(x_i,0,0,\pm x_i)$, $f(x_i)$ are the parton distribution functions, $\sqrt{s}$ is the collider center-of-mass energy, $\sigma^{\textrm{ fiducial}}_{\xi}(\boldsymbol\alpha)$ is the total cross section for the process $\xi$ (with $\boldsymbol\alpha$) times the detector acceptance, $d\Phi({\bf y})$ is the phase space density factor, $\mathcal{M}_{\xi}({\bf y}|\boldsymbol\alpha)$ is the matrix element (typically at leading-order (LO)), and $W({\bf x}, {\bf y})$ is the probability density (aka ``transfer function'') that a selected event ${\bf y}$ ends up as a measured event ${\bf x}$.
One can use calculations of Eq.~\ref{eqn:MEProb} in a number of ways (e.g. likelihood functions) to search for new phenomena at particle colliders.

%%%

As stated in Sect.~\ref{sec:applications-MEM}, the ME method has three notable features: it (1) does not require training data being an \emph{ab initio} calculation of event probabilities, (2) incorporates all available kinematic information of a hypothesized process, including all correlations, and (3) has a clear physical meaning in terms of the transition probabilities within the framework of quantum field theory.\\

In reference to point (1), the matrix element $\mathcal{M}_{\xi}({\bf y}|\boldsymbol\alpha)$ in the method involves all partons in the $n\rightarrow m$ process, so when the 4-momentum of particles are not completely measured experimentally (e.g. neutrinos), one must integrate over the missing information which increases the dimensionality of the integration.
In reference to point (2), a clever technique to re-map the phase space in order to reduce the sharpness of integrate in that space in an automated way ({\sf MADWEIGHT}~\cite{Artoisenet:2010cn}) is often used in conjunction with a matrix element calculation package ({\sf MADGRAPH\_aMC\@NLO}~\cite{Alwall:2014hca}).
In practice, evaluation of definite integrals by the ME approach invokes techniques such as importance sampling (see {\sf VEGAS}~\cite{PETERLEPAGE1978192,Ohl:1998jn} and {\sf FOAM}~\cite{JADACH200355}) or recursive stratified sampling (see MISER~\cite{Press:1989vk}) Monte Carlo integration.
Acceleration of some of these techniques on modern computing architectures has been achieved, for example concurrent phase space sampling in VEGAS on GPUs.


\documentclass[12pt, a4paper]{article}
\usepackage[margin=2.5cm]{geometry}
\usepackage{amssymb, amsmath}

\begin{document}
\subsection*{Some interesting formulae to try out:}
\begin{itemize}
\item
$z_{n+1} = e^{z_n^2 e^{i\varphi}}+c$
\item
$z_{n+1} = \left(\frac{1}{z_n^2 - e^{i\varphi}}\right)^2 + c$
\item
$z_{n+1} = \tan(z_n^2 e^{i\varphi}) + c$
\item
$z_{n+1} = (z_n^2 - e^{i\varphi})^3 + c$
\end{itemize}
\end{document}


\section{Preliminaries}
\subsection{Proofs using the Curry-Howard Correspondence}
The Curry-Howard correspondence is a relationship that can be used to interpret typed computer programs as mathematical proofs \cite{chc}. This is done by representing false statements as empty types, and true statements as non-empty types. For example, take  \verb|IsTrue b|  where  \verb|b| is some boolean expression. The type is constructed in such a way that it is empty if  \verb|b| is false, and non-empty if b is true.  So if there exists a value of type  \verb|IsTrue b|, this value is a proof that  \verb|b| must be true.

These proofs can be used as function arguments, constructor arguments or even as a function result. Since Agda is dependently typed, the proof can also refer to other arguments of a function. For example, this function may only be called when \verb|n| is greater than 5:
\begin{minted}{agda}
takesGtFive : (n : Nat) -> IsTrue (n > 5) -> ?
\end{minted}

\subsection{Lenses}
The QuadTree library makes extensive use of Lenses. Lenses are composable functional references \cite{lens}. Using lenses, data in a data-structure can be accessed and modified. This paper chooses to use the Van Laarhoven representation \cite{laarhovenlens}, since this is what the Haskell implementation of the QuadTree library uses. It is defined as:
\begin{minted}{haskell}
type Lens s a = forall f. Functor f => (a -> f a) -> s -> f s
\end{minted}
Using this representation, \verb|Lens a b| means that given an object of type \verb|a|, we can view or modify an inner object of type \verb|b|.
The functions to interact with lenses are:
\begin{minted}{haskell}
-- Get the value at this lens
view :: Lens a b -> a -> b
-- Set the value at this lens
set :: Lens a b -> b -> a -> a
-- Map the value at this lens
over :: Lens a b -> (b -> b) -> a -> a
-- Compose two lenses (Note: This is actually just regular function composition!)
compose :: Lens a b -> Lens b c -> Lens a c
\end{minted}

\subsection{QuadTrees}
\begin{wrapfigure}{r}{0.3\textwidth} %this figure will be at the right
	\vspace{-40pt}
	\includegraphics[width=0.3\textwidth]{graphics/test.png}
	\caption{An example QuadTree}
	\label{quadtree_img}
	\vspace{-90pt}
\end{wrapfigure}

The QuadTree is a data structure that is used for storing two-dimensional information in a functional way \cite{Finkel1974}. It is defined as:
\begin{minted}{haskell}
data Quadrant t = Leaf t | Node (Quadrant t)
	(Quadrant t) (Quadrant t) (Quadrant t)

data QuadTree t = Wrapper (Nat, Nat) (Quadrant t)
\end{minted}

A QuadTree consists of the size (width  ×  height) of the QuadTree, and the root quadrant. A quadrant is either a leaf (in which case all the values inside the region of the quadrant are the same), or four subquadrants. The four subquadrants are then called A (top left), B (top right), C (bottom left), and D (bottom right). Notice that in Figure \ref{quadtree_img}, space is consistently split into four quadrants.

There are five functions that can be used to interact with QuadTrees:
\begin{minted}{haskell}
-- Create a new QuadTree with the specified size
makeTree :: (Nat, Nat) -> t -> QuadTree t
-- Obtain a lens to the specified location
atLocation :: (Nat, Nat) -> Lens (QuadTree t) t
-- Get the value at the specified location
getLocation :: (Nat, Nat) -> QuadTree t -> t
-- Set the value at the specified location
setLocation :: (Nat, Nat) -> t -> QuadTree t -> QuadTree t
-- Map the value at the specified location
mapLocation :: (Nat, Nat) -> (t -> t) -> QuadTree t -> QuadTree t
\end{minted}


\section{Rigid Bodies}
\begin{definition}
    A rigit body is an extended mass with a finite volume as a system of particles that are constrained such that the mutual distances between them does not change.
\end{definition}
\begin{definition}
    An isometry is a distance-preserving map in the space, e.g. rotation, translation, etc..
\end{definition}
So a rigid body is a system of particle moving under isometries.
\subsection{Angular Velocity}
Recall that we can have a vector angular velocity $\underline{\omega}$ which points to the axis of rotation and has magnitude equal to the scalar angular velocity $\omega$ of the point mass $\underline{r}$.
So $\underline{\dot{r}}=\underline{\omega}\times\underline{r}$.
If the particle has mass $m$, we can write down the kinetic energy $T=m|\underline{\dot{r}}|^2/2=m\omega^2r_\perp^2/2$ where $r_\perp=|\underline{n}\times\underline{r}|$ where $\underline{n}$ is a unit vector and $\underline{\omega}=\omega\underline{n}$.
We write $I=mr_\perp^2$ as the moment of inertia, so $T=I\omega^2/2$.
Note that the moment of inertia is dependent on the axis.
\subsection{Moment of Inertia for a Rigid Body}
Consider a rigid body made up of $N$ particles following the notation we introduced earlier.
The body (i.e. all the particles within it) would rotate about an axis through the origin with angular velocity $\underline{\omega}$.
For particle $i$, we have $\underline{\dot{r}_i}=\underline{\omega}\times\underline{r_i}$.
Note that
$$\frac{\mathrm d}{\mathrm dt}|\underline{r_i}-\underline{r_j}|^2=2((\omega\times(\underline{r_i}-\underline{r_j}))\cdot(\underline{r_i}-\underline{r_j}))=0$$
So the particles do stay the same distance apart.
The kinetic energy of the rotating body is then going to be
$$T=\sum_{i=1}^N\frac{1}{2}m_i|\underline{\dot{r}_i}|^2=\frac{1}{2}\omega^2\sum_{i=1}^Nm_i|\underline{n}\times\underline{r_i}|^2=\frac{1}{2}\omega^2\sum_{i=1}^Nm_i(r_i)_\perp^2=\frac{1}{2}\omega^2I$$
where $I$ is called the moment of inertia for the body.
Correspondingly, we can consider the angular momentum, where
$$\underline{L}=\sum_{i=1}^N\underline{L_i}=\sum_{i=1}^Nm_i\underline{r_i}\times(\underline{\omega}\times\underline{r_i})=\omega\sum_{i=1}^Nm_i\underline{r_i}\times(\underline{n}\times\underline{r_i})$$
Consider its component in the direction of the axis of rotation,
$$\underline{L}\cdot\underline{n}=\omega\sum_{i=1}^Nm_i\underline{n}\cdot(\underline{r_i}\times(\underline{n}\times\underline{r_i}))=\omega\sum_{i=1}^nm_i|\underline{n}\times\underline{r_i}|^2=I\omega$$
So the direction of $\underline{L}$ in the direction of the axis of rotation is $I\omega$.
In general, $\underline{L}$ is not parallel to $\underline{\omega}$, so we need to go back to the vector expression.
Observe that $\underline{L}$ as a function of $\omega$ is linear, so
$$\underline{L}=\sum_{i=1}^Nm_i\underline{r_i}\times(\underline{\omega}\times\underline{r_i})=\sum_{i=1}^Nm_i(|\underline{r_i}|^2\underline{\omega}-|\underline{r_i}\cdot\underline{\omega}|\underline{r_i})=I\underline{\omega}$$
where $I$ here is a tensor (i.e. a (multi)linear map) which in this case is a $3\times 3$ matrix.
So under suffix notation, $L_\alpha=I_{\alpha\beta}\omega_\beta$.
$I$ is a symmetric tensor (matrix) by symmetry.
We have
$$I_{\alpha\beta}=\sum_{i=1}^Nm_i(|\underline{r_i}|^2\delta_{\alpha\beta})-(\underline{r_i})_\alpha(\underline{r_i})_\beta$$
Now $I$ is diagonalizable so we can choose our favourite basis (principal axes) to make $I$ diagonal.
To get $\underline{L}$ to be at the same direction as $\underline{\omega}$, we need the object to rotate wrt a principal axis
\subsection{Calculation of Moment of Inertia}
For a solid body, we replace mass-weighted sums by mass-weighted volume integrals.
Consider a body with volume $V$ with density $\rho(\underline{r})$, so its mass, center of mass and moment of inertia are
$$M=\int_V\rho\,\mathrm dV,\underline{R}=\frac{1}{M}\int_V\rho(\underline{r})\underline{r}\,\mathrm dV,I=\int_V\rho(\underline{r})|\underline{r}_\perp|^2\,\mathrm dV=\int_V\rho(\underline{r})|\underline{n}\times\underline{r}|^2\,\mathrm dV$$
For curves and surfaces, we can use line and area integrals accordingly.
\begin{example}
    1. For uniform thin ring of mass $M$ and radius $a$ with rotation axis $\underline{n}$ through the center of the ring and perpendicular to the plane where the ring is on.
    In this case, we can reduce volume integral to line integral.
    We have $\rho=M/(2\pi a)$, so
    $$I=\int_0^{2\pi}\left( \frac{M}{2\pi a} \right)a^2a\,\mathrm d\theta=Ma^2$$
    Every point in the body is of the same distance from the axis $|\underline{r}_\perp|=|\underline{n}\times\underline{r}|=a$.\\
    2. Consider a uniform thin rod of mass $M$ and length $l$ with axis of rotation through one end and perpendicular to the rod.
    So
    $$I=\int_0^l\left( \frac{M}{l} \right)x^2\,\mathrm dx=\frac{1}{3}Ml^2$$
    3. Consider a uniform thin disk with mass $M$ and radius $a$ with the axis of rotation through its center and perpendicular to the plane where the disk is in.
    So we use an area integral
    $$I=\int_0^a\int_0^{2\pi}\left( \frac{M}{\pi a^2} \right)r^2r\,\mathrm d\theta\,\mathrm dr=\frac{Ma^2}{2}$$
    4. Using the same disk but choose the axis to be one through the center and in the same plane as the disk.
    In this case,
    $$I=\int_0^a\int_0^{2\pi}\left( \frac{M}{\pi a^2} \right)(r^2\sin^2\theta)r\,\mathrm d\theta\,\mathrm dr=\frac{1}{4}Ma^2$$
    5. Consider a solid sphere (a ball) of mass $M$ and radius $a$ with axis of rotation through its center, so spherical polars will be a good choice.
    We assume WLOG that $\underline{n}$ is the $z$ direction (so $\theta=0$ along $\underline{n}$).
    By uniform density, we have $\rho=3M/(4\pi a^3)$, therefore
    $$I=\int_0^a\int_0^\pi\int_0^{2\pi}\frac{3M}{4\pi a^3}(r^2\sin^2\theta)r^2\sin\theta\,\mathrm d\phi\,\mathrm d\theta\,\mathrm dr=\frac{2}{5}Ma^2$$
\end{example}
There are a few simple but general results to simplify calculation moment of inertia.
\begin{theorem}[Perpendicular Axis Theorem]
    For a two dimensional body on a plane (aka lamina),
    $$I_z=I_x+I_y$$
    where $I_z$ is the moment of inertia along the $z$ axis chosen to be a normal to the plane and $I_x,I_y$ are the moments of inertia along two chosen perpendicular axes on the plane so that all three axes meet at the origin.
\end{theorem}
\begin{proof}
    We have
    $$I_x=\int_A\rho y^2\,\mathrm dA,I_y=\int_A\rho x^2\,\mathrm dA$$
    But
    $$I_z=\int_A\rho r^2\,\mathrm dA=\int_A\rho(x^2+y^2)\,\mathrm dA=I_x+I_y$$
    As desired
\end{proof}
Sometimes the lamina is symmetric enough such that $I_x=I_y$, so $I_z=2I_x$.
This corresponds to the example of a disk.
Note that this theorem works for lamina but does not work for $3$ dimensional bodies.
\begin{theorem}[Parallel Axes Theorem]
    If a rigid body of mass $M$ has moment of inertia $I_c$ about an axis through its center of mass, then for another axis parallel to the original axis with a distance $d$ away, then the moment of inertia $I$ about the new axis is $I=I_c+Md^2$.
\end{theorem}
\begin{proof}
    Choose Cartesian axes such that the centre of mass is at the origin and the rotation axis along $z$-axis.
    Also, choose $x,y$-axes such that the second axes of rotation is through the point $d\underline{\hat{x}}=(d,0,0)$, then
    \begin{align*}
        I_c+Md^2&=\int_V\rho(x^2+y^2)\,\mathrm dV+Md^2\\
        &=\int_V\rho((x-d)^2+y^2)\,\mathrm dV+2d\int_V\rho x\,\mathrm dV\\
        &=\int_V\rho((x-d)^2+y^2)\,\mathrm dV=I
    \end{align*}
    Since the axes are chosen in a way that the origin is the center of mass.
\end{proof}
\begin{example}
    Consider a uniform disk as before with the axis of rotation perpendicular to it through a point on the edge has $I=3Ma^2/2$.
\end{example}
\subsection{Motion of a Rigid Body}
General motion of a rigit body can be described by the composition of translation (of the center of mass) following some trajectory $\underline{R}(t)$ together with a rotation about the center of mass.
Following the previous discussion, we specify points in the body relative to the center of mass by writing $\underline{r_i}=\underline{R}+\underline{s_i}$.
Also recall that $\sum_im_i\underline{r_i}=M\underline{R}$, therefore $\sum_im_i\underline{s_i}=0$.
If a body rotates about its center of mass, with angular velocity $\underline{\omega}$, so $\underline{\dot{s}_i}=\underline{\omega}\times\underline{s_i}$ and $\underline{\dot{r}_i}=\underline{\dot{R}}+\underline{\omega}\times\underline{s_i}$.
The kinetic energy, as we recall, satisfies
$$T=\frac{1}{2}M|\underline{\dot{R}}|^2+\frac{1}{2}\sum_{i=1}^Nm_i|\underline{s_i}|^2=\frac{1}{2}M|\underline{\dot{R}}|^2+\frac{1}{2}I_c\omega^2$$
where $I_c$ is the moment of inertia parallel to $\underline{\omega}$ and through the center of mass.
So $T$ is the sum of translational KE and rotational KE.
We have also shown before that for a general multiparticle system, linear and angular momentum obey $\underline{\dot{P}}=\underline{F},\underline{\dot{L}}=\underline{G}$ where $\underline{F},\underline{G}$ are the total external applied force and torque respectively.
For a rigit body, these two equations determine the translational and rotational motion.
Sometimes, we can exploit the conservation of energy as an easier method of solution.\\
$\underline{L},\underline{G}$ depend on the choice of origin, and we can the origin to be any point fixed in an inertial frame (shown previously).
Or, we can define $\underline{L}$ and $\underline{G}$ about the center of mass, and the equation above, as we shall show, still holds.
Take
\begin{align*}
    \underline{G}&=\frac{\mathrm d}{\mathrm dt}\left(M\underline{R}\times\underline{\dot{R}}+\sum_{i=1}^Nm_i\underline{s_i}\times\underline{\dot{s}_i}\right)\\
    &=M\underline{R}\times\underline{\ddot{R}}+\frac{\mathrm d}{\mathrm dt}\left( \sum_{i=1}^Nm_i\underline{s_i}\times\underline{\dot{s}_i} \right)\\
    &=\underline{R}\times\underline{F}^{\rm ext}+\frac{\mathrm d}{\mathrm dt}\left( \sum_{i=1}^Nm_i\underline{s_i}\times\underline{\dot{s}_i} \right)
\end{align*}
Therefore
\begin{align*}
    \frac{\mathrm d}{\mathrm dt}\left( \sum_{i=1}^Nm_i\underline{s_i}\times\underline{\dot{s}_i} \right)&=\underline{G}-\underline{R}\times\underline{F}^{\rm ext}\\
    &=\sum_{i=1}^N\underline{r_i}\times\underline{F_i}^{\rm ext}-\underline{R}\times\sum_{i=1}^N\underline{F_i}^{\rm ext}\\
    &=\sum_{i=1}^N(\underline{r_i}-\underline{R})\times\underline{F_i}^{\rm ext}\\
    &=\underline{G_c}
\end{align*}
Consider now the motion in a uniform gravitational field with acceleration due to gravity $\underline{g}$, then the total gravitational force and torque acting on a rigit body would be the same as if it is acting on a particle of mass $m$ located in the center of mass (hence it is also called the center of gravity).
So
$$\underline{F}=\sum_{i=1}^N\underline{F_i}^{\rm ext}=\sum_{i=1}^Nm_i\underline{g}=M\underline{g}$$
similarly
$$\underline{G}=\sum_{i=1}^N\underline{G_i}^{\rm ext}=\sum_{i=1}^N\underline{r_i}\times(m_i\underline{g})=M\underline{R}\times\underline{g}$$
Note that the gravitational torque about the center of mass is zero since
$$\underline{G_c}=\sum_{i=1}^N\underline{s_i}\times(m_i\underline{g})=\left( \sum_{i=1}^Nm_i\underline{s_i} \right)\times\underline{g}=0$$
Consider the gravitatioinal potential $-m\underline{r}\cdot\underline{g}$, then
$$V^{\rm ext}=\sum_{i=1}^NV_i^{\rm ext}=\sum_{i=1}^N(-m_i\underline{r_i}\cdot\underline{g})=-M\underline{R}\cdot\underline{G}$$
\begin{example}
    1. Throw a stick in the air.
    So the center of mass follows a parabolic curve and the angular velocity of the stick about center of mass is constant by conservation of energy (or because gravitational torque about center of mass is $0$).\\
    2. A uniform rod of length $l$ and mass $M$ fixed at a pivot point $O$ at one end and makes an angle $\theta$ with the downward vertical.
    We say this is a compound pendulum since the mass is distributed instead of concentrated.
    Consider the angular velocity and angular momentum about the pivot.
    We have $\omega=\dot\theta,L=I\dot\theta=Ml^2\dot\theta/3$.
    So the gravitational torque about $O$ becomes $-Mgl\sin\theta/2$, so $\dot{L}=G\implies I\ddot\theta=-Mgl\sin\theta/2$, so
    $$\ddot\theta=-\frac{3}{2}\frac{g}{l}\sin\theta$$
    which just looks like a simple pendulum of length $2l/3$ (in fact equivalent to it).
    So for small oscillations, the frequency is $f=\sqrt{3g/(2l)}$ and period $2\pi/f$.\\
    Alternatively we can think of the energy, then
    $$E=T+V=\frac{1}{2}I\omega^2-\frac{Mgl}{2}\cos\theta$$
    So
    $$0=\frac{\mathrm dE}{\mathrm dt}=\dot\theta\left(I\ddot\theta+\frac{Mgl}{2}\sin\theta\right)=0$$
    which produces the same result as above.
\end{example}
\subsection{Sliding and Rolling}
Consider a cylinder or sphere with radius $a$ moving along a stationary horizontal surface, then the general motion is a translation of the center of mass with velocity $v$ together with rotation about the center of mass with angular velocity $\omega$.
Let $P$ be the instantaneous point of contact, then the horizontal velocity of this point is given by $v_{\rm slip}=v-a\omega$.\\
There are two extreme cases:\\
1. Pure sliding, where we have $\omega=0,v_{\rm slip}=v\neq 0$.
So the point of contact slips through the surface (probably due to a kinetic frictional force).\\
2. Pure rolling, where we have $\omega,v\neq 0$ but $v_{\rm slip}=v-a\omega=0$.
In this case, the contact point is stationary at any point, which produces rolling without sliding.\\
Instantaneously, we can view the motion of the body as the rotation of the body about the contact point.
Also note that these also apply to inclined plane.
\begin{example}
    Consider a cylinder of radius $a$ and mass $m$ rolling through inclined plane at angle $\alpha$ to the horizontal.
    Let $x$ be the distance down slope travelled by the center of mass, $v=\dot{x}$ and $Mg$ the gravitational force, $N$ the normal reaction and $F$ the frictional force.
    For the cylinder to be purely rolling, we must have $v-a\omega=0$, so $v=a\omega$.\\
    The kinetic energy has
    $$T=\frac{1}{2}Mv^2+\frac{1}{2}I\omega^2=\frac{1}{2}\left(M+\frac{I}{a^2}\right)v^2$$
    Note that due to their directions the normal and frictional force (in the case where $v_{\rm slip}=0$) do no work.
    Now the energy $T+V$ is conserved where $V=-Mgx\sin\alpha$, so
    \begin{align*}
        0&=\frac{\mathrm d(T+V)}{t}\\
        &=\frac{\mathrm d}{\mathrm dt}\left( \frac{M+I/a^2}{2}\dot{x}^2-Mgx\sin\alpha \right)\\
        &=(M+I/a^2)\dot{x}\ddot{x}-Mg\dot{x}\sin\alpha\\
        \implies \left( M+\frac{I}{a^2} \right)\ddot{x}&=Mg\sin\alpha
    \end{align*}
    Note that when $I=0$, this is exactly the equation for a frictionless particle, therefore the rotation makes acceleration smaller.
    Now for a cylinder in question, we have $I=Ma^2/2$, hence
    $$\ddot{x}=\frac{2}{3}g\sin\alpha$$
    We can also obtain the result by using forces and torques.
    By considering the rate of change of linear momentum along the plane, we have $M\dot{v}=Mg\sin\alpha-F$ and the rate of change of angular momentum about the center of mass then gives $I\dot\omega=aF$.
    So as it is rolling, $\dot{v}=a\dot\omega$, whence
    $$M\dot{v}=Mg\sin\alpha-\frac{I\dot{v}}{a^2}$$
    Thus $(M+I/a^2)\dot{v}=Mg\sin\alpha$ as above.\\
    There is yet another way to do this:
    Consider the torque about $P$, we have $I_P=I+Ma^2$ by the parallel axis theroem, also the gravitational torque has $I_P\dot\omega=Mga\sin\alpha$.
    So $v=a\omega$ gives $(I+Ma^2)\dot{v}/a=Mga\sin\alpha$.
\end{example}
\begin{example}
    We want to study the transition from a sliding motion to a rolling one.
    Consider a snooker ball on a horizontal plane hit by a cue instantaneously which gives it an initial velocity $v_0$.
    Initially $v=v_0$ and $\omega_0$, where sliding occurs (so no rotation at $t=0$).
    The kinetic frictional force obeys $F=\mu N=\mu Mg$ where $\mu$ is a constant (coefficient of kinetic friction).
    The linear motion has $M\dot{v}=-F$ and the angular motion $I\dot\omega=aF$.
    Also for a sphere $I=2Ma^2/5$, hence we have, by integrating,
    $$\begin{cases}
        v=v_0-\mu gt\\
        \omega=5\mu gt/(2a)
    \end{cases}$$
    So when the ball is still moving,
    $$0\le v_{\rm slip}=v-a\omega=v_0-\frac{7}{2}\mu gt$$
    So the total time of rolling is $t_{\rm roll}=2v_0/(7\mu g)$.
    During $0\le t\le t_{\rm roll}$, the friction acts to decrease $v$ and increase $\omega$ till the no-slip condition is satisfied, when $t=t_{\rm roll}$ and $v=v_{\rm roll}=5v_0/7$.
    But at $t_{\rm roll}$, the rolling could as well persist but the friction does no further work.
    At $t=t_{\rm roll}$, the kinetic energy is
    $$T=\frac{1}{2}Mv^2+\frac{1}{2}I\omega^2=\frac{1}{2}M\left( 1+\frac{2}{5} \right)v_{\rm roll}^2=\frac{5}{7}\left( \frac{1}{2}Mv_0^2 \right)$$
    So the loss of KE due to friction has a total of
    $$\int_0^{t_{\rm roll}}Fv_{\rm slip}\,\mathrm dt=\int_0^{t_{\rm roll}}F\left( v_0-\frac{7}{2}\mu gt \right)\,\mathrm dt=\frac{1}{7}Mv_0^2$$
\end{example}