As stated in the introduction, metasurfaces are 2D materials with some sub-wavelength structure. Their properties are determined by the geometry of this structure which is usually a simple shape, like a rectangle or oval, on a periodic grid. The shape is called metaatom and in the case of plasmonic metasurfaces it is made of metal.
With the $S$-matrix calculus and SASA we can fully describe stacks of homogeneous isotropic materials but we still have no understanding of what happens when we add a metasurface to a stack. To build some intuition for how a specific metasurface will affect the transmission spectrum of a stack, we will need a little more theory on optics in materials. As stated before, all optical properties of a material a captured by the $\vb D$ and $\vb H$ fields and in our context
\begin{equation} \label{eq:bg:D}
    \vb D =
    \epsilon_0 \, \vb E + \vb P =
    \epsilon_0\, \underbrace{\qty(1 + \chi)}_{
         := \epsilon
    }\, \vb E
\end{equation}
so the $\vb D$ field is determined by the $\vb E$ field and dielectric function $\epsilon$. In the following sections we will discuss which materials have which kind of dielectric functions and how we can use these to gain some understanding of the optical mechanisms in a metasurface.

\paragraph{Dielectric Function}~\\
In the simplest model we can describe electrons in a material as an ensemble of harmonic oscillators

\begin{equation} \label{eq:bg:lorentz}
    m \ddot{\vb x} + m \gamma \dot{\vb x} + m \omega_0^2 \vb x = -e \vb E,
\end{equation}

with the displacement $\vb x$, electron mass $m$, electron charge $e$, dampening factor $\gamma$ and resonance frequency $\omega_0$.
In this model the macroscopic polarization $\vb P$ is directly caused by this displacement $\vb x$ through
\begin{equation} \label{eq:bg:P}
    \vb P = - \rho e \vb x,
\end{equation}

where $\rho$ is the density of electrons.
If we assume a harmonic time dependency $\vb E(t) = \vb E_0 e^{i \omega t}$ equation \eqref{eq:bg:lorentz} is solved by
\begin{equation}
    \vb x(t) = \frac{e}{m(\omega^2 + i \gamma \omega - \omega_0^2)} \vb E(t)
\end{equation}

and using equation \eqref{eq:bg:D} and \eqref{eq:bg:P} this results in the $\vb D$ field
\begin{equation}
    \vb D =
    \epsilon_0 \underbrace{
    \qty(1 - \frac{f}{\omega^2 + i \gamma \omega - \omega_0^2})
    }_{
        =\epsilon
    }
    \vb E,
\end{equation}

where $f = \rho e^2 / \epsilon_0 m$ is the oscillator strength.
In a real material there are always multiple resonance frequencies $\omega_m$ but in good approximation these do not influence each other and the total dielectric function can be obtained simply by summing over all $m$ \cite{FOMO}
\begin{equation}
    \epsilon(\omega) :=
    \epsilon'(\omega) + i \epsilon''(\omega) =
    1 + \sum_m \frac{f_m}{\omega_m^2 - \omega_0^2 - i \gamma \omega}
\end{equation}

\begin{figure}[H]
    \floatbox[{
    \capbeside
    \thisfloatsetup{capbesideposition={right,top}}}]{figure}[\FBwidth]
    {\caption{
        Example plots for the real and imaginary part of the refractive index
        $n = \eta + i\kappa$
        as calculated by the multiple resonance Lorentz model.
        \cite{FOMO}
    }
    \label{fig:bg:lorentz}}
    {\includegraphics[width=0.5\textwidth]{bg_lorentz}}
\end{figure}

\begin{figure}[H]
    \centering
    \begin{subfigure}{.5\textwidth}
        \centering
        \includegraphics[width=.8\linewidth]{bg_dielectric}
        \caption{}
        \label{fig:bg:dielectric}
    \end{subfigure}%
    \begin{subfigure}{.5\textwidth}
        \centering
        \includegraphics[width=.8\linewidth]{bg_metal}
        \caption{}
        \label{fig:bg:metal}
    \end{subfigure}
    \caption{Imaginary and real part of the dielectric function
    $\epsilon = \epsilon' + i \epsilon''$
    for dielectrics (a) and metals (b) in the visible spectrum.
    \cite{FOMO}}
    \label{fig:bg:dm}
    \end{figure}

In section \ref{sec:s_mats} we have already discussed how the dielectric function is related to the complex refractive index by
\begin{equation}
    \epsilon = n^2 = (\eta + i \kappa)^2
\end{equation}

and this relationship is used to generate figure \ref{fig:bg:lorentz}. We observe high $\kappa$, so high absorption, at the resonance frequencies in the from of Lorentz peaks. Between resonance frequencies generally
$\partial \eta / \partial \omega > 0$.
This is called normal dispersion and explains, for example, why red light is diffracted stronger than blue light.
Figure \ref{fig:bg:lorentz} is just a general example. To understand metasurfaces we need to discuss how the resonance frequencies are distributed in real materials.
\\

\indent
Dielectrics in the visible spectrum are well described by two resonance frequencies. One in the IR and one in the UV range. That means in the visible spectrum we have a vanishing imaginary part $\epsilon''$ of the dielectric function and normal dispersion as seen in figure \ref{fig:bg:dielectric}.
Metals on the other hand are characterized by their high availability in free charges, thereby eliminating the restoring force ($\omega_0 = 0$) and reducing the dielectric function to
\begin{equation}
    \epsilon(\omega) = 1 - \frac{\omega_\s p^2}{\omega^2 + i \gamma \omega}\, ,
\end{equation}

with the plasma frequency
$\omega_\s p^2 = f = \rho e^2 / \epsilon_0 m$.
This results in a large negative $\epsilon'$ and a positive $\epsilon''$ as seen in figure \ref{fig:bg:metal}. The negative $\epsilon'$ is responsible for the reflection of visible light at metallic surfaces.

\paragraph{Surface Plasmon Polaritons}~\\
Finally, we can discuss the mechanism behind the optical properties of plasmonic metasurfaces. These work by allowing the excitation of a special electromagnetic wave at a metal dielectric interface called Surface Plasmon Polariton (SPP). This wave is confined to and travels along the interface until its energy is lost via absorption or scattering. That means it enables some photons to be absorbed by coupling into the interface. At which wavelengths this conversion occurs depends on the metasurface geometry and can thus be tailored for a specific optical target.

\begin{figure}[H]
    \centering
    \includegraphics[width=.6\linewidth]{bg_md_interface}
    \caption{Metal dielectric interface with light in a TM mode.}
    \label{fig:bg:md_inderface}
\end{figure}

Suppose the described wave exists as a TM mode, as can be seen in figure \ref{fig:bg:md_inderface}, and has the form

\begin{equation}
\begin{aligned}
    E_x &= E_0 e^{i k_x x - k_z z} \\
    E_z &= E_0 \frac{k_x}{k_z} e^{i k_x x - k_z z}\\
    H_y &= H_0 e^{i k_x x - k_z z}
\end{aligned}
\end{equation}

That is, propagating along the $x$ axis and evanescently decaying along the $z$ axis. This wave does indeed satisfy Maxwell's equation and the continuity conditions discussed in section \ref{sec:s_mats} if and only if \cite{Maier2007}

\begin{equation} \label{eq:bg:con1}
    \frac{\epsilon_1}{k_{z1}} = - \frac{\epsilon_2}{k_{z2}}
\end{equation}

and

\begin{equation} \label{eq:bg:con2}
    k_x^2 + k_{zn}^2 = \epsilon_n \frac{\omega^2}{c^2}
    \qq{for}
    n = 1,2.
\end{equation}

Equation \eqref{eq:bg:con1} explains why SPP's are only possible at metal dielectric interfaces because $\epsilon'_1$ and $\epsilon'_2$ need to have different signs and this is fulfilled precisely for metals and dielectrics as can be seen in figure \ref{fig:bg:dm}. If we solve equation \eqref{eq:bg:con1} and \eqref{eq:bg:con2}, we can obtain the dispersion relation for SPP's

\begin{equation}
    k_x = \frac{\omega}{c}
    \qty(\frac{\epsilon_1 \epsilon_2}{\epsilon_1 + \epsilon_2})^{1/2}.
\end{equation}

This dispersion relation is shown in figure \ref{fig:bg:dispersion}. For small wave vectors SPP's behave like light but then they start to have increasingly lower energy $E = \hbar \omega$ compared to a free photon at the same momentum $\vb p = \hbar \vb k$. However, for a conversion Photon $\rightarrow$ SPP both energy and momentum have to be conserved.
Thus excitation of SPP's is not possible at a simple smooth metallic surface, but rather the wave vector of an incident photon has to be matched to that of an SPP of the same energy by the surface geometry.
\\

\indent
To summarize, photons can couple into a plasmonic metasurface by exiting a special electromagnetic wave called Surface Plasmon Polariton confined to the interface. The exact wavelength at which this conversion is possible depends on the metaatom geometry and can thus be easily tuned to create different optical behaviors.

\begin{figure}[H]
    \floatbox[{
    \capbeside
    \thisfloatsetup{capbesideposition={right,top}}}]{figure}[\FBwidth]
    {\caption{
        Dispersion relation for light and Surface Plasmon Polaritons in air (grey) and silica (black). Frequency and wave vector are normalized to the plasma frequency $\omega_\s p$ \cite{Maier2007}.
    }
    \label{fig:bg:dispersion}}
    {\includegraphics[width=0.55\textwidth]{bg_dispersion}}
\end{figure}


\subsection{Bit-Blasting}

\begin{frame}
  \frametitle{Bit-Blasting}

Our goal is to devise an automatic decision procedure (\smtsolver)
to check the satisfiability of a given \bitvector formula
\vfill
\pause
State-of-the-art techniques are based on reduction to SAT. It is
called {\bf bit-blasting}
\vfill
\pause
\begin{itemize}
  \item the formula is seen as a circuit, in which
        variables and constants are inputs, while
        other terms are intermediate nodes. The outermost
        Boolean connective or predicate represents the output
\vfill
\pause

  \item each variable is assigned to a vector of Boolean variables
        ($n$ variables for a variable of sort \SBv{n})
\vfill
\pause

  \item each intermediate node is assigned to a vector of
	Boolean \formulae
        ($n$ \formulae for a term of sort \SBv{n})
\end{itemize}

\end{frame}

\begin{frame}
  \frametitle{Bit-Blasting}

  $$
  (\w{a}{2} \band \w{b}{4}[1:0]) = (\w{c}{2} + \w{d}{2})
  $$
  \vfill
  \begin{overlayarea}{\textwidth}{5cm}
  \begin{center}
    \only<1|handout:0>{\scalebox{.35}{\input{bb_example_0.pdf_t}}}
    \only<2|handout:0>{\scalebox{.35}{\input{bb_example_1.pdf_t}}}
    \only<3|handout:0>{\scalebox{.35}{\input{bb_example_2.pdf_t}}}
    \only<4|handout:0>{\scalebox{.35}{\input{bb_example_3.pdf_t}}}
    \only<5|handout:0>{\scalebox{.35}{\input{bb_example_4.pdf_t}}}
    \only<6|handout:0>{\scalebox{.35}{\input{bb_example_5.pdf_t}}}
    \only<7|handout:0>{\scalebox{.35}{\input{bb_example_6.pdf_t}}}
    \only<8|handout:0>{\scalebox{.35}{\input{bb_example_7.pdf_t}}}
    \only<9->{\scalebox{.35}{\input{bb_example_8.pdf_t}}}
  \end{center}
  \end{overlayarea}
  \vfill
  \onslide<10->{
    $((b^1 \wedge a^1) \leftrightarrow (c^1 \oplus d^1 \oplus (c^0 \wedge d^0))) \wedge ((b^0 \wedge a^0) \leftrightarrow (c^0 \oplus d^0))$
  }

\end{frame}

\begin{frame}[fragile]
  \frametitle{Bit-Blasing Algorithm (1)}

  \tiny

  \begin{tabbing}
    \= asdf \= asdfasdfasdfasdfasdf \= asdfasdfasdf \= asdf \kill
    \> BB := $\{ \}$, C := $\{ \}$ \\
    \> \\
    \> {\bf Procedure} Bit-Blast-Term( $t$ : \SBv{n} term ) \\
    \> \\
    \> {\bf if} ( $t \in $ C ) {\bf return}; \> \> // If already in cache, skip \\
    \> {\bf else} C := C $\cup\ t$     \> \> // Put in cache \\
    \> \\ \pause
    \> {\bf if} ( $t$ is a \SBv{n} variable ) \\
    \> \> // Let $x$ be the name of the variable \\
    \> \> BB := BB $\cup\ \{ x \mapsto [\bit{x}{{n-1}}, \ldots, \bit{x}{0}] \}$ \\
    \> \> // where $\bit{x}{i}$ are fresh Boolean variables \\
    \> \\ \pause
    \> {\bf else if} ( $t$ is a \SBv{n} constant ) \\
    \> \> // Let $c$ be the constant \\
    \> \> BB := BB $\cup\ \{ c \mapsto [\bit{c}{{n-1}}, \ldots, \bit{c}{0}] \} $ \\
    \> \> // where $\bit{c}{i}$ is $\bot$ if the i-th bit of $c$ is $0$, $\top$ otherwise \\
    \> \\ \pause
    \> {\bf else if} ( $t$ is ($t_1 \band t_2$), and $t_1, t_2$ are \SBv{n} terms ) \\
    \> \> Bit-Blast-Term( $t_1$ ) \\
    \> \> Bit-Blast-Term( $t_2$ ) \\
    \> \> BB := BB $\cup\ \{ t \mapsto [\mbox{BB}(t_1, n-1) \wedge \mbox{BB}(t_2, n-1), \ldots, \mbox{BB}(t_1, 0) \wedge \mbox{BB}(t_2, 0)] \} $ \\
    \ldots
  \end{tabbing}

  where $\mbox{BB}(t, i)$ means:
  \begin{enumerate}
    \item retrieve the correspondence $t \mapsto [\bit{t}{n-1}, \ldots, \bit{t}{0}]$, and
    \item return $\bit{t}{i}$
  \end{enumerate}

\end{frame}

\begin{frame}[fragile]
  \frametitle{Bit-Blasing Algorithm (2)}

  \scriptsize

  \begin{tabbing}
    \= asdf \= asdf \= asdf \= asdf \kill
    \> {\bf Procedure} Bit-Blast( $\varphi$ : \SBv{n} formula ) \\
    \> \\
    \> {\bf if} ( $\varphi$ is $(t_1 = t_2)$, and $t_1, t_2$ are \SBv{n} terms ) \\
    \> \> Bit-Blast-Term( $t_1$ ) \\
    \> \> Bit-Blast-Term( $t_2$ ) \\
    \> \> BB := BB $\cup\ \{ \varphi \mapsto ((\mbox{BB}(t_1, n-1) \leftrightarrow \mbox{BB}(t_2, n-1)) \wedge
	                                      \ldots \wedge
				              (\mbox{BB}(t_1, 0) \leftrightarrow \mbox{BB}(t_2, 0))) \} $ \\
    \> \\ \pause
    \> {\bf else if} ( $\varphi$ is $(t_1 <_u t_2)$, and $t_1, t_2$ are \SBv{n} terms ) \\
    \> \> Bit-Blast-Term( $t_1$ ) \\
    \> \> Bit-Blast-Term( $t_2$ ) \\
    \> \> BB := BB $\cup\ \{ \ldots \}$ \\
    \> \\ \pause
    \> {\bf else if} ( $\varphi$ is $\varphi_1 \wedge \varphi_2$ are \SBv{n} formula ) \\
    \> \> Bit-Blast( $\varphi_1$ ) \\
    \> \> Bit-Blast( $\varphi_2$ ) \\
    \> \> BB := BB $\cup\ \{ \varphi \mapsto (\mbox{BB}(\varphi_1) \wedge \mbox{BB}(\varphi_2)) \}$ \\
    \ldots
  \end{tabbing}

  where $BB(\varphi)$ means:
  \begin{enumerate}
    \item retrieve the correspondence $\varphi \mapsto \psi$, and
    \item return $\psi$
  \end{enumerate}

\end{frame}

\begin{frame}
  \frametitle{SMT via Bit-Blasing}

  \begin{center}
  \scalebox{.5}{\input{bitblasting.pdf_t}}
  \end{center}

\end{frame}

\begin{frame}
  \frametitle{Bit-Blasing pros and cons}

  \scriptsize

  Pros
  \begin{itemize}
    \item Very easy to write, if compared to write a native \bitvector solver \pause
    \vfill
    \item Boolean model from SAT can be mapped back to a model for each \bitvector variable.
	  If $\w{x}{n}$ was bit-blasted as $x^{n-1}, \ldots, x^0$
	  \begin{itemize}
	    \item retrieve SAT assignment for each $x^i$ (e.g., $x^0 = \top$, $x^1 = \bot$)
	    \item construct actual value for $\w{x}{n}$ by mapping $\top$ to $1$ and $\bot$ to $0$ (e.g., $\w{x}{2} = 01$)
	  \end{itemize}
  \end{itemize}
  \vfill
  \pause
  Cons
  \begin{itemize}
    \item Does not scale very well. Consider the formula $\neg(\w{x}{n} = \w{0}{32}) \wedge (\w{x}{n} \band \w{y}{n}) = (\w{x}{n} + \w{y}{n})$.
          It is unsat for {\bf every} $n$. \pause But to prove it for $n=32$ requires $64$ Boolean variables, to prove it with $n=1024$
	  requires $2048$ Boolean variables
    \vfill
    \pause
    \item It destroys the structure of the formula. In the encoding $\w{x}{32}$ is not seen as a ``single object'' but each
          $x^i$ is unrelated and independent
  \end{itemize}

\end{frame}


\chapter{Statistical Mechanics Problems}
\begin{enumerate}
	\item A system of $\mathrm{N}$ particles has only two allowed state $\mathrm{A}$ and $\mathrm{B}$. The probability for $\mathrm{A}$ is $\mathrm{P}$ and for $\mathrm{B}$ is 1-P? What is the probability for the system to be in macrostate defined by the distribution of $(r, N-r) ?$
	\begin{answer}
		 \begin{align*}
		 	\text{The probability of finding }&\mathrm{r}\text{ particle in state } \mathrm{A}=\mathrm{P}^{r}\\
		 	\text{ The probability of finding }&N-r\text{ particle in state } B=(1-P)^{N-r}\\
		 \text{	The Total no. of-ways in which}&\text{ r particle can be choosen from }N \text{- particle is }{ }^{N} C_{r}=N ! / r !(N-r) !\\
		 \text{ The probability in which r particle }&\text{are in state $A$ and $N-r$ particle in state $B$ is }=\frac{N !}{r !(N-r) !} P^{r}(1-P)^{N-r}
		 \end{align*}
	\end{answer}
	\item A one dimensiional random walker takes step to left or right with equal probability. The probability that the random walker starting from orign is back to orign after $N$ even number of step,is
	 \begin{tasks}(2)
		\task[\textbf{a.}]$\frac{N !}{\left(\frac{N}{2}\right) !\left(\frac{N}{2}\right) !}\left(\frac{1}{2}\right)^{N}$
		\task[\textbf{b.}]$\frac{N!}{(\frac{N}{2})!(\frac{N}{2})!}$
		\task[\textbf{c.}]$2N!(\frac{1}{2})^{2N}$
		\task[\textbf{d.}] $N!(\frac{1}{2})^{N}$
	\end{tasks}
	\begin{answer}
		\begin{align*}
		\text{Probability} \quad P&=\frac{N!}{r~(N-r)}P(1-P)^{N-r}\\
		&=\frac{N !}{\left(\frac{N}{2}\right) !\left(\frac{N}{2}\right) !}\left(\frac{1}{2}\right)^{\frac{N}{2}}\left(1-\frac{1}{2}\right)^{N-\frac{N}{2}}=\frac{N!}{(\frac{N}{2})!(\frac{N}{2})!}\left( \frac{1}{2}\right) ^N
		\end{align*}
		Option \textbf{(a)} is correct
	\end{answer}
	\item Calculate the no. of microstates for a configuration of a system of $N$ distinguishable particles in which there are $n_1$ particles in a particle state $1 \& n_2$ particle in state $2,n_3$ particle .....$n_1$ particle in the $i$ th state.
	\begin{answer}
		\begin{align*}
		\text{Total No. of particle }&=N\\
		\text{No: of microstate for state}-1&={{N_C}_{n1}}\\
	\text{	No. of microstate for state }-2&=\mathrm{N}-\mathrm{n}_{1}{ }_{\mathrm{C}_{\mathrm{n}} 2}\\
\text{	No. of microstate for state }i^{\text {th }}&=N-n_{1}-n_{2} \ldots . n_{i}-1_{c_{n_{i}}}
\intertext{So\quad total number of microstate is}
\mathrm{N}_{\mathrm{C}_{\mathrm{n}_{1}}} \times \mathrm{N}-\mathrm{n}_{1} \mathrm{C}_{\mathrm{n}_{2}} &\times \mathrm{N}-\mathrm{n}_{1}-\mathrm{n}_{2} \mathrm{C}_{\mathrm{n}_{3}} \times \ldots \ldots \ldots \mathrm{N}-\mathrm{n}_{1}-\mathrm{n}_{2}-\mathrm{n}_{3}-\mathrm{ni}-1_{\mathrm{n}_{\mathrm{i}}}\\
=\frac{N !}{n_{1} !\left(N-n_{1}\right) !} &\times \frac{(N-n) !}{n_{2} !\left(N-n_{1}-n_{2}\right) !} \times \ldots \ldots \cdot \frac{\left(N-n_{1}-n_{2} \ldots n_{i}\right) !}{n_{i} !\left(N-n_{1}-n_{2} \ldots n_{i}\right) !}\\
=\frac{N !}{n_{i} !\left(N-n_{1}-n_{2} \ldots . n_{i}\right) !}&
-\text{ For distinguishable particle}\\
=\frac{1}{n_{i} !\left(N-n_{1}-n_{2} \ldots n_{i} !\right) !}& \rightarrow
\text{For indistinguishable particle}
		\end{align*}
	\end{answer}
	\item Four distinguishable coins are tossed a large no. of time write down the different microstate which may be observed \& the macrostate into which they would fall. Give the probability of the most probable macrostate.\\
	\begin{center}
	\begin{tabular}{|p{2.5cm}|p{2.7cm}|p{2.5cm}|p{2.7cm}|p{2.5cm}|}
		\hline Macrostate & Microstate coins having head up & Microstate coins having tail up & No.of microstate & Probability \\
		\hline $\mathrm{n}_{1}=4, \mathrm{n}_{2}=0$ & $\mathrm{a} \mathrm{b} \mathrm{c} \mathrm{d}$ & $-$ & 1 & $\frac{1}{16}$ \\
		\hline & $\mathrm{abc}$ & $\mathrm{d}$ & & \\
		$\mathrm{n}_{1}=3, \mathrm{n}_{2}=1$ & $\mathrm{bcd}$ & $\mathrm{a}$ & 4 & $\frac{4}{16}$ \\
		& $\mathrm{cda}$ & $\mathrm{b}$ & & \\
		& $\mathrm{dab}$ & $\mathrm{c}$ & & \\
		\hline
		 &$\mathrm{ab}$&$\mathrm{cd}$ & &\\
		 &$\mathrm{ac}$&$\mathrm{bd}$ & &\\
		 $n_1=2,n_2=3$&$\mathrm{ad}$&$\mathrm{bc}$ &6&$\frac{6}{16}$\\
		 &$\mathrm{bc}$&$\mathrm{ad}$ & &\\
		 &$\mathrm{bd}$&$\mathrm{ac}$ & &\\
		 &$\mathrm{cd}$&$\mathrm{ab}$ & &\\
		 \hline
		  &$\mathrm{a}$&$\mathrm{bcd}$ & &\\
		$n_1=1,n_2=3$&$\mathrm{b}$&$\mathrm{acd}$ &4 &$\frac{4}{16}$\\
		  &$\mathrm{c}$&$\mathrm{abd}$ & &\\
		 &$\mathrm{d}$&$\mathrm{abc}$ & &\\
		  \hline
		  	$n_1=0,n_2=1$&$-$&$\mathrm{abcd}$ &1 &$\frac{1}{16}$\\
		  	 \hline
	\end{tabular}
\end{center}
\begin{answer}
	\begin{align*}
	\text{Total no. of microstate }&=6\\
	\text{The probability of most probable state }&=6/16
	\end{align*}
\end{answer}
	\item An isolated system consist of two non-interacting spin $-1/2$ particles a \& b fixed in space \& kept in magnetic field B. Find out the total number of microstates allowed in the system.

	\begin{answer}
		$\left. \right. $\\
		\begin{table}[H]
	\begin{tabular}{|p{2.5cm}|p{2.5cm}|p{2.5cm}|p{2.5cm}|p{2.5cm}|}
		\hline
		Systemstate or macrostate  &Particlestate or Microstate & Magnetic moment &Energy  &No.of microstate  \\ \hline
		1& U \quad U&$2\mu_0$ & $-2\mu_0 B$& 1   \\ \hline
	 \multirow{ 2}{*}{1} & U \quad D & 0 & 0 & \multirow{ 2}{*}{2}\\
		 & U \quad D & 0 & 0 &  \\ \hline
		 3&D\quad D&$-2\mu_0$& $+-2\mu_0B$&1\\
		 \hline
	\end{tabular}
\end{table}
\begin{align*}
\intertext{Total no. of microstate $=4$ for $\operatorname{Spin}-\frac{1}{2}$ two particle system no. of accessible microstate corresponding to energy.}
E=0\text{ is }2&
\intertext{The no. of microstate for $\mathrm{N}$ no. of Spin S particle}
(2 S+1)^{N}&
\intertext{The probability of getting a macrostate in which there are r particle out of N Spin $1 / 2$ particle in spin up state}
\mathrm{N}_{\mathrm{C}_{\mathrm{r}}} \times \frac{1}{2^{\mathrm{N}}}
\end{align*}
	\end{answer}
\item We have equal amount of two identical ideal gases at the same temperature $T$ but at different pressure $P_1$ and $P_2$ in two containers of volume $V_1$ and $V_2$ respectively. The containers are connected. Find the change in entropy (The temperature after mixing also ramains same)
\begin{answer}
	\begin{align*}
	S&=Nk_B \ \ell n\left( \frac{V}{N}\right)+\frac{3}{2}Nk_B \left[\frac{5}{3}+\ell n \left(\frac{2\pi mk_BT}{h^2} \right) \right] \\
	\sum\limits_{i=1}^{2}Si&=N_1k_B\ \ell n\frac{V_i}{N}+\frac{3}{2}N_1k_B \left[\frac{5}{3}+\ell n \left(\frac{2\pi mk_BT}{h^2} \right) \right] \\
	&=Nk_B\ \ell n\frac{V_1}{N}+\frac{3}{2}Nk_B \left[\frac{5}{3}+\ell n \left(\frac{2\pi mk_BT}{h^2} \right) \right]+\\
		&=Nk_B\ \ell n\frac{V_2}{N}+\frac{3}{2}Nk_B \left[\frac{5}{3}+\ell n \left(\frac{2\pi mk_BT}{h^2} \right) \right]+\\
		&\left\{\begin{array}{l}P_{1} V_{1}=N k_{B} T \Rightarrow \frac{V_1}{N}=\frac{k_B T}{P_1}\\ P_{2} V_{2}=N k_{B} T \Rightarrow \frac{V_2}{N}=\frac{k_B T}{P_2} \end{array}\right.\\
		\sum\limits_{i=1}^{2}Si&=Nk_B\ \ell n\frac{k_B T}{P_1}+\frac{3}{2}Nk_B \left[\frac{5}{3}+\ell n \left(\frac{2\pi mk_BT}{h^2} \right) \right] +\\
		&=Nk_B\ \ell n\frac{k_B T}{P_2}+\frac{3}{2}Nk_B \left[\frac{5}{3}+\ell n \left(\frac{2\pi mk_BT}{h^2} \right) \right]
		\intertext{\textbf{Total entropy:}}
		S_T&=2Nk_B\ \ell n\frac{V_1+V_2}{2N}+\frac{3}{2}\times 2Nk_B \left[\frac{5}{3}+\ell n \left(\frac{2\pi mk_BT}{h^2} \right) \right]
		\intertext{Change in entropy}
		\Delta S&=S_T-\sum\limits_{i=1}^{2}Si=2Nk_B\ \ell n\frac{V_1+V_2}{2N}-Nk_B \ \ell n\frac{k_B T}{P_1}-NK \ \ell n\frac{k_B T}{P_2}
		\intertext{Let after mixing pressure $=P$ and after mixing total No of particle $=N+N=2 N$}
		\end{align*}
		\begin{align*}
		&\mathrm{P}\left(\mathrm{V}_{1}+\mathrm{V}_{2}\right)=2 \mathrm{Nk}_{\mathrm{B}} \mathrm{T}\hspace{6cm}\mathrm{P}\left(\mathrm{V}_{1}+\mathrm{V}_{2}\right)=2 \mathrm{Nk}_{\mathrm{B}} \mathrm{T}\\
		&\frac{\left(\mathrm{V}_{1}+\mathrm{V}_{2}\right)}{2 \mathrm{~N}}=\frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{P}}\hspace{6.5cm}\frac{1}{\mathrm{P}}=\frac{\mathrm{V}_{1}+\mathrm{V}_{2}}{2 \mathrm{Nk}_{\mathrm{B}} \mathrm{T}}=\frac{\mathrm{V}_{1}}{2 \mathrm{Nk}_{\mathrm{B}} \mathrm{T}}+\frac{\mathrm{V}_{2}}{2 \mathrm{Nk}_{\mathrm{B}} \mathrm{T}}\\
		&\Delta \mathrm{S}=2 \mathrm{Nk}_{\mathrm{B}} \ \ell n \frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{P}}-\mathrm{Nk}_{\mathrm{B}} \ell \mathrm{n} \frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{P}_{1}}-\mathrm{Nk}_{\mathrm{B}}  \ \ell n \frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{P}_{2}}\hspace{2cm}\frac{1}{\mathrm{P}}=\frac{1}{2}\left(\frac{1}{\mathrm{P}_{1}}+\frac{1}{\mathrm{P}_{2}}\right)\\
		&=N k_{\mathrm{B}}  \ \ell n \left(\frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{P}}\right)^{2}-\mathrm{Nk}_{\mathrm{B}}  \ \ell n \frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{P}_{1}}-\mathrm{Nk}_{\mathrm{B}}  \ \ell n \frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{P}_{2}}\hspace{2cm}P=\frac{2 P_{1} P_{2}}{P_{1}+P_{2}}\\
		&=N k_{B}  \ \ell n \frac{\left(\frac{k_{B} T}{P}\right)^{2}}{\left(\frac{k_{B} T}{P_{1}}\right)\left(\frac{k_{B} T}{P_{2}}\right)}\\
		&=N k_{B}  \ \ell n \left[\frac{\left(k_{B} T\right)^{2}\left(P_{1}+P_{2}\right)^{2}}{\left(2 P_{1} P_{2}\right)^{2}} \times \frac{P_{1} P_{2}}{\left(k_{B} T\right)^{2}}\right]\\
		\Delta \mathrm{S}&=\mathrm{Nk}_{\mathrm{B}} \ell \mathrm{n} \frac{\left(\mathrm{P}_{1}+\mathrm{P}_{2}\right)^{2}}{4 \mathrm{P}_{1} \mathrm{P}_{2}}\\
		\Delta \mathrm{S}&=\mathrm{Nk}_{\mathrm{B}}  \ \ell n \frac{\left(\mathrm{P}_{1}+\mathrm{P}_{2}\right)^{2}}{4 \mathrm{P}_{1} \mathrm{P}_{2}}\quad\text{ Here we get }\Delta S>0\\
		\intertext{(We are considering identical particle. So it is a reversible case only when the particle density is same otherwise for identical gas also we will get $\Delta S>0$)}
	\end{align*}
\end{answer}
	\item Calculate the number of microstate for a free particle in 3-dimension that have momentum $p$.
	\begin{answer}
		\begin{align*}
		\mathrm{d} \tau&=\mathrm{dx} d y d z d p_{x} d p_{y} d p_{z}\\
	\textbf{	Allowed Phase Space Volume}=&\int \mathrm{d} \tau=\int \mathrm{dxdydz} \int \mathrm{dp}_{\mathrm{x}} \mathrm{dp}_{\mathrm{y}} \mathrm{dp}_{\mathrm{z}}\\
	&=V \int p^{2} d p \int_{0}^{\pi} \sin \theta d q \int_{0}^{2 \pi} d \phi=V \frac{4}{3} \pi p^{3}\\
\text{	No. of microstate }&=\frac{\mathrm{V}}{\mathrm{h}^{3}} \times \frac{4}{3} \pi \mathrm{p}^{3}
\intertext{Note : for 2. dimension.}
\int \mathrm{dx}&=\int \mathrm{dx} \int \mathrm{dy} \int \mathrm{dp}_{\mathrm{x}} \int \mathrm{dp}_{\mathrm{x}}\\
&=\int \mathrm{d}^{2} \mathrm{q} \cdot \mathrm{d}^{2} \mathrm{p} \quad=\mathrm{A} \cdot \pi \mathrm{p}^{2} \quad\{\text{ where }\mathrm{A}= \text{area} \}
\intertext{Since wave polarize in 2 dimension}
\int \mathrm{dr}&=2 \mathrm{~A} \pi \mathrm{p}^{2}
\intertext{If momentum lies between $p$ and $p+d p$}
\therefore\text{ No. of microstate }&=\frac{2 \mathrm{~A}}{\mathrm{~h}^{2}} 2 \pi \mathrm{pdp}=\frac{4 \mathrm{~A} \pi}{\mathrm{h}^{2}} \mathrm{pdp}
\intertext{If momentum lies between $\mathrm{p}$ and $\mathrm{p}+\mathrm{dp}$ then no. of microstate $=\frac{\mathrm{V}}{\mathrm{h}^{3}} 4 \pi \mathrm{p}^{2} \mathrm{dp}$}
\text{Density of state }&=\frac{\text { no. of state }}{\text { Volume }}\text{ or} \frac{\text { no. of state }}{\text { energy int erval }}
		\end{align*}
	\end{answer}
	\item Calculate the number of microstates accessible to the photon having frequency between $\nu$ and $\nu+d\nu$ confined to a 3 dimentional cavity of volume $V$
	 \begin{answer}
	 	\begin{align*}
	 	\intertext{No. of microstate in frequency range 0 to $\nu$}
	 	\Omega&=\frac{V}{h^{3}} \frac{4}{3} \pi(2 m E)^{3 / 2} \hspace{3cm}
	 	E=\frac{P^{2}}{2 m} \Rightarrow P=\sqrt{2 m} E\\
	 	&=\frac{\mathrm{V}}{\mathrm{h}^{3}} \frac{4}{3} \pi\left(\mathrm{P}^{2}\right)^{3 / 2}\hspace{3cm} \mathrm{P}=\frac{\mathrm{E}}{\mathrm{C}}=\frac{\mathrm{h\nu}}{\mathrm{C}}\\
	 	\Omega&=\frac{\mathrm{V}}{\mathrm{h}^{3}} \frac{4}{3} \pi \mathrm{P}^{3}=\frac{\mathrm{V}}{\mathrm{h}^{3}} \frac{4}{3} \pi\left(\frac{\mathrm{h\nu}}{\mathrm{c}}\right)^{3}=\frac{4}{3} \frac{\pi \mathrm{V\nu}^{3}}{\mathrm{c}^{3}}
	 	\intertext{No. of microstates in the frequency range $\nu$ and $v+d \nu$ is}
	 	\mathrm{d} \Omega&=\frac{4}{3}\  \frac{\pi \mathrm{V}}{\mathrm{c}^{3}} 3 \mathrm{\nu}^{2} \mathrm{~d} \mathrm{v}=\frac{4 \pi \mathrm{V}}{\mathrm{c}^{3}} \mathrm{\nu}^{2} \mathrm{~d} \mathrm{\nu}
	 	\intertext{Density of state}
	 	g(E)&=\text{ no. of states per unit energy range}\\
	 \text{	we have }\quad
	 	\Omega&=\frac{V}{h^{3}} \ \frac{4}{3} \ \pi(2 m E)^{3 / 2}\\
	 	\mathrm{d} \Omega&=\frac{\mathrm{V}}{\mathrm{h}^{3}} \ \frac{4}{3}\  \cdot \frac{3}{2} \pi(2 \mathrm{mE})^{1 / 2} 2 \mathrm{~m} \mathrm{dE}\\
	 	&=\frac{V}{h^{3}} \cdot 2 \pi(2 m)^{3 / 2} E^{1 / 2} d E\\
	 	&\left[\mathrm{E}=\frac{\mathrm{P}^{2}}{2 \mathrm{~m}} \Rightarrow 2 \mathrm{~m}\frac{P^2}{E}-\frac{E^2}{c^2 E}=\frac{E}{c^2}\right]\\
	 	&=\frac{V}{h^{3}} 2 \pi \left(\frac{E}{c^2} \right) ^{\frac{3}{2}}E^{\frac{1}{2}}dE\\
	 	&=\frac{V}{h^3}\ \frac{2\pi}{c^3}E^2dE\\
	 	g(E)&=\frac{d\Omega}{dE}=\frac{2\pi VE^2}{h^3 c^3}
	 	\end{align*}
	 \end{answer}
 \section{Canonical Ensemble}
	\item Show that the partition fuction of two independent (non-interacting)
	system $i$ and $j$ is given by
	$$Z_{ij}=Z_i\times Z_j$$
	\begin{answer}
		\begin{align*}
		\intertext{We know that}
		P_{i}&=\frac{e^{-\beta E_{i}}}{Z_{i}}, P_{j}=\frac{e^{-\beta E_{j}}}{Z_{j}}\\
		E&=E_{i}+E_{j}, P_{i j}=\frac{e^{-\beta\left(E_{i}+E_{j}\right)}}{Z_{i j}}\\
		P_{i j}&=\frac{e^{-\beta E}}{Z_{i i}}=\frac{e^{-\beta\left(E_{i}+E_{j}\right)}}{Z_{i i}}=\frac{e^{-\beta E_{i}} \times e^{-\beta E_{j}}}{\dot{Z}_{i i}}\\
		\mathbf{P}_{\mathrm{ij}}&=\mathrm{P}_{\mathrm{i}} \times \mathrm{P}_{\mathrm{j}}\\
		\frac{e^{-\beta E_{i}} \times e^{-\beta E_{j}}}{Z_{i j}}&=\frac{e^{-\beta E_{i}}}{Z_{i}} \times \frac{e^{-\beta E_{j}}}{Z_{i}}\\
		\text{So  }\quad Z_{\mathrm{ij}}&=Z_{\mathrm{i}} \times Z_{\mathrm{j}}\\
	\text{	In general }Z_{i j k} \cdots \cdots&=Z_{i} \times Z_{j} \times Z_{k} \times \cdots \\
	\text{Total partition function }Z&=Z_{\text {Rotational motion }} \times Z_{\text {Translational motion }} \times Z_{\text {Vibrational motion }} \times \ldots \ldots \ldots \ldots \ldots
		\end{align*}
	\end{answer}
	\item A system consist of three independent particles localised in space. Each particle have two states of energy $O$ and $E$. When the system is in thermal equilibrium with a heat bath at temperature $T$. Calculate its partition function?
 	\begin{answer}
 		Method :-1
 		\begin{align*}
 		\mathrm{Q}_{\mathrm{N}}(\mathrm{V}, \mathrm{T})&=\sum_{\mathrm{r}} \mathrm{g}_{\mathrm{r}} \mathrm{e}^{-\beta \mathrm{E}_{\mathrm{r}}}\\
 		Q_1 (V,T)=\sum e^{\beta E_r}=1+e^{\beta E}\colorbox{red}{Not completed}
 		\end{align*}
 	\end{answer}
	\item The partition function for two Bose particle each of which can occupy any of the enetgy level $0$ and $E$
	 \begin{tasks}(2)
		\task[\textbf{a.}]$1+e^{-2 E / K T}+2 e^{-E / K T}$
		\task[\textbf{b.}] $1+e^{-2 E / K T}+e^{-E / K T}$
		\task[\textbf{c.}]$2 \mathrm{e}^{-2 \mathrm{E} / \mathrm{KT}}+\mathrm{e}^{-\mathrm{E} / \mathrm{KT}}$
		\task[\textbf{d.}] $e^{-2 E / K T}+e^{-E / K T}$
	\end{tasks}
	\begin{answer}$\left. \right. $\\
		\renewcommand*{\arraystretch}{2}
		\begin{tabular}{llllll}
			& $\mathbf{0}$ & $\mathbf{E}$ & Total energy & \multicolumn{2}{c}{ degeneracy } \\
			1 & aa & 0 & 0 & 1 & $\left(g_{1}\right)$ \\
			2 & a & a & E & 1 & $\left(g_{2}\right)$ \\
			3 & 0 & a & 2 E & 1 & $\left(g_{3}\right)$
		\end{tabular}
	\begin{align*}
	\mathrm{Q}_{\mathrm{N}}(\mathrm{V}, \mathrm{T})&=\sum_{\mathrm{r}} \mathrm{g}_{\mathrm{r}} \mathrm{e}^{-\beta \mathrm{E}_{\mathrm{r}}}=\mathrm{g}_{1} \mathrm{e}^{-\beta \mathrm{E}_{1}}+\mathrm{g}_{2} \mathrm{e}^{-\beta \mathrm{E}_{2}}+\mathrm{g}_{3} \mathrm{e}^{-\beta \mathrm{E}_{3}}\\
	&=1+\mathrm{e}^{-\beta E}+\mathrm{e}^{-2 \beta E}
	\end{align*}
	Correct answer is option \textbf{(b)}
	\end{answer}
	\item The partition function of single gas molecule is $Z\alpha$. The partition fuction of $N$ such non-interacting gas molecule is given by
	 \begin{tasks}(2)
		\task[\textbf{a.}]$\frac{(Z \alpha)^{N}}{N !}$
		\task[\textbf{b.}]$(Z \alpha)^{N}$
		\task[\textbf{c.}] $\mathrm{NZ} \alpha$
		\task[\textbf{d.}] $(Z \alpha)^{N} / N$
	\end{tasks}
	\begin{answer}
		\begin{align*}
		\text{For indistinguishable particle }&\frac{(Z \alpha)^{N}}{N !}\\
		\text{for distinguishable particle }&(Z \alpha)^{N}
		\end{align*}
	\end{answer}
	\item A system has energy level $E_0,2E_0,3E_0....$ where the excited state are triply degenerate. Four non-interacting Bosons are placed in this system. If the total energy of these Bosons is $5E_0$. The number of microstate is
	 \begin{tasks}(2)
		\task[\textbf{a.}]2
		\task[\textbf{b.}]3
		\task[\textbf{c.}]4
		\task[\textbf{d.}]5
	\end{tasks}
	\begin{answer}
		\begin{align*}
		\text{Total energy }&=5 \mathrm{E}_{0}
		\intertext{For this 3 bosons must be in $E_0$ state and $1$ in $2E_0$ state.}
		\end{align*}
		\begin{figure}[H]
			\centering
			\includegraphics[height=4cm,width=4cm]{SM-problem-01}
		\end{figure}
		Correct answer is option \textbf{(b)}
	\end{answer}
	\item An ensemble of quantum harmonic oscillator is kept at a finite temperature $T$
	$$\mathrm{T}=\frac{1}{\mathrm{k}_{\mathrm{B}} \beta} \quad
 k_B	\text{-Boltzmann constant}$$
 The partition function of a single oscillator with energy $\left( n+1/2\right) \hbar\omega$ is given by
  \begin{tasks}(2)
 	\task[\textbf{a.}] $Z=\frac{e^{-\beta \hbar \omega / 2}}{1-e^{-\beta \hbar \omega}}$
 	\task[\textbf{b.}]$Z=\frac{e^{-\beta \hbar \omega / 2}}{1+e^{-\beta \hbar \omega}}$
 	\task[\textbf{c.}]$Z=\frac{1}{1-e^{-\beta \hbar \omega}}$
 	\task[\textbf{d.}] $Z=\frac{1}{1+e^{-\beta \hbar \omega}}$
 \end{tasks}
	\begin{answer}
		\begin{align*}
		\text{(i) }Q_{N}\left(V_{1} T\right)&=\sum_{r} e^{-\beta E}=\sum_{n} e^{-\beta(n+1 / 2) \hbar w}\\
		&=e^{-\beta \hbar \omega / 2}+e^{-3 / 2 \beta \hbar \omega}+e^{-5 / 2^{\beta \hbar \omega}}+\ldots \ldots\\
		&=\mathrm{e}^{-\beta \hbar \omega / 2}\left(1+\mathrm{e}^{-\beta \hbar \omega}+\mathrm{e}^{-\beta \hbar \omega}+\ldots \ldots\right.\\
		&=\mathrm{e}^{-\beta \hbar \omega / 2}\left(\frac{1}{1-\mathrm{e}^{-\beta \hbar \omega}}\right)\text{ use G.P, formula}\hspace{0.5cm}\left\{\begin{array}{lll}R=1+r+\pi^{2} \ldots \ldots \ldots \ldots \ldots \infty & \text { G.P. } \\ \sum_{0}^{\infty} R=\frac{1}{1-r} & r<1 &  \end{array}\right\}
		\end{align*}
	\end{answer}
	\item The average number of energy quanta of the oscillator is given by
	 \begin{tasks}(2)
		\task[\textbf{a.}] $\langle\mathrm{n}\rangle=\frac{1}{\mathrm{e}^{\beta \hbar \omega}-1}$
		\task[\textbf{b.}]$\langle\mathrm{n}\rangle=\frac{\mathrm{e}^{-\beta \hbar \omega}}{\mathrm{e}^{\beta \hbar \omega}-1}$
		\task[\textbf{c.}]$\langle\mathrm{n}\rangle=\frac{1}{\mathrm{e}^{\beta \hbar \omega}+1}$
		\task[\textbf{d.}] $\langle n\rangle=\frac{e^{-\beta \hbar \omega}}{e^{\beta \hbar \omega}+1}$
	\end{tasks}
	\begin{answer}
		\begin{align*}
		\intertext{(i) The average energy of a quantum oscillator is given by}
		\mathrm{U}&=\mathrm{N}\left[1 / 2 \hbar \omega+\frac{\hbar \omega}{\mathrm{e}^{\beta \hbar \omega-1}}\right]
		\intertext{If we neglect the zero point energy the average energy bocome}
	\mathrm{U}&=\frac{\mathrm{N} \hbar \omega}{\mathrm{e}^{\beta \hbar \omega}-1}\\
	\text{Avarege number of energy quanta}&=\frac{U}{N\hbar \omega}\\\colorbox{red}{Not completed}
		\end{align*}
	\end{answer}
	\item In a particular salt among $n$ atoms each atom has a spin $1/2$ and is associated with a magnetic dipolemoment $\mu_B$. The distance between the magnetic atoms is large enough to make the interaction between them negligible. In otherwords our system is an idealised spin system. The salt is placed in an external magnetic field $B$. Find the partition function of the individual atom and that for the salt
	\begin{answer}
		\begin{align*}
		\mathrm{Q}_{1}(\mathrm{~V}, \mathrm{~T})&=\sum_{\mathrm{r}} \mathrm{e}^{-\beta \mathrm{E}_{\mathrm{r}}}\qquad E=\mu_B\ B =\text{for} S=-\frac{1}{2}\qquad \left\lbrace\text{ where }\mu_B=\text{ Bohr magneton }\right\rbrace \\
		&=e^{-\beta\mu_B\ B}+e^{\beta\mu_B\ B}
		\intertext{Since the salt consist of $N$ number of particles}
		Q_N(V,T)&=[Q_1(V,T)]^N=[e^{-\beta\mu_B\ B}+e^{\beta\mu_B\ B}]^N \rightarrow \text{distinguishable}\\
		\colorbox{red}{Not completed}
		\end{align*}
	\end{answer}
	\item Consider a system of two identical particles which may occupy any of the three levels. The lowest energy state $\varepsilon_0=0$ is double degenerates. The system is in thermal equilibrium at temperature T. Determine the partition function and everage energy of the system if particles obey
	(i) M-B statistics\\
	(ii) B-E statistics\\
	(iii) F-D statistics.
	\begin{answer}
		\begin{align*}
		\text{(i) For $M-B$ }&\text{statistics, $g_{i}=16$}\\
		\mathrm{Q}_{\mathrm{N}}(\mathrm{V}, \mathrm{T})&=\Sigma g_{r} e^{-\beta E_{r}}=4+4 e^{-\beta \epsilon}+5 e^{-2 \beta \epsilon}+2 e^{-3 \beta \epsilon}+e^{-4 \beta \epsilon}\\
		\langle\mathrm{E}\rangle&=-\frac{\partial}{\partial \beta} \ln \mathrm{Q}_{\mathrm{N}}(\mathrm{V}, \mathrm{T})=\frac{\in\left[4 e^{-\beta \epsilon}+10 e^{-2 \beta \epsilon}+6 e^{-3 \beta \epsilon}+4 e^{-4 \beta \epsilon}\right]}{Q_{N}(V, T)}\\
		\langle E\rangle&=\frac{\in\left[2 e^{-\beta \epsilon}+6 e^{-2 \beta \epsilon}+3 e^{-3 \beta \epsilon}+4 e^{-4 \beta \epsilon}\right]}{Q_{N}(V, T)}
		\intertext{(iii) For F-D statistics:}
		Q_{N}(V, T)&=1+2 e^{-\beta \epsilon}+2 e^{-\beta \epsilon}+e^{-3 \beta \epsilon}\\
		\langle E\rangle&=\frac{\in\left(2 e^{-\beta \epsilon}+4 e^{-2 \beta \epsilon}+3 e^{-3 \beta \epsilon}\right)}{Q_{N}(V, T)}
		\end{align*}
	\end{answer}
\end{enumerate}
\section{Thermodynamics of Ensemble}
\begin{enumerate}
	\item Consider a system of an extreme relativistic gas consisting of $3 \mathrm{~N}$ particles moving in 1 dimension, Show that the partition function in this case is given by $\frac{1}{3 \mathrm{~N} !}\left[2 \mathrm{~L}\left(\frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{hc}}\right)\right]^{3 \mathrm{~N}} . \mathbf{L}$ being the length of the space available. Study the thermodynamics and check that weather $\mathrm{PV}=\frac{1}{3} \mathrm{U} \cdot \frac{\mathrm{U}}{\mathrm{N}}=3 \mathrm{k}_{\mathrm{B}} \mathrm{T}, \gamma=\frac{4}{3}$ or not.
	\begin{answer}
		\begin{align*}
		\mathrm{Q}_{\mathrm{N}}(\mathrm{V}, \mathrm{T})&=\left[\mathrm{Q}_{1}(\mathrm{~V}, \mathrm{~T})\right]^{\mathrm{N}}=\frac{1}{\mathrm{~N} !}\left[\frac{1}{\mathrm{~h}^{3}} \int \mathrm{e}^{-\beta \mathrm{H}} \mathrm{d} \omega\right]^{\mathrm{N}}\\
		&=\frac{1}{N ! h^{3 N}} \int e^{-\beta P c} d^{3 N} p d^{3 N} q\\
		\text{For extreme relativistic case }\mathrm{H}&=\mathrm{pc}\text{ and } \mathrm{d} \omega=\mathrm{d}^{3 \mathrm{~N}} \mathrm{p} \mathrm{d}^{3 \mathrm{~N}} \mathrm{q}\\
		\mathrm{Q}_{\mathrm{N}}(\mathrm{V}, \mathrm{T})&=\frac{\mathrm{V}^{\mathrm{N}}}{\mathrm{N} ! \mathrm{h}^{3 \mathrm{~N}}}\left[\int \mathrm{e}^{\frac{-\mathrm{pc}}{\mathrm{kT}}} \mathrm{d}^{3 \mathrm{~N}} \mathrm{p}\right]^{\mathrm{N}}\left(\int d^{3 \mathrm{~N}} \mathrm{q}=\int \mathrm{d}^{3} \mathrm{q}_{1} \mathrm{~d}^{3} \mathrm{q}_{\mathrm{2}} .....\mathrm{d}^{3} \mathrm{q}_{N}\right)\\
		&=\frac{\mathrm{V}^{\mathrm{N}}}{\mathrm{N} ! \mathrm{h}^{3 \mathrm{~N}}}\left[\int\limits_{0}^{\infty} \mathrm{e}^{\frac{-\mathrm{pc}}{\mathrm{kT}}}{ 4\pi p }^2\ pd \right]^N\\
		&=\frac{(4 \pi V)^{N}}{N ! h^{3 N}}\left[\int\limits_{0}^{\infty} \mathrm{e}^{\frac{-\mathrm{pc}}{\mathrm{kT}}}{  p }^2\ pd \right]^N\\
		\text{Let }\frac{\mathrm{pc}}{\mathrm{k}_{\mathrm{B}} \mathrm{T}}&=\mathrm{x} \Rightarrow \frac{c}{k_B T} dp=dx\\
		&\frac{(4 \pi \mathrm{V})^{\mathrm{N}}}{\mathrm{N} ! \mathrm{h}^{3 \mathrm{~N}}}\left[\int_{0}^{\infty} \mathrm{e}^{-\mathrm{x}}\left(\frac{k_B Tx}{c}\right)\ \frac{k_B T}{c}dx    \right]=\frac{(4\pi V)}{N!h^{3N}}\left[ \left(\frac{k_B T}{c} \right) \int e^{-x}x^2\ dx\right] ^N\\
		&\frac{(4 \pi \mathrm{V})^{\mathrm{N}}}{\mathrm{N} ! \mathrm{h}^{3 \mathrm{~N}}}\left[ \left( \frac{k_B T}{c}\right)^3 2!\right] ^N\\
		\mathrm{Q}_{\mathrm{N}}(\mathrm{V}, \mathrm{T})&=\frac{1}{\mathrm{~N} !}\left[8 \pi \mathrm{V}\left(\frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{hc}}\right)^{3}\right]^{\mathrm{N}}\\
		\ell n \ \mathrm{Q}_{\mathrm{N}}(\mathrm{V}, \mathrm{T})&=\mathrm{N} \ell n 8 \pi \mathrm{V}\left(\frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{hc}}\right)^{3}-\log N !=N \ell n 8 \pi V\left(\frac{k_{B} T}{h c}\right)^{3}-N \ell n N+N\\
		&=\mathrm{N} \ell \mathrm{n} \frac{8 \pi \mathrm{V}}{\mathrm{N}}\left(\frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{hc}}\right)^{3}+\mathrm{N}\\
		A&=-k_{B} T \ell n Q_{N}(V, T)=-N k_{B} T \ell n \frac{8 \pi V}{N}\left(\frac{k_{B} T}{h c}\right)^{3}-N k_{B} T\\
		P&=-\left(\frac{\partial A}{\partial V}\right)=N k_{B} T \frac{\frac{8 \pi}{N}\left(\frac{k_{B} T}{h c}\right)^{3}}{\frac{8 \pi V}{N}\left(\frac{k_{B} T}{h c}\right)^{3}}=\frac{N k_{B} T}{V}\\
		\Rightarrow \quad U&=-T^{2} \frac{\partial}{\partial T}\left(\frac{A}{T}\right)\\
	\text{	Now, }U&=T^{2} \frac{\partial}{\partial T}\left[N k_{B} \ln \frac{8 \pi V}{N}\left(\frac{k_{B} T}{h c}\right)^{3}+N k_{B}\right]\\
	&=\mathrm{T}^{2} \mathrm{Nk} \times \frac{\frac{8 \pi \mathrm{V}}{\mathrm{N}}\left(\frac{\mathrm{k}_{\mathrm{B}}}{\mathrm{hc}}\right)^{3}, 3 \mathrm{~T}^{2}}{\frac{8 \pi \mathrm{V}}{\mathrm{N}}\left(\frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{hc}}\right)^{3}}=\frac{3 \mathrm{k}_{\mathrm{B}} \mathrm{NT}^{4}}{\mathrm{~T}^{3}}=3 \mathrm{Nk}_{\mathrm{B}} \mathrm{T}\\
	\mathrm{U}&=3 \mathrm{Nk}_{\mathrm{B}} \mathrm{T}\\
\text{	Now, }\quad \mathrm{P}&=\left(\frac{\partial \mathrm{A}}{\partial \mathrm{V}}\right)_{\mathrm{T}, \mathrm{M}}=\mathrm{PV}=\frac{Nk_B T}{V}\times V=N k_B T\\
\text{and }\frac{1}{3} U&=\frac{3 U k_{B} T}{3}=N k_{B} T\\
\text{So we get }\mathrm{PV}&=\frac{1}{3} \mathrm{U} \quad \text{or}\quad P=\frac{1}{3}\ \frac{U}{V}\\
\text{Now }\mathrm{dQ}&=\mathrm{dU}+\mathrm{PdV}\\
\text{For adiabic process }&\mathrm{dQ}=0\\
\mathrm{dU}+\mathrm{PdV}&=0 \Rightarrow \mathrm{d}(3PV)+PdV=0\\
3 \mathrm{PdV}&+3 \mathrm{VdP}+\mathrm{PdV}=0 \quad \text{or} \quad 4PdV=3VdP\\
\frac{\mathrm{dP}}{\mathrm{P}}&=-\frac{4}{3} \frac{\mathrm{dV}}{\mathrm{V}}\\
\text{Integrating }\ell nP&=-\frac{4}{3} \ell n V+c-\ell n V^{4 / 3}+c\\
\ell n\mathrm{P}+\ell n \mathrm{V}^{4 / 3}&=\text{ constant,}\quad \mathrm{PV}^{4 / 3}= \text{ constant}\\
\text{For this we know }\gamma&=\frac{4}{3}
		\end{align*}
		\begin{align*}
		\text{(2) }\mathrm{Q}_{\mathrm{N}}(\mathrm{V}, \mathrm{T})&=\left[\mathrm{Q}_{1}(\mathrm{~V}, \mathrm{~T})\right]^{3 \mathrm{~N}}=\frac{1}{3 \mathrm{~N} !}\left[\frac{1}{\mathrm{~h}} \int_{\infty}^{\infty} \mathrm{e}^{-\beta \mathrm{Pc}} \mathrm{dp} \mathrm{dq}\right]^{3 \mathrm{~N}}\\
		&=\frac{1}{3 N ! h^{3 N}} L^{3 N}\left|\int^{\infty} e^{-\frac{P_{c}}{k T}} d p\right|^{3 N} \quad\{L \rightarrow\text{ Length }\}\\
		\text{Let }\frac{\mathrm{pc}}{\mathrm{k}_{\mathrm{B}} \mathrm{T}}&=\mathrm{x} \Rightarrow \mathrm{dp}=\frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{c}} \mathrm{dx}\\
		 \mathrm{Q}_{\mathrm{N}}(\mathrm{V}, \mathrm{T}) &=\frac{1}{3 \mathrm{~N} ! \mathrm{h}^{3 \mathrm{~N}}} \mathrm{~L}^{3 \mathrm{~N}}\left[2 \int_{0}^{\infty} \mathrm{e}^{-\mathrm{x}} \frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{c}} \mathrm{dx}\right]^{3 \mathrm{~N}} \\
		  &=\frac{1}{3 \mathrm{~N} ! \mathrm{h}^{3 \mathrm{~N}}} \mathrm{~L}^{3 \mathrm{~N}}\left(\frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{c}}\right)^{3 \mathrm{~N}}\left[2 \int_{0}^{\infty} \mathrm{e}^{-\mathrm{x}} \mathrm{dx}\right]^{3 \mathrm{~N}} \\
		  &=\frac{1}{3 \mathrm{~N} ! \mathrm{h}^{3 \mathrm{~N}}} \mathrm{~L}^{3 \mathrm{~N}}\left(\frac{\mathrm{k}_{\mathrm{B}} \mathrm{T}}{\mathrm{c}}\right)^{3 \mathrm{~N}}\left[\left[-2 \mathrm{e}^{-\mathrm{x}}\right]_{0}^{\infty}\right]^{3 \mathrm{~N}}\\
		  &=\frac{1}{3 N ! h^{3 N}} \mathrm{~L}^{3 N}\left(\frac{\mathrm{k}_{B} T}{c}\right)^{3 N}2^{3 N}\\
		  \mathrm{Q}_{\mathrm{N}}(\mathrm{V}, \mathrm{T})&=\frac{1}{3N!}\left[ 2L \left( \frac{k_B T}{hc}\right) \right] ^{3N}
		\end{align*}
	\end{answer}
\end{enumerate}
\section{Grand canonical emsemble}
\begin{enumerate}
	\item Consider a system may be either unoccupied or occoupied by one particle with energy $0$ and $E$. Find out \\
	1) partition function of the system \\
	2) Thermal average energy of the system. Thermal average occupancy of the system
	\begin{answer}
		\begin{align*}
			\begin{array}{llll}\mathbf{0} & \mathbf{E} & \mathbf{N_{r}} &E_s\\ 0 & 0 & 0 &0\\ 1 & 0 & 1 &0\\ 0 & 1 & 1&E\end{array}\\
			\mathrm{Z}(\mu, \mathrm{V}, \mathrm{T})&=\sum_{\mathrm{rs}}e^{-\alpha N_1-\beta E_s}\\
			&=e^{-\alpha\times0-\beta\times0}+e^{-\alpha\times1-\beta\times0}+e^{-\alpha\times1-\beta\times1}\\
			1+\mathrm{e}^{-\alpha}+\mathrm{e}^{-\alpha-\beta \epsilon}=&=1+e^{+\frac{\mu}{k_{B} T}}+e^{\left(\frac{\mu}{k_{B} T}-\frac{E}{k_{B} T}\right)}\\
			U&=\left\langle E_{S}\right\rangle=\frac{\sum_{r s} \in_{S} e^{-\alpha N_{r}-\beta E_{S}}}{\sum_{r s} e^{-\alpha N_{r}-\beta E_{S}}}=\sum_{r s} E_{S} P_{r s}\\
			\bar{N}&=\left\langle N_{r}\right\rangle=\frac{\sum_{r s} N_{r} e^{-\alpha N_{r}-\beta E S}}{\sum_{r s} e^{-\alpha N_{r}-\beta E_{S}}}=\sum_{r s} N_{r} P_{r S}\\
			\overline{\mathrm{N}}&=-\frac{\partial}{\partial N} \ln \mathrm{Z}(\mu, \mathrm{V}, \mathrm{T})_{\beta, \mathrm{V}}=\mathrm{e}^{-\alpha}-\mathrm{e}^{-\alpha-\beta \epsilon} / 1+\mathrm{e}^{-\alpha}+\mathrm{e}^{-\alpha-\beta \epsilon}\\
			\mathrm{E}&=-\frac{\partial}{\partial \beta} \ln \mathrm{Z}(\mu, \mathrm{V}, \mathrm{T})_{\alpha, \mathrm{N}}=\frac{\epsilon \mathrm{e}^{(\mu-\epsilon) \beta}}{1+\mathrm{e}^{\mu \beta}+\mathrm{e}^{(\mu-\epsilon) \beta}}
		\end{align*}
	\end{answer}
 \item Repeat the above problem if each state may be occupied by one particle of the same time
 \begin{answer}
 	\begin{align*}
 	\begin{array}{llll} & \mathbf{E} & \mathbf{N}_{\mathrm{r}} & \mathbf{E}_{\mathrm{s}} \\ 0 & 0 & 0 & 0 \\ 1 & 0 & 1 & 0 \\ 0 & 1 & 1 & \mathrm{E} \\ 0 & 1 & 2 & \mathrm{E}\end{array}\\
 	\mathrm{Z}_{\mathrm{N}}(\mu, \mathrm{V}, \mathrm{T})&=1+\mathrm{e}^{-\alpha}+\mathrm{e}^{-\alpha-\beta \epsilon}+\mathrm{e}^{-2 \alpha-\beta \epsilon}
 	\end{align*}
 \end{answer}
	\item In a system in thermal equilibrium at absolute temperature T, two states with energy difference $4.83 \times 10^{-21} \mathrm{~J}$ occour with relative probability $\mathrm{e}^{2}$. Deduce the temperature $\left(\mathrm{k}_{\mathrm{B}}=1.38 \times 10^{-23} \mathrm{~J} / \mathrm{K}\right)$
	\begin{answer}
		\begin{align*}
		\Delta \mathrm{E}&=4.83 \times 10^{-23}\\
		\frac{\mathrm{P}_{1}}{\mathrm{P}_{2}}&=\mathrm{e}^{2}\\
		\mathrm{P}_{\mathrm{i}}&=\mathrm{Ce}^{-\beta \mathrm{E}_{\mathrm{i}}}\\
		\mathrm{P}_{1}&=\mathrm{Ce}^{-\beta \mathrm{E}_{1}}\qquad \mathrm{P}_{2}=\mathrm{Ce}^{-\beta \mathrm{E}_{2}}\\
		\frac{\mathrm{P}_{1}}{\mathrm{P}_{2}}&=\frac{\mathrm{Ce}^{-\beta \mathrm{E}_{1}}}{\mathrm{Ce}^{-\beta \mathrm{E}_{2}}}=\mathrm{e}^{-\beta\left(\mathrm{E}_{1}-\mathrm{E}_{2}\right)}=e^{-\beta (E_2-E_1)}\\
		e^{2}&=e^{\left(E_{2}-E_{1}\right)}\\
		2&=\beta\left(\mathrm{E}_{2}-\mathrm{E}_{1}\right)\\
		2&=\frac{1}{\mathrm{k}_{\mathrm{B}} \mathrm{T}}\left(\mathrm{E}_{2}-\mathrm{E}_{1}\right)\\
		2 \mathrm{k}_{\mathrm{B}} \mathrm{T}&=\mathrm{E}_{2}-\mathrm{E}_{1}\Rightarrow T=\frac{E_2-E_1}{2k_B}\\
		&=\frac{4.83 \times 10^{-23J}}{2 \times 1.38 \times 10^{-23}J/K}=1.75 K
		\end{align*}
	\end{answer}
	\item A diatomic gas in thermal equilibrium have temperature $500K$. Its rotational constant $2.0 cm^{-1}$. Calculate the relative populations in the rotational states.
	\begin{answer}
		\begin{align*}
		\text{Population}\quad E&=\frac{h^2}{8\pi^2 I}J(J+I)\qquad B=\frac{h}{8\pi^2IC}\\
		\color{red}{???}\\
		\mathrm{N}_{\mathrm{J}}&=\frac{\mathrm{N}(2 \mathrm{~J}+1) \mathrm{e}^{-\beta \mathrm{E}_{\mathrm{J}}}}{\sum_{\mathrm{J}} \mathrm{e}^{-\beta \mathrm{E}_{\mathrm{J}}}}\\
	\text{	But }
		\mathrm{BhC}&=3.96 \times 10^{-16} \mathrm{erg}\\
		\frac{\mathrm{N}_{10}}{\mathrm{~N}_{0}}&=91.28 \text{Ans.}\\
		\frac{\mathrm{N}_{20}}{\mathrm{~N}_{0}}&=3.371\text{ Ans}
		\end{align*}
	\end{answer}

\end{enumerate}


% !Mode:: "TeX:UTF-8"
% !TEX program  = xelatex
\title{Assignment 7}

\section{Question 1}
\begin{statebox}{Timescale Invariance}{question-1}
    \begin{align*}
        d_1 &= \frac{\log(S/E)+(r+\sigma^2/2)(T-t)}{\sigma\sqrt{T-t}} \\
        d_1 &= \frac{\log(S/E)+(r-\sigma^2/2)(T-t)}{\sigma\sqrt{T-t}}
    \end{align*}
    Prove the following identity:
    \[
    	d_2 = d_1 - \sigma\sqrt{T-t}
    \]
\end{statebox}

\Python{}{code/7-a.py}

\section{Question 2}
\begin{statebox}{Put-Call Parity}{question-2}
    With $t=0$, $S_0=5$, $E=4$, $T=1$, $\sigma=0.3$ and $r=0.05$, find the option values and verify the put-call parity.
\end{statebox}

\Python{}{code/7-b.py}

\section{Question 3}
\begin{statebox}{Study Note}{question-3}
    \begin{enumerate}
    	\item Write a study note of Black-Scholes' 73 paper.
    	\item Find a sequence of works following this paper and sort out hot topics nowadays.
    \end{enumerate}
\end{statebox}

To be continued.


\subsection{Norms}\label{subsec:norms}

\begin{remark}\label{rem:normed_fields}
  Norms generalize distances of points in a plane, while absolute values generalize the absolute value over either \( \BbbR \) or \( \BbbC \). The axioms themselves differ minimally. Absolute values in a field are multiplicative norms over the field, however we cannot define absolute values in terms of norms since absolute values are needed for defining norms. Still, we will refer to fields with absolute values as \term{normed fields}.
\end{remark}

\begin{definition}\label{def:absolute_value}\mcite{nLab:absolute_value}
  Let \( R \) be a \hyperref[def:semiring]{semiring}. We say that the function \( \abs{\cdot}: V \to \BbbR_{>0} \) is an \term{absolute value} or a \term{semiring norm} if
  \begin{thmenum}
    \thmitem[def:absolute_value/RN1]{RN1}(identity) \( x = 0_R \) if and only if \( \abs{x} = 0 \)
    \thmitem[def:absolute_value/RN2]{RN2}(multiplicativity) For any \( x, y \in V \),
    \begin{equation*}
      \abs{xy} = \abs{x} \cdot \abs{y}
    \end{equation*}

    \thmitem[def:absolute_value/RN3]{RN3}(subadditivity) For any \( x, y \in V \),
    \begin{equation*}
      \abs{x + y} \leq \abs{x} + \abs{y}
    \end{equation*}
  \end{thmenum}
\end{definition}

\begin{definition}\label{def:norm}
  Let \( M \) be an \( R \)-module with absolute value \( \abs{\cdot} \). We say that the function \( \norm{\cdot}: M \to \BbbR_{\geq 0} \) is a \term{norm} if
  \begin{thmenum}
    \thmitem[def:norm/N1]{N1}(identity) \( x = 0_M \) if and only if \( \norm x = 0_{\BbbR} \)

    \thmitem[def:norm/N2]{N2}(absolute homogeneity)
    \begin{equation*}
      \norm{t x} = \abs{t} \norm{x} \text{ for all } t \in R \text{ and } x \in M
    \end{equation*}

    \thmitem[def:norm/N3]{N3}(subadditivity)
    \begin{equation*}
      \norm{x + y} \leq \norm{x} + \norm{y} \text{ for all } x, y \in M
    \end{equation*}
  \end{thmenum}

  If we remove \fullref{def:norm/N1}, then \( \norm{\cdot} \) is called a \term{seminorm}.

  If instead \( V \) is an \hyperref[def:algebra_over_ring]{associative} and \( \norm{\cdot} \) satisfies the additional axiom
  \begin{thmenum}
    \thmitem{def:norm/multiplicativity}(multiplicativity)
    \begin{equation*}
      \norm{xy} = \norm{x} \cdot \norm{y} \text{ for all } x, y \in M,
    \end{equation*}
  \end{thmenum}
  we say that it is a \term{multiplicative norm}.
\end{definition}

\begin{definition}\label{def:norm_induced_metric}
  A norm \( \norm \cdot \) on a real or complex vector space \( V \) induces the \hyperref[def:vector_space]{metric}
  \begin{balign*}
     & \rho: V \times V \to \BbbR_{\geq 0}  \\
     & \rho(x, y) \coloneqq \norm{x - y}.
  \end{balign*}
\end{definition}
\begin{proof}
  The function is positive definite since \( \norm \cdot \) is positive definite; we will show that the function is a metric.

  \SubProofOf{def:metric_space/M1} Follows from \fullref{def:norm/N1}.

  \SubProofOf{def:metric_space/M2} By \fullref{def:norm/N2},
  \begin{equation*}
    \rho(x, y) = \norm{ x - y } = \norm{ (-1) (y - x) } = \abs{-1} \norm{y - x} = \rho(y, x).
  \end{equation*}

  \SubProofOf{def:metric_space/M3}
  \begin{equation*}
    \rho(x, y) + \rho(y, z) = \norm{x - y} + \norm{y - z} \geq \norm{x - z} = \rho(x, z).
  \end{equation*}
\end{proof}

\begin{definition}\label{def:duality_mapping}\mcite[exmpl. 2.26]{Phelps1993}
  We define the \term{duality mapping}
  \begin{balign*}
     & D: E \multto X^*,                                                                                \\
     & D(x) \coloneqq \{ x^* \in X^* \colon \norm x = \norm {x^*} \text{ and } \inprod{x^*} x = \norm {x^*} \norm x \}.
  \end{balign*}

  We will usually use this mapping for unit vectors, so we may as well consider its restriction to the unit spheres, where
  \begin{balign*}
     & D': S_X \multto S_{X^*},                                       \\
     & D'(x) \coloneqq \{ x^* \in S_{X^*} \colon \inprod{x^*} x = 1 \}.
  \end{balign*}
\end{definition}

\begin{definition}\label{def:smooth_norm}\mcite[def. 2.36]{Phelps1993}
  The norm \( \norm \cdot \) on \( X \) is called \term{smooth} if any of  if for each \( x \in S_X \) the duality mapping is single-valued.
\end{definition}

\begin{definition}\label{def:rotund_norm}\mcite[def. 2.36]{Phelps1993}
  The norm \( \norm \cdot \) on \( X \) is called \term{rotund} or \term{strictly convex} if any of the following equivalent conditions hold:
  \begin{thmenum}
    \thmitem{def:rotund_norm/no_sphere_segments} There are no line segments in the unit sphere \( S_X \).
    \thmitem{def:rotund_norm/least_norm} Every convex subset of \( X \) has at most one point of least norm.
    \thmitem{def:rotund_norm/linearly_dependent}
    \begin{balign}\label{def:rotund_norm/linearly_dependent/equation}
      \norm{x + y} = \norm x + \norm y \implies x \text{ and } y \text{ are linearly dependent}.
    \end{balign}
  \end{thmenum}
\end{definition}
\begin{proof}
  \ImplicationSubProof{def:rotund_norm/no_sphere_segments}{def:rotund_norm/least_norm} Let the norm in \( E \) be rotund and let \( C \subseteq E \) be a (potentially empty) convex set. We will prove that \( C \) contains at most one point of least norm.

  If \( C \) is empty or otherwise contains no element of least norm, trivially contains at most one point of least norm.

  Now let \( C \) contain at least one element \( x \in C \) of least norm. Assume that \( y \in C \) is another element of least norm. Necessarily \( \norm x = \norm y \).

  Fix \( t \in (0, 1) \) and define \( z \coloneqq tx + (1-t)y \). Since \( C \) is convex, it contains \( z \). Since \( x \) and \( y \) are elements of least norm, we have \( \norm z \geq \norm x \). By the triangle inequality,
  \begin{balign*}
    \norm{z}
    =
    \norm{tx + (1-t)y}
    \leq
    t \norm x + (1-t) \norm y
    =
    \norm{x},
  \end{balign*}
  thus \( \norm z = \norm x \).

  This implies that the entire segment \( [x, y] \) are elements of least norm in \( C \). Hence, the segment \( [x, y] \) is contained in the sphere \( \norm x S_E \), which contradicts the rotundity of the norm \( \norm{\cdot} \).

  Hence, \( C \) contains at most one element of least norm.

  \ImplicationSubProof{def:rotund_norm/least_norm}{def:rotund_norm/no_sphere_segments} Let every convex set \( C \subseteq E \) have at most one element of least norm.

  Assume that the norm \( \norm{\cdot} \) is not rotund. Then the unit sphere \( S_E \) contains a line segment \( [x, y], x \neq y \). The set \( [x, y] \) is compact and, by the Weierstrass extreme value theorem, the norm attains its minimum on the segment in a point \( z \in [x, y] \). Since the segment is also convex and we assumed that convex sets have at most one element of least norm, it follows that this element \( z \) is unique.

  Then for any point \( s \in [x, y], s \neq z \), we have \( \norm s > \norm z = 1 \), thus \( s \) cannot be an element of the unit sphere. The obtained contradiction shows that the norm \( \norm{\cdot} \) is rotund.

  \ImplicationSubProof{def:rotund_norm/no_sphere_segments}{def:rotund_norm/linearly_dependent} Let \( E \) be rotund let \( x, y \in E \) be distinct vectors such that
  \begin{balign}\label{def:rotund_norm/linearly_dependent/assumption}
    \norm{x + y} = \norm x + \norm y.
  \end{balign}

  If either of them is the zero vector, then they are trivially linearly dependent.

  Assume that both \( x \) and \( y \) are nonzero and define
  \begin{balign*}
    \xi \coloneqq \frac x {\norm x}
     &  &
    \eta \coloneqq \frac y {\norm y}
     &  &
    t \coloneqq \frac {\norm x} {\norm{x + y}}
  \end{balign*}

  \Fullref{def:rotund_norm/linearly_dependent/assumption} implies that
  \begin{equation*}
    1 - t = 1 - \frac {\norm x} {\norm{x + y}} = \frac {\norm{x + y} - \norm x} {\norm{x + y}} = \frac {\norm y} {\norm{x+y}}.
  \end{equation*}

  Since both \( \xi \) and \( \eta \) are in \( S_E \), by rotundity, their convex combination
  \begin{equation*}
    \nu \coloneqq t \xi + (1-t)\eta
  \end{equation*}
  should not be contained in \( S_E \) unless \( \xi = \eta \).

  Calculating the norm, we obtain
  \begin{balign*}
    \norm{\nu}
     & =
    \norm{t \xi + (1-t)\eta}
    =    \\ &=
    \norm{\frac {\norm x \xi} {\norm{x + y}} + \frac {\norm y \eta} {\norm{x + y}}}
    =    \\ &=
    \norm{\frac {x + y} {\norm{x + y}}}
    = 1,
  \end{balign*}
  hence \( \nu \in S_E \). Thus, \( \xi = \eta \) and \( x = \frac {\norm x} {\norm y} y \), so \( x \) and \( y \) are linearly dependent.

  \ImplicationSubProof{def:rotund_norm/linearly_dependent}{def:rotund_norm/no_sphere_segments} Let \fullref{def:rotund_norm/linearly_dependent/equation} hold and fix \( x, y \in S_E, t \in (0, 1) \). Define \( z \coloneqq tx + (1-t)y \).
  First, assume that the vectors \( tx \) and \( (1-t)y \) satisfy the left part of \fullref{def:rotund_norm/linearly_dependent/equation}, i.e.
  \begin{equation*}
    \norm z = \norm{tx + (1-t)y} = t \norm x + (1-t) \norm y = 1.
  \end{equation*}

  This does not refute rotundity since \( x \) and \( y \) are not necessarily distinct. It follows from \fullref{def:rotund_norm/linearly_dependent/equation} that \( tx \) and \( (1-t)y \) are linearly dependent, hence \( x \) and \( y \) are also linearly dependent. Since \( x \) and \( y \) both have unit norm, either \( y = x \) or \( y = -x \).

  If we assume that \( y = -x \), then
  \begin{balign*}
    \norm z
    =
    \norm{tx + (1-t)y}
    =
    (2t - 1) \norm x
    =
    2t - 1,
  \end{balign*}
  which is only possible if \( t = 1 \) since \( \norm z = 1 \). But \( t \) is strictly less than 1.

  Hence, \( y \neq -x \) and the only remaining possibility is that \( y = x \).

  Now assume that the vectors \( tx \) and \( (1-t)y \) do not satisfy the left part of \fullref{def:rotund_norm/linearly_dependent/equation}. This implies \( \norm z < 1 \). Thus, \( x \) and \( y \) are necessarily distinct, but \( z \) is not contained in the unit sphere and the segment \( [x, y] \) is not contained in \( S_E \).

  We have shown that \( x, y \in S_E \) implies that either \( y = x \) or that the segment \( [x, y] \) is not contained in \( S_E \), thus the norm in \( E \) is rotund.
\end{proof}

\begin{theorem}\label{thm:smooth_rotund_norm_duality}\mcite[exer. 2.37(a)]{Phelps1993}
  If the norm in a Banach space \( X \) is such that its dual norm in \( X^* \) is rotund (resp. smooth), then it is itself smooth (resp. rotund).
\end{theorem}
\begin{proof}
  \begin{enumerate}
    \item First, let the dual norm \( \norm{\cdot}^* \) be rotund and assume that \( \norm{\cdot} \) is not smooth.

          Fix \( x \in S_X \). Since \( D(x) \) is nonempty (by \fullref{thm:hahn_banach_implies_duality_mapping_nonempty}) and since \( \norm{\cdot} \) is not smooth, then there exist two different functionals \( x^*, y^* \in D(x) \), such that
          \begin{balign*}
            \inprod {x^*} x
            =
            \inprod {y^*} x
            =
            1.
          \end{balign*}

          We will show that the segment \( [x^*, y^*] \) is contained in \( S_{X^*} \), i.e. that the dual norm is not rotund.

          Fix any \( t \in (0, 1) \) and define \( z^* \coloneqq t x^* + (1-t) y^* \). We only need to show that \( \norm{z^*} = 1 \).

          By the triangle inequality, we have
          \begin{balign*}
            \norm{z^*}
            =
            \norm{t x^* + (1-t) y^*}
            \leq
            t \norm{x^*} + (1-t) \norm{y^*}
            =
            t + (1-t)
            =
            1.
          \end{balign*}

          For the reverse inequality, note that
          \begin{balign*}
            \norm{z^*}
            \geq
            \inprod {z^*} x
            =
            t \inprod {x^*} x + (1-t) \inprod {y^*} x
            =
            t + (1-t)
            =
            1,
          \end{balign*}
          thus \( \norm{z^*} = 1 \). Hence, \( [x^*, y^*] \) is contained in \( S_{X^*} \) and the dual space is not smooth. The obtained contradiction proves that the norm in \( X \) is rotund.

    \item Now let the dual norm \( \norm{\cdot}^* \) be smooth and assume that \( \norm{\cdot} \) is not rotund. Then there exist points \( x, y \in S_X \) such that the while segment \( [x, y] \) is contained in \( S_X \).

          Fix \( t \in (0, 1) \) and define \( z \coloneqq tx + (1-t)y \in S_X \). Denote by \( J: X \to X^{**} \) the canonical embedding into the double-dual. By \fullref{thm:hahn_banach_implies_duality_mapping_nonempty}, there exists a functional \( z^* \in X^* \), such that
          \begin{balign*}
            \inprod {J(z)} {z^*}
            =
            \inprod{z^*} z
            =
            1.
          \end{balign*}

          Because the dual norm \( \norm{\cdot}^* \) is smooth, we cannot have \( \inprod{J(x)} {z^*} =  \inprod{z^*} x = 1 \) or \( \inprod{J(y)} {z^*} = \inprod{z^*} y = 1 \) and since \( \norm{z^*} = 1 \), necessarily
          \begin{equation*}
            \inprod{z^*} x < 1 \text{ and } \inprod{z^*} y < 1.
          \end{equation*}

          If follows that
          \begin{balign*}
            1
            =
            \inprod{z^*} z
            =
            t \inprod{z^*} x + (1-t) \inprod{z^*} y
            <
            t + (1-t)
            =
            1,
          \end{balign*}
          which is a contradiction. Hence, \( \norm{\cdot} \) is rotund.
  \end{enumerate}
\end{proof}

\begin{proposition}\label{thm:hilbert_space_smooth_rotund}\mcite[exer. 2.37(c)]{Phelps1993}
  Norms in Hilbert spaces are both smooth and rotund.
\end{proposition}
\begin{proof}
  Let \( X \) be a Hilbert space, i.e. the norm is generated by an inner product and, due to Riesz's theorem, we identify the space \( X \) with its continuous dual \( X^* \).

  To prove that \( X \) is rotund, choose \( x, y \in S_X, x \neq y \). We will show that the segment \( [x, y] \) is not contained in \( S_X \).

  If \( x \) and \( y \) are linearly dependent, necessarily \( y = -x \) and all non-trivial convex combinations of \( x \) and \( y \) are contained in the open unit ball, hence \( [x, y] \not\subseteq S_X \).

  Not let \( x \) and \( y \) be linearly independent. By the Cauchy-Bunyakovsky-Schwarz inequality, we have
  \begin{balign}\label{eq:hilbert_cauchy_inequality}
    \inprod x y \leq \abs{\inprod x y} < \norm x \norm y = 1.
  \end{balign}

  Fix \( t \in (0, 1) \) and let \( z \coloneqq tx + (1-t)y \). We will show that \( z \not\in S_X \). Indeed,
  \begin{balign*}
    \norm{z}^2
    =
    \inprod z z
     & =
    t^2 \norm x^2 + t(1-t) \inprod x y + (1-t) t \inprod y x + (1-t)^2 \norm y^2
    =    \\ &=
    t^2 + (1-t)^2 + 2 t(1-t) \inprod x y
    <    \\ &\reloset {(\ref{eq:hilbert_cauchy_inequality})} <
    t^2 + (1-t)^2 + 2 t(1-t)
    =    \\ &=
    t^2 + 1 - 2t + t^2 + 2t - t^2
    =
    1.
  \end{balign*}

  Thus, \( \norm{z}^2 < 1 \) and \( \norm z < 1 \) and \( z \not\in S_X \).

  In both cases, no interior point of the segment \( [x, y] \) is contained in \( S_X \), hence the norm in \( X \) is rotund.

  Since we identify \( X \) with its dual, the norm in \( X^* \) is also rotund and by \fullref{thm:smooth_rotund_norm_duality}, the norm in \( X \) is also smooth.
\end{proof}

\begin{example}\label{thm:c0_l1_not_smooth_rotund}\mcite[exer. 2.37(c)]{Phelps1993}
  The norms in \( c_0 \) and \( l^1 \) are neither smooth nor rotund.
\end{example}
\begin{proof}
  Consider the space \( c_0 \) of all real sequences that converge to zero equipped with the uniform norm
  \begin{equation*}
    \norm{x}_{c_0} \coloneqq \sup_i \abs{x_i}.
  \end{equation*}

  Note that the dual space of \( c_0 \) is (isometrically isomorphic to) the space \( l^1 \) of absolutely summable sequences with norm
  \begin{equation*}
    \norm{x}_{l^1} \coloneqq \sum_i \abs{x_i}.
  \end{equation*}

  Let \( \{ e_n \}_{n=1}^\infty \) be the canonical basis of \( c_0 \), i.e. the coordinates \( e^{(i)}_n \) of \( e_n \) are given by the Dirac delta function, \( e^{(i)}_n \coloneqq \delta_{i,n} \).

  For every natural \( n \geq 1 \), define \( x_n \) to be the same as \( e_n \) except that the first coordinate of \( x_n \) is always \( 1 \).

  The corresponding norms of \( e_n \) are all equal to 1 and the norms of \( x_n \) are
  \begin{balign*}
    \norm{x_n}_{c_0} = 1
     &  &
    \norm{x_n}_{l^1} = 2.
  \end{balign*}

  For every \( n \) we have
  \begin{equation*}
    \inprod {e_1} {x_n} = \inprod {e_n} {x_n} = 1,
  \end{equation*}
  hence \( J_{c_0}(x_n) \) has at least two elements \( e_1 \) and \( e_n \) and the norm in \( c_0 \) is not smooth.

  Given that \( \{ x_1, x_2, \ldots \} \subseteq S_{c_0} \), consider the convex combinations of \( x_2 \) and \( x_3 \):
  \begin{balign*}
    tx_2 + (1-t)x_3
    =
    (1, t, (1-t), 0, 0, \ldots).
  \end{balign*}

  Evidently \( tx_2 + (1-t)x_3 \in S_{c_0} \) for every \( t \in (0, 1) \), hence the norm in \( c_0 \) is not rotund.

  The contrapositions to the statements in \fullref{thm:smooth_rotund_norm_duality} say that if \( X \) is not rotund (resp. smooth), then the dual space \( X^* \) is not smooth (resp. rotund). Thus, \( l^1 \) is neither smooth or rotund as the dual of \( c_0 \).
\end{proof}


\subsection{Descriptions of Error}\label{sec:set-error}
If we assume $\observed$ is absolutely continuous with respect to $\dmeas$, we can describe $\observed$ with a density $\observedP$. Then, for any partition $\set{D_\idisc}_{\idisc=1}^{\ndiscs}$ of $\dspace$,
\[
\observed (D_\idisc) = \int_{D_\idisc} \observed \, \dmeas, \quad \text{ for } \idisc = 1, \hdots, \ndiscs.
\]

We often use Monte Carlo approximations to compute the approximations $p_{\dspace, \idisc}=\observed(D_\idisc)$ in the first for-loop in Algorithm~\ref{alg:inv_density}.
These samples are generated on $\dspace$ and do not require numerical solutions to the model.
We therefore assume that for any discretization of $\dspace$, these approximations can be made sufficiently accurate and neglect the error in this computation.

We denote the exact solution to the SIP associated with this partitioning of $\dspace$ by $\PP_{\pspace, \ndiscs}$.
In situations where $\qoi(\param^{(\iparam)})$ is estimated (e.g. by application of a functional on a finite-element solution to a PDE), the approximate solutions to the SIP given in the final for-loop of Algorithm~\ref{alg:inv_density} are denoted by $\PP_{\pspace, \ndiscs, \nsamps, h}$.
Here, the $h$ is in reference to a mesh or other numerical parameter that determines the accuracy of the numerical solution $u_h(\param^{(\iparam)})\approx u(\param^{(\iparam)})$, and subsequently the accuracy in the computations of $\qoi_\iparam = \qoi(\param^{(\iparam)})$ in Algorithm~\ref{alg:inv_density}.
Then, by repeated application of the triangle inequality,
\begin{equation}
\label{eq:set-triangleineq}
d(\PP_{\pspace, \ndiscs, \nsamps, h}, \paramP) \leq
\underset{ \text{(E1)} }{\underbrace{d(\PP_{\pspace, \ndiscs, \nsamps, h},\PP_{\pspace, \ndiscs, \nsamps})}} +
\underset{ \text{(E2)} }{\underbrace{d(\PP_{\pspace, \ndiscs, \nsamps}, \PP_{\pspace, \ndiscs}) }}+
\underset{ \text{(E3)} }{\underbrace{d(\PP_{\pspace, \ndiscs}, \paramP) }}.
\end{equation}

The term (E1) describes the effect of the error in the numerically evaluated $\qoi_\iparam$ on the solution to the SIP.
The term (E2) describes the effect of finite sampling error in $\pspace$ on the solution to the SIP and (E3) describes the effect of discretization error of $\observed$ on the solution to the SIP.

We assume that $h$ is tunable so that for any $A\in \pborel$,
\[
\lim\limits_{h \downarrow 0} \PP_{\pspace, \ndiscs, \nsamps, \imesh} (A) = \PP_{\pspace, \ndiscs, \nsamps} (A).
\]
It is possible to prove the convergence of $\PP_{\pspace, \ndiscs, \nsamps, \imesh} (A) \to \paramP (A)$ for some $A\in \pborel$ and on estimating the error in $\PP_{\pspace, \ndiscs, \nsamps, h}(A)$.
For example, in \cite{BGE+15}, adjoint-based a posteriori estimates in the computed QoI are combined with a statistical analysis to both estimate and bound the error in $\PP_{\pspace, \ndiscs, \nsamps, \imesh} (A)$.
In \cite{JNME19}, adjoints are used to compute both error and derivative estimates of $\qoi(\param^{(\iparam)})$ to improve the accuracy in $\PP_{\pspace, \ndiscs, \nsamps, \imesh} (A)$.
Since the error due to $\imesh$ can be estimated as described in previous studies, and $\ndiscs$ can be made arbitrarily large, we neglect (E1) and (E3) here.
Thus, we limit our focus to (E2), where certain geometric properties of the QoI map (namely, skewness), are known to significantly impact this term.

\FloatBarrier


\input{../../assignment-header}

%========================================================================
\title{Assignment 3:  Simulation using Runge--Kutta}
\date{Assigned:  Jan. 30  ---  Due:  Feb. 8}
\author{Tufts ME 149:  Optimal Control For Robotics}
%========================================================================
\begin{document}
\maketitle
%=================================================

\section*{Introduction}

In this assignment you will implement a general-purpose simulation function
which implements a few different Runge--Kutta integration methods.
You will then use this function to generate simulations for two different dynamical systems:
a driven-damped pendulum and a passive double pendulum.
The final part of the assignment is to create two plots, one for each system,
showing the solution as well as the absolute error for each method.

\section*{Write-up}

Instead of submitting a separate write-up,
please fill-in the header block in \texttt{hw\_03\_studentName.m}.
Like previous assignments, this includes
your name, information about the assignment, collaborators and references,
how long the assignment took to complete, and
concludes with an outline of your code.

\section*{Deliverables}

Please upload each of the following files to Trunk ---
you do not need to zip them to a single file before uploading.
If you choose to create any additional Matlab files as part of you assignment,
then please upload those as well.
You do not need to include any files (or functions)
that are part of the \texttt{codeLibrary} for this course.

\vspace{-0.6em} \begin{itemize}  \setlength\itemsep{0em}
 \item \texttt{hw\_03\_studentName.m} --- main function for the assignment
 \item \texttt{runSimulation.m} --- general simulation function using Runge--Kutta methods
 \item \texttt{hw\_03\_driven\_damped\_pendulum.pdf} --- plot for the driven damped pendulum results
 \item \texttt{hw\_03\_passive\_double\_pendulum.pdf} --- plot for the passive double pendulum results
\end{itemize}

See the following pages for additional details on how to create each file.

%=================================================
\pagebreak

\section*{Details: \texttt{runSimulation.m}}

\lstinputlisting{runSimulation.m}

\subsection*{Requirements}

This function provides an interface that makes it easy to run a simulation
with a variety of different integration methods.
You need to implement Euler's method,
at least one of the second-order methods,
and ``The'' forth-order Runge--Kutta method.
While working on your implementation please follow good programming practices.

\subsection*{Programming Challenge}

The easiest way to implement this function is to have a separate sub-function
for each of the three method orders (first, second and forth).
You will receive full credit for this style of implementation.

\par
An alternate implementation is to write a truely general-purpose sub-function
that will work for \textit{any} explicit Runge--Kutta method. Such a function
would work by accepting the method parameters (in the form of a Butcher tableau)
as arguments. The top level function would then simply set the correct method
parameters before calling the sub-function.
If you choose to implement this method, then try experimenting with more
complicated methods (email me for examples).

%=================================================
\pagebreak

\section*{Details: \texttt{hw\_03\_studentName.m}}

\lstinputlisting{hw_03_studentName.m}

\subsection*{System Dynamics}

For this assignment you will need to simulate two systems:
the driven-damped pendulum and a passive double pendulum.
You will find functions that implement the system dynamics for both of these systems
in the code library for the course:\\
\texttt{ME149/codeLibrary/modelSystems/systemName/systemNameDynamics.m}\\
You can add the entire code library to your matlab path by running:\\
\texttt{ME149/codeLibrary/addLibraryToPath.m}\\
\\
You are permitted (and encouraged) to use any file
within the code library for your assignment.
You do not need to upload these files to Trunk.

\subsection*{Double Pendulum Dynamics}

The double pendulum dynamics have a different set of arguments
than are required by the standard simulation functions.
Use a function handle to pass through the correct arguments.

\vspace{-0.6em} \begin{itemize}  \setlength\itemsep{0em}
  \item \textbf{time} --- doublePendulumDynamics does not use time, you can ommit it.
  \item \textbf{control} --- doublePendulumDynamics accepts control torques, as well as the state.
    To make the system passive you will need to set both control torques to zero.
  \item \textbf{parameterss} --- doublePendulumDynamics requires that you pass system parameters,
    which can all be set to one. Use \texttt{>> help doublePendulumDynamics} to find the parameter names.
\end{itemize}

\textbf{Note:} If you've taken a dynamics course,
then I suggest that you take a look at the
script for deriving the dynamics function using the Matlab symbolic toolbox.
The techniques used there can be applied to any continuous dynamical system,
with a few small modifications, provided that it lives in two-dimensions.

\subsection*{Solution with \texttt{ode45()}}

You will need to compute the error associated with each
simulation method that you implement in \texttt{runSimulation.m}.
We will use Matlab's \texttt{ode45} function to approximation the true solution.
To do this, use \texttt{ode45} to run a simulation for each system, and
specify \texttt{'RelTol' = 1e-10} and \texttt{'AbsTol' = 1e-10}
using the \texttt{options} argument and \texttt{odeset}.

\subsection*{Simulation Parameters}

The driven-damped pendulum simulation should
start at $t = 0$ and finish at $t = 20$ using $150$ grid points.
Its initial angle should be $1.2$ radians and the
initial angular rate should be $0.8$ radians per second.

The passive double pendulum simulation should have a
duration of $10$ seconds and use $100$ grid points.
The initial angles for first link should be $0.6$
and the second link should be $0.9$.
Both links are initially stationary.
Link lengths, masses, and gravity should all be set to one.

\subsection*{Plots}

Create one figure for the driven-damped pendulum results and a
second figure for the passive double pendulum results.
Both figures should have the same formatting and outline.

\par
The top row of subplots should show the state while the bottom row shows the error.
Each subplot should have four curves: one for the solution (via \texttt{ode45}),
and the remaining three for each method that you implemented in
\texttt{runSimulation.m}: first-, second-, and forth-order
Be sure to include axis labels and a legend for each subplot.

\par The error plots should show the difference between your simulations and \texttt{ode45}.
You should plot the absolute value of the error, and the y-axis should be on a log scale.
Since the tolerance for \texttt{ode45} is set to $1e-10$, it makes not sense to have
any error values smaller than that: please clamp the error curves to have a minimum value of $1e-10$.

\par Following this outline,
the driven-damped pendulum figure will have a two-by-two
array of subplots, while the passive double pendulum figure will have
two rows and four columns of subplots.
Be sure to make your figures full-screen before saving them to pdf.

%=================================================
\end{document}


\documentclass{beamer}
\usepackage[utf8]{inputenc}

\input{../../shared.tex}

\title{Newton's and Quasi-Newton Methods}
\date{\today}

\begin{document}
\maketitle
\frame{\tableofcontents[]}

\section{Introduction}

\begin{frame}
  \frametitle{$1$-dimensional case: Newton-Raphson method}
  \textcolor{blue}{Objective:} Find zero of differentiable $f: \R \to \R$.
  \vspace{0.3cm}
  \begin{minipage}{0.48\textwidth}
    \textcolor{blue}{Strategy:} Solve
    \begin{equation}
      f(x_k) + f'(x_k) (x - x_k) = 0.
    \end{equation}
    \textcolor{blue}{Method:} Gives
    \begin{equation}
      x_{k+1} = x_k - \frac{f(x_k)}{f'(x_k)}
    \end{equation}
  \end{minipage}
  \hfill
  \begin{minipage}{0.48\textwidth}
    \begin{figure}[ht]
      \centering
      \includegraphics[width=\textwidth,height=\textheight,keepaspectratio]{newton-raphson}
      % \caption{\label{fig:label} }
    \end{figure}
  \end{minipage}
\end{frame}

\begin{frame}
  \frametitle{The Babylonian method}
  \begin{itemize}
    \item \textcolor{blue}{compute square root} of $R \in \R_+$
    \item find zero of $f(x) = x^2 - R$
    \item use Newton-Raphson:
          \begin{equation}
            x_{k+1} = x_k - \frac{f(x_k)}{f'(x_k)} = x_k - \frac{x_k^2 - R}{2 x_k} = \frac12 \left( x_k + \frac{R}{x_k} \right)
          \end{equation}
    \item Starting from $x_0 > 0$ we have
          \begin{equation}
            x_{k+1} = \frac12 \left( x_k + \frac{R}{x_k} \right) \ge \frac{x_k}{2}.
          \end{equation}
    \item Starting from $x_0 = R \ge 1$, it takes $\mathcal{O}(\log R)$ steps to get to $x_k - \sqrt{R} < \frac12$.
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{The Babylonian method - Takeoff}
  \onslide<1>{%
    Note that
    \begin{equation}
      x_{k+1} - \sqrt{R} = \frac12 \left( x_k + \frac{R}{x_k} \right) - \sqrt{R} = \frac{x_k}{2} + \frac{R}{2 x_k} - \sqrt{R} = \frac{1}{2 x_k} {\left( x_k - \sqrt{R} \right)}^2
    \end{equation}
    For simplicity $R \ge 1/4$, then $x_k \ge \sqrt{R} \ge 1/2$. Hence
    \begin{equation}
      x_{k+1} - \sqrt{R} = \frac{1}{2 x_k} {\left( x_k - \sqrt{R} \right)}^2 \le {\left( x_k - \sqrt{R} \right)}^2
    \end{equation}
  }

  \onslide<2->{%
    If $x_0 - \sqrt{R} < \frac12$ (ensured after $\mathcal{O}(\log R)$ steps).
    \begin{equation}
      x_{k} - \sqrt{R} \le {\left( x_0 - \sqrt{R} \right)}^{2^k} \le {\left(\frac12\right)}^{2^k}
    \end{equation}
    To achieve $x_k - \sqrt{R} < \epsilon$ we only need $k = \log \log (\epsilon^{-1})$ steps!
  }
\end{frame}

\begin{frame}
  \frametitle{The Babylonian method - Example}
  $R=1000$, in double arithmetic
  \begin{itemize}
    \item $7$ steps to get to $x_7 - \sqrt{1000} < 1/2$
          \item $3$ steps to get to $\sqrt{1000}$ up to \textit{machine precision}
          \item First phase: $\approx$ \textcolor{blue}{one more correct digit} per iteration
          \item Second phase: $\approx$ \textcolor{blue}{double the number of correct digits} per iteration
  \end{itemize}

  \begin{center}
    In practice: $\log \log x \le 5$.
  \end{center}

\end{frame}

\section{Newton's method}%

\begin{frame}
  \frametitle{Newton's method for optimization}
  \begin{itemize}
    \item \textcolor{blue}{Goal:} Find global minimum $x^*$ of convex, differentiable function $f$.
    \item \textcolor{blue}{Strategy:} Search for zero of derivative.
    \item \textbf{$1$-dimensional case:} Apply Newton-Raphson method to $f'$:
          \begin{equation}
            x_{k+1} = x_k - \frac{f'(x_k)}{f''(x_k)} = x_k - {f''(x_k)}^{-1} f(x_k)
          \end{equation}
          (requires \textcolor{blue}{twice} differentiable and $f'' > 0$)

    \item \textbf{$d$-dimensional case:} Newtons methods for minimizing convex $f: \R^d \to \R$:
          \begin{equation}
            x_{k+1} = x_k - \nabla^2 f(x_k) \nabla f(x_k).
          \end{equation}
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Newton's method as adaptive gradient descent}
  General update scheme:
  \begin{equation}
    x_{k+1} = x_k - H(x_k) \nabla f(x_k)
  \end{equation}
  for some matrix $H(x) \in \R^{d \times d}$.
  \begin{itemize}
    \item \textcolor{blue}{Newton's method}: $H = {\nabla^2 f(x_k)}^{-1}$.
    \item \textcolor{blue}{Gradient descent}: $H = \alpha \Id$
  \end{itemize}
  \vspace{1cm}
 \begin{block}{}
  Newton's methods \textbf{adapts} to the local geometry of $f$ at $x_k$ \\
 \end{block}
  $\rightarrow$ \textit{no need for choosing a stepsize}.
\end{frame}

\begin{frame}
  \frametitle{Convergence in one step on quadratic functions}
  A \textcolor{blue}{quadratic} function
  \begin{equation}
    f(x) = \frac12 x^T M x + q^T x + c
  \end{equation}
  is called \textit{nondegenerate} if $M \in \R^{d \times d}$ is invertible.
  \begin{itemize}
    \item $x^* := M^{-1}q$ is the unique solution of $\nabla f(x) = 0$
    \item $x^*$ is the unique global minimum if $f$ is convex
  \end{itemize}
  \begin{lemma}%
    On nondegenerate quadratic functions with arbitrary starting point $x_0$, Newtons method yields $x_1=x^*$
  \end{lemma}
  \begin{proof}
    We have $\nabla f(x) = Mx -q$ and $\nabla^2 f(x) = M$. Therefore
    \begin{equation}
      x_1 = x_0 - \nabla^2 f(x_0) \nabla f(x_0) = x_0 - M^{-1}(M x_0 - q) = M^{-1}q = x^*. \qed\qedhere
    \end{equation}
  \end{proof}
\end{frame}

\begin{frame}
  \frametitle{Affine Invariance}
  Newton’s method is \textcolor{blue}{affine invariant}
  (invariant under any invertible affine transformation):
  Denote the Newton step for $h$ by
  \begin{equation}
    N_h(x) := x - {\nabla^2 h }^{-1} \nabla h(x).
  \end{equation}
  \begin{lemma}%
    Let $f: \R^d \to \R$ be twice differentiable, $A \in \R^{d \times d}$ an invertible matrix and $b \in \R^d$.
    \begin{equation}
      g(x) = Ax + b.
    \end{equation}
    Then
    \begin{equation}
      N_{f \circ g} = g^{-1} \circ N_f \circ g
    \end{equation}
  \end{lemma}
\end{frame}

\begin{frame}
  \frametitle{}
  \begin{figure}[ht]
    \centering
    \includegraphics[width=\textwidth,height=\textheight,keepaspectratio]{newton_aff_invariance}
    % \caption{\label{fig:label} }
  \end{figure}
  Gradient descent suffers if coordinates are at different scales; Newton's method doesn't.
\end{frame}

\begin{frame}
  \frametitle{Minimizing the second-order Taylor approximation}
  Alternative interpretation of Newton's method:
  Minimize (local) \textcolor{blue}{quadratic model} of $f$.
  \begin{lemma}%
    Let $f$ be conve, twice differentiable and $\nabla^2 f(x) \succ 0$. Then $x_{k+1}$ resulting from \textbf{Newton's step} satisfies
    \begin{equation}
      x_{k+1} = \argmin_{x \in \R^d} f(x_k) + \langle \nabla f(x_k), x-x_k \rangle + \frac12 \langle x-x_k, \nabla^2 f(x_k) (x-x_k)  \rangle
    \end{equation}
  \end{lemma}
\end{frame}

\begin{frame}
  \frametitle{Local Convergence}
  We will prove:\\
  Under suitable conditions on $f$ and \textcolor{blue}{close to the minimum} Newton's method approximates solution up to an error $\epsilon$ in \textcolor{blue}{$\log \log (1/\epsilon)$} iterations.
  \begin{itemize}
    \item much faster than anything so far..
    \item only locally
  \end{itemize}
  We call this a \textcolor{blue}{local convergence} result.\\

  \textcolor{blue}{Global convergence} statements are more difficult to obtain (some only recently).
\end{frame}

\section{Convergence analysis}%

\begin{frame}
  \frametitle{Theorem + Technical conditions }
  \begin{theorem}
    Let $f$ be convex with unique global minimum $x^*$, and $X$ a ball around $x^*$ s.t.
    \begin{enumerate}
      \item \textcolor{blue}{Bounded inverse Hessians:} There exists $\mu > 0$
            \begin{equation}
              \Vert {\nabla^2 f(x)}^{-1} \Vert \le \frac{1}{\mu}, \quad \forall x \in X
            \end{equation}
      \item \textcolor{blue}{Lipschitz continuous Hessians:} There exists $B>0$
            \begin{equation}
              \Vert \nabla^2 f(x) - \nabla^2 f(y) \Vert \le B \Vert x-y \Vert, \quad \forall x,y \in X
            \end{equation}
    \end{enumerate}
    Then, for $x_{k+1} = N_f(x_k)$ we have
    \begin{equation}
      \Vert x_{k+1} -x^* \Vert \le \frac{B}{2 \mu} \Vert x_k - x^* \Vert^2
    \end{equation}
  \end{theorem}
\end{frame}

\begin{frame}
  \frametitle{Super-exponential speed}
  \begin{corollary}%
    In the setting of previous theorem, if
    \begin{equation}
      \Vert x_k - x^* \Vert \le \frac{\mu}{B},
    \end{equation}
    then
    \begin{equation}
      \Vert x_k -x^*  \Vert \le \frac{\mu}{B} {\left( \frac{1}{2} \right)}^{2^k-1}
    \end{equation}
  \end{corollary}
  Close to the global minimum, we will reach distance to the minumum less than $\epsilon$ in at most $\log \log (1/\epsilon)$ steps.

  As for the last phase of Babylonian method.
\end{frame}

\begin{frame}
  \frametitle{Super-exponential speed - intuition}

  \begin{itemize}
    \item Almost constant Hessians close to optimality...
    \item so $f$ behaves almost like a quadratic
    \item on which Newton's converge in one step
  \end{itemize}

  \begin{lemma}%
    If
    \begin{equation}
      \Vert x_0 - x^* \Vert \le \frac{\mu}{B}
    \end{equation}
    the Hessians in Newton's method satisfy the \textcolor{blue}{relative error bound}
    \begin{equation}
      \frac{\Vert  \nabla^2 f(x_k) - \nabla^2 f(x^*) \Vert}{\Vert \nabla^2 f(x^*) \Vert} \le {\left( \frac12 \right)}^{2^k-1}.
    \end{equation}
  \end{lemma}
\end{frame}

\begin{frame}
  \frametitle{Proof of convergence theorem}
  We abbreviate $H = \nabla^2 f(x_k)$, $x=x_k$, $x^+ = x_{k+1}$
  \begin{equation}
    \begin{aligned}
      x^+ - x^* &= x - x^* - H^{-1} \nabla f(x) \\
      &= x - x^* + H^{-1}( \nabla f(x^*) - \nabla f(x)) \\
      &= x - x^* + H^{-1} \int_{0}^{1} H(x + t(x^* - x))(x^* - x) \diff t,
    \end{aligned}
  \end{equation}
  \onslide<2->{%
    where we used the fundamental theorem of calculus
    \begin{equation}
      \int_{a}^{b} h' (t) \diff t
    \end{equation}
    with
    \begin{align}
      h(t)  &= \nabla f(x + t(x^* -x )) \\
      h'(t) &= \nabla^2f (x + t(x^* -x ))(x^*-x).
    \end{align}
  }
\end{frame}

\begin{frame}
  \frametitle{Proof of convergence theorem II}
  So far
  \begin{equation}
      x^+ - x^* = x - x^* + H^{-1} \int_{0}^{1} H(x + t(x^* - x))(x^* - x) \diff t
  \end{equation}
  With
  \begin{equation}
    x- x^* = {H(x)}^{-1} \int_{0}^{1} - H(x)(x^*-x)
  \end{equation}
  we get
  \begin{equation}
    x^+ - x^* =  H^{-1} \int_{0}^{1} (H(x + t(x^* - x))- H(x))(x^* - x) \diff t.
  \end{equation}
  Using norms
  \begin{equation}
    \Vert x^+ - x^*  \Vert \le  \Vert H^{-1} \Vert \left\Vert  \int_{0}^{1} H(x + t(x^* - x))- H(x)(x^* - x) \diff t \right\Vert
  \end{equation}
\end{frame}

\begin{frame}
  \frametitle{Proof of convergence theorem III}

  \begin{align}
    \Vert x^+ - x^*  \Vert &=  \Vert H^{-1} \Vert \left\Vert  \int_{0}^{1} (H(x + t(x^* - x))- H(x))(x^* - x) \diff t \right\Vert \\
    &\le \Vert H^{-1} \Vert  \Vert x^* - x \Vert \int_{0}^{1} \left\Vert  (H(x + t(x^* - x))- H(x)) \right\Vert \diff t
  \end{align}
  Use \textbf{bounded inverse Hessians} and \textbf{Lipschitz continuity} of the Hessian to conclude
  \begin{align}
    \Vert x^+ - x^*  \Vert  &\le \frac{1}{\mu} \Vert x^* - x \Vert \int_{0}^{1} B \Vert t(x^*-x) \Vert \diff t  \\
    &= \frac{B}{\mu} \Vert x^*-x \Vert^2 \int_{0}^{1}t \diff t = \frac{B}{2\mu} \Vert x-x^* \Vert^2. \qed
  \end{align}

\end{frame}

\begin{frame}
  \frametitle{Strong convexity $\Rightarrow$ Bounded inverse Hessians}

  \begin{itemize}
    \item How to ensure bounded inverse Hessians?
  \end{itemize}

  \begin{lemma}%
    Let $f: \R^d \to \R$ be $C^2$ and \textbf{strongly convex} with parameter $\mu$, i.e.\
    \begin{equation}
      f(y) \ge f(x) \langle \nabla f(x), y-x \rangle + \frac{\mu}{2} \Vert y-x \Vert^2, \quad \forall x,y.
    \end{equation}
  Then, $\nabla^2 f(x)$ is invertible and $\Vert \nabla^2 f(x) \Vert^{-1} \le 1/\mu$ for all $x$.
  \end{lemma}

\end{frame}

\begin{frame}
  \frametitle{Downside of Newton's method}
  \textcolor{blue}{Computational bottleneck} in every step:
  \begin{itemize}
    \item compute Hessian
    \item invert Hessian or solve $\nabla^2 f(x_k) \Delta x = - \nabla f(x_k)$
  \end{itemize}
  \vspace{1cm}
  Matrix has size $d\times d$, taking $\mathcal{O}(d^3)$ to invert.\\
  In many applications the dimension $d$ is large (too large to even store Hessian).

  \vspace{1cm}
  When training a ML model $d$ is the \textit{number or parameters} of our ML model (number of features for linear model).
\end{frame}

\section{Quasi-Newton methods}%

\begin{frame}
  \frametitle{The secant method}
  Another iterative method for finding zeros in $1$-d.
  Recall Newton-Raphson:
  \begin{equation}
    x_{k+1} = x_k - \frac{f(x_k)}{f'(x_k)}
  \end{equation}
  Use \textcolor{blue}{finite difference approximation} of $f'(x_k)$:
  \begin{equation}
    f'(x_k) \approx \frac{f(x_k) - f(x_{k-1})}{x_k - x_{k-1}}.
  \end{equation}
  We obtain the \textcolor{blue}{\textbf{secant method}}:
  \begin{equation}
    x_{k+1} = x_k - f(x_k) \frac{x_k - x_{k-1}}{f(x_k) - f(x_{k-1})}.
  \end{equation}

\end{frame}

\begin{frame}
  \frametitle{The secant method II}
  \begin{figure}[ht]
    \centering
    \includegraphics[width=0.7\textwidth,keepaspectratio]{secant_method}
  \end{figure}
  Constructs the line through $(x_{k-1}, f(x_{k-1}))$ and $(x_k, f(x_k))$.
\end{frame}

\begin{frame}
  \frametitle{The secant method III}
  \begin{itemize}
    \item is a \emph{derivative-free} version of the Newton-Raphson method.
    \item \textcolor{blue}{For optimization}: Apply secant method to $f'$ to optimize $f$:
          \begin{equation}
            x_{k+1} = x_k - f'(x_k) \frac{x_k - x_{k-1}}{f'(x_k) - f'(x_{k-1})}.
          \end{equation}
    \item yields a \textcolor{blue}{second-derivative free} version of Newton's method.
  \end{itemize}

  \vspace{1cm}

  \onslide<2->{%
    \begin{center}
      \textit{What abouthigher dimensions? Can't divide vectors..}
    \end{center}
  }
\end{frame}

\begin{frame}
  \frametitle{The secant condition}
  In $1$-d:
  \begin{align}
    H_k &:= \frac{f'(x_k) - f'(x_{k-1})}{x_k - x_{k-1}} \approx f''(x_k) \\
    &\Leftrightarrow f'(x_k) - f'(x_{k-1}) = H_k (x_k - x_{k-1}),
  \end{align}
  the \textcolor{blue}{secant condition}.
  \begin{itemize}
    \item Newton's method: $x_{k+1} = x_k - {f''(x_k)}^{-1} f'(x_k)$
    \item Secant method: $x_{k+1} = x_k - {H_k}^{-1} f'(x_k)$
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Quasi-Newton methods}

  \begin{equation}
    \nabla f(x_k) - \nabla f(x_{k-1}) = H_k (x_k - x_{k-1}) \approx \nabla^2 f(x_k) (x_k - x_{k-1})
  \end{equation}
  We therefore hope that $H_k \approx \nabla^2 f(x_k)$.

  \begin{equation}
    \text{Secant:} \quad x_{k+1} = x_k = H_k^{-1} \nabla f(x_k)
  \end{equation}

  \begin{itemize}
    \item $d=1$: unique number $H_k$ satisfying the secant condition
    \item $d>0$: secant condition $\nabla f(x_k) - \nabla f(x_{k-1}) = H_k (x_k - x_{k-1})$ has \textbf{infinitely} many \textit{symmetric} solutions
  \end{itemize}

  \begin{block}{}
    \textit{Any scheme of choosing in each step of the secant method a symmetric $H_k$ that satisfies the secant condition defines a \textcolor{blue}{Quasi-Newton method}.}
  \end{block}

\end{frame}

\begin{frame}
  \frametitle{Quasi-Newton methods II}

  \begin{itemize}
    \item Newton’s method is a Quasi-Newton method if and only if f is a nondegenerate \textbf{quadratic} function.
    \item Hence, Quasi-Newton methods \textit{do not generalize} Newton’s method but form a
          family of \textit{related} algorithms.
    \item First Quasi-Newton method by William Davidon in 1956
    \item But the paper got rejected for lacking a convergence analysis,
    \item was finally officially published in 1991
    \item \textcolor{blue}{methods of choice} in a number of relevant machine learning applications
  \end{itemize}

\end{frame}

\begin{frame}
  \frametitle{Developing a Quasi-Newton method}
  \begin{itemize}
    \item We want to avoid matrix inversion
          $\Rightarrow$ directly deal with the inverse $H_k^{-1}$
    \item \textcolor{blue}{Given:} iterates $x_{k-1}, x_k$ and matrix $H_{k-1}^{-1}$
    \item \textcolor{blue}{Seeking:} next matrix $H_k^{-1}$ needed in next Quasi-Newton step
          \begin{equation}
            x_{k+1} = x_k = H_k^{-1} \nabla f(x_k)
          \end{equation}
    \item How to choose $H_k^{-1}$?
    \item Newton’s method: $\nabla^2 f(x_k)$ fluctuates only very little in the region of very fast convergence.
    \item Makes sense to have $H_k \approx H_{k-1}$ or $H_k^{-1} \approx H_{k-1}^{-1}$
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Greenstadt's family of Quasi-Newton methods}
  Greenstadt [Gre70]: Update
  \begin{equation}
    H_k^{-1} = H_{k-1}^{-1} + E_k,
  \end{equation}
  with $E_k$ an error matrix.
  \begin{itemize}
    \item Try to \textcolor{blue}{minimize error} subject to $H_k$ satisfying the \textbf{secant condition}!
  \end{itemize}

  Simple error measure: Frobenius norm
  \begin{equation}
    \Vert E \Vert_F^2 := \sum_{i=1}^{d} \sum_{j=1}^{d} E_{ij}^2
  \end{equation}
\end{frame}

\begin{frame}
  \frametitle{BFGS method}
  \begin{itemize}
    \item special version of Greenstadt
    \item is named after Broyden, Fletcher, Goldfarb and Shanno
          who all came up with it independently around 1970. Greenstadt’s name is
          mostly forgotten.
    \item Newton’s method needs to compute and invert
          Hessians $\rightarrow$ cost of $\mathcal{O}(d^3)$ per iteration
    \item Any method in Greenstadt’s family avoids computation of Hessian.
          \textbf{Only gradients are needed}.
    \item In the BFGS method, the cost per iteration drops to $\mathcal{O}(d^2)$.
    \item even this can be prohibitive $\rightarrow$ limited memory BFGS
    \item uses observation that we do not need $H_{k}^{-1}$, only $H_{k}^{-1} \nabla f(x_k)$
  \end{itemize}

\end{frame}
\end{document}


\section{The Generic MST Algorithm}	\label{section:mst-generic-alg}

%%%%%%%%%%%%%%%%%%%%
\begin{frame}{The Generic MST Algorithm}
  \begin{description}
	\setlength{\itemsep}{5pt}
	\item<1->[Overview:] Grow the MST one edge at a time
	\item<2->[State:] Manage a set of edges $X$
	\item<3->[\textcolor{red}{\bf Invariant:}] $X$ is a subset of some MST
	\item<4->[Init:] $X = \emptyset$
	\item<5->[Iteration:] Find \only<5>{an edge}\only<6->{a \textcolor{red}{\bf safe} edge} $e$ {\it s.t.}
	  \\ $X \cup \set{e}$ is also a subset of some MST
	\item<7->[Termination:] $(n-1)$ iterations
  \end{description}

  \vspace{0.20cm}
  \uncover<8->{
	\begin{center}
	  \textcolor{brown}{\large How to find a safe $e$ for $X$ in each iteration?}
	\end{center}
  }
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{The Cut Property}
	Given that $X$ is part of some MST $T$:
	\vspace{0.10cm}
    \begin{itemize}
	  \setlength{\itemsep}{3pt}
      \item<2-> A cut $(S, V \setminus S)$ \textcolor{red}{\bf respecting} $X$ ($X$ does not cross $(S, V \setminus S)$)
	  \item<6-> $e$: a lightest edge across $(S, V \setminus S)$
	\end{itemize}
	\vspace{0.10cm}
	Then, $X + \set{e}$ is also a part of some MST $T'$.

  \importikznocaption{0.50\textwidth}{0.50\textwidth}{tikz-in-beamer/cut-property-example-overlay.tex}
\end{frame}
%%%%%%%%%%%%%%%%%%%%
\begin{frame}{The Cut Property}
  \begin{enumerate}
	\item<1-> $X \subseteq T, \textcolor{blue}{e \in T} \Rightarrow X + \set{e} \subseteq T$
	\item<2-> $X \subseteq T, \textcolor{red}{e \notin T} \Rightarrow X + \set{e} \subseteq T'$.
  \end{enumerate}

  \vspace{-0.30cm}

  \importikznocaption{0.35\textwidth}{0.35\textwidth}{tikz-in-beamer/cut-property-proof-clrs-overlay.tex}

  \begin{itemize}
	\item<5-> $T + \set{e} \Longrightarrow$ cycle $C$ ($uv + P_{u \leadsto v}$)
	\item<6-> $\exists e' \in P_{u \leadsto v} \colon e'$ across the cut; $w(e') \geq w(e)$
    \item<7-> $T' = T + \set{e} - \set{e'}$ is an ST
    \item<8-> $w(T') \le w(T) \Rightarrow T' \textrm{ is an MST}$
	\item<9-> $e' \notin X {\scriptsize (\textcolor{red}{\bf respect})} \Rightarrow X + \set{e} \subseteq T'$
  \end{itemize}
\end{frame}
%%%%%%%%%%%%%%%%%%%%


\documentclass[12pt]{cdblatex}
\usepackage{exercises}
\usepackage{fancyhdr}
\usepackage{footer}

\begin{document}

% --------------------------------------------------------------------------------------------
\section*{Exercise 2.5 Combining rules -- a solution}

\begin{cadabra}
   {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u#}::Indices(position=independent).

   \nabla{#}::Derivative.
   \partial{#}::PartialDerivative.

   # rules for covariant derivatives of v

   deriv1 := \nabla_{a}{v^{b}} -> \partial_{a}{v^{b}}
                                + \Gamma^{b}_{d a} v^{d}.

   deriv2 := \nabla_{a}{\nabla_{b}{v^{c}}} -> \partial_{a}{\nabla_{b}{v^{c}}}
                                            + \Gamma^{c}_{d a} \nabla_{b}{v^{d}}
                                            - \Gamma^{d}_{b a} \nabla_{d}{v^{c}}.

   # second covariant derivative of v

   expr := v^{c}_{b a} -> \nabla_{a}{\nabla_{b}{v^{c}}}.  # cdb (ex-0205.101,expr)
   save := @(expr).

   # apply the rules, then simplify

   substitute     (expr,deriv2)       # cdb (ex-0205.102,expr)
   substitute     (expr,deriv1)       # cdb (ex-0205.103,expr)
   distribute     (expr)              # cdb (ex-0205.104,expr)
   product_rule   (expr)              # cdb (ex-0205.105,expr)
   canonicalise   (expr)              # cdb (ex-0205.107,expr)
   substitute     (expr,save)         # cdb (ex-0205.108,expr)
\end{cadabra}

The trick here is to introduce in line 17 a dummy left hand side, \verb|v^{c}{}_{b a}|,
that is invisible with respect to the substitution rules of lines 8 and 11. Thus lines 22
and 23 will only target the right hand side of \verb|expr|.

Notice how a copy of the initial expression is made in 18. This is used later in line 27 to
replace the dummy object \verb|v^{c}{}_{b a}| with \verb|\nabla_{a}{\nabla_{b}{v^{c}}}|
but this time acting on the left hand side of the rule. The result is a rule for second
covariant deriavtives.

\clearpage

\begin{align*}
   & \Cdb{ex-0205.101} \\
   & \Cdb{ex-0205.102} \\
   & \Cdb{ex-0205.103} \\
   & \Cdb{ex-0205.104} \\
   & \Cdb{ex-0205.105} \\
   & \Cdb{ex-0205.107} \\
   & \Cdb{ex-0205.108}
\end{align*}

\end{document}