\section{Monte Carlo Methods}
Monte Carlo methods learn state and action values by sampling and averaging returns (i.e. not from dynamics like DP). These methods learn from experience (real or simulated) and require no prior knowledge of the environments dynamics.\\

Monte Carlo methods thus require well defined returns, so we will consider them only for episodic tasks. Only on completion of an episode do values and policies change.\\

We still use the generalised policy iteration framework, but we adapt it so that we learn the value function from experience rather than compute it \emph{a priori}.

\subsection{Monte Carlo Prediction}
The idea is to average the returns following each state to get an estimate of the state value
\[
    v_\pi(s) = \Epi{} [G_{t+1} | S_t =s].
\]
Given enough observations, the sample average converges to the true state value under the policy $\pi$.\\

Given a policy $\pi$ and a set of episodes, here are two ways in which we might estimate state values
\begin{itemize}
    \item \emph{First Visit MC} average returns from first visit to state $s$ in order to estimate $v_\pi(s)$
    \item \emph{Every Visit MC} average returns following every visit to state $s$.
\end{itemize}

First visit MC generates iid estimates of $v_\pi(s)$ with finite variance, so the sequence of estimates converges to the expected value by the law of large numbers as visits to $s$ tend to $\infty$. Every visit MC does not generate independent estimates, but still converges.\\

An algorithm for first visit MS (what we will focus on) is below. Every visit is the same, just without the check for $S_k$ occurring earlier in the episode.\\

\includegraphics[width=\textwidth]{\ProjectDir/data/notes_images/first_visit_mc_algo.png} \\

Monte Carlo methods are often used even when the dynamics of the environment are knowable, e.g. in Blackjack. It is often much easier to create sample games than it is to calculate environment dynamics directly.\\

MC estimates for different states are independent (unlike bootstrapping in DP). This means that we can use MC to calculate the value function for a subset of the states, rather than the whole state space as with DP. Along with the ability to learn from experience and simulation, this is the another advantage that MC has over DP.

\subsection{Monte Carlo Estimation of Action Values}
If we don't have a model for the environment, then it is more useful to estimate action-values. With a model we can use state values to find a policy by searching possible actions, as with DP (value iteration, etc.). We can't do this without knowledge of the dynamics, so one of the primary goals of MC is to estimate $q_*$. We start with policy evaluation for action-values.

\subsubsection*{Policy Evaluation for Action-Values}
The policy evaluation problem for action-values is to estimate $q_\pi(s, a)$ for some $\pi$. This is essentially the same as for state values, only we now talk about state-action pairs being visited, i.e. taking action $a$ in state $s$, rather than just states being visited.\\

If $\pi$ is deterministic, then we will only estimate the values of actions that $\pi$ dictates. We therefore need to incorporate some exploration in order to have useful action-values (since, after all, we want to use them to make informed decisions).\\

One consideration is to make $\pi$ stochastic, e.g. $\varepsilon$-soft. Another is the assumption of \emph{exploring starts}, which specifies that ever state-action pair has non-zero probability of being selected as the starting state. Of course, this is not always posible in practice.\\

For now we assume exploring start. Later we will come back to the issue of \emph{maintaining exploration}

\subsection{Monte Carlo Control}
We make use of the GPI framework for action-values. Policy evaluation is done as described. Policy improvement is done by making the policy greedy with respect to the action-value function, so no model is needed for this step
\[
    \pi(s) \doteq \argmax_a q(s, a).
\]

We generate a sequence of policies $\pi_k$ each greedy with respect to $q_{\pi_{k-1}}(s, a)$. The policy improvement theorem applies: for all $s \in \mathcal{S}$
\begin{align*}
    q_{\pi_k}(s, a=\pi_{k+1}(s)) &= q_{\pi_k}(s, \argmax_a q_\pi(s, a)) \\
                    &= \max_a q_{\pi_k}(s, a) \\
                    &\geq q_{\pi_k}(s, \pi_k(s))\\
                    &= v_{\pi_k}(s)
\end{align*}

So $\pi_{k+1}$ uniformly better than $\pi_k$ or it is optimal. \\

The above procedure's convergence depends on assumptions of exploring starts and infinitely many episodes. We will relax the first later, but we will address the second now.\\

Two approaches to avoid infinitely many episodes:
\begin{enumerate}
    \item Stop the algorithm once the $q_{\pi_k}$ stop moving within a certain error. (In practice this is only useful on the smallest problems.)
    \item Stop policy evaluation after a certain number of episodes, moving the action value towards $q_{\pi_k}$, then go to policy improvement.
\end{enumerate}

For MC policy evaluation, it is natural to alternate policy evaluation and improvement on a episode by episode basis. We give such an algorithm below (with the assumption of exploring starts).\\

\includegraphics[width=\textwidth]{\ProjectDir/data/notes_images/mc_policy_iteration_exploring_starts.png}\\

It is easy to see that optimal policies are a fixed point of this algorithm. Whether this algorithm converges in general is still, however, an open question.

\subsection{Monte Carlo Control without Exploring Starts}
\subsubsection*{On Policy vs. Off Policy}
On-policy methods evaluate or improve the policy that is used to make decisions, whereas off-policy methods evaluate or improve one that is different than the one used to generate the data.

\subsubsection*{On-Policy Techniques without Exploring Starts}
We consider $\varepsilon$-greedy policies that put probability $1 - \varepsilon + \frac{\varepsilon}{|\mathcal{A}(s)|}$ on the maximal action and $\frac{\varepsilon}{|\mathcal{A}(s)|}$ on each of the others. These are examples of $\varepsilon$-soft policies in which $\pi(a|s) \geq \frac{\varepsilon}{|\mathcal{A}(s)|}$.\\

We use this idea in the GPI framework:\\

\includegraphics[width=\textwidth]{\ProjectDir/data/notes_images/on_policy_mc_algo.png}\\

We now show that an $\varepsilon$-greedy policy with respect to $q_\pi$, $\pi'$, is an improvement over any $\varepsilon$-soft policy $\pi$. For any $s \in \mathcal{S}$
\begin{align}
    q_\pi(s, \pi'(s)) &= \sum_a \pi'(a|s) q_\pi(s, a) \\
                      &= \frac{\varepsilon}{|\mathcal{A}(s)|} \sum_a q_\pi(s, a) + (1 - \varepsilon)\max_a q_\pi(s, a) \\
                      &\geq \frac{\varepsilon}{|\mathcal{A}(s)|} \sum_a q_\pi(s, a) + (1 - \varepsilon)\sum_a\frac{\pi(a|s) - \frac{\varepsilon}{|\mathcal{A}(s)|}}{1 - \varepsilon} q_\pi(s, a) \\
                      &= \sum_a \pi(a|s) q_\pi(s, a)\\
                      &= v_\pi(s)
\end{align}
(where line 3 follows because a weighted average with weights $w_i \geq 0$ and $\sum_i w_i = 1$ is $\leq$ the max term).\\

This satisfies the condition of the policy improvement theorem so we now know that $\pi' \geq \pi$.\\

Previously, with deterministic greedy policies, we would get automatically that fixed points of policy iteration are optimal policies since
\[
    v_*(s) \doteq \max_\pi v_\pi(s) \quad \forall s \in \mathcal{S}.
\]
Now our policies are not deterministically greedy, our value updates do not take this form. We note, however, that we can consider an equivalent problem where we change the environment to select state and reward transitions at random with probability $\varepsilon$ and do what our agent asks with probability $1 - \varepsilon$. We have moved the stochasticity of the policy into the environment, creating an equivalent problem. The optimal value function in the new problem satisfies its Bellman equation
\begin{align}
    \tilde{v}_\pi(s) &= (1 - \varepsilon) \max_a \tilde{q}_\pi(s, a) + \frac{\varepsilon}{|\mathcal{A}(s)|}\sum_a \tilde{q}_\pi(s, a) \\
                     &= (1 - \varepsilon) \max_a \sum_{s', r} p(s', r|s, a)[r + \gamma \tilde{v}_\pi(s')] + \frac{\varepsilon}{|\mathcal{A}(s)|} \sum_a \sum_{s', r} p(s', r|s, a)[r + \gamma \tilde{v}_\pi(s')].
\end{align}
We also know that at fixed points of our algorithm
\begin{align}
    v_\pi(s) &= (1 - \varepsilon) \max_a q_\pi(s, a) + \frac{\varepsilon}{|\mathcal{A}(s)|}\sum_a q_\pi(s, a) \\
                     &= (1 - \varepsilon) \max_a \sum_{s', r} p(s', r|s, a)[r + \gamma v_\pi(s')] + \frac{\varepsilon}{|\mathcal{A}(s)|} \sum_a \sum_{s', r} p(s', r|s, a)[r + \gamma v_\pi(s')].
\end{align}
This is the same equation as above, so by uniqueness of solutions to the Bellman equation we have that $v_\pi = \tilde{v}_\pi$ and so $\pi$ is optimal.

\subsection{Off-Policy Prediction via Importance Sampling}
Off-policy learning uses information gained by sampling the \emph{behaviour policy} $b$ to learn the \emph{target policy} $\pi$. The behaviour policy explores the environment for us during training and we update the target policy accordingly.\\

In this section we consider the prediction problem: estimating $v_\pi$ or $q_\pi$ for a fixed and known $\pi$ using returns from $b$. In order to do this we need the assumption of coverage:
\begin{equation}
    \pi(a|s) \geq 0 \implies b(a|s) \geq 0.
\end{equation}
This implies that $b$ must be stochastic wherever it is not identical to $\pi$. The target policy $\pi$ may itself be deterministic, e.g. greedy with respect to action-value estimates.

\subsubsection*{Importance Sampling}
We use \emph{importance sampling} to evaluate expected returns from $\pi$ given returns from $b$.\\

Define the importance sampling ratio as the relative probability of a certain trajectory from $S_t$
\begin{align}
    \rho_{t:T-1} &= \frac{\P{}(A_t, S_{t+1}, A_{t+1}, \dots)| S_t, A_{t:T-1} \sim \pi}{\P{}(A_t, S_{t+1}, A_{t+1}, \dots)| S_t, A_{t:T-1} \sim b} \\
                 &= \frac{\prod_{k=t}^{T-1}\pi(A_k|S_k)\P{}(S_{k+1}|S_k, A_k)}{\prod_{k=t}^{T-1}b(A_k|S_k)\P{}(S_{k+1}|S_k, A_k)}\\
                 &=\prod_{k=t}^{T-1}\frac{\pi(A_k|S_k)}{b(A_k|S_k)}
\end{align}
where the state transition dynamics $\P{}$ cancel out.\\

If we have returns $G_t$ from evaluating policy $b$, so $v_b(s) = \E[G_t|S_t=s]$, then we can calculate
\[
    v_\pi(s) = \E[\rho_{t:T-1}G_t|S_t=s]
\]

\subsubsection*{Estimation}
Introduce new notation:
\begin{itemize}
    \item Label all time steps in a single scheme. So maybe episode 1 is $t=1, \dots, 100$ and episode 2 is $t = 101, \dots, 200$, etc.
    \item Denote the set times of first/every visit to $s$ by $\mathcal{T}(s)$ (spanning episodes).
    \item Let $T(t)$ be the first termination after $t$
    \item Let $G_t$ be the returns from $t$ to $T(t)$
\end{itemize}

We can now give two methods of values for $\pi$ from returns from $b$:\\
{\bfseries{Ordinary Importance Sampling}}
\begin{equation}
    V(s) \doteq \frac{\sum_{t\in \mathcal{T}(s)} \rho_{t:T-1}G_t}{|\mathcal{T}(s)|}
\end{equation}
{\bfseries{Weighted Importance Sampling}}
\begin{equation}
    V(s) \doteq \frac{\sum_{t\in \mathcal{T}(s)} \rho_{t:T-1}G_t}{\sum_{t\in \mathcal{T}(s)} \rho_{t:T-1}}
\end{equation}
or 0 if the denominator is 0.\\

Weighted importance sampling is biased (e.g. it's expectation is $v_b(s)$ after 1 episode) but has bounded variance. The ordinary importance sampling ratio is unbiased, but has possibly infinite variance, because the variance of the importance sampling ratios themselves is unbounded.\\

Assuming bounded returns, the variance of the weighted importance sampling estimator converges to 0 even if the variance of the importance sampling ratios is infinite. In practice, this estimator usually has dramatically lower variance and is strongly preferred.

\subsection{Incremental Implementation}
We look for incremental calculations of the averages that make up the estimates, as in Chapter 2.\\

For on-policy methods the incremental averaging is the same as in Chapter 2. For off-policy methods, but with ordinary importance sampling, we only need to multiply the returns by the importance sampling ratio and then we can average as before.\\

We will now consider weighted importance sampling. We have a sequence of returns $G_i$, all starting in the same state $s$ and each with a random weight $W_i$ (e.g. $W_i = \rho_{i:T(i)-1}$). We want to iteratively calculate (for $n \geq 2$)
\[
    V_n = \frac{\sum_{k=1}^{n-1}W_kG_k}{\sum_{k=1}^{n-1}W_k}.
\]
We can do this with the following update rules
\begin{align}
    V_{n+1} &= V_n + \frac{W_n}{C_n}[G_n - V_n]\\
    C_{n+1} &= C_n + W_{n+1}
\end{align}
where $C_0 = 0$ and $V_1$ is arbitrary (notice that it cancels out as $V_2 = G_1$).\\

Below is an algorithm for off-policy weighted importance sampling (set $b=\pi$ for on policy). The estimator $Q$ converges to $q_\pi$ for all encountered state-action pairs.\\

\includegraphics[width=\textwidth]{\ProjectDir/data/notes_images/off_policy_mc_prediction_algo.png}\\

\subsection{Off-Policy Monte Carlo Control}
Below is an algorithm for estimating $\pi_*$ and $q_*$ in the GPI framework. The target policy $\pi$ is the greedy policy with respect to $Q$, which is an estimate of $q_\pi$. This algorithm converges to $q_\pi$ as long as an infinite number of returns are observed for each state-action pair. This can be achieved by making $b$ $\varepsilon$-soft. The policy $\pi$ converges to $\pi_*$ at all encountered states even if $b$ changes (to another $\varepsilon$-soft policy) between or within episodes.\\

\includegraphics[width=\textwidth]{\ProjectDir/data/notes_images/off_policy_mc_control_algo.png}\\

Notice that this policy only learns from episodes in which $b$ selects only greedy actions after some timestep. This can greatly slow learning.

\subsection{*Discounting Aware Importance Sampling}
We present a method of importance sampling that recognises the return as a discounted sum of rewards. This can help in estimation, since if an episode is of length 100 and $\gamma = 0$ then the final 99 terms of the importance sampling ration contribute nothing to the expected value of our estimator (they have expected value of 1) but can greatly increase its variance. We therefore construct a method of importance sampling that takes into account discounting.\\

Introduce the \emph{flat partial returns}
\[
    \bar{G}_{t:h} \doteq \sum_{i=t+1}^h R_{i} \quad 0 \leq t \leq h \leq T
\]
then it can be shown (by rearranging) that
\begin{align}
    G_t &\doteq \gamma^{i-t}R_{i+1}\\
        &= (1 - \gamma)\sum_{h=t+1}^{T-1}\gamma^{h-t-1}\bar{G}_{t:h} + \gamma^{T-t-1}\bar{G}_{t:T}.
\end{align}
Now we can scale each flat partial return by a truncated importance sampling ratio (hence reducing variance).\\

{\bfseries{Ordinary Importance Sampling Ratio}}
\begin{equation}
    V(s) \doteq \frac{\sum_{t\in \mathcal{T}(s)} \left[ (1 - \gamma) \sum_{h=t+1}^{T(t-1)} \gamma^{h - t - 1}\rho_{t:h-1}\bar{G}_{t:h} + \gamma^{T(t) - t - 1}\rho_{t:T(t) - 1} \bar{G}_{t:T(t)} \right]}{|\mathcal{T}(s)|}
\end{equation}

{\bfseries{Weighted Importance Sampling Ratio}}
\begin{equation}
    V(s) \doteq \frac{\sum_{t\in \mathcal{T}(s)} \left[ (1 - \gamma) \sum_{h=t+1}^{T(t-1)} \gamma^{h - t - 1}\rho_{t:h-1}\bar{G}_{t:h} + \gamma^{T(t) - t - 1}\rho_{t:T(t) - 1} \bar{G}_{t:T(t)} \right]}{\sum_{t\in \mathcal{T}(s)} \left[ (1 - \gamma) \sum_{h=t+1}^{T(t-1)} \gamma^{h - t - 1}\rho_{t:h-1} + \gamma^{T(t) - t - 1}\rho_{t:T(t) - 1} \right]}
\end{equation}

\subsection{*Per-Decision Importance Sampling}
There is another way in which we may be able to reduce variance in off-policy importance sapling, even in the absence of discounting ($\gamma = 1$). Notice that the off-policy estimators are made up of terms like
\[
    \rho_{t:T-1}G_t = \rho_{t:T-1} (R_{t+1} + \gamma R_{t+2} + \dots+ \gamma^{T-t-1}R_{T})
\]
and that each of these terms is of the form
\[
    \rho_{t:T-1}R_{t+1} = \frac{\pi(A_t|S_t)}{b(A_t|S_t)}\dots\frac{\pi(A_{T-1}|S_{T-1})}{b(A_{T-1}|S_{T-1})}R_{t+1}.
\]
Now notice that only the first and last terms here are correlated, while all the others have expected value 1 (taken with respect to $b$). Clearly this is also the case at each $t$. This means that
\[
    \E{}[\rho_{t:T-1}R_{t+k}] = \E{}[\rho_{t:t+k-1}R_{t+k}]
\]
therefore
\[
    \E{}[\rho{t:T-1}G_t] = \E{}[\tilde{G}_t]
\]
where
\[
    \tilde{G}_t \doteq \sum_{i=t}^{T-1}\gamma^{i-t}\rho_{t:i}R_{i+1}.
\]
Now we can write the ordinary importance sampling estimator as
\[
    V(s) \doteq \frac{\sum_{t\in\mathcal{T}(s)} \tilde{G}_t}{|\mathcal{T}(s)|}
\]
possibly reducing variance in the estimator. \\

The weighted importance sampling estimators of this form that have so far been found have been shown to not be consistent (in the statistical sense). We don't know if a consistent weighted average form of this exists.


\appendix
\appendixpage
\section{Terms That Need More Explanation Then A Footnote}
\subsection{Potential} \label{sec:potential}
Potential is the energy change that occurs when the position of an object changes \cite{potential}. There are many potentials, like electric potential, gravitational potential and elastic
potential. Let me explain the concept with an example. Say you are walking on a set of stairs in the upwards direction. As your muscles move to bring you one step upwards, energy that is used
by your muscles is converted into gravitational potential. Now imagine you turn around and go downwards instead. Notice how that is easier? That is due to the gravitational potential being
converted back into energy so your muscles have to deliver less energy to get you down. The potential is usually tied to a force, like the gravitational force.

\subsection{Asymptotic Runtime} \label{sec:runtime}
Asymptotic runtime is what we use in computer science to indicate how fast an algorithm works. We do it this way because concrete time indications (seconds, minutes, hours) are very machine
dependent. It matters a lot if your CPU, RAM and GPU are fast or not for the runtime. Therefore, we needed something to compare algorithms by which is machine independent. That is what asymptotic
runtime is. We have 3 notations for asymptotic runtime, $\Omega$ which is the lower bound of the runtime: not faster than; $O$ which is the upperbound of the runtime: not slower than; and we
have $\Theta$ which is the tight bound: not slower but also not faster than. After these 3 notations we usually denote the runtime in algebraic letters which stand for the input size. $O(n)$ for
instance means that for an input of size $n$ the algorithm will not run slower than $n$ operations. Whereas $\Omega(n^3)$ means that the algorithm needs for an input of size $n$ at least $n^3$
operations. Now this is not an exact match, as there are constants and other terms in the real runtime, but for asymptotic runtime we look at the most dominant factor, as that outgrows all the
other factors if the input size increases. You can compare this by plotting the functions $y = x$ and $z = x^2$ on a graphical calculator. No matter which constant $a$ you put in front of the $x$,
$z = x^2$ will at some point (note we don't specify when or where) be larger than $y = ax$. What you need to remember for all this is that polynomials are faster than exponentials ($n^2 < 2^n$)
and logarithms are faster than polynomials ($\log(n) < n$). $n!$ is very slow, $\log(n)$ is very fast.

\subsection{Complex Numbers} \label{sec:complex}
As you all know in the real numbers ($\mathbb{R}$) negative roots are not allowed as they do not exist. But what would happen if we would allow them to exist? Then we move into the area of
complex numbers. A complex number consists out of two parts, a real part and an imaginary part in the form $a + bi$ where $i = \sqrt{-1}$. Complex numbers have all kinds of properties, but what
we need them for are rotations. This is captured in Euler's formula $e^{it} = \cos(x) + i\sin(x)$ \cite{eulerFormula}. Which means that for time $t$ we rotate around the origin (of the complex
plane) forming a circle with radius one (the unit circle). Now if you would set $t = \pi$ then the result is $0$. What this means is that we have come full circle (hah) when $t = 2\pi$.


\section{The Argument Principle, Local Degree and Rouch\'e's Theorem}
\begin{definition}
    Let $D$ be a domain, we say a closed curve $\gamma:[0,1]\to\mathbb C$ bounds $D$ if $I(\gamma;w)=1$ for any $w\in D$ and $I(\gamma;w)=0$ for any $w\notin D\cup\gamma([0,1])$.
\end{definition}
Note that the orientation matters here.
$D$ is indeed bounded since $\gamma([0,1])$ is contained in $D_R(0)$ for a large enough $R$, but $I(\gamma;w)=0$ for any $w\notin D_R(0)$, so $D\subset D_R(0)$, hence is bounded.
\begin{theorem}[The Argument Principle]
    Let $\gamma$ be a closed curve bounding a domain $D$ and suppose that $f$ is meromorphic in some open set $U\subset \bar{D}\cup\gamma([0,1])$ such that $f$ has no pole or zero on $\gamma([0,1])$.
    If $f$ has precisely $N$ zeros and $P$ poles in $D$ (both counting with multiplicity), then
    $$N-P=\frac{1}{2\pi i}\int_\gamma\frac{f^\prime(z)}{f(z)}\,\mathrm dz=I(\Gamma;0)$$
    where $\Gamma=f\circ\gamma$.
\end{theorem}
\begin{proof}
    $N$,$P$ are finite since $D$ is bounded and zeros and poles are isolated points.
    Also note that $0\notin\Gamma([0,1])$ since $f\circ\gamma$ is never $0$.
    So
    $$I(\Gamma,0)=\frac{1}{2\pi i}\int_\Gamma\frac{1}{z}\,\mathrm dz=\frac{1}{2\pi i}\int_0^1\frac{1}{f(\gamma(t))}f^\prime(\gamma(t))\gamma^\prime(t)\,\mathrm dt=\frac{1}{2\pi i}\int_\gamma\frac{f^\prime(z)}{f(z)}\,\mathrm dz$$
    which is the second equality.\\
    As for the other part, note that if $a$ is neither a pole or a zero at $f$, then $f^\prime/f$ is holomorphic at $a$.
    Also, if $a$ is a zero (or a pole) of order $k$ at $z=a$, then $f^\prime/f$ has a simple pole at $z=a$ with $\operatorname{Res}_{f^\prime/f}(a)=k$ (or $-k$ respectively).
    Now $I(\gamma,w)=0$ for any $w\in D$ by hypothesis, so applying the residue theorem to $f^\prime/f$ shall give the result.
\end{proof}
\begin{definition}
    Let $f:D_R(a)\to\mathbb C$ be holomorphic and nonconstant, then the local degree of $f$ at $a$, denoted $\deg_f(a)$, is the order of zero of $f(z)-f(a)$ at $0$.
\end{definition}
Note that the local degree has to be positive.
\begin{theorem}[Local Degree Theorem]
    Consider $f:D_R(a)\to\mathbb C$ nonconstant and holomorphic.
    Suppose $\deg_f(a)=d>0$, then for every sufficiently small $r>0$, there is $\epsilon>0$ such that for any $w$ with $0<|w-f(a)|<\epsilon$, the equation $f(z)=w$ has precisely $d$ roots in $D(a,r)\setminus\{a\}$ that are all distinct.
\end{theorem}
\begin{proof}
    Since $f$ is nonconstant, by the principle of isolated zeros, we can choose $r>0$ such that $f(z)-f(a)\neq 0$ and $f^\prime(z)\neq 0$ for any $z\in B_r(a)\setminus\{a\}$.
    Now take $\gamma(t)=a+re^{it}$ for $t\in [0,2\pi]$, then $f(\gamma(t))\neq f(a)$ for any $t$, so $\Gamma=f\circ\gamma$ misses $f(a)$, so we can choose $\epsilon$ such that $\Gamma$ never enters $D_\epsilon(f(a))$.
    Then for $w\in D_\epsilon(f(a))\setminus\{f(a)\}$, by the argument principle, the number of zeros counting multiplicity of $f(z)-w$ in $D_r(a)$ is $I(\Gamma;w)=I(\Gamma;f(a))=d$.
    These zeros must all be in $D_r(a)\setminus\{a\}$ and none of them has multiplicity more than one as $f^\prime(a)\neq 0$.
\end{proof}
\begin{corollary}
    A nonconstant holomorphic function on a domain is an open map.
\end{corollary}
\begin{proof}
    Follows directly.
\end{proof}
\begin{theorem}[Rouch\'e's Theorem]
    Let $\gamma:[0,1]\to\mathbb C$ be a curve that bounds a domain $D$ and $f,g$ holomorphic on a open set $U$ containing $\bar D\cup\gamma([0,1])$.
    If $|f|>|g|$, then $f,f+g$ have the same number of zeros on $D$.
\end{theorem}
\begin{proof}
    $|f|>|g|$ hence $f,f+g$ are nowhere zero on $\gamma$, so we apply the argument principle on $h=(f+g)/f=1+g/f$.
    We have $I(h\circ\gamma;0)=0$, so $h$ has $N=P$, so $f+g,f$ has the same number of zeros.
\end{proof}


\section{Foundation of Quantum Mechanics}
\subsection{Quantum Mechanics of a Particle}
In classical dynamics, a particle is characterised by its position $\underline{x}$ and momentum $\underline{p}$.
We formulate classical laws in this framework.
E.g. in Newton's Second Law, we have $F(\underline{x})=m\underline{\ddot{x}}$.
A simple consequence of this is that the position and momentum of a particle at $t=0$ determines its motion under physical laws formulated in this way.\\
But in Quantum Mechanics, things are different.
\begin{postulate}[State and Wavefunction]
    A particle in space is described by its state $\psi:\mathbb R^3\times\mathbb R\to\mathbb C$ such that
    $$\int_{\mathbb R^3}|\psi(\underline{x},t)|^2\,\mathrm dV=N\in\mathbb R_+$$
    The probability amplitude of finding a particle at a place $\underline{x}$ and a given time $t$ is given by the state as a wavefunction $\psi(\underline{x},t)$.
\end{postulate}
There are subtle differences when we talk about state and when we talk about a wavefunction.
But we don't care.
\begin{postulate}[Born's Rule]
    $|\bar\psi(\underline{x},t)|^2\,\mathrm dV$ is the probability of finding the particle in $\mathrm dV$.
    Here, $\bar\psi$ is the normalised wavefunction $\bar\psi=\psi/\sqrt{N}$.
\end{postulate}
\begin{postulate}[Time-Dependent Schr\"odinger's Equation (TDSE)]
    For a particle of mass $m$ in potential $U(\underline{x})$, in the time evolution of $\psi$ we have
    $$i\hbar\frac{\partial\psi}{\partial t}=-\frac{\hbar^2}{2m}\nabla^2\psi+U\psi$$
\end{postulate}
Observe that the equation is first order in $t$ and second order in $\underline{x}$.
It is also linear, so it still holds if we replace $\psi$ by a scalar multiple of it, for example $\bar\psi$.
Heuristically, one can derive this equation (in dimension one) in the following way:\\
Consider the particle as a de Broglie wave with wavefunction $e^{i(kx-\omega t)}$.
So for a free particle with $U(x)=0$, we have $E=p^2/2m$, therefore the wavefunction is
$$\exp(i(kx-\omega t))=\exp\left( \frac{1}{\hbar}(px-Et) \right)=\exp\left( \frac{1}{\hbar}\left(px-\frac{p^2}{2m}t\right) \right)$$
which can be verified a solution to the equation.
But this is just heuristic, the de Broglie wave is not even normalisable.
We will (hopefully) see the actual stuff later.\\
One thing that can easily go wrong with these postulate is that we do not know if the normalised wavefunction will continue to be normalised as time goes, so we need some work on that.
It suffices show that TDSE guarantees that $N$ does not depend on $t$.
\begin{proposition}
    $N$ does not depend on $t$ assuming TDSE.
\end{proposition}
The proof involves the use of some techniques that need some analytical justification.
But this is an applied course, so nobody cares.
\begin{proof}
    Just differentiate
    \begin{align*}
        \frac{\mathrm dN}{\mathrm dt}&=\frac{\mathrm d}{\mathrm dt}\int_{\mathbb R^3}|\psi(\underline{x},t)|^2\,\mathrm dV\\
        &=\int_{\mathbb R^3}\frac{\partial}{\partial t}\left( \psi^*(\underline{x},t)\psi(\underline{x},t) \right)\,\mathrm dV\\
        &=\int_{\mathbb R^3}\psi^*\frac{\partial\psi}{\partial t}+\psi\frac{\partial \psi^*}{\partial t}\,\mathrm dV\\
        &=\int_{\mathbb R^3}\psi^*\left( \frac{i\hbar}{2m}\nabla^2\psi+\frac{i}{\hbar}U\psi\right)+\psi\left( -\frac{i\hbar}{2m}\nabla^2\psi^*-\frac{i}{\hbar}U\psi^* \right)\,\mathrm dV\\
        &=\frac{i\hbar}{2m}\int_{\mathbb R^3}\psi^*\nabla^2\psi-\psi\nabla^2\psi^*\,\mathrm dV\\
        &=\frac{i\hbar}{2m}\int_{\mathbb R^3}\nabla\cdot( \psi^*\nabla\psi-\psi\nabla\psi^*)\,\mathrm dV\\
        &=\frac{i\hbar}{2m}\int_{\partial V}(\psi^*\nabla\psi-\psi\nabla\psi^*)\cdot\mathrm d\underline{S}\\
        &=0
    \end{align*}
    by our boundary condition.
\end{proof}
\begin{remark}
    Assume that $\psi$ is normalised.
    Write $\rho(\underline{x},t)=|\psi(\underline{x},t)|^2$ as the probability density and
    $$\underline{J}=-\frac{i\hbar}{2m}(\psi^*\nabla\psi-\psi\nabla\psi^*)$$
    the probability current.
    Then we have the conservation law
    $$\frac{\partial\rho}{\partial t}+\nabla\cdot\underline{J}=0$$
    by the calculation involved in the above proof.
\end{remark}
\subsection{Principle of Superpositions}
As TDSE is linear in $t$, for states $\phi_1,\phi_2$ satisfying it, $a_1\phi_1+a_2\phi_2$ also satisfies TDSE for any $a_1,a_2\in\mathbb C$.
Obviously, we want to show that this superposition is either $0$ or also normalisable.
\begin{proposition}
    If $\phi_1,\phi_2$ are normalisable, so is $a_1\phi_1+a_2\phi_2$ for any $a_1,a_2\in\mathbb C$ given that it is nonzero.
\end{proposition}
\begin{proof}
    Quite obvious.
\end{proof}
\begin{corollary}
    The set of all states, together with the zero function, is a vector space.
\end{corollary}
This vector space is often denoted by $\mathcal H$.
\begin{proof}
    Immediate.
\end{proof}
We can equip a complex inner product
$$(\psi,\phi)=\int_{\mathbb R^3}\psi^*\phi\,\mathrm dV$$
on this space.
If we also know that it is complete with respect to that inner product, then $\mathcal H$ is a (complex) Hilbert space.
\subsection{Expectation Values and Operators}
For a (one dimensional) particle in state $\psi$, the average value of its position $x$ is then
$$\langle x\rangle=\int_{\mathbb R}x\rho(x,t)\,\mathrm dx=\int_{\mathbb R}x|\bar\psi(x,t)|^2\,\mathrm dx$$
which can be interpreted as the average of repeated measurement on an ensemble of identically prepared systems.
Classically $p=m\dot{x}$ is the momentum.
In quantum mechanics, we define the average momentum to be
$$\langle p\rangle=m\frac{\mathrm d\langle x\rangle}{\mathrm dt}=m\frac{\mathrm d}{\mathrm dt}\int_{\mathbb R}x|\bar\psi(x,t)|^2\,\mathrm dx$$
We can calculate it and using the conservation rule we found earlier and assuming the boundary condition that $\psi$ decays fast enough as $x\to\pm\infty$,
\begin{align*}
    \langle p\rangle&=m\frac{\mathrm d}{\mathrm dt}\int_{\mathbb R}x|\bar\psi(x,t)|^2\,\mathrm dx\\
    &=m\int_{\mathbb R}x\frac{\partial(\bar\psi^*\bar\psi)}{\partial t}\,\mathrm dx\\
    &=\frac{i\hbar}{2}\int_{\mathbb R}x\frac{\partial}{\partial x}\left( \bar\psi^*\frac{\partial\bar\psi}{\partial x}-\bar\psi\frac{\partial\bar\psi^*}{\partial x} \right)\,\mathrm dx\\
    &=\frac{i\hbar}{2}\left[x\bar\psi^*\frac{\partial\bar\psi}{\partial x}-x\bar\psi\frac{\partial\bar\psi^*}{\partial x}\right]_{-\infty}^\infty-\frac{i\hbar}{2}\int_{\mathbb R}\left( \bar\psi^*\frac{\partial\bar\psi}{\partial x}-\bar\psi\frac{\partial\bar\psi^*}{\partial x} \right)\,\mathrm dx\\
    &=-\frac{i\hbar}{2}\int_{\mathbb R}\left( \bar\psi^*\frac{\partial\bar\psi}{\partial x}-\bar\psi\frac{\partial\bar\psi^*}{\partial x} \right)\,\mathrm dx\\
    &=-i\hbar\int_{\mathbb R}\bar\psi^\ast\frac{\partial\bar\psi}{\partial x}\,\mathrm dx
\end{align*}
By a bad notation one can write
$$\langle p\rangle=\int_{\mathbb R}\psi^*\left( -i\hbar\frac{\partial}{\partial x} \right)\psi\,\mathrm dx$$
To make it even worse we can consider the functional $\hat{x}$ and $\hat{p}$ defined by $\hat{x}=x$ and $\hat{p}=-i\hbar\partial/\partial x$, which allows us to effectively confuse everybody by writing
$$\langle x\rangle=\int_{\mathbb R}\psi^*\hat{x}\psi\,\mathrm dx,\langle p\rangle=\int_{\mathbb R}\psi^*\hat{p}\psi\,\mathrm dx$$
In general dimensions, we can use the same idea with $\hat{\underline{x}}=\underline{x}$ and $\hat{\underline{p}}=-i\hbar\nabla$.
This is where we desperately try to justify these notations by considering them as (linear) operators in the Hilbert space $\mathcal H$, which works mathematically.
\footnote{However, the notation shall haunt you for a considerable proportion of your relationship with quantum mechanics.}
The kinetic energy operator is then
$$\hat{T}=\frac{\hat{p}^2}{2m}=-\frac{\hbar^2}{2m}\nabla^2$$
In fact, for any physical quantity $Q(x,p)$ , we are just gonna write $\hat{Q}=Q(\hat{x},\hat{p})$ and we can get
$$\langle Q(x,p)\rangle=\int_{\mathbb R}\psi^*\hat{Q}\psi\,\mathrm dx=\int_{\mathbb R}\psi^*Q\left( x,-i\hbar\frac{\partial}{\partial x} \right)\psi\,\mathrm dt$$
Also, one can check by direct calculation that
$$\frac{\mathrm d\langle p\rangle}{\mathrm dt}=\int_{\mathbb R}\psi^*\left( -\frac{\partial U}{\partial x} \right)\psi\,\mathrm dx=\langle -U_x\rangle$$
by using TDSE.
The Hamiltonian operator is $\hat{H}=\hat{T}+\hat{U}$, so
$$(\hat{H}\psi)(\underline{x},t)=-\frac{\hbar^2}{2m}\nabla^2\psi+U\psi$$
\subsection{Time-Independent Schr\"odinger Equation}
Note that we can rewrite the Schr\"odinger equation in the form
$$i\hbar\frac{\partial\psi}{\partial t}=\hat{H}\psi$$
SO if we seperate the variables $\psi(\underline{x},t)=X(\underline{x})T(t)$, then by some rearrangement
$$i\hbar T^{-1}T^\prime=(\hat{H}X)/X$$
The left hand side depends only on $t$ while the right hand side on $\underline{x}$, requiring them to be equal is just saying they are actually constants.
Denote this constant by $E$, then we get
$$\begin{cases}
    i\hbar T^{-1}T^\prime=E\\
    \hat{H}X=EX
\end{cases}$$
The first equation is easy to solve and gives $T(t)=e^{-iEt/\hbar}$.
The second equation is called the Time-Independent Schr\"odinger Equation (TISE).
Its solution $X$ is then interpreted as a physical state with energy $E$.
Note that $E$ must be real by our expression of $T$ and the condition that $T$ does not explode as $t\to\pm\infty$.
Also, TISE is essentially the eigenvalue problem of $\hat{H}$ which makes sense as it is a linear operator on the vector space of states and $0$.
So we have obtained the set of solutions $\psi=Xe^{-iEt/\hbar}$ where $X$ is an eigenfunction of $\hat{H}$ with eigenvalue $E$.
This set of solutions is called the stationary states.
\subsection{Stationary States}
For a stationary state, assuming $\psi$ is normalised, then
$$\rho(\underline{x},t)=|\psi(\underline{x},t)|^2=|X(\underline{x})|^2|e^{-iEt/\hbar}|^2=|X(\underline{x})|^2$$
So the stationary states are some particular solutions of TDSE whose induced probability distributions in space do not depend on time.
Of course, superpositions of this family of stationary states is also an allowed state (or zero).
What's more, if $\psi_1=X_1e^{-iE_1t/\hbar},\psi_2=X_2e^{-iE_2t/\hbar}$ are stationary states and $\psi=a_1\psi_1+a_2\psi_2$ is a superposition, then (assuming $a_i$ and $X_i$ are real),
\begin{align*}
    |\psi|^2&=(a_1^*X_1^*e^{iE_1t/\hbar}+a_2^*X_2^*e^{iE_2t/\hbar})(a_1X_1e^{-iE_1t/\hbar}+a_2X_2e^{-iE_2t/\hbar})\\
    &=|a_1|^2|X_1|^2+|a_2|^2|X_2|^2+a_1a_2X_1X_2(e^{i(E_1-E_2)t/\hbar}+e^{i(E_2-E_1)t/\hbar})\\
    &=|a_1|^2|X_1|^2+|a_2|^2|X_2|^2+2a_1a_2X_1X_2\cos\left( \frac{(E_1-E_2)t}{\hbar} \right)
\end{align*}
So $\psi$ is not a stationary state if $E_1\neq E_2$ as $|\psi|^2$ depends on time.
In fact, the stationary state is a basis of the vector space of states (and zero), that is each state $\psi$ can be expressed in the form
$$\psi(\underline{x},t)=\sum_{n=1}^Na_nX_n(x)e^{-iE_nt/\hbar}$$
where $X_n,E_n$ are stuff you expect them to be.
We then interpret $|a_n|^2$ to be the probability for the energy to be $E_n$.
\begin{remark}
    If we have a discrete and normalisable basis $X_n$ of the Hamiltonian $\hat{H}$.
    Then we might write something of the form
    $$\psi=\sum_{n=1}^\infty a_nX_ne^{-iE_nt/\hbar}$$
    which is a solution to TDSE if we can show that it converges nice enough.
    But this will require
    $$\lim_{R\to\infty}\int_{|\underline{x}|>R}|X_n|^2\,\mathrm dx=0$$
    So the particle cannot be too far from the origin.
    We call this a bounded state.\\
    How about a continuous basis (i.e. the basis is indexed by $(X_\alpha)_{\alpha\in I}$ where $I$ is an interval)?
    Then we might write
    $$\psi=\int_{\alpha\in I}A(\alpha)X_\alpha(\underline{x})e^{-iE_\alpha t/\hbar}\,\mathrm d\alpha$$
    So we interpret $|A(\alpha)|^2\,\mathrm d\alpha$ as the probability for the state to have energy $E_\alpha$.
    But the same limit condition does not have to hold, since even if the state themselves are not normalisable, we can choose an $A$ that decays fast enough to make the eventual superposition normalisable.
    This is called a scattering state.
\end{remark}


\section{Summary}
\label{sec:summary}

\begin{frame}{Multigroup Neutron Diffusion Equation}
    \begin{equation}
      \label{eq:multigroup_diffusion}
      \grad \cdot \current_g(\vr) + \Sigma_{r,g}(\vr) \phi_g(\vr)=
        \frac{\chi_g(\vr)}{\lambda}
        \sum_{g'=1}^{G} \nu\Sigma_{f,g'}(\vr) \phi_{g'}(\vr) +
        \sum_{\substack{g'=1 \\ g' \ne g}}^{G}
        \Sigma_{s,g' \rightarrow g}(\vr) \phi_{g'}(\vr)
    \end{equation}
    \vspace{-2\baselineskip}
    \begin{conditions} % custom environment designed for this purpose
      \vr & spatial position vector, \\
      \current_g(\vr) & net neutron current for energy group $g$
        \units{$\frac{1}{\text{cm}^2 \; \text{s}}$}, \\
      \phi_g(\vr) &
        \parbox[t]{\columnwidth}{fundamental eigenvector,  \\
        scalar neutron flux for energy group $g$
        \units{$\frac{1}{\text{cm}^2 \; \text{s}}$},} \\
      \Sigma_{r,g}(\vr) & macroscopic removal cross section for energy group $g$
        \units{$\frac{1}{\text{cm}}$}, \\
      \chi_g(\vr) & fission spectrum for energy group $g$,\\
      \lambda &
        \parbox[t]{\columnwidth}{fundamental eigenvalue, \\
        effective neutron multiplication factor,} \\
      \nu \Sigma_{f,g}(\vr) &
        \parbox[t]{\columnwidth}{number of fission neutrons times macroscopic
        fission \\
        cross section in energy group $g$ \units{$\frac{1}{\text{cm}}$},} \\
      \Sigma_{s,g' \rightarrow g} (\vr) &
        \parbox[t]{\columnwidth}{macroscopic scatter cross section from
        energy group $g'$ to \\
        energy group $g$ \units{$\frac{1}{\text{cm}}$},} \\
      G & total number of energy groups.
    \end{conditions}
\end{frame}

\begin{frame}{\glsentryshort{nem} Equations}
  Transverse integrated multigroup neutron diffusion equation. \\
  Note: node indices $i,j,k$ have been omitted.
  \begin{align}
    \label{eq:transverse_multigroup_diffusion}
    \frac{d \current_{g,u} (u)}{d u} + \overline{\Sigma_{r,g}}
      \phi_{g,u}(u) &= Q_{g,u}(u) - L_{g,u}(u) \\
    \label{eq:current_approximation}
    \current_{g,u}(u) &= - \overline{D_g} \, \frac{d \phi_{g,u}(u)}{du}
  \end{align}
  \vspace{-\baselineskip}
  \begin{conditions}
    u          & coordinate direction (i.e. $u = x,y,z$), \\
    \overline{\Sigma_{r,g}} & average value of $\Sigma_{r,g}(\vr)$ in node $i,j,k$, \\
    \overline{D_g} & average value of diffusion coefficient in node $i,j,k$, \\
    Q_{g,u}(u) & transverse integrated neutron source, \\
    L_{g,u}(u) & transverse leakage.
  \end{conditions}
\end{frame}

\begin{frame}{\glsentryshort{nem} Projections}
  Basis functions are typically polynomials.\\
  \citeauthor{qe2paper} select Legendre polynomials.
  \begin{align}
    \label{eq:flux_expansion}
    \phi_{g,u}(u) &= \sum_{n=0}^{N_{\phi} = 4} a_{g,u,n} \, f_{u,n}(u), \\
    \label{eq:source_expansion}
    Q_{g,u}(u)    &= \sum_{n=0}^{N_Q = 2}      q_{g,u,n} \, f_{u,n}(u), \\
    \label{eq:leakage_expansion}
    L_{g,u}(u)    &= \sum_{n=0}^{N_L = 2}      l_{g,u,n} \, f_{u,n}(u),
  \end{align}
  \vspace{-\baselineskip}
  \begin{conditions}
    a_{g,u,n} & expansion coefficient of $\phi_{g,u}(u)$, \\
    q_{g,u,n} & expansion coefficient of $Q_{g,u}(u)$, \\
    l_{g,u,n} & expansion coefficient of $L_{g,u}(u)$, \\
    f_{u,n}(u) & $n^{th}$ Legendre polynomial.
  \end{conditions}
\end{frame}

\begin{frame}{Local Elimination}
  \begin{itemize}
    \item Odd and even coefficients can be solved separately \cite{gehinThesis}.
    \item \citeauthor{qe2paper} show $a_{g,u,1}$ and $a_{g,u,3}$ can be written
      in terms of each other.
    \item $a_{g,u,1-3}$ and $a_{g,u,2-4}$ are introduced.
    \item Solution vector, $\Phi_g$ is length $10 \times N \times G$.
  \end{itemize}
  \begin{equation}
    \label{eq:solution_vector}
    \vPhi_g =
    \begin{pmatrix}
      \current_{g,x,+} \\
      \current_{g,y,+} \\
      \vspace{8pt}
      \current_{g,z,+} \\
      \overline{\phi_g} \\
      a_{g,x,1-3} \\
      a_{g,y,1-3} \\
      a_{g,z,1-3} \\
      a_{g,x,2-4} \\
      a_{g,y,2-4} \\
      a_{g,z,2-4}
    \end{pmatrix}
  \end{equation}
\end{frame}

\begin{frame}{\glsentryshort{jfnk} Theory}
  The $m^{th}$ Newton step.
  \begin{equation}
    \label{eq:newton_step}
    \jacobian (\vx^m) \cdot \step^m = - \residual(\vx^m)
  \end{equation}
  The step proceeds.
  \begin{equation}
    \vx^{m+1} = \vx^{m} + \step^{m}
  \end{equation}
\end{frame}

\begin{frame}{Inexact Newton Condition}
  The Newton step is solved with a Krylov solver. \\
  \glsentryshort{gmres} and \glsentryshort{bicgstab} are both investigated.
  \begin{equation}
    \label{eq:inexact_newton_condition}
    \| \residual(\vx^m) + \jacobian(\vx^m) \cdot \step^m \| \le
      \eta_m \| \residual(\vx^m) \|
  \end{equation}
  The Krylov solver does not require an explicit Jacobian, only the
  Jacobian-vector product which can be approximated with finite differences.
  \begin{equation}
    \label{eq:dirder}
    \jacobian(\vx^m) \cdot \vv \approx \frac{\residual(\vx^m + \epsilon \vv) -
      \residual(\vx^m)}{\epsilon}
  \end{equation}
  Typically, $\epsilon = \sqrt{\epsilon_{mach}} \approx 10^{-8}$.
\end{frame}

\begin{frame}{Choice of Physics-Based Preconditioner}
  \begin{equation}
    \label{eq:left_precondition}
    \| \mm_L^{-1} \residual(\vx^m) + \mm_L^{-1} \left( \jacobian(\vx^m)
      \cdot \step^m \right) \| \le \eta_m \| \mm_L^{-1} \residual(\vx^m) \|
  \end{equation}
  \begin{itemize}
    \item Preconditioner should approximate the Jacobian inverse
      \cite{textbookkelley}.
    \item \citeauthor{gill_azmy} investigate several choices of preconditioner
      and conclude that preconditioning with $\approx 5$~\glspl{pi} is ideal.
    \item \citeauthor{jfnk_wielandt} present similar results.
    \item \citeauthor{qe2paper} develop a preconditioner based on available
      data.
    \item Solved using \gls{tdma} and then \gls{adi} method.
    \item No preconditioner comparison provided.
  \end{itemize}
\end{frame}

\begin{frame}{Convergence Rates of \glsentryshort{jfnk} and \glsentryshort{pi}
  Methods}
  \begin{itemize}
    \item \gls{pi}.
    \begin{itemize}
      \item Converges linearly at a rate determined by the dominance ratio
        \cite{nakamura}.
      \begin{equation}
        d = \frac{\lambda_1}{\lambda_0}
      \end{equation}
      \item Typically, $d > 0.95$ is common and the \gls{ws} is used (to be
        discussed) \cite{gehinThesis}.
      \begin{equation}
        d' = \frac{\frac{1}{\lambda_0} - \frac{1}{\lambda'}}
          {\frac{1}{\lambda_1} - \frac{1}{\lambda'}}
      \end{equation}
      % if \lambda_0 = 1.0, \lambda_1 = 0.95, \lambda' = \lambda_0 + 0.03
      % d = 0.95, d' = 0.35625
    \end{itemize}
    \item \gls{jfnk}.
    \begin{itemize}
      \item Convergence rate determined by Jacobian properties (e.g. Lipschitz
        constant) \cite{textbookkelley}.
      \item Not affected by dominance ratio \cite{gill_azmy}.
      \item Will not be affected by \gls{ws} despite claim of
        \citeauthor{qe2paper}.
    \end{itemize}
  \end{itemize}
\end{frame}


\subsection{Bitshifts}

We can have other operations where bits do interact. An important operator is the bit shift. This takes a series of bits and shifts them to the right or left by one place. This pushes one bit off the end.

The new bit can take a \(0\) or \(1\).

Logic gates are not needed for bit shifts. Instead wiring of inputs to outputs achieves the same effect.


\chapter{Engel Expansions}
\label{ch:maximizing_target_node}

\section{Engel expansions maximize the node $v_{n+1}$}
A sequence $v_{n+1},v_n,\ldots,v_2,v_1$ describing a path in $H_{C,3}$ from $v_{n+1}$ down to $v_1$ allows at most one division by $2$ between two successive nodes. Dividing only once between two successive nodes, maximizes the $v_{n+1}$, but it is not obvious that this maximizes the product contained in condition~\ref{eq:condition_max}. Such a sequence forms the following ascending continued fraction (cf. also \cite[p.~11]{Ref_Laarhoven}):

\begin{equation}
\label{eq:asc_continued_fraction}
v_{n+1}=\cfrac{3\cfrac{3\cfrac{3\cfrac{3v_1+1}{2}+1}{2}+1}{2}+1}{2}\dotsb
=\frac{3^nv_1+\sum_{i=0}^{n-1}3^i2^{n-1-i}}{2^n}
=\frac{3^n(v_1+1)-2^n}{2^n}
\end{equation}

\par\medskip
The sum of the products of the powers of three and two, contained within the above term, can be simplified to the difference $3^n-2^n$ by converting the sum expression into the form $(x-1)(1+x+x^2+\cdots+x^{n-2}+x^{n-1})=x^n-1$ as follows:
\[
\frac{2^n}{2^n}(3-2)\sum_{i=0}^{n-1}3^i2^{n-1-i}
=\frac{2^n}{\cancel{2^{n-1}}}\cdot\frac{3-2}{2}\sum_{i=0}^{n-1}3^i2^{\cancel{n-1}-i}
=2^n\left(\frac{3}{2}-1\right)\sum_{i=0}^{n-1}\left(\frac{3}{2}\right)^i
=2^n\left(\left(\frac{3}{2}\right)^n-1\right)
\]

\begin{example}
	A concrete example for such a sequence is $v_1=31$, $v_2=47$, $v_3=71$, $v_4=107$, $v_5=161$. And, to follow that example, we can calculate the label of the vertex $v_5$ in a straightforward way:
	\[
	v_5=v_{n+1}=\frac{3^4(31+1)-2^4}{2^4}=161
	\]
	By choosing a vertex $v_1=2^{n+1}-1$, we are able to infinitely generate sequences which each form an ascending continued fraction. As per equation~\ref{eq:asc_continued_fraction} the last member in this sequence is the odd labeled vertex $v_{n+1}=3^n\cdot2-1$.
\end{example}

\begin{remark}
	Ascending variants of a continued fraction, such as used in equation~\ref{eq:asc_continued_fraction}, shall not be confused with continued fractions as treated for example in \cite{Ref_Moore}, \cite{Ref_Hensley}, \cite{Ref_Borwe_etal}. These ascending continued fractions correspond to the so-called "Engel Expansions" \cite{Ref_Kraaikamp_Wu}.
\end{remark}

\par\noindent
As illustrated below, we can formulate the ascending continued fractions in a generalized fashion, whereas the analogy to \ref{eq:asc_continued_fraction} is given by $b_1=b_2=b_3=b_4=2$ and $a_1=3^0$, $a_2=3^1$, $a_3=3^2$ and $a_4=3^3+3^4v_1$:
\[
\cfrac{a_1+\cfrac{a_2+\cfrac{a_3+\cfrac{a_4}{b_4}}{b_3}}{b_2}}{b_1}\dotsb=\frac{a_1}{b_1}+\frac{a_2}{b_1b_2}+\frac{a_3}{b_1b_2b_3}+\frac{a_4}{b_1b_2b_3b_4}+\cdots
\]

\par\medskip
The generalized form of equation~\ref{eq:asc_continued_fraction} may be used to compute any of the above-named ascending continued fraction that has $a_i=k^{i-1}$, $b_i=b$ for $i\in\mathbb{N}$ and $a_n=k^{n-1}+k^nv_1$:

\par\medskip
\begin{equation}
\label{eq:generalized_asc_continued_fraction}
v_{n+1}=\frac{k^n(kv_1-bv_1+1)-b^n}{b^n(k-b)}
\end{equation}

\section{Include more divisions by two into an Engel expansion}
\label{sec:include_divisions_engel_expansion}
In order to calculate the largest possible $v_{n+1}$, we have thus far considered Engel expansions which contain only $n$ division by two within a Collatz sequence of $n+1$ members. In the following, we include $m$ additional divisions by two and thus a total of $m+n$ divisions. Now we examine two corner cases:
\begin{itemize}
	\item the one where we do the additional $m$ divisions by $2$ at the end and
	\item the one where we do these additional divisions at the very beginning.
\end{itemize}

\par\noindent
\textbf{The first case} is our starting point to examine how swapping a division by two affects the node $v_{n+1}$. For this, let us compare the Engel expansion where we divide by $2^m$ afterwards with one where we divide by $2$ in the penultimate step and by $2^{m-1}$ in last step. One can immediately recognize the following inequality with a mere look:
\[
\cfrac{1+\cfrac{3+\cfrac{3^2+\cfrac{3^3+3^4v_1}{2}}{2}}{2}}{2\cdot2^m}
<
\cfrac{1+\cfrac{3+\cfrac{3^2+\cfrac{3^3+3^4v_1}{2}}{2}}{2\cdot\textcolor{red}{\mathbf{2}}}}{2\cdot2^{m-1}}
\]

To put it simply, in the expansion on the right side of the above-shown inequality we perform one division by two a little bit earlier as we do it in the expansion on the left side of the expansion. Almost all summands of both expansions cancel out each other:

\[
\frac{1}{2\cdot2^m}+\cancel{\frac{3}{2^2\cdot2^m}+\frac{3^2}{2^3\cdot2^m}+\frac{3^3+3^4v_1}{2^4\cdot2^m}}
<
\frac{1}{2\cdot2^{m-1}}+\cancel{\frac{3}{2^2\cdot\textcolor{red}{\mathbf{2}}\cdot2^{m-1}}+\frac{3^2}{2^3\cdot\textcolor{red}{\mathbf{2}}\cdot2^{m-1}}\frac{3^3+3^4v_1}{2^4\cdot\textcolor{red}{\mathbf{2}}\cdot2^{m-1}}}
\]

\par\bigskip\noindent
\textbf{The second case} deals with Engel expansions where we perform that additional $m$ divisions by two as early as possible. The resulting value decreases, when we make a division by two later:
\[
\cfrac{1+\cfrac{3+\cfrac{3^2+\cfrac{3^3+3^4v_1}{2\cdot2^{m-1}}}{2\cdot\textcolor{red}{\mathbf{2}}}}{2}}{2}
<
\cfrac{1+\cfrac{3+\cfrac{3^2+\cfrac{3^3+3^4v_1}{2\cdot2^m}}{2}}{2}}{2}
\]

Also here almost all summands of both Engel expansions cancel each other out:

\[
\cancel{\frac{1}{2}+\frac{3}{2^2}}+\frac{3^2}{2^3\cdot\textcolor{red}{\mathbf{2}}}+\cancel{\frac{3^3+3^4v_1}{2^4\cdot\textcolor{red}{\mathbf{2}}\cdot2^{m-1}}}
<
\cancel{\frac{1}{2}+\frac{3}{2^2}}+\frac{3^2}{2^3}+\cancel{\frac{3^3+3^4v_1}{2^4\cdot2^m}}
\]

\par\medskip
While the first case minimizes the value of the node $v_{n+1}$, the second case maximizes it. The difference between the maximum and the minimum is given by the following equation:

\begin{flalign*}
	&\frac{3^{n-1}\left(\frac{3v_1+1}{2\cdot2^m}+1\right)-2^{n-1}}{2^{n-1}}-\frac{3^n\left(v_1+1\right)-2^n}{2^{n+m}}\\
	=&\frac{3^{n-1}\cdot\left(3v_1+1+2^{m+1}\right)-2^{n-1}\cdot2^{m+1}-3^n\left(v_1+1\right)+2^n}{2^{m+1}\cdot2^{n-1}}\\
	=&\frac{3^{n-1}+3^{n-1}\cdot2^{m+1}-2^{n+m}-3^n+2^n}{2^{n+m}}=\frac{3^{n-1}-3\cdot3^{n-1}+3^{n-1}\cdot2^{m+1}-2^{n+m}+2^n}{2^{n+m}}\\
	=&\frac{-2\cdot3^{n-1}+3^{n-1}\cdot2^{m+1}-2^{n+m}+2^n}{2^{n+m}}=\frac{\left(2\cdot3^{n-1}-2^n\right)\left(2^m-1\right)}{2^n\cdot2^m}\\
	=&\left(\frac{3^{n-1}}{2^{n-1}}-1\right)\left(1-\frac{1}{2^m}\right)
\end{flalign*}

This has the consequence that for a given sequence consisting of $n+1$ members, between which a total of $n+m$ divisions have taken place, the permutation of these divisions has a very limited effect on the node $v_{n+1}$ as described by theorem~\ref{theo:permutation}.

\par\medskip
\begin{theorem}
	\label{theo:permutation}
	Let $v_{n+1},v_n,\ldots,v_2,v_1$ be a sequence in which a total of $n+m$ divisions took place (a path in which a total of $n+m$ edges has been contracted). No matter how these divisions are permuted, i.e. performed sooner or later, the node $v_{n+1}$ can differ at most by the following product:
	\[
	\left(\frac{3^{n-1}}{2^{n-1}}-1\right)\left(1-\frac{1}{2^m}\right)
	\]
\end{theorem}

\section{The product in the condition for alpha's upper limit}
Let us take a closer look at the product contained in condition~\ref{eq:condition_max} for the case $k=3$ and use the ascending continued fractions for examining this product. The exciting question is, does this product have a limit value even in the case where we only contract a single edge between successive nodes? Setting the according sequence, which maximizes $v_{n+1}$, into the product expressed by condition~\ref{eq:condition_max}, we obtain a product that is limited, or to be more specific, which in the worst case $v_1=1$ converges (for $n$ to infinity) towards $2$:
\begin{equation}
\label{eq:product_simplification_k3}
\prod_{i=1}^{n}\left(1+\frac{1}{3v_{i}}\right)
=\prod_{i=1}^{n}\left(1+\frac{1}{3\frac{3^{i-1}(v_1+1)-2^{i-1}}{2^{i-1}}}\right)
=\prod_{i=1}^{n}\frac{3^i(v_1+1)-2^i}{3^i(v_1+1)-3*2^{i-1}}
=\frac{1}{v_1}-\frac{1}{v_1}\left(\frac{2}{3}\right)^n+1
\end{equation}

The above-illustrated last forming step, simplifies this product significantly into an expression waiving a product formulation. A detailed breakdown including all intermediate steps of this simplification is shown in the appendix~\ref{appx:product_simplification_k3}. The correctness of this simplification can be proven inductively too, which we detail in appendix~\ref{appx:proof_product_simplification_k3}. The most important and the most interesting aspect of this result is, that the above simplified term cannot exceed the value $2$, whatever you choose to insert into $n$ or into $v_1$:
\[
\frac{1}{v_1}-\frac{1}{v_1}\left(\frac{2}{3}\right)^{n+1}+1<2
\]

As demonstrated above, since the product cannot exceed the value $2$, the logarithmic product expression in the condition~\ref{eq:condition_max} cannot exceed the value one and this condition becomes a consistently true statement:
\[
n\log_23-\lfloor n\log_23\rfloor<2-1
\]

Thus, for $k=3$ the condition~\ref{eq:condition_max} for alphas's upper limit is met for all sequences that maximize $v_{n+1}$.

\section{Include additional divisions into the product}
How does the product, contained in condition~\ref{eq:condition_max} look if we include the additional $m$ divisions into the Engel expansion as per section~\ref{sec:include_divisions_engel_expansion}? To answer this question, we consider the sequence $v_{n+1},v_n,v_{n-1},\ldots,v_2,v_1$ and we set $v_2=\frac{3v_1+1}{2\cdot2^{m}}$. Then reusing the continued fraction given by equation~\ref{eq:asc_continued_fraction}, we obtain:

{\setlength{\jot}{1.2em}
\begin{flalign}
\label{eq:asc_continued_fraction_m}
v_{n+1}&=\cfrac{3\cfrac{3\cfrac{3\cfrac{3v_1+1}{2\cdot2^m}+1}{2}+1}{2}+1}{2}\dotsb
=\cfrac{3\cfrac{3\cfrac{3v_2+1}{2}+1}{2}+1}{2}\dotsb
=\frac{3^{n-1}(v_2+1)-2^{n-1}}{2^{n-1}}\\
\notag
&=\frac{3^{n-1}(\frac{3v_1+1}{2\cdot2^{m}}+1)-2^{n-1}}{2^{n-1}}=\frac{3^nv_1+3^{n-1}+3^{n-1}2^{m+1}}{2^{m+n}}-1
\end{flalign}}

\par\noindent
The product will be calculated by using equation~\ref{eq:product_simplification_k3}:
\begin{flalign}
\label{eq:product_k3_m}
\prod_{i=1}^{n}\left(1+\frac{1}{3v_{i}}\right)&=\left(1+\frac{1}{3v_1}\right)\cdot\prod_{i=2}^{n}\left(1+\frac{1}{3v_{i}}\right)\\
\notag
&=\left(1+\frac{1}{3v_1}\right)\cdot\prod_{i=1}^{n-1}\left(1+\frac{1}{3v_{i+1}}\right)=\left(1+\frac{1}{3v_1}\right)\cdot\left(\frac{1}{v_2}-\frac{1}{v_2}\left(\frac{2}{3}\right)^{n-1}+1\right)
\end{flalign}

\par\noindent
Finally substituting $v_2=\frac{3v_1+1}{2\cdot2^{m}}$ into equation~\ref{eq:product_k3_m} leads to the simplified formula of the product:
\begin{equation}
\label{eq:product_simplification_k3_m}
\prod_{i=1}^{n}\left(1+\frac{1}{3v_{i}}\right)=\left(1+\frac{1}{3v_1}\right)\cdot\frac{1-\left(\frac{2}{3}\right)^{n-1}+v_2}{v_2}=\frac{1+2^{m+1}}{3v_1}-\frac{2^m}{v_1}\left(\frac{2}{3}\right)^n+1
\end{equation}

\begin{example}
	An example provides the sequence $v_1=661$, $v_2=31$, $v_3=47$, $v_4=71$, $v_5=107$. When we input $v_1=661$ with $m=5$ and $n=4$ into equation~\ref{eq:asc_continued_fraction_m} we retrieve the label of the vertex $v_5$:
	\[
	v_5=v_{n+1}=\frac{3^4\cdot661+3^3+3^3\cdot2^6}{2^9}-1=107
	\]
	In this sequence five $(m=5)$ additional divisions by two took place in the first step using $v_1$:
	\[
	\frac{3\cdot661-1}{2\cdot2^5}=v_2=31
	\]
	Let us now verify the formula for the product by taking this particular example. To this end, we input $v_1=661$ together with $m=5$ and $n=4$ into equation~\ref{eq:product_simplification_k3_m}:
	\[
	\left(1+\frac{1}{3\cdot661}\right)\left(1+\frac{1}{3\cdot31}\right)\left(1+\frac{1}{3\cdot47}\right)\left(1+\frac{1}{3\cdot71}\right)=\frac{1+2^{6}}{3\cdot661}-\frac{2^5}{661}\left(\frac{2}{3}\right)^4+1=1.023215853
	\]
\end{example}

\section{Condition for a limited growth of the Engel expansion}
\label{sec:condition_limited_growth}
Let us look now into the question of what condition must be met to prevent a greater growth than a decline in Collatz sequences. Specifically, we consider an Engel expansion comprising $n+1$ sequence members that include $m$ additional divisions by two at the beginning. The last member $v_{n+1}$ in such a sequence can be calculated by formula~\ref{eq:asc_continued_fraction_m}. In order to restrict the growth of this sequence, we require that the last member has to be smaller than the first one. For this we define the condition $v_{n+1}<v_1$:

\[
\frac{3^nv_1+3^{n-1}+3^{n-1}2^{m+1}}{2^{m+n}}-1<v_1
\]

By transforming this inequality, which is thoroughly described in the appendix~\ref{appx:condition_limited_growth} step by step, we obtain the condition:

\begin{equation}
\label{eq:condition_limited_growth}
	\frac{3^{n-1}\left(2^{m+1}-2\right)}{2^{m+n}-3^n}-1<v_1
\end{equation}

\section{Engel expansions maximize the product}
The question which sequence maximizes the target node $v_{n+1}$ ties into the question which sequence maximizes the product in the condition for cycle-alpha's upper limit given by equation~\ref{eq:condition_max}. The product formula that do not depend from all vertices $v_1,v_2,\ldots v_n$ has been evolved in appendix~\ref{appx:product_formula_depending_v1}. This formula depends only from $2^\alpha$, from the starting node $v_1$ and the target node $v_{n+1}$:

\[
\prod_{i=1}^{n}\left(1+\frac{1}{kv_i}\right)=\frac{2^{\alpha_1+\ldots+\alpha_n}v_{n+1}}{k^nv_1}
\]

In order to maximize this product, one needs to maximize the target node $v_{n+1}$, which is exactly what the Engel expansion does. Hence, for a given $v_1$, the Engel expansion is the worst case sequence maximizing the product in the condition for cycle-alpha's upper limit.


\documentclass[../main.tex]{subfiles}
\begin{document}
 %\subsection{Circuits in a graph}
 \begin{thm}
 Let $E$ be the edge set of a graph $G$ and let $\mathcal{C}$ be the edge sets of cycles in $G$.\\
 \noindent Then $\mathcal{C}$ is the set of circuits of a matroid.
 \end{thm}

\begin{proof}
Let $A, B \in \mathcal{C}, A \neq B $ and let $e \in A \cap B .$

\vspace{1mm}

\noindent We must now construct a minimal cycle of $G$ whose edge set is contained in $(A \cup B) \setminus \{e\}.$\\
\noindent For $ i = 1,2,3,...... $ let $P_1$ be a path whose edge set is $ A \setminus \{e\}.$\\
\noindent $ A \setminus \{e\} \in \mathcal{I} $ therefore $P_1$ is not a cycle of $G$. This path will traverse from the edge $a_j$ to $a_u$ where $u,j$ were the vertices connecting the edge e to $(A \cup B) \setminus \{e\}$ to make $A \cup B$.\\
\noindent Now perform the same procedure for a path $P_2$ whose edge set is $B \setminus \{e\}.$\\
\noindent $P_1$ and $P_2$ should meet at the junctions $u,v$,  where $e$ was removed to make $ (A \cup B) \setminus \{e\}.$\\
\noindent Therefore $P_1 \cup P_2$ should be a cycle of $G$.\\
\noindent $\implies$ (C3) holds.\\
\noindent $\implies \mathcal{C}$ is the set of circuits in $G$.\\
\end{proof}
\end{document}


\section*{Exercises}

\begin{ex} Let $z = 3 + 3i$ be a complex number written in standard form. Convert $z$ to polar form, and write it in the form $z = re^{i\theta}$.
%\begin{sol}
%\end{sol}
\end{ex}

\begin{ex} Let $z = 2i$ be a complex number written in standard form. Convert $z$ to polar form, and write it in the form $z = re^{i\theta}$.
%\begin{sol}
%\end{sol}
\end{ex}

\begin{ex} Let $z = 4e^{\frac{2\pi}{3}i}$ be a complex number written in polar form. Convert $z$ to standard form, and write it in the form $z = a+bi$.
%\begin{sol}
%\end{sol}
\end{ex}

\begin{ex} Let $z = -1e^{\frac{\pi}{6}i}$ be a complex number written in polar form. Convert $z$ to standard form, and write it in the form $z = a+bi$.
%\begin{sol}
%\end{sol}
\end{ex}

\begin{ex} If $z$ and $w$ are two complex numbers and the polar form of $z$
involves the angle $\theta $ while the polar form of $w$ involves the angle
$\phi$, show that in the polar form for $zw$ the angle involved is $\theta
+\phi$.
\begin{sol}
 You have $z=\abs{z}(\cos
\theta +i\sin \theta) $ and $w=\abs{w}(\cos
\phi +i\sin \phi)$. Then when you multiply these, you get
\begin{eqnarray*}
&&\abs{z}\abs{w}(\cos \theta +i\sin
\theta) (\cos \phi +i\sin \phi) \\
&=&\abs{z}\abs{w}(\cos \theta \cos
\phi -\sin \theta \sin \phi +i(\cos \theta \sin \phi +\cos \phi \sin
\theta)) \\
&=&\abs{z}\abs{w}(\cos (\theta
+\phi) +i\sin (\theta +\phi))
\end{eqnarray*}
\end{sol}
\end{ex}


\section{Appendix}

% ===
\emph{Complete the square:}
If $p(\bm x) \propto \exp(-\frac12 \bm x\!^\top \!\bm{{\color{OrangeRed} A}x} + \bm x\!^\top \begingroup \color{OrangeRed} \bm b \endgroup)$,
then $p(\bm x) = \Gauss{\bm x \mid \bm A^{-1} \bm b, \bm A^{-1}}$

% ===
\emph{Constrained optimisation:}
\\
\textit{primal}: \enspace $\min_{\bm x} f(\bm x)$ \enspace s.t. \enspace $g_i(\bm x) = 0$; \enspace $h_j(\bm x) \leq 0$
\\
\textbf{Lagrangian:} \enspace with each $\alpha_j \geq 0$\\
\enspace $\mathcal L(\bm x,\lambda,\alpha) = f(\bm x) + \sum_i \lambda_i g_i(\bm x) + \sum_j \alpha_j h_j(\bm x)$
\\
Solve: \: $\pderiv{\mathcal L}{\bm x} = 0$; \: $g_i(\bm x) = 0$; \: $\alpha_j \geq 0$; \: $h_j(\bm x) \leq 0$
\\
If \textbf{Slater's cond.} holds, $\exists \bm x : g_i(\bm x) = 0, h_j(\bm x) {\color{red}\,<\,} 0$, then we can solve the \textit{dual} instead:\\
\enskip $\max_{\bm \lambda, \bm \alpha} \brace{ \min_{\bm x} \mathcal L(\bm x, \bm \lambda, \bm \alpha) }$ \: s.t. \: $\alpha_j \geq 0$\\
Solve: \: $\pderiv{\mathcal L}{\bm x} = 0$; \: $\pderiv{\mathcal L}{\bm \lambda} = 0$; \: $\alpha_j h_j(\bm x) {\color{red}\,=\,} 0$; \: $\alpha_j \geq 0$

\iffalse
    \emph{Lagrange Multipliers: \color{red} OLD VERSION}
    \\
    Problem $\mathcal P : \begin{cases}
        \min f(\bm x),      & \bm x\in\mathbb R^d \\
        \text{s.t. } g_i(\bm x)=0,      & i\leq m \\
        \phantom{\text{s.t. }} h_j(\bm x) \leq 0,       & j\leq n
    \end{cases}$
    \\
    Lagrangian: $\mathcal L(\bm x,\lambda,\alpha) = f(\bm x) + \sum_{i\leq m} \lambda_i g_i(\bm x) + \sum_{j\leq n} \alpha_j h_j(\bm x)$ with each $\alpha_j \geq 0$.
    \\
    Solution must satisfy
    $\pderiv{\mathcal L}{\bm x} = 0$ and $\pderiv{\mathcal L}{\lambda} = 0$,
    $\alpha_j h_j(\bm x) = 0$, and
    $\alpha_j \geq 0, \forall j\leq n$.
\fi

\iffalse
    \emph{Euler-Lagrange:}
    Find extrema of functional $\mathcal F[f] = \int G(x, f(x), f’(x)) \diff x$,
    thus $\pderiv{\mathcal F}{f} \overset!= 0$.
    \\
    If $G$ is twice diff'able, then
    \\
    $\pderiv{\mathcal F}{f} = \pderiv{G}{f(x)} - \deriv{}{x} \paren*{ \pderiv{G}{f'(x)} } \overset{(\ast)}= \pderiv{G}{f(x)}$.
    \\
    $(\ast)$ : when $G$ does not depend on $f'$.
\fi

% ===
\emph{Metrics:}
$\mathit{acc} = \frac{\mathrm{TP} + \mathrm{TN}}{n}$
$\mathit{prec} = \frac{\mathrm{TP}}{\mathrm{TP} + \mathrm{FP}}$
$\mathit{FPR} = \frac{\mathrm{FP}}{\mathrm{FP} + \mathrm{TN}}$
$\mathit{Recall/TPR} = \frac{\mathrm{TP}}{\mathrm{TP} + \mathrm{FN}}$
$\mathit{balanced\:acc} = \frac1n \sum_i \mathit{TPR}_i$
$\mathit{F1\!-\!score} = \frac{2 \mathrm{TP}}{2 \mathrm{TP} + \mathrm{FP} + \mathrm{FN}}$
$\mathit{ROC} = \mathit{FPR} / \mathit{TPR}$

% ===
\emph{Conditional Gaussians:}\\
$P_{X,Y} = \begin{bmatrix} \bm X \\ \bm Y \end{bmatrix} \sim
\Gauss*[~]{
    \begin{bmatrix} \bm\mu_X \\ \bm\mu_Y \end{bmatrix} ,
    \begin{bmatrix} \bm\Sigma_{XX} & \bm\Sigma_{XY} \\ \bm\Sigma_{YX} & \bm\Sigma_{YY} \end{bmatrix}
}$,\enspace $\bm\Sigma_{ij}$ p.s.d.

\iffalse
    $\implies X\vert Y \sim \Gauss{\tilde{\bm\mu}, \tilde{\bm\Sigma}}$,\enskip where
    $\tilde{\bm\mu} = \bm\mu_X + \bm\Sigma_{XY} \bm\Sigma_{YY}^{-1} (Y - \bm\mu_Y)$,\enskip
    $\tilde{\bm\Sigma} = \bm\Sigma_{XX} - \bm\Sigma_{XY} \bm\Sigma_{YY}^{-1} \bm\Sigma_{YX})$
\fi

$\implies Y\vert X \sim \Gauss{\tilde{\bm\mu}, \tilde{\bm\Sigma}}$,\enskip where
$\tilde{\bm\mu} = \bm\mu_Y + \bm\Sigma_{YX} \bm\Sigma_{XX}^{-1} (X - \bm\mu_X)$,\enskip
$\tilde{\bm\Sigma} = \bm\Sigma_{YY} - \bm\Sigma_{YX} \bm\Sigma_{XX}^{-1} \bm\Sigma_{XY})$

% ===