\chapter{Multi-layer Multiple Kernel Learning}
\label{chap_mlmkl}
In this chapter we  discuss the Multi-layer Multiple Kernel Learning (ML-MKL\nomenclature{ML-MKL}{Multi-layer Multiple Kernel Learning}) framework developed using an unsupervised MKL algorithm. The organization of this chapter is as follows;  section \ref{sec_mkl} gives  the discussion of  traditional MKL algorithm  in a supervised learning settings, in section \ref{sec_umkl} the unsupervised MKL formulation is introduced, in section \ref{sec_rw} the related works in ML-MKL domain is discussed, in section \ref{sec_mlmkl} the proposed ML-MKL algorithm is discussed along with the experimental results and section \ref{sec_conc} concludes this chapter.

\section{Multiple Kernel Learning}
\label{sec_mkl}
Multiple Kernel Learning(MKL\nomenclature{MKL}{Multiple Kernel Learning}) aims at learning a convex combination of a set of predefined base kernels for choosing an optimum kernel(\cite{mkl} et al.). The primary aim of MKL algorithm is to automate the process of choosing the optimum kernel  for the learning task.

Typically multiple kernel learning is formulated in a supervised learning settings. Suppose we are given $n$ datapoints $\mathcal{D} = \{(x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)\}$, where $x_i \in \mathbb{R}^d$ is the $i^{th}$ input data vector and $y_i$ is the label of the class to which $x_i$ belongs to. Then the MKL learning task is formulated as the following optimization problem.
\begin{equation}
\min_{k \in \mathcal{K}} \min_{f \in \mathcal{H}_k} \lambda \norm{f}^2_{\mathcal{H}_k} + \sum_{i=1}^n l(y_i, f(x_i))
\end{equation}
where $l(\cdot)$ denotes the loss function(commonly used loss function is the hinge loss, defined as $l(t) = \max(0, 1-t)$), $\mathcal{H}_k$ is the RKHS corresponding to the kernel $k$, $\mathcal{K}$ denotes the optimization domain of the candidate kernels, and $\lambda > 0$ is the regularization parameter.

The optimizaion domain $\mathcal{K}$ is the convex combination of a set of predefined base kernels, defined as follows
\[ \mathcal{K} = \Bigg\{ k( \cdot \textrm{ , } \cdot ) = \sum_{t=1}^m \mu_t k_t(\cdot \textrm{ , } \cdot) \textrm{ : } \sum_{t=1}^m \mu_t = 1, \mu_t \geq 0 \Bigg\} \]
where $k_t$ is the $t^{th}$ base kernel and $\mu_t$ is the weight associated to the $t^{th}$ base kernel. The decision function $f(x)$ can be computed as a linear combination of kernel evaluations on all training samples;
\[ f(x) = \sum_{i=1}^n \alpha_i k(x_i, x) \]
where $\alpha_i$'s are the coefficients. As per the definition of kernel $k$ in MKL, the decision function of the conventional MKL is expressed as
\begin{equation*}
\begin{aligned}
f(x) &= \sum_{i=1}^n \alpha_i \sum_{t=1}^m \mu_t k_t(x_i, x) \\
&= \sum_{i=1}^n \sum_{t=1}^m \alpha_i \mu_t k_t(x_i, x)
\end{aligned}
\end{equation*}
In this basic formulation, the optimization task has to identify both the optimal kernel $k$ from the domain $\mathcal{K}$ and the optimal decision function $f$ from the RKHS $\mathcal{H}_k$ simultaneously. In order to eliminate this problem \cite{corinna} et al. proposed a two stage kernel learning algorithm which separates the kernel learning task from the decision function learning.

\section{Unsupervised MKL}
\label{sec_umkl}
In the KPCA based feature extraction stages, we were using only one kernel for the task. With the intuition that by using multiple kernels at each layer we would get more similarity information, we computed a convex combination of multiple kernels following the work in \cite{zhuang} et al. Since we are using kernels for unsupervised feature extraction, traditional MKLs following supervised paradigm cannot be used here.

The goal of an unsupervised multiple kernel learning task is to find an optimal linear combination of the $m$ kernel functions as, i.e, $k^*(\cdot \textrm{ , } \cdot) \in \mathcal{K}$. %where $K$ is defined as follows:
%\[ K = \Bigg\{ k( \cdot \textrm{ , } \cdot ) = \sum_{t=1}^m \mu_t k_t(\cdot \textrm{ , } \cdot) \textrm{ : } \sum_{t=1}^m \mu_t = 1, \mu_t \geq 0 \Bigg\} \]
%here $k_t$'s are the base kernels.
In order to determine the optimality of a linear combination of kernels, we used the following quality criteria\cite{zhuang} et al.:

\begin{itemize}
\item A  good kernel should enable each training instances to be well reconstructed from the localized bases weighted by the kernel values. Formulating this requirment mathematically, for each $x_i$ we expect the optimal kernel should minimize the approximation error $\norm{x_i-\sum_j k_{ij}x_j}^2$, where $k_{ij} = k(x_i, x_j)$.
\item A good kernel should induce kernel values that are coincided with the local geometry of the training data. This is equivalent to finding the optimal kernel that minimizes the distortion over all trainig data, computed as $\sum_{i,j}k_{ij} \norm{x_i-x_j}^2 $.
\end{itemize}

In addition to this, the locality preserving principle can be exploited  by using a set of local bases for each $x_i \in X$ denoted as $B_i$. By fixing the size of the local bases to some constant $N_B$,  the optimization problem of unsupervised MKL can be formulated as follows.
\[ \min_{k \in \mathcal{K}} \frac{1}{2}\sum_{i=1}^n \norm{x_i - \sum_{x_j \in B_i} k_{ij}x_j}^2 + \gamma* \sum_{i=1}^n \sum_{x_j \in B_i} k_{ij} \norm{x_i-x_j}^2 \]

where $\gamma$ is a tuning parameter, which controls the tradeoff between the coding error and the locality distortion. Converting to matrix notations the above problem becomes
\begin{equation}
  \min_{\mu \in \Delta,D} \frac{1}{2} \norm{X(I-K \circ D)}_F^2 + \gamma* \textrm{ tr }K\circ D\circ M(11^T)
  \label{obj_mat}
\end{equation}
%\begin{equation*}
 % \begin{aligned}
\[ \textrm{subject to } D \in \{0,1\}^{n \times n} \]
\[  \norm{d_i}_1 = N_B, i=1,2,\ldots,n \]
\[ \Delta = \Big\{\mu : \mu^T\textbf{1} = 1, \mu \geq 0 \Big\} \textrm{ and } \]
\[ [K]_{i,j} = \sum_{t=1}^m \mu_t k^t(x_i, x_j), 1\leq i,j \leq n \]
  %\end{aligned}
%\end{equation*}

The matrix $D \in \{0,1\}^{n \times n}$ contains information about local bases of each $x_i$ as a column vector. In particular, each column vector $d_i \in \{0, 1\}^n $ in $D$ has a 1 at those points $j$ where, $x_j \in B_i$ and zero elsewhere(or $B_i = \{ x_j : d_j \neq 0 \} $). The matrix M is defined as
\[ [M]_{ij} = x_i^Tx_i + x_j^Tx_j - 2x_i^Tx_j \]

In equation \ref{obj_mat} the notation `$\circ$' denotes elementwise multiplication of two matrices, $\norm{\cdot}_F^2$ denotes the Frobenius norm of a matrix and `tr' denotes the trace of a matrix.

In their implementation(\cite{zhuang}) Zhuang et.al solved the optimization problem in two stages in an alternating fashion, first by solving for $\mu$ with a fixed $D$ (using convex optimization) and then solving for $D$ by fixing $\mu$ (using a greedy mixed integer programming formulation). Since we are using a many layer architecture, the alternating optimization strategy is too costly; so we chose to do the optimization across $\mu$ only by choosing $D$ beforehand. Specifically, the matrix $D$ is computed beforehand by taking $k$ nearest neighbours of $x_i$ from the training set and putting a one in those positions for $d_i$. Rest of the positions are filled with zeros. The resulting optimization problem will be
\begin{equation}
  \begin{aligned}
  \min_{\mu \in \Delta} \frac{1}{2} \norm{X(I-K \circ D)}_F^2 + \gamma* \textrm{ tr }K\circ D\circ M(11^T)
  \end{aligned}
\end{equation}
\[ \textrm{subject to } D \in \{0,1\}^{n \times n}, \norm{d_i}_1 = N_B, i=1,2,\ldots,n \]
\[ \Delta = \Big\{\mu : \mu^T\textbf{1} = 1, \mu \geq 0 \Big\} \]

The objective function can be formulated as a convex quadratic programming problem w.r.t to kernel weights $\mu$ as shown below(derivation of the objective function $J(\mu)$ is shown in Appendix \ref{derivation1}).
\begin{equation}
J(\mu) = \mu^T \Bigg( \sum_{t=1}^m \sum_{i=1}^n k_{t,i}k_{t,i}^T \circ d_i d_i^T \circ P \Bigg)^T \mu + z^T \mu
\end{equation}
where $[z]_t = \sum_{i=1}^n (2 \gamma v_i \circ d_i - 2 p_i \circ d_i)^T \mathit{k}_{t,i} $, $P = X^TX$, and $\mathit{k}_{t,i} = \Big[ k^t(x_i, x_1), \ldots, k^t(x_i, x_n) \Big]^T $ is the $i^{th}$ column of the $t^{th}$ kernel matrix. $p$ and $v$ are columns of $P$ and $M$ corresponding to $x_i$ respectively.

\section{Related Works}
\label{sec_rw}

\cite{2l_mkl} et al. explored the idea of ML-MKL, in which at each layer they took a non-linear combination of kernels in the previous layer, forming a layered structure. They called such a model as deep multiple kernel learning framework. The architecture of their model is shown in figure \ref{fig_2l_mkl}. The kernel function obtained at layer $l$ is having the following domain.
\[ \mathcal{K}^{(l)} = \bigg\{ k^{(l)}(\cdot, \cdot) = g^{(l)}\Big( \big[k_1^{(l-1)},\ldots, k_m^{(l)}(\cdot, \cdot) \big] \Big)  \bigg\} \]

\begin{figure*}
  \centering
  \captionsetup{justification=centering,margin=0.1cm}
  \includegraphics[width=1.1\textwidth,height=8.5cm]{figures/2lmkl}
  \caption{Architecture of ML-MKL model proposed by \cite{2l_mkl} et al.}
  \label{fig_2l_mkl}
\end{figure*}

where $g^{(l)}$ is a function to combine multiple $(l-1)$ level kernels, which must ensure the resulting combination is a valid kernel. Though this combination can be applied for any number of layers, optimization problem is difficult to solve beyond two layers. Hence \cite{2l_mkl} et al. considered only 2-layer MKL in their empirical study.

In the 2-layer MKL, \cite{2l_mkl} et al. defined the kernel domain by using an RBF kernel for the combination function.
\[ \mathcal{K}^{(2)} = \bigg\{ k^{(2)}(x_i, x_j;\bm{\mu}) = exp\Big( \sum_{t=1}^m \mu_t k_t^{(1)}(x_i, x_j) \Big)  \bigg\}  \colon \bm{\mu} \in \mathbb{R}_+^m \]
$\mu_t$ is the weight of the $t^{(th)}$ antecedent layer kernel $k_t^{(1)}$(superscript (1) is used here because the antecedent layer is layer 1 in a 2-layer architecture). In order to prevent the kernel weight being too large, the kernel weights are also introduced into the optimization objective as a regularization term. Then the optimization objective becomes
\begin{equation}
\min_{k \in \mathcal{K}} \min_{f \in \mathcal{H}_k} \norm{f}^2_{\mathcal{H}_k} + \mathcal{C} \sum_{i=1}^n l(y_i, f(x_i)) + \sum_{t=1}^m \mu_t
\end{equation}
solving the Lagrangian, we will get the dual objective function as
\[ \min_{\bm{\mu}} \max_{\bm{\alpha}} \sum_{i=1}^n \alpha_i - \frac{1}{2}\sum_{i,j=1}^n \alpha_i \alpha_j y_i y_j k^{(2)}(x_i, x_j;\bm{\mu}) + \sum_{t=1}^m \mu_t \]
\[ \textrm{s.t } 0\leq \alpha_i\leq \mathcal{C}, \sum_{i=1}^n \alpha_i y_i = 0, \mu_t \geq 0, t=1, \ldots, m \]
where $\alpha = [\alpha_1, \alpha_2, \ldots, \alpha_n]^T$ is a vector of dual variables and $\mu = [\mu_1, \mu_2, \ldots, \mu_m]^T$. The final decision function of the 2-layer MKL is given as
\[f(x;\bm{\alpha}, \bm{\mu}) = \sum_{i=1}^n \alpha_i y_i k^{(2)}(x_i, x;\bm{\mu}) + b \]
where $b$ is the bias term. Rewriting the optimization objective in terms of $\alpha$ and $\mu$, we have
\[ \mathcal{J}(\alpha, \mu) = \frac{1}{2} \sum_{i,j=1}^n \alpha_i \alpha_j y_i y_j k^{(2)}(x_i, x_j;\bm{\mu}) - \sum_{i=1}^n \alpha_i - \sum_{t=1}^m \mu_t \]
The optimization problem is solved in two stages
\begin{itemize}
\item by fixing $\mu$ and solve for $\alpha$.
\item by fixing $\alpha$ and solve for $\mu$.
\end{itemize}
Since $k^{(2)}$ is positive definite $\mathcal{J}(\alpha, \mu)$ is convex over $\alpha$, thus an SVM solver can be used to solve the optimization over $\alpha$. The optimization over $\mu$ is solved using gradient ascent. In order to address the challenge of choosing the optimal set of base kernels, they proposed to choose base kernels iteratively inside $k^{(2)}$.

\cite{deep_mkl} et al. used backpropagation algorithm for designing the ML-MKL framework. The architecture of the model studied by them is shown in figure \ref{fig_deep_mkl}. They defined the kernel domain at level $l$ as

\begin{figure*}
  \centering
  \captionsetup{justification=centering,margin=0.1cm}
  \includegraphics[width=1\textwidth,height=6cm]{figures/back_prop_mlmkl}
  \caption{Architecture of ML-MKL model proposed by \cite{deep_mkl} et al. The kernels in each layer are linearly combined and the resulting Gram matrix is passed to the next layer as input.}
  \label{fig_deep_mkl}
\end{figure*}

\[ \mathcal{K}^{(l)} = \big\{ K^{(l)}(K^{(l-1)}; \mu^{(l)}) = \sum_{t=1}^m \mu_t^{(l)} k_t^{(l)}(K^{(l-1)})  \big\} \]
\[ \textrm{s.t } \mu_t^{(l)} \geq 0, l = 1, \ldots, L \]
where $k_t^{(l)}$ is the $k^{th}$ base kernel at layer $l$ and $\mu_t^{(l)}$ denotes the weight of the $k^{th}$ base kernel at layer $l$. $L$ is the total number of layers. The feature extracted in the antecedent layer are combined linearly and passed to the next layer as input. The kernel weights are obtained by minimizing the mean squre error(denotes as $\mathcal{E}$) of the predicted outputs.
\[ \mathcal{E} = \frac{1}{2n} \sum_{i=1}^n \norm{f(x_i) - y_i}^2 \]
where $f(x_i)$ is the value of the decision function for the input $x_i$, which is computed as
\[ f(x) = \sum_{i=1}^n \alpha_i y_i K^{(l)}(K^{(l-1)}; \mu^{(l)}) + b \]
where $b$ is the bias term. The weights $\mu^{(l)}$ in each layer are obtained using gradient descent algoritm.
\[ \mu^{(l)} := \mu^{(l)} + \eta \nabla \mathcal{E}  \textrm{ } l = 1, \ldots, L \]
\[ \nabla \mathcal{E} = \Bigg\{ \frac{\partial \mathcal{E}}{\partial \mu_1^{(1)}}, \cdots, \frac{\partial \mathcal{E}}{\partial \mu_m^{(1)}}; \cdots; \frac{\partial \mathcal{E}}{\partial \mu_1^{(L)}}, \cdots, \frac{\partial \mathcal{E}}{\partial \mu_m^{(L)}}  \Bigg\}  \]
Here $\eta$ is the learning rate parameter. The error obtained in the final layer are propagated back to all layers using backpropagation algorithm to update the kernel weight parameters $\mu^{(l)}$ in each layer.

\section{Multi-layer Multiple Kernel Learning}
\label{sec_mlmkl}
The architectue of the proposed ML-MKL framework is shown in figure \ref{fig_mlmkl}. It consists of many layers and in each layer the kernel PCA based feature extraction is performed using the combination of a set of predefined kernels. The dimensionality of the features thus obtained are reduced by using supervised feature selection techniques. The final output can be given to any classifier. Algorithm \ref{algo1} summarizes the proposed ML-MKL algorithm.

\begin{algorithm}
\caption{ML-MKL Algorithm}
\textbf{Input}: data X, true labels y, no. of layers L, base kernels for each layer $K_{base}^{(l)} = \{k_1^{(l)}, k_2^{(l)}, k_m^{(l)}\}$, $N_B$, $\gamma$\;
\textbf{Output}: kernel weights $\mu^l$ for each layer, predicted labels\;
1.Initialize $[M]_{ij} = x_i^Tx_i + x_j^Tx_j - 2x_i^Tx_j$, $\bm{D} = d_1, d_2, \ldots, d_n$ as row vectors, where $d_i = \{1 \textrm{ if } x_j \in B_i \textrm{ else } 0 \forall x_j \in X \}$, $\mu = \frac{1}{m}$, $\bm{P} = X^TX$\;
2.\For{each layer l}{
    a. $\bm{W} = \sum_{t=1}^m \sum_{i=1}^n k_{t,i}^{(l)}k_{t,i}^{(l)^{T}} \circ d_i d_i^T \circ P$\\
    b. $\bm{[z]}_t^l = \sum_{i=1}^n (2 \gamma v_i \circ d_i - 2 p_i \circ d_i)^T \mathit{k}_{t,i}^{(l)}$\\
    c. $\bm{\mu}^{*^{l}} = \mu^{l^{T}}W\mu^l + z^{l^{T}}\mu^l$\\
    d. $\bm{K}_{new} = \sum_{t=1}^m \mu_t^l * K_t^{(l)}$\\
    e. extract principal components with $\bm{K}_{new}$\\
    f. select most informative features for layer $l$($X_{new}$)\\
    g. $\bm{P} = X_{new} ^T X_{new}$
}
3.Give the final set of features to any classifier\;
\label{algo1}
\end{algorithm}

\begin{figure*}
  \centering
  \captionsetup{justification=centering,margin=0.1cm}
  \includegraphics[width=1.0\textwidth,height=5.5cm]{figures/mlmkm}
  \caption{An ML-MKL with L layers of transformations. Each layer consists of many kernels for feature  extraction using kernel PCA and a supervised feature selection module.}
  \label{fig_mlmkl}
\end{figure*}

The experimental setup used here was the same as described in \autoref{chap_mkm}. Table \ref{tab_mlmkl} lists the results obtained with the proposed ML-MKL framework. The classifier used was  SVM with arc-cosine kernel. For \textit{mnist-back-rand} dataset, the best result was obtained with a model consists of 4 layers and in each layer 7 kernels were used. In particular, each layer consisted of a mixture of one arc-cosine kernel and 6 gaussian kernels. For \textit{mnist-back-image} dataset, the best ML-MKL model obtained had 2 layers and 5 kernels in each layer. In each layer a mixture of one arc-cosine kernel and 4 polynomial kernels were used. In the case of \textit{mnist-rot-back-image} dataset the best result was fetched by a model having only one layer with 4 arc-cosine kernels in it. For \textit{rectangles-image} dataset the best result was shown by a one layer ML-MKL model with one arc-cosine kernel and 6 Gaussian kernels.

\renewcommand{\arraystretch}{2.1}
\begin{table*}
\centering
\begin{tabular}{|c|c|c|c|c|c|c|c|}
  \hline
  \multirow{2}{*}{\textbf{Dataset}} & \multicolumn{7}{ |c| }{\textbf{Loss in Percentage}} \\
  \cline{2-8}
  &$\textrm{SVM}_{\textrm{RBF}}$ & $\textrm{SVM}_{\textrm{Poly}}$ & NNet & DBN-3 & SAA-3 & DBN-1 & \textbf{ML-MKL}\\
  \hline
  \textit{back-rand} & 14.58 & 16.62 & 20.04 & \textbf{6.73} & 11.28 & 9.80 & 8.43$\pm 0.088$\\
  \hline
  \textit{back-image} & 22.61 & 24.01 & 27.41 & 16.31 & 23.00 & \textbf{16.15} & 20.92$\pm 0.092$\\
  \hline
  \textit{rot-back-image} & 55.18 & 56.41 & 62.16 & \textbf{47.39} & 51.93 & 52.21 & 51.21$\pm 0.811$\\
  \hline
  \textit{rect-image} & 24.04 & 24.05 & 33.20 & 23.69 & 24.05 & \textbf{22.50} & 22.88$\pm 0.124$\\
  \hline
\end{tabular}
\caption{Experimental Results of ML-MKL.}
\label{tab_mlmkl}
\end{table*}
\renewcommand{\arraystretch}{1}

Figures \ref{fig_mbr_layers} and \ref{fig_mbi_layers} illustrates the variation in classifier performance on \textit{mnist-back-rand} and \textit{mnist-back-image} datasets respectively when layers were added iteratively to the ML-MKL model. The value shown for each layer was the best error rate obtained after tuning the kernel parameters. The parameters were chosen greedily for each layer (with the expectation that subsequent layers would learn more valuable features from the current one) and no fine-tuning was performed with respect to the entire architecture.

\begin{figure}
  \centering
  \captionsetup{justification=centering,margin=0.1cm}
  \includegraphics[scale=0.8]{figures/mlmkl_rand}
  \caption{Change in classifier performance on \textit{mnist-back-rand} dataset when adding layers iteratively.}
  \label{fig_mbr_layers}
\end{figure}

\begin{figure}
  \centering
  \captionsetup{justification=centering,margin=0.1cm}
  \includegraphics[scale=0.8]{figures/mlmkl_image}
  \caption{Change in classifier performance on \textit{mnist-back-image} dataset when adding layers iteratively.}
  \label{fig_mbi_layers}
\end{figure}

Figure \ref{tsne_rand_mlmkl} shows the tSNE embedding of the features learned by ML-MKL algorithm. The visualization indicates that classes are well separated.

\begin{sidewaysfigure}
  \centering
  \captionsetup{justification=centering,margin=0.1cm}
  \includegraphics[scale=0.45]{figures/tsne_rand_mlmklbest}
  \caption{tSNE embedding of features obtained by ML-MKL algorithm for the \textit{mnist-back-rand} dataset.}
  \label{tsne_rand_mlmkl}
\end{sidewaysfigure}

Tables \ref{back_rand_kw} and \ref{back_image_kw} shows the kernel weights of each kernel in the mixture at every layer for \textit{mnist-back-rand} and \textit{mnist-back-image} datasets respectively. In both cases $k_1$ is an acr-cosine kernel, and the remaining are Gaussian kernels for \textit{mnist-back-rand} dataset and polynomial kernel for textit{mnist-back-image} dataset. The results in the table indicates that,  the contribution of individual kernels was highly varying in each layer (no single kernel had complete dominance over all layers in the feature learning process).

\renewcommand{\arraystretch}{2.3}
\begin{table*}
\centering
\begin{tabular}{|c|c|c|c|c|c|c|c|}
  \hline
  \multirow{2}{*}{\textbf{Layers}} & \multicolumn{7}{ |c| }{\textbf{Kernel Weights}} \\
  \cline{2-8}
  & $k_1$ & $k_2$ & $k_3$ & $k_4$ & $k_5$ & $k_6$ & $k_7$ \\
  \hline
  \textit{Layer 1} & 0.2007 & 0.1331 & 0.1331 & 0.1332 & 0.1332 & 0.1333 & 0.1333\\
  \hline
  \textit{Layer 2} & 0.2711 & 0.1160 & 0.1181 & 0.1203 & 0.1225 & 0.1248 & 0.1271\\
  \hline
  \textit{Layer 3} & 0.1764 & 0.0999 & 0.1125 & 0.1266 & 0.1426 & 0.1607 & 0.1811\\
  \hline
  \textit{Layer 4} & 0.0598 & 0.0524 & 0.0747 & 0.1071 & 0.1547 & 0.2245 & 0.3269\\
  \hline
  \textit{Layer 5} & 0.0514 & 0.0493 & 0.0726 & 0.1069 & 0.1569 & 0.2295 & 0.3334\\
  \hline
\end{tabular}
\caption{Kernel weights in each layer for the \textit{mnist-back-rand} dataset}
\label{back_rand_kw}
\end{table*}
\renewcommand{\arraystretch}{1}

\renewcommand{\arraystretch}{2.3}
\begin{table}
\centering
\begin{tabular}{|c|c|c|c|c|c|}
  \hline
  \multirow{2}{*}{\textbf{Layers}} & \multicolumn{5}{ |c| }{\textbf{Kernel Weights}} \\
  \cline{2-6}
  & $k_1$ & $k_2$ & $k_3$ & $k_4$ & $k_5$\\
  \hline
  \textit{Layer 1} & 0.2445 & 0.1887 & 0.1888 & 0.1889 & 0.1891\\
  \hline
  \textit{Layer 2} & 0.3421 & 0.1603 & 0.1630 & 0.1659 & 0.1688\\
  \hline
  \textit{Layer 3} & 0.2035 & 0.1615 & 0.1843 & 0.2104 & 0.2403\\
  \hline
  \textit{Layer 4} & 0.0843 & 0.1409 & 0.1877 & 0.2508 & 0.3362\\
  \hline
\end{tabular}
\caption{Kernel weights in each layer for the \textit{mnist-back-image} dataset}
\label{back_image_kw}
\end{table}
\renewcommand{\arraystretch}{1}

In order to evaluate the contribution of each kernel, exploratory analysis was carried out to monitor the performance of individual kernels in each layer. The individual kernels performance were compared with the combined kernel's performance. Tables \ref{back_rand_exp} and \ref{back_image_exp} summarizes the results of this exploratory analysis on \textit{mnist-back-rand} and \textit{mnist-back-image} datasets respectively (here $K_{conv}$ is the result of combined kernel). The results clearly indicates that the combination was not always improving the performance in all layers. However, the best result in both datasets were obtained by a combined kernel.

\renewcommand{\arraystretch}{2.1}
\begin{table*}
\centering
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
  \hline
  \multirow{2}{*}{\textbf{Layers}} & \multicolumn{8}{ |c| }{\textbf{Loss in Percentage}} \\
  \cline{2-9}
  & $k_1$ & $k_2$ & $k_3$ & $k_4$ & $k_5$ & $k_6$ & $k_7$ & $K_{conv}$\\
  \hline
  \textit{Layer 1} & 11.46 & 9.92 & 10.06 & 10.04 & 10.1 & 9.87 & 10.06 & 11.62\\
  \hline
  \textit{Layer 2} & 10.36 & 9.87 & 9.90 & 9.89 & 9.92 & 9.82 & 9.81 & 9.52\\
  \hline
  \textit{Layer 3} & 9.46 & 11.07 & 10.33 & 9.85 & 9.70 & 9.56 & 9.01 & 9.18\\
  \hline
  \textit{Layer 4} & 9.26 & 9.06 & 8.95 & 8.96 & 8.93 & 8.72 & 9.00 & \textbf{8.72}\\
  \hline
  \textit{Layer 5} & 9.18 & 9.07 & 9.02 & 9.20 & 9.22 & 9.36 & 9.47 & 9.15\\
  \hline
\end{tabular}
\caption{Individual kernels performance evaluation for \textit{mnist-back-rand} dataset}
\label{back_rand_exp}
\end{table*}
\renewcommand{\arraystretch}{1}

\renewcommand{\arraystretch}{2.1}
\begin{table}
\centering
\begin{tabular}{|c|c|c|c|c|c|c|}
  \hline
  \multirow{2}{*}{\textbf{Layers}} & \multicolumn{6}{ |c| }{\textbf{Loss in Percentage}} \\
  \cline{2-7}
  & $k_1$ & $k_2$ & $k_3$ & $k_4$ & $k_5$ & $K_{conv}$\\
  \hline
  \textit{Layer 1} & 20.91 & 21.82 & 21.82 & 21.83 & 21.77 & 21.15\\
  \hline
  \textit{Layer 2} & 21.27 & 21.00 & 21.09 & 21.04 & 21.04 & \textbf{20.83}\\
  \hline
  \textit{Layer 3} & 20.95 & 21.18 & 21.02 & 20.99 & 21.05 & 21.03\\
  \hline
  \textit{Layer 4} & 21.06 & 21.36 & 21.42 & 21.64 & 21.66 & 21.51\\
  \hline
\end{tabular}
\caption{Individual kernels performance evaluation for \textit{mnist-back-image} dataset}
\label{back_image_exp}
\end{table}
\renewcommand{\arraystretch}{1}

\section{Conclusion}
\label{sec_conc}
In this chapter we explored the concept of multiple kernel learning in MKMs. A linear combination of multiple kernels formulated purely from unlabelled data is used in each layer of MKMs. The learning process of the proposed ML-MKL algorithm employs a greedy layerwise training for each layer. Empirical results indicates that using (unsupervised) MKL  in MKMs improves the classifier performance. In our experimental analysis the classification accuracy of ML-MKL models was better than learning machines with shallow architectures and was comparable with existing deep architectures.

Exploratory analysis performed on the features learned by the ML-MKL model reveals much more interesting facts about the model. The contribution of kernels in each layer is measured in terms of individual kernel performance and the weights assigned to that kernel. However this information is insufficient to determine the optimal structural complexity required for modelling a problem.


\documentclass[../main.tex]{subfiles}

\begin{document}

\setcounter{chapter}{3}

\chapter{Schemes and algebraic spaces I}

In the previous lecture we saw the notion of a geometric context, and that this notion gives rise to manifolds and geometric spaces. We will now construct a context specific to algebraic geometry in which the manifolds are schemes and geometric spaces are algebraic spaces.

In order to understand the conrtuction of algebraic contexts we commence by recalling some elementary facts about (affine) algebraic varieties, .... Naively an algebraic variety is a set of solutions to a system of plynomial equations. ... An algebraic variety is determined a finite familyof polynomials $P_1, \cdot P_r \in \mathbb C[x_1, \cdots, x_n]$. X is then a solution a dset of solutions of the system $\{P_i(x) = 0\}_{1 \le i \le r}$, in ... terms we have

\[
X = \{ x \in \mathbb C^n ~|~ P_i(x) = 0 ~ \forall i\}
\]
We also then have the question of defining the ``algebraic functions'' on $X$. ... ring of algebraic functions on $X$ is witten as

\[
\mathcal O(X) = \mathcal C[x_1, \cdots x_n] / (P_1, \cdots, P_r)
\]

An algebraic variety $X$ corresponds to the ring $C[x_1, \cdots x_n] / (P_1, \cdots, P_r)$. This ring is a commutative $\mathbb C$-algebra of finite type (i.e. with a finite number of generators). Conversely, if $A$ is an commutative $C$-algebra of finite type, we can write

\[
A \simeq \mathbb C[y_1, \cdots, y_m]/(Q_1, \cdots, Q_s)
\]

and construct an algebraic variety in $\mathbb C^m$ with the equation $Q_1(y) = \cdots = Q_s(y) = 0$.

there's a correspondence between affines algebraic varies in $\mathbb C$ (i.e. subvarieties of $\mathbb C^n$ for some $n$), and commutative $\mathbb C$-algebras of finite type. The starting point for the theory of schemes and considering commutative rings A (not necessarily commutative $\mathbb C$-algebras of finite type) is the considering of the correspondenc between a geometric object $X$ and $A$ a ring of algebraic functions on $X$. The geometric objects associated with the ring are affine schemes, and a scheme is by definition a collection of affine schemes (much like a topological property is a collection of opens in $\mathbb R^n$). We have the following two principles

\begin{prcp*}
    Affine schemes are in correspondence with commutative rings
\end{prcp*}
\begin{prcp*}
    Schemes are obtained from collections of affines schemes
\end{prcp*}

From the combination two principles we obtain the following principles

\begin{prcp*}
    Schemes are obtained from collections of commutative rings
\end{prcp*}

\section{Geometric contexts}

\end{document}


\chapter{Explicit One-Step Methods and Convergence}

\section{Introduction}
\begin{example}[Euler's method]
  We begin this section with the method which serves as prototype for
  a whole class of schemes which solves an IVP or rather the Volterra
  integral equation numerically. Here, as always for problems with
  infinite dimensional solution spaces, numerical solution refers to
  finding an approximation by applying a discretization method, and
  studying the error of this method.

  Consider the following problem: given an IVP of the form of \slideref{Definition}{IVP},
  calculate the value $u(T)$ at a later point in time $T$.

  To this end, we note first of all that for an IVP at the initial
  point $0$, not only the function value $u(0) = u_0$ is known,
  but also the derivative $u'(0) = f(0, u_0)$. Thus we are capable
  to replace the solution $u(t)$ in blue by a straight line $y(t)$ in
  red, which we can see on the left of Figure~\ref{fig:explicit:Euler}.
  \begin{figure}[tp]
    \centering
    \includegraphics[width=.48\textwidth]{fig/euler1}
    \includegraphics[width=.48\textwidth]{fig/euler2}
    \caption{Derivation of the Euler method. Left: replacement of the
			solution of the IVP by a line with slope and initial point given
			by the IVP. Right: Euler method with three subintervals.}
    \label{fig:explicit:Euler}
  \end{figure}
  The figure suggests that in general the accuracy of this method may
  not be very good. The first improvement is that we do not draw the
  line through the whole interval from $0$ to $T$.  Instead, we
  insert intermediate points and apply the method to each subinterval,
  where we use the result of a previous interval as the initial point
  for the next subinterval.  As a result one obtains a
  chain of straight lines and the so-called \define{Euler method}.
\end{example}

\input{definitions/partitioning}

\begin{definition}
  In the following chapters we will regularly compare the solution of
  an IVP with the results of discretization methods. Therefore, we
  introduce the following convention for notations and symbols.

  \defindex{solution!continuous}\defindex{solution!exact} The solution
  of the IVP is called the \textbf{exact} or \define{continuous
    solution}\defindex{exact solution}. The term ``continuous''
  indicates here the solution of the non-discretized problem. Its
  symbol is in general $u$ and we set as abbreviation
  \begin{gather*}
    u_k = u(t_k).
  \end{gather*}
  If $u$ is vector-valued we also use the alternative superscript
  $u^{(k)}$ and $u_i^{(k)}$ for a entry of the vector $u(t_k)$.

  \defindex{solution!discrete} In general we write the
  \define{discrete solution} with the
  symbol $y$. We write $y_k$ or $y^{(k)}$ for the value of the
  discrete solution at the point in time $t_k$. In contrast to the
  continuous solution, $y$ only defined at discrete time steps, unless
  for special methods discussed later.
\end{definition}

\input{definitions/one-step}

\begin{remark}
  \label{remark:expl:first-step}
  For one-step methods every step is \emph{per definitionem} similar.
  Therefore, it is sufficient to consider the first step only.  Hence,
  we will define and analyze methods by stating the dependence of
  $y_1$ on $y_0$ which then can be transferred to the general step
  from $y_{n-1}$ to $y_{n}$. The general one-step method above then
  reduces to
  \begin{gather*}
    y_1 = y_0 + h \verfahren(t_0, y_0).
  \end{gather*}
  This implies that the values $y_k$ with $k\ge 2$ are computed
  through formula~\eqref{eq:explicit:8} with the respective $h_k$ and
  the same increment function.
\end{remark}

\input{theorems/euler-linear-1}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Error analysis}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{remark}
  In Figure~\ref{fig:explicit:Euler}, we observe that the error
  consists of two parts at a given time $t_{k+1}$. First, an error on
  the interval $I_k$ due to replacing the differential equation by the
  discrete method. Second, we have to add the error which results from
  the fact that our initial value $y_k$ is already not exact due to
  previous errors. This situation is displayed in
  Figure~\ref{fig:forward-errors}. After one time step, a local error
  has appeared. In the second time step, we already start with an
  erroneous initial value. Therefore, we split the error into the
  local error and an accumulated error. The local error compares
  continuous and discrete solutions on a single interval with the same
  initial value. In the analysis, we will have the options of using
  the exact (right figure) or the approximated initial value (left figure).

  \begin{figure}[tp]
    \centering
    \includegraphics[width=.49\textwidth]{fig/forward-error.tikz}
    \includegraphics[width=.49\textwidth]{fig/backward-error.tikz}
    \caption{Local and accumulated errors. Exact solution in black,
      the Euler method in red. On the left, in blue the exact solution
      of an IVP on the second interval with initial value $y_1$. On
      the right, in purple the second step of the Euler method, but with
      exact initial value $u_1$.}
    \label{fig:forward-errors}
  \end{figure}
\end{remark}

\input{definitions/globalerror}

\input{definitions/truncation-error}

\begin{example}[Euler method]
  To find out the order of consistency of the Euler method, we
  consider the Taylor expansion of the solution at the point $t_{n-1}$:
  \begin{equation*}
    u(t_n) = u(t_{n-1}) + h_n u'(t_{n-1}) + \frac12 h_n^2 u''(\zeta)
  \end{equation*}

  As a result the truncation error reduces to:

  \begin{align*}
    \tau_n & = \frac{u_n - u_{n-1}}{h_n} - F(h;t_{n-1},u(t_{n-1})) \\
    & = \frac{u_{n-1} + h_n f(t_{n-1},u_{n-1})
      + \frac12 h_n^2 u''(\zeta) - u_{n-1}}{h_n} - f(t_{n-1}; u_{n-1}) \\
    & = \frac12 h_n u''(\zeta)
  \end{align*}

% For $u''(\zeta)$ however there holds:
% \begin{equation*}
%   u''(\zeta) = \diffq[\zeta] f(\zeta,u(\zeta)) = \pdiffq[\zeta] f(\zeta,u(\zeta)) + \pdiffq[u] f(\zeta,u(\zeta)) \underbrace{u'(\zeta)}_{= f(\zeta,u(\zeta))}
% \end{equation*}

Under the assumption that $f \in C^1$ on a compact set around the graph
of $u$, this term is bounded, yielding.
\begin{gather*}
  | \tau_n | \le \frac{h_n}{2} \max_{\zeta\in I_n} \abs{u''(\zeta)}
  = \frac{h_n}{2} \max_{\zeta\in I_n} \bigl|\partial_x f(\zeta,
    u(\zeta)) + \partial_u  f(\zeta, u(\zeta)) f(\zeta, u(\zeta))\bigr|
\end{gather*}
Here, we enter the assumption that $f$ is sufficiently smooth to
conclude that the Euler method is consistent of order 1.
\end{example}

\input{theorems/gronwall-discrete}

\begin{proof}
  Define the functions $w(t)$, $a(t)$, and $b(t)$ such that for $k\ge
  1$ and $t\in [k-1,k)$ there holds
  \begin{gather*}
    w(t) = w(t_{k-1}),\quad
    a(t) = b(t_{k-1}),\quad
    b(t) = b(t_{k-1}).
  \end{gather*}
  These functions are bounded and piecewise continuous on any finite
  interval. Thus, they are integrable on $[0,n]$. Therefore, the
  continuous Grönwall inequality of Lemma~\ref{Lemma:gronwall} applies
  and proves the result.
\end{proof}

\input{theorems/stability-discrete}

\begin{proof}
  Subtracting the equations
  \begin{align*}
    \eta_k(y) &= y_k-y_{k-1} - \verfahren_{h_k}(t_{k-1},y_{k-1}),
    \\
    \eta_k(y) &= z_k-z_{k-1} - \verfahren_{h_k}(t_{k-1},z_{k-1}),
  \end{align*}
  we obtain
  \begin{multline*}
    y_k-z_k = y_{k-1} - z_{k-1} + \eta_k(y) - \eta_k(z)
    \\
    + h_k \bigl(
      F_{h_k}(t_{k-1},y_{k-1})-F_{h_k}(t_{k-1},z_{k-1})
    \bigr)
    .
  \end{multline*}
  Recursive application yields
  \begin{gather*}
    \abs{y_n-z_n} \le \abs{y_0-z_0}
    + \sum_{k=1}^n \abs{\eta_k(y)-\eta_k(z)}
    + \sum_{k=1}^n L_h h_{k} \abs{y_k-z_k}.
  \end{gather*}
  The estimate now follows from the discrete Grönwall inequality in
  Lemma~\ref{Lemma:gronwall-discrete}.
\end{proof}

\begin{Corollary*}{finite-precision}{One-step methods with finite precision}
  Let the one-step method $\verfahren$ be run on a computer, yielding
  a sequence $(z_n)$, such that each time step is executed in finite
  precision arithmetic. Let $(y_n)$ be the mathematically correct
  solution of the one-step method. Then, the difference
  equation~\eqref{eq:explicit:8} is fulfilled only up to machine
  accuracy $\epsilon_m$:
  \begin{align*}
    y_0-z_0 &\approx \epsilon_m \\
    \abs{\eta_k(y) - \eta_k(z)} &= \abs{\eta_k(z)} \approx \epsilon_m \abs{z_k}.
  \end{align*}
  Then, the error between the true solution of the one-step method
  $(y_n)$ and the computed solution is bounded by
  \begin{gather*}
    \abs{y_n-z_n} \le e^{L_h(t_n-t_0)} n \epsilon_m \max_k\abs{z_k}.
  \end{gather*}
\end{Corollary*}

\begin{Theorem*}{convergence-one-step}{Convergence of one-step methods}
  Let the one-step method $F(.,.)$ be consistent of order $p$ and
  discretely stable, that is, $F(.,.)$ is Lipschitz continuous in its
  second argument. Let $f(t,u)\in C^{p}$. Furthermore, let
  $y_0 = u_0$. Let $h = \max h_n$ and let there be a positive number
  $\gamma$ such that $\min h_n = \gamma h$. Then, the method converges
  with order $p$ and there holds
  \begin{equation}
    \label{eq:convergence-onestep:1}
    \abs{u_n - y_n} \le c e^{L_h(t_n-t_0)} h^p,
  \end{equation}
  where the constant $c$ is independent of $h$.
\end{Theorem*}

\begin{proof}
  Again we use the discrete stability theorem: with the definition of
  the order of the method, we obtain
  \begin{gather}
    \label{eq:convergence-onestep:2}
    \abs{\eta_k(u)-\eta_k(y)} = \abs{\eta_k(u)} \le c h^{p+1},
  \end{gather}
  where $c$ depends on the derivatives of $u$ (and thus of $f$), but
  not on $u_k-y_k$. On the other hand, we have
  \begin{gather*}
    n \le \frac{t_n-t_0}{\min h_n} \le \frac{t_n-t_0}{\gamma h}.
  \end{gather*}
  Thus, we obtain by summing up~\eqref{eq:convergence-onestep:2} over all $n$
  \begin{gather*}
    \abs{u_n-y_n} \le e^{L_h(t_n-t_0)}\sum_{k=1}^n h_k^{p+1}
    \le c e^{L_h(t_n-t_0)} h^p.
  \end{gather*}
\end{proof}

\begin{corollary}
  The Euler method converges of first order.
\end{corollary}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Runge-Kutta methods}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{intro}
  We are searching for methods which approximate the solution to an
  IVP numerically.  In fact we are not solving the IVP, but the Volterra
  integral equation~\eqref{eq:volterra}.  Hence we can consider
  solving differential equations as a quadrature problem; with the
  difficulty that the function, which we integrate, is not known. This
  consideration leads to a class of methods for IVP, the
  Runge-Kutta methods.
\end{intro}

\input{definitions/erk}

\begin{remark}
  Pursuant to remark~\ref{remark:expl:first-step} we present the
  formula for the calculation of $y_1$ from $y_0$ on the interval from
  $t_0=0$ to $t_1 = h$.  The formula for a later time step $k$ is
  obtained by replacing $y_0$ and $t_0=0$ by $y_k$ and $t_k$,
  respectively to obtain $y_{k+1}$.
\end{remark}

\begin{remark}
  The intermediate values $\rkg_i$ will not be saved separately in
  typical implementations, because it is possible to execute the
  method with the values $k_i$ alone. Nevertheless, the values
  $\rkg_i$ are useful for highlighting the structure of the method.
\end{remark}

\input{definitions/butcher-tableau-erk}

\begin{remark}
  The first row of the tableau is to read in such a manner, that
  $g_1 = y_0$ and $k_1$ is computed directly by $f(t_0, y_0)$. The
  coefficients $a_{1j}$ and $c_0$ do not appear in formulas~\eqref{eq:explicit:1a}
  and~\eqref{eq:explicit:1b} (or are considered zero).

  The further rows indicate the rules for the computation of the
  further values $k_i$ in each case according to the
  formulas~\eqref{eq:explicit:1a} and~\eqref{eq:explicit:1b}. The
  the method is explicit since the computation of $k_i$ only involves
  coefficients with index less than $i$.

  The last row below the line is then the short form of
  formula~\eqref{eq:explicit:1c} and lists quadrature weights.

  We see, that the coefficients $a_{ij}$ form the strict lower
  triangle of a square $s\times s$-matrix $A$. Therefore, in order to
  simplify the summation bounds, we implicitly complete this matrix
  with values $a_{ij} = 0$ for $j\ge i$.This way, the sum
  in~\eqref{eq:explicit:1a} can be taken from $1$ to $s$, independent
  of $i$. We will also associate to an $s$-stage method the vector $b
  = (b_1,\dots,b_s)^T$.
\end{remark}

\begin{example}
  The Euler method \putindex{Euler method} has the \putindex{Butcher
    tableau}:
  \begin{gather*}
    \begin{array}{c|c}
      0 & \\
      \hline
        & 1 \\
    \end{array}
  \end{gather*}
  That leads to the already known formula:
  \begin{equation*}
    y_1 = y_0 + h f(t_0, y_0)
  \end{equation*}
  The values $b_1=1$ and $c_1=0$ indicate that this is a quadrature
  rule with a single point at the left end of the interval. Such a
  rule is exact for constant polynomials and thus of order 1.
\end{example}

\input{definitions/rk2}

\begin{remark}
  The modified Euler method uses an approximation to the value of
  $f(h/2, u(h/2))$ in its quadrature, corresponding to the midpoint
  quadrature rule. The Heun method is constructed analogous to the
  trapezoidal rule. Both quadrature rules are of second order, and so
  are these one-step methods.  Both methods were discussed by Runge in
  his article of 1895~\cite{Runge95}.
\end{remark}

\begin{Lemma}{Heun-consistency}
  The Heun method and the modified Euler method are consistent of second
  order\footnote{Here and in the following proofs of consistency
    order, we will always assume that all necessary derivatives of $f$
  exist and are bounded. We say ``$f$ is sufficiently smooth''.}.
\end{Lemma}

\begin{proof}
  The proof uses Taylor expansion of the continuous solution $u$ and
  the discrete solution $y$ around $t_0$ with respect to $h$. First,
  abbreviating $f_t = \partial_t f(t_0,u_0)$ and
  $f_u = \partial_u f(t_0,u_0)$ and so forth\footnote{Note that $f_u$,
    $f_{u u}$ and so on are tensors of increasing rank.} and replacing
  $u'(t_0)=f(t_0,u_0) = f$:
  \begin{multline}
    \label{eq:explicit:4}
    u_1 = u(t_0+h) = u_0 + h f(t_0, u_0)
    + \frac{h^2}2\bigl(f_t + f_u f\bigr)
    \\
    + \frac{h^3}6\bigl(f_{t t}+2f_{t u}f+f_{u u}f^2+f_u f_t+f_u^2f\bigr)
    + \dots.
  \end{multline}
  For the discrete solution of the modified Euler step on the other
  hand, there holds
  \begin{align*}
    y_1 &= u_0 + h f\left(t_0+\frac h2, u_0+\frac h2 f(t_0,u_0)\right)
    \\
    &= u_0 + h f(t_0, u_0) + \frac{h^2}2 \bigl(f_t + f_u f\bigr)
    \\&\hphantom{...}
      + \frac{h^3}8\bigl(f_{t t}+2f_{t u}f+f_{u u}f^2+f_u f_t+f_u^2f\bigr)
        +\dots.
  \end{align*}
  Thus, $\abs{u_1-y_1} = \mathcal O(h^3)$ and the method is of second
  order. The proof for the Heun method is left as an exercise.
\end{proof}

\begin{Example}{rk3}
  \defindex{Runge-Kutta method!three-stage}
  The three stage Runge-Kutta method is

  \begin{gather*}
    \begin{aligned}
      k_1 & = f(t_0, y_0) \\
      k_2 & = f(t_0 + \frac12 h, y_0 + \frac12 h k_1) \\
      k_3 & = f(t_0 + h, y_0 - h k_1 + 2 h k_2) \\
      y_{n+1} & = y_0 + h ( \frac16 k_1 + \frac46 k_2 + \frac16 k_3 ) \\
    \end{aligned}
    \qquad
    \begin{array}{c|ccc}
      0 & \\
      \frac12 & \frac12 \\
      1 & -1 & 2 \\
      \hline
        & \frac16 & \frac46 & \frac16 \\
    \end{array}
  \end{gather*}
  This method is obviously based on the Simpson rule.
\end{Example}

\begin{remark}
  Computations become tedious very fast, in part due to the sum of
  partial derivatives of $f(t,u)$. This can be simplified by
  considering Runge-Kutta methods for the autonomized ODE (see
  Definition~\ref{Definition:autonomization})
  \begin{gather*}
    \begin{pmatrix} u' \\ t' \end{pmatrix} =
    \begin{pmatrix} f(t,u) \\ 1 \end{pmatrix}.
  \end{gather*}
  Then, the Runge-Kutta method~\eqref{eq:explicit:1} simplifies to
  \begin{gather}
    \label{eq:explicit:7a}
    \begin{split}
      \rkg_i &= y_0 +
      \sum_{j=1}^{i-1} \rka_{ij} h f(\rkg_j),\quad i=1,\dots,\rks
      \\
      y_1 &= y_0 + \sum_{j=1}^s b_j h f(g_j).
    \end{split}
  \end{gather}
\end{remark}

\begin{Lemma}{erk-autonomization}
  An ERK is invariant under autonomization (or in short,
  \putindex{autonomizable}), if and only if
  \begin{gather}
    \label{eq:explicit:9}
    c_i = \sum_{j=1}^{i-1} a_{ij}, \quad i=1,\dots,s.
  \end{gather}
\end{Lemma}

\begin{proof}
  Observing the last component of the vector $u$ in the previous
  remark and the method applied to it yields the condition.
\end{proof}

%HNW p. 145 and pp. 135/137
\begin{Lemma}{rk-order-3-4}
  An autonomizable ERK with $s$ stages is consistent of third order, if
  and only if the following conditions are met:
  \begin{subequations}
    \label{eq:explicit:11}
    \begin{align}
      \label{eq:explicit:12}
      b_1 + \dots + b_s &= 1, \\
      \label{eq:explicit:13}
      b_1c_1 + \dots + b_s c_s &= 1/2, \\
      \label{eq:explicit:14}
      b_1c_1^2 + \dots + b_s c_s^2 &= 1/3, \\
      \label{eq:explicit:15}
      \sum\nolimits_{i,j} b_i a_{ij}c_j &= 1/6.
    \end{align}
    It is consistent of fourth order, if and only if additionally
    \begin{align}
    \label{eq:explicit:16}
    b_1c_1^3 + \dots + b_s c_s^3 &= 1/4, \\
    \label{eq:explicit:18}
    \sum\nolimits_{i,j} b_i a_{ij} c_j^2 &= 1/12, \\
    \label{eq:explicit:19}
    \sum\nolimits_{i,j,k} b_i a_{ij} a_{jk} c_k &= 1/24,\\
    \label{eq:explicit:17}
    \sum\nolimits_{i,j} b_i c_i a_{ij} c_j &= 1/8.
  \end{align}
  \end{subequations}
\end{Lemma}

\begin{remark}
  We can rephrase these conditions, such that an ERK is of order $k$ if
  the quadrature with
  support points $c_i$ and corresponding weights $b_i$ is exact for
  polynomials of degree $k-1$:
  \begin{gather*}
    \sum_{i=1}^{s} b_i p(c_i) = \int_0^1 p(t)\dt,
    \qquad \forall p\in \P_{k-1}.
  \end{gather*}
  Furthermore, for $k \ge 3$
  \begin{gather*}
    \sum_{ij} b_i a_{ij} p(c_j) = \int_0^1\int_0^t p(s)\ds\dt,
    \qquad \forall p\in \P_{k-2}.
  \end{gather*}
  Additionally, for $k\ge 4$
  \begin{xalignat*}2
    \sum_{ijk} b_i a_{ij} a_{jk} p(c_k) &= \int_0^1\int_0^t\int_0^s p(r)\dr \ds \dt,
    &&\forall p\in \P_{k-3},\\
    \sum_{ij} b_i p(c_i) a_{ij} q(c_j)
    &= \int_0^1 p(t) \int_0^t q(s) \ds\dt
    &&\forall p\in \P_{k_1}, q\in\P_{k_2}, k_1+k_2 = k-2.
  \end{xalignat*}
\end{remark}

\input{theorems/taylor-u}
\begin{proof}
  Taking derivatives of $u$ and replacing every occurrence of $u'$ by
  $f(u)$. For scalar valued functions, we clarify this at the example
  \begin{align*}
    u'(t) &= f(u(t)) \\
    u''(t) &= \bigl(u'(t)\bigr)'
    = f(u(t))' = f'(u(t))u'(t)
    \\& = f'(u(t)) f(u(t))
    \\
    u^{(3)} &= \bigl(u''(t)\bigr)' = \bigl(f'(u(t)) f(u(t))\bigr)'
    \\&= f''(u(t)) u'(t) f(u(t)) + f'(u(t)) f'(u(t)) u'(t)
    \\& = f''(u(t)) f(u(t))^2 + f'(u(t))^2 f(u(t)).
  \end{align*}
  After the concept is clear, we have to keep track of the vector
  indices and compute with brute force.
  It may be worth noting, that in the 4th order term, we used the fact
  that we can swap summation indices and get
  \begin{gather*}
    \sum_{\lambda,\mu,\nu} \partial_{\lambda\mu} f_n \partial_\nu f_\lambda f_\mu f_\nu
    = \sum_{\lambda,\mu,\nu} \partial_{\lambda\mu} f_n \partial_\nu f_\mu f_\lambda f_\nu
    = \sum_{\lambda,\mu,\nu} \partial_{\lambda\nu} f_n \partial_\mu f_\lambda f_\mu f_\nu
    .
  \end{gather*}
\end{proof}

\input{theorems/taylor-y}

\begin{proof}
  We begin with the observation that for an arbitrary function $\phi$ holds
  \begin{gather*}
     \frac{d^{q}}{d h^{q}}\bigl(h\phi(h)\bigr)\bigg|_{h=0}
     = \left[ h \frac{d^{q}}{d h^{q}} \phi(h) + q h'
       \frac{d^{q-1}}{d h^{q-1}} \phi + \binom{q}{2}
       h''\dots\right]_{h=0}
     = q \frac{d^{q-1}}{d h^{q-1}} \phi.
  \end{gather*}
  Next, we use~\eqref{eq:explicit:1c} to obtain
  \begin{align*}
    y(h) &= u_0, \\
    y^{(q)}(h)\big|_{h=0} &= q \sum_{j=1}^s b_j \frac{d^{q-1}}{d h^{q-1}} f(g_j)\bigg|_{h=0}.
  \end{align*}
  We observe $g_i(0) = u_0$. Further, from~\eqref{eq:explicit:1a}, we
  obtain
  \begin{align*}
    g_i(h)\big|_{h=0} &= u_0, \\
    g_{i;n}^{(q)}(h)\big|_{h=0} &= q \sum_{j=1}^{i-1} a_{ij}
                                  \frac{d^{q-1}}{d h^{q-1}} f_n(g_{j})\bigg|_{h=0}.
  \end{align*}
  Here, $g_{i;n}$ refers to the component $k$ of vector
  $g_i$. Finally, we need
  \begin{align*}
    \frac{d}{d h} f_n(g_i(h))\big|_{h=0}
    & = \sum_{\lambda} \partial_{\lambda} f_n g'_{i;\lambda}\\
    \frac{d^2}{d h^2} f_n(g_i(h))\big|_{h=0}
    & = \sum_{\lambda,\mu} \partial_{\lambda\mu} f_n g'_{i;\lambda} g'_{i;\mu}
      + \sum_k \partial_{\lambda} f_n g''_{i;\lambda}.
  \end{align*}
  Summarizing, we obtain
  \begin{align*}
    y'_n &= \sum_{j=1}^s b_j f_n(g_j)
    \\
    y''_n &= 2 \sum_{j=1}^s b_j\frac{d}{d h} f_n(g_j)
    = \sum_{j=1}^s \sum_{k=1}^{j-1} b_j a_{j k}
            \sum_{\lambda} \partial_{\lambda} f_n f_\lambda
    \\
    y'''_n &= 3 \sum_{j=1}^s b_j\frac{d^2}{d h^2} f_n(g_j)
    \\&= 3 \sum_{j=1}^s b_j \left[
      \sum_{\lambda,\mu} \partial_{\lambda\mu} f_n g'_{j;\lambda} g'_{j;\mu}
      + \sum_\lambda \partial_{\lambda} f_n g''_{j;\lambda}\right]
    \\&= 3 \sum_{j=1}^s b_j \left[
        \sum_{\lambda,\mu} \partial_{\lambda\mu} f_n \sum_{k=1}^{j-1} a_{j k}
        f_\lambda \sum_{k=1}^{j-1} a_{j k}
        f_\mu + \sum_{\lambda} \partial_{\lambda} f_n 2 \sum_{k=1}^{j-1}
        a_{j k} \sum_{l=1}^{k-1} a_{kl} \sum_\mu \partial_\mu f_\lambda f_\mu
        \right]
  \end{align*}
\end{proof}

\begin{proof}[Proof of Lemma~\ref{Lemma:rk-order-3-4}]
  The proof utilizes Taylor expansion of $u_1$ and $y_1$ provided in
  Lemmas~\ref{Lemma:taylor-u} and~\ref{Lemma:taylor-y},
  respectively. Once we have computed these expansions, we compare
  coefficients in front of equal derivatives in order to get the result.
\end{proof}

\begin{remark}
  Butcher introduced a graph theoretical method for order conditions
  based on trees. While this simplifies the process of deriving these
  conditions for higher order methods considerably, it is beyond the
  scope of this course.
\end{remark}

\input{definitions/rk4}

\begin{remark}[Order conditions and quadrature]
  The order conditions derived by excessive Taylor expansion have a
  very natural interpretation through the analysis of quadrature
  formulas for the Volterra integral equation, where $(h c_i)$ are the
  quadrature points and the other values are quadrature weights.
  First, we observe that
  \begin{gather*}
    \sum_i b_i f(g_i) \quad\text{approximates}\quad
    \frac1h\int_0^1 f(u(h s)\ds.
  \end{gather*}
  In this view,
  conditions~\eqref{eq:explicit:12}--\eqref{eq:explicit:14}
  and~\eqref{eq:explicit:16} state that the formula $\sum_i b_i
  p(c_i)$ is an exact integral for polynomials of degree up to 3. In a
  previous semester, we have made use of this property to prove that
  the formula is of 4th order.

  Equally, we deduce from formula~\eqref{eq:explicit:1a} for $g_i$ that
  \begin{gather*}
    \sum_j a_{ij} f(g_j) \quad\text{approximates}\quad
    \frac1h\int_0^{c_i} f(u(h s)\ds.
  \end{gather*}
  The condition~\eqref{eq:explicit:9} that the method be autonomizable
  states nothing but that this be exact for constant functions. For
  higher order, the accuracy of the value of $g_i$ only implicitly
  enters the accuracy of the Runge-Kutta method by integrating this
  value again. Thus, we actually look at approximations of integrals
  of the form
  \begin{gather*}
    \int_0^1 \phi(s) \int_0^s \psi(r) \dr\ds.
  \end{gather*}
  Condition~\eqref{eq:explicit:15} for 3rd order states, that this
  condition must be true for linear polynomials $\psi(r)$ and constant
  $\phi(s)$, thus, after the interior integration again a polynomial
  of second order. Equally, conditions~\eqref{eq:explicit:17}
  and~\eqref{eq:explicit:18} state this for linear polynomials
  $\psi(r)$ with linear $\phi(s)$ and for quadratic polynomials
  $\psi(r)$ with constant $\phi(s)$, respectively. Finally,
  condition~\eqref{eq:explicit:19} states that the quadrature has to
  be exact for any linear polynomial $\phi(\tau)$ in
  \begin{gather*}
    \int_0^1 \int_0^s \int_0^r \phi(\tau) \,\diffd \tau\dr\ds.
  \end{gather*}
\end{remark}

\begin{remark}[Butcher barriers]
  The maximal order of an explicit Runge-Kutta method is limited
  through the number of stages, or vice versa, a minimum number of
  stages is required for a certain order. The \define{Butcher barriers} state
  that in order to achieve order $p$ one requires $\rks$ stages, where
  $p$ and $\rks$ relate as follows:
  \begin{center}
    \input{tables/butcher-barriers}
  \end{center}

  These order bounds refer to systems of differential equations.  For
  a simple equation they may be better. For instance, there exists a
  five-stage method which solves the one dimensional IVP with order 5.

  For $p = 10$ there is only known a method with $s = 17$ until now.
  It is possible that there exists a method that needs less stages,
  because currently no proof for a minimal number of stages is
  available.
\end{remark}

\begin{Lemma}{erk-Lipschitz}
  Let $f(t,u)$ admit the uniform \putindex{Lipschitz condition}. Then,
  every autonomizable ERK which is consistent of order one admits a
  uniform Lipschitz condition.
\end{Lemma}

\begin{proof}
  We observe that the \putindex{increment function} is
  \begin{gather}
    \label{eq:explicit:7}
    \verfahren(0, y) = \sum_{j=1}^s \rkb_j f(h\rkc_i, \rkg_i(y)),
  \end{gather}
  with $g_i(y)$ defined recursively by
  \begin{gather*}
     g_i(y) = y + h \sum_{j=1}^{i-1} \rka_{ij} f(h\rkc_j, \rkg_j(y)).
  \end{gather*}
  Let $L$ be the Lipschitz constant of $f$. Let
  $d_i = \abs{g_i(x)-g_i(y)}/\abs{x-y}$. We have
  \begin{align*}
    d_1 &= 1
    \\
    d_2 &= \abs{x-y + h \rka_{21}
          \Bigl(f\bigl(h\rkc_2,g_1(x)\bigr)
          -f\bigl(\rkc_2,g_1(x)\bigr)
          \Bigr)}/\abs{x-y}
    \\ &\le (1+ha_{21}L)
         = (1+h\rkc_{1}L)
    \\
    d_3 & \le \Bigl(1+
          h L\bigl(\rka_{31}+\rka_{32}(1+ha_{21}L)\bigr)\Bigr)
    \\ &\le \bigl(1+h L c_3(1+h c_2L)\bigr)
    \\
    d_4 & \le \Bigl(1+
          h L c_4\bigl(1+h L c_3(1+h L c_2)\bigr)\Bigr)
    \\
    d_s & \le
          \Bigl(1+ h L c_s\bigl(
          1+\dots(1+h L c_1)\dots\bigl)\Bigl).
  \end{align*}
  Since $c_i \le 1$, the factor is bounded by
  $d_s\le(1+h L)^{s-1}$. Moreover, if $h L \le 1$, we realize that
  \begin{gather*}
    d_s = \Bigl(1+ h L \bigl(1+\dots(1+h L)\dots\bigl)\Bigl) \le s.
  \end{gather*}
  Finally, we enter this result into~\eqref{eq:explicit:7} to obtain
  \begin{align*}
    \abs{\verfahren(0,x)-\verfahren(0,y)}
    & \le \sum_{j=1}^s b_j L d_j \abs{x-y}
    \\
    & \le d_s L \abs{x-y}.
  \end{align*}
  Thus, the increment function $\verfahren$ admits a Lipschitz
  condition with constant $L_h = L(1+h L)^{s-1}$ for general step size
  $h$ and $L_h = s L$ for $h\le 1/L$.
\end{proof}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Estimates of the local error
  and time step control}
  \label{section:step_size_control}

\begin{intro}
  In the preceding paragraphs, we have used a crude a priori estimate
  of the local error based on high order derivatives of the right hand
  side $f(t,u)$. In the case of a complex nonlinear system, such an
  estimate is bound to be inefficient, since it involves global bounds
  on the derivatives. Obviously, the local error cannot be computed
  exactly either, because that would require or imply the knowledge of
  the exact solution.

  In this section, we discuss two methods which allow an estimate of
  the truncation error from computed solutions. These estimates are
  local in nature and therefore usually much sharper. Thus, they can
  be used to control the step size, which in turn gives good control
  over the balance of accuracy and effort. Nevertheless, it should be
  pointed out that in these estimates there is an implicit assumption
  that the true solution $u$ is sufficiently regular and the step size
  is sufficiently small, such that the local error already follows
  the theoretically predicted order.

  Given an estimate for the local error, we can devise an algorithm
  step size control, which controls the local error and thus in a
  certain way the global error.
\end{intro}

\begin{algorithm}[Adaptive step size control]
  Let there be an estimate for the local error based
  on $\abs{y_1 - \hat y_1}$ . Then, the following algorithm can be used
  to guarantee that the local error of a one-step method remains below
  a threshold $\epsilon$ in every time step:

  \begin{enumerate}
  \item Given $y_{k-1}$, compute $y_k$ and $\hat y_k$ with time step $h_k$.
  \item Compute
    \begin{gather}
      \label{eq:explicit:30}
      h_{\text{opt}} = h_k \left(\frac{\epsilon}{y_k - \hat
          y_k}\right)^{\frac1{p+1}}.
    \end{gather}
  \item If $h_{\text{opt}} < h_k$ the time step is rejected: let
    $h_k = h_{\text{opt}}$ and recompute $y_k$ and $\hat y_k$.
  \item If the time step was accepted, let $h_{k+1} = h_{\text{opt}}$.
    \begin{enumerate}
    \item If $t_k+h_{k+1} > t_n$, let $h_{k+1} = t_n-t_k$.
    \end{enumerate}
    Increase $k$ by one and proceed with the first step.
  \end{enumerate}
\end{algorithm}

\begin{remark}
  It might happen, that the value $t_k$ is just below $t_n$ with a
  difference close to machine accuracy. As a result, the next time step
  with $h_{k+1} \approx \epsilon_m$ would suffer from round-off
  errors. Therefore, it is advisable to avoid this situation by
  expanding the last time step, if $t_n-t_k \le c h_{k+1}$ where $c$
  is a moderate constant of size around $1.1$.
\end{remark}

\begin{remark}
  This algorithm controls and equilibrates the local
  error. Nevertheless, the global estimate still retains the
  exponential term. The error estimation techniques in this section
  are thus not optimal controlling the global error, which involves
  considerably more effort and will be discussed in a later course.
\end{remark}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Extrapolation methods}

\begin{intro}
  Here, we estimate the \putindex{local error} by a method called
  \putindex{Richardson extrapolation}. It is based on computing two
  approximations with the same method, but different step size, say
  an approximation $y_2$ with two steps of size $h$ and an
  approximation $\hat y_2$ with one step of size $2h$.
\end{intro}

\input{theorems/richardson}

\begin{proof}
  For the proof we need a refined version of the local error estimates
  as well as the global error estimate in
  Theorem~\eqref{Theorem:convergence-one-step} which can be obtained by
  adding one more step of Taylor expansion. Then, we get for the local
  error of a method of order $p$ estimates of the form
  \begin{gather}
    \label{eq:explicit:21}
    e_1 = u_1 - y_1 = C h^{p+1} + \mathcal O(h^{p+2}),
  \end{gather}
  with a constant (vector) $C$ with not necessarily positive
  entries. In the same way, we refine the estimate for error
  propagation from basic Lipschitz continuity to
  \begin{gather}
    \label{eq:explicit:22}
    e_{2;\text{acc}} = \left( \identity + h \frac{\partial f}{\partial y} +
      \mathcal O(h^2)\right) \bigl(u_1-y_1).
  \end{gather}
  The local error on the second interval is of the same structure
  as~\eqref{eq:explicit:21}, but on the interval starting at $t_1$
  with initial value $y_1 = y_0 + \mathcal O(h)$.

  Thus, we obtain for the error after two steps of size $h$:
  \begin{multline}
    \label{eq:explicit:25}
    u_2-y_2
    = \underbrace{\bigl(\identity + \mathcal O(h)\bigr) C
      h^{p+1}}_{\text{local 2}}
    + \bigl(C+\mathcal O(h)\bigr) h^{p+1} + \mathcal
      O(h^{p+2})
    \\  = 2C h^{p+2} + \mathcal O(h^{p+2}).
  \end{multline}
  We compare this to a single step for $\hat y$ with
  \begin{gather}
    \label{eq:explicit:23}
    u_2 - \hat y_2 = C (2h)^{p+1} + \mathcal O(h^{p+2}).
  \end{gather}
  Subtracting equations~\eqref{eq:explicit:25}
  and~\eqref{eq:explicit:23}, we obtain
  \begin{gather*}
    y_2 - \hat y_2 = \bigl(2-2^{p+1}\bigr)C h^{p+1} + \mathcal O(h^{p+2}),
  \end{gather*}
  such that
  \begin{gather}
    \label{eq:explicit:26}
    C h^{p+1} = \frac{y_2 - \hat y_2}{2^{p+1}-2} + \mathcal O(h^{p+2}).
  \end{gather}
  We enter this result into~\eqref{eq:explicit:25} to conclude
  \begin{gather}
    \label{eq:explicit:27}
    u_2 - y_2 = \frac{y_2 - \hat y_2}{2^{p}-1} + \mathcal O(h^{p+2}).
  \end{gather}
  Adding $y_2$ on both sides, we see that
  \begin{gather}
    \label{eq:explicit:28}
    \tilde y_2 = y_2 + \frac{y_2 - \hat y_2}{2^{p}-1}
  \end{gather}
  approximates $u_2$ of order $\mathcal O(h^{p+2})$ and thus one order
  better than $y_2$.
\end{proof}

\begin{remark}
  Formula~\eqref{eq:explicit:10} can be evaluated after computation of
  $y_2$ and $\hat y_2$ in order to obtain an estimate for the local
  error of $y_2$. This estimate can be used to control the step size
  control according to the algorithm above. We do not have an
  estimate for the error of $\tilde y$. Nevertheless, we expect its
  values to be more accurate, such that we should use $\tilde y$ as
  approximation and initial value for the next time step.
\end{remark}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Embedded Runge-Kutta methods}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Instead of estimating the \putindex{local error} by doubling the step size,
embedded Runge-Kutta methods use two methods of different order to
achieve the same effect. The key to efficiency is here, that the
computed stages $g_i$ are the same for both methods, and only the
quadrature weights $b_i$ differ.

\begin{definition}[Embedded Runge-Kutta methods]
  \defindex{Runge-Kutta method!embedded} An embedded $\rks$-stage
  Runge-Kutta method with orders of consistence $p$ and $\hat p$
  computes two solutions $y$ and $\hat y$ with the same function
  evaluations. For this purpose we will first compute contributions
  $g_i$ and $k_i$ for $i=1,\dots,\rks$ as in the normal Runge-Kutta
  method of stage $\rks$. The function values at the end of the time
  step result as follows
  \begin{gather}
    \begin{split}
      y_1 &= y_0 + h \sum \rkb_i k_i \\
      \hat y_1 &= y_0 + h \sum \hat \rkb_i k_i.
    \end{split}
  \end{gather}
  The methods for $y$ and $\hat y$ are consistent of order $p$ and
  $\hat p$, respectively. We let $\hat p < p$, for example
  $\hat p = p-1$.

  This can be achieved by e.g. using the same mathod twice and omitting
  $\hat{b}_j$ for one $j \in \{1, \dots, s\}$.
\end{definition}

\input{definitions/embedded-butcher}

\begin{remark}
  For higher order methods or functions $f(t,u)$ with complicated
  evaluation, most of the work lies in computation of the
  stages. Thus, the additional quadrature for the computation of $\hat
  y$ is almost for free. Nevertheless, due to the different orders of
  approximation, $y$ is much more accurate and we obtain
  \begin{gather}
    \label{eq:explicit:29}
    u_1 - \hat y_1 = y_1 - \hat y_1 + \mathcal O(h^p).
  \end{gather}
  Thus, $y_1 - \hat y_1$ is a good estimate for the \putindex{local
    error} of $\hat y_1$. This is the error which is used in step size
  control below. Similar to Richardson extrapolation above, we use the
  more accurate value $y_1$ for further computation, even if we do not
  have a computable estimate for its local error.
\end{remark}

\input{definitions/dormand-prince-45.tex}

\begin{remark}
  The Dormand-Prince method of orders 4 and 5 has become a standard
  tool for the integration of IVP. It is the backbone of
  \texttt{ode45} in Matlab.
\end{remark}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Continuous Runge-Kutta methods}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{intro}
  The Runge-Kutta methods discussed so far compute highly accurate
  approximations to the solution $u(t)$ in the discrete points
  $(t_k)_{k=1,\dots,n}$. Such approximations are useful, if only the
  value at the interval end $t_n=T$ is needed, or if the time steps
  are sufficiently small in order to generate a plot of the solution
  history. There are some problems though, where the values in
  discrete points are not sufficient:
  \begin{enumerate}
  \item The step size control managed to use very large steps from
    which for instance a plot cannot read easily. Thus, we require the
    solution in the continuum between two time steps.
  \item Accurate approximations inside an interval are required. This
    may be due to the fact that we want to measure the length of a
    period of a periodic solution, or that the equation contains
    switches which change discretely when the solution attains certain
    values.
  \end{enumerate}
  In all of these cases, we need an interpolation formula for these
  intermediate values. Unfortunately, as the example of the classical
  Runge-Kutta method shows, the values $g_i$ have questionable value
  in this business. Therefore, in order to be better than linear
  interpolation between $y_{k-1}$ and $y_k$, we have to consider
  formulas, which provide the information for accurate interpolation
  with low additional cost.
\end{intro}

\begin{definition}
  \defindex{Runge-Kutta method!continuous}
  A continuous Runge-Kutta method is a method of the same type as in
  definition~\ref{Definition:erk}, for which the coefficients $\rkb_i$ are replaced
	by continuous functions $\rkb_i(\theta)$ on the interval $[0,1]$.
  For this reason the equation~\eqref{eq:explicit:1c} is augmented by
  \begin{equation}
    \label{eq:explicit:24}
    y(t_0+ \theta h) = y_0 + \sum\limits_{i=1}^{s^*} \rkb_i (\theta) k_i.
  \end{equation}
  Here the stage number $\rks^*$ may be higher than $\rks$.
  Then additional intermediate values $k_i$ have to be generated.
\end{definition}

\begin{remark}
  The local error $u(t_0+\theta h) - y(t_0+\theta h)$ is of order
  $p^*$ if the derivative $\partial_\theta^k y$ approximates
  $\partial_\theta^k u$ with an error of order $h^{p^*-k+1}$. Note that
  the first derivatives involve the derivatives of $g_i$ with respect
  to $h$.

  For a later time, we observe that the error of the initial value of
  an interval is only $\mathcal O(h^p)$. Thus, an optimal continuous
  formula balancing the global error of the original method with the
  local error of the continuous method should be of order
  $p^*=p-1$.
\end{remark}

\begin{remark}
  If $\rks^* > \rks$ choose $k_{\rks+1} = k_1 = f(t_n,y_1)$ of the next time step.
\end{remark}

\input{definitions/rk4-continuous}
\input{definitions/dormand-prince-45-continuous}

\begin{remark}
  Collocation methods will provide a natural way to obtain continuous
  method in the next chapter.
\end{remark}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "notes"
%%% End:


\documentclass[a4paper]{scrartcl}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{pgfplots}
\usepackage{amsmath, enumerate, amssymb, multirow, fancyhdr, color, graphicx, lastpage, listings, tikz, pdflscape, subfigure, float, polynom, hyperref, tabularx, forloop, geometry, listings, fancybox, tikz, forest, tabstackengine, cancel, bbm}
\input kvmacros
\geometry{a4paper,left=3cm, right=3cm, top=3cm, bottom=3cm}
\pagestyle {fancy}

\begin{document}

We assume $\sigma$ is the weight matrix of a weighted undirected graph, so $\sigma_{ij} = \sigma_{ji}$ and $\sigma_{ij} = 1$ if there is an edge between $i$ to $j$ and $\sigma_{ij} = 0$ if there is no edge between $i$ and $j$.
For simplicity we assume that nodes are not self connected, that is $\sigma_{ii} = 0$.

\section*{(a)}

{
\color{green}
TODO: Note the $\leq$ instead of the $=$ in the following. This propagates through all formulas below!
}

For $M$ to be a matching, it needs to hold by definition, that $$\forall i:  \sum_j \sigma_{ij} \leq 1$$
This is equivalent to saying $$ \prod_i \mathbbm{1}_{\left[\sum_j \sigma_{ij} \leq 1\right]} = 1 $$
where $\mathbbm{1}$ is the indicator function.

So we find the probability distribution

\begin{align}
    P(M) = \frac{1}{Z}\prod_i \mathbbm{1}_{\left[\sum_j \sigma_{ij} \leq 1\right]}
\end{align}

Where $Z$ is the normalization wich is the count of all matchings
\begin{align}
    Z = \sum_{M \in \mathcal{G}} \mathbbm{1}_{\left[ \prod_i \mathbbm{1}_{\left[\sum_j \sigma_{ij} \leq 1\right]} = 1 \right]}
\end{align}
Here $\mathcal{G}$ is the set of all subsets of the graph.

\section*{(b)}

The size of a matching $\lvert M \rvert$ is the count of its edges. With the adjecency matrix $\sigma$ that is the sum of all its elements divided by two: $$ \lvert M \rvert = \frac{\sum_{i} \sum_{j} \sigma_{ij}}{2}$$

We simply introduce the size of a matching $\lvert M \rvert$ as a scaling factor in the probability distribution, to give larger weight to larger matchings:

\begin{align}
    P(M) = \frac{1}{Z} \lvert M \rvert \prod_i \mathbbm{1}_{\left[\sum_j \sigma_{ij} \leq 1\right]}
\end{align}

which has new normalization factor

\begin{align}
    Z = \sum_{M \in \mathcal{G}} \lvert M \rvert \cdot \mathbbm{1}_{\left[ \prod_i \mathbbm{1}_{\left[\sum_j \sigma_{ij} \leq 1\right]} = 1 \right]}
\end{align}

% We have the size $S = M=\sum_{ij} \sigma^{ij}$ and the normalized version $S^*=\frac{M}{\lvert E \rvert}$.

{\color{red}
TODO I feel like there would be a way to integrate the scalar $\lvert M \rvert$ into the factor (in the product) but i can't find it. (Nevermind i don't think so anymore, with clarification 2 we don't need it anyways)
}

\section*{(c)}

The Graph is given as:

\begin{tikzpicture}
    [scale=1,auto=left,every node/.style={circle,draw}]
    \node (n1) at (1,3) {1};
    \node (n2) at (3,3)  {2};
    \node (n3) at (4,2)  {3};
    \node (n4) at (3,1) {4};
    \node (n5) at (1,1)  {5};
    \node (n6) at (0,2)  {6};

    \foreach \from/\to in {n1/n2,n1/n3,n2/n3,n1/n4,n2/n5,n3/n6}
      \draw (\from) -- (\to);

\end{tikzpicture}

An ordered drawing of this graph is:

\begin{tikzpicture}
    [scale=1,auto=left,every node/.style={circle,draw}]
    \node (n1) at (1,4) {1};
    \node (n2) at (3,3)  {2};
    \node (n3) at (5,4)  {3};
    \node (n4) at (1,1) {4};
    \node (n5) at (3,1)  {5};
    \node (n6) at (5,1)  {6};

    \foreach \from/\to in {n1/n2,n1/n3,n2/n3,n1/n4,n2/n5,n3/n6}
      \draw (\from) -- (\to);

\end{tikzpicture}

In general the probability distribution found above is given as
\begin{align}
    P(M) = \frac{1}{Z}\prod_i \mathbbm{1}_{\left[\sum_j \sigma_{ij} \leq 1\right]}
\end{align}
Which we can write as
\begin{align}
    P(M) = \frac{1}{Z} \prod_i \psi_i(\partial M)
\end{align}
with factors
\begin{align}
    \psi_i(\partial M) = \mathbbm{1}_{\left[ \sum_j \sigma_{ij} \leq 1 \right]}
\end{align}

By $\partial M$ we denote, that each factor $\psi_i$ is only connected to all nodes $\sigma_{ij}$ for which there is a connection from $i$ to $j$, so for all nodes for which $\sigma_{ij} = 1$ in the initial graph $G$.

In general to draw a factor graph we would start with a graph with all $\sigma_{ij}$ connected to all factors $\psi_i$. Then we can omit all $\sigma_{ij}$ where $\sigma_{ij} = 0$ in the initial graph $G$. Furthermore we can omit all connections from factors $\psi_i$ to $\sigma_{ij}$ if $i$ and $j$ are not connected.

For the graph given here this yields the factor graph:

\begin{tikzpicture}
    [scale=1,auto=left,every node/.style={circle,draw}]
    \node (s12) at (4,6) {$\sigma_{12}$};
    \node (s13) at (4,5) {$\sigma_{13}$};
    \node (s23) at (4,4) {$\sigma_{23}$};
    \node (s14) at (4,3) {$\sigma_{14}$};
    \node (s25) at (4,2) {$\sigma_{25}$};
    \node (s36) at (4,1) {$\sigma_{36}$};

    \node[rectangle] (f1) at (1,6) {$\psi_{1}$};
    \node[rectangle] (f2) at (1,5) {$\psi_{2}$};
    \node[rectangle] (f3) at (1,4) {$\psi_{3}$};
    \node[rectangle] (f4) at (1,3) {$\psi_{4}$};
    \node[rectangle] (f5) at (1,2) {$\psi_{5}$};
    \node[rectangle] (f6) at (1,1) {$\psi_{6}$};

    \foreach \from/\to in {f1/s12,f1/s13,f1/s14,f2/s12,f2/s23,f2/s25,f3/s13,f3/s23,f3/s36,f4/s14,f5/s25,f6/s36}
      \draw (\from) -- (\to);

\end{tikzpicture}

{\color{red}
TODO Is this also so simple for the probability distribution scaled by matching size $\lvert M \rvert$? (Nevermind, is saw Clarification 2)
}

\section*{(d)}

The general definitions are given as:
\begin{align}
    \nu_{i \rightarrow a}^{(t + 1)}(x_i) &\cong \prod_{b \in \partial i \backslash a} \hat{\nu}_{b \rightarrow i}^{(t)}(x_i) \\
    \hat{\nu}_{a \rightarrow i}^{(t)}(x_i) &\cong \sum_{\textbf{x}_{\partial a} \backslash i} \psi_a (\textbf{x}_{\partial a}) \prod_{j \in \partial a \backslash i} \nu_{j \rightarrow a}^{(t)}(x_j) \\
    P(x_i) &= \nu_i^{(t)}(x_i) \cong \prod_{a \in \partial i} \hat{\nu}_{a \rightarrow i}^{(t-1)}(x_i)
\end{align}

In our case we have messages from variable to factor:
\begin{align}
    \nu_{\sigma_{ij}}^{(ij) \rightarrow i} &\cong \prod_{b \in \partial (ij) \backslash i} \hat{\nu}_{\sigma_{ij}}^{b \rightarrow (ij)} \\
    &= \hat{\nu}_{\sigma_{ij}}^{j \rightarrow (ij)}
\end{align}
This simplification comes, since variable $(ij)$ only has one other neighbouring factor except $i$, which is $j$. \\

And messages from factor to variable:
\begin{align}
    \hat{\nu}_{\sigma_{ij}}^{i \rightarrow (ij)} \cong \sum_{\boldsymbol{\sigma}_{\partial i} \backslash (ij)}
    \psi_i(\boldsymbol{\sigma}_{\partial i}) \prod_{(ik) \in \partial i \backslash (ij)} \nu_{\sigma_{ik}}^{(ik) \rightarrow i}
\end{align}

For our problem this leads to messages from factor to variable:
\begin{align}
    \hat{\nu}_{\sigma_{ij}}^{i \rightarrow (ij)} &\cong \sum_{\boldsymbol{\sigma}_{\partial i} \backslash (ij)} \mathbbm{1}_{\left[\sum_l \sigma_{il} \leq 1 \right]} \prod_{(ik) \in \partial i \backslash (ij)} \nu_{\sigma_{ik}}^{(ik) \rightarrow i}\\
\end{align}

{
\color{green}
Note here that the sum is over all the values that the $\sigma$ which are neighbours of $i$ could assume.
}

The marginal is:
\begin{align}
    P(\sigma_{ij}) &\cong \prod_{b \in \partial (ij)} \hat{\nu}_{\sigma_{ij}}^{b \rightarrow (ij)} \\
    &= \hat{\nu}_{\sigma_{ij}}^{i \rightarrow (ij)} \cdot \hat{\nu}_{\sigma_{ij}}^{j \rightarrow (ij)}
\end{align}

{
\color{green}
TODO: I don't know if we have to write something for the second hint, but this would be the following?
}

In the following we find some derivations for the adjecent edges if $\sigma_{ij} = 1$ and $\sigma_{ij} = 0$:

For $\sigma_{ij} = 1$ this leads to
\begin{align}
    P(\sigma_{ij} = 1) &\cong \hat{\nu}_{\sigma_{ij}}^{i \rightarrow (ij)} \cdot \hat{\nu}_{\sigma_{ij}}^{j \rightarrow (ij)} \\
    &= \left[ \sum_{\sigma_{\partial i} \backslash (ij)} \mathbbm{1}_{\left[\sum_l \sigma_{il} \leq 1 \right]} \prod_{(ik) \in \partial i \backslash (ij)} \nu_{\sigma_{ik}}^{(ik) \rightarrow i} \right]
     \cdot \left[ \sum_{\sigma_{\partial j} \backslash (ij)} \mathbbm{1}_{\left[\sum_l \sigma_{jl} \leq 1 \right]} \prod_{(jk) \in \partial j \backslash (ij)} \nu_{\sigma_{jk}}^{(jk) \rightarrow j} \right]
\end{align}
Which can only be $\geq 0$ if
\begin{align}
    \sum_{\sigma_{\partial i} \backslash (ij)} \mathbbm{1}_{\left[\sum_l \sigma_{il} \leq 1 \right]} &= 1 \\
    \sum_{\sigma_{\partial i} \backslash (ij)} \mathbbm{1}_{\left[\sum_{l \backslash (l = j)} \sigma_{il} + 1 \leq 1 \right]} &= 1 \\
\end{align}
and
\begin{align}
    \sum_{\sigma_{\partial j} \backslash (ij)} \mathbbm{1}_{\left[\sum_l \sigma_{jl} \leq 1 \right]} &= 1 \\
    \sum_{\sigma_{\partial j} \backslash (ij)} \mathbbm{1}_{\left[\sum_{l \backslash (l = i)} \sigma_{jl} + 1 \leq 1 \right]} &= 1
\end{align}

This is saying that all edges adjecent to $i$ and $j$ except edge $(ij)$ have to be zero, meaning non existant.

For $\sigma_{ij} = 0$ we find with similar reasoning:

\begin{align}
    P(\sigma_{ij} = 0) &\cong \hat{\nu}_{\sigma_{ij}}^{i \rightarrow (ij)} \cdot \hat{\nu}_{\sigma_{ij}}^{j \rightarrow (ij)} \\
    &= \left[ \sum_{\sigma_{\partial i} \backslash (ij)} \mathbbm{1}_{\left[\sum_l \sigma_{il} \leq 1 \right]} \prod_{(ik) \in \partial i \backslash (ij)} \nu_{\sigma_{ik}}^{(ik) \rightarrow i} \right]
     \cdot \left[ \sum_{\sigma_{\partial j} \backslash (ij)} \mathbbm{1}_{\left[\sum_l \sigma_{jl} \leq 1 \right]} \prod_{(jk) \in \partial j \backslash (ij)} \nu_{\sigma_{jk}}^{(jk) \rightarrow j} \right]
\end{align}
Which can only be $\geq 0$ if
\begin{align}
    \sum_{\sigma_{\partial i} \backslash (ij)} \mathbbm{1}_{\left[\sum_l \sigma_{il} \leq 1 \right]} &= 1 \\
    \sum_{\sigma_{\partial i} \backslash (ij)} \mathbbm{1}_{\left[\sum_{l \backslash (l = j)} \sigma_{il} + 0 \leq 1 \right]} &= 1 \\
\end{align}
and
\begin{align}
    \sum_{\sigma_{\partial j} \backslash (ij)} \mathbbm{1}_{\left[\sum_l \sigma_{jl} \leq 1 \right]} &= 1 \\
    \sum_{\sigma_{\partial j} \backslash (ij)} \mathbbm{1}_{\left[\sum_{l \backslash (l = i)} \sigma_{jl} + 0 \leq 1 \right]} &= 1
\end{align}

Which is saying that any one, but no more than one of the edges adjecent to $i$ except $(ij)$ may be 1 and that any one, but no more than one of the edges adjecent to $j$ except $(ij)$ may be 1.
If we use $c_i$ and $c_j$ as the count of edges adjecent to $i$ and $j$ except for edge $(ij)$ respectively, we can say that this allows $c_i+1$ configurations for the edges adjecent to $i$ and $c_j + 1$ configurations for the edges adjecent to $j$ (Any one may be 1 or all are 0).

These derivations are intuitively consistent with the definitions of the matching.

{
\color{green}
TODO i don't quite get the following, from the reasoning it makes sense but i don't see how it follows from the formulas

For $\sigma_{ij}=1$, this leads to:
\begin{align}
    P(\sigma_{ij}=1) =  \prod_{\sigma_{ik} \in \partial a \backslash i} v_{\sigma_{ik} \rightarrow i}(0)
\end{align}
wich intuitively makes sense, since all the others have to be 0 in order for $\sigma_{ij}$ to be 1.\\
For $\sigma_{ij}=0$, this leads to:
\begin{align}
    P(\sigma_{ij}=1) = \sum_{\sigma_{ik} \in \partial i \backslash \sigma_{ij}} (\mathbbm{1}^{(\sum_{l \backslash i} \sigma_{il} = 1)} \prod_{\sigma_{ik} \in \partial a \backslash i} v_{\sigma_{ik} \rightarrow i}(\sigma_{ik}))\\
\end{align}
here the main difference is that the factor $\Psi_{i}$ doesn't sum over $\sigma_{ij}$ anymore, since it is set to 0.
}

{
\color{red}
We have the messages from variable to factor:
\begin{align}
    \hat{v}_{i \rightarrow \alpha}(x_i) = \prod_{\beta \in \partial \alpha \backslash i} \hat{v}_{b \rightarrow i}(x_i)
\end{align}
and factor to variable:
\begin{align}
    \hat{v}_{\alpha \rightarrow i}(x_i) = \sum_{x \in \partial \alpha \backslash i} (\Psi_a(X_{\partial a}) \prod_{j \in \partial a \backslash i} v_{j \rightarrow a}(x_j))
\end{align}

The general BP marginal is:
\begin{align}
    P(x_i) = v_i(x_i) = \prod_{a \in \partial i} \hat{v}_{\alpha \rightarrow i}(x_i)
\end{align}

For our problem, this leads to to message from factor to variable:
\begin{align}
    \hat{v}_{i \rightarrow \sigma_{ij}}(\sigma_{ij}) &= \sum_{\sigma_{ik} \in \partial i \backslash \sigma{ij}} (\mathbbm{1}^{(\sum_l \sigma^{il} = 1)} \prod_{\sigma_{ik} \in \partial a \backslash i} v_{\sigma_{ik} \rightarrow i}(\sigma_{ik}))\\
\end{align}

This means the marginal is:
\begin{align}
    P(\sigma_{ij}) = \hat{v}_{i \rightarrow \sigma_{ij}}(\sigma_{ij}) \cdot \hat{v}_{j \rightarrow \sigma_{ij}}(\sigma_{ij})
\end{align}
For $\sigma_{ij}=1$, this leads to:
\begin{align}
    P(\sigma_{ij}=1) =  \prod_{\sigma_{ik} \in \partial a \backslash i} v_{\sigma_{ik} \rightarrow i}(0)
\end{align}
wich intuitively makes sense, since all the others have to be 0 in order for $\sigma_{ij}$ to be 1.\\
For $\sigma_{ij}=0$, this leads to:
\begin{align}
    P(\sigma_{ij}=1) = \sum_{\sigma_{ik} \in \partial i \backslash \sigma_{ij}} (\mathbbm{1}^{(\sum_{l \backslash i} \sigma_{il} = 1)} \prod_{\sigma_{ik} \in \partial a \backslash i} v_{\sigma_{ik} \rightarrow i}(\sigma_{ik}))\\
\end{align}
here the main difference is that the factor $\Psi_{i}$ doesn't sum over $\sigma_{ij}$ anymore, since it is set to 0.

}

\section*{(d)}
{
\color{green}
TODO: To get the full marginal we would need to do the full message passing. For the equations the following suffices:

}

For one point marginal, see (c):
\begin{align}
    P(\sigma_{ij}) &\cong \hat{\nu}_{\sigma_{ij}}^{i \rightarrow (ij)} \cdot \hat{\nu}_{\sigma_{ij}}^{j \rightarrow (ij)}
\end{align}

For two point marginal, we have:
\begin{align}
    P(\sigma_{ij}, \sigma_{kl}) = P(\sigma_{kl} \vert \sigma_{ij}) P(\sigma_{ij})
\end{align}

In addition to the one point marginal we need to  find $P(\sigma_{kl} \vert \sigma_{ij})$ which is defined by:
\begin{align}
    P(\sigma_{kl} \vert \sigma_{ij} = x) &\cong \prod_{a} \psi_a(\boldsymbol{\sigma}_{\partial a}) \mathbbm{1}_{\left[ \sigma_{ij} = x \right]} \\
    &= \prod_{a} \mathbbm{1}_{\left[ \sum_{l} \sigma_{al} \leq 1 \right]} \mathbbm{1}_{\left[ \sigma_{ij} = x \right]}
\end{align}

This is, we add to the original factor graph a node of degree 1 connected to the variable node $\sigma_{ij}$ which fixes $\sigma_{ij} = x$ and run BP on this modified factor graph.

\end{document}


\section{Gale Games}
\label{gale-games-sect}

We now apply Theorem \ref{origgale-thm} to the study of bimatrix games.
By Proposition \ref{nash-to-acl}, solving 2-{\sc Nash} can be reduced to
{\sc Another Completely Labeled Facet}; if the polytope $P^\Delta$ in
Theorem \ref{unit-vector-dual-thm} is cyclic, we can exploit
Theorem~\ref{origgale-thm} to translate this special case of 2-{\sc Nash}
to a problem on Gale strings.

First of all, we have to define a labeling on Gale strings such that a
completely labeled Gale string corresponds to a completely labeled facet
of~$P^\Delta$.
We say that $s\in G(d,n)$ is a {\em completely labeled Gale string}
for some labeling function $l_s:[n]\to[d]$ if
$\{ l_s(i)\,\mid\,s(i)=\1\text{ for }i\in [n] \}=[d]$.
Since a string in $G(d,n)$ has exacty $d$ bits equal to \1, a Gale string
is completely labeled if and only if for each $j\in [d]$ there is exactly
one $i\in [n]$ such that $s(i)=\1$ and $l_s(i)=j$.

Notice that, given a labeling $l_s:[n]\to[d]$, it may not always be
possible to find a Gale string $s\in G(d,n)$ that is completely labeled
by $l_s$.

\begin{example}
\label{no-clgs}
For $l_s = 121314$, there are no completely labeled Gale strings.

The labels $l_s(i)=2,3,4$ appear only once in $l_s$, so we must have
$s(2)=s(4)=s(6)=1$. We also must have $l_s(i)=1$ for exactly one $i=1,3,5$.
The candidate strings are then $s_1=\1\10\10\1$, $s_2=0\1\1\10\1$,
$s_3=0\10\1\1\1$, but none of these satisfies the Gale Evenness Condition.
\end{example}

A {\em Gale game} is a unit vector game $(U,B)$ where
$U=[e_{l(1)}\cdots e_{l(d)}]$ for some labeling $l:[n]\to [d]$ and
for which the dual of the best response polytope is a cyclic polytope
$P^\Delta=\conv\{ e_1,\ldots,e_d,c_1,\ldots,c_n \}$.
We define the problem {\sc Gale Nash} as in Table~\ref{gale-nash-pbl}

\begin{problem}
{Gale Nash}
{A Gale game.}
{A Nash equilibrium of the game.}
\label{gale-nash-pbl}
\end{problem}

Theorem
\ref{unit-vector-dual-thm} gives the labeling (\ref{vert-labeling-unitv})
for the $d+n$ vertices of $P^\Delta$ as
\[
\arraycolsep.2em
\begin{array}{rcll}
l_v(-e_i)&=&i\quad & \text{ for }i\in [d],\\
l_v(c_j)&=&l(j)\quad & \text{ for }j\in [n].
\end{array}
\]
We define the labeling $l_s:[d+n]\to [d]$ of $G(d,n)$ as
\begin{equation}
\label{gs-labeling-unitv}
\arraycolsep.2em
\begin{array}{rcll}
l_s(i)&=&i\quad & \text{ for }i\in [d],\\
l_s(d+j)&=&l(j)\quad & \text{ for }j\in [n].
\end{array}
\end{equation}
Then the Gale strings $s\in G(d,d+n)$ that are completely labeled by $l_s$
correspond exactly to facets of $P^\Delta$ that are completely labeled by $l_v$,
with the facet $F_0$ corresponding to the ``trivial'' completely labeled
string $\1^d 0^n$.

\begin{example}
\label{c46-123432-ex}
Given the string of labels $l_s=123432$, there are four associated completely
labeled Gale strings in $G(4,6)$:
$s_A=\1\1\1\100$, $s_B=\1\10\1\10$,
$s_C=\100\1\1\1$ and
$s_D=\10\1\10\1$. These correspond to the completely labeled facets
for the labeling shown in Figure \ref{c46-123432-fig} on the left.
\begin{figure}[hbt]
\strut\hfill
\includegraphics[width=50ex]{chapter-2/fig-gale-games/123432-facets.pdf}%
\hfill
\small
\begin{tabular}{c | c @{ } c @{ } c @{ } c @{ } c @{ } c @{ } c }
facet & {\bf 1} & {\bf 2} & {\bf 3} & {\bf 4} & {\bf 3} & {\bf 2}\\
\hline
{\bf\it{A}} & \1 & \1 & \1 & \1 & 0 & 0 \\
{\bf\it{B}} & \1 & \1 & 0 & \1 & \1 & 0 \\
{\bf\it{C}} & \1 & 0 & 0 & \1 & \1 & \1 \\
{\bf\it{D}} & \1 & 0 & \1 & \1 & 0 & \1
\end{tabular}
\hfill\strut
\caption[A labeling of $C_4(6)$ and its completely labeled facets]{%
The cyclic polytope $C_4(6)$, where the labeling of the vertices
corresponds to the labeling of $G(4,6)$ given by $l_s=123432$.

The
completely labeled
facets $A$, $B$, $C$ and $D$ correspond respectively to the completely Gale
strings  $s_A=\1\1\1\100$, $s_B=\1\10\1\10$, $s_C=\100\1\1\1$ and
$s_D=\10\1\10\1$.
}
\label{c46-123432-fig}
\end{figure}
\end{example}

From this point forward, we will assume that the labeling
$l_s:[d+n]\to [d]$ satisfies
$l_s(i)\neq l_s(i+1)$. This can be done without loss of generality,
given the following consideration.
Suppose that $l_s(i)=l_s(i + 1)$ for some index $i$, and let $s$ be a
completely labeled Gale string for $l_s$. Then only one of
$s(i)$ and $s(i+1)$ can be equal to \1 (it is possible that both
are equal to~0), so $s(i)s(i+1)$ will never be part of a run of even length
that ``interferes'' with the Gale Evenness Condition. Therefore, we can
identify the indices $i$ and $i + 1$.

We can now define the problem \anothergale\ as in Table
\ref{another-gale}. For brevity, we change the notation from $d+n$ to $n$.

\begin{problem}
{\anothergale}
{A labeling $l:[n]\to[d]$, where $d<n$.
A Gale string $s\in G(d,n)$, completely labeled by $l$.}
{A Gale string $s_0\in G(d,n)$, completely labeled by $l$,
such that $s_0 \neq s$.}
\label{another-gale}
\end{problem}

It takes polynomial time to translate the facets of the cyclic polytope
$C_d(d+n)$ % can be translated in polynomial time
into the corresponding Gale strings in $G(d,d+n)$, following
the proof of Theorem~\ref{origgale-thm}.
Defining the labeling $l_s$ from the labeling $l_v$ also takes
polynomial time: for the labels $i\in [d]$ it is immediate, for the labels
$d+j$, where $j\in [n]$, we have to check the $d\times n$ matrix $U$
of the imitation game.
Therefore, by Proposition \ref{nash-to-acl}, we have a reduction from
{\sc Gale Nash} to \anothergale.

\begin{proposition}
\label{galenash-to-another-gale}
The problem {\sc Gale Nash} of Table~\ref{gale-nash-pbl} is
polynomial-time reducible to the problem \anothergale\ of
Table~\ref{another-gale}.
\end{proposition}

Proposition~\ref{galenash-to-another-gale} can be improved: it is enough to
consider the case where $d$ is even.

\begin{proposition}
\label{d-even-another-gale}
The problem \anothergale\ of Table~\ref{another-gale} is reducible to
the case where $d$ is even.
\begin{proof}
Consider an instance of the problem \anothergale\ with $d$ odd.

Let $s_0'\in G(d+1,n+1)$ be the string defined as
\[
\arraycolsep.2em
\begin{array}{ll}
s_0'(i)=s_0(i) & \text{ for }i\in [n],\\
s_0'(n+1)=1.\quad &
\end{array}
\]
% $s_0'(i)=s_0(i)$ for $i\in [n]$ and $s_0'(n+1)=1$.
This is indeed a Gale string. It is trivial to see that there are
exactly $d+1$ bits equal to \1; furthermore, since $s_0$ is a Gale string,
the Gale Evenness Condition holds in all the interior runs of $s_0'$.
Let now $l':[n+1]\to[d+1]$ be the labeling defined as
\begin{equation}
\label{extended-labeling}
\arraycolsep.2em
\begin{array}{ll}
l'(i)=l(i) & \text{ for }i\in [n],\\
l'(n+1)=d+1.\quad &
\end{array}
\end{equation}
Notice that $s_0'$ is completely labeled by $l'$, since for every
for each $j\in [d]$ there is exactly one $i\in [n]$ such that
$s_0'(i)=s_0(i)=\1$ and $l_s(i)=j$, and the only occurrence of the label
$d+1$ is at index $n+1$, where $s_0'(n+1)=\1$.

Let $s'$ be any bitstring of length $n+1$ such that $s'(n+1)=1$, and let
$s$ be the bitstring of length $n$ such that $s(i)=s'(i)$ for $i\in [n]$.
First of all, notice that if $s'\in G(d+1,n+1)$ then $s\in G(d,n)$:
it is obtained by removing a bit that is equal to \1, and the Gale
Evenness Condition still holds in all the interior runs.
Furthermore, if $s'$ is completely labeled for $l'$, then $s'(n+1)=1$,
since the only occurrence of label $d+1$ is at index $n+1$ and for all
the other labels $j\in [d]$ there is exactly one $i\in [n]$ such that
$s(i)=\1$ and $l_s(i)=j$.

Therefore, a solution for the original instance of \anothergale\ can be
found by solving the problem \anothergale\ with input $s_0'$ and $l'$ and
output $s'$, then considering the corresponding completely labeled Gale
string $s\in G(d,n)$ defined as above.
\end{proof}
\end{proposition}