\lab{Iterative Solvers}{Iterative Solvers}
\label{lab:iter_methods}

\objective{
Many real-world problems of the form $A\x =\b$ have tens of thousands of parameters.
Solving such systems with Gaussian elimination or matrix factorizations could require trillions of floating point operations (FLOPs), which is of course infeasible.
Solutions of large systems must therefore be approximated iteratively.
In this lab, we implement three popular iterative methods for solving large systems: Jacobi, Gauss-Seidel, and Successive Over-Relaxation.}

% Though finding an exact solution is either very time intensive or unstable, iterative methods give a sufficiently close approximations and take much less time.

Iterative methods are often useful to solve large systems of equations.
In this lab, let $\x^{(k)}$ denote the $k$th iteration of the iterative method for solving the problem $A\x=\b$ for $\x$.
Furthermore, let $x_{i}$ be the $i$th component of $\x$ so that $x_{i}^{(k)}$ is the $i$th component of $\x$ in the $k$th iteration.
Like other iterative methods, there are two stopping parameters: a very small $\epsilon > 0$ and an integer $N\in\mathbb{N}$.
Iterations continue until either
\begin{equation} % Stopping Criteria.
\|\x^{(k-1)}-\x^{(k)}\| < \epsilon
\qquad \text{or} \qquad
k > N.
\label{eq:itersolve-stopping-criteria}
\end{equation}

\section*{The Jacobi Method} % ================================================

The \emph{Jacobi Method} is a simple but powerful method used for solving certain kinds of large linear systems.
The main idea is simple: solve for each variable in terms of the others, then use the previous values to update each approximation.
As a (very small) example, consider the $3 \times 3$
\begin{align*}
\begin{array}{ccccccr}
  2x_1 &   &      & - & x_3  & = & 3,  \\
  -x_1 & + & 3x_2 & + & 2x_3 & = & 3,  \\
       & + & x_2  & + & 3x_3 & = & -1. \\
\end{array}
\end{align*}

Solving the first equation for $x_1$, the second for $x_2$, and the third for $x_3$ yields
\begin{align*}
\begin{array}{ccc}
    x_1 & = & \frac{1}{2}(3 + x_3), \\
    x_2 & = & \frac{1}{3}(3 + x_1 - 2x_3), \\
    x_3 & = & \frac{1}{3}(-1 - x_2).
\end{array}
\end{align*}

Now begin with an initial guess $\x^{(0)} = [x^{(0)}_1, x^{(0)}_2, x^{(0)}_3]\trp = [0,0,0]\trp$.
To compute the first approximation $\x^{(1)}$, use the entries of $\x^{(0)}$ as the variables on the right side of the previous equations:
\begin{align*}
\begin{array}{ccccccr}
    x^{(1)}_1 & = & \frac{1}{2}(3 + x^{(0)}_3) & = & \frac{1}{2} (3 + 0) & = & \frac{3}{2}, \\
    x^{(1)}_2 & = & \frac{1}{3}(3 + x^{(0)}_1 - 2x^{(0)}_3) & = & \frac{1}{3} (3 + 0 - 0) & = & 1, \\
    x^{(1)}_3 & = & \frac{1}{3}(-1 - x^{(0)}_2) & = & \frac{1}{3} (-1 - 0) & = & -\frac{1}{3}. \\
\end{array}
\end{align*}

Thus $\mathbf{x}^{(1)} = [\frac{3}{2}, 1, -\frac{1}{3}]\trp$.
Computing $\mathbf{x}^{(2)}$ is similar:
\begin{align*}
\begin{array}{ccccccc}
x^{(2)}_1 & = & \frac{1}{2} ( 3 + x^{(1)}_3)  & = & \frac{1}{2} (3 - \frac{1}{3})     & = & \frac{4}{3}, \\
x^{(2)}_2 & = & \frac{1}{3} ( 3 + x^{(1)}_1 - 2x^{(1)}_3) & = & \frac{1}{3} (3 + \frac{3}{2} + \frac{2}{3}) & = &  \frac{31}{18}, \\
x^{(2)}_3 & = & \frac{1}{3} ( -1 - x^{(1)}_2)       & = & \frac{1}{3} (-1 - 1)    & = & -\frac{2}{3}. \\
\end{array}
\end{align*}

The process is repeated until at least one of the two stopping criteria in (\ref{eq:itersolve-stopping-criteria}) is met.
For this particular problem, convergence to 8 decimal places ($\epsilon = 10^{-8}$) is reached in 29 iterations.

\begin{center}
\begin{tabular}{c|ccc}
    & $x^{(k)}_1$ & $x^{(k)}_2$ & $x^{(k)}_3$ \\
    \hline
    $\x^{(0)}$ & $0$ & $0$ & $0$ \\
    %\hline
    $\x^{(1)}$ & $1.5$ & $1$ & $-0.33333333$ \\
    %\hline
    $\x^{(2)}$  & $1.33333333$ & $1.72222222$ & $-0.6666666$7 \\
    $\x^{(3)}$  & $1.16666667$ & $1.88888889$ & $-0.9074074$1 \\
    $\x^{(4)}$  & $1.04629630$ & $1.99382716$ & $-0.9629629$6 \\
    \vdots      & \vdots       & \vdots       & \vdots      \\
    $\x^{(28)}$ & $0.99999999$ & $2.00000001$ & $-0.9999999$9 \\
    $\x^{(29)}$ & $1$          & $2$          & $-1$          \\
\end{tabular}
\end{center}

\subsection*{Matrix Representation} % -----------------------------------------

The iterative steps performed above can be expressed in matrix form.
First, decompose $A$ into its diagonal entries, its entries below the diagonal, and its entries above the diagonal, as $A = D + L + U$.
\begin{align*}
\begin{array}{ccccc}
    \left[\begin{array}{cccc}
        a_{11} & 0 & \ldots & 0 \\
        0 & a_{22} & \ldots & 0 \\
        \vdots & \vdots & \ddots & \vdots \\
        0 & 0 & \ldots & a_{nn} \\
    \end{array}\right]
    & &
    \left[\begin{array}{cccc}
    0 & 0 & \ldots & 0 \\
    a_{21} &  0 & \ldots & 0\\
     \vdots & \ddots & \ddots & \vdots \\
    a_{n1} & \ldots & a_{n,n-1} & 0 \\
    \end{array}\right]
    & &
    \left[\begin{array}{cccc}
    0 & a_{12} & \ldots & a_{1n} \\
    0 & 0 & \ddots & \vdots \\
     \vdots & \vdots & \ddots & a_{n-1,n} \\
    0 & 0 & \ldots & 0 \\
    \end{array}\right]
    \\D & & L & & U
\end{array}
\end{align*}

With this decomposition, $\x$ can be expressed in the following way.
\begin{align*}
A\x &= \b\\
(D + L + U)\x &= \b\\
D\x &= -(L+U)\x + \b\\
\x &= D^{-1}(-(L+U)\x + \b)
\end{align*}

Now using $\x^{(k)}$ as the variables on the right side of the equation to produce $\x^{(k+1)}$ on the left, and noting that $L+U=A-D$, we have the following.
\begin{align}
\nonumber \x^{(k+1)} &= D^{-1}(-(A-D)\x^{(k)} + \b) \\
\nonumber &= D^{-1}(D\x^{(k)} - A\x^{(k)}  + \b)\\
&= \x^{(k)} + D^{-1}(\b - A\x^{(k)})
\label{eq:jacobi-method}
\end{align}

There is a potential problem with (\ref{eq:jacobi-method}): calculating a matrix inverse is the cardinal sin of numerical linear algebra, yet the equation contains $D^{-1}$.
However, since $D$ is a diagonal matrix, $D^{-1}$ is also diagonal, and is easy to compute.

\[
D^{-1} =
\left[\begin{array}{cccc}
    \frac{1}{a_{11}} & 0                & \ldots & 0      \\
    0                & \frac{1}{a_{22}} & \ldots & 0      \\
    \vdots           & \vdots           & \ddots & \vdots \\
    0                & 0                & \ldots & \frac{1}{a_{nn}}
\end{array}\right]
\]
Because of this, the Jacobi method requires that $A$ have nonzero diagonal entries.

The diagonal $D$ can be represented by the 1-dimensional array $\mathbf{d}$ of the diagonal entries. %(instead of as a 2-dimensional array).
Then the matrix multiplication $D\x$ is equivalent to the component-wise vector multiplication $\mathbf{d}*\x = \x*\mathbf{d}$.
Likewise, the matrix multiplication $D^{-1}\x$ is equivalent to the component-wise ``vector division'' $\x/\mathbf{d}$.

\begin{comment}% This didn't seem super useful so I took it out.
\begin{lstlisting}
>>> import numpy as np

>>> D = np.array([[2,0],[0,16]])    # Let D be a diagonal matrix.
>>> d = np.diag(D)                  # Extract the diagonal as a 1-D array.
>>> x = np.random.random(2)
>>> np.allclose(D.dot(x), d*x)
<<True>>
\end{lstlisting}
\end{comment}

\begin{comment} % Jacobi algorithm box. Probably unnecessary.
\begin{algorithm}[H]
\begin{algorithmic}[1]
\Procedure{Jacobi Method}{$A$, $\b$, $\epsilon$, $N$}
    \State $m, n \gets \shape{A}$
        \Comment{Store the dimensions of $A$.}
    \State $\mathbf{d} \gets$ diag($A$)
        \Comment{Get the diagonal entries of $A$ with \li{np.diag()}.}
    \State $\x^{(0)} \gets \zeros{n}$
        \Comment{An array of $n$ zeros.}
    \For{$k=0 \ldots N-1$}
        \State $\x^{(k+1)} \gets (\b - A\x^{(k)} + \mathbf{d}*\x^{(k)})/\mathbf{d}$
        \If{$\|\mathbf{x}^{(k-1)} - \mathbf{x}^{(k)}\|_{\infty} < \epsilon$}
            \State \pseudoli{break}
        \EndIf
    \EndFor
    \State \pseudoli{return} $\x^{(k)}$
\EndProcedure
\end{algorithmic}
\caption{}
\label{alg:jacobi-method}
\end{algorithm}
\end{comment}

\begin{problem} % Implement the Jacobi Method.
Write a function that accepts a matrix $A$, a vector $\b$, a convergence tolerance \li{tol} defaulting to $10^{-8}$, and a maximum number of iterations \li{maxiter} defaulting to $100$.
Implement the Jacobi method using (\ref{eq:jacobi-method}), returning the approximate solution to the equation $A\x = \b$.

Run the iteration until $\|\mathbf{x}^{(k-1)} - \mathbf{x}^{(k)}\|_{\infty} <$ \li{tol}, and only iterate at most \li{maxiter} times.
Avoid using \li{la.inv()} to calculate $D^{-1}$, but use \li{la.norm()} to calculate the vector $\infty$-norm.

% \begin{lstlisting}
% >>> from scipy import linalg as la
%
% >>> x = np.random.random(10)
% >>> la.norm(x, <<ord>>=np.inf)          # Use la.norm() for ||x||.
% 0.74623726404168045
% >>> np.<<max>>(np.<<abs>>(x))               # The equivalent in NumPy for ||x||.
% 0.74623726404168045
% \end{lstlisting}

Your function should be robust enough to accept systems of any size.
To test your function, generate a random $\b$ with \li{np.random.random()} and use the following function to generate an $n\times n$ matrix $A$ for which the Jacobi method is guaranteed to converge.
Run the iteration, then check that $A\x^{(k)}$ and $\b$ are close using \li{np.allclose()}.

\begin{lstlisting}
def diag_dom(n, num_entries=None):
    """Generate a strictly diagonally dominant (n, n) matrix.
    Parameters:
        n (int): The dimension of the system.
        num_entries (int): The number of nonzero values.
            Defaults to n^(3/2)-n.
    Returns:
        A ((n,n) ndarray): A (n, n) strictly diagonally dominant matrix.
    """
    if num_entries is None:
        num_entries = int(n**1.5) - n
    A = np.zeros((n,n))
    rows = np.random.choice(np.arange(0,n), size=num_entries)
    cols = np.random.choice(np.arange(0,n), size=num_entries)
    data = np.random.randint(-4, 4, size=num_entries)
    for i in range(num_entries):
        A[rows[i], cols[i]] = data[i]
    for i in range(n):
        A[i,i] = np.<<sum>>(np.<<abs>>(A[i])) + 1
    return A
\end{lstlisting}

Also test your function on random $n \times n$ matrices.
If the iteration is non-convergent, the successive approximations will have increasingly large entries.

\label{prob:jacobi}
\end{problem}

\subsection*{Convergence} % ---------------------------------------------------

Most iterative methods only converge under certain conditions.
For the Jacobi method, convergence mostly depends on the nature of the matrix $A$.
If the entries $a_{ij}$ of $A$ satisfy the property
\[
|a_{ii}| > \sum_{j \neq i} |a_{ij}|\quad \text{for all}\quad i = 1,2,\ldots,n,
\]
then $A$ is called \emph{strictly diagonally dominant} (\li{diag_dom()} in Problem \ref{prob:jacobi} generates a strictly diagonally dominant $n\times n$ matrix).
If this is the case,\footnote{Although this seems like a strong requirement, most real-world linear systems can be represented by strictly diagonally dominant matrices.} then the Jacobi method always converges, regardless of the initial guess $\x_0$.
This is a very different convergence result than many other iterative methods such as Newton's method where convergence is highly sensitive to the initial guess.

There are a few ways to determine whether or not an iterative method is converging.
For example, since the approximation $\x^{(k)}$ should satisfy $A\x^{(k)} \approx \b$, the normed difference $\|A\x^{(k)} - \b\|_\infty$ should be small.
This value is called the \emph{absolute error} of the approximation.
If the iterative method converges, the absolute error should decrease to $\epsilon$.

\begin{problem}
Modify your Jacobi method function in the following ways.
\begin{enumerate}
    \item Add a keyword argument called \li{plot}, defaulting to \li{False}.
    \item Keep track of the absolute error $\|A\mathbf{x}^{(k)} - \mathbf{b}\|_{\infty}$ of the approximation at each iteration.
    \item If \li{plot} is {True}, produce a lin-log plot (use \li{plt.semilogy()}) of the error against iteration count.
    Remember to still return the approximate solution $\x$.
\end{enumerate}
If the iteration converges, your plot should resemble the following figure.

\begin{figure}[H]
    \includegraphics[width=.7\textwidth]{figures/jacobi_convergence.pdf}
\end{figure}

\label{prob:plot-iterative-convergence}
\end{problem}

\section*{The Gauss-Seidel Method} % ==========================================

The \emph{Gauss-Seidel method} is essentially a slight modification of the Jacobi method.
The main difference is that in Gauss-Seidel, new information is used immediately.
As an example, consider again the system from the previous section,
\begin{align*}
\begin{array}{ccccccr}
  2x_1 &   &      & - & x_3  & = & 3,  \\
  -x_1 & + & 3x_2 & + & 2x_3 & = & 3,  \\
       & + & x_2  & + & 3x_3 & = & -1. \\
\end{array}
\end{align*}

As with the Jacobi method, solve for $x_1$ in the first equation, $x_2$ in the second equation, and $x_3$ in the third equation:
\begin{align*}
\begin{array}{ccc}
    x_1 & = & \frac{1}{2}(3 + x_3), \\
    x_2 & = & \frac{1}{3}(3 + x_1 - 2x_3), \\
    x_3 & = & \frac{1}{3}(-1 - x_2).
\end{array}
\end{align*}
Using $\x^{(0)}$ to compute $x^{(1)}_1$ in the first equation as before,
\[x^{(1)}_1 = \frac{1}{2}(3 + x^{(0)}_3) = \frac{1}{2}(3 + 0) = \frac{3}{2}.\]
Now, however, use the updated value of $x^{(1)}_1$ in the calculation of $x^{(1)}_2$:
\[x^{(1)}_2 = \frac{1}{3}(3 + x^{(1)}_1 - 2x^{(0)}_3) = \frac{1}{3}(3 + \frac{3}{2} - 0) = \frac{3}{2}.\]
Likewise, use the updated values of $x^{(1)}_1$ and $x^{(1)}_2$ to calculate $x^{(1)}_3$:
\[x^{(1)}_3 = \frac{1}{3}(-1 - x^{(1)}_2) = \frac{1}{3}(-1 - \frac{3}{2}) = -\frac{5}{6}.\]
This process of using calculated information immediately is called \emph{forward substitution}, and causes the algorithm to (generally) converge much faster.

\begin{center}
\begin{tabular}{c|ccc}
    & $x^{(k)}_1$ & $x^{(k)}_2$ & $x^{(k)}_3$ \\
    \hline
      $x^{(0)}$ & $0$ & $0$ & $0$ \\
      %\hline
      $x^{(1)}$ & $1.5$ & $1.5$ & $-0.83333333$ \\
      %\hline
      $x^{(2)}$ & $1.08333333$ & $1.91666667$ & $-0.97222222$ \\
      $x^{(3)}$ & $1.01388889$ & $1.98611111$ & $-0.99537037$ \\
      $x^{(4)}$ & $1.00231481$ & $1.99768519$ & $-0.99922840$ \\
      \vdots    & \vdots       & \vdots       & \vdots     \\
      $x^{(11)}$ & $1.00000001$ & $1.99999999$ & $-1$ \\
      $x^{(12)}$ & $1$ & $2$ & $-1$ \\
\end{tabular}
\end{center}
Notice that Gauss-Seidel converges in less than half as many iterations as Jacobi does for this system.

\subsection*{Implementation} % ------------------------------------------------

% This matches the book (Ch. 13), but it may be confusing to the students here.
% We begin by solving for $\x$ in a slightly different way.
% \begin{align}
% \nonumber A\x &= \b\\
% \nonumber (D + L + U)\x &= \b\\
% \nonumber (D + L)\x &= -U\x + \b\\
% \x &= -(D + L)^{-1}(U\x + \b)
% \label{eq:gauss-seidel-method}
% \end{align}

Because Gauss-Seidel updates only one element of the solution vector at a time, the iteration cannot be summarized by a single matrix equation.
Instead, the process is most generally described by the equation
\begin{equation}
\label{eq:gauss-seidel-full}
x^{(k+1)}_i = \frac{1}{a_{ii}} \left(b_i - \sum_{j < i}a_{ij}x^{(k)}_j - \sum_{j > i}a_{ij}x^{(k)}_j \right).
\end{equation}

Let $\a_i$ be the $i$th \textbf{row} of $A$.
The two sums closely resemble the regular vector product of $\a_i$ and $\x^{(k)}$ without the $i^{\text{th}}$ term $a_{ii}x^{(k)}_i$.
This suggests the simplification
\begin{align}
\nonumber x^{(k+1)}_i &= \frac{1}{a_{ii}} \left(b_i - \a_i\trp\x^{(k)} + a_{ii}x^{(k)}_i \right)\\
&= x^{(k)}_i + \frac{1}{a_{ii}}\left( b_i - \a_i\trp\x^{(k)}\right).
\label{eq:gauss_seidel}
\end{align}
One sweep through all the entries of $\x$ completes one iteration.

\begin{problem} % Implement Gauss-Seidel.
Write a function that accepts a matrix $A$, a vector $\b$, a convergence tolerance \li{tol} defaulting to $10^{-8}$, a maximum number of iterations \li{maxiter} defaulting to $100$, and a keyword argument \li{plot} that defaults to \li{False}.
Implement the Gauss-Seidel method using (\ref{eq:gauss_seidel}), returning the approximate solution to the equation $A\x = \b$.

Use the same stopping criterion as in Problem \ref{prob:jacobi}.
Also keep track of the absolute errors of the iteration, as in Problem \ref{prob:plot-iterative-convergence}.
If \li{plot} is \li{True}, plot the error against iteration count.
Use \li{diag_dom()} to generate test cases.

\begin{warn} % Arrays are mutable!
Since the Gauss-Seidel algorithm operates on the approximation vector in place (modifying it one entry at a time), the previous approximation $\x^{(k-1)}$ must be stored at the beginning of the $k$th iteration in order to calculate $\|\x^{(k-1)} - \x^{(k)}\|_\infty$.
Additionally, since NumPy arrays are mutable, the past iteration must be stored as a \textbf{copy}.

\begin{lstlisting}
>>> x0 = np.random.random(5)        # Generate a random vector.
>>> x1 = x0                         # Attempt to make a copy.
>>> x1[3] = 1000                    # Modify the "copy" in place.
>>> np.allclose(x0, x1)             # But x0 was also changed!
<<True>>

# Instead, make a copy of x0 when creating x1.
>>> x0 = np.copy(x1)                # Make a copy.
>>> x1[3] = -1000
>>> np.allclose(x0, x1)
<<False>>
\end{lstlisting}
% It is good practice in most iterative methods to record a copy of $\x^{(k-1)}$ at the beginning of the $k$th iteration.
\end{warn}

\label{prob:gauss_seidel}
\end{problem}

\subsection*{Convergence} % ---------------------------------------------------

Whether or not the Gauss-Seidel method converges depends on the nature of $A$.
If all of the eigenvalues of $A$ are positive, $A$ is called \emph{positive definite}.
If $A$ is positive definite \emph{or} if it is strictly diagonally dominant, then the Gauss-Seidel method converges regardless of the initial guess $\x^{(0)}$.

\begin{comment} % This problem does not give good results.
\begin{problem} % Sparse implementation of Gauss Seidel.
The Gauss-Seidel method is faster than the standard system solver used by SciPy's \li{la.solve()} if the system is sufficiently large and sufficiently sparse.
For $n=5,6,\ldots,11$, generate a random $2^n \times 2^n$ matrix $A$ using \li{diag_dom()} and a random $2^n$ vector $\b$.
Time how long it takes to solve $A\x = \b$ using your Gauss-Seidel function from Problem \ref{prob:gauss_seidel}, and how long it takes to solve using \li{la.solve()}.
Make a log-log plot of the times against the system size.
\end{problem}
\end{comment}

\section*{Solving Sparse Systems Iteratively} % ===============================

Iterative solvers are best suited for solving very large sparse systems.
However, using the Gauss-Seidel method on sparse matrices requires translating code from NumPy to \li{scipy.sparse}.
The algorithm is the same, but there are some functions that are named differently between these two packages.%
\footnote{See the lab on Linear Systems for a review of \lif{scipy.sparse} matrices and syntax.}

\begin{problem} % Gauss-Seidel with Sparse matrices.
Write a new function that accepts a \textbf{sparse} matrix $A$, a vector $\b$, a convergence tolerance \li{tol}, and a maximum number of iterations \li{maxiter} (plotting the convergence is not required for this problem).
Implement the Gauss-Seidel method using (\ref{eq:gauss_seidel}), returning the approximate solution to the equation $A\x = \b$.
Use the usual default stopping criterion.

The Gauss-Seidel method requires extracting the rows $A_i$ from the matrix $A$ and computing $A_i\trp\x$.
There are many ways to do this that cause some fairly serious runtime issues, so we provide the code for this specific portion of the algorithm.

\begin{lstlisting}
# Get the indices of where the i-th row of A starts and ends if the
# nonzero entries of A were flattened.
rowstart = A.indptr[i]
rowend = A.indptr[i+1]

# Multiply only the nonzero elements of the i-th row of A with the
# corresponding elements of x.
Aix = A.data[rowstart:rowend] @ x[A.indices[rowstart:rowend]]
\end{lstlisting}

To test your function, cast the result of \li{diag_dom()} as a sparse matrix.

\begin{lstlisting}
>>> from scipy import sparse

>>> A = sparse.csr_matrix(diag_dom(50000))
>>> b = np.random.random(50000)
\end{lstlisting}
\end{problem}

\subsection*{Successive Over-Relaxation} % ------------------------------------

There are many systems that meet the requirements for convergence with the Gauss-Seidel method, but for which convergence is still relatively slow.
A slightly altered version of the Gauss-Seidel method, called \emph{Successive Over-Relaxation} (SOR), can result in faster convergence.
This is achieved by introducing a \emph{relaxation factor} $\omega \ge 1$ and modifying (\ref{eq:gauss-seidel-full}) as
\[
x^{(k+1)}_i = (1-\omega)x^{(k)}_i + \frac{\omega}{a_{ii}} \left(b_i - \sum_{j < i}a_{ij}x^{(k)}_j - \sum_{j > i}a_{ij}x^{(k)}_j \right).
\]
Simplifying the equation, we have
\begin{equation}
x^{(k+1)}_i = x^{(k)}_i + \frac{\omega}{a_{ii}}\left(b_i - \a_i\trp\x^{(k)}\right).
\label{eq:sor}
\end{equation}
Note that when $\omega = 1$, SOR reduces to Gauss-Seidel.
The relaxation factor $\omega$ weights the new iteration between the current best approximation and the next approximation in a way that can sometimes dramatically improve convergence.

\begin{problem} % SOR
Write a function that accepts a sparse matrix $A$, a vector $\b$, a relaxation factor $\omega$, a convergence tolerance \li{tol}, and a maximum number of iterations \li{maxiter}.
Implement SOR using (\ref{eq:sor}), compute the approximate solution to the equation $A\x = \b$.
Use the usual stopping criterion.
Return the approximate solution $\x$ as well as a boolean indicating whether the function converged and the number of iterations computed.
\\ (Hint: this requires changing only one line of code from the sparse Gauss-Seidel function.)
\label{prob:sor}
\end{problem}

\section*{A Finite Difference Method} % =======================================

\emph{Laplace's equation} is an important partial differential equation that arises often in both pure and applied mathematics.
In two dimensions, the equation has the following form.
\begin{equation}
\frac{\partial^2 u}{\partial x^2} + \frac{\partial^2 u}{\partial y^2} = 0
\label{eq:laplaces-equation}
\end{equation}

Laplace's equation can be used to model heat flow.
Consider a square metal plate where the top and bottom borders are fixed at $0^\circ$ Celsius and the left and right sides are fixed at $100^\circ$ Celsius.
Given these boundary conditions, we want to describe how heat diffuses through the rest of the plate.
The solution to Laplace's equation describes the plate when it is in a \emph{steady state}, meaning that the heat at a given part of the plate no longer changes with time.

It is possible to solve (\ref{eq:laplaces-equation}) analytically.
However, the problem can also be solved numerically using a \emph{finite difference method}.
To begin, we impose a discrete, square grid on the plate with uniform spacing.
% (see Figure \ref{fig:itersolve-finite-difference-grid}).
Denote the points on the grid by $(x_i,y_j)$ and the value of $u$ at these points (the heat) as $u(x_i,y_j) = U_{i,j}$.
Using the centered difference quotient for second derivatives to approximate the partial derivatives,
%
\begin{align}
\nonumber 0 &= \frac{\partial^2 u}{\partial x^2}+ \frac{\partial^2 u}{\partial y^2}\\
\nonumber &\approx \frac{U_{i+1,j}-2U_{i,j}+U_{i-1,j}}{h^2} + \frac{U_{i,j+1}-2U_{i,j}+U_{i,j-1}}{h^2}\\
&= \frac{1}{h^2}\left(-4U_{i,j} + U_{i+1,j} + U_{i-1,j} + U_{i,j+1} +  U_{i,j-1}\right),
\label{eq:laplace-finite-diff}
\end{align}
where $h = x_{i+1} - x_i = y_{j+1} - y_j$ is the distance between the grid points in either direction.
%
\begin{figure}
\centering
\begin{tikzpicture}
\begin{scope}[xshift=-3cm]
\foreach \x/\y in {-1.2/0,-.4/0,.4/0,1.2/0,-1.2/-.8,-.4/-.8,.4/-.8,1.2/-.8,-1.2/-1.6,-.4/-1.6,.4/-1.6,1.2/-1.6,-1.2/-2.4,-.4/-2.4,.4/-2.4,1.2/-2.4} \draw (\x,\y) circle (3pt);
\foreach \x/\y/\c in {-1.2/.8/blue,-.4/.8/blue,.4/.8/blue,1.2/.8/blue,-1.2/-3.2/blue,-.4/-3.2/blue,.4/-3.2/blue,1.2/-3.2/blue,-2/0/red,-2/-.8/red,-2/-1.6/red,-2/-2.4/red,2/0/red,2/-.8/red,2/-1.6/red,2/-2.4/red} \filldraw[\c] (\x,\y) circle (3pt);
\end{scope}
\begin{scope}[xshift=3cm]
\foreach \x/\y in {-1.2/0,-.4/0,.4/0,1.2/0,-1.2/-.8,-.4/-.8,.4/-.8,1.2/-.8,-1.2/-1.6,-.4/-1.6,1.2/-1.6,-1.2/-2.4} \draw (\x,\y) circle (3pt);
\foreach \x/\y/\c in {-1.2/.8/blue,-.4/.8/blue,.4/.8/blue,1.2/.8/blue,-1.2/-3.2/blue,-.4/-3.2/blue,1.2/-3.2/blue,-2/0/red,-2/-.8/red,-2/-1.6/red,-2/-2.4/red,2/0/red,2/-.8/red,2/-1.6/red,2/-2.4/red} \filldraw[\c] (\x,\y) circle (3pt);
\foreach \x/\y/\c/\i in {.4/-1.6/green/1,-.4/-2.4/green/2,.4/-2.4/black/3,1.2/-2.4/green/4,.4/-3.2/green/5} \node[circle,draw,fill=\c,inner sep=2.25pt] (\i) at (\x,\y) {};
\foreach \a in {1,2,4,5} \draw[->,>=stealth',thick] (\a) -- (3);
\end{scope}
\end{tikzpicture}
\caption{On the left, an example of a $6\times 6$ grid ($n=4$) where the red dots are hot boundary zones and the blue dots are cold boundary zones.
On the right, the green dots are the neighbors of the interior black dot that are used to approximate the heat at the black dot.}
\label{fig:itersolve-finite-difference-grid}
\end{figure}
%
This problem can be formulated as a linear system.
Suppose the grid has exactly $(n+2)\times (n+2)$ entries.
Then the interior of the grid (where $u(x,y)$ is unknown) is $n\times n$, and can be flattened into an $n^2\times 1$ vector $\u$.
The entire first row goes first, then the second row, proceeding to the $n$th row.
\[\u = \begin{bmatrix}U_{1,1} & U_{1,2} & \cdots & U_{1,n} & U_{2,1} & U_{2,2} & \cdots & U_{2,n} & \cdots & U_{n,n}\end{bmatrix}\trp\]
From (\ref{eq:laplace-finite-diff}), for an interior point $U_{i,j}$, we have
\begin{equation}
-4U_{i,j} + U_{i+1,j} + U_{i-1,j} + U_{i,j+1} +  U_{i,j-1} = 0.
\label{eq:hot-plate}
\end{equation}
If any of the neighbors to $U_{i,j}$ is a boundary point on the grid, its value is already determined by the boundary conditions.
For example, the neighbor $U_{3,0}$ of the gridpoint for $U_{3,1}$ is fixed at $U_{3,0} = 100$.
In this case, (\ref{eq:hot-plate}) becomes
\[
-4U_{3,1} + U_{2,1} + U_{3,2} + U_{4,1} = -100.
\]
The constants on the right side of (\ref{eq:hot-plate}) become the $n^2\times 1$ vector $\b$.
All nonzero entries of $\b$ correspond to interior points that touch the left or right boundaries.

As an example, writing (\ref{eq:hot-plate}) for the $16$ interior points of the grid in Figure \ref{fig:itersolve-finite-difference-grid} results in the following $16 \times 16$ system $A\u = \b$.
Note the block structure (empty blocks are all zeros).
\footnotesize
\begin{align*}
\left[\begin{array}{cccc|cccc|cccc|cccc}
\arrayrulecolor{lightgray}
-4 & 1 & 0 & 0 & 1 & 0 & 0 & 0 &   &   &   &   &   &   &   &  \\
1 & -4 & 1 & 0 & 0 & 1 & 0 & 0 &   &   &   &   &   &   &   &  \\
0 & 1 & -4 & 0 & 0 & 0 & 1 & 0 &   &   &   &   &   &   &   &  \\
0 & 0 & 1 & -4 & 0 & 0 & 0 & 1 &   &   &   &   &   &   &   &  \\ \hline
1 & 0 & 0 & 0 & -4 & 1 & 0 & 0 & 1 & 0 & 0 & 0 &   &   &   &  \\
0 & 1 & 0 & 0 & 1 & -4 & 1 & 0 & 0 & 1 & 0 & 0 &   &   &   &  \\
0 & 0 & 1 & 0 & 0 & 1 & -4 & 1 & 0 & 0 & 1 & 0 &   &   &   &  \\
0 & 0 & 0 & 1 & 0 & 0 & 1 & -4 & 0 & 0 & 0 & 1 &   &   &   &  \\ \hline
  &   &   &   & 1 & 0 & 0 & 0 & -4 & 1 & 0 & 0 & 1 & 0 & 0 & 0\\
  &   &   &   & 0 & 1 & 0 & 0 & 1 & -4 & 1 & 0 & 0 & 1 & 0 & 0\\
  &   &   &   & 0 & 0 & 1 & 0 & 0 & 1 & -4 & 1 & 0 & 0 & 1 & 0\\
  &   &   &   & 0 & 0 & 0 & 1 & 0 & 0 & 1 & -4 & 0 & 0 & 0 & 1\\ \hline
  &   &   &   &   &   &   &   & 1 & 0 & 0 & 0 & -4 & 1 & 0 & 0\\
  &   &   &   &   &   &   &   & 0 & 1 & 0 & 0 & 1 & -4 & 1 & 0\\
  &   &   &   &   &   &   &   & 0 & 0 & 1 & 0 & 0 & 1 & -4 & 1\\
  &   &   &   &   &   &   &   & 0 & 0 & 0 & 1 & 0 & 0 & 1 & -4
\end{array}\right]
\left[\begin{array}{c}
U_{1,1} \\ U_{1,2} \\ U_{1,3} \\ U_{1,4}\\
U_{2,1} \\ U_{2,2} \\ U_{2,3} \\ U_{2,4}\\
U_{3,1} \\ U_{3,2} \\ U_{3,3} \\ U_{3,4}\\
U_{4,1} \\ U_{4,2} \\ U_{4,3} \\ U_{4,4}
\end{array}\right]
=
\left[\begin{array}{c}
\arrayrulecolor{lightgray}
-100 \\ 0 \\ 0 \\ -100 \\ \hline
-100 \\ 0 \\ 0 \\ -100 \\ \hline
-100 \\ 0 \\ 0 \\ -100 \\ \hline
-100 \\ 0 \\ 0 \\ -100
\end{array}\right]
\end{align*}
\normalsize
More concisely, for any positive integer $n$, the matrix $A$ can be written as
\begin{align*}
A = \left[\begin{array}{ccccc}
B & I &      &        & \\
I & B &  I   &        & \\
  & I & \ddots & \ddots & \\
  &   & \ddots & \ddots & I \\
  &   &        &    I   & B
\end{array}\right],
\quad
\text{where}\ B = \left[\begin{array}{ccccc}
-4 &  1 &        &        &    \\
 1 & -4 &  1     &        &    \\
   &  1 & \ddots & \ddots &    \\
   &    & \ddots & \ddots & 1  \\
   &    &        &    1   & -4
\end{array}\right]
\ \text{is}\ n\times n.
\end{align*}
% Each $B$ is $n\times n$ and A is consequently $n^2 \times n^2$.

\begin{problem}
Write a function that accepts an integer $n$, a relaxation factor $\omega$, a convergence tolerance \li{tol} that defaults to $10^{-8}$, a maximum number of iterations \li{maxiter} that defaults to $100$, and a bool \li{plot} that defaults to \li{False}.
Generate and solve the corresponding system $A\u = \b$ using Problem \ref{prob:sor}.
Also return a boolean indicating whether the function converged and the number of iterations computed.
\\ (Hint: see Problem 5 of the Linear Systems lab for the construction of $A$.
Also, \li{np.tile()} may be useful for constructing $\b$.)

If \li{plot=True}, visualize the solution $\u$ with a heatmap using \li{plt.pcolormesh()} (the colormap \li{"coolwarm"} is a good choice in this case).
This shows the distribution of heat over the hot plate after it has reached its steady state.
Note that the $\u$ must be reshaped as an $n\times n$ array to properly visualize the result.
\label{prob:finite-difference-setup}
\end{problem}

\begin{problem}
To demonstrate how convergence is affected by the value of the relaxation factor $\omega$ in SOR, run your function from Problem \ref{prob:finite-difference-setup} with $\omega = 1,\ 1.05,\ 1.1,\ \dots , 1.9, 1.95$ and $n = 20$.
Plot the number of computed iterations as a function of $\omega$.
Return the value of $\omega$ that results in the least number of iterations.

Note that the matrix $A$ from Problem \ref{prob:finite-difference-setup} is not strictly diagonally dominant.
However, $A$ is positive definite, so the algorithm will converge.
Unfortunately, convergence for these kinds of systems usually requires more iterations than for strictly diagonally dominant systems.
Therefore, set \li{tol=1e-2} and \li{maxiter=1000}.

Recall that $\omega = 1$ corresponds to the Gauss-Seidel method.
Choosing a more optimal relaxation factor saves a large number of iterations.
This could translate to saving days or weeks of computation time while solving extremely large linear systems on a supercomputer.
\end{problem}

\begin{comment}
The following code generates a heatmap to help you visualize the steady-state heat flow across the interior of the plate:

\begin{lstlisting}
from matplotlib import pyplot as plt
import numpy as np

n = 100
A,b = finite_difference(n)
x = sparse_sor(A,b,1.9,maxiter=10000,tol=10**-2)
U = x.reshape((n,n))
x,y = np.linspace(0,10,n), np.linspace(0,10,n)
X,Y = np.meshgrid(x,y)

plt.pcolormesh(X,Y,U,cmap='coolwarm')
plt.show()
\end{lstlisting}
\end{comment}

%\newpage

%\section*{Additional Material} % =============================================

% END OF LAB ==================================================================

\begin{comment} % Migrated from NumPy lab. vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
\begin{algorithm} % Laplace's equation procedure.
\begin{algorithmic}[1]
\Procedure{Jacobi}{$n$, tol}
\State $U \gets$ an $n\times n$ array of zeros.
\State Set the boundaries of $U$.
\State $U^\prime \gets$ a copy of $U$.
\State diff $\gets$ tol
\While {diff $\ge$ tol}
    \State Set all interior points of $U^\prime$ equal to the average of their neighbors.
    \State diff $\gets$ the maximum of the absolute value of $U - U^\prime$.
    \State Interior of $U \gets$ interior of $U^\prime $.
\EndWhile
\State \pseudoli{return} $U$
\EndProcedure
\end{algorithmic}
\caption{The Jacobi method for solving Laplace's equation.}
\label{alg:jacobi}
\end{algorithm}

\begin{problem}
Laplace's equation is used to model steady-state heat flow on a square plate.
The plate can be approximated by a matrix, where each entry of the matrix represents the average temperature over a small square portion of the plate.
Suppose the plate starts at $0^\circ$ Celsius everywhere except on the east and west boundaries, which are held at a constant temperature of $100^\circ$.
In addition, the north and south boundaries are held constant at $0^\circ$.

\[
U = \left[\begin{array}{ccccc}
\textcolor[rgb]{0.,0.,1.}0 & \textcolor[rgb]{0.,0.,1.}0 & \textcolor[rgb]{0.,0.,1.}\cdots & \textcolor[rgb]{0.,0.,1.}0 & \textcolor[rgb]{0.,0.,1.}0\\
\textcolor[rgb]{1.,0.,0.}{100} & 0 & \cdots & 0 & \textcolor[rgb]{1.,0.,0.}{100}\\
\textcolor[rgb]{1.,0.,0.}\vdots & \vdots & \ddots & \vdots & \textcolor[rgb]{1.,0.,0.}\vdots \\
\textcolor[rgb]{1.,0.,0.}{100} & 0 & \cdots & 0 & \textcolor[rgb]{1.,0.,0.}{100}\\
\textcolor[rgb]{0.,0.,1.}0 & \textcolor[rgb]{0.,0.,1.}0 & \textcolor[rgb]{0.,0.,1.}\cdots & \textcolor[rgb]{0.,0.,1.}0 & \textcolor[rgb]{0.,0.,1.}0\\

\end{array}\right]
\]

The non-boundary, interior portion of the matrix (with black text) can be accessed with the slice \li{U[1:-1,1:-1]}.
To find the steady state of the hot plate, set each entry of the interior of $U$ equal to the average of its 4 immediate neighbors (the entries above, below, and to the left and right).
This step should take only \emph{one line} and should be based entirely on array slicing.

Continue updating the interior entries of $U$ until they stop changing significantly.
The entire procedure is summarized in Algorithm \ref{alg:jacobi}.

Hint: The slice \li{U[:-2,1:-1]} references the upper neighbors of the interior points of $U$ and \li{U[1:-1,2:]} references the right neighbors. How can you reference the lower and left neighbors?

Use the following code to visualize your results.
%% This does 3D visualization, but it's easier (and better) as a heat map.
% from mpl_toolkits.mplot3d import Axes 3D

%     x, y = np.linspace(0, 1, n), np.linspace(0, 1, n)
%     X, Y = np.meshgrid(x, y)
%     fig = plt.figure()
%     ax = fig.gca(projection='3d')
%     ax.plot_surface(X, Y, U, rstride=5)
%     plt.show()

\begin{lstlisting}
from matplotlib import pyplot as plt

def jacobi(n=100, tol=1e-8):

    # Perform the algorithm, storing the result in the array 'U'.

    # Visualize the results.
    plt.imshow(U)
    plt.show()
\end{lstlisting}

This Jacobi iteration is a \emph{finite difference method}, and is similar to many of the methods that we will use to find numerical solutions to differential equations in Volume 4.
\end{problem}

\begin{figure}[H]
\centering
\begin{subfigure}{.5\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/jacobi_small.pdf}
    \caption{$10\times 10$ approximation.}
\end{subfigure}%
\begin{subfigure}{.5\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/jacobi_big.pdf}
    \caption{$100\times 100$ approximation.}
\end{subfigure}
\caption{Hot plates in steady state with different resolutions.}
\end{figure}
\end{comment} % Used to be in NumPy ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


% corrected VD 80

\subsubsection{Gated Recurrent Unit}~\\

Identical to LSTM units, the gated recurrent unit (GRU) consists of gated units
which control the flow of information through the unit, although without having
a dedicated memory cell. Compared to LSTM units, GRUs don't have an output gate.
The output gate controls the amount of memory content seen by the other units in
the network. Thus GRU exposes its entire content without
control.~\cite{DBLP:journals/corr/ChungGCB14}\\

The activation of a GRU unit is computed with:
\begin{equation}
  h_t = (1-z_t)h_{t-1} + z_t\tilde{h}_t
\end{equation}
with an update gate $z_t$ which decides how much the unit updates its content.
Its content consists of the previous activation $h_{t-1}$ and the candidate
activation $\tilde{h}_t$. The update gate is given by the equation:
\begin{equation}
  z_t=\sigma(W_z x_t+U_z h_{t-1})
\end{equation}
This update gate is similar to the one used by LSTM. However, it exposes its
state each time step since it does not have a procedure to control how the state
is exposed. The candidate activation is given by:
\begin{equation}
  \tilde{h}_t=\tanh(Wx_t+U(r_t\odot h_{t-1}))
\end{equation}
where $r_t$ is a reset gate and is computed comparably to the update gate:
\begin{equation}
  r_t=\sigma(W_r x_t+U_r h_{t-1})
\end{equation}

See the illustration of a GRU unit in Figure~\ref{grucell}.

\input{sections/scientific/fr2/grucell.tex}


%
%

\chapter{Numerical solution of Markov models}  \label{SEC:Numerical}

\RELEASE{This chapter is still under revision.
More information can be obtained from the examples in
Chapter \ref{SEC:Examples} and from the various input files provided
with the {\smart} distribution.}

In this chapter, we consider systems whose underlying
stochastic process has a finite state space, and whose study can be carried
on through numerical methods.
Before describing the features implemented in {\smart} to solve such processes,
we discuss briefly a few key background notions in stochastic processes and
numerical linear algebra.

%        ===========================
\section{Discrete time Markov chains}
%        ===========================
A \index{discrete-time Markov chain (DTMC)}discrete-time Markov chain (DTMC)
is completely described by its transition
probability matrix $\P$ and its initial probability vector $\vp^{(0)}$,
where $\P[i,j]$ defines the probability
that a system in state $i$ will be in state $j$ at the next time step, and
$\vp^{(t)}[i]$ is the probability that the system is in state
$i$ at time $t \in \Naturals$.
Once $\P$ and $\vp^{(0)}$ have been specified explicitly or constructed from a
higher-level model, the following types of analysis are possible.
The interested reader is referred to \cite{Ross2003book} for more details.

%           =========================================
\subsection{Instantaneous transient analysis of DTMCs}\label{SEC:powerDTMC}
%           =========================================
The probability of being in state $j$ at time step $t \in \Naturals$
having started in state $i$ at time $0$ is the $(i,j)$-entry of
the $t\SUPTH$ power of the transition probability matrix, $\P^t[i,j]$.
However, powers of matrix $\P$ cannot be computed or stored in practice
due to the size of $\P$.
Thus, we make use of the knowledge of the initial probability vector
and compute instead the probability vector at time step $t$,
$\vp^{(t)} = \vp^{(0)} \cdot \P^t$, using the recursive relation
\[
  \vp^{(t+1)} = \vp^{(t)} \cdot \P.
\]

%           ======================================
\subsection{Cumulative transient analysis of DTMCs}
%           ======================================
If we are interested in the total time spent in each state during the first
$t \in \Naturals$ time steps, we compute
\[
   \vn^{(t)} = \sum_{y=0}^{t-1} \vp^{(y)}
\]
where $\vn^{(t)}[i]$ is the expected amount of time spent in state $i$
until time step $t$.

%           ==========================================
\subsection{Instantaneous stationary analysis of DTMCs}
%           ==========================================
Assuming that the DTMC is aperiodic\footnote{ The \emph{period} of
a state $i$ is the greatest common divisor of all time steps $t$
such that the probability of returning to state $i$ in $t$ time
steps, $\P^t[i,i]$, is nonzero. If a state has period greater than
one, it is said to be \index{periodic Markov chain}\emph{periodic},
otherwise it is \index{aperiodic Markov chain}\emph{aperiodic}.
The DTMC is aperiodic if all of its states are
aperiodic.}, {\smart} can compute the ``long-term'' probability of
being in each state. The method used in practice to compute the
\index{stationary probability distribution}stationary probability vector
$\vp = \lim_{t \to\infty} \vp^{(t)}$
depends on the properties of the DTMC.

\subsubsection*{Ergodic DTMCs}

\index{ergodic Markov chain}Ergodic DTMCs converge to a
\index{stationary probability distribution}stationary (invariant) probability distribution
independent of the initial state.  This is the case when a finite-state DTMC is aperiodic and
\index{irreducible Markov chain}irreducible\footnote{ A
Markov chain is irreducible if there is a path between every pair
of states.}.  Irreducibility on a finite state space
implies positive recurrence, which is necessary for a limiting distribution to exist.
Then, the stationary probability distribution (vector
$\vp$) uniquely satisfies $\vp = \vp \cdot \P$,
$\sum_{i}\vp[i]=1$.  The solution $\vp$ is strictly positive and
$\vp = \lim_{t \to\infty} \vp^{(t)}$.  In this case, {\smart} computes
$\vp$ by solving the homogeneous linear system
\begin{equation}
  \vp \cdot (\P - \I) = \0 \ \mbox{ subject to }\ \sum_{i}\vp[i]=1.
  \label{LS:ergDTMC}
\end{equation}

\subsubsection{\index{absorbing Markov chain}Absorbing DTMCs}

If the DTMC consists exclusively of transient and absorbing\footnote{
A state $i$ is \emph{transient} if there exists a state $j$ reachable from $i$
such that $i$ is not reachable from $j$.
A state $i$ is \emph{absorbing} if no other state is reachable from it,
that is, $\P[i,i] = 1$.
} states, we need to compute the
\index{stationary probability distribution}stationary probabilities only
for the absorbing states, since the transient states have zero probability.
However, these probabilities depend on the initial probability vector
$\vp^{(0)}$ in general.
Denoting the sets of absorbing and transient states by
$\aset$ and $\tset$, respectively, {\smart} computes $\vp$ as
\begin{eqnarray*}
 \vp[\Set{A}]
 & = &
 \vp^{(0)}[\Set{A}] + \vn[\Set{T}] \cdot \P[\Set{T}, \Set{A}]
 \\
 \vp[\Set{T}]
 & = &
 \0
\end{eqnarray*}
where $\vn[\Set{T}]$ is described in Sec.~\ref{SEC:fundamental}.

\subsubsection{Arbitrary aperiodic DTMCs}

In the general case, we only require that all states of the DTMC are
\index{aperiodic Markov chain}aperiodic.
The DTMC can consist of transient states and classes of
recurrent (mutually reachable) states, and can be seen as an
\index{absorbing Markov chain}absorbing DTMC where
each absorbing state is replaced by an \index{ergodic Markov chain}ergodic DTMC.
The \index{stationary probability distribution}stationary probability vector $\vp$
is then computed by first computing the stationary probability vectors of each
\index{ergodic}ergodic class in isolation,
then computing the probability of being absorbed in each ergodic class, and
finally combining the stationary probability vectors of each ergodic class
using the absorption probabilities as weights (of course, the probability
of transient states is zero in this case as well).

\subsubsection{Periodic DTMCs}

For \index{periodic Markov chain}periodic DTMCs with period $\delta > 1$, $\P^t[i, j]$ does not
have a limit as $t\to\infty$ except for $\P^t[i, j]\to 0$ when all
state are null.  However, the limits of $\P^{t\delta+u}[i, j]$
exist as $t\to\infty$ but depend on the initial state $i$.
Although there exists a unique solution satisfying $$\vp = \vp
\cdot \P,\quad \sum_{i}\vp[i]=\delta,$$ {\smart} does not check
explicitly that the DTMC is periodic, much less compute the
period $\delta$ needed to find the solution. Nevertheless,
numerical methods might still give the correct answer, but in a
time-averaged sense: the computed vector $\vp$ satisfies $\vp =
\lim_{t \to\infty} \vn^{(t)}/t$.

If the (iterative) solution methods are unable to compute a
solution, it is simply because convergence was not obtained within
the maximum number of iterations allowed, and a warning is issued
in this case.  In other words, periodicity does not lead to
incorrect answers in {\smart}.

%           =======================================
\subsection{Cumulative stationary analysis of DTMCs}\label{SEC:fundamental}
%           =======================================
We can also consider the (expected) total time spent in each state
by computing the vector $\vn = \lim_{t \to\infty} \vn^{(t)}$. Of
course, $\vn[i] = \infty$ for any state $i$ with a positive
\index{stationary probability distribution}stationary probability,
thus the entries in $\vn$ are meaningful
only for the transient states. In this case, $\vn[i]$ is the
expected time spent in transient state $i$ before the DTMC reaches
an absorbing state, or a class of \index{ergodic}ergodic states. The portion of
$\vn$ corresponding to the transient states is the solution of the
non-homogeneous linear system
\begin{equation}
  \vn[\Set{T}] \cdot (\P[\Set{T}, \Set{T}] - \I)
  = -\vp^{(0)}[\Set{T}]
  \label{LS:absDTMC}
\end{equation}
where $\P[\Set{T}, \Set{T}]$ is the submatrix of $\P$
corresponding to transitions between transient states.

%        =============================
\section{Continuous time Markov chains}
%        =============================
A \index{continuous-time Markov chain (CTMC)}continuous-time Markov chain (CTMC)
is completely described by its transition
rate matrix $\R$ and its initial probability vector
$\gvect{\pi}^{(0)}$, where $\R[i,j]$ defines the rate of going to state
$j$ given that the chain is in state $i$, and
$\gvect{\pi}^{(t)}[i]$ is the probability that the system is
in state $i$ at time $t \geq 0$.
A transition from a state to itself is meaningless in a CTMC,
thus, the diagonal elements of $\R$ are zero.
The infinitesimal generator matrix, $\Q$, is identical to the matrix $\R$
except for the diagonal elements, which are set so that each row of
$\Q$ sums to zero: $\Q[i,i] = - \sum_{j \neq i} \R[i,j]$.
Once $\Q$ (or $\R$) and $\gvect{\pi}^{(0)}$ have been specified explicitly
or constructed from a higher-level model, the following types of analysis
are possible.
The interested reader is referred to \cite{1993IMA-SRNs,Stewart1994book}
for more details.

%           =========================================
\subsection{Instantaneous transient analysis of CTMCs}\label{SEC:inst-trans-anal-CTMC}
%           =========================================
To compute $\gvect{\pi}(t)$, the probability vector at time $t$, we use a
technique called \index{uniformization (randomization)}\emph{uniformization}
(or \emph{randomization})
\cite{Grassmann1991}, which constructs the ``uniformized DTMC'' defined by
\[
   \P = \frac{\Q}{q} + \I, \quad \vp^{(0)} = \vpi(0)
\]
where $q$ satisfies $q \ge \max_i\{-\Q[i,i]\}$.  Observe that with this
choice for $q$, the CTMC will be sampled at a rate sufficient to observe the
fastest CTMC state transitions.  This means that the DTMC is constructed such that
states that are slower to transition to another state will have self loops.
Consequently, the DTMC is guaranteed to be aperiodic.
The probability vector $\gvect{\pi}(t)$ can then be written as
\begin{equation}
  \gvect{\pi}(t) = \sum_{n=0}^{\infty} \frac{e^{-qt}(qt)^n}{n!}
     \cdot \vp^{(n)}
  \label{SUM:unif-inst}
\end{equation}
where ${e^{-qt}(qt)^n}/{n!}$ is the Poisson probability of
observing $n$ state transitions of the CTMC ($n$ jumps of the DTMC) within time $t$,
and $\vp^{(n)}$ is the probability distribution of state occupancy after $n$ jumps.
Given $\P$, the vector $\vp^{(n)}$ is computed using the power method as discussed
in Sec.~\ref{SEC:powerDTMC}. {\smart} computes the Poisson probabilities using the
numerically stable Fox-Glynn algorithm \cite{Fox1988},
which provides a value of $N$ such that
$\overline{p}(N) \le \epsilon$
for a desired absolute precision $\epsilon>0$,
where $\overline{p}(N)$ is the probability that the Poisson value is larger than $N$:
\[
  \overline{p}(N) = 1 - \sum_{n=0}^{N} \frac{e^{-qt}(qt)^n}{n!}.
\]
{\smart} truncates the infinite sum in Eq.~\ref{SUM:unif-inst}
and computes instead the sum from 0 to $N$,
which guarantees that the elements of $\gvect{\pi}(t)$
sum to at least $1-\epsilon$.
\RELEASE{At present, {\smart} does not have an option to specify this $\epsilon$.}
\TBD{Add an option to specify this $\epsilon$.}

%           ======================================
\subsection{Cumulative transient analysis of CTMCs}\label{SEC:cumul-trans-anal-CTMC}
%           ======================================
{\smart} uses \index{uniformization (randomization)}uniformization also to compute $\vsigma(t) = \int_0^t
\vpi(u) du$, the cumulative sojourn time vector up to time $t$, or in other words,
the total amount of time spent in each state during the interval $(0,t)$.
Using integration by parts, $\gvect{\sigma}(t)$ can be reformulated as
\begin{eqnarray}
  \nonumber
  \vsigma(t)
  & = & \int_{0}^{t}
    \sum_{n=0}^{\infty} \frac{e^{-q u} (q u)^n}{n!} \vp^{[n]} d u
  ~ = ~ \sum_{n=0}^{\infty} \vect{p}^{[n]}
    \int_{0}^{t} \frac{e^{-q u} (q u)^n}{n!} d u
  ~ = ~ \frac{1}{q} \sum_{n=0}^{\infty} \vp^{(n)} \sum_{k=n+1}^{\infty}
  \frac{e^{-q t} (q t)^k}{k!}
  \\
  & = &
  \frac{1}{q} \sum_{n=0}^{\infty} \vp^{(n)} \overline{p}(n).
  \label{SUM:unif-acc}
\end{eqnarray}
Similar to the discussion in Sec.~\ref{SEC:inst-trans-anal-CTMC},
{\smart} computes a finite sum from $0$ to $N$
with $\overline{p}(N) \le \epsilon$,
where $\epsilon$ is a desired absolute precision.
\TBD{\ldots specified by option \Code{NeedToAddThisOption}.}

%           ======================
\subsection{Steady-state detection}
%           ======================
When computing the (truncated) sums in Eqs.~\ref{SUM:unif-inst} and \ref{SUM:unif-acc},
{\smart} checks for steady-state conditions,
just in case the CTMC has a limiting distribution,
$\gvect{\pi}=\lim_{u\to\infty} \gvect{\pi}(u)$, which may be dependent on the initial state.
If the solution time $t$ is large enough then $\vp^{(n)}$ may converge to a limiting distribution
as $n$ becomes large and before reaching the truncation point $N$.
{\smart} assumes steady-state conditions when the relative error between successive iterates,
$\vp^{(n)}$ and $\vp^{(n+1)}$, is less than or equal to a user-specified $\epsilon$.
\TBD{Need an option to specify this $\epsilon$.}
\TBD{Need an option to turn this off, for speed, e.g. if $0=\epsilon$.}

\TBD{HERE}

%           ==========================================
\subsection{Instantaneous stationary analysis of CTMCs}
%           ==========================================
Analogous to the discrete case, we can consider the ``long-term'' probability
of being in each state of a CTMC by computing the
\index{stationary probability distribution}stationary probability vector
\[
\gvect{\pi} = \lim_{t \rightarrow \infty} \gvect{\pi}(t).
\]
Like DTMCs, analysis of a general CTMC requires the analysis of many
\index{ergodic Markov chain}ergodic CTMCs and an \index{absorbing Markov chain}absorbing CTMC.

\subsubsection{Ergodic CTMCs}

Like ergodic DTMCs, \index{ergodic Markov chain}ergodic CTMCs also converge to a
\index{stationary probability distribution}stationary probability distribution
independent of the initial state.
In this case, a finite-state CTMC need only be
\index{irreducible Markov chain}irreducible since periodicity is not an issue
in continuous time.  Again, irreducibility on a finite state space
implies positive recurrence, which is necessary for a limiting distribution to exist.
%An ergodic CTMC has the property that for all pairs of states $i$ and $j$ in
%the CTMC, $j$ is reachable from $i$.
Then, the stationary probability vector, $\gvect{\pi}$, uniquely satisfies
\begin{equation}
  \gvect{\pi} \cdot \Q = \0 \ \mbox{ subject to }\ \sum_{i} \gvect{\pi}[i] = 1.
  \label{LS:ergCTMC}
\end{equation}

\MSG{New from here.}
Currently, {\smart} supports several different numerical algorithm to calculate the steady-state solutions
for ergodic CTMCs, which is set by option \emph{SolutionType} and \emph{Solver}.

\emph{SolutionType} has following options:
\begin{itemize}
\item NUMERICAL
\item APPROXIMATE\_EVMDD
\item EXACT\_EVMDD
\item SIMULATION
\end{itemize}

\emph{Solver} has following options:
\begin{itemize}
\item POWER
\item JACOBI
\item GAUSS\_SEIDEL
\end{itemize}

For example, {\smart} will utilize symbolic engine and Jacobi iteration by the following options:

\begin{verbatim}
# StateStorage MDD_SATURATION_LEVEL
# SolutionType EXACT_EVMDD
# Solver JACOBI
\end{verbatim}

\subsubsection{\index{absorbing Markov chain}Absorbing CTMCs}

If the CTMC consists only of transient and absorbing states,
we can compute the \index{stationary probability distribution}stationary probabilities
for the absorbing states by \emph{embedding} the CTMC, obtaining the DTMC
\[
  \P[i,j] = \left\{ \begin{array}{ll} %}
             \frac{\Q[i,j]}{-\Q[i,i]}
             &
             \mbox{if $i\neq j$}
             \\
             0
             &
             \mbox{otherwise}
                  \end{array} \right.
\]
with $\vp^{(0)} = \gvect{\pi}(0)$.
We then compute $\gvect{\pi} = \vp$ in a way analogous to the
discrete case.

%           =======================================
\subsection{Cumulative stationary analysis of CTMCs}
%           =======================================
Analogous the the DTMC case, we can compute the total expected time spent in
each state of a CTMC as the limit
\[
\gvect{\sigma} = \lim_{t \rightarrow \infty} \vsigma(t).
\]
For transient states, the entries in $\gvect{\sigma}$ are finite,
and represent the expected time spent in each transient state before
the CTMC reaches an absorbing state (or a class of \index{ergodic}ergodic states).
The portion of $\gvect{\sigma}$ corresponding to the transient states is the
solution to
\begin{equation}
  \gvect{\sigma}[\Set{T}] \cdot \Q[\Set{T}, \Set{T}]
  = -\gvect{\pi}(0)[\Set{T}]
  \label{LS:absCTMC}
\end{equation}
where $\Q[\Set{T}, \Set{T}]$ is the submatrix of $\Q$
corresponding to transitions from transient states to transient states.

\begin{developer}
%        ===========================
\section{Semi-regenerative processes}\label{SEC:semi-regen-processes}
%        ===========================
While more complex than simple Markov chains,
\index{semi-regenerative process}semi-regenerative processes may still be analyzed using
the traditional Markov chain methods discussed previously, but only under certain conditions.
Essentially, given a discrete-state, discrete-event model where the events have \emph{both}
discrete and continuous \index{phase-type distribution}phase-type distributed delays
(\index{ph int}\Code{ph int} and \index{ph real}\Code{ph real}, respectively),
the underlying process will be semi-regenerative
if the \Code{ph int} events are always \emph{synchronized} when active.
By \index{synchronization property}synchronized,
we mean that the phase changes of all active \Code{ph int} events occur at precisely the
same (discrete) time when active.
Then, the regeneration times can be defined by \Code{ph int} events
(simple phase changes or other state changes triggered by the event).
The only other active events that can trigger state changes between successive regeneration times
are of \Code{ph real} type.  Consequently, the subordinate processes that evolve between
regeneration times are simple CTMCs, and, as is always the case for semi-regenerative processes,
the embedded \index{Markov renewal process}Markov renewal process gives rise to an embedded DTMC.
Semi-regenerative processes encountered in this way can be decomposed into
DTMC and CTMC subproblems; however, because the DTMC and CTMC ``submodels'' interact
with each other, they must be solved through a concerted effort.
Note that the underlying process may still be semi-regenerative without the synchronization
assumption, but without a decomposition into Markov chain subproblems, numerical solution
may not be practical.

At present, {\smart} is capable of numerically computing the stationary probability distribution
of a semi-regenerative process with the synchronization assumption when
the embedded DTMC is \index{ergodic}ergodic with a stationary solution.
Synchronization among \Code{ph int} events is automatically checked while generating the
underlying state space.  A warning is issued to the user if the requisite synchronization
assumption is not present, which precludes a numerical solution.
In what follows, we will only present the key formulas for computing stationary solutions
and refer the interested reader to \cite{Jones-NASA-00}
for a discourse on this subject and a detailed presentation of the solution algorithms.

Let $\P$ be the transition matrix of the embedded DTMC with state space $\mathcal{E}$
that evolves at each regeneration time.
The matrix $\P$ is computed one row at a time by exploring from each known state $i\in\mathcal{E}$
and constructing the subordinate CTMC that evolves until the next regeneration time is reached
with new embedded states $j\in\mathcal{E}$ such that $\P[i, j]>0$.
Actually, only values $\P[i, j] > \Code{Pruning}$ are stored where the
user option \index{Pruning option}\Code{Pruning} is
considered ``small enough'' with a default value of $10^{-20}$.
Let $\Q_i$ be the infinitesimal generator matrix of the subordinate CTMC on the state space
$\mathcal{S}_i$ originating from embedded state $i$.  Take the basic step of all \Code{ph int}
to be unity.  Then, when both \Code{ph int} and \Code{ph real} events are active,
the entries of matrix $\P$ can be computed from
\[
  \P[i, j] = \sum_{k\in\mathcal{S}_i} e^{\Q_i}[i, k] \matr{\Delta}[k, j]
\]
where the matrix $e^{\Q_i}$ is the (matrix exponential) solution of the CTMC at time one,
which is actually computed using the \index{uniformization (randomization)}uniformization
procedure discussed in Sections \ref{SEC:inst-trans-anal-CTMC} and \ref{SEC:cumul-trans-anal-CTMC}.
The matrix $\matr{\Delta}[k, j]$ gives the probability of jumping from state $k$ to state $j$
at the regeneration time and is computed automatically from the model specification.
When only \Code{ph int} or only \Code{ph real} events are active,
a regeneration occurs after each state transition due to the strong Markov property, and we
simply replace $e^{\Q_i}[i, k]$ in the equation above with the indicator function $1\{i=k\}$.
Conditioned on each $i\in\mathcal{E}$, let the matrix $\gvect{\sigma}$ be the expected sojourn time
in each state $k\in\mathcal{S}_i$ until the next regeneration time, which is computed from
\[
  \gvect{\sigma}[i, k] =
  \begin{cases}
    \int_{0}^{1} e^{\Q_i u}[i, k]\,du
                               & \mbox{if both \Code{ph int} and \Code{ph real} are active}, \\
    1\{i=k\} (-Q_i[i, i])^{-1} & \mbox{if only \Code{ph real} events are active}, \\
    1\{i=k\}                   & \mbox{if only \Code{ph int} events are active},  \\
    0                          & \mbox{otherwise}.
  \end{cases}
\]
The computation in the first case can be performed at the same time $e^{\Q_i}$ is computed.
After constructing the matrix $\P$,
the stationary solution, $\vp$, of the embedded DTMC is computed from $\vp \cdot (\P - \I) = \0$.
The whole state space for the semi-regenerative process is defined by
$\mathcal{S}\equiv\bigcup_{i\in\mathcal{E}} \mathcal{S}_i$.
Finally, the stationary solution, $\gvect{\pi}$, of the semi-regenerative process is computed by
considering the average time spent in any state between regeneration times
(conditioned on each embedded state),
\[
  \gvect{\gamma}[k] = \sum_{i\in\mathcal{E}} \vp[i]\cdot\gvect{\sigma}[i, k],
\]
and then normalizing,
\[
  \gvect{\pi}[k] = \frac{\gvect{\gamma}[k]}{\sum_{j} \gvect{\gamma}[j]},
\]
to obtain the proportion of time spent in any state between regeneration times
for each state $k\in\mathcal{S}$.

\DEVELOPER{
{\smart} actually avoids having to compute and store the entire matrix
$\gvect{\sigma}\in\Reals^{|\mathcal{E}|\times|\mathcal{S}|}$ by being ``measure driven.''
That is, $\gvect{\sigma}[i, k]$ for each state $i\in\mathcal{E}$ in turn is used to
``distill'' the user-defined reward functions $f : \mathcal{S}\to\Reals$
(used to compute measures) into a new reward functions $\tilde{f} : \mathcal{E}\to\Reals$,
\[
  \tilde{f}(i) = \sum_{k\in\mathcal{S}_i} \gvect{\sigma}[i, k]\cdot f(k),
\]
defined only on the embedded state space $\mathcal{E}$, not the whole state space $\mathcal{S}$.
This way, $\gvect{\sigma}[i, \cdot]$ can be discarded once it is used to compute $\tilde{f}(i)$.
}

We have just described the \index{embedding strategies}``embedding strategy'' used when the
option \index{EmbedWithElim option}\Code{EmbedWithElim} = 0 is selected, which is the default.
Alternative embedding strategies, which attempt to reduce the size of the state space $\mathcal{E}$,
are also available that exploit the fact that oftentimes more than one embedded Markov process is
present \cite{2001PNPM-PDPN}.
Essentially, embedded states can be eliminated by being selective in our choice of
which embedded Markov process to observe.  At present, there are four possible
embedding strategies to choose from:
\begin{table}[h]
\centering
\begin{tabular}{|c|p{4.25in}|}
\hline
  \Code{EmbedWithElim} & \textbf{Description} \\
  \hline
  \hline
  0 & most basic, the one described above, 0 embedded state are eliminated \\ \hline
  1 & eliminate embedded states that arise when only
      \Code{ph int} or only \Code{ph real} events are active \\ \hline
  2 & eliminate embedded states that only record phase changes
      when both \Code{ph int} and \Code{ph real} events are active \\ \hline
  3 & 1+2=3, use both 1 and 2 together \\
  \hline
\end{tabular}
\end{table}

Another user option called
\index{Epsilon option}\Code{Epsilon} comes into play when using embedding strategies 2 or 3.
Embedding strategy 2 (and hence 3) tries to find \emph{all possible} regeneration times that:
\begin{itemize}
  \item comprise a single step in $\P$,
  \item are (mostly) due to synchronized events with \Code{ph int} delays, and
  \item it is certain that the event will trigger a state change where more than just the
        phase component changes, such as a change in the marking in the case of \Code{spn} models.
\end{itemize}
Because {\smart} will try to consider the entire (pmf) support of active \Code{ph int} events
in search of regeneration times with all of the above properties, a truncation point must be
introduced in case the support is infinite in length, such as the infinite support of the
geometric distribution.
The user option \index{Epsilon option}\Code{Epsilon} $\in$ $(0, 1)$ provides a truncation
point by stopping the search when the smallest pmf tail of any active \Code{ph int} event
becomes less than or equal to \Code{Epsilon}. \Code{Epsilon} has a default value of $10^{-10}$.

Finally, the following print options are provided that
take boolean values $\{\Code{true}, \Code{false}\}$,
all \Code{false} by default:
\index{ShowEMCdata, ShowEMCprob, ShowEMCsize, ShowSpecial options}
\begin{table}[h]
\centering
\begin{tabular}{|c|p{4.25in}|}
\hline
  \textbf{Option} & \textbf{Prints the following if \Code{true}} \\
  \hline
  \hline
  \Code{ShowEMCsize} & size of the embedded state space, $|\mathcal{E}|$ \\
  \Code{ShowEMCdata} & all nonzero entries of the transition matrix $\P\in\Reals^{|\mathcal{E}|\times|\mathcal{E}|}$ \\
  \Code{ShowEMCprob} & $\vp[i]$ and $\sum_k\gvect{\sigma}[i, k]$ for each state $i\in\mathcal{E}$ \\
  \Code{ShowSpecial} & data about the computational effort \\
  \hline
\end{tabular}
\end{table}

% FIX AND MOVE THIS TO AN EARLIER SECTION ON "UNDERLYING STOCHASTIC PROCESSES"...
% Perhaps the best way to describe a semi-regenerative process is by beginning from
%processes that are already familiar.  The simplest stochastic process is a
%sequence of independent and identically distributed (i.i.d.) random variables
%referred to as a renewal process.
%Markov chains generalize renewal processes somewhat by allowing a limited amount
%of dependency.
%Consider now a stochastic process that starts over independently and identically
%at certain random times forming an increasing sequence $T=\{T_n : n \geq 0\}$.
%Processes that restart like this are called regenerative processes and the times $T$
%are called regeneration times.
%example with Markov chains at fixed state....
%
%So, regenerative processes generalize renewal processes,
%but regenerative processes can be generalized further just like renewal processes were
%generalized by Markov chains.
%Semi-regenerative processes generalize regenerative processes by allowing the future
%after regeneration times to depend on the state entered just after such times.
%It follows that ...
%Semi-regenerative There is an
\end{developer}

\DEVELOPER{
%        ======================
\section{Solving linear systems}
%        ======================
For many types of analysis, {\smart} must solve a linear system of the form
\[
  \vx \A = \vb
\]
for the vector $\vx$.
Since the matrix $\A$ is typically very large and
quite sparse, and is not always explicitly stored, {\smart} uses
implicit, iterative techniques \cite{Stewart1994book} for computing
$\vx$. Since the iterative methods compute $\vx$ as a fixpoint,
options analogous to the \index{converge option}\Code{converge} options are used to
adjust the computation: \Code{NumericalPrecision} defines the
desired precision $\epsilon$, \Code{NumericalPrecisionTest}
defines the \emph{stopping criterion} (either element-wise
relative or element-wise absolute precision), and
\Code{MaxNumericalIters} defines the maximum number of allowed
iterations.

\MSG{discuss \Code{RELAXATION}}

The type of iteration performed is specified using the
\index{Solver option}\Code{Solver} option,
which can be one of the following.

\begin{description}

\item{\Code{POWER} :}  The power method.
This works for systems of the form $\vp = \vp \cdot \P$, and
simply computes $\vp^{(n+1)} = \vp^{(n)} \cdot \P$ starting
with $n=0$ and continuing until subsequent vectors are within a desired
precision.

\MSG{This can be used for transient solutions of DTMCs,...}

\MSG{Does this use relaxation?}

\item{\Code{JACOBI} :}  The Jacobi iteration.
Newly computed elements of $\vx$ are not used until the next iteration.
As a result, an auxiliary vector $\vx'$ is required to store the new
values.
{\smart} is able to use the Jacobi iteration when the matrix $\A$ is stored
in a sparse format, either by rows or by columns.
{\bf This is the default.}
\MSG{Should it be GAUSS\_SEIDEL instead?}

\DEVELOPER{
\item{\Code{BLOCK\_JACOBI} :}  An experimental block algorithm.
  This should not be used yet, but eventually will be used with distributed
  solutions.
}

\item{\Code{GAUSS\_SEIDEL} :}  The Gauss-Seidel iteration.
This is similar to Jacobi, except the newly computed elements of $\vx$
are used right away.
As a result, the auxiliary vector required by Jacobi is not needed with
Gauss-Seidel.
However, this requires the computation of one element of $\vx$ at a time;
as a result, the Gauss-Seidel iteration requires access to the columns of
matrix $\A$.
Therefore, Gauss-Seidel can only be used with data structures for $\A$
that allow efficient access to the matrix columns.

\MSG{Discuss relative number of iterations.}

\DEVELOPER{
\item{\Code{BLOCK\_GAUSS} :}  An experimental block algorithm, just as
experimental and not working as block Jacobi.
}

\item{\Code{SOR} :} A Gauss-Seidel iteration which adaptively adjusts the
relaxation parameter.
\cite{1993IMA-SRNs}
The initial relaxation parameter is taken from the option
\index{RELAXATION option}\Code{RELAXATION}.

\DEVELOPER{
\item{\Code{BICGSTAB} :} Biconjugate Gradient Stabilizer.
Also experimental (i.e., not thoroughly tested and debugged yet).
}

\end{description}

%        ===========================
\section{Representing large matrices}
%        ===========================
Since the matrices are large, blah blah blah.

{\smart} can use a variety of methods for representing the transition matrix of a Markov chain.
The methods, described below, are selected based on the option
\index{MarkovStorage option}\Code{MarkovStorage}.

%           ==============
\subsection{Sparse storage}
%           ==============
Mention rows vs columns.
Used for high-level formalisms and manipulation of phase-type vars.
For high level models, we do a memory-efficient, two-pass approach:
first count the number of entries in each row or column, so that we can
allocate an array of entries instead of using linked lists.
The second pass fills the array.
This is the default.

%           ==========
\subsection{On the fly}
%           ==========
Only high-level formalisms with expo or immediate transitions.
Only by rows.
\cite{Deavours1997onthefly}

%           =========================
\subsection{Kronecker representations}
%           =========================
Only high-level formalisms with expo or \emph{local} immediate transitions.
Mention rows vs columns.
Options to mention: \Code{MarkovStorage}, \Code{UseHoldingArray},
\Code{MatrixByRows}

The consistent requirement.

%           ===============
\subsection{Matrix diagrams}
%           ===============
Only high-level formalisms.
Mention rows vs columns.

Kronecker \cite{1999PNPM-MatrixDiagrams, MinerThesis2000} vs. non-Kronecker
\cite{Miner2001canonicalMDs}

Options to mention: \Code{MarkovStorage}, \Code{UseHoldingArray},
\Code{MergeLevel}

}


\documentclass[12pt, a4paper]{article}
\usepackage[margin=1.25in]{geometry}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{float}
\usepackage{listings}
\usepackage{physics}
\usepackage{mathrsfs}
\usepackage[shortlabels]{enumitem}

\setlength\parindent{0pt}
\newcommand{\code}{\lstinline[basicstyle=\small]}
\lstset{
    language=Python,
    basicstyle=\scriptsize
}

\title{EE2703: Applied Programming Lab \\ \Large Assignment 9: Spectra of Non-Periodic Signals}
\author{Soham Roy \\ \normalsize EE20B130}
\date{\today}
\begin{document}

\maketitle % Insert the title, author and date

\section{Introduction}
This assignment continues the examination of signals using Fourier Transforms, with non-periodic functions.
When these functions are periodically extended, discontinuities arise. Due to the Gibbs phenomenon, the discontinuities create
Fourier components in non-harmonic frequencies which decay as $\frac{1}{\omega}$. A Hamming window has been used to solve this
problem.

\section{Questions}
\subsection{Examples}
Spectrum of $\sin(\sqrt{2}t)$:
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.7]{eg1.png}
\end{figure}

Graph of $\sin(\sqrt{2}t)$:
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.7]{eg2.png}
\end{figure}

$\sin(\sqrt{2}t)$ represented by the DFT:
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.7]{eg3.png}
\end{figure}
\pagebreak

These discontinuities lead to non-harmonic components in the FFT which decay as $\frac{1}{\omega}$.
To confirm this, the spectrum of the periodic ramp has been plotted:
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.65]{eg4.png}
\end{figure}

\subsubsection{Hamming window}
The Hamming window removes discontinuities by attenuating the high frequency components that cause the discontinuities.
The Hamming window function is given by:
\begin{equation*}
    x[n] = 0.54 + 0.46\cos(\frac{2\pi n}{N - 1})
\end{equation*}

We now multiply our signal with the Hamming window and periodically extend it. We observe that the discontinuities nearly vanish:
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.65]{eg5.png}
\end{figure}

The spectrum that is obtained with a time period $2\pi$ is given below:
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.7]{eg6.png}
\end{figure}

The spectrum that is obtained with a time period $8\pi$ has a slightly sharper peak:
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.7]{eg7.png}
\end{figure}
\pagebreak

\section{Questions}
\subsection{Helper Functions}
We define the following helper functions:
\begin{lstlisting}
    def spectrum(f, n, lim, windowing=True, t=None):
        """Evaluates the DFT spectrum of a function f(t)."""
        if t is None:
            t, dt = np.linspace(-lim, lim, n, endpoint=False, retstep=True)
        else:
            dt = t[1] - t[0]
        f_max = 1 / dt
        w = np.linspace(-np.pi * f_max, np.pi * f_max, n, endpoint=False)
        y = f(t)
        if windowing:
            y *= fftshift(0.54 + 0.46 * np.cos(2 * np.pi * np.arange(n) / n))
        y[0] = 0  # the sample corresponding to -tmax should be set zero
        y = fftshift(y)  # make y start with y(t=0)
        Y = fftshift(fft(y)) / n

        return w, Y

    def plotter(w, Y, title, lim, out, xlabel, ylabels):
        """Plots the passed DFT spectrum."""
        plt.figure()
        plt.subplot(2, 1, 1)
        plt.title(title)
        plt.ylabel(ylabels[0], size=16)
        plt.plot(w, np.abs(Y), lw=2)
        plt.xlim(-lim, lim)
        plt.grid(True)

        plt.subplot(2, 1, 2)
        plt.xlabel(xlabel, size=16)
        plt.ylabel(ylabels[1], size=16)
        phase = np.angle(Y)
        phase[np.where(np.abs(Y) < 3e-3)] = 0
        plt.plot(w, phase, "ro", lw=2)
        plt.xlim(-lim, lim)
        plt.grid(True)

        plt.savefig("Assignment_09/LaTeX/" + out)
\end{lstlisting}

\subsection{Spectrum of $\cos^3(0.86t)$}
The FFT with and without the Hamming window have been plotted:
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.7]{q2a.png}
\end{figure}
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.7]{q2b.png}
\end{figure}
It is observed that a large part of the energy is stored in frequencies that are not a part of the signal.
After windowing, these frequencies are attenuated and hence the peaks are sharper in the windowed function.
However, it is not an impulse because convolution with the Fourier transform of the windowed function smears out the peak.

\subsection{Estimate $\omega_0$ and $\delta$}\label{ssec:estimate}
We need to estimate $\omega_0$ and $\delta$ for a signal $\cos(\omega_0 t + \delta)$ for 128 samples between $[-\pi,\pi)$,
and find the two peaks at $\pm\omega_0$, and estimate $\omega_0$ and $\delta$. We estimate $\omega_0$ using a weighted average
over $|Y(\omega)|^2$, and $\delta$ is the phase at the frequency nearest to $\omega_0$.
\begin{lstlisting}
    def estimate_params(w, Y):
    """Estimates the parameters omega and delta of cos(omega*t + delta)."""
        ii = np.where(w > 0)
        omega = np.sum(np.abs(Y[ii]) ** 2 * w[ii]) / np.sum(np.abs(Y[ii]) ** 2)
        i = np.argmin(np.abs(w - omega))
        delta = np.angle(Y[i])

        return omega, delta
\end{lstlisting}
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.7]{q3.png}
\end{figure}

\subsection{$\cos^3(0.86t)$ with White Gaussian Noise}
We perform the same process as \ref{ssec:estimate} but with noise added to the original signal.
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.7]{q4.png}
\end{figure}

For the true value of $\omega_0 = 1.5$ and $\delta = 0.5$, the estimated values are:
\begin{lstlisting}
    Estimated w_0 = 1.516318, delta = 0.506776 without noise
    Estimated w_0 = 2.098451, delta = 0.481129 with noise
\end{lstlisting}

\subsection{DFT of Chirped Signal}
We analyze a chirp signal -- an FM signal where frequency is directly proportional to time:
\begin{equation*}
    f(t) = \cos(16t\left(1.5 + \frac{t}{2\pi}\right))
\end{equation*}

We observe that the frequency response is spread between 5 and 50 rad/s.
A large section of this range apears due to Gibbs phenomenon.
On windowing, only frequencies between 16 and 32 rad/s remain.
The FFT of the chirp is as follows:
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.7]{q5a.png}
\end{figure}
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.7]{q5b.png}
\end{figure}

\subsection{Surface Plot}
For the same chirped signal, the 1024 vector is broken into pieces that are 64 samples wide. Then we extract the
DFT of each and store as a column in a 2D array. Then we plot the array as a surface plot to show how the
frequency of the signal varies with time:
\begin{figure}[H]
    \centering
    \includegraphics[scale=0.7]{q6.png}
\end{figure}

\section{Conclusion}
We investigated the need of windowing for DFTs in the case of non-periodic signals in this assignment.
This is done to reduce the effect of Gibbs phenomena caused by the discontinuous nature of the series
$\tilde{x}[n]$ generated by a discrete fourier transform.

The final question concerns chirped signals, in which we plot fourier spectra for various time slices of a
signal. We took more closely spaced slices after noticing the instance of a small number of slices.

The existence of two peaks, disappearing of chirp effects in case of a windowed transform, and a
phase plot that regularly varies with reduced phase near maximas are all visible aspects of a fourier spectra
for a chirped signal in time changing plots.

\end{document}


%! Author = tstreule

\section{Ultrasound Imaging \textnormal{\normalsize $c\ped{sound} \simeq \unitfrac[1540]{m}{s}$}}

Frequencies of 1-50 MHz $\implies$ $\lambda = 1 .... 0.03 \unit{mm}$

Wave formula: $\nabla^2 p - \frac{1}{c^2} \frac{\partial^2 p}{\partial t^2} = 0$ \quad Max. p = compressional pressure= $p_c$, \quad Min. p = rarefactional pressure = $p_r$

$\kappa$ = compressibility, $\rho$ = density, $u_z$ = particle velocity\\
$c = \frac{1}{\sqrt{\kappa \rho}}$, \quad $p = \rho c u_z$, \quad $Z = p/u_z = \rho c = \sqrt{\frac{\rho}{\kappa}}$, \quad $I = p u_z / 2$

Pressure \textbf{coeff.}: \highlight{$\displaystyle r = \frac{Z_2 - Z_1}{Z_2 + Z_1}$} \highlight{$\displaystyle t = \frac{2Z_2}{Z_2 + Z_1}$}\\
\textbf{Refraction:} $\theta_i = \theta_r$, \quad $\sin\theta_i/\sin\theta_t = c_i/c_t$

\textbf{Rayleigh scattering} by structures smaller $\lambda$ $\to$ speckles/noise ($\sigma_S \propto \lambda^{-4}$). It has a char. length of $\lambda/2$:

\textbf{Attenuation} (due to scattering and absorption): \\
\highlight{$\displaystyle p(z) = p_0 e^{-\alpha z} \approx p_0\,10^{-\frac{\textrm{att}}{\unit[20]{dB} }}$} ($[z]=\unit{cm}$, $\textrm{att} = \alpha_0\cdot z = \textrm{const}\cdot f\cdot z$) \\
\textbf{Decibel notation}: $\alpha_0 = 20 \log \left( \frac{p_0}{p(z)} \right) \frac{1}{z} = 8.686 \alpha [dB/cm]$

\textbf{Transducer}: Tx/Rx switch, Damping, matching layer, $f_0$
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Beam geometry}
%
\textbf{Near field (Fresnel)}zone with const. beamwidth of $2r$.\\
\textbf{Far field (Fraunhofer)} zone which starts at \highlight{$\displaystyle \textrm{NFB} \approx \frac{r^2}{\lambda_{tissue}}$}.\vspace{-3mm}\\
Beamwidth = lateral resolution \vspace{1mm}

\textbf{Angle} of beam: \highlight{$\displaystyle \theta = 2\arcsin\left( \frac{0.61 \lambda}{r}\right)$}

\textbf{Focusing} with acoustic lens: Focal distance, lateral resolution, aperture dimension. Depth of focus: Over which distance is it narrow.\vspace{-1mm} \\
\textbf{Axial resolution}: \highlight{$\textstyle \Delta z \geq \frac{\lambda}{2} = \frac{p_dc\ped{sound}}{2}$} \quad $\lambda$: pulse length

\textcolor{gray}{ \textbf{Range Gain}: The longer signals take to come back, the $\downarrow$. To compensate: Amplify the late signals exponentially }

\textbf{Thermal noise}: \fbox{$P_N = k_B \cdot T \cdot BW [\unit{W}]$} \hfill (BW: typ. 1MHz)

\textbf{SNR in dB} = Transmitted Power - power losses - $P_N$.\\
Power losses: Attenuation \& reflection coefficient (factor 20)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Transducer setups}
%
\textbf{Linear Array}. \textbf{pitch} $d \approx \lambda$ and \textbf{kerf} (gap). Sweep through. Good resolution $\implies$ small pitch $\implies$ large $\theta$

\begin{minipage}{0.64\linewidth}
    \textbf{Phased parallel operation}:\\
    \highlight{$\displaystyle \Delta \Phi = k \Delta s = kd\sin\alpha \mod 2\pi$}. Multiple solutions for \fbox{$d > \lambda/2$ $\to$} \textbf{grating lobes} (weaker since not time aligned)
\end{minipage} \hfill
\begin{minipage}{0.35\linewidth}
    \includegraphics[width=\linewidth]{US_PhasedParallelMode}
\end{minipage}

Receiving analog: delay elements, then sum all up.

\textbf{Variable focusing} (with shifting) and a combination of all

\textbf{Multi-dim. arrays}: 2D or 1.5D ($1/2$ D = ``\textbf{elevation}'').

\textbf{Curved arrays} (instead of flat): For small acoustic windows

\textbf{Annular Array} (circular sections): Simpler adjustable focus and circular symmetry = more \textbf{isotropic depiction}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Scanning modes}
%
\textbf{A-Mode} (Amplitude): time (distance) vs amplitude. \\
Usage: e.g. measuring the thickness

\textbf{M-Mode} (Motion): time vs depth. Amplitude with brightness. Must be tilted manually

\textbf{B-Mode} (Brightness): 2D spacial. Amplitude with brightness

\textbf{Scanning Procedures}:\\
$\bullet$ \textbf{Parallel scan} for large acoustic window.
$\bullet$ \textbf{Sector scan} for small ac. window.
$\bullet$ \textbf{Radial scan} for transd. in blood vessels (measuring the vessel wall).
$\bullet$ \textbf{Compound scan} from different directions $\to$ redundancy and reduce speckles.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Measuring the blood flow}
%
Observer at velocity $v$, receives $f\ap{eff} = f \frac{c + v}{c}$

Blood vessel flows under the receiver with angle $\theta$ rel. to the vertical to transducer:
\fbox{$f\ped{rec} = f_i \frac{2 f_i \cos \theta}{c} + \frac{f_i v^2 \cos^2 \theta}{c^2}$}\\
\begin{minipage}{0.6\linewidth}
    \highlight{$\displaystyle f_D = f_i - f\ped{rec} \approx \frac{2 f_i v \cos \theta}{c}$}
\end{minipage}
\begin{minipage}{0.4\linewidth}
    Must know $\theta$ \\$\implies$ do B-Mode scan first
\end{minipage}

Red blood cells $\approx 7 \unit{\mu m}$ wide $\implies$ Use high freq., typ. 5MHz

\textbf{Cont. setup}: Separated receiver/transmitter. Quadr. encoder: Mult. with cos ($\to$ real part) and sin ($\to$ imag part) $\to$ double sided spectrum. Neg. side is \textit{negative flow}. Then \textit{low-pass + high-pass} to remove (quasi) stationary echos.

\textbf{Pulsed measurement}: 1 transducer. Only receive signal during a narrow time window. Its delay = depth.

\begin{minipage}{\linewidth}
    \raggedleft
    \vspace{2mm}
    \includegraphics[width = \linewidth]{US_DopplerPulsed} \vspace{-37mm}\\
    \highlight{$\displaystyle -f\ped{prf}/2 < f\ped{D} < f\ped{prf}/2$}
    \vspace{37mm}
\end{minipage}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Appendix}
%
\begin{tabular}{ c@{$\;\laplace\;$}c c c@{$\;\laplace\;$}c }
    $f(at)$			& $\frac{1}{\abs{a}}F(s/a)$	&& $f(t-a)$		& $\eu^{-as}F(s)$\\
    $f(t)\eu^{at}$	& $F(s-a)$					&& $f'(t)$		& $sF(s) - f(0^+)$\\
    $t^n$			& $n!/s^{n+1}$				&& $t^n f(t)$	& $(-1)^n F^{(n)}(s)$\\
    $\sin(at)$		& $\frac{a}{s^2 + a^2}$		&& $\cos(at)$	& $\frac{s}{s^2 + a^2}$\\
    $\eu^{at}$		& $\frac{1}{s - a}$			&& $t^n \eu^{at}$& $\frac{n!}{(s-a)^{n+1}}$
\end{tabular}

\textbf{Constants}:\\
\begin{tabular}{r@{$\;=\;$}l}
    $h$			& $\unit[6.626\E{-34}]{JS} = \unit[4.135\E{-15}]{eV\,s}$,\qquad $\hbar = \frac{h}{2\pi}$\\
    $\epsilon_0$& $\unitfrac[8.85\E{-5}]{As}{Vm}$\\
    $\mu_0$		& $\unitfrac[4\pi\E{-7}]{N}{A^2}$\\
    $k\ped{B}$	& $\unitfrac[1.38\E{-23}]{J}{K} = \unitfrac[8.617\E{-5}]{eV}{K}$\\
    $q$			& $\unit[1.602\E{-19}]{C}$, \quad $m_e = \unit[9.109\E{-31}]{kg}$, \quad $m_p = \unit[1.672\E{-27}]{kg}$\\
    $m_ec^2$	& $\unit[511]{eV}$
\end{tabular}
$\unit[0]{^\circ C} = \unit[273.15]{K}$


\documentclass[a4paper,11pt]{book}
\usepackage{import}
\usepackage{preamb}

\makeindex

\begin{document}

\input{head}
\newpage
\input{Title}
% \section{Blocks and Community structure}

\begin{subbox}{subbox}{}
\centering
\Large{\textbf{Networks as Matrices}}
\end{subbox}

\begin{textbox}{Matrices in short}
Matrices are mathematical objects that can be thought as \textit{tables} of numbers. The size of a matrix is expressed as $m \times n$, for a matrix with $m$ rows and $n$ columns. \textbf{The order (row/column) is important}.

\textbf{$M_{ij}$} is a notation representing the element on \textbf{row} $i$ and \textbf{column} $j$.

%\centering

%\includegraphics[width=0.5\textwidth]{Matris.png}
%\textbf{Indices in a matrix}
\end{textbox}

\begin{textbox}{$A$ - Adjacency matrix}
The most natural way to represent a graph as a matrix is called the Adjacency matrix $A$. It is defined as a square matrix, such as the number of rows (and the number of columns) is equal to the number of nodes $N$ in the graph. Nodes of the graph are numbered from 1 to $N$, and there is an edge between nodes $i$ and $j$ if the corresponding position of the matrix $A_{ij}$ is not $0$.

\begin{itemize}
    \item A value on the diagonal means that the corresponding node has a \textbf{self-loop}
    \item the graph is \textbf{undirected}, the matrix is \textbf{symmetric}: $A_{ij}=A_{ji}$ for any $i,j$.
    \item In an \textbf{unweighted} network, and edge is represented by the value $1$.
    \item In a \textbf{weighted} network, the value $A_{ij}$ represents the \textbf{weight} of the edge $(i,j)$

\end{itemize}
\end{textbox}

\begin{textbox}{Typical operations on $A$}
Some operations on Adjacency matrices have straightforward interpretations and are frequently used, such as \textbf{Multiplying} $A$ by \textbf{itself} and \textbf{Multiplying} $A$ by a \textbf{column vector}
\end{textbox}

\begin{textbox}{Multiplying $A$ by itself}

\textbf{Multiplying} $A$ by \textbf{itself} allows to know the number of walks of a given length that exist between any pair of nodes: $A^2_{ij}$ corresponds to the number of walks of length 2 from node $i$ to node $j$, $A^3_{ij}$ to the number of walks of length 3, etc.
\end{textbox}

\begin{textbox}{Multiplying $A$ by a column vector}

\textbf{Multiplying} $A$ by a \textbf{column vector} $W$ of length $1\times N$ can be thought as setting the $i$ th value of the vector to the $i$th node, and each node \textit{sending} its value to its neighbors (for undirected graphs). The result is a column vector with $N$ elements, the $i$th element corresponding to the sum of the values of its neighbors in $W$. This is convenient when working with \textbf{random walks} or \textbf{diffusion} phenomenon.
\end{textbox}

\begin{textbox}{Spectral properties of $A$}
\textbf{Spectral Graph Theory} is a whole field in itself, and beyond the scope of this class. A few elements for those with a \textit{linear algebra} background:

\begin{itemize}
    \item The adjacency matrix of an undirected simple graph is symmetric, and therefore has a complete set of real eigenvalues and an orthogonal eigenvector basis.
    \item The set of eigenvalues of a graph is the spectrum of the graph.
    \item The $n$ eigenvalues are denoted as $\lambda_0 \leq \lambda_1 \leq \lambda_2 \leq \dots \lambda_{\max}$
    \item The largest eigenvalue $\lambda_{\max}$ lies between the average and maximum degrees.
    \item In a large, sparse random graph, $\lambda_{\max}\approx \langle k \rangle$
    \item The number of closed walks of length $k$ in $G$ equals $\sum^n_{i=0} \lambda^k_i$
    \item A graph is bipartite if and only if its spectrum is symmetric (i.e., if $\lambda$ is an eigenvalue, then so is $-\lambda$
    \item If $G$ is connected, then the diameter of $G$ is strictly less than its number of distinct eigenvalues
\end{itemize}

\end{textbox}

\begin{textbox}{Graph Laplacian}
The \textbf{Graph Laplacian}, or \textbf{Laplacian Matrix} of a graph is a variant of the Adjacency matrix, often used in \textit{Spectral Graph Theory}.

It is defined as $D-A$, with $D$ the \textit{Degree matrix} of the graph, defined as a $N \times N$ matrix with $D_{ii}=k_i$ and zeros everywhere else.
\end{textbox}

\begin{textbox}{Matrix notation - Example}
\begin{multibox}{2}
\begin{subbox}{white}{\textcolor{black}{Graph}}
\centering
%\includegraphics[width=\textwidth]{CheatSheet_pic.pdf}
\adjustbox{width=0.8\textwidth }{
%\scalebox{1}[0.5]{
%\rotatebox[90]{
\begin{tikzpicture}[scale=0.3,rotate=0][every node/.style={inner sep=0,outer sep=0}]
\clip (0,-0.5) rectangle (6,6.5);
\Vertex[x=1.781,y=1.331,size=0.3,opacity=0.8,label=1]{0}
\Vertex[x=3.083,y=3.151,size=0.3,opacity=0.8,label=2]{1}
\Vertex[x=2.831,y=0.200,size=0.3,opacity=0.8,label=6]{5}
\Vertex[x=4.058,y=1.136,size=0.3,opacity=0.8,label=5]{4}
\Vertex[x=4.219,y=5.643,size=0.3,opacity=0.8,label=3]{2}
\Vertex[x=2.386,y=5.800,size=0.3,opacity=0.8,label=4]{3}
\Edge[](0)(1)
\Edge[](0)(5)
\Edge[](0)(4)
\Edge[](1)(2)
\Edge[](1)(3)
\Edge[](1)(4)
\Edge[](1)(5)
\Edge[](5)(4)
\Edge[](4)(4)
\Edge[](2)(3)
\end{tikzpicture}
}%\end{resizebox}
\end{subbox}
\begin{subbox}{white}{\textcolor{black}{$A$ - Adjacency Mat.}}
\centering
\footnotesize
\[\begin{pmatrix}
  0  &  1  &  0  &  0  &  1  &  1 \\
  1  &  0  &  1  &  1  &  1  &  1 \\
  0  &  1  &  0  &  1  &  0  &  0 \\
  0  &  1  &  1  &  0  &  0  &  0 \\
  1  &  1  &  0  &  0  &  1  &  1 \\
  1  &  1  &  0  &  0  &  1  &  0
\end{pmatrix}
\]
\end{subbox}

\begin{subbox}{white}{\textcolor{black}{$D$ - Degree Matrix}}
\centering
\footnotesize
\[\begin{pmatrix}
  3  &  0  &  0  &  0  &  0  &  0 \\
  0  &  5  &  0  &  0  &  0  &  0 \\
  0  &  0  &  2  &  0  &  0  &  0 \\
  0  &  0  &  0  &  2  &  0  &  0 \\
  0  &  0  &  0  &  0  &  5  &  0 \\
  0  &  0  &  0  &  0  &  0  &  3
\end{pmatrix}
\]
\end{subbox}
\begin{subbox}{white}{\textcolor{black}{$L$ - Laplacian }}
\centering
\footnotesize
\[
\setlength\arraycolsep{1pt}
\begin{pmatrix}
  3  & -1  &  0  &  0  & -1  & -1 \\
 -1  &  5  & -1  & -1  & -1  & -1 \\
  0  & -1  &  2  & -1  &  0  &  0 \\
  0  & -1  & -1  &  2  &  0  &  0 \\
 -1  & -1  &  0  &  0  &  4  & -1 \\
 -1  & -1  &  0  &  0  & -1  &  3
\end{pmatrix}
\]
\end{subbox}
\begin{subbox}{white}{\textcolor{black}{ $A^2$}}
\centering
\footnotesize
\[\begin{pmatrix}
  3  &  2  &  1  &  1  &  3  &  2 \\
  2  &  5  &  1  &  1  &  3  &  2 \\
  1  &  1  &  2  &  1  &  1  &  1 \\
  1  &  1  &  1  &  2  &  1  &  1 \\
  3  &  3  &  1  &  1  &  4  &  3 \\
  2  &  2  &  1  &  1  &  3  &  3
\end{pmatrix}
\]
\end{subbox}
\begin{subbox}{white}{\textcolor{black}{ Random W. mat.}}
\centering
\footnotesize
\setlength\arraycolsep{3pt}
\[\begin{pmatrix}
  0 &  \frac{1}{5} &  0 &  0 &  \frac{1}{4} &  \frac{1}{3}\\[6pt]
  \frac{1}{3} &  0 &  \frac{1}{2} &  \frac{1}{2} &  \frac{1}{4} &  \frac{1}{3}\\[6pt]
  0 &  \frac{1}{5} &  0 &  \frac{1}{2} &  0 &  0\\[6pt]
  0 &  \frac{1}{5} &  \frac{1}{2} &  0 &  0 &  0\\[6pt]
  \frac{1}{3} & \frac{1}{5} &  0 &  0 &  \frac{1}{4} &  \frac{1}{3}\\[6pt]
  \frac{1}{3} &  \frac{1}{5} &  0 &  0 &  \frac{1}{4} &  0
 \end{pmatrix}
\]
\end{subbox}

\end{multibox}
\end{textbox}

\begin{textbox}{Laplace Operator}

Intuitively, the Laplace operator is a generalization of the second derivative, and is defined in discrete situations, for each value, as the sum of differences between the value and its "neighbors". e.g., in time, the 2\textsuperscript{nd} derivative \textit{acceleration} is the difference between current speed and previous speed. In a B\&W picture, it's the difference between the greylevel on current pixel and the greylevel of 4 or 8 closest pixels, and perform \textit{edge detection}. On a graph, with $W$ a column vector representing values on nodes, $LW$ computes for each node the difference to neighbors.
\end{textbox}

\begin{textbox}{Spectral properties of $L$}
Eigenvalues of the Laplacian have many applications, such as \textit{spectral clustering}, \textit{graph matching}, \textit{embedding}, etc. Assuming $G$ undirected with eigenvalues $\lambda_0 \leq \lambda_1 \leq \lambda_2 \leq \dots \lambda_n$, here are some interesting properties:

\begin{itemize}
    \item The smallest eigenvalue $\lambda_i$ equals 0
    \item The number of 0 eigenvalues gives the number of connected components
\end{itemize}

\end{textbox}

\begin{textbox}{Random Walk matrix}
Another useful matrix of a graph is the \textbf{Random Walk Transition Matrix} $R$. It is the column normalized version of the adjacency matrix. $R_{ij}$ can be understood as the probability for a random walker located on node $i$ to move to $j$.

\end{textbox}

\begin{textbox}{Going Further}
\begin{itemize}
    \item Introduction to spectral graph theory (\cite{nica2016brief})
    \item Survey on Graph Spectral Theory (\cite{spielman2012spectral})

    \item Book on Graph Spectral Theory (\cite{chung1997spectral})
    \item Spectral graph Clustering (\cite{nascimento2011spectral})
    \item Wavelets on graph (\cite{hammond2011wavelets})

\end{itemize}

\end{textbox}

\input{tail}

\end{document}