%----------------------------------------------------------------------
% ml-tools documentation
% Polynomial fit derivation
% Author: Daniel Clark
%
% References:
% Pattern Recognition and Machine Learning - Christopher Bishop
%----------------------------------------------------------------------

%----------------------------------------------------------------------
% Document class, packages, and formatting
%----------------------------------------------------------------------
\documentclass{article}

%% Packages
\usepackage{fancyhdr}    % Required for custom headers
\usepackage{lastpage}    % Required to determine the last page for the footer
\usepackage{extramarks}  % Required for headers and footers
\usepackage{graphicx}    % Allow figures to be scaled and inserted
\usepackage{float}       % Allow figures to be inserted immediately
%\usepackage{indentfirst} % Allow indentation of all paragraphs, including first, by default
\usepackage{amsmath}     % Allow for extended math symbols library
\usepackage{amssymb}     % allow for special math binary operator characters
\usepackage{hyper ref}   % Hyperlinks
\usepackage[all]{hypcap} % Go to top of the image in link
\usepackage{amsfonts}    % Allow for extended fonts library
\usepackage{dsfont}      % Allow for digit formatting for pulse equation
\usepackage{upgreek}     % allow for variants on greek letters (e.g. tau)
\usepackage{bm}          % allow for bold math symbols via \bm
\usepackage{url}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{listings}    % code format listing
\usepackage{xcolor}

%% Margins
\topmargin=-0.45in
\evensidemargin=0in
\oddsidemargin=0in
\textwidth=6.5in
\textheight=9.0in
\headsep=0.1in

%% Set up the header and footer
\pagestyle{fancy}
\rhead{Daniel Clark} % Top right header
\chead{} % Top center header
\lhead{}
\cfoot{} % Bottom center footer
\rfoot{Page\ \thepage\ of \pageref{LastPage}} % Bottom right footer
\renewcommand\headrulewidth{0.4pt} % Size of the header rule
\renewcommand\footrulewidth{0.4pt} % Size of the footer rule

%% Other formatting

%% Inputs

%----------------------------------------------------------------------
% Title Block
%----------------------------------------------------------------------

% Title
\title{Polynomial fit derivation notes\\
Used in \emph{polyfit.py}}

% Author name
\author{Daniel Clark}

\begin{document}

% Make the title, author name, date
\maketitle

%\newpage
%\tableofcontents
%\newpage
%-----
\section*{Polynomial curve-fitting}
%-----
Given a real-valued input variable $x$, we wish to predict a real-valued target variable $t$.
\\\\
We can fit a polynomial to an existing set of $N$ data points, with row vector inputs $\bm{x} \equiv (x_1, ..., x_n)$ and target variables $\bm{t} \equiv (t_1, ..., t_n) $. For a given input $x_n$
% Polynomial fit y(x,w)
\begin{align}
    t_n \approx y(x_n, \mathbf{w}) = \sum_{j=0}^M w_j x_n^j,
\end{align}
where $\mathbf{w} \equiv (w_0, ..., w_M)^\top$ and $y(x_n, \mathbf{w})$ predicts the target $t_n$ using an $M+1$-dimensional vector of weights, corresponding to each term of the $M$-order polynomial (plus the line offset $w_0$). *Note that $y(x_n, \mathbf{w})$ is a nonlinear function of $x_n$, but a linear function of the coefficients $\{w_j\}$; these type of models are known as \emph{linear models}.
\\\\
These weights can be determined by minimizing an \emph{error function}, which measures the misfit between the approximation $y(x_n, \mathbf{w})$ and the training set for any given value of $\mathbf{w}$. A widely used error function is the sum-of-squares, measuring the sum of the distance-squared between the predicted point $y(x_n, \mathbf{w})$ and the actual target $t_n$, from $n=1, ..., N$.

% Sum-of-squares error function E(w)
\begin{align}
    E(\mathbf{w}) = \frac{1}{2}\sum_{n=1}^N \hspace{1pt}\big[y(x_n, \mathbf{w}) - t_n\big]^2
\end{align}

The error function can be solved in closed form to find the optimal solution, $\mathbf{w}^*$, where $E(\mathbf{w}^*)$ is minimized.
%-----
\subsubsection*{Derivation}
%-----
Setting $y_n \equiv y(x_n, \mathbf{w})$, we have

% Derive w*
\begin{align*}
    E(\mathbf{w}) &= \frac{1}{2}\sum_{n=1}^N \hspace{1pt} \big[y(x_n, \mathbf{w}) - t_n \big]^2 \\
                  &= \frac{1}{2}\sum_{n=1}^N \hspace{1pt} \big[y_n - t_n \big]^2 \\
                  &= \frac{1}{2}\sum_{n=1}^N \hspace{1pt} \big[y_n^2 - 2y_nt_n + t_n^2 \big] \\
                  &= \frac{1}{2}\sum_{n=1}^N y_n^2 - \sum_{n=1}^Ny_nt_n + \frac{1}{2}\sum_{n=1}^Nt_n^2 \\
                  &= \frac{1}{2}\sum_{n=1}^N y(x_n, \mathbf{w})y(x_n, \mathbf{w}) - \sum_{n=1}^Ny(x_n, \mathbf{w})t_n + \frac{1}{2}\sum_{n=1}^Nt_n^2
\end{align*}

Since $E(\mathbf{w})$ is a quadratic function of $\mathbf{w}$, its global minimum is found by setting its derivative with respect to $\mathbf{w}$ to 0. First, let`s take the partial derivative of $y(x_n, \mathbf{w})$ with respect to any component of $\mathbf{w}$, $w_i$

% Deriv of y(x_n, w)
\begin{align*}
    \frac{\partial}{\partial w_i}y(x_n, \mathbf{w}) &= \frac{\partial}{\partial w_i} \sum_{j=0}^Mw_jx^j \\
                                                    &= \frac{\partial}{\partial w_i} \bigg(... + w_{i-1}x^{i-1} + w_ix^i + w_{i+1}x^{i+1} + ... \bigg) \\
                                                    &= ... + \frac{\partial}{\partial w_i}w_{i-1}x^{i-1} + \frac{\partial}{\partial w_i}w_ix^i + \frac{\partial}{\partial w_i}w_{i+1}x^{i+1} + ... \\
                                                    &= ... + 0 + x^i + 0 + ... \\
    \frac{\partial}{\partial w_i}y_n = \frac{\partial}{\partial w_i}y(x_n, \mathbf{w}) &= x^i
\end{align*}

And we can take the derivative of $y(x_n, \mathbf{w})y(x_n, \mathbf{w})$ using the product rule, where

% Deriv of y(x_n, w) y(x_n, w)
\begin{align*}
    \frac{\partial}{\partial w_i} \big[y(x_n, \mathbf{w}) y(x_n, \mathbf{w}) \big] &= \frac{\partial}{\partial w_i}y(x_n, \mathbf{w}) \cdot y(x_n, \mathbf{w}) + y(x_n, \mathbf{w}) \cdot \frac{\partial}{\partial w_i}y(x_n, \mathbf{w}) \\
                                                                                &= 2 \frac{\partial}{\partial w_i} y(x_n, \mathbf{w}) \cdot y(x_n, \mathbf{w}) \\
                                                                                &= 2 x^i y(x_n, \mathbf{w})
\end{align*}

Finally, taking $\frac{\partial}{\partial w_i} E(\mathbf{w})$ and setting it equal to 0 (where we are using the sum rule, $(f + g)' = f' + g'$, to take the derivatives on the inside of the summations),

% Deriv of E(w)
\begin{align*}
    \frac{\partial}{\partial w_i}E(x_n, \mathbf{w}^*) = 0 &= \frac{1}{2}\sum_{n=1}^N \frac{\partial}{\partial w_i} \big[y(x_n, \mathbf{w}^*)y(x_n, \mathbf{w}^*) \big] - \sum_{n=1}^N\frac{\partial}{\partial w_i}y(x_n, \mathbf{w}^*)t_n + \frac{1}{2}\sum_{n=1}^N\frac{\partial}{\partial w_i}t_n^2 \\
                                                  0       &= \frac{1}{2}\sum_{n=1}^N 2x_n^i y(x_n, \mathbf{w}^*) - \sum_{n=1}^Nx_n^it_n + 0 \\
                                                          &= \sum_{n=1}^N x_n^i y(x_n, \mathbf{w}^*) - \sum_{n=1}^Nx_n^it_n \\
                                     \sum_{n=1}^Nx_n^it_n &= \sum_{n=1}^N x_n^i \sum_{j=0}^Mw_j^*x_n^j \\
                                     \sum_{n=1}^Nx_n^it_n &= \sum_{n=1}^N \sum_{j=0}^Mw_j^*(x_n)^{i+j} \\
                                     \sum_{n=1}^Nx_n^it_n &= \sum_{j=0}^Mw_j^*\sum_{n=1}^N(x_n)^{i+j}
\end{align*}

We can re-arrange the above and represent the equations with vectors and matrices in the form of $\mathbf{A}\mathbf{w} = \mathbf{b}$, where

% Aw = b
\begin{align*}
    \sum_{j=0}^Mw_j^* \underbrace{\sum_{n=1}^N(x_n)^{i+j}}_{a_{ij}} &= \underbrace{\sum_{n=1}^Nx_n^it_n}_{b_i} \\
    \sum_{j=0}^Ma_{ij} w_j^* &= b_i \\
    \text{where $i$ are the vectors' index, from $0..M$}\rightarrow \mathbf{A} \mathbf{w}^* &= \mathbf{b} \\
    \rightarrow \mathbf{w}^* &= \mathbf{A}^{-1}\mathbf{b}
\end{align*}

To solve this efficiently, we can use matrix algebra to create $\mathbf{A}$ and $\mathbf{b}$ using a $N\times M+1$ matrix $\mathbf{X}$, where

% X => Aw = b
\begin{align*}
    \mathbf{X} =
    \begin{bmatrix}
        x_1^0 & x_1^1 & ... & x_1^M \\
        x_2^0 & x_2^1 & ... & x_2^M \\
        \vdots & \vdots & ... & \vdots \\
        x_N^0 & x_N^1 & ... & x_N^M
    \end{bmatrix}, &~\bm{t} =
        \begin{bmatrix}
            t_1 & ... & t_N
        \end{bmatrix} \\
        \text{and}~\mathbf{A} = \mathbf{X}^\top\mathbf{X}, &~\mathbf{b} = \bm{t}\mathbf{X}
\end{align*}.

There's also a problem when choosing the degree of the polynomial, $M$. A high $M$ can lead to over-fitting; we want to achieve a good \emph{generalization}.
\\\\
One way of testing the model is by comparison of $E(\mathbf{w}^*)$ across model parameters (e.g. size of $M$). A good way to incorparte different-sized datasets is via root-mean-squared error (RMS). With $E(\mathbf{w}^*) \equiv \frac{1}{2}\sum_{n=1}^N [y(x_n, \mathbf{w}^*) - t_n]^2$,

% RMS
\begin{align}
    E_{RMS} &= \sqrt{2E(\mathbf{w}^*)/N}
\end{align}

\end{document}


\subsection{Defining Work on a Gas}
As you may remember from the previous unit, work is generally defined as
\begin{equation}
    W = \int_{x_1}^{x_2} Fdx
\end{equation}
However, in thermodynamics we don't have position or force as variables to work with. Instead, we have pressure and volume. So, we need to find a way to define force and position in terms of pressure and volume. Starting with force, I can say that $F = PA$, so substituting that into the equation we get
\begin{equation*}
    W = \int_{x_1}^{x_2} PAdx
\end{equation*}
The product of area and distance is volume (Think of a cube. What's the area of on of the cube's faces times the length of its side?). So, since dx is a tiny displacement, which ultimately is a length, Adx is a tiny change in volume! Therefore:
\begin{equation*}
    W = \int_{V_1}^{V_2} PdV
\end{equation*}
However, there's one more issue with this. The way I've defined work so far was based on the definition of the amount of work done \textbf{by} a system on another system. We want the amount of work done \textbf{on} the system. Due to conservation of energy, I can just throw a negative sign in front to get
\begin{equation}
    W = -\int_{V_1}^{V_2} PdV
\end{equation}
This is the proper definition of the work done on a gas. One thing to be careful of here; It looks like the pressure $P$ is just a constant I can take out of the integral. This is only the case if the pressure is \textit{actually a constant over the entire process}. Otherwise, the pressure really is a function of volume, and so we keep it inside the integral; to be extra clear that the pressure is changing as a function of volume, we should write the equation as:
\begin{equation}
    \label{eqn:(16)}
    W = -\int_{V_1}^{V_2} P(V)dV
\end{equation}
The below diagram illustrates gas in a box with a piston, and shows the work done by the gas (because it's easier to draw):
\begin{center}
    \begin{tikzpicture}[scale=4]
    \filldraw[fill=pink, draw = black] (0,0) rectangle (1,1);
    \draw[black] (0,1) rectangle (1,1.2);
    \draw (0,1.2) -- (0,1.5);
    \draw (1,1.2) -- (1,1.5);
    \draw[->] (0.5,0.8) -- (0.5,1);
    \node[right] at (0.5,0.9) {Force $F$};
    \node[left] at (0,1) {$x_1$};
    \draw (0.5,0.5) node {Gas of volume $V_1$};
    \draw (0.5, 1.1) node {Piston of area $A$};
    \draw[->, blue, thick] (1.2,0.75) -- (1.8, 0.75);
    \node[above] at (1.5,0.75) {Gas does work $W$};
    \node[below] at (1.5,0.75) {$\displaystyle W = \int_{x_1}^{x_2} Fdx$};
    \node[below] at (1.5,0.4) {$\displaystyle \phantom{W }=  \int_{V_1}^{V_2} PdV$};
    \filldraw[fill=pink, draw = black] (2,0) rectangle (3,1.1);
    \draw[black] (2,1.1) rectangle (3,1.3);
    \draw[dashed] (2,1) -- (3,1);
    \draw (2,1.2) -- (2,1.5);
    \draw (3,1.2) -- (3,1.5);
    \node[left] at (2,1.1) {$x_2$};
    \draw (2.5,0.5) node {Gas of volume $V_2$};
    \draw (2.5, 1.2) node {Piston of area $A$};
    \end{tikzpicture}
\end{center}


\chapter{Bases}

The plural of \enquote{basis} is \enquote{bases}.
Don't confuse it with the plural of \enquote{base}.

\Table{
    \Columns{ll}
    \Caption{Basis-related notations}
    \Head{object & type}
    \Body{
        vector & \(V\)
        \\ tuple & \( \Real^n \)
        \\ parametric curve & \(\Real \to V\)
        \\ basis & \( \Real^n \to V \)
        \\ covector, scalar field & \(V \to \Real\)
        \\ cobasis & \( V \to \Real^n \)
        \\ vector change, vector field & \(V \to V\)
        \\ basis change & \( \Real^n \to \Real^n \)
    }
}

A basis invertibly maps a tuple to a vector.

Let \(V\) be a vector space.

A basis of \(V\) is an invertible function with type \( \Real^n \to V \).

With a basis, we can describe a vector by writing a tuple of numbers instead of drawing an arrow.

\section*{Example basis}

Let \(V\) be the space of all two-dimensional Euclidean vectors.

Thus the basis we describe here will have the type \( \Real^2 \to V \).

Let \(i\) be the unit vector pointing east (right).

Let \(j\) be the unit vector pointing north (up).

Note that \(i\) and \(j\) are orthogonal.

Then we can choose a basis \(e\) such that \(e(x,y) = xi+yj\).

In this basis, the tuple \((1,1)\) describes the vector
that has length \(\sqrt{2}\) and points northeast.

\section*{Example basis change}

Suppose that we rotate the basis \(e\)
(we rotate the coordinate axes) by 90 degrees counterclockwise.
Let the new basis be \(f\).
Then \(f(x,y) = -yi + xj\).

\section*{Don't confuse vectors and tuples}

The tuple is not the vector itself.
The tuple describes the vector.
The tuple and the vector are two different things.
They are related by the basis.

But we can indeed form an abstract-algebraic vector space \(\Real^n\) over the abstract-algebraic field \(\Real\),
so a real tuple is a vector.

\section{Linear basis}

A basis \(e : \Real^n \to V\) is linear iff \(e(a+b) = e(a) + e(b)\).
Note that we overload the plus sign.
The left plus sign is tuple addition.
The right plus sign is vector addition.

We can explode a linear basis \(e : \Real^n \to V\) to \(n\) vectors \(e_1,\ldots,e_n\),
each of the type \(V\), in this way:
\begin{align*}
    e(x_1,\ldots,x_n) &= e_1 x_1 + \ldots + e_n x_n
    \\ &= \Matrix{e_1 & \ldots & e_n}\Matrix{x_1 \\ \vdots \\ x_n}
    \\ &= EX
\end{align*}
and thus the linear basis \(e\) can be represented by the matrix \(E\) where
\[
    E = \Matrix{e_1 & \ldots & e_n}
\]

\section{Describing every vector as a linear combination of basis vectors}

We are talking about the two-dimensional Euclidean space here.
We can imagine it as an unbounded flat sheet of paper.

Let \(V^2\) be the set of all two-dimensional Euclidean vectors.

From \(V^2\), pick any two non-collinear vectors \(e_1\) and \(e_2\).

Let \(x_1,x_2\in\Real\).

The linear combination \(x_1 e_1 + x_2 e_2\) describes a vector in \(V^2\).

If we pick a basis,
we can represent every vector in \(V^2\) using two real numbers.
We can describe the entire \(V^2\) using \(\Real^2\)
as \( V^2 = \{ x_1 e_1 + x_2 e_2 ~|~ (x_1,x_2) \in \Real^2 \} \).

We say that \(E = \{e_1,e_2\}\) is a basis of the two-dimensional Euclidean space.

We say that \((x_1,x_2)\) is the coordinate tuple of vector \(v\) according to basis \(E\).
We can also say that \((x_1,x_2)\) is the \(E\)-coordinates of \(v\).

A basis of \(V^n\) is a set of \(n\) basis vectors
in which every pair of basis vectors are non-collinear.
With such basis, we can describe every vector in \(V^n\)
as a linear combination of those basis vectors.

\section{Representing a coordinate tuple by a column matrix}

We can write \((x,y,z)\) or we can write
\[
    \Matrix{x \\ y \\ z}
\]

\section{Scaling a vector}

If \(k\) is a number and \(v\) is a vector,
then \(kv\) is a vector that has the same direction as \(v\),
but the length of \(kv\) is \(k\) times the length of \(v\),
that is, \( \norm{k v} = k \norm{v} \).

We can think of \(-v\) (the negation of \(v\)) as scaling \(v\) by \(-1\).

If \(v = \sum_k x_k e_k\) then \(cv = \sum_k (c x_k) e_k \).

\section{The relationship between vectors and coordinates}

We have two choices

\(V \to \Real^n\)

\(\Real^n \to V\)

\section{Exploding a cobasis to covectors}

We can explode a cobasis \( e : V \to \Real^n \) to \(n\) covectors \( e_1, \ldots, e_n \),
each having type \( V \to \Real \), in this way:
\[
    e(v) = (e_1(v), \ldots, e_n(v))
\]

\section{Why are the basis and cobasis not the other way around?}

\enquote{A choice of an ordered basis for \(V\) is equivalent to a choice of a linear isomorphism \(\varphi\)
from the coordinate space \(F^n\) to \(V\).}%
\footnote{\url{https://en.wikipedia.org/wiki/Basis_(linear_algebra)\#Ordered_bases_and_coordinates}}

\section{Radial basis? Polar coordinates?}

Let \(e\) be the radial basis. Then \(e(v+w) \neq e(v) + e(w)\) in general.

\section{The cross product? The Levi-Civita symbol?}

The vector \( a \times b \) is the vector that is orthogonal to \(a\), orthogonal to \(b\).
Follow the right hand rule.
If \(a\) is represented by the thumb pointing right,
and \(b\) is represented by the index finger pointing forward,
then \(a \times b\) is represented by the middle finger pointing up.

\( a \times b = \sum_i\sum_j\sum_k \epsilon_{ijk} e_i a^j b^k \) ?

\footnote{\url{https://en.wikipedia.org/wiki/Levi-Civita_symbol\#Cross_product_(two_vectors)}}

\section{The Pythagorean theorem}

Let there be a right triangle.
Let \(c\) be its hypothenuse.
Let \(a\) and \(b\) be the other two sides.
Then, \( a^2 + b^2 = c^2 \).

Many proofs of this theorem are on the Internet.

\section{Closing}

With a basis, we can define vectors with numbers without drawing.
After we have a basis, we can use the calculus of infinitesimals on vectors.


\chapter{The Stiff String}\label{ch:stiffString}
In earlier chapters, the case of the ideal string was presented, modelled using the 1D wave equation. This system, if the CFL condition is satisfied with equality, generates an output with harmonic partials that are integer multiples of the fundamental frequency. In the real world, however, strings exhibit a phenomenon called \textit{frequency dispersion} due to stiffness in the material, hence the name \textit{stiff string}. This phenomenon causes another effect known as \textit{inharmonicity}: the ``harmonic'' partials get exponentially further apart the higher their frequency is. The stiffness in a string is dependent on its material properties and geometry and will be elaborated on in this chapter. The stiff string played a prominent part in the following papers: \citeP[A], \citeP[B], \citeP[C], \citeP[D] and \citeP[E].

This chapter presents the PDE of the stiff string in continuous time, and goes through the discretisation process. The analysis techniques presented in Chapter \ref{ch:analysis} will then be applied to the resulting (explicit) FD scheme and derived in detail. Finally, an example of an implicit scheme will be given and comparison to the earlier FD scheme will be made. Unless denoted otherwise, this chapter follows \cite{theBible}.

\section{Continuous time}
Consider a lossless stiff string of length $L$ and with a circular cross-section. Its transverse displacement is described by $u=u(x,t)$ (in m) defined for $x\in \D$ with domain $\D = [0, L]$ and time $t\geq 0$. The PDE describing its motion is
\begin{equation}\label{eq:stiffStringPDENoLosses}
    \rho A \ptt u = T \pxx u - EI \pxxxx u
\end{equation}
parametrised by material density $\rho$ (in kg/m$^3$), cross-sectional area $A = \pi r^2$ (in m$^2$), radius $r$ (in m), tension $T$ (in N), Young's modulus $E$ (in Pa) and area moment of inertia $I = \pi r^4/4$ (in m$^4$). If $E = 0$, Eq \eqref{eq:stiffStringPDENoLosses} reduces to the 1D wave equation in Eq. \eqref{eq:1DwavePDE} where $c = \sqrt{T/\rho A}$, i.e., the ideal string. If instead $T = 0$, Eq. \eqref{eq:stiffStringPDENoLosses} reduces to the \textit{ideal bar} equation. A more compact way to write Eq. \eqref{eq:stiffStringPDENoLosses} is
\begin{equation}
    \ptt u = c^2 \pxx u - \kappa^2 \pxxxx u
\end{equation}
with wave speed $c = \sqrt{T/\rho A}$ (in m/s) and stiffness coefficient $\kappa = \sqrt{EI / \rho A}$.

The difference between the ideal string and the stiff string is the term containing a 4\thOrder spatial derivative. This term adds \textit{stiffness} to the system and causes frequency dispersion. As opposed to unwanted numerical dispersion due to numerical error (see Section \ref{sec:quality1DWave}) this type of dispersion is physical and thus something desired in the model. This phenomenon causes higher frequencies to travel faster through a medium than lower frequencies. See Figure \ref{fig:dispersion}. Furthermore, frequency dispersion is closely tied to \textit{inharmonicity}, an effect where `harmonic' partials get exponentially further apart as frequency increases. For low values of $\kappa$, the frequency of these partials can be expressed in terms of the fundamental frequency $f_0 = c/2L$ (as in Eq. \eqref{eq:fundamentalFreq}) and frequency of partial $p$ (in Hz) is defined as \cite{theBible}
\begin{equation}\label{eq:inharmonicityEquation}
    f_p = f_0 p \sqrt{1 + B p^2},
\end{equation}
with inharmonicity coefficient
\begin{equation*}
    B = \frac{\kappa^2 \pi^2}{c^2}.
\end{equation*}
Frequency dispersion and inharmonicity will be further discussed in Section \ref{sec:paramsAndOutput}.

\def\figWidth{0.32}
\begin{figure}[h]
    \centering
    \subfloat[$t = 0$ ms.\label{fig:dispersion1}]{\includegraphics[width=\figWidth\textwidth]{figures/resonators/dispersion1.eps}}\hfill
    \subfloat[$t = 1$ ms.\label{fig:dispersion2}]{\includegraphics[width=\figWidth\textwidth]{figures/resonators/dispersion2.eps}}\hfill
    \subfloat[$t = 2$ ms.\label{fig:dispersion3}]{\includegraphics[width=\figWidth\textwidth]{figures/resonators/dispersion3.eps}}
    \caption{Frequency dispersion in a stiff string due to stiffness.\label{fig:dispersion}}
\end{figure}

\subsubsection{Dispersion Analysis}
The 4th-order spatial derivative models \textit{frequency dispersion}, a phenomenon that causes different frequencies to travel at different speeds. As opposed to the undesired numerical dispersion

\subsection{Adding losses}
Before moving on to the discretisation of the PDE in Eq. \eqref{eq:stiffStringPDENoLosses}, losses can be added to the system. In the physical world, strings lose energy through fx. air viscosity and thermoelastic effects. All frequencies lose energy and die out (damp) over time, but higher frequencies do so at a much faster rate. This phenomenon is called \textit{frequency-dependent damping} and can be modelled using a mixed derivative $\pt \pxx$. This way of frequency-dependent damping first appeared in \cite{Bensa2003} and has been used extensively in the literature since \cite{Valimaki2006,theBible}. A damped stiff string can be modelled as
\begin{equation}\label{eq:stiffStringPDE}
    \rho A \ptt u = T \pxx u - EI \pxxxx u - 2 \sz \rho A \pt u + 2 \so \rho A\pt \pxx u
\end{equation}
where the non-negative loss coefficients $\sz$ (in s$^{-1}$) and $\so$ (in m$^2$/s) determine the frequency-independent and frequency-dependent losses respectively. Appendix \ref{app:intuitionSigma1} attempts to provide some intuition on workings of these damping terms.

A more compact way to write Eq. \eqref{eq:stiffStringPDE}, and is also found often in the literature \cite{theBible}\todo{etc.}, is to divide all terms by $\rho A$ to get
\begin{equation}\label{eq:stiffStringPDECompact}
    \ptt u = c^2 \pxx u - \kappa^2 \pxxxx u - 2 \sz \pt u + 2 \so \pt \pxx u,
\end{equation}
where $c=\sqrt{T/\rho A}$ is the wave speed \todo{FULL DOC SWEEP: check wavespeed or wave speed (entire document)} (in m/s) as in the 1D wave equation in \eqref{eq:1DwavePDE} and $\kappa = \sqrt{EI / \rho A}$ is referred to as the stiffness coefficient (in m$^2$/s).

\subsubsection{Intuition}
Although Eq. \eqref{eq:stiffStringPDE} might look daunting at first, the principle of Newton's second law remains the same.

Something about the 4th spatial derivative and the loss terms here...

\subsubsection{Boundary conditions}
Section \ref{sec:1DWave} presents two types of boundary conditions for the 1D wave equation in Eq. \eqref{eq:boundaryCond1DWave}. In the case of the stiff string, these can be extended to
\begin{subequations}\label{eq:stiffStringBoundConds}
    \begin{align}
        u = \px u &= 0 \quad \text{(clamped)}\label{eq:BCclamped}\\
        u = \pxx u &= 0 \quad \text{(simply supported)}\label{eq:BCsimplySupported}\\
        \pxx u = \pxxx u &= 0 \quad \text{(free)}\label{eq:BCfree}
    \end{align}
\end{subequations}
at $x = 0, L$. See Figure \ref{fig:boundaryCondsStiffString} for plots of the first modal shape for each respective boundary condition.
\def\figWidth{0.32}
\begin{figure}[h]
    \centering
    \subfloat[Clamped.\label{fig:clamped}]{\includegraphics[width=\figWidth\textwidth]{figures/resonators/clamped.eps}}\hfill
    \subfloat[Simply supported.\label{fig:simplySupported}]{\includegraphics[width=\figWidth\textwidth]{figures/resonators/simplySupported.eps}}\hfill
    \subfloat[Free.\label{fig:free}]{\includegraphics[width=\figWidth\textwidth]{figures/resonators/free.eps}}
    \caption{Plots of the first (normalised) modal shape for the three boundary conditions in Eqs. \eqref{eq:stiffStringBoundConds}. The extremes are indicated with solid black and dashed grey lines respectively. \label{fig:boundaryCondsStiffString}}
\end{figure}

\section{Discrete time}\label{sec:stiffStringDiscrete}
For the sake of compactness, Eq. \eqref{eq:stiffStringPDECompact} will be used in the following, rather than Eq. \eqref{eq:stiffStringPDE}. Naturally, the same process can be followed for the latter, the only difference being a multiplication by $\rho A$ of all terms.

Following Section \ref{sec:gridFunctions} and using the FD operators presented in Section \ref{sec:FDoperators}, Eq. \eqref{eq:stiffStringPDECompact} can be discretised as
\begin{equation}\label{eq:stiffStringFDS}
    \dtt \uln = c^2 \dxx \uln - \kappa^2 \dxxxx \uln - 2 \sz \dtd \uln + 2 \so\dtm\dxx \uln,
\end{equation}
for domain $l\in\{0, \hdots, N\}$ and number of grid points $N+1$.

The $\dxxxx$ operator is the second-order spatial difference in Eq. \eqref{eq:discSecondSpace} applied to itself
\begin{equation}\label{eq:dxxxx}
    \dxxxx = \dxx\dxx = \frac{1}{h^4}\left(e_{x+}^2 - 4e_{x+}+6 - 4e_{x-}+e_{x-}^2\right).
\end{equation}
A multiplication of two shift operators applied to a grid function simply means to apply each shift individually. The $\dxxxx$ operator applied to $\uln$ thus becomes
\begin{equation}
    \dxxxx\uln = \frac{1}{h^4}\left(u_{l+2}^n - 4u_{l+1}^n+6\uln - 4u_{l-1}^n+u_{l-2}^n\right).
\end{equation}

A definition for the mixed-derivative operator can similarly be found.
Recalling the definitions for $\dtm$ in Eq. \eqref{eq:backwardTimeOperator} and $\dxx$ Eq. \eqref{eq:discSecondSpace}, their combination results in
\begin{align}
    \dtm\dxx &= \frac{1}{k}\left(1-e_{t-}\right)\frac{1}{h^2}\left(e_{x+}-2+e_{x-}\right),\nonumber \\
    &= \frac{1}{kh^2}\left(e_{x+}-2+e_{x-} - e_{t-}(e_{x+}-2+e_{x-})\right).
\end{align}
To have two different shift operators multiplied together still simply means to apply each of them to the grid function individually. The $\dtm\dxx$ operator applied to $\uln$ thus yields
\begin{equation}
    \dtm \dxx \uln = \frac{1}{hk^2}\left(u_{l+1}^n - 2 \uln + u_{l-1}^n - u_{l+1}^{n-1} + 2 u_l^{n-1} - u_{l-1}^{n-1}\right).
\end{equation}
The reason a backwards difference is used here is to keep the system \textit{explicit}. A scheme is explicit if the values of $u_l^{n+1}$ can be calculated from known values at times $n$ and $n-1$. If this is not the case and values of fx. $u_{l+1}^{n+1}$ and $u_{l-1}^{n+1}$ are required to calculate $u_l^{n+1}$, the scheme is called \textit{implicit}. An example of an implicit scheme using the centred operator for the temporal derivative in the frequency-dependent damping term instead can be found in Section \ref{sec:implicitStiffString}.

Using the definitions above,
the operators in scheme \eqref{eq:stiffStringFDS} can be expanded to get
\begin{equation}
    \begin{aligned}\label{eq:expandedStringFDS}
        \frac{1}{k^2}\big(u_l^{n+1} - 2\uln & + u_l^{n-1} \big) =\frac{c^2}{h^2}\left(u_{l+1}^n - 2\uln + u_{l-1}^n\right) \\
        &- \frac{\kappa^2}{h^4}\left(u_{l+2}^n - 4u_{l+1}^n+6\uln - 4u_{l-1}^n+u_{l-2}^n\right) \\
        &- \frac{\sz}{k} \left(u_l^{n+1} - u_l^{n-1}\right)\\
        & + \frac{2\so}{kh^2}\left(u_{l+1}^n - 2\uln + u_{l-1}^n - u_{l+1}^{n-1} + 2u_l^{n-1} + u_{l-1}^{n-1}\right),
    \end{aligned}
\end{equation}
and after multiplication by $k^2$ and collecting the terms yields
\begin{equation}\label{eq:stiffStringUpdate}
    \begin{aligned}
        (1+\sz k) u_l^{n+1} =&\ \left(2 - 2\lambda^2 - 6\mu^2 - \frac{4\so k}{h^2}\right) \uln\\
        & + \left(\lambda^2 + 4\mu^2 + \frac{2\so k}{h^2}\right) (u_{l+1}^n + u_{l-1}^n) \\
        &- \mu^2 (u_{l+2}^n + u_{l-2}^n) + \left(-1+\sz k + \frac{4\so k}{h^2}\right)u_l^{n-1}\\
        & - \frac{2\so k}{h^2}(u_{l+1}^{n-1} + u_{l-1}^{n-1}),
    \end{aligned}
\end{equation}
with
\begin{equation}\label{eq:stiffStringCourant}
    \lambda = \frac{ck}{h} \qaq \mu = \frac{\kappa k}{h^2}.
\end{equation}
The update equation follows by dividing both sides by $(1+\sz k)$.

The stability condition for the FD scheme in \eqref{eq:stiffStringFDS} is defined as
\begin{equation}\label{eq:stiffStringStability}
    h \geq \sqrt{\frac{c^2k^2+4\so k + \sqrt{(c^2k^2 + 4\so k)^2+16\kappa^2k^2}}{2}},
\end{equation}
and will be derived in Section \ref{sec:stiffStringStability} using von Neumann analysis.
This condition can then be used to calculate the number of intervals $N$ in a similar fashion as for the 1D wave equation shown in Eq. \eqref{eq:orderOfCalc}. First, Eq. \eqref{eq:stiffStringStability} should be satisfied with equality, after which
\begin{equation*}
    N := \floor[\frac{L}{h}], \qaq h := \frac{L}{N}
\end{equation*}
which can then be used to calculate $\lambda$ and $\mu$ in \eqref{eq:stiffStringCourant}.

\subsubsection{Stencil}
As done in Section \ref{sec:1DWaveDisc}, a stencil for the FD scheme in Eq.
\eqref{eq:stiffStringFDS} can be created. This is shown in Figure \ref{fig:stencilStiffString}. In order to calculate $u_l^{n+1}$, $5$ points at the current time step are needed due to the 4\thOrder spatial derivative. Due to the mixed derivative in the frequency-dependent damping term, neighbouring points at the previous time step are also required.
%, the coefficient multiplied onto $u_l^{n+1}$.

% and the terms collected to obtain the following update equation
% \begin{equation}
%     \begin{aligned}
%     % (1+\sz k) u_l^{n+1} =&\ \left(2 - 2\lambda^2 - 6\mu^2 - \frac{4\so k}{h^2}\right) \uln\\
%     % & + \left(\lambda^2 + 4\mu^2 + \frac{2\so k}{h^2}\right) (u_{l+1}^n + u_{l-1}^n) \\
%     % &- \mu^2 (u_{l+2}^n + u_{l-2}^n) + \left(-1+\sz k + \frac{4\sz k}{h^2}\right)u_l^{n-1}\\
%     % & - \frac{2\so k}{h^2}(u_{l+1}^{n-1} + u_{l-1}^{n-1})
%     % \end{aligned}
%     Au_l^{n+1} = &\ B_0 \uln + B_1 (u_{l+1}^n + u_{l-1}^n) + B_2 (u_{l+2}^n + u_{l-2}^n) \\
%     &+ C_0 u_l^{n-1} + C_1(u_{l+1}^{n-1} + u_{l-1}^{n-1})
%     \end{aligned}
% \end{equation}
% with coefficients
% \begin{gather*}
%     B_0 = 2 - 2\lambda^2 - 6\mu^2 - \frac{4\so k}{h^2}, \quad B_1 = \lambda^2 + 4\mu^2 + \frac{2\so k}{h^2}, \quad B_2 =- \mu^2, \\[1em]
%     C_0 =  -1+\sz k + \frac{4\so k}{h^2},\quad C_1 = - \frac{2\so k}{h^2}, \qaq A = 1+\sz k.
% \end{gather*}
% Note that for clarity the division by $A$ has been left for implementation.
\begin{figure}[h]
    \centering
    \includegraphics[width=0.8\textwidth]{figures/resonators/stencilDampedStiffString.eps}
    \caption{The stencil for the damped stiff string scheme in Eq. \eqref{eq:stiffStringFDS} (adapted from \citeP[A]).\label{fig:stencilStiffString}}
\end{figure}

\subsection{Boundary conditions}\label{sec:stiffStringBoundaryConditions}
Due to the 4\thOrder spatial derivative, two virtual grid points need to be accounted for at the boundaries of the system. Discretising the boundary conditions in \eqref{eq:stiffStringBoundConds} yields
\begin{subequations}\label{eq:stiffStringBoundCondsDisc}
    \begin{align}
        \uln = \delta_{x\pm} \uln &= 0 \quad \text{(clamped)}\label{eq:BCclampedDisc}\\
        \uln = \dxx \uln &= 0 \quad \text{(simply supported)}\label{eq:BCsimplySupportedDisc}\\
        \dxx \uln = \dxd\dxx \uln &= 0 \quad \text{(free)}\label{eq:BCfreeDisc}
    \end{align}
\end{subequations}
at $l = 0, N$. The operator in the clamped condition uses the $\dxp$ operator at the left boundary ($l = 0$) and $\dxm$ at the right ($l = N$). Note that to discretise $\px^3$ in the free boundary condition in Eq. \eqref{eq:BCfree}, the more accurate $\dxd\dxx$ operator has been chosen over the less accurate $\dxm \dxx$ and $\dxp \dxx$ operators for the left and right boundary respectively.

Below, the boundary conditions are expanded to get definitions for the virtual grid points.

\todo{insert figure showing virtual grid points somewhere in this section}

\subsubsection{Clamped}
Expanding the operators for the clamped condition yields
\begin{equation}
    u_0^n = u_1^n = 0 \qaq u_{N-1}^n = u_N^n = 0.
\end{equation}
This can be simplified by reducing the range of calculation to $l\in \{ 2, \hdots, N-2\}$.

\subsubsection{Simply supported}
As the states of the end points of a system with simply supported boundary conditions are $0$ at all times, the range of calculation can be reduced to $l\in \{ 1, \hdots, N-1\}$. Evaluating the update equation in Eq. \eqref{eq:stiffStringUpdate} at $l=1$ and $l=N-1$ shows that definitions for the virtual grid points $u_{-1}^n$ and $u_{N+1}^n$ are required. A definition for $u_{-1}^n$ can be found by expanding Eq. \eqref{eq:BCsimplySupportedDisc} at $l = 0$:
\begin{align}
    &\frac{1}{h^2}\left(u_1^n - 2 u_0^n + u_{-1}^n\right) = 0,\nonumber\\[-1em]
    \xLeftrightarrow{\mystrut\ u^n_0 = 0\ } \quad & u_1^n + u_{-1}^n = 0,\nonumber\\[0.25em]
    &u_{-1}^n = -u_1^n,\label{eq:simplySupportedResult}
\end{align}
and similarly for $u_{N+1}^n$ by expanding the condition at $l=N$:
\begin{equation*}
    u_{N+1}^n = -u_{N-1}^n.
\end{equation*}
Substituting the first definition into the expanded scheme in Eq. \eqref{eq:expandedStringFDS} at $l=1$ and collecting the terms thereafter yields
\begin{equation}
    \begin{aligned}
        \!\!\!\!\!(1+\sz k) u_1^{n+1} =&\ \left(2 - 2\lambda^2 - 5\mu^2 - \frac{4\so k}{h^2}\right) u_1^n + \left(\lambda^2 + 4\mu^2 + \frac{2\so k}{h^2}\right) u_{2}^n \\
        &+ \left(-1+\sz k + \frac{4\sz k}{h^2}\right)u_1^{n-1} - \frac{2\so k}{h^2}u_{2}^{n-1}.
    \end{aligned}
\end{equation}
Doing the same for $l=N-1$ yields
\begin{equation}
    \begin{aligned}
        \!\!(1+\sz k) u_{N-1}^{n+1} =&\ \left(2 - 2\lambda^2 - 5\mu^2 - \frac{4\so k}{h^2}\right) u_{N-1}^n + \left(\lambda^2 + 4\mu^2 + \frac{2\so k}{h^2}\right) u_{N-2}^n \\
        &-\mu^2u_{N-3}^n+ \left(-1+\sz k + \frac{4\sz k}{h^2}\right)u_{N-1}^{n-1} - \frac{2\so k}{h^2}u_{N-2}^{n-1}.
    \end{aligned}
\end{equation}

\subsubsection{Free}
Finally, the free boundary condition requires all points to be calculated and the range of calculation remains $l\in\{0, \hdots, N\}$. At each respective boundary, two virtual grid points are needed: $u_{-1}^n$ and $u_{-2}^n$ at the left and $u_{N+1}^n$ and $u_{N+2}^n$ at the right boundary respectively.
The third-order spatial FD operator in Eq. \eqref{eq:BCfreeDisc} is defined as:
\begin{align}
    \dxd\dxx &= \frac{1}{2h^3}\left(e_{x+}-e_{x-}\right)\left(e_{x+}-2+e_{x-}\right),\nonumber\\
    &=\frac{1}{2h^3}\left(e_{x+}^2 - 2e_{x+} + 1 - (1 - 2e_{x-} + e_{x-}^2\right),\nonumber\\
    &=\frac{1}{2h^3}\left(e_{x+}^2 - 2e_{x+} + 2e_{x-} -e_{x-}^2\right).
\end{align}
and can be used to solve for $u_{-2}^n$ at $l=0$:
\begin{align*}
    \frac{1}{2h^3} &\left(u_2^n - 2 u_1^n + 2u_{-1}^n - u_{-2}^n\right) = 0,\\%[0.25em]
    u_{-2}^n &= u_2^n - 2 u_1^n + 2u_{-1}^n.
    % \\[-1em]
    % \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\xLeftrightarrow{\mystrut\ \dxx \uln = 0\ \Rightarrow\ \text{ Eq. \eqref{eq:simplySupportedResult}}\ }
    % \quad u_{-2}^n &= u_2^n - 4 u_1^n
\end{align*}
As $u_0^n$ is not necessarily $0$ at all times, solving the first part of the boundary condition (i.e., $\dxx u_0^n = 0$) yields a different result than in the simply supported case:
\begin{align*}
    \frac{1}{2h^3} &\left(u_2^n - 2 u_1^n + u_{-1}^n - u_{-2}^n\right) = 0\\[0.25em]
    u_{-2}^n &= u_2^n - 2 u_1^n + u_{-1}^n\\[-1em]
    \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\xLeftrightarrow{\mystrut\ \dxx \uln = 0\ \Rightarrow\ \text{ Eq. \eqref{eq:simplySupportedResult}}\ }
    \quad u_{-2}^n &= u_2^n - 3 u_1^n
\end{align*}
The same can be done at $l=N$ to get the following definitions for the virtual grid points
\begin{equation*}
    u_{N+2}^n = u_{N-2}^n - 2 u_{N-1}^n + 2u_{N+1}^n
    \qaq u_{N+1}^n = 2u_N^n - u_{N-1}^n.
\end{equation*}
The update equations for the boundary points will not be given here. Instead the matrix form of the FD scheme with free boundaries will be provided below.
% The update equation at $l=0$ then becomes
% \begin{equation}
%     \begin{aligned}
%         \!\!\!\!\!(1+\sz k) u_0^{n+1} =&\ \left(2 - 2\lambda^2 - 2 \mu^2 - \frac{4\so k}{h^2}\right) u_0^n + \left(\lambda^2 +4\mu^2 + \frac{4\so k}{h^2}\right) u_1^n \\
%         &-2\mu^2u_2^n+ \left(-1+\sz k + \frac{4\sz k}{h^2}\right)u_0^{n-1} - \frac{4\so k}{h^2}u_1^{n-1}.
%     \end{aligned}
% \end{equation}
% at $l=1$
% \begin{equation}
%     \begin{aligned}
%         \!\!\!\!\!(1+\sz k) u_1^{n+1} =&\ \left(2 - 2\lambda^2 - 5\mu^2 - \frac{4\so k}{h^2}\right) u_1^n + \left(\lambda^2 + 4\mu^2 + \frac{2\so k}{h^2}\right) u_{2}^n \\
%         &-2\mu^2u_3^n+ \left(-1+\sz k + \frac{4\sz k}{h^2}\right)u_1^{n-1} - \frac{2\so k}{h^2}u_{2}^{n-1}.
%     \end{aligned}
% \end{equation}
\\

In practice, the simply supported boundary condition is mostly chosen as this most realistically reflects string terminations in the real world. The clamped condition could be chosen for simplicity as this does not require an alternative update at the boundaries. The free boundary condition is more often used to model a (damped) ideal bar, (Eq. \eqref{eq:stiffStringPDE} with $T = 0$).

\subsection{Implementation and matrix form}\label{sec:implementationStiffString}
When using \texttt{MATLAB}, for a more compact implementation, it is useful to write the scheme in matrix form (see Section \ref{sec:matrixForm}). The FD scheme of the stiff string in \eqref{eq:stiffStringFDS} can be written as
\begin{equation}\label{eq:matrixFormStiffString}
    A\u^{n+1} = \B\u^n + \C \u^{n-1}
\end{equation}
where
\begin{equation}
    \begin{gathered}
    A = (1+\sz k), \quad \B = 2\I + c^2 k^2 \Dxx - \kappa^2 k^2 \Dxxxx + 2 \so k \Dxx, \\
    \text{and} \quad \C = -(1-\sz k)\I - 2\so k \Dxx.
    \end{gathered}
\end{equation}
Notice that $A$ is a scalar rather than a matrix.

The size of the state vectors and the matrix-form operators depend on the boundary conditions. For clamped conditions, the state vectors ($\u^{n+1}$, $\u^n$ and $\u^{n-1}$) and matrices will be of size $(N-3) \times 1$ and $(N-3) \times (N-3)$ respectively. The $\Dxx$ matrix will be of the form given in Eq. \eqref{eq:DxxDef} and the matrix form of the $\dxxxx$ operator is
\begin{equation}
    \mathbf{D}_{xxxx} = \frac{1}{h^4}\begin{bmatrix}
        6& -4 & 1 & & \mathbf{0} \\
        -4 & 6 &\ddots &\ddots & \\
        1& \ddots & \ddots & \ddots&1 \\
        &\ddots & \ddots & 6 & -4 \\
        \mathbf{0} & & 1& -4 & 6 \\
    \end{bmatrix}.
\end{equation}

For simply supported conditions, the state vectors and matrices will be of size $(N-1) \times 1$ and $(N-1) \times (N-1)$ respectively. Again, $\Dxx$ is as defined in Eq. \eqref{eq:DxxDef} and $\Dxxxx$ can be obtained by multiplying two $\Dxx$ matrices according to
\begin{equation}
    \Dxx\Dxx = \mathbf{D}_{xxxx} = \frac{1}{h^4}\begin{bmatrix}
        5& -4 & 1 & & & \mathbf{0}& \\
        -4 & 6 &\ddots &\ddots & & & \\
        1& \ddots & \ddots & -4 & 1 & & \\
        & \ddots& -4 & 6 & -4 & \ddots& \\
        & & 1 & -4 & \ddots & \ddots &1 \\
        & & & \ddots & \ddots & 6 & -4 \\
        & \mathbf{0} & & & 1& -4 & 5 \\
    \end{bmatrix}.
\end{equation}

Finally for free boundary conditions given in Eq. \eqref{eq:BCfreeDisc}, the state vectors and matrices are $(N+1)\times 1$ and $(N+1)\times (N+1)$ respectively. Now, the $\Dxx$ matrix is of the form in Eq. \eqref{eq:DxxDefNeumann} instead, and

\setstackgap{L}{16pt}
\setstacktabbedgap{8pt}
\def\lrgap{\kern3pt}
\fixTABwidth{T}
\begin{equation}
    \mathbf{D}_{xxxx} = \frac{1}{h^4}\xbracketMatrixstack{
        2& -4 & 2 & & & \mathbf{0}& \\
        -2 & 5 & -4 & 1 & & & \\
        1& -4 & 6 & -4 & 1 & & \\
        & \ddots & \ddots & \ddots & \ddots &\ddots & \\
        & & 1 & -4 & 6 & -4 &1 \\
        & & & 1 & -4 & 5 & -2 \\
        & \mathbf{0} & & & 2& -4 & 2}.
\end{equation}

\subsection{Parameters and output}\label{sec:paramsAndOutput}
The values of the parameters naturally determine the properties of the output sound. Where in the 1D wave equation, only the fundamental frequency $f_0$ could be affected (through $c$ and $L$ in Eq. \eqref{eq:fundamentalFreq}), the stiff string has many more aspects that can be changed. See Table \ref{tab:stiffStringParams} for parameters most commonly used in this project.
\begin{table}[h]
    \begin{center}
    \begin{tabular}{|l|c|c|}
        \hline
        Name & Symbol (unit) & Value\\ \hline
        Length & $L$ (m) & $1$\\
        Material density & $\rho$ (kg/m$^3$) & $7850$\\
        Radius & $r$ (m) & $5\cdot10^{-4}$\\
        Tension & $T$ (N) &$100 \leq T \leq 1.5\cdot 10^4$\\
        Young's modulus & $E$ (Pa) & $2\cdot10^{11}$\\
        Freq.-independent damping & $\sz$ (s$^{-1}$) & $1$\\
        Freq.-dependent damping & $\so$ (m$^2$/s) & $0.005$\\\hline
    \end{tabular}
    \caption{Parameters and their values most commonly used over the course of this project.\label{tab:stiffStringParams}}
    \end{center}
\end{table}
{\renewcommand{\arraystretch}{1}

A formula exists to calculate the loss coefficients $\sz$ and $\so$ from $T_{60}$ values at different frequencies (see \cite[Eq (7.29)]{theBible}). During this project, however, these values have been tuned by ear and are usually set to be approximately those found in Table \ref{tab:stiffStringParams}.

\subsubsection{Output}
Figure \ref{fig:stiffStringOutput} shows the time domain and frequency domain output (retrieved at $l = 3$) of an implementation of the stiff string excited using a raised-cosine. The parameters used can be found in Table \ref{tab:stiffStringParams}
with $T = 1129$ N. Furthermore, $E = 7\cdot 10^{11}$ Pa to highlight dispersive effects. Finally, simply supported boundary conditions are chosen. From the left panel, one can observe that over time, dispersive effects show where higher-frequency components in the excitation travel faster through the medium than lower-frequency components. In the frequency domain (the right panel in Figure \ref{fig:stiffStringOutput}) this shows in the fact that the partials are not perfect integer multiples of the fundamental. Notice that the partials are closer to each other for lower frequencies and further apart as their frequency increases. Finally, the frequency-dependent damping term causes higher frequencies to have a lower amplitude than lower frequencies.

Apart from the obvious material properties such as density, stiffness and geometry, perceptual qualities of the sound are surprisingly much determined by $\so$, and for lower values the output can become extremely metallic.

\begin{figure}[h]
    \includegraphics[width=\textwidth]{figures/resonators/outputFFT.eps}
    \caption{The time-domain and frequency domain output of the stiff string. The parameters are set as in Table \ref{tab:stiffStringParams} with $E = 7\cdot 10^{11}$ Pa to highlight dispersive effects and $T = 1129$ N. %From the left panel, one can observe that over time, dispersive effects show due to the stiffness in the string. In frequency domain (right panel) this shows through the partials not being perfect integer multiples of the fundamental. Lastly, higher frequency components have a lower amplitude due to the frequency-dependent damping.
    \label{fig:stiffStringOutput}}
\end{figure}

\section{von Neumann analysis and stability condition}\label{sec:stiffStringStability}
In order to obtain the stability condition for the damped stiff string, one can perform von Neumann analysis as presented in Section \ref{sec:stabilityAnalysis} on the FD scheme in Eq. \eqref{eq:stiffStringFDS}.

Using the definitions found in Eq. \eqref{eq:temporalAnsatz} for the temporal operators and Eqs. \eqref{eq:dxxAnsatz} and \eqref{eq:dxxxxAnsatz} for the spatial operators, the frequency domain representation of Eq. \eqref{eq:stiffStringFDS} can be obtained:
\begin{align*}
    \!\!\!\!\frac{1}{k^2}\left(z - 2 + z^{-1}\right) =&-\frac{4c^2}{h^2}\sin^2(\beta h/2) - \frac{16\kappa^2}{h^4}\sin^4(\beta h/2) - \frac{\sz}{k}z + \frac{\sz}{k}z^{-1}\\
    & - \frac{8 \so}{kh^2}\sin^2(\beta h/2) + \frac{8 \so}{kh^2} \sin^2(\beta h/2)z^{-1}
\end{align*}
and after collecting the terms, the characteristic equation is as follows
\begin{gather}
    (1+\sz k)z + \left(16\mu^2\sin^4(\beta h/2)+\left(4\lambda^2+\frac{8\so k}{h^2}\right)\sin^2(\beta h/2) - 2\right)\nonumber\\
    +\left(1-\sz k-\frac{8\so k}{h^2}\sin^2(\beta h/2)\right)z^{-1}=0.\label{eq:charDampedString}
\end{gather}
Rewriting this to the form \eqref{eq:polynomialForm}, and using $\S = \sin^2(\beta h /2)$ for brevity, yields
\begin{equation*}
    z^2 + \left(\frac{16\mu^2\S^2+\left(4\lambda^2+\frac{8\so k}{h^2}\right)\S - 2}{1 + \sz k}\right)z+\frac{1-\sz k-\frac{8\so k}{h^2}\S}{1 + \sz k}=0.
\end{equation*}
Stability of the system can then be proven using condition \eqref{eq:condition214}, and substituting the coefficients into this condition yields
\begin{equation}\nonumber
    \begin{aligned}
        \left|\frac{16\mu^2\S^2+\left(4\lambda^2+\frac{8\so k}{h^2}\right)\S - 2}{1+\sz k}\right|-1 &\leq \frac{1-\sz k-\frac{8\so k}{h^2}\S}{1+\sz k}\leq 1,\\
        \left|16\mu^2\S^2+\left(4\lambda^2+\frac{8\so k}{h^2}\right)\S - 2\right|-(1+\sz k) &\leq 1-\sz k-\frac{8\so k}{h^2}\S\leq 1+\sz k,\\
        \left|16\mu^2\S^2+\left(4\lambda^2+\frac{8\so k}{h^2}\right)\S - 2\right|&\leq2-\frac{8\so k}{h^2}\S\leq2+2\sz k.
    \end{aligned}
\end{equation}
The second condition is always true due to the fact that $\sigma_0,\sigma_1 \geq 0$. Continuing with the first condition:
\begin{align*}
    -2+\frac{8\so k}{h^2}\S&\leq 16\mu^2\S^2+\left(4\lambda^2+\frac{8\so k}{h^2}\right)\S - 2\leq 2-\frac{8\so k}{h^2}\S,\\
    0&\leq 16\mu^2\S^2+4\lambda^2\S\leq 4-\frac{16\so k}{h^2}\S.
\end{align*}
As $16\mu^2\S^2+4\lambda^2\S$ is positive definite, the first condition is always satisfied. Continuing with the second condition:
\begin{align*}
    16\mu^2\S^2+\left(4\lambda^2+ \frac{16\so k}{h^2}\right)\S &\leq 4,\\
    4\mu^2\S^2+\left(\lambda^2+ \frac{4\so k}{h^2}\right)\S&\leq 1.
\end{align*}
As $\S$ is bounded by $1$ for all values of $\beta$, one can set $\S = 1$ \SWcomment[check with Stefan]. Continuing with the substituted definitions for $\lambda$ and $\mu$ from Eq. \eqref{eq:stiffStringCourant} yields
\begin{align*}
    \frac{4\kappa^2k^2}{h^4}+\frac{c^2k^2 + 4\so k}{h^2}&\leq 1, \\
    4\kappa^2k^2+(c^2k^2+ 4\so k)h^2&\leq h^4,\\
    h^4- (c^2k^2+ 4\so k)h^2 - 4\kappa^2k^2 &\geq 0 ,
\end{align*}
which is a quadratic equation in $h^2$ with $h$ bounded by
\begin{equation}
    h \geq \sqrt{\frac{c^2k^2+4\so k + \sqrt{(c^2k^2 + 4\so k)^2+16\kappa^2k^2}}{2}}.
\end{equation}
This is the stability condition for the damped stiff string also shown in Eq. \eqref{eq:stiffStringStability}.

\section{Energy analysis}\label{sec:energyAnalysisString}
As mentioned in Section \ref{sec:energyAnalysis}, it is useful to perform the energy analysis on the scheme with all physical parameters written out. Discretising the PDE in Eq. \eqref{eq:stiffStringPDE} yields
\begin{equation}
    \rho A \dtt \uln = T \dxx \uln - EI \dxxxx \uln - 2\sz \rho A \dtd \uln + 2 \so \rho A \dtm \dxx \uln,
\end{equation}
defined for $l\in d$ with discrete domain $d = \{0, \hdots, N\}$. This section will follow the 4 steps described in Section \ref{sec:energyAnalysis}.

This section will be divided into $5$ steps, as done in Section \ref{sec:energyAnalysis}.

\subsubsection{Step 2: Identify energy types and isolate $\dtp$}
As there is damping present in the system, and the system is distributed, the energy balance will be of the form
\begin{equation}
    \dtp \h = \mathfrak{b}-\mathfrak{q}.
\end{equation}
with boundary term $\b$ and damping term $\q$. The latter is defined as \todo{virtual grid points needed for freq-dep damping term..}
\begin{equation}\label{eq:dampingTermStiffString}
    \mathfrak{q} = 2\sz \rho A \lVert\dtd\uln\rVert_d^2 - 2 \so \rho A \langle \dtd \uln, \dtm \dxx \uln \rangle_d,
\end{equation}
and $\mathfrak{b}$ appears after rewriting Eq. \eqref{eq:rOCStiffString} using summation by parts (see Section \ref{sec:summationByParts}). Specifically, using Eq. \eqref{eq:summationByPartsMinusBar} for the second term and Eq. \eqref{eq:summationByPartsTwiceReduced} for the third, yields
\begin{align*}
    \dtp \h &= \rho A \langle \dtd \uln, \dtt \uln \rangle_d + T \langle \dtd \dxp \uln, \dxp \uln\rangle_{\underline{d}} + EI \langle \dtd \dxx \uln, \dxx \uln \rangle_{\overline{\underline{d}}} \\
    &= \mathfrak{b} - \mathfrak{q}
\end{align*}
where the boundary term becomes
\begin{align*}
    \mathfrak{b} =&\ T\Big((\dtd u_N^n)(\dxp u_N^n) - (\dtd u_0^n) (\dxp u_{-1}^n)\Big) \\
    &+ EI \Big((\dtd u_N^n)(\dxp \dxx u_N^n) - (\dxx u_N^n)(\dxm \dtd u_N^n) \Big)\\
    &- EI \Big((\dtd u_0^n)(\dxm \dxx u_0^n) - (\dxx u_0^n)(\dxp \dtd u_0^n)\Big).
\end{align*}
For the clamped and simply supported boundary conditions in \eqref{eq:BCclampedDisc} and \eqref{eq:BCsimplySupportedDisc} it can easily be shown that $\mathfrak{b} = 0$. If free conditions as in Eq. \eqref{eq:BCfreeDisc} are used, the boundary conditions will vanish when the primed inner product in Eq. \eqref{eq:primedInnerProd} is used in Step 1 and identity \eqref{eq:summationByPartsTwicePrimed} is used when performing summation by parts. Here, only the clamped / simply supported case will be considered.

\subsubsection{Step 3: Check units}
Comparing the acquired energy balance in Eq. \eqref{eq:energyBalanceStiffString} to the energy balance for the 1D wave equation in Eq. \eqref{eq:energyBalance1DWave}, one can observe that the balances are nearly identical, the only difference being the second term in the definition for $\v$ in Eq. \eqref{eq:energyBalanceStiffString}.
Writing this term out in units, and recalling that Pa (the unit for $E$) in SI units is kg$\cdot$m$^{-1}\cdot$s$^{-2}$, yields
\begin{align*}
    \frac{EI}{2}\langle\dxx\uln, e_{t-}\dxx\uln\rangle_{\overline{\underline{d}}}\ \overset{\text{in units}}{\xrightarrow{\hspace*{1cm}}}& \quad \text{Pa}\cdot \text{m}^4 \cdot \text{m} \cdot (\text{m}^{-2} \cdot \text{m} \cdot \text{m}^{-2} \cdot \text{m}) \\
    & = \text{kg} \cdot \text{m}^2 \cdot \text{s}^{-2},
\end{align*}
and indeed has the correct units.

As described in Section \ref{sec:energyAnalysis}, the damping terms in $\mathfrak{q}$ need to have units of Joules per second, or kg $\cdot$ m$^2 \cdot$ s$^{-3}$. Writing the terms in Eq. \eqref{eq:dampingTermStiffString} out in their units yields
\begin{align*}
    2\sz \rho A \lVert\dtd\uln\rVert_d^2\ \overset{\text{in units}}{\xrightarrow{\hspace*{1cm}}}&\quad \text{s}^{-1}\cdot\text{kg}\cdot\text{m}^{-3}\cdot\text{m}^2\cdot \text{m}\cdot(\text{s}^{-1}\cdot\text{m})^2 \\
    &= \text{kg} \cdot \text{m}^2 \cdot \text{s}^{-3},\\
    - 2 \so \rho A \langle \dtd \uln, \dtm \dxx \uln \rangle_d\  \overset{\text{in units}}{\xrightarrow{\hspace*{1cm}}}&\quad \text{m}^2\cdot\text{s}^{-1}\cdot\text{kg}\cdot\text{m}^{-3}\cdot\text{m}^2\\
    &\qquad\cdot\text{m}\cdot(\text{s}^{-1}\cdot\text{m})(\text{s}^{-1}\cdot\text{m}^{-2}\cdot \text{m}),\\
    &= \text{kg} \cdot \text{m}^2 \cdot \text{s}^{-3},
\end{align*}
and also have the correct units.
\subsubsection{Step 4: Implementation}
An implementation of the energy calculation for the simply supported boundary condition is given in Algorithm \ref{alg:stiffStringEnergy}. The damping is ignored but can be found in Appendix \ref{app:stiffstring}. \todo{Add to appendix or refer to a gist} Figure \ref{fig:energyStiffString} shows that the damping present in the system causes $\h$ to decrease in the left panel. The right panel shows that the deviation of the total energy calculated using Eq. \eqref{eq:normalisedEnergyDamping} is within machine precision.

\setlstMAT
\begin{lstlisting}[caption=Calculating $\h$ for the simply supported boundary condition., label=alg:stiffStringEnergy]
%%%% Before the main loop: %%%%

% Initialise Dx+ operator to calculate potential energy due to tension
% As the domain is reduced by one, the matrix needs to be of size N x N
Dxp = sparse(1:N, 1:N, -ones(1, N), N, N) + ...
        sparse(1:N-1, 2:N, ones(1, N-1), N, N);

%%%% In the main loop: %%%%

% energy in the system
kinEnergy(n) = rho * A * h / 2 * sum((1/k * (u - uPrev)).^2);
potEnergy(n) = T / 2 * h * sum((Dxp * [0; u]) .* (Dxp * [0; uPrev])) ... + E * I * h / 2 * sum((Dxx * u) .* (Dxx * uPrev));
\end{lstlisting}

\section{Modal analysis}
% To find an expression for the modal frequencies one can ignore the damping terms for now and follow the process in Section \ref{sec:modalAnalysis} to obtain
% \begin{equation}
%     f_p = \frac{1}{\pi k}\sin^{-1}\left(\frac{1}{2}\sqrt{-\eig_p(c^2k^2\Dxx - \kappa^2k^2\Dxxxx)}\right).
% \end{equation}

To be able to perform a modal analysis on the FD scheme in \eqref{eq:stiffStringFDS}, it must be written in one-step form -- introduced in Section \ref{sec:oneStepForm} -- due to the damping present in the system. Using the matrix form of the damped stiff string in Eq. \eqref{eq:matrixFormStiffString}, the one-step form can be written as
%needs be used. As there are damping terms present in the system, it is useful to write the update in one-step form as explained in Section \ref{sec:oneStepForm}.
\begin{equation}\label{eq:oneStepFormStiffSTring}
    \underbrace{\begin{bmatrix}
        \u^{n+1}\\
        \u^n
    \end{bmatrix}}_{\w^{n+1}} =
    \underbrace{\begin{bmatrix}
        \B/A & \C/A\\
        \I & \mathbf{0}
    \end{bmatrix}}_{\Q}
    \underbrace{\begin{bmatrix}
        \u^n\\
        \u^{n-1}
    \end{bmatrix}}_{\w^n},
\end{equation}
where the definitions for $\B$, $\C$ and $A$ can be found in Section \ref{sec:implementationStiffString}. In this analysis, the definitions for $\Dxx$ and $\Dxxxx$ for simply supported boundary conditions will be used.

Assuming test solutions of the form $\w^n = z^n\boldPhi$, and recalling that $z=e^{sk}$ and complex frequency $s = j\omega + \sigma$, yields the following eigenvalue problem (see Section \ref{sec:eigenValueProblems})
\begin{equation}
    z\boldPhi = \Q \boldPhi,
\end{equation}
which can be solved for the $p$\th complex modal frequency
\begin{equation}
    s_p = \frac{1}{k}\ln \left(\eig_p(\Q)\right).
\end{equation}
The (angular) frequency of the $p$\th mode can then be obtained using $\mathfrak{I}(s_p)$ and the damping per mode as $\mathfrak{R}(s_p)$. Only selecting the non-negative frequencies obtained from $\mathfrak{I}(s_p)$, these can be plotted and are shown in Figure \ref{fig:modesStiffString}. The parameters used are the ones found in Table \ref{tab:stiffStringParams} with $T = 1885$ N, and $E = 2\cdot 10^{14}$ Pa to highlight inharmonic behaviour. The left panel shows that the system is indeed inharmonic, i.e., modal frequencies increase more as the modal number increases. The right panel shows that higher modes exhibit a higher amount of damping. This is due to the frequency-dependent damping term. If $\sigma_1 = 0$ in \eqref{eq:stiffStringFDS}, it can be shown that $\sigma_p = \sigma_0$ for every mode $p$ (in this case $\sigma_0 = -1$).

\begin{figure}[h]
    \centering
    \includegraphics[width=\textwidth]{figures/resonators/modesStiffString.eps}
    \caption{The modal frequencies and damping per mode for the stiff string using the values in Table \ref{tab:stiffStringParams} and $T = 1885$ N and $E = 2\cdot 10^{14}$ Pa to highlight effects of stiffness. % Notice form the left panel that the frequency increases exponentially with the mode number. The right panel shows that higher modes exhibit a greater amount of damping due to the frequency-dependent damping term.
    \label{fig:modesStiffString}}
\end{figure}

\section{Implicit scheme}\label{sec:implicitStiffString}
Although not used in the published work of this project, it is useful to touch upon an example of an implicit scheme. Consider a discretisation of Eq. \eqref{eq:stiffStringPDECompact} where the (more accurate) centred operator is used for the frequency-dependent damping term:
\begin{equation}\label{eq:stiffStringFDSImplicit}
    \dtt \uln = c^2 \dxx \uln - \kappa^2 \dxxxx \uln - 2 \sz \dtd \uln + 2 \so\dtd\dxx \uln.
\end{equation}
Using the centred operator in the mixed-spatio-temporal operator renders the system \textit{implicit}, meaning that a definition for $u_l^{n+1}$ can not explicitly be found from known values. The stencil in Figure \ref{fig:stencilStiffStringImplicit} also shows this: in order to calculate $u_l^{n+1}$, neighbouring points at the next time step $u_{l+1}^{n+1}$ and $u_{l-1}^{n+1}$ are needed. The issue is that these values are unknown at the time of calculation.

Luckily, as the scheme is linear, it can be treated as a system of linear equations and solved following the technique described in Section \ref{sec:linearEquations}. The drawback is that this requires one matrix inversion per iteration which can be extremely costly (see Section \ref{sec:RTmatrixInversion}). However, both von Neumann and modal analysis (below) show that using the centred instead of the backwards operator has a positive effect on the stability and the modal behaviour of the scheme.

Considering simply supported boundary conditions such that $l \in \{1, \hdots, N-1\}$, the system will have $N-1$ unknowns ($u_l^{n+1}$ for $l \in \{1, \hdots, N-1\}$) that can be calculated using $N-1$ (update) equations. Writing this in matrix form using column vector $\u^n = [u_1^n, u_2^n, \hdots, u_{N-1}^n]$ yields

\begin{equation}\label{eq:matrixFormStiffStringImplicit}
    \A\u^{n+1} = \B\u^n + \C \u^{n-1}
\end{equation}
where
\begin{equation*}
    \begin{gathered}
    \A = (1+\sz k)\I - \so k\Dxx, \quad \B = c^2 k^2 \Dxx - \kappa^2 k^2 \Dxxxx \\
    \text{and} \quad \C = -(1-\sz k)\I - \so k \Dxx.
    \end{gathered}
\end{equation*}
Equation \eqref{eq:matrixFormStiffStringImplicit} can be considered a system of linear equations (see Section \ref{sec:linearEquations}) and the state at the next time step $\u^{n+1}$ can then be retrieved using a matrix inversion (see \ref{sec:inverse})
\begin{equation}
    \u^{n+1} = \A^{-1}\left(\B\u^n + \C \u^{n-1}\right).
\end{equation}
\begin{figure}[h]
    \centering
    \includegraphics[width=0.8\textwidth]{figures/resonators/stencilImplicitStiffString.eps}
    \caption{The stencil for the damped stiff string scheme in \eqref{eq:stiffStringFDSImplicit}.\label{fig:stencilStiffStringImplicit}}
\end{figure}

\subsection{von Neumann analysis}
Using the same process as in Section \ref{sec:stiffStringStability}, the definitions in Section \ref{sec:stabilityAnalysis} can be used to obtain a frequency domain representation of the FD scheme in Eq. \eqref{eq:stiffStringFDSImplicit}:
\begin{align}
    \!\!\!\!\frac{1}{k^2}\left(z - 2 + z^{-1}\right) =&-\frac{4c^2}{h^2}\sin^2(\beta h/2) - \frac{16\kappa^2}{h^4}\sin^4(\beta h/2) - \sz kz + \sz k z^{-1}\nonumber\\
    & - \frac{4 \so  k}{h^2}\sin^2(\beta h/2)z + \frac{4 \so  k}{h^2} \sin^2(\beta h/2)z^{-1}
\end{align}
and after collecting the terms, the characteristic equation follows
\begin{gather}
\left(1+\sz k + \frac{4\so k}{h^2}\sin^2(\beta h/2)\right)z + \left(16\mu^2\sin^4(\beta h/2)+4\lambda^2\sin^2(\beta h/2) - 2\right)\nonumber\\
+ \left(1-\sz k - \frac{4\so k}{h^2}\sin^2(\beta h/2)\right)z^{-1} = 0.\label{eq:charDampedStiffSTring}
\end{gather}
Rewriting this to the form \eqref{eq:polynomialForm} and, again, using $\S = \sin^2(\beta h / 2)$ yields:
\begin{equation*}
z^2 + \frac{16\mu^2\S^2+4\lambda^2\S - 2}{1+\sz k + \frac{4\so k}{h^2}\S}z+\frac{1-\sz k - \frac{4\so k}{h^2}\S}{1+\sz k + \frac{4\so k}{h^2}\S} = 0.
\end{equation*}
stability of the system can be proven using condition \eqref{eq:condition214}. Continuing with $\S = \sin^2(\beta h/2)$ and substituting the coefficients into this condition yields
\begin{align*}
\left|\frac{16\mu^2\S^2+4\lambda^2\S - 2}{1+\sz k + \frac{4\so k}{h^2}\S} \right|-1 &\leq \frac{1-\sz k - \frac{4\so k}{h^2}\S}{1+\sz k + \frac{4\so k}{h^2}\S}\leq 1,\\[1em]
\left|16\mu^2\S^2+4\lambda^2\S - 2 \right| - \left(1+\sz k + \frac{4\so k}{h^2}\S\right) &\leq 1-\sz k - \frac{4\so k}{h^2}\S\\
&\qquad\leq 1+\sz k + \frac{4\so k}{h^2}\S,\\[1em]
\left|16\mu^2\S^2+4\lambda^2\S - 2 \right|&\leq 2 \leq 2+2\sz k + \frac{8\so k}{h^2}\S.
\end{align*}
Because $\sz, \so, k, \S$ and $h$ are all non-negative, the last condition is always satisfied. Continuing with the first condition:
\begin{align*}
    -2\leq 16\mu^2\S^2+4\lambda^2\S - 2 &\leq 2,\\
    0\leq 16\mu^2\S^2+4\lambda^2\S &\leq 4.
\end{align*}
Again, the first condition is always satisfied due to the non-negativity of all coefficients. Continuing with the second condition yields
\begin{equation*}
    4\mu^2\S^2+\lambda^2\S \leq 1,
\end{equation*}
and knowing that $\S$ is bounded by $1$ for all $\beta$, the process can be finalised:
\begin{align*}
    4\mu^2+\lambda^2 &\leq 1,\\
    \frac{4\kappa^2k^2}{h^4}+\frac{c^2k^2}{h^2} &\leq 1,\\
    h^4 - c^2k^2h^2 - 4\kappa^2k^2 &\geq 0,
\end{align*}
and yields the following stability condition:
\begin{equation}\label{eq:implicitStability}
    h \geq \sqrt{\frac{c^2k^2 + \sqrt{c^4k^4 + 16\kappa^2k^2}}{2}}.
\end{equation}
Comparing this to the stability condition for the explicit scheme in Eq. \eqref{eq:stiffStringStability}, one can observe that the terms containing $\so$ have vanished. It can thus be concluded that if the centred (rather than the backwards) difference is used to discretise the temporal derivative in the frequency-dependent damping term, $\so$ no longer influences the stability of the scheme and the condition is more relaxed. What this means in terms of behaviour of the scheme will be elaborated on in the following section.

\subsection{Modal analysis}
As the matrix form of the implicit FD scheme in Eq. \eqref{eq:matrixFormStiffStringImplicit} matches the form in Eq. \eqref{eq:modalForm}, one can perform a modal analysis by writing the scheme in one-step form as explained in Section \ref{sec:oneStepForm}. The results of the analysis are shown in Figure \ref{fig:implicitModes}. To highlight the difference between using the backwards and centred difference for the frequency-dependent damping term, $\so $ has been set to $1$, which is much higher than one would normally use.

One can observe from Figure \ref{fig:implicitModes} that especially higher-frequency modes in the explicit scheme are affected by $\so$.
In the continuous case, the modal frequencies should only be affected by values for $c$ and $\kappa$ as per Eq. \eqref{eq:inharmonicityEquation} and the damping should not influence the frequencies of the partials, as one could expect. However, as $\so$ increases, $h$ increases due to Eq. \eqref{eq:stiffStringStability}, causing $\lambda$ and $\mu$ to decrease. This introduces numerical dispersion as explained in Section \ref{sec:quality1DWave}, and the higher the value of $\so$, the more numerical dispersion is introduced.

As the stability condition for the implicit scheme in Eq. \eqref{eq:implicitStability} does not contain $\so$, this value will not affect $\lambda$ and $\mu$ and will thus not affect the modal frequencies. As can be observed from the figure, it even allows for one more grid point to be included in the simulation. It can be concluded that because the frequency-dependent damping term no longer affects the stability condition for the implicit scheme, a more accurate simulation can be obtained with fewer numerically dispersive effects.

\begin{figure}[h]
    \centering
    \includegraphics[width=\textwidth]{figures/resonators/implicitModes.eps}
    \caption{A comparison between the modal frequencies and damping per mode of the explicit (blue) and implicit (red) scheme. Here, $T = 1885$ N, $E = 2\cdot10^{14}$ Pa and $\so = 1$ m$^2$/s to highlight differences between the two schemes. %One can observe that the modes of the implicit scheme follow the expected exponential pattern for the stiff string, where the explicit scheme shows numerically dispersive effects. Furthermore, due to the absence of $\so$ in the stability condition in Eq. \eqref{eq:implicitStability} and allows for one more grid point
    \label{fig:implicitModes}}
\end{figure}

\subsection{Conclusion}
This section presented an implicit discretisation of the stiff string where the centred operator has been used to discretise the temporal derivative in the frequency-dependent damping term. By means of stability analysis and modal analysis several advantages that the implicit scheme has over its explicit counterpart (presented in Section \ref{sec:stiffStringDiscrete}) have been shown.

As these advantages only show for higher values of $\so$, much higher than the ones used in this project, it has been chosen to use the explicit scheme for all further implementation. The decrease in accuracy is negligible for lower values of $\so$ and the calculation of the scheme becomes orders of magnitude more computationally expensive if the implicit scheme is used.


\documentclass[12pt]{article}

\usepackage{graphicx}
\usepackage{amsmath}

\begin{document}

\section{Least-squares fitting}

\subsection{Model}

\begin{equation}
y=\alpha_{0}+\alpha_{1}x
\end{equation}

\subsection{Data}

\begin{table}[ht]
\begin{centering}
\begin{tabular}{ccc}
i & $x_{i}$ & $y_{i}$\\
1 & 0 & 0 \\
2 & 2 & 3 \\
3 & 4 & 5 \\
\end{tabular}
\par\end{centering}
\caption{Data.}
\end{table}

\begin{figure}[ht]
\begin{centering}
\includegraphics[width=\columnwidth]{data.png}
\end{centering}
\caption{Data.}
\end{figure}

\section{Solution}

\begin{align*}
\alpha_{0}+\alpha_{1}0 & =0\\
\alpha_{0}+\alpha_{1}2 & =3\\
\alpha_{0}+\alpha_{1}4 & =5
\end{align*}

\[
\left[\begin{array}{cc}
1 & 0\\
1 & 2\\
1 & 4
\end{array}\right]\left[\begin{array}{c}
\alpha_{0}\\
\alpha_{1}
\end{array}\right]=\left[\begin{array}{c}
0\\
3\\
5
\end{array}\right]
\]

\begin{align*}
\boldsymbol{X}\boldsymbol{\alpha} & =\boldsymbol{y}\qquad|\boldsymbol{X}^{\mathrm{T}}\cdot\left(\cdot\right)\\
\boldsymbol{X}^{\mathrm{T}}\boldsymbol{X}\boldsymbol{\alpha} & =\boldsymbol{X}^{\mathrm{T}}\boldsymbol{y}\qquad|\left(\boldsymbol{X}^{\mathrm{T}}\boldsymbol{X}\right)^{-1}\cdot\left(\cdot\right)\\
\boldsymbol{\alpha} & =\left(\boldsymbol{X}^{\mathrm{T}}\boldsymbol{X}\right)^{-1}\boldsymbol{X}^{\mathrm{T}}\boldsymbol{y}
\end{align*}

\begin{figure}[ht]
\begin{centering}
\includegraphics[width=\columnwidth]{solution.png}
\end{centering}
\caption{Solution.}
\end{figure}

\end{document}


\documentclass[a4paper]{article}

\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{hyperref}
\usepackage[colorinlistoftodos]{todonotes}

\title{Playing with function iteration}
\author{Slavomir Kaslev}

\begin{document}
\maketitle

\section{General problem}

The equations $f(x) = x + f(x^2 + x^3)$ and $f(x) = x + f(\frac{1}{1-x})$ are instances of the more general equation
\begin{equation}
f(x) = x + f(g(x))\label{eq:eq1}
\end{equation}
We want to find a function $f(x)$ such that \eqref{eq:eq1} holds for all $x$, where the function $g(x)$ is known. Telescoping the equation gives
$$f(x) = x + g(x) + g(g(x)) + g(g(g(x))) + \dots$$
Let's use the following notation for iterating a function
\begin{align*}
g^{[0]}(x) &= x\\
g^{[1]}(x) &= g(x)\\
g^{[n]}(x) &= g(g^{[n-1]}(x))
\end{align*}
to rewrite equation \eqref{eq:eq1} as
\begin{equation}
f(x) = \sum_{n=0}^{\infty}{g^{[n]}(x)}\label{eq:eq2}
\end{equation}
We've reduced the problem from solving equation \eqref{eq:eq1} to calculating the infinite sum \eqref{eq:eq2}.

\section{Power series sleight of hand}

The next step is instead of solving equation \eqref{eq:eq1} we'll change the problem by inserting a small parameter $\epsilon$
\begin{equation}
f(x) = x + \epsilon f(g(x))\label{eq:eq3}
\end{equation}
Repeating the steps above, we telescope \eqref{eq:eq3}
$$f(x) = x + \epsilon g(x) + \epsilon^2 g(g(x)) + \epsilon^3 g(g(g(x))) + \dots$$
and finally we get $f(x)$ as power series of $\epsilon$
\begin{equation}\label{eq4}
f(x) = \sum_{n=0}^{\infty}{g^{[n]}(x)} \epsilon^n
\end{equation}

Now we have two procedures for computing $f(x)$ defined by equations \eqref{eq:eq2} and \eqref{eq4}. Of course, the two procedures will agree only if we finally set $\epsilon$ to $1$.

\section{Special case $g(x) = \frac{1}{1-x}$}

Equation \eqref{eq:eq3} takes the form
\begin{equation}\label{eq5}
f(x) = x + \epsilon f(\frac{1}{1-x})
\end{equation}
Note that the function $g^{[n]}(x)$ is periodic in $n$ with period 3. That is
\begin{align*}
g^{[0]}(x) &= x\\
g^{[1]}(x) &= \frac{1}{1-x}\\
g^{[2]}(x) &= 1 - \frac{1}{x}\\
g^{[3]}(x) &= x
\end{align*}
So the series in \eqref{eq4} have the form
\begin{equation}\label{eq6}
f(x) = x + \epsilon \frac{1}{1-x} + \epsilon^2 (1 - \frac{1}{x}) + \epsilon^3 x + \epsilon^4 \frac{1}{1-x} + \dots
\end{equation}
We'll sum this series using a generic summation procedure $S$ with the properties
\begin{subequations}
\begin{align}
  S(a_0 + a_1 + a_2 + \dots) &= a_0 + S(a_1 + a_2 + a_3 + \dots)\label{eq:sum1}\\
  S(\sum{\alpha a_n} + \sum{\beta b_n}) &= \alpha S(\sum{a_n}) + \beta S(\sum{b_n})\label{eq:sum2}
\end{align}
\end{subequations}
To simplify the calculation we'll substitute
\begin{equation*}
s = f(x)
\qquad
a = x
\qquad
b = \frac{1}{1-x}
\qquad
c = 1 - \frac{1}{x}
\end{equation*}
in \eqref{eq6} to obtain
$$s = S(a + b \epsilon + c \epsilon^2 + a \epsilon^3 + b \epsilon^4 + c \epsilon^5 + \dots)$$
Next we'll pull the first term out of the sum two times by \eqref{eq:sum1} and bring $\epsilon$ outside the sum by \eqref{eq:sum2}
\begin{align*}
s = S&(a + b \epsilon + c \epsilon^2 + a \epsilon^3 + b \epsilon^4 + c \epsilon^5 + \dots)\\
s = a + \epsilon S&(b + c \epsilon + a \epsilon^2 + b\epsilon^3 + c\epsilon^4 + a \epsilon^5 + \dots)\\
s = a + b \epsilon + \epsilon^2 S&(c + a \epsilon + b\epsilon^2 + c\epsilon^3 + a \epsilon^4 + b \epsilon^5+ \dots)\\
\end{align*}
If we multiply the first equation by $\epsilon^2$, the second by $\epsilon$ then add them all up, the result is
$$(1 + \epsilon + \epsilon^2)s = (1+\epsilon)a + \epsilon b + \epsilon^2 (a+b+c) S(1 + \epsilon + \epsilon^2 + \epsilon^3 + \dots)$$
Notice that $S(1 + \epsilon + \epsilon^2 + \epsilon^3 + \dots) = \frac{1}{1-\epsilon}$ and now we have an explicit formula for $s$
$$s=\frac{(1+\epsilon) a + \epsilon b}{1+\epsilon+\epsilon^2} + \frac{\epsilon^2}{1-\epsilon^3}(a+b+c)$$
Therefore the solution of \eqref{eq5} is
\begin{equation}
f(x) = \frac{(1+\epsilon)x + \epsilon\frac{1}{1-x}}{1+\epsilon+\epsilon^2} + \frac{\epsilon^2}{1-\epsilon^3}\frac{x^3-3x+1}{x(x-1)}\label{eq:soleps}
\end{equation}
We're interested in the solution of this equation
\begin{equation}
f(x) = x + f(\frac{1}{1-x})\label{eq9}
\end{equation}
which can be derived from \eqref{eq:soleps} by seting $\epsilon$ to $1$.
Notice that when $\epsilon=1$, the second term in the equation becomes infinite unless $x^3-3x+1=0$.

We conclude that the solution $f(x)$ of equation \eqref{eq9} is infinite everywhere except for a finite set of points given by the roots of the equation
$$x^3-3x+1=0$$
namely
\begin{equation*}
x_1 \approx -1.87938524
\qquad
x_2 \approx 0.34729636
\qquad
x_3 \approx 1.53208889
\end{equation*}
where $f(x)$ has an explicit formula
$$f(x)=\frac{1}{3}\;\frac{1 + 2x - 2x^2}{1-x}$$
and can be evalutated directly
\begin{equation*}
f(x_1) \approx -1.13715804
\qquad
f(x_2) \approx 0.7422272
\qquad
f(x_3) \approx 0.39493084
\end{equation*}
Notice that iterating $g(x)$ over the numbers $x_1,x_2,x_3$ forms a cycle
\begin{equation*}
g(x_1) = x_2
\qquad
g(x_2) = x_3
\qquad
g(x_3) = x_1
\end{equation*}
and
\begin{equation*}
\sum_{k=1}^{3}{x_k} = 0
\qquad
\sum_{k=1}^{3}{f(x_k)} = 0
\qquad
\sum_{k=1}^{3}{g(x_k)} = 0
\end{equation*}

\end{document}


\documentclass{beamer}

\usepackage[utf8]{inputenc}
\usepackage{subfigure}
\usepackage{bbm}

\usetheme{Madrid}
\usecolortheme{beaver}

\title{Spectral clustering}
\author{%
  Davide Riva
  (\texttt{driva95@protonmail.com})
}

\begin{document}

\frame{\titlepage}

\begin{frame}
  \frametitle{Table of Contents}
  \tableofcontents
\end{frame}

\section{Graphs with 1 connected component}
\begin{frame}
  \frametitle{Graphs with 1 connected component}
  \centering
  \includegraphics[width=0.5\linewidth]{figures/one-component.eps}
\end{frame}
\begin{frame}
  \begin{block}{Laplacian matrix}
    \[ L = D - W \]
    \[ D_{ij} = \begin{cases} \sum_{j=1}^n W_{ij} \; \text{if} \; i = j \\ 0 \; \text{if} \; i \neq j \end{cases} \]
  \end{block}

  \begin{alertblock}{One eigenvalue equals to zero, its eigenvector equals to $a [1, 1, \dots, 1]$}
    \[ f^T L f = \frac{1}{2} \sum_{i=1}^n \sum_{j=1}^n W_{ij} (f_i - f_j)^2, \; f \in \mathit{R}^n
      \implies f^T L f \geq 0 \; \forall f \]
  \end{alertblock}
\end{frame}

\section{Graphs with multiple connected components}
\begin{frame}
  \frametitle{Graphs with multiple connected components}
  \centering
  \includegraphics[width=0.5\linewidth]{figures/multiple-component.eps}
\end{frame}

\begin{frame}
  \frametitle{Number of eigenvalues equal to zero}
  \begin{figure}
    \hfill
    \subfigure[Graphs with two connected components]{\includegraphics[width=0.3\linewidth]{figures/2-components.eps}}
    \hfill
    \subfigure[Graphs with four connected components]{\includegraphics[width=0.3\linewidth]{figures/4-components.eps}}
    \hfill
    \subfigure[Graphs with eight connected components]{\includegraphics[width=0.3\linewidth]{figures/8-components.eps}}
    \hfill
    \label{figure:mcomp}
    \caption{Box plot of the first 10 eigenvalues of 100 graphs with 64 nodes and different connected components, sorted in ascending order for each graph.}
  \end{figure}
\end{frame}

\begin{frame}
  \frametitle{Eigenvectors as indicator vectors}
  \begin{figure}
    \hfill
    \subfigure{\includegraphics[width=0.3\linewidth]{figures/0-eigenvectors.eps}}%
    \hfill
    \subfigure{\includegraphics[width=0.3\linewidth]{figures/1-eigenvectors.eps}}%
    \hfill
    \subfigure{\includegraphics[width=0.3\linewidth]{figures/2-eigenvectors.eps}}%
    \hfill
    \label{figure:eivects}%
  \end{figure}
\end{frame}

\section{Considerations about noise}
\begin{frame}
  \frametitle{Considerations about noise}
  \begin{figure}
    \centering
    \includegraphics[width=0.5\linewidth]{figures/adding-noise.eps}
    \caption{Example of the effect of noise in the eigenvalues intensities in a graph with 3 connected components}
    \label{figure:addingnoise}
  \end{figure}
\end{frame}

\section{Spectral clustering when data is not a graph}
\begin{frame}
  \frametitle{Spectral clustering when data is not a graph}
  Similarity function $s(x_i, x_j) \geq 0$:
  \begin{itemize}
    \item $s(x_i, x_j) = e^{ - \frac{||x_i - x_j||^2}{2\sigma^2}}$ (Gaussian similarity function)
    \item $s(x_i, x_j) = ||x_i - x_j||_2 < \epsilon$
    \item $s(x_i, x_j) = \frac{1}{\sqrt{2}} \sqrt{\sum_k (\sqrt{x_i(k)} - \sqrt{x_j(k)})^2}$ (Hellinger distance)
    \item \dots
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Why not K-means?}
  \begin{figure}[ht]
    \hfill
    \subfigure[Ground truth]{\includegraphics[width=0.3\linewidth]{figures/ground-truth.eps}}%
    \hfill
    \subfigure[Clustering using K-means]{\includegraphics[width=0.3\linewidth]{figures/kmeans.eps}}%
    \hfill
    \subfigure[Clustering using spectral clustering]{\includegraphics[width=0.3\linewidth]{figures/spectral-on-comparison.eps}}%

  \end{figure}
\end{frame}

\begin{frame}
  \frametitle{Is it possible to avoid hyperparameters tuning?}
  \begin{block}{Modularity}
    \[ Q = \frac{1}{2m} \sum_{i=1}^n \sum_{j=1}^n (A_{ij} - \frac{k_i k_j}{2m}) \mathbbm{1}_{[C_i == C_j]} \]
  \end{block}

  \begin{figure}
    \subfigure[Raw data]{\includegraphics[width=0.2\linewidth]{figures/mall-raw.png}}%
    \hfill
    \subfigure[Spectral clustering with modularity]{\includegraphics[width=0.2\linewidth]{figures/mall-spectral.png}}%
    \hfill
    \subfigure[Spectral clustering]{\includegraphics[width=0.2\linewidth]{figures/mall-manual.png}}%
  \end{figure}
\end{frame}

\begin{frame}
  \frametitle{Conclusions}
  Spectral clustering outperforms K-means, but you need hyperparameter tuning
\end{frame}

\end{document}


\vsssub
\subsubsection{~$S_{ice}$: Damping by sea ice (Mosig et al.)} \label{sec:ICE5}
\vsssub

\opthead{IC5}{U. of Otago MATLAB code}{Q. Liu, E. Rogers, A. Babanin}

\noindent
The fifth method for representing ice-induced wave decay is based on another viscoelastic-type model, i.e., the EFS ice layer model described in \citet{art:MMS15}. The authors introduced viscosity into the thin elastic plate model of \citet{art:FS1994} and restricted it to one horizontal dimension (replacing a plate by a beam). The dispersion relation given by the EFS model can be written in the form
\begin{equation}
Q g k \tanh(k d)-\sigma^2 = 0,
\label{eq:ic5a}
\end{equation}
%
\begin{equation}
Q = \frac{G_{\eta} h_i^3}{6 \rho_w g} (1+\nu) k^4 - \frac{\rho_i h_i \sigma^2}{\rho_w g} + 1.
\label{eq:ic5b}
\end{equation}
%
In Eq.~(\ref{eq:ic5a})$-$(\ref{eq:ic5b}), $G_{\eta} = G - i \sigma \rho_i \eta$ is the complex shear modulus, where $G$ is the \emph{effective} elastic shear modulus and $\eta$ is the \emph{effective} viscosity; $\rho_w$ ($\rho_i$) is the density of water (ice), $d$ is water depth, $h_i$ is the ice cover thickness, $\sigma$ is the radian frequency, $k=k_r + i k_i$ is the complex wavenumber, $g$ is the gravitational acceleration and $\nu=0.3$ refers to the Poisson ratio of sea ice.

Same as {\code IC3}, {\code IC5} requires four ice parameters as input: $C_{ice, 1}$ for ice thickness $h_i$ (m), $C_{ice, 2}$ for the \emph{effective} viscosity $\eta$ (m$^2$ s$^{-1}$), $C_{ice, 3}$ for ice density $\rho_i$ (kg m$^{-3}$) and $C_{ice, 4}$ for the \emph{effective} shear modulus $G$ (Pa). For example, as shown in \citet[][see their Fig. 8]{art:MMS15}, a setting of $C_{ice, 1,...,4}=[1.0\ \mathrm{m},\ 5.0\times10^7\ \mathrm{m^2\ s^{-1}},\ 917.0\ \mathrm{kg\ m^{-3}},\ 4.9\times10^{12}\ \mathrm{Pa}]$ (with a water depth $d$ of 4300 m) can be used to fit the observed wave attenuation rates reported in \citet{art:MBK14}. The application of the EFS model to two realistic case studies is presented in \citet{Liu2018ic5}.

The dispersion relation shown above is solved iteratively using the Newton-Raphson method. The numerical solver, however, may fail for small wave periods in some rare cases (particularly for shallow water depth $d$ and low $G$). In such cases, the estimated wavelength $k_r$ is unreasonably low. Several namelist variables (limiters) are introduced to improve the code stability:
\begin{clist}
\cit{IC5MINIG} {the minimum allowed shear modulus $G$; Default= 1 Pa (i.e., zero $G$ is not allowed).}
%
\cit{IC5MINWT} {the minimum allowed wave periods $T$; Default=0 s (i.e., by default, this option is not used).}
%
\cit{IC5MAXKRATIO} {the maximum allowed $k_{ow}/k_r$, where $k_{ow}$ is the open-water wavenumber; Default=1E9 (i.e., by default, this option is not used).}
%
\cit{IC5MAXKI} {the maximum allowed $k_i$; Default=100 m$^{-1}$ (i.e., by default, this option is not used).}
%
\cit{IC5MINHW} {the minimum allowed water depth $d$; Default=300 m (this basically limits {\code IC5} to the deep-water case).}
%
\cit{IC5MAXITER} {the maximum allowed \# of iteration; Default=100.}
\end{clist}

Note that the EFS model used here regards the ice cover as a continuous homogeneous medium and characterizes various ice types with two \emph{totally empirical} rheological parameters, namely the elastic shear modulus $G$ and the viscosity $\eta$. As argued in \citet{art:MMS15}, these two parameters \emph{``cannot be measured directly as they do not represent observable physical processes''}. Therefore, ``no restrictions on the acceptable values of the rheological parameters, \emph{except positiveness}, can be imposed.'' So strictly speaking, the EFS model is better termed as an \emph{effective medium} model rather than a \emph{viscoelastic} model.


\subsection{Relative risk aversion}

Absolute risk aversion is:

\(A(x)=-\dfrac{u''(x)}{u'(x)}\)

\(R(x)=xA(x)\)

\(R(x)=-\dfrac{xu''(x)}{u'(x)}\)


%---------------------------Scaled Jacobian---------------------------
\section{Scaled Jacobian}

This metric is the minimum determinant of the Jacobian matrix
evaluated at each corner and the center of the element,
divided by the corresponding edge lengths.
\[
q = \min_{i\in\{0,1,\ldots,8\}}\left\{\hat\alpha_i\right\}.
\]

Note that if ${L_{\min}}^2 \leq DBL\_MIN$, we set $q = DBL\_MAX$.

\hexmetrictable{scaled Jacobian}%
{$1$}%                                        Dimension
{$[0.5,1]$}%                                  Acceptable range
{$[-1,1]$}%                                   Normal range
{$[-1,DBL\_MAX]$}%                            Full range
{$1$}%                                        Cube
{\cite{knu:00}}%                              Citation
{v\_hex\_scaled\_jacobian}%                   Verdict function name


\section{Points and vectors}

\begin{outcome}
  \begin{enumerate}
  \item Understand the geometric and algebraic meaning of points and
    vectors in $\R^n$.
  \item Find the position vector of a point in $\R^n$.
  \item Determine whether two vectors are equal.
  \end{enumerate}
\end{outcome}

In this section, we define points and vectors in $n$-dimensional
space, and discuss some of their interpretations. We start with a
brief review of Cartesian coordinate systems.
\bigskip

\noindent\textbf{Points in $n$-dimensional space.}
You are probably already familiar with Cartesian coordinates%
\index{Cartesian coordinates}%
\index{coordinate system!Cartesian}, which let you describe points%
\index{point} in $2$- or $3$-dimensional space. Consider the familiar
coordinate plane, with an $x$-axis and a $y$-axis. Any point within
this coordinate plane is identified by its $x$- and $y$-coordinates%
\index{coordinate!of a point}. For example, the point $P$ in the
following diagram has $x$-coordinate~$2$ and $y$-coordinate~$1$.  We
write these coordinates as an ordered pair $P=(2,1)$. Here,
``ordered'' means that the $x$-coordinate comes first, and then the
$y$-coordinate, i.e., $(1,2)$ is not the same point as
$(2,1)$. Coordinates can be positive, negative, or zero. The special
point with coordinates $(0,0)$ is called the \textbf{origin}%
\index{origin of a coordinate system} of the coordinate system, and
also written as $0$.
\begin{center}
  \begin{tikzpicture}[scale=0.6]
    \draw[thick,->](-4,0)--(4,0);
    \draw[thick,->](0,-1)--(0,5);
    \draw(2,0.2)--(2,-0.2);
    \draw(-3,0.2)--(-3,-0.2);
    \draw(-0.2,1)--(0.2,1);
    \draw(-0.2,4)--(0.2,4);
    \draw[help lines] (-3,0)--(-3,4)--(0,4);
    \draw[help lines](0,1)--(2,1)--(2,0);
    \draw[fill](-3,4) circle [radius=3pt];
    \draw[fill](2,1) circle [radius=3pt];
    \node[below right] at (4,0){$x$};
    \node[left] at(0,5){$y$};
    \node[below] at (-3,-0.5){$-3$};
    \node[below] at (2,-0.5){$2$};
    \node[left] at (-0.5, 1){$1$};
    \node[left] at (-0.5, 4){$4$};
    \node[above] at (-3,4){$Q = (-3,4)$};
    \node[above] at (2,1){$P = (2,1)$};
  \end{tikzpicture}
\end{center}
The situation in $3$ dimensions is analogous. Here, the coordinate
system has three axes, and each point is described by a triple of
coordinates, which we can write as $(x,y,z)$.  We can extend these
ideas beyond $n=3$. A coordinate system for $n$-dimensional space has
$n$ axes, which we may call $x_1,\ldots,x_n$ (as there are not enough
letters in the alphabet to continue after $z$). A point of
$n$-dimensional space is described by an ordered $n$-tuple
$(x_1,\ldots,x_n)$ of coordinates. For example, $P=(2,1,0,-1)$ is a
point in $4$-dimensional space which has $x_1$-coordinate $2$,
$x_2$-coordinate $1$, and so on. While most people cannot really
picture space beyond $3$ dimensions, it is easy to imagine tuples of
$n$ real numbers. Thus, although we may not be able to ``see'' the
points in higher dimensions, we can still talk about their coordinates.
\bigskip

\noindent\textbf{Vectors in $n$-dimensional space.}
Unlike a point, which describes a location in a coordinate system, a
vector%
\index{vector!geometric meaning} describes an {\em offset}%
\index{offset} or a {\em distance and direction}. We usually picture a
vector as an arrow, starting at one point (called the \textbf{tail}%
\index{tail of a vector}%
\index{vector!tail} of the arrow) and ending at another point (called
the \textbf{tip}%
\index{tip of a vector}%
\index{vector!tip} of the arrow).
\begin{center}
  \begin{tikzpicture}[scale=0.8]
    \draw[->, thick, blue](1,0)--+(1.5,1.125);
    \draw[->, thick, blue](2,1.2)--+(1.5,1.125);
    \draw[->, thick, blue](2,0)--+(1.5,1.125);
    \draw[->, thick, blue](0.7,0.7)--+(1.5,1.125);
  \end{tikzpicture}
\end{center}
Two vectors are considered equal if they have the same direction and
length. Thus, all four blue arrows in the above image describe exactly
the same vector. Mathematically, a vector in $2$-dimensional space is
described as an offset in the $x$-direction and an offset in the
$y$-direction.  For example, a certain vector $\vect{v}$ may be
described by the instruction: ``move $4$ units in the direction
parallel to the $x$-axis, and move $3$ units in the direction parallel
to the $y$-axis''. This situation is pictured here:
\begin{center}
  \begin{tikzpicture}[scale=0.5]
    \draw[->, thick, blue](0,0) -- node[above]{$\vect{v}$} +(4,3);
    \draw(0,0) -- node[below]{4 units} (4,0) -- node[right]{3 units} (4,3);
  \end{tikzpicture}
\end{center}
The numbers $4$ and $3$ are also called the $x$-component%
\index{component!of a vector} and the $y$-component of the
vector. Notice that a point has ``coordinates'', but a vector has
``components''. We write the components of a vector as an ordered
column within square brackets:
$\vect{v}=\begin{mysmallmatrix}{c}\scriptstyle4\\\scriptstyle3\end{mysmallmatrix}$. Note
that components can also be negative; for example, a negative
$x$-component indicates to move left instead of right, and a negative
$y$-component indicates to move down instead of up. The vector with
all components equal to $0$ is called the \textbf{zero vector}%
\index{zero vector}, and is written $\vect{0}$.

The situation in $3$ dimensions is similar. Here, a vector is
described by three components, namely, its $x$-component,
$y$-component, and $z$-component. The three components are written as
$\begin{mysmallmatrix}{c}\scriptstyle x\\\scriptstyle y\\\scriptstyle
  z\end{mysmallmatrix}$. The same idea generalizes to $n$-dimensional
vectors when $n$ is greater than $3$.

\begin{definition}{Column vectors and $\R^n$}{column-vector}
  A $n$-dimensional%
  \index{dimension!of a vector}%
  \index{vector!dimension} \textbf{column vector}%
  \index{column vector}%
  \index{vector!column vector}, often simply called a \textbf{vector}%
  \index{vector}, is an ordered list of $n$ real numbers, written as a
  column within square brackets:
  \begin{equation*}
    \begin{mymatrix}{c}
      x_1 \\
      x_2 \\
      \vdots \\
      x_n
    \end{mymatrix}.
  \end{equation*}
  We write $\R^n$%
  \index{Rn@$\R^n$} for the set of all $n$-dimensional column
  vectors. It is also known as \textbf{$n$-dimensional Euclidean
    space}%
  \index{Euclidean space}.
\end{definition}
Vectors are usually denoted by boldface lower-case letters such as
$\vect{v}$, $\vect{w}$, $\vect{a}$, $\vect{b}$. Some people write a
small arrow above the vector, but we do not do this here.
\bigskip

\noindent\textbf{Points vs. vectors.}
What is the relationship between points and vectors? Algebraically,
they seem to be almost the same thing, because a point $(x,y)$ and a
vector
$\begin{mysmallmatrix}{c}\scriptstyle x\\\scriptstyle
  y\end{mysmallmatrix}$ are both an ordered pair of real numbers,
written in a slightly different way. On the other hand, geometrically,
a point is a location in space, and has neither a length nor
direction, whereas a vector has length and direction, but is not fixed
at any particular location. Indeed, to convince yourself that despite
the similarity in their notation, points and vectors are different
kinds of objects, imagine that we moved the origin of the coordinate
system to a different location. Then the coordinates of all the points
would change, whereas the components of all the vectors would remain
the same. To describe the components of a vector, we require axes and
a scale, but no origin. To describe the coordinates of a point, we
require axes, a scale, and an origin.  \bigskip

\noindent\textbf{Vectors from points.}
If $Q$ and $P$ are two points in $n$-dimensional space, we can define
a \textbf{vector from $Q$ to $P$}%
\index{vector!from a point to a point}. This vector is written
$\longvect{QP}$, and is described by the arrow whose tail is at $Q$
and whose tip is at $P$, as in the following picture:
\begin{center}
  \begin{tikzpicture}[scale=0.55]
    \draw[fill] (0,0) circle [radius=1.8pt] node[below]{$Q$};
    \draw[fill] (4,0.5) circle [radius=1.8pt] node[below]{$P$};
    \draw[thick, blue, ->](0,0) -- node[above, pos=0.45]{$\longvect{QP}$} (4,0.5);
  \end{tikzpicture}
\end{center}
If the point $Q$ has coordinates $(q_1,\ldots,q_n)$ and the point $P$
has coordinates $(p_1,\ldots,p_n)$, then the components of
$\longvect{QP}$ are
\begin{equation*}
  \longvect{QP} =
  \begin{mymatrix}{c}
    p_1 - q_1 \\
    \vdots    \\
    p_n - q_n
  \end{mymatrix}.
\end{equation*}
An important special case of this is the case when the point $Q$ is
the origin. The following definition is concerned with that situation.

\begin{definition}{The position vector of a point}{position-vector}
  Let $P$ be a point in $n$-dimensional space. The \textbf{position
    vector}%
  \index{position vector} of $P$ is the vector
  $\vect{p} = \longvect{0P}$ whose tail is at the origin and whose tip
  is at $P$.
  \begin{center}
    \begin{tikzpicture}[scale=0.6]
      \draw[<->](2,0,0)--(0,0,0)--(0,2,0);
      \draw[fill] (0,0) circle [radius=1.8pt] node[below]{$0$};
      \draw[fill] (3,2) circle [radius=1.8pt] node[below]{$P$};
      \draw[thick, blue, ->](0,0) -- node[above left]{$\vect{p}$} (3,2);
    \end{tikzpicture}
  \end{center}
  If the point $P$ has coordinates $(p_1,\ldots,p_n)$, then the
  components of the position vector are
  \begin{equation*}
    \vect{p} =
    \begin{mymatrix}{c}
      p_1    \\
      \vdots \\
      p_n
    \end{mymatrix}.
  \end{equation*}
  Thus, the coordinates of a point are the same as the components of
  its position vector. For this reason, the position vector is also
  sometimes called the \textbf{coordinate vector}%
  \index{coordinate vector} of $P$.
\end{definition}

\noindent\textbf{Points from vectors.} Conversely, given any vector
$\vect{p}$, we may find a point $P$ that has $\vect{p}$ as its
position vector. To do so geometrically, we first have the move the
vector $\vect{p}$ around until its tail is at the origin. The point
$P$ will then be located at its tip.
\begin{center}
  \begin{tikzpicture}[scale=0.8]
    \draw[<->](2,0,0)--(0,0,0)--(0,2,0);
    \draw[fill] (0,0) circle [radius=1.8pt] node[below]{$0$};
    \draw[fill] (3,2) circle [radius=1.8pt] node[below]{$P$};
    \draw[thick, blue, ->](-2,1) -- node[above left]{$\vect{p}$} +(3,2);
    \draw[thick, blue, ->](2.5,0.5) -- node[below right]{$\vect{p}$} +(3,2);
    \draw[thick, blue, ->](0,0) -- node[above left]{$\vect{p}$} +(3,2);
  \end{tikzpicture}
\end{center}
Algebraically, if the vector $\vect{p}$ has components
\begin{equation*}
  \vect{p} =
  \begin{mymatrix}{c}
    p_1    \\
    \vdots \\
    p_n
  \end{mymatrix},
\end{equation*}
then the point $P$ will have coordinates $(p_1,\ldots,p_n)$. This is
just the opposite process of Definition~\ref{def:position-vector}.

So although we went to some lengths to point out that vectors and
points are different geometric objects, as soon as an origin of a
coordinate system has been fixed, we can always talk about a point by
talking about its coordinate vector. We will systematically do so, and
eventually the distinction between a point and its coordinate vector
will become blurred, so that we will be able to talk about $\R^n$ as
``a set of points'' or ``a set of vectors'' interchangeably.  \bigskip

\noindent\textbf{Equality of vectors.}
Two vectors are equal%
\index{equality!of vectors}%
\index{vector!equality} precisely when all corresponding components
are equal. In symbols, if
\begin{equation*}
  \vect{u} =
  \begin{mymatrix}{c}
    u_1 \\
    \vdots \\
    u_n
  \end{mymatrix}
  \quad\mbox{and}\quad
  \vect{v} =
  \begin{mymatrix}{c}
    v_1 \\
    \vdots \\
    v_n
  \end{mymatrix},
\end{equation*}
then $\vect{u}=\vect{v}$ if and only if $u_1=v_1$ and $u_2=v_2$ and
\ldots and $u_n=v_n$.
\bigskip

\noindent\textbf{Notation.}
In the text, it is often awkward to write column vectors, because they
take up so much space. To save space, we sometimes use a superscript
``$T$'' to denote a column vector. For example, we write
$\begin{mymatrix}{rrr}1&2&3\end{mymatrix}^T$, or sometimes
$\mat{1, 2, 3}^T$, to denote the vector
\begin{equation*}
  \begin{mymatrix}{c}
    1 \\
    2 \\
    3 \\
  \end{mymatrix}.
\end{equation*}
The letter ``$T$'' stands for ``transpose''. To transpose a vector
means to turn a row into a column or vice versa.

% ----------------------------------------------------------------------