%begin-include

\section{Propositional logic. General definitions}

\begin{para}[A note on circularity]
From now on, we will be working with propositional logic and predicate logic as object theories.
This unavoidably leads one to wonder what kind of reasoning they should be able to use in their metatheory.
I mean, it does not seem legit to use propositional logic in order to study propositional logic, right?

Turns out we can safely do it.
What we will be studying is not propositional logic itself, but a formalisation of propositional logic.
It should be out of question to any rational being that propositional logic is perfectly valid (and, thus, that there is no harm in using it).
What will not be obvious, however, is that the formalisation of propositional logic that we are about to present is correct and faithfully represents it.
\end{para}

\begin{definition}
The formal system $\mathsf{P}$ of propositional logic is defined by the tuple $(L_P,\Xi_P,R_P)$ where $L_P$ is the language defined in \ref{lp}, the set of axioms $\Xi_P$ consists of all the formulas of the form
\begin{axioms}[P]
\item \label{axp1} $A\limplies (B \limplies A)$,
\item \label{axp2} $(A\limplies(B\limplies C))\limplies ((A\limplies B)\limplies (A\limplies C))$,
\item \label{axp3} $(\lnot A \limplies \lnot B) \limplies (B\limplies A)$,
\end{axioms}
for $A,B,C\in F_P$, and $R_P$ has a single rule of inference:
\[ \{A,(A\limplies B)\} \vdash_{\mathsf{P}} B\]
for all $A,B\in F_P$. This rule is known as \emph{modus ponens}, MP for short.
\end{definition}

\begin{para}
As you may have already noticed, axioms such as \ref{axp1}, \ref{axp2} and \ref{axp3} are not axioms per se. Instead, they are ``rules'' for constructing the axioms; that is why they are called \emph{axiom schemata} instead of axioms. The axioms that are constructed using a particular axiom schema are said to be \emph{instances} of the schema. For example, the axiom $(p_1\limplies p_2)\limplies p_1$ is an instance of the axiom schema \ref{axp1}.
\end{para}

\begin{definition}
An interpretation of $L_P$ based on the semantics of propositional logic is a function $i : T_P \longrightarrow \{0,1\}$. Each interpretation $i$ induces a \emph{valuation} function $v_i: F_P \longrightarrow \{0,1\}$  verifying the following semantic rules:
\begin{enumerate}
\item The function $v_i$ takes the same values as $i$ on the set of propositional variables $T_P \subseteq F_P$.
\item If $A\in F_P$, then $v_i(A) \neq v_i(\lnot A)$.
\item If $A,B\in F_P$, then $v_i(A\limplies B) = 0$ if and only if $v_i(A) = 1$ and $v_i(B) = 0$.
\end{enumerate}
Given any propositional form $A$, we say that $A$ is true in $i$ if $v_i(A) = 1$. If, instead, $v_i(A) = 0$, we say that $A$ is false.

Valid propositional forms --- that is, valid formulas in the context of propositional logic --- are referred to as \emph{tautologies}.
Propositional forms that are false under any interpretation of propositional logic are said to be \emph{contradictions}.
\end{definition}

\begin{para}
It should be obvious that an interpretation is nothing more than an assignment of truth values. The interpretation function $i$ defined for the propositional symbols represents the assignment and then the ``truth value'' of the remaining formulas, given by their image under $v_i$, is obtained inductively by applying the semantic rules that define the connectives $\lnot$ and $\limplies$.

It should be obvious, according to the semantic rules that we have used, that getting the value of $v_i(A)$ for a propositional form $A$ and an interpretation $i$ is the same as getting the truth value of $A$ under the assignment of truth values induced by $i$. Thus, all the informal methods that can used to decide the truth or falsity of a propositional forms can be safely used to compute valuations.

Notice, by the way, how we have redefined tautologies. The underlying meaning, however, is the same.
\end{para}

\begin{definition}
Let $L = (\Sigma,T,F, \Lambda)$ be a language and $\mathsf{FS} = (L,\Xi,R)$ be a formal system containing all instances of axiom schemata \ref{axp1}, \ref{axp2} and \ref{axp3} among its axioms.
\begin{itemize}
\item If there exists no formula $A \in F$ such that $\vdash_\mathsf{FS} A $ and $\vdash_\mathsf{FS} \lnot A$, the system $\mathsf{FS}$ is said to be \emph{consistent}.
\item If, for every sentence $A\in \Lambda$, we have $\vdash_\mathsf{FS} A$ or $\vdash \lnot A$, we say that $\mathsf{FS}$ is \emph{syntactically complete} or, for short, \emph{complete}.
\item The system $\mathsf{FS}$ is said to be \emph{semantically complete} if every valid formula is a theorem in $\mathsf{FS}$. Conversely, if every theorem is a valid formula, $\mathsf{FS}$ is said to be \emph{sound}.
\end{itemize}
Let $A$ be an axiom and let $\mathsf{FS}^*$ be the formal system obtained by removing the axiom $A$ from $\mathsf{FS}$. If neither $A$ nor $\lnot A$ are theorems of $\mathsf{FS}^*$, $A$ is said to be an \emph{independent axiom}. Ideally, we want all axioms to be independent in order to avoid redundancy.
\end{definition}

\begin{example} \label{dedaa}
We will prove that $\vdash A \limplies A$ for any formula $A$. We will write down all the steps of the deduction together with their justification.

\begin{deduction}{dtp}
\dstep[a1]{\ref{axp1}}{A\limplies([A\limplies A]\limplies A),}
\dstep[a2]{\ref{axp2}}{(A\limplies([A\limplies A]\limplies A))\limplies( (A\limplies[A\limplies A])\limplies(A\limplies A)),}
\dstep[a3]{MP on \dref{a1}, \dref{a2}}{( (A\limplies[A\limplies A])\limplies(A\limplies A)),}
\dstep[a4]{\ref{axp1}}{A\limplies(A\limplies A),}
\dstep{MP on \dref{a4}, \dref{a3}}{A\limplies A.}
\end{deduction}
\end{example}

\begin{theorem}[Deduction theorem in $\mathsf{P}$]
\label{dedthmprop}
For any collection of propositional forms $\Gamma\subseteq F_P$ and any formulas $A,B\in F_P$, one can deduce $\Gamma\cup\{A\}\vdash_{\mathsf{P}} B$ if and only if $\Gamma \vdash_\mathsf{P} (A\limplies B)$.
\label{}
\end{theorem}

\begin{proof}

We will first show that $\Gamma \cup \{A\} \vdash B$ implies $\Gamma \vdash (A\limplies B)$ following a proof by induction on the length $n$ of the deduction of $\Gamma\cup\{A\}\vdash B$.
If $n = 1$, there are only three possibilities: either $B = A$, $B\in \Gamma$, or $B\in \Xi_P$.

According to \ref{dedaa}, any formula $A$ verifies $\vdash (A\limplies A)$, so, in particular, $\Gamma \vdash (A\limplies A)$.
Thus, if $B = A$, it is obvious that $\Gamma\cup\{A\}\vdash A$ implies $\Gamma \vdash (A\limplies A)$.

If $B\in \Gamma$ or $B\in \Xi_P$, $\Gamma \vdash (A\limplies B)$ follows from a trivial application of MP to $B$ and \ref{axp1}.\footnote{Be aware that, when applying \ref{axp1}, ``$A$'' in \ref{axp1} should be substituted by ``$B$'' and ``$B$'' by ``$A$.''}

Let us now assume the result to hold for deductions of an arbitrary length $n$ and prove it for those of length $n+1$.
If the deduction is of length $n+1$, the formula $B$ may be, as in the base case, an axiom, equal to $A$, or an element of $\Gamma$.
But it may also have been obtained from an application of MP on two previous elements of the deduction.
In this case, those elements have deductions of length smaller than $n+1$ and, therefore, they satisfy the result by the inductive hypothesis. Let us then assume that $B$ has been obtained from an application of MP to two formulas of the form $X$ and $X\limplies B$ verifying $\Gamma \vdash (A\limplies X)$ and $\Gamma \vdash (A\limplies(X \limplies B))$. Under these conditions, we can make the following deduction from $\Gamma$.

\begin{deduction}{thdedprop1}
\dstep[h1]{By hypothesis, can be deduced from $\Gamma$}{A\limplies X,}
\dstep[h2]{By hypothesis, can be deduced from $\Gamma$}{A\limplies (X\limplies B),}
\dstep[a1]{\ref{axp1}}{(A \limplies (X\limplies B))\limplies ( (A\limplies X) \limplies (A\limplies B) ),}
\dstep[c1]{MP on \dref{h2}, \dref{a1}}{(A\limplies X)\limplies (A\limplies B),}
\dstep[c2]{MP on \dref{h1}, \dref{c1}}{A\limplies B.}
\end{deduction}

We shall now prove the converse: assuming that $\Gamma \vdash (A\limplies B)$, we will show that $\Gamma\cup \{A\} \vdash B$.
This is easy. Keeping in mind that $\Gamma \vdash (A\limplies B)$ and, therefore, that $\Gamma\cup\{A\}\vdash (A\limplies B)$, we can write an explicit deduction of $\Gamma\cup\{A\}\vdash B$:
\begin{deduction}{thdedprop2}
\dstep[h1]{By hypothesis, can be deduced from $\Gamma\cup\{A\}$}{A\limplies B,}
\dstep[h2]{Belongs to $\Gamma\cup\{A\}$}{A,}
\dstep{MP on \dref{h2}, \dref{h1}}{B.}
\end{deduction}
This concludes the proof.
\end{proof}

\begin{para}
\label{remarkdedp}
The deduction theorem is, perhaps, one of the most significant results in this section, for it explains the confusion that the implication connective $\limplies$ generates.

The formula $A\limplies B$ is a formula in the object language, full stop.
What we have shown is that the meta-theoretic statement $\vdash (A\limplies B)$ meaning ``$(A\limplies B)$ is a theorem in $\mathsf{P}$'' is equivalent to the meta-theoretic statement $A\vdash B$ meaning ``$B$ can be deduced from $A$''.

We will later introduce the first-order version of this metatheorem, which will shed even more light on this matter.
\end{para}

\begin{theorem}
\label{pprop}
The following metatheorems about propositional logic are true:
\begin{statements}
\item \label{pprop:sound} If $\Gamma \subseteq F_P$ and $X\in F_P$ are such that $\Gamma \vdash X$, then $\Gamma\vDash X$. In particular, if a formula $X\in F$ is a theorem in $\mathsf{P}$, it is a tautology; which is to say that $\mathsf{P}$ is sound.
\item The formal system $\mathsf{P}$ is consistent.
\item Every tautology is a theorem in $\mathsf{P}$. In other words, $\mathsf{P}$ is semantically complete.
\end{statements}
These results show beyond any doubt that $\mathsf{P}$ is a correct formalisation of propositional logic.
\end{theorem}

\begin{proof}
\begin{parlist}
\item We proceed by induction on the length $n$ of the deduction.
If $n = 1$, then either $X \in \Gamma$ (in which case the result is obvious) or $X$ is an axiom of $\mathsf{P}$.
In order for the result to hold, we need to see that every axiom of $\mathsf{P}$ is true under any interpretation, i.e., that it is a tautology.

We shall first analyse \ref{axp1}.
I think we can both agree that, given any $A,B\in F$, the formula $A\limplies (B\limplies A)$ either is or is not a tautology.
Thus, we just need to show that it is impossible for $A\limplies (B\limplies A)$ not to be a tautology.
Were that formula not a tautology, there would necessarily exist an interpretation $i$ under which it would be false.
Nonetheless, according to the semantic rules of propositional logic, that would mean that $v_i(A) = 1$ yet $v_i(B\limplies A) = 0$.
But, by those same rules, $v_i(B\limplies A) = 0$ can only mean that $v_i(B) = 1$ and $v_i(A) = 0$.\
Consequently, $A\limplies (B\limplies A)$ can only be false under an interpretation $i$ verifying both $v_i(A) = 0$ and $v_i(A) = 1$.
As that is impossible, we can safely conclude that all the axioms defined by the schema \ref{axp1} are tautologies.

We can proceed in a similar fashion regarding \ref{axp2}.
Let $A,B,C\in F_P$ be formulas.
If an interpretation $i$ is such that
\[ v_i( \underbrace{(A\limplies(B\limplies C))}_{D_1} \limplies \overbrace{((A\limplies B) \limplies (A\limplies C))}^{D_2}) = 0,\]
then we necessarily have $v_i(D_1) = 1$ and $v_i(D_2) = 0$.
Having $v_i(D_2) = 0$ implies that $v_i(A\limplies C) = 0$ and $v_i(A\limplies B) = 1$, which can only mean that $v_i(A) = 1$, that $v_i(C) = 0$ and that $v_i(B) = 1$.
Simultaneously, $v_i(D_1) = 1$ with $v_i(A) = 1$ leads to $v_i(B\limplies C) = 1$, which --- with $v_i(C) = 0$ --- could only be the case if $v_i(B) = 0$.
Having reached a contradiction, we can conclude that all the axioms defined by \ref{axp2} need be true under any interpretation and, therefore, that they are all tautologies.

Finally, let us tackle \ref{axp3}. Given any two formulas $A,B\in F_P$, if we assume an interpretation $i$ to exist such that
\[ v_i\left( (\lnot A \limplies \lnot B) \limplies (B \limplies A) \right) = 0,\]
it will need to verify $v_i(B \limplies A) = 0$ and $v_i(\lnot A\limplies \lnot B) = 1$.
The former of these conditions implies that $v_i(A) = 0$ and $v_i(B) = 1$, which --- according to the semantic rules --- is equivalent to having $v_i(\lnot A) = 1$ and $v_i(\lnot B) = 0$. If $v_i(\lnot A\limplies \lnot B) = 1$ and, as we have just shown, $v_i(\lnot A) = 1$, then we necessarily have $v_i(\lnot B) = 1$, which is, as expected, a contradiction.
This proves that all the axioms defined by \ref{axp3} are tautologies.

Now that we have completed the base case, let us assume the result to hold for deductions of length equal to or smaller than $n$, and we will prove it for those of length $n+1$.
If the deduction has length $n+1$, $X$ may be an element of $\Gamma$ or an axiom as in the base case, or it may have been obtained from an application of MP on two previous elements of the deduction that, therefore, verify the result according to the inductive hypothesis.
Thus, we need to show that, given any two formulas of the form $A$ and $A\limplies B$, if they are true in any particular interpretation $i$, so is $B$.

If $v_i(A) = 1$ and $v_i(A\limplies B) = 1$, it is obvious that we need to have $v_i(B) = 0$. Indeed, if $v_i(B) = 0$ and $v_i(A) = 1$, that would yield $v_i(A\limplies B) = 0$.
Thus, we have shown that, for any interpretation $i$ in which any two formulas $A$ and $A\limplies B$ are true, $B$ is true too.

\item Let $A$ be a theorem in $\mathsf{P}$.
By \ref{pprop:sound}, $A$ need be a tautology and, therefore, for an interpretation $i$, we will have $v_i(A) = 1$.
According to the semantic rules, this means that, under any interpretation $i$, $v_i(\lnot A) = 0$.
Consequently, $\lnot A$ is not a tautology and since, by \ref{pprop:sound}, being a tautology is a necessary condition for any formula to be a theorem in $\mathsf{P}$, $\lnot A$ cannot be a theorem.

This shows that no formula $A$ can verify both $\vdash_{\mathsf{P}} A$ and $\vdash_{\mathsf{P}} \lnot A$, and, therefore, that $\mathsf{P}$ is consistent.

\item The details of this proof are pretty tedious to go through. If you feel motivated enough to do it, feel free to visit \ref[appendix]{psemcom} in the appendices.
\end{parlist}
\end{proof}

\begin{lemma}[Conjunction introduction rule]
Let $\Gamma$ be a collection of propositional forms and let $A$ and $B$ be two arbitrary propositional forms. One can deduce $\Gamma\vdash_\mathsf{P} (A\land B)$ if and only if one can deduce both $\Gamma \vdash_{\mathsf{P}}A$ and $\Gamma \vdash_{\mathsf{P}}B$.
\label{cinr}
\end{lemma}

\begin{proof}
If $\Gamma \vdash (A\land B)$, then, since $(A\land B) \limplies A$ and $(A\land B)\limplies B$ are both tautologies --- and, therefore, theorems in $\mathsf{P}$ --- it follows by a direct application of the MP rule that $\Gamma \vdash A$ and $\Gamma \vdash B$.

Conversely, if $\Gamma \vdash A$ and $\Gamma \vdash B$, we know $A\limplies (B\limplies (A\land B))$ to be another tautology. Two applications of MP yield $\Gamma \vdash (A\land B)$.
\end{proof}

\begin{proposition}
\label{piff}
Given any propositional forms $A$ and $B$, $\Gamma \vdash_{\mathsf{P}} (A \liff B)$ if and only if $\Gamma  \vdash_\mathsf{P} (A\limplies B)$ and $\Gamma \vdash_\mathsf{P} (B\limplies A)$.

In particular, if $\Gamma = \emptyset$, $A\liff B$ is a theorem of $\mathsf{P}$ if and only if so are $A\limplies B$ and $B\limplies A$.
\end{proposition}

\begin{proof}
If we have both $\Gamma\vdash (A\limplies B)$ and $\Gamma \vdash (B\limplies A)$, by \ref{cinr}, we know that $\Gamma \vdash ( (A\limplies B) \land (B\limplies A))$, which is, according to our definition of $\liff$, the same as $\Gamma \vdash (A\liff B)$.
The converse is also a direct consequence of \ref{cinr}.
\end{proof}

\begin{lemma}
Let $A$, $X$ and $Y$ be propositional forms. If $X\liff Y$ is a tautology and $A'$ denotes the propositional form resulting from replacing each appearance of $X$ in $A$ by $Y$, then $A\liff A'$ is a tautology and, therefore, a theorem in $\mathsf{P}$.
\label{replacetautology}
\end{lemma}

\begin{proof}
Let $i$ be any interpretation.
It suffices to notice that, as $X \liff Y$ is a tautology, we always have $v_i(X) = v_i(Y)$.
Therefore, as $A'$ is obtained by replacing every occurrence of $X$ by an occurrence of $Y$, we necessarily have $v_i(A) = v_i(A')$.
Consequently, $v_i(A\liff A') = 1$.
\end{proof}

\begin{para}
\label{lpinformal}
It is very easy to see that the following formulas are tautologies for any propositional forms $A$, $B$ and $C$:
\begin{statements}
\item $(A\liff B) \liff (B\liff A)$,
\item $(A\land B) \liff (B\land A)$,
\item $((A\land B) \land C) \liff (A\land (B \land C))$,
\item $((A\lor B) \lor C) \liff (A\lor (B \lor C))$,
\end{statements}
This, together with \ref{replacetautology}, should be enough to convince you that the conventions we introduced in \ref[prel]{hierarchy} can be safely used when working with $L_P$ in $\mathsf{P}$.
In particular, this shows that there is no harm in swapping formulas around the $\lor$, $\land$ and $\liff$ connectives.
\end{para}

\begin{proposition}
\label{impsystemp}
Let $A$, $B$, $A_1$, $A_2$, $B_1$ and $B_2$ be propositional forms and $\Gamma$ a set of propositional forms.
\begin{statements}
\item One can deduce $\Gamma \vdash_\mathsf{P} (A \limplies (B_1\land B_2))$ if and only if one can deduce both $\Gamma \vdash_\mathsf{P} (A\limplies B_1)$ and $\Gamma \vdash_\mathsf{P} (A\limplies B_2)$. In particular, if $\Gamma = \emptyset$, $A\limplies (B_1\land B_2)$ is a theorem of $\mathsf{P}$ if and only if so are $A\limplies B_1$ and $A \limplies B_2$.

\item One can deduce $\Gamma \vdash_\mathsf{P} ( (A_1\land A_2) \limplies B_1 )$ if and only if one can deduce $\Gamma \vdash_\mathsf{P} (A_1 \limplies (A_2 \limplies B))$ or $\Gamma \vdash_\mathsf{P} (A_2 \limplies (A_1 \limplies B)$. In particular, if $\Gamma = \emptyset$, $(A_1\land A_2) \limplies B$ is a theorem if and only if so are $A_1\limplies (A_2\limplies B)$ or $A_2 \limplies (A_1 \limplies B)$.
\end{statements}
\label{implesmani}
\end{proposition}

\begin{proof}
\begin{parlist}
\item Follows from a direct application of MP taking into account \ref{piff} and the fact that \[(A\limplies (B_1\land B_2)) \liff ((A\limplies B_1) \land (A\limplies B_2))\] is a tautology.
\item Follows from a direct application of MP taking \ref{piff} into account together with the fact that both
\begin{gather*}
((A_1 \land A_2)\limplies B) \liff (A_1 \limplies (A_2 \limplies B)),\\
((A_1 \land A_2)\limplies B) \liff (A_2 \limplies (A_1 \limplies B))
\end{gather*}
are tautologies.
\end{parlist}
\end{proof}

\begin{proposition}[Principle of explosion]
\label{pexpprop}
Anything can be deduced from a false premise: given any two propositional forms $A,B\in F_P$, we have $A,\lnot A \vdash B$.
\end{proposition}

\begin{proof}
It is easy to see that the formulas $A\limplies (A\lor B)$ and $\lnot A \limplies ( (A\lor B) \limplies B )$ are tautologies. We know $\mathsf{P}$ to be semantically complete and, therefore, we know those tautologies to be theorems of $\mathsf{P}$. The deduction of $B$ from $A,\lnot A$ is then very simple.
\begin{deduction}{dedpexpprop}
\dstep[h1]{Premise}{A,}
\dstep[h2]{Premise}{\lnot A,}
\dstep[t1]{Theorem}{A\limplies (A\lor B),}
\dstep[t2]{Theorem}{\lnot A \limplies ( (A\lor B) \limplies B),}
\dstep[d1]{MP on \dref{h1}, \dref{t1}}{A \lor B,}
\dstep[d2]{MP on \dref{h2}, \dref{t2}}{(A\lor B) \limplies B,}
\dstep{MP on \dref{d1}, \dref{d2}}{B.}
\end{deduction}
This completes the proof.
\end{proof}


\documentclass[11pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{hyperref}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{tikz}
\usepackage{geometry}
\usepackage{mathtools}

\newtheorem{theorem}{Theorem}[section]
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{corollary}[theorem]{Corollary}

\title{Sample LaTex Document}
\author{Tom Grubb}
\date{January 2021}

\begin{document}

\maketitle

\abstract{Abstract abstract abstract abstract abstract wow what a useful informative abstract.}

\section{Introduction}
This is a very basic LaTex document. It was created for UCSD's Math 157 class to highlight some of LaTex's functionality. The .tex file starts with a \emph{preamble}, which includes details on the title of this document, as well as several import statements. The result of compiling that .tex file is this pdf. This document was created online using Overleaf.

\section{A Theorem, a Proof, and the Align Environment}
In this section we will prove a basic fact about \textbf{natural numbers}. Look at how the phrase natural numbers is \emph{bolded} and how the word bolded is italicized; you can see how this was done in the source file.

\begin{theorem}
For an integer $n,$ $n\geq 1$, we have
$$
\sum_{k = 1}^{n} k = \frac{n(n+1)}{2}
$$
\end{theorem}

\begin{proof}
We induct on $n$. In the base case, $n=1$. In this case the sum on the left hand side has a single term equal to $1$ in it. The right hand side simplifies to
$$
\frac{1\cdot 2}{2} = 1.
$$
Thus the base case holds.

For the induction step, assume the theorem holds for some integer $n$. Then
$$
\sum_{k=1}^{n+1} = (n+1) + \sum_{k=1}^nk.
$$
By the induction hypothesis,
$$
\sum_{k=1}^{n}k = \frac{n(n+1)}{2},
$$
and thus
$$
\sum_{k=1}^{n+1} = (n+1)+ \frac{n(n+1)}{2}.
$$
Simplifying the right hand side gives
\begin{align*}
\sum_{k=1}^{n+1} &= \frac{2(n+1)+n(n+1)}{2} \\
&=\frac{(n+1)(n+2)}{2}.
\end{align*}
This finishes the proof of our theorem.
\end{proof}

In the next section we will explore other LaTex environments.

\section{LaTex Lists}
In LaTex you can make bulleted and numbered lists. A bulleted list uses the itemize environment:
\begin{itemize}
    \item This is the first entry in a bulleted list.
    \item This is the second entry in a bulleted list.
\end{itemize}
A numbered list uses the enumerate environment:
\begin{enumerate}
    \item This is the first entry in a numbered list.
    \item This is the second entry in a numbered list. It has a sublist in it!
    \begin{enumerate}
        \item This is a sublist within my numbered list! It uses a nested enumerate environment.
        \item Wow!
    \end{enumerate}
\end{enumerate}

\section{LaTex Tables}
There are two ways of making tables in LaTex. The tabular environment is for making \emph{text based tables}. Often you highlight a tabular environment using the \emph{center} environment:
\begin{center}
    \begin{tabular}{c|c}
    The columns are separated by ampersands & This is column 2 \\
    The rows are separated by two backslashes.  & This is row 2, column 2
\end{tabular}
\end{center}
You can add horizontal lines to the table using the hline command. Vertical lines are formed by adding pipes into the table metadata: \{ $\vert$ c $\vert$ c $\vert$\} means you have two columns which are \emph{centrally aligned} and have vertical lines in between each column:
\begin{center}
    \begin{tabular}{|c|c|}
    \hline
    The columns are separated by ampersands & This is column 2 \\
    \hline
    The rows are separated by two backslashes.  & This is row 2, column 2\\
    \hline
\end{tabular}
\end{center}
You could optionally change the alignment using l, c, or r in the table metadata:
\begin{center}
    \begin{tabular}{|l|c|r|}
    \hline
    Col 1& Col 2& Col3\\
    \hline
    Left aligned & Centrally aligned & Right aligned \\
    \hline

\end{tabular}
\end{center}

An array environment is similar to a tabular environment, but it is for math mode/numeric data. In particular, it should go between double dollar signs, or in an equation environment:
\begin{equation}
    \begin{array}{|c|c|}
    \hline
        \pi & \tau \\
        \hline
         3.14\dots&6.28\dots \\
         \hline
    \end{array}
\end{equation}
Note that the equation environment implicitly numbers the equations. You can change this by messing around with the settings if you wanted.

\section{Tikz}
If you want to get fancy with your files, you can use Tikz to create images/plots within your file. This is more advanced and will not be necessary for this class, but here are some examples that I have taken from Overleaf's documentation (available here: \url{https://www.overleaf.com/learn/latex/TikZ_package}):
\begin{center}
\begin{tikzpicture}

\draw (-2,0) -- (2,0);
\filldraw [gray] (0,0) circle (2pt);
\draw (-2,-2) .. controls (0,0) .. (2,-2);
\draw (-2,2) .. controls (-1,0) and (1,0) .. (2,2);

\end{tikzpicture}

\end{center}

\begin{center}
\begin{tikzpicture}[
roundnode/.style={circle, draw=green!60, fill=green!5, very thick, minimum size=7mm},
squarednode/.style={rectangle, draw=red!60, fill=red!5, very thick, minimum size=5mm},
]
%Nodes
\node[squarednode]      (maintopic)                              {2};
\node[roundnode]        (uppercircle)       [above of=maintopic] {1};
\node[squarednode]      (rightsquare)       [right of=maintopic] {3};
\node[roundnode]        (lowercircle)       [below of=maintopic] {4};

%Lines
\draw[->] (uppercircle.south) -- (maintopic.north);
\draw[->] (maintopic.east) -- (rightsquare.west);
\draw[->] (rightsquare.south) .. controls +(down:7mm) and +(right:7mm) .. (lowercircle.east);
\end{tikzpicture}
\end{center}
For more on TikZ, I recommend Google or Office Hours.
\section{Conclusion}
Hopefully this document helps! LaTex is very well documented online, so Googling is always a good way to get more information. But you can always reach out to course staff for a more in depth discussion of LaTex!

\end{document}


\documentclass{article}
\usepackage{bbold}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{commath}
\usepackage{bm}
\usepackage[makeroom]{cancel}

\begin{document}

\section{TCRen scoring}

\subsection{Deriving empirical contact scoring matrices}

Let $\tilde{n}_{ab}$ denote number of contacts between residues of amino acid $a$ of TCR CDR regions and $b$ of an epitope. We compute the symmetrized matrix $n_{ab} = \frac{1}{2} \left(\tilde{n}_{ab} + \tilde{n}_{ba}\right)$ and marginals $n_a = \sum_{b} n_{ab}$ and $n = \sum_a n_{a}$.

We define an observed-to-expected contact frequency matrix as

\begin{equation}
\phi_{ab} = n^{obs}_{ab} /  n^{exp}_{ab} = \frac{n_{ab} n}{n_a n _b}
\end{equation}

and use it to derive scores for amino acid pairs similar to BLOSUM matrix derivation where scores are calculated as $S_{ab} \sim \log \phi_{ab}$.

We decompose $\phi_{ab}$ into (empirical) baseline interaction score $S_{a}$ for a given amino acid defined as

\begin{equation}
S_{a} = \frac{1}{20} \sum_{b} \log \phi_{ab}
\end{equation}

and specific interaction score $S_{ab}$ defined based on the formula

\begin{equation}
\log \phi_{ab}= S_{ab} + S_{a} + S_{b}
\end{equation}

effectively decoupling pairwise interactions from raw amino acid frequencies in contacting regions. E.g. by using this correction we account for Glycine residues that are frequent in the central region of CDR3$\beta$ that is almost always close to an antigen in any TCR:pMHC complex.

In present study we compute a single scoring matrix based on CDR1,2,3 regions of both TCR$\alpha$ and $\beta$ chains.

\subsection{Scoring TCR:pMHC pairs}

We score each TCR:pMHC pair having residues $\left\{a_i\right\}$ of the TCR and $\left\{b_i\right\}$ of the antigen using by subtracting the non-specific binding score from the specific one:

\begin{equation}
S\left(\left\{a_i\right\},\left\{b_i\right\}\right)=\sum_{i,j} S_{a_{i}b_{j}} \delta_{ij} - \sum_{i} S_{a_{i}} - \sum_{j} S_{b_{j}}
\end{equation}

where $\delta_{ij} \in \left\{0, 1\right\}$ is $1$ for contacting residues and $0$ otherwise. We subtract baseline scores in order to account for intrinsic properties of amino acids to bind in a non-specific manner. As four flanking residues from both sides of CDR3 are involved in the $\Omega$-loop formation, are almost never involved in TCR:antigen contacts and are subject to strong bias from V/J segment choice, we would substitute $\left\{a_i\right\}$ by $\left\{a_i\right\} / \left\{a_{1..4}\right\} \cup \left\{a_{-4..-1}\right\}$ for CDR3 regions in our calculations.

\subsection{Physical interpretation of scoring}

(Experimental, not sure it should be included) The scoring can be decomposed into the following components

\begin{align}
\begin{split}
S\left(\left\{a_i\right\},\left\{b_i\right\}\right) &= \sum_{i,j} S_{a_{i}b_{j}} \delta_{ij} \\ &- \sum_{i} S_{a_{i}} \left(1 - \delta_i\right) - \sum_{j} S_{b_{j}} \left(1 - \delta_j\right) \\ &- \sum_{i} S_{a_{i}} \delta_i - \sum_{j} S_{b_{j}} \delta_j \\
 &= -\left(E_{bound} - E_{unbound} - E_{bound,nonspec}\right) \\
 &= -\left(\Delta E_{bound} - E_{bound,nonspec}\right) = -\Delta \Delta E
\end{split}
\end{align}

where $\delta_i = \max_{j} \delta_{ij}$ identifying a residue having at least one contact, assuming $E_{unbound,nonspec}=0$.

\end{document}


\section{Basic advance}
To change between the different ways of advancing the solution, change
the code in advance() between the godunov, molrk2 and molrk3 options.

MOLRK2 follows the same RK2 algorithm as CNS.
MOLRK3 follows the algorithm described in gottlieb, et al.
$$
u^(1) = u^n +  + \dt L(u^n)  \\
u^(2) = (3/4) u^n + (1/4) u^(1)  + (1/4) L(u^(1))
u^{n+1} = (1/3) u^n + (2/3) u^(2)  + (2/3) L(u(2))
$$

\subsection{Flux register factors}

Since
$$
u^{n+1} = (1/3) u^n + (2/3) u^(2)  + (2/3) \dt L(u(2))
$$

This means
$$
u^{n+1} = (1/3) u^n + (2/3)((3/4)(u^n) + (1/4)(u^1) + (1/4)\dt L(u^1))  + (2/3)( \dt L(u(2)))
$$

which, in turn, means
$$
u^{n+1} = (1/3) u^n + (2/3)((3/4)(u^n) + (1/4)(u^n + \dt L(u^n)) + (1/4)\dt L(u^1))  + (2/3)( \dt L(u^2))
$$

So the coefficient of $L(u^n) = (2/3)(1/4)\dt$, so I should set the
flux register coefficient for the first evaluation $c_n = (1/6)\dt$.
The coeffcient of $L(u^1) = (2/3)(1/4)\dt = 1/6 \dt$
The coeffcient of $L(u^2) = (2/3) \dt$.   So for the three evaluations
of the operator, the time step  that fluxes should see are $\dt/6,
dt/6, 2\dt/3$, respectively.

\section{Stuff I am not sure about}

\begin{itemize}
  \item I am not sure where to set the time levels  in the fill patch thing.
  \item I am not sure what time to use for time dependent
    vellocities.
\end{itemize}

\section{Basic Algorithm}

We start with cell averaged scalar $<\phi>$ and a pointwise velocity at faces.

\begin{itemize}
  \item Get pointwise $\phi$ because I will need this to get higher
    order gradient at faces.
$$
\phi = <\phi> -\frac{h^2}{24}(L(<\phi>)),
$$
   where $L$ is the standard second order Laplacian.
\item Get the average of the scalar over the faces .
$$
<\phi>_{i+1/2} =  \frac{7}{12}(<\phi>_i    + <\phi>_{i+1})
                -\frac{1}{12}(<\phi>_{i-1} + <\phi>_{i+2})
$$

\item Get the point values of the scalar on faces from the face
  average:
$$
\phi_{i+1/2} = <\phi>_{i+1/2} - \frac{1}{24} L^T(<phi>)_{i+1/2},
$$
where $L^T$ is the Laplacian that does not include terms normal to the
face (what McCorquodale refers to as the ``transverse Laplacian'').

\item Compute pointwise fluxes $F$ (including diffusion fluxes).
$$
F_{i+1/2} = u_{i+1/2} \phi_{i+1/2}  - nu G^x(phi)_{i+1/2}
$$
where the normal gradient $G^x$ is computed using fourth order finite
differences  of the pointwise cell-centered scalar $\phi$.
$$
G^x(\phi)_{i+1/2} = \frac{\partial \phi}{\partial x} + O(h^4)
= (1/h)(\phi_{i+1}-\phi_{i})
- (1/(4h))((\phi_{i+2} + \phi_i - 2\phi_{i+1}) - (\phi_{i+1} + \phi_{i-1} - 2\phi_{i}))
$$

\item Transform the pointwise fluxes $F$ into face-averaged fluxes $<F>$
$$
<F> = F - (1/24) L^T(F).
$$

Algorithm seems to work.  To fully test, I need to
\begin{itemize}
  \item Write MPI=based convergence script.
  \item Run tests (with and without AMR), record results.
  \item Write up document.
\end{itemize}
\end{itemize}


\documentclass[10pt,a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\begin{document}
$\tau \sim (\rho H R^2_p/\sigma)^{1/2}$

\section*{Charge}
We find that
\[q = k E_0 V_d^{2/3},\]
with $k=5.01 \times 10^{-11} \pm  2.85 \times 10^{-11}$ and $R^2 = 0.946$.
\\
\begin{eqnarray*}
q = k E_0 V_d^{2/3},
\end{eqnarray*}
with $k \approx 1.3 \times 10^{-10}$ (Griffiths, 1999).
\\
\[q = 4 \pi \epsilon_0 \beta E_0 R_d^2, \]
with $\beta \approx 2.63$, (Takamatsu \emph{et al.}, 1981).
\\
Surface potential $\varphi_s$ is related to characteristic electric field by
\[ E_0  = \frac{\varphi_s \kappa}{4 \pi \epsilon l}. \]

\section*{Parameter Estimation}
We can't directly measure $q$ experimentally, but we can estimate it by fitting the dynamical model to the data. Bayes' theorem:
\[\begin{array}{lll}
& \centering \mbox{prob}(X|D, I) & \mbox{posterior probability density function},\\
& \centering \mbox{prob}(D|X, I) & \mbox{likelihood function},\\
& \centering \mbox{prob}(X|I) &  \mbox{prior probability density function},\\
& \centering \mbox{prob}(D|I) &  \mbox{evidence}.
\end{array}
\]
The Maximum Likelihood Estimate of the model parameters is found by maximizing the probability
\[\mathcal{M} = \ln(\mathcal{L}) = \ln(\mbox{prob}(D|X, I)) = \mbox{const} - \frac{\chi^2}{2}.\]

\section*{Parameter Estimation}
Mathematically we state that we find the parameters $\mathbf{x}$ that solve the inverse problem $G(\mathbf{x}) = \mathbf{d}$, using a direct search method (\emph{Nelder-Mead}).
\[
\mbox{min} \hspace{2 mm} \chi^2 = \mbox{min} \hspace{2 mm} \sum^n_{i=1} \frac{\left({y_d(t)}_i - y_G(t, \mathbf{x})_i \right)^2}{y_G(t, \mathbf{x})_i}
\]
\begin{eqnarray*} \mbox{} \hspace{2 mm} \begin{split} \mathbf{x} = \left\{ \begin{array}{ll}      & q\\
		  &	V_d\\
          & \sigma
          \end{array} \right.
          \end{split} \hspace{2 mm} \mbox{subject to constraints} \hspace{2 mm} \begin{split}
          g = \left\{ \begin{array}{ll}
           V_d &\pm \hspace{2 mm} u_{exp}\\
      	   \sigma &\pm  \hspace{2 mm} u_{exp}\\
      	   y_0 &\pm \hspace{2 mm} u_{exp}\\
      	   t_0 &\pm \hspace{2 mm} u_{exp}\\
          \end{array} \right.
          \end{split}
\end{eqnarray*}
where $y_G(t, \mathbf{x})$ is a numerical solution of the equation of motion
\[
m y'' = \frac{1}{2} \rho C_D A_d {y'}^2 + q E(y) + K q^2 y^{-2} \]

\section*{Governing Equation}
\begin{equation*}
m y'' = - \mathbf{F}_D - \mathbf{F}_E, \hspace{5 mm} y(0) = R_d, \hspace{5 mm} y'(0) = U_0,
\label{gov_eqn}
\end{equation*}
\\
\begin{equation*}\label{force_density}
\mathbf{F}_E = \rho_f \mathbf{E} + \frac{1}{2} \left| E \right|^2 \nabla \epsilon - \nabla \left( \frac{1}{2} \rho \left( \frac{\partial \epsilon}{\partial \rho} \right)_T \left| E \right|^2 \right) .
\end{equation*}
\\
\begin{eqnarray*}
 \mathbf{F}_E &=& q \mathbf{E} + \mathbf{F}_{DEP} + \mathbf{F}_I \\
 &=& q \mathbf{E} + \frac{k q^2}{16 \pi \epsilon_0 } y^{-2} \hat{\mathbf{j}} + 2 \pi R_d^3 \kappa_1 \epsilon_0 K \nabla E^2,
\end{eqnarray*}
\\
\begin{eqnarray*} \label{gov_eqn_subs}
&m y'' = - \frac{1}{2} C_D \rho A {y'}^2 - q E - \frac{k q^2}{16 \pi \epsilon_0} y^{-2}- 2 \pi R_d^3 \kappa_1 \epsilon_0 K \nabla E^2,& \nonumber \\
&y(0) = R, \hspace{1 mm} y'(0) = U_0 .&
\end{eqnarray*}
\\
We note we can neglect polarization stresses when
\begin{eqnarray}
\frac{ \kappa_2 \epsilon_0 K R_d^2 E_0}{q} \ll 1. \nonumber
\end{eqnarray}

\newpage
\section*{Scaling}
Introducing the scaled variables
\begin{equation*}
 \bar{t} = \frac{t}{t_c}, \hspace{10 mm} \bar{y} = \frac{y}{y_c},
 \end{equation*}
where $y_c$ and $t_c$ are characteristic length and time scales respectively, and using the coordinate transformation $y(0) - R = 0$, the governing equation becomes
\begin{eqnarray*}
& \bar{y}'' = - \mathbf{\Pi}_1 \bar{y}'^2
- \mathbf{\Pi}_2 \bar{E} ( \bar{y} )
- \mathbf{\Pi}_3 \left( \mathbf{\Pi}_4  \bar{y} + 1 \right)^{-2}, & \nonumber \\
& \bar{y}(0) = 0, \hspace{1 mm} \bar{y}'(0) = \mathbf{\Pi}_5&, \label{scaled_eqn}
\end{eqnarray*}
with
\[ \mathbf{\Pi}_1 = \frac{C_D \rho A y_c}{2 m}, \hspace{5 mm}
\mathbf{\Pi}_2 = \frac{q E_0 t_c^2}{m y_c}, \hspace{5 mm}
\mathbf{\Pi}_3 = \frac{k q^2 t_c^2}{16 \pi \epsilon_0 R^2 m y_c}, \hspace{5 mm}
\mathbf{\Pi}_4 = \frac{y_c}{R}, \hspace{5 mm}
\mathbf{\Pi}_5 = \frac{U_0 t_c}{y_c}.\]
\section*{Scaling: short-times limit}
With $y_c \sim U_0 t_c$ and picking $t_c$ such that Coulombic force $\mathbf{\Pi}_2 \sim \mathcal{O}(1)$, the intrinsic scales become
\[ t_c \sim \frac{m U_0}{q E_0}, \hspace{5 mm}
y_c \sim \frac{m U_0^2}{q E_0} .
\]
With these scales the governing equation is
\begin{eqnarray*}
& \bar{y}'' = -1 - \mathbb{I}\mbox{m} \left( \mathbb{E}\mbox{u}\bar{y} + 1 \right)^{-2} ,& \nonumber \\
& \bar{y}(0) = 0, \hspace{1 mm} \bar{y}'(0) = 1 .& \label{img_limit}
\end{eqnarray*}
with
\[ \mathbb{I}\mbox{m} \equiv \frac{k q}{16 \pi \epsilon_0 R_d^2 E_0} = \mathbf{\Pi}_3 = \frac{\mbox{Image force}}{\mbox{Coulomb force}}, \hspace{5 mm}
\mathbb{E}\mbox{u} \equiv \frac{m U_0^2}{q E_0 R_d} = \mathbf{\Pi}_4 = \frac{\mbox{Inertia}}{\mbox{Coulomb force}}.
\]
\section*{Scaling: long-times limit}
\[ t_c \sim \frac{R_d^2}{L^2} \frac{4 \pi m U_0}{q E_0}, \hspace{5mm} y_c \sim \frac{R_d^2}{L^2} \frac{4 \pi m U_0^2}{q E_0}.
\]
With this scaling the non-dimensional governing equation is
\begin{eqnarray*}
&\bar{y}'' = - \mathbb{D}\mbox{g} \mathbb{E}\mbox{u}_+ \bar{y}'^2 - \left( \mathbb{E}\mbox{u}_+ \bar{y} + 1 \right)^{-2}, & \nonumber \\
& \bar{y}(0) = 0, \hspace{1 mm} \bar{y}'(0) = 1 & \label{drag_limit}
\end{eqnarray*}
where $\mathbb{D}\mbox{g}$ the drag number, $\mathbb{D}\mbox{g} \equiv \frac{C_D \rho_a}{\rho_l} = \mathbf{\Pi}_1 {\mathbb{E}\mbox{u}}_+^{-1}$ and $\mathbb{E}\mbox{u}_+ = 4 \pi \frac{R_d^2}{L^2} \mathbb{E}\mbox{u}$.

\newpage
\section*{Escape Velocity}
As ${\mathbb{E}\mbox{u}}_+$ grows, the time-of-flight grows rapidly, approaching an asymptote at a certain critical velocity; this is an electrostatic escape velocity, $U_e$. From
\[ m u' = - \frac{q E_0 y_c^2}{y^2}, \]
we find
\[ u(y) = \pm U_0 \left[1 + \frac{2q E_0 y_c^2}{m U_0^2} \left( \frac{1}{y} - \frac{1}{R_d} \right) \right]^{1/2}.
\]
This equation has an asymptotic velocity, $U_{\infty}$ at $y = \infty$ which is real if
\[ U_0 \geq  U_e = y_c \sqrt{\frac{2 q E_0 }{m R_d}}.
\]
Drops will escape the electric field when
\begin{equation*}\label{escape}
{\mbox{B}^2 \mathbb{E}\mbox{u}}^3 > 1,
\end{equation*}
where $B^2= 8 \pi\frac{ R_d^2}{L^2}$.
\newpage
\section*{Asymptotic Estimate of Trajectory Apoapse}
The scaled equation is still non-linear but we can get an approximate analytical solution by means of a regular perturbation, using the naive expansion
\[ \bar{y}(\bar{t}) \sim \bar{y}_0(\bar{t}) + \epsilon \bar{y}_1(\bar{t}) + \epsilon^2 \bar{y}_2(\bar{t}) \ldots \epsilon^n\bar{y}_n(\bar{t})
,\]
we obtain
\begin{eqnarray*}
&\bar{y}(\bar{t}) = \bar{t} + \frac{\bar{t}^{2}}{2} \left(-1 - \alpha\right) + \epsilon \left(\frac{\alpha \bar{t}^{3}}{3} + \frac{\alpha \bar{t}^{4}}{12} \left(-1 - \alpha\right)\right)& \\
&+ \epsilon^{2} \left(- \frac{\alpha \bar{t}^{4}}{4} + \frac{\alpha \bar{t}^{5}}{60} \left(9 + 11 \alpha\right) + \frac{\alpha \bar{t}^{6}}{360} \left(-9 - 20 \alpha - 11 \alpha^{2}\right)\right) + \mathcal{O}(\epsilon^3).&
\end{eqnarray*}

\end{document}


%!TEX root =  ../main.tex

\subsection{Numerical, Graphical, Algebraic, and Verbal}

\objective{Use functions defined numerically, graphically, algebraically, or verbally.}

A \gls{function} is a relation that uniquely associates members of one \gls{set}
with members of another set.
More simply, a function connects members of a \gls{domain} to members of a \gls{range},
with the proviso that each element in the domain maps onto only one element in the range.
When graphing a function, it is customary to plot the \emph{input} or \gls{independent variable}
on the $x$-axis (going left to right), and the \emph{output} or \gls{dependent variable} on
the $y$-axis (going down to up).  The independent variable is so-called because it may
be arbitrarily chosen from within the domain.  The dependent
variable is then forced to be a particular value (by the function).

\subsubsection{Numerically}
Typically, information is gathered about the physical world as \gls{discrete} numbers,
occurring at fixed points. Such information may be represented as in table~\ref{tab:Numerically}.

\columntable[-0.75in]{1.6in}{
  \begin{tabular}{ lr }
  \hline
  \textbf{x} (months since) & \textbf{y} (100's of \$)\\
  \hline
  0 & 1200.00 \\
  1 & 840.00 \\
  10 & 1680.24 \\
  16 & 2072.98 \\
  23 & 2264.72 \\
  36 & 4303.93 \\
  48 & 6770.53 \\
  \hline
  \end{tabular}
   }{Monthly income represented numerically\label{tab:Numerically}}

\subsubsection{Graphical}
A helpful way to visualize relationships between input and output
sets is a \gls{graph} of the function.  Similar techniques
include diagrams (see Figure for an example) and maps (see Figure for an example).
``Mapping diagrams'' blur the line between numerical and graphical data
(see Figure ~\ref{fig:graphically} for an example).

\begin{figure}[h]
\centering
\begin{tikzpicture}
\begin{axis}[samples=500,domain=0:48,restrict y to domain =0:8500]
\addplot[very thick,orange ]plot (\x, {1000*(pow(1.04,\x))+200*cos((\x r)*3.1415)});
\end{axis}
\end{tikzpicture}
\caption[Graph]{Monthly Income  - Graphically\label{fig:graphically}}
\end{figure}

\subsubsection{Algebraically}
Typically, graphs come from humans or machines plotting many,
many points derived from an \gls{algebraic} formula.
Algebra is a powerful tool for describing natural phenomena,
but it cannot do everything.  Important numbers
--- such as \gls{pi} or $e$ --- cannot be defined via algebraic techniques.
Worse, many confuse manipulation of the symbols with actual mathematics.
However, the symbols are very concise, as you might see in Tab.~\ref{tab:algebra}.

\columntable[-0.75in]{0.3in}{
\begin{center}
$y=1000\cdot1.04^x+200\cos(\pi x)$.
\end{center}}
{Monthly Income - Algebraically\label{tab:algebra}}

\subsubsection{Verbally}
Ordinary language serves us well most of the time, with an average level of
precision and an average level of
emotional content.  Sometimes, however, what we wish to convey is more
meaningful and deep than
regular wording can describe.  Conversely, we might wish to exclude a whole
host of meanings, and narrowly
zoom in on precise terminology, in order to avoid error to the fullest extent possible
(see Fig.~\ref{fig:lol}).

\marginfig[-0in]{\chapdir/pics/levelsoflanguage}{Levels of language\label{fig:lol}}

Most of us encounter poetic language\index{language} in the form of songs, since ours is an
age sadly bereft of
poetry, something that was not true a century ago or more.  Another common
phenomenon is to meet
technical language when it is not expected or welcome, such as when we read the
instruction manual for new
hardware or legal documents.  However, such documents are typically not aiming at
easy comprehension, but
precision.  Like most jargon, they tend to use mostly ordinary words in extraordinary
ways, similar but opposite
to poetry.

Mathematics is no different.  This textbook is a formal setting, and as such,
it uses technical language, carefully
choosing each word according the conventions and definitions ``in house''.
As you cross the threshold into the
beginning stages of advanced mathematics, you must cultivate the skill of
using this ``jargon'', something you
do not ordinarily do.  It may help your comprehension to paraphrase
what you are learning into the ``vernacular,''
but you should not consider a topic concluded until you are able to
re-articulate the mater in mathematical terminology.
For example, the equation above might be described as in Fig.~\ref{fig:verbally}.

\begin{figure}
\emph{The income ($y$) may be modeled by the number of months since
inception ($x$) as a sinusoid with an amplitude of 200, a period of 2, and a
midline of an exponential growth rate of 4\% beginning at \$1,000.}
\caption{Monthly Income - Verbally\label{fig:verbally}}
\end{figure}

\subsubsection{Mathematical Models}
Functions that can be used to make predictions and
interpretations of real-world phenomena are called \textbf{mathematical
models}.
It is important to be clear which variable is being used as input
and which is output, so that a meaningful
relationship between independent and dependent variables
can be ascertained.  In the model of income above,
time does not depend upon money, but rather money upon time.
The domain of the function is the initial time
index of zero until four years later, so in months $0\le x \le 48$.
The income varies from as low as \$840 but
generally goes up from there, so the range is $y\ge840$.

\begin{example}
	\exProblem
There are only so many hours in a day.  If you decide to stay up \emph{all} night studying,
you will surely get
a bad grade on the test tomorrow, due to sleep deprivation.
Sketch a reasonable graph showing hours spent sleep the day before
a test and the grade you will get on said test.  Give the domain and range of the function.

	\exSolution
One cannot possibly get less than zero hours of sleep, and presumedly around
12 or so hours of sleep is
long enough to have slept through the exam and angered one's parents!
Test scores are typically measured in percents, ranging from 0 to 100.

\marginfig[-1.8in]{\chapdir/pics/sleep.png}{Possible graph of sleep vs. grade}
\end{example}

~\vfill


\chapter[ICA on the Sphere]{Independent Component Analysis on the Sphere}
\label{ch_mrs_ica}

\section{Introduction}
\index{blind source separation}
\index{ICA}

Blind Source Separation (BSS) is a problem that occurs in multi-dimensional data processing. The overall goal is to recover
unobserved signals, images or \emph{sources} $S$ from mixtures of these sources $X$ observed typically at the output of an
array of sensors. The simplest mixture model takes the form:
\begin{equation}\label{model0}
X = A S
\end{equation}
where $X$ and $S$ are random vectors of respective sizes $m \times 1$, $n \times 1$ and $A$ is an $m \times n$ matrix. The
entries of $S$ are assumed to be independent random variables. Multiplying $S$ by $A$ linearly mixes the $n$ sources into
$m$ observed processes. \\

Independent Component Analysis methods were developed to solve the BSS problem, \emph{i.e.} given a batch of $T$ observed
samples of $X$, estimate the mixing matrix $A$ and reconstruct the corresponding $T$ samples of the source vector $S$, relying
mostly on the statistical independence of the source processes. Note that with the above model, the independent sources can
only be recovered up to a multiplication by a \emph{non-mixing} matrix \emph{i.e.} up to a permutation and a scaling of the
entries of $S$. Although independence is a strong assumption, it is in many cases physically plausible. The point is that
it goes beyond the simple second order decorrelation obtained for instance using Principal Component Analysis (PCA) : decorrelation
is not enough to recover the source processes since any rotation of a white random vector remains a white random vector.\\

Algorithms for blind component separation and mixing matrix estimation depend on the model used for the probability distribution
of the sources~\cite{ica:3easy}. In a first set of techniques, source separation is achieved in a noise-less setting, based on the
non-Gaussianity of all but possibly one of the components. Most mainstream ICA techniques belong to this category : JADE~\cite{ica:jade},
FastICA, Infomax~\cite{ica:icabook}. In a second set of blind techniques, the components are modeled as Gaussian processes, either
stationary or non stationary and, in a given representation, separation requires that the sources have diverse, \emph{i.e.} non
proportional, variance profiles. The Spectral Matching ICA method (SMICA) ~\cite{ica:Del2003}, considers in this sense the case of
mixed stationary Gaussian components and goes further than the above model (Eq.~\ref{model0}) by taking into account additive
\emph{instrumental } noise $N$:
\begin{equation}\label{model1}
X = A S + N
\end{equation}
Moving to a Fourier representation, the idea is that colored components can be separated based on the diversity of their power spectra.\\

The next two sections give a short overview of two significant ICA methods mentioned above and implemented in the \mrs package:
JADE and SMICA. A shorter description of FastICA ~\cite{ica:icabook} is also given. This is followed by a description of ways to
combine wavelets and ICA techniques. Some useful properties of wavelet transforms can indeed come enhance the performance of ICA
methods in several situations.

\section{JADE}
\index{ICA!jade}

The Joint Approximate Diagonalization of Eigenmatrices method (JADE) assumes the observed data $X$ follows the noiseless mixture
model~(\ref{model0}) where the independent sources $S$ are non-Gaussian \emph{i.i.d.}\footnote{The letters \emph{i.i.d.} stand for
independently and identically distributed meaning that each entries of $X$ at a given time $t$ are independent of $X$ at any other
time $t'$ and that the distribution of $X$ does not depend on time. } random processes. The mixing matrix is assumed to be square
and invertible so that (de)mixing is actually just a change of basis.

As mentioned above, second order statistics do not retain enough information for source separation in this context: finding a change of
basis in which the data covariance matrix is diagonal will not in general enable to identify the independent sources properly. Nevertheless,
decorrelation is \emph{half the job}~\cite{ica:tutorial} and one may seek the basis in which the data is represented by maximally independent
processes among those bases in which the data is decorrelated. This leads to so-called orthogonal algorithms: after a proper whitening of
the data by multiplication with the inverse of a square root of the covariance matrix of the data $W$, one is then seeking a rotation $R$
(which leaves things white) so that $\hat{ S}$ defined by
\begin{equation}
\hat{ S} = W^{-1} \, Y =  W^{-1}\, R \, X_{\textrm{white}}  = W^{-1}\, R \, W \, X
\end{equation}
and $\hat{B} = \widehat{A^{-1}} =  W^{-1}\, R \, W$ are estimations of the sources and of the inverse of the mixing matrix.\\

JADE is such an orthogonal ICA method and, like most mainstream ICA techniques, it exploits higher order statistics so as to achieve some
sort of \emph{ non linear decorrelation}. Precisely, in the case of JADE, statistical independence is assessed using fourth order cross cumulants :
\begin{eqnarray}  \nonumber
F_{ijkl} & = & \textrm{cum}( y_i, y_j, y_k, y_l )   \nonumber    \\
  & =& \mathcal{E} (y_i y_j y_k y_l) - \mathcal{E} (y_i y_j)\mathcal{E} (y_k y_l)\nonumber \\
  & & -\mathcal{E} (y_iy_l)\mathcal{E} ( y_j y_k)-\mathcal{E} (y_iy_k)\mathcal{E} (y_j y_k)
\end{eqnarray}
where $\mathcal{E}$ stands for statistical expectation and the $y_i$'s are the entries of vector $Y$ modeled as random variables,
and the correct change of basis (\emph{i. e.} rotation) is found by somehow \emph{diagonalizing} the fourth order cumulant tensor.
Indeed, if the $y_i$'s were independent, all the cumulants with at least two different indices would be zero. As a consequence of
the independence assumption of the source processes $S$ and of the \emph{whiteness} of $Y$ for all rotations $R$, the fourth order
tensor $F$ is well structured: JADE was precisely devised to take advantage of the algebraic properties of $F$. JADE's objective
function is given by
\begin{eqnarray}  \nonumber
%\mathcal{J}_{\textrm{jade}}( R )   &=& \sum _{ijkl \ne ijkk}  \textrm{cum}(  y_i, y_j, y_k, y_l )^2  \nonumber    \\
  \mathcal{J}_{\textrm{jade}}( R ) & =&  \sum _{ij}   \sum_{k \ne l} \textrm{cum}(  y_i, y_j, y_k, y_l )^2
\end{eqnarray}
which can be interpreted as a joint diagonalization criterion. Fast and robust algorithms are available for the minimization
of $\mathcal{J}_{\textrm{jade}}( R )$ with respect to $R$ based on Jacobi's method for matrix diagonalization~\cite{ica:pham2001}.
More details on JADE can be found in~\cite{ica:jade,ica:tutorial,ica:icabook}.

\subsubsection{JADE for spherical maps}

Applying JADE on multichannel data mapped to the sphere does not require any particular modification of the algorithm. Indeed, JADE estimates
the fourth order cumulant tensor from the available data samples assuming an \emph{i.i.d.} random field. Hence, given a pixelization scheme on
the sphere such as provided by the Healpix package, JADE can be directly applied to the multichannel spherical data pixels.

\section{FastICA}
\index{ICA!fastica}

FastICA is by now a standard technique in ICA. Like JADE, it is meant for the analysis of mixtures of independent non-Gaussian sources in
a noise-less setting. A complete description of this method can be found in \cite{ica:icabook} and references therein. Many papers on this
algorithm are available at \emph{http://www.cs.helsinki.fi/ u/ahyvarin/papers/fastica.shtml}. We give here a brief and simplified account
of the algorithm. FastICA, again like JADE, is a so-called orthogonal ICA method: the independent components are sought by maximizing a
measure of non-Gaussianity under the constraint that they are decorrelated. Intuitively, one should understand that mixtures of independent
non-Gaussian random variables tend to \emph{ look more Gaussian}. An enlightening view on the relation between mutual information, which is
a natural measure of independence, decorrelation and non-Gaussianity can be found in~\cite{ica:3easy,ica:geomindep}. Non-Gaussianity is assessed
in FastICA using a contrast function $G$ based on a non-linear approximation to \emph{negentropy}~\cite{ica:icabook}. In practice, depending
on the application, different approximations or non-linear (non-quadratic) functions should be experimented with. In a simple deflation scheme,
for sphered data, the directions are found sequentially : a direction $r$ of maximal non-Gaussianity is sought by maximizing
\begin{equation}
J_G(r) = \Big( \mathcal{E} \{ G(r^T x_{\textrm{white}}  ) \} - \mathcal{E} \{ G(\nu ) \} \Big)^2
\end{equation}
where $\nu$ stands for centered unit variance Gaussian variable, under the constraint that $r$ has unit norm and that $r$ is orthogonal
to the directions found previously. The contrast function $G$ can for instance be chosen among the following~\cite{ica:icabook}:
\begin{eqnarray}
G_0 (u)  & = &  \frac{1}{a} \textrm{log}\,\textrm{cosh} (a u ) \nonumber    \\
G_1 (u)  & = &  -\frac{1}{a} \textrm{exp}(- a u^2 / 2 )   \nonumber	   \\
G_2 (u)  & = &  \frac{1}{4} u^4 \nonumber    \\
\end{eqnarray}
where $a$ is a constant to be determined depending on the application. It can be shown that the maxima of $J_G$ occur at certain maxima
of $\mathcal{E} \{ G(r^T x_{\textrm{white}} ) \} $. These are obtained for $r$ solution to :
\begin{equation}
\mathcal{E} \{ x_{\textrm{white}} g(r^T x_{\textrm{white}}  ) \} - \lambda r = 0
\end{equation}
where $\lambda$ is a constant easily expressed in terms of the optimal direction $r_0$, and $g$ is the derivative of $G$. Solving this
equation using Newton's method, and a few approximations, a \emph{fixed-point} algorithm is derived which consists in repeating the
following two steps until convergence :
\begin{eqnarray}
r  & \leftarrow & \mathcal{E} \{ x_{\textrm{white}} g(r^T x_{\textrm{white}}  ) \} - \mathcal{E} \{ g'(r^T x_{\textrm{white}}  ) \} r     \nonumber  \\
r  & \leftarrow  &  \frac{r}{\| r \|}   \nonumber	   \\
\end{eqnarray}
A simple implementation of this algorithm is included in the present package. It is largely based on the $\textbf{Matlab}^{TM}$ code
available at \emph{www.cis.hut.fi/projects/ica/fastica/}.

%\section{SMICA}
%\index{smica}
%\index{ICA!smica}
%
%Spectral Matching ICA (SMICA) was designed to address some of the general problems raised by Cosmic Microwave Background data analysis
%where the major component of interest (CMB itself) is well modeled by an isotropic stationary Gaussian random field. Although standard
%ICA methods may be used in this context, they are not expected to perform as well as methods based on Gaussian model especially in the
%presence of additive Gaussian instrumental noise as in (\ref{model1}).
%
%SMICA belongs to a set of blind source separation techniques where the components are modeled in a given representation as \emph{locally i.i.d.}
%centered Gaussian processes. Independent Gaussian sources can then be separated based on their statistical independence (which obviously reduces
%\emph{locally} to decorrelation) provided they have diverse (\emph{i.e.} non proportional) variance profiles \emph{i.e.} energy distributions in
%that representation. SMICA considers in this sense the case of mixed stationary Gaussian components in a noisy context as in model~(\ref{model1}) :
%moving to a Fourier representation, colored components can be separated based on the diversity of their power spectra.
%
%\subsection{SMICA's objective function}
%
%In order to derive the Spectral Matching ICA criterion, we assume that, in the Fourier domain, the \emph{locally i.i.d.} Gaussian sources $S$
%and noise $N$ processes in (\ref{model1}) actually have constant spectral covariance matrices $R_f^S(q) \in \mathbb{R}^{n\times n}$ and $R_f^N(q)
%\in \mathbb{R}^{m \times m} $ on each of a set of $Q$ frequency bands. Clearly, the appropriate notion of frequency should be used, depending on
%whether the data $X$ is a set of time series, a set of 2D maps, etc. The assumption of statistical independence between the components $S$ implies
%that the $R_f^S(q)$ are diagonal matrices. With a similar assumption regarding  the noise processes in the $m$ different channels, the $R_f^N(q)$
%also are diagonal matrices. Applying a Fourier transform on (\ref{model1}) does not affect the mixing matrix $A$, so that the model covariance
%matrix of the observations $X$ in the $q^{\textrm{th}}$ frequency band is structured as
%\begin{equation}
%\label{structure}
%R_f^X(q) = A R_f^S(q) A^{\dagger} +   R_f^N(q)
%\end{equation}
%where we assumed that the instrumental noise is independent of the sources. Then, provided estimates $\widehat{R}_f^X(q) $ of $R_f^X(q) $ can be
%obtained from the available data (\emph{e.g.} empirical covariance estimator), SMICA consists in minimizing
%\begin{equation}\label{Cost_fourier}
% \Phi_f (\theta) =  \sum _{q=1}^{Q}  \alpha_q \mathcal{D} \left( \widehat{R}_f^X(q) \ , A R_f^S(q) A^{\dagger} +   R_f^N(q) \right)
%\end{equation}
%for some sensible choice of the weights $\alpha_q$ and of the matrix mismatch measure $\mathcal{D}$, with respect to the full set of parameters
%$\theta = (A,R_f^S(q), R_f^N(q) )$ or a subset thereof. As discussed in ~\cite{ica:Del2003}, a good choice for $\mathcal{D}$ is
%\begin{equation}\label{eq:kl}
%  \mathcal{D}_{KL} (R_1, R_2 )  =  \frac{1}{2} \Big( \mathrm{tr} (R_1R_2^{-1}) - \log\det (R_1R_2^{-1}) - m  \Big)
%\end{equation}
%which is the Kullback-Leibler divergence between two $m$-variate zero-mean Gaussian distributions with covariance matrices $R_1$ and $R_2$.
%With this mismatch measure, the SMICA criterion is shown to be related to the likelihood of the data in a Gaussian model, so that we can resort
%to the EM algorithm to minimize~(\ref{Cost_fourier}). The weights $\alpha_q$ should be chosen to reflect the variability of the estimate of the
%corresponding covariance matrix. Following the derivation in~\cite{ica:Del2003}, these are taken to be the number of Fourier modes in each band $q$. \\
%
%\subsection{Source map estimation}\label{sect:mapesti}
%
%As a result of applying SMICA, power densities in each frequency band are estimated for the sources and detector noise along with
%the estimated mixing matrix. These may be used in reconstructing the source maps \emph{via} for instance Wiener filtering in each band:
%a Fourier mode $X(\nu)$ in frequency band $q$ is used to reconstruct the maps according to
%\begin{equation}
%  \widehat{S}(\nu)
%  =
%  (\widehat{A}\adj \widehat{R}_f^N(q)^{-1} \widehat{A} + \widehat{R}_f^S(q)^{-1})\inv
%  \widehat{A}\adj \widehat{R}_f^N(q)^{-1} X(\nu)
%  \label{Wiener}
%\end{equation}
%In the limiting case where noise is small compared to signal components, this filter reduces to
%\begin{equation}
% \widehat{S}(\nu)  =
%  (\widehat{A}\adj \widehat{R}_f^N(q)^{-1} \widehat{A} )\inv
%  \widehat{A}\adj \widehat{R}_f^N(q)^{-1} X(\nu)
%  \label{Wiener1}
%\end{equation}
%Clearly, the above Wiener filter is optimal only in front of stationary Gaussian processes. For non Gaussian maps, such as given by
%the Sunyaev Zel'dovich effect, better reconstruction can be expected from non linear methods.
%
%\subsection{SMICA for spherical maps}\label{sect:smicas}
%
%In the linear mixture model~(\ref{model1}), $X$ now stands for an array of observed spherical maps, $S$ is now an array of spherical source maps
%to be recovered and $N$ is an array of spherical noise maps. The mixing matrix $A$ achieves a pixelwise linear mixing of the source maps, in the
%Healpix scheme for instance.
%
%Extending SMICA to deal with multichannel data mapped to the sphere is straightforward~\cite{ica:Del2003}. The idea is simply to substitute the
%spherical harmonics transform to the Fourier transform used in the above description of SMICA. Then, data covariance matrices are estimated in
%this representation over $Q$ intervals in multipole number $l$, assuming the components are stationary and isotropic over the sphere. These $Q$
%covariance matrices are still structured according to~(\ref{structure}) and source separation can be achieved by minimizing the spectral matching
%criterion~(\ref{Cost_fourier}). Source maps reconstruction follows as in section~\ref{sect:mapesti}.\\
%
%SMICA has already been applied in astrophysical data analysis, showing significant success for CMB spectral estimation in multidetector
%experiments~\cite{ica:Del2003,ica:patanchon}. Working in the frequency domain does offer several benefits such as easy handling of detector
%dependent point spread functions. However, the non locality of the Fourier or the spherical harmonics transform will have some undesired
%effects when dealing with non-stationary components or noise, or with incomplete data maps. The latter is a common issue in astrophysical
%data analysis : either the instrument scanned only a fraction of the sky or some regions of the sky were masked due to localized strong
%astrophysical sources of contamination (compact radio-sources or galaxies, strong emitting regions in the galactic plane). A simple way
%to overcome these effects is to move instead to a wavelet representation so as to benefit from the localization property of wavelet filters.
%This leads to WSMICA~\cite{starck:yassir05}, an extension of SMICA which is reviewed in section~\ref{sect:wsmica} below.

\section{ICA and Wavelets}
\index{wavelet!ICA}
\index{ICA!wavelet}

Several properties of wavelets have been recognized as particularly useful in multichannel data processing : bringing wavelets and
independent component analysis together has proven quite profitable. Extensions WJADE and WSMICA of the two ICA methods described
previously are discussed in this section.

Wavelets are remarkable at data compression meaning data that is structured in the initial representation requires fewer significant
coefficients in a wavelet representation. In imprecise and general terms, wavelets grab the coherence between coefficients of the
structured data and produces a smaller set of significant coefficients which are then less coherent and which have a sparser statistical
distribution. Then, the super-Gaussian\footnote{A super-Gaussian distribution is also called a lepto-kurtic distribution, referring to a
distribution with a narrow central peak and heavy tails. A typical example is the Laplacian distribution.} \emph{i.i.d.} statistical model
which appears in most standard ICA methods may better suit the wavelet coefficients of the data than the data samples in the initial representation.

Wavelets have been developed for the analysis of non-stationary and singular data in order to overcome certain difficulties attached to
the Fourier transform. Wavelets are widely used to reveal variations in the spectral content of time series or images as they permit to
single out regions in direct space while retaining localization in the frequency domain. Astrophysical data analysis has much to gain in
avoiding the assumption of stationarity underlying Fourier analysis. Moreover, observed data maps are commonly imperfectly shaped and
incomplete with missing or masked patches due to experimental settings, scanning strategies, etc. This will impair direct application of
the Spectral Matching ICA method described previously. One might consider resorting to wavelets.

%In dealing with non stationary data or incomplete data, an attractive feature of  wavelet filters over the spherical harmonic transform is that they are well localized in the initial representation.

%Dealing with non stationary data or incomplete data is made simpler due to the good localization of wavelet coefficients in the initial representation.In the smaller scales, most of the samples in the filtered signal will  be unaffected by the presence of gaps and only these samples should be used to estimate the data covariance matrices on each wavelet scale.

\subsection{WJADE}\label{sec:wjade}
\index{ICA!wjade}
\index{jade!wavelet}

Wavelets come into play as a sparsifying transform. Applying a wavelet transform on both sides of~(\ref{model0}) does not affect the
mixing matrix and the model structure is preserved. Also, moving the data to a wavelet representation does not affect its information
content. However, the statistical distribution of the data coefficients in the new representation is different: wavelets are known to
lead to sparse \emph{i.i.d.} representations of structured data. Further, the \emph{local} (coefficient wise) signal to noise ratio
depends on the choice of a representation. A wavelet transform tends to grab the informative coherence between pixels while averaging
the noise contributions, thus enhancing structures in the data. Although the standard ICA model~(\ref{model0}) is for a noiseless setting,
the derived methods can be applied to real data. Performance will depend on the detectability of significant coefficients \emph{i.e.} on
the sparsity of the statistical distribution of the coefficients. Moving to a wavelet representation will often lead to more robustness to noise.

Once the data has been transformed to a proper representation (\emph{e.g.} wavelets but also ridgelets and curvelets in the case of strongly
anisotropic 2D or 3D data), WJADE consists in applying the standard JADE method to the new multichannel coefficients. Once the mixing matrix
is estimated, the initial source maps are obtained using the adequate inverse transform after some non linear denoising or thresholding of
the coefficients if necessary.

\subsection{Covariance matching in wavelet space : WSMICA}\label{sect:wsmica}
\index{ICA!wsmica}
\index{smica!wavelet}
\index{wavelet!transform}

Let us consider the case of spherical maps as in \ref{sect:smicas} but possibly incomplete, partly masked or non-stationary. As a model case,
we actually consider incomplete data maps in which the positions of the missing pixels are known in advance. Moving to a wavelet representation,
it is possible to keep track of the missing pixels on each scale so that we can derive a covariance matching ICA criterion, WSMICA, robust to
gaps in the data. Indeed, an attractive feature of wavelet filters over the spherical harmonic transform is that they are well localized in the
initial representation. Provided the wavelet filter response on scale $j$ is short enough compared to data size and gap widths, most of the
samples in the filtered signal will then be unaffected by the presence of gaps. Using exclusively these samples yields an estimated covariance
matrix $\widehat{R}_w^X(j)$ which is not biased by the missing data. The price to pay is a possibly slight increase in variance which depends on scale $j$.

With the Isotropic Undecimated Wavelet Transform on the Sphere (UWTS) decribed in section~\ref{sect_wts}, the multichannel data $X$ is decomposed
into $J$ detail maps $X_{j}^w$ and a smooth approximation map $X_{J+1}^w$ over a dyadic resolution scale which simply sum back as:
\begin{equation}
X(\vartheta, \varphi) = X_{J+1}^w (\vartheta, \varphi) + \sum_{j=1}^{J} X_{j}^w(\vartheta, \varphi)
\end{equation}
Denoting $l_j$ the size of the set $\mathcal{M}_j$ of wavelet coefficients unaffected by the gaps at scale $j$, the wavelet covariances are empirically estimated using
\begin{equation}
\widehat{R}_w^X (j) = \frac{1}{l_j }  \sum_{t \in  \mathcal{M}_j } X_j^w( \vartheta_t, \varphi_t )X_j^w( \vartheta_t, \varphi_t ) ^\dagger
\end{equation}

Clearly, applying the above UWTS on both sides of~(\ref{model1}) does not affect the mixing matrix $A$ so that
the model covariance matrix of the observations at scale $j$, is still structured as
\begin{equation}
R_w^X(j) = A R_w^S(j) A^{\dagger} +   R_w^N(j)
\end{equation}
where $R_w^S(j)$ and $R_w^N(j)$ are the model diagonal spectral covariance matrices in the wavelet representation of $S$ and $N$ respectively
at scale $j$. Given an estimation of $R_w^X(j)$ from the data, $\widehat{R}_w^X(j)$, source separation follows from  minimizing the following
covariance matching criterion WSMICA-S in this \emph{spherical wavelets} representation:
\begin{equation}\label{Cost_wavelet}
 \Phi (\theta) =  \sum _{j=1}^{J+1}  \alpha_j \mathcal{D} \left( \widehat{R}_w^X(j), \,
    A R_w^S(j) A^{\dagger} + R_w^N(j) \right)
\end{equation}
with respect to the full set of parameters $\theta = (A,R_w^S(j), R_w^N(j) )$ or a subset thereof. Again, a good choice for $\mathcal{D}$ is
the Kullback-Leibler divergence given in equation~\ref{eq:kl}. With this mismatch measure, we can again resort to the EM algorithm to minimize~(\ref{Cost_wavelet}).

The weights in the covariance mismatch~(\ref{Cost_wavelet}) should be chosen to reflect the variability of the estimate of the corresponding
covariance matrix. Since WSMICA-S uses wavelet filters with only limited overlap, in the case of complete data maps we follow the derivation
in~\cite{ica:Del2003} and take $\alpha_j$ to be proportional to the number of spherical harmonic modes in the spectral domain covered at
scale $j$. In the case of data with gaps, we must further take into account that only a fraction $\beta_j$ of the wavelet coefficients are
unaffected so that the $\alpha_j$ should be modified in the same ratio.

Source maps may be reconstructed outside the possible gaps by Wiener filtering on each scale prior to inverting the wavelet transform, following
the procedure described in section~\ref{sect:mapesti}. Non linear filtering techniques may yield better results in the case of non-Gaussian components.

\subsubsection{WSMICA for \emph{flat} maps}

WSMICA can be easily implemented in the case of flat 2D data maps by substituting the 2D isotropic undecimated \emph{\`a trous} algorithm with
the cubic box-spline~\cite{starck:book02} as scaling function to the UWTS used in the previous section to deal with incomplete spherical maps.
This transform has several favorable properties for astrophysical data analysis. In particular, it is a shift invariant transform, the wavelet
coefficient maps on each scale are the same size as the initial image, and the wavelet and scaling functions have small compact supports in the
initial representation. As in the case of spherical maps, these properties allow  missing patches in the data maps to be handled easily.

WSMICA was used in~\cite{starck:yassir05} to process realistic CMB multichannel data as expected from the Planck experiment however on small enough
maps so that curvature could be neglected. The reported numerical experiments clearly confirm the benefits of correctly processing existing gaps.
Wavelets are able to correctly grab the spectral content of partly masked data maps and from there allow for better component separation.

\section{Applications}

\subsection{CMB data analysis}\label{sect:NUMEXP}
\index{CMB}
\index{CMB!ICA}
\index{ICA!CMB}

As an application of WSMICA on the sphere, we consider here the problem of CMB data analysis but in the special case where the use of a galactic
mask is a cause of non-stationarity which impairs the use of the spherical harmonics transform.

The simulated CMB, galactic dust and Sunyaev Zel'dovich (SZ) maps used, shown on the left-hand side of figure~\ref{components}, were obtained as
described in~\cite{ica:Del2003}. The problem of instrumental point spread functions is not addressed here, and all maps are assumed to have the
same resolution.The high level foreground emissions from the galactic plane region were discarded using the $Kp2$ mask from the WMAP team
website\footnote{\emph{http://lambda.gsfc.nasa.gov/product/map/intensity\_mask.cfm}}. These three \emph{incomplete} maps were mixed using the matrix
in table~\ref{MatrixA}, in order to simulate observations in the six channels of the Planck high frequency instrument~(HFI).

\begin{table}[!h]
  \begin{center}
   \footnotesize {
    \begin{tabular}{@{} cccc|c @{}}
      CMB & DUST & SZ &  & channel \\
      & & & &\\
      \hline
      & & & &\\
      $\quad1.0\quad$  &  $ 1.0 $    		&  $\quad-1.51\quad$	 	&& 100~GHz\\
      $\quad1.0 \quad$ & $ 2.20$   		& $\quad-1.05\quad$  		&& 143~GHz\\
      $\quad1.0 \quad$ &  $ 7.16 $  		& $\quad0.0\quad $ 			&& 217~GHz\\
      $\quad1.0 \quad$ &   $56.96 $		&  $\quad 2.22\quad$ 	&&353~GHz\\
      $\quad1.0 \quad$ &  $1.1\times10^{3}$ 		&   $\quad5.56\quad$  	&&545~GHz\\
      $\quad1.0\quad $ &  $1.47\times10^{5}$  	& $ \quad11.03\quad $	&& 857~GHz\\
    \end{tabular}}
    \caption{Entries of $A$, the mixing matrix used in our simulations.}\label{MatrixA}
  \end{center}
\end{table}

Gaussian \emph{instrumental} noise was added in each channel according to model~(\ref{model0}). The relative noise standard deviations between channels
were set according to the nominal values of the Planck HFI given in table~\ref{NoiseScale}.

\begin{table}[!h]
  \begin{center}
     \footnotesize {
    \begin{tabular}{@{} cccccc|c @{}}
      100&143&217&353&545&857&channel \\
          & & & & & &\\
  \hline
          & & & & & &\\
    $ 2.65\!\!\times\!\!10^{-6}$&$2.33\!\!\times\!\!10^{-6}$&$3.44\!\!\times\!\!10^{-6}$&$1.05\!\!\times\!\!10^{-5}$&$1.07\!\!\times\!\!10^{-4}$& $4.84\!\!\times\!\!10^{-3}$ &noise std\\
          & & & & & &\\
  % \hline
   %    & & & \\
   %   353~GHz&545~GHz  &857~GHz  &   channel\\
       %   & & &  \\
  %\hline
  %    & & & \\
   %$1.05\times10^{-5}$&$1.07\times10^{-4}$ & $4.84\times10^{-3}$ &   noise std\\
     % & & &  \\
      \end{tabular}}
    \caption{Nominal noise standard deviations in the six channels of the Planck HFI.}\label{NoiseScale}
  \end{center}
\end{table}

The synthetic observations were decomposed into six scales using the isotropic UWTS and WSMICA was used to obtain estimates of the mixing matrix
and of the initial source templates. The resulting component maps estimated using WSMICA, for nominal noise levels, are shown on the right-hand
side of figure~\ref{components} where the quality of reconstruction can be visually assessed by comparison to the initial components. The component
separation was also performed with SMICA based on Fourier statistics computed in the same six dyadic bands imposed by our choice of wavelet transform,
and with JADE. In figure~\ref{resultats}, the performances of SMICA, WSMICA and JADE, are compared in the particular case of CMB map estimation,
in terms of the relative standard deviation of the reconstruction error, $MQE$, defined by
 \begin{equation}
  MQE  = \frac{\mathbf{std}  ( CMB(\vartheta, \varphi)  - \alpha \times \widehat{CMB}(\vartheta, \varphi)  )}{\mathbf{std}  ( CMB(\vartheta, \varphi)   ) }
  \label{MQE}
\end{equation}
\index{jade}
\index{smica}
\index{smica!wavelet}
\index{jade!wavelet}
where $\mathbf{std}$ stands for empirical standard deviation  obviously computed outside the masked regions), and $\alpha$ is a linear regression
coefficient estimated in the least squares sense. As expected, since it is meant to be used in a noiseless setting, JADE performs well when noise
is very low. However, as the noise level increases, its performance degrades quite rapidly compared to the covariance matching methods. Further,
these results clearly show that using wavelet-based covariance matrices provides a simple and efficient way to cancel the bad impact that gaps
have on the performance of source separation using statistics based on the non local Fourier representation.

\begin{figure*}[htb]
% \begin{center}
% \includegraphics[width=7cm]{CMB_map}
% \includegraphics[width= 7cm]{CMB_w0db}
% \includegraphics[width= 7cm]{hist_equal_dust}
% \includegraphics[width= 7cm]{hist_equal_dust_w_0db}
% \includegraphics[width= 7cm]{sz_map3}
% \includegraphics[width= 7cm]{sz_w_0db_bis}
% \caption{%%
% The maps on the left are the templates for CMB, galactic dust and SZ used in the experiment
  % described in section~\ref{sec:NUMEXP}. The maps on the right were estimated using wSMICA
  %  and scalewise Wiener filtering. (The different maps are drawn here in different color scales
   % in order to enhance structures and ease visual comparisons).  }
% \label{components}
% \end{center}
\vbox{
\centerline{
\hbox{
% \psfig{figure=CMB_w0db.ps,bbllx=0.5cm,bblly=7.5cm,bburx=21.5cm,bbury=20cm,width=7cm,clip=}
\psfig{figure=CMB_map.pdf,bbllx=0.5cm,bblly=7.5cm,bburx=21.5cm,bbury=20cm,width=7cm,clip=}
\psfig{figure=CMB_w0db.pdf,bbllx=0.5cm,bblly=7.5cm,bburx=21.5cm,bbury=20cm,width=7cm,clip=}
}}
\centerline{
\hbox{
\psfig{figure=hist_equal_dust.pdf,bbllx=0.5cm,bblly=7.5cm,bburx=21.5cm,bbury=20cm,width=7cm,clip=}
\psfig{figure=hist_equal_dust_w_0db.pdf,bbllx=0.5cm,bblly=7.5cm,bburx=21.5cm,bbury=20cm,width=7cm,clip=}
}}
\centerline{
\hbox{
\psfig{figure=sz_map3.pdf,bbllx=0.5cm,bblly=7.5cm,bburx=21.5cm,bbury=20cm,width=7cm,clip=}
\psfig{figure=sz_w_0db_bis.pdf,bbllx=0.5cm,bblly=7.5cm,bburx=21.5cm,bbury=20cm,width=7cm,clip=}
}}}
\caption{The maps on the left are zero mean templates for CMB ($\sigma =  4.17\times 10^{-5}$), galactic dust ($\sigma = 8.61\times 10^{-6}$)
and SZ ($\sigma = 3.32\times 10^{-6}$) used in the experiment described in section~\ref{sect:NUMEXP}. The standard deviations given are for
the region outside the galactic mask. The maps on the right were estimated using WSMICA and scalewise Wiener filtering as is explained in Appendix~2.
Obviously, map reconstruction using Wiener filtering is optimal only in front of stationary Gaussian processes. For non Gaussian maps, such as
given by the Sunyaev Zel'dovich effect, better reconstruction can be expected from non linear methods. The different maps are drawn here in
different color scales in order to enhance structures and ease visual comparisons. }
\label{components}
\end{figure*}

\begin{figure}
\begin{center}
\includegraphics[width=8cm]{resultats_aa}
\caption{Relative reconstruction error defined by~(\ref{MQE}) of the CMB component map using SMICA, WSMICA and JADE as a function of the instrumental noise level in dB relative to the nominal values in table~\ref{NoiseScale}.}
\label{resultats}
\end{center}
\end{figure}

\subsection{Sunyaev-Zeldovich cluster detection}

Another application of component separation techniques in astrophysics and cosmology is in the reconstruction of Sunyaev-Zel'dovich (SZ)
galaxy clusters in future SZ-survey experiments such as Olimpo, APEX, or Planck which will use multiband bolometer cameras. The goal is
to optimize SZ-Cluster extraction from the multichannel observed noisy maps. Resorting to blind methods is again attractive.

A complete description of the method we developed is given in~\cite{cluster:sz_cluster}. Before the actual detection of the SZ clusters,
the multichannel data maps are combined and filtered to produce a clean map of the SZ component, with greater signal to noise ratio. We used
an ICA approach to estimate the mixing matrix and perform the separation. A non linear filtering technique in the wavelet domain was used
for the purpose of denoising. Finally a detection algorithm extracts the SZ clusters candidates from the restored SZ map. This is a new
application of ICA to multichannel astrophysical data analysis.

In the 100 to 600~GHz range, the brightest components of the sky are the Cosmic Microwave Background (CMB), the Infrared Point Sources,
the Galactic dust emission and, swamped in the previous ones, the SZ clusters. It follows that the true sky map $X_{\nu}(\vartheta, \varphi)$,
in a given optic band centered on $\nu$, can be modeled as a sum of distinct astrophysical radiations as in
\begin{equation} \label{SkyBandModel1}
	X_{\nu}(\vartheta, \varphi)  =  CMB_{\nu}(\vartheta, \varphi)  + IR_{\nu}(\vartheta, \varphi)  + Gal_{\nu}(\vartheta, \varphi)  + SZ_{\nu}(\vartheta, \varphi)
\end{equation}
where $\vartheta, \varphi$ denote spatial or angular indexes on 2D or spherical maps. Assuming again that the radiative properties of the sources
are completely isotropic in the sense that they do not depend on the direction of observation, the above model can be rewritten in the following factored form:
\begin{equation} \label{SkyBandModel2}
X_{\nu}(\vartheta, \varphi) =  \sum_{i} a_{\nu, i} S_{i}(\vartheta, \varphi)  \,\,+\,\, N_{\nu}(\vartheta, \varphi)
\end{equation}
where $S_{i}$ is the spatial template and $a_{\nu, i}$  the emission law of the $i\,{\textrm{th}}$ astrophysical component. Although this is
a coarse approximation in the case of Infrared Point Sources, it is mostly valid for the other three components. With observations available
in $m$ channels, assuming the beam varies only slightly as a function of $\nu$, equation~(\ref{SkyBandModel2}) can be written in matrix form :
\begin{equation} \label{SkyBandModel3}
X(\vartheta, \varphi) = A\,\, S(\vartheta, \varphi)  \,\,+\,\, N(\vartheta, \varphi)
\end{equation}
where $X(\vartheta, \varphi)$ is a vector in $\mathbb{R}^m$, $A$ is an $m\times n$ matrix, n is the number of contributing astrophysical components,
$S(\vartheta, \varphi)$ is now a vector in  $\mathbb{R}^n$ and $N(\vartheta, \varphi)$ in $\mathbb{R}^m$. Equation~(\ref{SkyBandModel3}) expresses
that the observations consist of linear mixtures of astrophysical components with different weights and additive noise.

Simulations of the major contributions in the frequency range considered here were generated according to the procedure described
in~\cite{cluster:sz_cluster}. Figure~\ref{RawMaps} shows such typical simulations. These four physical components are linearly
combined into a ``true" sky maps which are then convolved  with the experimental beam. Then instrumental noise is added. The resulting
noisy mixture maps, as shown on figure~\ref{NoisyMaps} would be what the analysis team would recover from the data, after pointing
reconstruction, outlier removal, de-correlation of instrumental systematics in the data, and map-making.

\begin{figure*}[htp]
\vbox{
\centerline{
\hbox{
\psfig{figure=CMB.pdf,bbllx=3.cm,bblly=6.cm,bburx=19.1cm,bbury=21.7cm,height=7.5cm,width=6.5cm,clip=}
\hspace{0.2cm}
\psfig{figure=SZOld.pdf,bbllx=3.cm,bblly=6.cm,bburx=19.1cm,bbury=21.7cm,height=7.5cm,width=6.5cm,clip=}
}}
\vspace{0.3cm}
\centerline{
\hbox{
\psfig{figure=IR.pdf,bbllx=3.cm,bblly=6.cm,bburx=19.1cm,bbury=21.7cm,height=7.5cm,width=6.5cm,clip=}
\hspace{0.2cm}
\psfig{figure=TheGalax.pdf,bbllx=3.cm,bblly=6.cm,bburx=19.1cm,bbury=21.7cm,height=7.5cm,width=6.5cm,clip=}
}}
}
\caption{The 4 physical components of the sky included in our simulation:
{\bf a)} is a map of the CMB's anisotropies in unit of $\mu$K,
{\bf b)} is the SZ Cluster map, in unit of y Compton,
{\bf c)} is the IR point source map, convolved with a beam of 2 arcmin, in Jy at 350GHz, finally
{\bf d)} is the Galactic dust map in unit of MJy/st at 100 $\mu$m.}
\label{RawMaps}
\end{figure*}

%
%
%
\begin{figure*}[htp]
\vbox{
\centerline{
\hbox{
\psfig{figure=c143GHz.pdf,bbllx=3.cm,bblly=6.cm,bburx=19.1cm,bbury=21.7cm,height=7.0cm,width=6.0cm,clip=}
\hspace{0.2cm}
\psfig{figure=c217GHz.pdf,bbllx=3.cm,bblly=6.cm,bburx=19.1cm,bbury=21.7cm,height=7.0cm,width=6.0cm,clip=}
}}
\vspace{0.2cm}
\centerline{
\hbox{
\psfig{figure=c385GHz.pdf,bbllx=3.cm,bblly=6.cm,bburx=19.1cm,bbury=21.7cm,height=7.0cm,width=6.0cm,clip=}
\hspace{0.2cm}
\psfig{figure=c600GHz.pdf,bbllx=3.cm,bblly=6.cm,bburx=19.1cm,bbury=21.7cm,height=7.0cm,width=6.0cm,clip=}
}}
}
\caption{Simulated maps in Olimpo's four frequency bands. {\bf upper right} is the 147 GHz Band, {\bf upper left)} is the 217 GHz Band, {\bf lower right} is
the 385 GHz Band: CMB anisotropies, IR point sources and Galactic Dust blend in this band {\bf lower left} 500 GHz: IR point sources and Galactic Dust are
the dominant features at high frequencies. SZ cluster signal is dominated by other astrophysical sources at all frequencies.}
\label{NoisyMaps}
\end{figure*}
%
%
%

Any mainstream ICA algorithm for sparse sources, based on maximizing non-Gaussianity, would probably be successful at separating the SZ component map.
We chose to use JADE in the wavelet domain, as described in section~\ref{sec:wjade}. Wavelets come into play as a sparsifying transform : data is sparse
on a basis when this basis allows to describe that signal with a small number of coefficients. This is a highly desirable property, since noise is not
expected to be sparse at the same time on such a basis. Choosing a sparsifying basis thus allows to enhance signal to noise ratio. Moving the data to
a wavelet representation does not affect its information content and applying a wavelet transform on both sides of~(\ref{model0}) does not affect the
mixing matrix and the model structure is preserved. However, the statistical distribution of the data coefficients in the new representation is different:
wavelets are known to lead to sparse approximately \emph{i.i.d.} representations of structured data. Further, the \emph{local} (coefficient wise) signal
to noise ratio depends on the choice of a representation. A wavelet transform tends to grab the informative coherence between pixels while averaging
the noise contributions, thus enhancing structures in the data. Although the standard ICA model is for a noiseless setting, the derived methods can be
applied to real data. Performance will depend on the detectability of significant coefficients \emph{i.e.} on the sparsity of the statistical distribution
of the coefficients. Moving to a wavelet representation will then often lead to more robustness to noise. We noted that the estimation of the mixing matrix
could be slightly enhanced by prefiltering the data using a Gaussian with the same width as the optical beam. The resulting SZ component map is shown
on figure~\ref{JADEMaps}. Different filtering techniques were applied on this map and the results of a quantitative comparison are given in~\cite{cluster:sz_cluster}.

\begin{figure}[htp]
 \centerline{
\hbox{
%\psfig{figure=SZJadeG1_69w.ps}
\psfig{figure=SZJadeG1_69w.pdf,bbllx=3.cm,bblly=6.cm,bburx=19.1cm,bbury=21.8cm,height=6.5cm,width=6.5cm,clip=}
}
}
\caption{SZ component map extracted by JADE from the four observed noisy maps. The SZ cluster signal, subdominant at all observed frequencies,
now appears clearly. No obvious leftovers from other astrophysical sources are seen. \textbf{Remaining noise is small, because we prefiltered
data before JADE processing, and we simulated the nominal noise levels of an ambitious project: Olimpo.}}
\label{JADEMaps}
\end{figure}


\documentclass{article}

\usepackage{lipsum}
\usepackage[margin=1.5in]{geometry}
\usepackage{titlesec}
\usepackage{graphicx}
\usepackage{amsmath}

\usepackage{mathtools, amssymb, nccmath}
\usepackage{bigstrut, changepage, lipsum}
\usepackage{mathtools}

\newcommand{\code}{\texttt}
\newcommand{\norm}[1]{\left\lVert#1\right\rVert}

\usepackage{siunitx} % Required for alignment

% Specify images directory
\graphicspath{ {./report-images/} }

% Header and Footer stuff
\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhead{}
\fancyfoot{}
\fancyfoot[R]{ \thepage\ }
\renewcommand{\headrulewidth}{0pt}
\renewcommand{\footrulewidth}{0pt}
\newcommand{\sectionbreak}{\clearpage}
\setlength{\parindent}{0pt}

%

\begin{document}

%----------------------------------------------------------------------------------------
%	TITLE PAGE
%----------------------------------------------------------------------------------------

\begin{titlepage} % Suppresses displaying the page number on the title page and the subsequent page counts as page 1
	\newcommand{\HRule}{\rule{\linewidth}{0.5mm}}% Defines a new command for horizontal lines, change thickness here

	\center % Centre everything on the page

	%------------------------------------------------
	%	Headings
	%------------------------------------------------

	\textsc{\Large Basic orthogonal and periodic functions}\\[0.5cm] % Major heading such as course name

	\textsc{\large Exercise 3}\\[0.5cm] % Minor heading such as course title

	%------------------------------------------------
	%	Title
	%------------------------------------------------

	\HRule\\[0.6cm]

	{\huge\bfseries Least-squares Approximation of cos(2x) and cos(4x) Using Chebyshev Polynomials}\\[0.25cm] % Title of your document

	\HRule\\[1.5cm]

	%------------------------------------------------
	%	Author(s)
	%------------------------------------------------

	\begin{minipage}{0.4\textwidth}
		\begin{flushleft}
			\large
			\textit{Author}\\
			\textsc{Cesare De Cal} % Your name
		\end{flushleft}
	\end{minipage}
	~
	\begin{minipage}{0.4\textwidth}
		\begin{flushright}
			\large
			\textit{Professor}\\
			\textsc{Annie Cuyt}\\ % Supervisor's name
			[0.25cm]
			\textit{Assistant Professor}\\
			\textsc{Ferre Knaepkens} % Supervisor's name

		\end{flushright}
	\end{minipage}

	\vfill\vfill\vfill

	{\large\today}

	\vfill

\end{titlepage}

%----------------------------------------------- Introduction ------------------------------------------------------
\section{Introduction}\label{sec:intro}
This exercise asks to compute the Chebyshev approximation
$$t(x)=\sum^n_{j=0}{a_jT_j(x)}$$

for the functions $f(x)=\cos(2x)$ and $f(x)=cos(4x)$ over the interval $[-\pi,\pi]$ for $n = 6$ and then plot the original functions with the approximations and draw conclusions on the results.\\

As we've seen in class, Chebyshev polynomials are a set of orthogonal polynomials that can be used to approximate to a least squares fit. For this exercise, I'll first compute the roots of the polynomial (also called Chebyshev nodes), then calculate its coefficients for both the functions. Finally, I'll calculate the y values of the approximation to see how well it approximates the original function. During this process, I'll make sure to choose the correct data points and the correct number of data points to exploit the properties of these basis functions.

%---------------------------------- Tools ---------------------------------------------------------------------------
\section{Tools}
To solve this problem, I've used MATLAB as requested by the exercise. To make the computation more efficient, I wrote my own function which calculates the Chebyshev polynomial instead of using the built-in MATLAB function $\code{chebyshevT}$ which was noticeably slower. Writing my own function was just a matter of using the compact closed-form expression for the Chebyshev polynomials:

$$T_i(x)=\cos{(i\arccos(x))}$$

Built-in MATLAB functions have been used such as $\code{cos(x)}$ and $\code{acos(x)}$ to calculate the cosine and the inverse cosine function. $\code{linspace(start, end, nrOfPoints)}$ was used to create an array of equidistant points for plotting. The functions $\code{plot(x, y)}$, $\code{legend()}$, $\code{axis([xmin xmax ymin ymax])}$, $\code{ylabel(label)}$, and $\code{xlabel(label)}$ were also used for plotting.

\section{Computation}
Given $n=6$, I first calculate the nodes of the Chebyshev polynomial by using the formula:

$$x_j=\cos \Big( \frac{2j-1}{2n}\pi \Big), j=1,\dots,n+1$$

The resulting zeros are:

$$
\begin{bmatrix}
9.749279121818236e-01\\
7.818314824680298e-01\\
4.338837391175582e-01\\
6.123233995736766e-17\\
-4.338837391175581e-01\\
-7.818314824680295e-01\\
-9.749279121818237e-01\\
\end{bmatrix}
$$

Let's first analyze the function $\cos(2x)$. We've seen in class we can calculate the coefficients with the formula given by:

$$c_j=\frac{2}{N}\sum_{k=0}^{n}f(x_j)T_j(x_k)$$

To better exploit the properties of Chebyshev polynomials, I rescale the interval from $[-\pi,\pi]$ to $[-1,1]$. This is done in the function calculation $f(x_j)$ by multiplying the argument by $\pi$. I find the following coefficients:

$$
\begin{bmatrix}
4.407675118300712e-01\\
3.489272363107635e-16\\
5.739892333362830e-01\\
-2.537652627714643e-16\\
6.516380108719009e-01\\
-1.554312234475219e-15\\
-7.019674665493458e-01\\
\end{bmatrix}
$$

The polynomial for the first function $\cos{2x}$ looks like this:

\begin{equation}
  \begin{array}{l}
    t(x) = 4.407675118300712e-01 \times T_0(x)+ 3.489272363107635e-16 \times T_1(x) + \\
    5.739892333362830e-01 \times T_2(x) + \dots -7.019674665493458e-01 \times T_6(x)
  \end{array}
\end{equation}

To plot these coefficients in the graph, I need to calculate their associated y values:

$$
\begin{bmatrix}
9.876173836808355e-01\\
1.986723718905843e-01\\
-9.149466098687946e-01\\
9.999999999999993e-01\\
-9.149466098687926e-01\\
1.986723718905768e-01\\
9.876173836808371e-01\\
\end{bmatrix}
$$

Calculating the coefficients for the $\cos(4x)$ function now only requires minor changes in the code. The following are the coefficients of the polynomial:
$$
\begin{bmatrix}
6.879841220704104e-01\\
2.537652627714643e-16\\
-1.535576743556636e-01\\
-1.459150260935920e-15\\
1.012919200612446e+00\\
-1.094362695701940e-15\\
5.104689360033142e-01
\end{bmatrix}
$$

The polynomial for the second function $\cos{4x}$ looks like this:

\begin{equation}
  \begin{array}{l}
    t(x) = 6.879841220704104e-01 \times T_0(x) + 2.537652627714643e-16 \times T_1(x) \\
    -1.535576743556636e-01 \times T_2(x) + \dots + 5.104689360033142e-01 \times T_6(x)
  \end{array}
\end{equation}

The associated y values for the given roots are the following:

$$
\begin{bmatrix}
9.507761930971577e-01\\
-9.210585772947382e-01\\
6.742545978208001e-01\\
1.000000000000000e+00\\
6.742545978207990e-01\\
-9.210585772947411e-01\\
9.507761930971639e-01\\
\end{bmatrix}
$$

In order to properly calculate the associated y values, I had to subtract half of the first coefficient from the calculation. This is done because the formula I used to calculate the coefficients isn't correct for the first coefficient. The plots for both functions have been added to the next section.\\

\section{Plots}
Function $\cos(2x)$ and $\cos(4x)$ respectively:\\
\includegraphics[width=\textwidth,height=\textheight,keepaspectratio]{cos2x.jpg}\\
\includegraphics[width=\textwidth,height=\textheight,keepaspectratio]{cos4x.jpg}
\section{Observations}
I first calculated the zeros of the polynomial so I could use them as nodes in polynomial interpolation because the resulting interpolation polynomial minimizes the effect of Runge's phenomenon. The functions, however, still display a little bit of Runge's phenomenon. \\

The Chebyshev approximation for the function $\cos(2x)$ is pretty accurate, whereas the approximation for $\cos(4x)$ not so much as it doesn't take into account some of the spikes of the original function.

\end{document}