\documentclass[letterpaper,12pt,addpoints,answers]{exam}
%\usepackage[utf8]{inputenc}
\usepackage[english]{babel}

\usepackage[top=1in, bottom=1in, left=0.75in, right=0.75in]{geometry}
\usepackage{amsmath,amssymb,graphicx,xcolor,asymptote}
\usepackage{embedall}

\newcommand{\university}{SANTA MONICA COLLEGE}
\newcommand{\faculty}{Department of Mathematics}
\newcommand{\class}{Math 10}
\newcommand{\examnum}{Midterm Exam}
\newcommand{\examdate}{April 8, 2022}
\newcommand{\timelimit}{24 hours}

\newcommand{\tf}{$\quad\quad T\quad F$}
\newcommand{\tft}{$\quad\quad \boxed T\quad F$}
\newcommand{\tff}{$\quad\quad T\quad \boxed F$}
\newcommand{\tfnt}{$\quad\quad \boxed T\quad F\quad$neither}
\newcommand{\tfnf}{$\quad\quad T\quad \boxed F\quad$neither}
\newcommand{\tfnn}{$\quad\quad T\quad F\quad \boxed{\text{ neither}}$}
\newcommand{\tfn}{$\quad\quad T\quad F\quad$neither}

\pagestyle{headandfoot}
\firstpageheader{}{}{}
\firstpagefooter{}{Page \thepage\ of \numpages}{}
\runningheader{\class}{\examnum}{\examdate}
\runningheadrule
\runningfooter{}{Page \thepage\ of \numpages}{}

\begin{document}

\title{\Large \textbf{\university\\ \faculty\\
        \bigskip
        \class -- \examnum }}
\author{Instructor: Kevin Arlin}
\date{\examdate}
\maketitle
\noindent \rule{\textwidth}{1pt}

Submit your work as you would any homework assignment: direct submission of a .pdf is ideal.
The exam is open book and open notes. \textcolor{red}{It is not open Internet.}  I take
academic honesty during COVID even more seriously than usual, and have been known to
be rather vindictive when I see solutions that were garnered online. Nobody wants that.
Don't do it. Not even if you're having lots of trouble with the test. It is fine if you only get
a 65\%; you might even still get an 'A' that way. I am generous with curves as appropriate,
and may both award bonus points to the toughest problems here and also curve grades toward the
end of the course.

Show your work. Partial credit will be given when earned. Explanations are
required unless explicitly not requested.

\emph{Enter your name below to signal your assent to the following honor statement:}

I, \fillin[\emph{Elias Schablowski}][2.5in] affirm that I have not discussed the contents of this exam with anyone, whether in person or online, except perhaps Kevin Arlin, and that I will not do so before Tuesday, April 12.

\begin{center}
    \textbf{Distribution of Points}\\
    \medskip
    \multicolumngradetable{2}[questions]
\end{center}

\clearpage

This page is intentionally left blank to accommodate work that wouldn't fit elsewhere and/or scratch work.
\clearpage
\emph{Note: this problem serves as your entire quiz 6, but it'll be graded with partial credit.}

\begin{questions}
    \section*{Quiz 6}

    \question[4] Prove that, if $n>0$ is any positive integer, then $\sum_{j=1}^n 4j\cdot 3^j=(6n-3)\cdot 3^n+3.$

    \begin{solutionorbox}[7in]
        \paragraph{Definitions:}
        Let $f(n)=\sum_{j=1}^n 4j\cdot 3^j$.
        $$f(1) = 12$$
        $$f(n) = f(n-1) + 4n \cdot 3^n$$

        \paragraph{Base Step:} $f(1) = 12 = (6(1) - 3) \cdot 3^{(1)} + 3 = (3) \cdot 3 + 3 = 9 + 3 = 12$
        \paragraph{Inductive Step:} Assume
        $f(k) = (6k-3)\cdot 3^k+3$ when $k \geq 1$.
        Then $f(k+1) = f(k) + 4(k+1) \cdot 3^{(k+1)}$.
        Substituting $f(k)$ we get $f(k+1) = (6k-3)\cdot 3^k+3 + 4(k+1) \cdot 3^{(k+1)}$.
        Exapnding $f(k+1) = 6k \cdot 3^k + 4k \cdot 3^{(k+1)} + 4 \cdot 3^{(k+1)} - 3 \cdot 3^k + 3$.
        Substitute $3^k = \frac{3^{k+1}}{3}$ we get $f(k+1) = 2k \cdot 3^{k+1} + 4k \cdot 3^{(k+1)} + 4 \cdot 3^{(k+1)} - 3^{k+1} + 3$.
        Simplify $f(k+1) = 6k \cdot 3^{(k+1)} + 3 \cdot 3^{(k+1)} + 3$.
        Simplifying further we get $f(k+1) = (6k + 3) \cdot 3^{(k+1)} + 3 = (6(k + 1) - 3) \cdot 3^{(k+1)} + 3$.
    \end{solutionorbox}

    \clearpage

    \section*{Midterm}

    \question[10] The $abc$ conjecture states that there is a certain positive real number $\varepsilon$ such that
    whenever $a,b,c$ are natural numbers with $c=a+b,$ all bigger than a certain $k\in \mathbb N,$ then $c$ is no bigger than the $\mathrm{rad}(abc)^{1+\varepsilon}.$ Here $\mathrm{rad}:\mathbb N\to \mathbb N$ is a certain function
    whose definition you don't have to worry about. A student attempts to formalize this conjecture in predicate
    logic as follows:
    \[\exists\varepsilon\in (0,\infty)\forall (a,b,c)\in \mathbb N^3\exists k\in \mathbb N (a>k\wedge b>k\wedge c>k\to c\le \mathrm{rad}(abc)^{1+\varepsilon}.) \]
    There are exactly two errors in the student's formalization. Explain what they are and how to fix them.

    \begin{solutionorlines}[1in]
        The students logic equates the sentence "There exists a positive real number $\varepsilon$ such that for all $a$, $b$, and $c$ in $\mathbb{N}$, there exists a $k$ in $N$ such that $a$, $b$, and $c$ are larger than $k$ then $c\le \mathrm{rad}(abc)^{1+\varepsilon}$."\\
        This has two problems,
        primarily it does not say anything about $c=a+b$ (which can be fixed by either adding that $c=a+b$ to his logic or by substituting $c$ for $a+b$ which would also reduce the variables),
        and furthermore the student switched the relationship between $k$ and $a$, $b$, and $c$, which can be fixed by moving the $k$ before $a$, $b$, and $c$.

        \[\exists\varepsilon\in (0,\infty)\exists k\in \mathbb N\forall (a,b)\in \{(x,y) | (x, y) \in \mathbb N^2 \land x > k \land y > k\} \exists c \in \{a + b\}(c\le \mathrm{rad}(abc)^{1+\varepsilon}) \]
    \end{solutionorlines}
    \question
    \begin{parts}
        \part[3] How many elements are in $\mathcal P(\{\mathrm{purdy},\{\mathrm{purdy}\}\})\cap \{\mathrm{purdy},\{\mathrm{purdy}\}\},$ where purdy is my anxious tabby cat? Explain why.

        \begin{solutionorlines}[1in]
            $$\mathcal P(\{\mathrm{purdy},\{\mathrm{purdy}\}\}) = \{\emptyset, \{\mathrm{purdy}\}, \{\{\mathrm{purdy}\}\}, \{\mathrm{purdy}, \{\mathrm{purdy}\}\}$$

            The only common element between them is $\{\mathrm{purdy}\}$ so \linebreak $\mathcal P(\{\mathrm{purdy},\{\mathrm{purdy}\}\})\cap \{\mathrm{purdy},\{\mathrm{purdy}\}\} = \{\{\mathrm{purdy}\}\}$, which is one element.
        \end{solutionorlines}

        \part[7] Suppose $A$ and $B$ are finite sets with cardinalities $|A|=a$ and $|B|=b,$ with $b\ge a.$ In terms of $a$ and
        $b,$ what are all the possibilities for $|A\cup B|?$ Show that all the possibilities you claim are possible,
        and that no other possibilities are possible.

        \begin{solutionorlines}[2in]
            \paragraph{Case $A \subseteq B$:} This means that $\forall x \in A(x \in B)$. Therefore $A \cup B = B$ and therefore $|A\cup B| = |B| = b$.
            \paragraph{Case $A \subset B$:} Let $C = A \cap B$ and $c = |C|$. Then $|A \cup B| = a + b - c$ where $c < a \land c < b$ since all of $B$'s and some of $A$'s elements are counted. Therefore $b < |A \cup B| < a + b$.
            \paragraph{Case $A \nsubseteq B$:} Then $|A \cup B| = a + b$ as all elements from both $A$ and $B$ are counted.

            \vspace{0.5cm}

            Thus we have proben: $b \leq |A\cup B| \leq a + b$.
        \end{solutionorlines}

    \end{parts}

    \question Say that a function $f:A\to B$ has the epi property if the following holds:
    \[\forall X\in\mathrm{Set}\forall g,h:B\to X (g\circ f=h\circ f\to g=h.)\]
    Here $\mathrm{Set}$ is the collection of all possible sets, so the first quantifier is over all sets
    $X$ and the second, over all functions $B\to X.$
    \begin{parts}
        \part[2] Show that $f:\{a,b,c\}\to \{d,e\}$ given by $f(a)=d,f(b)=d,f(c)=e$ has the epi property.

        \begin{solutionorlines}[3in]
            Let $z_1, z_2, z_3, z_4 \in X$.
            Let $g(d) = z_1$, $g(e) = z_2$, $h(d) = z_3$ and $h(e) = z_4$.
            Then $g(f(a)) = g(f(b)) = z_1$, $h(f(a)) = h(f(b)) = z_3$, and $g(f(a)) = g(f(b)) = h(f(a)) = h(f(b))$, therefore $z_1 = z_3$.
            Also $g(f(c)) = g(f(c)) = z_1$, $h(f(c)) = z_4$, and $g(f(c)) = h(f(c))$, therefore $z_2 = z_4$.
            Therefore $g = h$ and $f$ has the epi property.
        \end{solutionorlines}

        \part[2] Show that $f:\{1,2,3\}\to \{4,5\}$ given by $f(1)=f(2)=f(3)=4$ does not have the epi property.
        \begin{solutionorlines}[1.5in]
            Let $z_1, z_2 \in X$.
            Let $g(e) = z_1$, $h(e) = z_2$.
            Since $g(e)$ and $h(e)$ do not exist in $g \circ f$ and $h \circ f$, $z_1$ does not have to equal $z_2$ and therefore $g$ does not have to equal $h$.
            Thus $f$ does not have the epi property.
        \end{solutionorlines}
        \part[6] Conjecture and prove a more familiar way of describing a function with the epi property.
        \begin{solutionorlines}[3in]
            If and only if $\forall b \in B \exists a \in A(f(a) = b)$ ($f$ is \textbf{subjunctive}), then $f$ has the epi property.

            $g = h \leftrightarrow \forall b \in B(g(b) = h(b))$, therefore if $f$ is subjunctive and $g\circ f=h\circ f$, then $g = h$ as $\forall b \in B \exists a \in A(g(f(a)) = g(f(a)))$.

            Furthermore if $f$ isn't subjective, $\exists b \in B \forall a \in A(f(a) \notin B \to g(b) \neq h(b))$
        \end{solutionorlines}
    \end{parts}

    \question For any $n\in \mathbb Z_{>0}$ and $k\in \mathbb Z_n,$ let $\mu_k:\mathbb Z_n\to \mathbb Z_n$ be the multiplication-by-$k$ function, so $\mu_k(m)=k\times_n m.$
    \begin{parts}
        \part[2] Give an example of an $n$ and a $k\ne 0$ for which $\mu_k$ is not injective. (Explain why not.)
        \begin{solutionorlines}[0.5in]
            Let $n = 10$, $k = 5$.

            Since $\mu_5(5) = 25 \mod 10 = 5 = \mu_k(1)$
        \end{solutionorlines}
        \part[2] Give an example of an $n$ and a $k\ne 1$ for which $\mu_k$ is injective. (Prove it.)
        \begin{solutionorlines}[1in]
            Let $n = 3$ and $k = 2$.
            \paragraph{0} $\mu_2(0) = 0 \mod 3 = 0$
            \paragraph{1} $\mu_2(1) = 2 \mod 3 = 2$
            \paragraph{2} $\mu_2(2) = 4 \mod 3 = 1$
        \end{solutionorlines}
        \part[6] Do your best to state and prove necessary and sufficient conditions on $n$ and $k$ which guarantee $\mu_k$ is injective.
        \begin{solutionorlines}[3in]
            $$\forall x \in Z_n \forall y \in Z_n(\mu_k(x) = \mu_k(y) \to x = y)$$
            Therefore $kx = ky (\mod{n}) \to x = y$.

            Assume $k = n - 1 \lor k = 1$ is the sufficient and necessary condition for $\mu_k$ to be injective.

            Let $f(a) = (na - a) \mod n$
            $$f(a) = ((na \mod n) - (a \mod n)) \mod n$$
            $$ = ((((n \mod n)(a \mod n)) \mod n) - (a \mod n)) \mod n$$
            $$ = ((((0)(a \mod n)) \mod n) - (a \mod n)) \mod n$$
            $$ = (((0) \mod n) - (a \mod n)) \mod n$$
            $$ = ((0 - (a \mod n)) \mod n$$
            $$ = ((-(a \mod n))$$
            $$ = -(a \mod n)$$

            \paragraph{$k = n - 1$:}
            $$(n - 1)x = (n - 1)y (\mod{n}) \to x = y$$
            $$nx - x = ny - y (\mod{n}) \to x = y$$
            $$(nx - x) \mod{n} = (ny - y) \mod n \to x = y$$
            $$f(x) = f(y) \to x = y$$
            $$-x = -y \to x = y$$
            \paragraph{$k = 1$:}
            $$(1)x = (1)y \to x = y$$
            $$x = y \to x = y$$
        \end{solutionorlines}
    \end{parts}

    \question Let $C$ be the set of real numbers defined recursively as follows:
    \begin{itemize}
        \item (Base case) $\mathbb Q\subseteq C$
        \item (Recursive step) If $x,y\in C$ then $x+y,x-y,xy,x/y$ are in $C,$ and if $x\ge 0$ then so is $\sqrt x.$
    \end{itemize}
    \begin{parts}
        \part[2] Give three examples of elements of $C$ that are significantly different-looking from each other.
        \answerline[$34$, $\frac{321}{456}$, $\sqrt{345}$, $\frac{85\sqrt{432}}{453}$]
        \part[1] Give an example of a real number which is not in $C.$
        \answerline[$e$, $\pi$, $\tau$, etc.]
        \part[7] Suppose we want to define a function $d:C\to \mathbb N$ such that $d(x)$ is the number of nested square
        roots needed to express $x.$ What is wrong with this definition, and how could you improve it?
        \begin{itemize}
            \item $d(x)=0,$ if $x\in \mathbb Q$
            \item $d(x+y)=d(x-y)=d(xy)=d(x/y)=\max(d(x),d(y)),$ and $d(\sqrt x)=d(x)+1,$ when $x>0.$
        \end{itemize}
        \begin{solutionorlines}[1in]
            The problem is that it is undefined what happens if $x < 0$,
            one way to mitigate this is to add another case that $d(xy) = d(y)$ when $x < 0$.
        \end{solutionorlines}
    \end{parts}

    \bonusquestion (Bonus:) Consider an alphabet
    $\Sigma.$ In principle, $\Sigma$ can be any
    set, but you should think $\Sigma=\{0,1\}$
    or $\Sigma = \{a,b,\ldots,z\}.$

    The set $\mathcal R_\Sigma$
    of \emph{regular languages} over $\Sigma$ is defined
    as follows. Every element of $\mathcal R_\Sigma$ is
    in particular a set of strings over $\Sigma;$ that is, $\mathcal R_\Sigma\subseteq \mathcal P(\Sigma^*).$
    \begin{itemize}
        \item $\emptyset\in \mathcal R_\Sigma$
        \item $\forall s\in \Sigma, \{s\}\in \mathcal R_\Sigma$
        \item $\forall S \in \mathcal R_\Sigma, S^*\in \mathcal R_\Sigma,$ where $S^*$ is the smallest set
              containing $S$ and closed under concatenation. (So it contains all concatenations of $n$ strings from $S,$ for every $n.$)
        \item The union $S\cup T$ and concatenation $S\bullet T=\{st\mid s\in S,t\in T\}$ are in $\mathcal R_\Sigma$ whenever $S$ and $T$ are.
    \end{itemize}

    Prove that, with $\Sigma=\{0,1\},$ the set $S$ of
    bit strings that flip bits at most once (so
    $S$ contains $000011111$ and $11111$ but not $010$) is
    a regular language.
    \begin{solutionorlines}[7in]
        Let $S_k$ be the set of bit strings of $k$ bits that flip at most once.
        Let $T_1 = \{1\}$.
        Let $T_k$ be the set of $k$ $1$s.
        Let $F_1 = \{0\}$.
        Let $F_k$ be the set of $k$ $0$s.

        \paragraph{Base Step:} Since $\forall s \in \Sigma, \{s\} \in \mathcal R_\Sigma$, $T_0 = \{1\}$ is a regular language.
        \paragraph{Inductive Step:}
        Assume that $T_k \in \mathcal R_\Sigma$.
        Since $S\bullet T=\{st\mid s\in S,t\in T\}$ are in $\mathcal R_\Sigma$,
        if we substitute $T_0$ for $S$ and $T_k$ for $T$,
        we get $T_{k+1} = T_0 \bullet T_k = {t_0t_k | t_0 \in T_0, t_k \in T_k}$
        which is all in $\mathcal R_\Sigma$.

        \paragraph{Base Step:} Since $\forall s \in \Sigma, \{s\} \in \mathcal R_\Sigma$, $F_0 = \{0\}$ is a regular language.
        \paragraph{Inductive Step:}
        Assume that $F_k \in \mathcal R_\Sigma$.
        Since $S\bullet F=\{st\mid s\in S,t\in F\}$ are in $\mathcal R_\Sigma$,
        if we substitute $F_0$ for $S$ and $F_k$ for $F$,
        we get $F_{k+1} = F_0 \bullet F_k = {f_0f_k | f_0 \in F_0, f_k \in F_k}$
        which is all in $\mathcal R_\Sigma$.

        \paragraph{Base Step:}
        Since $S \in \mathcal R_\Sigma \land T \mathcal R_\Sigma \implies S \cup T \in \mathcal R_\Sigma$,
        and $S_0 = T_0 \cup F_0$ and $T_0, F_0  \in \mathcal R_\Sigma$, $S_0  \in \mathcal R_\Sigma$.

        \paragraph{Inductive Step:}
        Let $a, b \in \mathbb N(a + b = k)$.
        Since $S\bullet F=\{st\mid s\in S,t\in F\}$ and $S\cup T$ are in $\mathcal R_\Sigma$ if $S$ and $T$ are,
        if we substitute $T_a \cup F_a$ for $S$ and $T_b \cup F_b$ for $F$,
        we get $S_k = (T_a \cup F_a) \bullet (T_b \cup F_b)$ which is in $\mathcal R_\Sigma$.

        Since $S = \bigcup_i^\infty S_i$, $S \in \mathcal R_\Sigma$ as $S_k in \mathcal R_\Sigma$ and $S,T \in \mathcal R_\Sigma \to S \cup T$
    \end{solutionorlines}
\end{questions}
\end{document}


\section{Introduction} Instrumental variables estimation has gained considerable traction in recent decades as a tool for causal inference, particularly amongst empirical researchers. However, this has also highlighted the importance of taking a deeper look at the theoretical properties underlying one's estimator of choice, whether it is instrumental variable estimator or any other estimator. A case-in-point is the well-known 1991 quarter-of-birth study on returns to schooling estimates by \cite{angrist1991does} and the equally famous rebuttal of their results published by \cite{bound1995problems}.

\par The \cite{bound1995problems} critique of the quarter-of-birth study in essence formed the starting point of the weak instruments literature. They provided simulation evidence that the instruments used by \cite{angrist1991does} were very weak and this in turn led to misleading results. Interestingly, their study was not the first time that the poor finite-sample behavior of the instrumental variables estimator (two-stage least squares estimator in particular) was discussed. Previously, numerous authors had presented results on the magnitude of the finite-sample bias of the 2SLS estimator towards OLS, such as \cite{nagar1959bias}, \cite{basmann1960asymptotic}, \cite{richardson1968exact} and \cite{sawa1969exact}.
However, their critique was the first to suggest some practices for the detection of possible presence of weak instruments (reporting first-stage F-statistic and the $R^2$ of the first stage regression), which have since become standard practice for using instrumental variables in empirical studies.

\par The literature on weak instruments have since developed considerably and continues to do so. \cite{staiger1997stock} showed that conventional asymptotic approaches fail to provide good approximations for weak instruments, and formalized the problem of weak instruments by introducing `weak-instrument asymptotics' which mimics the situation of weak instruments better. \cite{stock2002survey} provide a comprehensive survey of the weak instruments literature, wherein they emphasize that the problem of bias of the two-stage least-squares estimator is not
solely a small sample problem. It affects estimates carried out on large sample sizes as well, as is the case with the quarter-of-birth study which has a dataset of around 300,000 observations.

\par Further, given the poor finite-sample properties of the two-stage least squares estimator, numerous alternate estimators have also been proposed. These include the USSIV estimator suggested by \cite{angrist1995split}, Fuller-k estimator by \cite{fuller1977some}, bias-adjusted 2SLS estimator by \cite{donald2001choosing}, JIVE estimators by \cite{angrist1999jackknife} and \cite{blomquist1999small}. Another estimator, the LIML was formalized by \cite{anderson1949estimation}. In our paper, we take a deeper look at two of these estimators - JIVE and LIML estimators.

\par We introduce the two conditions for instrumental variables estimation, the first of which will be a recurring theme in our paper, since it connects to the problem of weak instruments. Consider the standard population regression model:
\begin{align*}
Y_i &= \beta_0 + \beta_1X_i + \varepsilon_i, \: \: \: \: i = 1,2,...n
\end{align*}
where $\varepsilon_i$ is the error term representing omitted factors that determine $Y_i$. Variables correlated with the error term are termed endogenous variables and those uncorrelated with the error term are exogenous. A valid instrument $Z_i$ must satisfy two conditions:
\begin{itemize}
    \item Instrument Relevance: corr ($Z_i$, $X_i$) $\neq 0.$\\
    It is the implications of this condition, specifically the strength (or weakness) of this correlation between the instrument and the endogenous regressor, that we are interested in. Instruments that explain little of the variation in X are called weak instruments. A precise definition is introduced in section 3.1.
    \item Instrument Exogeneity: corr ($Z_i$, $\varepsilon_i$) = 0\\
    Note that this condition cannot be statistically tested, since it involves the covariance between $Z_i$ and the unobserved $\varepsilon_i$.
\end{itemize}
\par If number of instruments (K) equals the number
of endogenous regressors (L) we have the just or exactly identified case.  If the number of instruments exceeds the number of endogenous regressors, K $\geq$ L, then we have the over-identified case.

\par The rest of the paper is sectioned as follows. Chapter 2 presents a derivation of the 2SLS estimator and its limitations in the weak instruments case. Chapter 3 discusses methods to test for weak instruments. Chapter 4 presents the alternative estimators (JIVE and LIML). Chapter 5 presents the simulation study. Chapter 6 presents the application to returns to schooling. Chapter 7 concludes.


\documentclass[12pt,pdf,singlespace]{article}

\usepackage{times}

\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{listings}
\usepackage{verbatim}

\title{Using geodesic Levenberg-Marquardt}

\author{Colin Clement \& Mark K.~Transtrum}

\begin{document}
{
\maketitle
}

\section{General strategy:}

The first thing to do is examine the document string in the \texttt{geodesiclm} subroutine found in \texttt{geodesiclm.f90}, and make sure you provide at least the function which computes the residuals (\texttt{func}). Be careful to use the form specified in the code's documentation.

The strategy for use we suggest is to start by turning everything fancy off, so that this algorithm functions as a standard Levenberg-Marquardt (LM) algorithm. I.e. start with \texttt{iaccel}=0, \texttt{ibold}=0 and \texttt{ibroyden}=0. When you do this, you should pay most attention to \texttt{factoraccept} and \texttt{factorreject}. I recommend starting with 5 and 2, respectively. Standard LM algorithms set both of these parameters to 10. The problem with making them equal is that the optimizer can get stuck in cycles. For more nuanced reasons (we call it `delayed gratification' \cite{transtrum2012improvements}) we also recommend that the accept factor be larger than the reject factor. Play around with these until you are satisfied with the convergence rate.

You might find at first that the default convergence criterion (tolerances and goals, see the list below) are causing the algorithm to give up before true convergence. The procedure for diagnosing and ameliorating this is as follows: After the algorithm gives up, read the convergence message. The integer provided will represent a specific convergence criterion being hit (the list in is \texttt{geodesiclm.f90}), and you should decrease the offending condition to force the algorithm to work harder. Do so until the algorithm hits a different convergence criterion, and repeat until either you are satisfied with the minimum, or the speed of convergence becomes your principal concern. If you are faced with the latter problem, read on for useful options for speeding convergence.

Once you get things working reasonably well with standard LM settings, try turning on geodesic acceleration by setting \texttt{iaccel}=1. If you are not providing a routine to calculate the second directional derivative (see \cite{transtrum2012improvements,transtrum2012geodesic, PRE83} for details), then you need to set \texttt{analytic\_avv} = \texttt{.false.} so that finite differences are used to estimate it. You will need to pay attention to the finite difference step size \texttt{h2} to make sure your calculation is stable. We suggest using \texttt{h2}=0.1. Since the acceleration should be a correction to standard LM, the parameter \texttt{avmax} limits the length of the acceleration vector in proportion to the proposed LM step size. As such, \texttt{avmax}$<1.$ are recommended.

If you are satisfied with the behavior at this point, experiment with allowing uphill steps by setting \texttt{ibold} from 1 (least `bold'  uphill allowances) to 4 (most `bold' uphill allowances). You can also play with setting \texttt{ibroyden}=1, which will employ an estimate of the Jacobian to speed up some iterations, sacrificing some accuracy.

For more details on why least-squares fitting is so challenging, and for intuition on how various parts of this algorithm work, see \cite{PRL104, PRE83}. See \cite{transtrum2012improvements} for details specific to implementation and benchmarked performance of the algorithm.

\section{Parameter defaults and details:}

These parameters are ordered by their relative importance. A lot of this information is a repeat of what can be found in the documentation to the source code in \texttt{geodesiclm.f90}.
~\\

\noindent {\bf Format:}
\begin{itemize}
	\item \texttt{parameter\_name} = $<$suggested default$>$ : Some words you might find useful.
\end{itemize}

\noindent {\bf Basic Levenberg-Marquardt (LM) Parameters:}
\begin{itemize}
	\item \texttt{factoraccept} = 5 : This is the factor by which the LM parameter (often denoted $\lambda$) is {\bf decreased after a step is accepted.}
	\item \texttt{factorreject} = 2 : This is the factor by which the LM parameter is {\bf increased after a step is rejected.}
	\item \texttt{maxlam} = 1.E7 : The routine will never let $\lambda$ become larger than this.
	\item \texttt{imethod} = 0 : This is an integer which specifies the exact way $\lambda$ is updated. See \texttt{geodesiclm.f90} and \cite{transtrum2012improvements} for more details.
	\item \texttt{initialfactor} = 1 : The initial $\lambda$.
	\item \texttt{mode} = 0 : 0 or 1. Chooses between the `Levenberg' and `Marquardt' style of matrix added to $J^T J$ \texttt{geodesiclm.f90}.
\end{itemize}
\noindent {\bf Basic convergence criterion:}

\begin{itemize}
	\item \texttt{maxiter} = 500 : The total number of steps the algorithm will attempt. This should depend on how long a function and Jacobian evaluation takes, and how long you are willing to wait.
	\item \texttt{maxfev} = 0 : The maximum number of allowed residual function (\texttt{func}) evaluations. 0 indicates no limit.
	\item \texttt{Cgoal} = 1 : The goal for the minimum squared error.
	\item \texttt{maxjev} = 0 : The maximum number of allowed Jacobian (\texttt{jacobian}) evaluations. 0 indicates no limit.
\end{itemize}

\noindent {\bf Geodesic acceleration parameters:}

\begin{itemize}
	\item \texttt{iaccel} = 1 : 1 or 0 for geodesic acceleration on or off.
	\item \texttt{h2} = 0.1 :  Finite-difference step size for second-directional derivative estimation.
	\item \texttt{avmax} = 0.75 : The `acceleration' should be a correction to the `velocity' and this enforces that.
	\item \texttt{maxaev} = 0 : Maximum number of analytic second directional derivative (\texttt{avv}) evaluations. 0 indicates no limit.
\end{itemize}

\noindent {\bf Extra flags:}

\begin{itemize}

\item \texttt{print\_level} = 5 : 0-1, how much information to print to \texttt{print\_unit}.
\item \texttt{print\_unit} = 6 : The integer representing the place to print messages. Either open a file or set to 6 for standard out.

\item \texttt{analytic\_jac} = \texttt{.true.} : True if you supply a routine to compute the Jacobian (\texttt{jacobian}).
\item \texttt{analytic\_avv} = \texttt{.false.} : True if you supply a routine to compute the second directional-derivative (\texttt{avv}).
\item \texttt{center\_diff} = \texttt{.true.} : True if you want to use centered-difference. More accurate but slower than forward-difference.

\end{itemize}

\noindent {\bf Advanced Jacobian update and convergence criterion:}

\begin{itemize}

\item \texttt{ibold} = 0 : (0-4) If nonzero this allows some uphill steps which can speed convergence.
\item \texttt{ibroyden} = 0 : If nonzero this employs Broyden approximate Jacobian updates. Can speed up algorithm with cost of accuracy.
\item \texttt{eps} = 1.5E-6 : Precision of evaluation of residuals in \texttt{func}.

\item \texttt{h1} = 1.0E-5 : Step size for finite-difference Jacobian estimate.
\item \texttt{artol} = 1.E-3 : See \texttt{geodesiclm.f90} for details on these convergence tolerances.
\item \texttt{gtol} = 1.5E-8
\item \texttt{xtol} = 1.E-10
\item \texttt{xrtol} = 1.5E-8
\item \texttt{frtol} = 1.5E-8

\end{itemize}

\input{Bibliography}

\end{document}


\documentclass[a4paper,12pt]{article}

\usepackage{amsmath,amssymb,amsthm,tikz}
\usetikzlibrary{calc,arrows.meta}
\usepackage[margin=20mm]{geometry}
\usepackage{hyperref}

\setlength{\parindent}{0pt}
\setlength{\columnsep}{1cm}

\begin{document}

\twocolumn

\thispagestyle{empty}

\begin{center}
{\Large Assignment 15}\\
{\Large Published on 2020-12-06,}\\
{\em Estimated Time: 30 minutes,}\\
{\em Max.grade 10\textperthousand}
\end{center}

\section{Prim's Algorithm}

(Goodrich2011, p.651) defines Prim's algorithm.
See also \url{https://bit.ly/2VLz3DK}.
It is an efficient algorithm; it requires $O((m+n)\log_2 n)$ time, if
we use priority queues.
In this exercise you do not need to implement a priority queue;
assume that you can always compute the minimums in your head and
grow the MST accordingly.

\section{Problem}

We start with the graph shown in Figure~\ref{fig:problem-graph}.

\begin{figure}[!htb]
\center{\includegraphics[width=3in]{assignment15-mst-prim/problem-graph.png}}
\caption{\label{fig:problem-graph} Graph diagram}
\end{figure}

\vspace{10pt}
{\bf (A)} Redraw the graph Figure~\ref{fig:problem-graph},
replace the edge weights $a+b$, $b+c$ and $c+a$ by the from your Student ID).
Vertex $A$ will be your source vertex.

\vspace{10pt}
{\bf (B)} Draw $9$ copies of the graph (no need to rewrite
the edge weights all the time \textendash{} just keep them before your eyes
as you run the Prim's algorithm).
Show how the Prim's MST (Minimum Spanning Tree grows) one edge at a time.
Highlight the edges selected for MST (make them bold or color them differently).
The first graph in this sequence contains no selected edges of MST, but
the last graph in this sequence contains the full MST.
(Goodrich2011, p.652) contains sample pictures how to run the Prim's algorithm.

\vspace{10pt}
{\bf (C)} Add up the total weight of the obtained MST and
write this in your answer (it should be the minimum value among all the
possible spanning trees in this graph).

\end{document}


\documentclass[12pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{float}
\usepackage{amsmath}

\usepackage[hmargin=3cm,vmargin=6.0cm]{geometry}
%\topmargin=0cm
\topmargin=-2cm
\addtolength{\textheight}{6.5cm}
\addtolength{\textwidth}{2.0cm}
%\setlength{\leftmargin}{-5cm}
\setlength{\oddsidemargin}{0.0cm}
\setlength{\evensidemargin}{0.0cm}

%misc libraries goes here
\usepackage{amssymb}

\begin{document}

\section*{Student Information }
%Write your full name and id number between the colon and newline
%Put one empty space character after colon and before newline
Full Name : Yavuz Selim YEŞİLYURT \\
Id Number : 2259166 \\

% Write your answers below the section tags
\section*{Answer 1}
\hspace{5mm} a) Using membership table to prove $A \cap B \subseteq (A \cup \overline B) \cap (\overline A \cup B)$:
\begin{table}[H]
\small
\centering
\begin{tabular}{|c|c|c|c|c|c|c|c|c}
\hline
$A$ & $B$ & $ \overline A $ & $ \overline B$ & $A \cap B$ & $A \cup \overline B$ & $\overline A \cup B$ & $ (A \cup \overline B) \cap (\overline A \cup B)$\\
\hline
1 & 1 & 0 & 0 & 1 & 1 & 1 & 1\\
1 & 0 & 0 & 1 & 0 & 1 & 0 & 0\\
0 & 1 & 1 & 0 & 0 & 0 & 1 & 0\\
0 & 0 & 1 & 1 & 0 & 1 & 1 & 1\\
\hline
\end{tabular}
\end{table}
b) Using membership table to prove $\overline A \cap \overline B \subseteq (A \cup \overline B) \cap (\overline A \cup B)$:
\begin{table}[H]
\small
\centering
\begin{tabular}{|c|c|c|c|c|c|c|c|c}
\hline
$A$ & $B$ & $ \overline A $ & $ \overline B$ & $\overline A \cap \overline B $ & $A \cup \overline B$ & $\overline A \cup B$ & $ (A \cup \overline B) \cap (\overline A \cup B)$\\
\hline
1 & 1 & 0 & 0 & 0 & 1 & 1 & 1\\
1 & 0 & 0 & 1 & 0 & 1 & 0 & 0\\
0 & 1 & 1 & 0 & 0 & 0 & 1 & 0\\
0 & 0 & 1 & 1 & 1 & 1 & 1 & 1\\
\hline
\end{tabular}
\end{table}

\section*{Answer 2}
\begin{align*}
f^{-1}((A \cap B) \times C) &= f^{-1}(A \times B) \cap f^{-1}(B \times C) \\
f^{-1}((A \times C) \cap (B \times C)) &= f^{-1}(A \times C) \cap f^{-1}(B \times C) \\
\end{align*}

\hspace*{5mm}\textbf{Part 1}, show that $f^{-1}((A \times C) \cap (B \times C)) \subseteq  f^{-1}(A \times C) \cap f^{-1}(B \times C)$;\\
Assume that $x \in f^{-1}((A \times C) \cap (B \times C))$, then $f(x) \in ((A \times C) \cap (B \times C))$,\\ so $f(x) \in (A \times C)$ and $f(x) \in (B \times C)$, therefore $x \in f^{-1}(A \times C)$ and $x \in  f^{-1}(B \times C)$,\\ so $ x \in ( f^{-1}(A \times C) \cap  f^{-1}(B \times C))$\\\\

\hspace*{5mm}\textbf{Part 2}, show that $ f^{-1}(A \times C) \cap f^{-1}(B \times C) \subseteq f^{-1}((A \times C) \cap (B \times C))$;\\
Assume that $x \in f^{-1}(A \times C) \cap f^{-1}(B \times C)$, then $x \in  f^{-1}(A \times C)$ and $x \in  f^{-1}(B \times C)$,\\ so $f(x) \in (A \times C)$ and $f(x) \in (B \times C)$, which implies $ f(x) \in ((A \times C) \cap (B \times C))$, \\ therefore $x \in  f^{-1}((A \times C) \cap (B \times C))$\\\\

\hspace*{5mm}Hence by part1 and part2 $f^{-1}((A \times C) \cap (B \times C)) =  f^{-1}(A \times C) \cap f^{-1}(B \times C)$\\
\hspace*{5mm}\quad \ Therefore $f^{-1}((A \cap B) \times C) = f^{-1}(A \times B) \cap f^{-1}(B \times C)$

\section*{Answer 3}
\hspace*{5mm}a)	 From the definition of one to one functions, if a function is one to one then for all x and for all y; if $f(x) = f(y)$ then $x = y$. We can clearly see this is not the case for this function. Since the function has $x^2$ term in its interior, it maps the negative items from $R$ to their positive equivalents in $R$ (i.e $(-x)^2 = x^2$ while $x \neq -x$ , even function). So we see that there is at least one situation which contradicts with our assumption, therefore $f(x)$ can not be one-to-one. We know that $f : A \rightarrow B$ is onto if $f(A) = B$. Which means if a function f is onto then every element in its codomain is being mapped. This function is defined from $R$ to $R$ so if $f(x)$ is onto then every item on $R$ must be mapped by $f(x)$. But negative values on $R$ will not be mapped by $f(x)$ . Therefore $f(x)$ can not be onto.\\\\
\hspace*{5mm}b)	From the definitions of one-to-one functions and onto functions on part a, $f(x)$ is one-to-one, since the function maps every distinct item from $R$ to a distinct item in $R$. But $f(x)$ is not onto, since the $0$ in $R$ will remain unmapped from $f(x)$. Therefore $f(x)$ can not be onto.

\section*{Answer 4}

\hspace{6mm}a)
Let A and/or B countably infinite sets such that $A=\{a_1,a_2...\}$ and
$B=\{b_1,b_2...\}$ we can write the elements of $A\times B$ as;\\\\
\begin{tabular}{ c| c c c c c}
 $A \setminus B$  &$ b_1$ & $b_2$ & $b_3$ & ...  \\
   \hline
 $ a_1$ & $(a_1,b_1)$ & $(a_1,b_2)$ & $(a_1,b_3)$ &...\\
 $ a_2$ & $(a_2,b_1)$ & $(a_2,b_2)$ & $(a_2,b_3)$ &...\\
 $ a_3$ & $(a_3,b_1)$ & $(a_3,b_2)$ & $(a_3,b_3)$ &...\\
 ... & ... & ... & ... & ...
\end{tabular}
\\\\
In table, first row, above the horizontal line contains the countably infinite elements of set $B$, first column, left of vertical line contains the countably infinite elements of set $A$. We can write $A \times B$ on this table and we can count the elements of $A \times B$ diagonally (i.e. 1st element: $(a_1,b_1)$, 2nd element: $(a_2,b_1)$, 3rd element $(a_1,b_2)$, 4th element: $(a_1,b_3)$, 5th element: $(a_2,b_2)$, 6th element: $(a_3,b_1)$ and so on. ). With this way of counting we will eventually get all elements in $A \times B$, then we can index these elements with unique numbers from $\mathbb{N}$, therefore we can get the one-to-one correspondence between $A \times B$ and  $\mathbb{N}$. Therefore we can infer that $A \times B$ is countable.\\

b)	Prove by contradiction, suppose that $B$ were countable, say with elements $b1, b2 , ...$ .Then since $A \subseteq B$ , we can list the elements of $A$ using the order in which they appear in this listing of $B$. Therefore $A$ is countable. But this contradicts with the hypothesis ($A$ is uncountable). Thus $B$ is not countable.\\

c)	If $B$ is countable with the elements $b1, b2 , ...$ and $A \subseteq B$, then we can list the elements of $A$ using the order in which they appear in this listing of $B$. Therefore $A$ is countable.

\newpage

\section*{Answer 5}
\hspace{6mm}a)	Let $f_1(x) = 7x^2$ and $f_2(x)=x^3$ ($f_1(x)$ is $\mathcal{O}(f_2(x)$) and they are increasing functions). \\
Prove by the Big-Oh definition,  $ln|f_1(x)|$ is $\mathcal{O}(ln|f_2(x)|)$ if $ln|f_1(x)|\leq  c.(ln|f_2(x)|)$ for some $x>k$. Let us check this condition: If  $ln|7x^2|$ is $\mathcal{O}(ln|x^3|)$ then $ln|7x^2|\leq c.ln|x^3|$ . Exponential both sides  $e^{ln|7x^2|} \leq e^{c.ln|x^3|}$, eliminate the terms  $7x^2 \leq e^c.x^3$, note that when $x\geq k=7$ and $c \geq 1$, we have $7x^2 \leq e^c.x^3$, consequently, we can take $c=1$ and $k=7$ as witnesses to establish the relationship $ln|7x^2|$ is $\mathcal{O}(ln|x^3|)$. Therefore $ln|f_1(x)|$ is $\mathcal{O}(ln|f_2(x)|)$.\\

b)	Let $f_1(x) = 7x^2$ and $f_2(x)=x^3$ ($f_1(x)$ is $\mathcal{O}(f_2(x)$) and they are increasing functions). \\
Prove by the Big-Oh definition, $3^{f_1(x)}$ is $\mathcal{O}(3^{f_2(x)})$ if  $3^{f_1(x)} \leq c.3^{f_2(x)}$ for some $x>k$. Let us check this condition: If $3^{7x^2}$ is $\mathcal{O}(3^{x^3})$ then $3^{7x^2} \leq c.3^{x^3}$ . Take the log of both sides $7x^2.log3 \leq x^3.log3+logc$, divide log3 both sides $7x^2 \leq x^3 + log(c-3)$, note that when $x\geq k=7$ and $c \geq 4$, we have $7x^2 \leq x^3 + log(c-3)$, consequently, we can take $c=4$ and $k=7$ as witnesses to establish the relationship $3^{7x^2}$ is $\mathcal{O}(3^{x^3})$. Therefore $3^{f_1(x)}$ is $\mathcal{O}(3^{f_2(x)})$.\\

\section*{Answer 6}
\hspace{6mm}a)
\begin{equation*}
\begin{split}
(3^x -1 )\ mod(3^y -1) &= 3^{(x \ mod \ y)} -1 \\
3^x -1		 &= k.(3^y -1) + 3^{(x \ mod \ y)}-1 \ \ \ \ \ \ \ where \  k \in \mathbb{Z} \\
3^x 			 &= k.(3^y -1) + 3^{(x \ mod \ y)} \\
log_3(3^x)		 &= log_3(k.(3^y -1) + 3^{(x \ mod \ y)}) \\
x			 &= log_3(k.(3^y -1) + 3^{(x \ mod \ y)})  \\
\end{split}
\end{equation*} \\
\quad Now substitute $x$ in original equation:\\
\begin{equation*}
\begin{split}
(3^{log_3(k.(3^y -1) + 3^{(x \ mod \ y)})}-1)\ mod(3^y -1) &= 3^{(x \ mod \ y)} -1 \\
k.(3^y -1) + 3^{(x \ mod \ y)}-1		 				&= k.(3^y -1) + 3^{(x \ mod \ y)}-1  \\
0											&= 0 \ \ \ \ \ \ \ (1) \\
\end{split}
\end{equation*} \\
\quad Since we get (1) in the last step, $(3^x -1 )\ mod(3^y -1) = 3^{(x \ mod \ y)} -1$ is true.\\

b)
\begin{equation*}
\begin{split}
gcd(123,277) &= gcd(277,123 \ mod\ 277) \\
			 &= gcd(277,123) \\
			 &= gcd(123,277 \ mod\ 123) \\
			 &= gcd(123,31) \\
			 &= gcd(31,123 \ mod\ 31)  \\
			 &= gcd(31,30)  \\
			 &= gcd(30,31 \ mod\ 30)   \\
		 	 &= gcd(30,1)   \\
		 	 &= gcd(1,30 \ mod\ 1)    \\
			 &= gcd(1,0)    \\
			 &= 1           \\
\end{split}
\end{equation*} \\ \\

\end{document}


\lab{Exceptions and File Input/Ouput}{Exceptions and File Input/Output}
\objective{
In Python, an \emph{exception} is an error detected during execution.
Exceptions are important for regulating program usage and for correctly reporting problems to the programmer and end user.
An understanding of exceptions is essential to safely read data from and write data to external files, and being able to interact with external files is important for analyzing data and communicating results.
In this lab we learn exception syntax and file interaction protocols.
}

\section*{Exceptions} % =======================================================

An \emph{exception} formally indicates an error and terminates the program early.
Some of the more common exception types are listed below, along with the kinds of problems that they typically indicate.
%
\begin{table}[H]
\centering
\begin{tabular}{r|l}
    Exception & Indication \\ \hline
    \li{AttributeError} & An attribute reference or assignment failed.\\
    \li{ImportError} & An \li{import} statement failed.\\
    \li{IndexError} & A sequence subscript was out of range.\\
    \li{NameError} & A local or global name was not found.\\
    % \li{SyntaxError} & The parser encountered a syntax error.\\
    \li{TypeError} & An operation or function was applied to an object of \\ &inappropriate type.\\
    \li{ValueError} & An operation or function received an argument that had \\ & the right type but an inappropriate value.\\
    \li{ZeroDivisionError} & The second argument of a division or modulo operation was zero.
\end{tabular}
\end{table}
%
\begin{lstlisting}
>>> print(x)
<<Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined>>

>>> [1, 2, 3].fly()
<<Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute 'fly'>>
\end{lstlisting}
% See \url{https://docs.python.org/3/library/exceptions.html} for the complete list of built-in exception classes.

\subsection*{Raising Exceptions} % --------------------------------------------

Most exceptions are due to coding mistakes and typos.
However, exceptions can also be used intentionally to indicate a problem to the user or programmer.
To create an exception, use the keyword \li{raise}, followed by the name of the exception class.
As soon as an exception is raised, the program stops running unless the exception is handled properly.

% Exception objects can be initialized with any number of arguments.
% These arguments are stored as a tuple attribute called \li{args}, which serves as the string representation of the object.
% We typically provide a single string detailing the reasons for the error.

\begin{lstlisting}
>>> if 7 is not 7.0:                # Raise an exception with an error message.
...     raise Exception("ints and floats are different!")
...
<<Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
Exception: ints and floats are different!>>

>>> for x in range(10):
...     if x > 5:                   # Raise a specific kind of exception.
...         raise ValueError("'x' should not exceed 5.")
...     print(x, end=' ')
...
0 1 2 3 4 5
<<Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
ValueError: 'x' should not exceed 5.>>
\end{lstlisting}

\begin{problem} % Raising Exceptions during magic trick.
Consider the following arithmetic ``magic'' trick.
\begin{enumerate}
    \item Choose a 3-digit number where the first and last digits differ by 2 or more (say, $123$).
    \label{magic-trick-step-1}
    \item Reverse this number by reading it backwards ($321$).
    \item Calculate the positive difference of these numbers ($321 - 123 = 198$).
    \item Add the reverse of the result to itself ($198 + 891 = 1089$).
\end{enumerate}
The result of the last step will always be $1089$, regardless of the original number chosen in step \ref{magic-trick-step-1} (can you explain why?).

The following function prompts the user for input at each step of the magic trick, but does not check that the user's inputs are correct.

\begin{lstlisting}
def arithmagic():
    step_1 = input("Enter a 3-digit number where the first and last "
                                           "digits differ by 2 or more: ")
    step_2 = input("Enter the reverse of the first number, obtained "
                                              "by reading it backwards: ")
    step_3 = input("Enter the positive difference of these numbers: ")
    step_4 = input("Enter the reverse of the previous result: ")
    print(str(step_3), "+", str(step_4), "= 1089 (ta-da!)")
\end{lstlisting}

Modify \li{arithmagic()} so that it verifies the user's input at each step.
Raise a \li{ValueError} with an informative error message if any of the following occur:
\begin{itemize}
\item The first number (\li{step_1}) is not a 3-digit number.
\item The first number's first and last digits differ by less than $2$.
\item The second number (\li{step_2}) is not the reverse of the first number.
\item The third number (\li{step_3}) is not the positive difference of the first two numbers.
\item The fourth number (\li{step_4}) is not the reverse of the third number.
\end{itemize}
(Hint: \li{input()} always returns a string, so each variable is a string initially.
Use \li{int()} to cast the variables as integers when necessary.
The built-in function \li{abs()} may also be useful.)
\end{problem}

\subsection*{Handling Exceptions} % -------------------------------------------

To prevent an exception from halting the program, it must be handled by
placing the problematic lines of code in a \li{try} block.
An \li{except} block then follows with instructions for what to do in the event of an exception.

\begin{lstlisting}
# The 'try' block should hold any lines of code that might raise an exception.
>>> try:
...     print("Entering try block...")
...     raise Exception("for no reason")
...     print("No problem!")        # This line gets skipped.
... # The 'except' block is executed just after the exception is raised.
... except Exception as e:
...     print("There was a problem:", e)
...
<<Entering try block...
There was a problem: for no reason>>
>>> # The program then continues on.
\end{lstlisting}

In this example, the name \li{e} represents the exception within the \li{except} block.
Printing \li{e} displays its error message.
If desired, \li{e} can be raised again with \li{raise e} or just \li{raise}.

The try-except control flow can be expanded with two other blocks, forming a code structure similar to a sequence of \li{if}-\li{elif}-\li{else} blocks.
%
\begin{enumerate}
\item The \li{try} block is executed until an exception is raised (if at all).
% Unlike an \li{if} block, a \li{try} block must always be followed by at least one \li{except} or \li{finally} block.

\item An \li{except} statement specifying the same kind of exception that was raised in the try block ``catches'' the exception, and the block is then executed.
There may be multiple except blocks following a single try block (similiar to having several \li{elif} statements following a single \li{if} statement), and a single except statement may specify multiple kinds of exceptions to catch.

\item The \li{else} block is executed if an exception was \textbf{not} raised in the try block.

\item The \li{finally} block is always executed if it is included.
\end{enumerate}

\begin{lstlisting}
>>> try:
...     print("Entering try block...", end='')
...     house_on_fire = False
...     raise ValueError("The house is on fire!")
... # Check for multiple kinds of exceptions using parentheses.
... except (ValueError, TypeError) as e:
...     print("caught an exception.")
...     house_on_fire = True
... else:                           # Skipped due to the exception.
...     print("no exceptions raised.")
... finally:
...     print("The house is on fire:", house_on_fire)
...
<<Entering try block...caught an exception.
The house is on fire: True>>

>>> try:
...     print("Entering try block...", end='')
...     house_on_fire = False
... except ValueError as e:         # Skipped because there was no exception.
...     print("caught a ValueError.")
...     house_on_fire = True
... except TypeError as e:          # Also skipped.
...     print("caught a TypeError.")
...     house_on_fire = True
... else:
...     print("no exceptions raised.")
... finally:
...     print("The house is on fire:", house_on_fire)
...
<<Entering try block...no exceptions raised.
The house is on fire: False>>
\end{lstlisting}

The code in the \li{finally} block is always executed, even if a \li{return} statement or an uncaught exception occurs in any block following the try statement.

\begin{lstlisting}
>>> def implode():
...     try:                        # Try to return immediately...
...         return
...     finally:                    # ...but 'finally' goes before 'return'.
...         print("Goodbye, world!")
...
>>> implode()
<<Goodbye, world!>>
\end{lstlisting}

See \url{https://docs.python.org/3/tutorial/errors.html} for more examples.

\begin{warn} % Don't use except by itself; specify an exception class.
An \li{except} statement with no specified exception type catches \textbf{any} exception raised in the corresponding \li{try} block.
This approach can mistakenly mask unexpected errors.
Always be specific about the kinds of exceptions you expect to encounter.

\begin{lstlisting}
>>> def divider(x, y):
...     try:
...         return x / yy           # The mispelled yy raises a NameError.
...     except:                     # Catch ANY exception.
...         print("y must not equal zero!")
...
>>> divider(2, 3)
<<y must not equal zero!>>

>>> def divider(x, y):
...     try:
...         return x / yy
...     except ZeroDivisionError:   # Specify an exception type.
...         print("y must not equal zero!")
...
>>> divider(2, 3)
<<Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in divider
NameError: name 'yy' is not defined>>     # Now the mistake is obvious.
\end{lstlisting}
\end{warn}

\begin{problem} % Catch a KeyboardInterrupt to stop a random walk.
A \emph{random walk} is a path created by a sequence of random steps.
% Random walks are used in many applications, including molecular movement, image segmentation, and (somewhat disturbingly) stock market prices.
The following function simulates a random walk by repeatedly adding or subtracting $1$ to a running total.

\begin{lstlisting}
from random import choice

def random_walk(max_iters=1e12):
    walk = 0
    directions = [1, -1]
    for i in range(int(max_iters)):
        walk += choice(directions)
    return walk
\end{lstlisting}

A \li{KeyboardInterrupt} is a special exception that can be triggered at any time by entering \li{ctrl+c} (on most systems) in the keyboard.
Modify \li{random_walk()} so that if the user raises a \li{KeyboardInterrupt} by pressing \li{ctrl+c} while the program is running, the function catches the exception and prints ``Process interrupted at iteration $i$''.
If no \li{KeyboardInterrupt} is raised, print ``Process completed''.
In both cases, return \li{walk} as before.
\end{problem}

\begin{info} % Exception Hierarchy
The built-in exceptions are organized into a class hierarchy.
% This can lead to some confusing behavior because of class inheritance.
For example, the \li{ValueError} class inherits from the generic \li{Exception} class.
Thus a \li{ValueError} \textbf{is} a \li{Exception}, but a \li{Exception} is \textbf{not} a \li{ValueError}.

\begin{lstlisting}
>>> try:
...     raise ValueError("caught!")
... except Exception as e:          # A ValueError is a Exception.
...     print(e)
...
<<caught!>>                             # The exception was caught.

>>> try:
...     raise Exception("not caught!")
... except ValueError as e:         # A Exception is not a ValueError.
...     print(e)
...
<<Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
Exception: not caught!>>            # The exception wasn't caught!
\end{lstlisting}

See \url{https://docs.python.org/3/library/exceptions.html} for the complete list of built-in exceptions and the exception class hierarchy.
\end{info}

\section*{File Input and Output} % ============================================

A file object acts as an interface to a file stream, meaning it allows a program to read from or write to external files.
The built-in function \li{open()} creates a file object.
It accepts the name of the file to open and an editing mode.
The mode determines the kind of access that the user has to the file.
There are four common modes:
%
\begin{description}
\item \li{'r'}: \textbf{read}.
Open an existing file for reading.
The file must already exist, or \li{open()} raises a \li{FileNotFoundError}.
This is the default mode.
\item \li{'w'}: \textbf{write}.
Create a new file or \textbf{overwrite an existing file} (careful!) and open it for writing.
\item \li{'x'}: \textbf{write new}.
Create a new file and open it for writing.
If the file already exists, \li{open()} raises a \li{FileExistsError}.
This is a safer form of \li{'w'} because it never overwrites existing files.
\item \li{'a'}: \textbf{append}.
Open a file for writing, appending new data to the end of the file if it already exists.
\end{description}

\begin{lstlisting}
>>> myfile = open("hello_world.txt", 'r')   # Open a file for reading.
>>> print(myfile.read())                    # Print the contents of the file.
Hello,                                      # (it's a really small file.)
World!

>>> myfile.close()                          # Close the file connection.
\end{lstlisting}

\subsection*{The With Statement} % --------------------------------------------

% An opened file needs to be closed.
An \li{IOError} indicates that some input or output operation has failed.
A simple \li{try}-\li{finally} control flow can ensure that a file stream is closed safely.

The \li{with} statement provides an alternative method for safely opening and closing files.
Use \li{with open(<filename>, <mode>) as <alias>:} to create an indented block in which the file is open and available under the specified alias.
At the end of the block, the file is automatically and safely closed, even in the event of an exception.
This is the preferred file-reading method when a file only needs to be accessed briefly.

\begin{lstlisting}
>>> myfile = open("hello_world.txt", 'r')   # Open a file for reading.
>>> try:
...     contents = myfile.readlines()       # Read in the content by line.
... finally:
...     myfile.close()                      # Explicitly close the file.

# Equivalently, use a 'with' statement to take care of errors.
>>> with open("hello_world.txt", 'r') as myfile:
...    contents = myfile.readlines()
...                                         # The file is closed automatically.
\end{lstlisting}

In both cases, if the file \texttt{hello\_world.txt} does not exist in the current directory, \li{open()} raises a \li{FileNotFoundError}.
However, errors in the \li{try} or \li{with} blocks do not prevent the file from being safely closed.

\subsection*{Reading and Writing} % -------------------------------------------

Open file objects have an implicit \emph{cursor} that determines the location in the file to read from or write to.
After the entire file has been read once, either the file must be closed and reopened, or the cursor must be reset to the beginning of the file with \li{seek(0)} before it can be read again.

Some of more important file object attributes and methods are listed below.

\begin{table}[H]
\begin{tabular}{r|l}
Attribute & Description \\
\hline
\li{closed} & \li{True} if the object is closed.\\
\li{mode} & The access mode used to open the file object.\\
\li{name} & The name of the file.\\ \\
Method & Description \\
\hline
\li{close()} & Close the connection to the file.\\%Flush any delayed writes
% \li{flush()} & Flush the internal I/O buffer.\\
\li{read()} & Read a given number of bytes; with no input, read the entire file.\\
\li{readline()} & Read a line of the file, including the newline character at the end.\\
\li{readlines()} & Call \li{readline()} repeatedly and return a list of the resulting lines.\\
\li{seek()} & Move the cursor to a new position.\\
\li{tell()} & Report the current position of the cursor.\\
\li{write()} & Write a single string to the file (spaces are \textbf{not} added).\\
\li{writelines()} & Write a list of strings to the file (newline characters are \textbf{not} added).\\
\end{tabular}
\end{table}

Only strings can be written to files; to write a non-string type, first cast it as a string with \li{str()}.
Be mindful of spaces and newlines to separate the data.

\begin{lstlisting}
>>> with open("out.txt", 'w') as outfile:   # Open 'out.txt' for writing.
...     for i in range(10):
...         outfile.write(str(i**2)+' ')    # Write some strings (and spaces).
...
>>> outfile.closed                          # The file is closed automatically.
<<True>>
\end{lstlisting}

\begin{problem} % ContentFilter.__init__().
Define a class called \li{ContentFilter}.
Implement the constructor so that it accepts the name of a file to be read.
%
\begin{enumerate}
\item If the file name is invalid in any way, prompt the user for another filename using \li{input()}.
Continue prompting the user until they provide a valid filename.
\begin{lstlisting}
>>> cf1 = ContentFilter("hello_world.txt")  # File exists.
>>> cf2 = ContentFilter("not-a-file.txt")   # File doesn't exist.
<<Please enter a valid file name: >><r<still-not-a-file.txt>r>
<<Please enter a valid file name: >><r<hello_world.txt>r>
>>> cf3 = ContentFilter([1, 2, 3])          # Not even a string.
<<Please enter a valid file name: >><r<hello_world.txt>r>
\end{lstlisting}
(Hint: \li{open()} might raise a \li{FileNotFoundError}, a \li{TypeError}, or an \li{OSError}.)

\item Read the file and store its name and contents as attributes (store the contents as a single string).
Make sure the file is securely closed.
\end{enumerate}
\end{problem}

\section*{String Formatting} % ================================================

The \li{str} class has several useful methods for parsing and formatting strings.
They are particularly useful for processing data from a source file and for preparing data to be written to an external file.

\begin{table}[H]
\begin{tabular}{r|l}
Method & Returns \\ \hline
\li{count()} & The number of times a given substring occurs within the string.\\
\li{find()} & The lowest index where a given substring is found.\\
\li{isalpha()} & \li{True} if all characters in the string are alphabetic (a, b, c, \ldots).\\
\li{isdigit()} & \li{True} if all characters in the string are digits (0, 1, 2, \ldots).\\
\li{isspace()} & \li{True} if all characters in the string are whitespace (\li{" "}, \li{'\\t'}, \li{'\\n'}).\\
% \li{capitalize()} & A copy of the string with only its first character capitalized. \\
\li{join()} & The concatenation of the strings in a given iterable with a\\&specified separator between entries.\\
\li{lower()} & A copy of the string converted to lowercase. \\
\li{upper()} & A copy of the string converted to uppercase. \\
\li{replace()} & A copy of the string with occurrences of a given substring\\&replaced by a different specified substring.\\
\li{split()} & A list of segments of the string, using a given character or string \\
 & as a delimiter.\\
\li{strip()} & A copy of the string with leading and trailing whitespace removed. \\
\end{tabular}
\end{table}

The \li{join()} method translates a list of strings into a single string by concatenating the entries of the list and placing the principal string between the entries.
Conversely, \li{split()} translates the principal string into a list of substrings, with the separation determined by the a single input.

\begin{lstlisting}
# str.join() puts the string between the entries of a list.
>>> words = ["state", "of", "the", "art"]
>>> "-".join(words)
<<'state-of-the-art'>>

# str.split() creates a list out of a string, given a delimiter.
>>> "One fish\nTwo fish\nRed fish\nBlue fish\n".split('\n')
<<['One fish', 'Two fish', 'Red fish', 'Blue fish', '']>>

# If no delimiter is provided, the string is split by whitespace characters.
>>> "One fish\nTwo fish\nRed fish\nBlue fish\n".split()
<<['One', 'fish', 'Two', 'fish', 'Red', 'fish', 'Blue', 'fish']>>
\end{lstlisting}

Can you tell the difference between the following routines?

\begin{lstlisting}
>>> with open("hello_world.txt", 'r') as myfile:
...     contents = myfile.readlines()
...
>>> with open("hello_world.txt", 'r') as myfile:
...     contents = myfile.read().split('\n')
\end{lstlisting}

\begin{problem} % ContentFilter methods.
Add the following methods to the \li{ContentFilter} class for writing the contents of the original file to new files.
Each method should accept a the name of a file to write to and a keyword argument \li{mode} that specifies the file access mode, defaulting to \li{'w'}.
If \li{mode} is not \li{'w'}, \li{'x'}, or \li{'a'}, raise a \li{ValueError} with an informative message.

\begin{enumerate}
\item \li{uniform()}: write the data to the outfile with uniform case. Include an additional keyword argument \li{case} that defaults to \li{"upper"}.

If \li{case="upper"}, write the data in upper case.
If \li{case="lower"}, write the data in lower case.
If \li{case} is not one of these two values, raise a \li{ValueError}.

\item \li{reverse()}: write the data to the outfile in reverse order. Include an additional keyword argument \li{unit} that defaults to \li{"line"}.

If \li{unit="word"}, reverse the ordering of the words in each line, but write the lines in the same order as the original file.
If \li{unit="line"}, reverse the ordering of the lines, but do not change the ordering of the words on each individual line.
If \li{unit} is not one of these two values, raise a \li{ValueError}.

\item \li{transpose()}: write a ``transposed'' version of the data to the outfile.
That is, write the first word of each line of the data to the first line of the new file, the second word of each line of the data to the second line of the new file, and so on.
Viewed as a matrix of words, the rows of the input file then become the columns of the output file, and vice versa.
You may assume that there are an equal number of words on each line of the input file.
\end{enumerate}

Also implement the \li{__str__()} magic method so that printing a \li{ContentFilter} object yields the following output.
You may want to calculate these statistics in the constructor.
\begin{lstlisting}
<<Source file:              <filename>
Total characters:         <The total number of characters in the file>
Alphabetic characters:    <The number of letters>
Numerical characters:     <The number of digits>
Whitespace characters:    <The number of spaces, tabs, and newlines>
Number of lines:          <The number of lines>>>
\end{lstlisting}
(Hint: list comprehensions are \textbf{very} useful for some of these functions.
For example, what does \li{[line[::-1] for line in lines]} do?
What about \li{sum([s.isspace() for s in data])}?)

Compare your class to the following example.

\begin{lstlisting}
# cf_example1.txt
A b C
d E f
\end{lstlisting}

\begin{lstlisting}
>>> cf = ContentFilter("cf_example1.txt")
>>> cf.uniform("uniform.txt", mode='w', case="upper")
>>> cf.uniform("uniform.txt", mode='a', case="lower")
>>> cf.reverse("reverse.txt", mode='w', unit="word")
>>> cf.reverse("reverse.txt", mode='a', unit="line")
>>> cf.transpose("transpose.txt", mode='w')
\end{lstlisting}
% >>> print(cf)
% <<Source file:              cf_example1.txt
% Total characters:         12
% Alphabetic characters:    6
% Numerical characters:     0
% Whitespace characters:    6
% Number of lines:          2>> % TODO: check this.

\begin{lstlisting}
# uniform.txt
A B C
D E F
a b c
d e f
\end{lstlisting}

\begin{lstlisting}
# reverse.txt
C b A
f E d
d E f
A b C
\end{lstlisting}

\begin{lstlisting}
# transpose.txt
A d
b E
C f
\end{lstlisting}
\end{problem}

\newpage

\section*{Additional Material} % ==============================================

\subsection*{Custom Exception Classes} % --------------------------------------

Custom exceptions can be defined by writing a class that inherits from some existing exception class.
The generic \li{Exception} class is typically the parent class of choice.

\begin{lstlisting}
>>> class TooHardError(Exception):
...     pass
...
>>> raise TooHardError("This lab is impossible!")
<<Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
__main__.TooHardError: This lab is impossible!>>
\end{lstlisting}

This may seem like a trivial extension of the \li{Exception} class, but it is useful to do because the interpreter never automatically raises a \li{TooHardError}.
Any \li{TooHardError} must have originated from a hand-written \li{raise} command, making it easier to identify the exact source of the problem.

\subsection*{Chaining Exceptions} % -------------------------------------------

Sometimes, especially in large programs, it is useful raise one kind of exception just after catching another.
The two exceptions can be linked together using the \li{from} statement.
This syntax makes it possible to see where the error originated from and to ``pass it up'' to another part of the program
(\textbf{warning}: this feature was added in Python 3).

\begin{lstlisting}
>>> try:
...     raise TooHardError("This lab is impossible!")
... except TooHardError as e:
...     raise NotImplementedError("Lab is incomplete") from e
...
<<Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
__main__.TooHardError: This lab is impossible!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
NotImplementedError: Lab is incomplete>>
\end{lstlisting}

\subsection*{More String Formatting Tools} % ----------------------------------

Concatenating string values with non-string values can cumbersome and tedious.
The \li{str} class's \li{<<format()>>} method makes it easier to insert non-string values into the middle of a string.
Write the desired output in its entirety, replacing non-string values with curly braces \li{\{\}}.
Then use the \li{<<format()>>} method, entering each replaced value in order.

\begin{lstlisting}
# Join the data using string concatenation.
>>> day, month, year = 10, "June", 2017
>>> print("Is today", day, str(month) + ',', str(year) + "?")
Is today 10 June, 2017?

# Join the data using str.format().
>>> print("Is today {} {}, {}?".<<format>>(day, month, year))
Is today 10 June, 2017?
\end{lstlisting}

This method is extremely flexible and provides many convenient ways to format string output nicely.
Consider the following code for printing out a simple progress bar from within a loop.

\begin{lstlisting}
>>> iters = int(1e7)
>>> chunk = iters // 20
>>> for i in range(iters):
...     print("\r[{:<20}] i = {}".<<format>>('='*((i//chunk)+1), i),
...                                                     end='', flush=True)
...
\end{lstlisting}

Here the string \li{"\\r[\{:<20\}]"} used in conjunction with the \li{<<format()>>} method tells the cursor to go back to the beginning of the line, print an opening bracket, then print the first argument of \li{<<format()>>} left-aligned with at least $20$ total spaces before printing the closing bracket.
The comma after the print command suppresses the automatic newline character, keeping the output of each individual print statement on the same line.

Printing at each iteration dramatically slows down the progression through the loop.
How does the following code solve that problem?

\begin{lstlisting}
>>> for i in range(iters):
...     if not i % chunk:
...         print("\r[{:<20}] i = {}".format('='*((i//chunk)+1),i),
...                                                     end='', flush=True)
...
\end{lstlisting}

See \url{https://docs.python.org/3/library/string.html\#format-string-syntax} for more examples and specific syntax for using \li{str.<<format()>>}.
For a more robust progress bar printer, research the \li{tqdm} module.

\subsection*{Standard Library Modules for I/O} % ------------------------------

The standard library has other tools for input and output operations.
For details on each module, see \url{https://docs.python.org/3/library}.

\begin{table}[H]
\centering
\begin{tabular}{r|l}
    Module & Description \\ \hline
    \li{csv} & CSV (comma separated value) file writing and parsing.\\
    \li{io} & Support for file objects and \li{open()}.\\
    \li{os} & Communication with the operating system.\\
    \li{os.path} & Common path operations such as checking for file existence.\\
    \li{pickle} & Create portable serialized representations of Python objects.
\end{tabular}
\end{table}

% The CSV format (comma separated value) is a common file format for spreadsheets and grid-like data.


\chapter{Performance comparison}

This report will use the MATLAB profiler to discuss the runtime performance of MATLAB code implementations of the Block MLDU algorithm. The runtime of a piece of code is dependent on both the efficiency of the code and the hardware on which it is executed. This means that the runtime performance results will most likely vary from computer to computer. In order to verify if the achieved results on any given computer are in line with expectations, one must also be able to compare the general MATLAB related performance of the computer.\\

\noindent MATLAB provides a means to achieve this comparison by way of the \texttt{bench()} function. It runs a micro-benchmark with a variety of "typical" MATLAB workloads, namely:

\begin{itemize}
    \item LU decomposition
    \item FFT
    \item ODE with \texttt{ode45}
    \item Sparse
    \item 2D
    \item 3D
\end{itemize}

\noindent The most important benchmarks for this algorithm are LU decomposition, FFT and Sparse, because they touch on hardware performance elements that are similar to the the Block MLDU algorithm. More information can be found by running: \texttt{doc bench}\\

\noindent It should be noted that these benchmarks are far from an ideal way to compare different computers, but it can give a rough indication. It is recommended to run the benchmarks at least $10$ times, as there can be some significant performance variance.

\section{Test machine hardware specifications}

A more detailed performance comparison between computers requires the hardware specifications. The test machine specifications are provided below:

\begin{itemize}
    \item CPU: Intel Xeon E5-1650 v3 6 core 12 thread overclocked to 3.8 GHz
    \item Memory: 32 GB 4 channel DDR4 registered ECC running at 2133 MHz.
    \item Graphics: Two NVIDIA GTX Titan (Kepler) running in SLI
\end{itemize}

\newpage

\section{Test machine benchmark results}

\begin{figure}[h!]
    \includegraphics[width=\linewidth]{figures/Performance_1.eps}
    \centering
\end{figure}

\begin{figure}[h!]
    \includegraphics[width=\linewidth]{figures/Performance_2.eps}
    \centering
\end{figure}

\noindent As can bee seen, the performance of the test machine is generally very high. The second thing to note is that the performance varies quite significantly, sometimes as much as about $33 \%$. That is why this benchmark was run $20$ times, even though it only shows the results from the $10$ best runs.