\lab{Plot Customization and Matplotlib Syntax Guide}{Matplotlib Customization}
\label{mpltables}

\objective{
The documentation for Matplotlib can be a little difficult to maneuver and basic information is sometimes difficult to find.
This appendix condenses and demonstrates some of the more applicable and useful information on plot customizations.
For an introduction to Matplotlib, see lab \ref{lab:Matplotlib}.
}

\section*{Colors} % ===========================================================

By default, every plot is assigned a different color specified by the ``color cycle''.
It can be overwritten by specifying what color is desired in a few different ways.
\begin{itemize}
    \item \textbf{}
\end{itemize}
Matplotlib recognizes some basic built-in colors.

\begin{table}[H] % Basic colors.
\centering
\begin{tabular}{r|l}
    Code & Color \\
    \hline
    \li{'b'} & blue\\
    \li{'g'} & green\\
    \li{'r'} & red\\
    \li{'c'} & cyan\\
    \li{'m'} & magenta\\
    \li{'y'} & yellow\\
    \li{'k'} & black\\
    \li{'w'} & white
\end{tabular}
\end{table}

The following displays how these colors can be implemented.
The result is displayed in Figure \ref{colors}.

\lstinputlisting[style=fromfile]{colors.py}

\begin{figure}  % Colors.
    \includegraphics[width=\textwidth]{colors.pdf}
    \caption{A display of all the built-in colors.}
    \label{colors}
\end{figure}

There are many other ways to specific colors.
A popular method to access colors that are not built-in is to use a \li{RGB} tuple.
Colors can also be specified using an html hex string or its associated html color name like \li{"DarkOliveGreen", "FireBrick", "LemonChiffon", "MidnightBlue", "PapayaWhip",} or \li{"SeaGreen"}.

\section*{Window Limits} % ====================================================

You may have noticed the use of \li{plt.ylim([ymin, ymax])} in the previous code. This explicitly sets the boundary of the y-axis. Similarily, \li{plt.xlim([xmin, xmax])} can be used to set the boundary of the x-axis.
Doing both commands simultaneously is possible with the \li{plt.axis([xmin, xmax, ymin, ymax])}.
Remember that these commands must be executed after the plot.

\begin{comment} % pulled this in from matplotlib lab.
\begin{table}[H]
\centering
\begin{tabular}{r|l}
    Function & Description\\
    \hline
    % \li{annotate} & adds a commentary at a given point on the plot & annotate('text',(x,y))\\
    % \li{arrow} & draws an arrow from a given point on the plot & arrow(x,y,dx,dy)\\
    % \li{axhline} & draws a horizontal line at y from xmin to xmax & axhline(y=0, xmin=0, xmax=1)\\
    % \li{axvline} & draws a vertical line at x from ymin to ymax & axvline(x=0, ymin=0, ymax=1)\\
    % \li{axhspan} & draws a rectangle from xmin to xmax and ymin to ymax, if no xmin and xmax are given it goes across the plot & axhspan(ymin, ymax, xmin=0, xmax=1)\\
    % \li{axvspan} & draws a rectangle from ymin to ymax and xmin to xmax, if no ymin and ymax are given it goes across the entire plot & axvspan(xmin, xmax, ymin=0, ymin=1)\\
    % \li{grid()} & Add gridlines\\
    \li{legend()} & Place a legend in the plot\\
    % \li{text()} & Add text at a given position on the plot\\
    \li{title()} & Add a title to the plot\\
    \li{xlim()} & Set the limits of the $x$-axis\\
    \li{ylim()} & Set the limits of the $y$-axis\\
    % \li{xticks()} & set the location of the tick marks on the x axis, returns current locations if no arguments are given\\
    % \li{yticks()} & set the location of the tick marks on the y axis, returns current locations if no arguments are given\\
    \li{xlabel()} & Add a label to the $x$-axis\\
    \li{ylabel()} & Add a label to the $y$-axis
\end{tabular}
% \caption{Some Functions to Set Plotting Options}
% \label{mpl:useful_functions}
\end{table}
\end{comment}

% BAD! plt.axis("tight") has a bug that messes with OS X on Macs.
% Another popular command is \li{plt.axis("tight")} which adjusts the axes so all data is visible and centered.

\section*{Lines} % ============================================================

\subsection*{Thickness} % -----------------------------------------------------

You may have noticed that the width of the lines above seemed thin considering we wanted to inspect the line color. \li{linewidth} is a keyword argument that is defaulted to be \li{None} but can be given any real number to adjust the line width.

The following displays how \li{linewidth} is implemented.
It is displayed in Figure \ref{linewidth}.

\lstinputlisting[style=fromfile]{linewidth.py}

\begin{figure} % Line width results
\includegraphics[width=\textwidth]{linewidth.pdf}
\caption{plot of varying linewidths.}
\label{linewidth}
\end{figure}

% Talk about markersize and demonstrate markers of varying sizes.

\subsection*{Style} % ---------------------------------------------------------

By default, plots are drawn with a solid line.
The following are accepted format string characters to indicate line style.

\begin{table}[H] % Line style
\centering
\begin{tabular}{c|l}
    character & description \\
    \hline
    - & solid line style \\
    -{}- & dashed line style\\
    -. & dash-dot line style \\
    : & dotted line style \\
    . & point marker \\
    , & pixel marker \\
    o & circle marker \\
    v & triangle\_down marker \\
    \^{} & triangle\_up marker \\
    $<$ & triangle\_left marker \\
    $>$ & triangle\_right marker \\
    1 & tri\_down marker \\
    2 & tri\_up marker \\
    3 & tri\_left marker \\
    4 & tri\_right marker \\
    s & square marker \\
    p & pentagon marker \\
    * & star marker \\
    h & hexagon1 marker \\
    H & hexagon2 marker \\
    + & plus marker \\
    x & x marker \\
    D & diamond marker \\
    d & thin\_diamond marker \\
    $|$ & vline marker \\
    \_{} & hline marker \\
\end{tabular}
\end{table}

The following displays how \li{linestyle} can be implemented.
It is displayed in Figure \ref{linestyle}.

\lstinputlisting[style=fromfile]{linestyle.py}

\begin{figure} % Line styles
\includegraphics[width=\textwidth]{linestyle.pdf}
\caption{plot of varying linestyles.}
\label{linestyle}
\end{figure}

\section*{Text} % =============================================================

It is also possible to add text to your plots.
To label your axes, the \li{plt.xlabel()} and the \li{plt.ylabel()} can both be used.
The function \li{plt.title()} will add a title to a plot.
If you are working with subplots, this command will add a title to the subplot you are currently modifying.
To add a title above the entire figure, use \li{plt.suptitle()}.

All of the \li{text()} commands can be customized with \li{fontsize} and \li{color} keyword arguments.

We can add these elements to our previous example.
It is displayed in Figure \ref{text}.

\lstinputlisting[style=fromfile]{text.py}

\begin{figure}
\includegraphics[width=\textwidth]{text.pdf}
\caption{plot of varying linestyles using text labels.}
\label{text}
\end{figure}

\begin{comment} % PLOTTING COMMANDS
\begin{table} % Different plotting commands. Should be part of the appendix.
\centering
\begin{tabular}{|l|p{7cm}|p{3cm}|}
    \hline
    Function & Description & Usage\\
    \hline
    \li{bar} & makes a bar graph & bar(left,height)\\
    \li{barh} & makes a horizontal bar graph & barh(bottom,width)\\
    \li{fill} & plots lines with shading under the curve & fill(x,y)\\
    \li{fill\_between} & plots lines with shading between two given y
    values & fill\_ between(x,y1, y2=0)\\
    \li{hist} & plots a histogram from data & hist(data)\\
    \li{pie} & makes a pie chart & pie(x)\\
    \li{plot} & plots lines and data on standard axes & plot(x,y)\\
    \li{polar} & plots lines and data on polar axes & polar(theta,r)\\
    \li{loglog} & plots lines and data on logarithmic x and y axes &
    loglog(x,y)\\
    \li{scatter} & plots data, has more options for scatter plots than
    the plot function & scatter(x,y)\\
    \li{semilogx} & plots lines and data with a log scaled x axis &
    semilogx(x,y)\\
    \li{semilogy} & plots lines and data with a log scaled y axis &
    semilogy(x,y)\\
    \li{specgram} & makes a spectogram from data & specgram(x)\\
    \li{spy} & plots the sparsity pattern of a 2D array & spy(Z)\\
    \li{triplot} & plots triangulation between given points &
    triplot(x,y)\\
\end{tabular}
\end{center}
\caption{Some basic functions in Matplotlib.}
\label{mpl:basics}
\end{table}
\end{comment}

See \url{http://matplotlib.org} for Matplotlib documentation.


%% -*- coding:utf-8 -*-
\chapter*{Introduction}

You just looked at yet another introduction to Category Theory. The
subject mostly consists of a lot of definitions that are related each
others and I wrote the book to collect all of them in one
place to be easy checked and updated in future when I decide to refresh
my knowledge about the field of math. Therefore the book was written mostly
for my category theory studying
purposes but I will appreciate if somebody else find it useful.

The topics(chapters) cover the base definitions
(\mynameref{def:object}, \mynameref{def:morphism} and
\mynameref{def:category}), \mynameref{def:functor},
\mynameref{def:nt}, \mynameref{def:monad} and also include important
results from the category theory such as Yoneda's lemma (see
\cref{sec:yoneda}) and Curry-Howard-Lambek correspondence (see
\cref{sec:curry_howard_lambek}).

There are a lot of
examples in each chapter. The examples cover different category
theory application areas. I assume that the reader is familiar with
the corresponding area and the example(s) can be passed if not. I.e.
anyone can choose the suitable example(s) for (s)he.

The most important examples are related to the set theory. The set
theory and category theory are very close related. Each one can be
considered as an alternative view to another one.

There are a lot of examples from programming languages which include
Haskell, Scala, C++. The source files for programming languages
examples (Haskell, C++, Scala) can be found on github repository
\cite{bib:github:ivanmurashko}.

The examples from physics are related to quantum mechanics that is the
most known for me. For the examples I am inspired by the Bob Coecke
article \cite{bib:arxiv:Bob_Coecke_2008}.

The text is distributed under \textbf{Creative Common Public License}
(see the text of the license at the end of the book)
i.e. any reader has right to copy, store, modify, distribute or build
upon the book until the original author is pointed in the derivative
products.


\documentclass[main.tex]{subfiles}
\begin{document}

\marginpar{Wednesday\\ 2020-11-25, \\ compiled \\ \today}

We wanted to estimate the magnetic field \(B\) and characteristic age \(\tau _c\) of the NS: we have found that \(B_P \propto \sqrt{P \dot{P}}\) and \(\tau _c \propto P / \dot{P}\).

Now we will define the \textbf{braking index}: recall that the evolution of the angular velocity can be written as \(\dot{\Omega} = - k \Omega^3\).
This can be generalized to a law of the type \(\dot{\Omega} = -k \Omega^{n}\); the spin-down due to the emission of GWs is similarly written with \(n = 6\).

Taking the derivative of this law, we find
%
\begin{align}
\ddot{\Omega} &= - kn \dot{\Omega} \Omega^{n-1}  \\
&= \frac{n}{\Omega } \dot{\Omega} (-k \Omega^{n}) = n \frac{\dot{\Omega}^2}{\Omega }   \\
n &= \frac{\ddot{\Omega} \Omega }{\dot{\Omega}^2}
\,.
\end{align}

By measuring \(\Omega \), \(\dot{\Omega}\) and \(\ddot{\Omega}\) we can estimate \(n\), which will allow us to find the braking index \(n\).
However, as the order of the derivative increases the difficulty of the measurement increases. The index \(n\) is known for around 10 NSs, and it was measured to be \(n < 3\), which is evidence against magnetorotational losses as the main cause of the spin-down.

We make a plot, with  \(\log P\) on the \(x\) axis and \(\log \dot{P}\) on the \(y\) axis: this is the \(P-\dot{P}\) diagram.
We can scatter-plot known NSs in it: see figure \ref{fig:p-pdot-diagram}.

\begin{figure}[ht]
\centering
\includegraphics[width=.8\textwidth]{figures/p-pdot-diagram}
\caption{An example \(P-\dot{P}\) diagram from \textcite[]{beckerAutonomousSpacecraftNavigation2013}.}
\label{fig:p-pdot-diagram}
\end{figure}

Equal-\(B_p\) lines and equal-\(\tau _c\) lines can be drawn:
the first are in the form \(\log P = - \log \dot{P} + k\), the second are in the form \(\log P = \log \dot{P} + k\).
We can track the motion of NSs through this diagram: they will move down and to the right as they evolve, with a slope of \(2 - n\), since
%
\begin{align}
\dv{t}( \frac{1}{P}) \propto - \frac{1}{P^{n}} \implies \dot{P} \propto P^{2-n}
\,.
\end{align}

\section{Interior structure of NSs}

Are they actually made of neutrons? Mostly, yes, but not exclusively.

A free neutron will \(\beta \)-decay:
%
\begin{align}
\ce{n} \to e^{-} + \ce{p} + \overline{\nu}
\,
\end{align}
%
within around \SI{15}{min}. Neutrons are also formed through inverse \(\beta \)-decay (also called electron capture):
%
\begin{align}
e^{-} + \ce{p} \to \ce{n} + \nu
\,.
\end{align}

This process, however, is endothermic: \(m_e + m_p < m_n \).
Specifically, \(Q = (m_n - m_p)c^2 \approx 2.54 m_e c^2\approx \SI{1.3}{MeV}\) is the energy the electron must have in order to achieve the reaction.

The thermal energy is definitely not that high inside a NS!
In order to describe what goes on, we will make some assumptions:
\begin{enumerate}
    \item the interior is only made of neutrons, protons and electrons (the \(npe\) model) (this is not really true, but it will allow us to illustrate the point);
    \item the number densities of these species are similar: \(n_e \approx n_p \approx n_n\).
\end{enumerate}

Matter inside a neutron star is highly degenerate: the energy of each electron is much smaller than the Fermi energy.
The Fermi energy can be written as \(E_F \approx m_e c^2 \sqrt{1 + x_F^2}\), where \(x_F = p_F / m_ec\), and
%
\begin{align}
p_F = \sqrt[3]{\frac{3 h^3 n_e}{8 \pi }}
\,.
\end{align}

Therefore,
%
\begin{align}
x_F = \sqrt[3]{\frac{3 h^3 n_e}{8 \pi }} \frac{1}{m_e c}
\approx \num{e-2} \sqrt[3]{\frac{\rho}{\mu }}
\,.
\end{align}

If we want the electrons to be relativistic, we require \(x_F \gtrsim 1\), which means \(\rho / \mu _e \gtrsim \SI{e6}{g / cm^3}\).

If the Fermi energy is as high as the \(Q\)-value of the reaction, \(E_F \gtrsim \num{2.54} m_e c^2\), then we must have
%
\begin{align}
\sqrt{1 + x_F^2} \gtrsim \num{2.54} \implies x_F \gtrsim \num{2.3}
\,,
\end{align}
%
which can be solved in terms of the rest mass matter density: we get
%
\begin{align}
\rho \gtrsim \SI{1.7e7}{g / cm^3} \times \mu _e
\,,
\end{align}
%
which is high in absolute terms, but rather low for a NS, since they typically will have densities several orders of magnitude higher.

At these densities, not only electrons, but protons and neutrons are degenerate as well, however these more massive particles are not relativistic.
In fact, \(x_F \propto n / m\) holds across particle species: protons and neutrons will have \(x_F^{(p, n)} \sim x_F^{(e)} / 2000 \), since they are \(\sim 2000\) times more massive.

The densities needed for the neutrons and protons to be relativistic might be achieved in the very core of the star, but they are typically not.

So, the electrons are relativistic and the reaction can happen back and forth at equilibrium: this means that we can equate their chemical potentials,
%
\begin{align}
\mu _p + \mu _e = \mu _n (+ \mu _\nu )
\,,
\end{align}
%
where the neutrinos can be neglected since they generally fly away after a reaction, unable to interact further.
Under complete degeneracy we have \(\mu = E_F\).
Then, we can write
%
\begin{align}
m_p c^2 \sqrt{1 + x_p^2} +
m_e c^2 \sqrt{1 + x_e^2} =
m_n c^2 \sqrt{1 + x_n^2}
\,,
\end{align}
%
but by charge neutrality we must have \(n_e = n_p\).
This is to be complemented with \(n \propto p_F^3\) and \(x = p_F / mc\): thus, \(x_e / x_p = m_p / m_e\).

Working through the algebra,\footnote{The passages to follow are: eliminate \(x_e\) in the previous equation with the relation \(x_e / x_p = m_p / m_e\), approximate \(\sqrt{1 + x^2}\) as \(x\) for electrons and as \(1 + x^2/2\) for protons and neutrons (thus assuming that they are nonrelativistic); solve for \(x_n / x_p\).}  we find
%
\begin{align}
R_{np} = \frac{n_n}{n_p} = x_n^3 \qty[
    \frac{4 (1 + x_n^2)}{x_n^4 + \frac{4 Q x_n^3}{m_n} + 4\frac{(Q^2 -m_e^2)}{m_n^2}}
]^{3/2}
= \frac{x_n^3}{x_p^3}
\,.
\end{align}

This is plotted on a log-log scale as a function of \(x_n\) in figure \ref{fig:neutron-proton-ratio}.

The density is given by
%
\begin{align}
\rho &= m_e n_e + m_n n_n + m_p n_p \approx m (n_p + n_n)
\marginnote{We define \(m = m_n \approx m_p\).}
\\
&\approx m \frac{8 \pi }{3} \qty(\frac{mc}{h})^3
\qty[x_p^3 + x_n^3]  \\
&= m \frac{8 \pi }{3} \qty(\frac{mc}{h})^3
x_n^3 \qty[ \frac{1}{R_{np} (x_n)} + 1]
\,.
\end{align}
%

This yields a parametric expression for \(R_{np}\) in terms of \(\rho \); if \(\rho \) is very small then \(n_n \ll n_p\), approaching \(n_n \sim 0\), around \(\rho \approx \SI{e12}{g / cm^3}\) there is a maximum of \(R_{np}\approx \num{e4}\), and it seems to approach \(R_{np} \sim 10 \) asymptotically. It is shown in figure \ref{fig:neutron-proton-ratio-density}.

Taking the limit of the expression for  \(\rho \) as \(n_n \to 0\) we find
%
\begin{align}
\rho &\approx \frac{m 8 \pi }{3} \qty(\frac{mc}{h})^3 \qty[ \frac{Q^2 - m_e^2}{m^2}]^{3/2}  \\
&\approx \frac{8 \pi m}{3} \qty( \frac{c}{h}\sqrt{Q^2 - m_e^2})^3
\,.
\end{align}

This is around \SI{1.2e7}{g / cm^3}. It is expected, since below this density there will be no neutrons anymore.
If we plot \(\rho (x_n)\), we see a plateau until about \(x_n \sim \num{e-2}\), after which we start to see \(\rho \propto x_n^3\).

\end{document}


%%
%% Automatically generated file from DocOnce source
%% (https://github.com/hplgit/doconce/)
%%
%%

%-------------------- begin preamble ----------------------

\documentclass[%
oneside,                 % oneside: electronic viewing, twoside: printing
final,                   % draft: marks overfull hboxes, figures with paths
10pt]{article}

\listfiles               %  print all files needed to compile this document

\usepackage{relsize,makeidx,color,setspace,amsmath,amsfonts,amssymb}
\usepackage[table]{xcolor}
\usepackage{bm,ltablex,microtype}

\usepackage[pdftex]{graphicx}

\usepackage[T1]{fontenc}
%\usepackage[latin1]{inputenc}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}

\usepackage{lmodern}         % Latin Modern fonts derived from Computer Modern

% Hyperlinks in PDF:
\definecolor{linkcolor}{rgb}{0,0,0.4}
\usepackage{hyperref}
\hypersetup{
    breaklinks=true,
    colorlinks=true,
    linkcolor=linkcolor,
    urlcolor=linkcolor,
    citecolor=black,
    filecolor=black,
    %filecolor=blue,
    pdfmenubar=true,
    pdftoolbar=true,
    bookmarksdepth=3   % Uncomment (and tweak) for PDF bookmarks with more levels than the TOC
    }
%\hyperbaseurl{}   % hyperlinks are relative to this root

\setcounter{tocdepth}{2}  % levels in table of contents

% --- fancyhdr package for fancy headers ---
\usepackage{fancyhdr}
\fancyhf{} % sets both header and footer to nothing
\renewcommand{\headrulewidth}{0pt}
\fancyfoot[LE,RO]{\thepage}
% Ensure copyright on titlepage (article style) and chapter pages (book style)
\fancypagestyle{plain}{
  \fancyhf{}
  \fancyfoot[C]{{\footnotesize \copyright\ 1999-2019, "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license}}
%  \renewcommand{\footrulewidth}{0mm}
  \renewcommand{\headrulewidth}{0mm}
}
% Ensure copyright on titlepages with \thispagestyle{empty}
\fancypagestyle{empty}{
  \fancyhf{}
  \fancyfoot[C]{{\footnotesize \copyright\ 1999-2019, "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license}}
  \renewcommand{\footrulewidth}{0mm}
  \renewcommand{\headrulewidth}{0mm}
}

\pagestyle{fancy}

% prevent orhpans and widows
\clubpenalty = 10000
\widowpenalty = 10000

% --- end of standard preamble for documents ---

% insert custom LaTeX commands...

\raggedbottom
\makeindex
\usepackage[totoc]{idxlayout}   % for index in the toc
\usepackage[nottoc]{tocbibind}  % for references/bibliography in the toc

%-------------------- end preamble ----------------------

\begin{document}

% matching end for #ifdef PREAMBLE

\newcommand{\exercisesection}[1]{\subsection*{#1}}

% ------------------- main content ----------------------

% ----------------- title -------------------------

\thispagestyle{empty}

\begin{center}
{\LARGE\bf
\begin{spacing}{1.25}
Project 2 on Machine Learning, deadline November 8
\end{spacing}
}
\end{center}

% ----------------- author(s) -------------------------

\begin{center}
{\bf \href{{http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html}}{Data Analysis and Machine Learning FYS-STK3155/FYS4155}}
\end{center}

    \begin{center}
% List of all institutions:
\centerline{{\small Department of Physics, University of Oslo, Norway}}
\end{center}

% ----------------- end author(s) -------------------------

% --- begin date ---
\begin{center}
Oct 17, 2019
\end{center}
% --- end date ---

\vspace{1cm}

\subsection*{Classification and Regression, from linear and logistic regression to neural networks}

The main aim of this project is to study both classification and
regression problems, where we can reuse the regression algorithms studied
in project 1. We will include logistic regresion for classification
problems and write our own multilayer perceptron code for studying
both regression and classification problems.  The codes developed in
project 1, including bootstrap and/or cross-validation as well as the
computation of the mean-squared error and/or the $R2$ or the accuracy score (classification problems) functions can
also be utilized in the present analysis.

The data sets that we propose here are (the default sets)

\begin{itemize}
\item Regression (fitting a continuous function). In this part you will need to bring up your results from project 1 and compare these with what you get from you Neural Network code to be developed here. The data sets could be
\begin{enumerate}

 \item Either the Franke function or the terrain data from project 1, or data sets your propose.

\end{enumerate}

\noindent
\item Classification. Here you will also need to develop a Logistic regression code that you will use to compare with the Neural Network code. The data set we propose are
\begin{enumerate}

 \item The credit card data set from \href{{https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients}}{UCI}. This data set links to a \href{{https://bradzzz.gitbooks.io/ga-seattle-dsi/content/dsi/dsi_05_classification_databases/2.1-lesson/assets/datasets/DefaultCreditCardClients_yeh_2009.pdf}}{recent scientific article}.  Furthermore, in the lecture slides on \href{{https://compphysics.github.io/MachineLearning/doc/pub/LogReg/html/LogReg.html}}{Logistic Regression}, you will find an example code for the Credit Card data. You could use this code as an example on how to read the data and use \textbf{Scikit-Learn} to run a classification problem.
\end{enumerate}

\noindent
\end{itemize}

\noindent
However, if you would like to study other data sets, feel free to
propose other sets. What we listed here are mere suggestions from our
side. If you opt for another data set, consider using a set which
has been studied in the scientific literature. This makes it easier
for you to compare and analyze your results. It is also an essential
elements of the scientific discussion.

In particular, when developing your own Logistic Regression code for classification problems, the so-called Wisconsin Cancer data (which is a binary problem, benign or malignant tumors) may be studied. You find more information about this at the \href{{https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_breast_cancer.html}}{Scikit-Learn site} or at the \href{{https://archive.ics.uci.edu/ml/datasets/breast+cancer+wisconsin+(original)}}{University of California at Irvine}. The \href{{https://compphysics.github.io/MachineLearning/doc/pub/DimRed/html/DimRed.html}}{lecture slides on dimensionality reduction have several code examples on this data set}.

\paragraph{Part a): Write your Logistic Regression code, first step.}
If you opt for the credit card data, your first task is to familiarize yourself with the data set and the scientific article.
We recommend also that you study the code example in the \href{{https://compphysics.github.io/MachineLearning/doc/pub/LogReg/html/LogReg.html}}{Logistic Regression}.

Write the part of the code which reads in the data and sets up the relevant data sets.

\paragraph{Part b): Write your Logistic Regression code, second step.}
Now you should write your Logistic Regression code with the aim to
reproduce the Logistic Regression analysis of the \href{{https://bradzzz.gitbooks.io/ga-seattle-dsi/content/dsi/dsi_05_classification_databases/2.1-lesson/assets/datasets/DefaultCreditCardClients_yeh_2009.pdf}}{scientific
article}.

Define your cost function and the design matrix before you start writing your code.

In order to find the optimal parameters of your logistic regressor you
should include a gradient descent solver, as discussed in the
\href{{https://compphysics.github.io/MachineLearning/doc/pub/Splines/html/Splines-bs.html}}{gradient descent
lectures}.
Since we don't have so many data points, you may just code the
standard gradient descent with a given learning rate, or even attempt
to use the Newton-Raphson method.  Alternatively, it may be useful for
the next part on neural networks to implement a stochastic gradient
descent with and without mini-batches. Stochastic gradient with
mini-batches may give the best results. You could finally compare your
code with the output from \textbf{scikit-learn}'s toolbox for optimization
methods applied to logistic regression.

To measure the performance of our classification problem we use the
so-called \emph{accuracy} score.  The accuracy is as you would expect just
the number of correctly guessed targets $t_i$ divided by the total
number of targets. A perfect classifier will have an accuracy score of
$1$.

\[
\text{Accuracy} = \frac{\sum_{i=1}^n I(t_i = y_i)}{n} ,
\]

where $I$ is the indicator function, $1$ if $t_i = y_i$ and $0$
otherwise if we have a binary classifcation problem. Here $t_i$
represents the target and $y_i$ the outputs of your Logistic
Regression code.

You can compare your own results with those obtained using
\textbf{scikit-learn}.

As stated in the introduction, it can also be useful to study other datasets. In particular, when developing your own	Logistic Regression code for classification problems, the so-called \
Wisconsin Cancer data (which is a binary problem, benign or malignant tumors) may be studied. You find more \
information about this at the \href{{https://scikit-learn.org/stable/modules/generated/sklearn\
.datasets.load_breast_cancer.html}}{Scikit-Learn site} or at the "University	of California at Irvine":"https://archive.ics.uci.e\
du/ml/datasets/breast+cancer+wisconsin+(original)". The	\href{{https://compphysics.github.io/MachineLearning/doc/pub/DimRed/html/DimR\
ed.html}}{lecture slides on dimensionality reduction have sev\
eral code examples on this data set}.

\paragraph{Part c): Writing your own Neural Network code.}
Your aim now, and this is the central part of this project, is to
write to your own Feed Forward Neural Network  code implementing the back
propagation algorithm discussed in the \href{{https://compphysics.github.io/MachineLearning/doc/pub/NeuralNet/html/NeuralNet-bs.html}}{lecture
slides}. We
start with the Logistic Regression  case and the data set discussed in parts a) and b) but train
now the network to find the optimal weights and biases. You are free
to use the codes in the above lecture slides as starting points.

Discuss again your choice of cost function.

Train your network and compare the results with those from your Logistic  Regression code.
You should test your results against a similar code using \textbf{Scikit-Learn} (see the examples in the above lecture notes) or \textbf{tensorflow/keras}.

Comment your results and give a critical discussion of the results
obtained with the Logistic Regression code and your own Neural Network
code.  Make an analysis of the regularization parameters and the learning rates employed to find the optimal accurary score.

A useful reference on the back progagation algorithm is \href{{http://neuralnetworksanddeeplearning.com/}}{Nielsen's
book}. It is an excellent
read.

\paragraph{Part d): Regression analysis using neural networks.}
Here we will change the cost function for our neural network code
developed in part c) in order to perform a regression (fitting a
function or some data set) analysis. As stated above, our default data
sets could be either the Franke function or the terrain data from
project 1.

Compare you results from the neural network regression analysis (with a discussion of learning rates and regularization parameters)
with those you obtained in project 1. Alternatively, if you opt for other data sets, you would need to run your standard ordinary least squares, Ridge and Lasso calculations using your codes from project 1.

Again, we strongly recommend that you compare your own neural Network code and results against a similar code using \textbf{Scikit-Learn} (see the examples in the above lecture notes) or \textbf{tensorflow/keras}.

\paragraph{Part e) Critical evaluation of the various algorithms.}
After all these glorious calculations, you should now summarize the
various algorithms and come with a critical evaluation of their pros
and cons. Which algorithm works best for the regression case and which
is best for the classification case. These codes can also be part of
your final project 3, but now applied to other data sets.

\subsection*{Background literature}

\begin{enumerate}
\item The text of Michael Nielsen is highly recommended, see \href{{http://neuralnetworksanddeeplearning.com/}}{Nielsen's book}. It is an excellent read.

\item The textbook of \href{{https://www.springer.com/gp/book/9780387848570}}{Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer}, chapters 3 and 7 are the most relevant ones for the analysis here.

\item \href{{https://arxiv.org/abs/1803.08823}}{Mehta et al, arXiv 1803.08823}, \emph{A high-bias, low-variance introduction to Machine Learning for physicists}, ArXiv:1803.08823.
\end{enumerate}

\noindent
\subsection*{Introduction to numerical projects}

Here follows a brief recipe and recommendation on how to write a report for each
project.

\begin{itemize}
  \item Give a short description of the nature of the problem and the eventual  numerical methods you have used.

  \item Describe the algorithm you have used and/or developed. Here you may find it convenient to use pseudocoding. In many cases you can describe the algorithm in the program itself.

  \item Include the source code of your program. Comment your program properly.

  \item If possible, try to find analytic solutions, or known limits in order to test your program when developing the code.

  \item Include your results either in figure form or in a table. Remember to        label your results. All tables and figures should have relevant captions        and labels on the axes.

  \item Try to evaluate the reliabilty and numerical stability/precision of your results. If possible, include a qualitative and/or quantitative discussion of the numerical stability, eventual loss of precision etc.

  \item Try to give an interpretation of you results in your answers to  the problems.

  \item Critique: if possible include your comments and reflections about the  exercise, whether you felt you learnt something, ideas for improvements and  other thoughts you've made when solving the exercise. We wish to keep this course at the interactive level and your comments can help us improve it.

  \item Try to establish a practice where you log your work at the  computerlab. You may find such a logbook very handy at later stages in your work, especially when you don't properly remember  what a previous test version  of your program did. Here you could also record  the time spent on solving the exercise, various algorithms you may have tested or other topics which you feel worthy of mentioning.
\end{itemize}

\noindent
\subsection*{Format for electronic delivery of report and programs}

The preferred format for the report is a PDF file. You can also use DOC or postscript formats or as an ipython notebook file.  As programming language we prefer that you choose between C/C++, Fortran2008 or Python. The following prescription should be followed when preparing the report:

\begin{itemize}
  \item Use Devilry to hand in your projects, log in  at  \href{{http://devilry.ifi.uio.no}}{\nolinkurl{http://devilry.ifi.uio.no}} with your normal UiO username and password and choose either 'fysstk3155' or 'fysstk4155'. There you can load up the files within the deadline.

  \item Upload \textbf{only} the report file!  For the source code file(s) you have developed please provide us with your link to your github domain.  The report file should include all of your discussions and a list of the codes you have developed.  Do not include library files which are available at the course homepage, unless you have made specific changes to them.

  \item In your git repository, please include a folder which contains selected results. These can be in the form of output from your code for a selected set of runs and input parameters.

  \item In this and all later projects, you should include tests (for example unit tests) of your code(s).

  \item Comments  from us on your projects, approval or not, corrections to be made  etc can be found under your Devilry domain and are only visible to you and the teachers of the course.
\end{itemize}

\noindent
Finally,
we encourage you to collaborate. Optimal working groups consist of
2-3 students. You can then hand in a common report.

\subsection*{Software and needed installations}

If you have Python installed (we recommend Python3) and you feel pretty familiar with installing different packages,
we recommend that you install the following Python packages via \textbf{pip} as
\begin{enumerate}
\item pip install numpy scipy matplotlib ipython scikit-learn tensorflow sympy pandas pillow
\end{enumerate}

\noindent
For Python3, replace \textbf{pip} with \textbf{pip3}.

See below for a discussion of \textbf{tensorflow} and \textbf{scikit-learn}.

For OSX users we recommend also, after having installed Xcode, to install \textbf{brew}. Brew allows
for a seamless installation of additional software via for example
\begin{enumerate}
\item brew install python3
\end{enumerate}

\noindent
For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution
you can use \textbf{pip} as well and simply install Python as
\begin{enumerate}
\item sudo apt-get install python3  (or python for python2.7)
\end{enumerate}

\noindent
etc etc.

If you don't want to install various Python packages with their dependencies separately, we recommend two widely used distrubutions which set up  all relevant dependencies for Python, namely
\begin{enumerate}
\item \href{{https://docs.anaconda.com/}}{Anaconda} Anaconda is an open source distribution of the Python and R programming languages for large-scale data processing, predictive analytics, and scientific computing, that aims to simplify package management and deployment. Package versions are managed by the package management system \textbf{conda}

\item \href{{https://www.enthought.com/product/canopy/}}{Enthought canopy}  is a Python distribution for scientific and analytic computing distribution and analysis environment, available for free and under a commercial license.
\end{enumerate}

\noindent
Popular software packages written in Python for ML are

\begin{itemize}
\item \href{{http://scikit-learn.org/stable/}}{Scikit-learn},

\item \href{{https://www.tensorflow.org/}}{Tensorflow},

\item \href{{http://pytorch.org/}}{PyTorch} and

\item \href{{https://keras.io/}}{Keras}.
\end{itemize}

\noindent
These are all freely available at their respective GitHub sites. They
encompass communities of developers in the thousands or more. And the number
of code developers and contributors keeps increasing.

\begin{enumerate}
\item For project 3, you should feel free to use your own codes from projects 1 and 2, eventually write your own for SVMs and/or Decision trees and random forests' or use the available functionality of \textbf{scikit-learn}, \textbf{tensorflow}, etc.

\item The estimates you used and tested in projects 1 and 2 should also be included, that is the $R2$-score, \textbf{MSE}, cross-validation and/or bootstrap if these are relevant.

\item If possible, you should link the data sets with exisiting research and analyses thereof. Scientific articles which have used Machine Learning algorithms to analyze the data are highly welcome. Perhaps you can improve previous analyses and even publish a new article?

\item A critical assessment of the methods with ditto perspectives and recommendations is also something you need to include.
\end{enumerate}

\noindent

% ------------------- end of main content ---------------

\end{document}


\documentclass[bigger]{beamer}

\input{header-beam} % change to header-handout for handouts

% ====================
\title[Lecture 13]{Logic I F13 Lecture 13}
\date{October 24, 2013}
% ====================

\include{header}

\setlength{\fitchprfwidth}{5em}
\setlength{\fitchargwidth}{25em}

\section{Review}

\subsec{Properties and Quantifiers}{

\bit
\item Wff $A(x)$ expresses a \emph{property}
\item The sentence
\[
\forall x\, A(x)
\]
is true iff
\emph{every object} in the domain has the property expressed by A(x),
\item The sentence
\[\sf
\exists x\, A(x)
\]
is true iff
\emph{at least one object} in the domain has the property expressed by A(x),
\eit

}

\subsec{The Aristotelian Forms}{

\bit
\item ``Some A is B''
\[
\sf \exists x(A(x) \land B(x)
\]
\item ``Every A is B'' (``All A are B'')
\[\sf \forall x(A(x) \to B(x))\]
\item ``No A is B''
\begin{align*}
\sf \forall x(A(x) & \to \sf\lnot B(x))\\
\sf \lnot\exists x(A(x) & \sf\land B(x))
\end{align*}
\item ``Some A is not B'' (``Not all A are B'')
\begin{align*}
\sf \exists x(A(x) & \land \sf\lnot B(x)) \\
\sf \lnot\forall x(A(x) & \to \sf B(x))
\end{align*}

\eit

}

\subsec{Only, Except}{

\bit
\item ``Only A are B''
\begin{align*}
\sf \forall x(B(x) & \sf\to A(x))\\
\sf \lnot\exists x(\lnot A(x) & \sf\land B(x))
\end{align*}
\item ``Only a is large''
\[
\sf \forall x(Large(x) \iff x = a)
\]
\eit
}

\subsec{Existential Import and Implicature}{

\bit
\item ``All A are B'' does not imply ``There are A''
\item $\sf \forall x(A(x) \impl B(x))$ is true if nothing satisfies $A(x)$\\
(vacuous quantification)
\item ``Some A are B'' does not imply ``Not all A are B''
\item \dots but they do implicate it
\eit

}

\subsec{Quantifiers and Function Symbols}{

\bits
\item ``The leftmost block in the same row as any cube is small''
\bits
\item ``For every cube, the leftmost block in the same row as it is small''
\item
\[
\sf \forall x(Cube(x) \to Small(lm(x)))
\]
\eit
\item ``Some multiples of 3 are even, and some aren't''
\bits
\item Note: every multiple of 3 is the multiple of 3 \emph{by some number}
\item ``There is a number such that the multiple of 3 by it is even''
\item
$
\sf \exists x\,Even(x \times (1 + (1+1)))
$
\item
$
\sf \exists x\, Even(x \times (1 + (1+1))) \land \exists x\, \lnot Even(x \times (1 + (1+1)))
$
\eit
\eit
}

\section{Arguments involving Quantifiers}

\subsec{Example}{

\ben
\item If an action x is morally wrong then
A is blameworthy for freely doing x.
\item If x is rationally optimal (there is no action which A has
  reason to think there is more reason for A to do), then A is not
  blameworthy for freely doing x.
\item Therefore, if x is morally wrong, then
x is not rationally optimal.
(Principle of moral categoricity.)
\een

\begin{raggedleft}
\small(John Skorupski, \textit{Ethical Explorations}, 2000 (\href{http://books.google.ca/books?id=bxIzZYqRZdwC&lpg=PP1&pg=PA170\#v=onepage&q&f=false}{link} \href{http://peasoup.typepad.com/peasoup/2008/07/logic-help.html}{link}))
 \end{raggedleft}

}

\subsec{Example}{

\fitcharg{
\nline{\forall x(Wrong(x) \to BlameFreelyDoing(A, x))}\\
\nline{\forall x(ROpt(x) \to \lnot BlameFreelyDoing(A, x))}}{
\nline{\forall x(Wrong(x) \to \lnot ROpt(x))}\\
}

\bigskip

\fitcharg{\tline{All A are B}\\
\tline{No C are B \qquad ($\Leftrightarrow$ No B are C)}}{
\tline{No A are C}}
}

\subsec{FOL and the Truth Table Method}{

\bit
\item Some arguments valid only in virtue of the meanings of quantifiers
\item Some arguments valid independently of the meanings of quantifiers, e.g.,
\fitcharg{\tline{If there are cubes, then there are also tetrahedra}\\
\tline{There are no tetrahedra}}{\tline{There are no cubes}}
\item Challenge: how does the truth-table method apply?  When should we say an argument is \emph{tautologically} valid even though it contains quantifiers?

\eit

}

\subsec{The Truth-Functional Form Algorithm}{

\bit
\item Underline every quantifier and its entire scope (unless it is in the scope of another quantifier)
\item Label every underlined bit with a schematic sentence letter (A, B, C, \dots), use the same letter if the underlined parts are \emph{identical}
\item Replace underlined parts by sentence letters
\item Result is the \emph{truth-functional form} of the argument
\item An argument (with quantifiers) is \emph{tautologically valid} if its truthfunctional form is tautologically valid
\eit

}

\subsec{The Truth-Functional Form Algorithm}{

\fitcharg{
\nline{\underline{\exists x\, Cube(x)}_A \to \underline{\exists x\, Tet(x)}_B}\\
\nline{\lnot\underline{\exists x\, Tet(x)}_B}}{
\nline{\lnot\underline{\exists x\, Cube(x)}_A}}

\bigskip

\fitcharg{
\nline{A \to B}\\
\nline{\lnot B}}{\nline{\lnot A}}

}

\subsec{The Truth-Functional Form Algorithm}{

\fitcharg{
\nline{\underline{\exists x\, Cube(x)}_A \to \underline{\exists x\, Tet(x)}_B}\\
\nline{\underline{\forall x\, \lnot Tet(x)}_C}}{
\nline{\underline{\forall x\, \lnot Cube(x)}_D}}

\bigskip

\fitcharg{
\nline{A \to B}\\
\nline{C}}{\nline{D}}

}

\subsec{First-order Validity}{

\bit
\item Want to capture validity \emph{in virtue of the meanings of the
  connectives and the quantifiers} (but ignoring meanings of predicate
  symbols other than =)
\item So we want to ignore any restrictions the predicate symbols place on their extensions
\item Hence: allow \emph{any} extension in a potential counterexample
\item An argument is \emph{first-order valid} if there is no \emph{first-order structure} in which the premises are true and the conclusion false

\item Variant interpretation: \emph{first-order structure}
\eit

}

\subsec{The Replacement Method}{

\bit
\item Existing meanings of predicate symbols may confuse you, so:
\item Replace each predicate symbol (other than =) with a ``nonsense predicate'': A, B, C, \dots
\item Arity the same as original predicate
\eit
}

\subsec{First-order Structure}{

\bit
\item Specify domain (e.g., ``Canadian cities'', ``the natural numbers'', graphically using a collection of dots, \dots)
\item Specify extension of A(x), B(x, y), C(x, y, z), \dots{} by
\bit
\item a diagram (as usual)
\item enumeration of elements, pairs of elements, triples of elements of the domain, e.g., ``Calgary, Edmonton, Lethbridge'', ``(0, 1), (0, 2), (0, 3), \dots''
\item determinate predicate (expression involving x, y, z), e.g., ``x is entirely located in Alberta'', ``x evenly divides y'', \dots
\eit
\item Specify referents of constant symbols, e.g., ``c: Calgary'', ``d: 5'', graphically by writing the constant next to the dot named by it
\eit
}

\section{FO Consequence and FO Structures}

\subsec{First-order Logical Notions}{

\bit
\item An argument is \emph{first-order valid}, and its conclusion is a \emph{first-order consequence} of its premises,
if there is no \emph{first-order structure} in which the premises are true and the conclusion is false
\item A sentence is a \emph{first-order validity} if it is true in every first-order stucture
\item Two sentences are \emph{first-order equivalent} if there is no first-order structure in which they have different truth values.
\item A (set of) sentence(s) is \emph{first-order satisfiable} if it is (all) true in at least one first-order structure
\item NB: There is \emph{no mechanical test} for any of these (The Church-Turing Undecidability Theorem), by contrast to the truth-table case!
\eit

}

\subsec{First-order Structure}{

\bit
\item $\sf \forall x\,(P(x) \impl R(x, b))$
\item We need:
\begin{itemize}
\item A domain: objects which can take the value of \textsf x.
\item Extension of $\sf P(x)$ (or a determinate property of the objects in that domain as an
interpretation for $\sf P(x)$).
\item Extension of $\sf R(x, y)$ (or a determinate 2-place relation as an interpretation of the
predicate $\sf R(x, y)$).
\item An object to be named by \textsf a.
\end{itemize}
\eit
}

\subsec{FO Validity of Arguments}{

\fitcharg{
\nline{\forall x\, Cube(x) \lor \forall x\, Tet(x)}\\
\nline{\exists x\, \lnot Cube(x) \land \forall y(Tet(y) \to Small(y))}}{
\nline{\lnot \exists x\, Large(x)}}
}

\subsec{FO (In)Validity of Arguments}{

\fitcharg{
\nline{\forall x\, A(x) \lor \forall x\, B(x)}\\
\nline{\exists x\, \lnot A(x) \land \forall y(B(y) \to C(y))}}{
\nline{\lnot \exists x D(x)}}

\begin{itemize}
\item Domain: the planets
\item $\sf A(x)$ \dots \textsf x is inhabited
\item $\sf B(x)$ \dots \textsf x is smaller than the Sun
\item $\sf C(x)$ \dots \textsf x orbits the Sun
\item $\sf D(x)$ \dots \textsf x has a moon
\end{itemize}
Another FO counterexample given by an Euler diagram:
\[
\begin{array}{|lc|}
\hline
\sf B = C = D & \\
& \begin{array}{|lc|}
\hline
\sf A & \cdot 1\\
\hline
\end{array}\quad \\
\cdot 2 & \\ \hline
\end{array}
\]

}

\subsec{Making ``All As are Bs'' True}{

\begin{align*}
\sf \forall x\, (A(x) & \impl \sf B(x)) \\
\sf \lnot\exists x(A(x) & \land \sf\lnot B(x))
\end{align*}

\[
\begin{array}{|lc|}
\hline
\sf B & \cdots \\
& \begin{array}{|c|}
\hline
\sf A \cdots \\
\hline
\end{array} \quad \\[1ex]
\hline
\end{array}
\qquad\qquad
\begin{array}{|lc|}
\hline
\sf B = A & \\
& \cdots \\
\hline
\end{array}
\qquad\qquad
\begin{array}{c}
\begin{array}{|lc|}
\hline
\sf B & \\
& \cdots \\
\hline
\end{array} \\[2ex]
\sf A = \emptyset
\end{array}
\]

Makes ``Some As are not Bs'' \emph{false}
}

\subsec{Making ``All As are Bs'' False}{

\begin{align*}
\sf \lnot\forall x\, (A(x) & \impl \sf B(x)) \\
\sf \exists x(A(x) & \land \sf\lnot B(x))
\end{align*}

\[
\begin{array}{lcr|}
  \cline{1-2}
  \multicolumn{2}{|c|}{\sf A \quad \cdots}\\
  \cline{2-3}
  \multicolumn{1}{|c|}{} & \multicolumn{1}{|c|}{\cdots} &
  \multicolumn{1}{|c|}{} \\
  \cline{1-2}
  & \multicolumn{2}{|c|}{\sf B \quad \cdots}\\
  \cline{2-3}
\end{array}
\qquad\qquad
\begin{array}{|lc|c|lc|}
\cline{1-2}\cline{4-5}
\sf A  & & & \sf B & \\
& \cdots & & & \cdots\\
\cline{1-2}\cline{4-5}
\end{array}
\qquad\qquad
\begin{array}{c}
\begin{array}{|lc|}
\hline
\sf A & \\
& \cdots \\
\hline
\end{array} \\[1ex]
\sf B = \emptyset
\end{array}
\]

Makes ``Some As are not Bs'' \emph{true}

}

\subsec{Making ``Some As are Bs'' True}{

\begin{align*}
\sf \exists x (A(x) & \sf\land B(x)) \\
\sf \lnot\forall x(A(x) & \impl\sf \lnot B(x))
\end{align*}

\[
\begin{array}{lcr|}
  \cline{1-2}
  \multicolumn{2}{|c|}{\sf A \quad \cdots}\\
  \cline{2-3}
  \multicolumn{1}{|c|}{} & \multicolumn{1}{|c|}{\cdots} &
  \multicolumn{1}{|c|}{} \\
  \cline{1-2}
  & \multicolumn{2}{|c|}{\sf B \quad \cdots}\\
  \cline{2-3}
\end{array}
\qquad\qquad
\begin{array}{|lc|}
\hline
\sf A & \cdots \\
& \begin{array}{|c|}
\hline
\sf B \cdots \\
\hline
\end{array} \quad \\[1ex]
\hline
\end{array}
\qquad\qquad
\begin{array}{|lc|}
\hline
\sf B & \cdots \\
& \begin{array}{|c|}
\hline
\sf A \cdots \\
\hline
\end{array} \quad \\[1ex]
\hline
\end{array}
\qquad\qquad
\begin{array}{|lc|}
\hline
\sf B = A & \\
& \cdots \\
\hline
\end{array}
\]

Makes ``No As are Bs'' \emph{false}

}

\subsec{Making ``Some As are Bs'' False}{

\begin{align*}
\sf \lnot\exists x (A(x) & \sf\land B(x)) \\
\sf \forall x(A(x) & \impl\sf \lnot B(x))
\end{align*}

\[
\begin{array}{|lc|c|lc|}
\cline{1-2}\cline{4-5}
\sf A  & & & \sf B & \\
& \cdots & & & \cdots\\
\cline{1-2}\cline{4-5}
\end{array}
\qquad\qquad
\begin{array}{c}
\begin{array}{|lc|}
\hline
\sf A & \\
& \cdots \\
\hline
\end{array}\\[2ex]
\sf B = \emptyset
\end{array}
\qquad\qquad
\begin{array}{c}
\begin{array}{|lc|}
\hline
\sf B & \\
& \cdots \\
\hline
\end{array} \\[2ex]
\sf A = \emptyset
\end{array}
\]

Makes ``No As are Bs'' \emph{true}
}

\subsec{Skorupski Revisited}{

\fitchprf{\pline[1.]{\text{If $\sf x$ is morally wrong, then}}\\
\pline{\qquad \text{$\sf A$ is blameworthy for freely doing $\sf x$}}\\
  \pline[2.]{\text{If $\sf x$ is
  rationally optimal, then }}\\
\pline{\qquad\text{A is not blameworthy for freely doing x.}}}
{\pline[3.]{\text{If $\sf x$ is morally wrong, then $\sf x$ is not rationally
      optimal.}}}

\bigskip

\fitchprf{\pline[1.]{\forall x(Wrong(x) \lif BlameFreelyDoing(a, x))}\\
  \pline[2.]{\forall x(ROpt(x) \lif \lnot BlameFreelyDoing(a, x))}}
{\pline[3.]{\forall x(Wrong(x) \lif \lnot ROpt(x))}}
}

\subsec{Skorupski Revisited}{

\href{http://books.google.ca/books?id=bxIzZYqRZdwC&lpg=PP1&pg=PA170\#v=onepage&q&f=false}{Actual argument} from Skurupski's book:

\fitchprf{\pline[1.]{\text{If $\sf x$ is morally wrong and $\sf A$ freely does $\sf x$,
    then}}\\
\pline{\qquad\text{$\sf A$ is blameworthy for doing $\sf x$.}}\\
  \pline[2.]{\text{If doing $\sf x$ is
  rationally optimal, then}}\\
\pline{\qquad\text{there is no reason to blame $\sf A$ for freely doing
  $\sf x$.}}}
{\pline[3.]{\text{If $\sf x$ is morally wrong, then $\sf x$ is not rationally
      optimal.}}}

\bigskip

\fitchprf{\pline[1.]{\forall x((Wrong(x) \land DoesF(a, x)) \lif Blame(a, x))}\\
  \pline[2.]{\forall x((ROpt(x) \land DoesF(a, x)) \lif \lnot Blame(a, x))}}
{\pline[3.]{\forall x(Wrong(x) \lif \lnot ROpt(x))}}

}

\subsec{Skorupski Revisited}{

\fitchprf{\pline[1.]{\forall x((Wrong(x) \land DoesF(a, x)) \lif Blame(a, x))}\\
  \pline[2.]{\forall x((ROpt(x) \land DoesF(a, x)) \lif \lnot Blame(a, x))}}
{\pline[3.]{\forall x(Wrong(x) \lif \lnot ROpt(x))}}

\bigskip

\fitchprf{\pline[1.]{\forall x((Cube(x) \land LeftOf(a, x)) \lif SameRow(a, x))}\\
  \pline[2.]{\forall x((Small(x) \land LeftOf(a, x)) \lif \lnot SameRow(a, x))}}
{\pline[3.]{\forall x(Cube(x) \lif \lnot Small(x))}}
}

\end{document}


\clearpage
\subsection{For Loop} % (fold)
\label{sub:for_loop}

As has been shown in previous chapters, Computers can only perform simple actions. They cannot perform an action on all of the elements in our arrays. For example, a computer cannot sum \textbf{all} of the values in an array. What you need to do is think of these tasks so that they can be performed \textbf{for \emph{each}} value in the array. So the sum would become, for each of the numbers in the array, add the number to a running total. When this has been performed for each element in the array you have the sum of all of the elements.

The for loop is a \nameref{sub:pre_test_loop} that repeats a block of code a number of times. You can think of it like a counting loop, with it counting from a start value to an end value. The for loop has a \textbf{control variable} that has the number of the current loop as its value.

\begin{figure}[h]
   \centering
   \includegraphics[width=\textwidth]{./topics/arrays/diagrams/For}
   \caption{The for loop can be used to loop through the elements of an array}
   \label{fig:for-loop}
\end{figure}

\mynote{
\begin{itemize}
  \item A for loop is an \textbf{action}, a kind of statement you can use to command the computer to perform an action.
  \item The key is to think about processing \textbf{each} element in an array, rather than thinking about \emph{all} elements of an array.
  \item The for loop can then provide the infrastructure to repeat this code \emph{for each} element in the array.
  \item The for loop is designed to work well with the array. The values in the \emph{control variable} can be used to access the individual elements of the array.
  \item When processing the elements of an array you have it loop from the lowest index value (0) to the highest index value (n - 1).
\end{itemize}
}

% subsection for_loop (end)


\lab{Data Visualization}{Data Visualization}
\label{lab:DataVis}
\objective{Correctly presenting and interpreting data with visualizations is a fundamental skills for all scientists.
In this lab we discuss and demonstrate the principles of good data visualization, including the key characteristics of good graphs, common mistakes, and how to avoid unethical visualizations.}

\section*{The Importance of Visualizations} % =================================

Using visualizations to understand data often reveals insights that may not immediately obvious from simple statistics.
The following data set, known as \emph{Anscombe's quartet}, is a famous example of the importance of graphing data.

% \begin{comment} % TODO: change the paragraph before this to account for taking the table out.
\begin{table}[H]
\small{
\begin{tabular}{rr|rr|rr|rr}

\multicolumn{2}{c|}{I}    & \multicolumn{2}{|c|}{II} &
\multicolumn{2}{|c|}{III} & \multicolumn{2}{|c}{IV} \\
\multicolumn{1}{c}{$x$}   & \multicolumn{1}{c|}{$y$} &
\multicolumn{1}{|c}{$x$}  & \multicolumn{1}{c|}{$y$} &
\multicolumn{1}{|c}{$x$}  & \multicolumn{1}{c|}{$y$} &
\multicolumn{1}{|c}{$x$}  & \multicolumn{1}{c}{$y$} \\
\hline
10.0 & 8.04  & 10.0 & 9.14 & 10.0 & 7.46  & 8.0  & 6.58  \\
8.0  & 6.95  & 8.0  & 8.14 & 8.0  & 6.77  & 8.0  & 5.76  \\
13.0 & 7.58  & 13.0 & 8.74 & 13.0 & 12.74 & 8.0  & 7.71  \\
9.0  & 8.81  & 9.0  & 8.77 & 9.0  & 7.11  & 8.0  & 8.84  \\
11.0 & 8.33  & 11.0 & 9.26 & 11.0 & 7.81  & 8.0  & 8.47  \\
14.0 & 9.96  & 14.0 & 8.10 & 14.0 & 8.84  & 8.0  & 7.04  \\
6.0  & 7.24  & 6.0  & 6.13 & 6.0  & 6.08  & 8.0  & 5.25  \\
4.0  & 4.26  & 4.0  & 3.10 & 4.0  & 5.39  & 19.0 & 12.50 \\
12.0 & 10.84 & 12.0 & 9.13 & 12.0 & 8.15  & 8.0  & 5.56  \\
7.0  & 4.82  & 7.0  & 7.26 & 7.0  & 6.42  & 8.0  & 7.91  \\
5.0  & 5.68  & 5.0  & 4.74 & 5.0  & 5.73  & 8.0  & 6.89  \\
\end{tabular}}
\end{table}

Each section of Anscombe's quartet the following statistical properties:
\begin{itemize}
\setlength\itemsep{0em}
\item The means are $9$ for $x$ and $7.5$ for $y$.
\item The sample variances are $11$ for $x$ and $3.75$ for $y$.
\item The correlation between $x$ and $y$ is $.816$.
\item The linear least squares regression line is $y=\frac{1}{2}x+3$.
\end{itemize}

Despite these similarities, each section is quite different from the others.

\begin{problem} % Describe Anscombe's quartet.
The file \texttt{anscombe.npy} contains Anscombe's quartet in the same format as displayed above.
Plot each section of the quartet separately as a scatter plot.
Also plot the regression line $y = \frac{1}{2}x + 3$ on the domain $x\in[0,20]$ over each scatter plot.

Write a few sentences describing what makes each section unique.
% How would you effectivly describe each section to another person?
\label{prob:anscombes-quartet}
\end{problem}

\begin{comment}
\section*{Principles of Good Data Visualization} % ============================

Problem \ref{prob:anscombes-quartet} demonstrate how visualizations can describe data much more accurately than simple statistics.
However, a poor or unethical visualizations can easily misrepresent data.

\begin{enumerate}
    \item \textbf{Clarity}: Use titles, label axes, and include a legend. Graphs should be self-explanatory.
    Where appropriate, use words to carefully explain the insights represented
    in the visualization.
    \item \textbf{Simplicity}: Remove unnecessary elements of a graph.
    Don't add 3D effects in for no reason.
    \item \textbf{Integrity}. Tell the truth, the whole truth, and nothing but the truth.

    Every visualization should be presented alongside all relevant information including who created it, where the data was obtained, how it was collected, whether it was cleaned or transformed, and whether there are conflicts of interest or possible biases present.

    It is almost always very easy to manipulate a visualization so that it supports a desired interpretation.

    As in any form of communication, integrity is important in data visualization.

\end{enumerate}

\end{comment}

\section*{Choosing the Right Visualization} % =================================

Most data sets are more naturally described with one type of visualization than another.
Below we review several common visualizations, their standard usages, and questions to consider during their construction.

\small{ % Get this section to fit on one page.
\begin{itemize}
\item \textbf{Line Plot}: Connects ordered $(x,y)$ points with straight lines.

Used to visualize one or two ordered arrays, such as functional outputs over an ordered domain or a sequence of values over time.
%
\begin{itemize}
    % \item What do the axes represent? How should they be labeled?
    \item Are linear or logarithmic scales more effective?
    \item Is the line an appropriate thickness and color?
\end{itemize}

% Use \li{plt.plot()} to create a line plot.

\item \textbf{Scatter Plot}: Draws $(x,y)$ points without connecting them.

Best for displaying points without a natural order or where each point is an individual instance and not inherently related to the other points in the set.
%
\begin{itemize}
    % \item What do the axes represent? How should they be labeled?
    \item Which marker is most appropriate?
    \item Are the markers an appropriate size and color?
\end{itemize}

% Use \li{plt.plot()} with a point marker (\li{'.'}, \li{','}, \li{'o'}, etc.) or \li{plt.scatter()} to draw a scatter plot.

\begin{comment} % Window size, also should go later (with ethics).
Figure \ref{fig:scatter_correlation} displays two scatter plots.
The first appears to have a weak correlation and the second appears to have a strong correlation.
However, the same data is being plotted and the only difference is the scale and window size.
Manipulating these can change your interpretation and should be done with careful consideration.
\end{comment}

\item \textbf{Histogram}: Partitions an interval into a number of bins and counts the number of values that fall into each bin.

Best for visualizing a single unordered array, such as draws from a probability distribution.
%
\begin{itemize}
    \item How many bins should be used?
    \item Over what range should the bins be?
    \item Is a linear or a logarithmic scale more appropriate for the frequency axis?
\end{itemize}

% Use \li{plt.hist()} to create a histogram.
% The arguments \li{bins} and \li{<<range>>} specify the number of bins to draw and over what domain.

\begin{comment} % Details for later.
A histogram with only a few bins or too many bins will fail to give a clear view of the distribution of the data, therefore it is important to choose the number of bins that gives the best view of the distribution of the data.

Similarly, reference lines are distracting and focus on unnecessary details instead of the shape of the distribution.
If more precise details are desired, exact figures can be presented alongside the histogram in a table.
\end{comment}

\item \textbf{Heat Map}: Assigns a color to each point on a 2-D domain, providing a 2-D picture representing a 3-D surface.

Best for visualizing functions $f:\mathbb{R}^2\rightarrow\mathbb{R}$, usually over a domain constructed with \li{np.meshgrid()}.
%
\begin{itemize}
    \item Is the domain sufficiently refined?
    \item Which color scheme is most clear and effective?
    \item Is a linear or a logarithmic scale more appropriate for the color?
\end{itemize}

\item \textbf{Contour Map}: Draws level curves of a function $f:\mathbb{R}^2\rightarrow\mathbb{R}$.

A filled contour map is discretized version of a heat map.
%
\begin{itemize}
    \item How many / which contour lines should be drawn?
    % \item Is the color scheme clear and effective?
    \item Is a linear or a logarithmic scale more appropriate for the color?
\end{itemize}

\end{itemize}}
%
% \begin{table}[H]
% \begin{tabular}{rcc}
% & \textbf{One Array} & \textbf{Two Arrays} \\
% \cline{2-3}
% \textbf{Ordered Data} &
% \multicolumn{1}{|c|}{Line Plot} & \multicolumn{1}{|c|}{Line Plot} \\
% \cline{2-3}
% \textbf{Unordered Data} &
% \multicolumn{1}{|c|}{Histogram} & \multicolumn{1}{|c|}{Scatter Plot} \\
% \cline{2-3}
% % \textbf{Categorical Data} & \multicolumn{1}{|c|}{Bar Chart} & \multicolumn{1}{|c|}{Scatter Plot} \\ \cline{2-3}
% \end{tabular}
% \end{table}

\subsection*{Scatter Plots} % -------------------------------------------------

\begin{comment}
% Example of using a marker and a line.
Line plots and scatter plots are sometimes used together.
A line connecting points implies a relationship or correlation between the points, while drawing point markers emphasizes that each point is a unique instance of data.
Use \li{plt.plot()} and specify a marker \emph{and} a line style in the third argument to do both at once.
% The keywords \li{linewidth} (or \li{ls}) and \li{markersize} (or \li{ms}) control the thickness of the line and the size of the markers, respectively.

\begin{lstlisting}
>>> x = np.linspace(0, 4, 8)
>>> data = x**2

>>> plt.subplot(131)                # Regular line plot of the data.
>>> plt.plot(x, data, 'b-', linewidth=2)

>>> plt.subplot(132)                # Scatter plot of the data.
>>> plt.plot(x, data, 'g.', markersize=15)

>>> plt.subplot(133)                # Line and scatter plot.
>>> plt.plot(x, data, 'r.-', linewidth=2, markersize=15)
\end{lstlisting}
\end{comment}

% Explain plt.scatter(), especially the 'c' and 's' keyword arguments.
While \li{plt.plot()} is an easy way to create scatter plots, \li{plt.scatter()} provides a few extra tools.
The keywords \li{s} and \li{c} correspond to marker size and marker color, respectively.
Each keyword can either be a single entry, producing uniform markers, or an array, specifying the sizes or colors of the individual markers.
This allows a scatter plot to have up to four dimensions of information.

Consider again Anscombe's quartet.
The pattern in section II can be made more pronounced by tying the size of the marker to the value of the $y$ coordinate.
Since adjusting the size of the markers may lead to overlapping markers, we specify the \emph{alpha value} of the color to make the markers transparent.
The keyword \li{alpha} accepts a value in the interval $[0,1]$; 0 makes the markers completely transparent, while a $1$ makes the markers completely opaque.

\begin{lstlisting}
>>> anscombe = np.load("anscombe.npy")
>>> x2, y2 = anscombe[:,2], anscombe[:,3]

>>> plt.subplot(121)
>>> plt.scatter(x2, y2, c='b', s=500)

>>> plt.subplot(122)
>>> plt.scatter(x2, y2, c='b', s=.5*y**4, alpha=.8)

>>> plt.show()
\end{lstlisting}

\begin{figure}[H] % Different sizing in a scatter plot.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/scatter_1_bad.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/scatter_1_good.pdf}
\end{subfigure}
\end{figure}

Controlling the colors of the individual markers can also be illuminating. % TODO: The goal is to present 3-dimensional data, not show patterns...
As with heat maps and contour plots, a color bar can be added to indicate the values that the colors represent.

\begin{lstlisting}
>>> x3, y3 = anscombe[:,4], anscombe[:,5]

>>> plt.subplot(121)
>>> plt.scatter(x3, y3, s=500, c='b')

>>> plt.subplot(122)
>>> plt.scatter(x, y, s=500, c=y, alpha=.8)
>>> cbar = plt.colorbar()           # Draw and label the color bar.
>>> cbar.set_label("y")
\end{lstlisting}

\begin{figure}[H] % Different coloring in a scatter plot.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/scatter_2_bad.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/scatter_2_good.pdf}
\end{subfigure}
\end{figure}

% Talk about connecting the dots, then using a regression line instead.

\begin{problem} % Scatter plots with visualizations.
The file \texttt{MLB.npy} contains measurements from over 1,000 recent Major League Baseball players, compiled by UCLA.\footnote{See \url{http://wiki.stat.ucla.edu/socr/index.php/SOCR_Data_MLB_HeightsWeights}.}
Each row in the array represents a different player; the columns are the player's height (in inches), weight (in pounds), and age (in years), in that order.

Describe the data with at least one visualization.
Your graph(s) should demonstrate whether height, weight, or age correlated with each other in the MLB.
\end{problem}

\subsection*{Histograms} % ----------------------------------------------------

Line plots should \emph{not} be used for data without a natural progression.
For example, suppose we would like to make a large number of random draws from a distribution to visualize its probability density function.
In this case, a plain line plot is completely useless, but the histogram provides an accurate picture of the statistical distribution.

\begin{comment}
\begin{lstlisting}
>>> import numpy as np
>>> from matplotlib import pyplot as plt

# Get 10,000 random samples from the standard normal distribution.
>>> data = np.random.normal(size=10000)

>>> plt.subplot(121)                # Regular line plot of the data.
>>> plt.plot(data)

>>> plt.subplot(122)                # Histogram of the data.
>>> plt.hist(data, bins=30)

>>> plt.show()
\end{lstlisting}

\begin{figure}[H] % line plot versus a histogram.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/hist_1_bad.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/hist_1_good.pdf}
\end{subfigure}
\end{figure}
\end{comment}

Less is more.
In the case of a statistical distribution, all we really want to see is the general shape of the distribution.
The lines separating the bins, axis tick marks, and even the labels on the $y$ axis are all unnecessary (and potentially distracting) details.

Removing the lines between bins is easy: set the line width to $0$.
To ensure the gaps where the lines used to be are filled, specify the \li{histtype} as \li{"stepfilled"} (using \li{histtype="step"} will draw the outline without filling it in).
Finally, to get rid of extra markings on the axes, use \li{plt.tick_params()}.
Some of this function's keyword arguments are summarized below.

\begin{table}[H]
\begin{tabular}{r|c|l}
    \multicolumn{1}{c|}{Argument} & Options & \multicolumn{1}{|c}{Description}
    \\ \hline
    \li{axis} & \li{'x'}, \li{'y'}, \li{"both"} & Axis on which to operate. \\
    \li{which} & \li{"major"}, \li{"minor"}, \li{"both"} & Operate on major or minor ticks.\\
    \li{color} & Any Matplotlib color & Tick color. \\
    \li{labelcolor} & Any Matplotlib color & Tick label color. \\
    \li{bottom}, \li{top}, \li{left}, \li{right} & \li{"on"}, \li{"off"} & Turn ticks on or off. \\
    \li{labelbottom}, \li{labeltop}, & \li{"on"}, \li{"off"} & Turn tick labels on or off. \\
    \li{labelleft}, \li{labelright} & &
\end{tabular}
\end{table}

\begin{lstlisting}
# Get 10,000 random samples from the standard exponential distribution.
>>> data = np.random.exponential(size=N)

>>> plt.subplot(121)
>>> plt.hist(data, bins=30)

>>> plt.subplot(122)                # Histogram with no lines or tick marks.
>>> plt.hist(data, bins=30, lw=0, histtype="stepfilled")
>>> plt.tick_params(axis="both", which="both", labelleft='off',
...                              left="off", top="off", right="off")

>>> plt.show()
\end{lstlisting}

\begin{figure}[H] % Cleaning up a histogram.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/hist_2_bad.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/hist_2_good.pdf}
\end{subfigure}
\end{figure}

A histogram plots an array of values against the frequency with which the values occur.
To plot a line plot of values against frequency (without drawing a histogram), use \li{np.histogram()}.
This function returns the number of values in each bin and the locations of the edges of the bins.
Use the edges of the bins to calculate the center of the bins, then use a line plot to connect the two.

\begin{lstlisting}
# Get samples from a beta distribution and draw its PDF.
>>> for i, N in enumerate([10000, 10000000]):
...     plt.subplot(1,2,i+1)
...     data = np.random.beta(a=5, b=2, size=N)
...     # Draw a line plot using the bin centers and frequencies.
...     frequencies, bin_edges = np.histogram(data, bins=50)
...     bin_centers = (bin_edges[:-1] + bin_edges[1:])/2.
...     plt.plot(bin_centers, frequencies, 'b-', lw=4)
...     # Plot the faded histogram for comparison.
...     plt.hist(data, bins=50, alpha=.1)
...     plt.tick_params(axis="both", which="both", labelleft='off',
...                                  left="off", top="off", right="off")
...
>>> plt.show()
\end{lstlisting}

\begin{figure}[H] % Histogram Line Plot Hybrid
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/hist_3_bad.pdf}
\end{subfigure}
%
\begin{subfigure}{.49\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/hist_3_good.pdf}
\end{subfigure}
\end{figure}

\begin{problem} % Earthquake data
The file \texttt{earthquakes.npy} contains data from the 17,065 earthquakes between 2000 and 2010 that were at least a 5 on the Richter scale.\footnote{See \url{http://earthquake.usgs.gov/earthquakes/search/}.}
Each row in the array represents a different earthquake;
the columns are the earthquake's date (as a fraction of the year), magnitude, longitude, and latitude, in that order.

Because each earthquake is a distinct event, a good way to start visualizing this data might be a scatter plot of the years versus the magnitudes of each earthquake.

\begin{lstlisting}
>>> year, magnitude, longitude, latitude = np.load("earthquakes.npy").T
>>> plt.plot(year, magnitude, '.')
>>> plt.xlabel("Year")
>>> plt.ylabel("Magnitude")
>>> plt.show()
\end{lstlisting}

\begin{figure}[H] % Bad visualization: earthquake data, year vs magnitude.
    \centering
    \includegraphics[width=.7\textwidth]{figures/earthquake.pdf}
\end{figure}

Unfortunately, this plot communicates very little information because the data is so cluttered.
Describe the data with two or three better visualizations.
Your graphs should clearly answer the following questions:
\begin{enumerate}
    \item How many earthquakes happened every year?
    \item How often do stronger earthquakes happen compared to weaker ones?
    \item Where do earthquakes happen? Where do the strongest earthquakes happen?
\end{enumerate}
% (Hint: \li{plt.gca().set_aspect("equal")} fixes the aspect ratio, which may improve comparisons between longitude and latitude.)
\end{problem}

% Consider Figure \ref{fig:logexample} which shows data about large earthquakes from the United States Geological Survey.
% It appears to be somewhat random however, when transformed, the frequency of the magnitudes on a logarithmic scale show a fairly strong linear relationship between the variables.

\subsection*{Heat and Contour Maps} % -----------------------------------------

\begin{figure}[h]
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{contour_map.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{contour_map_filled.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{contour_map_rainbow_filled.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{contour_map_rainbow.png}
\end{subfigure}
\caption{Contour Maps.  A contour map (top left) and a heat map (top right).
The color choices of the bottom row are less intuitive and make the visualization harder to interpret.}
\label{fig:contour}
\end{figure}

Three-dimensional data can be displayed with a contour plot.
This draws lines in the plane where the third value is constant---like a topographical map.
Coloring the space between the contour lines produces a heat map.
Heat maps work best when only one or two colors are used.

\begin{figure}[h]
\centering
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=\textwidth]{heatmap_color.png}
\end{subfigure}%
\begin{subfigure}{.5\textwidth}
  \centering
  \includegraphics[width=\textwidth]{heatmap_hot.png}
\end{subfigure}
%\includegraphics[width=\textwidth]{heatmap.png}
\caption{Both of these pseudocolor plots depict the function
    $z = \sin(x)\sin(y)$ on the domain $[-1,4] \times [-4,1]$.
The plot on the left uses a rainbow gradient, which has no meaningful relationship to the data.
The plot on the right uses the colormap \li{afmhot}.  The lighter and darker colors are a more intuitive
representation of higher and lower values.}

\label{fig:heatmap}
\end{figure}

% TODO: In the next release of Matplotlib (version 2.0) viridis is the default.
% TODO: Reword (awkward).
A good default colormap is Matplotlib's \li{viridis}.
Other colormaps that are appropriate for the data can be selected using the argument \li{cmap='NAME\_OF\_COLORMAP'}, where \li{NAME\_OF\_COLORMAP} is one of Matplotlib's sequential or diverging colormaps.
Two sequential colormaps are \li{afmhot} and \li{cool}. Two diverging colormaps are \li{seismic} and \li{bwr}.
These colormaps along with their color scheme can be found on
\url{http://matplotlib.org/examples/color/colormaps_reference.html}.
Although three-dimensional data can be plotted as a surface in 3-space, it is often difficult to find a view where none of the important features of the data are obscured.
A heatmap avoids this problem and is easier to understand.

You can create a contour plot in Matplotlib with \li{plt.contour()}.

You can also create a pseudocolor plot in Matplotlib with the \li{plt.pcolormesh()} command.

% # This code corresponds to the figure on the top right of the contour maps.
\begin{lstlisting}
import numpy as np
from matplotlib import pyplot as plt

n=400
xran = np.linspace(-1.5,1.5,n)
yran = np.linspace(-1.5,1.5,n)
X, Y = np.meshgrid(xran,yran)
F = Y**2 - X**3 + X**2
plt.contourf(X, Y, F, [-2,-1,0.0001,1,2,3,4,5] ,cmap=plt.get_cmap('afmhot'))
\end{lstlisting}

\begin{problem} % Contour map problem. TODO: don't specify number of intervals?
Using 400 intervals, plot a filled contour map of the function \[f(x,y) = \sin(x) + \sin(y)\] over the domain $[0,12\pi]\times[0,12\pi]$.

Be sure to use an appropriate color map.
\end{problem}

\section*{Communicating Insights with Visualizations} % =======================

\begin{comment} % USE THIS AS AN EXAMPLE LATER.
\begin{figure}[H] % Histogram with and without busy information.
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{good_hist_example.png}
\end{subfigure}
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{hist_reference_grid.png}
\end{subfigure}
\caption{Histogram with and without reference lines. It is harder to see the shape of the distribution in the histogram with reference lines.}
\label{fig:histogram}
\end{figure}

Plot two histograms with data drawn randomly from a normal and exponential distribution.

\begin{lstlisting}
from numpy.random import normal
from numpy.random import exponential
normal_data = normal(size=1000)
exp_data = exponential(size=1000)
\end{lstlisting}

For each distribution, plot a histogram using 30 bins, 15 bins and 5 bins.
Does this change how we interpret the distribution of the data?
\end{comment}

\begin{comment}
\begin{figure}[h] % Line plots.
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{line_plot_bad_thin.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{line_plot_good.png}
\end{subfigure}
\caption{Line Plot.  The left plot is plotted with \li{lw=1} whereas the right plot is plotted with \li{lw=2}.  Notice that the right plot is better because it is easier to see the line.} % Irrelevant for right here.
\label{fig:lineplot_bad}
\end{figure}

\begin{figure}[h]
\centering
\begin{subfigure}{.5\textwidth}
\centering
\includegraphics[width=\textwidth]{line_plot_bad_X.png}
\end{subfigure}
\caption{Line plot. Notice that this graph appears to show that the x-values are ordered. However, in reality, they are unordered. This is a bad practice and misrepresents the data.}
\label{fig:lineplotbadX}
\end{figure}
\end{comment}

\begin{comment} % MOVE BELOW.
\begin{figure}[h] % Choose an ethical scale and window.
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{scale_scatter_zoomed_in.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{scale_scatter_zoomed_out.png}
\end{subfigure}
\caption{Scatter Plots. The data in both plots are the same but there appears to be a weaker
correlation in the first plot compared to the second.}
\label{fig:scatter_correlation}
\end{figure}
\end{comment}

\begin{comment}
\begin{figure}[h]
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{earthquake_zoomed.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{earthquake_data.png}
\end{subfigure}
\caption{USGS Earthquake data. The scatter plot on the left only plots earthquakes with
magnitude 7.5 or greater from 1980-2010 and appears to have a correlation.
However, the plot on the right shows all recorded earthquakes from 1900-2010 and shows that the correlation
is not a true correlation.}
\label{fig:scatterquake}
\end{figure}

Scatter plots may reveal insights other than correlation.
Consider Figure \ref{fig:scatterquake} which plots earthquake data from the United States Geological Survey (USGS).
The USGS data began including earthquakes with magnitude 6.5 or greater; this is clearly seen
in the scatter plot of the entire dataset.
Plots like this may also reveal errors in the dataset that are harder to detect in a table of values.
\end{comment}

\begin{comment}
\begin{figure}[h] % Bad visualization: earthquake data, year vs magnitude.
\captionsetup[subfigure]{justification=centering}
\centering
\begin{subfigure}{.5\textwidth}
    \centering
    \includegraphics[width=\linewidth]{figures/earthquake_bad.pdf}
\end{subfigure}%
\begin{subfigure}{.5\textwidth}
    \centering
    \includegraphics[width=\linewidth]{freq_earthquake_log.png}
\end{subfigure}
\caption{The left plot shows earthquakes of magnitudes 7.5 or higher over time. The right plot shows the frequencies of
different magnitudes. Note the scale of the y-axis is logarithmic.}
\label{fig:logexample}
\end{figure}
\end{comment}

\begin{comment} % This needs to go somewhere else.
Understanding a data set through visualizations is an iterative process.
Start with an initial visualization---usually a broad overview---and examine the data, then adjust the visualization or create other visualizations based on what you observe.
Ask the following questions as you search for insights:
\begin{enumerate}
\item Does the data make sense?
\item Would a different visualization communicate more information?
\item Would visualizing a subset of the data provide more information?
\item Would transforming the data reveal a hidden pattern?
\end{enumerate}
\end{comment}

As in any form of communication, integrity is important in data visualization.
Every visualization should be presented alongside all relevant information including who created it, where the data was obtained, how it was collected, whether it was cleaned or transformed, and whether there are conflicts of
interest or possible biases present.
Where appropriate, use words to carefully explain the insights represented
in the visualization.
These practices will help the reader correctly interpret the visualization.

\subsection*{Things to Avoid} % -----------------------------------------------

Some popular visualizations (pie charts, radar graphs, stacked bar charts and others) interfere with our ability to interpret the information they display.
For example, it is more difficult to detect differences in a pie chart than on a bar chart with the same data.
This is because differences in area are more difficult to detect than differences in length.
As a result, charts that depend on detecting variation in area will be less effective.
Instead, choose visualizations that compare the data to the a simple standard that is easy to understand and interpret.
Figures \ref{fig:pievsbar} and \ref{fig:stacked} are examples of poorly chosen visualizations.

\begin{figure} % Bad graphs.
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{bad_pie_chart.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{bar_chart_horizontal_sorted.png}
\end{subfigure}
\caption{A pie chart on the left that has been improved by a transformation into a bar chart on the right.}
\label{fig:pievsbar}
\end{figure}

\subsection*{Keep It Simple} % ------------------------------------------------

It may be tempting to stylize your visualization by adding icons and pictures or by making certain features
look 3D.  This is called chartjunk, a term coined by Edward Tufte. Chartjunk is anything in a plot that fails to add to or
misrepresents the data.  Avoid making visualizations that are overly fancy or cluttered with chartjunk. This
includes 3D effects (as in \ref{fig:pievsbar}), bad font choices, and unnecessary icons and pictures.
Visualizations are at their best when they are as simple as possible and allow the reader to easily
interpret the information being presented.

Reference lines can distract from the visualization (see Figure \ref{fig:histogram}).
For example, we can remove axis lines, tick marks, and bar lines
with the following code:

\begin{lstlisting}
# Get current axis instance
axis = plt.gca()

# Hide top and right spines
axis.spines['right'].set_visible(False)
axis.spines['top'].set_visible(False)

# Only show bottom and left tick marks
axis.yaxis.set_ticks_position('left')
axis.xaxis.set_ticks_position('bottom')
\end{lstlisting}

\begin{problem} % Problem: convert a pie chart to a horizontal bar chart.
Regraph the pie chart in Figure \ref{fig:pievsbar} as a horizontal bar chart using the follwing data.
Spam: 21, Eggs: 20, Ham: 22, Sausage: 21, Bacon: 20, Baked Beans: 11, Lobster Thermidor: 10
\end{problem}

% TODO: WHY IS THIS PROBLEM WAY DOWN HERE??? Shouldn't this be part of the histogram section?
\begin{problem} % Problem: histogram of earthquake magnitudes.
Plot a histogram of earthquake magnitudes using the data in the file \li{earthquake.txt}.
Remove reference lines, tick marks and bar lines.
Use the numpy function \li{np.loadtxt()} to read in the data.
Is the data distributed normally or exponentially?
\end{problem}

\begin{figure} % Stacked v unstacked bar charts
\centering
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{stackedbar.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
\includegraphics[width=\textwidth]{unstackedbar.png}
\end{subfigure}
\caption{Stacked Bar Chart. The relative values are harder to compare when stacked (left) instead of adjacent (right).}
\label{fig:stacked}
\end{figure}

\subsection*{Small Multiples} % -----------------------------------------------

When we have too much information to display cleanly in one chart, we can split the visualization into many smaller visualizations and place them near each other so they can be easily compared.
This method is called \emph{small multiples} and was made famous by Edward Tufte.

As an example, consider the first six Chebyshev polynomials.
Plotting them all on the same graph makes the visualization cluttered.
Instead, each polynomial can be given its own graph and laid out in a way that makes them easy to compare, (see Figure \ref{fig:PoT}).

% add more reference material??

\begin{figure} % Chebyshev polynomials
\centering
\begin{subfigure}{.45\textwidth}
  \centering
  \includegraphics[width=\textwidth]{PoT.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
\centering
  \includegraphics[width=\textwidth]{PoT_separate.png}
\end{subfigure}
\caption{Chebyshev polynomials graphed together on the left and in small multiples on the right.
The small multiples are much easier to interpret.}
\label{fig:PoT}
\end{figure}

To make small multiples, use the Matplotlib command \li{ax = plt.subplot()} which sets up multiple plots one at a time.
% Some example code is provided below.

\begin{lstlisting}
import numpy as np
from matplotlib import pyplot as plt
from numpy.polynomial import Chebyshev as T

def Chebyshev_subplots():
    fig = plt.figure(dpi=100)
    fig.set_size_inches(10,10)
    fig.suptitle('Chebyshev Polynomials', fontsize=20)

    x = np.linspace(-1,1,500)

    for i in range(9):
        ax = plt.subplot(3,3,i+1)
        ax.plot(x, T.basis(i)(x), lw=2, label="$T_%d$"%i)
        plt.xlim(-1.1,1.1)
        plt.ylim(-1.1,1.1)
        ax.set_title('$T_%d$'%i)
        if i%3:
            # remove the inner y-ticks and labels.
            ax.set_yticklabels([])
            ax.yaxis.set_ticks_position('none')
        if i<6:
            # remove the inner x-ticks and labels.
            ax.xaxis.set_ticks_position('none')
            ax.set_xticklabels([])

    plt.show()
\end{lstlisting}

\begin{problem} % Problem: plot the Bernstein polynomials.
The $n+1$ Bernstein basis polynomials of degree $n$ are defined as

$$b_{v,n} = {{n} \choose {v}} x^v (1-x)^{n-v}, v = 0, 1,..., n.$$

Plot the Bernstein basis polynomials for $v,n$ in [0,3] as small multiples and compare that to the cluttered version of plotting them on top of each other.
\end{problem}

\begin{comment}

\subsection*{Additional Material} % ===========================================

There are many software packages that facilitate the visual exploration of data.
One Python library is Glue (see \cite{glue}).

Some packages for making nicer looking plots include \li{Seaborn} and \li{prettyplotlib}.  TODO get refs

For more about visualization of data, we highly recommend the following books and websites:

% TODO get full refs for the follwoing

\begin{itemize}
    \item \emph{How to Lie with Statistics} by Darrell Huff (1954)
    \item \emph{Envisioning Information} by Edward Tufte
    \item \emph{The Visual Display of Quantitative Information} by Edward Tufte (2nd edition)
    \item \emph{Beautiful Evidence} by Edward Tufte
    \item \emph{Now you see it} by Stephen Few
    \item \url{http://www.informationisbeautiful.net/}
\end{itemize}

\end{comment}

%\noindent\emph{At their best, graphics are instruments for reasoning about quantitative information.} \small{---Edward Tufte}
% from the introduction of his book 'The Visual Display of Quantitative Information'

\begin{comment} % Move to 'Additional Material'.
\textbf{Bar Chart}: Best for small sets of discrete, one-dimensional, categorical data.
Usually, a horizontal bar chart is preferred over a vertical bar chart because it's harder to read vertical labels than horizontal labels.
Also, horizontal labels on a vertical bar chart may not fit on the graph well (see Figure \ref{fig:barchart}).

The data in a bar chart should be sorted in a logical way.
If the chart is being used to compare the values of different data points, it is best to sort by size; if the chart is being used to look up specific values, the labels should be sorted in a way that makes it easy to find specific values (for example, alphabetically).
Use \li{plt.bar()} or \li{plt.barh()} to create a bar chart in Matplotlib.

\begin{figure}[h] % Bar chart styles
\centering
\begin{subfigure}{.45\textwidth}
  \centering
  \includegraphics[width=\textwidth]{bar_chart_horizontal_sorted.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
  \centering
  \includegraphics[width=\textwidth]{bar_chart_unsorted.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
  \centering
  \includegraphics[width=\textwidth]{bar_chart_vertical_bars_horizontal_labels.png}
\end{subfigure}
\begin{subfigure}{.45\textwidth}
  \centering
  \includegraphics[width=\textwidth]{bar_chart_vertical_bars_vertical_labels.png}
\end{subfigure}

\caption{Bar Charts.
The example in the upper left is easier to use for comparison than the upper right, and easier to read than the bottom two charts.
The labels overlap in the bottom left and the labels are vertical in the bottom right, making them hard to read.}
\label{fig:barchart}
\end{figure}

\begin{lstlisting}
# This code created the top right bar chart.
labels = 'Spam', 'Eggs', 'Ham', 'Sausage', 'Bacon', 'Baked beans','Lobster thermidor'
val = [10, 11, 18, 19, 20, 21, 22]    # the bar lengths
pos = np.arange(7)+.5    # the bar centers on the y axis
plt.barh(pos,val, align='center')
plt.yticks(pos, labels[::-1])
plt.show()
\end{lstlisting}
\end{comment}