\Lecture{Jayalal Sarma }{ 2 october, 2020}{11}{More on PIE, PIE -Tuttes-Matrix-Tree-Theorem-Part2}{Shrinidhi Bajpayee}{$\alpha$}{JS}

\section{Introduction}
We finished Tutte's Matrix Tree Theorem in non trivial way.Let's just recall Tutte's Theorem.We have directed graph $G(V,E)$.We have $V={v_1,v_2,....v_n}$.\\
A $n \times n$ matrix $L(G)$ called the \textit{Laplacian matrix} of $G$ as follows.
\[
  L(G)_{ij} =
  \begin{cases}
    indeg(v_i) & \mbox{ if } i = j \\
    -1   & \mbox{ if }~ i \neq j ~\text{and}~(v_i,v_j)\in E \\
    0    & \mbox{ otherwise (When there is no edge) }
  \end{cases}
\]
What theorem says,the number of spanning arborescences rooted at some vertex n is given by exactly equal to $det(L_G[i])$ Here i=n.\\
Quickly Recap Definition of Spanning arborescences :An \textit{Arborescence} is a directed graph in which a vertex $u$ is called the root and for every other vertex $v$ in the graph, there is exactly one directed path from $u$ to $v$. In simpler terms, an arborescence is an directed tree in which all the edges are directed away from the root. A \textit{Spanning Arborescence} $S(V,E)$ of a directed graph $G(V',E')$ is an arborescence such that $V=V'$ and $E\subseteq E'$.\\
One condition is that there is a directed path from V to every vertex in G within E'.\\
Second condition is underlying undirected graph should be a tree. \\ \\
Recall the following from the last lecture.
\begin{align}
  det(L_G[n]) = \sum_{\sigma \in S_{n-1}}~
  Sign(\sigma)~\prod_{i=1}^{n-1}L_{i\sigma(i)}
  \label{eq:to_prove:Tutte}
\end{align} \\
$S(n-1)$ is notation for permutation of $n-1$.
This is expression of determinant that we have seen in last lecture.
So we want to show it is equal to spanning arborescence we introduce concept called Spreg.So what is Spreg $?$.
We quickly define Spreg.
\begin{definition}\textbf{Spregs:} Single prdecessor graphs or \textit{Spregs} with distinguished vertex $v$ of a directed graph $G(V,E)$ is a subgraph $T(V,E')$, $E' \subseteq E$, such that each vertex in $T$ except the vertex $v$ has exactly one predecessor and the vertex $v$ has no predecessors. In other words; in the spreg T, $indeg(v) = 0$ and for every $u \neq v$, $indeg(u)=1$.
\end{definition}
Every vertex other than distinguished vertex must be indegree 1.Subgraph of G is called Spregs.So there are several spregs are possible similar to several spanning arborescence possible.\\
We want to count spanning arborescence using spregs.That will be very nice combinatorial interpretation of this topic,that is plan.\\

%1. We want to count the number of spanning arborescence rooted at Vn.\\ \\
2.We want to count the number of spregs distinguished vertex Vn.\\
We associated with last lecture that every spanning arborescence corresponds to spregs .It turns out there are more spregs.There is some structure,Spregs looks like are of the form arborescence $+$ weekly connected component 1 $+$ weekly connected component 2......\\
V can not be part of cycle.V can be vertex in component.Weekly connecetd component has exactly 1 cycle.\\
So what we want to count spregs in inner circle with distinguished vertex Vn.Ok so already know what we have to count.Basic strategy is as follows:\\
Strategy is that count the number of spregs with distinguished vertex Vn which does not have any cycle.So this is what we want to count.We want to use Inclusion Exclusion.In fact we will not only count through Inclusion exclusion but we will go through each term of the determinant expression corresponds to our counting terms.\\
Let's demonstrate by writing down Inclusion exclusion.\\
In order to define Inclusion Exclusion formulation we need to define the \\ \\
1.Universe (Called as X).\\
2.Component $A_i$.\\
To define that Let's consider,notice that we want to count spregs without any cycle.\\
We need to be set of all spregs with distinguished vertex $V_n$.\\
Let $C_1,C_2,C_3, \dots , C_n$ be the set of cycles in graph G.We looking at simple cycle.\\
$A_i$ define as spregs which contain the cycle $C_1$.The number of spregs which does not contain any cycle.This what we want to count.So,\\

$|X|$ $-$ $|\bigcup_{i=1}^{n} A_i|$
It is same as,
$$|X| - \sum_{I \subseteq [n] , I \neq \emptyset} (-1)^{|I|+1}|\bigcap_{i \in I} A_i|$$
So already applied Inclusion Exclusion,Now write it as
$$ |X| + \sum_{I \subseteq [n] , I \neq \emptyset} (-1)^{|I|+1}|\bigcap_{i \in I} A_i| $$

So basically we need to count this.We can say that each term in these expression are exactly same as term used in determinant expression.As number of Spregs are equal to number of spanning arborescence,hence theorem is called.This is strategy.Hence each term is associated with terms of determinant theorem.\\
\textbf{Observation:}Let's consider C1,C2 be two cycles.\\
What can say about $$ A1 \bigcap A2 = \emptyset iff C1
  \bigcap C2 \neq \emptyset.$$\\
Every spregs looks like arborescence + uniquely weekly connected component 1 +uniquely weekly connected component 2.\\
$$ If C1 \bigcap C2 \neq \emptyset then |A1 \bigcap A2| =0.$$\\
Using the observation,\\
The number of spregs with distinguished vertex Vn which is acyclic is nothing but \\
$$ |X| + \sum_{I \subseteq [n] ,I \neq \emptyset ,for J,K \in I, J \neq K, cj \bigcap ck =\emptyset}       (-1)^{|I|+1}|\bigcap_{i \in I} A_i| $$

\textbf{Recall:} We counted |X| =Total number of spregs\\
$$det(L_G[n]) = \sum_{\sigma \in S_n} Sign(\sigma)  \prod_{i=1}^{n} a_{i\sigma(i)}$$\\
This is determinant expression. term   corresponds   to   $$ \sigma=id $$
$$\prod_{i=1}^{n-1} (L_i) = \prod_{i=1}^{n-1} deg_in(L_i) $$

Now we count the singleton sized I.
Spregs that contain exactly one cycle.
Fix,I={1},We are counting spregs which contain only C1.
Let  CI be (Vi1,Vi2,Vi3,....Vin)

$$\sigma(i1)=i2$$
$$\sigma(i2)=i3$$
$$\sigma(ik-1)=ik$$
$$\sigma(ik)=iI$$
$$\sigma \in S_n-1$$
$$\forall other  i \in {1...n} $$
$$ \sigma(i)=i $$

This is permutation which contain one cycle.
Now we just able to do following

\textbf{Claim:} Term in the determinant expression for
$$LG[n] corresponds to \sigma=number of spregs which contain C1$$
Right now we are talking about one circle.
Let's see proof of this.
Proof of this is very natural.

$$sign(\sigma)\prod_{i=1}^{n-1} L_i,\sigma(i   $$
sign of permutation is exactly equal to
$(-1)^{(l-1)} $.Here length is l.

Second term is
$$ (-1)^{(l-1)} (\prod_i,|\sigma(i)|=i L_i,i)(\prod_i,|\sigma(i)| \neq i L_i,\sigma(i)) $$
$$ L_i,\sigma(i) =-1,if V_i,V_\sigma(i) is edge in graph \in E\\ ootherwise  $$
If corresponding edge is present then this term is non zero otherwise zero.\\
$$ (-1)^{(l-1)}(\prod_i,|\sigma(i)| deg_in(v_i))*(-1)^{l} $$
$$= (-1)^{(2l-1)} |number of spregs which contain C1|  $$
$$= -|number of spregs which contain C1 | $$
Proposition for $$ \sigma \in S_n-1 comsisting of single cycle.   $$$$ \sigma =(i1,i2,.....il)   $$
Associate$ C\sigma $ as a cycle corresponds to vertex sequence (Vi1,Vi2,Vi3.....Vil,Vii)\\
Then the term in the determinant corresponds to $\sigma$ satisfies following.\\
$$sgn(\sigma)(\prod_{i=1}^{n-1}L_i,\sigma(i))$$

$$-(\prod_{i,\sigma(i)=i}deg_in(v_i))C_\sigma \subseteq G(V,E) $$\\
$$ 0 if C\sigma \nsubseteq G(V,E) $$ \\
$$-1  if C\sigma \subseteq G(V,E) if \forall i \sigma(i) \neq i     $$\\

\textbf{Corollary:}$$For \sigma,C\sigma as above $$\\
$$ |\prod_{i=1}^{n-1}L_i,\sigma(i)|=number of spregs which contain C_\sigma   $$\\
Now this is case for single cycle.\\Now we will generalise case for multiple cycle.\\
So it is very natural.

\textbf{Question:}Suppose $\sigma \in S_{n-1}$ is the product of $k>0$ disjoint cycle.
$$\sigma=(i11,i12,i13.....iil) (i21,i22,i23....(ikl,ik2,ik3...ikl)   $$
k is number of cycles. So now we are associate
$$ C_\sigma = \cup  {j=1}^{k}C_j  where C_j is (V_ij1....V_ijLj.....VijI)$$

Then the term  corresponds to $\sigma$ in $det(L_G[n])$
$$sgn(\sigma) = (-1)^{K} \prod_{i|\sigma(i)|}deg_in(V_i) if C_\sigma \subseteq G   $$\\
$$ o if C\sigma \nsubseteq G $$

\textbf{Corollary:}If we look at\\
$$|\prod_{i=1}^{n}L_i,\sigma(i)|=number of spregs which contain C_\sigma = \bigcup_{j=1}^{k}C_j   $$\\
$$ det(L_G[n]) =\sum_{\sigma \in S_n-1}sig(\sigma) \prod_{i=1}^{n} L_i\sigma      $$\\
This is how determinant expression looks like.\\
Number of spanning arborescence rooted at Vn  as distinguished vertex and not containing cycle.\\
%$$&=|X| + \sum_{I \subseteq [n] ,I \neq \emptyset, cj \bigcap   ck=\emptyset ,j,k \in I}(-1)^I    $$\\
This is strategy.
This lecture is combinatorial application of PIE.

\Lecture{Jayalal Sarma}{Oct 10, 2020}{12}{Algorithmic Application of PIE}{Shrinidhi Bajpayee}{$\alpha$}{JS}

\section{Introduction} We talk about counting problem.Its about given a $n*n$ bipartite graph,We want to count the number of perfect matching in it?\\
subset of edges such that every vertex has exactly one edge incident on it from the
subset.\\
Bipartite adjacency matrix is different from adjacency matrix.\\ \\
\textbf{Trivial Algorithm:}Run through all n sized subsets of E.\\
$${n^{2} \choose n } \sim {n^{2} \choose n}^{n}    \sim n^{n} \sim 2^{nlogn}$$\\

$$ {n \choose k}^{k} \leq {n \choose k} \leq {n_e \choose k}^{k} $$\\
e=natural log base \\
\section{Decision Problem:}
\textbf{Binpacking Problem:}
Given a positive integer (bin capacity B),positive integer k\\
We have n item with weight S1,S2,S3.....,Sn\\
partition the items into u1,u2,u3.....,uk\\
Capacity is such that sum of weights of items in each ui is almost B.\\
This is binpacking problem.\\
It is NP-problem.\\
By PIE Time complexity is $O(nB2^{n})$\\
n=number of items\\
B=capacity\\
Now we will reformulate PIE.\\

Given a collection of N combinatorial objects \\
Let p(1),p(2),p(3)...p(n) be properties.\\
$p_i$ is essential function \\
$$ N \to {0,1}   $$\\
It is possible that some object doesn't have properties of that.\\
$$N_i \to Number of objects among N with property P(i).$$\\
$$ {i_1,i_2,....i_r} \subseteq {1,2,3,...N} $$\\
$$N_{i_1,i_2,i_3....i_r} = Number of objects with properties p(i_1) p(i_2)....p(i_r) $$
N(0)=Number of objects not having any of the property.\\
$$ N(0)=N-\sum_{i=1}^{n} N_{i_1} +\sum_{i_1<i_2}N_{i_1,i_2} -\sum_{i_1 < i_2 <i_3}.......(-1)^{j} \sum_{i_1<i_2<...i_j}N_{i_1,i_2...i_j} (-1)^{x}N_1...r   $$\\
Restatement (Complementary term)(For the discussion)\\
$$N \to Number of objects.$$\\
$$Q(1),Q(2),....Q(n) $$be properties that some of these objects have.\\
$$W \subseteq {1,2,....n}  $$\\
Let N(W) be the number of objects having none of the properties Q(i).\\
$$i \in W   $$\\
Now what we want to count the other set.\\
$$X \to number of objects which has all the properties.$$\\
$$X = \sum_{W \subseteq [n]}(-1)^{|W|} N(W)   $$\\
\textbf{Pf:}$Define P(i) iff \neg Q(i)$\\
Now the above restatement is applicable.\\
Given a bipartite graph $G(u,v,E)\\
  |u|=|v|=n $\\
count the number of perfect matching.\\
\textbf{I/p:}Bipartite adjacency matrix.\\
\includegraphics{images/graph.jpg}\\ \\
There are 2 perfect matching is in these graph.\\
Matching 1: There is bijection.\\
There are 4!=24 permutation.But matching 1 and matching 2 are two permutation for perfect matching.\\
How's the algebric way of writing itself.\\
Let's write $\sigma \in S_n$ \\
Consider,\\
$$ \prod_{i=1}{n} a_i \sigma(i)$$\\
When $a_i \sigma_i$ is non zero value.
When $i\sigma _i$ is true.\\
$$\prod a_ii = a11 .a22 .a33. a44$$\\
$$ 1 $$\\
What will be product of \\
$$\prod_{i=1}{n} a_i \sigma(i) $$\\
$$ a12. a21.a33.a44  $$\\
$$ 0  $$\\
If corresponding edge is there in graph then value will be 1 otherwise 0.\\
a12=0 since no edge from 1 to 2.\\
Let's write,\\
Permanant of A=\\
$$ \sum_{\sigma \in S_n}  = Number of perfect matching in the graph.$$\\
Given a matrix A,\\
Its two types are:\\
!)permutataion (A) that is per(A): It is number of perfect matching in graph.\\
2)Determinant (A) that is det(A)\\
\\
\textbf{Problem:}Given a matrix A o/p the value of per(A)\\
\textbf{Trivial Algorithm:}Run over all $\sigma \in S_n$ compute the product \\
$$\prod_{i=1}{n}a_i \sigma(i)$$ and add.\\
$$O(n!) \sim O(n ^{n}) \sim 2^{nlogn} $$\\
\textbf{Lemma:}A is bipartite adjacency matrix of a bipartite graph G\\
$$Per(A) = \sum_{W \in {1...n}}(-1)^{|W|} \prod_{i=1}^{n}(\sum_{j \notin W}a_ij)$$\\
Applying Lemma, $2^{n}$ W's and $n^{2}=remaining Computation$\\
$2^{n}n^{2}$ time algorithm.\\
\textbf{PF:}To apply complementary form PIE.\\
Define N objects: $M \in N$\\
$M \subseteq E$ such that for  every \\
$\forall i \in {1...n}$ the vertex $X_i$ is the endpoint of some vertex in M.\\
$\forall j \in [n] Q(j)$: Vertex $Y_j$ is the endpoint of some vertex in M.\\
We want to count number of objects from underlined part which satisfies Q(1) .....Q(n)\\
The number of perfect matching=per(A)=number of objects which satisfies all of Q(1) Q(2)....Q(n)\\
Q(1) says first one is matched.\\
Q(2) says second one is matched.\\
Like this etc etc.\\
When all vertex are matched from both side.\\
$$ \sum_{W \in {1...n}}(-1)^{|W|}$$\\
N(W)=Number of objects which satisfies none of property in W.\\
\includegraphics{images/img.jpg}\\ \\
Per(A)=Number of perfect matching.\\
$$ \sum_{W \subseteq {1...n}}(-1)^{|W|}(\prod_{i=1}^{n}(\sum_{j \in w}aij)) $$\\
This completes the proof of the Lemma.\\
This lemma gives $O(2^{n}n^{2})$ algorithm for computing number of perfect matching in a given bipartite graph.\\

\textbf{Quickly Recap:}\\
\textbf{Binpacking problem:}\\
$Bin capacity B,We have number of bins K,We have n items S1....Sn are sizes.$
Question is Can we partition items to $u1....uk[n]$ such that
%$$\forall_{i\leq j \leq k} j\sum_{i \in u}Si \leq B}$$
  We want to count this.\\
  \textbf{Algorithm:}\\
  \textbf{Trivial Algorithm:}Run through partition.\\
$$ n \choose k$$\\
$O(nB2^{n})$ time space.\\
A partition of [n] into u1....uk is said to be feasible if the sum of the sizes of item $\leq$ B in uj.\\
\textbf{Relax:}Items can appear more than once.\\
\textbf{observation:}A feasible solution with relaxation will remain feasible without relaxation.\\
\textbf{Relaxed Solution:}ordered set of K lists of items from ${1...n}$.\\
1.Each of the elements in ${1...n}$ appears atleast in one list.\\
2.For each list $a1,a2....ap$
$$\sum_{h=1}{p}S_ak \leq B $$\\
To apply PIE,we need to define objects $Q1....$\\
\textbf{Objects:}Ordered sets of K list of elements from {1...n} such that for each list (a1...ap) $$\sum_{h=1}{p}S_an \leq B $$\\
Q(1):=The ordered list of k lists contain 1 in atleast one list in it.\\
.\\
.\\
.\\
Q(W):=W  is contained in atleast one of the lists in the ordered set of list.\\
.\\
.\\
.\\
Q(n)\\
X=Set of objects which satisfy all of the properties Q(1)....Q(n)\\
$$X=\sum_{w \subseteq [n]}(-1)^{|w|} N(W) $$\\
\textbf{Aim:}To compute N(W)\\
A(W):=Number of list a1....ap of elements list in W.\\
$$ \sum_{h=1}{p}S(an) \leq B $$\\
$$N(W)=(A(W))^{k}  $$\\
\textbf{Aim:}Compute A(W)\\
$P_w(j)$ =number of list a1....ap of which element not in w such that
$$\sum_{h=1}{p}S(ak)=j  $$\\
$$A(w)=\sum_{j=0}{B}P_w(j)$$\\
$$P_w(j)=\sum_{i \notin w}P_W(j-s(w)) $$\\
$$ p_W(j)=0, j<0 $$\\
$$P_w(0)=1 $$\\
Time taken is $O(nB2^{n})$

% Lecture 3

\Lecture{Jayalal Sarma}{Oct 10, 2020}{13}{From Principle of Inclusion-Exclusions to Mobius Inversion}{Sandip Saha}{$\alpha$}{JS}

% \section{Introduction}
The journey so far has been that we have started with pigeon hole principle and its applications then we used double counting and bijections to establish different identities then we moved to PIE and its different applications.
% Then we came to Principle of Inclusion-Exclusion(PIE) as a consequence of a bijection argument.
In this lecture, we will look at a more generalized version of PIE and move gradually towards the Mobius Inversion.

\subsection{PIE Revisited}
Previously we have used PIE to compute $\left| \bigcup^n _{i=n} A_i \right|$ as follows,
\[ \left| \bigcup^n _{i=n} A_i \right|=\sum_{\emptyset \neq I \subseteq [n]} (-1)^{|I|+1}\left|\bigcap_{i\in I} A_i\right|\]
Now, we are interested to compute $\left|\bigcap_{i=1}^n  \overline{A_i}\right|$, which can be rewritten as,
\begin{align}
  \left|\bigcap_{i=1}^n  \overline{A_i}\right| & = \left|
  \overline{\bigcup_{i=1}^n  A_i}
  \right| \nonumber \tag{using De-morgan's law}                                                                                \\
                                               & = \left| X \right| - \left| \bigcup^n _{i=n} A_i \right| \nonumber                                                \\
                                               & = \left| X \right| -\sum_{\emptyset \neq I \subseteq [n]} (-1)^{|I|+1}\left|\bigcap_{i\in I} A_i\right| \nonumber \\
                                               & = \sum_{ I \subseteq [n]} (-1)^{|I|}\left|\bigcap_{i\in I} A_i\right|
\end{align}

Note that $\bigcap_{i\in \emptyset} A_i =X$. This thing can also be stated in the following way.

\begin{theorem}
  Let $X$ be a finite set and $P_1,\dots, P_m$ properties. Further define for $S \subseteq [m]$ the set $N(S) = \{x\in X \mid \forall i \in S: x \text{ has property } P_i\}$  then, number of elements in $X$ satisfying none of the properties $P_1,\dots, P_m$ is given by,
  $$ \sum_{S \subseteq [m]} (-1)^{|S|} \left| N(S) \right|$$
\end{theorem}

\subsection{Stronger version of PIE}
The Inclusion-Exclusion Principle has a stronger version which is as follows.

\begin{theorem}[Stronger PIE]
  \label{stronger PIE}
  Let $f,g:2^{[n]}\longrightarrow \mathbb{R}$ are functions assigning real numbers to subsets of $[n]$ with the property that for any $A\subseteq [n]$

  \[ g(A)=\sum_{S\subseteq A} f(S)\]

  Then,
  \[ f(A) = \sum_{S \subseteq A} (-1)^{|A|-|S|}g(S)\]

\end{theorem}

\begin{proof}
  Let $f$ and $g$ are functions from the powerset of $[n]$ to real numbers and for all $A \subseteq [n]$
  \[g(A)=\sum_{S\subseteq A}f(S)\]

  \begin{align}
    \sum _{s\subseteq A}( -1)^{|A|-|S|} g( S) & =\sum _{s\subseteq A}( -1)^{|A|-|S|}\left(\sum _{T\subseteq A} f( T)\right) \nonumber \\
                                              & =\sum _{T\subseteq A} C_{T} f( T)
  \end{align}
  Where $C_T$ is appropriate signed number. Our aim is now to find $C_T$ for different $T$

  \begin{description}
    \item{\textbf{Case 1: }($T=A$)} $C_T=1$, since $f(A)$ is only encountered for $T=S=A$.

    \item{\textbf{Case 2: }($T \neq A$)} choosing a set between $T$ and $A$ is equivalent of choosing a set from $A\setminus T$
          \[C_T=\sum _{T \subseteq S\subseteq A}( -1)^{|A|-|S|}=\sum^k_{i=0}(-1)^{k-1} {k \choose i} =0 \]
  \end{description}
  This proves the claim.
  \[\boxed{\therefore f(A) =  \sum_{S \subseteq A} (-1)^{|A|-|S|}g(S)}\]
\end{proof}

\textbf{Why it is strong version?} it implies PIE.
Assume the strong version, we can derive PIE.
\begin{proof}
  Properties $P_1,\dots, P_m$ of elements of $X$ . $X_1,\dots, X_m$ are the subsets of $X$ satisfying the respective property i.e. $X_i=\{ x\in X \mid x \text{ satisfy } P_i\}$.
  then for $S \subseteq [m]$  we define $f(S)$ to be the number of elements in $X$ having all properties $P_i$ such that $i \notin S$ and none of the properties $P_j$ such that $ j \in S$ i.e.
  \[ f(S) = \left| \bigcap_{i\in [m] \setminus S} X_i \setminus \bigcup_{i\in S} X_i \right| \]
  We are interested in counting the number of elements in $X$ which does not satisfy any property in $P_1,\dots, P_m$ i.e. $f([m])$. We define,

  \begin{align*}
    g(A) & = \sum_{S\subseteq A} f(S)                         \\
         & = \left| \bigcap_{i \in [m] \setminus A}X_i\right| \\
         & = N([m]\setminus A)
  \end{align*}
  $g(A)$ counts $x\in X$ if the property that $x$ does not satisfy forms a subset of $A$. By \ref{stronger PIE}
  \begin{align*}
    f([m]) & = \sum_{S\subseteq [m]}(-1)^{m-|S|} g(S) \\
           & = \sum_{\substack{
    S'\subseteq [m]\setminus S                        \\
        S\subseteq [m]}}
    (-1)^{|S'|} N(S')                                 \\
           & = \sum_{S \subseteq [m]} (-1)^{|S|} N(S)
  \end{align*}
  This concludes the proof.
\end{proof}


\documentclass{article}
\usepackage{fullpage}
\usepackage{nopageno}
\begin{document}
\title{Section 1.1 Problems 2, 18, 26, 46}
\author{Jon Allen}
\date{September 4, 2013}
\maketitle
\section*{\#2}
Determine if each of the following equations is an ODE or a PDE. If the equation is an ODE, then determine (a) the order of the ODE and (b) if the equation is leinear or nonlinear.

$y\frac{\mathrm{d}y}{\mathrm{d}x}+y^4=\sin x$

The equation is an ODE of order one. It is nonlinear as neither the $y\frac{\mathrm{d}y}{\mathrm{d}x}$ term or the $y^4$ term fit the definition for linearity.
\section*{\#18}
Verify that the given function is a solution to the corresponding differential equation.

$\mathrm{d}y/\mathrm{d}x+xy=0$, $y(x)=e^{-x^2/2}$

We find $\frac{\mathrm{d}y}{\mathrm{d}x}=-xe^{-x^2/2}$. Substituting this result into the differential equation we find $-xe^{-x^2/2}+x\cdot e^{-x^2/2}=0$ which is true. The given function is a solution to the differential equation.
\section*{\#26}
Verify that the given function is a solution to the corresponding differential equation.

$y'''-2y''=0$, $y=A+Bt+Ce^{2t}$

We start by finding the first, second and third derivatives of $y$. $y'=B+2Ce^{2t}$, $y''=4Ce^{2t}$, $y'''=8Ce^{2t}$. Substituting $y'''$ and $y''$ into the differential we have $8Ce^{2t}-2\cdot4Ce^{2t}=0$. We see that the function $y$ is a solution to the differential equation regardless of the values of $A$, $B$, and $C$.
\section*{\#46}
Use the indicated conditions with the indicated solution to determine the solution to the given problem.

$\mathrm{d}y/\mathrm{d}t+y=\sin t$, $y(0)=-1$, $y(t)=Ae^{-t}-\frac{1}{2}\cos t+\frac{1}{2}\sin t$

First we substitute the initial value of $y(0)=-1$ into $y(t)$ to obtain $-1=Ae^0-\frac{1}{2}\cdot1+\frac{1}{2}\cdot0=A-\frac{1}{2}$ or $A=-\frac{1}{2}$. Our solution then is $y(t)=-\frac{1}{2}\left(e^{-t}+\cos t-\sin t\right)$
\end{document}


% This is part of the TFTB Reference Manual.
% Copyright (C) 1996 CNRS (France) and Rice University (US).
% See the file refguide.tex for copying conditions.

\markright{mexhat}
\section*{\hspace*{-1.6cm} mexhat}

\vspace*{-.4cm}
\hspace*{-1.6cm}\rule[0in]{16.5cm}{.02cm}
\vspace*{.2cm}

{\bf \large \sf Purpose}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
Mexican hat wavelet in time domain.
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Synopsis}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
h = mexhat
h = mexhat(nu)
\end{verbatim}
\end{minipage}
\vspace*{.5cm}

{\bf \large \sf Description}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
        {\ty mexhat} returns the mexican hat wavelet, with central
        frequency {\ty nu} ({\ty nu} is a normalized frequency). Its
        expression writes
\[h(t)=\nu\ \frac{\sqrt{\pi}}{2}\ (1-2(\pi\nu t)^2)\ \exp[-(\pi\nu\ t)^2].\]

\hspace*{-.5cm}\begin{tabular*}{14cm}{p{1.5cm} p{8.5cm} c}
Name & Description & Default value\\
\hline
        {\ty nu} & any real between 0 and 0.5         & {\ty 0.05} \\
 \hline {\ty h}  & time vector containing the mexhat samples\\
                 & {\ty length(h)=2*ceil(1.5/nu)+1}\\
\hline
\end{tabular*}

\end{minipage}
\vspace*{1cm}

{\bf \large \sf Example}
\begin{verbatim}
         plot(mexhat);
\end{verbatim}
\vspace*{.5cm}

{\bf \large \sf See Also}\\
\hspace*{1.5cm}
\begin{minipage}[t]{13.5cm}
\begin{verbatim}
klauder.
\end{verbatim}
\end{minipage}


\documentclass{scrartcl}

\input{../../shared.tex}

\begin{document}

\section*{Exercises on (quasi) Newton methods}

\paragraph{Exercise (i)} (2P) Show that Newton's method applied to a nondegenerate quadratic function yields a Quasi-Newton method.\\
\textbf{Bonus points:} Show that this is an \textit{if and only if}.

\paragraph{Exercise (ii)} (2P) Consider a step of the secant method:
        \begin{equation}
          x_{k+1} = x_k - f(x_k)\frac{x_k-x_{k-1}}{f(x_k)-f(x_{k-1})}.
        \end{equation}
        Assuming that $x_k\neq x_{k-1}$ and $f(x_k)\neq f(x_{k-1})$, prove that the line through the two points $(x_{k-1}, f(x_{k-1}))$ and $(x_k, f(x_k))$ intersects the $x$-axis at the point $x=x_{k+1}$.
\paragraph{Exercise (iii)} (6P) Complete the notebook.

\end{document}


\section[sct:distance]{Extracting Distances}

\distance{} prints distances between nodes, in various ways.  By default, it
prints the distance from the root of the tree to each labeled leaf, in \nw{}
order. Let's look at distances in the catarrhinian tree:

\svgFigure{display_3}

\txtCmdOutput{dist_1}

\noindent{}This means that the distance from the root to \id{Gorilla} is
56, etc. The distances are in the same units as the tree's branch lengths --
usually substitutions per site, but this is not specified in the tree itself.
If the tree is a cladogram, the distances are expressed in numbers of
ancestors.  Option \code{-n} shows the labels:

\txtCmdOutput{dist_2}

There are two main parameters to \distance: the {\em method} and the
{\em selection}. The method determines how to compute the distance (from what
node to what node), and the selection determines for which nodes the program is
to compute distances. Let's look at examples.

\subsection{Selection}

In this section we will show the different selection types, using the default
distance method (\ie{}, from the tree's root -- see below). The selection type
is the argument to option \code{-s}.  The nodes appear in the same order as
in the Newick tree, except when they are specified on the command line (see
below).

To illustrate the selection types, we need a tree that has both labeled and
unlabeled leaves and inner nodes. Here it is:

\svgCmdOutput{dist_3}

We will use option \code{-n} to see the node labels.

\subsubsection{All labeled leaves}

The selection consists of all leaves with a label. This is the default, as
leaves will mostly be labeled and we're generally more interested in leaves
than inner nodes.
\txtCmdOutput{dist_4}

\subsubsection{All labeled nodes}

Option \code{-s l}. This takes all labeled nodes into account, whether they are leaves or inner nodes.
\txtCmdOutput{dist_5}

\subsubsection{All leaves}

Option \code{-s f}. Selects all leaves, whether they are
labeled or not.
\txtCmdOutput{dist_6}

\subsubsection{All inner nodes}

Option \code{-s i}. Selects the inner nodes, labeled or not.
\txtCmdOutput{dist_9}

\subsubsection{All nodes}

Option \code{-s a}. All nodes are selected.
\txtCmdOutput{dist_7}

\subsubsection{Command line selection}

The selection consists of the nodes whose labels are passed as arguments on the
command line (after the file name). The distances are printed in the same
order.

\txtCmdOutput{dist_8}

\subsection{Methods}

In this section we will take the default selection and vary the method. The
method is passed as argument to option \code{-m}. I will also use an
\adhoc{} tree to illustrate the methods:

\svgCmdOutput{dist_11}

\noindent{}As explained above, the default selection consists of all labeled
leaves -- in our case, nodes \id{A}, \id{B} and \id{C}.

\subsubsection{Distance from the tree's root}

This is the default method: for each node in the selection, the program prints
the distance from the tree's root to the node. This was shown above, so I
won't repeat it here.

\subsubsection{Distance from the last common ancestor}
Option \code{-m l}. The program computes the \lca{} of all nodes in the
selection (in our case, node \id{e}), and prints out the distance from
that node to all nodes in the selection.
\txtCmdOutput{dist_10}

\subsubsection{Distance from the parent}
Option \code{-m p}. The program prints the length of each selected node's
parent branch.
\txtCmdOutput{dist_12}

\subsubsection{Matrix}
Option \code{-m m}. Computes the pairwise distances between all nodes in the
selection, and prints it out as a matrix.
\txtCmdOutput{dist_13}

\subsection{Alternative formats}

Option \code{-t} changes the output format. For matrix output, (\code{-m
m}), the matrix is triangular.
\txtCmdOutput{dist_16}
When labels are printed (option \code{-n}), the diagonal is shown
\txtCmdOutput{dist_14}

For all other formats, the values are printed in a line, separated by
\cap{tab}s.
\txtCmdOutput{dist_15}


\SecDef{minimalist}{Minimalist Quadratic Masking Scheme}

In this section I show a first-order algebraically secure quadratic masking scheme. Then I describe concrete circuits which can be verified to be first-order algebraically secure gadgets using \AlgRef{verify}.

\subsubsection{Minimalist Quadratic Masking.}
Since the decoding function has to be at least quadratic, we need at least two bits to encode a single bit. For two bits all nonlinear decoding functions are linear equivalent to a quadratic monomial being simply the product of the two input bits. Unfortunately, this decoding function is vulnerable to the linear algebra attack. Any quadratic function with 2-bit input is unbalanced. Therefore, one of the reference bit values can be encoded by 3 different values and the other value has only 1 possible encoding. For example, if the value is equal to 1 and the decoding function is simply AND, the input has to be equal to (1, 1). In this case there is no randomness involved and the hidden value is leaked.
The conclusion is that any value of the original bit should include randomness in its encoding. In particular, the decoding function can not be a point function.

We move on to 3-bit encodings. The simplest quadratic function using all 3 input bits $a,b,c$ is $ab\oplus c$. Note the similarity with the broken 2-bit scheme: the quadratic monomial $ab$ is simply linearly masked by $c$. However, this linear mask is enough to prevent the attack: in this case $\dec(a,b,c)=1$ does not imply $a=1$ or $b=1$. In fact, such $\dec$ is balanced: both 0 and 1 have exactly 4 preimages. We first describe an insecure yet simple masking scheme based on this decoding function in \FigRef{insecure}. It is easy to verify that $Eval_{XOR}$ and $Eval_{AND}$ satisfy the requirements from Definition~\ref{wba.def:masking}. In addition, $Refresh(a, r)$ returns fresh random encoding of $a$, meaning that $\dec(a) = \dec(Refresh(a, r))$ for any $r$ and new encoding reveals no information about the old encoding.

\begin{figure}
\vspace{-0.5em}
\FigDef{insecure}{An Insecure Quadratic Masking Scheme.}
\vspace{-0.5em}
\begin{align}
    \enc(x,r_a,r_b)              &~=~ (r_a,r_b,r_ar_b \oplus x),\\
    \dec(a,b,c)                  &~=~ ab \oplus c,\\
    Eval_{XOR}((a,b,c),(d,e,f)) &~=~ (a\oplus d,~ b \oplus e,~  ae \oplus bd \oplus c \oplus f),\\
    Eval_{AND}((a,b,c),(d,e,f)) &~=~ (ae,~        bd,~         (cd)e \oplus a(bf) \oplus cf),\\
    Refresh((a,b,c),(r_a,r_b))  &~=~ (a \oplus r_a,~ b \oplus r_b,~ c \oplus r_ab \oplus r_ba \oplus r_a r_b).
\end{align}
\vspace{-0.5em}
\end{figure}

We now observe that $Refresh$ is not $\maxcor$-1-AS for any $\maxcor < 1$: the computed term $r_a b$ is constant when $b$ is fixed to 0 and equals to $r_a$ otherwise (leading to $\maxcor = 1$). This can be fixed by using an extra random bit $r_c$ to mask $a,b$ through the computations:
\begin{multline}
    Refresh((a,b,c),(r_a,r_b,r_c)) ~=~ \\
    \big(
    a \oplus r_a,~
    b \oplus r_b,~
    c \oplus r_a(b \oplus r_c) \oplus r_b (a \oplus r_c) \oplus (r_a \oplus r_c)(r_b \oplus r_c) \oplus r_c
    \big).
\end{multline}

The new $Refresh$ function can be verified to be secure using the algorithm from \SecRef{verify}. Moreover, the circuit computing $Eval_{XOR}$ applied to refreshed inputs is secure as well. However, $Eval_{AND}$ is not secure even if composed with the fixed $Refresh$ gadget. Consider the linear combination of computed terms $a(bf) \oplus cf = (ab \oplus c)f$. Here the variables are refreshed masks and can not be fixed by the adversary. However, the refreshing function does not change the hidden value. Therefore, $ab \oplus c$ would be equal to the value hidden by initial non-refreshed shares which can be fixed. Fixing the hidden value to 0 makes the combination $f(ab \oplus c)$ equal to 0 and be equal to the random share $f$ when the hidden value is fixed to 1. We observe that it is possible to use a trick similar to the one used to fix the $Refresh$ function. In fact, the extra random shares added to fix the $Refresh$ function may be reused to fix the $Eval_{AND}$ function. As a result, we obtain a fully secure masking scheme. The complete description is given in \AlgRef{mqms}.

\begin{algorithm}[ht]
    \AlgDef{mqms}{Minimalist Quadratic Masking Scheme.}
    \begin{algorithmic}[1]
        \Function{Encode}{$x,r_a,r_b$}
            \State \Return{$(r_a,r_b,r_ar_b \oplus x)$}
        \EndFunction
        \Statex
        \Function{Decode}{$a,b,c$}
            \State \Return{$ab\oplus c$}
        \EndFunction
        \Statex
        \Function{EvalXOR}{$(a,b,c),(d,e,f),(r_a,r_b,r_c),(r_d,r_e,r_f)$}
            \State $(a,b,c) \gets \textsc{Refresh}((a,b,c),(r_a,r_b,r_c))$
            \State $(d,e,f) \gets \textsc{Refresh}((d,e,f),(r_d,r_e,r_f))$
            \State $x \gets a \oplus d$
            \State $y \gets b \oplus e$
            \State $z \gets c \oplus f \oplus ae \oplus bd$
            \State \Return{$(x,y,z)$}
        \EndFunction
        \Statex
        \Function{EvalAND}{$(a,b,c),(d,e,f),(r_a,r_b,r_c),(r_d,r_e,r_f)$}
            \State $(a,b,c) \gets \textsc{Refresh}((a,b,c),(r_a,r_b,r_c))$
            \State $(d,e,f) \gets \textsc{Refresh}((d,e,f),(r_d,r_e,r_f))$
            \State $m_a \gets bf \oplus r_c e$
            \State $m_d \gets ce \oplus r_f b$
            \State $x \gets ae \oplus r_f$
            \State $y \gets bd \oplus r_c$
            \State $z \gets am_a \oplus dm_d \oplus r_c r_f \oplus cf$
            \State \Return{$(x,y,z)$}
        \EndFunction
        \Statex
        \Function{Refresh}{$(a,b,c),(r_a,r_b,r_c)$}
            \State $m_a \gets r_a \cdot (b \oplus r_c)$
            \State $m_b \gets r_b \cdot (a \oplus r_c)$
            \State $r_c \gets
                m_a \oplus m_b \oplus
                (r_a \oplus r_c) (r_b \oplus r_c) \oplus
                r_c$
            \State $a \gets a \oplus r_a$
            \State $b \gets b \oplus r_b$
            \State $c \gets c \oplus r_c$
            \State \Return{$(a,b,c)$}
        \EndFunction
    \end{algorithmic}
\end{algorithm}

\paragraph{Security.}
First, we verify $Eval_{XOR}$ and $Eval_{AND}$ gadgets using \AlgRef{verify}. We obtain that they are $\maxcor$-1-AS circuits for some $\maxcor < 1$. Then we construct the ANFs of intermediate functions. The maximum degree is equal to 4. It is achieved for example in the term $cf$ in the gadget $Eval_{AND}$: its ANF contains the term $r_ar_br_dr_e$. Therefore, $Eval_{AND}$ is $\maxcor$-1-AS with $\maxcor \le 7/8$. The gadget $Eval_{XOR}$ has degree 2 and is $1/2$-1-AS. Unfortunately, we do not have a pen-and-paper proof for security of the gadgets and rely solely on the verification algorithm (which is able to spot the described weaknesses in the insecure versions of the gadgets).

Verifying security of the encoding function $\enc$ can be done in the same way. Clearly, no linear combination of $r_a,r_b,r_ar_b\oplus x$ is constant for any fixed $x$. The coordinate $r_ar_b\oplus x$ has degree 2 and its absolute correlation is equal to $1/2$. Therefore, $\enc$ is an $\maxcor$-1-AS encoding with $\maxcor=1/2$.

By applying \PropRef{combine}, we obtain that for any adversary $\adv$, for any circuit $C$ build from the gadgets $Eval_{XOR}, Eval_{AND}$ and for the described $\enc$ encoding we have:
\begin{equation}
    \advanPS[\adv] \le min(2^{Q-\rC}, 2^{eQ}),
\end{equation}

where $e = \log_2{(1+7/8)/2}\approx -0.093$. According to Corollary~\Ref{cor:seclevel}, in order to achieve provable 80-bit security we need to have $\rC \ge 80(1 - 1/e) \approx 940$ random bits in the circuit. Note that it does not depend on the actual size of the circuit, i.e. 940 random bits are enough for an arbitrary-sized circuit. However, the adversary should not be able to shrink the window so that it contains less than 940 random bits. This is expected to be guaranteed by a \SH{} protection. Finally, we remark that the bounds are rather loose and more fine-grained analysis should improve the bound significantly.

\subsection{Implementation}

We applied our masking scheme to an AES-128 implementation to estimate the overhead. Our reference AES circuit contains 31,783 gates. It is based on Canright's S-Box implementation~\cite{Canright} and naive implementation of MixColumns. After applying our nonlinear masking scheme and a first-order linear masking scheme on top the circuit expands to 2,588,743 gates of which 409,664 gates are special gates modeling external random bits. The circuit can be encoded in 16.5 MB. Extra RAM needed for computations is less than 1KB. On a common laptop it takes 0.05 seconds to encrypt 1 block. Since the implementation is bitwise, 64 blocks can be done in parallel at the same time on 64-bit platforms. There is still a large room for optimizations. We used the Daredevil CPA tool~\cite{SCM} to test our implementation. Due to the first-order linear masking on top we did not detect any leakage. Pure nonlinear masking scheme does leak the key so the combination of both is needed as we suggested in \SecRef{method}. The implementation code is publicly available~\cite{OurWhiteboxCode}. We remark that it is a proof-of-concept and not a secure white-box implementation; it can be broken in various ways.


\section{Dijkstra's Algorithm as Skeleton}

%%%%%%%%%%
\begin{frame}[fragile]{Dijkstra's algorithm}
  Invariant: maintain $R \subseteq V$: $\forall u \in R: s \leadsto u$ is known
  \begin{exampleblock}{3. How to update $\text{dist}[w]$ for $(v,w) \in E \land w \notin R$?}
    new estimator for $\text{dist}[w]$
    \begin{verbatim}
      forall (e = (v,w) in E) and (w not in R)
      if dist[w] > dist[v] + w(v,w)
        dist[w] = dist[v] + w(v,w)
    \end{verbatim}
  \end{exampleblock}
\end{frame}
%%%%%%%%%%
\begin{frame}[fragile]{Dijkstra's algorithm as skeleton}
  \begin{verbatim}
    Initialization:
      dist[s] = 0;
      dist[v] = infty for others

    Q = MakePriorityQueue(V) with dist[v] as keys
                       (using min-heap)

    while (Q is not empty)
      v = deleteMin(Q)
      foreach edge (v,w) in E // w in Q
        if dist[w] > dist[v] + w(v,w)
          dist[w] = dist[v] + w(v,w)
          decreaseKey(Q, w)
        else if dist[w] = dist[v] + w(v,w)
        //  do nothing
  \end{verbatim}

  \begin{alertblock}{How to use this skeleton?}
    \begin{enumerate}
      \item What does $R/Q$ maintain?
      \item How to update estimators?
	\begin{itemize}
	  \item invalidated; following the new one
	  \item amending the old one
	\end{itemize}
    \end{enumerate}
  \end{alertblock}
\end{frame}
%%%%%%%%%%
\begin{frame}[fragile]{Dijkstra's algorithm as skeleton}
  \begin{exampleblock}{Uniqueness of shortest path \pno{3.7.7}}
    \begin{itemize}
      \item $G = (V, E, w), s \in V, w(e) > 0$
      \item \textcolor{gray}{Is shortest path $s \leadsto t$ unique?}
      \item $\forall t$: compute the number of shortest paths from $s$ to $t$.
    \end{itemize}
  \end{exampleblock}
\end{frame}
%%%%%%%%%%
\begin{frame}[fragile]{Dijkstra's algorithm as skeleton}
  \begin{block}{Solution.}
    \begin{verbatim}
      Initialization:
        num[s] = 1; num[v] = 0 for others
        usp[s] = true; usp[v] = false for others

      // update num[w] and usp[w]:
      if dist[w] > dist[v] + w(v,w)
        dist[w] = dist[v] + w(v,w)
        num[w] = num[v]
        usp[w] = usp[v]
      else if dist[w] = dist[v] + w(v,w)
        num[w] = num[w] + num[v]
        usp[w] = false
    \end{verbatim}
  \end{block}
\end{frame}
%%%%%%%%%%
\begin{frame}[fragile]{Dijkstra's algorithm as skeleton}
  \begin{exampleblock}{Shortest path with fewest edges \pno{3.7.19}}
    \begin{itemize}
      \item $G = (V, E, w), w(e) > 0, s \in V$
      \item $\text{best}[u]$: minimum number of edges in a shortest path from $s$ to $u$
      \item (an example here)
    \end{itemize}
  \end{exampleblock}

  \begin{block}{Solution.}
    \begin{verbatim}
      Initialization:
        best[s] = 0; best[v] = infty for others
      // update best[w]
      if dist[w] > dist[v] + w(v,w)
        dist[w] = dist[v] + w(v,w)
        best[w] = best[v] + 1
      else if dist[w] = dist[v] + w(v,w)
        if best[w] > best[v] + 1
          best[w] = best[v] + 1
    \end{verbatim}
    an example for ``amending''
  \end{block}
\end{frame}
%%%%%%%%%%
\begin{frame}[fragile]{Dijkstra's algorithm as skeleton}
  \begin{exampleblock}{Bottleneck shortest path \pno{3.7.20}}
    \begin{itemize}
      \item min-max path: bottleneck length and bottleneck distance
      \item single source, all-pairs
    \end{itemize}
  \end{exampleblock}

  \begin{block}{Solution.}
    \begin{verbatim}
    Q = MakePQ(V) with b-dist[v] as keys (using min-heap)
    v = deleteMin(Q)

    if b-dist[w] > max(b-dist[v], w(v,w))
       b-dist[w] = max(b-dist[v], w(v,w))
    \end{verbatim}
  \end{block}

  \begin{alertblock}{For max-min path \pno{3.7.21 (3.7.23, 3.7.24)}}

  \end{alertblock}
\end{frame}
%%%%%%%%%%


\documentclass{article}  %Need this.

\usepackage{amsmath,amsthm,amssymb}

\newtheorem*{thm}{Theorem}
\newtheorem*{cnj}{Conjecture}
\newtheorem*{lem}{Lemma}
\newtheorem*{cor}{Corollary}
\newtheorem*{prop}{Proposition}

\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\R}{\mathbb{R}}

\title{Practice}
\author{--Name --}
\date{}

\begin{document}
\maketitle  %This will add the title, author, and date located above

\begin{thm}
If $x$ and $y$ are odd integers then $x+y$ is an even integer.
\end{thm}

\begin{proof}
We assume that $x$ and $y$ are odd integers and will prove that $x+y$ is an even integer. Since $x$ and $y$ are odd, there exist integers $m$ and $n$ such that $x=2m+1$ and $y=2n+1$.  By substitution and algebra we obtain
\begin{align*}
x+y &= 2m+1 + 2n + 1\\
&= 2m+2n+2\\
&=2(m+n+1).
\end{align*}
Define $q=m+n+1$.  Since $m$ and $n$ are integers and the integers are closed under addition, we conclude that $q$ is an integer. Since $x+y=2q$ for the integer $q$ we conclude that $x+y$ is an even integer.
\end{proof}

\vspace{.15in}

\section*{Challenge Typing}

Suppose that $f:(-1,1)\to \R$ and $f$ is differentiable at $0$. Let sequences $(\alpha_n)_{n\geq1}$ and $(\beta_n)_{n\geq1}$ satisify $-1<\alpha_n<\beta_n<1$ for all $n\geq 1$ and $\displaystyle{\lim_{n\to\infty}} \alpha_n = \displaystyle{\lim_{n\to\infty}} \beta_n = 0$. Set

		$$\lambda_n = \frac{f(\beta_n) -f(\alpha_n)}{\beta_n-\alpha_n} . $$

\newpage

\begin{thm}
The set $\{ x\in\Z : |x-2.5| = 2 \}$ is the empty set.
\end{thm}

\begin{proof}
Let $y$ be an integer such that $y \in \{ x\in\Z : |x-2.5| = 2 \}$  Then $y\in \mathbb{Z}$ and $|y-2.5| = 2$.  Since $|y-2.5| = 2$ then $y = 4.5$ or $y=-.5$.  But then $y$ is not an integer.  Therefore the set $\{ x\in\Z : |x-2.5| = 2 \}$ has no elements and
\[\{ x\in\Z : |x-2.5| = 2 \} = \emptyset.\]
\end{proof}

\begin{thm}
There exist two positive irrational numbers $s$ and $t$ such that $s^t$ is rational.
\end{thm}
\begin{proof}
We will consider two cases.  For the first case, suppose that $\sqrt{2}^{\sqrt{2}}$ is rational.  Then we may take $s= t = \sqrt{2}$.  For the second case, suppose that $\sqrt{2}^{\sqrt{2}}$ is irrational.  Let $s= \sqrt{2}^{\sqrt{2}}$ and $t=\sqrt{2}$.  Then
$$\left(\sqrt{2}^{\sqrt{2}}\right)^{\sqrt{2}} = \left(\sqrt{2}\right)^2 = 2.$$
Since $2$ is rational, $s^t$ is rational.  Therefore, there exists irrational numbers $s$ and $t$ such that $s^t$ is rational.
\end{proof}

\begin{thm}
Let $n$ be a natural number.  Then $$\sum_{k=1}^n k = \frac{n(n+1)}{2}.$$
\end{thm}

Consider the following matrix,
$$\left( \begin{array}{ccc}
1 &2 &3\\
4 &5 &6\\
7 &8 &9
\end{array}\right)$$

\end{document}


\subsection*{List Support}

The following list processing functions are supported:

\begin{itemize}
\item \lstinline{pair(x, y)}: \textit{primitive}, makes a pair from \lstinline{x} and \lstinline{y}.
\item \lstinline{is_pair(x)}: \textit{primitive}, returns \lstinline{true} if \lstinline{x} is a
  pair and \lstinline{false} otherwise.
\item \lstinline{head(x)}: \textit{primitive}, returns the head (first component) of the pair \lstinline{x}.
\item \lstinline{tail(x)}: \textit{primitive}, returns the tail (second component) of the
  pair \lstinline{x}.
\item \lstinline{is_null(xs)}: \textit{primitive}, returns \lstinline{true} if \lstinline{xs} is the
  empty list \lstinline{null}, and \lstinline{false} otherwise.
\item \lstinline{is_list(x)}: Returns \lstinline{true} if
  \lstinline{x} is a list as defined in the lectures, and
  \lstinline{false} otherwise. Iterative process;
time: $O(n)$, space: $O(1)$, where $n$ is the length of the
chain of \lstinline{tail} operations that can be applied to \lstinline{x}.
\item \lstinline{list(x1, x2,..., xn)}: \textit{primitive}, returns a list with $n$ elements. The
first element is \lstinline{x1}, the second \lstinline{x2}, etc. Iterative
process; time: $O(n)$, space: $O(n)$, since the constructed list data structure
consists of $n$ pairs, each of which takes up a constant amount of space.
\item \lstinline{draw_data(x)}: \textit{primitive}, visualizes \lstinline{x} in a separate drawing
  area in the Source Academy using a box-and-pointer diagram; time, space:
  $O(n)$, where $n$ is the number of data structures such as
  pairs in \lstinline{x}.
\item \lstinline{equal(x1, x2)}: Returns \lstinline{true} if both
  have the same structure with respect to \lstinline{pair},
  and the same numbers, boolean values, functions or empty list
  at corresponding leave positions (places that are not themselves pairs),
  and \lstinline{false} otherwise; time, space:
  $O(n)$, where $n$ is the number of pairs in \lstinline{x}.
\item \lstinline{length(xs)}: Returns the length of the list
  \lstinline{xs}.
Iterative process; time: $O(n)$, space: $O(1)$, where $n$ is the length of \lstinline{xs}.
\item \lstinline{map(f, xs)}: Returns a list that results from list
  \lstinline{xs} by element-wise application of \lstinline{f}.
Recursive process; time: $O(n)$, space: $O(n)$, where $n$ is the length of \lstinline{xs}.
\item \lstinline{build_list(n, f)}: Makes a list with \lstinline{n}
elements by applying the unary function \lstinline{f} to the numbers 0 to \lstinline{n - 1}.
Recursive process; time: $O(n)$, space: $O(n)$.
\item \lstinline{for_each(f, xs)}: Applies \lstinline{f} to every
  element of the list \lstinline{xs}, and then returns
  \lstinline{true}.
Iterative process; time: $O(n)$, space: $O(1)$, where $n$ is the length of \lstinline{xs}.
\item \lstinline{list_to_string(xs)}: Returns a string that represents
list \lstinline{xs} using the text-based box-and-pointer notation \lstinline{[...]}.
\item \lstinline{reverse(xs)}: Returns list \lstinline{xs} in reverse
  order. Iterative process; time: $O(n)$, space: $O(n)$, where $n$ is the length of \lstinline{xs}.
The process is iterative, but consumes space $O(n)$ because of the result list.
\item \lstinline{append(xs, ys)}: Returns a list that results from
appending the list \lstinline{ys} to the list \lstinline{xs}.
Recursive process; time: $O(n)$, space: $O(n)$, where $n$ is the length of \lstinline{xs}.
\item \lstinline{member(x, xs)}: Returns first postfix sublist
whose head is identical to
\lstinline{x} (\lstinline{===}); returns \lstinline{[]} if the
element does not occur in the list.
Iterative process; time: $O(n)$, space: $O(1)$, where $n$ is the length of \lstinline{xs}.
\item \lstinline{remove(x, xs)}: Returns a list that results from
\lstinline{xs} by removing the first item from \lstinline{xs} that
is identical (\lstinline{===}) to \lstinline{x}. Recursive process;
time: $O(n)$, space: $O(n)$, where $n$ is the length of \lstinline{xs}.
\item \lstinline{remove_all(x, xs)}: Returns a list that results from
\lstinline{xs} by removing all items from \lstinline{xs} that
are identical (\lstinline{===}) to \lstinline{x}.
Recursive process;
time: $O(n)$, space: $O(n)$, where $n$ is the length of \lstinline{xs}.
\item \lstinline{filter(pred, xs)}: Returns a list that contains
only those elements for which the one-argument function
\lstinline{pred}
returns \lstinline{true}.
Recursive process;
time: $O(n)$, space: $O(n)$, where $n$ is the length of \lstinline{xs}.
\item \lstinline{enum_list(start, end)}: Returns a list that enumerates
numbers starting from \lstinline{start} using a step size of 1, until
the number exceeds (\lstinline{>}) \lstinline{end}.
Recursive process;
time: $O(n)$, space: $O(n)$, where $n$ is the length of \lstinline{xs}.
\item \lstinline{list_ref(xs, n)}: Returns the element
of list \lstinline{xs} at position \lstinline{n},
where the first element has index 0.
Iterative process;
time: $O(n)$, space: $O(1)$, where $n$ is the length of \lstinline{xs}.
\item \lstinline{accumulate(op, initial, xs)}: Applies binary
function \lstinline{op} to the elements of \lstinline{xs} from
right-to-left order, first applying \lstinline{op} to the last element
and the value \lstinline{initial}, resulting in $r_1$, then to the
second-last element and $r_1$, resulting in $r_2$, etc, and finally
to the first element and $r_{n-1}$, where $n$ is the length of the
list. Thus, \lstinline{accumulate(op,zero,list(1,2,3))} results in
\lstinline{op(1, op(2, op(3, zero)))}.
Recursive process;
time: $O(n)$, space: $O(n)$, where $n$ is the length of \lstinline{xs},
assuming \lstinline{op} takes constant time.
\end{itemize}


\subsection{Optimizing Expected 1-call@k}
Before we present the result on optimizing expected $n$-call@$k$ for
general $n$, we start off from the simplest case, expected 1-call@$k$
(where $n=1$):
% Definition of 1-call
\begin{align}
    \ExpOneCall(S_k,\vec{q}) & = \mathbb{E}[R_k\geq 1|s_1,\dots,s_k,\vec{q}]
\end{align}
Since $R_k\geq 1$ is satisfied as long as any one of the $r_i$ is $1$,
the objective can be rewritten in terms of logical notation:
\begin{align}
\label{eq:setRelevance}
    \ExpOneCall(S_k,\vec{q}) & = \mathbb{E} \left[\left. \bigvee_{i=1}^{k}r_i=1 \right| s_{1},\dots, s_{k},\vec{q} \right]
\end{align}

Since jointly optimizing $\ExpOneCall(S_k,\vec{q})$ is NP-hard, we
take a greedy approach similar to MMR where we choose the best $s_k^*$
assuming that $S_{k-1}^*$ is given.  Then following~\cite{chen06Less},
we can greedily optimize this objective as
follows:
\begin{align}
s_k^* & = \argmax_{s_k} \; \ExpOneCall(S_{k-1}^* \cup \{ s_k \},\vec{q}) \nonumber \\
   & = \argmax_{s_k} \mathbb{E}\left[\left. \bigvee_{i=1}^{k} r_i=1 \right| S_{k-1}^*, s_{k},\vec{q}\right] \nonumber
\end{align}
Next we can apply a logical equivalence and exploit the additivity of
mutually exclusive events to split $\bigvee_{i=1}^{k} r_i=1$ into mutually exclusive
disjoint subsets:
\begin{align}
s_k^* & = \argmax_{s_k} \mathbb{E}\Bigg[  (r_1=1) \vee (r_2 =1 \wedge r_1=0) \vee (r_3 =1 \wedge r_2=0 \wedge r_1=0) \vee \cdots \vee \nonumber \\
   & \hspace{19mm} \left(r_k=1 \wedge \bigwedge_{i=1}^{k-1} r_i=0 \right) \, \Bigg| \, S_{k-1}^*,s_k,\vec{q} \Bigg] \label{eq:logicalsplit}
\end{align}
Since these events are binary and disjoint, we can rewrite the expectation as probability.
This gives us the sum of the probabilities of each individual event.
We further simplify it by grouping all $r_j = 0$.\footnote{The notation
$\{ \cdot \}_C$ refers to a (possibly empty) set of
variables (or variable assignments) $\cdot$ that meet constraints $C$.}
We factorize each joint probability into a conditional and prior, and remove terms and factors that do not contain $s_k$, noting that these terms only act as constants when optimizing for $s_k$:
\begin{align}
s_k^* & = \argmax_{s_k} \sum_{i=1}^{k} P(  r_i=1, \{ r_{j}=0 \}_{j<i} \, | \, \{ s^*_j \}_{j\leq i,j<k},\{ s_{k} \}_{k=i},\vec{q}) \nonumber \\
   &= \argmax_{s_k} \sum_{i=1}^{k} P(  r_i=1 \, | \, \{ r_{j}=0 \}_{j<i} , \{ s^*_j \}_{j\leq i,j<k},\{ s_k \}_{k=i},\vec{q}) \nonumber\\
   & \hspace{21.5mm} P(\{ r_{j}=0 \}_{j<i} \, | \, \{ s^*_j \}_{j\leq i,j<k}, \vec{q}) \nonumber \\
   &= \argmax_{s_k} P( r_k=1 \, | \, \{ r_{j}=0 \}_{j<k}, S_{k-1}^*,s_k,\vec{q}) \label{eq:set-objective}
\end{align}
From~\eqref{eq:set-objective}, to optimize for $s_k$, we need only to maximize $s_k$'s probability of relevance conditioned on the previous selections (which are assumed irrelevant, $r_j=0$) and the query.

Next we evaluate the final query from~\eqref{eq:set-objective} w.r.t.\
our actual graphical model of subtopic relevance from Figure~\ref{fig:gm}:
\begin{align}
s_k^* & = \argmax_{s_{k}} P( r_k=1 \, | \, \{ r_{j}=0 \}_{j<k}, S_{k-1}^*,s_k,\vec{q}) \nonumber \\
& = \argmax_{s_{k}} \sum_{t, t_1, \cdots, t_{k}} P(t|\vec{q}) P(t_k|s_k) \I[t_k = t] \prod_{i=1}^{k-1} P(t_i|s_i^*) \I[t_i\neq t]\nonumber \\
& = \argmax_{s_{k}} \sum_{t} P(t|\vec{q}) \sum_{t_{k}} P(t_k|s_k) \I[t_k = t] \prod_{i=1}^{k-1} \sum_{t_{i}} P(t_i|s_i^*) \I[t_i \neq t]\nonumber \\
& = \argmax_{s_{k}} \sum_{t} P(t|\vec{q}) P(t_k=t|s_k) \fbox{$\prod\limits_{i=1}^{k-1} (1 - P(t_i=t|s_i^*))$} \label{eq:partial_simp}
\end{align}
Here we have used the following equality:
\begin{align*}
& \sum_{t_{i}} P(t_i|s_i) \I[t_i = t] =  P(t_i=t|s_i) \\
& \sum_{t_{i}} P(t_i|s_i) \I[t_i \neq t] = 1 - P(t_i=t|s_i)
\end{align*}

Defining $\tilde{P}(t | S_{k-1}^*) = 1 - \Box = 1 - \prod_{i=1}^{k-1} (1 -
  P(t_i=t|s_i^*))$, this is the probability that
set $S_{k-1}^*$ already \emph{covers} topic $t$
w.r.t.\ a \emph{noisy-or} interpretation.  Substituting
$(1 - \tilde{P}(t | S_{k-1}^*))$ for $\Box$ since
$(1 - \tilde{P}(t | S_{k-1}^*)) = 1 - (1 - \Box) = \Box$, we obtain
\begin{align}
s_k^* & = \argmax_{s_{k}} \sum_{t} P(t|\vec{q}) P(t_k=t|s_k) \left( 1 - \tilde{P}(t | S_{k-1}^*) \right) \nonumber \\
      & = \argmax_{s_{k}} \sum_{t} \underbrace{P(t|\vec{q}) P(t_k=t|s_k)}_{\mbox{\footnotesize query similarity}} - \sum_{t} \underbrace{P(t|\vec{q}) P(t_k=t|s_k) \tilde{P}(t | S_{k-1}^*)}_{\mbox{\footnotesize query-reweighted diversity}}. \label{eq:1call}
\end{align}

From~\eqref{eq:1call}, we can see that optimizing expected 1-call@$k$
give us a greedy algorithm that somewhat resembles MMR
in~\eqref{eq:MMR} with an implicit equal balancing of the similarity
and diversity objectives (i.e., $\lambda = \frac{1}{2}$).  After presenting our
general derivation for expected $n$-call@$k$ in the next subsection
for $n \geq 1$, we return to a general comparison between expected
$n$-call@$k$ and MMR.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Optimizing Expected n-call@k}

\label{subsec:ncall}

Having analyzed the special case of $n$-call@$k$ for $n=1$, we now
attempt to generalize our approach to general $n \geq 1$.  In contrast
to $n=1$, however, we note that it is not straightforward to use the
logical notation of the expected $n$-call@$k$ objective for $n=1$
given in~\eqref{eq:setRelevance} when $n > 1$: even for $n=2$, the
number of disjoint events that satisfy $R_k \geq 2$ grows by a factor
of $(k-1)/2$.\footnote{$\binom{k}{2} / \binom{k}{1} = (k-1)/2$}

Next we provide our main result on optimizing expected $n$-call@$k$
for general $n \geq 1$.  For the interested reader, we note that a
simpler (logic-based) derivation for the special case of $n=2$ is
provided in Appendix~\ref{appendix.2call}, which provides some guiding
intuitions for the general derivation deveoped in this section.

In the following derivation, we adopt a more abstract approach
(working with $R_k$ directly) while utilizing the same principle as in
expected 1-call@$k$.
As done previously, we take a greedy approach that
selects $s_k$ assuming that $S_{k-1}^*$ is already chosen:
\begin{align}
  s_k^* & = \argmax_{s_k} \ExpNCall{n}(S_{k-1}^* \cup \{ s_k \},\vec{q}) \nonumber\\
  s_k^* & = \argmax_{s_k} \mathbb{E}[R_k\geq n|S_{k-1}^*,s_k,\vec{q}] \nonumber\\
  & = \argmax_{s_k} P(R_k\geq n|S_{k-1}^*,s_k,\vec{q}) \nonumber
\end{align}

Here, we have exploited the binary ($0,1$) nature of $R_k \geq n$ to rewrite the objective
directly as a probabilistic query.
This query can be evaluated w.r.t.\ our latent subtopic binary relevance
model in Figure~\ref{fig:gm} as follows, where we marginalize out
all non-query, non-evidence variables $T_k$ $\big(\mbox{define } T_k\!=\!\{t,t_1,\dots,t_k\}$ and
$\sum_{T_k} \circ = \sum_t \sum_{t_1} \cdots \sum_{t_k} \circ \big)$:
\begin{align}
  s_k^* = & \argmax_{s_k} \!\sum_{T_k} \Bigl( P(t|\vec{q}) \,P(t_k|s_k) \prod_{i=1}^{k-1} P(t_i|s_i^*) \cdot P(R_k\geq n|T_k,S_{k-1}^*,s_k,\vec{q}) \Bigr) \nonumber
\end{align}

We split $R_k \geq n$ into two disjoint (additive) events
$(r_k \! \geq \! 0, R_{k\!-\!1}\!\geq \!n)$, $(r_k\!\!=\!\!1, R_{k\!-\!1}\!\!=\!\!n\!-\!1)$ based on $R_{k-1}$. (If $R_{k-1}$ is equal to $n\!-\!1$, $r_k$ must be $1$; if $R_{k-1}$ is greater or equal to $n$, then $r_k$ can be either $0$ or $1$).
\begin{align}
 s_k^* = & \argmax_{s_k} \!\sum_{T_k} P(t|\vec{q}) \,P(t_k|s_k) \prod_{i=1}^{k-1} P(t_i|s_i^*) \cdot \Bigl( P(r_k\geq 0, R_{k-1}\geq n|T_k,S_{k-1}^*,s_k,\vec{q}) \nonumber \\
  & \hspace{56mm} + P(r_k = 1, R_{k-1} = n-1|T_k,S_{k-1}^*,s_k,\vec{q}) \Big) \nonumber
\end{align}
We then write the joint probability into a conditioned and prior, conditioned on $R_k$:
% Also note that all $r_i$ are D-separated:
\begin{align}
 s_k^* = & \argmax_{s_k} \!\sum_{T_k} P(t|\vec{q}) \,P(t_k|s_k) \prod_{i=1}^{k-1} P(t_i|s_i^*) \cdot \Bigl( \mbox{$\underbrace{P(r_k\!\geq\!0|R_{k-\!1}\!\geq\!n,t_k,t)}_{1}$} P(\!R_{k-\!1}\!\geq\!n|\TlessK) \nonumber \\
  & \hspace{50mm} + P(r_k=1|R_{k-1}\!=\!n\!-\!1,t_k,t) P(\!R_{k-\!1}\!=\!n\!-\!1|\TlessK) \Big) \nonumber
\end{align}
We distribute initial terms over the summands noting that
$\sum_{t_k} \!\! P(t_k|s_k) P(r_k\!\!=\!\!1|t_k,t) \! = \!\! \sum_{t_k} \!\! P(t_k|s_k) \I[t_k\!\!=\!\!t] \! = \!\! P(t_k\!\!=\!\!t|s_k)$,
and $r_k$ is independent to $R_k$ given $T_k$:
\begin{align}
 s_k^* = & \argmax_{s_k} \Bigg( \sum_{\TlessK} \bigg[ \mbox{$\underbrace{ \sum_{t_k} \!P(t_k|s_k) }_{1}$} \bigg] P(\!R_{\!k-\!1}\!\geq\!n|\TlessK) P(t|\vec{q}) \prod_{i=1}^{k-1} P(t_i|s_i^*) \nonumber \\
  & \hspace{13mm} + \sum_{t} \!P(t|\vec{q}) P(t_k\!=\!t|s_k) \hspace{-4mm} \sum_{t_1, \dots, t_{k-1}} \hspace{-3mm} P(R_{k-\!1}\!=\!n\!-\!1|\TlessK) \prod_{i=1}^{k-1} \!P(t_i|s_i^*) \Bigg) \nonumber
\end{align}
Next we proceed to drop the first summand since it is not a function of $s_k$ (\emph{i.e.},
it has no influence in determining $s_k^*$):
% This give us the
%simplified optimization objective:
\begin{align}
s_k^* = & \argmax_{s_k} \!\sum_{t} \!P(t|\vec{q}) P(t_k\!=\!t|s_k) P(\!R_{k-\!1}\!\!=\!n\!-\!1|S_{k-1}^*, t) \label{eq.ncall}
\end{align}

By similar reasoning, we can derive that the last probability
needed in~\eqref{eq.ncall} is recursively defined as
\begin{align*}
P(R_k=n|S_k,t)=
\begin{cases}
n \geq 1, k > 1:  &  \bigl( 1\!-\!P(t_k\!=\!t|s_k) \bigr) P(R_{k-1}\!=\!n|S_{k-1},t) \nonumber \\
  & \hspace{5mm} + P(t_k\!=\!t|s_k) P(R_{k-\!1}\!=\!n\!-\!1|S_{k-\!1},t) \\
%%
n = 0, k > 1:   & \bigl( 1\!-\!P(t_k\!=\!t|s_k) \bigr) P(R_{k-\!1}\!=\!0|S_{k-\!1},t) \\
%%
n = 1, k = 1:   & P(t_1\!=\!t|s_1) \\
n = 0, k = 1:   & 1 - P(t_1\!=\!t|s_1) \\
n > k:			& 0
\end{cases}
%       \bigl 1-P(t_1\!=\!t|s_1) \bigr) = \bigl 1-P(t_1\!=\!t|s_1) \bigr) \\
%  & \hspace{2mm} P(R_1\!=\!1|S_1,t) = P(t_1\!=\!t|s_1)
\end{align*}

We can now rewrite~\eqref{eq.ncall} by unrolling its recursive definition.
For expected $n$-call@$k$ where $n \leq k/2$ %, n \! \neq \! 1$
(a symmetrical result holds for $k/2 < n \leq k$),
the explicit unrolled objective is
\begin{align}
  & s_k^* = \argmax_{s_k} \sum_t \Biggl( P(t|\vec{q}) \, P(t_k=t|s_k) \cdot \hspace{-2mm} \sum_{j_1, \dots, j_{n-\!1}} \hspace{-14mm} \prod_{\hspace{14.5mm} l \in \{j_1, \dots, j_{n-\!1}\}} \hspace{-14mm} P(t_l\!=\!t|s_l^*) \hspace{-13mm} \prod_{\substack{i=1 \\ \hspace{14mm} i \notin \{j_1, \dots, j_{n-\!1}\}}}^{k-1} \hspace{-13mm} \!\bigl( 1 - P(t_i\!=\!t|s_i^*) \bigr) \!\Biggr) \label{eq.ncall.alt}
\end{align}
where $j_1, \dots, j_{n-1} \in \{1,\ldots,k-1\}$ satisfy
that $j_i < j_{i+1}$ (\emph{i.e.},
an ordered permutation of $n-1$ result set indices).
Note that~\eqref{eq.ncall.alt} reduces to~\eqref{eq:partial_simp} when $n=1$.

% ===============================================================================

%From here we focus on the last product in~\eqref{eq.ncall.alt}.  We note that
If we assume each document covers a single subtopic of the query (\emph{e.g.},
a subtopic represents an intent of an ambiguous query) then we can assume that
$\forall i \; P(t_i|s_i) \in \{0,1\}$ and $P(t|\vec{q}) \in \{0,1\}$.  This
allows us to convert a $\prod$ to a $\max$
\begin{align*}
  \hspace{-13mm} \prod_{\substack{i=1 \\ \hspace{14mm} i \notin \{j_1, \dots, j_{n-\!1}\}}}^{k-1} \hspace{-14mm} \bigl( 1 - \!P(t_i\!=\!t|s_i^*) \bigr) & =
1 - \Biggl( 1 - \hspace{-14mm} \prod_{\substack{i=1 \\ \hspace{14mm} i \notin \{j_1, \dots, j_{n-\!1}\}}}^{k-1} \hspace{-13mm} \bigl( 1 - P(t_i\!=\!t|s_i^*) \bigr) \Biggr)
   = 1 - \Bigl( \hspace{-6mm} \max_{\substack{i \in [1,k-1] \\ \hspace{7mm} i \notin \{j_1, \dots, j_{n-1}\}}} \hspace{-6mm} P(t_i\!=\!t|s_i^*) \Bigr)
\end{align*}
and by substituting this into~\eqref{eq.ncall.alt} and distributing, we get
\begin{align}
s_k^* =  & \, \argmax_{s_k} \sum_t \Biggl( P(t|\vec{q}) P(t_k\!=\!t|s_k)  \sum_{\hspace{-1mm} j_1, \dots, j_{n-\!1}} \hspace{-13.5mm} \prod_{\hspace{14.5mm} l \in \{j_1, \dots, j_{n-1}\}} \hspace{-14mm} P(t_l\!=\!t|s_l^*) \nonumber \\
  & \hspace{20mm} - \!P(t|\vec{q}) P(t_k\!=\!t|s_k) \sum_{\hspace{-1mm} j_1, \dots, j_{n-\!1}} \hspace{-14mm} \prod_{\hspace{14.5mm} l \in \{j_1, \dots, j_{n-\!1}\}} \hspace{-13mm} P(t_l\!=\!t|s_l^*) \hspace{-11.5mm} \max_{\substack{\hspace{5mm} i \in [1,k-1] \\ \hspace{11.5mm} i \notin \{j_1, \dots, j_{n-\!1}\}}} \hspace{-8.5mm} P(t_i\!=\!t|s_i^*) \!\!\Biggr) . \label{eq.ncall.alt2}
\end{align}

Assuming $m$ of the selected documents $(S_{k-1}^*)$ are relevant
then the top term
(specifically $\prod_l$) is non-zero $\binom{m}{n-1}$ times.  For the
bottom term, it takes $n-1$ relevant $S_{k-1}^*$ to satisfy its
$\prod_l$, and one additional relevant document to satisfy the
$\max_i$ making it non-zero $\binom{m}{n}$ times.  Factoring out the
$\max$ element from the bottom and pushing the $\sum_t$ inwards (all legal
due to the $\{0,1\}$ subtopic probability assumption) we get
\begin{align}
s_k^* = & \argmax_{s_k} \binom{m}{n\!-\!1} \underbrace{\sum_t P(t|\vec{q}) P(t_k\!=\!t|s_k)}_{\textrm{relevance}: \; \Sim_1(s_k,\vec{q})} - \binom{m}{n} \max_{s_i \in S_{k-1}^*} \underbrace{\sum_t \!P(t|\vec{q}) P(t_k\!=\!t|s_k) P(t_i\!=\!t|s_i)}_{\textrm{diversity}: \; \Sim_2(s_k,s_i,\vec{q})} \nonumber .
\end{align}

From here we can
normalize by $\binom{m}{n-1} + \binom{m}{n} = \binom{m+1}{n}$
(Pascal's rule), leading to the simplified result:
\begin{align}
s_k^* =  & \argmax_{s_k} \!\! \frac{n}{m\!+\!1} \Sim_1(s_k,\vec{q}) - \frac{m\!-\!n\!+\!1}{m+1} \max_{s_i \in S_{k-1}^*} \! \Sim_2(s_k,s_i,\vec{q}) \label{eq:ncall-result-almost}
\end{align}
Fortuitously, we note that the $\binom{m+1}{n}$ divisor cancelled with
the numerators, yielding this elegant and interpretable result.
In practice, we can make one additional
simplifying assumption: under the greedy approach of
selecting $s_k$ given $S_{k-1}^*$ when $k \geq n + 1$, we expect that there are already $n$
relevant documents chosen in the set $S_{k-1}^* = \{s_1^*, \dots,
s_{k-1}^*\}$, and hence in expectation $m=n$.  Assuming $m = n$ we obtain
\begin{align}
s_k^* =  & \argmax_{s_k} \!\! \frac{n}{n\!+\!1} \Sim_1(s_k,\vec{q}) - \frac{1}{n+1} \max_{s_i \in S_{k-1}^*} \! \Sim_2(s_k,s_i,\vec{q}) \label{eq:ncall-result}
\end{align}
leading to $\lambda = \frac{n}{n\!+\!1}$ when comparing to MMR
in~\eqref{eq:MMR}, which achieves our goal of formally expressing the
relevance vs. diversity tradeoff as a function of $n$.

As a reality check, we see that the result $\lambda = \frac{n}{n+1}$
coincides with the result of $\lambda=\frac{1}{2}$ for $n=1$ as
derived in the previous section for expected $1$-call@$k$.  Hence, we
have achieved one of our key theoretical results and have shown that
when optimizing expected $n$-call@$k$ w.r.t.\ a latent subtopic binary
relevance model diversification decreases as $n \to 1$, thus
confirming the empirical observations regarding $n$-call@$k$ in
\cite{wang09PortfolioTheory} (Figure 2c).

%$n=1$, $m=1$.  Overall we have achieved our goal and have shown that
%indeed, diversificiation in expected $n$-call@$k$ decreases linearly
%as $n \to 1$.

% ===============================================================================
% This subsection is copied from Section 4.1
\subsection{Connections to Maximal Marginal Relevance}

\label{subsec:mmr_comp}

The result in~\eqref{eq:ncall-result} is strikingly similar to maximal marginal
relevance (MMR)~\cite{carbonell98MMR} --- it
contains two terms, one for query similarity and the other for result
set diversification, where each term represents a similarity kernel
--- more specifically a \emph{probability product kernel}
(PPK)~\cite{prodprobkernel} that is an inner product of probability
vectors (or more generally, functions).  More formally, let
$\vec{T}'$, $\vec{T}_k$, and $\vec{T}_{S_{k-1}^*}$ be respective topic
probability vectors $P(t'=t|\vec{q})$, $P(t_k=t|s_k)$ and
%$\tilde{P}(t_k=t | S_{k-1}^*)$
$P(t_i\!=\!t|s_i)$
with vector indices for each topic $t
\in T$.  Then the similarity and diversity terms from~\eqref{eq:ncall-result}
can be respectively written as
\begin{align}
\Sim_1(\vec{q},s_k) = &
\sum_{t \in T} P(t'=t|\vec{q}) P(t_k=t|s_{k}) \; = \; \langle \vec{T}',\vec{T}_k \rangle \label{eq:sim_term} \; \mbox{and}\\
\Sim_2(s_k,S_{k-1}) = &
\sum_{t \in T} P(t|\vec{q}) P(t_k=t|s_k) \tilde{P}(t | S_{k-1}^*) \; = \; \langle \vec{T}_k, \vec{T}_{S_{k-1}^*} \rangle_{\vec{T}'}. \label{eq:div_term}
\end{align}
Here, we let $\langle \cdot,\cdot \rangle$ denote an inner product of
two vectors and $\langle \cdot,\cdot \rangle_\vec{v}$ a
\emph{$\vec{v}$-reweighted} inner product, defined as
in~\eqref{eq:div_term}.

While having similarity and diversity terms similar to MMR,
Exp-$n$-call@$k$ in~\eqref{eq:ncall-result} differs from MMR
in two notable ways:
\begin{enumerate}
\item While MMR's definition allows for any similarity function, not
  just PPKs, we note that \emph{equating words to subtopics}, popular
  kernels like TF and TFIDF~\cite{salton83Introduction} can be viewed
  directly as PPKs if the TF and TFIDF vectors are $L_1$ normalized to
  represent probability vectors.  Hence we can avoid directly answering the question of how
  subtopics and subtopic probabilities are specified by using the
  ``kernel trick'' and kernels with a PPK interpretation like TF or
  TFIDF to \emph{directly} compute the required optimization quantities
  in~\eqref{eq:sim_term} and~\eqref{eq:div_term}.
%% FOLLOWING NO LONGER RELEVANT for n-call@k (not 1-call@k)
%%
%\item MMR uses a maximization term for
%diversity, whereas optimization of Exp-$n$-call@$k$ instead calls for
%a product (noisy-or) diversity term. %$\tilde{P}(t | S_{k-1}^*)$.
%We note that a noisy-or reduced to a max when the subtopic
%probabilities are deterministic (0 or 1).
%\item While MMR proposes a $\lambda$ term to explicitly
%trade off the similarity and diversity terms, the greedy optimization
%of Exp-$n$-call@$k$ in~\eqref{eq:ncall-result} yields such trade-off term
%as a function of $m, n$ and $k$.
\item Optimizing Exp-$n$-call@$k$ introduces query-specific relevance
  into the diversification term as shown by the query topic
  ($\vec{T}'$) reweighted diversity function in~\eqref{eq:div_term}.
  Indeed this modification to MMR is quite intuitive and may prove
  advantageous since result set diversification need only be
  considered over the subtopics that are actually relevant to the
  query.
\end{enumerate}

Despite these minor differences, however, we note the strong similarity
between the optimization of expected $n$-call@$k$ and MMR where we formally
derived that $\lambda = \frac{n}{n+1}$.


%        File: doc.tex
%     Created: Lun mar 29 09:00  2021 C
% Last Change: Lun mar 29 09:00  2021 C
%
\documentclass[a4paper]{article}

\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage[]{amsmath}

\newcommand{\rom}[1]{\mathrm{#1}}
\newcommand{\bold}[1]{\mathbf{#1}}
\newcommand{\dd}{\mathrm{d}}

\begin{document}

\section{Slater-type orbitals}

A real Slater-type orbital (STO) is defined as follows:

\begin{equation}
  \chi_{nlm}(\zeta, \bold{r}) = N_n(\zeta) R_n(r, \zeta) S_{lm}(\theta, \phi)
  \label{eq:real_sto}
\end{equation}

where

\begin{equation}
  N_n(\zeta) = \frac{(2\zeta)^{n+\frac{1}{2}}}{\sqrt{(2n)!}}
  \label{eq:norm_sto}
\end{equation}

is the normalization factor,

\begin{equation}
  R_n(r,\zeta) = r^{n-1} e^{-\zeta r}
  \label{eq:sto_radial_part}
\end{equation}

is the radial part and $S_{lm}(\theta, \phi)$ is a real spherial harmonic.

\section{Prolate spheroidal coordinates}

For two-center integrals, it is more convenient to align the coordinate
system of the two centers along the $z$-axis and use prolate spheroidal
coordinates. In order to switch from spherical coordinates to the new
coordinate system, we use the following identities:

\begin{equation}
  \begin{aligned}
    r_a = \frac{R(\mu+\nu)}{2}; \quad
    \cos\theta_a = \frac{1+\mu\nu}{\mu+\nu}; \quad &
    \sin\theta_a = \frac{\sqrt{(\mu^2-1)(1-\nu^2)}}{\mu+\nu}; \\
    r_b = \frac{R(\mu-\nu)}{2}; \quad
    \cos\theta_b = \frac{1-\mu\nu}{\mu-\nu}; \quad &
    \sin\theta_b = \frac{\sqrt{(\mu^2-1)(1-\nu^2)}}{\mu-\nu}
  \end{aligned}
  \label{eq:prolate_spheroidal_coordinates}
\end{equation}

Inside the integrals this change gives

\begin{equation}
  \dd V = \frac{R^3}{8} (\mu+\nu) (\mu-\nu) \dd\mu \dd\nu \dd\phi
\label{eq:prolate_spheroidal_coordinates_integration}
\end{equation}

\section{Overlap integrals}

The one-center overlap integrals

\begin{equation}
  S_{nlm,n'l'm'}(\zeta, \zeta') =
  \int \chi_{nlm}^{\star}(\zeta, \bold{r})
  \chi_{n'm'l'}(\zeta', \bold{r}) \dd V
  \label{eq:1c_overlap_integral}
\end{equation}

is the most straightforward molecular integrals and is readily evaluated in
spherical coordinates. The integration of the radial part yields

\begin{equation}
  \int_0^{\infty} R_n(\zeta,r) R_{n'}(\zeta',r) r^2 \dd r =
  \frac{(n+n')!}{(\zeta+\zeta')^{n+n'+1}}
  \label{eq:1c_overlap_integral_radial_part_eval}
\end{equation}

and using the orthonormality condition of the spherical harmonics, the analytical
form of one-center overlap is

\begin{equation}
  S_{nlm,n'l'm'}(\zeta, \zeta') =
  N_n(\zeta) N_{n'}(\zeta')
  \frac{(n+n')!}{(\zeta+\zeta')^{n+n'+1}}
  \delta_{ll'}\delta_{mm'}
\label{eq:1c_overlap_integral_result}
\end{equation}

\end{document}