%
% 337
%
\chapter{The Confluent Hypergeometric Function}

\Section{16}{1}{The confluence of two singularities of Riemanns equation}

We have seen \hardsectionref{10}{8}) that the linear differential equation with two
regular singularities only can be integrated in terms of elementary
functions; while the solution of the linear differential equation
with three regular singularities is substantially the topic of Chapter
xiv. As the next type in order of complexity, we shall consider a
modified form of the differential equation which is obtained from
Riemann's equation by the confluence of two of the singularities. This
confluence gives an equation with an irregular singularity
(corresponding to the confluent singularities of Riemann's equation)
and a regular singularity corresponding to the third singularity of
Riemann's equation.

The confluent equation is obtained by making c -* oo in the equation
defined by the scheme

X c

-.--m -c c - k

P

1;

2

.7 - ni k

The equation in question is readily found to be

We modify this equation by writing TODO and obtain as the equation*
for Wkmi)

The reader will verify that the singularities of this equation are at
6 and X, the former being regular and the latter irregular; and when
2m is not an integer, two integrals of equation (B) which are regular
near and valid for all finite values of z are given by the series

* This equation was given by Whittaker, Bulletin American Math. Soc.
x. (1904), pp. 125-134. W. M. A. 22

%
% 338
%

These series obviously form a fundamental system of solutions.

[Note. Series of the type in \{ \} have been considered by Kummer* and
more recently by Jticobsthalt and Barnes J; the special series in
which k = had been investigated by Lagrange in 1762-1765 (Oeuvres, I.
p. 480). In the notation of Kummer, modified by Barnes, they would be
written iF-i \{h±ni - k; ± 2m + 1; 2\}; the reason for discussing

solutions of equation (B) rather than those of the equation 2

which iFi (a; p; z) is a solution, is the greater appearance of
symmetry in the formulae, together with a simplicity in the equations
giving various functions of Applied Mathe- matics (see \hardsectionref{16}{2}) in
terms "of solutions of equation (B).]

\Subsection{16}{1}{1}{Kummer s formulae.}

(I) We shall now shew that, if '2'm is not a negative integer, then

---il/,,() = (-)-i\~''W,,,(-), that is to say,

., (I + m - k) (f + m - k)

 "" -" 2 ! (2w + 1) (271 + 2)

h

+ m -

k

1!

\{2m +

1)

i

+ m +

k

  . ( + w + k) (f + m + k) .,

l!(2m + l) 2 ! (2m + 1) (2m +2) ' '■**

For, replacing TODO by its expansion in powers of z, the coefficient
of TODO in the product of absolutely convergent series on the left is

by \hardsubsectionref{14}{1}{1}, and this is the coefficient of " on the right§; we have
thus obtained the required result.

This will be called Kummej-'s first formula.

(II) The equation

valid when 2m is not a negative integer, will be called Kummer s
second formula.

To prove it we observe that the coefficient of £"+"'+? in the product
TODO

* Journal fur Math. xv. (1836), p. 139. t Math. Ann. lvi. (1903), pp.
129-154. X Trans. Camb. Phil. Soc. xx. (1908), pp. 253-279.

§ The result is still true when ?h +  + k is a negative integer, by
a slight modification of the analysis of \hardsubsectionref{14}{1}{1}.

%
% 339
%

of which the second and third factors possess absolutely convergent
expansions, is \hardsubsectionref{3}{7}{3})

nl\{2m+l)\{2m + 2) ...\{27n+7i); - -

by Kummer's relation*

TODO

valid when a' i; and so the coefficient of 2"+'""'" (by
\hardsubsectionref{14}{1}{1}) is

(| + w)(§ + m) ... \{n-m-- ) T \{-n + h-m)r \{ ) n (2i+l)(2m +
2)... (2m + 7i) T \{-m-hi)T\{-n)

n (2m + 1 ) \{2m -- 2) . . . \{2m + n) r ( - w - hi) r ( - 1?0 '
and when n is odd this vanishes; for even values of n\{ = 2p) it is

r(i-/)(-)(-|)...(i-p)

l.S...\{2p-l) 1

2pl 23p(m + l)(m + 2)...(m+p) 2*P.p \{m + 1) \{m + 2) ... \{m+p)'

\Subsection{16}{1}{2}{Definition f of the function Wic,m (2) }

The solutions -il/t.imC') of equation (B) of \hardsectionref{16}{1} are not,
however, the most convenient to take as the standard solutions, on
account of the disappearance of one of them when 2m is an integer.

The integral obtained by confluence from that of \hardsectionref{14}{6}, when
multiplied by a constant multiple of TODO is TODO:

It is supposed that arg has its principal value and that the contour
is so chosen that the point t = - z is outside it. The integrand is
rendered one- valued by taking; arg (- ) |  tt and taking that
value of arg (1 + t/z) which tends to zero as i -*- by a path lying
inside the contour.

Under these circumstances it follows from \hardsubsectionref{5}{3}{2} that the integral is
an analytic function of z. To shew that it satisfies equation (B),
write

* See Chapter xiv, examples 12 and 13, p. 298.

+ The function TODO was defined by means of an integral in this manner
by Whittaker, ioc. cit. p. 125.

* A suitable contour has been chosen and the variable t of \hardsectionref{14}{6}
replaced by - 1.

22-2

%
% 340
%

and we have without difficulty*

TODO

since the expression in \{ \} tends to zero as i -* + oo; and this is
the condition that TODO should satisfy (B).

Accordingly the function Wk, m \{z) defined by the integral

is a solution of the differential equation (B).

The formula for TfA;,m () becomes nugatory when ' - 2 - w is a
negative integer. To overcome this difficulty, we observe that
whenever

and k - - m is not an integer, we may transform the contour integral
into an infinite integral, after the manner of \hardsubsectionref{12}{2}{2}; and so, when

Rik -i-:- 7n 0,

This formula suffices to define Wimiz) in the critical cases when m
+ 2->t is a positive integer, and so Wk,m\{z) is defined for all
values of k and m and all values of z except negative real values f.
Example. Solve the equation

du ( b c

in terms of functions of the type Tft, (2), where a, 6, c are any
constants.

\Section{16}{2}{Expression of various functions by functions of the
  type Wk,n(z)}

It has been shewn:!: that various functions employed in Applied Mathe-
matics are expressible by means of the function Wk,m (2); the
following are a few examples :

* The differentiations under the sign of integration are legitimate by \hardsubsectionref{4}{4}{4} corollary.

t When z is real and negative, TODO may be defined to be either TODO
or TODO whichever is more convenient.

+ Whittaker, Bulletin American Math. Soc. x.; this paper contains a
more complete account than is given here.

%
% 341
%
(I) The Error function* which occurs in connexion with the theories of
Probability, Errors of Observation, Refraction and Conduction of Heat
is defined by the equation

Erfc()=[ e-''dt, where a; is real.

Writing t = x\{w - l) and then iu = s/x in the integral for TT.j.
.(a), we get

Ji = TODO

J X

and so the error function is given by the formula

Erfc (a;) = TODO

Other integrals which occur in connexion with the theory of Conduction
rb of Heat, e.g. TODO, can be expressed in terms of error functions,
and

J a

SO in terms of TODO functions.

Exaviple. Shew that the formula for the eri'or function is true for
complex values of x.

(II) The Incomplete Gamma function, studied by Legendre and othersf,
is defined by the equation

y\{n, x)= j f'-'e-dt. Jo

By writing t = s - x in the integral for Wi.,,i in(x), the reader
will

verify that

TODO

(III) The Logarithmic-integral function, which has been discussed by
Euler and others :|:, is defined, when | arg \{- log ■j < tt, by the
equation

* This name is also applied to the function

Erf(x)= I ''e-«'d( = 7r-Erfc(j-).

\{ Legendre, Exercices, i. p. 339; Hocevar, Zeitschrift fiir Math,
und Phys. xxi. (1876), p. 449; Schlomilch; Zeitschrift fur Math, und
Phys. xvi. (1871), p. 261; Prym, Journal filr Math, lxxxii. (1877),
p. 165.

X Euler, Inst. Calc. Int. i.; Soldner, Monatliche- Correspondenz, von
Zach (1811), p. 182; Briefwechsel zwischen Gauss und Bessel (1880),
pp. 114-120; Bessel, Konigsherger Archiv, i. (1812), pp. 369-405;
Laguerre, Bulletin de la Soc.Math.de France, vii. (1879), p. 72;
Stieltjes, Ann. de VEcole norm. sup. (3), iii. (1886). The
logarithmic-integral function is of considerable importance in the
higher parts of the Theory of Prime Numbers. See Landau, Primzahlen,
p. 11.

%
% 342
%

On writing 5 - log z = u and then u = - log t in the integral for

it may be verified that

TODO

It will appear later that Weber's Parabolic Cylinder functions
(\hardsectionref{16}{5})
and Bessel's Circular Cylinder functions (Chapter xvii) are
particular cases of the Wkrn function. Other functions of like
nature are given in the Miscellaneous Examples at the end of this
chapter.

[Note. The error function has been tabulated by Encke, Berliner ast.
Jahrbuch, 1834, pp. 248-304, and Burgess, Tram. Roy. Soc. Edin. xxxix.
(1900), p. 257. The logarithmic- integral function has been tabulated
by Bessel and by Soldner. Jahnke und Emde, Fiinktionentafeln (Leipzig,
1909), and Glaisher, Factor Tables (London, 1883), should also be
consulted.]

\Section{16}{3}{The asymptotic expansion of TT'a;,w \{z), when z is large}
From the contour integral by which Wk,rn,\{z) was defined, it is
possible to obtain an asymptotic expansion for W]cm\{z) valid when
|arg| < tt.

For this purpose, we employ the result given in Chap, v, example 6,
that

1+-) =1+- + ...+ - - + Rn (t, Z),

z) TODO

Substituting this in the formula of \hardsubsectionref{16}{1}{2}, and integrating
term-by-term, it follows from the result of \hardsubsectionref{12}{2}{2} that

yyjc,m\{2) =  -z ji + YY +- 212 +

[m - (k - y-] \{m' -(k- f] . . . (m -\{k-n-if f]

where

n ! z

I ««

provided that n be taken so large that R \{n - k---- m j > 0.

Now, if I arg 2 |  tt - a and z > 1, then

11(1 + /0)11+ R\{z)Q I (1 + tjz) 1  sin a R (z)  O] '

and so*

TODO

* It is supposed that is real; the inequality has to be slightly
modified for complex values of

%
% 343
%

Therefore Rn\{t,z)

TODO

since 1 +;< < 1 + t

Therefore, when j  | > 1,

= TODO

= TODO

since the integral converges. The constant implied in the symbol is
independent of arg2, but depends on a, and tends to infinity as a-
0.

That is to say, the asymptotic expansion of TODO is given by tlie
formula

TODO

for large values ofz when arg 2; |  tt - a < tt.

\Subsection{16}{3}{1}{The second solution of the equation for TODO}

The differential equation (B) of \hardsectionref{16}{1} satisfied by Wk,m\{z) is
unaltered if the signs of z and k are changed throughout.

Hence, if TODO is a solution of the equation.

Since, when $absval{\arg z} < Tr$,

TODO

whereas, when | arg ( - 2) | < tt,

W.k,.n\{-z) = e'H-z)-[l + 0\{z-%

the ratio W]cm\{z)jW-]cjn\{- z) cannot be a constant, and so
Wkm\{z) and -k,m\{- z) form a fundamental system of solutions of
the differential equation.

\Section{16}{4}{Contour integrals of Barnes type for Wc,n(z)}

Consider now

TODO

where | arg; ] < - tt, and neither of the numbers k + m + 5 is a
positive integer

%
% 344
%

or zero*; the contour has loops if necessary so that the poles of T
(s) and those of r ( - s - k-m + 5 ) F (- s - k + m + j are on
opposite sides of it.

It is easily verified, by \hardsectionref{13}{6}, that, as s->cc on the contour,

TODO

and so the integi-al represents a function of 2 which is analytic at
all points f in the domain j arg z TODO

Now choose N so that the poles oi T (- s - k - m + j T (- s - k+ m +
j

are on the right of the line R(s) = - N-; and consider the integral
taken

round the rectangle whose corners are TODO, where  is
positive J and large.

The reader will verify that, when 1 arg z <x, the integrals TODO

tend to zero as - > 00; and so, by Cauchy's theorem,

TODO

where Rn is the residue of the integrand at s = - n.

Write s = - N - I + it, and the modulus of the last integrand is

where the constant implied in the symbol is independent of z.

Since TODO converges, we find that

TODO

* In these cases the series of \hardsectionref{16}{3} terminates and TODO is a
combination of elementary' functions.

t The integral is rendered one- valued when J? (z) <0 by specifying
arg z.

X The line joining ±j may have loops to avoid poles of the integrand
as explained above.

%
% 345
%
But, on calculating the residue Rn, we get

TODO

and so TODO has the same asymptotic expansion as TODO.

Further / satisfies the differential equation for Wk, m () ! for, on

substituting 1 r\{s) F (- s - k - m + j F (- s - k + m + j zds
for v in

the expression (given in \hardsubsectionref{16}{1}{2})

TODO

we get

TODO

Since there are no poles of the last integrand between the contours,
and since the integrand tends to zero as | * j - > oo, s being
between the contours, the expression under consideration vanishes, by
Cauchy's theorem; and so / satisfies the equation for TTjt, m \{2).

Therefore TODO,

where A and B are constants. Making $\absval{TODO} \rightarrow \infty$
when R(z)>0 we see, from the asymptotic expansions obtained for / and
W±k,m\{± ). that

 = 1, 5 = 0.

Accordingly, by the theory of analytic continuation, the equality

I=W,,,n(z) persists for all values of z such that argr|<7r; and, for
values* of arg' such that TT < I arg  I < I TT, Wk,m (z) may be
defined to be the expression /.

Example 1. Shew that

taken along a suitable contour.

* It would have been possible, by modifying the path of integration in \hardsectionref{16}{3}, to have shewn that that integral could be made to define an
analytic function when $\arg z < TODO$. But the reader will see that
it is unnecessary to do so, as Barnes' integral affords a simpler
definition of the function.

%
% 346
%

Example 2. Obtain Barnes' integral for TODO by writing for TODO in the
integral of TODO and changing the order of integration.

TODO

\Subsection{16}{4}{1}{Relations betiveen Wk,m\{z) '-nd Mk,±m\{z)- }
If we take the
expression

TODO

which occurs in Barnes' integral for TODO. and write it in the form

TODO

r(s + k + m + )r\{s + k - m + )cos\{s + k + m) ir cos \{s--k - m)
it ' we see, by \hardsectionref{13}{6}, that, when B (s)  0, we have, as, 5 | - > x
,

F\{s) = exp|C-s--2'jlog5 +

sec \{s + k -- m) tt sec \{s + k - m) ir.

Hence, if | arg 2 | <  tt, jF\{s)z--ds, taken round a semicircle
on the

right of the imaginary axis, tends to zero as the radius of the
semicircle tends to infinity, provided the lower bound of the distance
of the serai- circle from the poles of the integrand is positive (not
zero).

Therefore Tf,,..(.) = - r(,,, +)r(-yfc + m + f) '

where SR' denotes the sum of the residues of F(s) at its poles on the
right of the contour (cf.\hardsectionref{14}{5}) which occurs in equation (C) of §
16*4.

Evaluating these residues we find without difficulty that, when

I arg 2 : < I TT, and 2m is not an integer*,

,,, T(-2m),.,, r(2m),,,

Example 1. Shew that, when | arg ( -s) | <|7r and 2m is not an
integer,

\addexamplecitation{Earnest.} Example 2. AVhen - -stt < arg s < f tt and - f tt < arg ( -
) < tt, shew that

* When 1m is an integer some of the poles are generally double poles,
and their residues involve logarithms of z. The result has not been
proved when fe- |i 7h. is a positive integer or zero, but may be
obtained for such values of k and m by comparing the terminating
series for ')t,m (2) with the series for Mj.±, (2).

t Barnes' results are given in the notation explained in \hardsectionref{16}{1}.

%
% 347
%

Example 3. Obtain Kummer's first formula \hardsubsectionref{16}{1}{1}) from the result

TODO \addexamplecitation{Barnes.}

Inl J -xi

\Section{16}{5}{The parabolic cylinder functions. Weber's equation}

Consider the differential equation satisfied by TODO it is

TODO

this reduces to TODO

Therefore the function satisfies the differential equation

Accordingly Dn(z) is one of the functions associated with the
parabolic cylinder in harmonic analysis*; the equation satisfied by it
will be called Weber's equation.

From \hardsubsectionref{16}{4}{1}, it follows that

Q

when I arg z < -tt.

But

z

4 -i

and these are one-valued analytic functions of z throughout the
TODO-plane. Accordingly Dn (z) is a one-valued function of z
throughout the TODO-plane; and,

by \hardsectionref{16}{4}, its asymptotic expansion when arg  < - tt is

TODO

\Subsection{16}{5}{1}{The second solution of Weber's equation.}

Since Weber's equation is unaltered if we simultaneously replace n and
z by - n - 1 and + iz respectively, it follows that Dn-i (iz) and
D-n-i (- iz) are solutions of Weber's equation, as is also Dn (- z).

* Weber, Math. Ann. i. (1869), pp. 1-36; Whittaker, Proc, London Math.
Soc. xxxv. (1903), pp. 417-427.

%
% 348
%

It is obvious from the asymptotic expansions of Dn\{z) and
Z)i(2e'*), valid in the range -  tt < arg z < -ir, that the
ratio of these two solutions is not a constant.

16'511. The relation between the functions Dn\{z), Dn (+ iz).

From the theory of linear diiSerential equations, a relation of the
form Dn \{z) = aDn-i \{iz) + h Di (- iz) must hold when the
ratio of the functions on the right is not a constant.

To obtain this relation, we observe that if the functions involved be
expanded in ascending powers of z, the expansions are

H i=nr - 1 -  - z + ... ■

Comparing the first two terms we get

a = (27r) - * r (w + 1) e'''' h = (27r) "  T (w + 1) e " and so

r(n + l)

I>n\{z) =

TODO

\Subsection{16}{5}{2}{The general asymptotic expansion of Dn \{z). }
So far the
asymptotic expansion of D (z) for large values of z has only

been given (§ ] 6*5) in the sector arg z < jTt. To obtain its form for
values

of arg z not comprised in this range we write - iz for z and -n - 1
for 7i in the formula of the preceding section, and get

TODO

Now, if TODO, we can assign to TODO and TODO arguments between

TODO + J TT; and arg (- z) = arg z - ir, arg (- iz) = arg z - tt;
and then, applying the asymptotic expansion of \hardsectionref{16}{5} to Dn\{- z) and
Dn-i\{-iz), we see that, TODO

%
% 349
%

This formula is not inconsistent with that of \hardsectionref{16}{5} since in their
common range of validity, viz. TODO for all positive values of w.

To obtain a formula valid in the range TODO, we use the formula

and we get an asymptotic expansion which differs from that which has
just been obtained only in containing e""'' in place of e'"'.

Since Dn(z) is one-valued and one or other of the expansions obtained
is valid for all values of arg z in the range - tt  arg z tt, the
complete asymptotic expansion of i) (z) has been obtained.

\Section{16}{6}{A contour integral for Dn(z)}

TODO Consider TODO, where TODO; it represents a one-valued

analytic function of z throughout the -plane \hardsubsectionref{5}{3}{2}) and further

the differentiations under the sign of integration being easily
justified; accordingly the integral satisfies the differential
equation satisfied by e \~ i Z) (2); and therefore

e-'- e-'f-i-t)-'-'dt = aB\{z) + bDi\{iz),

where a and b are constants.

Now, if the expression on the right be called E (2)5 we have

En\{0)= e-¥\{-t)--dt, En'\{0)= e-¥-t)-dt.

To evaluate these integrals, which are analytic functions of n, we
suppose first that R \{n) <0; then, deforming the paths of
integration, we get

TODO

Similarly TODO.

Both sides of these equations being analytic functions of TODO, the
equations are true for all values of n; and therefore

TODO

Therefore TODO.

%
% 350
%

\Subsection{16}{6}{1}{Recurrence formulae for Dn \{z). }
From the equation

after using \hardsectionref{16}{6}, we see that

Dn+, (z) - z Dn (z) + n Dn-, \{z) = 0. Further, by differentiating the
integral of \hardsectionref{16}{6}, it follows that

D \{z) + zDn \{z) - nDn-, \{z) = 0. Example. Obtain these results
from the ascendiug power series of \hardsectionref{16}{5}.

\Section{16}{7}{Properties of Dn (z) when n is an integer}

When n is an integer, we may write the integral of \hardsectionref{16}{6} in the form

TODO

If now we write t = v - z, we get

TODO

a result due to Hermite*.

Also, if m and n be unequal integers, we see from the differential
equations that

Dn \{z) Dm" (z) - D, \{z) Dn" (z) + (m - n) Dm (z) Dn (z) = 0, and so

Dn\{z)DJ\{z)-Dm(z)Dn'\{z)

\{m - 71) I Di (z) Dn (z) dz =

J -ex

= 0,

by the expansion of \hardsectionref{16}{5} in descending powers of z (which terminates
and is valid for all values of arg z when n is a positive integer).

Therefore if m and n are unequal positive integers

D,iz)Dn\{z)dz=0.

■>

Comptes Rendus, lviii. (1864), pp. 266-273.

%
% 351
%

On the other hand, when 7n = n, we have

J - cc

= D, (Z) Dn+, (Z)] + I \l zDn iz) D,,, \{Z) - Dn+ (z) D' (z) dz

=r [D,,\{z)Ydz,

J -X

on using the recurrence formula, integrating by parts and then using
the recurrence formula again.

It follows by induction that

f \{Dr,(z)Y-dz = nir [D,\{z)Ydz

J -CO

= (27r)n!, by \hardsubsectionref{12}{1}{4} corollary 1 and \hardsectionref{12}{2}.

It follows at once that if, for a function /(), an expansion of the
form

/(z) = aoDo (z) + a, D,\{z)+... + aDn \{z)+ ...

exists, and if it is legitimate to integrate term-by-term between the
limits - oc and oo, then

TODO

REFERENCES.

W. Jacobsthal, Math. Ann. LVi. (1903), pp. 129-154.

E. W. Barnes, Trans. Camb. Phil. Soe. xx. (1908), pp. 253-279.

E. T. Whittaker, Bulletin American Math. Soc. x. (1904), pp. 125-134.

H. Weber, Math. Ann. i. (1869), pp. 1-36.

A. Adamoff, Ann. de VInstitut Polytechnique de St Petershourg, v.
(1906), pp. 127-143.

E. T. Whittaker, Proc. London Math. Soc. xxxv. (1903), pp. 417-427.

G. N. Watson, Proc. London Math. Soc. (2), viii. (1910), pp. 393-421;
xvn. (1919), pp. 116-148.

H. E. J. CuRZON, Proc. London Math. Soc. (2), xii. (1913), pp.
236-259.

A. Milne, Proc. Edinburgh Math. Soc. xxxii. (1914), pp. 2-14; xxxill.
(1915), pp. 48-64.

N. Nielsen, Meddelelser K. Danske Videnskabernes Selskab, i. (1918),
no. 6.

%
% 352
%

Miscellaneous Examples.

1. Shew that, if the integral is convergent, then

TODO

2. Shew that TODO

3. Obtain the recurrence formulae

4. Prove that Ht,,1(2) is the integral of an elementary function when
either of the numbers k-h + m is a negative integer,

5. Shew that, by a suitable change of variables, the equation can be
brought to the form

\{a.2 + b2.v) + \{ai + biX)-£ + \{ao + box)i/ =

derive this equation from the equation for F\{a, b; c; x) by writing x
= lb and making 6-*-« .

6. Shew that the cosine integral of Schlomilch and Besso \{Oiornale di
Matematiche, VI.), defined by the equation

TODO

is equal to TODO

Shew also that Schlomilch's function, defined \{Zeitschrift filr Math,
und Physik, iv. (1859), p. 390) by the equations

S\{v,z)=j \{l + t)-''e-'dt = z''-e' I du,

is equal to i" - 1 i W  1  1  1  (2).

7. Express in terms of W,ci functions the two functions

TODO

Jot J z t

8. Shew that Sonine's polynomial, defined \{Math. Ann. xvi. p. 41) by
the equation TODO

is equal to TODO

%
% 353
%

9. Shew that the function TODO defined by Lagrange in 1762-1765
\{Oeuv)-es, i. p. 520) and by Abel (Oeuvres, 1881, p. 284) as the
coefficient of /('*' in the expansion of (1 - A)-i e-''Mi-ft) is eqnal
to

10*. Shew that the Pearson-Cunningham function \{Proc. Royal Soc.
Lxxxi. p. 310), <>>n,m (■')) defined as

T\{n

n- hm

is equal to  TODO

11. Shew that, if | arg z < n, and | arg (1 + ) | < tt,

\addexamplecitation{Whittaker.}

12. Shew that, if n be not a positive integer and if ! arg z < frr,
then

TODO

and that this result holds for all values of args if the integral be /
, the contours enclosing the poles of r ( - but not those of r \{t
-n).

13. Shew that, if j arg a | < it,

J tc

= - r, n(-*". hn-¥i<;hn-hn + - l-ia-).

r(-w)r(m-i + l)aH'+i)

14. Deduce from example 13 that, if the integral is convergent, then

|J e - i'' z""' A«+i (2) c/2 = (v/2)-i-'" r (i + 1) sin ( - \{m) n.

\addexamplecitation{Watson.}

15. Shew that, if n be a positive integer, and if

E, \{x) = T J- '' \{z - .v) - 1 n,, (z) dz,

then TODO.

the upper or lower signs being taken according as the imaginary part
of x is positive or negative. \addexamplecitation{Watson.}

16. Shew that, if n be a positive integer,

TODO

Jo sm

where fi is n or J(n- 1), whichever is an integer, and the cosine or
sine is taken as n is even or odd. \addexamplecitation{AdamoflF.}

* The results of examples 8, 9, 10 were communicated to us by Mr
Bateman. W. M. A. 23

%
% 354
%

17. Shew that, if n be a positive integer,

where TODO

TODO \addexamplecitation{Adamoff}

18. With the notation of the preceding examples, shew that, when x is
real,

TODO

while Ja satisfies both the inequalities

TODO

Shew also that as v increases from to 1, o- (/') decreases from to a
minimum at- TODO and then increases to at i'=l; and as v increases
from 1 to $\infty$, o-(v) increases to a maximum at l + /?2 and then
decreases, its limit being zero; where

TODO \addexamplecitation{Adamoff.}

19. By employing the second mean value theorem when necessary, shew
that

A.(.') = V2.(v'0 e

cos(.r«2 -:-)iTr) + '

s'n J'

where co,j(.r) satisfies both the inequalities

I X I 'TT D

when X is real and n is an integer greater than 2. \addexamplecitation{AdamofF.}

3-35... ia;2,,,, .1 i

20. Shew that, if n be positive but otherwise unrestricted, and if m
be a positive integer (or zero), then the equation in z

has m positive roots when TODO. \addexamplecitation{Milne.}


\section{More inductive types}

Analogous to the type of natural numbers, many types can be specified as inductive types. In this section we introduce some further examples of inductive types: the unit type, the empty type, the booleans, coproducts, dependent pair types, and cartesian products. We also introduce the type of integers.

\subsection{The idea of general inductive types}

Just like the type of natural numbers, other inductive types are also specified by their \emph{constructors}, an \emph{induction principle}, and their \emph{computation rules}:
\begin{enumerate}
\item The constructors tell what structure the inductive type comes equipped with. There may any finite number of constructors, even no constructors at all, in the specification of an inductive type.
\item The induction principle specifies the data that should be provided in order to construct a section of an arbitrary type family over the inductive type.
\item The computation rules assert that the inductively defined section agrees on the constructors with the data that was used to define the section. Thus, there is a computation rule for every constructor.
\end{enumerate}
The induction principle and computation rules can be generated automatically once the constructors are specified, but it goes beyond the scope of our course to describe general inductive types.
%For a more general treatment of inductive types, we refer to Chapter 5 of \cite{hottbook}.

\subsection{The unit type}
\index{unit type|(}
\index{inductive type!unit type|(}
A straightforward example of an inductive type is the \emph{unit type}, which has just one constructor.
Its induction principle is analogous to just the base case of induction on the natural numbers.

\begin{defn}
We define the \define{unit type}\index{1 @{$\unit$}|see {unit type}}\index{unit type} to be a closed type $\unit$\index{unit type!is a closed type} equipped with a closed term\index{unit type!star@{$\ttt$}}
\begin{equation*}
\ttt:\unit,
\end{equation*}
satisfying the induction principle\index{induction principle!of unit type}\index{unit type!induction principle} that for any type family of types $P(x)$ indexed by $x:\unit$, there is a term\index{ind 1@{$\indunit$}}\index{unit type!indunit@{$\indunit$}}
\begin{equation*}
\indunit : P(\ttt)\to\prd{x:\unit}P(x)
\end{equation*}
for which the computation rule\index{computation rules!of unit type}\index{unit type!computation rules}
\begin{equation*}
\indunit(p,\ttt) \jdeq p
\end{equation*}
holds. Sometimes we write $\lam{\ttt}p$ for $\indunit(p)$.
\end{defn}

The induction principle can also be used to define ordinary functions out of the unit type. Indeed, given a type $A$ we can first weaken it to obtain the constant family over $\unit$, with value $A$. Then the induction principle of the unit type provides a function
\begin{equation*}
  \indunit : A \to (\unit\to A).
\end{equation*}
In other words, by the induction principle for the unit type we obtain for every $x:A$ a function $\pt_x\defeq\indunit(x):\unit\to A$.\index{ptx@{$\pt_x$}}
\index{unit type|)}
\index{inductive type!unit type|)}

\subsection{The empty type}
\index{empty type|(}
\index{inductive type!empty type|(}
The empty type is a degenerate example of an inductive type. It does \emph{not} come equipped with any constructors, and therefore there are also no computation rules. The induction principle merely asserts that any type family has a section. In other words: if we assume the empty type has a term, then we can prove anything.

\begin{defn}
We define the \define{empty type}\index{0 @{$\emptyt$}|see {empty type}} to be a type $\emptyt$ satisfying the induction principle\index{induction principle!of empty type}\index{empty type!induction principle} that for any family of types $P(x)$ indexed by $x:\empty$, there is a term\index{ind 0@{$\indempty$}}\index{empty type!indempty@{$\indempty$}}
\begin{equation*}
\indempty : \prd{x:\emptyt}P(x).
\end{equation*}
\end{defn}

The induction principle for the empty type can also be used to construct a function
\begin{equation*}
  \emptyt\to A
\end{equation*}
for any type $A$. Indeed, to obtain this function one first weakens $A$ to obtain the constant family over $\emptyt$ with value $A$, and then the induction principle gives the desired function.

Thus we see that from the empty type anything follows. Therefore, we we see that anything follows from $A$, if we have a function from $A$ to the empty type. This motivates the following definition.

\begin{defn}
  For any type $A$ we define \define{negation}\index{negation!of types}\index{neg (A)@{$\neg A$}|see {negation}} of $A$ by
  \begin{equation*}
    \neg A\defeq A\to\emptyt.
  \end{equation*}
\end{defn}

Since $\neg A$ is the type of functions from $A$ to $\emptyt$, a proof of $\neg A$ is given by assuming that $A$ holds, and then deriving a contradiction. This proof technique is called \define{proof of negation}\index{proof of negation}. Proofs of negation are not to be confused with \emph{proofs by contradiction}\index{proof by contradiction}. In type theory there is no way of obtaining a term of type $A$ from a term of type $(A\to \emptyt)\to\emptyt$.
\index{empty type|)}
\index{inductive type!empty type|)}

\subsection{The booleans}
\index{booleans}
\index{inductive type!booleans}

\begin{defn}
We define the \define{booleans}\index{2 @{$\bool$}|see {booleans}} to be a type $\bool$ that comes equipped with\index{booleans!btrue@{$\btrue$}}\index{booleans!bfalse@{$\bfalse$}}\index{0 2@{$\bfalse$}}\index{1 2@{$\btrue$}}
\begin{align*}
\bfalse & : \bool \\
\btrue & : \bool
\end{align*}
satisfying the induction principle\index{induction principle!of booleans}\index{booleans!induction principle} that for any family of types $P(x)$ indexed by $x:\bool$, there is a term\index{ind 2@{$\indbool$}}
\begin{equation*}
\indbool : P(\bfalse)\to \Big(P(\btrue)\to \prd{x:\bool}P(x)\Big)
\end{equation*}
for which the computation rules\index{computation rules!of booleans}\index{booleans!computation rules}
\begin{align*}
\indbool(p_0,p_1,\bfalse) & \jdeq p_0 \\
\indbool(p_0,p_1,\btrue) & \jdeq p_1
\end{align*}
hold.
\end{defn}

Just as in the cases for the unit type and the empty type, the induction principle for the booleans can also be used to construct an ordinary function $\bool\to A$, provided that we can construct two terms of type $A$. Indeed, by the induction principle for the booleans there is a function
\begin{equation*}
  \indbool : A \to (A\to A^\bool)
\end{equation*}
for any type $A$.

\begin{eg}\label{eg:boolean-ops}
  \index{boolean operations|(}\index{boolean logic|(}
  Using the induction principle of $\bool$ we can define all the operations of Boolean algebra\index{boolean algebra}. For example, the \define{boolean negation}\index{booleans!negation} operation $\negbool : \bool \to \bool$\index{negation function!on booleans}\index{neg 2@{$\negbool$}}\index{booleans!neg 2@{$\negbool$}} is defined by
  \begin{align*}
    \negbool(\btrue) & \defeq \bfalse & \negbool(\bfalse) & \defeq \btrue.
  \end{align*}
  The \define{boolean conjunction}\index{booleans!conjunction} operation $\blank\land\blank : \bool \to (\bool\to \bool)$ is defined by
  \begin{align*}
    \btrue\land\btrue & \defeq \btrue & \bfalse\land\btrue & \defeq \bfalse \\
    \btrue\land\bfalse & \defeq \bfalse & \bfalse\land\bfalse & \defeq \bfalse.
  \end{align*}
  The \define{boolean disjunction}\index{booleans!disjunction} operation $\blank\lor\blank : \bool \to (\bool\to \bool)$ is defined by
  \begin{align*}
    \btrue\lor\btrue & \defeq \btrue & \bfalse\lor\btrue & \defeq \btrue \\
    \btrue\lor\bfalse & \defeq \btrue & \bfalse\lor\bfalse & \defeq \bfalse.
  \end{align*}
  We leave the definitions of some of the other boolean operations as \cref{ex:boolean-operation}. Note that the method of defining the boolean operations by the induction principle of $\bool$ is not that different from defining them by truth tables\index{truth tables}.

  Boolean logic is important, but it won't be very prominent in this course. The reason is simple: in type theory it is more natural to use the `logic' of types that is provided by the inference rules.\index{boolean operations|)}\index{boolean logic|)}
\end{eg}
\index{booleans|)}
\index{inductive type!booleans|)}

\subsection{Coproducts and the type of integers}
\index{coproduct|(}
\index{inductive type!coproduct|(}
\begin{defn}
Let $A$ and $B$ be types. We define the \define{coproduct}\index{disjoint sum|see {coproduct}} $A+B$\index{A + B@{$A+B$}|see {coproduct}} to be a type that comes equipped with\index{inl@{$\inl$}}\index{coproduct!inl@{$\inl$}}\index{inr@{$\inr$}}\index{coproduct!inr@{$\inr$}}
\begin{align*}
\inl & : A \to A+B \\
\inr & : B \to A+B
\end{align*}
satisfying the induction principle\index{induction principle!of coproduct}\index{coproduct!induction principle} that for any family of types $P(x)$ indexed by $x:A+B$, there is a term\index{ind +@{$\ind{+}$}}\index{coproduct!ind+@{$\ind{+}$}}
\begin{equation*}
\ind{+} : \Big(\prd{x:A}P(\inl(x))\Big)\to\Big(\prd{y:B}P(\inr(y))\Big)\to\prd{z:A+B}P(z)
\end{equation*}
for which the computation rules\index{computation rules!of coproduct}\index{coproduct!computation rules}
\begin{align*}
\ind{+}(f,g,\inl(x)) & \jdeq f(x) \\
\inr{+}(f,g,\inr(y)) & \jdeq g(y)
\end{align*}
hold. Sometimes we write $[f,g]$ for $\ind{+}(f,g)$.
\end{defn}

The coproduct of two types is sometimes also called the \define{disjoint sum}. By the induction principle of coproducts it follows that we have a function
\begin{equation*}
  (A\to X) \to \big((B\to X) \to (A+B\to X)\big)
\end{equation*}
for any type $X$. Note that this special case of the induction principle of coproducts is very much like the elimination rule of disjunction in first order logic: if $P$, $P'$, and $Q$ are propositions, then we have
\begin{equation*}
  (P\to Q)\to \big((P'\to Q)\to (P\lor P'\to Q)\big).
\end{equation*}
Indeed, we can think of \emph{propositions as types} and of terms as their constructive proofs. Under this interpretation of type theory the coproduct is indeed the disjunction.

\index{integers|(}
An important example of a type that can be defined using coproducts is the type $\Z$ of integers.\index{coproduct!Z@{$\Z$}}

\begin{defn}
  We define the \define{integers}\index{Z@{$\Z$}|see {integers}} to be the type $\Z\defeq\N+(\unit+\N)$. The type of integers comes equipped with inclusion functions of the positive and negative integers\index{integers!in-pos@{$\inpos$}}\index{integers!in-neg@{$\inneg$}}
  \begin{align*}
    \inpos & \defeq \inr\circ\inr \\
    \inneg & \defeq \inl,
  \end{align*}
  which are both of type $\N\to\Z$, and the constants\index{integers!-1 Z@{$-1_\Z$}}\index{integers!0 Z@{$0_\Z$}}\index{integers!1 Z@{$1_\Z$}}\index{-1 Z@{$-1_\Z$}}\index{0 Z@{$0_\Z$}}\index{1 Z@{$1_{\Z}$}}
  \begin{align*}
    -1_\Z & \defeq \inneg(0)\\
    0_\Z & \defeq \inr(\inl(\ttt))\\
    1_\Z & \defeq \inpos(0).
  \end{align*}
\end{defn}

In the following lemma we derive an induction principle\index{induction principle!of Z@{of $\Z$}}\index{integers!induction principle} for $\Z$, which can be used in many familiar constructions on $\Z$, such as in the definitions of addition and multiplication.

\begin{lem}\label{lem:Z_ind}
  Consider a type family $P$ over $\Z$. If we are given
  \begin{align*}
    p_{-1} & :P(-1_\Z) \\
    p_{-S} & : \prd{n:\N}P(\inneg(n))\to P(\inneg(\succN(n)))\\
    p_{0} & : P(0_\Z) \\
    p_{1} & : P(1_\Z) \\
    p_{S} & : \prd{n:\N}P(\inpos(n))\to P(\inpos(\succN(n))),
  \end{align*}
  then we can construct a dependent function $f:\prd{k:\Z}P(k)$ for which the following judgmental equalities hold:\index{integers!computation rules}\index{computation rules!of Z@{of $\Z$}}
  \begin{align*}
    f(-1_\Z) & \jdeq p_{-1} \\
    f(\inneg(\succN(n))) & \jdeq p_{-S}(n,f(\inneg(n))) \\
    f(0_\Z) & \jdeq p_{0} \\
    f(1_\Z) & \jdeq p_{1} \\
    f(\inpos(\succN(n))) & \jdeq p_S(n,f(\inpos(n))).
  \end{align*}
\end{lem}

\begin{proof}
  Since $\Z$ is the coproduct of $\N$ and $\unit+\N$, it suffices to define
  \begin{align*}
    p_{inl} & : \prd{n:\N}P(\inl(n)) \\
    p_{inr} & : \prd{t:\unit+\N}P(\inr(t)).
  \end{align*}
  Note that $\inneg\jdeq\inl$ and $-1_\Z\jdeq \inneg(\zeroN)$. In order to define $p_{inl}$ we use induction on the natural numbers, so it suffices to define
  \begin{align*}
    p_{-1} & : P(-1) \\
    p_{-S} & : \prd{n:\N} P(\inneg(n))\to P(\inneg(\succN(n))).
  \end{align*}
  Similarly, we proceed by coproduct induction, followed by induction on $\unit$ in the left case and induction on $\N$ on the right case, in order to define $p_{inr}$.
\end{proof}

As an application we define the successor function on the integers.

\begin{defn}
We define the \define{successor function}\index{successor function!on Z@{on $\Z$}}\index{function!succ Z@{$\succZ$}} on the integers $\succZ:\Z\to\Z$\index{succ Z@{$\succZ$}}\index{integers!succ Z@{$\succZ$}} using the induction principle of \cref{lem:Z_ind}, taking
\begin{align*}
\succZ(-1_\Z) & \defeq 0_\N \\
\succZ(\inneg(\succN(n))) & \defeq \inneg(n) \\
\succZ(0_\Z) & \defeq 1_\N \\
\succZ(1_\Z) & \defeq \inpos(1_\N) \\
\succZ(\inpos(\succN(n))) & \defeq \inpos(\succN(\succN(n))).
\end{align*}
\end{defn}
\index{integers|)}
\index{coproduct|)}
\index{inductive type!coproduct|)}

\subsection{Dependent pair types}

\index{dependent pair type|(}
\index{inductive type!dependent pair type|(}

Given a type family $B$ over $A$, we may consider pairs $(a,b)$ of terms, where $a:A$ and $b:B(a)$. Note that the type of $b$ depends on the first term in the pair, so we call such a pair a \define{dependent pair}\index{dependent pair}.

The \emph{dependent pair type} is an inductive type that is generated by the dependent pairs.

\begin{defn}
  Consider a type family $B$ over $A$.
  The \define{dependent pair type} (or $\Sigma$-type) \index{Sigma-type@{$\Sigma$-type}|see {dependent pair type}}is defined to be the inductive type $\sm{x:A}B(x)$ equipped with a \define{pairing function}\index{pairing function}\index{(-,-)@{$(\blank,\blank)$}}\index{dependent pair type!(-,-)@{$(\blank,\blank)$}}
\begin{equation*}
(\blank,\blank):\prd{x:A} \Big(B(x)\to \sm{y:A}B(y)\Big).
\end{equation*}
The induction principle\index{induction principle!of Sigma types@{of $\Sigma$-types}}\index{dependent pair type!induction principle} for $\sm{x:A}B(x)$ asserts that for any family of types $P(p)$ indexed by $p:\sm{x:A}B(x)$, there is a function\index{dependent pair type!indSigma@{$\ind{\Sigma}$}}\index{ind Sigma@{$\ind{\Sigma}$}}
\begin{equation*}
\ind{\Sigma}:\Big(\prd{x:A}\prd{y:B(x)}P(x,y)\Big)\to\Big(\prd{p:\sm{x:A}B(x)}P(p)\Big).
\end{equation*}
satisfying the computation rule\index{computation rules!of Sigma types@{of $\Sigma$-types}}\index{dependent pair type!computation rule}
\begin{equation*}
\ind{\Sigma}(f,(x,y))\jdeq f(x,y).
\end{equation*}
Sometimes we write $\lam{(x,y)}f(x,y)$ for $\ind{\Sigma}(\lam{x}\lam{y}f(x,y))$.
\end{defn}

\begin{defn}
Given a type $A$ and a type family $B$ over $A$, the \define{first projection map}\index{first projection map}\index{projection maps!first projection}\index{dependent pair type!pr 1@{$\proj 1$}}\index{pr 1@{$\proj 1$}}\index{function!pr 1@{$\proj 1$}}
\begin{equation*}
\proj 1:\Big(\sm{x:A}B(x)\Big)\to A
\end{equation*}
is defined by induction as
\begin{equation*}
\proj 1\defeq \lam{(x,y)}x.
\end{equation*}
The \define{second projection map}\index{second projection map}\index{projection map!second projection}\index{dependent pair type!pr 2@{$\proj 2$}}\index{pr 2@{$\proj 2$}}\index{function!pr 2@{$\proj 2$}} is a dependent function
\begin{equation*}
\proj 2 : \prd{p:\sm{x:A}B(x)} B(\proj 1(p))
\end{equation*}
defined by induction as
\begin{equation*}
\proj 2\defeq \lam{(x,y)}y.
\end{equation*}
By the computation rule we have
\begin{align*}
\proj 1 (x,y) & \jdeq x \\
\proj 2 (x,y) & \jdeq y.
\end{align*}
\end{defn}
\index{dependent pair type|)}
\index{inductive type!dependent pair type|)}

\subsection{Cartesian products}

\index{cartesian product|(}
\index{inductive type!cartesian product|(}
A special case of the $\Sigma$-type occurs when the $B$ is a constant family over $A$, i.e., when $B$ is just a type.
In this case, the inductive type $\sm{x:A}B(x)$ is generated by \emph{ordinary} pairs $(x,y)$ where $x:A$ and $y:B$. In other words, if $B$ does not depend on $A$, then the type $\sm{x:A}B$ is the \emph{(cartesian) product} $A\times B$.
The cartesian product is a very common special case of the dependent pair type, just as the type $A\to B$ of ordinary functions from $A\to B$ is a common special case of the dependent product. Therefore we provide its specification along with the induction principle for cartesian products.

\begin{defn}
Consider two types $A$ and $B$. The \define{(cartesian) product}\index{product of types}\index{A x B@{$A\times B$}|see {cartesian product}} of $A$ and $B$ is defined as the inductive type $A\times B$ with constructor
\begin{equation*}
(\blank,\blank):A\to (B\to A\times B).
\end{equation*}
The induction principle\index{induction principle!of cartesian products}\index{cartesian product!induction principle} for $A\times B$ asserts that for any type family $P$ over $A\times B$, one has\index{ind times@{$\ind{\times}$}}\index{cartesian product!indtimes@{$\ind{\times}$}}
\begin{equation*}
\ind{\times} : \Big(\prd{x:A}\prd{y:B}P(a,b)\Big)\to\Big(\prd{p:A\times B} P(p)\Big)
\end{equation*}
satisfying the computation rule\index{computation rules!of cartesian product}\index{cartesian product!computation rule} that
\begin{align*}
\ind{\times}(f,(x,y)) & \jdeq f(x,y).
\end{align*}
\end{defn}

The projection maps are defined similarly to the projection maps of $\Sigma$-types. When one thinks of types as propositions\index{propositions as types!conjunction}, then $A\times B$ is interpreted as the conjunction of $A$ and $B$.
\index{cartesian product|)}
\index{inductive type!cartesian product|)}

\begin{exercises}
\exercise
  \index{rules!for unit type}\index{unit type!rules}
  \index{rules!for empty type}\index{empty type!rules}
  \index{rules!for booleans}\index{booleans!rules}
  \index{rules!for coproduct}\index{coproduct!rules}
  \index{rules!for dependent pair type}\index{dependent pair type!rules}
  \index{rules!for cartesian product}\index{cartesian product!rules}
  Write the rules for $\unit$, $\emptyt$, $\bool$, $A+B$, $\sm{x:A}B(x)$, and $A\times B$. As usual, present the rules in four sets:
  \begin{enumerate}
  \item A formation rule.
  \item Introduction rules.
  \item An elimination rule.
  \item Computation rules.
  \end{enumerate}
  \exercise Let $P$ and $Q$ be types. Use the fact that $\neg P$\index{negation} is defined as the type $P\to\emptyt$ of functions from $P$ to the empty type\index{empty type}, to give type theoretic proofs of the following taugologies\index{tautologies} of constructive logic\index{constructive logic}.\label{ex:dne-dec}
  \begin{subexenum}
  \item $P\to\neg\neg P$
  \item $(P\to Q)\to(\neg\neg P\to\neg\neg Q)$
  \item $(P+\neg P)\to(\neg\neg P\to P)$
  \item $\neg\neg(P+\neg P)$
  \item $\neg\neg(\neg\neg P \to P)$
  \item $(P\to \neg\neg Q)\to (\neg\neg P \to\neg\neg Q)$
  \item $\neg\neg\neg P \to \neg P$
  \item $\neg\neg(P \to \neg\neg Q)\to (P\to\neg\neg Q)$
  \item $\neg\neg((\neg\neg P)\times(\neg\neg Q))\to (\neg\neg P)\times(\neg\neg Q)$
  \end{subexenum}
\exercise \label{ex:boolean-operation}Define the following operations of Boolean algebra:\index{boolean algebra}\index{booleans!exclusive disjunction}\index{booleans!implication}\index{booleans!if and only if}\index{booleans!Peirce's arrow}\index{booleans!Sheffer stroke}
  \begin{center}
    \begin{tabular}{ll}
      exclusive disjunction & $p \oplus q$ \\
      implication & $p \Rightarrow q$ \\
      if and only if & $p \Leftrightarrow q$ \\
      Peirce's arrow (neither \dots{} nor) & $p \downarrow q$ \\
      Sheffer stroke (not both) & $p\mid q$.
    \end{tabular}
  \end{center}
  Here $p$ and $q$ range over $\bool$.
\exercise \label{ex:int_pred}\index{integers|(}\index{predecessor function}\index{function!pred Z@{$\predZ$}}\index{integers!pred Z@{$\predZ$}}\index{pred Z@{$\predZ$}}Define the predecessor function $\predZ:\Z\to \Z$.
\exercise \label{ex:int_group_ops}\index{group operations!on Z@{on $\Z$}}Define the group operations\index{add Z@{$\addZ$}}\index{integers!add Z@{$\addZ$}}\index{neg Z@{$\negZ$}}\index{integers!neg Z@{$\negZ$}}\index{mul Z@{$\mulZ$}}\index{integers!mul Z@{$\mulZ$}}
  \begin{align*}
    \addZ & : \Z \to (\Z \to \Z) \\
    \negZ & : \Z \to \Z,
    \intertext{and define the multiplication}
    \mulZ & : \Z \to (\Z \to \Z).
  \end{align*}
\exercise Construct a function $F:\Z\to\Z$ that extends the Fibonacci sequence\index{Fibonacci sequence}\index{integers!Fibonacci sequence} to the negative integers
  \begin{equation*}
    \ldots,5,-3,2,-1,1,0,1,1,2,3,5,8,13,\ldots
  \end{equation*}
  in the expected way.\index{integers|)}
\exercise \label{ex:one_plus_one} Show that $\unit+\unit$ satisfies the same induction principle\index{induction principle!of booleans} as $\bool$, i.e., define
  \begin{align*}
    t_0 & : \unit + \unit \\
    t_1 & : \unit + \unit,
  \end{align*}
  and show that for any type family $P$ over $\unit+\unit$ there is a function
  \begin{align*}
    \ind{\unit+\unit}:P(t_0)\to \Big(P(t_1)\to \prd{t:\unit+\unit}P(t)\Big)
  \end{align*}
  satisfying
  \begin{align*}
    \ind{\unit+\unit}(p_0,p_1,t_0) & \jdeq p_0 \\
    \ind{\unit+\unit}(p_0,p_1,t_1) & \jdeq p_1.
  \end{align*}
  In other words, \emph{type theory cannot distinguish between the types $\bool$ and $\unit+\unit$.}
\exercise \label{ex:lists}For any type $A$ we can define the type $\lst(A)$\index{list A@{$\lst(A)$}|see {lists in $A$}} of \define{lists}\index{lists in A @{lists in $A$}}\index{inductive type!list A@{$\lst(A)$}} elements of $A$ as the inductive type with constructors\index{lists in A@{lists in $A$}!nil@{$\nil$}}\index{nil@{$\nil$}}\index{cons(a,l)@{$\cons(a,l)$}}\index{lists in A@{lists in $A$}!cons@{$\cons$}}
  \begin{align*}
    \nil & : \lst(A) \\
    \cons & : A \to (\lst(A) \to \lst(A)).
  \end{align*}
  \begin{subexenum}
  \item Write down the induction principle and the computation rules for $\lst(A)$.\index{induction principle!list A@{$\lst(A)$}}\index{lists in A@{lists in $A$}!induction principle}
  \item Let $A$ and $B$ be types, suppose that $b:B$, and consider a binary operation $\mu:A\to (B \to B)$. Define a function\index{fold-list@{$\foldlist$}}\index{lists in A@{lists in $A$}!fold-list@{$\foldlist$}}
    \begin{equation*}
      \foldlist(\mu) : \lst(A)\to B
    \end{equation*}
    that iterates the operation $\mu$, starting with $\foldlist(\mu,\nil)\defeq b$.
  \item Define a function $\lengthlist:\lst(A)\to\N$.\index{length-list@{$\lengthlist$}}\index{lists in A@{lists in $A$}!length-list@{$\lengthlist$}}
  \item Define a function\index{sum-list@{$\sumlist$}}\index{lists in A@{lists in $A$}!sum-list@{$\sumlist$}}
    \begin{equation*}
      \sumlist : \lst(\N) \to \N
    \end{equation*}
    that adds all the elements in a list of natural numbers.
  \item Define a function\index{concat-list@{$\concatlist$}}\index{lists in A@{lists in $A$}!concat-list@{$\concatlist$}}\index{concatenation!of lists}
    \begin{equation*}
      \concatlist : \lst(A) \to (\lst(A) \to \lst(A))
    \end{equation*}
    that concatenates any two lists of elements in $A$.
  \item Define a function\index{flatten-list@{$\flattenlist$}}\index{lists in A@{lists in $A$}!flatten-list@{$\flattenlist$}}
    \begin{equation*}
      \flattenlist : \lst(\lst(A)) \to \lst(A)
    \end{equation*}
    that concatenates all the lists in a lists of lists in $A$.
  \item Define a function $\reverselist : \lst(A) \to \lst(A)$ that reverses the order of the elements in any list.\index{reverse-list@{$\reverselist$}}\index{lists in A@{lists in $A$}!reverse-list@{$\reverselist$}}
  \end{subexenum}
\end{exercises}


% !TEX root = scombinatorics.tex
\documentclass[scombinatorics.tex]{subfiles}
\begin{document}
\chapter{Small transversals}
\label{fractional}

\def\medrel#1{\parbox[t]{5ex}{$\displaystyle\hfil #1$}}
\def\ceq#1#2#3{\parbox[t]{23ex}{$\displaystyle #1$}\medrel{#2}{$\displaystyle #3$}}
Unlike the rest of these notes, this chapter is not self contained, as it relies on the duality of linear programming.
The reader can use the result as a black box.
Otherwise, we recommend~\cite{LPmatousek}*{Chapter 6}, a lively and conceptual introduction to linear programming (a rarity for an otherwise rather dry subject).

%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%
\section{Transversals and packings}\label{Transversals_Packings}

Let $\phi(x\,;z)$ be given.
Let $A\subseteq\U$ and $B\subseteq\V$ be finite sets.
%(It seems that most of the section smoothly generalizes to rational valued formulas and multisets, see below.)

A subset $A'\subseteq A$ is a \emph{transversal\/} if $\phi(A',b)\neq\0$ for every $b\in B$. Equivalently, if the sets $\phi(a,B)_{a\in A'}$ cover $B$, i.e.

\ceq{\hfill B}
{=}
{\bigcup_{a\in A'}\phi(a,B).}

The \emph{transversal number\/} is the smallest cardinality of a transversal $A'$.
It denoted by \emph{$\tau$.}

A subset $B'\subseteq B$ is a \emph{packing\/} if $\phi(A,b)\cap\phi(A,b')=\0$ for every distinct $b,b'\in B'$. Equivalently if $|\phi(a,B')|\le1$ for every $a\in A$.
The \emph{packing number\/} is the largest cardinality of a packing $B'$.
It is denoted by \emph{$\nu$.}

We may write \emph{$\tau_{\phi(A,B)}$} and  \emph{$\nu_{\phi(A,B)}$} when ambiguity is of concern.

If $A'$ is a transversal and $B'\subseteq B$, then the sets $\phi(a,B')_{a\in A'}$ cover $B'$.
Now, suppose $B'$ is a packing, then these sets contain at most one element, hence $|B'|\le|A'|$.
Therefore, we always have $\nu\le\tau$.
Very little can be said in general about the reverse direction.

\begin{example}\label{expl_tr_pk}
  Let $\U=\RR^2$ and $\V$ is the set of lines in $\RR^2$.
  Let $\phi(x\,;z)$ be the incidence (that is, membership) relation.
  Let $A\subseteq\U$ and let $B\subseteq\V$ be a set of $n$ lines in generic position
  (any to lines intersect and every point is contained in at most two lines).
  Then $\tau=\lceil n/2\rceil$, as each point belongs to at most two lines, while $\nu=1$, as any two lines intersect.\QED
\end{example}

A (fractional) multiset over $\U$ is a real-valued function $A':\U\to\RR$.
The support \emph{support} of a multiset $A'$ is the set where it takes nonzero values.
In this chapter will only consider nonnegative multisets with finite support.
These can be interpreted as measures concentrated on a finite set.

If $A'':\U\to\RR$ is another multiset, we write \emph{$A'\cdot A''$\/} for the pointwise product of the two.
We write \emph{$A'\le A''$\/} if $A'(a)\le A''(a)$ for every $a\in \U$.

We define the \emph{size\/} of $A'$ to be\smallskip

\ceq{\hfill\emph{$|A'|$}}
{=}
{\sum_{a\in\U}A'(a)}

We write \emph{$\phi(A',b)$\/} for the multiset $A'\cdot\Indicator_{\phi(\U,b)}$.

Multisets over $\V$ are defined analogously.

%seiIf $A'$ and $A''$ are multisets we write $A'\le A''$ if $A'(a)\le A''(a)$ holds for every $a$.
A \emph{fractional transversal\/} is a multiset $A'\le\Indicator_A$ such that $|\phi(A', b)|\ge1$ for every $b\in B$.
The \emph{fractional transversal number\/} of $\phi(x\,;z)$, denoted by $\tau^*$, is the  infimum of the size of the fractional transversals of $\phi(x\,;z)$.

A fractional multiset $B'\le\Indicator_B$ over $\V$ is a \emph{fractional packing\/} if $|\phi(a,B')|\le1$ for every $a\in A$.
The \emph{fractional packing number\/} of $\phi(x\,;z)$, denoted by $\nu^*$, is the supremum of the size of the fractional packings of $\phi(x\,;z)$.

\begin{example}
  The sets $\U,\V$ and the relation $\phi(A\,;B)$ are as in the example~\ref{expl_tr_pk}.
  Let $B'$ be a multiset that assigns $1/2$ to every line in $B$. Then $|\phi(a,B')|\le1$ holds because each point is contained in at most two lines.
  Then $\nu^*\ge |B'|=n/2$.
  It is easy to see that $\tau^*\ge n/2$.
  We claim that $\tau\le n/2$.
  If $n$ is even, use the same transversal $A'$ as in Example~\ref{expl_tr_pk} is even.
  If $n$ is odd, take $3$ any lines, and assign $1/2$ to the three intersection points.
  Proceed as in the even case with the other lines.\QED
\end{example}

\begin{exercise}
Let $\U=\RR$ and $\V$ is a set of finitely many closed intervals.
Let $\phi(x\,;z)$ be the membership relation.
Then $\nu=\tau$.
Hint: use induction on $\nu$.\QED
\end{exercise}

\begin{theorem}\label{thm_fractional_nu=tau}
 For all $\phi(x\,;z)$ and all finite sets $A\subseteq\U$ and $B\subseteq\V$, we have $\nu^*=\tau^*$ and this value is rational.
\end{theorem}
\begin{proof}
Let $A=\{a_1,\dots,a_m\}$ and $B=\{b_1,\dots,b_n\}$.
Let $F$ be the $m\times n$-matrix with entries $\Indicator_{\phi(a_i,b_j)}$.
A multi-set over $A$ is a naturally associated to a vector $0\le x\in\RR^m$.
A multi-set over $B$ is associated to a vector $0\le y\in\RR^n$.
Then it is easy to verify that

\ceq{\hfill\tau^*}
{=}
{\inf\,\big\{\,1_m^{\rm T}\;x\ :\ F^{\rm T}\,x\ge 1_n,\ 0\le x\};}

\ceq{\hfill\nu^*}
{=}
{\sup\big\{\,1_n^{\rm T}\ y\,\  :\ \ F\,y\le 1_m,\,\  0\le y\}.}

Therefore, by he duality of linear programming $\nu^*=\tau^*$.

As $\tau^*$ is the minimum of the linear function $x\mapsto 1_m^{\rm T}\;x$ over a polyhedron, such minimum is attained at vertex.
The inequalities describing the polyhedron have rational coefficients, so also the vertices have rational coordinates (elaborate on this).
\end{proof}

When $\phi(x\,;z)$ has finite \vc-dimension, the transversal number $\tau$ is bounded by a function of $\tau^*$.

\begin{proposition}\label{prop_bound_fractional_trans}
  Let $\phi(x\,;z)$ have \vc-dimension $k$.
  Then for all finite sets $A\subseteq\U$ and $B\subseteq\V$

  \ceq{\hfill\tau}{\le}{ c\,k\,(\tau^*)^2\,\ln (k\,\tau^*)}

  where $c$ is an absolute constant.
\end{proposition}

\begin{proof}
  Let $A'$ be an optimal fractional transversal.
  After normalizing, $A'$ defines a probability measure $\Pr$ on $\U$.
  Namely, $\Pr(\{a\})=A'(a)/\tau^*$ for $a\in\U$.
  By the definition of fractional transversal, every set $\phi(A\,;b)_{b\in B}$ has measure at last $1/\tau^*$.
  By Proposition~\ref{prop_vc_sample}, for every $\epsilon>0$ there is a sample $s$ of size

  \ceq{\hfill n}
  {\le}
  {c\,\frac{k}{\epsilon^2}\log\frac{k}{\epsilon}}

  If we set $\epsilon=1/\tau^*$, then $\range(s)$ is a transversal and we obtain the required bound.
\end{proof}

The bound in the proposition above can be improved.
One can replace $(\tau^*)^2$ with  $\tau^*$ at the cost of a more difficult proof.

\section{Helly-type properties}

We now investigate methods of bounding $\tau^*=\nu^*$.
As motivation we cite a classical theorem of Helly.

\begin{proposition}[(Helly Theorem)]
Let $\Phi$ be a finite family of convex sets in $\RR^d$.
Assume that any $d+1$ sets from $\Phi$ have non-empty intersection.
Then the whole family $\Phi$ has non-empty intersection.\QED
\end{proposition}

Note that Helly's theorem does not hold for families of finite \vc-dimension.
A counter example of \vc-dimension $2$ can be constructed with a family containing sets that are unions of two finite intervals of the real line.

We will deal with the following property, which is more robust. It says that if there is \textit{plenty\/} of \textit{small\/} collections of sets with nonempty intersection, then there is a \textit{large\/} collection with nonempty intersection.

\begin{definition}
We say that $\phi(x\,;z)$ \emph{has fractional Helly number $k$\/} if for all $\alpha>0$ there is a $\beta>0$ such that  for every finite $A\subseteq\U$ and $B\subseteq\V$ the following holds (write $n$ for $|B|$):

\ceq{(1)\hfill \bigcap_{b\in B'}\phi(A,b)}{\neq}{\0}\hfill for at least \ $\displaystyle\alpha{n\choose k}$ \ sets \ $\displaystyle B'\in{B\choose k}$

then

\ceq{(2)\hfill\bigcap_{b\in B''}\phi(A,b)}{\neq}{\0}\hfill for some $B''\subseteq B$ of cardinality $\ge\beta\,n$.

We say that $\phi$ has the \emph{fractional Helly property\/} if it has fractional Helly number $k$ for some finite $k$.
The \emph{fractional Helly number\/} of $\phi(x\,;z)$ is the smallest number $k$ satisfying the property above.\QED
\end{definition}

For further reference, we note that (2) in the definition above can be rewritten as

\ceq{(2$'$)\hfill\big|\phi(a,B)\big|}{\geq}{\beta\,n}\hfill for some $a\in A$.

The following theorem proves that \nip{} formulas have the fractionally Helly property (in a strong sense).

\begin{theorem}[(Matou\v{s}ek)]\label{thm_matousek}
Let  $\phi(x\,;z)$ have \vc-codimension $<k$.
Then $\phi(x\,;z)$ has fractional Helly number $k$.
Moreover, $\beta$ in the definition above only depends on $\alpha$ and $k$.
\end{theorem}

\begin{proof}
Let $\alpha$ be arbitrary and set $\beta=1/2m$ where $m$ is such that

\ceq{\hfill \bigsum^{k-1}_{i=0} \binom{m}{i}}{<}{\frac\alpha4{m\choose k}.}

Note that, by the Sauer-Shelah Lemma~\ref{lem_sauer}, the r.h.s.\@ is strictly larger than $\pi^*_\phi(m)$ for all $\phi(x\,;z)$ with \vc-codimension $<k$.

Assume for a contradiction that some finite $A\subseteq\U$ and $B\subseteq\V$ contradicts the definition above.
That is,

\ceq{(1)\hfill \bigcap_{b\in B'}\phi(A,b)}{\neq}{\0}\hfill for at least \ $\displaystyle\alpha{n\choose k}$ \ sets \ $\displaystyle B'\in{B\choose k}$

and

\ceq{(2)\hfill\big|\phi(a\,;B)\big|}{<}{\beta\,n}\hfill for all $a\in A$.

Note that we can assume that $n>2m$ otherwise $\beta\,n<1$ and (2) never occur.
We will find a set $B''\subseteq B$ of cardinality $m$ with more than $\pi^*(m)$ distinct $\phi(x\,;z)^{\rm op}$-definable subsets, a contradiction.

Let $P$ be the set of pairs $B'\subseteq B''\subseteq B$ such that $|B'|=k$ and $|B''|=m$.
We say that a pair $B'\subseteq B''$ in $P$ is \textit{good\/} if there is $a\in A$ such that $B'=\phi(a\,;B'')$.
That is, $B'$ is a $\phi(x\,;z)^{\rm op}$-definable subset of $B''$.

Claim 1.
Assume the uniform probability on $P$.
Then the probability that a random pair is good is $\ge\alpha/4$.

Assume Claim~1 for now and continue with the proof.
We can think that the random pair in $P$ is chosen by first picking $B''\in{B\choose m}$ with the uniform distribution and than $B'\in{B''\choose k\phantom{'}}$ again with the uniform distribution.
(To put it more pedantically, we are applying the theorem of total probability.)
If the the probability that a pair is good is $\ge\alpha/4$, then for at least one $B''\in{B\choose m}$ the probability of finding a good subset $B'$ is $\ge\alpha/4$.
Therefore, $B''$ has $\ge\frac\alpha4{m\choose k}>\pi^*(m)$ good subsets.
A contradiction which proves the theorem given the claim.

We now prove the claim.
There is another equivalent way to pick a random pair in $P$.
First we choose at random $B'\subseteq B$ of cardinality $k$ then obtain $B''$ by adding $m-k$ random elements from $B\sm B'$.
By (1), the probability that $B'$ is such that $\bigcap_{b\in B'}\phi(A,b)\neq\0$ is at least $\alpha$.
So, assume that $B'$ is such, and fix any $a$ is this intersection.
By (2), there are $|\phi(a,B)|<\beta n$.
Then the probability that all $b\in B''\sm B'$ are such that $\neg\phi(a,b)$ is at least\smallskip

\ceq{\hfill{n-\beta n\choose m-k}\bigg/{n-k\choose m-k}}
{=}
{\prod^{m-k-1}_{i=0}\frac{n-\beta n-i}{n-k-i}}
\hfill we write $\beta n$ for $\lfloor \beta n \rfloor$

\ceq{}
{\ge}
{\prod^{m-k-1}_{i=0}\frac{n-\beta n-m}{n-m}}

\ceq{}
{=}
{\bigg(\frac{n-\beta n-m}{n-m}\bigg)^m}

\ceq{}
{=}
{\bigg(1-\frac{\beta n}{n-m}\bigg)^m}

\ceq{}
{\ge}
{\displaystyle(1-2\beta)^m\vphantom{\bigg)}}
\hfill because $n>2m$

\ceq{}
{\ge}
{\bigg(1-\frac{1}{m}\bigg)^m}
\hfill because $\beta\le1/2m.$

As we can assume $m\ge 2$, the probability that a random pair in $P$ is good is at least $\alpha/4$.
This proves Claim~1 and with it the theorem.
\end{proof}

%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{The (p,q)-theorem}

For integers $p\ge q$ we say that $\phi(x\,;z)$ has the \emph{$(p,q)$-property\/} if for every finite $A\subseteq\U$ and every $B\subseteq\V$ of cardinality $p$ there is some $B'\subseteq B$ of cardinality $q$ such that

\ceq{\hfill \bigcap_{b\in B'}\phi(A\,;b)}
{\neq}
{\0}

For ease of speaking we call $\phi(A\,;b)_{b\in B}$ a $p$-collection of definable subsets of $A$.
Note that, strictly speaking, these is not collection of sets but collection of parameters defining sets (though not intrinsically relevant, it is convenient not to assume extensionality).

Hence, we can rephrase the $(p,q)$-property in plain English: out of any $p$-collection of definable sets there are at least $q$ sets with nonempty intersection.

Helly's theorem says that any finite collection of convex sets in $\RR^d$ satisfying the $(d + 1, d + 1)$-property has non-empty intersection, i.e.\@ admits a transversal of size $1$.
A generalization of this was conjectured by Hadwiger and Debrunner, and many
years later proved by Alon and Kleitman~\cite{AK1},~\cite{AK2}.
Subsequently, after proving Theorem~\ref{thm_matousek}, Matou\v{s}ek~\cite{matousek} noted that the method in~\cite{AK2} applies also to collections of sets with finite \vc-dimension.
More precisely, Matou\v{s}ek proved the existence of a bound to the cardinality of the transversal number $\tau$ of {\nip} formulas with the $(p,q)$-property, and that this bound depends only on $p,q$ and the \vc-codimension of the formula.
(Formally, the results~\cite{AK2} and~\cite{matousek} do not imply each other.)

\begin{theorem}[(Alon, Kleitman + Matou\v{s}ek)]
  Let $p\ge q> k$ be natural numbers.
  There is a number $N=N(k,p,q)$ such that
  $\tau_{\phi(A\,;B)}\le N$ for all formulas $\phi(x\,;z)$ with the $(p,q)$-property and \vc-codimension $< k$, and every $A\subseteq\U$ and $B\subseteq\V$.
\end{theorem}

\begin{proof}
As we are not trying to optimize $N$, we may prove the theorem for $q=k+1$.
By Proposition~\ref{prop_bound_fractional_trans}, the transversal number is bounded by a function of $\tau^*$, so it is enough to bound $\tau^*$. By Theorem~\ref{thm_fractional_nu=tau}, we can equivalently bound the fractional packing number $\nu^*$ because it coincides with $\tau^*$.

Let $B'\subseteq B$ be an optimal fractional packing.  That is, $\nu^*=|B'|$ where $B'\le\Indicator_B$ is such that $|\phi(a,B')|\le1$ for every $a\in A$. As we rather work with regular sets than with fractional multisets, we apply a trick that allows to replace $B'$ with a regular set $C'$.

By Theorem~\ref{thm_fractional_nu=tau} we may assume that $B'$ is rational valued. Therefore $B'=(1/m)\,C$ where $m$ is a positive integer and $C$ is a integral valued multiset over $\V$.
Replace $\V$ with $\V\times[m]$.
Define $\phi\!_{_\times}\!(a\,;b,i)$ to be the relation that holds if and only if $\phi(a\,;b)$ holds.
Then we can replace the multiset $C$ with a regular set $C'\subseteq\V\times[m]$ such that $|\phi\!_{_\times}\!(a\,;C')|\le m$ for every $a\in A$.

If write $n$ for $|C'|$, then $\nu^*=n/m$.

\smallskip
Claim~1. $\phi\!_{_\times}\!(x\,;z,y)$ satisfies the $(qp,q)$-property.

Let $D\subseteq\V\times[m]$ have cardinality $qp$.
If the $qp$-collection $\phi\!_{_\times}\!(A\,;b,i)_{b,i\in D}$ contains $q$ copies of the same definable set $\phi(A\,;b)$, then we immediately have the required $q$-collection with nonempty intersection.
So, suppose not.
Then  $\phi\!_{_\times}\!(A\,;b,i)_{b,i\in D}$ contains $p$ distinct sets $\phi(A,b_1),\dots,\phi(A,b_p)$.
Then the $q$-collection with nonempty intersection is obtained from the $(p,q)$-property of $\phi(x\,;z)$.

\smallskip
Claim 2.
There is an $\alpha=\alpha(p,q)>0$ such that

\ceq{\hfill \bigcap_{b,i\in D}\phi\!_{_\times}\!(A\,;b,i)}{\neq}{\0}\hfill for at least \ $\displaystyle\alpha{n\choose q}$ \ sets \ $\displaystyle D\in{C'\choose q}$.

By Claim~1, every $qp$-collection of $\phi\!_{_\times}\!$-definable sets contains at least one $q$-collection with non-empty intersection.
Every $q$-collection is contained in ${n-q\choose qp-q}$ many $qp$-collections.
Therefore the number $q$-collections with non-empty intersection is at least

\ceq{\hfill{n\choose qp}\Big/{n-q\choose qp-q}}{=}{{n\choose q}\Big/{qp\choose q}.}

Therefore, the claim holds with $\displaystyle1/\alpha={qp\choose q}$.

Now we can resume the proof of the theorem (recall that our goal is to bound $\nu^*$ by a function of $p,q$, and $k$).
Let $\beta=\beta(\alpha,k)$ be as in Theorem~\ref{thm_matousek}.
As $\phi\!_{_\times}\!(x\,;z,y)$ has the same \vc-codimension as $\phi(x\,;z)$, by Claim~2 there is an $a\in A$ such that $\phi(a,C')$ has cardinality at least $\beta\,n$. So, from $\beta\,n\le |\phi(a,C')|\le m$ we obtain $\nu^*\le 1/\beta$.
\end{proof}

\end{document}


\section{Distributionally Robust Optimization}
\label{sec:DROCapitalBudgeting}

We consider another risk-averse decision making approach
using distributionally robust optimization (DRO). To this end, we begin with
a nominal stochastic optimization problem:

\begin{equation}
\max_{s\in S} \sum_{\sigma\in \Sigma} q^\sigma f(s,\xi^\sigma)
\end{equation}

In our context, the nominal model is a stochastic capital budgeting problem
in which we maximize expected NPV, and we could have
$f(s,\xi^\sigma)=NPV(s,\xi^\sigma)$. In this context ``$s\in S$'' simply
indicates the constraints that a prioritized solution must satisfy, wherein
we prioritize project selection subject to uncertainty in costs and the NPV
of each project as well as uncertainty in resource availability. The goal is
to prioritize so as to maximize expected NPV, assuming that nominal distribution,
specified by the probability mass function $q^\sigma, \sigma \in \Sigma$.

We suppose that $\xi$ is a discrete random variable with finite sample space $\Omega$,
so that $\xi^\omega, \omega \in \Omega$ enumerates all possible realizations.
We further suppose that we only have observations of
$\xi^\sigma, \sigma \in \Sigma \subset \Omega$,
i.e., possibly a strict subset, which may arise in a data-driven setting. In such a
data-driven setting we could have, for example, probability mass $q^\sigma = 1/|\Sigma|$
for all $\sigma \in \Sigma$.
A DRO variant of this nominal stochastic optimization model is then given by:

\begin{equation}
\max_{s\in S} \min_{p\in P} \sum_{\omega\in \Omega} p^\omega f(s,\xi^\omega)
\end{equation}

Here, we may view this DRO model as playing a ``game'' against nature. First, we select
$s\in S$, and then knowing $s$, nature selects a worst-case probability distribution,
$p\in P$, to minimize the expected NPV, which we seek to maximize. We will make precise
what we mean by the Distributional Uncertainty Set (DUS), denoted by $P$, below, but
for the moment it is enough to think of the set as representing a neighborhood of
probability distributions centered on the given probability mass function, $q$,
with the radius of the neighborhood specified by parameter $\varepsilon$. If
$\varepsilon = 0$ then the DRO model reduces to the nominal model. If $\varepsilon$ is very
large then nature will select the single worst-case scenario, e.g., the scenario
with lowest budgets, highest costs, and lowest NPVs. This is too conservative to be useful
(i.e. if we are living in this world, it is very likely that the plant would be
uneconomical no matter what decisions are made). However, with moderate values of
$\varepsilon$ we obtain solutions that hedge against deviations from $q$ without being
excessively conservative.

Importantly, we do not view nature as malevolent, despite occasional evidence to the contrary.
Rather, we use ``$min_{p\in P}$'' to combat over-adapting our solution to a specific assumption
about the probability distribution. In this sense, DRO plays the role of a ``regularizer'' to
combat over-fitting that is analogous to regularizers used in high-dimensional statistics
and statistical machine learning. Our approach to DRO requires further mathematical and
intuitive development before we can analyze solutions to the DRO problem.

\subsection{Defining a Distributional Uncertainty Set via the Wasserstein Distance}
\label{subsec:WassersteinDistance}
By the constraints denoted by $p\in P$ we require that nature select a probability
mass function, $p$, that is ``close'' to the nominal or empirical data-driven distribution
$q$. We define the DUS as follows:

\begin{equation}
P=\{p: D(p,q)\le \varepsilon, \sum_{\omega\in \Omega} p^\omega = 1, p^\omega \ge 0, \omega \in \Omega \}
\end{equation}

where $D(p, q)$ is the distance between nature’s choice, $p$, and the nominal data-driven distribution,
$q$. As indicated above, the radius parameter $\varepsilon$ governs the latitude we give nature,
which in turn governs the degree of conservatism that we face when selecting decision $s\in S$.

There are multiple ways to measure the ``distance'', $D(p, q)$, between two probability distributions,
which include the Kolmogorov-Smirnov distance, Kullback-Leibler divergence, chi-squared distances,
total variation, and more general $\psi$-divergences. The Wasserstein distance, which is based on
the idea of optimal transport, is a particularly useful way to measure such a distance in the
context of distributionally robust optimization. For a distribution with known finite support,
the Wasserstein distance, $D(p, q)$, between a given distribution, $q^\sigma, \sigma \in \Sigma$,
and a given candidate robust distribution, $p^\omega, \omega \in \Omega$, is provided by the
optimal value of the transportation problem:

\begin{subequations}\label{WassersteinEq}
\begin{eqnarray}
& & D(q, p) = \min_{z} \sum_{\sigma \in \Sigma, \omega \in \Omega} d_{\sigma, \omega} z_{\sigma, \omega} \\
& & s.t. \sum_{\omega \in \Omega} z_{\sigma, \omega} = q^\sigma, \sigma \in \Sigma \\
& & \sum_{\sigma \in \Sigma} z_{\sigma, \omega} = p^\omega, \omega \in \Omega \\
& & z_{\sigma, \omega} \ge 0, \sigma \in \Sigma, \omega \in \Omega
\end{eqnarray}
\end{subequations}

The intuition behind this measure concerns the magnitude of probability mass, $q^\sigma$,
that must be transported distance $d_{\sigma, \omega}$ from vector $\xi^\sigma$ to vector $\xi^\omega$
via variable $z_{\sigma, \omega}$. In one extreme case, if the two sample spaces and probability
mass functions coincide, i.e., $\Omega = \Sigma$ and $p^\omega = q^\omega$, and $d_{\omega, \omega} = 0$
for all $\omega \in \Omega$ then $D(p, q) = 0$.

To fully specify $D(p, q)$ we must define $d_{\sigma, \omega}= dist(\xi^\sigma, \xi^\omega)$.
To do so we can select $dist(\cdot, \cdot)$, for example, to be the two-norm distance, or a more
general $\eta$-norm distance, between the vectors, $\xi^\sigma$ and $\xi^\omega$,
i.e., $dist(\cdot, \cdot) = ||\xi^\sigma - \xi^\omega||_\eta$.

With the Wasserstein distance, if we are given distribution, $q$, we can then define:

\begin{equation}
P=\{p: D(p,q)\le \varepsilon, \sum_{\omega\in \Omega} p^\omega = 1, p^\omega \ge 0, \omega \in \Omega \}
\end{equation}

for a given radius $\varepsilon$. Here, we think of $P$ as a ball, or neighborhood, of
probability distributions centered on $q$, where the neighborhood has radius $\varepsilon$.
With $\Sigma \subset \Omega$, if $\varepsilon = 0$ then $P$ is the singleton ${q}$, and
larger values of $\varepsilon$ lead to increasingly large neighborhoods. In the context
of robust optimization, if $\varepsilon = 0$ then we will simply be solving the nominal
stochastic optimization model, and as $\varepsilon$ grows large we will consider
increasingly conservative models.

We can now represent the set $P$ via the following so-called extended-variable set of constraints:

\begin{subequations}\label{WassersteinConstraints}
\begin{eqnarray}
& & \sum_{\sigma \in \Sigma, \omega \in \Omega} d_{\sigma, \omega} z_{\sigma, \omega} \le \varepsilon \\
& & \sum_{\omega \in \Omega} z_{\sigma, \omega} = q^\sigma, \sigma \in \Sigma \\
& & \sum_{\sigma \in \Sigma} z_{\sigma, \omega} = p^\omega, \omega \in \Omega \\
& & z_{\sigma, \omega} \ge 0, \sigma \in \Sigma, \omega \in \Omega
\end{eqnarray}
\end{subequations}

\subsection{Towards a Computationally Tractable Reformulation}
\label{subsec:TractableReformulation}
Due to the max min construct in the DRO model, the model is not amenable to direct solution
via optimization software. So, we reformulate the model to facilitate computation.
For the moment let $s\in S$ be fixed so that $f(s, \xi^\omega)$ is just a known numerical
value for each $\omega \in \Omega$. Then, nature’s problem may be written:

\begin{subequations}
\begin{eqnarray}
& & \min_{p, z} \sum_{\omega \in \Omega} p^\omega f(s, \xi^\omega) \\
& & s.t. \sum_{\sigma \in \Sigma, \omega \in \Omega} d_{\sigma, \omega} z_{\sigma, \omega} \le \varepsilon : [-\gamma] \\
& & \sum_{\omega \in \Omega} z_{\sigma, \omega} = q^\sigma, \sigma \in \Sigma : [\nu^\sigma]\\
& & \sum_{\sigma \in \Sigma} z_{\sigma, \omega} = p^\omega, \omega \in \Omega : [\beta^\omega]\\
& & z_{\sigma, \omega} \ge 0, \sigma \in \Sigma, \omega \in \Omega
\end{eqnarray}
\end{subequations}

Here, $\gamma, \nu^\sigma$, and $\beta^\omega$ denote dual variables.
In this model, nature optimizes over $z$ and over $p$ to select a worst-case distribution
within radius $\varepsilon$ of $q$.

Taking the dual of the linear DRO program, and substituting out the dual variable
$\beta^\omega= -f(s, \xi^\omega)$, we obtain the following:

\begin{subequations}\label{dualProlem}
\begin{eqnarray}
& & \max_{\gamma, \nu} -\gamma \varepsilon + \sum_{\sigma\in \Sigma} \nu^\sigma q^\sigma \\
& & s.t. -\gamma d_{\sigma, \omega} + \nu^\sigma \le f(s, \xi^\omega), \sigma \in \Sigma, \omega \in \Omega \\
& & \gamma \ge 0
\end{eqnarray}
\end{subequations}

For a better understanding of the model, consider two extreme cases, $\varepsilon = 0$ and $\varepsilon = \infty$.
If $\varepsilon = 0$ then there is no penalty in the objective function for allowing $\gamma$
to grow large. As $gamma$ grows large, the constraint becomes vacuous for all $\sigma \ne \omega$;
however, for $\sigma = \omega$ we have $d_{\sigma, \sigma} = ||\xi^\sigma - \xi^\omega|| = 0$,
and hence the constraint reduces to $\nu^\sigma \le f(s, \xi^\omega)$, and coupled with the objective
function the optimal value reduces to $\sum_\sigma q^\sigma f(s, \xi^\sigma) = \sum_\omega p^\omega f(s, \xi^\omega)$,
i.e., it reduces to the objective function value of the nominal stochastic optimization model,
as it must with $\varepsilon = 0$.

In the other extreme, as $\varepsilon$ grows sufficiently large we must have $\gamma = 0$
to avoid a huge penalty in the objective function. Thus, for each $\sigma \in \Sigma$,
the constraint reduces to $\nu^\sigma \le f(s, \xi^\omega)$, i.e.,
the objective function reduces to:

\begin{equation}
\sum_\sigma \min_\omega f(s, \xi^\omega) q^\sigma = \min_\omega f(s, \xi^\omega) \sum_\sigma q^\sigma = \min_\omega f(s, \xi^\omega)
\end{equation}

Again, this matches what it must: if $\varepsilon = \infty$ then nature has enough latitude
to place a probability of one on the single worst-case scenario.

Specializing $s\in S$ to be the constraints for prioritization, and specializing
$f(s,\xi^\omega)$ to define the NPV under scenario $\omega\in \Omega$, the DRO variant of the
stochastic capital budgeting problem is as follows:

\begin{subequations}\label{fullDRO}
\begin{eqnarray}
& & \max_{x, y, \gamma, \nu} -\gamma \varepsilon + \sum_{\sigma\in \Sigma} \nu^\sigma q^\sigma \\
& & s.t. -\gamma d_{\sigma, \omega} + \nu^\sigma \le \sum _{i \in I}^{} \sum _{j \in J_{i}}^{}a_{ij}^{ \omega }x_{ij}^{ \omega }, \sigma \in \Sigma, \omega \in \Omega \\
& & y_{ii^{'}}+y_{i^{'}i} \geq 1,~ i<i^{'}\text{, i, }i^{'} \in I \\
& & \sum_{j=1}^{J_i} x_{ij}^\omega \geq \sum_{j=1}^{J_i} x_{i'j}^\omega + y_{ii'} -1,~ i \neq i^{'}\text{, i, }i^{'} \in I,  \omega  \in  \Omega \\
& & \sum _{i \in I}^{} \sum _{j \in J_{i}}^{}\text{~ c}_{ijkt}^{ \omega }x_{ij}^{ \omega }~  \leq  b_{kt}^{ \omega },~ k \in K, t \in T,  \omega  \in  \Omega \\
& & \sum_{j\in J_i} x_{ij}^{ \omega } \leq 1,~ i \in I, \omega  \in  \Omega \\
& & \gamma \ge 0
\end{eqnarray}
\end{subequations}

\subsection{LOGOS Settings for DRO Problems}
\label{subsec:DROSettings}
DRO approach is an extension for stochastic optimization approach discussed in
Section~\ref{sec:StochasticCapitalBudgeting}. Both of them share the same input
structures except the \xmlNode{Settings} block. In both cases, the user need to
specify a collection of scenarios via \xmlNode{Uncertainties} block. The
\xmlNode{problem\_type} within \xmlNode{Settings} block is used to select the
type of DRO problems. The currently available DRO problem types are:
\xmlString{droskp}, \xmlString{dromkp}, and \xmlString{dromckp}. The user can
use \xmlNode{radius\_ambiguity} to control the Wasserstein distance for DRO problems.

Example LOGOS input XML for DRO:
\begin{lstlisting}[style=XML]
<Settings>
<Logos>
  <solver>cbc</solver>
  <solverOptions>
    <StochSolver>EF</StochSolver>
    <!-- epsilon radius -->
    <radius_ambiguity>0.1</radius_ambiguity>
  </solverOptions>
  <sense>maximize</sense>
  <problem_type>droskp</problem_type>
</Settings>
</Logos>
\end{lstlisting}

\subsection{DRO for Single Knapsack Problem}
\label{subsec:DRO_SKP}

\vst \noi {\em Model Formulation:}
\begin{subequations}\label{DROSimpleKP}
\begin{eqnarray}
& & \max_{x, y} \max_{\gamma, \nu} (-\gamma \varepsilon + \sum_{\sigma \in \Sigma} \nu^\sigma q^\sigma)  \\
& &  -\gamma d_{\sigma, \omega} + \nu^\sigma \le \sum _{i \in I} a_{i}^{ \omega }x_{i}^{ \omega }, \sigma \in \Sigma, \omega \in \Omega \\
& & \sum_{i \in I} c_{i}^\omega x_{i}^\omega \leq b^\omega \\
& & y_{ii'} + y_{i'i} \geq 1, i<i'  \\
& & x_{i}^\omega \geq x_{i'}^\omega + y_{ii'}-1, i\neq i' \\
& & x_{i}^\omega, y_{ii'}^\omega \in {0, 1} \\
& & \gamma \ge 0
\end{eqnarray}
\end{subequations}

See next section~\ref{subsec:DRO_DKP} for the example of LOGOS input file, since multi-dimensional Knapsack problem
is just a simple extension of a single-dimensional Knapsack problem, and both of them belong to the same
\xmlNode{problem\_type}: \xmlString{droskp}.

\subsection{DRO for Multi-Dimensional Knapsack Problem}
\label{subsec:DRO_DKP}

\vst \noi {\em Model Formulation:}
\begin{subequations}\label{DROMultiDKP}
\begin{eqnarray}
& & \max_{x, y} \max_{\gamma, \nu} (-\gamma \varepsilon + \sum_{\sigma \in \Sigma} \nu^\sigma q^\sigma)  \\
& &  -\gamma d_{\sigma, \omega} + \nu^\sigma \le \sum _{i \in I} a_{i}^{ \omega }x_{i}^{ \omega }, \sigma \in \Sigma, \omega \in \Omega \\
& & \sum_{i \in I} c_{it}^\omega x_{i}^\omega \leq b_{t}^\omega, t\in T \\
& & y_{ii'} + y_{i'i} \geq 1, i<i'  \\
& & x_{i}^\omega \geq x_{i'}^\omega + y_{ii'}-1, i\neq i' \\
& & x_{i}^\omega, y_{ii'}^\omega \in {0, 1} \\
& & \gamma \ge 0
\end{eqnarray}
\end{subequations}

Example LOGOS input XML:
\begin{lstlisting}[style=XML]
<Logos>
  <Sets>
    <investments>
      1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16
    </investments>
    <time_periods>
      1,2,3,4,5
    </time_periods>
  </Sets>

  <Parameters>
    <net_present_values index="investments">
      2.315,0.824,22.459,60.589,0.667,5.173,4.003,0.582,0.122,
      -2.870,-0.102,-0.278,-0.322,-3.996,-0.246,-20.155
    </net_present_values>
    <costs index="investments, time_periods">
      0.219,0.257,0.085,0.0,0.0,
      0.0,0.0,0.122,0.103,0.013,
      5.044,1.839,0.0,0.0,0.0,
      6.74,6.134,10.442,0.0,0.0,
      0.425,0.0,0.0,0.0,0.0,
      2.125,2.122,0.0,0.0,0.0,
      2.387,0.19,0.012,2.383,0.192,
      0.0,0.95,0.0,0.0,0.0,
      0.03,0.03,0.688,0.0,0.0,
      0,0.2,0.763,0.739,2.539,
      0.081,0.032,0,0,0,
      0.3,0,0,0,0,
      0.347,0,0,0,0,
      4.025,0.297,0,0,0,
      0.095,0.095,0.095,0,0,
      5.487,5.664,0.5,6.803,6.778
    </costs>
    <available_capitals index="time_periods">
      18,18,18,18,18
    </available_capitals>
  </Parameters>

  <Uncertainties>
    <available_capitals>
      <totalScenarios>10</totalScenarios>
      <probabilities>
        0.012, 0.019, 0.032, 0.052, 0.086, 0.142, 0.235, 0.188, 0.141, 0.093
      </probabilities>
      <!--
        scenarios is ordered by numberScenarios * parametersIndex,
        the number of scenarios is determined by
        the number of elements in <probabilities>, for this case:
        numberScenarios * time_periods = 10 * 5
      -->
      <scenarios>
        11, 11, 11, 11, 11,
        12, 12, 12, 12, 12,
        13, 13, 13, 13, 13,
        14, 14, 14, 14, 14,
        15, 15, 15, 15, 15,
        16, 16, 16, 16, 16,
        17, 17, 17, 17, 17,
        18, 18, 18, 18, 18,
        19, 19, 19, 19, 19,
        20, 20, 20, 20, 20
      </scenarios>
    </available_capitals>
  </Uncertainties>

  <Settings>
    <mandatory>10,11,12,13,14,15,16</mandatory>
    <solver>cbc</solver>
    <solverOptions>
      <StochSolver>EF</StochSolver>
      <!-- epsilon radius -->
      <radius_ambiguity>0.1</radius_ambiguity>
    </solverOptions>
    <sense>maximize</sense>
    <problem_type>droskp</problem_type>
  </Settings>
</Logos>
\end{lstlisting}

\subsection{DRO for Multiple Knapsack Problem}
\label{subsec:DRO_MKP}

\vst \noi {\em Model Formulation:}
\begin{subequations}\label{DROMKP}
\begin{eqnarray}
& & \max_{x, y} \max_{\gamma, \nu} (-\gamma \varepsilon + \sum_{\sigma \in \Sigma} \nu^\sigma q^\sigma)  \\
& &  -\gamma d_{\sigma, \omega} + \nu^\sigma \le \sum _{i \in I} \sum _{m \in M}a_{i}^{ \omega }x_{im}^{ \omega }, \sigma \in \Sigma, \omega \in \Omega \\
& & \sum _{i \in I}^{} c_{i}^{ \omega }x_{im}^{ \omega }~  \leq  b_{m}^{ \omega },~ m \in M,  \omega  \in  \Omega \\
& & y_{ii'} + y_{i'i} \geq 1, i<i'  \\
& & \sum_{m=1}^{M} x_{im}^\omega \geq \sum_{m=1}^{M} x_{i'm}^\omega + y_{ii'} -1,~ i \neq i^{'}\text{, i, }i^{'} \in I,  \omega  \in  \Omega \\
& & x_{i,m}^\omega, y_{ii'}^\omega \in {0, 1} \\
& & \gamma \ge 0
\end{eqnarray}
\end{subequations}

Example LOGOS input XML:
\begin{lstlisting}[style=XML]
<Logos>
  <Sets>
    <investments>
      1,2,3,4,5,6,7,8,9,10
    </investments>
    <capitals>
      unit_1, unit_2
    </capitals>
  </Sets>

  <Parameters>
    <net_present_values index="investments">
      78, 35, 89, 36, 94, 75, 74, 79, 80, 16
    </net_present_values>
    <costs index="investments">
      18, 9, 23, 20, 59, 61, 70, 75, 76, 30
    </costs>
    <available_capitals index="capitals">
      103, 156
    </available_capitals>
  </Parameters>

  <Uncertainties>
    <available_capitals>
      <totalScenarios>10</totalScenarios>
      <probabilities>
        0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
      </probabilities>
      <scenarios>
        101, 154,
        102, 155,
        103, 156,
        104, 157,
        105, 158,
        106, 159,
        107, 160,
        108, 161,
        109, 162,
        110, 163
      </scenarios>
    </available_capitals>
  </Uncertainties>

  <Settings>
    <solver>cbc</solver>
    <solverOptions>
      <StochSolver>EF</StochSolver>
      <!-- epsilon radius -->
      <radius_ambiguity>0.1</radius_ambiguity>
    </solverOptions>
    <sense>maximize</sense>
    <problem_type>dromkp</problem_type>
  </Settings>
</Logos>
\end{lstlisting}

\subsection{DRO for Multiple-Choice Knapsack Problem}
\label{subsec:DRO_MCKP}

\vst \noi {\em Model Formulation:}
\begin{subequations}\label{DROMCKP}
\begin{eqnarray}
& & \max_{x, y} \max_{\gamma, \nu} (-\gamma \varepsilon + \sum_{\sigma \in \Sigma} \nu^\sigma q^\sigma)  \\
& &  -\gamma d_{\sigma, \omega} + \nu^\sigma \le \sum _{i \in I}^{} \sum _{j \in J_{i}}^{}a_{ij}^{ \omega }x_{ij}^{ \omega }, \sigma \in \Sigma, \omega \in \Omega \\
& & \sum _{i \in I}^{} \sum _{j \in J_{i}}^{}\text{~ c}_{ijkt}^{ \omega }x_{ij}^{ \omega }~  \leq  b_{kt}^{ \omega },~ k \in K, t \in T,  \omega  \in  \Omega \\
& & y_{ii'} + y_{i'i} \geq 1, i<i'  \\
& & \sum_{j=1}^{J_i} x_{ij}^\omega \geq \sum_{j=1}^{J_i} x_{i'j}^\omega + y_{ii'} -1,~ i \neq i^{'}\text{, i, }i^{'} \in I,  \omega  \in  \Omega \\
& & x_{i,j}^\omega, y_{ii'}^\omega \in {0, 1} \\
& & \gamma \ge 0
\end{eqnarray}
\end{subequations}

Example LOGOS input XML:
\begin{lstlisting}[style=XML]
<Logos>
  <Sets>
    <investments>
      1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17
    </investments>
    <options index='investments'>
      1;
      1;
      1;
      1,2,3;
      1,2,3,4;
      1,2,3,4,5,6,7;
      1;
      1;
      1;
      1;
      1;
      1;
      1;
      1;
      1;
      1;
      1
    </options>
  </Sets>

  <Parameters>
    <net_present_values index='options'>
      2.046
      2.679
      2.489
      2.61
      2.313
      1.02
      3.013
      2.55
      3.351
      3.423
      3.781
      2.525
      2.169
      2.267
      2.747
      4.309
      6.452
      2.849
      7.945
      2.538
      1.761
      3.002
      3.449
      2.865
      3.999
      2.283
      0.9
      8.608
    </net_present_values>
    <costs index='options'>
      36538462
      83849038
      4615385
      2788461538
      2692307692
      5480769231
      1634615385
      2981730768
      7211538462
      9038461538
      649038462
      650000000
      216346154
      212500000
      3076923077
      3942307692
      1144230769
      675721154
      1442307692
      99711538
      4807692
      123076923
      138461538
      86538462
      108653846
      75092404
      6413462
      147932692
    </costs>
    <available_capitals>
      15E9
    </available_capitals>
  </Parameters>

  <Uncertainties>
    <available_capitals>
      <totalScenarios>3</totalScenarios>
      <probabilities>
        0.2,0.6,0.2
      </probabilities>
      <scenarios>
        5E9,10E9,15E9
      </scenarios>
    </available_capitals>
  </Uncertainties>

  <Settings>
    <solver>glpk</solver>
    <solverOptions>
      <StochSolver>EF</StochSolver>
      <!-- epsilon radius -->
      <radius_ambiguity>0.1</radius_ambiguity>
    </solverOptions>
    <sense>maximize</sense>
    <problem_type>dromckp</problem_type>
  </Settings>
</Logos>
\end{lstlisting}


\section{Properties of the Dual Map and Double Dual}
\begin{lemma}
    Let $V,W$ be vector spaces over $F$ and $\alpha\in L(V,W)$.
    Let $\alpha^\ast\in L(W^\ast,V^\ast)$ be the dual map, then:\\1
    1. $\ker\alpha^\ast=(\operatorname{Im}\alpha)^\circ$, so $\alpha^\ast$ is injective iff $\alpha$ is surjective.\\
    2. $\operatorname{Im}\alpha^\ast\le(\ker\alpha)^\circ$ with equality if $V,W$ are finite dimensional, in which case it implies that $\alpha^\ast$ is surjective iff $\alpha$ is injective.
\end{lemma}
This is very important as it shows how we can understand $\alpha$ from $\alpha^\ast$, which is often simpler.
\begin{proof}
    1. Pick $\epsilon\in W^\ast$, then $\epsilon\in\ker\alpha^\ast$ iff $\alpha^\ast(\epsilon)=0$ iff $\epsilon\circ\alpha=0$ iff $\epsilon\in(\operatorname{Im}\alpha)^\circ$.\\
    2. We first show that $\operatorname{Im}\alpha^\ast\le(\ker\alpha)^\circ$.
    Indeed, for any $\epsilon\in\operatorname{Im}\alpha^\ast$, we have $\epsilon=\alpha^\ast(\phi)=\phi\circ\alpha$ for some $\phi\in W^\ast$.
    But then for any $u\in \ker\alpha$ we have $\epsilon(u)=\phi\circ\alpha(u)=\phi(0)=0$, which means $\epsilon\in(\ker\alpha)^\circ$.
    In finite dimension, pick bases $B,C$ of $V,W$ and we get
    \begin{align*}
        \dim\operatorname{Im}\alpha^\ast&=r(\alpha^\ast)=r([\alpha^\ast]_{C^\ast,B^\ast})=r([\alpha]_{B,C}^\top)\\
        &=r([\alpha]_{B,C})=r(\alpha)\\
        &=\dim V-\dim\ker\alpha\\
        &=\dim (\ker\alpha)^\circ
    \end{align*}
    So they have the same dimension, hence equal.
\end{proof}
We now turn to a very important concept known as double dual.
$V^\ast$ is a vector space too, so we can also construct its dual
$$V^{\ast\ast}=L(V^\ast,F)=(V^\ast)^\ast$$
Why is it important?
Well, not much in finite dimensions, but in infinite dimensonal spaces, it is very hard to find obvious relations between $V$ and $V^\ast$.
However, there is a canonical embedding of $V$ into $V^{\ast\ast}$.
Indeed, pick $v\in V$, consider $\hat{v}:V^\ast\to F$ via $\epsilon\mapsto\epsilon(v)$, which is a well-defined element of $V^{\ast\ast}$.
Quite ironically, our first theorem on this topic is about finite-dimensional spaces.
\begin{theorem}
    If $V$ is finite dimensional, then this operation $\hat{}:V\to V^{\ast\ast}$ we just described is an isomorphism of vector spaces.
\end{theorem}
So we can just identify $V^{\ast\ast}$ with $V$.
\begin{proof}
    Linearity is standard.
    To see it is injective, let $e\in V\setminus\{0\}$ and extend $\{e\}$ to a basis $\{e,e_2,\ldots,e_n\}$ of $V$.
    So the dual basis $(\epsilon,\epsilon_2,\ldots,\epsilon_n)$ would have $\hat{e}(\epsilon)=\epsilon(e)=1$.
    Therefore $\hat{}$ has trivial kernel, hence injective.
    It then follows that it is an isomorphism as $\dim V=\dim V^\ast=\dim V^{\ast\ast}$.
\end{proof}
\begin{remark}
    In further linear analysis and functional analysis, we will see that $\hat{}$ remains injective for a huge class of infinite dimensional vector spaces (those of interests are often space of functions).
    And there are many of them (called reflexive spaces) where $\hat{}$ is actually an isomorphism.
    The theories emerged from here have numerous applications in analysis.
\end{remark}
\begin{lemma}
    Let $V$ be a finite dimensional vector space over $F$ and $U\le V$.
    Define $\hat{U}=\{\hat{u}:u\in U\}\le V^{\ast\ast}$.
    Then $\hat{U}=U^{\circ\circ}=(U^\circ)^\circ$.
\end{lemma}
Thus we can identify $U^{\circ\circ}$ with $U$ too.
\begin{proof}
    Trivial.
\end{proof}
\begin{lemma}
    Let $V$ be finite dimensional vector space over $F$ and $U_1,U_2\le V$, then:\\
    1. $(U_1+U_2)^\circ=U_1^\circ\cap U_2^\circ$.\\
    2. $(U_1\cap U_2)^\circ=U_1^\circ+U_2^\circ$.
\end{lemma}
\begin{proof}
    Just write it out.
\end{proof}