% Chapter X

\chapter{Creating models} % Chapter title

\label{Creating models} % For referencing the chapter elsewhere, use

The previous section described the basic building blocks of Figaro models. Out of these building blocks, a wide variety of models can be created. This section describes how to build a range of models.

\section{Basic models}

One of the first things you can do with an element is to assign it to a Scala value:

\begin{flushleft}
\texttt{val burglary = Flip(0.01)}
\end{flushleft}

\marginpar{\texttt{val} in Figaro represents an immutable value. When a thing is assigned to a val, data inside the thing can change but the reference stored in the val is constant}

A \texttt{val} represents a field (in this case burglary) that takes on an immutable value (in this case the element \texttt{Flip(0.01)}). A field is not a variable; its value cannot be changed (Note that the scala assignment of the field \texttt{burglary} cannot change, but the value of the Figaro element that is assigned to it, \texttt{Flip(0.01)}, \emph{can} change). You can use the value of a field by referring to its name:

\begin{flushleft}
\texttt{val alarm = If(burglary, Flip(0.9), Flip(0.1))}
\end{flushleft}

Recall that an element defines a process that probabilistically produces a value. If an element is referred to multiple times, it must produce the same value everywhere it appears. Consider:

\begin{flushleft}
\texttt{val x = Flip(0.5)
\newline val y = x === x}
\end{flushleft}

Although we don't know the value, \texttt{x} must produce the same value on both sides of the equality test. Therefore, \texttt{y} produces the value \texttt{true} with probability 1. In contrast, in

\begin{flushleft}
\texttt{val y = Flip(0.5) === Flip(0.5)}
\end{flushleft}

the left and right hand sides are distinct elements (each call produces a new \texttt{Flip}), so they need not produce the same value. Therefore, \texttt{y} will produce \texttt{true} with probability 0.5.

With the tools we have defined so far, we can easily create a Bayesian network. In the following code, \texttt{CPD} is a library element (based on \texttt{Chain}) that makes it easy to define conditional probability distributions:

\begin{flushleft}
\marginpar{This example is found in Burglary.scala}
\texttt{import com.cra.figaro.language.\_
\newline import com.cra.figaro.library.compound.CPD
\newline
\newline val burglary = Flip(0.01)
\newline
\newline val earthquake = Flip(0.0001)
\marginpar{Scala statements can be written on multiple lines}
\newline
\newline val alarm = CPD(burglary, earthquake,
\newline \tab (false, false) -> Flip(0.001),
\newline \tab (false, true) -> Flip(0.1),
\newline \tab (true, false) -> Flip(0.9),
\newline \tab (true, true) -> Flip(0.99))
\newline
\newline val johnCalls = CPD(alarm,
\newline \tab false -> Flip(0.01),
\newline \tab true -> Flip(0.7))}
\end{flushleft}

With \texttt{CPD}, every single combination of values of the parents needs to be listed. \texttt{RichCPD} provides a more flexible format that allows for specification of structures such as context specific independence. Each clause in a \texttt{RichCPD} consists of a tuple of cases, one for each parent. A case can be \texttt{OneOf} a set of values, \texttt{NoneOf} a set of values (meaning that it matches all values except for the ones listed), or *, meaning that it accepts all values. For example:

\begin{flushleft}
\texttt{import com.cra.figaro.language.\_}
\newline \texttt{import com.cra.figaro.library.compound.RichCPD}
\newline \texttt{val x1 = Select(0.1 -> 1, 0.2 -> 2, 0.3 -> 3, 0.4 -> 4)}
\newline \texttt{val x2 = Flip(0.6)
\newline val x3 = Constant (5)
\newline val x4 = Flip(0.8)
\newline val y = RichCPD(x1, x2, x3, x4,
\newline \tab (OneOf(1, 2), *, OneOf(5), *) -> Flip(0.1),
\newline \tab (NoneOf(4), OneOf(false), *, *) -> Flip(0.7),
\newline \tab (*, *, NoneOf(6, 7), OneOf(true)) -> Flip(0.9),
\newline \tab (*, *, *, OneOf(false)) -> Constant (true))}
\end{flushleft}

A particular combination of values of the parents is matched against each row in turn, and the first match is chosen. For example, the combination (1, false, 5, true) matches the first three rows, so the first result (\texttt{Flip(0.1)}) is chosen. All possible values of the parent still need to be accounted for in the argument list using a combination of \texttt{OneOf}, \texttt{NoneOf} and *.

\section{Conditions and constraints}

So far, we have described models that generate the values of elements. It is also possible to influence the values of elements by imposing conditions or constraints on them.

A \emph{condition} represents something the value of the element must satisfy. Only values that satisfy the condition are possible. Every element has a condition, which is a function from a value of the element to a Boolean. If the element is of type \texttt{Element[T]}, the condition is of type \texttt{T => Boolean}. Conditions can have multiple purposes. One is to assert evidence, by specifying something that is known about an element. Alternatively, a condition can specify a structural property of a model, for example, that only one of two teams playing a game can be the winner.

The default condition of an element returns true for all values. The condition can be changed using \texttt{setCondition}:

\begin{flushleft}
\texttt{val x1 = Select(0.1 -> 1, 0.2 -> 2, 0.3 -> 3, 0.4 -> 4)
x1.setCondition((i: Int) => i == 1 || i == 4)}
\end{flushleft}

which says that \texttt{x1} must have value 1 or 4. We can add a condition on top of existing conditions using the \texttt{addCondition} method. For example, the following code says that not only must \texttt{x1} equal 1 or 4, it must also be odd:

\begin{flushleft}
\texttt{x1.addCondition((i: Int) => i \% 2 == 1)}
\end{flushleft}

The \texttt{observe} method provides an easy way to specify a condition that only allows a single value. For example, to specify that \texttt{x1} must have the value 2, we can use

\begin{flushleft}
\texttt{x1.observe(2)}
\end{flushleft}

Note that using \texttt{observe} will remove all previous conditions on an element.

A \emph{constraint} provides a way to specify a potential or weighting over an element. It is a function from a value of the element to a Double, so if the element has type \texttt{Element[T]}, the constraint is of type \texttt{T => Double}.

Constraints serve multiple purposes in Figaro. One is to specify soft evidence on an element. For example, if in the above Bayesian network we think we heard John call but we're not sure, we might introduce the constraint

\begin{flushleft}
\texttt{johnCalls.setConstraint((b: Boolean) =>
\newline \tab if (b) 1.0; else 0.1)}
\end{flushleft}

This line will have the effect of making John calling 10 times more likely than not, all else being equal. Another purpose of constraints is to define some probabilistic relationships conveniently that are more difficult to express without them. Consider the following example, in which we are modeling the process of firms bidding for a contract and one of them being selected as the winner.

\begin{flushleft}
\marginpar{This example is found in Firms.scala}
\texttt{import com.cra.figaro.language.\_}
\newline \texttt{import com.cra.figaro.library.atomic.\_}
\newline \texttt{import com.cra.figaro.library.compound.If}
\newline
\newline \texttt{class Firm \{
\newline \tab val efficient = Flip(0.3)
\newline \tab val bid = If(efficient, continuous.Uniform(5, 15),
\newline continuous.Uniform(10, 20))
\newline \}
\newline
\newline \texttt{val firms = Array.fill(20)(new Firm)
\newline val winner = discrete.Uniform(firms:\_*)
\newline val winningBid = Chain(winner, (f: Firm) => f.bid)
\newline winningBid.setConstraint((d: Double) => 20 $-$ d)}
}
\end{flushleft}

This example shows some new Scala features. First, we have a class definition (the \texttt{Firm} class). A class creates a type that can be instantiated to create instances. The \texttt{Firm} class has two fields, \texttt{efficient} and \texttt{bid}. Note that bid makes use of \texttt{continuous.Uniform}. This is the continuous uniform element defined in the \texttt{library.atomic.continuous} package, but we did not import the members of this package, only the members of the \texttt{library.atomic} package. The reason we did things this way is that later in the example, we use the discrete uniform, and we want to be explicit about which uniform element we mean at each point.

Once we have defined the \texttt{Firm} class, we create an array named \texttt{firms} consisting of 20 instances of \texttt{Firm.Array.fill(20)(new Firm)} creates an array filled with the result of 20 different invocations of \texttt{new Firm}, each of which creates a separate instance of  \texttt{Firm} (and separate Figaro elements in each class). We then define the winner to be one of the firms, chosen uniformly. Note the notation \texttt{firms:\_*.} The element \texttt{discrete.Uniform} takes as arguments an explicit sequence of values of variable length, for example, \texttt{discrete.Uniform(1, 2, 5)} or \texttt{discrete.Uniform("x")}. Since \texttt{firms} is a single field representing an array, we must convert it into a sequence of arguments, which is accomplished using the \texttt{:\_*} notation. The field \texttt{winner} represents an \texttt{Element[Firm]}; it is intended to mean the winning bidder, although so far we have done nothing to relate the winner to its bid.

The next line is interesting. It allows us to identify the bid of the winning bidder as an element with a name, even though we don't know who the winner is. We can do this because even though we don't know who the winner is, we can refer to the \texttt{winner} field, and because the value of \texttt{winner}, whatever it is, is a \texttt{Firm} that has a \texttt{bid} field, which is an element that can be referred to. It is important to realize that this \texttt{Chain} does not create a new element but rather refers to the element \texttt{f.bid} that was created previously.

Finally, we introduce the constraint, which says that a winning bid of d has weight \texttt{20 $-$ d}. This means that a winning bid of 5 is 15 times more likely than a winning bid of 19. The effect is to make the winning bid more likely to be low. Note that in this model, the winning bid is not necessarily the lowest bid. For various reasons, the lowest bidder might not win the contract, perhaps because they offer a poor quality service or they don't have the right connections. Using a constraint, the model is specified very simply using a discrete uniform selection and a simple constraint.

Constraints are also useful for expressing undirected models such as relational Markov networks or Markov logic networks. To illustrate, we will use a version of the friends and smokers example. This example involves a number of people and their smoking habits. People have some propensity to smoke, and people are likely to have the same smoking habit as their friends.

\begin{flushleft}
\marginpar{This example is found in Smokers.scala}
\texttt{import com.cra.figaro.language.Flip
\newline import com.cra.figaro.library.compound.\textasciicircum \textasciicircum
\newline
\newline class Person \{
\newline \tab val smokes = Flip(0.6)
\newline \}
\newline
\newline val alice, bob, clara = new Person
\newline val friends = List((alice, bob), (bob, clara))
\newline clara.smokes.observe(true)
\newline
\marginpar{Single line function definitions in Scala do not need bracketing}
\newline def smokingInfluence(pair: (Boolean, Boolean)) =
\newline \tab if (pair.\_1 == pair.\_2) 3.0; else 1.0
\newline
\newline for \{ (p1, p2) <- friends \} \{
\newline \tab \textasciicircum \textasciicircum(p1.smokes, p2.smokes).setConstraint(smokingInfluence)
\newline \} }
\end{flushleft}

First, we create a \texttt{Person} class with a \texttt{smokes} field. We create three different people and a network of friends, represented by a list of pairs of people. We also observe that one of the people smokes.

Now we create the constraint function \texttt{smokingInfluence}. This function takes a pair of Booleans, and returns 3.0 if they are the same, 1.0 if different. The intended meaning of this function is to compare the smoking habit of two friends, and say that having the same smoking habit is three times as likely as a different smoking habit, all else being equal.

Finally, we apply the constraint to all the pairs of friends. The code uses a Scala feature called a "for comprehension". The notation for \texttt{\{ (p1, p2) <- friends \}} { "do something" } iterates through all pairs of people in the \texttt{friends} list and executes "do something" for each pair. In this case, "do something" is "add the constraint on their smoking habits to the pair of friends". The notation \texttt{\textasciicircum \textasciicircum(p1.smokes, p2.smokes)} takes each pair of friends and creates the pair element consisting of their smoking habits. We then assign the \texttt{smokingInfluence} constraint to this pair.

\section{Classes, instances, and relationships}

The object-oriented nature of Scala makes Figaro ideal for representing probabilistic models involving objects and relationships such as probabilistic relational models (PRMs).
In the following example, we will see how to define general classes of object, and create instances of a class by using a subclass of the class specially designed for the instance.

In this example, we are given two possible sources and a sample that came from one of the sources, and want to determine which source the sample came from based on the strength of the match with each
source.

\begin{flushleft}
\marginpar{Abstract classes in Scala are similar as in Java; they cannot be instantiated}
\texttt{class Source(val name: String)
\newline
\newline abstract class Sample \{
\newline \tab val fromSource : Element[Source]
\newline \}
\newline
\newline class Pair(val source: Source, val sample: Sample) \{
\newline \tab val isTheRightSource = Apply(sample.fromSource, (s: Source) => s == source)
\newline \tab val distance = If(isTheRightSource,
\newline \tab Normal(0.0, 1.0),
\newline \tab Uniform(0.0, 10.0))
\newline \}
\newline
\marginpar{Defining class contents at instantiation time will override undefined values}
\newline val source1 = new Source("Source 1")
\newline val source2 = new Source("Source 2")
\newline val sample1 = new Sample \{
\newline val fromSource = Select(0.5 -> source1, 0.5 -> source2)
\newline \}
\newline val pair1 = new Pair(source1, sample1)
\newline val pair2 = new Pair(source2, sample1)
\newline
\newline pair1.distance.setCondition((d:Double) => (d > 0.15 \&\& d < 0.25))
\newline pair2.distance.setCondition((d:Double) => (d > 1.45 \&\& d < 1.55)) }
\end{flushleft}

We begin by creating classes representing sources and samples, where each sample comes from a source. Note that \texttt{Sample} is an abstract class, because in this class we do not say anything about what source the sample came from (the from \texttt{Source} field has not been assigned an \texttt{Element[Source]} yet). We then create the \texttt{Pair} class representing a pair of a source and a sample. \texttt{Pair} has two fields: \texttt{isTheRightSource}, which produces true if the sample is from the source in the pair, and \texttt{distance}, which measures the closeness of the match between the sample and the source (lower distance means better match). The distance will tend to be smaller if the sample is from the right source but will not always be so.

Now it's time to create some instances. Note that the \texttt{Source} class takes an argument which is the name of the source. When we create instances \texttt{source1} and \texttt{source2} of this class, we supply the name argument. Next, we create an instance of \texttt{Sample}. Since \texttt{Sample} is abstract, we need to supply a definition of \texttt{fromSource}. We can do that right in line here, specifying that \texttt{sample1} could come either from \texttt{source1} or \texttt{source2}, each with probability 0.5. Finally, we create pairs pairing both of the sources to \texttt{sample1} and create conditions about the distances. The conditions are ranges rather than exact observations because exact observations on continuous elements can be problematic for many types of inference algorithms.

Using similar techniques, we can create a PRM. The following example shows the classical actors and movies PRM. There are three classes: actors, movies, and appearances relating actors to movies. Whether an actor receives an award for an appearance depends on the fame of the actor and the quality of the movie. The Figaro code for this example is as follows:

\begin{flushleft}
\marginpar{This example can be found in SimpleMovie.scala}
\texttt{import com.cra.figaro.library.compound.CPD
\newline import com.cra.figaro.language.\_
\newline
\newline class Actor \{
\newline \tab val famous = Flip(0.1)
\newline \}
\newline
\newline class Movie \{
\newline \tab val quality = Select(0.3 -> 'low, 0.5 -> 'medium, 0.2 -> 'high)
\newline \}
\newline
\marginpar{The ' in front of a string creates a Scala symbol, which are treated like \texttt{String} constants}
\newline class Appearance(actor: Actor, movie: Movie) \{
\newline \tab def probAward(quality: Symbol, famous: Boolean) =
\newline \tab  \tab  (quality, famous) match \{
\newline \tab \tab  case ('low, false) => 0.001
\newline \tab \tab  case ('low, true) => 0.01
\newline \tab \tab  case ('medium, false) => 0.01
\newline \tab \tab  case ('medium, true) => 0.05
\newline \tab \tab  case ('high, false) => 0.05
\newline \tab \tab  case ('high, true) => 0.2
\newline \tab \tab \}
\newline \tab  val award = SwitchingFlip(Apply(movie.quality, actor.famous, (q: Symbol, f: Boolean) => probAward(q, f)))
\newline \}
\newline
\newline val actor1 = new Actor
\newline val actor2 = new Actor
\newline val actor3 = new Actor
\newline val movie1 = new Movie
\newline val movie2 = new Movie
\newline val appearance1 = new Appearance(actor1, movie1)
\newline val appearance2 = new Appearance(actor2, movie2)
\newline val appearance3 = new Appearance(actor3, movie2)
\newline actor3.famous.observe(true)
\newline movie2.quality.observe('high)
\newline
\marginpar{The \texttt{\_.award} notation is Scala shorthand to retrieve the award value of each element of the map}
\newline // Ensure that exactly one appearance gets an award.
\newline def uniqueAwardCondition(awards: List[Boolean]) =
\newline \tab awards.count((b: Boolean) => b) == 1
\newline val allAwards: Element[List[Boolean]] =
\newline \tab Inject(appearances.map(\_.award):\_*)
\newline allAwards.setCondition(uniqueAwardCondition)}
\end{flushleft}

The code is self-explanatory except for the last few lines, which enforce the condition that an award is given to exactly one appearance. The function \texttt{uniqueAwardCondition} takes a list of award Booleans and returns true if exactly one Boolean in the list is true. The count method counts the number of elements in the list that satisfy the predicate contained in its argument. In this case the predicate is \texttt{(b: Boolean) => b} which is true precisely when the element of the list is true. So \texttt{awards.count((b: Boolean) => b)} counts the number of elements in the list that are true.

We then define the \texttt{allAwards} element to be the element over lists of Booleans consisting of the award field of all the appearances.  Here we have a new notation: \texttt{appearances.map(\_.award)}. We have already seen the \texttt{map} method, which applies a function to every element of a list and returns a new list consisting of the results. In this case, the argument to map is the function \texttt{\_.award}. This is shorthand for a function of one argument in which the argument appears once in the body and in which the type of the argument is known. Here, the type of the argument is clearly an appearance. We could have used \texttt{appearance => appearance.award}. The notation \texttt{\_.award} is short for this. Finally, we impose the \texttt{uniqueAwardCondition} on \texttt{allAwards}, ensuring that exactly one appearance is awarded.

\section{Mutable fields}

Up to this point, all our Figaro programs have been purely functional. All elements have been defined by a \texttt{val}, and they have been immutable. In principle, all programs can be written in a purely functional style. However, this can make it quite inconvenient to represent situations in which different entities refer to each other. Scala supports both functional and non-functional styles of programming, allowing us to gain the benefits of both.

For example, let's expand the actors and movies example so that actors have a skill, and the quality of a movie depends on the skill of the actors in it. In turn, the fame of an actor depends on the quality of the movies in which he or she has appeared. We have created a mutual dependence of actors on movies which is hard to represent in a purely functional style. We can capture it in Figaro using the following code:

\begin{flushleft}
\marginpar{This example can be found in MutableMovie.scala}
\texttt{import com.cra.figaro.library.compound.CPD
\newline import com.cra.figaro.language.\_
\newline
\newline class Actor \{
\newline \tab var movies: List[Movie] = List()
\newline \tab lazy val skillful = Flip(0.1)
\newline \tab lazy val famous =
\newline \tab Flip(Apply(Inject(movies.map(\_.quality):\_*), probFamous \_))
\newline \tab private def probFamous(qualities: Seq[Symbol]) =
\newline \tab if (qualities.count(\_ == 'high) >= 2) 0.8; else 0.1
\newline \}
\newline
\newline class Movie \{
\newline \tab var actors: List[Actor] = List()
\newline \tab lazy val actorsAllGood = Apply(Inject(actors.map(\_.skillful):\_*), (s: Seq[Boolean]) => !(s.contains(false)))
\newline lazy val probLow =
\newline Apply(actorsAllGood, (b: Boolean) => if (b) 0.2; else 0.5)
\newline lazy val probHigh =
\newline Apply(actorsAllGood, (b: Boolean) => if (b) 0.5; else 0.2)
\newline lazy val quality =
\newline Select(probLow -> 'low, Constant(0.3) -> 'medium, probHigh -> 'high)
\newline \}
\newline
\newline class Appearance(actor: Actor, movie: Movie) \{
\newline \tab actor.movies ::= movie
\marginpar{\texttt{::=} is Scala shorthand for list concatenation}
\newline \tab movie.actors ::= actor
\newline
\newline \tab def probAward(quality: Symbol, famous: Boolean) =
\newline \tab \tab (quality, famous) match \{
\newline \tab \tab case ('low, false) => 0.001
\newline \tab \tab case ('low, true) => 0.01
\newline \tab \tab case ('medium, false) => 0.01
\newline \tab \tab case ('medium, true) => 0.05
\newline \tab \tab case ('high, false) => 0.05
\newline \tab \tab case ('high, true) => 0.2
\newline \tab \tab \}
\newline \tab lazy val award = SwitchingFlip(Apply(movie.quality, actor.famous, (q: Symbol, f: Boolean) => probAward(q, f)))
\newline \}
\newline
\newline val actor1 = new Actor
\newline val actor2 = new Actor
\newline val actor3 = new Actor
\newline val movie1 = new Movie
\newline val movie2 = new Movie
\newline val appearance1 = new Appearance(actor1, movie1)
\newline val appearance2 = new Appearance(actor2, movie2)
\newline val appearance3 = new Appearance(actor3, movie2)
\newline actor3.famous.observe(true)
\newline movie2.quality.observe('high)
\newline
\newline // Ensure that exactly one appearance gets an award.
\newline def uniqueAwardCondition(awards: List[Boolean]) =
\newline \tab awards.count((b: Boolean) => b) == 1
\newline val allAwards: Element[List[Boolean]] =
\newline \tab Inject(appearances.map(\_.award):\_*)
\newline allAwards.setCondition(uniqueAwardCondition)
}
\end{flushleft}

First, note that the \texttt{Actor} class has a \texttt{movies} field, whose purpose is to indicate the list of movies the actor has appeared in. Likewise, the \texttt{Movie} class has an \texttt{actors} field to represent the actors who appear in it. If these fields were immutable, we would need to create all the movies an actor appears in before we create the actor, and we would need to create all the actors appearing in a movie before the movie, which is impossible. Therefore, we use mutable variables, which are indicated in Scala by the \texttt{var} keyword.

The initial value of both \texttt{movies} and \texttt{actors} is an empty list. We add elements to them later. In fact, whenever we create an appearance, we make sure to add the movie to the actor's list of movies and vice versa. This is achieved by the first two lines of the \texttt{Appearance} class. The notation \texttt{actor.movies ::= movie} is short for \texttt{actor.movies = movie :: actor.movies}, which prepends movie to the current \texttt{actor.movies} list, and replaces the current list with the new list. The \texttt{::=} notation is a variant of the familiar \texttt{+=} notation common in many languages.

The \texttt{Actor} class has \texttt{skillful} and \texttt{famous} fields. Rather than an ordinary \texttt{val}, each of these fields is defined to be \texttt{lazy val}, which means that their contents are not determined until they are required by some other computation. This is necessary for us because their contents can depend on the list of movies the actor appears in. For example, whether the actor is famous depends on whether at least two movies have high quality, as defined by \texttt{probFamous}. (Comment on the notation: the underscore after \texttt{probFamous} is required here to tell Scala that what is desired is the \texttt{probFamous} function itself, not its application to arguments.)   If \texttt{famous} was an ordinary \texttt{val}, its value (an \texttt{Element[Boolean]}) would be computed at the point it is defined, so it would use an empty list of movies. Because we want to use the correct list of movies in defining it, we postpone evaluating it until the movies list has been filled. For \texttt{actor3}, this will happen when we make the observation \texttt{actor3.famous.observe(true)}, which we make sure to delay until after all the appearances have been created. For other actors, the \texttt{famous} field will be evaluated even later, during inference. Care should be taken with declaring elements as lazy. Side effects and unintended consequences can occur if a lazy element declared outside a Chain is first required (i.e., created) during the execution of a Chain.

Do not hesitate to use mutation if it will help you organize your program in a logical way. In one application, we have found it convenient to use a hash table that maps concepts to their associated elements. This allowed us to create the element associated with a concept as the concept was introduced. If we later had to refer to the same concept again, we could easily access its element.

\section{Universes}

A central concept in Figaro is a \emph{universe}. A universe is simply a collection of elements. Reasoning algorithms operate on a universe (or, as we shall see for dependent universe reasoning, on multiple connected universes). Most of the time while using Figaro, you will not need to create a new universe and can rely on the default universe, which is just called \texttt{universe}. It can be accessed using

\begin{flushleft}
\texttt{import com.cra.figaro.language.\_
\newline import com.cra.figaro.language.Universe.\_}
\end{flushleft}

If you do need a different universe, you can call \texttt{Universe.create\-New()}. This creates a new universe and sets the default universe to it. If you are going to need the old default universe, you will need a way to refer to it. You could use

\begin{flushleft}
\texttt{val u1 = Universe.universe
\newline val u2 = Universe.createNew()}
\end{flushleft}

\texttt{u1} will now refer to the old default universe while \texttt{u2} refers to the new one. Every element belongs to exactly one universe. Ordinarily, when an element is created, it is assigned to the current default universe. As we will see below when we talk about element collections, it is possible to assign a particular element to a different universe from the current default.

Elements can be activated or deactivated. Elements that are inactive are not operated on by reasoning algorithms. Elements are active when created. To deactivate an element \texttt{e} use \texttt{e.deactivate()}; to reactivate it, use \texttt{e.activate()}. When a compound element is created that uses a parent element, the parent must already be active.

You can get a list of all active elements in universe \texttt{u} using \texttt{u.active\-Elements}. There are many more methods of a universe that are useful for writing reasoning algorithms. See the documentation in Universe.scala for more details.

\section{Names, element collections, and references}

Suppose we want to create a PRM in which we are uncertain about the value of an attribute whose value is itself an instance of another class (which is called reference uncertainty). For example, suppose we have the following classes and instances:

\begin{flushleft}
\marginpar{This example can be found in CarAndEngine.scala}
\texttt{import com.cra.figaro.language.\_
\newline
\newline abstract class Engine \{ val power : Element[Symbol] \}
\newline class V8 extends Engine \{
\newline \tab val power = Select(0.8 -> 'low, 0.2 -> 'high)
\newline \}
\newline class V6 extends Engine \{
\newline \tab val power = Select(0.8 -> 'low, 0.2 -> 'high)
\newline \}
\newline
\newline object MySuperEngine extends V8 \{
\newline \tab val power = Constant('high)
\newline \}
\newline class Car \{
\newline \tab val engine = Uniform[Engine](new V8, new V6, MySuperEngine)
\newline \tab val speed = CPD(?,
\newline \tab 'high -> Constant(90.0),
\newline \tab 'medium -> Constant(80.0),
\newline \tab 'low -> Constant(70.0))
\newline \}
}
\end{flushleft}

We want the speed of the car to depend on the power of its engine, but we have uncertainty over what the engine actually is. What should we put in place of the question mark? The obvious choice is \texttt{engine.power}, but this does not work because engine is an \texttt{Element\-[Engine]}, not an instance of \texttt{Engine}.

To get around this problem, Figaro provides \emph{names} and \emph{element collections}. Every element has a name and belongs to an element collection. By default, the name is the empty string and the element collection is the default universe at the time the element is created, which works because universes are element collections. So, most of the time, as in the tutorial to this point, you don't have to worry about the name and element collection of an element. To assign a name and element collection to an element explicitly, you provide an extra pair of arguments when creating it.

We can give the engine a name and make it belong to the car as an element collection as follows:

\begin{flushleft}
\texttt{class Car extends ElementCollection \{
\newline \tab val engine = Uniform[Engine](new V8, new V6, MySuperEngine)("engine", this)
\newline \}
}
\end{flushleft}

In the first line we make the \texttt{Car} class inherit from \texttt{ElementCollect\-ion}, so that every instance of \texttt{Car} is an element collection. In the fourth line, we assign engine the name "engine" and add it to the instance of Car being created, which is referred to by \texttt{this} within the \texttt{Car} class. We similarly make the abstract \texttt{Engine} class inherit  element collections and assign \texttt{power} the name "power" within \texttt{V*, V6, and MySuperEngine} within each subclass of  \texttt{Engine}.

An element collection, like a universe, is simply a set of elements. The difference is that a universe is also a set of elements on which a reasoning algorithm operates. An element collection provides the ability to refer to an element by name. For example, if \texttt{car} is an instance of \texttt{Car}, we can use \texttt{car.get[Engine]("engine")} to get at the element named "engine". The \texttt{get} method takes a type parameter, which is the value type of the element being referred to. The notation \texttt{[Engine]} specifies this type parameter, and serves to make sure that the expression \texttt{car.get[Engine]("engine")} has type \texttt{Element[Engine]}.

The key ability of element collections that allows them to solve our puzzle is their ability to get at elements embedded in the value of an element.  It uses \emph{references} to do this. A reference is a series of names separated by dots. For example, "engine.power" is a reference. When we call \texttt{car.get[Symbol]("engine.power")}, it refers to the element named "power" within the \emph{value} of the element named "engine" within the car. The value of this expression is a \texttt{ReferenceElement} that captures the uncertainty about which power element is actually being referred to. In a particular state of the world, i.e., an assignment of values to all elements, it is possible to determine the value of engine and therefore which \texttt{power} element is being referred to. So a \texttt{ReferenceElement} is a deterministic element that defines a way to get its value in any possible world.

So, finally, the answer to our puzzle is that in place of the question mark, we put \texttt{get[Symbol]("engine.power")}. This applies the get method to the instance of \texttt{Car} being created. Here is the full example:

\begin{flushleft}
\marginpar{This example can be found in MultiValuedReferenceUncertainty.scala}
\texttt{import com.cra.figaro.language.\_
\newline
\newline abstract class Engine extends ElementCollection \{
\newline \tab val power : Element[Symbol]
\newline \}
\newline class V8 extends Engine \{
\newline \tab val power = Select(0.8 -> 'low, 0.2 -> 'high)("power", this)
\newline \}
\newline class V6 extends Engine \{
\newline \tab val power = Select(0.8 -> 'low, 0.2 -> 'high)("power", this)
\newline \}
\newline object MySuperEngine extends V8 \{
\newline \tab val power = Constant('high)("power", this)
\newline \}
\newline class Car extends ElementCollection \{
\newline \tab val engine = Uniform[Engine](new V8, new V6, MySuperEngine)("engine", this)
\newline \tab val speed = CPD(
\newline \tab get[Symbol]("engine.power"),
\newline \tab 'high -> Constant(90.0),
\newline \tab 'medium -> Constant(80.0),
\newline \tab 'low -> Constant(70.0))
\newline )
\newline \}
}
\end{flushleft}

\section{Multi-valued references and aggregates}

The previous subsection described how to refer to elements using references that identify a single element. A feature of PRMs is the ability to define multi-valued relationships, where an entity is related to multiple entities via an attribute. In Figaro, we use multi-valued references and aggregates to capture these kinds of situations. For example:

\begin{flushleft}
\texttt{import com.cra.figaro.language.\_
\newline
\newline class Component extends ElementCollection \{
\newline \tab val f = Select(0.2 -> 2, 0.3 -> 3, 0.5 -> 5)("f", this)
\newline \}
\newline
\newline val specialComponent1 = new Component
\newline val specialComponent2 = new Component
\newline
\newline def makeComponent() =
\newline \tab Select(0.1 -> specialComponent1,
\newline \tab 0.2 -> specialComponent2,
\newline \tab 0.7 -> new Component)
\newline
\newline class Container extends ElementCollection \{
\newline \tab  val components = MakeList(Select(0.5 -> 1, 0.5 -> 2), makeComponent)("components", this)
\newline
\marginpar{The body of the sum function is shorthand notation for Scala's fold function. Fold iterates through a sequence and applies a function to the previous
result and each new entry in turn. The (\_ + \_ ) notation will add the previous value to each value in \texttt{xs}.}
\newline val sum = getAggregate((xs: MultiSet[Int]) => (0 /: xs)(\_ + \_))("components.f")
\newline \}
}
\end{flushleft}

First, we create a \texttt{Component} class with an element named "f". We then define two specific instances of \texttt{Component}. Next, we define a \texttt{makeComponent} function that either produces one of the specific instances or a new instance of \texttt{Component} that is distinct from all other instances. We then define a \texttt{Container} class that contains components. Now, the contained components are a list that has either one or two elements, each produced by \texttt{makeComponent}. We then create a \texttt{sum} element that aggregates the values of all elements referred to by "components.f"; that is, the values of the elements named "f" in all the values of the element named "components".

Multi-valued references have "set semantics". If the same element appears more than once as the target of the reference, it only contributes one value to the aggregate. So, if the components list has two components, both of which are \texttt{specialComponent1}, whose value is 2, the value of the aggregate will be 2, not 4. On the other hand, if two different target elements both have the same value, both values contribute to the aggregate. For example, if the components are \texttt{specialComponent1} and \texttt{specialComponent2}, and both have value 2, the value of the aggregate is 4.

A comment on the code: The code defining \texttt{sum} might look mysterious. This code takes a list of integers and returns their sum. This is a standard Scala idiom that unfortunately is a bit obscure if you're not familiar with it. It is used to "fold" a function through a list. We begin with 0 and then repeatedly add the current result to the next element of the list until the list is exhausted. The notation \texttt{(\_ + \_)} is shorthand for the function that takes two arguments and adds them. The notation \texttt{(0 /: xs}) means that this function should be folded through \texttt{xs}, starting from 0.

\section{Open Universe Models}

We close this section by showing how Figaro can be used to represent "open universe" situations. An open universe situation is one in which we don't know what objects are there, how many there are, which objects are the same as which other objects, and so on. In our example situation, there are an unknown number of sources that is geometrically distributed. Each source is uniformly distributed between 0 and 1. There is some number of observed samples, each drawn from a single unknown source. This is the classic data association problem in which we want to determine which sample comes from which source, and in particular which two samples actually come from the same source. The Figaro code for the example is as follows:

\begin{flushleft}
\marginpar{This example can be found in OpenUniverse.scala}
\texttt{import com.cra.figaro.language.\_
\newline import com.cra.figaro.library.atomic.continuous.Uniform
\newline import com.cra.figaro.library.atomic.continuous.Normal
\newline import com.cra.figaro.library.atomic.discrete.Geometric
\newline import com.cra.figaro.library.compound.\{MakeList, IntSelector\}
\newline
\newline def source(): Element[Double] = Uniform(0.0, 1.0)
\newline
\newline val numSources = Geometric(0.9)
\newline
\newline val sources = MakeList(numSources, source \_)
\newline
\newline class Sample \{
\newline \tab val sourceNum = IntSelector(numSources)
\newline \tab val source =
\newline \tab Apply(sources, sourceNum, (s: Seq[Double], i: Int) => s(i))
\newline \tab val position =
\newline \tab NonCachingChain(source, (x: Double) => Normal(x, 1.0))
\newline \}
\newline
\newline val sample1 = new Sample
\newline val sample2 = new Sample
\newline
\newline val equal = sample1.source === sample2.source
\newline
\newline sample1.position.addCondition((y: Double) => y >= 0.7 \&\& y < 0.8)
\newline sample2.position.addCondition((y: Double) => y >= 0.7 \&\& y < 0.8)
}
\end{flushleft}

Most of this should be self-explanatory at this point. There are a couple of interesting new element classes being used. \texttt{MakeList} takes an element over integers and a function that generates elements over a certain type (in this case doubles). It returns an element over lists of the appropriate type (in this case lists of doubles) whose length is distributed according to the first argument and in which each element is generated according to the second argument. In our example, \texttt{sources} is a list of sources whose length is geometrically distributed and in which each source is generated according to the source model. A notable aspect of the \texttt{MakeList} class, which is important for reasoning algorithms, is that the elements generating the values in the list are stored as an infinite lazy stream. Depending on the value of the first argument, the value of the \texttt{MakeList} is a finite prefix of the values of elements in the stream. As a result of this design, we don't create a completely fresh list each time the length of the list changes. \texttt{MakeList} could also have been used in the previous section's example to define the \texttt{components} element of the \texttt{Container} class.

The second new element class is \texttt{IntSelector} which takes an element over integers and returns an element that produces uniformly a number between 0 and the value of its argument (exclusive). This element can be used to generate a random index into a list produced by \texttt{MakeList}. \texttt{IntSelector} also has an interesting implementation that has benefits for reasoning algorithms (especially Metropolis- Hastings). The \texttt{Randomness} is an infinite stream of uniformly distributed doubles between 0 and 1. Given a particular value of the integer argument, the selected index is the one with the highest randomness value in the finite portion of the stream defined by the argument.


%!TEX root = ../notes.tex
\section{March 10, 2022}
\subsection{Jacobi Symbol \emph{continued}}
\recall
\begin{definition*}[Jacobi Symbol]
    Let $b\in\ZZ_+$ be odd, and let $a\in\ZZ$. We write $b = p-1p_2\cdots p_m$, where $p_i$ are primes (not necessarily distinct). Then we have
    \[\lege{a}{b} = \lege{a}{p_1}\lege{a}{p_2}\cdots \lege{a}{p_m}\]
    is called the \ul{Jacobi symbol}.
\end{definition*}

This generalizes the Legendre symbol. We have basic properties that
\begin{align*}
    \lege{a_1a_2}{b} & = \lege{a_1}{b}\lege{a_2}{b} \\
    \lege{a}{b_1b_2} & = \lege{a}{b_1}\lege{a}{b_2}
\end{align*}
We noted that $\lege{a}{b} = -1$ implies that $a$ is \emph{not} a quadratic residue mod $b$ but $\lege{a}{b} = 1$ does not imply $a$ is a quadratic residue mod $b$.

We also stated analogues of the reciprocity laws for the Legendre symbol.

\begin{lemma}\label{lemma:multiplicativity-for-jacobi}
    Let $r, s\in\ZZ_+$ be odd. Then
    \begin{enumerate}[(a)]
        \item $\displaystyle\frac{rs-1}{2}\equiv \frac{r-1}{2} + \frac{s-1}{2}\mod 2$.
        \item $\displaystyle\frac{r^2s^2 - 1}{8} \equiv \frac{r^2-1}{8} + \frac{s^2 - 1}{8}\mod 2$.
    \end{enumerate}
\end{lemma}
\begin{proof}
    ~\begin{enumerate}[(a)]
        \item $(r-1)(s-1)\equiv 0\mod 4$. Hence
              \begin{align*}
                  rs - 1 & \equiv (r-1)(s-1) + r + s - 2\mod 4          \\
                         & \equiv r + s - 2\mod{4}                      \\
                         & \equiv (r-1) + (s-1)\mod{4}                  \\
                         & \equiv \frac{r-1}{2} + \frac{s-1}{2} \mod{2}
              \end{align*}
              which gives (a).
        \item We follow the same procedure, more or less. $(r^2-1)(s^2-1)\equiv 0\mod{16}$, so
              \begin{align*}
                  r^2s^2-1 & \equiv (r^2-1)(s^2-1)+r^2+s^2-2\mod{16}         \\
                           & \equiv (r^2-1)+(s^2-1)\mod{16}                  \\
                           & \equiv \frac{r^2-1}{8} + \frac{s^2-1}{8}\mod{2}
              \end{align*}
              which gives (b).
    \end{enumerate}
\end{proof}

\begin{corollary}
    let $r_1, r_2, \dots, r_m\in\ZZ_+$ be odd. Then
    \begin{enumerate}[(a)]
        \item $\displaystyle
                  \sum_{i=1}^m \frac{r_i-1}{2} \equiv \frac{r_1r_2\cdots r_m - 1}{2}\mod 2
              $.
        \item $\displaystyle
                  \sum_{i=1}^m \frac{r_i^2 - 1}{2}\equiv \frac{r_1^2r_2^2\cdots r_m^2 - 1}{8}\mod 2
              $.
    \end{enumerate}
\end{corollary}
\begin{proof}
    By induction on $m$ from \cref{lemma:multiplicativity-for-jacobi}.
\end{proof}

We restate the reciprocity laws but for Jacobi symbols, \cref{prop:5.2.2}:
\begin{proposition*}[5.2.2 of Text]
    We have the following properties about the Jacobi symbol:
    \begin{enumerate}[(a)]
        \item \[\lege{-1}{b} = (-1)^{\frac{b-1}{2}}\]
        \item \[\lege{2}{b} = (-1)^{\frac{b^2 - 1}{8}}\]
        \item If $a, b\in\ZZ_+$, then
              \[\lege{a}{b}\lege{b}{a} = (-1)^{\frac{a-1}{2}\frac{b-1}{2}}\]
    \end{enumerate}
\end{proposition*}
\begin{proof}[Proof of \cref{prop:5.2.2}]
    ~\begin{enumerate}
        \item[(a) $+$ (b)] are immediate from the corollary (factor $b$, sum exponents and take the parity of the exponent) and the supplemental laws of quadratic reciprocity.
        \item[(c)] Let
            \begin{align*}
                a & =q_1q_2\cdots q_l   \\
                b & = p_1p_2\cdots p_m.
            \end{align*}
            Then
            \begin{align*}
                \lege{a}{b}\lege{b}{a} & = \prod_{i}\prod_{j} \lege{q_i}{p_j}\lege{p_i}{q_j}                                     \\
                                       & = (-1)^{\sum_i \sum_j \left(\frac{q_i-1}{2}\right)\left(\frac{p_j-1}{2}\right)}
                \intertext{Applying the corollary, }
                                       & = (-1)^{\left(\sum_i \frac{q_i - 1}{2}\right)\left(\sum_j \frac{p_j - 1}{2}\right)}     \\
                                       & = (-1)^{\left(\frac{(\sum_i q_i) - 1}{2}\right)\left(\frac{(\sum_j p_j) - 1}{2}\right)} \\
                                       & = (-1)^{\left(\frac{a-1}{2}\right)\left(\frac{b-1}{2}\right)}
            \end{align*}
            which is as desired!
    \end{enumerate}
\end{proof}
\begin{example}
    We try to compute with the Jacobi symbol. Recall \cref{example:legendre-symbol}
    \[\lege{219}{383}\]
    where we repeatedly factored and flipped. With a Jacobi symbol, we don't need to start with factoring; we can forego factorization of top argument and simply repeatedly flip:
    \begin{align*}
        \lege{219}{383} & = - \lege{383}{219} = -\lege{164}{219} = - \lege{4}{219}\lege{41}{219} = - \lege{41}{219} \\
                        & = - \lege{219}{41} = - \lege{14}{41} = - \lege{2}{41}\lege{7}{41} = -\lege{7}{41}         \\
                        & = -\lege{41}{7} = -\lege{-1}{7} = \boxed{1}.
    \end{align*}
\end{example}
What we did here is to exploit the fact that \emph{all} Legendre symbols agree with Jacobi symbols, we treat it as a Jacobi symbol and do `Jacobi-\emph{like}' manipulations on it.
\begin{center}
    \includegraphics[width=0.4\textwidth]{images/jacobi_legendre_reveal.jpeg}
\end{center}

\emph{This marks the dividing line between the first half and latter half of the course! Everything up to this point is fair game on the midterm. We also now switch to Stewart and Tall.}

\subsection{Number Fields}
\begin{definition}[Algebraic Numbers]
    A complex number $x$ is called \ul{algebraic} if it is algebraic over $\QQ$, i.e., if it satisfies a nonzero polynomial equation over $\QQ$.

    We denote the set of algebraic numbers over $\QQ$ as $\overline{\QQ}$.
\end{definition}
\begin{proposition}
    The set $\overline{\QQ}$ of algebraic numbers is a subfield of $\CC$. That is, addition and multiplication is closed, and we have inverses for nonzero elements.
\end{proposition}
\begin{proof}
    The key point is that if $L / K$ is a field extension, then $\alpha\in L$ is algebraic over $K$ iff $K(\alpha)/K$ is finite.

    So suppose $\alpha, \beta\in\overline{\QQ}$. Then $\QQ(\alpha)/\QQ$ and $\QQ(\beta)/\QQ$ are finite. Thus $\QQ(\alpha, \beta)/\QQ$ is finite (and all pieces of the associated diamond are finite extensions as well).

    \[
        \xymatrix{
            \QQ \ar[d] \ar[r] &\QQ(\alpha)\ar[d]\\
            \QQ(\beta) \ar[r] &\QQ(\alpha, \beta)}
    \]

    Since $\alpha + \beta$, $\alpha - \beta$, $\alpha\beta$ and for $\beta \neq 0$, $\alpha/\beta \in \QQ(\alpha, \beta)$. This means that all of these elements are algebraic over $\QQ$.
\end{proof}

\begin{definition}
    A \ul{number field} is a subfield $K$ of $\CC$ such that $[K : \QQ] < \infty$.
\end{definition}

Thus every element of a number field is algebraic, so $K\subseteq \overline{Q}$.

By the definition of a finite extension, every number field has the form
\[K = \QQ(\alpha_1, \alpha_2, \dots, \alpha_N)\text{ for some }\alpha_1, \dots, \alpha_n\in\overline{\QQ}\]

However, something stronger than this holds.

\begin{theorem}[Primitive Element Theorem]
    If $K$ is a number field, then $K = \QQ(\theta)$ for some $\theta\in\overline{\QQ}$.
\end{theorem}
\begin{proof}[Proof sketch]
    It is enough to show that if
    \[K = K_1(\alpha, \beta), \]
    then $K = K_1(\theta)$ for some $\theta\in\overline{\QQ}$.

    Suppose the minimum polynomials (over $\QQ$) of $\alpha$ and $\beta$ respectively are (factored over roots in $\CC$):
    \begin{align*}
        (t-\alpha_1)(t-\alpha_2)\cdots (t-\alpha_n)\qquad \alpha_1 & = \alpha \\
        (t-\beta_1)(t-\beta_2)\cdots (t-\beta_n)\qquad \beta_1     & = \beta  \\
    \end{align*}
    These polynomials are separable. Hence for each $i$ and each $k\neq 1$, there exists at most one $x\in K_1$ such that
    \[\alpha_i + x\beta_k = \alpha_1 + x\beta_1.\]
    (This only holds for more $x$ when you have $\beta_k$ and $\beta_1$ colliding). There are only finitely many of these equations, so we can choose a nonzero $c\in K_1$ such that
    \[\alpha_i + c\beta_k \neq \alpha_1 + c\beta_1\]
    for any $1\leq i\leq n$ and $2\leq k\leq m$.

    Define $\theta = \alpha + c\beta$. We claim that
    \[K_1(\alpha, \beta) = K_1(\theta)\]
    for which the proof is on page 39 of Stewart Tall.
\end{proof}


\documentclass[11pt]{article}

\usepackage{latexsym}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{graphicx}
\usepackage{float}
\usepackage{wrapfig}
\usepackage{pseudocode}
\usepackage{url}
\usepackage[backref, colorlinks=true, citecolor=red, urlcolor=blue, pdfauthor={Jyh-Ming Lien}]{hyperref}
\graphicspath{{images/}}

\newcommand{\handout}[5]{
  \noindent
  \begin{center}
  \framebox{
    \vbox{
      \hbox to 5.78in { {\bf } \hfill #2 }
      \vspace{4mm}
      \hbox to 5.78in { {\Large \hfill #5  \hfill} }
      \vspace{2mm}
      \hbox to 5.78in { {\em #3 \hfill #4} }
    }
  }
  \end{center}
  \vspace*{4mm}
}

\newcommand{\lecture}[4]{\handout{#1}{#2}{#3}{#4}{#1}}

\newtheorem{theorem}{Theorem}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{observation}[theorem]{Observation}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{fact}[theorem]{Fact}
\newtheorem{assumption}[theorem]{Assumption}

% 1-inch margins, from fullpage.sty by H.Partl, Version 2, Dec. 15, 1988.
\topmargin 0pt
\advance \topmargin by -\headheight
\advance \topmargin by -\headsep
\textheight 8.9in
\oddsidemargin 0pt
\evensidemargin \oddsidemargin
\marginparwidth 0.5in
\textwidth 6.5in

\parindent 0in
\parskip 1.5ex
%\renewcommand{\baselinestretch}{1.25}

\begin{document}

\lecture{Voronoi Stippling Report}{Fall 2017}{Anthony Sermania}{Computational Geometry}

\section{Summary of the two methods}

\subsection{hedcuter method}
The hedcuter method is as follows:

Hedcut:
\begin{enumerate}
\item Create a set of n random initial points (keep on random probability, favoring darker areas)
\item Compute the Weighted Centroidal Voronoi Tessellation (CVT)
\item Generate Disks
\end{enumerate}

Computing The CVT:
\begin{enumerate}
	\item While (we have fewer iterations than the limit AND the max-displacement is greater than some threshold:
	\begin{enumerate}
	\item Computer the Voronoi
	\item Shift Voronoi cell sites towards cell centroids (and record displacement
	\end{enumerate}
\end{enumerate}

Compute the Voronoi
\begin{enumerate}
\item For each input site
	\begin{enumerate}
	\item redirect site to corresponding position in resized image (fake enhanced resolution)
	\item determine distance based on color of site
	\item add site to open list/heap
	\end{enumerate}

\item while heap is not empty
\begin{enumerate}
	\item cell <- max heap element (pop)
	\item if distance has been updated or cell has been visited
		\begin{enumerate}
		\item skip
		\end{enumerate}

	\item for each neighbor of cell
	\begin{enumerate}
	\item if cell.distance+neighbor.distance < neighbor.distance
	\begin{enumerate}
	\item set neighbor.distance <- cell.distance+neighbor.distance
			\item make neighbor.root <- cell.root
			\item push neighbor into heap
	\end{enumerate}

	\end{enumerate}

\end{enumerate}

\item For each cell in image
	\begin{enumerate}
	\item determine root of cell
	\item add cell to cell.root's coverage
	\end{enumerate}

\item Remove all cells with coverage.empty
\end{enumerate}

Move voronoi cell sites to their new locations (and record displacement)
\begin{enumerate}
\item For each cell
\begin{enumerate}
\item calculate new position as average of positions in coverage
\item Generate high res img
\item newpos <- (0, 0), total <- 0
\item for each cell c in cell.coverage
\begin{enumerate}
\item newpos += c.distance*c.pos
\item total += c.distance
\end{enumerate}

\item newpos /= total to normalize
\item newpos /= subpixels to redirect to original image
\item store manhattan distance from new to old position
\end{enumerate}

\item return the max displacement (manhattan distance) or average of displacements (depending on termination mode)
\end{enumerate}

Generate Disks
\begin{enumerate}
\item Collect the darkness/color of the area in a voronoi cell
\item Create a disk at cell site (center)
\item Color/darken cell according to the average color/darkness of the pixels i the cell
\end{enumerate}

\subsection{voronoi method}
This one was a little bit tougher to work through mentally. At the highest level it uses Lloyd's Algorithm as so:

While generating points (xi) not converged to centroid
\begin{enumerate}
\item compute voronoi diagram of xi
\item compute centroid Ci
\item move generating point xi to centroid Ci
\end{enumerate}

The Voronoi diagram computation is done differently here, utilizing an entire Voronoi Generation class that appears to implement a half-edge mechanism for doing it's work after some initial set up. It is extremely well commented; rather than just essentially replicating the comments, here is an overview of the half-edge voronoi generation:

\begin{enumerate}
\item While there are events:
\begin{enumerate}
\item If the lowest site is lower than the lowest vector intersection process the site
\begin{enumerate}
\item Get left and right half edges, plus the edge of the left halfedge (or the bottom site if there's no edge)
\item create a bisector for the left edge (or bottom site) and if it intersects the left edge shuffle the vertices so the left edge and bisector share a vertex.
\item Create a new half edge right of the first (only at the moment) bisector
\item if this new bisector intersects with the right halfedge, add the bisector halfedge to the ordered linked list of verts
\item get next site
\end{enumerate}

\item else process the lowest vector intersection
\begin{enumerate}
\item Get the halfedge for the intersection vector, as well as the surrounding halfedges
\item get the sites of the of the exterior halfedges
\item consider a circle passing through the triple of sites obtained
\item get the vertex that caused the event
\item set the endpoints of the left and right half edges to be this vertex
\item mark the lowest half edge for deletion
\item clear events for right halfedge
\item mark right halfedge for deletion
\item if the site to left of the event is higher than site to the right, swap them
\item create an edge between the two sites
\item create a half edge for this edge
\item insert the new bisector to the right of the left halfedge
\item cleanup (reposition half-edges not intersecting the bisector that should)
\end{enumerate}

\end{enumerate}

\end{enumerate}

That still got pretty verbose, but it's not quite done. Once this routine has run, the generator that ran it is used to collect the edges before completing (this process is straightforward and is not detailed here).

Now it moves to step 2, redistributing the stipples (bringing them closer to the centroids of the Voronoi cells). This is done in parallel where applicable.

\begin{enumerate}
\item Do some setup (make pairs of points and edge lists)
\item For each pair:
\begin{enumerate}
\item Calculate Centroid
\item update Radius and vertex X/Y positions to be that of the calculated centroid
\item determine displacement of point
\end{enumerate}
\item average displacement of all points
\end{enumerate}

Calculating a Centroid:
According to the paper, \textit{Weighted Voronoi Stippling} by Adrian Secord, there are some integrals being done here. The implementation provided however seems to work around that, operating primarily on distance constraints:

\begin{enumerate}
\item Compute the Clip Lines
\item For all points in this space if a point is inside it's polygon, it contributes to the area density
\item If a cell is not completely white, shift the centroid according to the weighted-spot:area density ratio (that's a weird way to put it, but it's what's happening)
\item Iterate over the available edges and determine the nearest and farthest, using one of those distances to determine the size of the stipple (nearest if NoOverlap, Farthest otherwise) and scale by the area:max-area ratio
\item Return the pair of the original point and the radius.
\end{enumerate}

\section{Comparison of the two methods}
Both methods saw fair success. I will however claim the voronoi code to be superior overall.

Let's start with hedcut though. For one thing, hedcut was not consistent across multiple runs with the same parameters. Take for instance the following images:

\begin{figure}[H]
\includegraphics[scale=0.4]{fairyeyes-10000-r4-1.png}
\includegraphics[scale=0.4]{fairyeyes-10000-r4-2.png}
\end{figure}

We can see here that the two images, run with the same parameters (radius 4, 10,000 stipples) produced different stipplings. By contrast, here are two images from the voronoi code with the equivalent parameters (4,000 stipples, color, overlap)

\begin{figure}[H]
\includegraphics[scale=0.4]{fairyeyes-4k-color-2.png}
\includegraphics[scale=0.4]{fairyeyes-4k-color-3.png}
\end{figure}

And two more from voronoi, this time with no overlap.

\begin{figure}[H]
\includegraphics[scale=0.4]{fairyeyes-noverlap-4k-color.png}
\includegraphics[scale=0.4]{fairyeyes-noverlap-4k-color-2.png}
\end{figure}

It is worth noting I discovered later that hedcut might by consistent with sufficiently many iterations (see Improvements).

In terms of speed, hedcut may be faster in the simplest circumstances, coming out at around the same computatin time for 10,000 stipples as voronoi did for 4,000 and 8,000, however voronoi never seemed to vary much with number of dots- just by image. Hedcut on the other hand does take longer the more dots there are. The two images below are both 100,000 stipples. The image on the left is the hedcut image, created in 16.22 seconds; the one on the right is from voronoi, created in 4.43 seconds (not substantially slower than the 8,000 or 4,000 stipple runs for voronoi).

\begin{figure}[H]
\includegraphics[scale=0.4]{fairyeyes-hedcut-100000.png}
\includegraphics[scale=0.4]{fairyeyes-voronoi-100000.png}
\end{figure}

Beyond that time difference, the level of image fidelity with this many stipples is vastly superior in the voronoi image. I suspect the difference in speed comes from the use of so many loops in the hedcut code. The difference in quality is undoubtedly due to the stronger implementation of varied stipple sizes (based purely on the voronoi cell size) in the voronoi code. The hedcut code uses a base radius which is scaled, resulting in much weirder stipple sizes. It does however improve it's image fidelity with successive iterations (see below), perhaps even gaining on voronoi for image quality.

\section{Improvement of hedcuter method}
While I technically made two improvements to hedcut, I can only claim one of them was significant. The insignificant one was some compiler optimizations; this ended up not amounting to any consistently measurable improvement, and moreover wasn't a significant revamp to the code, so I would understand considering it poorly. Given that I was working on Windows in the end, and this required some working knowledge of Visual Studio (2013), maybe it's worth something? I ultimately did not try a third attempt due to time- much of which was wasted either trying to get the project to run (copy-paste .dll saved the day) or interviewing CS 110 students one at a time.

I did however make one very significant change that yielded marvelous improvements. There was a point in move-sites where the code was creating a higher resolution version of our source image to do centroid-relocation math in before redirecting the result back to the source image resolution. Being located in move-sites, it was being run for every voronoi cell, every iteration of the algorithm. This struck me as highly inefficient. So I relocated it to just before we began running the algorithm in full- we never change the source image, so the higher res version should never change either. I was initially confused because I was mistakenly attributing the benefits of greater iterations to this change, but I did discover my folly.

\begin{figure}[H]
\includegraphics[scale=0.5]{sunbros-plain-itr1-100000.png}
\includegraphics[scale=0.5]{sunbros-plain-itr3-100000.png}
\caption{Original Code. Left is 1 iteration; Right is 3 iterations. A fair improvement can be seen by increasing the number of iterations.}
\end{figure}

\begin{figure}[H]
\includegraphics[scale=0.5]{sunbros-lessrescale-itr1-100000.png}
\includegraphics[scale=0.5]{sunbros-lessrescale-itr3-100000.png}
\caption{The revised version. As before, left is 1 iteration and right is 3. The images are quite similar to their original counterparts. It is possible the hedcut algorithm gains consistency with increasing iterations- a costly but evidently worthwhile gain.}
\end{figure}

So, No significant changes in the images from this. However, consider for a moment the following tables, indicating run times for various runs using the sunbros.png image for 1 or 3 iterations, before and after the change.

\begin{figure}[H]
\includegraphics[scale=0.8]{hedcut-table.png}
\end{figure}
\begin{figure}[H]
\includegraphics[scale=0.8]{hedcut-results.png}
\end{figure}

A 99 percent improvement! That image rescaling must have been taking a hefty toll to run for so many iterations, seemingly unnecessarily!

After seeing the improvements made here, I decided to accept the negligible impact of the compiler optimizations; this now performs even better than the voronoi implementation in terms of run time.

I did make several other efforts that ultimately were not completed. I made a few attempts to determine the stipple size solely using the voronoi cell area, but concluded that I would have to backtrack a litte further than I was comfortable doing and risk breaking other key parts of the code. I also swapped the random number generator for the uniform gaussian to the included MT19937 generator during the sample point creation. This had no noticeable difference however (pictures omitted because I'm already lost in ths LaTeX thing).

\bibliographystyle{plain}
\bibliography{report}

\end{document}


\documentclass[11pt,a4paper]{report}
\usepackage{amsmath,amsfonts,amssymb,amsthm,epsfig,epstopdf,titling,url,array}
\usepackage{enumitem}
\usepackage{changepage}
\usepackage{graphicx}
\usepackage{caption}
\theoremstyle{plain}
\newtheorem{thm}{Theorem}[section]
\newtheorem{prop}[thm]{Proposition}
\newtheorem*{cor}{Corollary}
\theoremstyle{definition}
\newtheorem{defn}{Definition}[section]
\newtheorem{conj}{Conjecture}[section]
\newtheorem{exmp}{Example}[section]
\newtheorem{exercise}{Exercise}[section]
\theoremstyle{remark}
\newtheorem*{rem}{Remark}
\newtheorem*{lem}{Lemma}
\newtheorem*{note}{Note}
\def\changemargin#1#2{\list{}{\rightmargin#2\leftmargin#1}\item[]}
\let\endchangemargin=\endlist
\begin{document}

\section*{Problem}
a)  Let $x_0 , ... , x_{n-1}$ and $M$ be positive real numbers.  Find real numbers $a_0, ..., a_n$ so that $\sum_{i = 0}^{n-1} x_i a_i$ is maximized subject to the constraint that $\sum_{i = 0}^{n-1} a_i ^2 = M$
\\
b) If you used the definition of the cosine between vectors as part of your solution in a), prove independently that your solutiion is correct (the definition depends on the result being established here).

\section*{Solution (contributed by Juan Carlos Ramirez)}
Consider the equivalent problem (controlling over $a$):
minimize $-x^Ta$
subject to $a^Ta=M$.
Let is relax the constraint to consider the problem
minimize $-a^Tx$
subject to $a^Ta \leq M$.
If we find that the minimum of this relaxed version satisfies $a^Ta=M$ then it is also a minimum of the original problem (since it contains the feasible set for the original problem). The feasible set is a closed ball, so it is compact and since $f_0(a)=-x^Tc is$  linear, it is also continuous and it achieves a minimum in the closed ball (the minimum exists). Let us also note $f_0$ is convex and infinitely differentiable.
Let $f_1(a) = a^Ta-M$. Then $f_1$ is also convex, inifinitely differentiable, and there is a point $a_s=0$ such that $f_1(a_s)<0$. So the problem satisfies Slater's condition, and since $f_1$ is convex, any local minimum of the problem will satisfy KKT conditions (In particular any global minima), i.e.  $(a^*,\lambda^*)$:
$f_1(a^*)<=0$ (Primal Feasibility constraint)
$\lambda^*>=0$ (Dual Feasibility constraint)
$\lambda^* f_1(a^*)=0$ (Complimentary slackness CS)
$\nabla f_0(a^*)= -\lambda^* f_1(a^*)$ (First order condition FOC)
Using the FOC, we see that $\lambda^*$ cannot be 0, since then $-x=0$, but x is a prechosen vector with all positive entries. Therefore, $\lambda^*>0$ and by CS condition $f_1(x^*)=0$ (this gives us that $*(a^*)^Ta^*=M$ and therefore both minimization problems are have the same minimum).
Using FOC again we get
$$ -x=-\lambda^*(2a*)$$

or
$$a^*=x/(2\lambda^*)$$

and therefore
$$(a^*)^Ta^*=x^Tx/(2\lambda^*)^2$$

So
$$(\lambda^*)=\sqrt{x^Tx}/(2\sqrt{M}$$

and
$$a^*=\sqrt{M}x/\sqrt{x^Tx}$$

\end{document}


% Preamble.
\documentclass[12pt]{article}
\usepackage[margin=1.25in]{geometry}
\usepackage[fleqn]{amsmath}
\usepackage{textcomp}
\usepackage{gensymb}
\usepackage{amsfonts}
\usepackage{enumitem}
%\usepackage{tikz}  % Include for figures.
%\usepackage{subfiles}  % Include for subfiles.

%% Title macros.
\newcommand{\HOMEWORKNUM}{17}
\newcommand{\NAME}{D. Choi}
\newcommand{\DATE}{2020-06-18}

\title{\vspace{-2\baselineskip}MATH 225 - Homework \#\HOMEWORKNUM}
\author{\NAME}
\date{\DATE}

%% Formatting options.
%\pagenumbering{gobble}  % Include for single-page document.

% Document.
\begin{document}
\maketitle

\section*{1.}
\textit{Find the matrix that rotates space $30 \degree$ counterclockwise about
the vector $\vec{n} = \begin{pmatrix} 1 \\ 0 \\ -1 \end{pmatrix}$.}
\\[\baselineskip]
The plane normal to $\vec{n}$ can be represented by the equation
\begin{equation*}
	0
	=
	\vec{n} \cdot \begin{pmatrix} x \\ y \\ z \end{pmatrix}
	=
	\begin{pmatrix} 1 \\ 0 \\ -1 \end{pmatrix}
	\cdot
	\begin{pmatrix} x \\ y \\ z \end{pmatrix}
	=
	x - z
	.
\end{equation*}
Let $\vec{v}_1 = \vec{n}$. \\
A vector that is orthogonal to $\vec{v}_1$ can be found on the plane
$0 = x - z$:
\begin{equation*}
	\vec{v}_2 = \begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix}
	.
\end{equation*}
The vector that is orthogonal to both $\vec{v}_1$ and $\vec{v}_2$ can be found
by computing the cross product of $\vec{v}_1$ and $\vec{v}_2$:
\begin{equation*}
	\vec{v}_3 = \vec{v}_1 \times \vec{v}_2 =
	\begin{pmatrix} 0 \\ -2 \\ 0 \end{pmatrix}
	.
\end{equation*}
A convenient basis for describing the rotation can be constructed from
normalizing the orthogonal triple $\vec{v}_1$, $\vec{v}_2$, and $\vec{v}_3$.
Let such a basis be $\mathcal{B}$, where
\begin{equation*}
	\mathcal{B}
	=
	\left\{ \hat{v}_1, \hat{v}_2, \hat{v}_3 \right\}
	=
	\left\{
		\begin{pmatrix}
			\frac{1}{\sqrt{2}} \\
			0 \\
			-\frac{1}{\sqrt{2}}
		\end{pmatrix}
		,
		\begin{pmatrix}
			\frac{1}{\sqrt{2}} \\
			0 \\
			\frac{1}{\sqrt{2}}
		\end{pmatrix}
		,
		\begin{pmatrix}
			0 \\
			-1 \\
			0
		\end{pmatrix}
	\right\}
	,
\end{equation*}
in contrast to the standard basis $\mathcal{S}$, where
\begin{equation*}
	\mathcal{S}
	=
	\left\{
		\begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix},
		\begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix},
		\begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix}
	\right\}
	.
\end{equation*}
\newpage \noindent
In basis $\mathcal{B}$ the rotation of interest is a $30 \degree$
counterclockwise rotation about $\hat{v}_1$. \\
Let ${\{R\}}_{\mathcal{B}}^{\mathcal{B}}$ be the matrix for such a rotation in
basis $\mathcal{B}$ so that
\begin{equation*}
	{\{R\}}_{\mathcal{B}}^{\mathcal{B}}
	=
	\begin{pmatrix}
		1 & 0 & 0 \\
		0 & \cos(30 \degree) & -\sin(30 \degree) \\
		0 & \sin(30 \degree) & \cos(30 \degree)
	\end{pmatrix}
	.
\end{equation*}
Let ${\{I\}}_{\mathcal{B}}^{\mathcal{S}}$ be the matrix with columns
$\hat{v}_1$, $\hat{v}_2$, and $\hat{v}_3$ so that
\begin{equation*}
	{\{I\}}_{\mathcal{B}}^{\mathcal{S}}
	=
	\begin{pmatrix}
		\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} & 0 \\
		0 & 0 & -1 \\
		-\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} & 0
	\end{pmatrix}
	.
\end{equation*}
${\{I\}}_{\mathcal{B}}^{\mathcal{S}}$ can be thought of as the matrix that
converts an input ${\{\vec{x}\}}_{\mathcal{B}}$ (a vector described in basis
$\mathcal{B}$) to ${\{\vec{x}\}}_{\mathcal{S}}$ (the same vector but
described in basis $\mathcal{S}$). \\
Similarly, let ${\{I\}}_{\mathcal{S}}^{\mathcal{B}}$ be the matrix that
converts from basis $\mathcal{S}$ to $\mathcal{B}$, so that
\begin{equation*}
	{\{I\}}_{\mathcal{S}}^{\mathcal{B}}
	=
	{\left( {\{I\}}_{\mathcal{B}}^{\mathcal{S}} \right)}^{-1}
	=
	{\left( {\{I\}}_{\mathcal{B}}^{\mathcal{S}} \right)}^{\text{T}}
	=
	\begin{pmatrix}
		\frac{1}{\sqrt{2}} & 0 & -\frac{1}{\sqrt{2}} \\
		\frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \\
		0 & -1 & 0
	\end{pmatrix}
	.
\end{equation*}
Let ${\{R\}}_{\mathcal{S}}^{\mathcal{S}}$ be the rotation of interest in
standard basis. \\
${\{R\}}_{\mathcal{S}}^{\mathcal{S}}$ can be computed as the composition of
firstly the transformation from basis $\mathcal{S}$ to $\mathcal{B}$, then the
rotation of interest under basis $\mathcal{B}$, and finally the transformation
from basis $\mathcal{B}$ back to $\mathcal{S}$:
\begin{align*}
	{\{R\}}_{\mathcal{S}}^{\mathcal{S}}
	&=
	{{\{I\}}_{\mathcal{B}}^{\mathcal{S}}}
	\cdot
	{{\{R\}}_{\mathcal{B}}^{\mathcal{B}}}
	\cdot
	{{\{I\}}_{\mathcal{S}}^{\mathcal{B}}}
	\\
	&=
	\boxed{
		\begin{pmatrix}
			\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} & 0 \\
			0 & 0 & -1 \\
			-\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} & 0
		\end{pmatrix}
		\begin{pmatrix}
			1 & 0 & 0 \\
			0 & \cos(30 \degree) & -\sin(30 \degree) \\
			0 & \sin(30 \degree) & \cos(30 \degree)
		\end{pmatrix}
		\begin{pmatrix}
			\frac{1}{\sqrt{2}} & 0 & -\frac{1}{\sqrt{2}} \\
			\frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \\
			0 & -1 & 0
		\end{pmatrix}
	}
	.
\end{align*}

\section*{2.}
\textit{Let $\vec{a} = \begin{pmatrix} 5 \\ 0 \\ 0 \end{pmatrix}$ and
$\vec{b} = \begin{pmatrix} 0 \\ 3 \\ 4 \end{pmatrix}$. \\
Find the vector $\vec{v}$ that is $\vec{a}$ rotated $40 \degree$ toward
$\vec{b}$.} \\[\baselineskip]
Note that $\vec{a} \cdot \vec{b} = 0$, meaning that the two vectors are
orthogonal. \\
A vector $\vec{c}$ in the axis of rotation can be found by computing
\begin{equation*}
	\vec{c}
	=
	\vec{a} \times \vec{b}
	=
	\begin{pmatrix} 0 \\ -20 \\ 15 \end{pmatrix}
	.
\end{equation*}
Let the basis $\mathcal{B}$ be constructed from the triple $\vec{a}$,
$\vec{b}$, and $\vec{c}$:
\begin{equation*}
	\mathcal{B}
	=
	\left\{ \vec{a}, \vec{b}, \vec{c} \right\}
	=
	\left\{
		\begin{pmatrix} 5 \\ 0 \\ 0 \end{pmatrix},
		\begin{pmatrix} 0 \\ 3 \\ 4 \end{pmatrix},
		\begin{pmatrix} 0 \\ -20 \\ 15 \end{pmatrix}
	\right\}
	.
\end{equation*}
Let ${\{R\}}_{\mathcal{B}}^{\mathcal{B}}$ be the matrix for the rotation of
interest about $\vec{c}$ in basis $\mathcal{B}$:
\begin{equation*}
	{\{R\}}_{\mathcal{B}}^{\mathcal{B}}
	=
	\begin{pmatrix}
		\cos(40 \degree) & -\sin(40 \degree) & 0 \\
		\sin(40 \degree) & \cos(40 \degree) & 0 \\
		0 & 0 & 1
	\end{pmatrix}
	.
\end{equation*}
Using ${\{R\}}_{\mathcal{B}}^{\mathcal{B}}$,
\begin{gather*}
	{\{\vec{v}\}}_{\mathcal{B}}
	=
	{\{R\}}_{\mathcal{B}}^{\mathcal{B}}
	{\{\vec{a}\}}_{\mathcal{B}}
	=
	{\{R\}}_{\mathcal{B}}^{\mathcal{B}}
	\begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix}
	=
	\begin{pmatrix}
		\cos(40 \degree) \\
		\sin(40 \degree) \\
		0
	\end{pmatrix}
	,
	\\
	\vec{v}
	=
	\cos(40 \degree)
	\begin{pmatrix} 5 \\ 0 \\ 0 \end{pmatrix}
	+
	\sin(40 \degree)
	\begin{pmatrix} 0 \\ 3 \\ 4 \end{pmatrix}
	+
	0
	\begin{pmatrix} 0 \\ -20 \\ 15 \end{pmatrix}
	=
	\boxed{
		\begin{pmatrix}
			5 \cos(40 \degree) \\
			3 \sin(40 \degree) \\
			4 \sin(40 \degree)
		\end{pmatrix}
	}
	.
\end{gather*}

\end{document}


\chapter{Permutation representations}
\section {Basic Results}
{\bf Lemma 1:}
Let $H \le G$ and $Hg_1 , ..., Hg_n$ be the cosets; the map
$\pi(g): \langle Hg_1, ..., Hg_n \rangle \mapsto \langle Hg_1 g, ..., Hg_n g \rangle $
is a map from $G$ to $\Sigma_n$
whose kernel is the largest normal subgroup of $G$ in $H$; in fact,
$ker(\pi) = \bigcap_{i=1}^n H^{x_i}$.
\\
\\
{\bf Corollary:}  If $G$ is simple and $G > H$ with $|G:H| = k$ then
$|G| \mid k!$.
\section {Imprimitivity}
{\bf Definition 1:}
A \emph{system of imprimitivity for permutation
group $G$:} is a set, ${\cal B} = \{ \Delta_i\}$ $|\Delta_i| > 1$, with
the property that for $\Delta \in {\cal B}, g \in G$ either
$\Delta \cap g \Delta = \emptyset$ or
$\Delta = g \Delta$.
\\
\\
{\bf Theorem: 1}
If $H<G$ and $G$ is simple then $|G| \mid |G:H|!$.
\begin{quote}
\emph{Proof:}
Let  $\pi: G \rightarrow Perm(G/H)$ be the map from $G$ to the permutations on the right
cosets of $H$ in $G$.  $G/ker(\pi)$ is an injection into $S_{|G:H|}$ and $ker(\pi)=1$ so
$|G| \mid |G:H|!$.
\end{quote}
{\bf Definition 2:}
A permutation is \emph{primitive} if there is no non-trivial set of imprimitivity.
\\
\\
{\bf Definition 3:}
$\Gamma$ is $G$ invariant if $\Gamma^G= \Gamma$ so $\Gamma$ is a union of $G$ orbits.
$G/G_{\Gamma} \equiv G^{\Gamma}$.
\\
\\
{\bf Normal Subgroups of Primitive Groups Theorem:}
Let $G$ be primitive on $\Omega$ and $1 \ne N \lhd G$.  Then either $N \subseteq G_a$ or
$N$ acts transitively on $\Omega$.  If the action is regular, $N$ is a minimal normal subgroup.
\begin{quote}
\emph{Proof:}
Let $\Delta(a)= Na$.  If $\Delta(a)= \{ a \}$, $N \subseteq G_a$ and we're done.
Suppose $a \ne b \in Na$.  Since $G$ is transitive, $\exists g \in G: ga=b$ so
$g \Delta(a)= gNa= (gN g^{-1} ) ga= Nb = \Delta(b)$.  But $b \in Na$ so $Nb = Na$.
Thus $\Delta(a)$ is an imprimitive block.  Since $G^{\Omega}$ is primitive, $\Delta(a)= \Omega$
and $N$ is transitive.
\end{quote}
{\bf Iwasawa's Theorem:}  Let $G$ be a primitive permutation group and suppose (1) $G'=G$ and
(2) $\exists A$ with $A$ solvable  and $A \lhd G_a: G=
\langle A^G \rangle $ then $G$ is simple.
\begin{quote}
\emph{Proof:}
Let $1 < N \lhd G$.  Since $N$ is transitive by the previous result, $NG_a=G$.  Further,
$G_a \subseteq N_G(NA)$ and $N \subseteq N_G(NA)$ and so $G= NG_a= N_G(NA)$. So $NA \lhd G$ and
$(NA)^G =NA \lhd G$ but $ \langle A^G \rangle = G$ so $NA=G$.
Since $A$ is solvable, so is $G/N$ and
$(G/N)'=(G/N)$ and so $N = G$.
\end{quote}
{\bf Theorem 2:}
If $\Delta \subseteq \Omega$ and $\alpha \in \Omega$ then
$\psi( \alpha ) = \bigcap_{\alpha \in g \Delta} g \Delta$ is a block of a transitive
group $G \subseteq Sym(\Omega)$.
\begin{quote}
\emph{Proof:}
Let $\beta= h \alpha$ and
note that
$h \psi( \alpha ) = \bigcap_{h \alpha \in hg \Delta} hg \Delta= \psi( \beta )$ so
$ |\psi( \alpha )|= |\psi( \beta )| $.   Now suppose $\beta \in \psi( \alpha )$ then
$\alpha \in g \Delta \rightarrow \beta \in g \Delta$ so
$\bigcap_{\beta \in g \Delta} g \Delta \supseteq \bigcap_{\alpha \in g \Delta} g \Delta$ and
so, $\psi( \beta ) = \psi ( \alpha )$ since they both have the same number of elements.  We have
shown $\beta \in \psi( \alpha ) \rightarrow  \psi( \beta ) = \psi( \alpha )$ and thus
$\psi( \alpha ) \cap \psi( \beta ) = \emptyset$  or
$\psi( \alpha ) = \psi( \beta )$.
\end{quote}
{\bf Theorem 3:}
$G$ is primitive iff $G_{\alpha}$ is maximal.
A transitive group is imprimitive iff $\exists Z$: $G_{\alpha} < Z < G$.
\begin{quote}
\emph{Proof:}
Suppose $b \ne a$.  Suppose $a, b \in \Delta_i$ with $\{ \Delta_i \}$ a set of
imprimitivity and suppose $g a= b$.  Then $g \Delta_i = \Delta_i$ and
$ \langle g, G_a \rangle > G_a$
stabilises the block containing $a$.  So if $G_a$ is not primitive then $G_a$ is not maximal.
If $G > M > G_a$ and $M$ is maximal, $Ma$ is a set of imprimitivity so $G$ is not primitive.
\end{quote}
{\bf Definition 4:}
$G$ acts \emph{regularly} on $\Omega$ if $\forall \alpha, \beta \in \Omega,
\exists ! g: \alpha^g= \beta$.
\\
\\
{\bf Theorem 4:}  Let $G$ be $n-$fold transitive on $\Omega$, $n \ge 2$ and $N$ a regular
normal subgroup of $G$ then $n \le 4$ and
(1) If $n=2$, $N$ is an elementary abelian $p-$group;
(2) If $n=3$, $N$ is an elementary abelian $2-$group or $N=C_3$ and $G=S_3$;
(3) If $n=4$, $N= C_2 \times C_2$ and $G=S_4$.
\begin{quote}
\emph{Proof:}
Let $n \geq 2$ and $\alpha \in \Omega$.  $G_{\alpha}$ is $(n-1)$-fold transitive on $\Omega - \{ \alpha \}$ and
hence on $N^{\#}$ so $\exists g: x^g=y, x,y \in N^{\#}$.  Thus every element of $N^{\#}$ has the same order, $p$.
$N= {\mathbb Z}(N)$ and $N$ is abelian (and hence elementary abelian).
If $n \geq 3$, $3 \leq |N|=|\Omega|$.  If $n=3$, $G=S_3$.  If $p \geq 3$, $x_1 \neq x_1^{-1}=x_2$.  Let $x_3$ be another
element.  Then $x_1^g= x_2, x_2^g=x_3$ for some $g$ and this is contradictory so $p=2$.
If $n \geq 4$, $N$ is an elementary abelian $2$-group.  Let $U= \langle x_1 \rangle \times \langle x_2 \rangle$.
If $N \neq U$, put $x_3=x_1 x_2$ and choose $x_4 \notin U$.  Then $\exists g \in G:$
$x_1^g= x_1$,
$x_2^g= x_2$, and
$x_3^g= x_4$ which is again contradictory.
\end{quote}
{\bf Definition 5:}
Define ${\cal G}(G, \Omega )$ as the graph of $G$ acting on $\Omega$ as follows:
$G$ acts on $\Omega \times \Omega$.  Diagonal orbital is
$\Delta_1 = \{ (\alpha , \alpha )\}$. If $\Delta = \{ (\alpha , \beta ) \}$,
$\Delta^*= \{ (\beta, \alpha) \}$.  Self paired if $\Delta^* = \Delta$.
$\Delta(\alpha)= \{ \beta : (\alpha , \beta ) \in \Delta \}$ --- corresponds to
orbits of $G_{\alpha}$. The \emph{rank of the permutation group} is number of orbitals.
$\Delta^p= \{ (y,x): (x, y) \in \Delta \}$ is called the \emph{paired orbit}.
\\
\\
{\bf Theorem 5:}
On a self-paired orbit $\Delta$, the graph ${\cal G}=(G, X,\Delta)$ is
symmetric and $G$ is transitive on edges.
\begin{quote}
\emph{Proof:}
Since $\Delta$ is self-paired, the graph is symmetric.
\end{quote}
{\bf Theorem 6:}
$G$ is primitive iff ${\cal G}$ is connected.
\begin{quote}
\emph{Proof:}
$G$ is maximal so $G= \langle G_x , G_y \rangle , x \ne y$.
\end{quote}
{\bf Definition 6:}
A transitive permutation group is \emph{regular} if $|X|= |G^X|$ or, equivalently
$|G_x|=1, \forall x \in X$ and $G^X$, transitive.
\\
\\
{\bf Theorem 7:}
Let $X$ be a faithful primitive $G-set$ with $G_x$ simple.  Then either $G$ is simple or
every non-trivial normal subgroup $H$ of $G$ is a regular normal subgroup.
\begin{quote}
\emph{Proof:}
Suppose $1 \ne H \lhd G$.  Since $G$ is primitive, $H$ is transitive.  $H_x= H \cap G_x \lhd G_x$
so, since $G_x$ is simple,  $H_x = 1$ of $H_x= G_x$.  If $H_x=1$, $H$ is a regular normal subgroup
of $G$.  So $H_x = G_x$ and since $H$ is transitive, $|H:H_x|= |G:G_x|$, so $G$ is simple.
\end{quote}
{\bf Definition 7:} $A$ acts \emph{semi-regularly} on $G$ if $C_G(a)=1, \forall a \in A^{\#}$.
\\
\\
{\bf Theorem 8:}
Suppose $A$ acts semi-regularly
on $G$.  Then (1) $|G| = 1 \jmod{|A|}$, (2) $A$ is semi-regular on each $A-$invariant
subgroups factor group of $G$, (3) $\forall p \in \pi(G)$, $\exists! A-$invariant
Sylow $p-$subgroup of $G$, (4) $\forall a \in A, g \mapsto [g,a]$ is a permutation of
$G$, (5) if $2 | |A|, \exists t: |t|=2, t \in A: g^t = g^{-1}, g \in G$ and
$G^{(1)}=1$.
\begin{quote}
\emph{Proof:}
(1) each orbit of $A$ on $G^{\#}$ has length $|A|$.  (2) is clear (second part comes from coprime action).
(3) By Schur, the set of $A$-invariant Sylow $p$-groups is non-empty.  Since $C_G(A)= 1$ is transitive on this
set, the subgroup is unique. (4) $[g,a]= [h,a]$ iff $h g^{-1} \in C_G(a)$.  The commutator map is already an
injection and since $G$ is finite, it is a bijection. (5)  Let $t \in Inv(A)$ and $g \in G$.  By previous result,
$g= [h,t]$ for some $h \in G$ and $g^t = (h^{-1} h^t)^t= g^{-1}$.  Therefore $x^t =x^{-1}$ and $G$ is abelian.
Finally, if $s \in Inv(A)$, $s$ inverts $G$ so $st \in C_A(G)=1$ and $t$ is unique.
\end{quote}
{\bf Theorem 9:}
Let $\Delta$ be an orbit of $G$ and let $\delta \in \Delta$.  For each $\gamma \in \Delta$
let $v(\gamma ) \in G$ be such that $\delta \mapsto \gamma$.  Finally, suppose $S$
generates $G$.
Then $G_{\delta}= \langle v(\gamma)sv(\gamma^{s})^{-1} | \gamma \in \Delta, s \in S \rangle$.
\begin{quote}
\emph{Proof:}
See, Stellmacher.
\end{quote}
\section {Fixed point free automorphisms}
{\bf Fixed Point Free:}  An automorphism, $\phi$ acting on $G$ is fixed point free if $C_G(\phi) =1$.
\\
\\
{\bf Lemma 1:} Let $\phi$ be a fixed point free automorphism acting on $G$ with $|\phi|=n$.  Then
(1) $y \in G$ then $y= x^{-1}(x \phi)$ for some $x$, and
(2) $\forall x \in G, x(x \phi) (x \phi^2) \ldots (x \phi^{n-1}) = 1$.
\begin{quote}
\emph{Proof:}
$x^{-1}(x\phi) = y^{-1}(y\phi)$ $\rightarrow$ $y x^{-1} = (y x^{-1})\phi$, so $x=y$.  Thus
$|{x^{-1}(x\phi): x \in G}| = |G|-1$ and (1) holds. Since $\exists y: x= y^{-1}(y\phi)$, so
$x(x \phi) (x \phi^2) \ldots (x \phi^{n-1}) =
y^{-1}(y \phi) (y \phi^2) \ldots (y \phi^{n-1})^{n-1}) = y^{-1}y = 1$.
\end{quote}
{\bf Lemma 2:} Let $\phi$ be a fixed point free automorphism acting on $G$ then $\phi$ leaves a unique
$S_p$subgroup of $G$ invariant.
\begin{quote}
\emph{Proof:}
Let $Q \in S_p(G), (Q)\phi = y^{-1}Qy$.  $y^{-1}= (z\phi)z^{-1}$ and $y = z(z^{-1}]phi)$.
$Q\phi = (z\phi) (z^{-1}Q z) (z^{-1}\phi)$ and $(z^{-1}Qz)\phi= z^{-1}Qz$ and thus
$\phi$ leaves $P=z^{-1}Qz$ fixed.  If both $P, Q$ are $\phi$-invariant and
$Q=x^{-1}Px$ then
$Q=(x^{-1}\phi)P(x(phi)$ and $(x\phi)x^{-1} \in N(P) = N$ but $N$ is $\phi$-invariant and
$\phi$ is fixed point free on $N$ so $\exists z: y=(z\phi)z^{-1}$.
$(z\phi)z^{-1}=
(x\phi)x^{-1}$ $\rightarrow$ $x=z$.
\end{quote}
{\bf Lemma 3:} Let $\phi$ be a fixed point free automorphism acting on $G$, $H \lhd G$ and $H= H\phi$ then
$\phi$ is fixed point free on $G/H$.
\begin{quote}
\emph{Proof:}
Let ${\overline G} = G/H$ and ${\overline x} = {\overline x}\phi$.  ${\overline x}^{-1}({\overline x}\phi)$
so $y= x^{-1}(x\phi) \in H$.  Since $\phi$ induces a fixed point free automorphism on $H$, $y= z^{-1}(z\phi)$.
So $x = z$, $x \in H$ and ${\overline x} = 1$.  Thus $\phi$ induces a fixed point free automorphism on
${\overline G}$.
\end{quote}
{\bf Lemma 4:} Let $\phi$ be a fixed point free automorphism acting on $G$ of order $2$, then $G$ is abelian.
\begin{quote}
\emph{Proof:}
Since $x(x\phi) = 1$, $x^{-1} = (x\phi)$.  If
$x, y \in G$, $y^{-1}x^{-1} = (xy)^{-1}= ((x\phi)(y\phi))^{-1} = x^{-1} y^{-1}$, so $G$ is abelian.
\end{quote}
{\bf Lemma 5:} Let $\phi$ be a fixed point free automorphism acting on $G$ of order $3$, then $G$ is
nilpotent and $[x, x\phi]=1, \forall x \in G$.
\begin{quote}
\emph{Proof:}
$x(x\phi)(x\phi^2)=1$, so $x(x\phi)= (x\phi^2)^{-1}$ and $x$ commutes with $x\phi$ for any $x \in G$.
Let $P$ be the unique $\phi$-invariant $S_p$ subgroup of $G, \forall p \in \pi(G)$.  We show
$P \lhd G$.
Suppose not and
$Q \ne P \in S_p(G)$ and pick $x \in Q \setminus P$, put $H=\langle x, x\phi \rangle$.
$H$ is a $p$-group since $[x, x\phi]=1$ and $H'=1$ and $H \phi = H$.  So $H \subseteq P$ but then
$x \in P$.
\end{quote}
{\bf Thompson:}
Let $G$ be a transitive permutation group on $X$ and $1 \ne g \in G$ fixes
no more than one element then $N= \{g: X_g= \emptyset \}$ is a normal subgroup of $G$.
Thompson showed any finite group having a fixed point free automorphism of prime order is nilpotent.
\begin{quote}
\emph{Proof:}
Let $G$ be a counterexample of minimal order and $\phi$ be a fixed point free automorphism of prime order $r$.
$G$ has a proper normal subgroup, $H \ne 1$ with $H\phi=H$.  By induction, $H$ is nilpotent and $\phi$
is a fixed point free automorphism on $G/H$ so $G/H$ is nilpotent and $G$ is solvable.  If $G$ has no non-trivial
normal subgroup, which is $H$-invariant.  Let $P \in S_p(G), p \ne 2$ with $P\phi=P$.  Put
$N= N_G({\mathbb Z}(J(P)))$, $ {\mathbb Z}(J(P))) \; char \; P$.  If $N < G$, $N$ is nilpotent:
$N$ has a normal $p$-complement, $K$.  By the Thompson $p$-complement theorem, $G=KP$, $K\phi = K$ so
$K = 1$ and $P=G$ so $G$ is nilpotent.  We may assume $G$ is solvable.
Suppose $H_1, H_2 \lhd G$ $H_i\phi = H_i$, $H_1 \cap H_2 = 1$.  ${\overline G_i} = G/H_i$ is nilpotent
so ${\overline L} = {\overline G_1} \times {\overline G_2}$ is too.  $x\phi = (H_1x, H_2x)$.
$\psi: G \rightarrow {\overline G_1} \times {\overline G_2}$.  $G\psi$ is nilpotent.
Let $N$ a minimal normal subgroup of $G$, $N$ is elementary abelian and ${\overline G} = G/N$ is nilpotent.
${\overline G}$ is not a $p$-group.  Let ${\overline Q} \in S_q({\overline G})$ with
${\overline Q} = {\overline Q} \phi$, $q \ne p$.  Let ${\overline M}$ be a minimal $\phi$-invariant
subgroup of $\Omega_1({\mathbb Z}({\overline Q}))$.  ${\overline M} \ne 1$ and ${\overline M} \lhd {\overline G}$
since ${\overline G}$ is nilpotent.  Let $H$ be the inverse image of ${\overline M}$ then
$H=NM$ and $M$ is a non-trivial elementary abelian $q$-group.  $H \lhd G$, $H\phi = H$ and $M\phi=M$.
$\phi$ acts irreducibly on $M$.  $H \subset G$ then $H$ is nilpotent.
$M \; char \; H \lhd G$ and $M$ and $N$ are two minimal normal $\phi$-invariant subgroup and
$M \cap N \ne 1$.  $C_M(N)\phi = C_M(N)$ since $\phi$ acts irreducibly on $M$.  Either
$C_M(N) = 1$ or $C_M(N) = N$.
If $C_M(N) = N$, $G$ is nilpotent so $C_M(N) = 1$.  Let $G^*$ be the semidirect product of
$M$ by $\langle \phi \rangle$.  $G^*$ acts irreducibly on $N$ as a vector space since
$C_M(N) = 1$ and $\phi$ is fixed point free.  But $G^*$ is a $p'$-group $C_{M^*}(M) = M$ and
$G^*/M$ has order $q$ so $C_N(\phi) \ne 1$.
\end{quote}
{\bf O-Nan-Scott:}  Let $G$ be a finite primitive permutation group of degree $n$ and
$H=soc(G)$.  Then either (1) $H$ is a regular elementary abelian $p$ group for
some $p$ and $G$ is isomorphic to a subgroup of
$AGL_m(P)$ ; or, (2) $H$ is isomorphic to $T^m$ where $T$ is a non-abelian simple group
with a bunch of conditions.
\section {Mathieu groups are simple}
We will use one result from a future section here.
\\
\\
{\bf Theorem 1:}
Let $N$ be a finite group $G \subseteq Aut(N)$.  Then $G$ acts as a permutation group on $N^{\#}$.  Further, \\
(i) If $G$ is transitive, then $N$ is an elementary abelian $p$-group.
\\
(ii) If $G$ is $2$-transitive, then $N$ is either an elementary abelian $2$-group or $|N|=3$.
\\
(iii) If $G$ is $3$-transitive, then $|N|=4$.
\\
(iv) $G$ cannot be $4$-transitive.
\\
\begin{quote}
\emph{Proof:}  These are all clear.
\end{quote}
{\bf Theorem 2:}
Let $G$ be an transitive permutation group and $N \lhd G$.  Then $N$ is ${\frac 1 2}$ transitive.  If
$N \ne 1$ and $G$ is primitive then $N$ is transitive.
\begin{quote}
\emph{Proof:}
Let $B$ be an orbit of $N$ of minimal length. $B^{Ng}= B^{gN} = B^g$, so $B^g$ is a union of orbits of $N$.
By minimality of $|B|$, $B^g$ is a single $N$ orbit.  These form a system of imprimitivity.
\end{quote}
{\bf Theorem 3:}
Let $G$ be primitive with no regular normal subgroups.  If $G_a$ is simple then $G$ is simple.
\begin{quote}
\emph{Proof:}
Let $N \lhd G$.  $N_a \leq (N \cap G_a) \lhd G_a$ so either $N_a =1$ or $N=G_a$.  If $N_a=1$, $N$
is a regular normal subgroup.
\end{quote}
{\bf Theorem 4:}
Let $G$ be $m$-transitive on $A$, $|A|=n$ with a regular normal subgroup, $N$.\\
(i) if $m=2$, then $n = |N|= p^k$
\\
(ii) if $m=3$, then $n = 3$ or $n = 2^k$ or $n = 3$.
\\
(iii) if $m=4$, then $n = 4$.
\\
(iv) We cannot have $m>4$.
\begin{quote}
\emph{Proof:}
\end{quote}
{\bf Theorem 5:}  Let $G$ be transitive and $H < G$ also be transitive.  The $G= H G_a$
\begin{quote}
\emph{Proof:}
Clear.
\end{quote}
{\bf Theorem:}  The Mathieu groups are simple.
\begin{quote}
\emph{Proof:}
\\
Step 1: $M_{11}$ is simple.\\
\\
Put $G = M_{11}$ so $|G| = 11 \cdot 10 \cdot 9 \cdot 8$.  Let $P=\langle x\rangle$ be a subgroup of order
$11$. $P$ acts transitively on $\Omega$.  If $P \subseteq A$, $A$, abelian, then $A$ is transitive and regualr so
$A=P$.  Thus $C_G(P)=P$.  Since $|Aut(P)|=10$, $|N_G(P)/P| \mid 10$.
If $2 \mid N_G(P)$ let $y \in N_G(P), |y|=2$. $y$ has a fixed point since $11$ is odd and $x^y = x^{-1}$.  Thus
$y$ is a product of $5$ transpositions thus $y \notin A_{11}$ which is a contradiction.  So $|N_G(P)/P|$ is $1$ or $5$.
Let $1 \ne H \lhd G, H \ne G$.  Since $G$ is primitive, $H$ is transitive and $11 \mid |H|, P \subseteq H$.
$G= H G_a$ and $G_a \subseteq N_G(P)$ so $G= H N_G(P)$ and $N_G(P) \nsubseteq H$ and $N_H(P)=P$.  So, in $H$,
$P$ is in the center of its normalizer and $H$ has a normal $11$ complement, $K$.  $K \lhd G$ and $11 \nmid |K|$ so
$K = 1$.  Thus $H=K$ and $|G|=55$, which is a contradiction.
\\
Step 2:  $PSL_3(4)$ is simple. This was already shown.
\\
\\
Step 3:
By the above result, $M_{12}, M_{22}, M_{23}, M_{24}$ cannot ahve a regular normal subgroup.
These groups are all $3$-transitive.  $M_{11} = (M_{12})_a$ so $M_{12}$ is simple.
$PSL_3(4)  = (M_{22})_a$, $M_{22} = (M_{23})_a$, and $M_{23} = (M_{24})_a$ proving their simplicity.
\end{quote}