\chapter{Supervised learning methods}

Note the supervised learning methods described in this chapter all require Numerical Python (numpy) to be installed.

\section{The Logistic Regression Model}
\label{sec:LogisticRegression}

\subsection{Background and Purpose}

Logistic regression is a supervised learning approach that attempts to distinguish $K$ classes from each other using a weighted sum of some predictor variables $x_i$. The logistic regression model is used to calculate the weights $\beta_i$ of the predictor variables. In Biopython, the logistic regression model is currently implemented for two classes only ($K = 2$); the number of predictor variables has no predefined limit.

As an example, let's try to predict the operon structure in bacteria. An operon is a set of adjacent genes on the same strand of DNA that  are transcribed into a single mRNA molecule. Translation of the single mRNA molecule then yields the individual proteins. For {\it Bacillus subtilis}, whose data we will be using, the average number of genes in an operon is about 2.4.

As a first step in understanding gene regulation in bacteria, we need to know the operon structure. For about 10\% of the genes in {\it Bacillus subtilis}, the operon structure is known from experiments. A supervised learning method can be used to predict the operon structure for the remaining 90\% of the genes.

For such a supervised learning approach, we need to choose some predictor variables $x_i$ that can be measured easily and are somehow related to the operon structure. One predictor variable might be the distance in base pairs between genes. Adjacent genes belonging to the same operon tend to be separated by a relatively short distance, whereas adjacent genes in different operons tend to have a larger space between them to allow for promoter and terminator sequences. Another predictor variable is based on gene expression measurements. By definition, genes belonging to the same operon have equal gene expression profiles, while genes in different operons are expected to have different expression profiles. In practice, the measured expression profiles of genes in the same operon are not quite identical due to the presence of measurement errors. To assess the similarity in the gene expression profiles, we assume that the measurement errors follow a normal distribution and calculate the corresponding log-likelihood score.

We now have two predictor variables that we can use to predict if two adjacent genes on the same strand of DNA belong to the same operon:
\begin{itemize}
\item $x_1$: the number of base pairs between them;
\item $x_2$: their similarity in expression profile.
\end{itemize}

In a logistic regression model, we use a weighted sum of these two predictors to calculate a joint score $S$:
\begin{equation}
S = \beta_0 + \beta_1 x_1 + \beta_2 x_2.
\end{equation}
The logistic regression model gives us appropriate values for the parameters $\beta_0$, $\beta_1$, $\beta_2$ using two sets of example genes:
\begin{itemize}
\item OP: Adjacent genes, on the same strand of DNA, known to belong to the same operon;
\item NOP: Adjacent genes, on the same strand of DNA, known to belong to different operons.
\end{itemize}

In the logistic regression model, the probability of belonging to a class depends on the score via the logistic function. For the two classes OP and NOP, we can write this as
\begin{eqnarray}
\Pr(\mathrm{OP}|x_1, x_2) & = & \frac{\exp(\beta_0 + \beta_1 x_1 + \beta_2 x_2)}{1+\exp(\beta_0 + \beta_1 x_1 + \beta_2 x_2)} \label{eq:OP} \\
\Pr(\mathrm{NOP}|x_1, x_2) & = & \frac{1}{1+\exp(\beta_0 + \beta_1 x_1 + \beta_2 x_2)} \label{eq:NOP}
\end{eqnarray}
Using a set of gene pairs for which it is known whether they belong to the same operon (class OP) or to different operons (class NOP), we can calculate the weights $\beta_0$, $\beta_1$, $\beta_2$ by maximizing the log-likelihood corresponding to the probability functions (\ref{eq:OP}) and (\ref{eq:NOP}).

\subsection{Training the logistic regression model}
\label{subsec:LogisticRegressionTraining}

\begin{table}[h]
\begin{center}
\caption{Adjacent gene pairs known to belong to the same operon (class OP) or to different operons (class NOP). Intergene distances are negative if the two genes overlap.}
\begin{tabular}{|c|c|c|c|}
\hline
Gene pair & Intergene distance ($x_1$) & Gene expression score ($x_2$) & Class\\
\hline
{\it cotJA} --- {\it cotJB} & -53 & -200.78 & OP\\
{\it yesK} --- {\it yesL} & 117 & -267.14 & OP\\
{\it lplA} --- {\it lplB} & 57 & -163.47 & OP\\
{\it lplB} --- {\it lplC} & 16 & -190.30 & OP\\
{\it lplC} --- {\it lplD} & 11 & -220.94 & OP\\
{\it lplD} --- {\it yetF} & 85 & -193.94 & OP\\
{\it yfmT} --- {\it yfmS} & 16 & -182.71 & OP\\
{\it yfmF} --- {\it yfmE} & 15 & -180.41 & OP\\
{\it citS} --- {\it citT} & -26 & -181.73 & OP\\
{\it citM} --- {\it yflN} & 58 & -259.87 & OP\\
{\it yfiI} --- {\it yfiJ} & 126 & -414.53 & NOP\\
{\it lipB} --- {\it yfiQ} & 191 & -249.57 & NOP\\
{\it yfiU} --- {\it yfiV} & 113 & -265.28 & NOP\\
{\it yfhH} --- {\it yfhI} & 145 & -312.99 & NOP\\
{\it cotY} --- {\it cotX} & 154 & -213.83 & NOP\\
{\it yjoB} --- {\it rapA} & 147 & -380.85 & NOP\\
{\it ptsI} --- {\it splA} & 93 & -291.13 & NOP \\
\hline
\end{tabular}
\label{table:training}
\end{center}
\end{table}

Table \ref{table:training} lists some of the {\it Bacillus subtilis} gene pairs for which the operon structure is known.
Let's calculate the logistic regression model from these data:

%NOTE - can't use this as a doctest in case NumPy is missing (Jython).
%Also the ... are missing but that means the user can copy/paste the example.
\begin{verbatim}
>>> from Bio import LogisticRegression
>>> xs = [[-53, -200.78],
          [117, -267.14],
          [57, -163.47],
          [16, -190.30],
          [11, -220.94],
          [85, -193.94],
          [16, -182.71],
          [15, -180.41],
          [-26, -181.73],
          [58, -259.87],
          [126, -414.53],
          [191, -249.57],
          [113, -265.28],
          [145, -312.99],
          [154, -213.83],
          [147, -380.85],
          [93, -291.13]]
>>> ys = [1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          0,
          0,
          0,
          0,
          0,
          0,
          0]
>>> model = LogisticRegression.train(xs, ys)
\end{verbatim}

Here, \verb+xs+ and \verb+ys+ are the training data: \verb+xs+ contains the predictor variables for each gene pair, and \verb+ys+ specifies if the gene pair belongs to the same operon (\verb+1+, class OP) or different operons (\verb+0+, class NOP). The resulting logistic regression model is stored in \verb+model+, which contains the weights $\beta_0$, $\beta_1$, and $\beta_2$:

\begin{verbatim}
>>> model.beta
[8.9830290157144681, -0.035968960444850887, 0.02181395662983519]
\end{verbatim}

Note that $\beta_1$ is negative, as gene pairs with a shorter intergene distance have a higher probability of belonging to the same operon (class OP). On the other hand, $\beta_2$ is positive, as gene pairs belonging to the same operon typically have a higher similarity score of their gene expression profiles.
The parameter $\beta_0$ is positive due to the higher prevalence of operon gene pairs than non-operon gene pairs in the training data.

The function \verb+train+ has two optional arguments: \verb+update_fn+ and \verb+typecode+. The \verb+update_fn+ can be used to specify a callback function, taking as arguments the iteration number and the log-likelihood. With the callback function, we can for example track the progress of the model calculation (which uses a Newton-Raphson iteration to maximize the log-likelihood function of the logistic regression model):

\begin{verbatim}
>>> def show_progress(iteration, loglikelihood):
        print("Iteration:", iteration, "Log-likelihood function:", loglikelihood)
>>>
>>> model = LogisticRegression.train(xs, ys, update_fn=show_progress)
Iteration: 0 Log-likelihood function: -11.7835020695
Iteration: 1 Log-likelihood function: -7.15886767672
Iteration: 2 Log-likelihood function: -5.76877209868
Iteration: 3 Log-likelihood function: -5.11362294338
Iteration: 4 Log-likelihood function: -4.74870642433
Iteration: 5 Log-likelihood function: -4.50026077146
Iteration: 6 Log-likelihood function: -4.31127773737
Iteration: 7 Log-likelihood function: -4.16015043396
Iteration: 8 Log-likelihood function: -4.03561719785
Iteration: 9 Log-likelihood function: -3.93073282192
Iteration: 10 Log-likelihood function: -3.84087660929
Iteration: 11 Log-likelihood function: -3.76282560605
Iteration: 12 Log-likelihood function: -3.69425027154
Iteration: 13 Log-likelihood function: -3.6334178602
Iteration: 14 Log-likelihood function: -3.57900855837
Iteration: 15 Log-likelihood function: -3.52999671386
Iteration: 16 Log-likelihood function: -3.48557145163
Iteration: 17 Log-likelihood function: -3.44508206139
Iteration: 18 Log-likelihood function: -3.40799948447
Iteration: 19 Log-likelihood function: -3.3738885624
Iteration: 20 Log-likelihood function: -3.3423876581
Iteration: 21 Log-likelihood function: -3.31319343769
Iteration: 22 Log-likelihood function: -3.2860493346
Iteration: 23 Log-likelihood function: -3.2607366863
Iteration: 24 Log-likelihood function: -3.23706784091
Iteration: 25 Log-likelihood function: -3.21488073614
Iteration: 26 Log-likelihood function: -3.19403459259
Iteration: 27 Log-likelihood function: -3.17440646052
Iteration: 28 Log-likelihood function: -3.15588842703
Iteration: 29 Log-likelihood function: -3.13838533947
Iteration: 30 Log-likelihood function: -3.12181293595
Iteration: 31 Log-likelihood function: -3.10609629966
Iteration: 32 Log-likelihood function: -3.09116857282
Iteration: 33 Log-likelihood function: -3.07696988017
Iteration: 34 Log-likelihood function: -3.06344642288
Iteration: 35 Log-likelihood function: -3.05054971191
Iteration: 36 Log-likelihood function: -3.03823591619
Iteration: 37 Log-likelihood function: -3.02646530573
Iteration: 38 Log-likelihood function: -3.01520177394
Iteration: 39 Log-likelihood function: -3.00441242601
Iteration: 40 Log-likelihood function: -2.99406722296
Iteration: 41 Log-likelihood function: -2.98413867259
\end{verbatim}

The iteration stops once the increase in the log-likelihood function is less than 0.01. If no convergence is reached after 500 iterations, the \verb+train+ function returns with an \verb+AssertionError+.

The optional keyword \verb+typecode+ can almost always be ignored. This keyword allows the user to choose the type of Numeric matrix to use. In particular, to avoid memory problems for very large problems, it may be necessary to use single-precision floats (Float8, Float16, etc.) rather than double, which is used by default.

\subsection{Using the logistic regression model for classification}

Classification is performed by calling the \verb+classify+ function. Given a logistic regression model and the values for $x_1$ and $x_2$ (e.g. for a gene pair of unknown operon structure), the \verb+classify+ function returns \verb+1+ or \verb+0+, corresponding to class OP and class NOP, respectively. For example, let's consider the gene pairs {\it yxcE}, {\it yxcD} and {\it yxiB}, {\it yxiA}:

\begin{table}[h]
\begin{center}
\caption{Adjacent gene pairs of unknown operon status.}
\begin{tabular}{|c|c|c|}
\hline
Gene pair & Intergene distance $x_1$ & Gene expression score $x_2$ \\
\hline
{\it yxcE} --- {\it yxcD} & 6 & -173.143442352 \\
{\it yxiB} --- {\it yxiA} & 309 & -271.005880394 \\
\hline
\end{tabular}
\end{center}
\end{table}

The logistic regression model classifies {\it yxcE}, {\it yxcD} as belonging to the same operon (class OP), while {\it yxiB}, {\it yxiA} are predicted to belong to different operons:
\begin{verbatim}
>>> print("yxcE, yxcD:", LogisticRegression.classify(model, [6, -173.143442352]))
yxcE, yxcD: 1
>>> print("yxiB, yxiA:", LogisticRegression.classify(model, [309, -271.005880394]))
yxiB, yxiA: 0
\end{verbatim}
(which, by the way, agrees with the biological literature).

To find out how confident we can be in these predictions, we can call the \verb+calculate+ function to obtain the probabilities (equations (\ref{eq:OP}) and \ref{eq:NOP}) for class OP and NOP. For {\it yxcE}, {\it yxcD} we find
\begin{verbatim}
>>> q, p = LogisticRegression.calculate(model, [6, -173.143442352])
>>> print("class OP: probability =", p, "class NOP: probability =", q)
class OP: probability = 0.993242163503 class NOP: probability = 0.00675783649744
\end{verbatim}
and for {\it yxiB}, {\it yxiA}
\begin{verbatim}
>>> q, p = LogisticRegression.calculate(model, [309, -271.005880394])
>>> print("class OP: probability =", p, "class NOP: probability =", q)
class OP: probability = 0.000321211251817 class NOP: probability = 0.999678788748
\end{verbatim}

To get some idea of the prediction accuracy of the logistic regression model, we can apply it to the training data:
\begin{verbatim}
>>> for i in range(len(ys)):
        print("True:", ys[i], "Predicted:", LogisticRegression.classify(model, xs[i]))
True: 1 Predicted: 1
True: 1 Predicted: 0
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
\end{verbatim}
showing that the prediction is correct for all but one of the gene pairs. A more reliable estimate of the prediction accuracy can be found from a leave-one-out analysis, in which the model is recalculated from the training data after removing the gene to be predicted:
\begin{verbatim}
>>> for i in range(len(ys)):
        model = LogisticRegression.train(xs[:i]+xs[i+1:], ys[:i]+ys[i+1:])
        print("True:", ys[i], "Predicted:", LogisticRegression.classify(model, xs[i]))
True: 1 Predicted: 1
True: 1 Predicted: 0
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 1
True: 0 Predicted: 0
True: 0 Predicted: 0
\end{verbatim}
The leave-one-out analysis shows that the prediction of the logistic regression model is incorrect for only two of the gene pairs, which corresponds to a prediction accuracy of 88\%.

\subsection{Logistic Regression, Linear Discriminant Analysis, and Support Vector Machines}

The logistic regression model is similar to linear discriminant analysis. In linear discriminant analysis, the class probabilities also follow equations (\ref{eq:OP}) and (\ref{eq:NOP}). However, instead of estimating the coefficients $\beta$ directly, we first fit a normal distribution to the predictor variables $x$. The coefficients $\beta$ are then calculated from the means and covariances of the normal distribution. If the distribution of $x$ is indeed normal, then we expect linear discriminant analysis to perform better than the logistic regression model. The logistic regression model, on the other hand, is more robust to deviations from normality.

Another similar approach is a support vector machine with a linear kernel. Such an SVM also uses a linear combination of the predictors, but estimates the coefficients $\beta$ from the predictor variables $x$ near the boundary region between the classes. If the logistic regression model (equations (\ref{eq:OP}) and (\ref{eq:NOP})) is a good description for $x$ away from the boundary region, we expect the logistic regression model to perform better than an SVM with a linear kernel, as it relies on more data. If not, an SVM with a linear kernel may perform better.

Trevor Hastie, Robert Tibshirani, and Jerome Friedman: {\it The Elements of Statistical Learning. Data Mining, Inference, and Prediction}. Springer Series in Statistics, 2001. Chapter 4.4.

\section{$k$-Nearest Neighbors}

\subsection{Background and purpose}

The $k$-nearest neighbors method is a supervised learning approach that does not need to fit a model to the data. Instead, data points are classified based on the categories of the $k$ nearest neighbors in the training data set.

In Biopython, the $k$-nearest neighbors method is available in \verb+Bio.kNN+. To illustrate the use of the $k$-nearest neighbor method in Biopython, we will use the same operon data set as in section \ref{sec:LogisticRegression}.

\subsection{Initializing a $k$-nearest neighbors model}

Using the data in Table \ref{table:training}, we create and initialize a $k$-nearest neighbors model as follows:

\begin{verbatim}
>>> from Bio import kNN
>>> k = 3
>>> model = kNN.train(xs, ys, k)
\end{verbatim}

where \verb+xs+ and \verb+ys+ are the same as in Section \ref{subsec:LogisticRegressionTraining}. Here, \verb+k+ is the number of neighbors $k$ that will be considered for the classification. For classification into two classes, choosing an odd number for $k$ lets you avoid tied votes. The function name \verb+train+ is a bit of a misnomer, since no model training is done: this function simply stores \verb+xs+, \verb+ys+, and \verb+k+ in \verb+model+.

\subsection{Using a $k$-nearest neighbors model for classification}

To classify new data using the $k$-nearest neighbors model, we use the \verb+classify+ function. This function takes a data point $(x_1,x_2)$ and finds the $k$-nearest neighbors in the training data set \verb+xs+. The data point $(x_1, x_2)$ is then classified based on which category (\verb+ys+) occurs most among the $k$ neighbors.

For the example of the gene pairs {\it yxcE}, {\it yxcD} and {\it yxiB}, {\it yxiA}, we find:
\begin{verbatim}
>>> x = [6, -173.143442352]
>>> print("yxcE, yxcD:", kNN.classify(model, x))
yxcE, yxcD: 1
>>> x = [309, -271.005880394]
>>> print("yxiB, yxiA:", kNN.classify(model, x))
yxiB, yxiA: 0
\end{verbatim}
In agreement with the logistic regression model, {\it yxcE}, {\it yxcD} are classified as belonging to the same operon (class OP), while {\it yxiB}, {\it yxiA} are predicted to belong to different operons.

The \verb+classify+ function lets us specify both a distance function and a weight function as optional arguments. The distance function affects which $k$ neighbors are chosen as the nearest neighbors, as these are defined as the neighbors with the smallest distance to the query point $(x, y)$. By default, the Euclidean distance is used. Instead, we could for example use the city-block (Manhattan) distance:

\begin{verbatim}
>>> def cityblock(x1, x2):
...    assert len(x1)==2
...    assert len(x2)==2
...    distance = abs(x1[0]-x2[0]) + abs(x1[1]-x2[1])
...    return distance
...
>>> x = [6, -173.143442352]
>>> print("yxcE, yxcD:", kNN.classify(model, x, distance_fn = cityblock))
yxcE, yxcD: 1
\end{verbatim}

The weight function can be used for weighted voting. For example, we may want to give closer neighbors a higher weight than neighbors that are further away:

\begin{verbatim}
>>> def weight(x1, x2):
...    assert len(x1)==2
...    assert len(x2)==2
...    return exp(-abs(x1[0]-x2[0]) - abs(x1[1]-x2[1]))
...
>>> x = [6, -173.143442352]
>>> print("yxcE, yxcD:", kNN.classify(model, x, weight_fn = weight))
yxcE, yxcD: 1
\end{verbatim}
By default, all neighbors are given an equal weight.

To find out how confident we can be in these predictions, we can call the \verb+calculate+ function, which will calculate the total weight assigned to the classes OP and NOP. For the default weighting scheme, this reduces to the number of neighbors in each category. For {\it yxcE}, {\it yxcD}, we find
\begin{verbatim}
>>> x = [6, -173.143442352]
>>> weight = kNN.calculate(model, x)
>>> print("class OP: weight =", weight[0], "class NOP: weight =", weight[1])
class OP: weight = 0.0 class NOP: weight = 3.0
\end{verbatim}
which means that all three neighbors of \verb+x1+, \verb+x2+ are in the NOP class. As another example, for {\it yesK}, {\it yesL} we find

\begin{verbatim}
>>> x = [117, -267.14]
>>> weight = kNN.calculate(model, x)
>>> print("class OP: weight =", weight[0], "class NOP: weight =", weight[1])
class OP: weight = 2.0 class NOP: weight = 1.0
\end{verbatim}
which means that two neighbors are operon pairs and one neighbor is a non-operon pair.

To get some idea of the prediction accuracy of the $k$-nearest neighbors approach, we can apply it to the training data:
\begin{verbatim}
>>> for i in range(len(ys)):
        print("True:", ys[i], "Predicted:", kNN.classify(model, xs[i]))
True: 1 Predicted: 1
True: 1 Predicted: 0
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
\end{verbatim}
showing that the prediction is correct for all but two of the gene pairs. A more reliable estimate of the prediction accuracy can be found from a leave-one-out analysis, in which the model is recalculated from the training data after removing the gene to be predicted:
\begin{verbatim}
>>> k = 3
>>> for i in range(len(ys)):
        model = kNN.train(xs[:i]+xs[i+1:], ys[:i]+ys[i+1:], k)
        print("True:", ys[i], "Predicted:", kNN.classify(model, xs[i]))
True: 1 Predicted: 1
True: 1 Predicted: 0
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 1
True: 1 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 1
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 0
True: 0 Predicted: 1
\end{verbatim}
The leave-one-out analysis shows that $k$-nearest neighbors model is correct for 13 out of 17 gene pairs, which corresponds to a prediction accuracy of 76\%.

\section{Na\"ive Bayes}

This section will describe the \verb|Bio.NaiveBayes| module.

\section{Maximum Entropy}

This section will describe the \verb|Bio.MaximumEntropy| module.

\section{Markov Models}

This section will describe the \verb|Bio.MarkovModel| and/or \verb|Bio.HMM.MarkovModel| modules.


%!TEX root =  ../main.tex

\mychapters{Polynomials}{polynomials}{\chapdir/pics/Sao_Paulo_Stock_Exchange.jpg}

Polynomials are among the most well-studied and understood parts of mathematics.
There is a tremendous amount we know and can describe about how they work
and what parts of the natural world they describe.  However, this is still so much
to discover about them and it is readily possible to generate unsolvable problems.

That being said, monstrously difficult problems become easily doable with the mastery
of a few, simple techniques.  Does this look easy?!

\begin{equation}
135x^{10} + 54x^9 + 5x^7 + 2x^6 - 135x^4 - 54x^3 - 5x - 2 = 0
\end{equation}

You might not believe it, but if we told you $-\frac{2}{5}$ was one of the solutions,
the rest will jump out at you in no time flat!

\newpage
\chapterminitoc

%									6 - 1
\newpage
\invisiblesection{Graphs}
\marginlessinput{\chapdir/0601p}
\newpage
\input{\chapdir/0601}
\newpage
\subsection{Exercises}
Kuta

%									6 - 2
\newpage
\section{Factoring}
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0602p.pdf}}
\input{\chapdir/0602}
\newpage
\subsection{Exercises}
In Kuta

%									6 - 3
\newpage
\invisiblesection{Mean, Max., Min.}
\marginlessinput{\chapdir/0603p}
\newpage
\input{\chapdir/0603}
\newpage
\subsection{Exercises}
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0603xA.pdf}}
\newpage
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0603xB.pdf}}
\newpage
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0603xC.pdf}}

%									6 - 4
%\newpage
\invisiblesection{Rational Functions}
\subsection{Problems}
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0604p.pdf}}
\input{\chapdir/0604}
\newpage
\subsection{Exercises}
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0604xA.pdf}}
\newpage
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0604xB.pdf}}
\newpage
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0604xC.pdf}}
\newpage
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0604xD.pdf}}

%									6 - 5
\newpage
\section{Asymptotes}
\noindent\makebox[\textwidth]{\includegraphics[width=\paperwidth]{\chapdir/0605p.pdf}}
\input{\chapdir/0605}
~\vfill
\newpage
\subsection{Exercises}
two parts, in Kuta

\newpage
\section{Review}
\subsection{Chapter Review}
\subsection{Chapter Test}


\documentclass[12pt]{article}
    \usepackage{mathtools}
    \usepackage{amsfonts}

    \usepackage{venndiagram}
    \usepackage{lastpage}
    \usepackage[hidelinks]{hyperref}
    \usepackage{color}
    \definecolor{darkred}{RGB}{200, 0, 0}

    \usepackage{fancyhdr}
    \pagestyle{fancy}
    \fancyhf{}
    \fancyhead[LO]{TMA4140: Homework Set 2}
    \fancyhead[RO]{Henry S. Sjøen}
    \fancyfoot[CO]{\thepage\ of \pageref{LastPage}}

    \usepackage{fancyref}

  \author{Henry S. Sjøen}
  \title{%
  \textbf{TMA4140 - Homework Set 2}\\
  Basic structures: Sets, Functions, Sequences and Sums\\
    \color{darkred}{\textbf{RETTES}}
  }
    \begin{document}
    \maketitle
    \thispagestyle{empty}
    % \pagebreak
    \tableofcontents

    \pagebreak
    \section{Chapter 2.1 - Sets}
    \subsection{Exercise 5}
    Determine whether each of these pairs are equal.\\
        a) $\{1,3,3,3,5,5,5,5\},\{5,3,1\} \Rightarrow True$.\\
        b) $\{\{1\}\},\{1,\{1\}\} \Rightarrow False.$\\
        c) $\emptyset,\{\emptyset\} \Rightarrow False$

    \subsection{Exercise 24}
    Determine wether each of these sets is the power set of a set, where \emph{a} and \emph{b} are distinct elements.\\
        a) $\emptyset$, is False. $P(\emptyset)=\{\emptyset\{\emptyset\}\}$\\
        b) $\{\emptyset,\{a\}\}$, is True. $P(\{a\})=\{\emptyset,\{a\}\}$\\
        c) $\{\emptyset, \{a\},\{\emptyset,a\}\}$, is False.\\
        d) $\{\emptyset,\{a\},\{b\},\{a,b\}\}$, is True.

        The powerset of any set $S$ is the set of all subsets of $S$, including the empty set $\emptyset$ and $S$ itself. \footnote{Wikipedia, 2018-09-14, 13:54, \url{https://en.wikipedia.org/wiki/Power_set}}
        If $a$ and $b$ are distinct elements of a set $A$. $A=\{a,b\}$
        Then the powerset of $A$ is $p(A)=\{\emptyset,\{a\},\{b\},\{a,b\}\}$.

    \pagebreak
    \section{Chapter 2.2 - Set Operations}
    \subsection{Exercise 18c}
    Let A, B and C be sets. Show that:
    $(A \cap B) \subseteq (A \cup B \cup C)$

    % \begin{equation}
    %     \begin{split}
    %         (A \cap B) \subseteq (A \cup B \cup C)
    %     \end{split}
    % \end{equation}

    \begin{figure}[h]
        \centering
        \begin{venndiagram3sets}
            \fillA
            \fillB
            \fillC
        \end{venndiagram3sets}
        \caption{$(A \cup B \cup C)$}
        \label{fig:Union}
    \end{figure}

    As illustrated in \fref{fig:Union} we can se the venndiagram for $(A \cup B \cup C)$. Coloring out $(A \cap B)$, as shown in  \fref{fig:ACapB}, we can see that $(A \cap B)$ is a subset of $(A \cup B \cup C)$.

    \begin{figure}[h]
        \centering
        \caption{$A \cap B$}
        \begin{venndiagram3sets}
            % [labelOnlyAB=$A \cap B$]
            \fillACapB
        \end{venndiagram3sets}
        \label{fig:ACapB}
    \end{figure}

        % \url{http://www3.wolframalpha.com/Calculate/MSP/MSP2891521ibi0a5gd30f95g000037hcb6a7hd034233?MSPStoreType=image/gif&s=44&w=471.&h=279.&cdf=Resizeable}

    \newpage
    \subsection{Exercise 18d}
    Let A, B and C be sets. Show that:
    $(A - B) - C \subseteq A - C$

    \begin{figure}[h]
        \centering
        \begin{venndiagram3sets}
            \fillOnlyA
        \end{venndiagram3sets}
        \caption{$(A - B) - C$}
        \label{fig:onlyA}
    \end{figure}

    \begin{figure}[h]
        \centering
        \begin{venndiagram3sets}
            \fillANotC
        \end{venndiagram3sets}
        \caption{$A - C$}
        \label{fig:ANotC}
    \end{figure}

    We can see that $(A - B)-C$ is a subset of $A-C$, see \fref{fig:onlyA} and \ref{fig:ANotC}.

    \newpage
    \subsection{Exercise 46}
    Show that if A, B, and C are sets, then:\footnote{This is a special case of the inclusion-exclusion principle, which will be studied in Chapter 8. Also... Wikipedia \url{https://en.wikipedia.org/wiki/Inclusion-exclusion_principle}}

    \begin{equation}
        |A \cup B \cup C| =
        |A| + |B| + |C| - |A \cap B| - |A \cap C| - |B \cap C| + |A \cap B \cap C|
    \end{equation}

    \begin{figure}[h]
        \centering
        \begin{venndiagram3sets}[
            labelOnlyA=1,labelOnlyB=1,labelOnlyC=1,
            labelOnlyAB=2,labelOnlyAC=2,labelOnlyBC=2,
            labelABC=3]
            \fillA
            \fillB
            \fillC
        \end{venndiagram3sets}
        \caption{$|A|+|B|+|C|$}
        \label{fig:u1}
    \end{figure}

    Here we have counted some elements more than once (\fref{fig:u1}), lets correct that.

    \begin{figure}[h]
        \centering
        \begin{venndiagram3sets}[
            labelOnlyA=1,labelOnlyB=1,labelOnlyC=1,
            labelOnlyAB=1,labelOnlyAC=1,labelOnlyBC=1,
            labelABC=0]
            \fillA
            \fillB
            \fillC
        \end{venndiagram3sets}
        \caption{$|A|+|B|+|C|-(|A \cap B| - |A \cap C| - |B \cap C|)$}
        \label{fig:u2}
    \end{figure}

    But now, the intersection of A,B and C is not counted. (\fref{fig:u2}). Lets count the intersection once $|A\cap B \cap C|$ as shown in \fref{fig:u3}.

    \begin{figure}[h]
        \centering
        \begin{venndiagram3sets}[
            labelOnlyA=1,labelOnlyB=1,labelOnlyC=1,
            labelOnlyAB=1,labelOnlyAC=1,labelOnlyBC=1,
            labelABC=1]
            \fillA
            \fillB
            \fillC
        \end{venndiagram3sets}
        \caption{$|A|+|B|+|C|-(|A \cap B| - |A \cap C| - |B \cap C|) + |A \cap B \cap C|$}
        \label{fig:u3}
    \end{figure}

    We have now shown that
    and we can intuitively see that it's the same as counting all elements inside the union of the three sets: $|A \cup B \cup C|$. (\fref{fig:u4})

    \begin{figure}[h]
        \centering
        \begin{venndiagram3sets}
            \fillA
            \fillB
            \fillC
        \end{venndiagram3sets}
        \caption{Union of the sets A,B and C}
        \label{fig:u4}
    \end{figure}

    % TODO: Show it with more amazing math.

    \pagebreak
    \section{Chapter 2.3 - Functions}
    \subsection{Exercise 12c}
    Determine whether each of these functions from $Z$ to $Z$ is one-to-one.

    \textbf{True.} $f(n) = n^3$, is One-to-One, because it passes both the vertical and horizontal line test.\footnote{\url{http://www.mathwords.com/o/one_to_one_function.htm}}

    \subsection{Exercise 38}
    % Todo: Look over later
    Let $f(x) = ax+b$ and $g(x)= cx+d$, where a,b,c, and d are constants. Determine necessary and sufficient conditions on the constants a,b,c, and d so that
    $f \cdot g = g \cdot f$.

    \begin{equation}
        \begin{split}
            f \cdot g = g \cdot f = a(cx+d)+b=acx+ad+b\\
            g\cdot f = g(f(x))=c(ax+b)+d=acx+cb+d\\
            f \cdot g =  g\cdot f \Leftrightarrow ad+b = cb+d
        \end{split}
    \end{equation}

    \subsection{Exercise 42}
    % Todo: Look over later
    Let $f$ be the function from $R$ to $R$ defined by $f(x)=x^2$. Find:\\
    a) $f^{-1}(\{1\}) = \pm 1$\\
    b) $f^{-1}(\{ x | 0 < x < 1\}) = \pm \{x|-1<x<1 \wedge x \neq 0 \}$\\
    c) $f^{-1}(\{ x | x > 4\}) = \{x|-2>x \wedge x>2\}$

    \pagebreak
    \section{Chapter 2.4 - Sequences and Summations}

    \subsection{Exercise 12c}
    % Todo: Look over later
    Show that the sequence {$a_n$} is a solution of the recurrence relation $a_n=-3a_{n-1}+4a_{n-2}$ if $a_n = (-4)^n$
    \begin{equation}
        \begin{split}
            a_n =& -3a_{n-1}+4a_{n-2} = (-4)^n\\
            =&-3a_{n-1} + 4a_{n-2}\\
            =& (-4)^{n-1}+4(-4)^{n-2}\\
            =& (-4)^{n-2}[(-3)(-4)+4]\\
            =& (-4)^{n-2}(16)\\
            =& (-4)^{n-2}(-4)^2\\
            =& (-4)^{n}
        \end{split}
    \end{equation}

    \subsection{Exercise 33d}
    Compute the double sum
    \begin{equation}
        \begin{split}
            \sum_{i=0}^{2}\sum_{j=1}^{3}ij &= (0*1+0*2+0*3)+(1*1+1*2+1*3)+(2*1+2*2+2*3) \\
            &= (0+0+0) + (1+2+3) + (2+4+6)\\
            &= 0 + 6 + 12\\
            &= 18
        \end{split}
    \end{equation}

    \pagebreak
    \section{Chapter 2.5 - Cardinality of Sets}
    \subsection{TODO: Exercise 16}
    \emph{Exercise}:
    Show that a subset of a countable set is also countable.\\
    \emph{Answer}:
    A set is countable if it is finite or is the same size as $\mathbb{N}$.
    To show that $A$ is countable, it is sufficient to show that there is an injection
    from $A$ to $\mathbb{N}$.

    \end{document}


\chapter{Preliminary Results} \label{ch-1}

\section{Minimal Subspace Polynomial and Multipoint Evaluation}

An algorithm for multipoint evaluation was presented in \cite{musschost}. Without loss of generality we may suppose $\rho$ has skew-degree $d$ a perfect square and let $\delta = \sqrt{d}$. The target skew polynomial $\rho$ can be decomposed as $\rho = \rho_0 + \tau^{\delta} \rho_1 + \ldots + \tau^{\delta(\delta - 1)}\rho_{\delta-1}$ where each $\rho_i$ has skew-degree at most $\delta - 1$. Set $\rho_i = \rho_{i,0} + \rho_{i,1}\tau + \ldots + \rho_{i,\delta - 1} \tau^{\delta-1}$. For each $0 \leq i \leq \delta -1$ and $1 \leq j \leq s$ set $v_{i,j} = \sigma^i(v_j)$, and each $\rho_i(v_j) = \sum_{k=0}^{\delta - 1} \rho_{i,k}v_{k,j}$. Horner's scheme can then be used to compute $\rho(v_j) = \rho_0(v_j) + \tau^{\delta}(\rho_1(v_j) + \tau^{\delta}(\rho_2(v_j) + \ldots $.

\begin{algorithm}
\caption{Multipoint Evaluation \cite{musschost}}
\label{alg:mpe2}
\hspace*{\algorithmicindent} \textbf{Input} A skew polynomial $\rho \in \F_{q^n}\{\tau \}$ of degree $d$, and a set of linearly independent evaluation points $\{ v_1, \ldots, v_s \} \subset \F_{q^n}$.\\
 \hspace*{\algorithmicindent} \textbf{Output} The set $\{ \rho(v_1), \ldots, \rho(v_s) \} \subset  \F_{q^n}$ \\
 \begin{algorithmic}[1]
  \State Compute $\rho_i = \rho_{i,0} + \rho_{i,1}\tau + \ldots + \rho_{i,\delta - 1} \tau^{\delta}$ such that $\rho = \rho_0 + \tau^{\delta} \rho_1 + \ldots + \tau^{\delta(\delta - 1)}\rho_{\delta-1}$
  \State Compute $v_{i,j} = \sigma^i(v_j)$ for each $0 \leq i \leq \delta-1$ and $1 \leq j \leq s$
  \State Compute the $\delta \times \delta$ and $\delta \times s$ matrix product:
  $
  \begin{bmatrix}
  \rho_{0,0} & \ldots & \rho_{0, \delta - 1} \\
  \vdots & & \vdots \\
  \rho_{\delta - 1, 0} & \ldots & \rho_{\delta - 1, \delta - 1}
  \end{bmatrix}\begin{bmatrix}
  v_{0,0}  & \ldots & v_{0,s} \\ \vdots & & \vdots \\
  v_{\delta - 1, 0} & \ldots & v_{\delta - 1, s}
  \end{bmatrix} = \begin{bmatrix}
  \rho_0(v_1) & \ldots & \rho_{0}(v_s) \\
  \vdots & & \vdots \\
  \rho_{\delta - 1}(v_1) & \ldots & \rho_{\delta - 1}(v_s)
  \end{bmatrix}
  $

  \State For $1 \leq j \leq s$ compute $\rho(v_j) = \rho_0(v_j) + \tau^{\delta}(\rho_1(v_j) + \tau^{\delta}(\rho_2(v_j) + \ldots $ via Horner's scheme

\end{algorithmic}

\end{algorithm}

Steps 1, 2, and 3 take time equal to the cost of $O(d + d^{1/2}s )$ applications of Frobenius powers. The cost of step 3 is either $O(ds^{\omega - 2})$ field operations in $\L$ if $s < \sqrt{d}$, and $O(d^{(\omega-1)/2}s)$ $\L$-operations otherwise.  Counting the complexity of modular compositions, the overall bit complexity of the algorithm is either $(ds^{\omega - 2} n^{\theta} \log q)^{1 + o(1)}$ if $s < \sqrt{d}$, and $(d^{(\omega-1)/2}sn^{\theta} \log q)^{1 + o(1)}$ otherwise.

Algorithm \ref{alg:mpe2} may then be leveraged as a subroutine for algorithm \ref{msp1}; the algebraic runtime of $\MSP(d)$ satisfies

\begin{equation*}
    \MSPr(d) = 2\MSPr(d/2) + O(d^{(\omega + 1)/2}).
\end{equation*}

Applying the master theorem, $\MSPr(d) = \Theta(d^{(\omega + 1)/2})$. In terms of bit complexity, we have $ \MSPr(d) = (d^{(\omega-1)/2}sn^{\theta} \log q)^{1 + o(1)}$.

\section{Narayanan's Algorithm for Rank-two Drinfeld Modules}

We propose an amendment to an additional algorithm for finding the characteristic polynomial of a rank-two Drinfeld module based on an approach originally due to Narayanan in \cite{Narayanan18} and later modified in \cite{musschost}. Recall that in the rank-two case, the characteristic polynomial of $\phi$ takes the form $X^2 - aX - b$. Moreover, since $\tau^{2n} - \phi_a\tau^n - \phi_b = \id - a(\phi_x) - b(\phi_x) = 0$, the polynomial $1 - a - b$ is an annihilator of $\phi_x$; in fact it is the characteristic polynomial of $\phi_x = \Delta_2 \tau^2 + \Delta_1 \tau + \gamma(x)$ \cite{GEKELE1991187}. If the minimal polynomial $\Gamma$ of $\phi_x$ coincides with the characteristic polynomial,then we can once again resort to a Wiedemann style argument: in this case, it suffices to compute the minimal polynomial $\Gamma_{\ell, \alpha}$ of a sequence $\{r_i\} = \{\ell(\phi_x^i(\alpha)) \}$ for a uniformly random $\alpha \in \L$ and $\F_q$-linear map $\ell: \L \to \F_q$ and argue that with probability greater than $\frac{1}{2}$ we have $\Gamma_{\ell, \alpha} = \Gamma$. In his original proposal, Narayanan used the automorphism projection algorithm of Kedlaya and Umans to compute $2n$ terms of the sequence $\{ r_i \}$, whereas it is unclear to us whether this approach and its complexity bound extends to the case where $\phi_x$ is not an automorphism of $\L$.

Instead, we propose the following alternative approach based on a ``block Wiedemann" style algorithm \cite{Coppersmith94}. For some positive integer $\mu$, pick $\mu$ many $\F_q$-linear maps $\ell_1, \ldots, \ell_{\mu}$ as before and $\mu$ elements $\alpha_i \in \L$. Construct a sequence of $\mu \times \mu$ matrices $R_{k}$ such that the $(i,j)^{th}$ entry of $R_k$ is $ \ell_i(\phi_x^k(\alpha_j))$. The generating series of this sequence can be expressed as a product of the form $Q^{-1}N$ for  $\mu \times \mu$ matrices matrices $Q,N$ taking coefficients in $\F_q[x]$ and with the entries of $Q$ having degree at most $\lceil n/\mu \rceil$. Computing $Q$ can be accomplished by first determining $2n/\mu$ matrices $R_k$ and using the PM basis algorithm given in \cite{GiJeVi03} in $(\mu^{\omega - 1}n\log q)^{1 + o(1)}$ bit operations. Fix additional constants $\frak{b}$and set $K = \lfloor (n/2\mu)^{\frak{b}} \rfloor$, $K' = \lceil n/2\mu K \rceil$. To compute the $R_k$, we turn to the following procedure:

\begin{enumerate}
    \item Compute the $\F_q$-linear maps $\ell_{i, \frak{s}} =  \phi_x^{\frak{s}}(\ell_i)$ for $i \leq \mu$ and $\frak{s} < K$.
    \item Compute $\phi_x^{K}$ and for $j \leq \mu$ and $\frak{t} \leq K'$ compute $\alpha_{\frak{t}, j} = \phi_x^{\frak{t}K}(\alpha_j)$.
    \item Let $L$ be the $K\mu \times n$ matrix whose rows correspond to the linear map $\ell_{i, \frak{s}}$, and $A$ the $n \times K'\mu$ matrix whose columns correspond to $\alpha_{\frak{t}, j}$. The entries of the matrix product $LA$ are exactly $\ell_i(\phi_x^{\frak{s} + \frak{t}K}(\alpha_j))$ for all $i,j \leq \mu$, $\frak{s} < K$, $\frak{t} < K'$.
\end{enumerate}

For well chosen $\mu, \frak{b}$, the procedure outlined above computes all entries of $R_k$ for $k \leq 2n/\mu$ in at most $(n^{1.885} \log q)^{1 + o(1)}$.

%Fix constants, $\mu$, $\frak{w}, \frak{v}$.

\section{Minimal Polynomials in Higher Ranks}

\subsection{Schoof-like Algorithms}

Our main results for this section is the following theorem.

\begin{theorem}\label{schooflike}
There exists a Monte Carlo algorithm such that when $\gcd(n,r) = 1$ or $r = 3$, then for a generic choice of rank $r$ Drinfeld module $\phi$ over a degree $n$ extension $\L$ of  $\F_q$, the algorithm computes the minimal polynomial of $\phi$ with bit complexity $r^{(\omega+3)/2}(n^2 \log q)^{1 + o(1)}$ and returns a correct result when $q > Cnr^2$, for some constant $C > 2$, with probability greater than $\frac{1}{2}$.
\end{theorem}

Recall the \textit{minimal polynomial} of $\phi$ is the unique monic polynomial $\Gamma = T^{d} - a_{d-1}T^{d-1} - \ldots - a_1T - a_0 \in \F_q[x][T]$ of least degree such that:

\begin{equation} \label{charpoly1}
 \tau^{nd} - \phi_{a_{d-1}}\tau^{n(d-1)} - \ldots - \phi_{a_1} \tau^n - \phi_{a_0} = 0
\end{equation}

The goal is to compute each $a_i$ modulo a set of polynomials which can then be used to reconstruct the true coefficients. This can be done over the right-quotient space $\L\{\tau \}/\phi_{x-e}$ since $\phi_{a \bmod b} \bmod \phi_b = \phi_a \bmod \phi_b$ for any $a, b \in \F_q[x]$. If $b = x - e$ for $e \in \F_q$, then in particular $a(c) = \phi_a \bmod \phi_b$. As in the rank 2 case, when $q \gg nr$, linear polynomials suffice and equation (\ref{charpoly1}) becomes

\begin{equation}\label{linsys}
\tau^{n\hatr} - a_{\hatr-1}(e)\tau^{n(\hatr-1)} - \ldots - a_1(e)\tau^n - a_0(e)  = 0 \mod \phi_{x - e}.
\end{equation}

$\L\{\tau \}/\phi_{x-e}$ is an $r$-dimensional vector space over $\L$ spanned by the computational basis $\{1, \tau, \ldots, \tau^{r-1}\}$, and let $\hat{\varphi}$ denote the column vector of $\varphi \in \L\{\tau \}/\phi_{x-e}$ in the computational basis. Writing (\ref{linsys}) as a linear system

\begin{equation}\label{system1} \begin{bmatrix} \hat{\tau}^{n(\hatr-1)} & \ldots & \hat{\tau}^{n} \end{bmatrix} \begin{bmatrix} a_{\hatr-1}(e) \\ \vdots \\ a_1(e) \end{bmatrix} = \begin{bmatrix}  -a_0(e) \\ 0 \\ \vdots \\ 0  \end{bmatrix} + \hat{\tau}^{n\hatr}.
\end{equation}

Let $ B(\psi) = \begin{bmatrix} \hat{\tau}^{n(\hatr-1)} & \ldots & \hat{\tau}^{n} \end{bmatrix}$ for modulus $\psi \in \L\{\tau\}$. The well-determinedness of the system encoded by $B(\phi_{x-e})$ can be characterized using the following lemma.

%Let $B_{\phi}^{n,j} := \begin{bmatrix} \tau^{n(j-1)} & \ldots & \tau^{n} \end{bmatrix} \bmod \phi_{x-e}$ be the $r \times (j -1)$ system encoding the coefficients of the $r$-dimensional vector $\tau^{ni} \mod \phi_{x -e}$ in the computational basis. The goal is to show the system given in $(\ref{system1})$ has a unique solution with high probability when $q \gg nr$ by applying the Schwartz-Zippel lemma to the polynomial in the coefficients of the Drinfeld module given by $\det B_{\phi}^{n,\hatr}$, where $B_{\phi}^{n,\hatr}$ is truncated to the first $\hatr - 1$ rows, as $\det B_{\phi}^{n,\hatr} \neq 0$ provides a sufficient condition for solvability.

%Let $ \Delta_i = \sum_{j=0}^{n-1}\alpha_{i,j}x^j$ for $i > 0$ and $ (\gamma(x) - e) = \sum_{j=0}^{n-1}\alpha_{0,j}x^j$. The goal of the next lemma is to bound the degree

\begin{lemma}\label{detdeg}
Suppose $Cn^2r < q$ for a constant $C>2$. Further suppose that there exists at least one skew-polynomial modulus $\psi$ of skew-degree $r$ such that the system in (\ref{system1}) is uniquely solvable. Then for choices of Drinfeld coefficients $\Delta_i$ and evaluation point $e$ except on an asymptotically vanishing set as $q \to \infty$, the system given in $(\ref{system1})$ is well-determined.
\end{lemma}

\begin{proof}
We consider general skew-polynomial modulus $\psi = \sum_{i=0}^{r} \Lambda_i \tau^i$ for equation $(\ref{system1})$ rather than the special Drinfeld case of $\Lambda_0 = x - e$. We assume the case $n,d \geq 3$ we can set $\mu = n(\hatr - 1)$, $\reduc = \mu - r$, $\omega = \mu - n > 0$. We can characterize the linear independence of the set $\{\tau^{ni} \}_{i =1}^{\hatr} \bmod \psi$ as follows: suppose to the contrary that

\begin{equation}\label{mult}
    \sum_{i=1}^{\hatr - 1} \beta_i \tau^{ni} = P \psi
\end{equation}

with $P = \sum_{i= 1}^{\reduc} p_ix^i \in \L\{\tau \}$ and $P \neq 0 $. The choice of $P$ is constrained by the requirement that the coefficient of $\tau^{j}$ in the product is 0 if $n$ doesn't divide $j$. The linear system representing the product $P\psi$ can be encoded as an $ (\omega + 1) \times (\omega - r + 1)$ matrix $M$ over $\L$

\begin{equation}
    \label{productsystem}
    \begin{bmatrix}
    \Lambda_{r}^{q^{\reduc}} & 0 & 0 & \ldots & 0 & \ldots & 0 \\
    \Lambda_{r - 1}^{q^{\reduc}} & \Lambda_{r}^{q^{\reduc - 1}} & 0 & \ldots & 0 & \ldots & 0 \\
    \vdots & \vdots & \vdots & \ddots & \vdots & \ddots & \vdots \\
    \Lambda_{0}^{q^{\reduc}} & \Lambda_{1}^{q^{\reduc - 1}} & \Lambda_{2}^{q^{\reduc - 2}} & \ldots & \Lambda_{r}^{q^{\reduc - r}} & \ldots & 0 \\
    0 & \Lambda_{0}^{q^{\reduc - 1}} & \Lambda_{1}^{q^{\reduc - 2}} & \ldots & \Lambda_{r-1}^{q^{\lambda - r}} & \ldots & 0 \\
    \vdots & \vdots & \vdots & \ddots & \vdots & \ddots & \vdots \\
    0 & 0 & 0 & \ldots & 0 & \ldots & \Lambda_0^{q^{n}}

    \end{bmatrix}
    \begin{bmatrix}
    p_{\reduc} \\ p_{\reduc - 1} \\ \vdots \\ p_{n}
    \end{bmatrix}
    =
    \begin{bmatrix}
    \beta_{\hatr-1} \\ 0 \\ \vdots \\ 0 \\ \beta_{1}
    \end{bmatrix}
\end{equation}

Deleting $\hatr - 1$ rows corresponding to the free parameters $\beta_i$ leaves the above as a well determined $ (\omega - \hatr + 2) \times (\omega - r + 1)$ linear system.

\begin{equation}
    \label{reduced}
    \begin{bmatrix}
    \Lambda_{r - 1}^{q^{\reduc}} & \Lambda_{r}^{q^{\reduc - 1}} & 0 & \ldots & 0 & \ldots & 0 \\
    \vdots & \vdots & \vdots & \ddots & \vdots & \ddots & \vdots \\
    \Lambda_{0}^{q^{\reduc}} & \Lambda_{1}^{q^{\reduc - 1}} & \Lambda_{2}^{q^{\reduc - 2}} & \ldots & \Lambda_{r}^{q^{\reduc - r}} & \ldots & 0 \\
    0 & \Lambda_{0}^{q^{\reduc - 1}} & \Lambda_{1}^{q^{\reduc - 2}} & \ldots & \Lambda_{r-1}^{q^{\lambda - r}} & \ldots & 0 \\
    \vdots & \vdots & \vdots & \ddots & \vdots & \ddots & \vdots \\

    \end{bmatrix}
    \begin{bmatrix}
    p_{\reduc} \\ p_{\reduc - 1} \\ \vdots \\ p_{n}
    \end{bmatrix}
    =
    \begin{bmatrix}
    0 \\ 0 \\ \vdots \\ 0 \\ 0
    \end{bmatrix}
\end{equation}

Deleting an additional $r - \hatr + 1$ rows, which can be chosen such that the resulting system derived from the modulus $\psi$ is invertible, and showing that for generic choices of Drinfeld module parameters the determinant of this resulting square matrix is non-zero is sufficient to establish $P = 0$. To that end, we ``vectorize" equation $(\ref{productsystem})$: fix an $\F_q$-basis of $\L$, under which elements $a \in \L$ can be represented by a length $n$ vector $\hat{a}$ with entries in $\F_q$ or by their $n\times n$ multiplication operator $\overline{a} \in \F_q^{n \times n}$. Moreover, let $\overline{\sigma}$ denote the $n\times n$ matrix of the Frobenius acting on vectors $\hat{a}$. This allows a rewriting of the truncated system $\overline{M}$

\begin{equation}
    \label{vectorsystem}
    \begin{bmatrix}
    \overline{\sigma}^{\lambda}\overline{\Lambda}_{r - 1} & \overline{\sigma}^{\lambda-1}\overline{\Lambda}_{r} & 0 & \ldots & 0 & \ldots & 0 \\
    \vdots & \vdots & \vdots & \ddots & \vdots & \ddots & \vdots \\
    \overline{\sigma}^{\lambda}\overline{\Lambda}_{0} & \overline{\sigma}^{\lambda -1}\overline{\Lambda}_{1} & \overline{\sigma}^{\lambda - 2}\overline{\Lambda}_{2} & \ldots & \overline{\sigma}^{\lambda - r}\overline{\Lambda}_{r} & \ldots & 0 \\
    0 & \overline{\sigma}^{\lambda - 1}\overline{\Lambda}_{0} & \overline{\sigma}^{\lambda - 2}\overline{\Lambda}_{1} & \ldots & \overline{\sigma}^{\lambda - r}\overline{\Lambda}_{r-1} & \ldots & 0 \\
    \vdots & \vdots & \vdots & \ddots & \vdots & \ddots & \vdots

    \end{bmatrix}
    \begin{bmatrix}
    \hat{p}_{\reduc} \\ \hat{p}_{\reduc - 1} \\ \vdots \\ \hat{p}_{n}
    \end{bmatrix}
    =
    \begin{bmatrix}
     \hat{0} \\ \vdots \\ \hat{0}
    \end{bmatrix}
\end{equation}

This system is parametrized by the $\F_q$-coefficients of $\Lambda_i = \sum_{j = 0}^{n-1} \alpha_{i,j}x^j$. The resulting vectorized system is an  $ n(\omega - r + 1) \times n(\omega - r + 1)$ matrix whose entries are degree 1 polynomials in the variables $\alpha_{i,j}$; therefore when the determinant is not exactly the zero polynomial in the $\alpha_{i,j}$, then it has degree $n(\omega - r + 1) = O(n^2r)$. This can't be the case since the system must be invertible for at least one choice of $\alpha_{i,j}$ corresponding to the choice of skew-polynomial $\psi$ for which a solution $z$ to $B(\psi)z = 0$ is uniquely determined. Applying the Schwartz-Zippel lemma to the determinant:

%Set $\alpha_i = \frac{\Delta_i}{\Delta_r}$ with coefficients $\alpha_i := \sum_{j=0}^{n-1}\alpha_{i,j}x^j$. Furthermore, let $\overline{\alpha_i}$ be the $n\times n$ matrix of the action of left multiplication by $\alpha_i$ on $\L/\F_q$, and let $\overline{\tau}$ denote the matrix representation of the Frobenius. If $\sum_{i=0}^{r-1}c_i \tau^i$ is any skew polynomial, the action of left multiplication by $\tau$ on $\L[\tau]/\phi_{x-e}$ is given by the following matrices:

%\begin{equation*}
%M :=  \begin{bmatrix}\overline{\alpha}_{r-1} & I & 0 & \ldots & 0 \\ \overline{\alpha}_{r-2} & 0 & I & \ldots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \overline{\alpha}_{0} & 0 & 0 & \ldots & I \end{bmatrix} \hspace{4mm} S := \begin{bmatrix} \overline{\tau} & & \\ & \ddots & \\ & & \overline{\tau} \end{bmatrix}
%\end{equation*}

%\begin{equation*}
%\begin{bmatrix}\overline{\alpha}_{r-1} & I & 0 & \ldots & 0 \\ \overline{\alpha}_{r-2} & 0 & I & \ldots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \overline{\alpha}_{0} & 0 & 0 & \ldots & I \end{bmatrix} \begin{bmatrix} \overline{\tau} & & \\ & \ddots & \\ & & \overline{\tau}\end{bmatrix} \begin{bmatrix} \overline{c}_{r-1} \\ \vdots \\ \overline{c}_{0} \end{bmatrix} = \begin{bmatrix} \overline{\alpha}_{r-1} \overline{\tau}(\overline{c}_{r-1}) + \overline{\tau}(\overline{c}_{r-2}) \\ \overline{\alpha}_{r-2} \overline{\tau} \overline{c}_{r-1} + \overline{\tau} \overline{c}_{r-3} \\ \vdots \\ \overline{\alpha}_{0} \overline{\tau}(\overline{c}_{r-1})  \end{bmatrix}
%\end{equation*}

%with each $\overline{c}_i \in \L$ being written as a size $n$ vector over $\mathbb{F}_q$. Then $\tau^{in} \mod \phi_{x-e}$ is given by

%\begin{equation*}
%(MS)^{in-r} \begin{bmatrix} \overline{1} \\ 0 \\ \vdots \\ 0 \end{bmatrix}.
%\end{equation*}

%The entries of this expression have degree $in - r$ in the coefficients $\alpha_{i,j}$, and the total degree of any term in the square determinant is $2\sum_{i=1}^{r-1} (in - r) \leq 2nr^2$.

\begin{equation*}
    \Pr\Big[\det \overline{M} = 0\Big] \leq \frac{n^2r}{q} < \frac{1}{C}.
\end{equation*}

\end{proof}

%If $\sqdet B_{\psi}^{n,r} \neq 0 $, then for generic choices of base field element $e$ and coefficients of $\phi_x$, the Schwartz-Zippel lemma implies

% $\{\alpha_{i,j}\}_{1 \leq i \leq r-1}^{0 \leq j \leq n-1}$, the Schwartz-Zippel lemma implies

The remainder of the proof of Theorem 1 is classifying pairs of parameters $n,r$ for which there exists at least one choice of modulus $\psi \in \sring$ that guarantees unique solvability of $B(\psi)$

\begin{prop}\label{coprime}
Suppose $\gcd(n,r) = 1$. There exists a Drinfeld module $\phi_x$ such that $ B(\phi_x)$ is uniquely solvable.  %e $\psi = \tau^n - 1$.
\end{prop}
\begin{proof}
Let $\phi_x = \tau^r - \gamma(x)$. For any $e \neq -\gamma(x)$, $\tau^{ni} = (\gamma(x) + e)^{\lfloor ni/r \rfloor} \tau^{ni \bmod r} \bmod \phi_{x} - e$. Since $\gcd(n,r) = 1$, the set $\{ n \bmod r, 2n \bmod r, \ldots, rn \bmod r\}$ is a reordering of $\{0, 1, \ldots, r-1\}$ and so the elements $\tau^{ni}$ form an $\L$ basis of $\L\{ \tau\}/\phi_{x - e}$.
\end{proof}

\begin{prop}\label{rank3}
If $r = 3$, then for any $n$ there exists a skew-polynomial $\psi \in \L\{\tau\}$ such that $B(\psi)$ is uniquely solvable.
\end{prop}
\begin{proof}
For a contradiction, let $\psi$ be any skew polynomial. The notion of an \textit{Azumaya algebra} and their connection to skew polynomials was previously studied in \cite{Ikehata1984AzumayaAA} and \cite{skewfactor}. In particular, we are interested in a distinguished map sending an Azumaya algebra to its center, the \textit{reduced norm}, which induces a map $\cN: \L\{\tau\} \to \comm$ while satisfying the following constraints.

\begin{enumerate}
    \item $\cN(\psi_1 \psi_2) = \cN(\psi_1) \cN(\psi_2)$
    \item $\deg_{\tau^n} \cN(\psi) \leq n \deg_{\tau} \psi$
    \item $\cN(\psi) = \psi^n$ if $\psi \in \L[\tau^n]$
    \item $\psi$ is irreducible in $\sring$ if and only if $\cN(\psi)$ is irreducible in $\F_q[\tau^n]$
\end{enumerate}

For details on the construction of the map and the proof of item (4) we refer to \cite[Prop. 2.1.17]{skewfactor}. Suppose $\tau^{2n} - \lambda \tau^n = 0$ for some $\lambda \in \F_q$. Then $\tau^{n} - \lambda = 0 \mod \psi$, and we must have a skew polynomial $D$ such that $\tau^n - \lambda = D \psi$. Taking reduced norms of both sides leads to

\begin{equation*}
(\tau^n - \lambda)^n = \cN(D)\cN(\psi)
\end{equation*}

The number of irreducible skew-polynomials of any degree was partly characterized in \cite{odoni_1999}, and in particular they exist for any skew degree. If we choose $\psi$ to be any degree 3 skew polynomial irreducible in $\L\{\tau \}$, item (4) implies $\cN(\psi)$ is an irreducible central skew-polynomial dividing $(\tau^n - \lambda)^n$ and so $\deg_{\tau}\cN(\psi) = n$. But we must then have $n^2 = \deg_{\tau} \cN(D) + \deg_{\tau} \cN(\psi) \leq (n-3)n + n = n^2 - 2n < n^2$, giving a contradiction.

\end{proof}

\subsection{Limitations}

One of the clearest limitations of this approach is the large base field requirement $q > Cn^2 r$. Unlike the case $r = 2$, working with arbitrary degree irreducibles in $\F_q[x]$ in place of degree 1 polynomials does not appear to repair the issues discussed in lemma (\ref{detdeg}).

Lemma (\ref{detdeg}) also appears limited to only the parameter cases given in propositions (\ref{coprime}) and (\ref{rank3}). As will be seen in the following example, there exist parameter choices for $n, r,$ and Drinfeld module $\phi_x$ outside these cases where the algorithm will never work.

\begin{example}\label{examplefail}
Let $n = 3$, $r =3$, $\phi_x = \tau^3 + x$. For $(\ref{system1})$ to be uniquely solvable for some $e$, whenever $\alpha_1 \tau^3 + \alpha_2 \tau^6 = P (\tau^3 + x - e)$ we must have $P = 0$. However for any $e$ setting $\alpha_1 = 1$, $\alpha_2 = x - e $ has the non-zero solution $P = \tau^3$.
\end{example}

It is possible to detect failure cases such as those in the preceding example by checking dimensionality of the kernel of the linear system in $(\ref{reduced})$.

\subsection{Complexity.}

The algorithm can be summarized with the following routine:

%\textit{Subroutine 1}
\begin{algorithm}[H]\label{schoof}
\caption{Pseudocode for Schoof-like algorithm}
\begin{enumerate}
    \item Compute $\tau^{ni} \mod \phi_{x-e}$ in the computational basis for $\lfloor\frac{r-1}{r}n \rfloor$ distinct $e \in \F_q$ and $1 \leq i \leq r$
    \item Solve the $r \times (\hatr-1)$ system given in (\ref{system1}) for each choice of $e$. If the system $B(\phi_{x-e})$ is uniquely solvable, apply a uniformly random projection $\ell : \L \to \F_q$ to the system and solve the system over $\F_q$.
    \item Interpolate $\{a_i\}_{i=1}^{\hatr-1}$ using the $\lfloor\frac{r-1}{r}n \rfloor$ evaluation points $\{a_i(e)\}_{i=1}^{r-1}$ computed in step (2).
    \end{enumerate}
\end{algorithm}

The bottleneck of subroutine 1 is step (1). Algorithms for skew-polynomial multiplication can be leveraged to compute the entire collection of $\tau^{ni} \mod \phi_{x - e}$ in $r^{(\omega+3)/2}(n \log q)^{1 + o(1)}$ bit operations for a single $e$, with an overall contribution to the algorithm of $r^{(\omega+3)/2}(n^2 \log q)^{1 + o(1)}$. Step (2) adds $r^{\omega}n\log q$ bit operations, and (3) is of negligible cost.

A generic choice of projection $\ell$ will preserve unique solvability of $B(\phi_{x-e})$ with probability at least $1 - \frac{\hatr}{q} \geq 1 - \frac{1}{Cn^2} $. This holds for $O(n)$ independent choices of projection with probability at least $(1 - \frac{1}{Cn^2})^n \to e^{-1/(C)} \to 1$ for large $n, C$. The remaining probabilistic element is the likelihood of genericity for the choice of Drinfeld module, base field element pair $(\phi, e)$ for $n$ choices of $e$, which happens with probability $(1 - \frac{1}{Cn})^{n}$ and which can be made arbitrarily close to 1. This completes the argument for Theorem 1.

When working with the prime field case $n = m$ or when $\gcd(r, s) = 1$, the minimal polynomial is known to have degree $r$ and the routine need only be invoked once. For the general case, including parameter choices not covered by theorem \ref{schooflike}, we speculate it may be possible to invoke the algorithm for choices of $\hat{d}$ such that $r = \hat{d} r'$ with $r' | s$ and invoking probabilistic polynomial testing to verify that the output for a given $\hat{d}$ is in fact an annihilating polynomial for $\tau^n$; taking the polynomial corresponding to the least such $\hat{d}$.

%\begin{enumerate}
%    \item Initialize $r_{upper} \leftarrow r$, $r_{lower} \leftarrow 1$.
%    \item If $r_{upper} = r_{lower}$, return $r_{upper}$. Otherwise set $\hatr \leftarrow \frac{r_{upper} + r_{lower}}{2}$.
%    \item Run Subroutine 1 on the estimate $\hatr$.
%    \item If the system in (\ref{system1}) is inconsistent, $r_{lower} \leftarrow \hatr$ and go to step 2.
%    \item If the system in (\ref{system1}) returns a solution, use polynomial identity testing to determine if $\tau^{n \hatr} - \sum_{i = 0}^{\hatr - 1} \phi_{a_i}\tau^{ni} = 0$. If this equation holds, set $r_{upper} \leftarrow \hatr$, otherwise $r_{lower} \leftarrow \hatr$. Go to step 2.

%\end{enumerate}

\subsection{From Hankel Systems}
Inspired by the Monte Carlo Algorithm given in $\cite{musschost}$, we will propose an additional algorithm which doesn't depend on the genericity of the chosen Drinfeld module required by the Zippel-Schwartz criteria, and therefore may be usable when the algorithm of theorem \ref{schooflike} fails, and possesses a similar complexity.
\begin{theorem}\label{hankel}
Let $\phi$ be a rank $r$ Drinfeld module over a degree $n$ extension $L/\F_q$ such that $\gcd(n, r) = 1$. There is a Monte-Carlo algorithm computing the minimal polynomial of $\phi_x$ with bit complexity $(r^{3}n^2 \log q)^{1+o(1)}$ and which returns a correct result with probability greater than $\frac{1}{2}$ .
\end{theorem}

Note that given any algorithm achieving a probability of success greater than $\frac{1}{2}$, another algorithm that succeeds with probability $\frac{1}{2} < p < 1$ for any $p$ can be constructed with the same complexity as the original algorithm. We can organize the characteristic equation into a Hankel system.

\begin{align*}
\sum_{i=1}^{r - 1} \phi_{a_i}\tau^{ni} = &  \tau^{nr} - \phi_{a_0}
%\end{equation*}
%\begin{equation}
%\label{chareq}
\\
\sum_{i=1}^{r - 1} \sum_{j=0}^{n(r-i)/r}a_{i,j}\phi_{x}^j\tau^{ni} = & \tau^{nr} - \phi_{a_0}
\end{align*}

With $a_{i,j} \in \F_q$, and with $a_0$ determined via $(\ref{norm})$. For integers $u, v > 0$, consider the operator Hankel blocks
\begin{align*}
H_{u,v} = & \begin{bmatrix}
\phi_x^u\tau^{n} & \phi_x^u\tau^{2n} & \ldots & \phi_x^u\tau^{n (r - 1 - \lfloor(ur/n) \rfloor)} \\
\phi_x^u\tau^{2n} & \phi_x^u\tau^{3n} & \ldots & \phi_x^u\tau^{(1 + r - 1 - \lfloor(ur/n) \rfloor)n} \\ \vdots & \vdots & \ddots & \vdots \\ \phi_x^u\tau^{vn} & \phi_x^u\tau^{(1+v)n} & \ldots & \phi_x^u\tau^{(v +  r - 1 - \lfloor(ur/n) \rfloor) n}
\end{bmatrix}
\\
%\end{equation*}
%\begin{equation*}
    G_{v} = &\begin{bmatrix}
    \tau^{nr} - \phi_{a_0} & \tau^{n(r+1)} - \phi_{a_0}\tau^n & \ldots \tau^{n(r+v -1)} - \phi_{a_0}\tau^{n(v-1)}
    \end{bmatrix}^T
\end{align*}

Define
\begin{align*}
%\begin{equation*}
H:= & \begin{bmatrix}
H_{0,v_1} & H_{1, v_1} & \ldots & H_{\lfloor n (r- 1)/r \rfloor, v_1} \\ \vdots & \vdots & \ddots & \vdots \\ H_{0,v_k} & H_{1, v_k} & \ldots & H_{\lfloor n (r-1)/r \rfloor, v_k}
\end{bmatrix}\\
G:= & \begin{bmatrix}
G_{v_1} \\ \vdots \\ G_{v_k}
\end{bmatrix}
%\end{equation*}
\end{align*}

We then obtain the system

\begin{equation*}
H
\begin{bmatrix}
a_{1,0} \\ a_{2,0} \\ \vdots \\ a_{1,1} \\ a_{2,1} \\ \vdots \\ a_{1, \lfloor n (r-1)/r \rfloor}
\end{bmatrix} = G
\end{equation*}

In general, non-trivial relations between the linear maps $\phi_x^i\tau^{jn}$ acting on $\L$ ensure that the space of solutions has dimension larger than 0. To eliminate these relations, we lift to an extension $\K$ over $\L$. Let $H_{i, j}(\alpha_{\ell})$ denote the entry-wise evaluation of $H_{i, j}$ at $\alpha_{\ell}$. The goal is to choose a set of evaluation points $\alpha_1, \ldots, \alpha_k \in K$ such that the over-determined

\begin{equation*}
    H(\alpha_1, \ldots, \alpha_k) := \begin{bmatrix}
H_{0,v_1}(\alpha_1) & H_{1, v_1}(\alpha_1) & \ldots & H_{\lfloor n (r-1)/r \rfloor, v_1}(\alpha_1) \\ \vdots & \vdots & \ddots & \vdots \\ H_{0,v_k}(\alpha_k) & H_{1, v_k}(\alpha_k) & \ldots & H_{\lfloor n (r-1)/r\rfloor, v_k}(\alpha_k)
\end{bmatrix}
\end{equation*}

has a well-determined solution. Note that $H$ is an $ (v_1 + \ldots + v_k) \times \bigg( \sum_{i=1}^{n(r-1)/r} r - 1 - \lfloor(i r/n) \rfloor \bigg)$ system, and that the number of columns is a most $nr$. Let $v_1 = v_2 = \ldots = v_k = r$, $k = n$, and $[\K:\L] = r$. So $[\K:\F_q] = nr$, and observe that if the following two conditions are satisfied:

\begin{enumerate}
    \item The linear maps $\phi_x^{u}\tau^{ni}$ on $K$ are independent for $1 \leq i \leq r-1$ and $0 \leq u \leq  \frac{r-i}{r} n < n$
    \item The set $B = \{\alpha_1, \tau^n(\alpha_1), \ldots , \tau^{n(r - 1)}(\alpha_1), \ldots , \alpha_k, \tau^n(\alpha_k), \ldots , $
    $\tau^{n(r - 1)}(\alpha_k) \}$ is a basis for $K/\F_q$
\end{enumerate}

Then the equation $H(\alpha_1, \ldots, \alpha_{n})z = G$ must have a unique solution, corresponding to the coefficients of the minimal polynomial of $phi_x$. The following lemma summarizes the core of this argument.

\begin{lemma}\label{charindlem}
Let $F/G$ be a finite field extension with $[F:G] = m$ and fix a basis $\alpha_1, \ldots, \alpha_m$ for $F$ over $G$. Let $\sigma_1, \ldots, \sigma_m$ be independent linear maps over $F$, fixing $G$. Then the following matrix is has no non-trivial solutions

\begin{equation*}
 \begin{bmatrix}
\sigma_1(\alpha_1) & \ldots & \sigma_m(\alpha_1) \\ \vdots & & \vdots \\ \sigma_1(\alpha_m) & \ldots & \sigma_m(\alpha_m)
\end{bmatrix}
\end{equation*}

\end{lemma}
\begin{proof}
Suppose there exist $\Lambda = \begin{bmatrix} \lambda_1 &  \ldots & \lambda_m \end{bmatrix}^T \in F^m$ with not all $\lambda_i = 0$ such that

\begin{equation*}
 \begin{bmatrix}
\sigma_1(\alpha_1) & \ldots & \sigma_m(\alpha_1) \\ \vdots & & \vdots \\ \sigma_1(\alpha_m) & \ldots & \sigma_m(\alpha_m) \end{bmatrix} \begin{bmatrix} \lambda_1 \\ \vdots \\ \lambda_m \end{bmatrix} = \begin{bmatrix}
0 \\ \vdots \\ 0
\end{bmatrix}
\end{equation*}

Then for any row-wise re-scaling with $\ell_1, \ldots, \ell_m \in G$ we must also have

\begin{equation*}
 \begin{bmatrix}
\ell_1\sigma_1(\alpha_1) & \ldots & \ell_1\sigma_m(\alpha_1) \\ \vdots & & \vdots \\ \ell_m\sigma_1(\alpha_m) & \ldots & \ell_m\sigma_m(\alpha_m) \end{bmatrix} \begin{bmatrix} \lambda_1 \\ \vdots \\ \lambda_m \end{bmatrix} = \begin{bmatrix}
0 \\ \vdots \\ 0
\end{bmatrix}
\end{equation*}

We multiply out the left-hand side and sum the rows to obtain

\begin{equation}\label{violation}
    \sum_{i=1}^m\sum_{j=1}^m\ell_i\lambda_j\sigma_j(\alpha_i) = \sum_{j=1}^m\lambda_j\sigma_j\bigg(\sum_{i=1}^m\ell_i\alpha_i\bigg)  = 0
\end{equation}

Observe now that (\ref{violation}) holds for any $\ell_1, \ldots, \ell_m $, violating linear independence.

\end{proof}

From Lemma (\ref{charindlem}), demonstrating that the system $H(\alpha_1, \ldots, \alpha_{n})z = G$ can be uniquely solved for the coefficients of the minimal polynomial of $\phi_x$ reduces to establishing the linear independence of $\phi_x^{u}\tau^{ni}$ over $\K$.  If $\gcd(n,r) = 1$, the degree of the lead term of $\phi_x^u = C\tau^{ru} + \ldots$ has a unique residue modulo $n$ for distinct choices of $u$, and so independence of $\{ \phi_x^{u}\tau^{ni} \}_{0 \leq u \leq n(r-i)/r }^{1 \leq i < r}$ follows from independence of $id, \tau, \tau^2, \ldots, \tau^{nr-1}$. The values $\alpha_1, \ldots, \alpha_r$ can be constructed from a normal basis for $K/\F_q$ by selecting $\alpha_i = \alpha^{q^i}$ for $0 \leq i \leq r - 1$.

\subsection{Complexity.} Summarizing the preceding results, we have the following procedure:

\begin{algorithm}[H]\label{hankel}
\caption{Computing the Minimal Polynomial via a Hankel System }

\begin{enumerate}
    \item Fix a degree $r$ extension $K$ of $L$ and a normal basis for $K$ over $\F_q$
    \item Compute the Hankel Blocks $H_{i,r}(\alpha_j)$ as well as the right-hand blocks $G_r(\alpha_j)$
    \item Fix a random projection $\ell: K \to \F_q$ and apply it to the entries of $H, G$
    \item Solve the resulting $O(nr) \times nr$ system over $\F_q$.
\end{enumerate}
\end{algorithm}
Determining a normal basis of size $nr$ of $K/\F_q$ can be done in sub-quadratic time in the degree of the extension. Constructing the Hankel system above involves computing $\phi_x^{u}\tau^{ni}(\alpha_j)$ for $O(nr^2)$ choices of $0 \leq u < \frac{r-1}{r}n, 1 \leq i \leq r, 0 \leq j \leq r$. The applications of $\tau$ can be applied for $O(1)$ cost on the normal basis, while applications of $\phi_x$ take $O(nr \log q)$ bit operations. The total bit complexity for the entire procedure to compute the block Hankel system is $O(n^2r^3\log q)$. We may again apply the uniformly selected linear projection $\ell$ at an additional cost of $O(n^2r^3\log q)$. Constructing the normal basis and solving the resulting $nr \times nr$ Hankel system consisting of $n$ block columns and $r$ block rows can be done in at most $O(n^2r^2\log q)$.

\subsection{Comparison and Limitations}

Unlike in the Schoof-like case, there is no dependence on the genericity of the choice of Drinfeld module; the algorithm only fails to return a result when a poor choice of projection is made. For cases other than $\gcd(n,r) = 1$, lemma (\ref{charindlem}) can be applied by directly by testing the linear independence of characters $\{ \phi_x^{u}\tau^{ni} \}_{0 \leq u \leq n(r-i)/r }^{1 \leq i < r}$, which can be done probabilistically using polynomial identity testing.


\documentclass[a4paper]{article}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{graphicx}
\usepackage{multicol}
\usepackage{amsmath}
\usepackage{hyperref}
\usepackage{amsthm}
\usepackage{geometry}
\geometry{a4paper}
\usepackage{fancyhdr}
\usepackage{xcolor}
\begin{document}
\author{\textbf{Elshimaa Ahmed}}
\title{\textbf{Lecture 2}}
\date {\today}
\maketitle
\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]
\section{Predicates}
\paragraph{}
predicates are statements involving variables such that " $x >3$ " . These statements are neither tru or false where the value of the variable is not specified.
The statement "$x$ is greater than 3" involving two parts . the variable part $x$ and the predicate(property) it self " great than 3" .its denoted by $P(x)$  once a value is assigned to $x$ the $P(x)$ become a preposition
\newline
\newline
\textbf{For example:}
\newline
let $P(x)$ denote the statement "$x > 3$" what are truth values of $P(4)$ and $P(2)$
\newline
\newline
\textbf{Solution: } we obtain $P(4)$ by setting $x = 4$ in the statement, Hence $P(4)$  is the statement "$3 > 4$" which is $True$ . $P(2)$ is the statement "$ 2 >3$" which is $False$
\section{Quantifiers}
\begin{definition}[\textbf{Universal Quantifier}]
    the universal Quantifier of $P(x)$ is the statement "$P(X)$ is $True$ for all values of $x$ in the domain". It is denoted by $\forall x P(x)$
\begin{definition}[\textbf{Esistential Quantifier}]
    the existential quantifier of $P(x)$ is the preposition " there exist an element $x$ such that $P(x)$ is $True$" . It is denoted by $\exists x P(x)$
\end{definition}
\end{definition}

\begin{tabular}{|c|c|c|}
    \hline
    statement & when true & when false \\
    \hline
    $\forall x P(x)$ & P(x) is $True$ for all values of $x$ & there is an $x$ for which $P(x)$ is $False$\\
    $\exists x P(x)$ & there is an $x$ such that $P(x)$ is $True$ & $P(x)$ is $False$ for all $x$ \\
    \hline
\end{tabular}
\newline\newline
\textbf{English phrases with quatifiers:}
\begin{itemize}
    \item "no one is P(x)" $\longleftrightarrow$ $\forall x \neg P(x)$ or $\neg(\exists x P(x))$
    \item "not every one is P(x)"$\longleftrightarrow$ $\neg(\forall x P(x))$ or $\exists x\neg P(x)$
    \item "exactly one is P(x) " $\longleftrightarrow$ $\exists x (P(x) \wedge \forall y (P(y) \rightarrow x = y)) $
    \item "all Q(x) is P(x) " $\longleftrightarrow$ $ \forall x (Q(x) \rightarrow P(x))$
    \item "all Q(x) isnot P(x)" $\longleftrightarrow$ $\forall x (Q(x)\rightarrow \neg P(x))$
    \item "some Q(x) are P(x)" $\longleftrightarrow$ $\exists x (Q(x)\wedge P(x))$
    \item "some Q(x) are not P(x)" $\longleftrightarrow$ $\exists x (Q(x)\wedge \neg P(x))$
\newline\newline
\end{itemize}

\noindent
\textbf{Nested Quantifiers}
\newline
\newline
\begin{tabular}{|p{2cm}|p{7cm}|p{7cm}|}
    \hline
    statement & when true? &when false? \\
    \hline
    $\forall \forall y P(x,y)$ & P(x,y) is true for all possible pairs & there is a pair (x,y) such that P(x,y) is false\\
    $\forall y \forall x P(x,y)$ & &\\
    \hline
    $\forall x \exists y P(x,y)$ & for every x there is y (not necessary same for different values of x) such that P(x,y) is true & there is an x such that P(x,y) is false for every y \\
    \hline
    $\exists y \forall x P(x,y)$ & there is exist a specific (same) y that P(x,y) is true for all x & for every y there is at leasy one  x that P(x,y) fails\\
    \hline
    $\exists x \exists y P(x,y)$ & therer is a pair (x,y) for which P(x,y) is true & for all pairs (x,y) P(x,y) is False\\
    $\exists y \exists x P(x,y) $&&\\
    \hline
\end{tabular}

\end{document}