text:
# Finite Connectivity Attractor Neural Networks

## 1 Introduction

Spin models of recurrent neural networks have been studied intensively within equilibrium and non-equilibrium statistical mechanics, especially after Hopfield  emphasized their link with spin-glasses. In Hopfield’s picture, the free energy minima of glassy systems (fixed-point attractors of the dynamics) are given an information processing interpretation: in recurrent neural networks they represent stored items of information (‘patterns’), whose locations in phase space are the result of suitable modification of the neuron interactions (‘learning’). Spin glass theory, especially the replica method , was shown to be an efficient tool with which to study the equilibrium properties of Hopfield type models (or ‘attractor neural networks’) with full connectivity . It was clear from the start that full connectivity is an undesirable simplification of biological reality, made only for mathematical convenience. However, it was also clear that solving neural network models with restricted range interactions in finite dimensions $`D`$ was either pointless (no phase transitions for $`D=1`$) or impossible (for $`D>1`$). This dilemma prompted the study of so-called diluted models, where for each spin all but a randomly selected subset of size $`c`$ of interactions are removed; such can be done either while preserving interaction symmetry, or asymmetrically (in the latter case detailed balance no longer holds). Different regimes for the scaling of $`c`$ with the system size $`N`$ were found to give different physics, and to require different mathematical techniques. The early models of  correspond to $`c=N`$. The Hopfield model with so-called extreme dilution (i.e. $`lim_N\mathrm{}c^1=lim_N\mathrm{}c/N=0`$, such as $`c=\mathrm{log}N`$) was studied in  for the case of fully asymmetric dilution, and in  for the case of symmetric dilution. Both in fully connected and in symmetrically extremely diluted networks close to saturation one finds a conventional replica theory á la , with the familiar replica order parameter matrix. An alternative route away from full connectivity, which preserves both the potential for phase transitions and solvability, was followed in  (the $`1+\mathrm{}`$ dimensional attractor networks). For recent reviews of the equilibrium and non-equilibrium statistical mechanics of recurrent neural networks see e.g. .

In this paper we study the as yet unsolved class of symmetrically diluted attractor neural networks with finite connectivity, where the average number $`c`$ of bonds per neuron is finite, independent of the system size $`N`$. For bond-disordered spin systems, the finite connectivity regime (which so far has been addressed only in the context of spin glasses , error correcting codes  and satisfiability problems ) requires order parameter functions, which generalize the replica matrices of . For finite $`c`$ the replica symmetry breaking theory (RSB) is still under development . Here we apply the finite connectivity spin glass replica techniques to a general family of attractor neural networks (which includes the Hopfield model, but also the so-called clipped Hebbian synapses, as in e.g. ), within the replica symmetry ansatz (RS). We obtain phase diagrams in the $`(\alpha ,T)`$ plane, for arbitrary finite $`c`$, where $`\alpha =p/c`$ ($`p`$ giving the number of stored patterns) and $`T`$ is the temperature. These diagrams contain a paramagnetic phase (P), a recall phase (R), and a spin-glass phase (SG), all separated by second order transitions. We also analyze the RS ground state, and show how for $`c\mathrm{}`$ and arbitrary $`T`$ one recovers the simpler results of . The most surprising outcome of our calculations is the low values of the connectivity $`c`$, only barely exceeding the percolation threshold, which are required for the system to operate effectively as an attractor neural network; equivalently, the robustness of such information processing systems against excessive dilution and/or physical damage.

## 2 Model Definitions

We study Ising spin neural network models, with microscopic states defined by the $`N`$-neuron state vector $`𝝈=(\sigma _1,\mathrm{},\sigma _N)\{1,1\}^N`$. Here $`\sigma _i=1`$ if neuron $`i`$ fires, and $`\sigma _i=1`$ if it is at rest. Upon imposing standard Glauber-type dynamics, where the spins align stochastically to local fields of the form $`h_i(𝝈)=_{ji}J_{ij}\sigma _j`$ with symmetric interactions $`\{J_{ij}\}`$, these systems will evolve to thermodynamic equilibrium, described by the Hamiltonian $`H(𝝈)=_{i<j}^NJ_{ij}\sigma _i\sigma _j`$ and the associated free energy

$$F=\beta ^1\mathrm{log}Z,Z=\underset{𝝈}{}e^{\beta H(𝝈)}$$
(1)
The interactions are defined as a diluted and generalized version (in the spirit of ) of the standard Hebbian recipe, with $`c_{ij}\{0,1\}`$ and $`c_{ij}=c_{ji}`$ for all $`(i,j)`$:

$$J_{ij}=\frac{c_{ij}}{c}\varphi \left(\underset{\mu =1}{\overset{p}{}}\xi _i^\mu \xi _j^\mu \right)\xi _i^\mu \{1,1\}\mathrm{for}\mathrm{all}(i,\mu )$$
(2)
The $`p`$ vectors $`𝝃^\mu =(\xi _1^\mu ,\mathrm{},\xi _N^\mu )\{1,1\}^N`$ represent patterns to be stored in the system. The $`c_{ij}\{0,1\}`$ define the connectivity of the network, and act as quenched disorder. They are drawn randomly and independently from

$$P(c_{ij})=\frac{c}{N}\delta _{c_{ij},1}+[1\frac{c}{N}]\delta _{c_{ij},0}$$
(3)
The average number of connections to any neuron is $`c`$. Averages over the realizations of the $`\{c_{ij}\}`$ will be denoted as $`\overline{\mathrm{}}`$. The function $`\varphi (x)`$ in (2) need not be specified at this stage; for $`\varphi (x)=x`$ one returns to the symmetrically diluted Hopfield model, for $`\varphi (x)=\sqrt{p}\mathrm{sgn}(x)`$ one finds diluted and clipped Hebbian synapses, etc. The relevant and nontrivial scaling regime is that where $`lim_N\mathrm{}p/c=\alpha `$, with $`0<\alpha <\mathrm{}`$.

Models of the type (2) have so far been studied in regimes where $`lim_N\mathrm{}c^1=0`$. Here we study the more complicated scaling regime where $`c=𝒪(N^0)`$. Our objective is to solve the model by calculating the disorder-averaged free energy per spin in the thermodynamic limit: $`f=lim_N\mathrm{}(\beta N)^1\overline{\mathrm{log}Z}`$. The replica identity $`\overline{\mathrm{log}Z}=lim_{n0}n^1\mathrm{log}\overline{Z^n}`$ allows us to write this in the standard manner as

$`f`$ $`=`$ $`\underset{N\mathrm{}}{lim}\underset{n0}{lim}{\displaystyle \frac{1}{\beta Nn}}\mathrm{log}{\displaystyle \underset{𝝈^1\mathrm{}𝝈^n}{}}\overline{e^{\frac{\beta }{c}_{i<j}c_{ij}\varphi (𝝃_i𝝃_j)_\alpha \sigma _i^\alpha \sigma _j^\alpha }}`$ (4)
Due to $`c=𝒪(N^0)`$, one can expand this expression for $`N\mathrm{}`$:

$$f=\underset{N\mathrm{}}{lim}\underset{n0}{lim}\frac{1}{\beta Nn}\mathrm{log}\underset{𝝈^1\mathrm{}𝝈^n}{}\mathrm{exp}\left\{\frac{c}{2N}\underset{ij}{}\left[e^{\frac{\beta }{c}\varphi (𝝃_i𝝃_j)_\alpha \sigma _i^\alpha \sigma _j^\alpha }1\right]+𝒪(N^0)\right\}$$
(5)
In non-diluted disordered spin systems, the sum in the exponent would have been quadratic in the spin variables; there the free energy can be linearized by a gaussian transformation, leading to a single spin problem with the conventional replica order parameter matrix $`\{q_{\alpha \beta }\}`$ . In systems with finite connectivity, in contrast, one finds more complicated order parameters which encode higher order correlations between replicas . In these models it is more convenient to use an order parameter *function* , describing the distribution of spin variables in the various replicas, from which all the conventional order parameters can be derived.

## 3 Calculation of the Free Energy

### 3.1 Replica Analysis and Sub-lattice Order Parameters

To work out (5) we exploit the fact that for $`c=𝒪(N^0)`$, also $`p=𝒪(N^0)`$. This allows us to use the concept of sublattices  (of which there are $`2^p`$):

$$I_𝝃=\{i|𝝃_i=𝝃\}p_𝝃=|I_𝝃|/N$$
(6)
We define $`𝝈_i=(\sigma _i^1,\mathrm{},\sigma _i^n)`$, and henceforth abbreviate averages over the sublattices as $`f(𝝃)_𝝃=_𝝃p_𝝃f(𝝃)`$. The trace in (5) can now be written as

$$\underset{𝝈^1\mathrm{}𝝈^n}{}\mathrm{exp}\left\{\frac{c}{2N}\underset{𝝈𝝈^{}}{}\underset{𝝃𝝃^{}}{}\left[e^{\frac{\beta }{c}\varphi (𝝃𝝃^{})_\alpha \sigma _\alpha \sigma _\alpha ^{}}1\right]\underset{iI_𝝃}{}\underset{jI_𝝃^{}}{}\delta _{𝝈,𝝈_i}\delta _{𝝈^{},𝝈_j}+𝒪(N^0)\right\}$$
(7)
We next introduce a spin distribution $`P_𝝃(𝝈)`$ within each sublattice, with $`𝝈=\{\sigma ^1,\mathrm{},\sigma ^n\}`$, to be isolated upon inserting suitable $`\delta `$-distributions into (5):

$$1=\underset{𝝃𝝈}{}\left\{dP_𝝃(𝝈)\delta \left[P_𝝃(𝝈)\frac{1}{|I_𝝃|}\underset{iI_𝝃}{}\delta _{𝝈,𝝈_i}\right]\right\}$$
(8)
From these distributions, one derives the more familiar types of observables such as replicated sublattice magnetizations $`m_𝝃^\alpha `$ and replicated pattern overlaps $`m^{\mu \alpha }`$:

$$m_𝝃^\alpha =\underset{𝝈}{}P_𝝃(𝝈)\sigma ^\alpha m^{\mu \alpha }=\xi ^\mu m_𝝃^\alpha _𝝃$$
(9)
The introduction of integral representations for the delta functions in (8) generates conjugate variables $`\widehat{P}_𝝃(𝝈)`$. The trace in (7) is now performed trivially, resulting in an integral to be evaluated by steepest descent:

$`f`$ $`=`$ $`\underset{N\mathrm{}}{lim}\underset{n0}{lim}{\displaystyle \frac{1}{\beta Nn}}\mathrm{log}{\displaystyle \left[\underset{𝝃𝝈}{}dP_𝝃(𝝈)d\widehat{P}_𝝃(𝝈)\right]\mathrm{exp}\left\{\text{}iN\underset{𝝈}{}\widehat{P}_𝝃(𝝈)P_𝝃(𝝈)_𝝃\right\}}`$ (10)
$`\times \mathrm{exp}\left\{{\displaystyle \frac{cN}{2}}{\displaystyle \underset{𝝈𝝈^{}}{}}P_𝝃(𝝈)P_𝝃^{}(𝝈^{})\left[e^{\frac{\beta }{c}\varphi (𝝃𝝃^{})(𝝈𝝈^{})}1\right]_{𝝃𝝃^{}}+𝒪(N^0)\right\}`$
$`\times \mathrm{exp}\left\{N\mathrm{log}\left[{\displaystyle \underset{𝝈}{}}e^{i\widehat{P}_𝝃(𝝈)}\right]_𝝃\right\}`$
$`=`$ $`\underset{n0}{lim}{\displaystyle \frac{1}{\beta n}}\mathrm{extr}_{\{P_𝝃(𝝈),\widehat{P}_𝝃(𝝈)\}}\{i{\displaystyle \underset{𝝈}{}}\widehat{P}_𝝃(𝝈)P_𝝃(𝝈)_𝝃+\mathrm{log}\left[{\displaystyle \underset{𝝈}{}}e^{i\widehat{P}_𝝃(𝝈)}\right]_𝝃`$
$`+{\displaystyle \frac{1}{2}}c{\displaystyle \underset{𝝈𝝈^{}}{}}P_𝝃(𝝈)P_𝝃^{}(𝝈^{})[e^{\frac{\beta }{c}\varphi (𝝃𝝃^{})(𝝈𝝈^{})}1]_{𝝃𝝃^{}}\}`$
Variation with respect to $`\{P_𝝃(𝝈)\}`$ gives an equation with which to eliminate the conjugate order parameters, resulting in $`f=lim_{n0}\mathrm{extr}_{\{P_𝝃(𝝈)\}}f[\{P_𝝃(𝝈)\}]`$, where

$`f[\mathrm{}]`$ $`=`$ $`{\displaystyle \frac{c}{2\beta n}}{\displaystyle \underset{𝝈𝝈^{}}{}}P_𝝃(𝝈)P_𝝃^{}(𝝈^{})\left[e^{\frac{\beta }{c}\varphi (𝝃𝝃^{})(𝝈𝝈^{})}1\right]_{𝝃𝝃^{}}`$ (11)
$`{\displaystyle \frac{1}{\beta n}}\mathrm{log}\left[{\displaystyle \underset{𝝈}{}}\mathrm{exp}\left\{c{\displaystyle \underset{𝝈^{}}{}}P_𝝃^{}(𝝈^{})\left[e^{\frac{\beta }{c}\varphi (𝝃𝝃^{})(𝝈𝝈^{})}1\right]_𝝃^{}\right\}\right]_𝝃`$
Further variation, again with respect to $`P_𝝃(𝝈)`$, then leads to the saddle point equation

$`P_𝝃(𝝈)={\displaystyle \frac{\mathrm{exp}\left\{c_𝝈^{}P_𝝃^{}(𝝈^{})\left[e^{\frac{\beta }{c}\varphi (𝝃𝝃^{})(𝝈𝝈^{})}1\right]_𝝃^{}\right\}}{_𝝈^{}\mathrm{exp}\left\{c_{𝝈^{\prime \prime }}P_{𝝃^{\prime \prime }}(𝝈^{\prime \prime })\left[e^{\frac{\beta }{c}\varphi (𝝃^{\prime \prime }𝝃)(𝝈^{\prime \prime }𝝈^{})}1\right]_{𝝃^{\prime \prime }}\right\}}}`$ (12)
When deriving (12) it first appears that one must also allow for adding to $`\{P_𝝃(𝝈)\}`$ any vector in the kernel of the $`2^{p+n}\times 2^{p+n}`$ matrix $`U(𝝃,𝝈;𝝃^{},𝝈^{})=\mathrm{exp}[\frac{\beta }{c}\varphi (𝝃𝝃^{})(𝝈𝝈^{})]1`$. However, adding such elements is seen to generate new expressions for $`\{P_𝝃(𝝈)\}`$ which again solve (12); hence (12) is always satisfied at the saddle-point.

An alternative route for calculating the free energy will turn out to be more convenient for calculating zero temperature properties, such as ground state energy and entropy. Here we first vary (10) with respect to $`\widehat{P}_𝝃(𝝈)`$, resulting in $`f=lim_{n0}\mathrm{extr}_{\{P_𝝃(𝝈)\}}f[\{P_𝝃(𝝈)\}]`$ subject to the constraints $`_𝝈P_𝝃(𝝈)=1`$ for all $`𝝃`$, with

$`f[\mathrm{}]`$ $`=`$ $`{\displaystyle \frac{c}{2\beta n}}{\displaystyle \underset{𝝈𝝈^{}}{}}P_𝝃(𝝈)P_𝝃^{}(𝝈^{})\left[e^{\frac{\beta }{c}\varphi (𝝃𝝃^{})(𝝈𝝈^{})}1\right]_{𝝃𝝃^{}}`$ (13)
$`+{\displaystyle \frac{1}{\beta n}}{\displaystyle \underset{𝝈}{}}P_𝝃(𝝈)\mathrm{log}P_𝝃(𝝈)_𝝃`$
The conjugate order parameter functions $`\widehat{P}_𝝃(𝝈)`$ are now found to act as Lagrange multipliers, imposing the normalization constraints on the distributions $`P_𝝃(𝝈)`$.

### 3.2 RS Order Parameter Equations and Free Energy

In order to take the $`n0`$ limit we make the replica symmetric (RS) ansatz for the order parameters. One anticipates RS to be broken in the low temperature phases , but to hold up to the first phase transition away from the paramagnetic phase, as in the Viana-Bray model . In the present context, RS is equivalent to assuming that the $`P_𝝃(𝝈)`$ are invariant under permutations of the components of $`𝝈`$, i.e. only depend on the sum $`_\alpha \sigma ^\alpha `$. Upon defining effective fields , viz. $`h_i\frac{1}{\beta }\mathrm{tanh}^1\sigma _i^\alpha _\beta `$, our RS order parameter functions can thus be written  as

$$P_𝝃(𝝈)=𝑑hW_𝝃(h)\frac{e^{\beta h_\alpha \sigma ^\alpha }}{[2\mathrm{cosh}(\beta h)]^n}$$
(14)
Here $`W_𝝃(h)0`$ and $`𝑑hW_𝝃(h)=1`$. One can write sub-lattice magnetizations and higher order observables in terms of the $`W_𝝃(h)`$:

$$\underset{𝝈}{}P_𝝃(𝝈)\sigma ^\alpha \sigma ^\beta \mathrm{}\sigma ^r=𝑑hW_𝝃(h)\mathrm{tanh}^r(\beta h)(\alpha <\beta <\mathrm{}<r)$$
(15)
The pattern overlaps $`m^{\mu \alpha }`$ are now independent of $`\alpha `$, $`m^{\mu \alpha }=m_\mu `$, where

$$m_\mu =\xi ^\mu 𝑑hW_𝝃(h)\mathrm{tanh}(\beta h)_𝝃$$
(16)
We substitute (14) into (12), and isolate the occurrences of $`_\alpha \sigma ^\alpha `$ by inserting

$$1=\underset{m=\mathrm{}}{\overset{\mathrm{}}{}}_0^{2\pi }\frac{d\widehat{m}}{2\pi }e^{i\widehat{m}(m_\alpha \sigma ^\alpha )}$$
(17)
After some mostly straightforward manipulations111The only technical subtlety is the need to take $`n0`$ for even $`n`$, to avoid in the denominator of (12) tricky terms like $`\mathrm{log}\mathrm{cos}(\widehat{m})`$ with $`\widehat{m}<0`$. The same issue arises when calculating the RS free energy. one can then take the $`n0`$ limit and find (12) converting into:

$$𝑑hW_𝝃(h)e^{\beta hm}=\mathrm{exp}\left\{c𝑑h^{}W_𝝃^{}(h^{})\left[e^{m\mathrm{tanh}^1[\mathrm{tanh}(\beta h^{})\mathrm{tanh}(\frac{\beta }{c}\varphi (𝝃𝝃^{}))]}1\right]_𝝃^{}\right\}$$
(18)
This is to hold for any real $`m`$. Provided the various integrals exist, we may now also put $`mim/\beta `$ and carry out an inverse Fourier transform (following ), leading to

$`W_𝝃(h)`$ $`=`$ $`{\displaystyle \frac{dm}{2\pi }e^{imh}}`$ (19)
$`\times \mathrm{exp}\left\{c{\displaystyle 𝑑h^{}W_𝝃^{}(h^{})\left[e^{\frac{im}{\beta }\mathrm{tanh}^1[\mathrm{tanh}(\beta h^{})\mathrm{tanh}(\frac{\beta }{c}\varphi (𝝃𝝃^{}))]}1\right]}_𝝃^{}\right\}`$
These are the final equations from which to solve the RS order parameters, i.e. the $`2^p`$ effective field distributions $`W_𝝃(h)`$.

Upon inserting the RS ansatz (14) into expression (11) for the free energy, one can again take the limit $`n0`$ (provided $`n`$ is even). However, the calculation of $`f`$ is found to be easier using the alternative expression (13) (see A for details). The result of the calculation is

$`f`$ $`=`$ $`{\displaystyle \frac{c}{2\beta }}{\displaystyle 𝑑h𝑑h^{}W_𝝃(h)W_𝝃^{}(h^{})\mathrm{log}\left[1+\mathrm{tanh}(\beta h)\mathrm{tanh}(\beta h^{})\mathrm{tanh}(\frac{\beta }{c}\varphi (𝝃𝝃^{}))\right]}_{𝝃𝝃^{}}`$ (20)
$`{\displaystyle \frac{c}{2\beta }}\mathrm{log}\mathrm{cosh}[{\displaystyle \frac{\beta }{c}}\varphi (𝝃𝝃^{})]_{𝝃𝝃^{}}{\displaystyle \frac{1}{\beta }}{\displaystyle 𝑑hW_𝝃(h)\mathrm{log}[2\mathrm{cosh}(\beta h)]}_𝝃`$
$`{\displaystyle \frac{c}{2\beta }}{\displaystyle 𝑑hW_𝝃(h)\mathrm{log}\left[1\mathrm{tanh}^2(\beta h)\mathrm{tanh}^2(\frac{\beta }{c}\varphi (𝝃𝝃^{}))\right]}_{𝝃𝝃^{}}`$
In deriving (20), which is reassuringly similar but not identical to the corresponding expression derived for the RS free energy of spin glasses with random interactions , we have used $`W_𝝃(h)`$ solving (19) and being normalized (for every $`𝝃`$).

As a simple test of our RS results, one may inspect the limit $`c\mathrm{}`$ for $`\varphi (x)=x`$. Here the interactions are of the Hopfield type, and, since $`p`$ is finite, one should recover the equations describing the Hopfield model away from saturation. Expansion of the saddle-point equation (19) in powers of $`1/c`$, keeping only the $`𝒪((\frac{1}{c})^0)`$ terms, results in

$$W_𝝃(h)=\delta (h\underset{\mu }{}\xi ^\mu m^\mu )$$
(21)
Upon using this expression to calculate the overlaps $`m^\mu =\xi ^\mu 𝑑hW_𝝃(h)\mathrm{tanh}(\beta h)_𝝃`$ one indeed recovers the saddle-point equations of the Hopfield model away from saturation. Similarly, (20) reduces to the correct corresponding free energy.

## 4 Analysis of Phase Transitions

### 4.1 Expansion of the saddle-point equations

The paramagnetic state $`W_𝝃(h)=\delta (h)𝝃`$ always solves (19). As the temperature is lowered from $`T=\mathrm{}`$, we expect other solutions to bifurcate away from the paramagnetic one. In finite connectivity spin glasses, phase transitions were found upon expanding the RS free energy in the order parameters (15) near the paramagnetic phase . A similar strategy can be applied here, following . We assume that, close to the transition, $`𝑑hW_𝝃(h)h^{\mathrm{}}=𝒪(ϵ^{\mathrm{}})`$ with $`0<ϵ1`$, and we expand both sides of (18) in powers of $`ϵ`$. This can be done self-consistently, for all al orders in $`ϵ`$. Identification of the lowest two orders $`ϵ`$ and $`ϵ^2`$ is then found to give, respectively

$`{\displaystyle 𝑑hW_𝝃(h)h}=c{\displaystyle 𝑑hW_𝝃^{}(h)h\mathrm{tanh}[\frac{\beta }{c}\varphi (𝝃𝝃^{})]}_𝝃^{}`$ (22)
$`{\displaystyle 𝑑hW_𝝃(h)h^2}\left[{\displaystyle 𝑑hW_𝝃(h)h}\right]^2=c{\displaystyle 𝑑hW_𝝃^{}(h)h^2\mathrm{tanh}^2[\frac{\beta }{c}\varphi (𝝃𝝃^{})]}_𝝃^{}`$ (23)
There are hence two types of transitions away from the paramagnetic (P) state. The first corresponds to the lowest order being $`ϵ`$; such transitions are marked by the appearance of nonzero solutions of (22). The second type have $`ϵ^2`$ as lowest non-zero order; these transitions are marked by non-trivial solutions of (23) with $`𝑑hW_𝝃(h)h=0`$. We conclude that these transitions are marked by the highest temperature for which the two $`2^p\times 2^p`$ matrices $`M_{𝝃𝝃^{}}`$ and $`Q_{𝝃𝝃^{}}`$, respectively, have an eigenvalue equal to 1:

$$M_{𝝃𝝃^{}}=cp_𝝃^{}\mathrm{tanh}[\frac{\beta }{c}\varphi (𝝃𝝃^{})]Q_{𝝃𝝃^{}}=cp_𝝃^{}\mathrm{tanh}^2[\frac{\beta }{c}\varphi (𝝃𝝃^{})]$$
(24)
It follows from (16) that bifurcation of $`M`$ eigenvectors corresponds to a transition towards a retrieval state (R), with nonzero pattern overlaps, whereas bifurcation of $`Q`$ eigenvectors corresponds to a transition towards a spin-glass state (SG).

### 4.2 Transition Temperatures for Random Patterns

For independently drawn random patterns, where $`p_𝝃=2^p`$ for all $`𝝃`$, both matrices in (24) are symmetric and can be diagonalized by exploiting symmetries. The desired eigenvectors have in fact already been calculated in  (in a different context). They are found to be universal for all $`2^p\times 2^p`$ symmetric matrices whose entries depend only on $`𝝃𝝃^{}`$, i.e. $`U_{𝝃𝝃^{}}=U(𝝃𝝃^{})`$. With each of the $`2^p`$ index subsets $`S\{1,2,\mathrm{},p\}`$ one can associate an eigenvector $`v_S`$, defined as $`v_S(𝝃)=_{\mu S}\xi ^\mu `$. The eigenvector corresponding to the empty set is defined as $`v_{\mathrm{}}(𝝃)=1`$ for all $`𝝃`$. One easily verifies that this gives all eigenvectors of $`U`$, with corresponding eigenvalues

$$\lambda _S=\underset{𝝃}{}U(\underset{\nu =1}{\overset{p}{}}\xi ^\nu )\underset{\mu S}{}\xi ^\mu $$
(25)
The eigenvalues depend only on the size $`|S|`$ of the subset. Application of (25) to our matrices $`M`$ and $`Q`$ gives the following eigenvalue spectra:

$$\lambda _S^M=c\left[\underset{\mu S}{}\xi ^\mu \right]\mathrm{tanh}[\frac{\beta }{c}\varphi (\underset{\nu }{}\xi _\nu )]_𝝃\lambda _S^Q=c\left[\underset{\mu S}{}\xi ^\mu \right]\mathrm{tanh}^2[\frac{\beta }{c}\varphi (\underset{\nu }{}\xi _\nu )]_𝝃$$

Clearly, $`lim_{\beta 0}\lambda _S^M=lim_{\beta 0}\lambda _S^Q=0`$. For monotonically non-decreasing functions $`\varphi (x)`$, the largest $`M`$ eigenvalue is found for $`|S|=1`$, whereas the largest $`Q`$ eigenvalue corresponds to $`S=\mathrm{}`$:

$`\lambda _{\mathrm{max}}^M`$ $`=`$ $`{\displaystyle \frac{c}{p}}({\displaystyle \underset{\mu }{}}\xi ^\mu )\mathrm{tanh}[{\displaystyle \frac{\beta }{c}}\varphi ({\displaystyle \underset{\nu }{}}\xi ^\nu )]_𝝃`$ (26)
$`\lambda _{\mathrm{max}}^Q`$ $`=`$ $`c\mathrm{tanh}^2[{\displaystyle \frac{\beta }{c}}\varphi ({\displaystyle \underset{\mu }{}}\xi ^\mu )]_𝝃`$ (27)
This leads us to the following equations for the P$``$R and P$``$SG transition lines:

$`\mathrm{P}\mathrm{R}:`$ $`{\displaystyle \frac{c}{p}}2^p{\displaystyle \underset{n=0}{\overset{p}{}}}(\begin{array}{c}p\\ n\end{array})(p2n)\mathrm{tanh}[{\displaystyle \frac{\beta }{c}}\varphi (p2n)]=1`$ (30)
$`\mathrm{P}\mathrm{SG}:`$ $`c2^p{\displaystyle \underset{n=0}{\overset{p}{}}}(\begin{array}{c}p\\ n\end{array})\mathrm{tanh}^2[{\displaystyle \frac{\beta }{c}}\varphi (p2n)]=1`$ (33)
Equations (30,33) will generally have to be solved numerically.

We may finally inspect the limit $`c\mathrm{}`$, with $`\alpha =p/c`$ fixed, where we expect to find a generalization of the results in . The central limit theorem now allows us to replace $`p^{\frac{1}{2}}_{\mu =1}^p\xi ^\mu `$ by a zero-average and unit-variance Gaussian variable, so (26,27) simplify to (provided the limits exist, and with $`Dy=(2\pi )^{\frac{1}{2}}e^{\frac{1}{2}y^2}`$):

$`\underset{p\mathrm{}}{lim}\lambda _{\mathrm{max}}^M`$ $`=`$ $`{\displaystyle \frac{1}{\alpha }}{\displaystyle Dyy\underset{p\mathrm{}}{lim}\left\{\sqrt{p}\mathrm{tanh}[\frac{\alpha \beta }{p}\varphi (y\sqrt{p})]\right\}}`$ (34)
$`\underset{p\mathrm{}}{lim}\lambda _{\mathrm{max}}^Q`$ $`=`$ $`{\displaystyle \frac{1}{\alpha }}{\displaystyle Dy\underset{p\mathrm{}}{lim}\left\{p\mathrm{tanh}^2[\frac{\alpha \beta }{p}\varphi (y\sqrt{p})]\right\}}`$ (35)
### 4.3 Application to Specific Synaptic Kernels

We now have to make explicit choices for the kernel $`\varphi (x)`$ in our expression for the interaction matrix, viz. $`J_{ij}=\frac{c_{ij}}{c}\varphi (𝝃_i𝝃_j)`$. We first inspect the finite connectivity Hopfield model $`J_{ij}=\frac{c_{ij}}{c}𝝃_i𝝃_j`$. Here we have $`\varphi (x)=x`$, and the $`c\mathrm{}`$ expressions for the P$``$R and P$``$SG transition temperatures (34,35) reduce to $`T_R=1`$ and $`T_{SG}=\sqrt{\alpha }`$ (in accordance with the extreme dilution results in ). In addition one finds that for $`c\mathrm{}`$ the effective field distributions $`W_𝝃(h)`$ become Gaussian, with mean $`\overline{h}_𝝃=𝑑hW_𝝃^{}\mathrm{tanh}(\beta h)(𝝃𝝃^{})_𝝃^{}`$ and variance $`\sigma ^2=\alpha 𝑑hW_𝝃^{}\mathrm{tanh}^2(\beta h)(𝝃𝝃^{})_𝝃^{}\overline{h}_𝝃^2`$, from which one immediately recovers the RS order parameter equations of  (describing states with a single ‘condensed’ pattern, for the scaling regime $`c\mathrm{}`$ with $`c/N0`$):

$$m=Dy\mathrm{tanh}[\beta (m+y\sqrt{\alpha q})]q=Dy\mathrm{tanh}^2[\beta (m+y\sqrt{\alpha q})]$$

The results of solving (30,33) numerically for finite $`c`$ are shown in figure 1. Note that $`\alpha =p/c\{\frac{1}{c},\frac{2}{c},\frac{3}{c},\mathrm{}\}`$, so only the actual markers in figure 1 correspond to physically realizable parameter values (the connecting line segments just provide a guide to the eye). We observe that for $`c=100`$ the transition lines away from the paramagnetic phase are already nearly identical to those corresponding to $`c\mathrm{}`$. For $`c=2`$, the system remains in a paramagnetic phase up to $`T=0`$ for $`\alpha =1`$. Here, interestingly, for $`\alpha =0.5`$ (i.e. $`p=1`$) there is a transition to a nonzero retrieval overlap at finite $`T`$. This is not what one would find in e.g. a $`1d`$ Ising chain, in which the connectivity is $`c=2`$, but where there is no phase transition at finite $`T`$; the difference is due to the fact that in our present model the connectivity equals $`2`$ only *on average*. Note that the value $`c=1`$ corresponds to the percolation transition . We will turn to the location of the more complicated R$``$SG transition in a subsequent section.

Our second application is obtained upon choosing a finite connectivity version of the so-called clipped Hebbian synapses: $`J_{ij}=\frac{c_{ij}}{c}\sqrt{p}\mathrm{sgn}(𝝃_i𝝃_j)`$ (the specific scaling with $`\sqrt{p}`$ ensures a proper limit $`c\mathrm{}`$). Here $`\varphi (x)=\sqrt{p}\mathrm{sgn}(x)`$, and the $`c\mathrm{}`$ expressions for the P$``$R and P$``$SG transition temperatures (34,35) now reduce to $`T_R=\sqrt{2/\pi }`$ and $`T_{SG}=\sqrt{\alpha }`$. The results of solving (30,33) numerically for finite $`c`$ are shown in figure 2. As before, only the actual markers in figure 2 correspond to physically realizable parameter values. As was the case for full connectivity , also with finite connectivity one finds, surprisingly from an information processing point of view, that the differences between Hebbian and clipped Hebbian synapses are only of a quantitative nature; limited largely to a rescaling of critical temperatures.

Our third and final finite connectivity network example is one which interpolates between Hebbian and clipped Hebbian synapses: $`\varphi (x)=x`$ (i.e. Hebbian) for $`|x|<\sqrt{p}`$, and $`\varphi (x)=\sqrt{p}\mathrm{sgn}(x)`$ (i.e. clipped Hebbian) for $`|x|\sqrt{p}`$. The value $`\sqrt{p}`$ is found to be the natural and most interesting scaling for the cut-off point in this definition. The $`c\mathrm{}`$ expressions for the P$``$R and P$``$SG transition temperatures (34,35) now reduce to $`T_R=\mathrm{Erf}(1/\sqrt{2})`$ and $`T_{SG}=[\alpha (1\sqrt{2/\pi e})]^{\frac{1}{2}}`$. The results of solving (30,33) numerically for finite $`c`$ are shown in figure 3. Again only the markers correspond to physically realizable parameter values. The apparent irregularities in the transition lines are due to the cut-off in the function $`\varphi (x)`$ at $`|𝝃𝝃^{}|=\sqrt{p}`$.

## 5 RS Ground State of the Finite Connectivity Hopfield Model

For large $`\alpha `$, one expects replica symmetry to break at low temperatures. In contrast to full connectivity models, for finite connectivity models RSB theory is still under development. Here we will therefore analyze as yet only the $`T0`$ limit of our RS equations. Apart from giving exact statements at least for small $`\alpha `$, a calculation of the RS ground state entropy leads for large $`\alpha `$ to a bound for the location of the RSB transition in the phase diagram, since the latter must come before the zero entropy line.

### 5.1 RS Order Parameters at $`T=0`$

We follow closely the RS ground state analysis carried out for finite connectivity spin glasses . At $`T=0`$ there are no thermal fluctuations, so the effective fields in (14) are identical to the true local fields. The latter can take only discrete values, due to the finite number of connections per spin, hence each $`W_𝝃(h)`$ is a sum of delta peaks:

$$W_𝝃(h)=\underset{\mathrm{}=\mathrm{}}{\overset{\mathrm{}}{}}K_𝝃(\mathrm{})\delta (h\frac{\mathrm{}}{c})$$
(36)
Upon taking the limit $`\beta \mathrm{}`$ in (19), using $`lim_\beta \mathrm{}\beta ^1\mathrm{tanh}^1[\mathrm{tanh}(\beta x)\mathrm{tanh}(\beta y)]=\frac{1}{2}|x+y|\frac{1}{2}|xy|`$, one verifies by substitution that (36) is indeed a solution. Insertion into the right hand side of (19) gives (with $`\overline{\delta }_{nm}=1\delta _{nm}`$):

$`W_𝝃(h)`$ $`=`$ $`{\displaystyle }{\displaystyle \frac{dy}{2\pi }}e^{iyh}\mathrm{exp}(c\{{\displaystyle \underset{\mathrm{}^{}=1}{\overset{|𝝃𝝃^{}|1}{}}}[K_𝝃^{}(\mathrm{}^{})e^{\frac{iy\mathrm{}^{}}{c}\mathrm{sgn}(𝝃𝝃^{})}+K_𝝃^{}(\mathrm{}^{})e^{\frac{iy\mathrm{}^{}}{c}\mathrm{sgn}(𝝃𝝃^{})}]`$ (37)
$`+{\displaystyle \underset{\mathrm{}^{}=|𝝃𝝃^{}|}{\overset{\mathrm{}}{}}}[K_𝝃^{}(\mathrm{}^{})e^{\frac{iy}{c}𝝃𝝃^{}}+K_𝝃^{}(\mathrm{}^{})e^{\frac{iy}{c}𝝃𝝃^{}}][1K_𝝃^{}(0)]\}\overline{\delta }_{𝝃𝝃^{},0}]_𝝃^{})`$
Upon expanding the exponent in the right-hand side, we would indeed recognize a sum of delta peaks. Integrating of both sides over an infinitesimally small interval around $`h=\mathrm{}/c`$ leads to equations for the factors $`K_𝝃^\pm (\mathrm{})`$:

$`K_𝝃(\mathrm{})`$ $`=`$ $`\underset{ϵ0}{lim}{\displaystyle }{\displaystyle \frac{dz}{\pi z}}\mathrm{sin}(z)e^{\frac{i\mathrm{}z}{cϵ}}\mathrm{exp}(c\{{\displaystyle \underset{\mathrm{}^{}=1}{\overset{|𝝃𝝃^{}|1}{}}}[K_𝝃^{}(\mathrm{}^{})e^{\frac{iz\mathrm{}^{}}{cϵ}\mathrm{sgn}(𝝃𝝃^{})}+K_𝝃^{}(\mathrm{}^{})e^{\frac{iz\mathrm{}^{}}{cϵ}\mathrm{sgn}(𝝃𝝃^{})}]`$ (38)
$`+{\displaystyle \underset{\mathrm{}^{}=|𝝃𝝃^{}|}{\overset{\mathrm{}}{}}}[K_𝝃^{}(\mathrm{}^{})e^{\frac{iz}{cϵ}𝝃𝝃^{}}+K_𝝃^{}(\mathrm{}^{})e^{\frac{iz}{cϵ}𝝃𝝃^{}}][1K_𝝃^{}(0)]\}\overline{\delta }_{𝝃𝝃^{},0}_𝝃^{})`$
For $`ϵ0`$, the periodic part of the integral (involving $`z/cϵ`$) oscillates rapidly and decouples from the non-oscillating part. We note that (38) is of the form $`lim_{ϵ0}\frac{dz}{\pi z}\mathrm{sin}(z)f(z/ϵ)`$, where $`f(z+2\pi )=f(z)`$. The function $`f`$ may be written as a Fourier series, $`f(z/ϵ)=_{n=\mathrm{}}^{\mathrm{}}a_ne^{inz/ϵ}`$. Consequently:

$$\underset{ϵ0}{lim}\frac{dz}{\pi z}\mathrm{sin}(z)f(\frac{z}{ϵ})=a_0\frac{dz}{\pi z}\mathrm{sin}(z)+\underset{n0}{}a_n\underset{ϵ0}{lim}\frac{dz}{\pi z}\mathrm{sin}(ϵz)e^{inz}=a_0$$

Since $`a_0=(2\pi )^1_\pi ^\pi 𝑑\varphi f(\varphi )`$, expression (38) can be simplified to

$`K_𝝃(\mathrm{})`$ $`=`$ $`e^{c[1K_𝝃^{}(0)]\overline{\delta }_{𝝃𝝃^{},0}_𝝃^{}}{\displaystyle _\pi ^\pi }{\displaystyle \frac{d\varphi }{2\pi }}\mathrm{exp}(c\{{\displaystyle \underset{\mathrm{}^{}=1}{\overset{|𝝃𝝃^{}|1}{}}}[K_𝝃^{}(\mathrm{}^{})+K_𝝃^{}(\mathrm{}^{})]\mathrm{cos}(\mathrm{}^{}\varphi )`$ (39)
$`+{\displaystyle \underset{\mathrm{}^{}=|𝝃𝝃^{}|}{\overset{\mathrm{}}{}}}[K_𝝃^{}(\mathrm{}^{})+K_𝝃^{}(\mathrm{}^{})]\mathrm{cos}(𝝃𝝃^{}\varphi )\}\overline{\delta }_{𝝃𝝃^{},0}_𝝃^{})`$
$`\times \mathrm{cos}(\mathrm{}\varphi +c\{{\displaystyle \underset{\mathrm{}^{}=1}{\overset{|𝝃𝝃^{}|1}{}}}[K_𝝃^{}(\mathrm{}^{})K_𝝃^{}(\mathrm{}^{})]\mathrm{sin}(\mathrm{}^{}\varphi )\mathrm{sgn}(𝝃𝝃^{})`$
$`+{\displaystyle \underset{\mathrm{}^{}=|𝝃𝝃^{}|}{\overset{\mathrm{}}{}}}[K_𝝃^{}(\mathrm{}^{})K_𝝃^{}(\mathrm{}^{})]\mathrm{sin}(𝝃𝝃^{}\varphi )\}\overline{\delta }_{𝝃𝝃^{},0}_𝝃^{})`$
Solutions to this set of equations can be found numerically, via iteration.

### 5.2 RS Entropy at $`T=0`$

We next calculate the RS zero temperature entropy per spin $`s_0`$, by expanding the free energy per spin up to order $`T`$:

$$f=f_{T=0}s_0T+𝒪(T^2)$$
(40)
Care is needed in taking the temperature derivative of the free energy, in view of the normalization constraint on the order parameters $`W_𝝃(h)`$. One generally has

$`{\displaystyle \frac{df}{dT}}={\displaystyle \frac{f[\mathrm{}]}{T}}|_{\mathrm{saddle}}+{\displaystyle \underset{𝝃^{}}{}}{\displaystyle }dh^{}{\displaystyle \frac{\delta f[\mathrm{}]}{\delta W_𝝃^{}(h^{})}}|_{\mathrm{saddle}}.{\displaystyle \frac{W_𝝃^{}^{}(h^{})}{T}}`$ (41)
where $`W^{}`$ denotes the saddle-point of $`f[\mathrm{}]`$. In contrast to unconstrained extremization, the functional derivative in the right hand side of (41) need not vanish, since extremization of $`f`$ is restricted to the subspace in which $`𝑑hW_𝝃(h)=1`$:

$$\frac{\delta f[\mathrm{}]}{\delta W_𝝃(h)}+\underset{𝝃^{}}{}\lambda _𝝃^{}\frac{\delta }{\delta W_𝝃(h)}\left[𝑑h^{}W_𝝃^{}(h^{})1\right]=0$$
(42)
Here $`\{\lambda _𝝃^{}\}`$ are Lagrange multipliers. As a consequence, we find $`\delta f[\mathrm{}]/\delta W_𝝃(h)=\lambda _𝝃`$ at the saddle point (i.e. independent of the field $`h`$), and hence

$`{\displaystyle \frac{df}{dT}}`$ $`=`$ $`{\displaystyle \frac{f[\mathrm{}]}{T}}|_{\mathrm{saddle}}+{\displaystyle \underset{𝝃^{}}{}}\lambda _𝝃^{}{\displaystyle \frac{}{T}}{\displaystyle 𝑑h^{}W_𝝃^{}^{}(h^{})}={\displaystyle \frac{f[\mathrm{}]}{T}}|_{\mathrm{saddle}}`$ (43)
For the purpose of calculating $`s_0`$, we may thus simply insert the $`T=0`$ saddle-point (36,39) into our expression (20) for the free energy, giving:

$`s_0`$ $`=`$ $`\underset{\beta \mathrm{}}{lim}\beta ^2{\displaystyle \frac{}{\beta }}{\displaystyle \frac{c}{2\beta }}{\displaystyle \underset{\mathrm{},\mathrm{}^{}0}{}}K_𝝃(\mathrm{})K_𝝃^{}(\mathrm{}^{})\mathrm{log}[1+\mathrm{tanh}({\displaystyle \frac{\beta \mathrm{}}{c}})\mathrm{tanh}({\displaystyle \frac{\beta \mathrm{}^{}}{c}})\mathrm{tanh}({\displaystyle \frac{\beta }{c}}𝝃𝝃^{})]`$ (44)
$`{\displaystyle \frac{c}{2\beta }}{\displaystyle \underset{\mathrm{}0}{}}K_𝝃(\mathrm{})\left[\mathrm{log}[1+\mathrm{tanh}({\displaystyle \frac{\beta \mathrm{}}{c}})\mathrm{tanh}({\displaystyle \frac{\beta }{c}}𝝃𝝃^{})]+\mathrm{log}[1\mathrm{tanh}({\displaystyle \frac{\beta \mathrm{}}{c}})\mathrm{tanh}({\displaystyle \frac{\beta }{c}}𝝃𝝃^{})]\right]`$
$`{\displaystyle \frac{c}{2\beta }}\mathrm{log}\mathrm{cosh}({\displaystyle \frac{\beta }{c}}𝝃𝝃^{}){\displaystyle \frac{1}{\beta }}{\displaystyle \underset{\mathrm{}0}{}}K_𝝃(\mathrm{})\mathrm{log}[2\mathrm{cosh}({\displaystyle \frac{\beta \mathrm{}}{c}})]_{𝝃𝝃^{}}`$
$`=`$ $`{\displaystyle \frac{c}{2}}\mathrm{log}2{\displaystyle \underset{\mathrm{}=0}{\overset{|𝝃𝝃^{}|1}{}}}\{K_𝝃(|𝝃𝝃^{}|)+K_𝝃(|𝝃𝝃^{}|)[K_𝝃(\mathrm{})+K_𝝃(\mathrm{})][1{\displaystyle \frac{1}{2}}\delta _{\mathrm{},0}]\}`$
$`\times [K_𝝃^{}(\mathrm{})+K_𝝃^{}(\mathrm{})][1{\displaystyle \frac{1}{2}}\delta _{\mathrm{},0}]\overline{\delta }_{𝝃𝝃^{},0}_{𝝃𝝃^{}}+\mathrm{log}2K_𝝃(0)_𝝃`$
$`+{\displaystyle \frac{c}{4}}[2\mathrm{log}2\mathrm{log}3][K_𝝃(|𝝃𝝃^{}|)+K_𝝃(|𝝃𝝃^{}|)]`$
$`\times [K_𝝃^{}(|𝝃𝝃^{}|)+K_𝝃^{}(|𝝃𝝃^{}|)]\overline{\delta }_{𝝃𝝃^{},0}_{𝝃𝝃^{}}`$
This result is reminiscent of the corresponding expression found for spin glasses in .

### 5.3 Zero Temperature Phase Transitions

We next try to solve the coupled equations (39), for randomly drawn patterns, at zero temperature. We know that in the case of perfect retrieval of a pattern $`\mu `$, half of the sublattice magnetizations (the ones with $`\xi ^\mu =1`$) must be $`1`$ and the other half must be $`1`$. At $`T=0`$ all spins align with their local fields, so it follows from (36) that $`K_𝝃(\mathrm{})=K(\xi ^\mu \mathrm{})`$ for some set of non-negative numbers $`K(\mathrm{})`$. We will now choose the property $`K_𝝃(\mathrm{})=K(\xi ^\mu \mathrm{})`$ for all $`𝝃\{1,1\}`$ and all integer $`\mathrm{}`$ as an ansatz. Upon restricting ourselves for simplicity to odd values of $`p`$ , our equations (39) subsequently simplify to

$`K(\mathrm{})`$ $`=`$ $`e^{c[1K(0)]}{\displaystyle _\pi ^\pi }{\displaystyle \frac{d\varphi }{2\pi }}e^{cA(\varphi ,\{K\})}\mathrm{cos}[\mathrm{}\varphi +cB(\varphi ,\{K\})]`$ (45)
with the abbreviations

$`A(\varphi ,\{K\})`$ $`=`$ $`2^{(p1)}{\displaystyle \underset{m=(p+1)/2}{\overset{p}{}}}(\begin{array}{c}p\\ m\end{array}){\displaystyle \underset{\mathrm{}=1}{\overset{2mp1}{}}}[K(\mathrm{})+K(\mathrm{})]\mathrm{cos}(\mathrm{}\varphi )`$ (51)
$`+2^p{\displaystyle \underset{m=0}{\overset{p}{}}}(\begin{array}{c}p\\ m\end{array})\mathrm{cos}[(p2m)\varphi ]{\displaystyle \underset{\mathrm{}=|p2m|}{\overset{\mathrm{}}{}}}[K(\mathrm{})+K(\mathrm{})]`$
$`B(\varphi ,\{K\})`$ $`=`$ $`2^p{\displaystyle \underset{m=0}{\overset{p}{}}}{\displaystyle \frac{p2m}{p}}(\begin{array}{c}p\\ m\end{array})\mathrm{sin}[(p2m)\varphi ]{\displaystyle \underset{\mathrm{}=|p2m|}{\overset{\mathrm{}}{}}}[K(\mathrm{})K(\mathrm{})]`$ (54)
The result of solving (45) numerically, and subsequently detecting the R$``$SG transition (which is found to be first order) is shown in figure 4 in the $`(p,\alpha )`$ plane. The R$``$SG transition line should for $`c\mathrm{}`$ (i.e. $`p\mathrm{}`$ with fixed $`\alpha `$) approach the value $`\alpha _c=2/\pi `$ ( $`0.637`$), in accordance with ; the latter limit is shown in the figure as a horizontal dashed line, which is indeed seen to be approached by our R$``$SG line as $`p\mathrm{}`$. As usual for the present type of system, we concentrated on bifurcation points. Due to the extensive energy barriers between the ergodic sectors in attractor neural networks, on the important time-scales local stability is more important than thermodynamic stability, so the thermodynamic transition lines (based on comparing values of free energies) are not relevant. Numerical evaluation of the free energy of the candidate solution in fact reveals that the bifurcation lines coincide with the thermodynamic transitions.

Figure 4 shows re-entrance phenomena. It should be noted that an increase of $`p`$ for fixed $`\alpha `$ implies a simultaneous increase of the number of stored patterns (with a detrimental effect on recall quality) and of the connectivity $`c`$ (expected to have a positive effect on recall quality); the non-monotonic appearance of the R$``$SG line reflects the competition of these opposite tendencies.

Finally, for the present type of zero temperature solution, our expression for the RS zero temperature entropy (44) reduces to

$`{\displaystyle \frac{s_0}{\mathrm{log}2}}`$ $`=`$ $`K(0)+c2^p{\displaystyle \underset{m=(p+1)/2}{\overset{p}{}}}(\begin{array}{c}p\\ m\end{array})\{{\displaystyle \underset{\mathrm{}=0}{\overset{2mp1}{}}}[K(2mp)+K(p2m)`$ (60)
$`[K(\mathrm{})+K(\mathrm{})][1{\displaystyle \frac{1}{2}}\delta _{\mathrm{},0}]][K(\mathrm{})+K(\mathrm{})][1{\displaystyle \frac{1}{2}}\delta _{\mathrm{},0}]\}`$
$`+c2^p(1{\displaystyle \frac{\mathrm{log}3}{2\mathrm{log}2}}){\displaystyle \underset{m=(p+1)/2}{\overset{p}{}}}(\begin{array}{c}p\\ m\end{array})[K(2mp)+K(p2m)]^2`$
Numerical evaluation of this expression reveals that the RS entropy at $`T=0`$ is never zero, for any combination of $`\alpha `$ and $`p`$. This at first sight somewhat surprising result is in fact in accordance with a similar observation made recently for finite connectivity spin-glasses in . It does, however, not imply that replica symmetry is not broken at low temperatures (which we know must happen for $`c\mathrm{}`$ ).

## 6 Discussion

In this paper we have shown how the mathematical framework of finite connectivity spin glass replica theory (involving order parameter functions in replica space, rather than matrices) can be combined with the concept of sub-lattices in order to solve a large class of finite connectivity Hopfield-type attractor neural network models near saturation. So far such networks appear to have been studied only in the two technically simpler regimes of full connectivity and so-called extreme dilution, both of which involve a diverging number of bonds per spin $`c`$ in the thermodynamic limit. We have restricted ourselves to a replica symmetric (RS) analysis. The replica symmetric theory is found to lead to $`2^p`$ coupled integral equations (19), whose solutions give the effective local field distributions in each of the $`2^p`$ sublattices of the system. Here $`p`$ denotes the number of stored patterns. In the limit $`c\mathrm{}`$ our equations are shown to reduce to the theory of so-called extremely diluted attractor neural networks, as expected. For $`T0`$ and below a certain critical value for $`\alpha `$ one should expect replica symmetry no longer to hold (it has been shown for $`c\mathrm{}`$ in ); going beyond replica symmetry would require extending the theory to include one or more steps of replica symmetry breaking, similar to the finite connectivity spin glass calculations in .

As is usual for attractor neural networks near saturation, our phase diagrams exhibit three phases: a paramagnetic phase (P), a retrieval phase (R), and a spin-glass phase (SG). We have calculated, for arbitrary values of the number of bonds $`c`$ per spin and a large class of Hebbian type synaptic kernels, the (second order) P$``$R and P$``$SG transition lines in the $`(\alpha ,T)`$ phase diagram. For the main member of our model class, the finite connectivity Hopfield model, we have also calculated the (second order) R$``$SG transition line at $`T=0`$. We find that the values of the average connectivity $`c`$ needed for the system to function as an attractor neural network are surprisingly small, barely exceeding the percolation threshold, even for clipped Hebbian synapses (where each bond carries only one bit of information). Figures 1 to 3, for instance, underline that already for $`c=3`$ the phase diagrams differ only in a relatively modest sense from those corresponding to $`c\mathrm{}`$. This underlines the robustness of recurrent neural networks as information processing systems.

Apart from the obvious extension of our present work, the inclusion of RSB solutions of our saddle-point equations and calculation of AT lines , it would also be an interesting challenge to attempt a dynamical solution. Within the generating functional analysis formalism of  this would for finite connectivity systems involve an effective single spin problem, with order parameters describing single-spin path probabilities and the effect on these path probabilities of single-spin external field perturbations.

## Acknowledgement

One of the authors (BW) gratefully acknowledges helpful discussions with Prof JRL De Almeida.

## References

## Appendix A Calculation of the RS Free Energy

In this section we calculate the RS free energy per spin, upon inserting (14) into (13). In doing so we will use the short-hands:

$$\gamma _m=_{\pi /2}^{\pi /2}\frac{d\varphi }{\pi }\mathrm{log}[\mathrm{cos}(\varphi )].\mathrm{cos}(2m\varphi )m=0,\pm 1,\pm 2,\mathrm{}$$

They obey:

$$\underset{m=\mathrm{}}{\overset{\mathrm{}}{}}\gamma _me^{2am}=\mathrm{log}\mathrm{cosh}(a)$$
(61)
Taking the limit $`n0`$ is again found to impose the need to restrict ourselves to even values of $`n`$. One then obtains, after some algebra, for the entropic term in (13):

$`\underset{n0}{lim}{\displaystyle \frac{1}{\beta n}}{\displaystyle \underset{𝝈}{}}P_𝝃(𝝈)\mathrm{log}P_𝝃(𝝈)_𝝃`$ $`=`$ $`\underset{n0}{lim}{\displaystyle \frac{1}{\beta n}}{\displaystyle \underset{m=\mathrm{}}{\overset{\mathrm{}}{}}}{\displaystyle _0^{2\pi }}{\displaystyle \frac{d\widehat{m}}{2\pi }}e^{i\widehat{m}m}{\displaystyle 𝑑he^{\beta hm}\left[\frac{\mathrm{cos}(\widehat{m})}{\mathrm{cosh}(\beta h)}\right]^n}`$ (62)
$`\times W_𝝃(h)\mathrm{log}\left[{\displaystyle 𝑑h^{}W_𝝃(h^{})\frac{e^{\beta h^{}m}}{[2\mathrm{cosh}(\beta h^{})]^n}}\right]_𝝃`$
$`={\displaystyle \frac{1}{\beta }}{\displaystyle \underset{m=\mathrm{}}{\overset{\mathrm{}}{}}}\gamma _m{\displaystyle 𝑑hW_𝝃(h)e^{2\beta hm}\mathrm{log}\left[𝑑h^{}W_𝝃(h^{})e^{2\beta h^{}m}\right]_𝝃}`$
$`{\displaystyle \frac{1}{\beta }}{\displaystyle 𝑑hW_𝝃(h)\mathrm{log}[2\mathrm{cosh}(\beta h)]_𝝃}`$
Similarly, for the energetic term $`U`$ in (13) one finds:

$`\underset{n0}{lim}{\displaystyle \frac{c}{2\beta n}}{\displaystyle \underset{𝝈𝝈^{}}{}}P_𝝃(𝝈)P_𝝃^{}(𝝈^{})\left[e^{\frac{\beta }{c}\varphi (𝝃𝝃^{})(𝝈𝝈^{})}1\right]_{𝝃𝝃^{}}`$ (63)
$`={\displaystyle \frac{c}{2\beta }}{\displaystyle 𝑑h𝑑h^{}W_𝝃(h)W_𝝃^{}(h^{})\mathrm{log}\left[\underset{\sigma \sigma ^{}}{}e^{\frac{\beta }{c}\varphi (𝝃𝝃^{})\sigma \sigma ^{}+\beta [h\sigma +h^{}\sigma ^{}]}\right]_{𝝃𝝃^{}}}`$
$`+{\displaystyle \frac{c}{\beta }}{\displaystyle 𝑑hW_𝝃(h)\mathrm{log}\mathrm{cosh}(\beta h)_𝝃}`$
$`={\displaystyle \frac{c}{2\beta }}{\displaystyle 𝑑h𝑑h^{}W_𝝃(h)W_𝝃^{}(h^{})\mathrm{log}\left[1+\mathrm{tanh}(\beta h)\mathrm{tanh}(\beta h^{})\mathrm{tanh}(\frac{\beta }{c}\varphi (𝝃𝝃^{}))\right]_{𝝃𝝃^{}}}`$
$`{\displaystyle \frac{c}{2\beta }}\mathrm{log}\mathrm{cosh}[{\displaystyle \frac{\beta }{c}}\varphi (𝝃𝝃^{})]_{𝝃𝝃^{}}`$
We use (18) to simplify the entropic term (62), and add the latter to the energetic term (63) to obtain the following expression for the RS free energy per spin:

$`f_{\mathrm{RS}}`$ $`=`$ $`{\displaystyle \frac{1}{\beta }}{\displaystyle 𝑑hW_𝝃(h)\mathrm{log}[2\mathrm{cosh}(\beta h)]_𝝃}{\displaystyle \frac{c}{2\beta }}\mathrm{log}\mathrm{cosh}[{\displaystyle \frac{\beta }{c}}\varphi (𝝃𝝃^{})]_{𝝃𝝃^{}}`$
$`{\displaystyle \frac{c}{2\beta }}{\displaystyle 𝑑h𝑑h^{}W_𝝃(h)W_𝝃^{}(h^{})\mathrm{log}\left[1+\mathrm{tanh}(\beta h)\mathrm{tanh}(\beta h^{})\mathrm{tanh}(\frac{\beta }{c}\varphi (𝝃𝝃^{}))\right]_{𝝃𝝃^{}}}`$
$`+{\displaystyle \frac{c}{\beta }}{\displaystyle \underset{m=\mathrm{}}{\overset{\mathrm{}}{}}}\gamma _m{\displaystyle 𝑑h𝑑h^{}W_𝝃(h)W_𝝃^{}(h^{})e^{2\beta hm}\left[e^{2m\mathrm{tanh}^1[\mathrm{tanh}(\beta h^{})\mathrm{tanh}(\frac{\beta }{c}\varphi (𝝃𝝃^{}))]}1\right]_{𝝃𝝃^{}}}`$
At this stage we may use relation (61) to carry out the summation over $`m`$. After some further re-arrangement of terms, and application of the simple identity $`\mathrm{cosh}[\mathrm{tanh}^1(y)]=(1y^2)^{1/2}`$, one then arrives at the final result (20).


format: text

text:
# Methods for Computing Normalisations of Affine Rings

## 1. Introduction

In this note we describe the available methods for computing the normalization of affine rings, that is reduced, finite, $`k`$-algebras, where $`k`$ is a computable field, and the implications for computing the integral closure of ideals. Our goal is to keep it simple. Much more is known for computing the normalization with assumptions on the ring or the ideal, as well as for the normalization of discrete valuation rings. We only address what can be done in the most general setting of affine rings.

Our main goal is to give examples so that users can fully exploit the implementations. We also briefly describe which algorithms have been implemented, some implementation issues and how to use these implementations to compute the integral closure of small ideals. All computations listed in this paper were done on a Pentium III, 600 MHZ machine with 256 MB of RAM.

At the time of this writing there are two implementations of normalization. One implementation is in Macaulay 2  and the other is in Singular . Both implementations are based on the information given in de Jong’s paper . The philosophical approach of which is seen in Vasconcelos’ papers  and  and a more detailed analysis of this approach is given in . A brief description as well as other examples for Singular are given in . Finally, there will be chapter on normalization, computing it in Singular, and related issues in the upcoming book of Greuel and Pfister . All of what is published is based on the implementation in Singular, where as this is the first document detailing what is done in Macaulay 2 as well.

Let $`A=R/I`$ be an affine ring. In the following we use the language of Matsumura . If $`A`$ is a domain and $`Q`$ its ring of fractions then the *integral closure* $`\overline{A}`$ of $`A`$ is the set of all elements in $`Q`$ that are integral over $`A`$ and $`A`$ is *integrally closed* if $`A=\overline{A}`$. An affine ring $`A`$ is *normal* if $`A_p`$ is an integrally closed domain at every prime $`p\mathrm{Spec}(A)`$. If $`A`$ is normal it is well known that $`AA/P_1\times \mathrm{}A/P_t`$ where $`P_1,\mathrm{}P_t`$ are the minimal primes for $`A`$ and $`A/P_i`$ is an integrally closed domain for $`1it`$. Since the implementations deal with affine rings in general this decomposition is often referred to as the *normalization* of $`A`$ rather than the integral closure. Macaulay 2 refers to the computation as integral closure where as Singular and all of the publications associated to its implementation call the computation normalization. To emphasize that we do not require the ring to be a domain, we also refer to the computation as normalization.

Serre’s criterion for normality implies that an affine ring $`A`$ is normal if and only if it is both $`S_2`$ and $`R_1`$ \[13, Theorem 23.8\]. An affine ring is $`S_2`$ if $`A_p`$ is Cohen-Macaulay for $`p\mathrm{Spec}(A)`$ and $`\mathrm{codim}(p)2`$, and has depth greater than or equal to 2 for $`p\mathrm{Spec}(A)`$ and $`\mathrm{codim}(p)>2`$. An affine ring is $`R_1`$ if it is regular in codimension one.

The philosophy of the approach is to enlarge $`A`$ recursively inside of $`Q`$ until the normalization of $`A`$ is obtained. The extension Vasconcelos uses in ,  is $`\mathrm{Hom}(\mathrm{Hom}(L,A),\mathrm{Hom}(L,A))`$, where $`L`$ is the Jacobian ideal of $`A`$. The following key theorem in de Jong’s paper  is originally due to Grauert and Remmert \[6, pp. 220-221\], \[7, pp. 125-127\] and describes the ring de Jong uses for this extension. The *non-normal locus* of $`A`$ is the set $`NNL\mathrm{Spec}(A)`$ such that for $`pNNL`$, $`A_p`$ is not an integrally closed domain.

###### Theorem 1.1.

Let $`A=R/I`$ be an affine ring and $`J`$ an ideal of $`A`$. Assume that the ideal $`J`$ contains a non-zero divisor, and has the following property:

$$NNLV(J),$$

where $`V(J)=\{p\mathrm{Spec}(A)|Jp\}`$ denotes, as usual, the zero set of $`J`$. Suppose moreover that $`J`$ has the property

(1.2)
$$\mathrm{Hom}_A(J,J)=\mathrm{Hom}_A(J,A)\overline{A}.$$

Then one has the following normality criterion:

$$A=\mathrm{Hom}_A(J,J)\text{ if and only if }A\text{ is normal. }$$

The implementations in both Macaulay 2 and Singular are split into two phases. First they find an ideal $`J`$ such that the non-normal locus of $`A`$ is contained in $`V(J)`$ and satisfying Equations (1.2). In the second phase $`\mathrm{Hom}(J,J)`$ is computed as a ring.

For the first phase, Theorem 2.2 in de Jong’s paper  establishes that any radical ideal containing a non-zero divisor satisfies Equation (1.2). Also, it is well known that if the Jacobian ideal $`L`$ of $`A`$ is not contained in $`Q\mathrm{Spec}(A)`$ then $`A_Q`$ is regular, so the non-normal locus of $`A`$ is contained in $`V(L)=V(\sqrt{L})`$. Hence $`\sqrt{L}`$ satisfies Theorem 1.1. Computing $`L`$ can be very complex in terms of space and time, so sometimes the programs may use $`\sqrt{f}`$ for $`fL`$ non-zero which also satisfies the hypotheses of Theorem 1.1.

For the second phase, there are two facts that are very helpful. The first, is the well known fact that if $`f`$ is a non-zero divisor in $`A`$, then $`(fJ:J)=f\mathrm{Hom}(J,J)`$. The second, is the presentation of $`\mathrm{Hom}(J,J)`$ as a ring that is due to Catanese  and is given in de Jong’s paper \[12, Theorem 3.1\]. Let $`v_1,\mathrm{},v_n`$ be a module basis for $`\mathrm{Hom}(J,J)`$. Map $`S=k[X_1,\mathrm{},X_n]`$ onto $`\mathrm{Hom}(J,J)`$ by sending $`X_i`$ to $`v_i`$ for $`1in`$. Then as a ring $`\mathrm{Hom}(J,J)`$ is defined by two sets of relations. There are linear equations which are the syzygies for the module, $`\alpha _iX_i=0`$ and quadratic equations which come from the fact that $`\mathrm{Hom}(J,J)`$ is a ring so there exists $`\beta _{ijk}`$, $`1kn`$ such that $`v_iv_j=\beta _{ijk}v_k`$. It is a theorem of Catanese stated clearly in  and  that the ideal generated by these two sets of relations is the full kernel of the map, thus inducing an isomorphism.

The implementations in Macaulay 2 and Singular were completed independently, but the pseudo-code for both is very close to the presentation in  so we will not include it here.

The casual user looking at the code in either program will see relatively complicated programs which is due to the recursive nature of the algorithm. The program must track the proper extensions and in the case of reduced rings, it must track the splitting of the ring into domains. Often for an ideal $`J`$ as in the theorem, while $`\mathrm{Hom}(J,J)`$ is a proper extension, it is not the normalization yet so the process must be repeated many times. It is unknown, given the ring, the number of extensions that must be computed. Moreover, experimental experience suggests that there are choices available that could reduce the number of extensions that need to be computed, but knowing when to use them is not always a priori clear. There is a new algorithm of Vasconcelos’ for which the number of recursions is bounded, but this algorithm is not yet implemented in any of the systems .

## 2. Examples

For the examples we will focus on a hypersurface given to us by Craig Huneke, a union of straight lines given in the Singular example files and an example of Huneke’s given in each of , , , and . Also the first and third examples were chosen because one satisfies Serre’s condition $`S_2`$ but does not satisfy Serre’s condition $`R_1`$ and the other is the opposite.

For each of the examples we give the input and output of both programs so that the user can simply type in the same information and see the same results. Occasionally, due to the width of the text we have had to slightly alter the output for it to fit in the space allotted. We hope that this is also enough information for the user to then perform similar computations on their own examples. Also, when possible we include the computation times in CPU seconds. If no time is listed it is because the time is negligible. We include the times because they are indicative of what can happen on larger examples and because they give more meaning to the times listed after Example 2.6. However, when taken by themselves the times may not have much meaning since even on the same machine times can vary up to a few seconds.

###### Example 2.1.

Let $`A=[x,y,z]/(x^6z^6y^2z^4)`$. This ring is not $`R1`$, but it is $`S2`$. Therefore, computing the normalization in the example forms the desingularization of $`A`$ in codimension one. The computation in Macaulay 2 follows.

```
i1 : A = QQ[x,y,z]/ideal(x^6-z^6-y^2*z^4)

o1 = A

o1 : QuotientRing

i2 : time integralClosure (A, Variable => symbol a)
     -- used 1.67 seconds

                 QQ [a , a , x, y, z]
                      7   6
o2 = ---------------------------------------------
       2                    2         2    2    2
     (x  - a z, a x - a z, a  - a x, a  - y  - z )
            6    6     7    6    7    7

o2 : QuotientRing

```

Computing the normalization of an affine ring requires new variables unless the ring is already normal. Macaulay 2 gives the user the opportunity to specify which letter to use for this new variable. This is done via

```
Variable => symbol a.

```

If this is not included, then Macaulay 2 uses $`w`$ by default. Using a letter to create the normalization which is a variable in the definition of $`R`$ can lead to problems with using the normalization, so we recommend supplying a variable not used to define $`R`$.

For Singular, we must first load the normalization library. Singular then lists the libraries loaded and we use elipses to indicate that more is printed on that line, but we have removed it to keep from having to make multiple lines due to the size of the paper. Also, we use nor to name the output of normal(I) to match the comments that singular outputs as part of this function. We then follow the other commands they list for continuity.

```
> LIB "normal.lib";
// ** loaded /usr/local/Singular/2-0-3/LIB/normal.lib ...
// ** loaded /usr/local/Singular/2-0-3/LIB/hnoether.lib ...
// ** loaded /usr/local/Singular/2-0-3/LIB/primitiv.lib ...
// ** loaded /usr/local/Singular/2-0-3/LIB/random.lib ...
// ** loaded /usr/local/Singular/2-0-3/LIB/matrix.lib ...
// ** loaded /usr/local/Singular/2-0-3/LIB/ring.lib ...
// ** loaded /usr/local/Singular/2-0-3/LIB/inout.lib ...
// ** loaded /usr/local/Singular/2-0-3/LIB/presolve.lib ...
// ** loaded /usr/local/Singular/2-0-3/LIB/elim.lib ...
// ** loaded /usr/local/Singular/2-0-3/LIB/primdec.lib ...
// ** loaded /usr/local/Singular/2-0-3/LIB/sing.lib ...
// ** loaded /usr/local/Singular/2-0-3/LIB/poly.lib ...
// ** loaded /usr/local/Singular/2-0-3/LIB/general.lib ...
> ring R = 0,(x,y,z),dp;
> ideal I= x6-z6-y2z4;
> list nor = normal(I);

// ’normal’ created a list of 1 ring(s).
// nor[1+1] is the delta-invariant in case of choose=wd.
// To see the rings, type (if the name of your list is nor):
     show( nor);
// To access the 1-st ring and map (similar for the others),
     type: def R = nor[1]; setring R;  norid; normap;
// R/norid is the 1-st ring of the normalization and
// normap the map from the original basering to R/norid
> timer=1;
//used time: 0.82 sec
> show(nor);

// list, 1 element(s):
[1]:
   // ring: (0),(T(1),T(2),T(3),T(4),T(5)),
                           (a(1,1,1,1,1),dp(5),C);
   // minpoly=0
// objects belonging to this ring:
// normap               [0]  ideal, 3 generator(s)
// norid                [0]  ideal, 4 generator(s)
> setring S; norid;
norid[1]=T(4)^2-T(1)*T(5)
norid[2]=T(1)*T(4)-T(3)*T(5)
norid[3]=T(2)^2+T(3)^2-T(5)^2
norid[4]=T(1)^2-T(3)*T(4)

```

The the output of the command show(nor); states that the normalization of $`R/I`$ requires only one ring because $`R/I`$ was a domain. If we denote the normalization of $`R/I`$ by $`S/L`$ then the next line tells us that $`S=[T(1),T(2),T(3),T(4),T(5)]`$, each variable has degree 1 and the ordering is degree reverse lexicographic. The next two lines tell us that in Singular there are two other objects assigned to this ring. The first defines the natural map from $`R/I`$ into $`S/L`$ and the second is the the defining ideal $`L`$ of the integral closure. To see generators of $`L`$ we execute the two commands setring S; and norid;.

The information stored in the outputs of the two programs is fundamentally the same, but the presentation of the output in the two programs is a little different. Macaulay 2 preserves the fact that $`A\mathrm{Hom}(J,J)`$ and thus computes a presentation that includes those variables from $`A`$ which still contribute to a minimal presentation. In contrast, Singular uses all new variables and in this example $`T(1)=x,T(2)=y,T(3)=z,T(4)=a_6,T(5)=a_7`$. The fractions computed in Example 2.4 can be used to find this correspondence. Macaulay 2 allows the user to choose the name of the new variables, but Singular does not have such an option. When Singular computes $`normal(I)`$ it gives information about the number of rings in the decomposition and stores this information in the list we called nor above and then a few commands are used to display this information. In this example there is one ring, but in the next example when the input is not a domain we are told there are 3 rings, so we do the display information three times. In Macaulay 2 for a domain, the domain that is the normalization is displayed unless we suppress it using a semicolon. When the ring is not a domain, a sequence of normal rings, such that the direct sum of these rings is the normalization of the reduced ring is displayed.

###### Example 2.2.

Let $`A`$ be a union of lines, $`A=[x,y,z]/((xy)(xz)(yz))`$.

```
i1 : A = QQ[x,y,z]/ideal((x-y)*(x-z)*(y-z))

i2 : time integralClosure(A,Variable => V)
     -- used 0.47 seconds

      QQ [V , y, z]
           0           R
o2 = (-------------, -----)
          2          y - z
         V  + V
          0    0

o2 : Sequence

```

Before proceeding to the Singular example we consider the presentation of the first ring. The output of Macaulay 2 will always give a normal ring, but not necessarily a domain. Here it gives

$$\frac{[V_0,y,z]}{V_0^2+V_0}\frac{[V_0,y,z]}{V_0+1}\frac{[V_0,y,z]}{V_0}$$

which is normal. In contrast, Macaulay 2 computes the normalization of $`[V_0,y,z]/(V_0^2+V_0)`$ as the isomorphic presentation.

Using Singular, the output of $`normal(I)`$ reveals that there are three rings. What follows reveals that each of the three rings is isomorphic to $`[x,y]`$. This is the information we gained from Macaulay 2 in an isomorphic format. Since we ran this example in the same session as the previous example, we do not need

```
> LIB "normal.lib";

```

this time. However, if the Singular session is restarted then this command is needed to get the same results.

```
> ring R=0,(x,y,z),dp; ideal I=(x-y)*(x-z)*(y-z);
> list nor=normal(I);

// ’normal’ created a list of 3 ring(s).
// nor[3+1] is the delta-invariant in case of choose=wd.
// To see the rings, type (if the name of your list is nor):
     show( nor);
// To access the 1-st ring and map (similar for the others),
     type: def R = nor[1]; setring R;  norid; normap;
// R/norid is the 1-st ring of the normalization and
// normap the map from the original basering to R/norid
> timer=1;
> show(nor);
// list, 3 element(s):
[1]:
   //ring: (0),(T(1),T(2)),(dp(2),C);
   // minpoly = 0
// objects belonging to this ring:
// normap               [0]  ideal, 3 generator(s)
// norid                [0]  ideal, 1 generator(s)
[1]:
   //ring: (0),(T(1),T(2)),(dp(2),C);
   // minpoly = 0
// objects belonging to this ring:
// normap               [0]  ideal, 3 generator(s)
// norid                [0]  ideal, 1 generator(s)
[1]:
   //ring: (0),(T(1),T(2)),(dp(2),C);
   // minpoly = 0
// objects belonging to this ring:
// normap               [0]  ideal, 3 generator(s)
// norid                [0]  ideal, 1 generator(s)
> def S1=nor[1]; def S2=nor[2]; def S3=nor[3];
> setring S1; norid; setring S2; norid; setring S3; norid;
norid[1]=0
norid[1]=0
norid[1]=0

```

Since there are three rings whose direct sum forms the normalization of $`A=R/I`$ the output of show(nor); gives the defining rings for all three and we see they are the same. Then we ask for the defining ideal and get that it is zero, so

$$\overline{A}[T(1),T(2)][T(1),T(2)][T(1),T(2)]$$

which is essentially the same information we got from Macaulay 2.

###### Example 2.3.

This example was chosen to because this ring has an isolated singularity, so it is $`R_1`$, but is not $`S_2`$ in contrast to the first example. Thus the normalization of this ring is its $`S_2`$-ification. This example also takes a little longer than the previous examples. Let $`I`$ be the radical of the ideal generated by

$$\begin{array}{c}\hfill ab^3c+bc^3d+a^3be+cd^3e+ade^3,\\ \hfill a^2bc^2+b^2cd^2+a^2d^2e+ab^2e^2+c^2de^2,\\ \hfill a^5+b^5+c^5+d^55abcde+e^5\end{array}$$

We use a few new commands in Macaulay 2 this time to make the output more presentable for this paper, however these are not needed when a user runs Macaulay 2 in emacs since the output is reasonable with wrap-around turned off. Even with these new commands we altered the presentation of the output a little for both programs to make them more readable in this paper.

```
i1 : R = QQ[a..e];

i2 : time I = radical(ideal(
a*b^3*c+b*c^3*d+a^3*b*e+c*d^3*e+a*d*e^3,
a^2*b*c^2+b^2*c*d^2+a^2*d^2*e+a*b^2*e^2+c^2*d*e^2,
a^5+b^5+c^5+d^5-5*a*b*c*d*e+e^5));
     -- used 5.77 seconds

i3 : time V = integralClosure (R/I, Variable => X);
     -- used 4.23 seconds

i4 : ring ideal V

o4 = QQ [X ,X ,a,b,c,d,e, Degrees =>
          0  1                 {{5},{5},{1},{1},{1},{1},{1}}]

o4 : PolynomialRing

i5 : toString ideal V

o5 = ideal(a^2*b*c^2+b^2*c*d^2+a^2*d^2*e+a*b^2*e^2+c^2*d*e^2,
a*b^3*c+b*c^3*d+a^3*b*e+c*d^3*e+a*d*e^3,
a^5+b^5+c^5+d^5-5*a*b*c*d*e+e^5,
a*b*c^4-b^4*c*d-X_0*e-a^2*b^2*d*e+a*c^2*d^2*e+b^2*c^2*e^2
       -b*d^2*e^3,
a*b^2*c^3+X_1*d+a*b*c*d^2*e-a^2*b*e^3-d*e^5,
a^3*b^2*c-b*c^2*d^3-X_1*e-b^5*e-d^5*e+2*a*b*c*d*e^2,
a^4*b*c+X_0*d-a*b^4*e-2*b^2*c^2*d*e+a^2*c*d*e^2+b*d^3*e^2,
X_1*c+b^5*c+a^2*b^3*e-a*b*c^2*d*e-a*d^3*e^2,
X_0*c-a^2*b^2*c*d-b^2*c^3*e-a^4*d*e+2*b*c*d^2*e^2+a*b*e^4,
X_1*b-b*c^5+2*a*b^2*c*d*e-c^3*d^2*e+a^3*d*e^2-b*e^5,
X_0*b+a*b*c^2*d^2-b^3*c^2*e+a*d^4*e-a^2*b*c*e^2+b^2*d^2*e^2
     -c*d*e^4,
X_1*a-b^3*c^2*d+c*d^2*e^3,
X_0*a-b*c*d^4+c^4*d*e,
X_1^2+b^5*c^5+b^4*c^3*d^2*e+b*c^2*d^3*e^4+b^5*e^5+d^5*e^5,
X_0*X_1+b^3*c^4*d^3-b^2*c^7*e+b^2*c^2*d^5*e-b*c^5*d^2*e^2
       -a*b^2*c*d^3*e^3+b^4*c*d*e^4+a^2*b^2*d*e^5
       -a*c^2*d^2*e^5-b^2*c^2*e^6+b*d^2*e^7,
X_0^2+b*c^3*d^6+2*b^5*c*d^3*e+c*d^8*e-b^4*c^4*e^2
     +a^3*c^3*d^2*e^2+2*a^2*b^3*d^3*e^2-5*a*b*c^2*d^4*e^2
     +4*b^3*c^2*d^2*e^3-3*a*d^6*e^3+5*a^2*b*c*d^2*e^4
     -b^2*d^4*e^4-2*b*c^3*d*e^5-a^3*b*e^6+3*c*d^3*e^6-a*d*e^8)

```

The command “ring ideal V” gives the polynomial ring and “toString ideal V” gives the defining ideal for the normalization.

In Macaulay 2 we computed the radical of the three equations and then computed the normalization. Singular and Macaulay 2 use different algorithms for the command radical which means that there are examples which run well in one program, but not the other. The radical computation done above in Macaulay 2 does not finish in Singular on the Pentium III, 600 MHZ machine with 256 MB of RAM, but this is a feature of this example. We could just as easily find an example that Singular completes and Macaulay 2 does not. In Singular we ran the normalization computation on the radical obtained from Macaulay 2. Also this time we do not do the command show(nor);, because the output is similar to that of the first example. Finally, as we did above, we alter the output a little to make it more readable for this paper.

```
> ring R = 0,(a,b,c,d,e),dp;
> ideal I = a2bc2+b2cd2+a2d2e+ab2e2+c2de2,
ab3c+bc3d+a3be+cd3e+ade3,
a5+b5+c5+d5-5abcde+e5,
a3b2cd-bc2d4+ab2c3e-b5de-d6e+3abcd2e2-a2be4-de6,
abc5-b4c2d-2a2b2cde+ac3d2e-a4de2+bcd2e3+abe5,
ab2c4-b5cd-a2b3de+2abc2d2e+ad4e2-a2bce3-cde5,
b6c+bc6+a2b4e-3ab2c2de+c4d2e-a3cde2-abd3e2+bce5,
a4b2c-abc2d3-ab5e-b3c2de-ad5e+2a2bcde2+cd2e4;
list nor = normal(I);

// ’normal’ created a list of 1 ring(s).
// nor[1+1] is the delta-invariant in case of choose=wd.
// To see the rings, type (if the name of your list is nor):
     show( nor);
// To access the 1-st ring and map (similar for the others),
     type: def R = nor[1]; setring R;  norid; normap;
// R/norid is the 1-st ring of the normalization and
// normap the map from the original basering to R/norid
> timer=1;
//used time: 3.21 sec
> def S = nor[1]; setring S; norid;
norid[1]=T(1)^2*T(2)*T(3)^2+T(2)^2*T(3)*T(4)^2
        +T(1)^2*T(4)^2*T(5)+T(1)*T(2)^2*T(5)^2
        +T(3)^2*T(4)*T(5)^2
norid[2]=T(1)*T(2)^3*T(3)+T(2)*T(3)^3*T(4)+T(1)^3*T(2)*T(5)
        +T(3)*T(4)^3*T(5)+T(1)*T(4)*T(5)^3
norid[3]=T(1)^5+T(2)^5+T(3)^5+T(4)^5
        -5*T(1)*T(2)*T(3)*T(4)*T(5)+T(5)^5
norid[4]=T(2)*T(3)*T(4)^4-T(3)^4*T(4)*T(5)-T(1)*T(6)
norid[5]=T(1)*T(2)*T(3)^2*T(4)^2-T(2)^3*T(3)^2*T(5)
        +T(1)*T(4)^4*T(5)-T(1)^2*T(2)*T(3)*T(5)^2
        +T(2)^2*T(4)^2*T(5)^2-T(3)*T(4)*T(5)^4+T(2)*T(6)
norid[6]=T(2)^3*T(3)^2*T(4)-T(3)*T(4)^2*T(5)^3-T(1)*T(7)
norid[7]=T(1)^2*T(2)^2*T(3)*T(4)+T(2)^2*T(3)^3*T(5)
+T(1)^4*T(4)*T(5)
        -2*T(2)*T(3)*T(4)^2*T(5)^2-T(1)*T(2)*T(5)^4-T(3)*T(6)
norid[8]=T(2)*T(3)^5-2*T(1)*T(2)^2*T(3)*T(4)*T(5)
+T(3)^3*T(4)^2*T(5)
        -T(1)^3*T(4)*T(5)^2+T(2)*T(5)^5-T(2)*T(7)
norid[9]=T(1)*T(2)*T(3)^4-T(2)^4*T(3)*T(4)
        -T(1)^2*T(2)^2*T(4)*T(5)+T(1)*T(3)^2*T(4)^2*T(5)
        +T(2)^2*T(3)^2*T(5)^2-T(2)*T(4)^2*T(5)^3-T(5)*T(6)
norid[10]=T(1)*T(2)^2*T(3)^3+T(1)*T(2)*T(3)*T(4)^2*T(5)
         -T(1)^2*T(2)*T(5)^3-T(4)*T(5)^5+T(4)*T(7)
norid[11]=T(2)^5*T(3)+T(1)^2*T(2)^3*T(5)
         -T(1)*T(2)*T(3)^2*T(4)*T(5)
         -T(1)*T(4)^3*T(5)^2+T(3)*T(7)
norid[12]=T(1)^3*T(2)^2*T(3)-T(2)*T(3)^2*T(4)^3-T(2)^5*T(5)
         -T(4)^5*T(5)+2*T(1)*T(2)*T(3)*T(4)*T(5)^2-T(5)*T(7)
norid[13]=T(1)^4*T(2)*T(3)-T(1)*T(2)^4*T(5)
         -2*T(2)^2*T(3)^2*T(4)*T(5)+T(1)^2*T(3)*T(4)*T(5)^2
         +T(2)*T(4)^3*T(5)^2+T(4)*T(6)
norid[14]=T(2)^4*T(3)*T(4)*T(5)^4+2*T(1)^3*T(4)^3*T(5)^4
         +4*T(1)^2*T(2)^2*T(4)*T(5)^5
         -4*T(1)*T(3)^2*T(4)^2*T(5)^5+2*T(2)*T(4)^2*T(5)^7
         +T(1)*T(2)*T(3)*T(4)*T(5)*T(6)
         +T(1)*T(3)^2*T(4)^2*T(7)-T(2)^2*T(3)^2*T(5)*T(7)
         -4*T(1)^2*T(3)*T(5)^2*T(7)-T(2)*T(4)^2*T(5)^2*T(7)
         +T(6)*T(7)
norid[15]=2*T(1)^2*T(2)*T(4)^4*T(5)^3
         -T(1)*T(2)^3*T(4)^2*T(5)^4-T(2)*T(3)^2*T(4)^3*T(5)^4
         +3*T(4)^5*T(5)^5+2*T(1)*T(2)*T(3)*T(4)*T(5)^6
         +3*T(2)^2*T(3)*T(4)^2*T(6)+2*T(1)*T(2)^2*T(5)^2*T(6)
         +2*T(3)^2*T(4)*T(5)^2*T(6)-2*T(3)^5*T(7)
         -3*T(4)^5*T(7)+3*T(1)*T(2)*T(3)*T(4)*T(5)*T(7)
         -2*T(5)^5*T(7)+2*T(7)^2
norid[16]=T(1)^3*T(2)*T(4)^5*T(5)-T(1)^3*T(3)^3*T(4)^2*T(5)^2
         -T(2)^2*T(4)^4*T(5)^4+T(2)*T(3)^3*T(4)*T(5)^5
         +T(1)^2*T(2)^2*T(4)*T(6)+T(2)^2*T(3)^2*T(5)*T(6)
         -2*T(2)*T(4)^2*T(5)^2*T(6)-T(6)^2

```

Singular used approximately 1 CPU second less than Macaulay 2 for this normalization computation. As examples get larger this could be significant in obtaining success.

There are two other operations we want to illustrate. Assume $`A=R/I`$ is an affine domain. These first three examples give the integral closure of $`A`$ in the form $`S/L`$ where $`S`$ is of the form $`R[Y_0,\mathrm{}Y_t]`$ and $`S/L\overline{A}`$. If $`A`$ is a domain a user may want the fractions from the quotient field that generate the normalization of $`A`$ as a finite algebra over $`A`$. This information can be obtained using both Macaulay 2 and Singular.

###### Example 2.4.

For simplicity we use the same hypersurface as Example 2.1.

```
i1 : A = QQ[x,y,z]/ideal(x^6-z^6-y^2*z^4);

i2 : time ICfractions(A)
     -- used 1.77 seconds

o2 = | x3/z2 x2/z x y z |

                  1            5
o2 : Matrix frac A  <--- frac A

```

Thus $`A[x^3/z^2,x^2/z]=\overline{A}`$.

In Singular the method for obtaining the fractions is considerably different. Several auxiliary functions are used. The library normal.lib is no longer used and instead we need reesclos.lib. Singular is very particular how you define the ideal defining the affine domain. It must be named ker or an error will be issued. The output has the following form.

```
fraction;
[1]:
  xyz8
[2]:
  yz8

```

Then the fraction we are looking for is  divided by , that is $`{\displaystyle \frac{xyz^8}{yz^8}}=x`$. The fractions found in this way in Singular are often not reduced.

We use a new library in this example and since we included the printed output that is given when a library is loaded into Singular in Example 2.1, we removed it this time.

```
> LIB "reesclos.lib";
> ring R=0,(x,y,z),lp; ideal ker=x6-z6-y2z4;
> list L=primeClosure(R); closureRingtower(L);
> setring R(6); poly f=T(1); closureFrac(L);
> setring R(1); fraction;
[1]:
   xyz8
[2]:
   yz8
> setring R(6); poly f=T(2); closureFrac(L);
> setring R(1); fraction;
// ** redefining f **
[1]:
   y2z8
[2]:
   yz8
> setring R(6); poly f=T(3); closureFrac(L);
> setring R(1); fraction;
// ** redefining f **
[1]:
   yz9
[2]:
   yz8
> setring R(6); poly f=T(4); closureFrac(L);
> setring R(1); fraction;
// ** redefining f **
[1]:
   x2yz7
[2]:
   yz8
> setring R(6); poly f=T(5); closureFrac(L);
> setring R(1); fraction;
// ** redefining f **
[1]:
   x3yz6
[2]:
   yz8
> timer=1;
//used time: 1.66 sec

```

We can use this output from Macaulay 2 and Singular to find a map from the variables used by Macaulay 2 to those used by Singular and vice versa. Also, given any element of the presentation of $`\overline{A}`$ as $`S/J`$, Singular can find its image in the ring of fractions of $`A`$. This is not currently set up in Macaulay 2.

Before computing the integral closure of an ideal, we point out one other subtle difference between the two programs. In Macaulay 2 it is possible to define multigraded rings and the integralClosure program is designed to handle such rings, while in Singular it is not possible to define such rings. Multigraded rings play an important role in computing the integral closure of an ideal in Macaulay 2. Since it is not possible to define a multigraded ring in Singular the implementation there must be fundamentally different from that in Macaulay 2.

It is possible, theoretically, to compute the integral closure of an ideal using both systems. We say theoretically because the computation is often much too complex to complete, either due to memory or time (mostly memory). The approach, in both systems, is the classical one, that is compute the Rees algebra first, find the normalization of the Rees algebra and then find the degree one piece of that graded algebra.

In Macaulay 2 a function to compute $`\overline{I}`$ is not yet implemented in the main distribution, but we include one in the appendix which is being submitted to Macaulay 2 for inclusion. To run this program, a program to compute the Rees algebra that preserves the natural multigrading is needed. We also include the code for this in the appendix. There are several such programs for Macaulay 2 in circulation. Besides the one in the appendix, one can be found in  and we have received yet another via personal communication from David Eisenbud. A package of such programs is being put together for inclusion with Macaulay 2. The one we include here is the only one that incorporates the natural multigrading, but is otherwise fundamentally the same as that in , which could be easily altered to use the grading. The program communicated by David Eisenbud is more general and is based on his paper with Huneke and Ulrich . Both computing blowups and integral closure of ideals are functions in Singular if the Rees library has been loaded.

We include two complete examples and discuss briefly two others. The first example is one that can be checked by hand using standard techniques. The second example is considerably more complicated and the remaining ideals illustrate how easily this process fails to complete. All the examples are small in the sense that they use at most three generators, use at most three variables and have degree at most seven.

###### Example 2.5.

Let $`R=[x,y]`$ and $`I=(x^3,x^2y^2,y^7)`$. Using the fact that the integral closure of a monomial ideal corresponds to the integer lattice points in the convex hull of the exponent vectors of the ideal, this example is easily computed by hand. The first two commands load files named IdealNormal.m2 and blowup.m2 which are the programs given in the appendix.

```
i1 : load "IdealNormal.m2"; load "blowup.m2";
--loaded IdealNormal.m2
--loaded blowup.m2

i2 : R = QQ[x,y]; I = ideal(x^2,x*y^4,y^5);

o3 : Ideal of R

i4 : time idealIC(I)
     -- used 0.98 seconds

             2     3   5
o4 = ideal (x , x*y , y )

o4 : Ideal of R

```

Computing the integral closure of an ideal in Singular uses the library reesclos.lib which we used in Example 2.4. The library must be loaded if it has not already been.

```
> ring R = 0,(x,y),dp; ideal I = x2,xy4,y5;
>  list J = normalI(I);  J;
[1]:
   _[1]=x2
   _[2]=y5
   _[3]=-xy3

```

The output following : is the generators of $`\overline{I}`$.

###### Example 2.6.

Let $`R=[x,y,z]`$ and $`I=(y^6+x^2z,x^6+y^4z^2)`$.

```
i5 : R = QQ[x,y,z, MonomialSize => 16];

i6 : I = ideal(y^6+x^2*z,-x^6+y^4*z^2); time idealIC(I)

o6 : Ideal of R
     -- used 3.6 seconds

             6    2    6    4 2   5 4      2 3
o7 = ideal (y  + x z, x  - y z , x y  + x*y z )

o7 : Ideal of R

```

```
> ring R=0,(x,y,z),dp; ideal I=y6+x2z,-x6+y4z2;
> list J=normalI(I);  J;
[1]:
   _[1]=y6+x2z
   _[2]=-x6+y4z2
   _[3]=x5y4+xy2z3

```

These examples show that both Macaulay 2 and Singular can compute the integral closure of a 2 generated ideal in 3 variables, but an ideal with three generators will often cause problems. For example, if $`R=[x,y,z]`$ and $`I=(y^6+x^2z,x^6+y^4z^2,x^3yz^3y)`$ then Macaulay 2 will give a monomial overflow error and Singular runs for about 7.1 hours and uses up the memory available on this machine. But, with two variables and two generators we can find a troublesome ideal in degree as low as seven. If $`R=[x,y]`$ and $`I=(y^7+x^4,y^6+xy^4)`$ then Macaulay 2 finishes in 905.29 CPU seconds and uses about 66% of the memory. The ideal we get is

$$\overline{I}=(x^2y^3+x^4,x^3y^2x^4,x^4y+39x^3y^238x^4,x^5x^4,y^6xy^4,xy^5+x^4).$$

We ran this same example in Singular and after 80,836.49 CPU seconds the machine quit the computation after using all of the memory. Both programs struggle with various computations of ideal integral closure due to computing the integral closure of the blow-up ring which has more variables than the original ring. Thus, while it is possible to compute the integral closure of some ideals using this method, for most cases it is impractical and the myriad of papers dealing with special cases must be consulted.

## 3. Appendix

The program idealIC, below, computes the integral closure of an ideal in Macaulay 2. This is the program used to compute the closures in Examples 2.52.6. Except for ICfractionsLong all of the functions used in idealIC are self-explanatory, are commented, or are clearly explained in the Macaulay 2 documentation. ICfractionsLong is similar to the function ICfractions used in Example 2.4 which computes the normalization of $`A`$ and then returns a minimal generating set for $`\overline{A}`$ as an algebra over $`A`$. During the normalization computation fractions are computed that may be extraneous and while ICfractions only returns a minimal set, ICfractionsLong keeps track of all the fractions generated by the computation. To ensure that the entire degree one component of the integral closure of the Rees algebra is computed, ICfractionsLong is used rather than ICfractions.

```
idealIC = method()
idealIC(Ideal) := Ideal => (I) -> (
     R := ring I;
     n1 := numgens R;
     J1 := blowup(I);  --defining ideal of blow-up
     R2 := ring J1;
     n2 := numgens R2;
     S := R2/J1;  -- the blow-up
     Sfrac1 := first entries ICfractionsLong S;
     -- The slow part is ICfractionsLong(S), as expected.
     Sfrac2 := apply(#Sfrac1-n2, i-> Sfrac1#i);
     toLift := select(Sfrac2, i-> (degree i)#1 == 1);
     -- toLift is the set of elements in the normalization
     -- of the Rees algebra that generate the degree one
     -- component.
     VarImage := flatten append (gens R,first entries gens I);
     -- We use VarImage to define the map from R2 -> R needed
     -- to lift the fractions back to R.
     LiftMap := map(R,R2,VarImage);
     NewNums := apply(toLift, i-> LiftMap(
               substitute(numerator i, R2)));
     NewDenoms := apply(toLift, i-> LiftMap(
               substitute(denominator i, R2)));
     NewGens := apply(#toLift, i-> substitute(
               (NewNums#i)/(NewDenoms#i),R));
     ideal mingens (I + ideal(NewGens))
     )

```

A program for computing the Rees algebra $`R[tI]`$ in Macaulay 2 which is used in the program IdealIC give above.

```
blowup = method(Options => {VarName => Y})
blowup(Ideal) := o-> (J)-> (
     -- Input:  J is any ideal.
     -- Output:  The result is the defining ideal of the
     --          blowup algebra R[tJ] where R is the ring
     --          of J.
     -- METHOD:  We construct a polynomial ring
     --          R[t,y_1,..,y_n] where n is the number of
     --          generators of J.  We then construct the ideal
     --          (Y_1-tJ_1,...,Y_n-tJ_n). Then R[tJ] is
     --          isomorphic to the polynomial ring
     --          R[Y_1,...,Y_n] mod the out put of this
     --          algorithm.
     R := ring(J);
     n1 := numgens(J);
     n2 := numgens(R);
     Degs1 := flatten degrees source gens J;
     -- Degs1 are the degrees of the generators of J.
     Degs1a := apply(n1,i->1+Degs1_i);
     Degs2 := flatten ((monoid R).Options.Degrees);
     -- Degs2 is the degrees of the variables of R.
     Degs := join({1},Degs2,Degs1a);
     S := coefficientRing(R)[t,gens R,
          (o.VarName)_(1)..(o.VarName)_(n1),
          MonomialOrder => Eliminate 1, Degrees => Degs];
     J1 := substitute(J,S);  -- Put J in the bigger ring S.
     M := matrix{apply(n1,i-> S_(i+n2+1)-S_0*J1_i)};
     -- This is the matrix (Y_1-tJ_1,...,Y_n-tJ_n)
     K := gb(M);
     -- Next eliminate t.
     L := ideal(selectInSubring(1,gens(K)));
     -- Set the Multidegrees.
     Degs1b := apply(n1,i->join({Degs1_i},{1}));
     Degs2b := apply(n2,i->join({Degs2_i},{0}));
     Degsb := join(Degs2b,Degs1b);
     Sb := coefficientRing(R)[gens R,
           (o.VarName)_1..(o.VarName)_(n1),
           MonomialOrder => ProductOrder{n2,n1},
           Degrees => Degsb];
     trim(substitute(L,Sb))
     )

```

## Acknowledgements

I thank Anna Guerrieri for encouraging me to write this article and Mike Stillman and Wolmer Vasconcelos for their helpful conversations. I thank David Eisenbud and C. Musili for giving me the opportunity to attend the conference in Hyderabad, India.