format: text

text:
# The Effects of Photoionization on Galaxy Formation — III: Environmental Dependence in the Luminosity Function

## 1 Introduction

Understanding the galaxy luminosity function has been a goal of galaxy formation theory for several decades (e.g. ???). A particularly interesting question is whether the luminosity function is universal or whether it depends on environmental factors such as the mass of the dark halo that hosts a particular galaxy population. Considerable attention has been paid to the faint end of the luminosity function which has a much flatter slope than the low mass end of the halo mass function predicted in cold dark matter (CDM) models of galaxy formation (e.g. ?).

The early work of ? showed that the number of faint galaxies must have been strongly affected by feedback processes that prevented most of the gas from cooling in small halos at early times. Some likely feedback mechanisms, such as the injection of energy into the interstellar medium in the course of stellar evolution, depend on the internal properties of the galaxy and so their effects may be expected to be independent of the large-scale environment. A number of observational studies, such as a recent analysis of the 2dF galaxy redshift survey (?), indeed find no significant difference between the luminosity functions of galaxies in rich clusters and in the field. Other studies, however, have found the opposite. For example, ? concluded that galaxies in rich clusters have luminosity functions with considerably steeper faint ends than galaxies in the field. More recently, ? have claimed that the faint end of the galaxy luminosity function varies systematically with environment, increasing in slope from small, diffuse systems like the Local Group, to massive, dense systems like the Coma cluster.

In the cold dark matter model of galaxy formation, dark matter halos retain considerable substructure after they collapse and virialize (e.g. ??) and some of these subhalos are associated with sites of galaxy formation. The mass function of subhalos appears to be relatively independent of the mass of the parent halo. Thus, trends such as those inferred by ? would require processes that either preferentially suppress the formation of dwarf galaxies in low mass systems, or destroy them after they form. An effective mechanism for suppressing the formation of small galaxies is the reheating of the intergalactic medium (IGM) caused by the reionization of the Universe at a redshift $`z6`$. ? have argued that this process could introduce an environmental dependence in the galaxy luminosity function on the grounds that a higher fraction of the low-mass halos that formed before reionization (when dwarf galaxy formation proceeded unimpeded by photoionization suppression) ended up in clusters today than in less massive systems.

The effect of reionization on the formation of galaxies has been the subject of several recent studies (???), aimed mostly at investigating the discrepancy between the large number of subhalos found in N-body simulations of galactic CDM halos and the small number of satellite galaxies observed in the Local Group. In this paper, we employ the CDM model of ? to calculate the luminosity function of galaxy populations residing in dark matter halos of different mass. We find that there are significant differences in these luminosity functions and we then explore how they combine together to build up the global luminosity function, with particular emphasis on the faint end slope. A partial study of luminosity functions in halos of different mass using a semi-analytic model of galaxy formation was carried out by ?.

To calculate galaxy luminosity functions in halos of different mass correctly, it is important to include tidal effects on satellite galaxies, a potential galaxy destruction mechanism. Our model treats these effects in considerably more detail than previous models of galaxy formation. We find that tidal effects are important in limiting the formation of massive galaxies at the centre of rich clusters.

In this paper, we compare the results of our calculations to the data of ? and assess whether feedback from reionization is a viable explanation of the trend claimed by these authors. The existing dataset is small, but forthcoming results from the 2dF and Sloan galaxy surveys will enable much more extensive comparisons with the theory.

The remainder of this paper is arranged as follows. In §2 we briefly outline our model of galaxy formation, in §3 we present results for the environmental dependence of the luminosity function and in §4 we compare our model with the available observational data. Finally, in §5 we present our conclusions. We present, in an Appendix, several simple models of photoionization suppression to elucidate how this mechanism works.

## 2 Model

We employ the semi-analytic model of galaxy formation described in detail by ? and Benson et al. (2002a; hereafter Paper I) to compute the properties of galaxies in a range of environments at $`z=0`$. The reader is referred to those papers for a complete and detailed description of the model. Briefly, the hierarchical formation of dark matter halos is calculated using the extended Press-Schechter formalism (???). The formation of galaxies in the resulting dark matter halo merger trees is followed by means of simple, physically motivated models of gas cooling, star formation and galaxy merging. Recent work has demonstrated that at least in so far as gas cooling is concerned these simplified calculations are in excellent agreement with the results of N-body/hydrodynamical simulations (???). Applying a stellar population synthesis model gives galaxy luminosities in different passbands. The model includes a prescription for supernovae feedback which drives gas out of galaxies at a rate proportional to the current star formation rate, with a constant of proportionality that is larger for less strongly bound systems. This negative feedback flattens the faint end slope of the luminosity function but, on its own, its effect is not strong enough to account for the measured function (Paper I). ? developed, and incorporated into this model, a detailed, self-consistent treatment of photoionization suppression of galaxy formation (hereafter abbreviated as PhS). By causing further suppression of faint galaxy formation, this mechanism brought the model luminosity function into excellent agreement with observations. In Paper I, we also included a much more detailed treatment than was previously possible of the evolution of satellite galaxies as they orbit within larger halos experiencing tidal forces and gravitational shock heating. (Hereafter, we abbreviate “tidal limitation as TiL.) The associated mass loss also affects the shape of the luminosity function, and so we will explore this process in this paper.

We adopt essentially the same model parameters as in Paper I, with the following differences. While the models of Paper I used the Press-Schechter halo mass function, we choose to adopt the function proposed by ? which gives a better match to the results of N-body simulations. Consequently, to produce a reasonable galaxy luminosity function, we find it necessary to adjust the parameters $`\mathrm{\Omega }_\mathrm{b}`$ from $`0.020`$ to $`0.024`$, $`ϵ_{}`$ (which determines the star formation rate) from 0.0050 to 0.0067, and $`\mathrm{{\rm Y}}`$ (which affects mass-to-light ratios) from 1.38 to 1.03 (since $`\mathrm{{\rm Y}}`$ affects the fraction of mass recycled in star formation, we adjust that fraction accordingly). Even with these changes, the ? mass function results in a somewhat worse fit to the bright end of the luminosity function than was obtained in Paper I. We defer further study of this aspect of the model to future work. We adopt an escaping fraction of ionizing photons from galaxies, $`f_{\mathrm{esc}}`$, of 12%. This produces a neutral hydrogen fraction for gas at zero overdensity in the IGM at $`z=6`$ of $`5\times 10^4`$, in agreement with the lower limit of $`3.4\times 10^4`$ derived by ?, while causing reionization to occur at as high a redshift as possible. This maximizes the associated suppression of galaxy formation111We could increase $`f_{\mathrm{esc}}`$ slightly further without violating the limit of ? but, since the neutral hydrogen fraction drops very rapidly in our model at these redshifts (Paper I), it makes almost no difference to the suppression of galaxy formation.. The resulting model is essentially the same as the $`f_{\mathrm{esc}}=10\%`$ model presented in Paper I.

## 3 The Galaxy Luminosity Function

### 3.1 Luminosity functions in halos of different mass

We begin by studying the luminosity functions of galaxies in individual dark matter halos of different mass, and then proceed to consider how these combine together to form the global galaxy luminosity function. Using the model described in §2, we simulate galaxy formation in a large number of halos of masses $`10^{11}`$, $`10^{12}`$, $`10^{13}`$, $`10^{14}`$ and $`10^{15}h^1M_{}`$222Throughout this paper we write Hubble’s constant as $`H_0=100h`$km s-1 Mpc-1.. This spans halo masses appropriate to systems ranging from the Local Group to rich galaxy clusters. (The $`10^{11}h^1M_{}`$ bin corresponds to halos less massive than that of the Local Group but we include it since it shows the most dramatic effects.) Merger trees are constructed with a resolution of $`10^5`$ of the final halo mass or $`10^8h^1M_{}`$, whichever is smaller. This is sufficient to follow accurately the formation and evolution of galaxies brighter than $`M_\mathrm{B}5\mathrm{log}h=10`$ except in the most massive clusters ($`10^{15}h^1M_{}`$) for which resolution begins to affect our results faintwards of $`M_\mathrm{B}5\mathrm{log}h=12`$; for these clusters, we show results only down to $`M_\mathrm{B}5\mathrm{log}h=13`$. We perform calculations for our “standard model” (i.e. including the effects of PhS and TiL of satellite galaxies), and repeat them switching off PhS and/or TiL (resulting in a total of four different sets of results), in order to assess the effects of each physical mechanism separately. We simulate a sufficiently large number of halos of each mass so that the average luminosity functions are reasonably smooth.

The solid lines in Figure 1 show the luminosity functions of galaxies in our simulated halos. The basic form of the standard model luminosity function is similar in all mass halos. It has two components: a near power-law distribution at faint magnitudes which is made up of satellite galaxies (i.e. those galaxies that do not reside at the centre of the final halo), and a strongly peaked distribution at bright magnitudes beyond which no more galaxies are found and which is made up of the central galaxy in each final halo. As we consider halos of increasing mass, both components shift to ever brighter luminosities, but the peaked component decreases in amplitude relative to the power-law component, indicating that, in low mass halos, a single galaxy tends to dominate, while in clusters the light is more equally shared among many galaxies.

Observed luminosity functions, $`\varphi (L)=\mathrm{d}n/\mathrm{d}L`$ (where $`n`$ is the number of galaxies per unit volume and $`L`$ is luminosity), are typically described by the Schechter function,

$$\varphi (L)\mathrm{d}L=\varphi _{}\left(\frac{L}{L_{}}\right)^\alpha \mathrm{exp}\left(\frac{L}{L_{}}\right)\frac{\mathrm{d}L}{L_{}},$$
(1)
where $`\varphi _{}`$, $`L_{}`$ and $`\alpha `$ are parameters. At magnitudes faint compared to the characteristic luminosity, $`L_{}`$, has the simple form $`\varphi (L)L^\alpha `$. Following normal practice, we will refer to $`\alpha `$ as “the faint end slope.” Although the luminosity functions in Fig. 1 are not well fit by the Schechter form, we can still obtain a formal value of $`\alpha `$ by fitting to the data in the first few bins plotted. We find slopes in the range $`1.57<\alpha <1.48`$, except in the lowest mass halos ($`10^{11}h^1M_{}`$) for which a much flatter slope, $`\alpha =1.06`$, is found (although it is possible that a steeper slope would be obtained also in this case if we probed to even fainter luminosities). There is little evidence for a strong variation of slope with halo mass, except perhaps for a weak (and rather insignificant) trend for *flatter* slopes in higher mass clusters. However, it is important to note that the slope is not constant with galaxy magnitude. This is clearly seen, for example, in the $`10^{12}h^1M_{}`$ halos for which, in the region $`18<M_\mathrm{B}5\mathrm{log}h<15`$, the slope is close to $`\alpha =1`$.

Switching off TiL in our model, and reverting to the simpler calculation of dynamical friction of ?, results in the dotted curves of Fig. 1. Clearly, TiL has little influence on the faint end of the luminosity function, although it does slightly reduce the luminosities of already faint galaxies (as is particularly noticeable for $`10^{12}h^1M_{}`$ halos). A more interesting effect of TiL is apparent in the massive clusters ($`10^{14}`$ and $`10^{15}h^1M_{}`$ halos). Here, switching off TiL results in a new population of very luminous galaxies, visible as a “bump” in the luminosity function at bright magnitudes. These highly luminous objects form through multiple mergers at the centres of cluster halos. When TiL is included, the merger rate of massive galaxies in clusters is significantly reduced, and none of these highly luminous objects form. As shown in the left-hand panel of Fig. 2, cluster central galaxies are about an order of magnitude less massive when our accurate calculation of dynamical friction is used. Note that the inclusion of TiL makes much less difference in $`10^{12}h^1M_{}`$ halos, due to the very different shape of the luminosity function in these systems (i.e. there is relatively little mass in satellites to merge into the central object). This difference between the merger rates in our model and that of ? cannot be fully reproduced by simply lengthening the dynamical friction timescale in the ? prescription by some factor (even if this factor is a function of halo mass). The shape of the distribution of merger times is significantly different between the models and, in our calculation, varies with halo mass.

An effect similar to that shown by the comparison of the solid and dotted lines in Fig. 1 was noticed by ? in cluster luminosity functions calculated using a different semi-analytic model. ? computed luminosity functions adopting both simple estimates of galaxy merger rates (similar to those assumed in computing the dotted curves in Fig. 1) and merger rates taken directly from a high resolution simulation of dark matter. They found that the simple scheme produced highly luminous galaxies through merging that were not present in the N-body calculation. Based on the number of massive mergers quoted by ?, we can conclude that a similar reduction in central galaxy mass occurred when our realistic merger rate calculation was employed. As suggested by ?, the cause of this difference is that the simple scheme underestimates the merging timescale of rather massive subhalos in clusters, leading to many massive galaxies merging with the central object in the final halo. In our more detailed calculation, as in the N-body simulation, these large subhalos loose a significant amount of their mass as they orbit within the cluster, leading to a reduced dynamical friction force and hence to a reduced merger rate.

The right-hand panel of Fig. 2 shows the mass functions of *surviving* substructure halos in clusters at $`z=0`$. The dotted histogram shows the model with no TiL, while the dashed histogram corresponds to the model with TiL included, but the mass of each substructure is plotted as it was *before* it experienced any tidal mass loss. Clearly, inclusion of TiL has resulted in fewer mergers since there are more surviving substructures of a given initial mass. The solid histogram gives the distribution of substructure masses after tidal mass loss. Comparing the dashed and solid histograms indicates that halos typically lose between 50% to 70% of their mass (with higher mass halos losing less than lower mass halos). The dynamical friction time scales as the mass of the satellite halo and is therefore increased by factors of 2 to 3 when TiL is included, greatly reducing the galaxy merger rates.

Switching off PhS (short-dashed lines in Fig. 1) has a more dramatic effect on the luminosity functions than switching off TiL. Firstly, the faint end slope steepens in almost all systems to values in the small range $`1.71<\alpha <1.64`$. (The exception are the lowest mass halos which have $`\alpha =1.38`$ because of the particularly strong effects of supernovae feedback in these halos.) The increase in the number of faint galaxies is more pronounced in lower mass systems. In high-mass systems there is a weak trend for the bright end of the luminosity function to be shifted slightly faintwards when PhS is ignored. In this case, more gas is locked up in the fainter galaxies, leaving less gas to form massive, bright galaxies. In the lowest mass systems the opposite effect occurs and the central galaxies become brighter when PhS is ignored. In these $`10^{11}h^1M_{}`$ halos, PhS suppresses the amount of gas that is able to accrete into the halo, thus reducing the luminosity of the associated galaxy. The way in which various processes associated with photoionisation shape the luminosity function in halos of different mass, particularly its faint end, is discussed in detail in the Appendix. There we also discuss the effect of various approximations in the treatment of photoionisation.

Long-dashed lines show results when both TiL and PhS are left out of our calculations. These are essentially accumulations of the effects seen when each process was switched off individually, and result in a further steepening of the faint end slopes. In these models, only feedback from supernovae affects the faint end of the luminosity function in a manner which is essentially independent of halo mass.

### 3.2 The global luminosity function

We now explore how the individual halo luminosity functions presented in Section 3.1 combine to produce the global luminosity function. An initially surprising aspect of the individual halo luminosity functions is that their faint end slopes are, in all cases, steeper than that of the global luminosity function. This, as we showed in Paper I, has a slope of around $`1.2`$ both in our models and in the real universe. How does PhS contrive to produce a flatter faint end slope for the total population than for any of the constituent individual halos? To understand this behaviour, we must consider the contribution of different mass halos to the global luminosity function.

The upper panels of Fig. 3 show the contributions to the global luminosity function from satellite and central galaxies (left and right-hand panels respectively) residing in halos of different mass. The slope of the combined satellite luminosity function reflects the power-law slope of the individual halo luminosity functions and is dominated over a wide range of luminosities by halos of $`10^{13}`$$`10^{14}h^1M_{}`$. The luminosity function of central galaxies, on the other hand, has a much flatter faint end slope, which is determined by the way in which central galaxy luminosity scales with halo mass. In our models, this scaling depends primarily on the combined effects of supernovae and PhS feedback. Since the central galaxy luminosity functions of individual halos are strongly peaked around a particular luminosity, the global central galaxy luminosity function is dominated by halos in a narrow range of mass at each luminosity. (The peak for galaxies in $`10^{10.5}`$ to $`10^{11.5}h^1M_{}`$ is much broader, since these mass scales are affected by PhS, causing strong suppression of galaxy formation in the lower mass halos in the range. Figure 1 shows much more clearly the sharply peaked luminosity distribution of central galaxies in halos of fixed mass.)

The lower left-hand panel of Fig. 3 shows how the satellite and central galaxy luminosity functions combine to produce the global luminosity function (which is compared to the recent observational determination from the 2dF galaxy redshift survey of ?). Brightwards of $`M_\mathrm{B}5\mathrm{log}h17`$, central galaxies dominate, while faintwards of this, the satellite contribution gradually takes over. The faint end slope of the global luminosity function ends up being intermediate between that of the satellite and central galaxy luminosity functions.

Finally, in the lower right-hand panel of Fig. 3 we show the contribution to the global luminosity function from halos of different mass. Clearly, the faint end slope of the global luminosity function is determined in part by the relation between central galaxy luminosity and halo mass, not just by the faint end slopes of the individual halo luminosity functions. This explains why the global galaxy luminosity function ends up having a flatter faint end than the individual halo luminosity functions. It is interesting to note that both the total and cluster (i.e. $`10^{15}h^1M_{}`$ halo) luminosity functions are reasonably well fit by Schechter functions, albeit with slightly different parameters ($`\alpha =1.31/1.45`$ and $`M_{}5\mathrm{log}h=20.12/19.60`$ for the total/cluster luminosity function; note that the correlation between the parameters $`\alpha `$ and $`M_{}`$ should be borne in mind when comparing these values).

## 4 Comparison with Observations

Our model predicts marked differences in the luminosity functions of galaxy populations residing in halos of different mass. Although the detailed form of these predictions depends on the details of our galaxy formation model, the gross differences seen in Fig. 1 are generic predictions that bear directly on basic features of the model such as hierarchical clustering from CDM initial conditions, gas cooling, photoionization, feedback, etc. Thus, in principle, individual halo luminosity functions provide a strong test of the CDM galaxy formation paradigm. Unfortunately, implementing such a test in practice is difficult because of the observational complications inherent in identifying galaxies attached to a particular dark matter halo. Gravitational lensing is a powerful and promising tool for detecting dark matter halos directly, but this technique is still in early stages of development (e.g. ?). A less direct, but still useful approach, consists of finding groups and clusters in large redshift catalogues such as the 2dF and SDSS surveys. When interpreted with the aid of cosmological simulations, it is possible to go some way towards identifying galaxy populations likely to be associated with single halos (and their subhalos) of a given mass (?). This approach will yield interesting data for comparison with our model predictions in the near future.

In the interim, ? have provided a limited compilation of observational data. They give estimates of the B-band luminosity functions of galaxies in the Local Group and in the Ursa Major, Virgo and Coma clusters. In our model, these systems are expected to reside in halos of mass similar to those we have simulated, i.e. $`10^{12}`$, $`10^{13}`$, $`10^{14}`$ and $`10^{15}h^1M_{}`$ respectively (????). We show Trentham & Hodgkin’s (2002) compilation in Fig. 1 as open circles. The vertical normalization of the datasets have been adjusted arbitrarily to permit an easier comparison of the shapes and faint end slopes of the model and observed luminosity functions. For the two most massive systems in the figure, our standard model is in reasonably good agreement with the observed luminosity function, including the faint end slope, but for Ursa Major and the Local Group, the observations indicate a much shallower slope than is produced by our model. The same discrepancy was noted by ? for the Local Group, and is also apparent in the predicted Local Group luminosity function of ? (which has $`\alpha 1.5`$ over the range $`20<M_\mathrm{V}5\mathrm{log}h_{70}<10`$, where $`h_{70}`$ is the Hubble constant in units of 70km/s/Mpc, for the case of reionization at $`z=8`$ which is the closest to our own calculations), suggesting that this is a rather robust prediction of PhS.

In making the comparison between theory and observations in Fig. 1, one should keep in mind the possibility that the two may not correspond to quite the same quantity. The theoretical predictions pertain to luminosity functions of galaxy populations residing in individual dark matter halos of a given mass. However, there is no guarantee that the samples selected by ? do indeed come from individual dark matter halos. For example, these samples might be contaminated by contributions from secondary halos which happen to be nearby the dominant halo. This could be particularly important for the Ursa Major cluster which is a rather diffuse object and is too large, given its mass, to be associated with a single, virialized halo (according to the standard theoretical definitions). There are ambiguities regarding the Local Group data as well. As argued by ?, many dwarfs nominally associated with the Local Group lie beyond the region which, according to our model, contains the (distinct, but similar mass) halos of the Milky Way and M31. In Fig. 1, we show as filled squares the luminosity function of the galaxies that ? would class as lying within the virial radius of the Milky Way or M31. This luminosity function also shows a fairly flat faint end. Nevertheless, selection effects such as these must be accounted for in a more thorough comparison of theory and observations.

The results of Fig. 1 are shown in a different way in Fig. 4 which more clearly shows the variation in the shapes of the luminosity functions at different luminosities. Here, we plot the effective luminosity function slope, $`\alpha _{\mathrm{eff}}=L(\mathrm{d}^2N/\mathrm{d}L^2)/(\mathrm{d}N/\mathrm{d}L)`$, as a function of absolute magnitude for all the systems we have simulated. (For clarity, we show results only for our standard model and for a model with both PhS and TiL switched off.) The luminosity functions with no PhS or TiL in $`10^{13}`$ and $`10^{14}h^1M_{}`$ halos show an extended region where $`\alpha _{\mathrm{eff}}`$ is almost constant, indicating a power-law luminosity function. With PhS and TiL switched on, the slope is shallower, but the luminosity function is no longer a power-law over an appreciable range of magnitudes. For the other mass systems, the luminosity function is nowhere well described by a power-law and, in every case, the inclusion of PhS makes the slope shallower, the effect being larger in the lower mass systems.

? characterized variations in luminosity function shapes using the “dwarf-to-giant ratio” (i.e. the number of galaxies in some range of faint magnitudes relative to the number at brighter magnitudes). We compute a similar ratio for our model with PhS and TiL and present the results in Table 1. (We use a slightly different magnitude range for the dwarfs because, for our most massive halos, our model is not complete over the range of magnitudes used by Trentham & Hodgkin.)

For the two most massive systems, our model results are in reasonably good accord with the observational determinations, but for the lower mass systems there are significant discrepancies. This analysis verifies and quantifies the comparison between the shapes of the predicted and observed luminosity functions made in Fig. 1.

## 5 Discussion

Using a semi-analytic model of galaxy formation, we have calculated the luminosity function of galaxy populations contained in halos of different mass. We find large differences in the shapes of these luminosity functions. In smaller mass halos ($`M<10^{13}h^1M_{})`$, the luminosity function has a “hump” at bright magnitudes and a roughly power-law shape at fainter magnitudes. In more massive systems, the shape approaches the familiar Schechter form. These differences reflect the relative contributions of “central” and “satellite” galaxies. In the smaller mass halos, the bright end of the luminosity function is dominated by a single or at most a few bright galaxies and the rest of the population consists predominantly of much fainter satellites. In the more massive halos, the light is more evenly distributed amongst a large number of galaxies. These differences reflect the complex interplay between the processes that establish the number of halos and subhalos of different mass (i.e. hierarchical clustering from CDM initial conditions) and the processes that light them up (i.e. gas cooling, stellar evolution, feedback, etc).

The global luminosity function is the weighted sum of the luminosity functions of individual halos. Interestingly, the faint end slopes of the individual luminosity functions are always steeper than the faint end slope of the global luminosity function. This can be understood in terms of the relative contributions to the global luminosity function of satellite and central galaxies from individual halos, whose luminosity functions have steep and shallow slopes respectively. The slope of the global function is intermediate between these two.

Our model of galaxy formation represents an advance over previous models because of its detailed treatment of two processes that are important in establishing the luminosity function: the suppression of galaxy formation in small halos at high redshift (PhS), as a result of the photoionization of the IGM by early generations of galaxies and quasars, and tidal mass loss from satellite galaxies (TiL). PhS is particularly relevant at the faint end and has the net effect of reducing the number of faint galaxies, giving rise to a flatter luminosity function than would otherwise be the case. This outcome is far from obvious. For example, a simplistic model of PhS in which galaxy formation is assumed not to occur in low mass halos below the reionization redshift results, in fact, in a steeper luminosity function (see Appendix A). In reality, the effects of PhS depend on the mass of the halo relative to the time-dependent “filtering mass” (defined as the halo mass which accretes only half the mass that it would have accreted in the absence of photoionization; ?). Galaxies can form even if their halos have mass smaller than the filtering mass, but with reduced luminosity. The lower the mass of the halo relative to the filtering mass, the fainter is the galaxy that forms. The filtering mass is an increasing function of time. Thus, although lower mass halos form at higher redshift, the redshift where the filtering mass becomes comparable to their mass is also higher, increasing the fraction of them which experience strong suppression. These two effects combined *do* tend to produce a flatter luminosity function but the degree of flattening depends on their detailed balance333A model in which a cut-off in circular velocity rather than mass is assumed gives results in quite good agreement with our detailed calculations (?).. As we have shown previously (?; see also ?), PhS on its own cannot account for the shallow faint end slope of the global luminosity function. Other processes, such as feedback from energy released by stellar winds and supernovae, need to be included in order to reduce the number of faint galaxies to the observed level.

The other process that we have treated in detail, TiL, is relevant at the bright end of the luminosity function, particularly in massive halos harbouring rich clusters. Mass loss from satellite galaxies, as they spiral into the centre of their halo, increases the dynamical friction time, so reducing the merger rate. This prevents the formation of some of the highly luminous and unobserved cluster galaxies that tend to appear in models of galaxy formation that do not take this process into account.

Keeping in mind the provisos of §4, we find that our model fails to reproduce the steepening of the faint end slope of the luminosity function with system richness found by (?). The model agrees quite well with the data for the Virgo and Coma clusters, but it predicts a steeper faint end than is inferred for the Local Group and Ursa Minor. Photoionization is an effective (and unavoidable) mechanism for suppressing galaxy formation in low mass halos, but it does not produce the kind of “environmental” variation seen by ?. At the low mass end, this failure of the model is related to the discrepancy that we (?, see also ?), found previously in our CDM models of the Local Group. Photoionization has a dramatic effect on the overall number of satellites that survive in galactic halos and can reduce it to the levels seen in the Local Group. However, the satellite luminosity function in the models is considerably steeper than is observed. This discrepancy contrasts with the success of CDM models in reproducing the structural properties of the satellites (?).

Taking at the face value the discrepancy between our models and the data of ?, it is interesting to speculate about the possible root causes of the problem. Leaving aside alternative theories for the nature of the dark matter or drastic changes to the initial power spectrum of density perturbations specifically designed to reduce the amount of small-scale power (e.g. ??), we focus on our modeling of baryonic processes within the context of the CDM cosmogony. This is undoubtedly simplified. For example, the effects of PhS depend upon both the time variation of the filtering mass and on the assumed functional form of the suppression, both of which are uncertain. As discussed in the Appendix, reasonable variations in our assumed mass dependence of PhS do not seem capable of producing a sufficiently flat faint end slope in our models of the Local Group. However, we cannot exclude the possibility that the simplifications made in our model are inappropriate. For example, we neglect the patchy nature of reionization, but if low density regions became ionized first, galaxy formation would be suppressed more effectively in these regions, altering the temporal evolution of the filtering mass. Localized PhS triggered, for example, by a nearby galaxy or quasar could also change our model predictions. These processes are poorly understood theoretically, but observational evidence (e.g. ?) suggests that they could be important in determining the thermodynamic state of the IGM at high redshift.

At a more basic level, there are approximations in our model which might be inappropriate in the regime of low mass halos. Thus, although comparisons with N-body/hydrodynamical simulations indicate that our estimates of gas cooling rates are accurate for relatively massive galaxies (???), it has not been shown that the same is also true for the low mass galaxies of interest here. Cooling times in these objects can become very short and this might conceivably render incorrect the assumption of cooling from an initial, spherically symmetric quasi-hydrostatic equilibrium. Further study of gas cooling in low mass systems is necessary to address this issue. Similarly, it is far from clear that the simple rules for star formation in our model, which are empirical parameterizations tuned to match observational constraints for relatively bright galaxies at low redshifts, remain valid when extrapolated to low mass galaxies and/or high redshifts. Other physical constraints on star formation not currently included in our models may also be important for these low mass galaxies (e.g. ?). Finally, our modeling of feedback is also simplified and, again, its extrapolation to low masses may be unrealistic. Possible sources of feedback such as strong outflows or heating by AGN which are currently neglected may be important on these scales.

In conclusion, we have found substantial differences in the luminosity functions of galaxies residing in dark matter halos of different mass. These result from the interplay of a variety of processes that affect the formation of galaxies of different luminosity. Although the exact predictions of our model no doubt depend on its details, the gross differences between the predicted luminosity functions are likely to be generic and to be present in a broad class of cold dark matter models of galaxy formation. Comparison with existing data gives mixed results: fair agreement with the luminosity functions in Virgo and Coma but substantial disagreement with the luminosity functions in the Local Group and Ursa Minor cluster. The much larger and better controlled samples that will be forthcoming from cluster analyses of the 2dF and SDSS galaxy redshift surveys will allow a more comprehensive test of the general class of cold dark matter models of galaxy formation.

## Acknowledgments

## Appendix A Photoionization and the Faint End of the Luminosity Function

The faint end of the luminosity function directly reflects the outcome of feedback effects. The two sources of feedback in our model, supernova-driven winds and photoionization, are both important in determining the number of faint galaxies. As we discussed in Paper I, photoionization inhibits the formation of faint galaxies for two reasons: it raises the temperature and pressure of the IGM, curtailing the ability of gas to accrete into halos, and it reduces the rate at which gas in halos can cool, inhibiting star formation. The first of these processes is the dominant effect. It can be conveniently described in terms of a “filtering mass,” $`M_\mathrm{F}`$, defined as the mass of a halo which accretes only half the baryonic mass that it would have accreted in the absence of photoionization. The mass of gas accreted by a halo of mass $`M_\mathrm{h}`$ is approximated by the following formula (?), based on the results of gas-dynamical simulations:

$$M_{\mathrm{gas}}=\frac{f_\mathrm{b}M_\mathrm{h}}{[1+(2^{1/3}1)M_\mathrm{F}/M_\mathrm{h}]^3},$$
(2)
where $`f_\mathrm{b}`$ is the universal baryon fraction and $`M_\mathrm{F}`$ is the filtering mass.

There are three different aspects to consider when evaluating the effect of the filtering mass on the faint end slope of the luminosity function:

1. Galaxy formation is significantly suppressed in halos less massive than the filtering mass.
2. The filtering mass is an increasing function of time (at least over the range of redshifts of interest here).
3. For halos formed at a given redshift, the suppression of galaxy formation is greater the smaller the halo mass is relative to the filtering mass.

The impact of these three factors will depend on the distribution of formation redshifts of dark matter halos of different mass in different environments444By “environment” we mean the mass of the halos into which that earlier halo has been incorporated by the present day.. According to the extended Press-Schechter formalism, it is always true (for a CDM power spectrum) that the typical formation redshift of a halo increases as the mass of the halo decreases, and also as the mass of the final host in which the halo resides increases. Further understanding of halo formation redshift distributions may be gained by considering eqn. 2.15 of ? that gives the mass function of progenitor halos at any redshift and which we reproduce in slightly different form below:

$`{\displaystyle \frac{\mathrm{d}N}{\mathrm{d}M}}(z)`$ $`=`$ $`{\displaystyle \frac{1}{\sqrt{2\pi }}}{\displaystyle \frac{M_{\mathrm{host}}}{M}}{\displaystyle \frac{(\delta _\mathrm{c}^{(\mathrm{z})}\delta _\mathrm{c}^{(0)})}{[\sigma ^2(M)\sigma ^2(M_{\mathrm{host}})]^{3/2}}}`$ (3)
$`\mathrm{exp}\left({\displaystyle \frac{(\delta _\mathrm{c}^{(\mathrm{z})}\delta _\mathrm{c}^{(0)})^2}{2[\sigma ^2(M)\sigma ^2(M_{\mathrm{host}})]}}\right){\displaystyle \frac{\mathrm{d}\sigma ^2}{\mathrm{d}M}},`$
where $`\delta _\mathrm{c}^{(\mathrm{z})}`$ is the critical value of the linear theory fractional overdensity for collapse at redshift $`z`$, $`M`$ is the mass of the progenitor halo at that redshift, $`\sigma ^2(M)`$ is the variance of the linear fractional overdensity in a sphere of mass $`M`$, and $`M_{\mathrm{host}}`$ is the mass of the halo into which that progenitor has been incorporated by the present day.

Consider galaxies living in relatively low mass halos at the present day (in which most of the galaxies that make up the faint end of the luminosity function reside). The relative numbers of halos of masses $`M_1`$ and $`M_2(<M_1)`$ at some particular redshift depends upon $`\sigma ^2(M_1)\sigma ^2(M_{\mathrm{host}})`$ and $`\sigma ^2(M_2)\sigma ^2(M_{\mathrm{host}})`$. As $`M_{\mathrm{host}}`$ increases, $`\sigma ^2(M_{\mathrm{host}})`$ decreases for a CDM power spectrum. Thus, for sufficiently large $`M_{\mathrm{host}}`$, the relative numbers of halos of masses $`M_1`$ and $`M_2`$ tends to a fixed value. For $`M_{\mathrm{host}}`$ comparable to $`M_1`$, the relative numbers of these halos become a very strong function of $`M_{\mathrm{host}}`$, and the number of mass $`M_1`$ halos is exponentially suppressed, as indicated by eqn. (3).

These theoretical expectations are clearly manifest in Fig. 5, where we plot the positions of galaxies in the halo-mass vs. formation-redshift plane. All of the galaxies we consider at $`z=0`$ are satellites in a larger host halo (of mass $`M_{\mathrm{host}}`$) due to the ranges of luminosity and $`M_{\mathrm{host}}`$ we have chosen to consider. Therefore, we plot the formation redshift and mass for the halo in which the satellite formed. We adopt the definition of formation redshift of ?, namely a halo is assumed to be newly formed if its mass exceeds twice the mass of each of its progenitors at their own formation time, and plot the halo mass at the formation redshift (it may have increased afterwards due to continued accretion and merging). (The formation redshifts appear quantized in this figure because of the finite timesteps in our calculations which, however, are sufficiently fine so as not to affect the results.) The halo mass of the satellite is then set equal to the mass of the halo in which it lived at redshift $`z_{\mathrm{form}}`$). The position of a halo in this plane, relative to the redshift-dependent filtering mass (shown as a solid curve), determines the degree of PhS it experiences. We examine faint galaxies in two luminosity ranges, $`13<M_\mathrm{B}5\mathrm{log}h<12`$ (dots) and $`15<M_\mathrm{B}5\mathrm{log}h<14`$ (small squares).

Consider first a model with no PhS (left-hand panels). In this case, faint galaxies currently hosted in a $`10^{14}h^1M_{}`$ cluster halo (lower panel) have a much more extended range of formation redshifts than galaxies in a $`10^{12}h^1M_{}`$ Milky Way-type halo (upper panel). The fainter galaxies located in lower mass halos have a more extended range of formation redshifts than their brighter counterparts, although this difference is rather small. The values of $`\alpha `$ for the two populations, given in each panel, indicate steep slopes in both environments. The filtering mass has no influence on these results, of course, but is shown to indicate which of halos might be expected to still form a galaxy when PhS is switched on.

In the right-hand panels we plot results for a model with PhS. The effect of the filtering mass can now be clearly seen: halos below the solid line form galaxies much less efficiently than before (although there are certainly still some halos that managed to make galaxies below this line). The values of $`\alpha `$ given in the panels indicate that some flattening of the luminosity function has occurred, but this effect is small.

Let us now examine the importance of these various processes using simplified models of PhS. Figure 6 shows luminosity functions in different mass halos constructed using the simplified models described below, together with our standard model of PhS (heavy solid line) and a model with no PhS (thin solid line).

In the simplest scenario, suppose the filtering mass jumps abruptly from zero to $`M_\mathrm{F}`$ at redshift $`z_{\mathrm{reion}}`$, and that any halo of smaller mass forming at lower redshift makes no galaxy at all, while galaxy formation in other halos proceeds unchanged. In this case, the luminosity function should be unaffected at bright magnitudes (since these galaxies form in halos which are more massive than $`M_\mathrm{F}`$), should plummet sharply at a magnitude corresponding to halo mass equal to $`M_\mathrm{F}`$, and should rise again to fainter magnitudes (since these galaxies inhabit lower mass halos which typically form at redshifts greater than $`z_{\mathrm{reion}}`$). On the basis of eqn. (3), the slope of this rise should be large for low $`M_{\mathrm{host}}`$ halos since these have the greatest variation in progenitor number with mass at $`z_{\mathrm{reion}}`$, and should approach some fixed value for large $`M_{\mathrm{host}}`$. This behaviour is borne out when we apply this simple prescription to our semi-analytic models, using values of $`M_\mathrm{F}=3\times 10^{10}h^1M_{}`$ (comparable to the value of $`M_\mathrm{F}`$ at the redshifts where the majority of the faint galaxies are formed in our standard model) and $`z_{\mathrm{reion}}=6.5`$ (the redshift of reionization in our standard model; dot-long-dashed lines in Fig. 6)—suppression is greatest for small $`M_{\mathrm{host}}`$, tends to steepen the faint end of the luminosity function, and causes the most steepening for the lowest $`M_{\mathrm{host}}`$. This model produces a discontinuity in the luminosity function and is a poor approximation to the full treatment.

At the next level of sophistication, we take into account the variation of the filtering mass with redshift as predicted by our model, but retain the simple prescription wherein any halo forming with mass below the filtering mass forms no galaxy at all. Since $`M_\mathrm{F}`$ increases with time, galaxy formation in lower mass halos is suppressed earlier, an effect which should tend to flatten luminosity function slopes. Since halos of a given mass form earlier in larger $`M_{\mathrm{host}}`$, we still expect the effects of filtering to be weaker for clusters than for the Local Group. At a given magnitude, this process would simply reduce the number of galaxies. However, the detailed consequences for the faint end slope are now harder to anticipate since they depend on just how $`M_\mathrm{F}`$ varies with redshift. Applying this prescription to our model (long-dashed lines in Fig. 6), we still find a discontinuity in the luminosity function (although this is much weaker than in model (i) since $`M_\mathrm{F}`$ varies smoothly with time) which is flattened relative to that without any PhS. Furthermore, the slopes are significantly flatter than in case (i), indicating that the time variation of $`M_\mathrm{F}`$ largely counteracts the steepening of the luminosity function produced by the variation in halo formation distributions. No environmental variation is introduced by these processes. The resulting luminosity functions are quite similar to those from our full PhS calculations, although the level of suppression is somewhat greater. Again, the effects are largest for the lowest $`M_{\mathrm{host}}`$.

Alternatively, we can consider a prescription which has no time variation in $`M_\mathrm{F}`$, but in which, instead, the degree of suppression varies smoothly with halo mass (as prescribed by ?)555? considered a similar model, but kept the circular velocity corresponding to $`M_\mathrm{F}`$ constant, resulting in a filtering mass that increased with time. This prescription, also adopted by ? provides a better match to the results of our standard model. Here, however, we are intersted only in exploring simplified models.. We find that this model also produces luminosity functions similar to those from our full PhS calculations (dot-short-dashed lines), and with about the right degree of suppression for suitable choices of $`M_\mathrm{F}`$ and $`z_{\mathrm{reion}}`$.

We next consider a model in which $`M_\mathrm{F}`$ varies with time and in which the degree of suppression varies smoothly with halo mass. This then differs from our full PhS calculations only in neglecting suppression due to photoheating by the ionizing background. In this model, photoionization redistributes galaxies which previously had a given magnitude over a range of fainter magnitudes. At a given magnitude, some galaxies are lost due to suppression, but also new ones appear as brighter galaxies are partially suppressed, thus becoming fainter. The net effect is therefore difficult to judge since it depends on the time variation of the filtering mass and on the previous shape of the luminosity function. We can, however, safely say that the effect is still expected to be larger for lower $`M_{\mathrm{host}}`$. Applying this prescription we reproduce the results of our full PhS calculation rather well (dotted lines in Fig. 6), finding slightly less suppression overall than in case (ii), but still with no environmental variation.

A smoother variation of suppression with halo mass would result in less flattening of the faint end slope. For example, the short-dashed lines in Fig. 6 show results for a suppression of the form

$$M_{\mathrm{gas}}=\frac{f_\mathrm{b}M_\mathrm{h}}{1+M_\mathrm{F}/M_\mathrm{h}},$$
(4)
with the same variation of filtering mass with redshift as in our standard model.

In conclusion, a simple model in which galaxy formation at $`z<z_{\mathrm{reion}}`$ is entirely suppressed below some fixed mass scale grossly overestimates the effects of PhS, and produces very different luminosity functions from those predicted by our full calculation. Including *either* a smoothly varying degree of suppression *or* a time-varying filtering mass produces better agreement with our full calculations, indicating that these two ingredients are of comparable importance. The degree of suppression (as characterised by the reduction in amplitude of the LF) for a given time dependence in $`M_\mathrm{F}`$ depends on the functional form used to characterize the variation of suppression with halo mass. The sharper the transition from weak to strong suppression, the greater the net effect on the luminosity function. Smoother transitions tend to produce steeper faint end slopes, but even the sharpest transition possible does not produce slopes as flat as are observed and we never find any significant environmental variation.


text:
# Critical analyses of order parameter and phase transitions at high density in Gross-Neveu model1footnote 11footnote 1 This work was partially supported by the National Natural Science Foundation of China.

## I Introduction

The symmetry restoring phase transition at high temperature and high density has been an interesting research topic which is closely relative to evolution of early universe and phase transition of the nuclear matter kn:1 ; kn:2 ; kn:3 ; kn:4 . A good laboratory to research the phase transition is the models of four-fermion interactions including the Gross-Neveu(GN) models in $`D=2`$ and $`D=3`$ kn:5  time-space dimensions. On the phase transitions in these models, extensive investigations have been made kn:6 ; kn:7 ; kn:8 ; kn:9 ; kn:10 ; kn:11 ; kn:12 ; kn:13 , some common conclusions have been reached but there is still particular question, e.g. whether first order phase transition exists or not in $`D=3`$ GN model, which needs to be further clarified.
For determining the order of a phase transition, one can analyze critical behaviors of dynamical fermion mass as the order parameter of symmetry breaking at a critical temperature and/or at a critical chemical potential. The phase transition will be second or first order if the dynamical fermion mass goes to zero at the critical point continuously or discontinuously.
The fundamental approach of the critical analyses should be using an effective potential. However, in many cases, the gap equation is a more simple and direct means. So called the gap equation we mean the essential part of the Schwinger-Dyson equation obeyed by the dynamical fermion mass as the order parameter of symmetry breaking. It can be independently derived from the fermion condensates induced by the four-fermion interactions without using an effective potential, though it has been proven that the gap equation can also come from the extreme value condition of an effective potential kn:14 . If a phase transition is second order and this normally happens when the gap equation has the only non-zero solution, then the critical analysis can be made simply by means of gap equation. This has been carried out in the demonstrations on high temperature second order phase transition for fixed chemical potential in $`D=4`$ Nambu-Jona-Lasinio model kn:15 ,$`D=3`$ kn:16  and $`D=2`$ kn:17  GN model. However, if a phase transition is first order and this is usually the case when the gap equation has two or more non-zero solutions, then how should one make the critical analysis ? Whether or in what conditions can one do that simply based on the gap equation ?
To answer the above questions, in this paper we will generalize critical analyses to the case including first order phase transitions. As far as the four-fermion models are concerned, it is most possible that the first order phase transition occurs in the case of high density. Hence we will investigate symmetry restoring phase transitions at high density for a given temperature, especially for zero temperature case, in $`D=2`$ and $`D=3`$ GN models. We will use both gap equations and effective potentials, and specially examine effectiveness and limitations of the gap equation for critical analyses of first order phase transition.
The Lagrangian of the models will be expressed by

$$(x)=\underset{k=1}{\overset{N}{}}\overline{\psi }^k(x)i\gamma ^\mu _\mu \psi _k(x)+\frac{g}{2}\underset{k=1}{\overset{N}{}}[\overline{\psi }^k(x)\psi _k(x)]^2,$$
(1)
where $`\psi _k(x)`$ are the fermion fields with $`N`$ ”color” components and $`g`$ is the four-fermion coupling constant. The discussions will be made in the fermion bubble diagram approximation which is equivalent to the leading order of the $`1/N`$ expansion. It is assumed that higher-order corrections in $`1/N`$ do not change the qualitative feature of the models, since our models involve only some discrete symmetries which will be spontaneously broken or restored kn:18 . Our basic starting point is the gap equations coming from the fermionic bilinear condensates induced by the four-fermion interactions in Eq.(1.1), besides the corresponding effective potentials. The gap equations have been given in Ref.kn:16 ; kn:17 . The corresponding effective potentials may be obtained from the relations between the gap equations and the effective potentials proven in Ref.kn:14 . Here we will directly use the known explicit results of them.
The paper is arranged as follows. In Sect. II we discuss $`D=2`$ GN model, including analyzing critically the second order high density phase transition at high temperature $`T`$ by the gap equation and in particular, the first order high density phase transition at $`T=0`$ by the gap equation combined with the corresponding effective potential. In Sect. III, similar discussions to ones in Sect. II will be made in $`D=3`$ GN model. The first order high density phase transition at $`T=0`$ in this model will be proven respectively by critical analyses based on the gap equation and the effective potential. A difference between it and general first order phase transition and this difference’s significance for usefulness of the gap equation analysis will be indicated. Finally in Sect. IV we come our conclusions.

## II $`D=2`$ Gross-Neveu model

In this modelkn:17 , there exists a matrix $`\gamma _5=\sigma ^3`$ anticommunicating with $`\gamma ^0=\sigma ^1`$ and $`\gamma ^1=i\sigma ^2`$, where $`\sigma ^i(i=1,2,3)`$ are the Pauli matrices. By the Mermin-Wagner-Coleman theorem, a continuous symmetry cannot be spontaneously broken in 2 dimensions kn:19 . Dynamical generation of the fermion mass will spontaneously break only the discrete chiral symmetry $`\chi _D:\psi (t,x)\stackrel{\chi _D}{}\gamma _5\psi (t,x)`$ and the special parity $`𝒫_1:\psi (t,x)\stackrel{𝒫_1}{}\gamma ^1\psi (t,x)`$. Hence possible phase transition at high temperature and high density will relate to restoration of only the $`\chi _D`$ and $`𝒫_1`$. However, in 2 dimensions, symmetry restoration could occur only in the mean-field approximation kn:6 ; kn:20  which is equivalent to fermion bubble diagram approximation kn:17 .Because the former can be used only if $`N\mathrm{}`$ or $`N`$ is large and only a finite segment of one-space dimension system is considered kn:20 , here it is assumed that the use of the fermion bubble diagram approximation will be in the above conditions. In 2 dimensions, Lagrangian (1.1) is perturbatively renormalizable. Assume the four-fermion interactions in Eq.(1.1) could lead to the condensates $`_{k=1}^N\overline{\psi }^k\psi _k0`$ when $`T=\mu =0`$ and the thermal condensates $`_{k=1}^N\overline{\psi }^k\psi _k_T0`$ when $`T0`$, where $`\mu `$ is the chemical potential of fermions, then we will obtain the renormalized gap equations in respective cases and finally reduce the gap equation at $`T0`$ to the following form

$$\mathrm{ln}\frac{m(0)}{m}=F_1(T,\mu ,m)=I_1(y,r)+I_1(y,r)$$
(2)
with the denotations

$$I_1(y,r)=_0^{\mathrm{}}\frac{dx}{\sqrt{x^2+y^2}}\frac{1}{\mathrm{exp}(\sqrt{x^2+y^2}r)+1},$$
(3)
where $`y=m/T`$, $`r=\mu /T`$, $`m(0)`$ and $`mm(T,\mu )`$ are respectively the dynamical fermion masses at $`T=\mu =0`$ and at $`T0`$ and/or $`\mu 0`$. It is easy to verify that $`F_1(T,\mu ,m)`$ is an increasing function of $`T`$ and $`\mu `$ but a decreasing function of $`m`$, and from Eq.(2.1), we can assume that at some critical temperature $`T_c`$ and critical chemical potential $`\mu _c`$, the dynamical fermion mass $`m`$ becomes zero. This will lead to a critical equation. At a high temperature $`T`$, by using the high temperature expansion of $`F_1(T,\mu ,m)`$ kn:15  we may reduce the gap equation (2.1) to

$$\begin{array}{c}\mathrm{ln}\frac{m(0)}{T\pi }+\gamma =g(T,\mu )(\frac{y}{2\pi })^2+h(T,\mu )+𝒪\left[(\frac{y}{2\pi })^4\right],\\ g(T,\mu )=\frac{7}{2}\zeta (3)93\zeta (5)(\frac{r}{2\pi })^2+\frac{1905}{2}\zeta (7)(\frac{r}{2\pi })^4,\\ h(T,\mu )=7\zeta (3)(\frac{r}{2\pi })^231\zeta (5)(\frac{r}{2\pi })^4+127\zeta (7)(\frac{r}{2\pi })^6,\end{array}$$
(4)
where $`\zeta (s)(s=3,5,7)`$ are the Riemann zeta functions and $`\gamma `$ is the Euler constant. Noting that the $`\mathrm{ln}m`$ term in Eq.(2.1) has been cancelled, thus we can set $`y=0`$ in Eq.(2.3) and obtain the critical equation

$$\mathrm{ln}\frac{m(0)}{T_c\pi }+\gamma =h(T_c,\mu _c).$$
(5)
In the derivation of Eq. (2.4) an implicit fact is that Eq. (2.3) has the only solution $`m`$ and $`m`$ can go to zero continuously as $`T`$ and $`\mu `$ vary. This is just the characteristic of a second order phase transition. In view of this, Eq. (2.4) can be called the critical equation of a second order phase transition. The critical points determined by Eq. (2.4) are all second order ones. For verifying this conclusion, we can discuss from Eqs.(2.3) and (2.4) critical behaviors of the squared order parameter $`m^2`$ at $`\mu _c`$ when $`T`$ is fixed. Replacing $`T_c`$ by $`T`$ in Eq.(2.4) and substituting it into Eq. (2.3), we get

$$m^2(\mu _c^2\mu ^2)f(T,\mu _c)/g(T,\mu ),\mathrm{when}\mu \mu _c,$$
(6)
where

$$f(T,\mu _c)=7\zeta (3)62\zeta (5)(\frac{\mu _c}{2\pi T})^2+381\zeta (7)(\frac{\mu _c}{2\pi T})^4.$$
(7)
Eq.(2.5) shows that the squared order parameter $`m^2`$ goes to zero continuously when $`\mu \mu _c`$ in the form of $`\mu _c\mu `$, hence for a given high temperature $`T`$, the symmetry restoration phase transition at $`\mu _c`$ is second order indeed.
Eq.(2.4) comes from high temperature expansion of $`F_1(T,\mu ,m)`$ and is inapplicable to discussion of high density phase transition at low temperature and $`T=0`$. For discussion of the latter, we have to use the effective potential. In the following we will only focus on the high density phase transition at $`T=0`$. Let us first go back to Eq.(2.1). Through changing the integral variable by $`z=(x^2/y^2+1)^{1/2}`$ we can express

$$F_1(T,\mu ,m)=_1^{\mathrm{}}\frac{dz}{\sqrt{z^21}}\left[\frac{1}{e^{y(z\alpha )}+1}+(\alpha \alpha )\right],$$
(8)
where $`\alpha =\frac{\mu }{m}`$. In the limit $`T0`$ or $`y\mathrm{}`$, we have

$$F_1(T=0,\mu ,m)=\theta (\mu m)\mathrm{ln}\frac{\mu +\sqrt{\mu ^2m^2}}{m}.$$
(9)
Substituting Eq.(2.8) into Eq.(2.1) we obtain

$$\mathrm{ln}\frac{m(0)}{m}=\theta (\mu m)\mathrm{ln}\frac{\mu +\sqrt{\mu ^2m^2}}{m}$$
(10)
which has the solutions

$$m=m(0),\mathrm{when}\mu m$$
(11)
and

$`m`$ $`=`$ $`\{m(0)[2\mu m(0)]\}^{1/2}m_1,`$ (12)
$`\mathrm{when}\mu >m\mathrm{and}{\displaystyle \frac{m(0)}{2}}\mu <m(0).`$
On the other hand, the effective potential $`V_{eff}^{(2)}(T,\mu ,m)`$ has been proven to satisfy the equation kn:14

$$\frac{V_{eff}^{(2)}(T,\mu ,m)}{m}=\frac{m}{\pi }\left[\mathrm{ln}\frac{m}{m(0)}+F_1(T,\mu ,m)\right],$$
(13)
noting that the gap equation (2.1) multiplied by $`m`$ is just the extreme conditions given by $`V_{eff}^{(2)}(T,\mu ,m)/m=0`$. Hence, when $`T=0`$, $`V_{eff}^{(2)}(T=0,\mu ,m)`$ for $`m(0)/2\mu <m(0)`$ will have the three possible extreme points $`m=0,`$ $`m_1`$ and $`m(0)`$. To determine which one of them corresponds to a minimum or a maximum of $`V_{eff}^{(2)}(T=0,\mu ,m)`$, we may calculate and obtain directly from Eq. (2.12)

$$\frac{^2V_{eff}^{(2)}(T=0,\mu ,m)}{m^2}=\frac{1}{\pi }\{\begin{array}{ccc}\mathrm{ln}\frac{m}{m(0)}+1& \mathrm{when}& \mu m\\ \mathrm{ln}\frac{\mu +\sqrt{\mu ^2m^2}}{m(0)}\frac{m^2}{(\mu +\sqrt{\mu ^2m^2})\sqrt{\mu ^2m^2}}& \mathrm{when}& \mu >m\end{array},$$
(14)
where Eq. (2.8) has been used. Eq. (2.13), combined with Eqs. (2.10) and (2.11), give that

$$\frac{^2V_{eff}^{(2)}(T=0,\mu ,m)}{m^2}|_{m=0}=\frac{1}{\pi }\mathrm{ln}\frac{2\mu }{m(0)}\{\begin{array}{ccc}<0& \mathrm{when}& \mu <m(0)/2\\ >0& \mathrm{when}& \mu >m(0)/2\end{array},$$
(15)
$$\frac{^2V_{eff}^{(2)}(T=0,\mu ,m)}{m^2}|_{m=m_1}=\frac{1}{\pi }\frac{2\mu m(0)}{m(0)\mu }<0,\mathrm{when}\frac{m(0)}{2}<\mu <m(0)$$
(16)
and

$$\frac{^2V_{eff}^{(2)}(T=0,\mu ,m)}{m^2}|_{m=m(0)}=\frac{1}{\pi }>0,\mathrm{when}\mu <m(0).$$
(17)
Eqs. (2.14)-(2.16) indicate that
1) When $`\mu <\frac{m(0)}{2}`$, $`V_{eff}^{(2)}(T=0,\mu ,m)`$ will have the maximum point $`m=0`$ and the minimum point $`m=m(0)`$, thus the model has the same spontaneous symmetry breaking as one at $`\mu =0`$.
2) When $`\frac{m(0)}{2}<\mu <m(0)`$, $`V_{eff}^{(2)}(T=0,\mu ,m)`$ will have three extreme points: $`m=0`$ (minimum),$`m=m_1`$ (maximum) and $`m=m(0)`$ (minimum), and this is just the characteristic of possible emergence of a first order phase transition.
It is emphasized that in the above analyses we only use the known relation (2.12) between the gap equation and the effective potential and the explicit expression of the gap equation (2.1), and did not use an explicit expression of $`V_{eff}^{(2)}(T,\mu ,m)`$. However, for determining the critical point, we have to find out the explicit expression of $`V_{eff}^{(2)}(T,\mu ,m)`$ from Eq.(2.12). The result is kn:14

$`V_{eff}^{(2)}(T=0,\mu ,m)`$ $`=`$ $`{\displaystyle \frac{m^2}{2\pi }}\left[\mathrm{ln}{\displaystyle \frac{m}{m(0)}}{\displaystyle \frac{1}{2}}\right]+{\displaystyle \frac{\mu ^2}{2\pi }}+{\displaystyle \frac{1}{2\pi }}\theta (\mu m)`$ (18)
$`\times \left(m^2\mathrm{ln}{\displaystyle \frac{\mu +\sqrt{\mu ^2m^2}}{m}}\mu \sqrt{\mu ^2m^2}\right).`$
Assuming that $`\frac{m(0)}{2}<\mu <m(0)`$, then for $`m=m(0)`$ among the three extreme points, we will get

$$V_{eff}^{(2)}[T=0,\mu ,m=m(0)]=\frac{1}{4\pi }[2\mu ^2m^2(0)]<0,$$

$$\mathrm{when}\mu <\frac{1}{\sqrt{2}}m(0).$$
(19)
Hence, if $`\mu <m(0)/\sqrt{2}`$, $`m=m(0)`$ will be the global minimum point of $`V_{eff}^{(2)}(T=0,\mu ,m)`$, because in this case, the minimum $`V_{eff}^{(2)}(T=0,\mu ,m=0)=0`$ and $`V_{eff}^{(2)}(T=0,\mu ,m=m_1)`$ is a maximum. Obviously, the first order phase transition point should be determined by

$$V_{eff}^{(2)}[T=0,\mu ,m=m(0)]=V_{eff}^{(2)}(T=0,\mu ,m=0)=0$$
(20)
and this will lead to

$$\mu =\frac{1}{\sqrt{2}}m(0)\mu _c,$$
(21)
i.e. when $`\mu `$ crosses over $`\mu _c`$, the global minimum point of $`V_{eff}^{(2)}(T=0,\mu ,m)`$ will jump from $`m(0)`$ down to 0 through a barrier tunneling between the minimum points $`m=m(0)`$ and $`m=0`$. The critical chemical potential $`\mu _c`$ of the first order phase transition at $`T=0`$ in $`D=2`$ GN model in the leading order of $`1/N`$ expansion given by Eq. (2.20) is consistent with the one obtained by other authors kn:7 ; kn:8 ; kn:9 ; kn:10 ; kn:11 ; kn:12 . Since the global minimum of $`V_{eff}^{(2)}(T=0,\mu ,m)`$ is also at $`m=m(0)`$ when $`\mu <m(0)/2`$, we can write the critical behavior of the order parameter $`m`$ by

$$m=\{\begin{array}{ccc}m(0)& \mathrm{when}& \mu \frac{1}{\sqrt{2}}m(0)\\ 0& \mathrm{when}& \mu >\frac{1}{\sqrt{2}}m(0)\end{array}.$$
(22)
The above discussions show that for critical analyses of the order parameter in a general first order phase transition, it is not sufficient only using the gap equation, we must use the gap equation combined with corresponding effective potential. This is also applicable to critical analyses of the phase transitions at low temperature $`T0`$ in $`D=2`$ GN model which are known to be first order. The effective potential at $`T0`$ may be derived from Eq.(2.12). By means of it and the gap equation (2.1), we will be able to obtain other results of the phase transition at low temperature in this model including location of the tricritical point etc., following similar methods to ones used in Ref kn:6 ; kn:7 ; kn:11 ; kn:12 .

## III $`D=3`$ Gross-Neveu model

In this case kn:16 , the dynamical fermion mass will spontaneously break the special parities $`𝒫_1`$ and $`𝒫_2`$: $`\psi (t,x^1,x^2)`$ $`\stackrel{𝒫_1}{}\gamma ^1\psi (t,x^1,x^2)`$ and $`\psi (t,x^1,x^2)\stackrel{𝒫_2}{}\gamma ^2\psi (t,x^1,x^2)`$ and the time reversal $`𝒯`$: $`\psi (t,x^1,x^2)\stackrel{𝒯}{}\gamma ^2\psi (t,x^1,x^2)`$222We no longer include the discrete symmetry $`Z_2`$: $`\psi _k^1(x)\stackrel{Z_2}{}\psi _k^2(x),\mathrm{if}\psi _k(x)=(\psi _k^1(x),\psi _k^2(x))`$, as was stated in Ref.kn:16 , since the kinetic energy terms in Eq. (1.1) actually have not separate $`Z_2`$ symmetry.. After the gap equation at $`T=\mu =0`$ is substituted, the gap equation at $`T0`$ becomes

$$m(0)=m+F_2(T,\mu ,m)$$
(23)
with

$$F_2(T,\mu ,m)=T\left\{\mathrm{ln}\left[1+e^{(m\mu )/T}\right]+(\mu \mu )\right\},$$
(24)
and the critical equation of second order phase transition will be

$$m(0)=T_c\left[\mathrm{ln}(1+e^{\mu _c/T_c})+\mathrm{ln}(1+e^{\mu _c/T_c})\right]$$
(25)
which determines critical points of second order phase transitions. For example, for a given finite $`T`$, when $`\mu \mu _c`$, $`m0`$, so we will have $`m/T1`$. Then Eq.(3.1) with (3.2) will lead to

$`m(0)`$ $`=`$ $`T\mathrm{ln}\left(1+e^{\mu /T}\right)+T\mathrm{ln}\left(1+e^{\mu /T}\right)`$ (26)
$`+{\displaystyle \frac{m^2}{2T[1+\mathrm{cosh}(\mu /T)]}}+𝒪\left({\displaystyle \frac{m^3}{T^3}}\right).`$
Substituting Eq.(3.3) with $`T_c`$ replaced by $`T`$ into Eq.(3.4) we get

$$m^2=2T\mathrm{sinh}(\mu /T)(\mu _c\mu ),\mathrm{for}T0.$$
(27)
Eq.(3.5) shows that for a given finite temperature, the symmetry restoration phase transition at $`\mu _c`$ is second order. The same has been shown for the phase transition at $`T_c`$ for a given $`\mu `$ kn:16 . These conclusions are valid for $`T0`$. For $`T=0`$, we must go back the gap equation (3.1). We note that when $`\mu m`$, $`\underset{T0}{lim}F_2(T,\mu ,m)=0`$ and by Eq.(3.1) this implies that $`m=m(0),\mathrm{if}\mu m`$, or furthermore, $`m=m(0),\mathrm{if}\mu m(0)`$. On the other hand, if $`\mu >m`$, then $`\underset{T0}{lim}F_2(T,\mu ,m)=\mu m`$. Substituting this result into Eq.(3.1) we will obtain that $`m(0)=\mu `$, but the presupposition leading to this result in fact should be $`\mu >m=m(0)`$, the both apparently contradict each other. This indicates that when $`\mu >m=m(0)`$, the gap equation (3.1) can not be satisfied, thus there is no longer symmetry breaking and the order parameter $`m`$ must be zero. The above results can be summarized as

$$m=\{\begin{array}{ccc}m(0)& \mathrm{when}& \mu m(0)\\ 0& \mathrm{when}& \mu >m(0)\end{array}.$$
(28)
Furthermore, when $`T0`$, the derivative $`m/\mu `$ will be

$`\underset{T0}{lim}{\displaystyle \frac{m}{\mu }}|_{m=m(0)}`$
$`=\underset{T0}{lim}{\displaystyle \frac{F_2(T,\mu ,m)}{\mu }}/\left[1+{\displaystyle \frac{F_2(T,\mu ,m)}{m}}\right]|_{m=m(0)}`$
$`=\underset{T0}{lim}e^{[m(0)\mu ]/T}=\{\begin{array}{cc}0,& \mathrm{when}\mu <m(0)\\ 1,& \mathrm{when}\mu =m(0)\\ \mathrm{},& \mathrm{when}\mu m(0)\end{array}`$
(noting that when $`\mu >m(0)`$, the gap equation (3.1) is no longer valid). Eqs.(3.6) and (3.7) show that when $`\mu <m(0)`$, $`m`$ does not change as $`\mu `$ varies and keeps its value $`m(0)`$ at $`\mu =0`$; however as soon as $`\mu `$ crosses over $`m(0)`$, $`m/\mu `$ will change from 0 into $`\mathrm{}`$ and $`m`$ will jump discontinuously from $`m(0)`$ to zero. This implies that when $`T=0`$ the phase transition at the critical chemical potential $`\mu _c=m(0)`$ is first order, i.e. $`(T,\mu )=(0,m(0))`$ will be a first order phase transition point in $`T\mu `$ phase diagram. On the other hand, we note that $`(T,\mu )=(0,m(0))`$ is a solution of the critical equation (3.3) corresponding to second order phase transition when we take the limit $`T0`$ in Eq.(3.3). Hence we may reasonably conclude that $`(T,\mu )=(0,m(0))`$ will be a tricritical point in the $`T\mu `$ phase giagram of $`D=3`$ GN model.
This conclusion coincides with the one in Ref. kn:13  but different from the one in Ref. kn:12  where the authors claimed that there is no first order phase transition in $`D=3`$ GN model. To clarify this problem, we can make further analysis of the phase transition at $`T=0`$ in the model by means of the effective potential which may be derived from the gap equation, as was made in $`D=2`$ GN model. It has been proven that the extreme value condition of the effective potential $`V_{eff}^{(3)}(T,\mu ,m)`$ is kn:14

$$\frac{V_{eff}^{(3)}(T,\mu ,m)}{m}=0$$
(30)
with

$$\frac{V_{eff}^{(3)}(T,\mu ,m)}{m}=\frac{m}{2\pi }\left[mm(0)+F_2(T,\mu ,m)\right],$$
(31)
noting that Eq.(3.8) is just the gap equation (3.1) multiplied by $`m`$. From Eq. (3.9) we may obtain $`V_{eff}^{(3)}(T,\mu ,m)`$ satisfying the condition $`V_{eff}^{(3)}(T,\mu ,m=0)=0`$ expressed by

$$V_{eff}^{(3)}(T=0,\mu ,m)=\frac{1}{2\pi }\left\{\frac{m^2}{2}[\mu \theta (\mu m)m(0)]+\theta (m\mu )\left(\frac{m^3}{3}+\frac{\mu ^3}{6}\right)\right\}.$$
(32)
For fixed $`\mu `$, from Eq.(3.10) or the $`T0`$ limit of Eq.(3.9),we find out

$$\frac{V_{eff}^{(3)}(T=0,\mu ,m)}{m}=\frac{m}{2\pi }\{[\mu m(0)]\theta (\mu m)+[mm(0)]\theta (m\mu )\}$$
(33)
and

$$\frac{^2V_{eff}^{(3)}(T=0,\mu ,m)}{m^2}=\frac{1}{2\pi }\{[\mu m(0)]\theta (\mu m)+[2mm(0)]\theta (m\mu )\}.$$
(34)
Variation of $`V_{eff}^{(3)}(T=0,\mu ,m)`$ as $`\mu `$ increases may be discussed by Eqs. (3.10)-(3.12).
1) $`\mu <m(0)`$. In this case, $`V_{eff}^{(3)}[T=0,\mu <m(0),m]`$ will have the maximum point $`m=0(m<\mu )`$ and the minimum point $`m=m(0)(m>\mu )`$. It is indicated that location of the minimum point keeps at $`m=m(0)`$ which does not change as $`\mu `$ increases. However, the minimum of the effective potential at $`m=m(0)`$

$$V_{eff}^{(3)}[T=0,\mu <m(0),m]|_{m=m(0)}=\frac{1}{12\pi }[\mu ^3m^3(0)]$$
(35)
will rise as $`\mu `$ increases. Nevertheless, these results only show the same spontaneous symmetry breaking as one at $`T=\mu =0`$.
2) $`\mu =m(0)`$. In this case, the effective potential will take the following form

$$V_{eff}^{(3)}[T=0,\mu =m(0),m]=\frac{1}{2\pi }\theta [mm(0)]$$

$$\times \left[\frac{m^3(0)}{6}\frac{m^2}{2}m(0)+\frac{m^3}{3}\right]$$
(36)
which indicates that in the total real axis segment $`0mm(0)`$, $`V_{eff}^{(3)}[T=0,\mu =m(0),m]=0`$, thus the total segment $`0mm(0)`$ including the original $`m=m(0)`$ when $`\mu <m(0)`$, will be the minimum points of $`V_{eff}^{(3)}[T=0,\mu =m(0),m]`$. This situation corresponds to that when $`\mu =m(0)`$, the $`T0`$ limit of the gap equation (3.1) has infinite non-zero solutions.
3) $`\mu >m(0)`$. In this case, $`V_{eff}^{(3)}[T=0,\mu >m(0),m]/m=0`$ will have the only solution $`m=0`$ and by Eq. (3.12),

$$\frac{^2V_{eff}^{(3)}[T=0,\mu >m(0),m]}{m^2}|_{m=0}=\frac{1}{2\pi }[\mu m(0)]>0.$$
(37)
Thus $`m=0`$ will be the only minimum point of $`V_{eff}^{(3)}[T=0,\mu >m(0),m]`$ and this implies that the broken symmetries will be restored. The critical chemical potential $`\mu _c`$ should be determined by the condition

$$\frac{^2V_{eff}^{(3)}(T=0,\mu ,m)}{m^2}|_{m=0}=0,$$
(38)
i.e. at $`\mu _c`$, $`m=0`$ must change from being a maximum point into being a minimum point. The result is that $`\mu _c=m(0)`$.
The above discussions show that as $`\mu `$ increases and finally crosses over $`\mu _c=m(0)`$, the global minimum point of the effective potential $`V_{eff}^{(3)}(T=0,\mu ,m)`$ will jump from $`m(0)`$ down to 0 thus Eq. (3.6) is reproduced. The phase transition is first order indeed. The analysis based on effective potential leads to the same conclusion as the one only based on the gap equation, however the latter is obviously more simple and direct in this model. We also indicate that the first order phase transition at $`T=0`$ in $`D=3`$ GN model has a distinct feature, i.e. the transit from broken phase to symmetry phase does not undergo a barrier tunneling, since when $`\mu `$ is increasing, as we have seen above, no barrier emerges from between the minimum points $`m=0`$ and $`m=m(0)`$ of $`V_{eff}^{(3)}(T=0,\mu ,m)`$. This is apparently different from general first order phase transition, e.g. the one in $`D=2`$ GN model discussed in Sect. II. It is assumed that it is just this feature that makes one be able to determine the critical behavior of the order parameter in the first order phase transition in $`D=3`$ GN model merely by the gap equation approach.

## IV Conclusions

In this paper, we have discussed symmetry restoring phase transitions at high density in $`D=2`$ and $`D=3`$ GN model by means of analyses of critical behaviors of the dynamical fermion mass as order parameter of symmetry breaking. We have used both gap equation and effective potential approach. In these approaches, the gap equations multiplied by the dynamical fermion mass are simply the Schwinger-Dyson equations obeyed by the dynamical fermion mass and the effective potentials will be derived from the gap equations based on the fact that a gap equation multiplied by the dynamical fermion mass can also come from the extreme value condition of corresponding effective potential. This shows that the Schwinger-Dyson equations obeyed by the dynamical fermion mass in fact contain the essential elements of the phase structure of the discussed models. We have found that for second order phase transitions and some specific first order phase transitions e.g. the one at high density and $`T=0`$ in $`D=3`$ GN model in which the jumping of the order parameter does not correspond to a barrier tunneling in an effective potential, the gap equation analysis alone has the same effect as the effective potential one. However, for general first order phase transitions, e.g. the one at high density and $`T=0`$ and low $`T0`$ in $`D=2`$ GN model, we must combine the gap equation with the corresponding effective potential so as to obtain correct conclusions.
Our analyses also reproduce some of known essential results of the phase transitions in $`D=2`$ and $`D=3`$ GN model. They include that in $`D=2`$ GN model, the phase transitions at high density will be second order when $`T`$ is high and first order when $`T0`$; in $`D=3`$ GN model, the phase transition at high density will be second order when $`T`$ is finite and first order when $`T=0`$ thus one can conclude that $`(T,\mu )=(0,m(0))`$ is a tricritical point. The latter further clarifies the theoretical divergence of that if first order phase transition exists in $`D=3`$ GN model.