text:
# PROGRESS IN CORRELATION FEMTOSCOPY11footnote 1 Work supported by grant 202/01/0779 of the Grant Agency of the Czech Republic.

## 1 Introduction

The momentum correlations of particles at small relative velocities are widely used to study space-time characteristics of the production processes, so serving as a correlation femtoscope. Particularly, for non-interacting identical particles, like photons or, to some extent, pions, these correlations result from the interference of the two–particle amplitudes due to the symmetrization requirement of quantum statistics (QS).222 There exists a deep analogy of the momentum QS correlations of photons with the space–time correlations of the intensities of classical electromagnetic fields used in astronomy to measure the angular radii of stellar objects based on the superposition principle (HBT effect). This analogy is sometimes misunderstood and the momentum correlations are mixed up with the HBT correlations in spite of their orthogonal character and the absence of the classical analogy for correlations of identical fermions.  The momentum QS correlations were first observed as an enhanced production of the pairs of identical pions with small opening angles (GGLP effect). Later on, Kopylov and Podgoretsky settled the basics of correlation femtoscopy; particularly, they suggested to study the interference effect in terms of the correlation function and clarified the role of the space–time characteristics of particle production in various physical situations.

The momentum correlations of particles emitted at nuclear distances are also influenced by the effect of final state interaction (FSI). Though the FSI effect complicates the correlation analysis, it is an important source of information allowing for the coalescence femtoscopy (see, e.g., and the talk by G. Melkumov), the correlation femtoscopy with unlike particles including the access to the relative space–time asymmetries in particle production and a study of particle strong interaction.

The two-particle correlation function $`(p_1,p_2)`$ is usually defined as a ratio of the measured two-particle distribution to the reference one obtained by mixing the particles from different events. It can be calculated- as a square of the properly symmetrized stationary solution $`\psi _𝐤^{}^{S(+)}(𝐫^{})`$ of the scattering problem averaged over the relative distance $`𝐫^{}`$ of the emitters in the pair c.m.s. ($`𝐤^{}=𝐩_1^{}=𝐩_2^{}𝐐/2`$) and over the pair total spin $`S`$.

It is well known that the directional and velocity dependence of the correlation function can be used to determine both the duration of the emission process and the form of the emission region, as well as - to reveal the details of the production dynamics (such as collective flows; see, e.g.,). The recent (puzzling) results on like pion correlations from BNL RHIC have been presented at this conference by M. Lisa and V. Okorokov.

## 2 Femtometry with unlike particles

The complicated dynamics of particle production, including resonance decays and particle rescatterings, leads to essentially non–Gaussian tail of the $`r^{}`$–distribution. Therefore, due to different $`r^{}`$–sensitivity of the QS, strong and Coulomb FSI effects, one has to be careful when analyzing the correlation functions in terms of simple models. Thus, the QS and strong FSI effects are influenced by the $`r^{}`$–tail mainly through the suppression parameter $`\lambda `$ while, the Coulomb FSI is sensitive to the distances as large as the pair Bohr radius $`|a|`$ (hundreds fm for the pairs containing pions). These problems can be at least partially overcome with the help of transport code simulations accounting for the dynamical evolution of the emission process and providing the phase space information required to calculate the QS and FSI effects on the correlation function.

Thus, in a preliminary analysis of the NA49 correlation data from central $`Pb+Pb`$ 158 AGeV collisions, the transport RQMD v.2.3 code was used. To account for a possible mismatch in $`r^{}`$, the correlation functions were calculated with the space–time coordinates of the emission points scaled by 0.7, 0.8 and 1. The scale parameter was then fitted using the quadratic interpolation. The fits of the $`\pi ^+\pi ^{}`$, $`\pi ^+p`$ and $`\pi ^{}p`$ correlation function indicate that RQMD overestimates the distances $`r^{}`$ by 10-20$`\%`$.

Recently, there appeared data on $`p\mathrm{\Lambda }`$ correlation functions from $`Au+Au`$ experiment E985 at AGS. As the Coulomb FSI is absent in this system, one avoids here the problem of its sensitivity to the $`r^{}`$–tail. Also, the absence of the Coulomb suppression of small relative momenta makes this system more sensitive to the radius parameters as compared with $`pp`$ correlations. In spite of rather large statistical errors, a significant enhancement is seen at low relative momentum, consistent with the known singlet and triplet $`p\mathrm{\Lambda }`$ s–wave scattering lengths. In fact, using the analytical expression for the correlation function (originally derived for $`pn`$ system), one gets a good fit of the combined (4, 6 and 8 AGeV) correlation function with the Gaussian radius $`r_0=4.5\pm 0.7`$ fm, in agreement with the radii of 3-4 fm obtained from $`pp`$ correlations in heavy ion collisions at GSI, AGS and SPS energies.

## 3 Accessing particle strong interaction

In case of a poor knowledge of the two–particle strong interaction, which is the case for exotic systems like ($`M=`$ meson) $`MM`$, $`M\mathrm{\Lambda }`$ or $`\mathrm{\Lambda }\mathrm{\Lambda }`$, the correlation measurements can be also used to study the latter.

In heavy ion collisions, the effective radius $`r_0`$ of the emission region can be considered much larger than the range of the strong interaction potential. The FSI contribution to the correlation function is then independent of the actual potential form. At small $`Q=2k^{}`$, it is determined by the s-wave scattering amplitudes $`f^S(k^{})`$. In case of $`|f^S|>r_0`$, this contribution is of the order of $`|f^S/r_0|^2`$ and dominates over the effect of QS. In the opposite case, the sensitivity of the correlation function to the scattering amplitude is determined by the linear term $`f^S/r_0`$.

The possibility of the correlation measurement of the scattering amplitudes has been demonstrated in a recent analysis of the NA49 $`\pi ^+\pi ^{}`$ correlation data within the RQMD model. The fitted strong interaction scale, redefining the original scattering length $`f_0=`$ 0.232 fm, appeared to be significantly lower than unity: $`0.63\pm 0.08`$. To a similar shift ($`20\%`$) point also the recent BNL data on $`K_{l4}`$ decays. These results are in agreement with the two–loop calculation in the chiral perturbation theory with a standard value of the quark condensate.

As for the $`\mathrm{\Lambda }\mathrm{\Lambda }`$ system, the singlet $`\mathrm{\Lambda }\mathrm{\Lambda }`$ s–wave scattering length $`f_0`$ has been recently estimated based on the NA49 data on $`\mathrm{\Lambda }\mathrm{\Lambda }`$ correlations in $`Pb+Pb`$ collisions at 158 AGeV. Using the analytical expression for the correlation function and fixing the purity of direct $`\mathrm{\Lambda }`$–pairs at the estimated value of 0.16 and varying the effective radius $`r_0`$ in the acceptable range of several fm, one gets e.g., $`f_0=2.4\pm 2.1`$ and $`3.2\pm 5.7`$ fm for $`r_0=2`$ and 4 fm respectively (we use the same sign convention as for meson–meson and meson–baryon systems). Though the fit results are not very restrictive, they likely exclude the possibility of a large positive singlet scattering length comparable to that of $``$20 fm for the two–nucleon system.

The important information is also coming from $`\mathrm{\Lambda }\mathrm{\Lambda }`$ correlations at LEP. Here the effective radius $`r_0`$ is substantially smaller than the range of the strong interaction potential, so the $`\mathrm{\Lambda }\mathrm{\Lambda }`$ correlation function is sensitive to the potential form. In fact, the observed strong decrease of the correlation function at small $`Q`$ can be considered as a direct evidence for the potential core; particularly, the Nijmegen potential NSC97e yields a reasonable agreement with this data.

## 4 Accessing relative space-time asymmetries

The correlation function of two non–identical particles, compared with the identical ones, contains a principally new piece of information on the relative space-time asymmetries in particle emission. Since this information enters in the two-particle amplitude $`\psi _𝐤^{}^{S(+)}(𝐫^{})`$ through the terms odd in $`𝐤^{}𝐫^{}𝐩_1^{}(𝐫_1^{}𝐫_2^{})`$, it can be accessed studying the correlation functions $`_{+i}`$ and $`_i`$ with positive and negative projection $`k_i^{}`$ on a given direction $`𝐢`$ or, - the ratio $`_{+i}/_i`$. For example, $`𝐢`$ can be the direction of the pair velocity or, any of the out (x), side (y), longitudinal (z) directions. Note that in the longitudinally comoving system (LCMS), one has $`r_i^{}=r_i`$ except for $`r_x^{}\mathrm{\Delta }x^{}=\gamma _t(\mathrm{\Delta }xv_t\mathrm{\Delta }t)`$, where $`\gamma _t`$ and $`v_t`$ are the pair LCMS Lorentz factor and velocity. One may see that the asymmetry in the out (x) direction depends on both space and time asymmetries $`\mathrm{\Delta }x`$ and $`\mathrm{\Delta }t`$. In case of a dominant Coulomb FSI, the intercept of the correlation function ratio is directly related with the asymmetry $`r_i^{}`$ scaled by the Bohr radius $`a=(\mu z_1z_2e^2)^1`$: $`_{+i}/_i1+2r_i^{}/a.`$

A review of the simulation studies of the method sensitivity and the experimental results can be found elsewhere. Here we discuss the out correlation asymmetries observed for $`\pi p`$ and $`\pi K`$ systems in heavy ion collisions at CERN SPS and BNL RHIC. These asymmetries are in agreement with practically charge independent meson production and a negative $`\mathrm{\Delta }x`$ or positive $`c\mathrm{\Delta }t`$ on the level of several fm (assuming $`m_1<m_2`$). The RHIC asymmetries seem to be overestimated by the RQMD model while the NA49 $`\pi ^+\pi ^{}`$ and $`\pi p`$ asymmetries in central $`Pb+Pb`$ collisions at 158 AGeV are in quantitative agreement with this model - it yields practically zero asymmetries for $`\pi ^+\pi ^{}`$ system while, for $`\pi ^\pm p`$ systems, $`\mathrm{\Delta }x5.2`$ fm, $`\mathrm{\Delta }t2.9`$ fm/c, $`\mathrm{\Delta }x^{}8.5fm`$. Besides, it predicts $`x`$ increasing with particle $`p_t`$ or $`u_t=p_t/m`$, starting from zero due to kinematic reasons. The asymmetry arises because of a faster increase with $`u_t`$ for heavier particle. In fact, the hierarchy $`x_\pi <x_K<x_p`$ is a signal of a universal transversal collective flow; one should simply take into account that the mean thermal velocity is smaller for heavier particle and thus washes out the positive shift due to the flow to a lesser extent.

## 5 Spin correlations

The information on the system size and the two–particle interaction can be achieved also with the help of spin correlation measurements using as a spin analyzer the asymmetric (weak) particle decay.- Since this technique requires no construction of the uncorrelated reference sample, it can serve as an important consistency check of the standard correlation measurements. Particularly, for two $`\mathrm{\Lambda }`$–particles decaying into the $`p\pi ^{}`$ channel, the distribution of the cosine of the relative angle $`\theta `$ between the directions of the decay protons in the respective $`\mathrm{\Lambda }`$ rest frames allows one to determine the triplet fraction $`\rho _t=_t/`$, where $`_t`$ is the triplet part of the correlation function.

The spin correlations allow also for a relatively simple test of the quantum–mechanical coherence based on Bell–type inequalities derived from the assumption of the factorizability of the two–particle density matrix, i.e. its reduction to a sum of the direct products of one–particle density matrices with the nonnegative coefficients. Clearly, such a form of the density matrix corresponds to a classical probabilistic description and cannot account for the coherent quantum–mechanical effects, particularly, for the production of two $`\mathrm{\Lambda }`$-particles in a singlet state. Thus the suppression of the triplet $`\mathrm{\Lambda }\mathrm{\Lambda }`$ fraction observed in multihadronic $`Z^0`$ decays at LEP indicates a violation of one of the Bell-type inequalities, $`\rho _t1/2`$.

## 6 Conclusions

The particle momentum and, recently, also spin correlations give unique information on the space–time production characteristics including collective flows. Rather direct evidence for a strong transverse flow in heavy ion collisions at SPS and RHIC is coming from unlike particle correlation asymmetries. Being sensitive to relative time delays and collective flows, the correlation asymmetries can be especially useful to study the effects of the quark–gluon plasma phase transition. The correlations yield also a valuable information on the particle strong interaction hardly accessible by other means.


format: text

text:
# The Las Campanas Infrared Survey. IV. The Photometric Redshift Survey and the Rest-frame 𝑅-band Galaxy Luminosity Function at 0.5≤𝑧≤1.5

## 1 Introduction

The galaxy luminosity function represents the product of a series of physical processes that govern galaxy evolution, including star formation and the recycling of metals through various heating and cooling processes. An accurate estimate of the galaxy luminosity function as a function of time therefore bears significantly on our understanding of how structures in the universe form and evolve. Independent measurements of the optical galaxy luminosity function based on various deep redshift surveys between $`z0`$ and $`z0.75`$ have yielded consistent results (Lilly et al. 1995; Ellis et al. 1996; Marzke & da Costa 1997; Bromley et al. 1998; Marzke et al. 1998; Lin et al. 1999; Madgwick et al. 2001). For example, the galaxy luminosity function is well represented by a Schechter function (Schechter 1976) with a steeper faint-end slope for galaxies of bluer optical colors at all epochs that have been studied. Comparisons of these measurements at different redshifts further show that these blue galaxies exhibit a stronger evolution than red galaxies both in their space density and in their rest-frame optical luminosity.

It is difficult, however, to apply these results to fully constrain galaxy formation models. While galaxy luminosities measured in the optical are sensitive to the transient star formation rate and therefore are a good tracer of the instantaneous star formation activity, they are not representative of the total stellar mass that has been assembled in the galaxies over time. Additional caveats exist in all studies that attempt to draw conclusions based on comparisons of luminosity function measurements obtained using different galaxy samples. For example, comparison of morphologically-selected and color-selected galaxy samples is inadequate, because even morphologically distinct galaxies can exhibit similar colors. Furthermore, it is impossible to unambiguously identify the same galaxy population at different epochs using the observed colors because the underlying stellar evolution sequence is unknown. Finally, other selection biases such as the cosmic surface brightness dimming effect (preventing accurate morphological classifications at high redshift), bandpass selection effect (galaxies selected in the same observed-frame bandpass may have different rest-frame properties), and the presence of dust (in which a young, star-forming galaxy masquerades as an old, quiescent one) further complicate the interpretations of comparisons of different galaxy surveys.

We have initiated the Las Campanas Infrared (LCIR) survey (Marzke et al. 1999; McCarthy et al. 2001; Chen et al. 2002; Firth et al. 2002) to study the history of mass assembly based on a uniform sample of near-infrared selected galaxies. The rest-frame near-infrared light is a good proxy for the total stellar mass, independent of age or galaxy type (Charlot 1996). Measurements of the galaxy luminosity function at rest-frame near-infrared wavelengths at different epochs may, therefore, be adopted to estimate the stellar mass density evolution. A robust measurement of the rest-frame near-infrared luminosity function at $`z1`$ requires a galaxy survey conducted in the $`K`$ band. We have so far completed the LCIR $`H`$-band survey and a complete $`K`$-band survey is underway. Here we present measurements of the rest-frame $`R`$-band luminosity functions in three redshift intervals over the range $`z=0.5`$–1.5 for the $`H`$-band selected galaxies using photometric redshifts. At $`z1.2`$, the observed-frame $`H`$-band approximately corresponds to rest-frame $`R`$-band, and therefore the calculation does not depend sensitively on the templates adopted to determine the $`k`$-correction. The primary objectives of the analysis are (1) to obtain measurements of the galaxy luminosity function at $`z1`$, where different galaxy formation scenarios begin to show distinct predictions in the space densities and masses of evolved galaxies, and (2) to study galaxy evolution at redshifts between $`z0.5`$ and $`z1.5`$ based on a uniform sample of near-infrared selected galaxies, thus extending the redshift range of existing deep redshift surveys to beyond $`z=1`$.

We first conducted a photometric redshift survey in the LCIRS galaxy sample to identify a complete sample of galaxies at $`z>0.5`$. The photometric redshift survey in two of the survey fields identified $`3000`$ galaxies at $`z0.5`$, yielding a statistically significant sample of intermediate-redshift galaxies for understanding the galaxy population at redshifts beyond $`z=1`$. The sample size is comparable to those of wide-field spectroscopic surveys carried out at $`z0.5`$ such as the Canadian Network for Observational Cosmology Field Galaxy Redshift Survey (CNOC2; e.g. Lin et al. 1999). The photometric redshift techniques determine galaxy redshifts based on the presence/absence of broad-band spectral discontinuities, rather than narrow-band emission or absorption line features. They are therefore of particular value for the identification of galaxies at $`z0.75`$. Complete spectroscopic surveys of faint galaxies in this redshift range are extremely challenging, as prominent spectral line features in galaxy spectra are redshifted to near-infrared wavelengths and are difficult to detect due to bright, contaminating sky lines and detector limitations. It has been demonstrated over the past several years that distant galaxies may be robustly identified using photometric redshift techniques that incorporate optical and near-infrared broad-band photometry (Connolly et al. 1997; Lanzetta, Fernández-Soto, & Yahil 1998; Fernández-Soto, Lanzetta, & Yahil 1999; Martini 2001; Fernández-Soto et al. 2001; Rudnick et al. 2001). In this paper, we show that the photometric redshift measurements of the galaxies identified in the LCIR survey are accurate, with an RMS dispersion between the photometric and spectroscopic redshifts of $`\sigma _z/(1+z)0.08`$.

Next, we developed a maximum likelihood method to measure galaxy luminosity function using photometric redshifts, which explicitly accounts for photometric redshift uncertainties of individual galaxies. Photometric redshift uncertainties together with a steep slope of the galaxy luminosity function would flatten the observed luminosity function and yield systematic biases. Uncertainties in photometric redshifts must therefore be accounted for in the luminosity function analysis. We demonstrate that systematic uncertainties in the derived galaxy luminosity function are significantly reduced when redshift error functions are taken into account. We show that (1) the galaxy luminosity function is consistent with a Schechter (1976) form; (2) the evolution of the co-moving luminosity density of the $`H`$-band selected galaxy is characterized by $`\mathrm{\Delta }\mathrm{log}\mathrm{}/\mathrm{\Delta }\mathrm{log}(1+z)=0.6\pm 0.1`$ at rest-frame 6800 Å; (3) galaxies with spectral energy distributions best characterized as E/S0 or Sab appear to have faded by $`0.5`$ mag from $`z1`$ to $`z0.5`$, consistent with the expected luminosity evolution from an exponentially declining star formation rate model; and (4) only moderate evolution in the space density and in the rest-frame, co-moving $`R`$-band luminosity density is found for the bright, color-selected early-type galaxies.

We adopt the following cosmology: $`\mathrm{\Omega }_\mathrm{M}=0.3`$ and $`\mathrm{\Omega }_\mathrm{\Lambda }=0.7`$ with a dimensionless Hubble constant $`h=H_0/(100\mathrm{km}\mathrm{s}^1\mathrm{Mpc}^1)`$ throughout the paper.

## 2 The Las Campanas Infrared Survey

The LCIR survey is a deep, wide-field near-infrared and optical imaging survey designed to identify a large number of galaxies dominated by an old stellar population at $`z1`$, while securing a uniform sample of galaxies of all types to $`z2`$ based on their broad-band optical and near-infrared colors. We have completed the $`H`$-band survey, covering one square degree of sky to a mean $`5\sigma `$ detection limit in a four arcsecond diameter aperture of $`H20.8`$. Initial catalogs of $``$ 54,000 galaxies identified over 1400 $`\mathrm{arcmin}^2`$ together with results from a comprehensive study of the survey incompleteness in each field are presented in Chen et al. (2002). The analysis presented in this paper is based on galaxies identified in two of the LCIR fields: the Hubble Deep Field South (HDFS) and Chandra Deep Field South (CDFS). To summarize, the galaxy sample contains $``$ 6,700 $`H`$-band selected galaxies over 847 $`\mathrm{arcmin}^2`$ in the HDFS region with complementary optical $`U`$, $`B`$, $`V`$, $`R`$, and $`I`$ colors, and $`7,400`$ $`H`$-band selected galaxies over 561 $`\mathrm{arcmin}^2`$ in the CDFS region with complementary optical $`V`$, $`R`$, $`I`$, and $`z^{}`$ colors.

To study the statistical properties of galaxies at $`z1`$ using the LCIR survey sample, we need to first determine redshifts for the galaxies. The photometric redshift techniques that determine galaxy redshifts based on their broad-band spectral energy distribution (SED) offer an efficient means to obtain robust redshift measurements together with well-understood measurement uncertainties for all galaxies in the survey.

### 2.1 Photometric Redshift Likelihood Analysis

We adopted the photometric redshift technique originally developed by Lanzetta and collaborators (e.g. Lanzetta, Yahil, & Fernández-Soto 1996), and modified the program for its application to ground-based imaging data. The technique determines galaxy redshifts by comparing the SED of a galaxy, which is established based on a compilation of broad-band photometric measurements, with a grid of spectrophotometric templates evaluated at different redshifts. It employs a maximum likelihood method, in which the likelihood of a galaxy matching a given spectral type $`T`$ at redshift $`z`$ based on $`n`$ photometric measurements is written as

$$_z(z,T)=\underset{i=1}{\overset{n}{}}\mathrm{exp}\left\{\frac{1}{2}\left[\frac{f_iAF_i(z,T)}{\sigma _i}\right]^2\right\},$$
(1)
where $`f_i`$ and $`F_i`$ are respectively the measured and model galaxy fluxes in bandpass $`i`$, $`A`$ is the normalization of the template, and $`\sigma _i`$ is the measurement uncertainty of $`f_i`$. In cases where the galaxy is not observed in $`m`$ of the $`n`$ bandpasses to the detection limits of the images, equation (1) is modified as

$$_z(z,T)=\left(\underset{i=1}{\overset{nm}{}}\mathrm{exp}\left\{\frac{1}{2}\left[\frac{f_iAF_i(z,T)}{\sigma _i}\right]^2\right\}\right)\left(\underset{i=1}{\overset{m}{}}_{f_{\mathrm{min}}}^{f_{\mathrm{max}}}𝑑f^{}\mathrm{exp}\left\{\frac{1}{2}\left[\frac{f^{}AF_i(z,T)}{\sigma _i}\right]^2\right\}\right),$$
(2)
where $`f_{\mathrm{min}}`$ and $`f_{\mathrm{max}}`$ are the minimum and maximum possible observed fluxes of the galaxy. We have chosen $`f_{\mathrm{min}}=0`$ and $`f_{\mathrm{max}}=2\sigma _i`$. The likelihood function $`_z(z,T)`$ is maximized with respect to the type $`T`$ at different redshifts to form $`_z(z)`$, which is then maximized with respect to the redshift $`z`$ to determine a best-fit photometric redshift $`z_{\mathrm{phot}}`$ for the galaxy.

### 2.2 Spectral Energy Distribution Templates

We included in our photometric redshift likelihood analysis six galaxy templates, a suite of stellar templates, and a QSO template in order to effectively identify the large number of stars and potential QSO/AGN candidates that appear in wide-field surveys. Every template covers a spectral range from $`\lambda =300`$ to 25,000 Å. The galaxy templates were adopted from Fernández-Soto et al. (1999) and Yahata et al. (2000) and spanned a range of spectral types—from elliptical or S0 (E/S0), Sab, Scd, irregular (Irr), to starburst (SB). A total of 150 stellar templates were compiled from the literature, spanning a wide range of stellar types—from early-type O and B stars to late-type L and T dwarfs (Oppenheimer et al. 1998; Pickles 1998; Fan et al. 2000; Leggett et al. 2000). The QSO template was formed by first adopting the composite QSO spectrum presented in Francis et al. (1991) and extrapolating the spectrum to infrared wavelengths using a simple power law. Because the Francis spectrum was derived based in part on a large number of QSOs at $`z2`$ and was not corrected for intervening absorption at wavelengths shortward of the redshifted $`\mathrm{Ly}\alpha `$ emission line, we replace the continua at $`\lambda 1450`$ Å with the composite spectrum presented in Zheng et al. (1997) based on $`100`$ spectra of QSOs at $`z1.6`$ obtained with the Faint Object Spectrograph aboard the Hubble Space Telescope. Our approach to separate stars from galaxies is described in Chen et al. (2002). The number-magnitude diagram of the stars identified in the HDFS and CDFS fields, in comparison to that of the galaxies, is presented in Chen et al. as well.

### 2.3 The Photometric Redshift Survey

We performed the redshift likelihood analysis and determined photometric redshifts for all galaxies identified in the LCIRsurvey using the available near-infrared and optical broad-band photometry. As discussed in § 2.4, uncertainties in galaxy photometry contribute to the uncertainties in photometric redshifts. Because galaxies vary in size, fixed-aperture photometry would include excess noise from sky for smaller objects and exclude flux from the outskirts of bigger objects, both of which would yield a reduced signal-to-noise (S/N) ratio in the photometric measurements. Fixed-aperture photometry is, therefore, not optimal for establishing the observed SEDs of individual galaxies to be compared with model templates in the photometric redshift analysis.

It is clear that to improve the accuracy of photometric redshifts, we need to first achieve an optimal S/N ratio for the photometric measurements of individual galaxies in all bandpasses, where by optimal measurements we mean those that do not include excess noise or exclude object fluxes. To reach the goal, we obtained a new set of photometric measurements using a varying-size aperture for different objects, which is different from the fix-aperture photometry published in Chen et al. (2000). The individual object apertures were adopted from the object segmentation determined by applying SExtractor (Bertin & Arnout 1996) to the “white light” image of each field. As described in Chen et al. (2002), the white light image is an unweighted sum of all the registered optical images that had been scaled to unit exposure time and filter throughput. Because of the improved signal, particularly in the low surface brightness regions, this image is more sensitive than any individual optical images for SExtractor to identify the true extent of each object at a given surface brightness detection threshold. We measured, for each galaxy, the total fluxes and associated uncertainties in different bandpasses using the same aperture determined in the white light image by SExtractor. This procedure allowed us to recover the total fluxes of each object in different bandpasses, without significantly compromising the precision of the photometric measurements.

Note that the main objective of the analysis here is to obtain accurate and precise broad-band colors. Accurate measurements of the total fluxes of individual galaxies are necessary for a robust estimate of the galaxy luminosity function, but not for the purpose of obtaining an accurate photometric redshift. We adopted the optimal photometric measurements for the photometric redshift analysis and the 4”-aperture photometric measurements for the luminosity function analysis. We present in Figure 1 the median residuals and the corresponding $`1\sigma `$ scatters between the “optimal” and 4”-aperture photometric measurements versus magnitude for the $`H`$-band selected objects in the CDFS region in the $`V`$, $`R`$, $`I`$, and $`H`$ bands. It was shown in Chen et al. (2002) that the 4”-aperture photometric measurements closely resembled the total magnitudes of individual galaxies—only $`<0.1`$ mag aperture correction was necessary. Figure 1 shows that the optimal photometry is consistent with the fixed aperture photometry in all bandpasses to within measurement uncertainties, although it systematically underestimates the total fluxes at faint limits. We note that the systematic biases of the optimal photometry at faint magnitudes do not affect our photometric redshift estimates, because they are almost entirely independent of the adopted bandpasses.

The products of the photometric redshift analysis for each galaxy include: the photometric redshift measurement, the redshift likelihood function, and the best-fit spectral template. Figure 2 shows examples of the results in pairs of panels for four HDFS galaxies that have been spectroscopically identified at different redshifts. The top panel of each pair shows the photometric measurements and uncertainties as solid points with error bars. The solid curves represent the best-fit spectral templates with the spectral type indicated in the upper-left corner. The open squares indicate the predicted fluxes in individual bandpasses, estimated by integrating the best-fit template over the corresponding system transmission functions. The dashed lines indicate the zero flux level. The bottom panel of each pair shows the redshift likelihood functions. The estimated photometric redshift ($`z_{\mathrm{phot}}`$) and spectroscopic redshift ($`z_{\mathrm{spec}}`$) are indicated in the lower-right corner of each of the top panels. In all four cases, we have successfully determined the galaxy redshifts to within the estimated uncertainties.

We studied the redshift distribution of the $`H`$-band selected sample. The left panel of Figure 3 shows the redshift distribution of the $`H`$-band selected HDFS galaxies with $`H20`$. The photometric redshifts were obtained based on the available $`U`$, $`B`$, $`V`$, $`R`$, $`I`$, and $`H`$ photometric measurements. The hatched region indicates the redshift distribution of galaxies with $`IH3`$. The $`IH3`$ color criterion is predicted by various evolutionary scenarios to identify evolved galaxies at $`z1`$ (Chen et al. 2002). The curves indicate the predicted redshift distributions of the total $`H`$-selected sample and the red population, which were calculated using the model that best fit the number–magnitude relation presented in Chen et al. (2002). Specifically, we calculated the predicted redshift distribution of the total $`H`$-band selected galaxies by adopting the $`K`$-band luminosity function obtained by Gardner et al. (1997), assuming a rest-frame color of $`HK0.2`$ (e.g. Mobasher et al. 1986) and a no-evolution scenario. The predicted redshift distribution of the red galaxies was estimated based on the observed population fraction of elliptical galaxies in the local universe (e.g. Binggeli, Sandage, & Tammann 1988). We first adopted the same luminosity function and scaled it according to $`M_{}(\mathrm{red})=M_{}1.2`$ and $`\varphi _{}(\mathrm{red})=0.2\varphi _{}`$ for identifying the red population. Next, we assumed an exponentially declining star formation rate history with a 1-Gyr e-folding time for galaxies formed at $`z_f=30`$ to model the pure luminosity evolution and added random photometric noise of $`0.25`$ mag in the model to account for the uncertainty of the $`IH`$ color selection criterion in the data. Finally, we calculated the integrated galaxy number counts versus redshift. Figure 3 shows that these models agree well with the observation.

In the right panel of Figure 3, we present the cumulative redshift distributions of the $`H20`$ galaxies in the HDFS (open) and CDFS (hatched histogram) regions. We exclude the CDFS galaxies identified at $`z0.75`$ from our analysis as they lack $`U`$\- and $`B`$-band data, which causes photometric redshifts to be less accurate in this range. We find that ($`7.3\pm 0.2`$) % (HDFS) and ($`16.7\pm 0.4`$) % (CDFS) of the $`H`$-band selected galaxies with $`H20`$ are at $`z1`$. The errors were estimated using a Monte Carlo simulation, which takes into account photometric uncertainties and photometric redshift uncertainties, the subject of the next section. The difference between the two fields in the number density of $`z>1`$ galaxies is likely due to the fluctuations in large-scale structures. In particular, our survey is sensitive to galaxies at the bright end of the galaxy luminosity function, which are believed to be more strongly clustered.

### 2.4 Uncertainties in Photometric Redshift Measurements

In order for any statistical analysis of the intrinsic properties of galaxies identified using photometric redshift techniques to be plausible, we must first quantify the uncertainties. As discussed in detail in Lanzetta et al. (1998) and Fernández-Soto et al. (2001, 2002), uncertainties in photometric redshift measurements are due to two sources: (1) for bright galaxies, the dominant uncertainty is due to the finite number of templates employed in the analysis; this is known as template-mismatch variance, and (2) for faint galaxies, an additional uncertainty is due to photometric measurement uncertainties. Fernández-Soto et al. (2001) demonstrated that uncertainties in photometric redshifts due to template mismatch is well characterized by a Gaussian distribution function with the 1-$`\sigma `$ width equal to the RMS residuals between photometric and spectroscopic redshifts. In addition, equation (1) explicitly shows that in the absence of template-mismatch variance the uncertainty of the redshift likelihood function depends entirely upon the accuracy and precision of the flux measurements. Namely, the smaller the $`\sigma _i`$’s are, the more precise the best-fit photometric redshift measurement. Uncertainties in photometric redshifts due to photometric errors are, therefore, well represented by the redshift likelihood functions of individual objects.

Photometric redshift uncertainties can be quantified by explicitly taking into account the redshift likelihood functions without adopting a parametric form. The error function of the photometric redshift measurement of galaxy $`i`$ is

$$p_i(zz_i;z_i)=_0^{\mathrm{}}\frac{1}{\sigma _z\sqrt{2\pi }}_z^i(z^{}z_i;z_i)\mathrm{exp}[(zz^{})^2/2\sigma _z^2]𝑑z^{}.$$
(3)
Equation (3) is a convolution of the redshift likelihood function $`_z^i(z)`$ and a Gaussian kernel of width $`\sigma _z=\widehat{\sigma }_z(1+z_i)`$ that characterizes the template-mismatch variance, where $`\widehat{\sigma }_z`$ is the RMS residuals between spectroscopic and photometric redshifts at zero redshift.

To assess the accuracy and reliability of our photometric redshifts, we first compared our photometric redshifts with known spectroscopic redshifts. There are over 100 objects in the HDFS region (Glazebrook 1998; Tresse et al. 1999; Palunas et al. 2000; Cristiani et al. 2000; Dennefeld 2002). Excluding the ambiguous spectroscopic redshift identifications indicated by these authors, we found 67 objects that have both reliable spectroscopic and photometric redshift measurements at $`z2.5`$, 13 of which are stars. There are 162 known spectroscopic redshifts in the CDFS region obtained by ourselves (McCarthy et al. 2002, in preparation). Excluding ambiguous photometric redshifts (objects with uncertain redshift likelihood distributions), we found 140 objects with both reliable spectroscopic and photometric redshifts.

Figure 4 shows the comparisons of photometric ($`z_{\mathrm{phot}}`$) and spectroscopic ($`z_{\mathrm{spec}}`$) redshifts for the HDFS and CDFS objects in the left panel. Solid points represent objects identified in the HDFS region and open circles represent those in the CDFS region. The solid line indicates $`z_{\mathrm{phot}}=z_{\mathrm{spec}}`$. The distributions of redshift residuals are shown in the right panel of Figure 4 for the HDFS (open histogram) and CDFS (shaded histogram) objects. We find an RMS dispersion of $`\sigma _z/(1+z)0.08`$ for the 67 HDFS objects at $`z1`$. Only one of the 13 stars was misidentified as a galaxy at $`z_{\mathrm{phot}}=0.09`$. Note that the mean redshift residuals are nearly zero at all redshifts that have been tested here, supporting that there is no systematic bias in our photometric redshifts. It demonstrates that the photometric redshifts obtained using $`UBVRIH`$ photometry are accurate at all redshifts below $`z1`$. The comparison based on the 34 CDFS objects at $`z0.75`$ shows $`\sigma _z/(1+z)0.08`$. It demonstrates that the photometric redshifts obtained using $`VRIz^{}H`$ photometry are accurate at redshifts beyond $`z0.75`$, although they become unreliable at lower redshifts due to the lack of $`U`$ and $`B`$ photometry.

There are only a small number of known spectroscopic redshifts at $`1z2.5`$ in both the HDFS and CDFS regions, but it is unlikely that the accuracy of the photometric redshifts at $`1z2.5`$ would be much worse than those at lower redshifts. It has been demonstrated by different groups that photometric redshifts are accurate to within $`\sigma _z/(1+z)<0.1`$ at $`1z1.5`$ for all galaxies that have been tested so far (Connolly et al. 1997; Fernández-Soto et al. 2001; Rudnick et al. 2001). In particular, Fernández-Soto et al. (2001) showed that using the same set of SED templates their photometric redshifts are consistent with spectroscopic redshifts for all 19 galaxies that have known spectroscopic redshifts at $`1z1.5`$ in the HDF north. This supports that the number of templates in our photometric redshift analysis is sufficient to minimize template mismatch uncertainties. A more critical examination of the accuracy of our photometric redshifts in this redshift range relies on the availability of more spectroscopic measurements.

Next, we performed a series of simulations designed to evaluate the uncertainties in the photometric redshifts due to purely photometric uncertainties. We first generated a set of photometric measurements in each bandpass for a given $`H`$-band magnitude and input redshift $`z_{\mathrm{in}}`$ based on an E/S0 template. We then perturbed the photometric measurements in individual bandpasses within the $`1\sigma `$ photometric uncertainties measured for the individual optical and $`H`$ images. Next, we determined a best-fit redshift $`z_{\mathrm{out}}`$ using the photometric redshift technique. Finally, we repeated this procedure 1000 times and calculated the histogram of the residuals between $`z_{\mathrm{in}}`$ and $`z_{\mathrm{out}}`$. We repeated the entire process for the Sab, Scd, and Irr spectral templates to assess the performance of the photometric redshift technique for galaxies of different spectral shape. Figure 5 shows an example of the simulation results based on the filter combination and image sensitivities of the HDFS data. The left panels show the histograms of the residuals between the input galaxy redshift $`z_{\mathrm{in}}`$ and the best-fit photometric redshift $`z_{\mathrm{out}}`$ and the right panels show the cumulative fractions versus absolute redshift residuals, $`\mathrm{\Delta }z=|z_{\mathrm{out}}z_{\mathrm{in}}|`$. The results indicate that while we could accurately identify an Sab-Irr galaxy of $`H=20`$ at $`z=1.2`$, there is an $`10`$% chance that we would misidentify an E/S0 galaxy of $`H=20`$ at $`z=1.2`$ as a starburst galaxy at $`z5`$. The bimodal distribution of the residuals is due to the confusion between a 4000-Å spectral discontinuity at low redshifts and a Ly$`\alpha `$ continuum break at high redshifts.

We repeated the simulations for a grid of redshift and $`H`$-band magnitude based on the filter combination and image sensitivities of both HDFS and CDFS data. The results are presented in Figure 6, each panel of which now shows the sum of the residual histograms over all four spectral types. Figure 6 indicates that, even in the absence of template-mismatch variance, photometric redshifts become progressively more uncertain at fainter apparent magnitudes and higher redshifts, as indicated by the broader distribution of the residuals. The bimodal distribution that appears, for example, for an E/S0 galaxy of $`H=20`$ at $`z=1.2`$ in the HDFS region does not exist for a galaxy of the same brightness at the same redshift range in the CDFS region, because the CDFS images reach to fainter magnitude limits. It is also clear, however, that there exists a non-negligible probability that galaxies at $`z0.4`$ in the CDFS region would be misidentified as galaxies at $`z3`$ even at bright magnitudes. The misidentifications occur primarily in cases when the input spectral template was an early-type galaxy. Because of the lack of $`U`$\- and $`B`$-band photometric measurements for galaxies in the CDFS region, the observed SEDs are poorly constrained at rest-frame wavelengths shortward of the 4000-Å break. The ambiguity disappears for the CDFS galaxies at $`z0.8`$, where the available $`V`$-band photometry begins to impose a strong constraint on the shape of the observed SEDs.

In Figure 7, we present the cumulative fractions of the simulated photometric redshifts versus absolute redshift residuals for the four $`z=1.2`$ cases presented in Figure 6. The left panels show the simulation results for the HDFS galaxies and the right panels for the CDFS galaxies. At $`z=1.2`$, it is clear that photometric redshifts are accurate to within $`\mathrm{\Delta }z=0.1`$ for the CDFS galaxies of $`H21`$, and that they become very uncertain for the HDFS galaxies at $`H21`$ because of their relatively larger photometric uncertainties. Specifically, there is a $``$ 30% chance that the photometric redshifts of these galaxies are off by more than $`\mathrm{\Delta }z=0.1`$.

In summary, we conclude that the photometric redshift measurements of the HDFS objects based on $`U`$, $`B`$, $`V`$, $`R`$, $`I`$, and $`H`$ photometry are robust for galaxies with $`H19`$ at all redshifts $`z1`$. The photometric redshift measurements of the CDFS objects based on $`V`$, $`R`$, $`I`$, $`z^{}`$, and $`H`$ photometry are robust for galaxies of $`H20`$ at all redshifts $`0.75z1`$. We exclude CDFS objects with photometric redshifts $`z0.75`$ from subsequent analysis because of the lack of photometric measurements in the observed $`U`$ and $`B`$ bands (see also Connolly et al. 1995). The robustness of photometric redshifts at $`z1`$ remains to be examined more critically, because only a small number of known spectroscopic redshifts are available in our survey fields. However, consistent redshift estimates of the HDF galaxies at $`1z1.5`$ support the photometric redshift technique using the six templates in our analysis for this redshift interval (e.g. Fernández-Soto et al. 2001). We note that redshift uncertainties due to dust reddening is not explicitly accounted for in our analysis. We consider the effect of dust as part of sources that lead to template mismatch. To further improve the reliability of the photometric redshift measurements and to reduce the number of low-redshift galaxies that are mis-identified as star-forming galaxies at high redshifts due to photometric uncertainties, we restricted the redshift range of the photometric redshift likelihood analysis to $`z2.5`$. This is appropriate because the fraction of $`H20`$ galaxies at $`z2.5`$ is $`\mathrm{\hspace{0.17em}0.1}`$ % (Lanzetta et al. 1999; Cohen et al. 2000; Shapley et al. 2001).

## 3 Rest-frame $`R`$-band Galaxy Luminosity Function

Here we present the rest-frame $`R`$-band luminosity function for three redshift ranges: $`0.5z0.75`$, $`0.75z1.0`$, and $`1.0z1.5`$. We emphasize that the results are based on photometric redshifts for $`3000`$ $`H`$-band selected galaxies with apparent magnitude $`17H20`$. We calculated the luminosity function separately for the total $`H`$-band selected sample and for a sub-sample of early-type galaxies that have a best-fit spectral type of E/S0 or Sab in the photometric redshift analysis. The primary objectives are (1) to obtain an estimate of the galaxy luminosity function at $`z1`$ in order to study the statistical properties of galaxies at this epoch, and (2) to study galaxy evolution at redshifts $`0.5z1.5`$ based on a uniform sample of near-infrared selected galaxies; this extends the redshift range of existing deep redshift surveys to beyond $`z=0.75`$.

The absolute $`R`$-band magnitude $`M_R`$ of a galaxy with an apparent magnitude $`H`$ at redshift $`z`$ is

$$M_R(z)=H25.05\times \mathrm{log}\frac{D_L(z)}{(\text{Mpc})}k^{}(z)+2.5\times \mathrm{log}(1+z),$$
(4)
where $`D_L(z)`$ is the luminosity distance to the galaxy in units of Mpc and $`k^{}(z)`$ is the $`k`$-correction term to account for the color difference between the observed-frame and corresponding rest-frame bandpasses. The $`k`$-correction term for each galaxy was calculated using the best-fit spectral template from the photometric redshift analysis. For galaxies at $`z1.2`$, the observed-frame $`H`$ band approximately corresponds to rest-frame $`R`$, so the calculation does not depend sensitively upon the adopted templates to determine the $`k`$-correction. For galaxies at $`z<1`$, the estimated $`k`$-correction ranges from between $`0.34`$ (SB) and $`0.9`$ (E/S0) at $`z=0.5`$ to between $`0.12`$ (SB) and $`0.43`$ (E/S0) at $`z=1.0`$.

Several methods to calculate the luminosity function have been proposed over the years. We examine three of these: (1) the $`1/V_{\mathrm{max}}`$ approach (Schmidt 1968; Felten 1976), (2) the Sandage-Tammann-Yahil (STY) approach (Sandage, Tammann, & Yahil 1979), and (3) the stepwise maximum likelihood (SWML) approach (Efstathiou, Ellis, & Peterson 1988) in the context of a photometric-redshift determination of the luminosity function. A foreseeable systematic error in the derived luminosity function using photometric redshifts is to overestimate the number density of galaxies at the bright end, producing a flattened, deduced galaxy luminosity function relative to the intrinsic luminosity function. This is due to the relatively large uncertainties associated with photometric redshifts and the steep slope at the bright end of the galaxy luminosity function. There are more intrinsically fainter galaxies scattered into the brighter end of the luminosity function than intrinsically brighter galaxies scattered into the fainter end, thus yielding an apparently flatter luminosity function. To obtain robust estimates of the “intrinsic” galaxy luminosity function using photometric redshifts, it is therefore important to account for these systematic biases. In the following sections we present a maximum likelihood analysis that explicitly takes into account the empirical redshift error functions of individual objects in the luminosity function calculation, and demonstrate, via a Monte Carlo simulation, that our approach allows us to reduce the systematic errors and to completely recover the bright end of the galaxy luminosity function. We summarize these three different luminosity function estimators, describe how we quantify and incorporate the photometric redshift uncertainties.

### 3.1 The $`1/V_{\mathrm{max}}`$ Approach

The $`1/V_{\mathrm{max}}`$ approach was first proposed by Schmidt (1968) and modified by Felten (1976). The galaxy luminosity function $`\varphi (M)`$ is related to the number of galaxies per co-moving volume expected in a survey defined by the limiting apparent magnitude range $`m_{\mathrm{min}}mm_{\mathrm{max}}`$ and the redshift range $`z_{\mathrm{min}}zz_{\mathrm{max}}`$ according to

$$\mathrm{\Delta }n=\varphi (M)(M_{\mathrm{max}}M_{\mathrm{min}})=\underset{i}{}\frac{1}{V_i},\text{for }M_{\mathrm{min}}M_iM_{\mathrm{max}}\text{ and }z_{\mathrm{min}}z_iz_{\mathrm{max}}.$$
(5)
The absolute magnitude range $`[M_{\mathrm{min}},M_{\mathrm{max}}]`$ corresponds to the apparent magnitude range imposed by the survey at the galaxy redshift $`z_i`$. The co-moving volume $`V_i`$ is the maximum accessible volume of the survey for a galaxy of $`M_i`$,

$$V_i=_{z_{\mathrm{min}}^{}}^{z_{\mathrm{max}}^{}}\frac{c\mathrm{\Theta }_i}{H_0}\frac{D_L^2}{(1+z)^2}\frac{dz}{\sqrt{\mathrm{\Omega }_M(1+z)^3(\mathrm{\Omega }_M+\mathrm{\Omega }_\mathrm{\Lambda }1)(1+z)^2+\mathrm{\Omega }_\mathrm{\Lambda }}},$$
(6)
where $`c`$ is the speed of light, $`\mathrm{\Theta }_i`$ is the angular area covered by the survey that is sensitive enough to detect the galaxy $`M_i`$ at $`z_i`$, and $`z_{\mathrm{min}}^{}`$ and $`z_{\mathrm{max}}^{}`$ are the minimum and maximum redshifts that satisfy both the limiting magnitude and redshift ranges of the survey. The angular area $`A_i`$ for each galaxy identified in the LCIR survey is determined from the results of the incompleteness analysis presented in Chen et al. (2002).

The $`1/V_{\mathrm{max}}`$ approach is essentially a maximum likelihood method that does not assume any parametric form for estimating the galaxy luminosity function (see also § 3.3 below). It has been shown, however, that large-scale fluctuations in the galaxy number density introduce systematic biases in the galaxy luminosity function obtained with this approach for a magnitude-limited sample (de Lapparent, Geller, & Huchra 1989). We show in § 3.4 that it is difficult to recover the intrinsic galaxy luminosity function with this approach for a galaxy sample identified in a photometric redshift survey.

### 3.2 The Sandage-Tammann-Yahil Approach

Previous studies have demonstrated that the galaxy luminosity function may be well represented by a Schechter (1976) function,

$$\mathrm{\Phi }(L;L_{})=\varphi _{}(L/L_{})^\alpha \mathrm{exp}(L/L_{})$$
(7)
or

$$\mathrm{\Phi }(M;M_{})=(0.4\mathrm{ln}10)\varphi _{}10^{0.4(M_{}M)(1+\alpha )}\mathrm{exp}(10^{0.4(M_{}M)}).$$
(8)
To accurately determine the faint-end slope $`\alpha `$ and $`M_{}`$ for the Schechter luminosity function in the presence of large-scale density fluctuations, Sandage, Tammann, & Yahil (1979) first introduced a maximum likelihood approach that calculates the cumulative probability of observing an ensemble of galaxies in a magnitude-limited survey for given $`\alpha `$ and $`M_{}`$. The probability of observing a galaxy of $`M_i(m_i,z_i)`$ in a magnitude-limited redshift survey given a Schechter luminosity function is

$$P_i(M_i;M_{})=\frac{10^{0.4(M_{}M_i(m_i,z_i))(1+\alpha )}\mathrm{exp}(10^{0.4(M_{}M_i(m_i,z_i))})}{\mathrm{\Theta }_i_{m_{\mathrm{min}}}^{m_{\mathrm{max}}}10^{0.4(M_{}M(m,z))(1+\alpha )}\mathrm{exp}(10^{0.4(M_{}M(m,z))})𝑑m},$$
(9)
where $`\mathrm{\Theta }_i`$ is the fraction of the angular area that is sensitive enough to detect the galaxy $`m_i`$ at redshift $`z_i`$ and $`[m_{\mathrm{min}},m_{\mathrm{max}}]`$ defines the magnitude sensitivity range of the survey. If the luminosity function remains the same for galaxies with different clustering properties at different redshifts, then the likelihood of obtaining an ensemble of $`N`$ galaxies in the survey is

$$_\varphi =\underset{i=1}{\overset{i=N}{}}P_i(M_i;M_{}).$$
(10)
The faint-end slope $`\alpha `$ and characteristic magnitude $`M_{}`$ in equation (8) are determined by maximizing the likelihood function $`_\varphi `$.

The normalization $`\varphi _{}`$ of the Schechter luminosity function is determined separately, according to

$$\varphi _{}=\underset{i=1}{\overset{i=N}{}}V^1/_{M_{\mathrm{lim}}(z_i)}^{\mathrm{}}\overline{\mathrm{\Phi }}(M;M_{})𝑑M,$$
(11)
where $`V`$ is the co-moving volume spanned from $`z_{\mathrm{min}}`$ to $`z_{\mathrm{max}}`$ in the survey, $`\overline{\mathrm{\Phi }}(M;M_{})`$ is the galaxy luminosity function defined in equation (8) with $`\varphi _{}=1`$, and $`M_{\mathrm{lim}}(z_i)`$ is the corresponding absolute magnitude limit of the survey at the galaxy redshift $`z_i`$. The uncertainty in $`\varphi _{}`$ may be estimated by bootstrap resampling, that is, randomly re-sampling the same number of galaxies from the parent catalog a large number of times (allowing for duplications) and measuring the variation of $`\varphi _{}`$ between different re-sampled catalogs.

### 3.3 The Stepwise Maximum Likelihood Approach

A different maximum likelihood approach was developed by Efstathiou, Ellis, & Peterson (1988), one which does not require a functional form in the analysis and is similarly insensitive to the presence of large-scale density fluctuations in the survey. In this approach, the galaxy luminosity function is parameterized according to

$$\mathrm{\Phi }(M)=\varphi _k(M_k),\text{if }M_k\frac{1}{2}\mathrm{\Delta }MM<M_k+\frac{1}{2}\mathrm{\Delta }M\text{, for }k=1,..,L.$$
(12)
The probability of observing a galaxy of $`M_i(m_i,z_i)`$, for which $`M_k\frac{1}{2}\mathrm{\Delta }MM_i(m_i,z_i)<M_k+\frac{1}{2}\mathrm{\Delta }M`$ is,

$$P_i(m_i,z_i)=\varphi _k(m_i,z_i)/\underset{k=k(m_{\mathrm{min}})}{\overset{k=k(m_{\mathrm{max}})}{}}\varphi _k(m,z_i)\mathrm{\Delta }m,$$
(13)
where $`k(m_{\mathrm{min}},z_i)`$ and $`k(m_{\mathrm{max}},z_i)`$ indicate the minimum and maximum absolute magnitude intervals defined by the apparent magnitude sensitivity range of the survey at $`z_i`$. The likelihood of observing $`N`$ galaxies in a magnitude-limited survey is then

$$_\varphi =\underset{i=1}{\overset{i=N}{}}P_i(m_i,z_i).$$
(14)
The best-fit $`\varphi _k`$’s are obtained by maximizing the likelihood function defined in equation (14).

In the simplest case where $`k(m_{\mathrm{min}})=k_0`$ and $`k(m_{\mathrm{max}})=k_f`$ for $`i=1`$$`N`$, it is straightforward to show that $`\varphi _km_k`$, where $`m_k`$ is the number of galaxies in each magnitude interval. Therefore, in the absence of large-scale density fluctuations, the SWML approach and the $`1/V_{\mathrm{max}}`$ approach are essentially the same.

### 3.4 Systematic Uncertainties

Photometric redshift techniques clearly provide an efficient means of identifying distant, faint galaxies in deep, wide-field surveys. It is, however, important to understand how the redshift measurement errors affect the derived luminosity functions of distant galaxies. Photometric redshift uncertainties propagate through the uncertainties in luminosity distances and consequently yield large uncertainties in the absolute magnitudes of individual galaxies. Here we neglect the effect of purely photometric uncertainties, because according to equation (4) a redshift uncertainty of $`\delta z=0.1`$ will result in a magnitude uncertainty of $`\mathrm{\Delta }M0.4`$ at $`z1`$ due to the uncertainty in the luminosity distance. It is clear that photometric redshift errors dominate the uncertainties in the absolute magnitude calculations of individual galaxies.

We performed a Monte Carlo simulation in order to understand the systematic uncertainties in the galaxy luminosity function calculations due to errors in photometric redshifts. First, we generated a model catalog of 8000 galaxies of different brightness at random redshifts between $`z=0.1`$ and $`z=1`$ according to an input Schechter luminosity function. Next, we assumed a redshift error function parameterized as a Gaussian distribution function of $`1\sigma `$ width $`0.15(1+z_i)`$ and formed an “observed” redshift catalog by perturbing the input galaxy redshift $`z_i`$ within the redshift error function. Finally, we determined the luminosity function for the galaxies at $`0.5z0.8`$ using the $`1/V_{\mathrm{max}}`$ and STY methods.

Figure 8 shows the derived galaxy luminosity function based on the simulated catalogs using different approaches. The input galaxy luminosity function is shown as the dash-dotted curve with the selected $`M_{}`$ and $`\alpha `$ indicated as a star in the inset. The solid circles represent measurements using the “observed” galaxy redshifts based on the $`1/V_{\mathrm{max}}`$ approach. The error bars are the associated $`1\sigma `$ uncertainties estimated with a bootstrap re-sampling technique that takes into account both the sampling and redshift uncertainties. The dotted curve represents the best-fit Schechter luminosity function to the “observed” redshift catalog. The best-fit $`M_{}`$ and $`\alpha `$ correspond to the upper solid point in the inset with the 99% uncertainties indicated by the dotted contour.

It is clear from Figure 8 that large redshift errors together with the steep slope at the bright end of the galaxy luminosity function tend to flatten the observed luminosity function and result here in a best-fit $`M_{}`$ that is 0.8 mag brighter than the input value. This confirms the statements above that more intrinsically fainter galaxies are scattered into the brighter end of the luminosity function than are intrinsically brighter galaxies scattered into the fainter end. This is equivalent to convolving the galaxy luminosity function with an error function in the galaxy absolute magnitudes induced by the redshift uncertainties. Evidently, redshift measurement uncertainties must be accounted for in order to obtain an accurate estimate of the intrinsic galaxy luminosity function.

### 3.5 A Modified Maximum Likelihood Analysis

In order to reduce the aforementioned bias we modified the likelihood analysis to incorporate the error functions of photometric redshift measurements into the probability calculations. Because objects are scattered in redshift space due to the relatively large redshift measurement errors, it is necessary to rewrite equations (9) and (13). The probability of observing a galaxy of $`M_i(m_i,z_i)`$ in a magnitude-limited photometric redshift survey is now the cumulative probability for a galaxy of apparent magnitude $`m_i`$ at $`z^{}`$ to be identified at $`z_i`$ according to the redshift error function $`p_i(z_iz;z_i)`$ defined in equation (3). Namely, it is a convolution of equations (9) and (13), respectively, with the error function of the photometric redshift measurement of the galaxy. Equation (9) defined for the STY approach is therefore

$$P_i(m_i,z_i;M_{})=\frac{_0^{z_f}\mathrm{\hspace{0.17em}10}^{0.4(M_{}M_i(m_i,z^{}))(1+\alpha )}\mathrm{exp}(10^{0.4(M_{}M_i(m_i,z^{}))})p_i(z_iz^{};z_i)𝑑z^{}}{\mathrm{\Theta }_{m_{\mathrm{min}}}^{m_{\mathrm{max}}}_0^{z_f}10^{0.4(M_{}M(m,z^{}))(1+\alpha )}\mathrm{exp}(10^{0.4(M_{}M(m,z^{}))})p_i(z_iz^{};z_i)𝑑z^{}𝑑m},$$
(15)
where $`z_f`$ is the redshift limit of the grid search in the photometric redshift analysis, and equation (13) defined for the SWML approach is

$$P_i(m_i,z_i)=\frac{_0^{z_f}\varphi _k(m_i,z^{})p_i(z_iz^{};z_i)𝑑z^{}}{_{k=k(m_{\mathrm{min}})}^{k=k(m_{\mathrm{max}})}_0^{z_f}\varphi _k(m,z^{})p_i(z_iz^{};z_i)𝑑z^{}\mathrm{\Delta }m}.$$
(16)
In cases where an accurate and precise spectroscopic redshift measurement is available, $`p_i(z_iz;z_i)=\delta (zz_i)`$ and equations (15) and (16) reduce to equations (9) and (13). The parameters that determine the Schechter luminosity function, $`\alpha `$ and $`M_{}`$ in the STY approach and the $`\varphi _k`$’s in the SWML approach, are then evaluated with the maximum likelihood functions (equations 10 and 14).

We applied this modified likelihood analysis to the simulated catalog described in § 3.4 in order to determine how accurately we recover the intrinsic luminosity function with this prescription for the “empirical” error functions of the photometric redshifts. Here the redshift error function of each galaxy is $`p_i(zz_i;z_i)=\frac{1}{\widehat{\sigma }_z^{}(1+z_i)\sqrt{2\pi }}\mathrm{exp}[(zz^{})^2/2\widehat{\sigma }_z^2(1+z_i)^2]`$ with $`\widehat{\sigma }_z^{}=0.15`$. The results are shown in Figure 8 as well. The solid curve shows the best-fit Schechter luminosity function. The best-fit $`M_{}`$ and $`\alpha `$ are the lower solid point in the inset with the 99% uncertainties indicated by the solid contour. The step function shows the best-fit stepwise luminosity function with the vertical bars indicating the associated $`3\sigma `$, one-parameter uncertainties. Comparison of the simulation results for the luminosity function measurements with the input model shows that we are able to significantly reduce the systematic uncertainties (by $``$ 0.7 mag) by taking into account an empirical redshift error function for each galaxy in the STY and SWML approaches. Although a lack of data means we could not constrain the faint-end galaxy luminosity function very well, we were able to recover the bright-end galaxy luminosity function to approximately one magnitude fainter than $`M_{}`$.

### 3.6 The Galaxy Luminosity Functions from the LCIR Photometric Redshift Survey

We applied the modified maximum likelihood analysis to determine the rest-frame $`R`$-band galaxy luminosity function for $`3000`$ $`H`$-band selected galaxies identified at $`0.5z1.5`$ in the LCIR survey. We adopted the redshift error function as formulated in equation (3), which is a convolution of the redshift likelihood function and a Gaussian kernel of width $`\widehat{\sigma }_z(1+z)`$ with $`\widehat{\sigma }_z=0.08`$ as determined empirically from the comparison of photometric and spectroscopic redshifts described in § 2.3.

The results of the analysis are presented in Figures 9 and 10 for the $`H`$-band detected galaxies. We calculated the luminosity functions for the total $`H`$-band selected sample (left panels) and for galaxies that have a best-fit spectral template of either E/S0 or Sab in the photometric redshift likelihood analysis (right panels). The solid curves in Figure 9 represent the best-fit Schechter luminosity functions at $`0.5z0.75`$ (top panels), $`0.75z1.0`$ (middle panels), and $`1.0z1.5`$ (bottom panels) determined with the modified STY approach. Only galaxies identified in the HDFS region were included in the luminosity function calculations at $`0.5z0.75`$ for the reasons discussed in § 2.4. For comparison, we include the $`r`$-band luminosity function determined for galaxies at $`z<0.2`$ from the Sloan Digital Sky Survey (SDSS; Blanton 2001; dotted curve) in the top-left panel. In addition, the best-fit luminosity functions of the LCIRs galaxies at $`0.5z0.75`$ are also indicated in the middle and bottom panels as the short-dashed curves. Because of the lack of sensitivity to intrinsically faint galaxies in the $`H`$-band survey, the solid curves shown in the middle and bottom panels were determined by fixing $`\alpha `$ to the best-fit value at $`0.5z0.75`$ and allowing only $`M_R_{}`$ to vary in the likelihood analysis. We have, however, attempted to determine both $`\alpha `$ and $`M_R_{}`$ for galaxies at $`0.75z1.0`$ and the results are shown as the long-dashed curves in the middle panels. The best-fit galaxy luminosity functions determined based on the modified SWML approach are shown as solid points in Figure 9. The vertical bars indicate the corresponding $`1\sigma `$ one-parameter uncertainties determined according to $`2\mathrm{ln}_\varphi =2\mathrm{ln}_{\varphi ,\mathrm{max}}\mathrm{\Delta }\chi ^2`$ with $`\mathrm{\Delta }\chi ^2=1`$.

The good agreement at the bright end between the two best-fit Schechter luminosity functions (the solid and long-dashed curves) in each of the two middle panels of Figure 9 shows that despite the lack of sensitivity at the faint end, we can determine the bright-end galaxy luminosity function and therefore $`M_R_{}`$ fairly reliably using the $`H`$-band selected LCIR survey sample. In addition, a comparison of the solid and short-dashed curves in each panel indicates that there is little or no evolution in the rest-frame $`R`$-band galaxy luminosity function for the entire $`H`$-band selected sample. The 99% error contours of the best-fit $`M_R_{}`$ and $`\alpha `$ using the modified STY analysis are shown in Figure 10 for galaxies in different redshift intervals. It is clear from the top panels that the faint-end slope for galaxies at these redshifts is poorly constrained because our survey is insensitive to intrinsically faint galaxies at $`z0.75`$.

We summarize the results of the modified STY analysis in Table 1, in which we list the redshift range, the best-fit faint-end slope $`\alpha `$, $`M_R_{}5\mathrm{log}h`$, and $`\varphi _{}`$ together with the associated $`1\sigma `$, one-parameter errors. The number of galaxies that are included in the analysis for all $`H`$-band selected galaxies and the $`H`$-band selected early-type galaxies are also listed. The results of the modified SWML analysis are summarized in Table 2, in which we list the rest-frame $`R`$-band magnitude, $`M_R5\mathrm{log}h`$, and the best-fit $`\mathrm{log}\varphi (M_R)/(h^3\mathrm{Mpc}^3)`$ together with the associated $`1\sigma `$, one-parameter errors in three different redshift ranges for the $`H`$-band selected total and early-type samples.

## 4 Discussion

We have developed a technique to obtain a robust estimate of the intrinsic luminosity function using photometric redshifts. We adopt an “empirical” estimate of the redshift error function for each galaxy, which is a convolution of the redshift likelihood function that characterizes the uncertainty due to photometric errors and a Gaussian kernel of 1-$`\sigma `$ width $`\sigma _z`$ that characterizes the uncertainty due to template mismatch. This technique takes into account the non-gaussian error characteristics of photometric redshifts, and thus differs from the approach of Subbarao et al. (1996), who used a simple Gaussian to model the photometric redshift uncertainties. The results of the simulation described in § 3.4 show that our approach allows us to completely recover the galaxy luminosity function to the sensitivity limit of the survey. Here we compare the derived galaxy luminosity functions at different redshifts and discuss the implications for the evolution of galaxies since $`z1.5`$.

### 4.1 Galaxy Evolution at $`0.5z1.5`$

It is clear from Figure 9 that the depth of the $`H`$-band survey is not sufficiently sensitive to constrain the statistical properties of galaxies fainter than $`L_{}`$ at $`z0.75`$, and we are therefore unable to determine the evolution of the faint galaxy population based on the $`H`$-band survey. However, the data are sufficient to determine the evolution of the bright galaxy population at $`0.5z1.5`$ based on the $`H`$-band survey data. The evolution of this population has significant implications for understanding how galaxies form because different formation scenarios make distinct predictions for the number density evolution of massive galaxies. For example, under monolithic collapse scenarios (e.g. Eggen, Lynden-Bell, & Sandage 1962), massive galaxies form early on a dynamical timescale and have a constant co-moving space density and gradually declining intrinsic luminosities. In contrast, hierarchical formation scenarios (e.g. White & Rees 1978) predict that massive galaxies form through the merger of lower-mass galaxies over a Hubble time and therefore the co-moving space density of massive galaxies increases as the universe evolves.

The luminosity functions presented in Figure 9 and Tables 1 and 2 show several interesting results. First, the best-fit Schechter luminosity functions agree with the stepwise luminosity functions to within the measurement uncertainties at all redshifts, including the luminosity function at $`0.5z0.75`$ where our survey is sufficiently sensitive to identify galaxies as faint as $`0.2L_{}`$. Second, there exists little or no evolution either in $`M_R_{}`$ or in $`\varphi _{}`$ for the total $`H`$-band selected population at $`0.5z1.5`$. Third, we find a moderate luminosity evolution for galaxies with an SED best characterized as E/S0 or Sab galaxies. Specifically, our measurements obtained from the modified STY approach suggest that from $`z1`$ to $`z0.5`$ these early-type galaxies have faded by $`0.5`$ mag. This is consistent with the expected luminosity evolution for a galaxy formed at $`z_f=30`$ and following an exponentially declining star formation rate model with $`\tau =1`$ Gyr (McCarthy et al. 2001; Chen et al. 2002). Finally, we find only a mild evolution in $`\varphi _{}`$ between $`z0.5`$ and $`z1`$ ($`40`$% decrease with redshift) for the color-selected early-type galaxies, but we cannot argue against the no evolution scenario at more than the $`2\sigma `$ level. When we consider the entire $`H`$-band selected sample at $`0.5z1.5`$, our estimates of $`\varphi _{}`$ are completely consistent with a no evolution scenario.

It appears that neither the total $`H`$-band selected sample nor the color-selected early-type galaxies have evolved significantly in intrinsic luminosity or space density since $`z1.5`$, contrary to the predictions of the hierarchical formation scenarios. But as with all luminosity function measurements, it is clear that there exists a strong degeneracy between $`M_R_{}`$ and $`\varphi _{}`$ at $`z0.75`$ in our analysis, because the $`H`$-band survey is not sensitive to sub-$`L_{}`$ galaxies at this redshift range and our estimates for these two parameters were determined from the space density of intrinsically bright galaxies with a fixed faint-end slope $`\alpha `$. For example, a bright $`M_R_{}`$ together with a small $`\varphi _{}`$ can produce the same bright-end luminosity function as a faint $`M_R_{}`$ with a large $`\varphi _{}`$. To estimate how significantly early-type galaxies have evolved since $`z1.5`$, we therefore calculated the rest-frame co-moving $`R`$-band luminosity density $`\mathrm{}_R`$ for galaxies brighter than $`M_{\mathrm{min}}`$ using the best-fit Schechter luminosity function,

$`\mathrm{}_R`$ $`=`$ $`{\displaystyle _{L_{min}}^{\mathrm{}}}L\mathrm{\Phi }(L;L_{})d(L/L_{})`$ (17)
$`=`$ $`{\displaystyle _{\mathrm{}}^{M_{\mathrm{min}}}}(0.4\mathrm{ln}10)\varphi _{}L_{}10^{0.4(M_{}M)(2+\alpha )}\mathrm{exp}(10^{0.4(M_{}M)})𝑑M.`$ (18)
This represents an integrated quantity that characterizes the brightest galaxy population and is least sensitive to faint galaxies.

We test different formation scenarios using the estimated co-moving luminosity density evolution of the bright, red galaxies. Including the luminosity function estimated for early-type galaxies at $`z=0.3`$ in the CNOC2 survey (Lin et al. 1999), we first compared $`\mathrm{}_R`$ calculated for a constant $`M_{\mathrm{min}}`$ at all redshifts. The results are presented in the left panel of Figure 11, which shows a flat $`\mathrm{}_R`$ versus $`z`$ for $`M_{\mathrm{min}}=20.5`$ (squares) , $`21.0`$ (triangles), and $`21.5`$ (circles). These roughly corresponds to $`L_{}`$, $`1.6L_{}`$, and $`2.5L_{}`$, respectively, at $`z0.3`$. However, because stars evolve with time, the corresponding luminosity evolution must be accounted for. To distinguish between a passively evolving population and hierarchical mergers, we estimated the amount of brightening ($`\mathrm{\Delta }M_R`$) with increasing redshift (or with decreasing age of the universe) for different stellar evolution models (see Table 3 for a summary). We considered a wide range of stellar evolution scenarios, from a single burst at $`z_f=30`$, to an exponentially declining star formation rate model with $`z_f=5`$ and $`\tau =1`$ Gyr. The former represents a classic passive evolution mode under the monolithic collapse scenariol. The latter represents the closest resemblance of a more continuous star-forming scenario in which some residual star formation is taking place at $`z<3`$, while there is still enough time for the red galaxy population to be present by $`z1`$. We then calculated $`\mathrm{}_R`$ for a suite of evolving $`M_{\mathrm{min}}(z)`$ according to different stellar evolution recipes.

The goal of this analysis is to examine whether or not mergers are the dominant process for the formation of red galaxies. Mergers would lead to brightening with time (smaller systems combined to form bigger ones), which compensates the expected fading due to stellar evolution. We would therefore expect to observe an increasing $`\mathrm{}_R`$ with decreasing redshift using ($`\mathrm{\Delta }M_R`$) presented in Table 3, if these red galaxies formed through a sequence of mergers. A flat redshift distribution of $`\mathrm{}_R`$ would then imply that mergers are insignificant in the redshift interval tested.

We present the results in Figure 11. The middle panel is for the $`z_f=30`$ single burst scenario and the right panel is for the $`z_f=5`$ exponentially declining star formation rate model. The results in these two panels have been scaled to have consistent $`M_{\mathrm{min}}`$ at $`z=0.3`$ as in the left panel. In addition, we exclude the brightest subsample (circles in the left panel) from these two panels, because the number of galaxies at the very bright end of the luminosity functions is very small beyond $`z0.75`$. Specifically, there are only eight galaxies in the red sample that are brighter than $`M_R=23`$ at $`1.0z1.5`$.

Figure 11 shows that $`\mathrm{}_R`$ evolves relatively slowly with redshift for different bright subsamples, and that it could have increased by at most a factor of $`6`$ from $`z1.2`$ to $`z0.3`$ for the brightest early-type sample under the most extreme stellar evolution model presented in the right panel. Taking into account the fact that these models do not have dust included, which may further reduce the amount of fading in intrinsic luminosity with redshift, we find no evidence to support that mergers dominate the formation of early-type galaxies over $`0.5z1.5`$. This is consistent with the findings of Firth et al. (2002), who directly compared the observed abundances of red galaxies at $`z1`$ with those from different semi-analytical models. They found that the models underpredict the number of red galaxies observed in the LCIR survey by at least a factor of three.

The rest-frame co-moving $`R`$-band luminosity densities for all $`H`$-band selected galaxies at different redshifts are presented in Figure 12. As in Figure 11, we calculated $`\mathrm{}_R`$ for three constant thresholds, $`M_{\mathrm{min}}=20.5`$ (squares) , $`21.0`$ (triangles), and $`21.5`$ (circles). Because galaxies in the total $`H`$-band selected sample have a wide range of star formation history, we cannot remove the effects of pure luminosity evolution by adopting a simple $`M_{\mathrm{min}}(z)`$. The results of Figure 12 show that there exists only a mild evolution in the rest-frame co-moving $`R`$-band luminosity density for the entire galaxy population over $`0.5z1.5`$.

It is also important to note that our analysis is based on an $`H`$-band selected galaxy sample, which roughly corresponds to galaxies selected in the rest-frame $`J`$, $`I`$, and $`R`$ bands at $`z0.5`$, 0.9, and 1.3, respectively. A potential systematic bias in our measurements is clearly the bandpass selection effect which, when combined with variation in star formation rate and stellar evolution with time, makes it difficult to interpret our results, particularly for the total $`H`$-band selected sample. At $`z1.3`$, the $`H`$-band survey preferentially selects optically bright (and therefore younger) galaxies, while it tends to select near-infrared bright (and therefore older) galaxies at $`z0.5`$, when the universe was twice as old. Because the total $`H`$-band selected sample consists of galaxies of different star formation histories in different redshift intervals, it is possible that galaxies of different colors, selected at different rest-frame wavelengths at different redshifts conspire to produce a luminosity function that appears to show little evolution. Consequently, it is impossible to rule out any evolution scenarios without a detailed understanding of the intrinsic properties of the galaxies in the total $`H`$-band sample. On the other hand, the bandpass selection bias has a much less serious effect for the early-type galaxy sample. Only galaxies with an observed SED (spanning a spectral range from the $`V`$ band or $`U`$ for the HDFS objects through the $`H`$ band) that is consistent or redder than a typical Sab galaxy were included in the analysis at all redshifts. Our measurements represent a true, empirical measure of how “red” galaxies evolve with time, and the results of our analysis suggest that early-type galaxies have not evolved significantly since $`z1.5`$.

### 4.2 Comparison with Previous Studies

The luminosity functions presented in § 3.6 cover the redshift range $`0.5z1.5`$. The upper end of this range extends beyond the reach of traditional spectroscopic redshift surveys, the most ambitious of which stretch to $`z=1.2`$ but become significantly incomplete beyond $`z=1`$ (Lilly et al. 1995; Cowie et al. 1996; Cohen 2002). Various groups have measured evolution in the luminosity function using spectroscopic redshifts over the redshift range $`0<z<1`$ for galaxies of different colors (e.g. Lilly et al. 1995), different spectral types (e.g. Heyl et al. 1997; Cohen 2002), or different morphology (Im et al. 1999). In addition, some groups have attempted to measure the luminosity function using photometric redshifts for galaxies at $`z4`$ (Gwyn & Hartwick 1996; Mobasher et al. 1996; Connolly et al. 1997; Sawicki et al. 1997). Different sample selection criteria make a quantitative comparison between these measurements difficult. Here we present a qualitative comparison between our measurements and the previous ones.

First, we consider previous measurements that were conducted in the rest-frame $`R`$ band. The local luminosity density is now well constrained by both the 2dF Redshift Survey (Cross et al. 2001) and the SDSS (Blanton 2001). It is encouraging that these surveys yielded consistent results in the $`B`$ band. The $`R`$-band luminosity density available from the SDSS is most directly relevant to our results from the $`H`$-selected LCIR survey, and is included in Figure 12 for the luminosity density evolution of the total $`H`$-band selected sample. If we consider only the LCIR measurements at $`0.5<z<1`$ and the SDSS measurement at $`z<0.2`$, then the evolution in the luminosity density from $`z0`$ to $`z1`$ is less than 0.2 dex for any reasonable choice of the faint-end slope, $`\alpha `$, indicating a very slow evolution.

Next, we compare our results with those from a number of very deep redshift surveys that covered a relatively smaller area but were sensitive to galaxies at redshift beyond $`z1`$. These are the Canada-France Redshift Survey (CFRS; e.g. Lilly et al. 1996), a deep $`K`$-band survey by Cowie, Songaila, & Barger (1999), and the Caltech Faint Galaxy Redshift Survey (CFGRS; Cohen et al. 2002), all of which showed that the bulk of the evolution in the luminosity function occured in bluer spectral types. In addition, they showed that galaxies with redder SEDs evolved more slowly, if at all, over this redshift range, a result with which the LCIR survey is at least qualitatively consistent. Note however that these results may change if adopting a different color selection criterion for red galaxies at high redshifts (Kauffmann, Charlot, & White 1996) or the incompleteness correction for absorption feature dominated galaxies was incorrect (Totani & Yoshii 1997), two effects that work in opposite ways.

A more quantitative comparison is possible with Lilly et al. (1996), which gave the evolution of the luminosity density between $`z=0`$ and $`z=1`$ for different CFRS samples. In particular, Figure 12 is most directly comparable to Figure 1 in Lilly et al. (1996). The CFRS luminosity densities were computed at rest-frame 2800 Å, 4400 Å, and 1 $`\mu `$m by interpolating their $`BVIK`$ photometry, which showed a decline by approximately 0.5 dex at 4400 Å and 0.4 dex at $`1\mu `$m from $`z1`$ to $`z0`$.<sup>*</sup><sup>*</sup>*We have subtracted 0.2 dex from the original measurements in Lilly et al. to account for the differences in the adopted cosmological models. This gives $`\mathrm{\Delta }\mathrm{log}\mathrm{}/\mathrm{\Delta }\mathrm{log}(1+z)=1.7\pm 0.5`$ at 4400 Å and $`1.3\pm 0.4`$ at $`1\mu `$m. We find from the LCIR survey that $`\mathrm{\Delta }\mathrm{log}\mathrm{}/\mathrm{\Delta }\mathrm{log}(1+z)=0.6\pm 0.1`$ at 6800 Å, if we assume a faint-end slope $`\alpha =1.0`$. If we assume a very steep faint-end slope $`\alpha =1.5`$, then we measure still only $`1.0\pm 0.2`$. Although our results are marginally consistent with the CFRS results, the $`H`$-selected LCIR survey appears to favor slower evolution in the overall galaxy population at $`z<1`$ (at the $`\mathrm{\hspace{0.17em}1}\sigma `$ level).

The slow evolution measured here is similar to the $`I`$-band evolution measured by Cowie, Songaila, & Barger (1999) from $`K`$-selected spectroscopic surveys at $`z<1.2`$. Based on a much larger sample covering a broader redshift range, we find that this slower evolution at red wavelengths extends to at least $`z=1.5`$. Our results are also consistent with the interpretation of the luminosity function evolution derived from the CFGRS, which represents by far the largest and most complete spectroscopic sample of galaxies at $`z>0.8`$. It is, however, still limited by small sample size (144 galaxies at $`0.8z1.05`$ and 18 galaxies at $`1.05z1.5`$). This illustrates the difficulties of carrying out a complete spectroscopic survey over a large area to very faint magnitude limits.

Next, we compare our results with those estimated based on photometric redshifts. At $`1<z<2`$, photometric redshift surveys prevail but have so far been limited to small areas of the sky. In addition, it is important to include near-infrared photometry in order for accurate photometric redshift measurements to be plausible. Connolly et al. (1997) showed that uncertainties of photometric redshifts are reduced by 40%, when incorporating near-infrared photometry in the photometric redshift analysis. Indeed, the reddest galaxies known in this reshift interval (Elston et al. 1988; McCarthy et al. 1992; Hu & Ridgeway 1994) are often exceedingly faint in all optical bandpasses bluer than the $`R`$ band, making photometric redshift measurements very uncertain. The luminosity function from Connolly et al. was for a $`J`$-band selected galaxy sample over $`1<z<2`$ in the Hubble Deep Field North (HDFN). Their luminosity function was computed, however, at rest-frame 2800 Å, which is more sensitive to young stellar populations than the rest-frame $`R`$ band. Furthermore, over the redshift range probed by the LCIR survey ($`0.5<z<1.5`$), the HDFN is primarily sensitive to the faint end of the luminosity function, while the LCIR survey is sensitive only to the bright end. Finally, the total survey area of HDFN was only 4.48 arcmin<sup>2</sup>, corresponding to a comoving volume of $`1.3\times 10^4`$ Mpc<sup>3</sup> over the $`1z2`$ interval. In contrast, our analysis is based on a survey volume that is more than two orders of magnitude larger than in the HDFN. Therefore, we have sufficient numbers of bright galaxies to $`z=1.5`$ to probe their evolution in greater detail. The larger volume is necessary to accurately measure the mean space density because the bright, red galaxies are now known to be strongly clustered (Daddi et al. 2000; McCarthy et al. 2001; Firth et al. 2002).

Other groups have attempted to study the evolution of the galaxy luminosity function at $`z1`$ based on a deep photometric redshift survey that incorporates near-infrared photometry (Fried et al. 2001; Poli et al. 2001), but these galaxy samples were selected in the $`I`$ band, which lends a bluer tint to the overall mix of SEDs than the $`H`$-selected LCIR survey galaxies. It is therefore not straightforward to compare our results with these other photometric redshift surveys.

Finally, we consider measurements obtained for early-type galaxies selected on the basis of quantitative morphological criteria such as smoothness, symmetry, and bulge-to-total ratio. Im et al. (1999) selected a sample of 145 E/S0 galaxies from deep HST imaging in the Groth Strip (Groth et al. 1994) Using a subset with spectroscopic redshifts as calibrators, they estimated redshifts for the remaining galaxies using the $`VI`$ color alone. The resulting luminosity functions (measured in the rest-frame $`B`$ band) suggest slow evolution in the E/S0 population at $`z<1`$, in qualitative agreement with the results discussed earlier. Brinchmann et al. (1998) and Menanteau et al. (1999) have, however,suggested that the bulk of morphologically selected early-type galaxies at $`z<1`$ have bluer colors than those predicted by different stellar evolution models, but the results are inconclusive because they depend sensitively on the adopted models and how photometric uncertainties are incorporated in these models.

In summary, we find that the rest-frame, co-moving optical luminosity density of the entire galaxy population has evolved only moderately (declining by a factor of $`2`$) from $`z1`$ to $`z0`$. The rest-frame, co-moving optical luminosity density of the color-selected, early-type galaxies has not evolved significantly since $`z1`$. Our new measurements of luminosity density evolution is based on a significantly larger sample selected in the near-infrared than previous surveys selected at a range of wavelengths. It is consistent with previous findings, but we place a much tighter constraint on the apparant lack of evolution in the co-moving luminosity density of early-type galaxies.

## 5 Summary

We have obtained an estimate of the galaxy luminosity function at $`z1`$ using photometric redshifts for galaxies identified in the HDFS and CDFS regions of the LCIR survey. We have also calculated the luminosity functions at redshifts between $`z0.5`$ and $`z1.0`$ using the same galaxy sample selected uniformly at near-infrared wavelengths. The luminosity function analysis is based on $`3000`$ $`H`$-band selected galaxies with apparent magnitude $`17H20`$ at $`0.5z1.5`$. We demonstrate that our photometric redshift measurements are accurate with an RMS dispersion between the photometric and spectroscopic redshifts of $`\sigma _z/(1+z)0.08`$. In addition, we show that the systematic uncertainty inherent in the luminosity function measurements due to uncertainties in photometric redshifts is non-negligible and therefore must be accounted for. We develop a technique to incorporate photometric redshift error functions of individual galaxies in the estimates of the intrinsic luminosity function. We show that this technique allows us to completely recover the galaxy luminosity function to the sensitivity limit of the survey and therefore to study the luminosity and space density evolution of near-infrared selected galaxies over a large redshift interval. The results of our study are:

1. On the basis of galaxies identified in the Hubble Deep Field South and Chandra Deep Field South regions, we find that ($`7.3\pm 0.2`$) % and ($`16.7\pm 0.4`$) %, respectively, of the $`H`$-band detected galaxies of $`H20`$ are at $`z1`$.

2. The galaxy luminosity functions determined via a modified SWML method are consistent with the Schechter functions determined via the modified STY method.

3. The best-fit $`M_R_{}`$ and $`\varphi _{}`$ for the total $`H`$-band selected population are consistent with a no-evolution model at $`0.5z1.5`$.

4. The evolution of the co-moving luminosity density of the $`H`$-band selected galaxies is characterized by $`\mathrm{\Delta }\mathrm{log}\mathrm{}/\mathrm{\Delta }\mathrm{log}(1+z)=0.7\pm 0.2`$ at rest-frame 6800 Å. This is marginally consistent with the CFRS results, but favors a slower evolution (at the $`\mathrm{\hspace{0.17em}1}\sigma `$ level).

5. Galaxies with spectral energy distributions best characterized as E/S0 or Sab appear to have faded by $`0.5`$ mag from $`z1`$ to $`z0.5`$, consistent with the expected luminosity evolution from an exponentially declining star formation rate model. In addition, the space density of the these color-selected early-type galaxies has evolved only mildly since $`z1`$ ($`40`$% decrease with redshift).

6. The derived rest-frame co-moving $`R`$-band luminosity density $`\mathrm{}_R`$ of color-selected early-type galaxies exhibits only moderate evolution with redshift. Specifically, we find that under the most extreme stellar evolution scenario the $`\mathrm{}_R`$ of early-type galaxies brighter than $`L_{}`$ ($`1.6L_{}`$) could have increased by at most a factor of $`3(6)`$ from $`z1.2`$ to $`z0.3`$. Our results place an upper limit to the merger rate of bright galaxies over this time interval.

We appreciate the expert assistance from the staffs of the Cerro Tololo Inter-American Observatory and the Las Campanas Observatory. We thank Andrew Firth, Chris Sabbey, Richard McMahon, and Kathleen Koviak for assistance with the observations. This research was supported by the National Science Foundation under grant AST-9900806. The CIRSI camera was made possible by the generous support of the Raymond and Beverly Sackler Foundation.