text:
# Ultradeep Near-Infrared ISAAC Observations of the Hubble Deep Field South: Observations, Reduction, Multicolor Catalog, and Photometric RedshiftsBased on service mode observations collected at the European Southern Observatory, Paranal, Chile (ESO Programme 164.O-0612). Also based on observations with the NASA/ESA Hubble Space Telescope, obtained at the Space Telescope Science Institute which is operated by AURA, Inc., under NASA contract NAS5-26555.

## 1 Introduction

In the past decade, our ability to routinely identify and systematically study distant galaxies has dramatically advanced our knowledge of the high-redshift universe. In particular, the efficient U-dropout technique (Steidel et al., 1996a, b) has enabled the selection of distant galaxies from optical imaging surveys using simple photometric criteria. Now more than 1000 of these Lyman break galaxies (LBGs) are spectroscopically confirmed at $`z2`$, and have been subject to targeted studies on spatial clustering (Giavalisco & Dickinson, 2001), internal kinematics (Pettini et al., 1998, 2001), dust properties (Adelberger & Steidel, 2000), and stellar composition (Shapley et al., 2001; Papovich, Dickinson, & Ferguson, 2001). Although LBGs are among the best studied classes of distant galaxies to date, many of their properties like their prior star formation history, stellar population ages, and masses are not well known.

More importantly, it is unclear if the ultraviolet-optical selection technique alone will give us a fair census of the galaxy population at $`z3`$ as it requires galaxies to have high far-ultraviolet surface brightnesses due to on-going spatially compact and relatively unobscured massive star formation. We know that there exist highly obscured galaxies, detected in sub-mm and radio surveys (Smail et al., 2000), and optically faint hard X-ray sources (Cowie et al., 2001; Barger et al., 2001) at high redshift that would not be selected as LBGs, but their number densities are low compared to LBGs and they might represent rare populations or transient evolutionary phases. In addition, the majority of present-day elliptical and spiral galaxies, when placed at $`z3`$, would not satisfy any of the current selection techniques for high-redshift galaxies. Specifically, they would not be selected as U-dropout galaxies because they are too faint in the rest-frame UV. It is much easier to detect such galaxies in the near-infrared (NIR), where one can access their rest-frame optical light. Furthermore, observations in the near-infrared allow the comparison of galaxies of different epochs at fixed rest-frame wavelengths where long-lived stars may dominate the integrated light. Compared to the rest-frame far-UV, the rest-frame optical light is less sensitive to the effects of dust extinction and on-going star formation, and provides a better tracer of stellar mass. By selecting galaxies in the near-infrared $`K_s`$-band, we expect to obtain a more complete census of the galaxies that dominate the stellar mass density in the high-redshift universe, thus tracing the build-up of stellar mass directly.

In this context we initiated the Faint InfraRed Extragalactic Survey (FIRES; Franx et al., 2000), a large public program carried out at the Very Large Telescope (VLT) consisting of very deep NIR imaging of two selected fields. We observed fields with existing deep optical WFPC2 imaging from the Hubble Space Telescope (HST): the WPFC2-field of Hubble Deep Field South (HDF-S), and the field around the $`z0.83`$ cluster MS1054-03. The addition of NIR data to the optical photometry is required not only to access the rest-frame optical, but also to determine the redshifts of faint galaxies from their broadband photometry alone. While it may be possible to go to even redder wavelengths from the ground, the gain in terms of effective wavelength leverage is less dramatic compared to the threefold increase going from the $`I`$ to $`K`$-band. This is because the $`K_s`$-band is currently the reddest band where achievable sensitivity and resolution are reasonably comparable to deep space-based optical data. Preliminary results from this program were presented by Rudnick et al. (2001, hereafter R01).

Here we present the full NIR data set of the HDF-S, together with a $`K_s`$-selected multicolor catalog of sources in the HDF-S with seven-band optical-to-infrared photometry (covering $`0.32.2\mathrm{\mu m}`$), unique in its image quality and depth. This paper focusses on the observations, data reduction and characteristic properties of the final images. We also describe the source detection and photometric measurement procedures and lay out the contents of the catalog, concluding with the NIR number counts, color distributions of sources, and their photometric redshifts. The results of the MS1054-03 field will be presented by Förster Schreiber et al. (2002) and a more detailed explanation of the photometric redshift technique can be found in Rudnick et al. (2001, 2002b). Throughout this paper, all magnitudes are expressed in the AB photometric system (Oke, 1971) unless explicitly stated otherwise.

## 2 Observations

### 2.1 Field Selection and Observing Strategy

The high Galactic latitude field of the HDF-S is a natural choice for follow-up in the near-infrared given the existing ultradeep WFPC2 data in four optical filters (Williams et al., 1996, 2000; Casertano et al., 2000). The Hubble Deep Fields (North and South) are specifically aimed at constraining cosmology and galaxy evolution models, and in these studies it is crucial to access rest-frame optical wavelengths at high redshift through deep infrared observations. Available ground-based NIR data from SOFI on the NTT (da Costa et al., 1998) are not deep enough to match the space-based data. To fully take advantage of the deep optical data requires extremely deep wide-field imaging in the infrared at the best possible image quality; a combination that in the southern hemisphere can only be delivered by the Infrared Spectrometer And Array Camera (ISAAC; Moorwood, 1997), mounted on the Nasmyth-B focus of the 8.2 meter VLT Antu telescope. The infrared camera has a $`2.5\mathrm{}\times 2.5\mathrm{}`$ field of view similar to that of the WFPC2 ($`2.7\mathrm{}\times 2.7\mathrm{}`$). ISAAC is equipped with a Rockwell Hawaii $`1024\times 1024`$ HgCdTe array, offering imaging with a pixel scale of $`0\stackrel{}{\mathrm{.}}147`$ pix-1 in various broad and narrow band filters.

Our NIR imaging consists of a single ISAAC pointing centered on the WFPC2 main-field of the HDF-S ($`\alpha =22^h32^m55.464`$, $`\delta =60\mathrm{°}33\mathrm{}05.01\mathrm{}`$, J2000) in the $`J_s`$, $`H`$ and $`K_s`$ filters, which gives good sampling of rest-frame optical wavelengths over the redshift range $`1<z<4`$. The $`J_s`$ filter is being established as the new standard broadband filter at $`1.24\mathrm{\mu m}`$ by most major observatories (Keck, Gemini, Subaru, ESO), and is photometrically more accurate than the classical $`J`$ because it is not cut off by atmospheric absorption. It is a top-hat filter with sharp edges, practically the same effective wavelength as the normal $`J`$ filter, and half-transmittance points at $`1.17\mathrm{\mu m}`$ and $`1.33\mathrm{\mu m}`$. We used the $`K_s`$ filter which is bluer and narrower than standard $`K`$, but gives a better signal-to-noise ratio (SNR) for faint sources because it is less affected by the high thermal background of the atmosphere and the telescope. The ISAAC $`H`$ and $`K_s`$ filters are close to those used to establish the faint IR standard star system (Persson et al., 1998), while the $`J_s`$ filter requires a small color correction. The WFPC2 filters that are used are $`F300W,F450W,F606W`$ and $`F814W`$ which we will call $`U_{300}`$, $`B_{450}`$, $`V_{606}`$ and $`I_{814}`$, respectively, where the subscript indicates the central wavelength in nanometers.

The observing strategy for the HDF-S follows established procedures for ground-based NIR imaging. The dominance of the sky background and its rapid variability in the infrared requires dithering of many short exposures. We used a $`20\mathrm{}`$ jitter box in which the telescope is moved in a random pattern of Poissonian offsets between successive exposures. This jitter size is a trade-off between keeping a large area at maximum depth and ensuring that each pixel has sufficient exposures on sky. Individual exposures have integration times of $`6\times 30`$s in $`J_s`$, $`6\times 20`$s in $`H`$, and $`6\times 10`$s in $`K_s`$ (subintegrations $`\times `$ detector integration times). We requested service mode observations amounting to 32 hours in each band with a seeing requirement of $`0\stackrel{}{\mathrm{.}}5`$, seeing conditions that are only available 25% of the time at Paranal. The observations were grouped in $`112`$ observation blocks (OBs), each of which uniquely defines a single observation of a target, including pointing, number of exposures in a sequence, and filter. The calibration plan for ISAAC provides the necessary calibration measurements for such blocks, including twilight flats, detector darks, and nightly zero points by observing LCO/Palomar NICMOS standard stars (Persson et al., 1998).

### 2.2 Observations

The HDF-S was observed from October to December 1999 and from April to October 2000 under ESO program identification 164.O-0612(A). A summary of the observations is shown in 1. We obtained a total of $`33.6`$, $`32.3`$ and $`35.6`$ hours in $`J_s`$, $`H`$ and $`K_s`$, distributed over $`33,34`$ and $`55`$ OBs, or $`1007,968`$ and $`2136`$ frames, respectively. This represents all usable data, including aborted and re-executed OBs that were outside weather specifications or seeing constraints. In the reduction process these data are included with appropriate weighting (see section 3.4). Sixty-eight percent of the data was obtained under photometric conditions and the average airmass of all data was 1.25. A detailed summary of observational parameters with pointing, observation date, image quality and photometric conditions can be found on the FIRES homepage on the World Wide Web (http://www.strw.leidenuniv.nl/{̃}fires).

An analysis of various observational parameters reveals some surprising trends in the data, whereas other expected relations are less apparent. An overview is given in Figure 1. The median seeing on the raw images is better than 0$`\stackrel{}{\mathrm{.}}`$5 in all bands, with the seeing of 90% of the images in the range $`0\stackrel{}{\mathrm{.}}40\stackrel{}{\mathrm{.}}65`$, as can be seen in Figure 1.

Seeing may vary strongly on short timescales but it is not related to any other parameter. The most drastic trend in the raw data is the change of sensitivity with date. Since the cleaning and re-alumization of the primary mirror in March 2000 the count rates of bright stars within a $`3\mathrm{}`$ aperture increased by $`+29`$% in $`J_s`$, $`+45`$% in $`H`$, and $`+45`$% $`K_s`$, which is reflected by a change in zero points before and after this date. Because the average NIR sky levels remained the same, this increase proportionally improved the achievable signal-to-noise for background-limited sources. The change in throughput was caused by light scattering, which explains why the sky level remained constant. Sky levels in $`J_s`$ and $`H`$, dominated by airglow from OH-emission lines in the upper atmosphere (typically 90 km altitude), vary unpredictably on the timescale of minutes, but also systematically with observed hour. The average sky level is highest at the beginning and end of each night with peak-to-peak amplitudes of the variation being 50% relative to the average sky brightnesses over the night. The background in $`K_s`$ is dominated by thermal emission of the telescope, instrument, and atmosphere and is mainly a function of temperature. The $`K_s`$ background is the most stable of all NIR bands and only weakly correlated with airmass; our data do not show a strong thermal atmospheric contribution, which should be proportional to atmospheric path length. We take into account the variations of the background and seeing through weighting in the data coadding process.

## 3 Data Reduction

The reduction process included the following steps: quality verification, flat-fielding, bad pixel correction, sky subtraction, distortion correction, registration, photometric calibration and weighting of individual frames, and combination into a single frame. We used a modified version of the DIMSUM111DIMSUM is the Deep Infrared Mosaicing Software package developed by Peter Eisenhardt, Mark Dickinson, Adam Stanford, and John Ward, and is available via ftp to ftp://iraf.noao.edu/contrib/dimsumV2/ package and standard routines in IRAF222IRAF is distributed by the National Optical Astronomy Observatories, which are operated by the AURA, Inc., under cooperative agreement with the NSF. for sky subtraction and coadding, and the ECLIPSE333ECLIPSE is an image processing package written by N. Devillard, and is available at ftp://ftp.hq.eso.org/pub/eclipse/ package for creating the flatfields and the initial bad pixel masks. We reduced the ISAAC observations several times with an increasing level of sophistication, applying corrections to remove instrumental features, scattered light, or clear artifacts when required. Here we describe the first version of the reduction (v1.0) and the last version (v3.0), leaving out the intermediate trial versions. The last version produced the final $`J_s`$, $`H`$ and $`K_s`$ images, on which the photometry (see section 5) and analysis (see section 8) is based.

### 3.1 Flatfields and Photometric Calibration

We constructed flatfields from images of the sky taken at dusk or dawn, grouped per night and in the relevant filters, using the flat routine in ECLIPSE, which also provided the bad pixel maps. We excluded a few flats of poor quality and flats that exhibited a large jump between the top row of the lower and bottom row of the upper half of the array, possibly caused by the varying bias levels of the Hawaii detector. We averaged the remaining nightly flats per month, and applied these to the individual frames of the OBs taken in the same month. If no flatfield was available for a given month we used an average flat of all months. The stability of these monthly flats is very good and the structure changes little and in a gradual way. We estimate the relative accuracy to be $`0.20.4`$% per pixel from the pixel-to-pixel rms variation between different monthly flats. Large scale gradients in the monthly flats do not exceed 2%. We checked that standard stars, which were observed at various locations on the detector, were consistent within the error after flatfielding.

Standard stars in the LCO/Palomar NICMOS list (Persson et al., 1998) were observed each night, in a wide five-point jitter pattern. For each star, on each night, and in each filter, we measured the instrumental counts in a circular aperture of radius $`20`$ pixel ($`2\stackrel{}{\mathrm{.}}94`$) and derived zero points per night from the magnitude of that star in the NICMOS list. We identify non-photometric nights after comparison with the median of the zero points over all nights before and after re-aluminization in March 2000 (see section 2.2). The photometric zero points exhibit a large increase after March 2000 but, apart from this, the night-to-night scatter is approximately 2%. We adopted the mean of the zero points after March 2000 as our reference value. See Table 2 for the list of the adopted zero points. By applying the nightly zero points to 4 bright unsaturated stars in the HDF-S, observed on the same night under photometric conditions, we obtain calibrated stellar magnitudes with a night-to-night rms variation of only $`11.5`$%. No corrections for atmospheric absorption were required because the majority of the science data were obtained at similar airmass as the standard star observations. In addition, instrumental count rates of HDF-S stars in individual observation blocks reveal no correlation with airmass. We used the calibrated magnitudes of the 4 reference stars, averaged over all photometric nights, to calibrate every individual exposure of the photometric and non-photometric OBs. The detector non-linearity, as described by Amico et al. (2001), affects the photometric calibration by $`1`$% in the $`H`$-band, where the exposure levels are highest. Because the effect is so small, we do not correct for this. We did not account for color terms due to differences between the ISAAC and standard filter systems. Amico et al. (2001) report that the ISAAC $`H`$ and $`K_s`$ filters match very well those used to establish the faint IR standard star system of Persson et al. (1998). Only the ISAAC $`J_s`$ filter is slightly redder than Persson’s $`J`$ and this may introduce a small color term, $`0.04(JK)_{LCO}`$. However, the theoretical transformation between ISAAC magnitudes and those of LCO/Palomar have never been experimentally verified. Furthermore, the predicted color correction is small and could not be reproduced with our data. In the absence of a better calibration we chose not to apply any color correction. We did apply galactic extinction correction when deriving the photometric redshifts, see section 6, but it is not applied to the catalog.

As a photometric sanity check, we compared $`2\mathrm{}`$ circular diameter aperture magnitudes of the brightest stars in the final (version 3.0, described below) images to magnitudes based on a small fraction of the data presented by R01. Each data set was independently reduced, the calibration based on different standard stars, and the shallower data were obtained before re-aluminization of the primary mirror. The magnitudes of the brightest sources in all bands agree within 1% between the versions, indicating that the internal photometric systematics are well under control. For the NIR data, the adopted transformations from the Johnson (1966) system to the AB system are taken from Bessell & Brett (1988) and we apply $`J_{s,AB}=J_{s,Vega}+0.90,H_{AB}=H_{Vega}+1.38`$ and $`K_{s,AB}=K_{s,Vega}+1.86`$.

### 3.2 Sky Subtraction and Cosmic Ray Removal

The rapidly varying sky, typically 25 thousand times brighter than the sources we aim to detect, is the primary limiting factor in deep NIR imaging. In the longest integrations, small errors in sky subtraction can severely diminish the achievable depth and affect faint source photometry. The IRAF package DIMSUM provides a two-pass routine to optimally separate sky and astronomical signal in the dithered images. We modified it to enable handling of large amounts of data and replaced its co-adding subroutine, which assumes that the images are undersampled, by the standard IRAF task IMAGES.IMMATCH.IMCOMBINE. The following is a brief summary of the steps performed by the REDUCE task in DIMSUM.

For every science image in a given OB a sky image is constructed. After scaling the exposures to a common median, the sky is determined at each pixel position from a maximum of 8 and a minimum of 3 adjacent frames in time. The lowest and highest values are rejected and the average of the remainder is taken as the sky value. These values are subtracted from the scaled image to create a sky subtracted image. A set of stars is then used to compute relative shifts, and the images are integer registered and averaged to produce an intermediate image. All astronomical sources are identified and a corresponding object mask is created. This mask is used in a second pass of sky subtraction where pixels covered by objects are excluded from the estimate of the sky.

The images show low-level pattern due to bias variations. Because they generally reproduce they are automatically removed in the skysubtraction step. We find cosmic rays with DIMSUM, using a simple threshold algorithm and replacing them by the local median, unless a pixel is found to have cosmic rays in more than frames per OB. In this case the pixel is added to the bad pixel map for that OB.

### 3.3 First Version and Quality Verification

The goal of the first reduction of the data set is to provide a non-optimized image, which we use to validate and to assess the improvements from more sophisticated image processing. The first version consists of registration on integer pixels and combination of the sky subtracted exposures per OB. For each of the $`122`$ OBs, we created an average and a median combined image to verify that cosmic rays and other outliers were removed correctly, and we visually inspected all $`4149`$ individual sky subtracted frames as well, finding that many required further processing as described in the following section. Finally, we generated the version 1.0 images (the first reduction of the full data set) by integer pixel shifting all OBs to a common reference frame, and coaveraging them into the $`J_s`$, $`H`$ and $`K_s`$ images. While this first reduction is not optimal in terms of depth and image quality, it is robust owing to its straightforward reduction procedure.

### 3.4 Additional Processing and Improvements

The individual sky subtracted frames are affected by a number of problems or instrumental features, which we briefly describe below, together with the applied solutions and additional improvements that lead to the version 3.0 images. The most important problems are:

* Detector bias residuals, most pronounced at the rows where the read-out of the detector starts at the bottom (rows $`1,2,\mathrm{}`$) and halfway (rows $`513,514,\mathrm{}`$), caused by the complex bias behaviour of the Rockwell Hawaii array. These variations are uniform along rows, and we removed the residual bias by subtracting the median along rows in individual sky subtracted exposures, after masking all sources.
* Imperfect sky subtraction, caused by stray light or rapid background variations. Strong variations in the backgrounds, reflection from high cirrus, reflected moonlight in the ISAAC optics or patterns of less obvious origin can lead to large scale residuals in the sky subtraction, particularly in $`J_s`$ and $`H`$. For some OBs, we succesfully removed the residual patterns by splitting the sequence in two (in case of a sudden appearance of stray light), or subtracting a two-piece cubic spline fit along rows and columns to the background in individual frames, after masking all sources. We rejected a few frames, or masked the affected areas, if this simple solution did not work.
* Unidentified cosmic rays or bad pixels. A small amount of bad pixels were not detected by ECLIPSE or DIMSUM routines but need to be identified because we average the final images without additional clipping or rejection. By combining the sky subtracted frames in a given OB without shifts and with the sources masked, we identified remaining cosmic rays or outliers through sigma clipping. We added $`60100`$ pixels per OB to the corresponding bad pixel map.

Several steps were taken to improve the quality and limiting depths of the version 1.0 images, the most important of which are:

* Distortion correction of the individual frames and direct registration to the $`3\times 3`$ blocked $`I_{814}`$ image ($`0.119\mathrm{}`$pixel-1), our preferred frame of reference. We obtained the geometrical distortion coefficients for the 3rd order polynomial solution from the ISAAC WWW-page444ISAAC home page: http://www.eso.org/instruments/isaac. The transformation procedure involves distortion correcting the ISAAC images, adjusting the frame-to-frame shifts, and finding the linear transformation to the WFPC2 $`I_{814}`$ frame of reference. This linear transformation is the best fit mapping of source positions in the blocked WFPC2 $`I_{814}`$ image to the corresponding positions in the corrected $`J_s`$-band image555We have noticed that the mapping solution changed slightly after the remount of ISAAC in March 2000, implying a 0.1% scale difference.. Compared to version 1.0 described in the previous section, this procedure increases registration accuracy and image quality, decreases image smearing at the edges introduced by the jittering and differential distortion. Given the small amplitude of the ISAAC field distortions, the effect on photometry is negligible. In the linear transformation and distortion correction step the image is resampled once using a third-order polynomial interpolation, with a minimal effect of the interpolant on the noise properties.
* Weighting of the images. We substantially improved the final image depth and quality by assigning weights to individual frames that take into account changes in seeing, sky transparency, and background noise. Two schemes were applied: one that optimizes the signal-to-noise ratio (SNR) within an aperture of the size of the seeing disk, and one that optimizes the SNR per pixel. The first improves the detection efficiency of point sources, the other optimizes the surface brightness photometry. The weights $`w_i`$ of the frames are proportional to either the inverse scaled variance $`zpscale_i\times var_i`$ within a seeing disk of size $`s_i`$, or to the inverse scaled variance per pixel, where the scaling $`zpscale_i`$ is the flux calibration applied to bring the instrumental counts of our four reference stars in the HDF-S to the calibrated magnitude.

$$w_{i,point}(zpscale_i\times var_i\times s_i^2)^1$$
(1)
$$w_{i,extended}(zpscale_i\times var_i)^1$$
(2)

### 3.5 Final Version and Post Processing

The final combined $`J_s`$, $`H`$, and $`K_s`$ images (version 3.0) were constructed from the individually registered, distortion corrected, weighted and unclipped average of the $`1007,968`$, and $`2136`$ NIR frames respectively. Ultimately, less than 3% of individual frames were excluded in the final images because of poor quality. In this step we also generated the weight maps, which contain the weighted exposure time per pixel. We produced three versions of the images, one with optimized weights for point sources, one with optimized weights for surface brightness, and one consisting of the best quartile seeing fraction of all exposures, also optimized for point sources. The weighting has improved the image quality by 10–15% and the background noise by 5–10%, and distortion correction resulted in subpixel registration accuracy between the NIR images and $`I_{814}`$-band image over the entire field of view.

The sky subtraction routine in DIMSUM and our additional fitting of rows and columns (see section 3.4) have introduced small negative biases in combined images, caused by systematic oversubtraction of the sky which was skewed by light of the faint extended PSF wings or very faint sources, undetectable in a single OB. Because of this, the flatness of the sky on large scales was limited to about 10-5. The negative bias was visible as clearly defined orthogonal stripes at P.A.$`6\mathrm{°}`$, as well as dark areas around the brightest stars or in the crowded parts of the images. To solve this, we rotated a copy of the final images back to the orientation in which we performed sky substraction, fitted a 3-piece cubic spline to the background along rows and columns (masking all sources), re-rotated the fit, and subtracted it. The sky in the final images is flat to a few $`\times 10^6`$ on large ($`>20\mathrm{}`$) scales.

## 4 Final Images

The reduced NIR $`J_s`$, $`H`$, and $`K_s`$ images and weight maps can be obtained from the FIRES-WWW homepage (http://www.strw.leidenuniv.nl/{̃}fires). Throughout the rest of the paper we will only consider the images optimized for point source detection which we will use to assemble the catalog of sources.

### 4.1 Properties

The pixel size in the NIR images equals that of the $`3\times 3`$ blocked WFPC2 $`I_{814}`$-band image at $`0.119\mathrm{}`$ pixel-1. The combined ISAAC images are aligned with the HST version 2 images (Casertano et al., 2000) with North up, and are normalized to instrumental counts per second. The images are shallower near the edges of the covered area because they received less exposure time in the dithering process, which is reflected in the weight map containing the fraction of total exposure time per pixel. The area of the ISAAC $`K_s`$-band image with weight per pixel $`w_K0.95,0.2,`$ and $`0.01`$ covers $`4.5,7.2`$ and $`8.3`$ arcmin2, while the area used for our preferred quality cut for photometry ($`w0.2`$ in all seven bands) is $`4.7`$ arcmin2. The NIR images have been trimmed where the relative exposure time per pixel is less than 1%.

Figure 2 shows the noise-equalized $`K_s`$-detection image obtained by division with the square root of the exposure-time map. The richness in faint compact sources and the flatness of the background are readily visible. Figure 3 shows a RGB color composite image of the $`I_{814}`$, $`J_s`$, and $`K_s`$ images. The PSF of the space-based $`I_{814}`$ image has been matched to that of the NIR images at FWHM$`0\stackrel{}{\mathrm{.}}46`$ (see section 4.2) and three adjacent WFPC2 $`I_{814}`$ flanking fields have been included for visual purposes. We set the linear stretch of both images to favor faint objects. Immediately striking is the rich variety in optical-NIR colors, even for the faint objects, indicating that the NIR observations are very deep and that there is a wide range of observed spectral shapes, which can result from different types of galaxies over a broad redshift range.

### 4.2 Image Quality

The NIR PSF is stable and symmetric over the field with a gaussian core profile and an average ellipticity $`<0.05`$ over the $`J_s`$, $`H`$, and $`K_s`$ images. The median FWHM of the profiles of ten selected isolated bright stars is $`0\stackrel{}{\mathrm{.}}45`$ in $`J_s`$, $`0\stackrel{}{\mathrm{.}}48`$ $`H`$, and $`0\stackrel{}{\mathrm{.}}46`$ $`K_s`$ with $`0\stackrel{}{\mathrm{.}}04`$ amplitude variation over the images.

For consistent photometry in all bands we convolved the measurement images to a common PSF, corresponding to that of the $`H`$-band which had worst effective seeing (FWHM $`=0\stackrel{}{\mathrm{.}}48`$). The similarity of PSF structure across the NIR images allowed simple gaussian smoothing for a near perfect match. The complex PSF structure of the WFPC2 requires convolving with a special kernel, which we constructed by deconvolving an average image of bright isolated non-saturated stars in the $`H`$-band with the average $`I_{814}`$-band image of the same stars. Division of the stellar growth curves of the convolved images by the $`H`$-band growth curve shows that the fractional enclosed flux agrees to within 3% at radii $`r0\stackrel{}{\mathrm{.}}35`$.

### 4.3 Astrometry

The relative registration between ISAAC and WFPC2 images needs to be very precise, preferably a fraction of an original ISAAC pixel over the whole field of view, to allow correct cross-identification of sources, accurate color information and morphological comparison between different bands. To verify our mapping of ISAAC to WFPC2 coordinates, we measured the positions of the 20 brightest stars and compact sources in all registered ISAAC exposures, and we compared their positions with those in the $`I_{814}`$ image. The rms variation in position of individual sources is about $`0.20.3`$ pixel at $`0.119\mathrm{}`$ pixel-1 ($`2535`$ mas), but for some sources systematic offsets between the NIR and the optical up to $`0.85`$ pixel ($`100`$ mas) remain. The origin of the residuals is unclear and we cannot fit them with low order polynomials. They could be real, intrinsic to the sources, or due to systematic errors in the field distortion correction of ISAAC or WFPC2666The ISAAC field distortion might have changed over the years, but this cannot be checked because recent distortion measurements are unavailable. The worst case errors of relative positions across the four WFPC2 chips can be $`0\stackrel{}{\mathrm{.}}1`$ (Voit et al., 1997), but is expected to be smaller for the HDF-S images.. However, for all our purposes, the effect of positional errors of this amplitude is unimportant. The error in absolute astrometry of the HST HDF-S coordinate system, estimated to be less than $`40`$ mas, is dominated by the systematic uncertainty in the positions of four reference stars (Casertano et al., 2000; Williams et al., 2000).

### 4.4 Backgrounds and Limiting Depths

The noise properties of the raw individual ISAAC images are well described by the variance of the signal collected in each pixel since both Poisson and read noise are uncorrelated. However, image processing, registration and combination have introduced correlations between neighbouring pixels and small errors in the background subtraction may also contribute to the noise. Understanding the noise properties well is crucial because limiting depths and photometric uncertainties rely on them.

Instead of a formal description based on the analysis of the covariance of correlated pixel pairs, we followed an empirical approach where we fit the dependence of the rms background variation in the image as a function of linear size $`N=\sqrt{A}`$ of apertures with area $`A`$. Directly measuring the effective flux variations in apertures of different sizes provides a more realistic estimate of signal variations than formal Gaussian scaling $`\sigma (N)=N\overline{\sigma }`$ of the pixel-to-pixel noise $`\overline{\sigma }`$, as is often done.

We measured fluxes in $`1200`$ non-overlapping circular apertures randomly placed on the registered convolved images, which were also used for photometry. We excluded all pixels belonging to sources detectable in $`K_s`$ at the 5$`\sigma `$ level (see section 5.1 for detection criteria). We used identical aperture positions for each band $`i`$ and measured fluxes for circular aperture diameters ranging from $`0\stackrel{}{\mathrm{.}}5`$ to $`3\mathrm{}`$. Then we obtained the flux dispersions by fitting a Gaussian distribution to the histogram of fluxes at each aperture size. Finally, we fitted a parameterized function of linear size to the different dispersions:

$$\sigma _i(N)=N\overline{\sigma }_i(a_i+b_iN)/\sqrt{w_i}$$
(3)
This equation describes the signal variation versus aperture size $`N`$ over the entire image, taking into account spatial variations as a result of relative weight $`w_i`$ for each passband $`i`$. As can be seen in Figure 4, it provides a good fit to the noise characteristics. The noise is significantly higher than expected from uncorrelated (Gaussian) noise, indicated by a dashed line in Figure 4b. Table 3 shows the best fit values in all bands and the corresponding limiting depths. The parameter $`a`$ reflects the correlations of neighbouring pixels ($`a>1`$), which is important in the WFPC2 images because of heavy smoothing, but also in the ISAAC images given the resampling from $`0\stackrel{}{\mathrm{.}}147`$ to $`0\stackrel{}{\mathrm{.}}119`$ pixel-1. The parameter $`b`$ accounts for large scale correlated variations in the background ($`b>0`$). This may be caused by the presence of sources at very faint flux levels (confusion noise) or instrumental features. Typically, the large scale correlated contribution per pixel is only 3-15% relative to the gaussian rms variation, but due to the $`N^2`$ proportionality the contribution to the variation in large apertures increases to significant levels. While the signal variations grow faster with area than expected from a Gaussian, at any specific scale the variation is consistent with a pure Gaussian.

From the analysis of the scaling relation of simulated colors we find that part of the large scale irregularities in the background are spatially correlated between bands. In particular, we measured the rms variation of the $`I_{814}V_{606}`$ colors directly by subtracting in registered apertures the $`I_{814}`$-band fluxes from the $`V_{606}`$ fluxes and fitting the dispersion of the difference at each linear size. On large scales rms variations are 30% smaller than predicted from Eq. 3 if the noise were uncorrelated. Yet, if we subtract the two fluxes in random apertures, the scaling of the background variation is consistent with the prediction. A similar effect is seen for the $`I_{814}J_s`$ color, but at a smaller amplitude. The spatial coherence of the background variations between filters and across cameras suggests that part of the background fluctuations may be associated with sources at very faint flux levels. Other contributions are likely similar flatfielding or skysubtraction residuals from one band to another.

## 5 Source Detection and Photometry

The detection of sources at very faint magnitudes against a noisy background forces us to trade off completeness and reliability. A very low detection threshold may generate the most complete catalog, but we must then apply additional criteria to assess the reliability of each detection given that such a catalog will contain many spurious sources. More conservatively, we choose the lowest possible threshold for which contamination by noise is unimportant. We aim to produce a catalog with reliable colors suitable for robustly modeling of the intrinsic spectral energy distribution. Using SExtractor version 2.2.2 (Bertin & Arnouts, 1996) with a detection procedure that optimizes sensitivity for point-like sources, we construct a $`K_s`$-band selected catalog with seven band optical-to-infrared photometry.

### 5.1 Detection

To detect objects with SExtractor using a constant signal-to-noise criterion over the entire image, including the shallower outer parts, we divide the point source optimized $`K_s`$-image by the square root of the weight (exposure time) map to create a noise-equalized detection image. A source enters the catalog if, after low-pass filtering of the detection image, at least one pixel is above $`5`$ times the standard deviation of the filtered background, corresponding to a total $`K_s`$-band magnitude limit for point sources of $`K_s26.0`$. This depth is reached for the central $`4.5`$ arcmin2. In total we have 833 detections in the entire survey area of $`8.3`$ arcmin2. Initially 820 sources are found but the detection software fails to detect sources lying in the extended wings of the brightest objects. To include these, we fit the surface brightness profiles of the brightest sources with the GALPHOT package (Franx et al., 1989) in IRAF, subtract the fit, and carry out a second detection pass with identical parameters. Thirteen new objects enter the catalog, and 9 sources detected in the first pass are replaced with improved photometry. The catalog identification numbers of all second-pass objects start at 10001, and the original entries of the updated sources are removed.

Filtering affects only the detection process and the isophotal parameters; other output parameters are affected only indirectly through barycenter and object extent. We chose a simple two-dimensional gaussian detection filter (FWHM$`=0\stackrel{}{\mathrm{.}}46`$), approximating the core of the effective $`K_s`$-band PSF well. Hence, we optimize detectability for point-like sources, introducing a small bias against faint extended objects. In principle it is possible to combine multiple catalogs created with different filter sizes but merging these catalogs consistently is a complicated and subjective process yielding a modest gain only in sensitivity for larger objects. We prefer the small filter size equal to the PSF in the detection map because the majority of faint sources that we detect are compact or unresolved in the NIR and because we wish to minimize the blending effect of filtering on the isophotal parameters and on the confusion of sources. SExtractor applies a multi-thresholding technique to separate overlapping sources based on the distribution of the filtered $`K_s`$-band light. About 20% of the sources are blended because of the low value of the isophotal threshold; in SExtractor this must always equal the detection threshold. With the deblending parameters we used, the algorithm succeeds in splitting close groups of separate galaxies, without “oversplitting” galaxies with rich internal structure.

We tested sensitivity to false detections by running SExtractor on a specially constructed $`K_s`$-band noise map created by subtracting, in pairs, individual $`K_s`$-band images of comparable seeing, after zero point scaling, and coaveraging the weighted difference images. This noise image has properties very similar to the noise in the original reduced image, including contributions from the detector and reduction process, but with no trace of astronomical sources. Our detection algorithm resulted in only 11 spurious sources over the full area.

### 5.2 Optical and NIR Photometry

We use SExtractor’s dual-image mode for spatially accurate and consistent photometry, where objects are detected and isophotal parameters are determined from the $`K_s`$-band detection image while the fluxes in all seven bands are measured in the registered and PSF matched images. We used fluxes measured in circular apertures $`APER(D)`$ with fixed-diameters $`D`$, isophotal apertures $`APER(ISO)`$ determined by the $`K_s`$-band detection isophote at the $`5\sigma `$ detection threshold, and $`APER(AUTO)`$ (autoscaling) apertures inspired by Kron (1980), which scales an elliptical aperture based on the first moments of the $`K_s`$-band light distribution. We select for each object the best aperture based on simple criteria to enable detailed control of photometry. We define two types of measurements:

* “color” flux, to obtain consistent and accurate colors. The optimal aperture is chosen based on the $`K_s`$ flux distribution, and this aperture is used to measure the flux in all other bands.
* “total” flux, only in the $`K_s`$-band, which gives the best estimate of the total $`K_s`$ flux.

For both measurements we treat blended sources differently from unblended ones, and consider a source blended when its BLENDED or BIAS flag is set by SExtractor, as described in Bertin & Arnouts (1996).

Our color aperture is chosen as follows, introducing the equivalent of a circular isophotal diameter $`D_{iso}=2\sqrt{A_{iso}/\pi }`$ based on $`A_{iso}`$, the measured non-circular isophotal area within the detection-isophote:

$$\begin{array}{ccc}\text{if the source is }\text{not}\text{ blended}\hfill & & \\ & & \\ APER(COLOR)=\{\begin{array}{cc}APER(ISO)\hfill & (0\stackrel{}{\mathrm{.}}7<D_{iso}<2\stackrel{}{\mathrm{.}}0)\hfill \\ APER(0\stackrel{}{\mathrm{.}}7)\hfill & (D_{iso}0\stackrel{}{\mathrm{.}}7)\hfill \\ APER(2\stackrel{}{\mathrm{.}}0)\hfill & (D_{iso}2\stackrel{}{\mathrm{.}}0)\hfill \end{array}\hfill & & \\ & & \\ \text{else if the source is blended}\hfill & & \\ & & \\ APER(COLOR)=\{\begin{array}{cc}APER(D_{iso}/s)\hfill & (0\stackrel{}{\mathrm{.}}7<D_{iso}/s<2\stackrel{}{\mathrm{.}}0)\hfill \\ APER(0\stackrel{}{\mathrm{.}}7)\hfill & (D_{iso}/s0\stackrel{}{\mathrm{.}}7)\hfill \\ APER(2\stackrel{}{\mathrm{.}}0)\hfill & (D_{iso}/s2\stackrel{}{\mathrm{.}}0)\hfill \end{array}\hfill & & \end{array}$$
(4)
The parameter $`s`$ is the factor with which we shrink the circular apertures centered on blended sources, increasing the separation to the blended neighbour such that mutual flux contamination is minimal. This factor depends on the data set, and for our ISAAC $`K_s`$ image we find that $`s=1.4`$ is most successful. The smallest aperture considered, APER($`0\stackrel{}{\mathrm{.}}7`$), $`1.5`$ FWHM of the effective PSF, optimizes the S/N for photometry of point sources in unweighted apertures and prevents smaller more error-prone apertures. The largest allowed aperture, APER($`2\stackrel{}{\mathrm{.}}0`$), prevents large and inaccurate isophotal apertures driven by the filtered $`K_s`$-light distribution. We continuously assessed the robustness and quality of color flux measurements by inspecting the fits of redshifted galaxy templates to the flux points, as described in detail in $`\mathrm{\S }`$6.

We calculate the total flux in the $`K_s`$-band from the flux measured in the AUTO aperture. We define a circularized AUTO diameter $`D_{auto}=2\sqrt{A_{auto}/\pi }`$ with $`A_{auto}`$ the area of the AUTO aperture, and define the total magnitude as:

$$\begin{array}{ccc}\text{if the source is }\text{not}\text{ blended}\hfill & & \\ & & \\ APER(TOTAL)=APER(AUTO)\hfill & & \\ & & \\ \text{else if the source is blended}\hfill & & \\ & & \\ APER(TOTAL)=APER(COLOR)\hfill & & \end{array}$$
(5)
Finally, we apply an aperture correction using the growth curve of brighter stars to correct for the flux lost because it fell outside the “total” aperture. This aperture correction is necessary because it is substantial for our faintest sources, as shown in Figure 5 where we compare different methods to estimate magnitude. The aperture correction reaches 0.7 mag at the faint end, therefore magnitudes are seriously underestimated if the aperture correction is ignored.

We derive the $`1\sigma `$ photometric error for all measurements from Eq. 3 with the best-fit values shown in Table 3. These errors may overestimate the uncertainty in colors of adjacent bands (see section 4.4) but it should represent well the photometric error over the entire $`0.3\mu 2.2\mathrm{\mu m}`$ wavelength range. The magnitudes may suffer from additional uncertainties because of surface brightness biases or possible biases in the sky subtraction procedure which could depend on object magnitude and size.

## 6 Photometric Redshifts

To physically interpret the seven-band photometry for our $`K_s`$-band selected sample, we use a photometric redshift ($`z_{phot}`$) technique explained in detail by R01. In summary, we correct the observed flux points for galactic extinction (see Schlegel, Finkbeiner, & Davis 1998) and we model the rest-frame colors of the galaxies by fitting a linear combination of redshifted empirical galaxy templates. The redshift with the lowest $`\chi ^2`$ statistic, where

$$\chi ^2(z)=\underset{i=1}{\overset{N_{filter}}{}}\left[\frac{F_i^{data}F_i^{model}}{\sigma _i^{data}}\right]^2$$
(6)
is then chosen as the most likely $`z_{phot}`$. Using a linear combination of SEDs as $`F^{model}`$ minimizes the a priori assumptions about the nature and stellar composition of the detected sources.

Our data set with three deep NIR bands samples the position of Balmer/4000Å break over $`1z4`$, allowing us to probe the redshift distribution of more evolved galaxy types that may have little rest-frame UV flux and hence a weak or virtually absent Lyman break.

### 6.1 Photometric Templates

We used the local Hubble type templates E, Sbc, Scd, and Im from Coleman, Wu, & Weedman (1980), the two starburst templates, SB1 and SB2, with low derived reddening from Kinney et al. (1996), and a 10 Myr old single age template model from Bruzual & Charlot (2002). The starburst templates are needed because many galaxies even in the nearby Universe have bluer colors than the bluest CWW templates. The observed templates are extended beyond their published wavelengths into the far-ultraviolet by power law extrapolation and into the NIR using stellar population synthesis models from Bruzual & Charlot (2002), with the initial mass functions and star-formation timescales for each template Hubble type from Pozzetti, Bruzual, & Zamorani (1996). We accounted for internal hydrogen absorption of each galaxy by setting the flux blueward of the $`912`$Å Lyman limit to zero, and for the redshift-dependent cosmic mean opacity due to neutral intergalactic hydrogen by following the prescriptions of Madau (1995).

### 6.2 $`Z_{phot}`$ Uncertainties

The best test of photometric redshifts is direct comparison to spectroscopic redshifts, but spectroscopic redshifts in the HDF-S are still scarce. We calculate the uncertainty in the photometric redshift due to the flux measurement errors using a Monte-Carlo (MC) technique derived from that used in R01 and fully explained in Rudnick et al. (2002b). At bright magnitudes template mismatch dominates the errors, something that is not modeled by the MC simulation. Hence, the MC error bars for bright galaxies are severe underestimates. At fainter magnitudes, the uncertainty is driven by errors in photometry (Fernández-Soto, Lanzetta, & Yahil, 1999) and the MC technique should provide accurate $`z_{phot}`$ uncertainties. Experience from R01 showed that two ways to correct for the template mismatch, setting a minimum fractional flux error or setting a minimum $`z_{phot}`$ error based on the mean disagreement with $`z_{spec}`$, either degrade the accuracy of the $`z_{phot}`$ measurement or reflect the systematic error only in the mean, while template mismatch can be a strong function of SED shape and redshift. A method based completely on Monte-Carlo techniques is preferable because it has a straightforwardly computable redshift probability function. This approach is desirable for estimating the rest-frame luminosities and colors (Rudnick et al., 2002b).

Therefore, we modify the MC errors directly using the FIRES photometry. In summary, we estimate the systematic component of the $`z_{phot}`$ uncertainty by scaling up all the photometric errors for a given galaxy with a constant to bring residuals of the fit in agreement with the errors. This will not change the best fit redshift and SED and will not modify the MC errorbars of faint objects, but it will enlarge the redshift interval over which the templates can satisfactorily fit the bright objects. Only in case of widely different photometric errors between the visible and infrared might the modified MC uncertainties still underestimate the true $`z_{phot}`$ uncertainty.

In Figure 6 we show a direct test of photometric redshifts of the 39 objects in the HDF-S with available spectroscopy and good photometry in all bands. The current set of spectroscopic redshifts in the HDF-S will appear in Rudnick et al. (2002a). For the small sample that we can directly compare, we find excellent agreement with no failures and with a mean $`\mathrm{\Delta }z/(1+z_{spec})0.08`$ with $`\mathrm{\Delta }z=|z_{spec}z_{phot}|`$. It is encouraging to see that the modified 68% error bars that were derived from the Monte Carlo simulations are consistent with the measured disagreement between $`z_{phot}`$ and the $`z_{spec}`$ in the HDF-S. However, large asymmetric uncertainties remain for some objects, clearly showing the presence of a second photometric redshift solution of comparable likelihood at a vastly different redshift, revealing limits on the applicability of the photometric redshift technique.

### 6.3 Stars

In a pencil beam survey at high Galactic latitude such as the HDF-S, a limited number of foreground stars are expected. We identify stars as those objects which have a better raw $`\chi ^2`$ for a single stellar template fit than the $`\chi ^2`$ for the galaxy template combination. The stellar templates are the NEXTGEN model atmospheres from Hauschildt et al. (1999) for main sequence stars with temperatures of $`3000`$ to $`10000`$K, assuming local thermodynamical equilibrium (LTE). Models of cooler and hotter stars cannot be included because non-LTE effects are important. We checked the resulting list of stars using the FWHM in the original $`B_{450}`$-band image and the $`J_sK_s`$ color, excluding two objects (catalog IDs 207 and 296) that were obviously extended in $`B_{450}`$, and we find a total of 57 stars. As shown in Figure 7, most galaxies are clearly separated from the stellar locus in $`I_{814}K_s`$ versus $`J_sK_s`$ color-color space. Other cooler stars might still resemble SEDs of redshifted compact galaxies but the latter are generally redder in the infrared $`J_sK_s`$ than most known M or methane dwarfs. Known cool L-dwarfs fall along a redder extension of the track traced by M-dwarfs in color-color space and have progressively redder $`J_sK_s`$ colors for later spectral types. However measurements by Kirkpatrick et al. (2000) show the L-dwarf sequence abruptly stopping at $`(J_sK_s)_J2.1`$ (the subscript noting Johnson magnitudes, see section 3.1 for the transformations to the AB system) whereas even cooler T-dwarfs have much bluer $`(J_sK_s)_J0`$ colors than expected from their temperatures due to strong molecular absorption. This is important because if we would apply a $`(J_sK_s)_J>2.3`$ photometric criterion to select $`z>2`$ galaxies (as discussed in section 8.2), then we should ensure that cool Galactic stars are not expected in such a sample. The published data on the lowest-mass stars suggest that they are too blue in infrared colors to be selected this way. Only heavily reddened stars with thick circumstellar dust shells, such as extreme carbon stars or Mira variables, or extremely metal-free stars having a hypothetical $`1500`$K blackbody spectrum could also have red $`(J_sK_s)_J>2.3`$ colors but it seems unlikely that the tiny field of the HDF-S would contain such unusual sources.

## 7 Catalog Parameters

The $`K_s`$-selected catalog of sources is published electronically. We describe here a subset of the photometry containing the most important parameters. The catalog with full photometry and explanation can be obtained from the FIRES homepage 777http://www.strw.leidenuniv.nl/{̃}fires.

* $`ID`$. — A running identification number in catalog order as reported by SExtractor. Sources added in the second detection pass have numbers higher than 10000.
* $`x,y`$. — The pixel positions of the objects corresponding to the coordinate system of the original (unblocked) WFPC2 version 2 images.
* $`RA,DEC`$. – The right ascension and declination in equinox J2000.0 coordinates of which only the minutes and seconds of right ascension, and negative arcminutes and arcseconds of declination are given. To these must be added $`22^h`$(R.A.) and $`60\mathrm{°}`$(DEC).
* $`f_{col,i}\pm \sigma _i`$. — The sum of counts in the “color” aperture $`f_{col,i}`$ in band $`i=\{U_{300},B_{450},V_{606},I_{814},J_s,H,K_s\}`$ and its simulated uncertainty $`\sigma _i`$, as described in $`\mathrm{\S }`$ 5.2. The fluxes are given in units of $`10^{31}`$ ergs s-1 Hz-1 cm-2.
* $`K_{tot}\pm \sigma (K_{tot})`$. — Estimate of the total $`K_s`$-band flux and its uncertainty. The sum of counts in the “total” aperture is corrected for missing flux assuming a PSF profile outside the aperture, as described in $`\mathrm{\S }`$ 5.2.
* $`ap\mathrm{\_}col`$. — An integer encoding the aperture type that was used to measure $`f_{col,i}`$. This is either a (1) $`0\stackrel{}{\mathrm{.}}7`$ diameter circular aperture, (2) $`2\stackrel{}{\mathrm{.}}0`$ diameter circular aperture, (3) isophotal aperture determined by the detection-image isophote, or a (4) circular aperture with a reduced isophotal diameter $`D=\sqrt{(A_{iso}/\pi )}/1.4`$.
* $`ap\mathrm{\_}tot`$. — An integer encoding the aperture type that was used to measure $`K_{tot}`$. This is either a (1) automatic Kron-like aperture, or a (2) circular aperture within a reduced isophotal diameter.
* $`r_{col},r_{tot}`$. — Circularized radii $`r=\sqrt{A/\pi }`$, corresponding to the area $`A`$ of the specified “color” or “total” aperture.
* $`A_{iso},A_{auto}`$. — Area of the detection isophote $`A_{iso}`$ and area of the autoscaling elliptical aperture $`A_{auto}=\pi ab`$ with semi-major axis $`a`$ and semi-minor axis $`b`$.
* $`FWHM_K,FWHM_I`$. — Full width at half maximum of a source in the $`K_s`$ detection image $`FWHM_K`$, and that of the brightest $`I_{814}`$-band source that lies in its detection isophote $`FWHM_I`$. We obtained the latter by running SExtractor separately on the original $`I_{814}`$-image and cross correlating the $`I_{814}`$-selected catalog with the $`K_s`$-limited catalog.
* $`w_i`$. — The weight $`w_i`$ represents, for each band $`i`$, the fraction of the total exposure time at the location of a source.
* $`flags`$.— Three binary flags are given. The $`bias`$ flag indicates either that the AUTO aperture measument is affected by nearby sources, or marks apertures containing more than 10% bad pixels. The $`blended`$ flag indicates overlapping sources, while the $`star`$ flag shows that the source SED is best fit with a stellar template (see section 6.3).

## 8 Analysis

### 8.1 Completeness and Number Counts

The completeness curves for point-sources in the $`J_s`$ and $`K_s`$-band as a function of input magnitude are shown in Figure 8. Our 90% and 50% completeness levels on the AB magnitude system are $`25.65`$ and $`26.25`$, respectively, in $`K_s`$, and $`26.30`$ and $`26.90`$ in $`J_s`$.

We derived the limits from simulations where we extracted a bright non-saturated star from the survey image and add it back $`30000`$ times at random locations, applying a random flux scaling drawn from a rising count slope (or an increasing surface density of galaxies with magnitude) to bring it to magnitudes between $`22K_{s,AB}28`$. We added the dimmed stars back in series of $`30`$ realizations so that they do not overlap each other. The rising count slope needs to be considered because the slope influences the number of recovered galaxies per apparent magnitude, as described below. The input count slope is based on the observed surface densities in the faint magnitude range where the signal-to-noise is $`60SNR10`$ (or $`23K_{s,tot}25`$) and where incompleteness does not yet play a role. We used only the deepest central $`4.5`$ arcmin2 of the $`J_s`$ and $`K_s`$ images ($`w>0.95`$) with near uniform image quality and exclude four small regions around the brightest stars. In the simulation images we extract sources following the same procedures as described in $`\mathrm{\S }`$5.1, but applying a reduced ($`3.5\sigma `$) detection threshold. We measure the recovered fraction of input sources against apparent magnitude, and from this we estimate the detection efficiency of point-like sources which we use to correct the observed number counts. We executed this procedure in the $`J_s`$ and $`K_s`$-band.

The resulting completeness curves assume that the true profile of the source is point-like and therefore they should be considered upper limits. An extended source would have brighter completeness limits depending on the true source size, its flux profile, and the filter that is used in the detection process. However, the detailed treatment of detection efficiency as a function of source morphology and detection criteria is beyond the scope of this paper.

When using the completeness simulations to correct the number counts, we choose a simple approach and apply a single correction down to $`50`$% completeness, based on the ratio of the simulated counts per input magnitude bin to the total recovered counts per observed total magnitude bin. More sophisticated modeling is possible but requires detailed knowledge of the intrinsic size and shape distribution of faint NIR galaxies. The simple approach corrects for all effects resulting from detection criteria, photometric scheme, incompleteness, and noise peaks. We find it works well if the total magnitude of sources is measured correctly, with little systematic difference between the input and recovered magnitudes, which is the case for our photometric scheme (see section 5.2). It is worth noticing that at $`K_{s,AB}>25`$ we actually recovered slightly more counts in the observed magnitude bins than we put in. This is caused by the fact that, in the case of a rising count slope, there are more faint galaxies boosted by positive noise peaks than bright galaxies lost on negative noise peaks. This effect is strong at low signal-to-noise fluxes and results in a slight excess of recovered counts. This is the main reason that we required little correction up to the detection threshold, except for the faintest $`0.5`$ mag bin centered on $`K_s=26.0`$, which contained false positive detections due to noise. After removing stars (see section 6.3), we plot in Figure 9 the raw and corrected source counts against total magnitude.

Figure 10 presents a compilation of other deep $`K`$-band number counts from a number of published studies. The FIRES counts follow a d$`log`$(N)/dm relation with a logarithmic slope $`\alpha 0.25`$ at $`20K_{s,J}22`$ (Johnson magnitudes) and decline at fainter magnitudes to $`\alpha 0.15`$ at $`22.0K_{s,J}24`$. This flattening of the slope has not been seen in other deep NIR surveys, where we emphasize that the FIRES HDFS field is the largest and the deepest amongst these surveys, and that only the counts in the last FIRES bin at $`K_{s,J}=24.25`$ were substantially corrected. It is remarkable that the SUBARU Deep Field count slope $`\alpha 0.23`$ of Maihara et al. (2001) looks smooth compared to the HDF-S although their survey area and the raw count statistics are slightly smaller.

Other authors (Djorgovski et al., 1995; Moustakas et al., 1997; Bershady, Lowenthal, & Koo, 1998) find logarithmic counts slopes in $`K`$ ranging from $`0.23`$ to $`0.36`$ over $`20K_J2324`$, however the counts in the faintest bins in these surveys were boosted by factors of $`23`$, based on completeness simulations. The origin of the faint-end discrepancies of the $`K`$ counts is unclear. Cosmic variance can play a role, because the survey areas never exceed a few armin2, but also differences in the used filters ($`K_s,K,K`$) and differences in the techniques and assumptions used to estimate the total magnitude (see 5) or to correct the counts for incompleteness may be important. Further analysis is needed to ascertain whether size-dependent biases in the completeness correction play a role in the faint-end count slope.

### 8.2 Color-Magnitude Distributions

Figures 1114 show color-magnitude diagrams of $`K_s`$-selected galaxies in the HDF-S. The $`I_{814}K_s`$ versus $`K_s`$ color-magnitude diagram in Figure 11 shows a large number of extremely red objects (EROs) with $`I_{814}K_s2.6`$ (on the AB system) or $`(I_{814}K_s)_J4`$ (Johnson). There appears to be an excess of EROs at total magnitudes $`K_{s,AB}23`$ compared to magnitudes $`K_{s,AB}24`$. This is not caused by the insufficient signal-to-noise ratio in the $`I_{814}`$ measurements. In a similar diagram for the $`J_sK_s`$ color shown in Figure 12, there is a striking presence at the same $`K_s`$ magnitudes of sources with very red $`(J_sK_s)_{AB}1.34`$ or $`(J_sK_s)_J2.3`$ colors. Such sources were also found by Saracco et al. (2001), using shallow NIR data, who suggested they might be dusty starbursts or ellipticals at $`z>2`$. Interestingly, any evolved galaxy with a prominent Balmer/4000Å discontinuity in their spectrum, like most present-day Hubble Type galaxies, would have such very red observed NIR colors if placed at redshifts $`z>2`$. While the $`(J_sK_s)_J2.3`$ sources we find are generally morphologically compact, with exceptions, we do not expect the sources with good photometry to be faint cool L-dwarf stars because known colors of such stars are $`(J_sK_s)_J2.1`$ (see section 6.3). The photometric redshifts of all red NIR galaxies are $`z_{phot}2`$, but they would be missed by ultraviolet-optical color selection techniques such as the U-dropout method, because most of them are barely detectable even in the deepest optical images. One bright NIR galaxy is completely undetected in the original WFPC2 images. The $`(J_sK_s)_J2.3`$ sources are studied in more detail by Franx et al. (2002) and the relative contributions of these galaxies and U-dropouts to the rest-frame optical luminosity density will be presented in Rudnick et al. (2002b).

If we select sources with $`1.95<z_{phot}<3.5`$, we find clear differences in the $`V_{606}H`$ versus $`H`$ color-magnitude diagram between our NIR-selected galaxies in the HDF-S and those of the HDF-N (compare Figure 14 to Figure 1 of Papovich, Dickinson, & Ferguson 2001). Over a similar survey area and to similar limiting depths, we find 7 galaxies redder than $`(V_{606}H)_{AB}3`$ and brighter than total magnitude $`H_{AB}25.5`$, compared to only one in the HDF-N.

While the surface density of such galaxies is not well known, it is clear that the HDF-N contains far fewer of them than the HDF-S. It remains to be seen if this is just field-to-field variation, or that one of the two fields is atypical. The results of the second much larger FIRES field centered on MS1054-03 (Förster Schreiber et al., 2002) should provide more insight into this issue. We note that all 7 $`(V_{606}H)_{AB}3`$ galaxies in the HDF-S are also amongst the brightest 16 $`(J_sK_s)_J>2.3`$ sources. Figure 15 shows the surface densities of EROs and galaxies with $`(J_sK_s)_J>2.3`$ colors as function of $`K_s`$-band total magnitude. The surface density of EROs peaks around $`K_{s,AB}23`$ and then drops or flattens at fainter magnitudes, contrary to the number of $`(J_sK_s)_J>2.3`$ galaxies which keeps rising to the faintest $`K_s`$ total magnitudes in our catalog.

## 9 Summary and Conclusions

We have presented the results of the FIRES deep NIR imaging of the WFPC2-field of the HDF-S obtained with ISAAC at the VLT: the deepest ground-based NIR data available, and the deepest $`K_s`$-band of any field. We constructed a $`K_s`$-selected multicolor catalog of galaxies, consisting of 833 objects with $`K_{s,AB}26`$ and photometry in seven-bands from $`0.3`$ to $`2.2\mu `$ for 624 of them. These data are available electronically together with photometric redshifts for 567 galaxies. Our unique combination of deep optical space-based data from the HST together with deep ground-based NIR data from the VLT allows us to sample light redder than the rest-frame V-band in galaxies with $`z3`$ and to select galaxies from their rest-frame optical properties, obtaining a more complete census of the stellar mass in the high-redshift universe. We summarize our main findings below:

* The $`K_s`$-band galaxy counts in HDF-S turn over at the faintest magnitudes and flatten from $`\alpha 0.25`$ at total AB magnitudes $`22K_s24`$ to $`\alpha 0.15`$ at $`24K_s26`$; this is flatter than counts in previously published deep NIR surveys, where the FIRES HDF-S field is largest and deepest amongst these surveys. The nature of the scatter in the faint-end counts is yet unclear but field-to-field varations as well as different analysis techniques likely play a role.
* The HDF-S contains 7 sources redder than $`(V_{606}H)_{AB}3`$ and brighter than total magnitude $`H_{AB}25.5`$ at photometric redshifts $`1.95<z_{phot}<3.5`$, while such galaxies were virtually absent in the HDF-N. They are much redder than regular U-dropout galaxies in the same field and are candidates for relatively massive, evolved systems at high redshift. The difference with the HDF-N might just reflect field-to-field variance, calling for more observations to similar limits with full optical-to-infrared coverage. Results from the second and larger FIRES field centered on MS1054-03 (Förster Schreiber et al., 2002) should provide more insight into this issue.
* We find substantial numbers of red galaxies with $`(J_sK_s)_J>2.3`$ that have photometric redshifts $`z_{phot}>2`$. These galaxies would be missed by ultraviolet-optical color selection techniques such as the U-dropout method because most of them are barely detectable even in the deepest optical images. The surface densities of these sources in our field keeps rising down to the detection limit in $`K_s`$, in contrast to the number counts of EROs which peak at $`K_{s,AB}23`$ and then drop or flatten at fainter magnitudes.

The results of the HDF-S presented in this paper demonstrate the necessity of extending optical observations to near-IR wavelengths for a more complete census of the early universe. Our deep $`K_s`$-band data prove invaluable for they probe well into the rest-frame optical at $`2<z<4`$, where long-lived stars may dominate the light of galaxies. We are pursuing follow-up programs to obtain more spectroscopic redshifts needed to confirm the above results. Updates on the FIRES programme and access to the reduced images and catalogues can be found at our website http://www.strw.leidenuniv.nl/{̃}fires.

The data here presented have been obtained as part of an ESO Service Mode program. We would very much like to thank the ESO staff for their kind assistance and enormous efforts in taking these data and making them available to us. This project would not be possible without their dedicated work. This work was supported by a grant from the Netherlands Organization for Scientific Research. We would like to thank the Lorentz Center of Leiden University for its hospitality during several workshops. The comments of the referee helped to improve the paper.


format: text

text:
# Relativistic effects in the search for high density symmetry energy

## Abstract

Intermediate energy heavy ion collisions open the unique possibility to explore the Equation of State ($`EOS`$) of nuclear matter far from saturation, in particular the density dependence of the symmetry energy. Within a relativistic transport model it is shown that the isovector-scalar $`\delta `$-meson, which affects the high density behavior of the symmetry energy density, influences the dynamics of heavy ion collisions in terms of isospin collective flows. The effect is largely enhanced by a relativistic mechanism related to the covariant nature of the fields contributing to the isovector channel. Results for reactions induced by $`{}_{}{}^{132}Sn`$ radioactive beams are presented. The elliptic flows of nucleons and light isobars appear to be quite sensitive to microscopic structure of the symmetry term, in particular for particles with large transverse momenta, since they represent an earlier emission from a compressed source. Thus future, more exclusive, experiments with relativistic radioactive beams should be able to set stringent constraints on the density dependence of the symmetry energy far from ground state nuclear matter.

, , , , and

thanks: On leave from Laboratori Nazionali del Sud INFN, Catania, Italy and Dipartimento di Fisica, Università degli Studi di Catania, partially supported by a grant of the A.LaRiccia Foundationthanks: On leave from NIPNE-HH and Bucharest University, Romaniathanks: On leave from Sektion Physik, Universität München, Germany, supported by the German Ministry of Education and Science, BMBW, grant 06LM981.

The high density behaviour of nuclear symmetry energy $`E_{sym}(\rho _B)`$ is very important for understanding many interesting astrophysical phenomena, but it is absolutely not constrained by the predictions from several relativistic and non-relativistic  models of nuclear matter. The results can be roughly classified into two groups, i.e., one where the $`E_{sym}(\rho _B)`$ rises, Asy-stiff, and one in which it falls with increasing density, Asy-soft . An increasing $`E_{sym}(\rho _B)`$ leads to a more proton-rich neutron star whereas a decreasing one would make it more pure in neutron content. As a consequence the chemical composition and cooling mechanism of protoneutron stars , mass-radius correlations , critical densities for kaon condensation in dense stellar matter  as well as the possibility of a mixed quark-hadron phase  in neutrons stars will all be rather different. It has recently been argued by means of simple thermodynamics considerations that the onset of the quark phase has a strong sensitivity to the behaviour of $`E_{sym}(\rho _B)`$ even for not very large asymmetries .

The search for $`E_{sym}(\rho _B)`$ around saturation density has driven a lot of theoretical and experimental efforts. It seems to be rather well established that heavy-ion collisions ($`HIC`$) at cyclotron energies can give the possibility to extract some information on the symmetry term of the nuclear Equation of State ($`EOS`$) in region below and/or sligthly above the normal density . On the other hand it is quite desirable to get information on the symmetry energy at higher density, where furthermore we cannot have complementary investigations from nuclear structure like in the case of the low density behaviour. Indeed $`HIC`$ provide a unique way to create asymmetric matter at high density in terrestrial laboratories. Calculations within transport theory show that $`HICs`$ around $`1AGeV`$ allow to reach a transient state of matter with more than twice the normal baryon density. Moreover, although the data are mostly of inclusive type (and the colliding nuclei not very neutron rich), quite clearly a dependence of some observables on charge asymmetry is emerging.

In this paper we show that a relativistic description of the nuclear mean field can account for an enhancement of isospin effects during the dynamics of heavy-ion collision. In particular future experiments with radioactive should be able to provide information on the vector part of isovector mean fields from collective flow analyses.

The isospin dependence of collective flows has been already discussed in a non-relativistic framework  using very different $`EOS`$ with opposite behaviours of the symmetry term at high densities, increasing repulsion (asy-stiff) vs. increasing attraction (asy-soft). The main new result shown here, in a fully relativistic scheme, is the importance at higher energies of the microscopic covariant structure of the effective interaction in the isovector channel: effective forces with very similar symmetry terms can give rise to very different flows in relativistic heavy ion collisions.

We start from the Relativistic Mean Field ($`RMF`$) picture of the hadronic phase of nuclear matter  which has been extensively used to the study of the EOS for symmetric and asymmetric matter. The $`RMF`$ describes an interacting system of nucleons (described as Dirac spinors) and meson classical fields. The most common treatment of the isospin dependent part of the interaction is based on the introduction of an effective $`\rho `$-meson field (vector-isovector) which can account for the known value of symmetry energy at normal density.

However, a full description in a relativistic framework in principle should rely on the balance between a scalar and a vector field as stressed in some papers within the Hartree approximation , and as naturally accounted for within the Dirac-Hartree-Fock ($`DHF`$)  or the Dirac-Brueckener-Hartree-Fock ($`DBHF`$)  schemes. One could argue that the scalar-isovector $`\delta `$ meson $`(a_0(980))`$ is too heavy or that the scalar field in the isoscalar channel may be due to the two pion correlation, while there is no equivalent possibility in the isovector channel. On the other hand we stress that the vector-isovector field in the $`RMF`$ effective picture has not to be viewed as coming only from the exchange of the $`\delta `$ meson. In fact in the nuclear system contributions to the isovector channel are mostly due to isoscalar mesons, as clearly shown by DHF or DBHF scheme , through important exchange and correlation terms. Therefore when in the following we will refer to $`\delta `$ field indeed we mean a $`\delta like`$ field, i.e. the scalar isovector part of the effective “interaction”.

In recent years some efforts have been devoted to the effects of the scalar-isovector channel in finite nuclei, . Such investigations have not shown a clear evidence for the $`\delta `$-field and this can be understood considering that in finite nuclei one can test the interaction properties mainly below the normal density, where the effect of the $`\delta `$channel on symmetry energy and on the effective masses is indeed small  and eventually could be absorbed into non linear terms of the $`\rho `$ field. Moreover even studies of the asymmetric nuclear matter by means of the Fermi Liquid Theory  and a linear response analysis have concluded that some properties, like the borderline and the dynamical response inside the spinodal instability region, are not affected by the $`\delta `$ field .

Here we show that heavy-ion collisions around $`1AGeV`$ with radioactive beams can provide instead a unique opportunity to spot the presence of the scalar isovector channel. In fact, due to the large counterstreaming nuclear currents one may exploit the different Lorentz nature of a scalar and a vector field.

All models, including $`RMF`$, allow to use the parabolic approximation for the description of the $`EOS`$ of asymmetric nuclear matter:

$$E(\rho _B,I)=E(\rho _B)+E_{sym}(\rho _B)I^2$$
(1)
where $`I=\frac{NZ}{A}`$ is the asymmetry parameter. When both $`\rho `$-like and $`\delta `$-like channels are considered $`E_{sym}(\rho _B)`$ can be written as :

$$E_{sym}(\rho _B)=\frac{1}{6}\frac{k_F^2}{E_F^{}}+\frac{1}{2}\left[f_\rho f_\delta \left(\frac{M^{}}{E_F^{}}\right)^2\right]\rho _BE_{sym}^{kin}+E_{sym}^{pot}$$
(2)
with $`E^{}=\sqrt{k_F^2+M^2}`$, $`M^{}`$ the effective Dirac mass and $`f_{\rho ,\delta }=(g_{\rho ,\delta }/m_{\rho ,\delta })^2`$ are the coupling constants of the isovector channels.

We see that, when $`\delta `$ is included, the observed $`a_4=E_{sym}(\rho _0)`$ value actually assigns the combination $`[f_\rho f_\delta (\frac{M^{}}{E_F^{}})^2]`$ of the $`(\rho ,\delta )`$ coupling constants, for futher details see ref.. In Fig.1 we report the density dependence of the symmetry energy for three different models: one including only the $`\rho `$ field ($`RMF\rho `$), the other with ($`\rho +\delta `$) fields ($`RMF(\rho +\delta )`$) and the last, ($`RMFD\rho `$), including only $`\rho `$ field but with a covariant density dependence of $`f_\rho `$, see ref. . This is tuned just to give at high density the same $`E_{sym}(\rho _B)`$ of the the ($`RMF(\rho +\delta )`$) case. As shown in the following, the latter is useful for disentagling in the reaction dynamics the effects due to the difference in $`E_{sym}(\rho _B)`$ from those directly linked to the strenght of the $`\rho `$ vector field.

Thus these models parametrize the isovector mean field either by only the vector field with $`f_\rho =1.1fm^2`$, or with a balance between a vector field with $`f_\rho =3.3fm^2`$ and a scalar one with $`f_\delta =2.4fm^2`$, or finally by a normal density coupling $`f_\rho (\rho _0)=1.1fm^2`$, but with an increasing density dependence as shown in Fig.1 (insert). We stress again that in $`RMF(\rho +\delta )`$ the symmetry energy is coming from a balance between a scalar attraction, ($`\delta like`$), and a vector repulsion, ($`\rho like`$), which is now roughly three times larger than in the $`RMF\rho `$ case.

The choice of $`f_\delta `$ is fixed relatively well by DBHF  and DHF  calculations. Therefore the effect described in the following is not artificially enhanced, but based on a reliable estimate available at the moment. In any case the aim of this work is to present some qualitative new features expected in the reaction dynamics, in particular for collective flows, just due to the introduction of a scalar effective field in the isovector channel.

Collective flows in heavy ion collisions give important information on the dynamic response of excited nuclear matter . In particular the proton-neutron differential flow $`F^{pn}(y)`$  has been found to be a very useful probe of the isovector part of the $`EOS`$ sine it appears rather insensitive to the isoscalar potential and the in medium nuclear cross section. The definition of the $`F^{pn}(y)`$ is

$$F^{pn}(y)\frac{1}{N(y)}\underset{i=1}{\overset{N(y)}{}}p_{x_i}\tau _i$$
(3)
where $`N(y)`$ is the total number of free nucleons at the rapidity $`y`$, $`p_{x_i}`$ is the transverse momentum of particle $`i`$ in the reaction plane, and $`\tau _i`$ is +1 and -1 for protons and neutrons, respectively.

For the theoretical description of heavy ion collisions we solve the covariant transport equation of the Boltzmann type within the Relativistic Landau Vlasov ($`RLV`$) method  (for the Vlasov part) and applying a Monte-Carlo procedure for the collision term. $`RLV`$ is a test particle method using covariant Gaussians in phase space for the test particles. The collision term includes elastic and inelastic processes involving the production/absorption of the $`\mathrm{\Delta }(1232MeV)`$ and $`N^{}(1440MeV)`$ resonances as well as their decays into one- and two-pion channels. Details about the used cross sections for all possible channels can be found in Ref. . An explicit isospin-dependent Pauli blocking term for the fermions is employed. Asymmetry effects are suitably accounted for in a self-consistent way with respect for the $`RMF`$ models discussed above.

A typical result for the $`{}_{}{}^{132}Sn+^{132}Sn`$ reaction at $`1.5AGeV`$ (semicentral collisions) is shown in Fig.2. The error bars are related to statistical fluctuations due to the Monte-Carlo nature of the simulations.

We notice that the differential flow in case of the $`RMF(\rho +\delta )`$ (full circles and solid line) presents a stiffer behaviour relative to the $`RMF\rho `$ (open circles) model, as expected from the more repulsive symmetry energy $`E_{sym}(\rho _B)`$ at high baryon densities, see Fig.1. On the other hand it is quite surprising that a relatively small difference at $`2\rho _0`$ can result in a such different collective flows. Indeed, we will see below that this is not the whole story.

We have repeated the calculation using the $`RMFD\rho `$ interaction, i.e. with only a $`\rho `$ contribution but tuned to reproduce the same $`EOS`$ of the $`RMF(\rho +\delta )`$ case. The results, short-dashed curve of Fig.2, are very similar to the ones of the $`RMF\rho `$ interaction. Therefore we can explain the large flow effect as mainly due to the different strengths of the vector-isovector field between $`RMF(\rho +\delta )`$ and $`RMF\rho ,D\rho `$ in the relativistic dynamics. In fact if a source is moving the vector field is enhanced (essentially by the local $`\gamma `$ Lorentz factor) relative to the scalar one.

In order to get the idea we write down, for an idealized situation, the “force” acting on a particle. From the isovector part of the interaction we get

$$\frac{d\stackrel{}{p}_i^{}}{d\tau }=\pm f_\rho \frac{E_i^{}}{M_i^{}}\stackrel{}{}\rho _3f_\delta \stackrel{}{}\rho _{S3}$$
(4)
for protons and (upper signs), respectively neutrons (lower signs). $`\stackrel{}{p}^{}`$ is the effective momentum, $`\tau `$ the particle proper time (averaged) and $`\rho _3=\rho _p\rho _n`$ the isovector baryon density (correspondingly $`\rho _{S3}`$ the scalar one). Here we have simplified the problem neglecting the contribution from the current gradient in the transverse direction and the current derivative with respect to time. We are interested in the difference between the force acting on a neutron and on a proton, respectively. Oversimplifying the $`HIC`$ dynamics we consider locally neutrons and protons with the same $`\gamma `$ factor (i.e. with the same speed). Then Eq.4 can be expressed approximately by the following transparent form ($`\rho _{S3}=\frac{M^{}}{E^{}}\rho _3`$):

$$\frac{d\stackrel{}{p}_p^{}}{d\tau }\frac{d\stackrel{}{p}_n^{}}{d\tau }2\left[\gamma f_\rho \frac{f_\delta }{\gamma }\right]\stackrel{}{}\rho _3$$
(5)
where $`\gamma `$ is the Lorentz factor for the collective motion of a given ideal cell.

Keeping in mind that $`RMF(\rho +\delta )`$ has a three times larger $`\rho `$ field it is clear that dynamically the vector-isovector mean field acting during the $`HIC`$ is much greater than the one of the $`RMF\rho ,D\rho `$ cases. Then the isospin effect is mostly caused by the different Lorentz structure of the “interaction” which results in a dynamical breaking of the balance between the $`\rho `$ vector and $`\delta `$ scalar fields, present in nuclear matter at equilibrium. This effect is analogous to the interplay between the isoscalar vector- and scalar-fields which is seen in the magnitude and energy dependence of the real part of the optical potential, ref. .

In order to characterize the effect on differential collective flows we have calculated the slope $`dF^{pn}(y)/d(y/y_{proj})`$ at mid-rapidity. Its value is $`46.7MeV/c`$ for $`RMF(\rho +\delta )`$ and $`23.4MeV/c`$ for $`RMF\rho `$, i.e. a factor two difference.

We have also performed some calculations at lower beam energies. We have found that up to $`500AMeV`$ there is no valuable difference in the differential flow predictions among the models discussed here. The effect coming from the strength of $`\rho `$ field starts to become important around $`1AGeV`$, as expected from the relativistic mechanism.

Another interesting observable is the elliptic flow $`v_2(y,p_t)`$, which is derived as the second coefficient from a Fourier expansion of the azimuthal distribution $`N(\varphi ,y,p_t)=v_0(1+v_1cos(\varphi )+2v_2cos(2\varphi ))`$. It can be expressed as

$$v_2=<\frac{p_x^2p_y^2}{p_t^2}>$$

where $`p_t=\sqrt{p_x^2+p_y^2}`$ is the transverse momentum .

A negative value of $`v_2`$ corresponds to the emission of matter perpendicular to the reaction plane, $`squeezeout`$ flow. The $`p_t`$-dependence of $`v_2`$, which has been recently investigated by various groups , is very sensitive to the high density behavior of the $`EOS`$ since highly energetic particles ($`p_t0.5`$) originate from the initial compressed and out-of-equilibrium phase of the collision, see e.g. Ref. .

In Fig.3 we present the $`p_t`$ dependence of the proton-neutron difference of the elliptic flow in the same very exotic $`{}_{}{}^{132}Sn+^{132}Sn`$ reaction at $`1.5AGeV`$ (semicentral collisions), for mid-rapidity emissions. The larger error bars correspond to a reduced statistics when a selection on different $`p_t`$ bins is introduced. The effect is increasing for larger $`p_t`$ values due to the smaller number of contributions.

From Fig.3 we see that, in spite of the statistical errors, in the $`(\rho +\delta )`$ dynamics the high-$`p_t`$ neutrons show a much larger $`squeezeout`$. This is fully consistent with an early emission (more spectator shadowing) due to the larger repulsive $`\rho `$-field. We can expect this appreciable effect since the relativistic enhancement discussed above is relevant just at the first stage of the collision. The $`v_2`$ observable, which is a good chronometer of the reaction dynamics, appears to be particularly sensitive to the Lorentz structure of the effective interaction.

We have repeated the same set of simulations for the more realistic $`{}_{}{}^{132}Sn+^{124}Sn`$ reaction at 1.5 AGeV (b=6fm), that likely could be studied with the new planned radioactive beam facilities at intermediate energies. The results are shown in Fig.4. The effect of the different structure of the isovector channel is still quite clear, of course with a reduction due to the smaller isospin density in the interaction region. Particularly evident is again the splitting in the high $`p_t`$ region of the elliptic flow.

In conclusion intermediate energy heavy-ion collisions with radioactive beams can give information on the symmetry energy at high baryon density and on its detailed microscopic structure. We have shown that such experiments provide a unique tool to investigate the strength of the $`\delta like`$ field. The sensitivity is enhanced relative to the static property $`E_{sym}(\rho _B)`$ because of the more fundamental covariant nature of the fields involved in $`HIC`$ dynamics.

Collective flows observables are found to be sensitive to isospin effects. Due to the time selectivity on the emitted particles the elliptic flow measurements appear to be the most appealing, especially for nucleons and light isobars at high transverse momentum.


text:
# Left 3-Engel elements in groups

## 1. Introduction and results

Let $`G`$ be any group and $`n`$ be a non-negative integer. For any two elements $`a`$ and $`b`$ of $`G`$ we define inductively $`[a,_nb]`$ the $`n`$-Engel commutator of the pair $`(a,b)`$, as follows:

$$[a,_0b]:=a,[a,b]:=a^1b^1ab,\text{and}[a,_nb]:=[[a,_{n1}b],b]\text{for all}n>0.$$

An element $`x`$ of $`G`$ is called a left $`n`$-Engel element if $`[g,_nx]=1`$ for all $`gG`$. We denote by $`L_n(G)`$, the set of all left $`n`$-Engel elements of $`G`$. So $`L_0(G)=1`$, $`L_1(G)=Z(G)`$ the centre of $`G`$, and it can be easily seen that

$$L_2(G)=\{xG|x^G\text{is abelian}\},$$

where $`x^G`$ denotes the normal closure of $`x`$ in $`G`$. Therefore $`L_2(G)`$ is contained in $`B(G)`$ the Baer radical of $`G`$, and in particular it is contained in $`HP(G)`$ the Hirsch-Plotkin radical of $`G`$. In general for an arbitrary group $`K`$ it is not necessary that $`L_n(K)HP(K)`$. For suppose, for a contradiction, that $`L_n(K)HP(K)`$ for all $`n`$ and all groups $`K`$. By a deep result of Ivanov , there is a finitely generated infinite group $`M`$ of exponent $`2^k`$ for some positive integer $`k`$. Suppose that $`k`$ is the least integer with this property, so every finitely generated group of exponent dividing $`2^{k1}`$ is finite. Let $`a`$ be any element of $`M`$ of order $`2`$ and $`x`$ an arbitrary element of $`M`$. It is easy to see that $`[x,_ma]=[x,a]^{(2)^{m1}}`$ for all positive integers $`m`$. Thus every element of order $`2`$ of $`M`$ is in $`L_{k+1}(M)`$. So by hypothesis, $`M/HP(M)`$ is of exponent dividing $`2^{k1}`$ and so it is finite. Since $`M`$ is finitely generated, $`HP(G)`$ is also. But this yields that $`HP(G)`$ is a periodic finitely generated nilpotent group and so it is finite. It follows that $`M`$ is finite, a contradiction.
Hence the question which naturally arises, is that: What is the least positive integer $`n`$ such that $`L_n(G)HP(G)`$? To study this question it should first study the case $`n=3`$, since

$$L_0(G)L_1(G)L_2(G)\mathrm{}L_n(G)\mathrm{}.$$

The main object of this paper is to study $`L_3(G)`$. As far as we know there is no example of a group $`G`$ for which $`L_3(G)HP(G)`$. The corresponding subset to $`L_n(G)`$ which can be similarly defined, is $`R_n(G)`$ the set of all right $`n`$-Engel elements of $`G`$: an element $`x`$ of $`G`$ is called a right $`n`$-Engel element if $`[x,_ng]=1`$ for all $`gG`$. There is a well-known relation between these two subsets of $`G`$ due to Heineken \[4, Theorem 7.11\]: $`(R_n(G))^1L_{n+1}(G)`$ for all integers $`n>0`$. It is clear that $`R_0(G)=1`$, $`R_1(G)=Z(G)`$ and by a result of Kappe \[4, Corollary 1 Theorem 7.13\], $`R_2(G)`$ is a characteristic subgroup of $`G`$. It is known also that $`R_2(G)L_2(G)`$ \[4, Theorem 7.13 (i)\]. Recently Newell  has shown that the normal closure of every element of $`R_3(G)`$ is nilpotent of class at most $`3`$. This shows, of course, that $`R_3(G)B(G)HP(G)`$. An early example was given by Macdonald  shows that the inverse and square of a right 3-Engel element need not be right 3-Engel, thus $`R_3(G)`$ is not necessarily a subgroup. But as we show in Corollary 2.2, $`L_3(G)`$ is closed under the taking all powers. It is interesting to note that $`(R_3(G))^2L_3(G)`$ and $`(R_3(G))^4R_3(G)`$ \[3, Remark to Theorem 1\].

The main results of this paper are the followings.

###### Theorem 1.1.

Let $`G`$ be a group and $`p`$ be a prime number. If $`xL_3(G)`$ and $`x^{p^n}=1`$ for some integer $`n>1`$, then $`x^p^G`$ is soluble of derived length at most $`n1`$ and $`x^p`$ belongs to $`B(G)`$, the Baer radical of $`G`$. In particular, $`x^p`$ belongs to $`HP(G)`$, the Hirsch-Plotkin radical of $`G`$.

###### Theorem 1.2.

Let $`G`$ be any group and $`a,bL_3(G)`$. Then $`a,b`$ is nilpotent of class at most $`4`$.

## 2. proofs

###### Lemma 2.1.

Suppose that $`G`$ is an arbitrary group and $`x,yG`$. Then $`[y,_3x]=[y^1,_3x]=1`$ if and only if $`x,x^y`$ is nilpotent of class at most $`2`$.

###### Proof.

Since

$$[y^ϵ,_3x]=[x^1,[x^1,[x^1,y^ϵ]]]^{x^3}$$

where $`ϵ\{1,1\}`$, we have

$`[y,_3x]=[y^1,_3x]=1`$
$`[x^1,[x^1,[x^1,y]]]=[x^1,[x^1,[x^1,y^1]]]=1`$
$`[x^{x^y},x^1]=[x^{x^{y^1}},x^1]=1`$
$`[x^y,_2x^1]=[x^{y^1},_2x^1]=1`$
$`[x^y,_2x^1]=[x^1,_2x^y]=1`$
$`x^1,x^y=x,x^y\text{is nilpotent of class at most}\mathrm{\hspace{0.33em}2}.`$
So we have the following characterization of left 3-Engel elements in a group. We denote by $`𝒩_2`$ the class of nilpotent groups of class at most $`2`$.

###### Corollary 2.2.

For an arbitrary group $`G`$,

$$L_3(G)=\{xG|x,x^y𝒩_2\text{for all}yG\}.$$

In particular every power of a left 3-Engel element is also a left $`3`$-Engel element.

###### Proposition 2.3.

A group generated by a set of left $`3`$-Engel elements of finite orders such that their orders are pairwise coprime is abelian.

###### Proof.

It is enough to show that $`[a,b]=1`$ for any two left 3-Engel elements $`a`$ and $`b`$ such that $`\mathrm{gcd}(|a|,|b|)=1`$. By Corollary 2.2, $`K=a,a^b`$ and $`H=b,b^a`$ are both nilpotent. Thus $`[a,b]=a^1a^b=(b^1)^abKH`$. Since $`\mathrm{gcd}(|a|,|b|)=1`$ and $`H`$ and $`K`$ are both nilpotent, $`\mathrm{gcd}(|H|,|K|)=1`$. It follows that $`[a,b]=1`$, as required. ∎

###### Lemma 2.4.

Let $`p`$ be a prime number, $`G`$ be a group and $`xL_3(G)`$. If $`x^{p^n}=1`$ for some integer $`n2`$, then $`x^{p^{n1}}L_2(G)`$.

###### Proof.

Let $`y`$ be an arbitrary element of $`G`$. By Corollary 2.2, $`x,x^y`$ is nilpotent of class at most $`2`$. Thus

$`[(x^y)^{p^{n1}},x^{p^{n1}}]=`$
$`[(x^y)^{p^{n2}},x^{p^{n1}}]^p=\text{since}[x^y,x]Z(x,x^y)`$
$`[(x^y)^{p^{n2}},x^{p^n}]=1`$
But $`[y,x^{p^{n1}},x^{p^{n1}}]=[(x^y)^{p^{n1}},x^{p^{n1}}]=1`$. This completes the proof. ∎

Proof of Theorem 1.1. By Lemma 2.4 and Corollary 2.2

$$1\mathrm{}x^{p^{n1}}^G\mathrm{}x^{p^{n2}}^G\mathrm{}\mathrm{}\mathrm{}x^p^G$$

is a series of normal subgroups of $`G`$ with abelian factors. This implies that $`K=x^p^G`$ is soluble of derived length at most $`n1`$. By Corollary 2.2, $`x^p`$ and so all its conjugates in $`G`$ belong to $`L_3(G)`$ and in particular they are in $`L_3(K)`$. Now a result of Gruenberg \[4, Theorem 7.35\] implies that $`B(K)=K`$. But $`K\mathrm{}G`$ which yields that $`x^pKB(G)HP(G)`$. $`\mathrm{}`$

In the following calculations, one must be careful with notation. As usual $`u^{g_1+g_2}`$ is shorthand notation for $`u^{g_1}u^{g_2}`$. This means that

$$u^{(g_1+g_2)(h_1+h_2)}=u^{(g_1+g_2)h_1}u^{(g_1+g_2)h_2}$$

which does not have to be equal to $`u^{g_1(h_1+h_2)}u^{g_2(h_1+h_2)}`$. We also have that

$$u^{(g_1+g_2)(h)}=((u^{g_1}u^{g_2})^1)^h$$

which is equal to $`u^{g_2hg_1h}`$. This does not have to be the same as $`u^{g_1hg_2h}`$.
The following remark easily follows from Corollary 2.2. We use in sequel this remark, sometimes without any reference.

###### Remark 1.

Let $`G`$ be any group and $`aL_3(G)`$. Then

$$[a,x]^{a^2}=[a,x]^{2a1}\text{and}[a,x]^{a^1}=[a,x]^{a+2}\text{for all}xG.$$

###### Lemma 2.5.

Let $`G`$ be any group and $`a,bL_3(G)`$. Then

$$[a,b,a,b]=[a,b],[a,b]^a,[a,b]^b,[a,b]^{ab}.$$

###### Proof.

It is enough to show that $`N=[a,b],[a,b]^a,[a,b]^b,[a,b]^{ab}`$ is a normal subgroup of $`a,b`$. We have

$`[a,b]^{a^2}=[a,b]^{2a1}N`$
$`[a,b]^{ba}=[a,b]^{ab[b,a]}=[a,b]^{1+ab1}N`$
$`[a,b]^{aba}=[a,b]^{a^2b[b,a]}=[a,b]^{1+a^2b1}`$ $`=[a,b]^{1+(2a1)b1}=[a,b]^{1+2abb1}N`$
Thus $`N^aN`$ (I). Also we can write

$`[a,b]^{a^1}=[a,b]^{a+2}N`$
$`[a,b]^{ba^1}=[a,b]^{a^1b[b,a^1]}=[a,b]^{a+a^1ba^1}=[a,b]^{aab+2b+a^1}N`$
$`[a,b]^{aba^1}=[a,b]^{b[b,a^1]}=[a,b]^{a^1+ba^1}N`$
Therefore $`N^{a^1}N`$ (II). It follows from (I) and (II) that $`N=N^a`$. Now since $`[a,b]^{ab}=[a,b]^{1+ba+1}N`$ we have $`M=[b,a],[b,a]^b,[b,a]^a,[b,a]^{ba}`$ is equal to $`N`$. Thus by the symmetry between $`a`$ and $`b`$, we have $`M^b=M`$ and so $`N^b=N`$. Hence $`N\mathrm{}a,b`$. This completes the proof. ∎

###### Lemma 2.6.

Let $`G`$ be any group and $`a,bL_3(G)`$. Then both of the commutators $`[a,b]`$ and $`[a,b]^{ab}`$ belong to $`C_G([a,b]^a,[a,b]^b)`$.

###### Proof.

By Lemma 2.1, $`a,a^b𝒩_2`$ and $`b,b^a𝒩_2`$, so $`[[a,b],[a,b]^a]=1`$ (I) and $`[[a,b],[a,b]^b]=1`$ (II). It follows from (I) that $`[[a,b]^b,[a,b]^{ab}]=1`$ and (II) yields that $`[[a,b]^a,[a,b]^{ba}]=1`$ (III). But $`[a,b]^{ba}=[a,b]^{1+ab1}`$ so (I) and (III) imply that $`[[a,b]^a,[a,b]^{ab}]=1`$. This completes the proof. ∎

Proof of Theorem 1.2. We first prove that the derived subgroup of $`a,b`$ is abelian. Since $`a,b^{}`$ is the normal closure of $`[a,b]`$ in $`a,b`$, it is enough to show that $`[a,b]`$ is in the centre of $`a,b^{}`$ and by Lemma 2.6, it suffices to prove that $`[[a,b],[a,b]^{ab}]=1`$.
Let $`a^b=a_1`$, $`a^{b^2}=a_2`$ and let $`c=[a,b]`$. Then $`a,a_1`$, $`a_1,a_2`$ and $`a_1,a_2`$ are each nilpotent of class at most 2, by Lemma 2.1.
Now $`c=a^1a_1`$ and $`c^b=a_1^1a_2`$ and these commute by Lemma 2.1. But

$`1=[c^b,c]`$ $`=[a_1^1a_2,a^1a_1]`$
$`=[a_1^1a_2,a_1][a_1^1a_2,a^1]^{a_1}`$
$`1`$ $`=[a_2,a_1][a_1,a]^{a_2a_1}[a_2,a]^{a_1}`$
$`1`$ $`=[a_2,a_1][a_1,a][a,a_2]`$
Hence $`[a_1,a,a_2]=1`$. Now $`[a_1,a]=[a,b,a]=[c,a]`$ and $`a_2=ac^2[c,b]`$. Since $`[c,a]`$ commutes with $`a`$ and $`c`$, it follows that $`[[c,a],[c,b]]=1`$. Now It follows from Lemma 2.6 that $`[c^a,c^b]=1`$. Now by Remark 1 we have $`c^{b^1}=c^bc^2`$. It follows that $`[c^{ab},c]=[c^a,c^{b^1}]=1`$.
Thus $`a,b`$ is metabelian. Now for completing the proof, it is enough to show that $`[a,b,x_1,x_2,x_3]=1`$ (\*) for all $`x_1,x_2,x_3\{a,b\}`$. Since $`a,b`$ is metabelian we have $`[a,b,x_1,x_2,x_3]=[a,b,x_{\sigma (1)},x_{\sigma (2)},x_{\sigma (3)}]`$ for all permutations $`\sigma `$ on the set $`\{1,2,3\}`$. Now $`a,bL_3(G)`$, it is easy to see that the equality (\*) is satisfied by all $`x_1,x_2,x_3\{a,b\}`$. This completes the proof. $`\mathrm{}`$

The author could not prove a similar result to Theorem 1.2 for groups generated by $`3`$ left 3-Engel elements. We finish this paper by the following question.

Question. Is there a function $`f:`$ such that every nilpotent group generated by $`d`$ left $`3`$-Engel elements is nilpotent of class at most $`f(d)`$?

Acknowledgement. The author is indebted to the referee who has read this paper carefully and patiently and pointed out a serious error in the origin proof of Theorem 1.2 and