4 Data reduction

For the data reduction we used the Australia Telescope National Facility (ATNF) release of the Multichannel Image Reconstruction, Image Analysis and Display (MIRIAD) software package (Sault et al. 1995). A number of steps in the reduction process are usually done interactively, specifically the removal of bad data ("flagging''), and, in a certain measure, cleaning and self calibration. Due to the large amount of data involved we found it more practical to develop a semi-automated reduction pipeline.

  
4.1 Flagging of bad data

 

 

Table 3: Main parameters for the 16 final mosaics

Mosaica	Fields	Tangent Pointb		Synthesized Beamc		Smin	$\sigma_{\mathrm{fit}}$	$<\sigma>$
fld x to y	$n \times m$	$\alpha_0$	$\delta_0$	$b_{\mathrm{min}}\times b_{\mathrm{maj}}$	PA	mJy	$\mu$Jy	$\mu$Jy
fld01to06	$6\times 4$	22 35 57.37	-39 59 15.0	$7.8\hbox{$^{\prime\prime}$ }\times 12.9\hbox{$^{\prime\prime}$ }$	$1\hbox{$^\circ$ }$	-0.49	78.7	$78.5\pm 3.0$
fld05to11	$7\times 4$	22 44 57.60	-39 59 15.0	$7.8\hbox{$^{\prime\prime}$ }\times 12.8\hbox{$^{\prime\prime}$ }$	$3\hbox{$^\circ$ }$	-0.96	77.8	$78.5\pm 3.8$
fld10to15	$6\times 4$	22 53 21.68	-39 59 15.0	$7.9\hbox{$^{\prime\prime}$ }\times 13.0\hbox{$^{\prime\prime}$ }$	$-1\hbox{$^\circ$ }$	-0.70	88.1	$84.1\pm 5.3$
fld20to25	$6\times 4$	23 34 56.09	-39 58 48.0	$8.5\hbox{$^{\prime\prime}$ }\times 16.8\hbox{$^{\prime\prime}$ }$	$1\hbox{$^\circ$ }$	-0.68	83.0	$80.3\pm 4.3$
fld24to30	$7\times 4$	23 43 38.26	-39 58 48.0	$9.5\hbox{$^{\prime\prime}$ }\times 16.6\hbox{$^{\prime\prime}$ }$	$-2\hbox{$^\circ$ }$	-0.50	82.8	$83.2\pm 6.4$
fld29to35	$7\times 4$	23 52 20.43	-39 58 48.0	$8.9\hbox{$^{\prime\prime}$ }\times 16.7\hbox{$^{\prime\prime}$ }$	$2\hbox{$^\circ$ }$	-0.53	79.2	$76.2\pm 2.9$
fld34to40	$7\times 4$	00 01 02.59	-39 58 48.0	$8.0\hbox{$^{\prime\prime}$ }\times 14.5\hbox{$^{\prime\prime}$ }$	$5\hbox{$^\circ$ }$	-0.48	76.3	$74.8\pm 4.0$
fld39to45	$7\times 4$	00 09 44.76	-39 58 48.0	$8.6\hbox{$^{\prime\prime}$ }\times 14.3\hbox{$^{\prime\prime}$ }$	$-10\hbox{$^\circ$ }$	-0.61	81.2	$80.4\pm 3.4$
fld44to50	$7\times 4$	00 18 26.93	-39 58 48.0	$7.4\hbox{$^{\prime\prime}$ }\times 14.0\hbox{$^{\prime\prime}$ }$	$7\hbox{$^\circ$ }$	-0.43	78.0	$76.4\pm 1.7$
fld49to55	$7\times 4$	00 27 09.10	-39 58 48.0	$8.0\hbox{$^{\prime\prime}$ }\times 12.9\hbox{$^{\prime\prime}$ }$	$10\hbox{$^\circ$ }$	-0.62	78.6	$77.1\pm 1.4$
fld54to60	$7\times 4$	00 35 51.26	-39 58 48.0	$8.2\hbox{$^{\prime\prime}$ }\times 12.6\hbox{$^{\prime\prime}$ }$	$4\hbox{$^\circ$ }$	-0.42	77.3	$75.8\pm 1.3$
fld59to65	$7\times 4$	00 44 33.42	-39 58 48.0	$7.7\hbox{$^{\prime\prime}$ }\times 12.5\hbox{$^{\prime\prime}$ }$	$4\hbox{$^\circ$ }$	-0.44	79.4	$77.8\pm 1.4$
fld64to70	$7\times 4$	00 53 15.58	-39 58 48.0	$7.7\hbox{$^{\prime\prime}$ }\times 12.6\hbox{$^{\prime\prime}$ }$	$1\hbox{$^\circ$ }$	-0.45	75.1	$74.6\pm 2.2$
fld69to75d	$7\times 4$	01 01 57.70	-39 58 48.0	$7.5\hbox{$^{\prime\prime}$ }\times 12.8\hbox{$^{\prime\prime}$ }$	$-2\hbox{$^\circ$ }$	-0.47	81.9	$79.3\pm 2.1$
fld74to80	$7\times 4$	01 10 39.86	-39 58 48.0	$7.0\hbox{$^{\prime\prime}$ }\times 15.0\hbox{$^{\prime\prime}$ }$	$1\hbox{$^\circ$ }$	-0.61	77.1	$76.3\pm 3.8$
fld79to84	$6\times 4$	01 19 22.03	-39 58 48.0	$6.8\hbox{$^{\prime\prime}$ }\times 14.3\hbox{$^{\prime\prime}$ }$	$7\hbox{$^\circ$ }$	-0.41	68.9	$67.6\pm 3.5$

^a x and y refer to the first and last field columns composing the mosaic. ^b J2000 reference frame.

^c PA is defined from North through East. ^d Reported values for S_min and noise refer to masked mosaic (see text).

We made a modified version of the MIRIAD task TVFLAG (inserted in MIRIAD as TVCLIP), which recursively flags visibilities with amplitudes exceeding a given threshold. The threshold was set as a convenient multiple of the average absolute deviation ( $\vert\Delta S\vert$) from the running median, evaluated separately for each baseline, each channel and each integration cycle (10 s).

For the primary calibrator the automated flagging procedure was applied before the calibration. This was necessary to avoid the calibration being affected by bad data. For the secondary calibrators and the mosaic data the bandpass and instrumental polarization calibration were applied before running TVCLIP. As we noticed that the shortest baselines introduced some low level, spatially correlated features in the images, which could affect the zero level for faint sources, we decided to remove all baselines shorter than 500 m from the data prior to the pipeline processing (rejection of $\sim 10\%$ of the visibilities). As a consequence, the ATESP survey becomes progressively insensitive to sources larger than $30\hbox{$^{\prime\prime}$ }$: assuming a Gaussian shape, only $50\%$ of the flux for a $30\hbox{$^{\prime\prime}$ }$ large source would appear in the ATESP images. However, the expected fraction of sources with angular sizes $\geq 30\hbox{$^{\prime\prime}$ }$ is very small: $\leq 2\%$ at fluxes $S\leq 1$ mJy according to the Windhorst et al. (1990) angular size distribution.

  
4.2 Cleaning and self-calibration

Since the primary beam response is frequency dependent, we did not merge the data from the two observing bands before imaging and cleaning. This results in a slightly poorer UV coverage but allows the cleaning process to succeed in subtracting correctly 100% of the source flux, and self-calibration to be more effective and reliable.

On the other hand, to improve UV coverage and sensitivity, for each field we have merged the (calibrated) data coming from all the observing runs.

In contrast with the imaging of extended sources, joint deconvolution is not needed for a point source survey. It is also very expensive computationally for high resolution images. We therefore reduced every field separately, simplifying imaging considerably.

For each field a $2048\times 2048$ pixel image (total emission only) was produced (pixel size = $2.5\hbox{$^{\prime\prime}$ }$). The entire image was then cleaned in order to deconvolve all the sources in the field. To improve sensitivity we used natural weighting, which gave a synthesized beam typically of the order of $8\hbox{$^{\prime\prime}$ }\times 14\hbox{$^{\prime\prime}$ }$.

Each image went through different cleaning cycles. First, we produced the list of the brightest components to use as model for self-calibration. Phase only self-calibration was applied. Usually two iterations were sufficient to remove phase error "stripes'' and improve the image quality.

The self-calibrated visibilities were then used to produce a deeper cleaned image. A serious problem arises if snapshot images are cleaned too deeply. Due to the incomplete UV coverage the number of CLEAN components can approach the number of independent UV points. At this point the cleaning algorithm is not well constrained anymore and can interprete noise (sidelobes, calibration errors, etc.) as CLEAN components. This process can redistribute the noise into the sources and, as a result, the process does not converge and, in principle, the image can be cleaned to zero flux. This produces many faint spurious sources, while the flux of real sources is systematically underestimated. This effect has been mentioned by Condon et al. (1998) and White et al. (1997) for deep snapshot observations with the VLA and is referred to as "clean bias''.

From our tests we found that cleaning down to $3\sigma$, the noise gets $\sim 20\%$ lower than theoretical. Down to $2\sigma$ it is a factor of two lower and at $1 \sigma$ can be 4 - 5 times lower. Thus we decided to stop any cleaning process when the peak flux residuals are of the order of 4 - 5 times the theoretical value (setting a cut-off of 0.5 mJy) to minimize the clean bias (a few percent effect expected, but see discussion in Sect. 5.3).

After this first phase of self-calibration and deep cleaning, we subtracted the sources from the visibility data and we repeated the flagging procedure on the residual visibilities in order to reduce the "birdies'' effect mentioned in Sect. 5.1 below.

We then proceeded with another phase of cleaning, and produced a half-resolution residual image covering 4 times the original area; only external parts of this image were cleaned (not the inner quarter, corresponding to the original field, which was already cleaned down to 0.5 mJy). This procedure allowed us to remove the sidelobes from more distant sources (belonging to adjacent fields). These new components were subtracted from visibility data before restoring the sources in the final cleaned image.

As a final step we checked for bright, extended sources in the field, which needed deeper cleaning. A small box containing such a source was cleaned, to a $2\sigma$ level. In general, the application of all these cleaning steps produced good quality single field images.

Figure 2: One of the $6\times 4$ fields mosaiced images. The rectangular box indicates the region corresponding to the ESP redshift survey

  
4.3 Mosaicing

The cleaned single field images were co-added in mosaics in order to improve the signal to noise ratio and get uniform sensitivity. Each set of $5\times 4$ fields observed in $2\times 12^{\mathrm{h}}$ observing blocks produces a separate independent mosaic; an overlap between adjacent mosaics was created by adding one (or two) extra column(s) of fields to the side(s) of each mosaic.

Before any mosaic is produced, every field was restored using the same values for the beam parameters. The restoring parameters were chosen as the average value of all the fields composing the mosaic at both frequencies.

The final mosaics were obtained in two steps. First a single frequency mosaiced image was produced for each of the two observing bands, then the final mosaic was obtained by averaging (pixel by pixel) the two initial mosaics in order to improve the sensitivity.

One of our final mosaics is shown in Fig. 2 as an example. The rectangular box indicates the region corresponding to the ESP redshift survey, where the radio survey was designed to give uniform noise. Such a region covers an area of $1.67\hbox{$^\circ$ }\times 1\hbox{$^\circ$ }$ or $2\hbox{$^\circ$ }\times 1\hbox{$^\circ$ }$for mosaics composed by $6\times 4$ or $7\times 4$ fields respectively (see Table 3). Hereafter we will always refer to the central box only in our mosaic analysis. All mosaics are available through the ATESP page at http://www.ira.bo.cnr.it