X-ray data reduction

The general outline of the developed XDCP X-ray reduction pipeline is illustrated in Fig. 6.8. Raw XMM-Newton data for each observation are organized in a so-called Ob- servation Data File (ODF), a directory with a collection of ∼ 200 files containing the uncalibrated data of all instruments, satellite attitude files, and calibration information. ODFs have unique identifications numbers (OBSID) and can be considered as the basic starting data set from which all science products can be derived. An arbitrarily long list of ODFs serves as the input for the X-ray pipeline and is passed on to the top-level script, which organizes the ODF handling and the calls of the individual reduction modules. In practice, 1–3 modules are applied to typically 100 data sets at a time, followed by qual- ity and completion checks of the intermediate data products. All data sets have been homogenously16 _{reduced with the XMM Science Analysis Software version SAS 6.5.}

The full XDCP pipeline can be subdivided into three main parts: (i) The actual X- ray data reduction modules, which create calibrated science images from the raw satellite data, (ii) thesource detectionand characterization procedures resulting in lists of extended sources, and (iii) theinteractive individual detailed source analysis and classification using X-ray-optical image overlays. In this sub-section, we will have a closer look at the data reduction modules17 _{of the first part.}

16_{An early reduction run of 120 XMM observations using SAS 6.1 has been repeated with SAS 6.5,}

released in August 2005.

17_{The individual X-ray reduction modules are adapted and extended versions of codes originally written}

Figure 6.6: Optimized X-ray source detection bands for galaxy clusters and groups as derived by Scharf (2002). Left: Upper (black solid lines) and lower (black dotted lines) detection band-pass versus object redshift for the XMM MOS detectors for systems with different temperatures. From top to bottom the assumed cluster/group temperature is 6, 4, 2, 1, 0.5, 0.2 keV. The blue solid lines illustrate the XDCP detection band with an energy range of 0.35–2.4 keV, which was chosen to yield an optimized high-redshift detection performance over a wide range of temperatures. Center: Same plot for the XMM PN detector. Right: Expected PN signal-to-noise ratio of the standard 0.5–2.0 keV band divided by the SNR of the optimized bands (black lines) in the center panel. The SNR gain is most notable for lower temperature systems atT <_∼2keV. Plots adapted from Scharf (2002).

Figure 6.7: Detection schemes. The relative photon weight factor is plotted against the energy for the unweighted single band scheme (red line), the three-band standard scheme (green), and the spectral matched filter scheme (blue) featuring different photon weights across the spectral range. Small offsets are applied for easier distinction of band boundaries.

X-ray Data Reduction ODF 1

CAL Data Preparation

organize satellite data & calibration files

XDCP X-ray Pipeline

Iterated Flare Cleaning

1) hard-band flare removal 2) soft-band flare removal

Source Detection

Exclusion Mask

mask out contaminating sources

X-ray-Optical Overlays

X-ray Contours

for combined and single detectors

Create Overlays

DSS+X-ray+NED source zoom

ODF 2 ODF 3

Data Processing

run processing chains for all detectors

Calibrated Event Files

Clean Statistics

Image Creation

for different bands and all detectors

Combination & Smoothing

PN+MOSs combined & smoothed images

Cleaned Event Files

Smoothed X-ray Images

Exposure+Background Maps

for all bands

Box Detection & ML Fitting

eboxdetect+emldetect for different methods

Wavelet Detection ewavelet Wavelet Detected Sources CAL Source Analysis+Extraction

display sources & extract source list

Extended Source Masterlist DSS NED X-ray- Optical Overlays

XMM data setup

The first task is to transform the XMM sets of raw satellite data, each ODF containing typically ∼500 MB of data, into a suitable format for further processing. This is achieved with few straightforwardSAS procedures which keep the complexity of the underlying raw data hidden from the user. The taskcifbuildmatches the input data with the appropriate calibration files, which are accessible through the so-called Calibration Access Layer (CAL).

odfingest extracts and organizes the satellite housekeeping data necessary to accurately reconstruct the observations. The instrument specific processing chains epchain for the PN and emchain for the MOS cameras generate calibrated photon event lists for the complete observation with reconstructed sky positions, energies, arrival times, and event characteristics for each individual X-ray photon.

In Sect. 6.2, one of the survey field selection criteria was a minimal nominal exposure time of 10 ksec, which is the period XMM was pointing at the target location. However, the nominal on-target times are not equivalent to the effective science-usable clean exposure times, which can be significantly lower for two main reasons. (i) Instrument calibration overheads at the beginning of an observation require about 0.5–1.5 h for the PN, and 10 min for the MOS instruments in order to compute theoptical loading of the CCD chips, i.e. the signal fraction attributed to optical light rather than X-ray photons. The more sensitive PN camera hence achieves on average about 1 h less on-target time than the MOS instruments. (ii) In addition to the constant hardware overheads, the more or less unpredictable space weathercan contaminate observations resulting in extended periods of lost time. The space weather effects and the different XMM background components are discussed in the following.

X-ray backgrounds and flare cleaning

The XMM background is the prime constraint for the achievable sensitivity limit of low surface brightness extended sources. For its importance, the main background compo- nents18 _{are shortly introduced followed by a discussion of the implemented flare cleaning}

procedure.

Cosmic X-ray Background (CXB): The cosmic X-ray background can be separated into two main contributions. (i) The hard component of the CXB dominates at energies_∼>1 keV and is mostly attributed to the integrated light of faint, unresolved extragalactic X-ray sources, predominantly Active Galactic Nuclei (AGN). Extremely deep 1 Msec pencil-beam observations with Chandra have resolved the largest fraction of the hard CXB into individual point sources (e.g.Mushotzky et al., 2000; Giacconi et al., 2001). The total CXB flux in the 2–10 keV hard band reaches an approximate surface brightness of 5×10−15_{erg s}−1_cm−2_arcmin−2_{. (ii) The soft component of the}

CXB, dominating below 1 keV, can be attributed to thermal diffuse emission of galac- tic origin. It originates from plasma of the Local Hot Bubble, the Galactic Disk, and 18_{All XMM background components are summarized by A. Read at http://www.star.le.ac.}

the Galactic Halo and is consistent with thermal emission of about T∼0.1–0.2 keV. The CXB is constant in time, but thesoft componentvaries spatially across the sky.

Flaring Particle Background: This background component was severely underesti- mated before the launch of XMM and is responsible for most of the lost science time. The flaring background has strong rapid variability and is attributed to soft protons of solar origin with energies of a few 100 keV. These protons, which seem to be organized in ‘clouds’ populating the Earth’s magnetosphere, are funnelled by the X-ray mirrors towards the detectors, where they typically deposit several keV of their energy in the CCD pixels. The occurrence of the soft proton flares is unpredictable but is related to the solar activity and the satellite position with respect to the mag- netosphere. Flare intensities can reach 1 000 times the quiescent background level implying that contaminated time intervals have to be carefully cleaned and removed from the science data. Since the soft protons pass the mirror system, they imprint a (partially) vignetted background component onto the detectors.

Quiescent Particle Induced Background: This second particle induced component dominates the quiescent background level. It is attributed to high-energy cosmic ray events of some 100 MeV, where a charged particle interacts with the detector material and the surrounding structures to produce fluorescence emission. This more stable internal background component typically varies only on the±15% level during obser- vations, but can be an order of magnitude enhanced in high radiation periods. The cosmic rays induce a background component originating from the detector material itself,i.e. the quiescent particle background is not vignetted, but reflects the spatial structure of the surrounding detector material. The most prominent instrumental fluorescence lines are due to the Al-Kα transition at 1.5 keV for all instruments and additional Si-Kα emission at 1.7 keV for the MOS detectors. Another intense line complex of several elements (e.g. Cu, Ni, Zn) arises around 8 keV for the PN and is hence beyond the considered bands for the survey.

Instrumental Noise: The last considered background contribution is electronic detector noise. This component is negligible in the harder bands but becomes significant at low energies of<_∼0.3 keV. The low energy band limit for the survey of 0.3 keV is thus motivated by the onset of increased instrumental noise in conjunction with enhanced galactic hydrogen absorption.

The top left panel of Fig. 6.9 shows a typical 100 sec-binned light curve in the PN 12- 14 keV hard band of a 47 ksec pointing. The soft proton flare during the central part of the observation is obvious and can be particularly well identified in the hardest band due to the flat nature of the flare spectrum. The pipeline flare removal procedure of the first cleaning stage automatically identifies the stable quiescent level based on a Gaussian peak-fit in the count-level histogram of the time series (center left panel of Fig. 6.9). All observation periods with background levels less than three standard deviations above the determined quiescent level are accepted as good time intervals (GTI) for the initial hard band cleaning.

Figure 6.9: X-ray flare cleaning procedure. Top row left: 100 sec-binned PN light curve of the complete field in the hard X-ray band at 12–14 keV. The flared periods are easily visible and are removed during the first cleaning stage. Right: 10 sec-binned PN ‘soft’ band light curve at 0.3– 10 keV. The missed rising flanks close to the onset of flare periods and short flares are identified during the second cleaning stage. Center row: Automatic good time interval analysis based on the identification of the quiescent level and specified κσ-clipping cut-levels for cleaning stage one (left) and stage two (right). Bottom row: Final light curve of the identified good time intervals for the PN (left) and MOS (right) detectors.

This first cleaning stage efficiently removes the largest fraction of flare-contaminated data. However, at lower energies the flares can exhibit a different time behavior than the main hard component, which is currently attributed to lower energy particles at the edges of the encountered proton ‘clouds’. The upper right panel of Fig. 6.9 displays the 10 sec- binned light curve in the full 0.3–10 keV soft X-ray band after the initial flare removal. The residual flare peaks are identified during the second soft-band cleaning stage19 _and

are removed from the remaining good time intervals via 5-σ clipping of the soft energy histogram (center right panel of Fig. 6.9).

The lower panels of Fig. 6.9 show the resulting light curves (100 sec-binned) after the iterated flare cleaning procedure for the PN instrument (left) and one of the MOS cameras (right). The contaminated time intervals are completely removed from the final cleaned photon event lists resulting in an effective science-usable exposure time of 29.6 ksec or 63% of the nominal time for this field.

The flare removal procedure is applied to all instruments independently since the higher low energy sensitivity of the PN camera results in a different temporal response of the system. For about 80% of all XMM fields, theautomatic double flare cleaning procedures yields good and stable results in identifying the good time intervals. If more than half of the observation is contaminated, the automatic determination of the quiescent background level might fail and require a manual definition of the GTI cuts.

X-ray images

With the cleaned photon event lists at hand, we can now create X-ray images for all subsequent analysis tasks. The energy information of the X-ray photon events allows the arbitrary definition of customized band-passes at this point. As discussed in Sect. 6.3.1, five different energy bands are considered for the XDCP survey. Photons of the 0.35–2.4 keV energy range are selected for the optimized single detection band, the ranges 0.3–0.5 keV, 0.5–2.0 keV, and 2.0–4.5 keV for the standard band method, and an additional 0.5–7.5 keV broad band for the spectral matched filter scheme, which uses all five bands.

The X-ray images are reconstructed with 4 arcsec pixels from the cleaned photon event lists by applying the appropriate energy cuts and conservatively selecting only well cal- ibrated events as characterized by event quality flags. The absolute astrometry, i.e. the sky coordinate information, is typically accurately determined to within approximately one arcsecond. Images are produced for each instrument individually resulting in a total of 15 band-detector combinations.

At this stage, the correction for out-of-time events of the PN detector is performed. Based on the observed count rate in each pixel, the smeared streaks of photons with mismatched Y position information can be statistically reconstructed in an OoT image20_.

This OoT frame is created for each band and subtracted from the corresponding raw PN data to yield a first order correction of the out-of-time image artifacts. Although the fraction of smeared photons is only 2.3% or 6.3% (depending on the imaging mode), the trails of very bright central target objects can mimic apparently extended sources and should therefore be corrected.

19_{The double flare cleaning procedure was originally proposed and applied by Pratt and Arnaud (2003).} 20_{A reconstructed OoT event list can be obtained with the taskepchain.}

Observations that have been interrupted,e.g.due to technical problems or strong flares, are often split up into several shorter data blocks per instrument. The separate event lists are individually reduced, cleaned, and transformed into X-ray images. These shorter exposures are co-added for each band-pass and instrument at this point in order to restore the full on-target integration time of the observation.

As a last step of the data reduction part, images of the same band are combined for the three detectors to yield the full XMM imaging information in the specified energy range. In order to enhance the contrast of weak sources for visual inspection, the combined images are additionally smoothed with a 4 arcsec Gaussian filter21_{. Figure 6.10 displays the raw}

(center left panel) and smoothed (upper left panel) combined 0.35–2.4 keV images of one of the deep XMM survey fields. The first part of the XDCP pipeline is now finished and we can proceed with the source detection module.