2.1.1 Images

Images from CFHT datasets belong to one of the four deep fields (noted D1, D2, D3 and D4) during the first three years of the survey: the D3 field. We use this field as a limited sample that allows us to test our contributions within the limits of our technical resources. These first three years, from 2004 to 2006, have available valuable results regarding the detection of SNIa among the images [^{Guy et al.(2010)}]. These images are obtained using the Megacam camera. The Megacam has 36 charge-coupled devices (CCD) capable of providing an image. The number of images per filter varies as shown in Table 1; the total number of images in the used field is 54614. It is noticeable that most images use filters r and i, which justifies why we focus some of our results on the former.

These deep fields have the following characteristics:

2.1.2 Light Curves

To validate our results, we use the SNIa reported by the SNLS and identified using spectroscopy. This survey has identified and measured the photometric properties of 252 SNIa with a redshift range of between 0.15 and 1.1 [^{Guy et al.(2010)}] and reported at least 485 SNIa candidates [^{Bazin et al.(2011)}]. The first SNLS sample provides one of the most uniform sets of SNIa available, but the actual number of objects per deep field is low, providing only 75 SNIa for the D3 field in the three years of observation. Nevertheless, these supernovae serve as the ground truth for the detection.

3.2.1 Image Pre-Processing Task

Firstly, the Image Pre-Processing task encompasses the calibration of the initial exposures, the artifact correction, the detection of visible sources, and the masking of saturated and edge pixels. This processing provides a relatively clean set of images with noise corrections, masked saturated pixels and cosmic rays. These images are used as inputs for the Image Coaddition tasks. The result of this first task is a set of calibrated exposures or calexps.

3.2.2 Image Coaddition and Input Selection

Typically, when using coadditions in LSSTsp, the final user is in charge of selecting calibrated exposure inputs and there are no enforced rules within the code to differentiate between a coaddition for a reference image and one for a science image. This process is reached through the use of two tasks and it is indifferent to the purpose of the output. In the pipeline, we opt to fuse both generation and assembly tasks together and then specialize each one to generate the coaddition for science and reference calexps. We call these the Day Coaddition and the Deep Coaddition tasks. This allows us more control over each process independently, without affecting the other. From a technical perspective, this also mitigates the impact of bottlenecks for global field processing and allows the new tasks to be more time efficient when generating the deep coadditions.

Selecting high quality inputs is vital to ensure a good coaddition and subsequently good difference images. Using low quality inputs without any sort of quality metric has the potential of propagating noticeable errors and aberrations from the coaddition, all the way to the detection of transient candidates. Errors on the images come from the sky background and increase as the square root of the signal, bad atmospheric conditions or deficient seeing. As the image subtraction operation alters the inherent noise of the inputs through convolution, they become problematic and the artifacts become more visible most of the time. We choose to address this issue for deep coadditions.

Day Coaddition Task

The objective of this task is to generate a high quality science coaddition image, as mentioned before. Day coadditions, or dayCoadds, are built by using the different exposures acquired on a given night. As there is a limited number of acquisitions per night (between 4 and 10 in most cases), we use all available exposures to ensure the best possible quality without performing a selection of high quality inputs. This way of generating day coadditions means that very short-lived objects (less than one night) are not registered in the detections or might appear as single point detections.

As with science images, we expect the transient and variable objects to be present here; thus, when performing the OIS, we match their PSF to that of the astrometrically corresponding deep coaddition to detect them after subtracting both.

Deep Coaddition Task

Deep coadditions created from a reference image set are referred to as refCoadds. These coadded exposures, which come from a wider time frame belonging to an outside acquisition year, help to detect all changes in science images during the subtraction. As explained before, the convolution Kernel that calculates the OIS when matching with the day coaddition modifies these images. High quality images provide better and less complex PSFs to be parametrized by the sum of Gaussians via the OIS, reducing the overall artifacts later when a subtraction is performed.

Fig. 2 First part of the pipeline: the generation of difference images. The tasks here are responsible for generating the reference and science image coadditions, and then performing the subtraction using the OIS algorithm. The Deep Coaddition task generates the reference image, whereas the Day Coaddition task creates the science image. The Image Subtraction task generates the difference image as final result. 

Ensuring the quality of the inputs is especially important when constructing the deep coaddition, because input datasets for these can span up to hundreds of different images acquired during several months of operation. Therefore, we construct these references guaranteeing high signal-to-noise on the result and thus obtaining high quality difference images. To achieve this, we perform an selection using the noise variance levels on the input images after warping and adjusting them to their respective patch in the skymap, but before assembling the final coaddition. As noise levels change locally within the image and can present variation between the original calexps and the smaller patches used for the coaddition, it follows that the optimal approach is to use the values in the patches as criteria for the selection. We call this process the Input Selection.

3.2.3 Optimal Image Subtraction

As already mentioned, we use the OIS [^{Alard & Lupton(1998)}, ^Alard(2000)] LSSTsp task with the new dayCoadds and refCoadds as inputs. We use the default parameters of 3 Gaussians with degrees 4, 2 and 2. As expected, the OIS generates a difference image that is the output of the current stage and will be the input of the next one

3.3.1 DIASource Selection

This first task of this stage is divided in two phases: The DIASource detection and the DIASource labeling.

DIASource Detection

The detection of sources in difference images is performed using the default single frame measurement algorithm of the LSSTsp that searches positive and negative local peaks, and using a detection threshold of 5σ. Each detected source is then stored into a catalog created for each patch of the defined skymap. Every source is characterized by its coordinates, its flux measurement and its flux error which are calculated from the aperture flux. We use an aperture with a radius of 4.5 pixels that has consistently yielded the most stable results of flux measurements while reducing the overall measurement error in our experiments.

Fig. 3 Second part of the pipeline: Light curve detection and selection. The tasks here include the detection of DIASources; and the detection and selection of light curves. 

Previous analysis using the LSSTsp suggest that an underestimation of noise in the CFHT images ends up generating a high number of detections during this step [^Slater(2016)]. To mitigate this, we re-estimate the value of background noise on the images using the algorithm in the LSSTsp and then recalculate the threshold value defined for the image in the given band. This process, even if time-consuming, ensures that there are less bogus detections in the images, and guarantees that the noise is not inflated.

When a DIASource is detected, one or two footprints are calculated on a 20×20 local neighborhood centered around the source. Footprints F are connected regions that start as contiguous patches of high-intensity pixels that are then expanded to include up to 4 pixels away from the initial pixels. As these footprints can be calculated in difference images, the possible footprints will group pixels with positive or negative values. Overlapping footprints with the same sign are merged. Figure 4 shows an example of positive footprints calculated on a calibrated image. Consequently, a given DIASource can have either one positive F ⁺ footprint, one negative F ^- footprint or both at the same time. Footprints are defined as follows:

where R’ is the result of a region growth over an initial group of pixels R ₊ or R _- such as:

Here, d((x',y'),(x,y)) is the Euclidean distance between pixels I(x',y') and I(x,y). g is the growth value of the region.

Fig. 4 Example of sources with footprints overlaid as semi-transparent blue groups of pixels. The color figure can be viewed online. 

R ₊ and R _- are initial groups of at least 5 pixels for which:

Here th is the threshold value, and x and y represent the pixel coordinates.

Notice that, as the region growing algorithm determined by equation 3 does not take into account pixel values, footprints F ⁺ and F ^- can have intersections.

DIASource Labeling

Once the footprints of a DIASource are calculated, the number of pixels and their flux are measured. These values are used to label the corresponding DIASource. The analysis of DIASources is a custom task independent from similar ones currently present in more recent versions of the LSSTsp such as DipoleAnalysis. The possible labels that we assign are:

Let us define:

To determine whether a DIASource is a Dipole or a Fringe, we define two properties: Geometric Dipoles GD(F¯) and Photometric Dipoles PD(F¯).

A Geometric Dipole GD(F¯) is defined using the following equation: @mathfonts

A Photometric Dipole PD(F¯) is defined using the following equation: @mathfonts

Therefore, using the properties defined in equations 7 and 8, we define a Dipole D(F¯) as:

And a Fringe Fr(F¯) as:

To summarize, we evaluate the amount of overlapping in terms of pixels and calculated flux to determine the label for a given footprint. The proportions are empirically determined, leaving the possibility of a more in-depth study and refinement of these categories. Visual examples of their aspect are presented in Table 2.

TABLE 2 Footprint labels defined for every source detection based on their footprints* 

Label	Visual Aspect	Description
Positive		Sources that only have positive footprints. Their presence usually is the result of a transient object in the science image.
Negative		Sources that only have negative footprints. These are rare residuals and they sometimes indicate transients present in the reference images.
Dipole		Sources with positive and negative footprints. These artifacts occur when there are alignment problems between the images.
Fringe		Sources with positive and negative footprints. These artifacts occur when there are inaccuracies in the PSF matching algorithm.
Artifact		Sources with both positive and negative footprints, but that do not fall in the Dipoles or Fringes categories.

*The Visual Aspect column shows a general idea of the type of objects labeled as such.

3.3.2 Light Curve Generation

From the detected and labeled DIASources obtained from the previous step, we match all DIASources that are located at the same coordinates within a 1 arcsec of radius in all the different images along all the seasons, under the assumption that all these detections belong to the same object. For each set of matched DIASources, we build a light curve and assign an object ID to it. Consequently, each light curve may be composed of several DIASources and always has at least one.

The output of this task is a catalog of the light curves for every filter. Each light curve is stored as set of DIASources with all the relevant information such as the coordinates, flux, error, magnitude, filter, and date (MJD format). The study of the DIASources and its classification is helpful to identify which type of object is being described by the light curve.

3.3.3 Light Curve Selection

A consequence of the previous task is that a detected light curve can technically contain, in the worst case, a single point. Isolated detections of an object do not strongly indicate the presence of supernova-like transients; in fact, they can be indicative of anomalous subtractions, short-lived events, or transients who vary in intensity and position at the same time. The Light Curve Selection task is implemented so that it chooses only curves that satisfy two criteria: at least five total detections and data present in at least two different bands. This ensures that the object has a certain continuity between observations and that its residual light intensity can be picked in different bands.

Although these criteria might appear to be simplistic, they perform a considerable reduction of the candidate set, while also being conservative enough not to reject true transients.

Real data

When we tested our pipeline with real data (CFHT images), it preserved among the candidates 85% of the supernovae reported by SNLS (64 out of 75).

We also determined the quality of the supernova-like candidates by measuring the proportions of DIASource labels per object. Results on the CFHT images show that, before Input Selection, at least 33% of the objects presented light curves with 70% or more detections labeled as artifacts, whereas after the Input Selection only around 6% presented this property. For the dipoles, the difference was less stringent, as some of the built light curves still had a mixture of dipole and positive detections, that can be probably attributed to subtraction problems and other imaging issues.

Before Input Selection, the distribution of positives per light curve was so low that there were no strong candidates with this quality. Our results show that 72% of the light curves have 80% of DIASources labeled as positive. The reduction of the number of artifacts with the extra options of the pipeline is notable: originally, at least 33% of objects we obtained have light curves with 70% or more detections that were labeled as artifacts, whereas for our pipeline only less than 1% presented this quality.

Simulated Data

In the case of simulations, we tested the Candidate Selection by injecting 5000 artificial supernovae. While obtaining 80% less total detections, we were able to find 75% of the injected supernovae. By creating simulations that avoided injecting SNIa too near the edges of patches we were able to increase the number of found SNIa to between 75% and 78%. The detection efficiency drops at a redshift of about 0.8.

Figure 8 illustrates that most of the light curves that were not found by the algorithm have a low number of individual points and a considerable number of high magnitude points. These qualities make these curves more susceptible to be missed, thus reducing the overall efficiency. By using curves with points with a lower magnitude, and a slightly higher number of points, we improved the pipeline’s response to the simulated events.

Fig. 8 Heatmap of light curves per configuration of maximum magnitude values vs number of points for the simulated curves that were not found (Top) and the simulated curves that were successfully found by the algorithm (Bottom). The color figure can be viewed online. 

The results in this case are highly dependant on the simulated curves and the magnitude used. It is possible to vary the threshold to allow more candidates, at the cost of more spurious candidates.

The distribution of positive residuals among the simulated type Ia supernova light curves showed that 95% of these detected supernovae had at least 90% or more of their individual detections labeled as positive residuals. Thus, they have an overall higher quality and demonstrate that we are able to obtain a cleaner candidate set.

It is important to notice that, even if we are able to greatly reduce the load by sacrificing some accuracy, other works provide better insights on how to improve these results. For instance, [^{Goldstein et al.(2015)}] use machine learning methods to reduce by 11% the load of candidates to evaluate with a loss of only 1%. We also believe that an adapted threshold for detection might mitigate this loss, at the cost of augmenting again the number of candidates.

Given the fact that the proportion of positive labels among all objects rose in the light curves we obtained, we assert that our enhancements go in the right direction. Reducing the number of DIASouces and increasing their quality provides a more accurate detection of transients. This accuracy is not only susceptible to increase by applying methods to automatically classify SNIa, but also when using high quality datasets with strategies such as Input Selection.