Automated Detection

Oncehotpantshas been run on the search and template image, theSExtractoris run on the resulting differenced image. This produces a list of sources, referred to as sdetec- tions, with signal-to-noise>4.0. During SV, the number of sdetections in an “standard"

image was ∼103, but pipeline and template improvements reduced this number by an

order of magnitude. The locations listed bySExtractorare then put through a piece of

code calledfilterObj, which cuts out a square 51 pixel stamp and analyses the sdetec-

tion on the pixel level. A variety of measurements of the stamp are made, and if those

measurements, such as the number of pixels 6σ below the noise level, fall outside the

desired parameter space then the sdetection is cut as junk. These junk s-detections are still recorded, but will not be used in downstream analysis. Anything that passes the cut is called a detection and inserted into an Oracle table called SNOBS, which contains loca-

tion information as well as the flux and magnitude calculated byfilterObj. filterObj

then goes through each detection on the chip, looking for preexisting candidates within 1.80 arcseconds of the detection’s location. If no match is found, the detection is then compared to all detection locations already in the SNOBS table. If a match is found within 1.08 arcseconds of the detection, then the average location is labelled as a new candidate and recorded in our database for further study.

OncefilterObj runs to completion, a monitoring script (NAME TO BE ADDED

LATER, runMon?) is applied to the various DIFFIM outputs that does several quick

quality checks on the images, which are then available through the DiffImg monitor- ing website to the collaborators. Every run is broken down by CCD to check for unusual variations. Additionally, the efficiency in detecting fake supernova for the run is also de- termined. A sufficiently low efficiency for a given chip (or a low efficiency for artificial

SNe below a magnitude of 20.0 ) indicates either a failure of theDIFFIMpipeline, which

may then require reprocessing, or a bad observation, which will require an obstac reset for that field on the following night, as described in 2.1.3.

Machine Learning

§2.2.4 and Appendix A will discuss the human transients detection methods that were used initially in during the SV and Y1 seasons. However, between SV and Y1 consider- able work was put into developing a machine learning algorithm that could be effectively used to remove junk from the DES supernova sample. During Y1, a random forest ma- chine learning algorithm (Statistics & Breiman 2001) was used only to prevent the worst non-transient artifacts from being viewed by the human scanners, thus reducing their overall workload. A very detailed description of the development and methology of the DES machine learning algorithm can be found in Goldstein et al. (2015), but we will summarize the machine learning system here in order to provide a complete picture of the DES supernova search strategy as well as show some additional examples of how the algorithm was improved using further analysis of SV and Y1 data.

The ML algorithm,snautoscanwas developed using thescikit-learnPedregosa

et al. (2012), which is an open source set of Python scripts. The random forest algorithm takes a set of parameters associated with each individual detection and identifies areas of a full parameter space that contain either “good" detections or “junk". In the case of a supernova search, good detection are variable astromical sources and junk detections are are what we call here artifacts. These areas in parameter space are mapped by training the ML classifier using raw imaging data from the differenced imaging pipeline. Such a training set can be created by selecting fake supernova detections or detections classified by humans as transients as to represent the good detections and either random field detec- tions (the vast majority of raw detections in SV and Y1 would be considered artifacts) or

tions as good and randomly selected non-fake field detections (the vast majority of which would be classified as artifacts by scanners anyway) as junk.

In the case of DES, the supernova survey developed its parameter set over both SV and Y1, eventually compiling 38 such parameters that could be used to assign a machine learning score to detections Goldstein et al. (2015). Some of these parameters would be useful to separate data sets by themselves while others are only useful in concert with

other parameters. The machine learning score,τ, Bailey et al. (2007) is derived from the

area of parameter space that the detection exists in represents the probability that a train- ing detection in that region of parameter space is considered good. A test set, composing a dataset that is separate from the training set, can then be used to verify the quality of the trained classifier. Care must be given to when aggregating the training and testing sets to make sure that they both fill the parameters space to roughly the same extent, or

the value of τ for future data is unlikely be a good representation of the actual proba-

bility that a real detection is “good". Figure 2.7 shows distributions ofτ from the latest

version of snautoscan for both fake and field detections, illustrating how the machine

learning score can be used to separate real transient data from artifacts without additional classification by people. The sample presented was chosen such that the signal-to-noise for the plotted detections was relatively low, between 3.5 to 6.0, because higher signal-to-

noise detections present less of a challenge for either humans orsnautoscanto classify.

If the training and testing sets don’t properly represent the true sign-to-noise ratio for

real data andsnautoscancan’t separate faint sources from faint junk then the classifier

would essentially just represent a higher S/N cut than we want applied. Once S/N>6.0

are included the separation between fakes and field detections becomes much more stark, although somewhat less sensitive to the details of the classifier. It should be noted, how-

ever, that the τ values presented here are not for supernova candidates (locations mea-

sured throughout the season), but just detections made on individual filter-epochs and post-processing was used to collect statistics regarding how many detections associated with a candidate passed our machine learning cuts.

0.0

0.2

0.4

0.6

0.8

1.0

τ

0

2

4

6

8

10

12

14

16

Distribution of Machine Learning Scores

Fake Detections

Field Detections

Figure 2.7: The distribution of machine learning scores for detections made in DES Y1, which are used as a filter when creating records of transients. Y1 required 2 detections withau>0.4 on two separate nights to be associated with the same location before a transients was deemed worthy of more attention, such as processing through codepsnid. Scores for fakes that were inserted into the images, which are used here as a

Post Processing

Once all of the fields for a specific night have been put throughDiffImg(successfully or

otherwise), all available candidates are put through machine learning and post-processing. The post-processing module counts all of the unique epochs and detections (or unique fil- ter epochs combinations of detections, if more than one detection from a given observa- tion is nearby) associated with a given candidate location (within 1.0 arcsecond)and uses that information to prioritize the candidates for human scanning. Because it could be run on all detections for a season in aggregate, post-processing became an invaluable tool that allowed for tweaking of the search radius, described above, without the need to reprocess images for a whole season through filterObj. This allowed for us to test several different values for the search radius based on both the minimum radius needed to make a candi- date as well as the distance between candidates and their respective detections. Figure 2.8 shows two plots of the probability distribution of angular distances between candidates and detections, with the overall angular distance PDF on the left and the minimum angu- lar distance PDF on the left. In those plots, the total fraction of candidates that would be retained is shown as a function of what is essentially the search radius cut. The minimum angular distance indicates the absolute minimum distance between a candidate and all of its related detections. By using the minimum angular distance as a measure, it would po-

tentially be possible to make a cut at 0.2 arcseconds which would retain>95% of actual

transients, but would ultimately cut out 50% of the total number of candidates (the vast

majority of which would be junk), without ever leveragingsnautoscanor human scan-

ning. The downside of such a small search radius, however, is that while it would result in little to no loss in discovery efficiency in the long term, when combined with the approx- imately 5 day cadence such a small search radius would ultimately reduce the number of Type Ia supernovae found before peak brightness, which would greatly reduce DES’s ability to obtain any peak brightness spectra of any of its candidates, which are expected to represent 10% of the total supernova sample. And while DES is largely a photometric survey, the ability to obtain spectra of a portion of the total sample is an important goal of

0.0 0.5 1.0 1.5 2.0 2.5 |Candidate - Detection | in arcseconds 0.0 0.2 0.4 0.6 0.8 1.0

Fraction of Detections within Radius

Fake Candidates Field Candidates

0.0 0.2 0.4 0.6 0.8 1.0 Minimum |Candidate - Detection | in arcseconds 0.0 0.2 0.4 0.6 0.8 1.0

Fraction of Candidates Detection

Fake Candidates Field Candidates

Figure 2.8: In order to determine a search radius within which to create candidates that

can be followed throughout the season, we had to determine a minimumΘto distinguish

junk from real astronomical events. On the left we plotted the angular distance between known candidate locations and related detections to determine the maximum angular dis- tance that no longer improves our ability to find candidates. The second plot shows the minimum distance between candidate location and related detections, for which ˜90% have a minimum angular distance below 1”.

the survey that cannot be compromised. This leaves the option of using the total angular distance distribution, where approximately 100% of real transients would be retained by using a search radius of 1.0 arcsecond, which would result in approximately a 30% junk

For the SV season the main requirement was the “numepochs" cut; a candidate needed to have multiple filter-epochs worth of detections within the post-processing search radius before being sent to the human scanning, where a filter-epoch is defined as an individual detection of a transient in one band in a single observation night. Barring the approxi- mately 1% overlap between fields, transient can therefore have up to 4 filter-epochs per night. This requirement alone prevented a significant portion of candidates from reaching future processing steps, but it was most valuable in removing asteroids and Kuiper Belt Objects from the SN candidate sample. During Y1, this same requirement was kept, but in addition to having at least two unique filter-epochs being detected, at least two of the

filter-epochs were also required to pass ansnautoscanτ cut of 0.5. A higher cut would

have provided signifantly less junk, but 0.5 was chosen as the cut value because it does not remove a significant portion of the fake supernovae candidate sample. Future processing runs and seasons might be able to use a higher cut to provide even better sample purity

with future iterations ofsnautoscan.

If the candidate passes the numepochs cut and hasn’t already been classified as a tran- sient and more than one unscanned detection is associated with the candidate then all of the unscanned detections are sent to human scanners for visual identification. Most of- ten this would result in candidates being scanned only one time, but it was possible for candidates to be scanned multiple times, something which happened in many cases ulti- mately at the expense of . Candidates that would be classified as artifacts were rescanned multiple times (every time unscanned detections passed the machine learning cut), which allowed for transients to be identified even in locations that were previously identified as “artifacts."