Subsequent to the transient database search (including all re-fitting, monitoring and re-assessment of variability), an essential part of the pipeline procedure is to in- spect the large quantity of information generated. I developed two methods to do this. One method was an interactive tool to browse through data using the FITS viewing package DS9. The second made use of HTML to produce large numbers of webpages which contained all lightcurves and information of potential transients. These were both exploited in the testing conducted throughout this thesis and I dis- cuss them in more detail in the following sections.
4.3.1
Interactive tools
There are two interactive query toolbars shown in Figure 4.6 (written in Python Tk- inter) that communicate between the LOFAR database and DS9. In the top toolbar clicking ‘Show Transient Candidates’ will send – via Python – the MySQL query to the database which will return the locations of all the transient candidates within a given sequence of images. It will then pipe the locations to DS9 and overlay them on a master image. Each source is labelled with its unique source id. number, and the relevant lightcurve can be retrieved using the toolbar. The lightcurve is gener- ated by querying the database for the correct flux (integrated and peak, both with errors) and time; the results are then plotted with Pylab. The toolbar can also show the locations of the persistent sources within the field, which is shown in Figure 4.6. The bottom toolbar uses a similar system but instead overlays only the transient candidates in the image which they are detected. The location of each FITS image is stored in the database (as metadata) such that each transient identification is linked to an image location. This information can be pulled from the database to view the transient locations specific to a given image. The toolbar can then be used (at speed) to look through a large sequence of images using the scroll buttons. Both these toolbars we used extensively in the VLA archival transient search presented in Chapter 5, inspecting over 5000 images.
Figure 4.6: Example visualisation GUI showing the use of a Python coded toolbar to interact with the database, in conjunction with DS9. Large numbers of images can be inspected – at speed – and the positions and source id. numbers of transient candidates can be viewed. Lightcurves can also be generated for any intersting source(s). These tools were used extensively in the VLA transient search presented in Chapter 5.
4.3.2
Webpages
An alternative to using the interactive tool was to generate HTML webpages (one per source) containing the all information relevant to the transient candidates. This algorithm was coded within the pipeline to post-process any transient detection (af- ter source extraction and the database search). The sources id. numbers of candi- dates were piped to a visualisation function which performed the following tasks (for each source independently).
• The locations of the stored FITS images was queried and returned from the database; the pipeline then used the FITSCUT1tool to extract a JPEG thumb- nail of 50 × 50 pixels around the transient location (for each image). The locations of the JPEG images were then added to the source webpage. It is possibly anticipated that data storage capabilities might be such that even
1FITSCUT is a tool to produce a cutout in the form of FITS, PNG or JPEG of a master FITS
storing FITS images with LOFAR might be impractical, therefore extracting thumbnails may be required.
• The lightcurve was extracted from the database, plotted and added onto the webpage.
• Metadata for each source was added (in text) to the webpage. This metadata included RA and DEC, detection level and timescale of variability (if detected multiple times). The peak and integrated fluxes were compared to assess if the source was a point source – a classification of ‘point’ or ‘resolved’ was then added to the page accordingly.
• If the source was a single epoch detection, all images were refitted with ellip- tical Gaussian and the lightcurve was added to the page.
• If there was an associated catalogued source (NVSS, WENSS or VLSS) the associated source id. number and distance to the source were also added to the webpage. This was used to check that sources had been correctly associated with the NVSS, WENSS and VLSS counterparts.
An example webpage is shown in Figure 4.7. The infrastructure described above could be modified to retrieve any information from the database (using MySQL), which can subsequently be used to generate outputs on the webpages.
All sources that were deemed variable or transient were processed by using the procedure described above. The final list of transient sources and the links to the webpages generated, were ordered based on either detection threshold (sin- gle epoch) or index of variability for variable sources. After the pipeline had fin- ished the post-processing, a firefox browser was automatically loaded to let the user browse the results.
The tools described above have been useful to heavily test various LTraP settings, to expose bugs within the pipeline code and to proceed with commissioning. They have also been important to test the possible future full scale visualisation needed for LOFAR. The tools developed for the work conducted within this thesis did how- ever have their pitfalls. The toolbars were good with large numbers of images, with a small number of sources. The webpages were useful for large number of sources with a small number of images i.e. it is inpractical to view large numbers (>100) of thumbnail images on a webpage. The final LOFAR visualisation toolkit may indeed include implementations of both webpages and software. One choice would to be to use the PHP dynamic internet language, which can execute embedded MySQL
Figure 4.7: Example webpage for a transient source generated from the visuali- sation library. Included on the webpage are: thumbnail cutouts of a region around the transient position for each epoch; an automatically generated lightcurve which included the fluxes of multi-wavelength counterparts; statistical information; and other diagnostics plots relevant to the detection.
commands; this could be used to generate source webpages, on-the-fly. However, it is still cumbersome when viewing large numbers of thumbnail images, which may therefore require specifically designed software.