Daily Processing Script Version 2.0

The inclusion of twelve additional CGPS stations during the re–analysis of the UK 25 CGPS station data set in the period from October 2001 to June 2002, meant that the complete network had to be divided into six rather than just two sub–networks. Further- more, the introduction of the NFS server–client workstation configuration (Figure 4.3), drove the development of the automated procedures Version 3.2 (§4.4.2). Both of these factors required an overhaul of the Perl script dailyproc.pl and a review of the format of its control file dailyproc.ctl, in order to make the script independent of network size and to account for any possible future developments.

In June 2002, Version 2.0 of the daily processing script dailyproc.pl and its control file were developed. The new script was based on the automated procedures Version 3.2 as opposed to Version 3.3, due to the problems outlined in §4.4.3. In general, the new Perl script carries out the same tasks as the previous version, and its structure has remained much the same as that shown in Figure 4.5. Figure4.6shows an outline of the processing tasks carried out by the daily processing script Version 2.0.

In order to carry out automated daily processing using the new Perl script dailyproc.pl Version 2.0 and Version 3.2 of the automated procedures, a project di- rectory, containing the following sub–directories must be created in the user environment on one of the processing workstations.

Chapter 4. Automated GPS Processing and Analysis 92

UNIX shell scripts Pre−processing Stage

UNIX shell scripts Network−Processing Stage

Post−Processing Stage UNIX shell scripts

start

Repeat for each Station

Repeat for each Sub−Network

Repeat for each Sub−Network

Post−Processing Stage UNIX shell scripts (Network Combination)

Final Corrected Coordinate Time Series Perl script

Cleaning Report

Network Report

Post−processing Report

Station Statistics Files

Combination Report

Figure 4.6: Outline of tasks carried out by the Daily Processing Script Version 2.0 for automated daily processing with GAS.

Chapter 4. Automated GPS Processing and Analysis 93

# ls

dailyproc netwproc postproc preproc

dailyproctemps netwprocscripts postprocscripts preprocscripts netwinfo netwproctemps postproctemps preproctemps

Directory dailyproc contains the Perl script and its control file. The directory dailyproctemps contains all template files for the UNIX .now, the GAS D and the GAS R shell scripts used directly by the Perl script dailyproc.pl. The importance of the remaining directories was described in detail in §4.4, however it should be pointed out that with this new directory structure the scripts for the pre–, the network– and post–processing stages are created by the daily processing Perl script.

Similar to Version 1.0 of the Perl script dailyproc.pl, the execution of Version 2.0 is carried out automatically using the UNIX cron utility. The first task the script carries out, is to read the specified control file, which holds the processing parameters for the dailyproc.pl execution. In order to accommodate for the increased number of stations and thus the need for more sub–networks, these are not defined in the control file anymore. Instead, the user is required to specify a network definition file in dailyproc.ctl and the file must be located in the directory netwinfo. This network definition file contains information on the network names, the network post–processing designations and a 3– character definition for the network–specific SINEX format solution files. Also included is the date3 on which the sub–network is to be processed the first time. At the moment the script assumes that once a sub–network is established it will always be included in the analysis thereafter. This will need some modifications in order to be more flexible, e.g. the use of episodic or quasi–continuous GPS stations will create the need for more sub–networks for which processing is started and stopped. Besides these four parameters, the network definition file must also contain a list of all CGPS stations attributed to a specific sub–network, excluding the base station. An example network definition file as used currently for automated daily processing with GAS is given below.

3

Chapter 4. Automated GPS Processing and Analysis 94

NETW POPR_NAME SNX_NAME NETW_START SUB_NETW_STATIONS (without base station) =============================================================================== cls1 c1kovw c_1 50567 kosg,onsa,vill,wtzr,lerw,camb,shee,sunb,bark cls2 c2kovw c_2 50567 kosg,onsa,vill,wtzr,lerw,camb,hers,abyw,hems cls3 c3kovw c_3 51074 kosg,onsa,vill,wtzr,lerw,camb,newl,aber,live cls4 c4kovw c_4 51222 kosg,onsa,vill,wtzr,lerw,camb,lowe,morp,nstg cls5 c5kovw c_5 51274 kosg,onsa,vill,wtzr,lerw,camb,brst,dunk,hurn cls6 c6kovw c_6 51772 kosg,onsa,vill,wtzr,lerw,camb,pers,port,npld

Besides the network definition file, the control file dailyproc.ctl also contains parame- ters for the names of a station times file and a station categories files, both of which must be located in the directory netwinfo. The station times file contains the MJD of the first ob- servation day of each station. If a station was abandoned during the processing period, e.g. Hemsby, then the last MJD must be included too, alternatively the string ‘99999’ will tell the Perl script dailyproc.pl that the station is still active. At the moment of writing, the station times file only allows start and stop times for one active period to be specified. This will need some modifications in order to be able to include episodic and quasi–continuous GPS stations in the automated daily processing. The top section of the currently used station times file is shown below, the complete file can be found on the CD–Rom in the directory /Software_Developments/Daily_Processing_Scripts_V2/netwinfo.

SITE STARTMJD ENDMJD ===================== aber 51074 99999 abyw 50913 99999 bark 50567 99999 * * * * * *

In order to be able to make the cleaning report introduced in Version 1.0 of the Perl script dailyproc.pl clearer, the cleaning reports of Version 2.0 have been divided according to the station categories assigned to each station in the station categories file. The top section of the currently used station categories file is shown below, the complete file can be found on the CD–Rom in the directory /Software_Developments/Daily_ Processing_Scripts_V2/netwinfo.

Chapter 4. Automated GPS Processing and Analysis 95 SITE CATEGORY ============== aber TG abyw MO bark OTHER * * * *

After the control file is read a series of directory path checks is carried out. Also the existence of the network definition, station times and station categories files in the directory netwinfo is evaluated. If any necessary template files for creating the UNIX .now, the GAS D or the GAS R files are missing in the directory dailyproctemps the process is stopped. However, once the processing environment is correctly set up by the user, definition errors are very unlikely.

The following GAS pre–processing stage is carried out in the same manner as in Perl script dailyproc.pl Version 1.0. However, a cleaning report for each station category is generated based on the assignment of each station in the station categories file. Currently, stations are separated into four different categories depending on the site, e.g. tide gauge sites (TG), IGS tracking sites (IGS), Met Office sites (MO) and any other sites (OTHER). Stations sponsored by the Environment Agency, by the University of Newcastle or the IESSG have been attributed to category OTHER, if they do not fall into any other category. Another modification to dailyproc.pl with respect to the cleaning reports is that a new report is started at the beginning of each calendar year, which should improve readability. The cleaning reports can be found on the CD-Rom in the directory /GPS_Processing_Reports/Preprocessing_Stage.

The network–processing stage is carried out much in the same way as in Version 1.0 of the Perl script dailyproc.pl. Due to the fact that the current CGPS network is divided into six sub–networks, a rigorous method for monitoring the results of this stage had to be introduced. Version 1.0 did not have a routine quality control tool for the network– processing stage. However, by obtaining the a–posteriori σ0for each PANIC least–squares adjustment process from the PANIC network solution files and writing them into a network report (Figure4.6), it is possible, to quickly evaluate the results of the adjustment process

Chapter 4. Automated GPS Processing and Analysis 96

for each sub–network. The annual network reports have been compiled on the CD-Rom in the directory /GPS_Processing_Reports/Networkprocessing_Stage

The post–processing stage of the Perl script dailyproc.pl Version 2.0 is generally carried out in the same manner as in Version 1.0. First, the automated procedures are executed for each sub–network as defined by the network definition file. Then in a second stage, the network combination is carried out by including GPS baselines of all networks in the CARNET least–squares adjustment. If the post–processing stage for each sub– network has already been carried out, Version 2.0 of the Perl script dailyproc.pl allows to only execute the combination run if necessary. This feature was introduced as a result of the re–analysis in 2001/2002, where each GAS processing stage was carried out for the whole data set prior to attempting the next stage.

Version 2.0 of Perl script dailyproc.pl also carries out the correction of the standard errors in the combined network for stations which have been used in several sub–networks. Some technical improvements in the Perl script have also been made, in order to allow any number of sub–networks to be processed.

The updated Perl scripts dailyproc.pl also provide, a report of the a–posteriori σ0 for each sub–network, denoted as a post–processing report. This report is based on the a–posteriori σ0 of the CARNET solution files for each individual network. The post– processing reports then contain the σ0 for all six networks for each day processed for one calendar year. If no CARNET solution file is found for a specific sub–network, then the space is left empty in the report. Using this report, post–processing problems can quickly be identified by the user.

Furthermore, to check the overall solution of the network combination, a combination report is formatted. In Version 2.0 of the daily processing script this report has also been divided into individual reports depending on station categories and checks both the CARNET solution files and the final coordinate time series files. A list of stations with a solution for a specific day is established from the CARNET solution file of the network combination and their correct standard errors are read from the coordinate time series files. Again, the 3–dimensional radial error for each station is computed and entered into the report. However, this list of stations is also compared with the list of active stations

Chapter 4. Automated GPS Processing and Analysis 97

from the station times file and and if a station is missing in the CARNET solution file, Perl script dailyproc.pl assigns a different set of characters as an indication for a problem. In addition, due to the possibility of expressing the final coordinate solutions in different reference frames, the combination reports created by Version 2.0 of the daily processing script are frame dependent. Examples of the post–processing and combination reports created by dailyproc.pl for the time span from 1997 to 2002 can be found on the CD– Rom in the directory /GPS_Processing_Reports/Postprocessing_Stage.

As an additional feature, email reporting has been implemented in Version 2.0 of the daily processing script. At the moment, an email is automatically generated containing the start and stop times of each GAS processing stage. From experience it is known, how long each processing stage takes on average to be carried out successfully. A shorter than usual processing time period in the email, will then indicate that something during the process has failed.

Based on the quality control reports implemented in Version 2.0 of the Perl script dailyproc.pl, it is possible to generate histograms, similar to other CGPS analyses e.g.

Hurst (2000), such as histograms of station coordinate repeatabilities. Using histograms of the σ0 of the ionospherically free residuals collected in the cleaning report, it is possible to investigate the long–term data quality of each CGPS station in the analysis, thus separating better stations from bad ones. These station specific histograms for the UK 25 CGPS station data set can be found in §5.3. An interesting aspect of the cleaning reports was identified, when sessions reported to be not cleanable, were correlated with periods of high ionospheric activities. Particularly for the CGPS station Lerwick, a good correlation between the inability of the scripts to clean the GPS observations for cycle slips and large K indices can be observed. The K index gives a local measure of geomagnetic activity of the ionosphere (§3.4.2) relative to a quiet day for a specific site (Dodson et al.,2001b).

In a similar manner, histograms of the sub–network dependent a–posteriori σ0, can give an indication for the quality of each adjustment computation and also identification of possible long–term variations. Similar graphs can also be generated for the two processing reports of the post–processing stage as a final quality check. This has not yet been

Chapter 4. Automated GPS Processing and Analysis 98

implemented by the author, however, will be suggested for further improving the daily processing script.