Automated Procedures Version 3.1

The automated procedures Version 3.1 are based on the Mark 3 automated procedures described in detail by Booth (2000). The developments involved, were mainly necessary because of changes to the processing data archive and the requirement to optimize the disk storage required for the results of the processing. In this respect, it was necessary to be more restrictive as to whether files were to be removed rather than just compressed in the daily processing directory structure. A close look was also taken at UNIX warnings and error messages sent to the UNIX standard error, which are normally shown on the screen display. Once processing runs were invoked using the UNIX cron utility without user interaction, this proved to be essential, as every warning or error message sent to the UNIX standard error was now re–directed into an email, sent to the user who owned the process. In order to reduce unnecessary messages and therefore cluttering of the display during process runs, more ‘if–then–else’ clauses were introduced and command arguments using UNIX wildcards, e.g. ‘*’ in file names, were replaced with commands using precise references, e.g. giving the complete file name.

All automated procedures associated with GAS can be attributed to four levels of scripts. At the top level is a UNIX Korn shell script with extension .now. Generally, this script defines the processing environment using variables for directory path definitions and the processing time span. This can be invoked either manually or automatically using the UNIX cron utility. The top level UNIX shell script then invokes a GAS D shell script, which can be identified by the file extension *.dates. This GAS D shell script checks the processing time span specified for plausibility and carries out the processing loop over each day specified by the time span in the UNIX .now shell script. Within the GAS D shell script, the GPS stations to be processed are also defined. Due to memory limitations in the GAS processing module PANIC, the network–processing stage can currently only

Chapter 4. Automated GPS Processing and Analysis 68 shell script UNIX *.now GAS D shell script GAS R shell script GAS F shell functions netwproc.now netwproc.dates netwproc.rprocess fdefcheck fmknetwdirs fgetdata2 funzipdata fnetwbline fnetwctlfilein fnetwindata fnetwsats fnetwproc fnetwstns fnetwatmos fnetwambig fnetwrun fzipdata fnetwsnx shell script UNIX *.now GAS D shell script GAS R shell script GAS F shell functions funzipdata postproc.now postproc.dates postproc.rprocess fpoprdefcheck fmkpoprdirs fpoprjdaycar fgetpoprjdayxyz fgetpoprjdayplh fzipdatantp fcleanupdirs shell script UNIX *.now preproc.now GAS D shell script GAS R shell script preproc.rprocess GAS F shell functions preproc.dates fmkdirs funhealth frnxwinchk frin2not fmkabasectlline fgetdata fcprsfiles2 fdetslips fclean fcleprep frunabaseline

Figure 4.2: Automated Procedures for GPS Processing with GAS (Version 3.1).

process a maximum of nine baselines simultaneously, which means that the CGPS network analysed in this study, had to be split into a series of sub–networks of ten stations each.

The GAS D shell script then calls the GAS R shell script, which is the third level script and is identified by the file extension .rprocess. A GAS R shell script contains a series of user–defined variables, which mainly define the GPS processing options depending on the processing stage. The GAS R shell script then calls a series of GAS F shell functions, which are effectively the fourth level scripts. The GAS F shell functions basically carry out the tasks for a processing stage. GAS F shell functions can easily be identified by their filename, which starts with the character ‘f’. The GAS F shell functions use template files that are stream edited and concatenated to produce specific control files for each GAS processing module. An overview of all shell scripts and functions is shown in Figure4.2. The figure lists filenames for the pre–, the network– and the post–processing stages of the automated procedures Version 3.1.

Chapter 4. Automated GPS Processing and Analysis 69

In preparation for an automated processing run, the shell scripts, shell functions and template files have to be arranged in a project directory containing following directory structure:

# ls

preproc netwproc postproc netwinfo

preprocscripts netwprocscripts postprocscripts preproctemps netwproctemps postproctemps

where the level one, two and three GAS shell scripts for each processing stage are in the preproc, netwproc and postproc directories respectively. GAS F shell functions are placed in directories with names ending -scripts and template files in the directories with names ending -temps. An additional directory netwinfo is needed for the automated procedures, which contains the GOF, a file assigning the correct antenna phase centre models, and reference frame specific coordinate files, for all stations in the project.

After the general structure for the automated procedures has been described, it is now possible to explain the different stages and their changes since Version 3.0 in more detail.

Pre–Processing Stage

The pre–processing stage is invoked using the top level UNIX shell script preproc.now. This script will execute the GAS D shell script preproc.dates, which will call the GAS R shell script preproc.rprocess for each day according to the time span defined by the user in preproc.now. The GAS R shell script will then call each GAS F shell function of the pre–processing stage as listed with their file names in Figure4.2.

The first GAS F shell function to be called is mkdirs, which creates the daily directory structure needed for any GAS processing run. Using the GAS F shell function getdata, the necessary RINEX format observation files, the IGS precise ephemeris and the daily GAF (§4.3.2) are obtained from the processing data archive. In 1999, this data archive was located on workstation ukcogr and was called tgarch. It used a directory structure (/wwww/yyddd/) based on the 4–character designator for the GPS week (wwww), the last two digits of the current calendar year (yy) and the 3–character designator for the day–

Chapter 4. Automated GPS Processing and Analysis 70

of–year (ddd), e.g. for week 1016, year 1999 and day–of–year 92, the directory would look like /1016/99092.

Mark 3 of the automated procedures used the UNIX remote copy command rcp to obtain all necessary daily files from the tgarch archive on workstation ukcogr. The new processing data archive procarch was based on a different directory structure, not including the GPS week in the path definition for the daily directories. As the automated procedures were modified to copy the daily observation and ephemeris files from this archive, changes had to be applied to the GAS F shell function getdata. When the processing data archive procarch was moved to the new workstation monix in 2001, changes to the GAS shell scripts were again necessary.

After the RINEX format observation files, the IGS ephemeris files for the previous, the current and the following day, and the daily GAF are copied to the correct processing directories, the GAS F shell function unhealth determines if any satellites observed in the RINEX format observation files are missing in the IGS ephemeris and which satellites are not common to all three IGS ephemeris files, in order to declare them to be unhealthy (Booth, 2000). The output of unhealth is a block of unhealthy satellites, which is used by the GAS F shell function rin2not at a later stage during the creation of the control file of the GAS processing module FILTER.

At the next step, the RINEX format observation files are checked for errors. The GAS F shell function rnxwinchk removes any RINEX format observation files that do not contain any raw GPS observations, i.e. an empty file or one which simply contains the header (Booth,2000).

As mentioned above, the GAS F shell function rin2not prepares the FILTER control file. FILTER is used to split the 24–hour RINEX format observation files into four 6–hour files in the NOTT2 observation file format. At this stage, known station coordinates are inserted into the NOTT2 format observation file headers based on the FILTER ancillary file filter.anc located in directory netwinfo.

The GAS F shell functions mkabasectline and runabaseline determine a set of base satellites, normally three satellites for each of the six hour observation sessions. The

Chapter 4. Automated GPS Processing and Analysis 71

satellites giving the greatest average elevation angle over the session and which are visible from each station for the greatest period are selected as input for the base satellite block in the PANIC control file. The control file itself, is then created in the next step by the GAS F shell function cleprep (Booth, 2000) which stream edits a number of template files in order to produce a PANIC control file.

Using the NOTT2 format observation files, the GAS F shell functions clean and detslips now carry out the cycle slip detection and correction. The GAS F shell func- tion clean invokes the GAS processing modules PANIC, FILTER and SLIPCOR, using function detslips and the auxiliary C program SCANRES. SCANRES scans the carrier phase residuals file, created by PANIC in cycle slip detection mode, for periods of noisy double difference observations. If such a period for a specific satellite is identified, then SCANRES produces a temporary FILTER control file, which can be used by FILTER to remove this satellite specific period of noisy data. Again the reader is referred to Booth

(2000) for a detailed description of this key step in the automated procedures for the GAS pre–processing stage.

All the procedures mentioned above create and use temporary files, which are not needed in the coming processing stages or can easily be re–created. To ensure efficient hard disk usage, files have to be removed or compressed. This is now carried out by the largely modified GAS F shell function cprsfiles2. This improved version of function cprsfiles, uses fewer UNIX wildcards, e.g. ‘*’, in file names and directory paths, and carries out more file checks using if–then–clauses. As a result, the new GAS F shell function cprsfiles2 is far more selective with respect to which files are being removed, rather than compressed.

Network–Processing Stage

The network–processing stage is invoked using the top level UNIX shell script netwproc.now. This script will execute the GAS D shell script netwproc.dates, which will call the GAS R shell script netwproc.rprocess for each day according to the time span defined by the user in netwproc.now. The GAS R shell script calls each GAS F shell function of the network–processing stage as listed by their file names in Figure 4.2.

Chapter 4. Automated GPS Processing and Analysis 72

The changes from Mark 3 to Version 3.1 of the automated procedures for the network– processing stage, are mainly related to the GAS F shell functions getdata2 and zipdata. As with the automated procedures for the pre–processing stage, this was due to changes in the processing data archives and a more restrictive policy with respect to which files were to be kept. Apart from that, modifications to the GAS processing module PANIC, which enabled the output of SINEX format solution files (§4.2.2) drove changes to the GAS F shell function netwrun, and the introduction of the new function netwsnx and the auxiliary Perl script compepoch.pl.

The first GAS F shell function to be invoked by the GAS R shell script netwproc.rprocess is defcheck. This function checks user–defined variables in the GAS R shell script and although very unlikely, terminates the execution, if any errors in the definitions occurred. The GAS F shell function mknetwdirs then creates the network solution directory, before the IGS ephemeris of the previous, current and following days and the GAF for the current day are copied from the processing data archive by the GAS F shell function getdata2. A modification in function getdata2 of the automated procedures Version 3.1 was made, in that no RINEX format observation files are copied, as these are not needed in the network–processing stage. Before the data can be used in the processing, the GAS F shell function unzipdata uncompresses the NOTT2 format observation files, produced by the pre–processing stage.

In the next step, the PANIC default control file and the PANIC control file are then created by stream editing and concatenating a number of template files. This is carried out by the GAS F shell functions netwctlfilein, netwindata, netwsats, netwproc, netwbline, netwstns, netwatmos, netwambig, and netwsnx. For a detailed description of these functions, except for netwsnx, the reader is referred to Booth (2000). After the GAS processing module PANIC was modified to export SINEX format solution files, a new GAS F shell function had to be developed which would create the SINEX block in the PANIC control file. This also involved the computation of the epoch of the solution, thus an additional script was required. Using the auxiliary Perl script compepoch.pl, which is called by GAS F shell function netwsnx, this simple computation is carried out and the correct reference epoch is included in the SINEX block in the PANIC control file. The

Chapter 4. Automated GPS Processing and Analysis 73

GAS F shell function netwrun then executes PANIC and renames output files for further use.

Since Version 3.1, the tropospheric scale factor output files (Stewart et al.,2002) and the newly generated SINEX format solution files have been archived. In order to resemble file naming conventions of the IGS as much as possible, the author decided to use the RINEX format file naming convention (Gurtner and Mader,1990), using the character ‘t’ as observation type indicator for the tropospheric scale factors in the file extension. The SINEX format solution file name should resemble IGS or EUREF weekly SINEX solution file names. Therefore, a 3–character SINEX solution designator was introduced in the GAS R shell script netwproc.rprocess. The SINEX file name is then a combination of this 3–character word, the GPS week, the day–of–week and the extension *.snx2.

By declaring user–defined variables in the GAS R shell script netwproc.rprocess, it is possible to select GPS network processing strategies (Booth,2000). For the analysis of the GPS observations in this research, the standard processing strategy included the mitigation of solid Earth tides (§3.4.4_{), ocean tide loading (§}3.4.4), antenna phase centre variations (§3.4.3_{), and tropospheric delay (§}3.4.2). These remained unchanged throughout the complete time span of available CGPS data. In all instances, the IGS final precise ephemeris was held fixed, and the a priori coordinates of any IGS stations were computed in the same realization of the ITRS as the satellite ephemeris, i.e. the ITRF94, ITRF96, ITRF97, or ITRF2000 (§3.3.2), and motioned to the observation epoch. The consistent use of common reference frames for station coordinates and satellite ephemeris is necessary in order to avoid offsets in the coordinate time series at epochs when the reference frame was updated (§3.4.1).

The GAS processing module PANIC can process a maximum of ten stations simulta- neously. For convenience, the base station for pre–processing was chosen to be the CGPS station at the IESSG, as it lies centrally to all of stations analysed. In addition, four IGS stations have been used in all of the networks analysed by the author. In all cases, this has meant that the network has been divided into a series of sub–networks (with 2_{e.g. for the sub–network cluster 1, the designator could be declared to be c 1, and with GPS week}

Chapter 4. Automated GPS Processing and Analysis 74

the IESG station and the four IGS stations included in each sub–network) and the GAS network–processing stage being carried out for each sub–network separately.

The network solution results consist of the daily GPS baseline vectors and their associ- ated variance–covariance information. These are obtained in the form of PANIC network solution files and the SINEX format solution files. The availability of both files allows for two different analysis approaches in the following post–processing stage, in that the PANIC network solution files can be used as input to the GAS module CARNET and the SINEX format solution files as input to the Bernese GPS Software modules SNXNEQ and ADDNEQ (Hugentobler et al.,2001).

In the final step of the network–processing stage, the GAS F shell function zipdata compresses all of the cleaned NOTT2 format observation files and the various solution files.

Post–Processing Stage

The post–processing stage is invoked using the top level UNIX shell script postproc.now. This script will execute the GAS D shell script postproc.dates, which will call the GAS R shell script postproc.rprocess for each day according to the time span defined by the user in postproc.now. The GAS R shell script calls each GAS F shell function of the post–processing stage as listed by their file names in Figure4.2.

Overall, the main difference between the Mark 3 and Version 3.1 of the automated procedures for the post–processing stage is the implementation of two different processing modes, i.e. the single network and the network combination mode. Due to limitations in the GAS processing module PANIC, the increasing UK CGPS network had to be divided into an increasing number of sub–networks, with a combined solution only achieved at the post–processing stage. In single network mode, the automated procedures carry out processing using GPS baseline vectors of a single sub–network. In network combination mode, all GPS baseline vectors of all sub–networks are used to compute a solution for the combined network.

Chapter 4. Automated GPS Processing and Analysis 75

Switching between the two processing modes is achieved by specifying the number of networks to be processed in the shell script postproc.now. If this variable is set for one network, then a network name must be specified in the GAS R shell script postproc.rprocess. For more than one network, names for all sub–network must be declared in the GAS R shell script instead.

The post–processing stage of the automated procedures Version 3.1 can only be carried out on a daily basis. Thus it is not possible to produce weighted weekly solutions or repeatabilities using CARNET and REPDIF, as with the Mark 3 automated procedures. Although, the original GAS F shell functions of the Mark 3 automated procedures are still included in Version 3.1, they were not modified with respect to the introduction of the network combination mode.

During a monitoring period of several years, the reference frame in which the precise ephemeris are computed can be updated several times. These changes in the reference frame would cause discontinuities in any coordinate time series based purely on solutions obtained at the network–processing stage. It becomes difficult, therefore, to base any further analysis on these time series, unless they are transformed to a common frame. The GAS R shell script postproc.rprocess also provides the possibility of expressing the daily coordinate solutions in a common reference frame.

The first GAS F shell function to be called by postproc.rprocess is poprdefcheck. This function carries out checks on the user definitions in the GAS R shell script. With the introduction of the network combination mode, poprdefcheck required some extra check routines included. The next two functions executed, un–compress the PANIC