Open-source software for geodetic imaging:
ROI_PAC for InSAR and pixel tracking
Authors: Matthew E. Pritchard; and contributors to the ROI_PAC wikiVersion: 0.4 May, 2014
Deformation Discovery: Interferogram from southern Peru showing ground displacement from at least 3 different origins within 50 km. The deformation in the upper left was expected as this interferogram spans an earthquake swarm that occurred in October, 2005. However, this data show that the locations from some global earthquake catalogs are in error by 10 km or more. The other two areas of deformation are newly discovered but based on their characteristics in a volcanic region (center) and near a Lake (lower right), volcanic and groundwater related sources of deformation are suggested. The dates of the two images used are 17 June 2006 and 4
Overview of Contents
Section 0. Acknowledgements Section I. Introduction
Section II. Getting started: Data & software Raw data format
Available software for InSAR and pixel tracking Appendix 0: Some basic LINUX commands
Appendix 1: Getting and installing ROI_PAC Appendix 2: ROI_PAC file types and file formats
Appendix 3: ROI_PAC processing flow: Script-by-Script Appendix 4: User defined options for ROI_PAC processing
Appendix 5: Which scripts created each file generated by ROI_PAC? Appendix 6: What is the information in the RSC files?
Appendix 7: Access to data and special processing considerations for each satellite Appendix 8: Viewing ROI_PAC output files and moving them to other programs Appendix 9: ROI_PAC Troubleshooting
Appendix 10: What ROI_PAC can’t do (yet)
Appendix 11: Annotated bibliography of useful books and websites References
Section 0. Acknowledgements
The impetus for this textbook was a realization that although there are many useful review papers and a few textbooks about InSAR and pixel tracking (e.g., Hansen, 2001; Kampes, 2006; Simons & Rosen, 2007; Massonnett & Feigl, 1998; Burgmann et al., 2000; Rosen et al., 2000; Chapter co-authored by Zhong Lu in Dzurisin, 2007; Feretti, 2014) there is no existing textbook on the rapidly developing field of imaging geodesy that explicitly helped new users get started using real data and software. Demand for such a textbook is high from attendees at the UNAVCO short courses and others. So, in 2009, Matt Pritchard proposed to write such a document as an online, free textbook as part of the education plan in a National Science
Foundation CAREER that was funded in 2010. The focus here is on practical concerns of using the software, and we refer the reader to the literature for theoretical developments. In particular, a chapter from the Ph.D. thesis of Sean Buckley (a modified version is available online:
http://www.geo.cornell.edu/eas/PeoplePlaces/Faculty/matt/ROI_PAC_doc.pdf) documents some of the equations used in the software described in this text. Further corrections and additions to the textbook are planned at least until the end of the grant in 2016, so please send any
suggestions or comments to [email protected].
The materials in this book have been merged from many sources and many authors – web pages, scientific papers and presentations, theses, classroom lectures and short courses, one-on-one training and so on. We thank all of the students and postdocs over the years who have commented on these documents and presentations and contributed to their development. We especially thank the approximately 170 attendees of the annual 2.5-3 day UNAVCO short courses from 2008-2014 entitled “InSAR: An introduction to Processing and Applications for Geoscientists (with an optional module for pixel tracking)” that were taught by Eric Fielding (JPL, 2008-2012), Paul Rosen (JPL, 2008-2012), Yuri Fialko (UCSD, 2008); Matt Pritchard (Cornell, 2009-2012), Scott Baker (UNAVCO 2012-2013), Walter Szeglia (Central Washington University, 2012-2013), and Piyush Agram (JPL, 2013) – we further thank the UNAVCO staff for helping to organize these courses.
The ROI_PAC software (Rosen et al., 2004) that is discussed extensively in this book was developed over several decades by researchers at the Jet Propulsion Laboratory (JPL) and the California Institute of Technology (Caltech) in Pasadena, CA. Some parts of ROI_PAC date back to the early 1980's and the pioneering spacebourne InSAR work done by Dick Goldstein, Howard Zebker and others (Goldstein et al, 1986). During the 1990's, Gilles Peltzer and colleagues started to put together the programs into a set of scripts. Through the collective efforts of the following people, ROI_PAC was created: Paul Rosen (JPL), Scott Hensley (JPL), Gilles Peltzer (JPL/UCLA), Francois Rogez (JPL), Frederic Crampe (JPL/UCLA), Eric Fielding (JPL), Mark Simons (Caltech), Rowena Lohman (Caltech undergrad/grad student, now at Cornell), and Matt Pritchard (Caltech grad student now at Cornell).
Section I. Introduction
For centuries, measurement of the shape of the Earth (called the science of geodesy) was necessarily time consuming. Even with new technologies like the Global Positioning System (GPS), vast portions of the Earth remain infrequently monitored for movement. Over the past 20 years, a new form of geodesy has rapidly developed whereby satellite images can be compared to infer movements of the Earth's surface. Called imaging geodesy, the synoptic aircraft or satellite views allow large regions to be surveyed densely without any human setting foot in the area. Imaging geodesy encompasses several different types of methods including Interferometric Synthetic Aperture Radar (InSAR, e.g., Rosen et al., 2000) as well as the automated comparison of SAR and optical images via the techniques of pixel or feature tracking (e.g, Leprince et al., 2007). InSAR can image sub-centimeter deformation of the Earth's surface every 20 meters over areas spanning hundreds to thousands of kilometers. InSAR has allowed vast areas of the Earth's surface to be monitored frequently for deformation for the first time and led to discoveries in a variety of fields (including signals caused by earthquakes, volcanoes, landslides, glaciers, groundwater, and human-induced ground deformation). Pixel tracking is a very complementary tool to InSAR-- although the sensitivity to deformation is less (decimeter instead of
sub-centimeter in a given image pair) and the horizontal spacing is coarser, it can be applied to both radar and optical images and often works when InSAR does not – for example, in areas that have large displacements or changes to the radar scattering properties of the ground.
Section II. Getting started with InSAR and pixel tracking: Data & software
Radar is a commonly used word, but it is actually an acronym for “RAdio Detection And Ranging” and involves the active transmission and reception of microwave energy to measure distance. The waves in the radar portion of the electromagnetic spectrum have frequencies from 0.3-30 GHz or wavelengths of 1 cm – 1 m. Different portions of this spectrum were divided into code names during World War II and these codes are still widely used. The most commonly used codes by satellite radars are X-band (~2.5-3.75 cm wavelength), C-band (3.75-7.5 cm), and L-band (15-30 cm), although the S band (7.5-15 cm) was used on NASA Magellan mission to Venus, the Cassini mission to Saturn uses Ku band (1.67-2.5 cm), and P-band (> 1 m) is
sometimes used on airplane imaging radar systems. For reference, ground-penetrating-radars are typically in the UHF to VHF bands of .05-1 GHz (30 cm to 6 m), weather radars can be in the K, X, C, and S bands, air traffic control radars can include L, S, and X bands and cell phones, Wi-Fi, television, and Bluetooth typically are in the UHF band that overlaps mostly with the P but partly with the L band.
Radars have a wide variety of uses, but we will be considering imaging radars that are able to distinguish fine details in two-dimensional arrays that constitute “images” of the ground. Specifically, we use Synthetic Aperture Radar (SAR), a common imaging technique used on a moving platforms (like an air- or spacecraft) to produce high resolution images in the along-track direction.
SAR Raw Data Format
The format of raw SAR data varies depending on the satellite and a compilation of documentation of several different formats is available here:
http://aws.roipac.org/cgi-Observation Satellites (CEOS) format. The CEOS-format SAR raw data nominally consists of a Volume Directory File (ASCII), SAR Leader File (mixed ASCII and Binary), Raw Data File (Binary), and a Null Volume File (ASCII). The Volume Directory File describes the arrangement of the data on the storage media (tape, CD, file directory, etc.). The SAR Leader File provides pertinent information about the specific SAR data set: Raw Data File size, spacecraft height and velocity, scene center latitude, longitude and time of acquisition, etc. The Raw Data File contains a header record plus the SAR raw data nominally stored as one data line per record. Each value in the header contains useful information about the parameters of the data collected (the satellite clock time, line number, etc.) that are documented by the space agencies
(http://aws.roipac.org/cgi-bin/moin.cgi/Raw_Formats). Each record line consists of a prefix (header – in the case of ERS, this is 412 bytes), raw data (see next paragraph), and a suffix (not always used).
The binary SAR raw data (excluding the header and suffix on each line) is commonly stored in two formats – as a complex number in I/Q (in-phase (real) and quadrature (imaginary)
components) or offset video (real numbers). We focus on the I/Q format, as it is the most common for the satellites we discuss here (the offset video was used for SEASAT). The choice of data formats is governed by how the radar signal returned to the satellite from the ground (the echo) is converted from an analog to digital. The I/Q format uses a complex converter while offset video requires a more complicated converter to save as real numbers – but the total number of data points in each format is the same. Each format has different advantages and requires different algorithms to process.
For each return sample, the received voltage is recorded as an integer byte values -- each byte is made up of 8 bits and each bit has a value of 0 or 1. For example, ERS SAR raw data is 5-bit quantized, i.e., the voltage is recorded as one of 32 values (25) -- between 0 and 31, and the JERS-1 data are quantized to 3 bits (or between 0 and 7). Thus, JERS and ERS data are
significantly smaller than the equivalent binary files composed of floating point numbers that are typically made up of 4 bytes each (each byte is 8 bits), and the data can be easily read on all types of computers (no matter if they are big endian or little endian). In order that negative voltages can be recorded, a “bias” is added to all returned voltages. For ERS, this bias is about 15.5 (small deviations from this nominal value can be determined from the raw data and are typically calculated by the satellite downlink facility and included in the leaderfile). To convert the raw data number (dn) to a voltage (V), use the expression: V = A * (dn-15.5) where A is a scaling constant depending on the gain of the radar receiver and analog to digital quantizer. Since the data is quantized directly as complex numbers, each returned sample is recorded as a real and imaginary component for the I/Q format – thus the signal from a given sample is complex(dn2*n-1-15.5,dn2*n-15.5) where n is the column number.
The raw data file can be viewed using the program mdx (Appendix 8) using the following command:
Mdx –i1 FILENAME LINELENGTH (show example figure). Available software
Pixel tracking with ROI_PAC: A set of python scripts (download here:
http://www.geo.cornell.edu/eas/gstudent/akm/pixelTrack.tar) have been released that use ROI_PAC to generate dense sub-pixel offsets of optical or SAR using the method of normalized cross correlation (NCC), which computes the offsets in the spatial domain (like the program IMCORR).
Cosi-Corr (Leprince et al., 2007) An open-source program to generate sub-pixel offsets is called COSI-CORR (available at
http://www.tectonics.caltech.edu/slip_history/spot_coseis/index.html) that generates sub-pixel offsets of optical images that can use either NCC or a different algorithm called the Fourier Shift Theorem. COSI-CORR requires the use of the GIS package ENVI and the commercial software package IDL, which are usually available for $2000-$3000 to academic users.
IMCORR (Scambos et al., 1992), Open source software (available at
http://nsidc.org/data/velmap/imcorr.html) that uses NCC to generate sub-pixel offsets between optical images.
InSAR software:
ROI_PAC: The Repeat Orbit Interferometry PACkage developed at JPL and Caltech, is a collection of fortran and C programs bound together with perl scripts to perform certain routine but important tasks in synthetic aperture radar (SAR) repeat orbit interferometry. Specifically, the individual programs perform everything from raw data conditioning, SAR image processing, interferogram formation, correlation estimation, phase
unwrapping, baseline determination, estimation of topography, removal of topography from a deformation interferogram, and geocoding. The perl scripts control certain prescribed ways of combining these programs to specifically create a geocoded deformation phase or topographic phase image from two satellite radar images and a digital elevation model, or create a deformation phase image from three radar images without a digital elevation model. The perl scripts allow considerable control over every step of the processing, such that individual programs or combinations of programs can be manipulated to produce a more favourable output without direct interaction with the program itself. ROI_PAC does not perform well with airborne sensors because it does not include motion compensation. The software is described in Rosen et al., (2004) and this is the article that should be cited when using the software. The software package is the most widely used for InSAR processing in the peer-reviewed scientific literature (over 3000 unique downloads since 2003 and 150 peer-reviewed publications, roipac.org/BiblioList)
ISCE: InSAR Scientific Computing Environment is a new Stanford/Caltech/JPL software package that builds on some of the FORTRAN and C programs in ROI_PAC but is faster, more accurate, and provides better geolocation (Gurrola et al., 2010; Rosen et al., 2009; Zebker et al., 2010). For example, it uses a motion-compensated geodetic
coordinate system from the start facilitates time series analysis because co-registration of images before they can be combined is not required (as in ROI_PAC). Another
difference is that ISCE uses python instead of Perl Scripts. Licensing for ISCE is available to WInSAR members through UNAVCO.
GMTSAR: http://topex.ucsd.edu/gmtsar/ Open source InSAR processing software based on the open source Generic Mapping Tool (GMT) package (Sandwell et al., 2011)
DORIS (http://doris.tudelft.nl/) is another open source suite of InSAR processing software that is being integrated in the NEST (Next ESA SAR Toolbox) software packaged (http://nest.array.ca/web/nest). Training in the use of the NEST software is available: (http://earth.eo.esa.int/workshops/fringe2011/index.php?page=164&type=s)
Open source software to create InSAR time series using multiple methods (Agram et al., 2013): GIANT (http://earthdef.caltech.edu/projects/giant/wiki)
Open source software for Atmospheric Phase Screen removal (Jolivet et al., 2011) Pyaps (http://earthdef.caltech.edu/projects/pyaps/wiki/Main)
Other notable InSAR processing software that is commercially available includes: GAMMA http://www.gamma-rs.ch/
Nigel Press Associates (NPA) (http://www.cgg.com/default.aspx?cid=5837); DIAPASON (http://www.altamira-information.com/html/1-18215-InSAR-Software--Training.php);
Appendix 0: Some basic LINUX commands
ROI_PAC requires users to use UNIX or LINUX and interact with their computer via the command line. For many people, use of the command line requires a period of adjustment usually lasting a day or two. The key point to remember is that LINUX is based on a directory structure so you need to constantly be aware of what directory you are currently in.
There are many good resources on the web that provide a quick introduction to LINUX -- for example: http://www.math.utah.edu/lab/unix/unix-tutorial.html,
http://linuxreviews.org/beginner/, and http://freeengineer.org/learnUNIXin10minutes.html. Here we provide a very short list of some essential commands.
The Top 16 Most Useful LINUX commands you will use everyday 1) “ls” List the files and subdirectories
“ls -alF” will list the files and directories with additional useful information (size, date of last modification, and who has permission to read/write)
For example, here is some example output:
-rw-r--r-- 1 matt staff 240758 Mar 23 2009 myfile.pdf drwxr-xr-x 6 matt staff 204 Jan 11 2011 Stephen/
An important trick in LINUX is the use of wildcards. If I wanted to display all the interferogram files, I would type “ls *.int”
On many computers, different types of files show up as different colors. Directories are one color, files that you can execute are another, files that are linked (see below) another, etc.
2) “man” Display the manual pages for any command.
For example, typing “man ls” would allow you to see all of the options you can type with the “ls” command including “ls -alF”
3) “cd” change directory
“cd” by itself takes you to your home directory (usually /home/yourname) “cd ..” moves you one directory up
“cd ./NAME” moves you down into directory NAME
“cd /FULL/PATH” moves you to the specific directory /FULL/PATH that may neither be directly up or down from your current location
The usual syntax is “cp OLDFILE NEWFILE” where OLDFILE is the existing file and NEWFILE is the name of the file you wish to create. You can create the NEWFILE in a different directory if you include the full path
5) “mv” Will move a file. The original version will not be saved. Useful when you do not want to have two copies of a large file.
6) “ln -s” Sometimes a file is so big that you do not want to move the whole thing, but instead want to link a large file from one directory to another. For example, ROI_PAC uses this command with the large single look complex files. The syntax is “ln -s ORIGINAL_FILE /NEW/FILE/DESTINATION”
7) “rm” Will remove or delete a file. USE WITH CAUTION
Useful flags are “rm –f” to force without having to be asked permission and “rm –r” to remove all files and subdirectories in a directory
8) “pwd” Will show you the path of the current directory you are in. It stands for print working directory.
9) Edit text files. There are many ways to do this depending on what computer you are on. If you are on a Mac, you can use the program “TextEdit”. Many machines have a program called “emacs” which allows you to interact with the text file in a familiar menu based manner (like Word). If you want to go old school you can learn how to edit using a program called “vi” that works on all platforms. A web search for vi will reveal all the commands you need to use this program.
10) “more” Prints the contents of a text file to screen. Use the spacebar to scroll forward and type “q” to exit.
11) “chmod” Allows you to change the permissions on a file (to give others the ability to read, write or execute the file).
12) “source” ROI_PAC sets up a list of file name shortcuts in a file called SAR_CONFIG. Any time you change this file you need to type “source SAR_CONFIG” to make the changes active. Every time you log in, your computer runs a program called “.bash_profile” or something similar if you are using a shell other than bash. You can set your computer to run this command
automatically for you every time you log in by typing this command in your .bash_profile file: “source /PATH/SAR_CONFIG” where PATH is the directory where the SAR_CONFIG file is located.
13) “df -k” Displays the amount of disk space currently being used (in terms of bytes and %) and how much is available. Important so you don’t overfill the disk.
14) “du -ks ./*” This command allows you to see the size of the all the files within a set of subdirectories. The “ls” command will show you the size of individual files but will not sum the
contents of an entire directory. This command is useful to see which directories are taking up the most disk space.
15) “grep” Extremely useful command for extracting information from text files by doing a search.
For example, if I want to find the pixel size of a certain file, I would type
“grep PIXEL_SIZE FILENAME” where it would search for the phrase PIXEL_SIZE in all the files named FILENAME
Appendix 1: Getting and installing ROI_PAC Getting ROI_PAC
ROI_pac is copyrighted software that requires a license. Licenses are available at no charge for non-commercial use from Open Channel Foundation
(http://www.openchannelfoundation.org/projects/ROI_PAC) Installation
The new release (v. 3.0.1) has a very different installation system compared to releases before v. 3.0. The easiest way to build it is explained in the
ROI_PAC_3_0_1/ROI_PAC/AAREADME_BUILD_ROIPAC file. 1. Expand Archive
Uncompress and untar the archived file. (Command line input is in red) % tar -xzf ROI_PAC_3_0_1.tgz
2. Read the documentation
Read through the text file ROI_PAC_3_0_1/AAREADME and the additional documentation found in ROI_PAC_3_0_1/ROI_PAC/DOC. The instructions for installing the software and running the scripts are described in ROI_PAC_3_0_1/ROI_PAC/DOC/Setup and
ROI_PAC_3_0_1/ROI_PAC/AAREADME_BUILD_ROIPAC. The most important steps are
summarized below.
% cd ROI_PAC_3_0_1 % less AAREADME 3. Compile Executables
Follow the instructions in the file ROI_PAC_3_0_1/ROI_PAC/AAREADME_BUILD_ROIPAC. Copy and paste each command individually into a shell. These commands will build and test the software package for multiple compilers. If problems are encountered, refer to the bottom of
AAREADME_BUILD_ROIPAC for platform specific issues or look through prior posts on the forum (http://www.openchannelfoundation.org/forum/forum.php?forum_id=99) at Open Channel Foundation. To see what shell you are using, type echo $SHELL on the command line.
% cd ROI_PAC_3_0_1/ROI_PAC % less AAREADME_BUILD_ROIPAC
% which ifort g95 f90 pgf95 f95 xlf gfortran cc gcc icc
% touch aclocal.m4 Makefile.in configure % ./contrib/install-fftw.sh CC=cc
% export FFTW_LIB_DIR=/software/ROI_PAC_3_0_1/ROI_PAC/NetInst/fftw-071005-1457/lib % export FFTW_INC_DIR=/software/ROI_PAC_3_0_1/ROI_PAC/NetInst/fftw-071005-1457/include % ./contrib/multibuild.sh etc.
When testing the build, you will need to gunzip the provided test data set prior to running the script multitest.sh.
% ls
ROI_PAC_3_0_1 ROI_PAC_3_0_1.tgz roi_pac_testdir.tar.gz % gunzip roi_pac_testdir.tar.gz % cd ROI_PAC_3_0_1/ROI_PAC/test-runs % ../contrib/multitest.sh /full_path/roi_pac_testdir.tar /full_path/ROI_PAC_3_0_1/ROI_PAC/multibuild-071005-1506/installs/share/roi_pac /full_path/ROI_PAC_3_0_1/ROI_PAC/multibuild-071005-1506/installs/defaults/bin /full_path/ROI_PAC_3_0_1/ROI_PAC/multibuild-071005-1506/installs/gfortran/bin Each test run should take ~40 minutes depending on your computer. If the test fails while
processing the DEM contained in roi_pac_testdir.tar, you may have to manually swap this DEM file with one that has the proper byte order for your machine (i.e., little-endian versus big-endian, http://en.wikipedia.org/wiki/Little_endian#Endianness_and_hardware see
TEST_DIR/AAREADME). If the test is successful, the finished product is the file geo_930110-950523.unw which should be buried within the directory structure (check in
test-runs/defaults/TEST_DIR/int_930110_950523). You can view the image file using the program
mdx, if you already have it installed. Otherwise, you can try to view the file with a different program that can display binary image files (i.e., Matlab, ENVI, ERmapper). The file is
composed of floating point values in a band-interleave format, i.e. alternating lines of magnitude and phase. The size of the file should be just under 10 MB. The meta-data for the file can be found in geo_930110-950523.unw.rsc. If you have progressed this far, congratulations! You have a working version of ROI_PAC.
4. Install Executables
Make a new directory called INT_BIN where you will deposit the final executables. % mkdir ROI_PAC_3_0_1/ROI_PAC/INT_BIN
You can copy the set of executables from the multi-build operation. Choose one of the successful builds found in ROI_PAC_3_0_1/ROI_PAC/multibuild*/installs.
% cd ROI_PAC_3_0_1/ROI_PAC/multibuild*/installs/defaults/bin % cp * ROI_PAC_3_0_1/ROI_PAC/INT_BIN/.
NOTE: If you plan to use ODR orbit files, then the getorb
(http://www.deos.tudelft.nl/ers/precorbs/tools/getorb_pack.shtml) executable (installed separately) must be located in INT_BIN. You can copy the executable into this directory or make a symbolic link.
NOTE: After finishing this step, please check the Patches ( http://aws.roipac.org/cgi-bin/moin.cgi/Patches) page to review/install any version specific corrections.
5. Configure SAR_CONFIG
The shell script SAR_CONFIG controls the location of the code and scripts. Once this is set properly the tools of ROI_PAC should be available to the user.
SAR_CONFIG describes how to establishes required environment variables pointing to the programs and perl scripts that ROI_PAC uses, as well as the location of orbit data and other ancillary data. ROI_PAC will absolutely not work if these are not set properly. Create your personal copy with the settings your like and source the file at login.
Example config variables are provided in csh and bash style. If you use a different shell you may need to change the SAR_CONFIG syntax. For example:
csh: setenv SAR /home/insar bash: export SAR=/home/insar
Read the instructions in ROI_PAC_3_0_1/ROI_PAC/DOC/SAR_CONFIG. Copy this file into the directory ROI_PAC_3_0_1/ROI_PAC and edit the appropriate lines. You will have to source this file every time that you want to use ROI_PAC. It will set your path environment so that the necessary scripts will be available on the command line.
% cp ROI_PAC_3_0_1/ROI_PAC/DOC/SAR_CONFIG ROI_PAC_3_0_1/ROI_PAC/. % vi ROI_PAC_3_0_1/ROI_PAC/SAR_CONFIG (edit paths)
% source SAR_CONFIG
In addition to the paths to ERS orbit files (SAR_PRC_DIR and SAR_ODR_DIR) or Envisat orbit files (POR_DIR and VOR_DIR), you will also need to define the INT_BIN and INT_SCR
directories. To do this, add the following lines to SAR_CONFIG.
export INT_BIN="/full_path/ROI_PAC_3_0_1/ROI_PAC/INT_BIN" export INT_SCR="/full_path/ROI_PAC_3_0_1/ROI_PAC/INT_SCR"
Instead of using export for the bash shell, use the setenv syntax if you are using the csh or sh shell. You can also add the path to mdx, getorb, or other related software that you might install. export MDX="/software/mdx/bin" export
FFTW_LIB="/software/ROI_PAC_3_0_1/NetInst/fftw-071005-1457/lib" export PATH="$PATH:$INT_BIN:$INT_SCR:$MDX"
6. Update Path To Perl
The path to the perl executable may be different on your machine than is used in the ROI_PAC scripts. You can check if this step is necessary by comparing the paths in the following outputs: % cd ROI_PAC_3_0_1/ROI_PAC/INT_SCR
% which perl /usr/bin/perl
The example shown above shows that no changes are necessary. If the paths are different, there is a simple script in ROI_PAC_3_0_1/ROI_PAC/INT_SCR that will edit this path for you in all of the perl scripts.
% cd ROI_PAC_3_0_1/ROI_PAC/INT_SCR % chgperlpath.pl
7. Celebrate
You are now ready to process your own SAR data. Create a workspace where you will process future data. You may also want to create a new directory elsewhere on your system where you can archive your DEM and orbit files. Update SAR_CONFIG with the new location of the orbit files. Remember to source your SAR_CONFIG in each new shell before you start to process data. Alternatively, you can add a line to the file that initializes your shell environment upon startup (such as .cshrc or .bash_profile).
Some notes on compiling ROI_PAC for Mac 10.6.x (Snow Leopard)
I recently compiled ROI_PAC on a 64-bit Mac (OS 10.6.8, little endian). Here are notes on the procedure.
1. Obtain Compilers
The Snow Leopard operating system does not include any compilers by default. In order to obtain compilers, you must install XCode, which is the Mac “developer” software. The newest version of XCode (4.2.x) is not available for free – it is $99 if you don’t already have a license. The older XCode (3.2.x) was free, but is no longer downloadable. If you can obtain a .dmg for XCode 3.2.x, this can be installed for free.
All versions of XCode include a gcc compiler. Therefore, simply installing any version of
XCode will allow you to install and compile the FFTW library. However, you will get stuck after this point if you have not also installed a separate fortran compiler (in other words, no fortran compiler is included with XCode 3.2.x). The following website has resources for installing gfortran: http://r.research.att.com/tools/
The first .dmg (4.2.3) on the website will not work with XCode – you will not be able to get ROI_PAC to link to FFTW during compilation if you install gfortran4.2.3. You must instead install the gfortran-42-5xxx version of gfortran (whichever matches your XCode version). 2. Build Without Multibuild
I found that building ROI_PAC with these compilers was easier without using multibuild.sh. Following the instructions in /ROI_PAC_3_0_1/ROI_PAC/INSTALL, I did the following:
% LDFLAGS=-L/$FFTW_LIB_DIR F77=gfortran ./configure --prefix=/ROI_PAC_3_0_1/ROI_PAC/INT_BIN
% mkdir /ROI_PAC_3_0_1/ROI_PAC/INT_BIN % sudo make
% sudo make install
I found that specifying the fortran compiler was necessary, as was setting the “LDFLAGS” option (even though $FFTW_LIB_DIR was already exported). Finally, I put the “prefix” option in order to direct the executables to INT_BIN (otherwise I’m not sure where they would end up). After this, I proceeded with the instructions above to set up SAR_CONFIG and so forth.
Appendix 2: ROI_PAC file types and file formats
In outline, the files output by ROI_PAC have 4 basic types:
1) ASCII files used for input or output by programs (suffixes: .out, .in, .off: more details below)
2) Binary files as real or complex files (see below)
• RMG format (Names for JPL radar pioneer Richard M. Goldstein) are made up of real*4 (32 byte) numbers in two arrays side-by-side. The two arrays often show the “magnitude” of the radar image and the “phase” – although not always (sometimes the “phase” is the correlation. The length and width of each array are given as lines in the metadata (.rsc) file. Thus the total width of the binary file is (2 * width) and the data are stored as (Magnitude, magnitude, magnitude…, Phase, Phase, Phase,…)
• complex or c8 files contain real and imaginary values that are stored in the binary file as (real, imaginary, real imaginary, etc.)
3) ASCII files associated with each binary file containing metadata (end with .rsc – resource file); and a record of the changes made to the metadata file (.rsc.hst – history file)
4) ASCII log files that record the commands that have been run in each directory where ROI_PAC is run. The log files provides a record of the commands that have been run and the times that these were run. During debugging you can often just copy and paste these commands onto the command line modifying one or more values. The log1 file shows the information that was written out by the scripts to the screen, but unfortunately it usually does not include error messages.
We now describe the specifics of these files in categories of input file (user-created) and output files (generated automatically from ROI_PAC) divided by the suffix to the file name:
•Input files:
• *.proc = ASCII text file specifying parameters used in processing and there are two main types: int*.proc files for creating the interferograms (required and the suffix is allowed to be different) and optional roi.proc or (DATE).proc files that are used to create the slc’s. See Appendix 4 for details on the different parameters that can be set.
• *.dem = binary file with digital elevation model in lat-long or UTM coordinates, signed integer 2-byte values (in meters unless scaling specified). An important question to consider when comparing InSAR results to GPS or others is whether the heights in the DEM file refer to heights relative to the ellipsoid or the Geoid (SRTM refers to Geoid heights). The .dem and .dem.rsc file can be automatically generated by the script
get_SRTM.pl available at http://aws.roipac.org/cgi-bin/moin.cgi/ContribSoftware or you can create your own files. The format for the .rsc file is as follows:
• WIDTH number of columns • FILE_LENGTH number of rows
• X_FIRST longitude of upper left corner (does it refer to center of upper left pixel?)
• Y_FIRST latitude of upper left corner (does it refer to center of upper left pixel?)
• X_STEP longitude pixel size in degrees for latlon, meters for UTM • Y_STEP latitude pixel size in degrees for latlon, meters for UTM • X_UNIT degrees or meters
• Y_UNIT degrees or meters •Output files:
• *.off = ASCII text file of offsets measured between two images
columns 1=X, 2=ΔX, 3= Y, 4=ΔY, 5=Signal-to-noise ratio, other columns are three components of the match covariance
• *.in = ASCII text file created by ROI_pac scripts used as input for a compiled program • *.out = ASCII text informational output of a program
• *.hgt = binary file with simulated SAR amplitude image and elevation in radar coordinates, real 4-byte values band interleaved by line (BIL or “rmg”)
• *.slc = complex real 8-byte binary file containing real and imaginary parts of the single-look complex (SLC) image (and multisingle-looked versions of the SLC)
• *.cor = binary file with average amplitude of SAR images used to form an interferogram and correlation measure of coherence, real 4-byte values band interleaved by line (BIL or “rmg”)
• *.raw = binary file of raw data, I,Q (see section II for explanation) 1-byte integer values for each echo sample
• *.aff = ASCII text file of affine transformation to map simulated image to actual SAR image
• *.unw = binary file with SAR amplitude image and unwrapped phase, real 4-byte values band interleaved by line (BIL or “rmg”). The units in the file are radians and can be converted to cm by multiplying by the radar wavelength and dividing by 4*pi. NOTE: phase change values in this file are relative – that is zero radians of phase change could actually be -1 m, +1 m, or some other value that must be otherwise constrained.
• *.flg = binary file with flags used and resulting from unwrapping with standard unwrapper, 1-byte values with flags set in bits
• * int = complex real 8-byte binary file containing real and imaginary parts of the interferogram (can also be read as real or float values band interleaved by pixel)
• .amp = binary file with amplitudes of the two SAR images used to form an
interferogram, real 4-byte values band interleaved by pixel (can also be read as complex 8-byte values)
• *.msk = binary file with SAR amplitude image and coherence with zeros in masked out areas, real 4-byte values band interleaved by line (BIL or “rmg”)
• *.trans = binary file with inverse mapping transformation from SAR to DEM coordinates, two bands are range and azimuth pixel locations of SAR for each DEM pixel, real 4-byte values band interleaved by line (BIL or “rmg”)
The file names before the suffix are descriptive of the steps that were taken to create that file. For example:
*rlks* Specifies the number of range looks taken from the full resolution interferogram or, for a geocoded file, from the resolution of the full resolution geocoded file
filt_* means that the file has been filtered by the filter.pl script
*-sim* means that the simulated topographic interferogram has been removed flat_* the file has been flattened (a.k.a. orbital effects have been removed) geo_* file has been geocoded
SIM_* The DEM has been converted to radar coordinates (simulated interferogram) using orbital information but no fine adjustment of the position has been made (see radar_*)
radar_* The DEM has been converted to radar coordinates (simulated interferogram) using orbital information and a fine adjustment of the position has been made (this file is created after SIM_*)
radar_*.unw is created by diffnsim.pl and includes the orbital & topographic effects
radar_*.hgt is created by rect.pl and includes only the topographic effects ramp_* The orbital phase ramp created by diffnsim.pl with no topographic effects *SIM* or *ODR* or *PRC* or *HDR* means that the orbital effects have been removed using the given type:
ODR = Delft PRC = DPAF
SIM = re-estimated baselines
Appendix 3: ROI_PAC processing flow: Script-by-Script Before you get started, you need the following 5 things:
1) Orbit files may need to be downloaded directly from a space agency or other institutions – it depends on which satellite you are using. See Appendix 6
2) InSAR processing software: see: Appendix 1 for installing ROI_PAC. To make sure it is working properly it is recommended to try the test data and ensure that the processing results are correct.
3) Digital Elevation Model (Definitely try SRTM if available (between 60 degrees north and 56 degrees south); The script get_SRTM.pl will allow you to download the software and convert it to ROI_PAC format by specifying a few simple parameters (lat/lon, global vs. US versions, etc.) The latest version is available at:
https://github.com/scottyhq/insar_scripts/tree/master/DEM
4) Viewing software: http://aws.roipac.org/cgi-bin/moin.cgi/Viewing_results
also see Appendix 8
5) Raw data: for information on how to download, see Appendix 7 Step 1:
Create the proper directory structure (two subdirectories with the raw data from the two different dates to be combined. Format could be YYMMDD). Then within each subdirectory, run the following command for ERS data, where DATE = YYMMDD (or whatever date format you use for the directory name).
Run make_raw.pl ORBIT_TYPE LEADER_FILE DATE
There are different scripts if you are using JERS, RADARSAT, ENVISAT, or ERS data downloaded in CEOS format from ASF. Those that are not available in ROI_PAC are available at: http://aws.roipac.org/cgi-bin/moin.cgi/SatelliteIssues. However, once you've made raw, the following steps are more-or-less the same for all data types.
Data ingestion: make_raw_(SAT).pl
The make_raw script will prepare the files for processing in ROI_PAC, performing an initial check of the file, reformatting the data, locating the appropriate orbit file, and performing a Doppler computation. There are different ingestion programs for each satellite type
• ERS-1 and ERS-2: use “make_raw.pl”
• For data formatted at the Alaska Satellite Facility use “make_raw_ASF.pl” • Envisat: use “make_raw_envi.pl”
• ALOS: use “make_raw_alos.pl” • JERS-1: use “make_raw_jers.pl”
• COSMO-SkyMed (CSK): (to be released in version 3.1) • raw, use “make_raw_csk.csh”
• SLC, use “make_slc_csk.csh”
• TerraSAR-X and TanDEM-X (TSX & TDX): (to be released in version 3.1) • SLC, use “make_slc_tsx.csh”
• Radarsat-1 has two data products from the Alaska Satellite Facility • CEOS: use “make_raw_RSAT-CEOS.pl”
• STF: use “make_raw_RADARSAT_swath.pl” (Not yet released)
You can run make_raw_(SAT).pl with no arguments to see a detailed usage page. As an example, to run with ERS, “cd” to the directory containing the master SAR scene and run the following command (using Delft “ODR” orbits, you could also use the “HDR”, header file orbits or “PRC” orbits):
$ make_raw.pl ODR SARLEADER(DATE1) (DATE1)
Repeat the steps: “cd” to the directory of the slave SAR scene, and run make_raw_XXX.pl adjusting the date and filename appropriately. You are now ready to produce an interferogram.
NOTE: make_raw_(SAT).pl can concatenate multiple frames together – you do not need to run each frame separately. For the concatenation to work, you need to have all of the frames together in the same directory and have their file names be in numerical order so that the program can figure out which goes first, second, third, etc. based just on the file names. For Envisat and ALOS data, the files usually come in the correct format for concatenation – the file time (Envisat) or frame number (ALOS) usually appear in the file name. For ERS, depending on how you receive the data, you may need to rename the binary data and leader files to have each frame be in the correct order.
The output files are:
• $date.raw–unpacked raw data 8 bits I,Q for each sample, one line per echo record, ERS has 412 bytes extra at start of lines
• $date.raw.rsc–metadata for raw data
• hdr_data_points_$date.rsc–orbit data reformatted
• other intermediate files, including dop.unw–output of programs to measure Doppler centroid of data (fit put into .raw.rsc)
The other programs called by make_raw_(SAT).pl include (information here may not be completely up-to-date, please read the scripts themselves for the more details);
•make_raw.pl: new_parse – finds missing lines and adds duplicates, finds overlapping lines and removes
delay_shift – shifts the lines in the raw data to remove offset (called swst) due to satellite geometry
leader2rsc – takes information from the leader file to put into the .rsc file state_vector.pl -- calculates satellite position & velocity at requested epoch dopiq
GetPeg.pl
•make_raw_envi.pl: asa_im_decode
state_vector.pl -- calculates satellite position & velocity at requested epoch scan_doppler.pl --
make_raw_jers.pl autofocus.pl
JoinJData – concatenate scenes and remove raw data header Calc_squint.pl
leader2rsc – takes information from the leader file to put into the .rsc file mlcc – used in Doppler centroid estimation
dopiq – used in Doppler centroid estimation •make_raw_alos.pl:
ALOS_pre_process ALOS_merge ALOS_fbs2fbd ALOS_fbd2fbs
state_vector.pl -- calculates satellite position & velocity at requested epoch •make_raw_RSAT-CEOS.pl:
state_vector_RSAT.pl -- calculates satellite position & velocity at requested epoch
extract_UT1_UTC
parse_RSAT.le or parse_RSAT.be – modify rough raw data into conditioned raw data
dopiq-new – used in Doppler centroid estimation
Step 2:
Create the proper processing file called "date1-date2.proc"
Run process_2pass.pl "date1-date2.proc" [DoItFrom] [EndItAt] (see table below for some possible [DoItFrom] or [EndItAt] points)
process_2pass.pl is the most commonly run script, but others
(http://geophysics.eas.gatech.edu/people/anewman/classes/MGM/InSAR/ROI_PAC_DO CS/process_index/perl_scripts.html) are sometimes needed [this list is incomplete and a little out of date, but it's better than nothing!].
If you are exceedingly lucky, process_2pass.pl can go from the raw data to making an interferogram to removing the orbital effects ("flattening") to removing the topographic effect to unwrapping to geocoding in a single run. More likely, you are going to start and stop this flow at various points to fix errors or otherwise change the parameters (see the table below). There are numerous such "start" and "stop" points listed below, but we highlight (in purple bold) the ones that you will use most frequently.
[DoItFrom] [EndItAt] point Comments
raw Always start here after running make_raw.pl roi_prep Create input files for SAR processor
orbbase Calculate baselines
slcs Create single-look-complex interferograms offsets Co-register two slcs: Frequently restart here
after (problem #2a on this list) resamp Resample slave scene to master coordinates
flatorb "flatten" scene and remove orbital ramp
full_res Good stopping point: have created
interferograms and correlation files done_sim_off Calculate offsets between interferogram and DEM
done_sim_removal Good stopping point: have removed the topographic effects from the flattened interferogram
begin_filt filter the interferogram
filtered done filtering
make_mask create mask for unwrapping
unwrapped Good stopping point: Flattened interferogram with topography removed is unwrapped
redo_base Empirically re-estimate the baselines sim_removal_bsim remove topo from newly re-flattened interferogram
unwrapped_bsim unwrap newly re-flattened interferogram
done
Final stopping point: Put flattened unwrapped interferogram with topography removed in
geographic coordinates
How long will it take? Processing a single frame interferogram takes on the order of an hour (using the default parameters, although it can take longer if you have a noisy interferogram) from raw data to the final interferogram with the orbital and topographic effects removed, unwrapped and placed in geocoded coordinates. Of course this does depend on your computer processing speed and whether you use $concurrent_roi=yes among other options. All output files and directories will be placed in the directory where param.proc tells them to. If the script terminates successfully, the final line of terminal output will be “That’s all folks.”
How much memory do I need? For most satellites besides ALOS, you can process most data with 2 Gb of RAM. For ALOS you need at least 4-8 Gb.
How much disk storage do I need? For a single frame of ERS data (other satellites may be bigger or smaller), the slc’s are each about 1.5 Gb, the full resolution interferograms are about 1 Gb. Thus, the complete int* directory is of order 6 Gb and thus including raw data and slc’s a single frame completely processed to done is about 10 Gb. Multiply this number by the number of frames to get an estimate.
process_2pass.pl calls secondary scripts that are listed here in the order with which they are called and a brief description of their purpose (information here may not be completely up-to-date, please read the scripts themselves for the more details):
[DoItFrom= raw]
raw2ampintcor.pl is a script that calls other scripts to go from the steps of making the slc’s, interferograms and correlation files
dopav.pl -- checks to make sure the bandwidth of the two slcs are within
reasonable limits. If not, it removes the slc files which kills the process_2pass.pl Calculates the mean Doppler centroid for the pair of images and common
bandwidth
uses: (date).raw.rsc files makes: roi.dop.rsc files
roi_prep.pl -- Creates the input files for the roi.pl script
uses: (date).raw; (date).raw.rsc; roi.dop; (date).proc [optional] roi.proc
[optional]
makes: (date).slc.rsc
(date)_roi.in -- Input file for the roi program
calls: GetPeg.pl – calculates peg point on the ground [EndItAt= roi_prep]
baseline.pl -- calculates baseline between image pair from orbits or header information
uses: (date1).slc.rsc; (date2).slc.rsc makes: (date1)_(date2)_baseline.rsc
calls: state_vector.pl (calculates satellite position & velocity at requested
epoch) [EndItAt= orbbase]
roi.pl -- Creates the slcs from the raw data
uses: (date).raw; (date).raw.rsc; roi.dop; (date).proc [optional] roi.proc
[optional]
makes: (date).slc; (date).slc.rsc
(date)_roi.out output file for the roi program
calls: the roi -- FORTRAN program creates the slc’s;
look.pl -- Takes specified number of “looks” of an imagefile. This script is called several times throughout process_2pass.pl, but we will only refer to it at this first usage
uses: (date).slc; (date).slc.rsc
makes: (date)_rlks.slc; (date)_rlks.slc.rsc [EndItAt= slcs]
[DoItFrom= slcs]
make_offset.pl -- computes offset between two slcs as a function of x, y in the reference image. The program calculates the offsets twice, once at large spacing between searches with a large search window in order to get a rough estimate of the offset (*ampcor_gross*) and then with a denser spacing between searches and a smaller search window (*ampcor*)
uses: both (date).slc and (date).slc.rsc files makes: gross_x; gross_y;
(date1)_ (date2)_ampcor_gross.in; (date1)_ (date2)_ampcor_gross.off; (date1)_ (date2)_cull_gross.off; (date1)_ (date2)_ampcor_gross.out; (date1)_ (date2)_ampcor.in; (date1)_ (date2)_ampcor.off; (date1)_ (date2)_cull.off; (date1)_ (date2)_ampcor.out; fitoff_ampcor_gross.out fitoff_ampcor.out;
calls: offset.pl -- which actually calculates the offsets once all files are set
up (this is the program that is called twice for the gross and fine offsets) fitoff – selects good points from the ampcor file and puts them in the *cull* file
[EndItAt= offsets] [DoItFrom= offsets]
resamp.pl -- makes the interferogram and amplitude files
uses: slc and slc.rsc files; (date1)_ (date2)_cull.off; makes: (date1)_ (date2).int; (date1)_ (date2).int.rsc
(date1)_ (date2).amp; (date1)_ (date2).amp.rsc
(date1)_ (date2)_resamp.in; (date1)_ (date2)_resamp.out
calls: resamp_roi -- FORTRAN program that actually makes the
interferogram after putting the slc’s in the same reference frame. Fits a second order polynomial to the culled offsets and uses this to resample the
slave slc to the master. Interferogram formed by multiplying the complex pixel of master scene by the complex conjugate of the slave scene
[EndItAt= resamp] [DoItFrom= resamp]
add_rmg.pl -- This is a utility program used throughout the scripts that works on two files of the same size and will add or subtract rmg them together or you can not add/subtract but mask certain values. You have the choice of outputting zero at a given pixel when one of the images has a zero in it or both do. In this
location in the script, the program creates a file that is the size of the
interferogram but that is all set to $ref_height (default is zero) to be used by diffnsim.pl to “flatten” the interferogram
uses: junk.unw
makes: reference.hgt; reference.hgt.rsc
diffnsim.pl -- are then used to “flatten” the interferogram, meaning that the orbital effects are removed but not the topographic effects
uses: (date1)_ (date2).int; makes: flat_(date1)_ (date2).int;
ramp_ORBIT_TYPE.unw diffnsim_flat*.int.in
calls: diffnsim -- computes the differential interferogram (in this instance,
it only removes the orbital effects) [EndItAt= flatorb]
[DoItFrom= flatorb]
make_cor.pl -- creates an rmg file with average amplitude and correlation between images
uses: flat_(date1)_ (date2).int; (date1)_ (date2).amp makes: (date1)_ (date2).cor;
calls cchz_wave -- which calculates the correlation using a 5x5 triangular
weighted window
replace mag.pl -- replaces the magnitude from infile1 with the magnitude from infile2. Replaces the magnitude of the correlation file with the average from the two scenes
uses: flat_(date1)_ (date2).int makes: (date1)_ (date2).cor;
separate rmg or cpx files into their magnitude and phase components and then combine them together again.
[EndItAt= full_res] [DoItFrom= full_res]
dem2diff.pl -- is a script that calls other scripts to go from the original interferogram to one that has both the orbital and topographic effects removed
make_sim.pl -- makes a simulated interferogram that only includes the effects of topography that will be removed from the interferogram created in the
raw2ampintcor.pl script.
Uses: .dem; .dem.rsc The dem file format is discussed in appendix 2 Makes: .dem.slp (if it doesn’t already exist) This is a dem slope file
SIM_raw.hgt first guess of dem converted into radar coordinates. This files has holes in it and must be interpolated using the Aik_resamp.in file and the Aik_resamp program to make SIM_rlks.hgt
SIM.orrm orbit information to determine the location of the satellite relative to the simulated interferogram
IntSim.in; IntSim.out
These files are created by are not used: GPS.in; GPS.out; simfile; simoutfile
Calls: make_orrm.pl -- creates the SIM.orrm file
Gradient.pl -- (if the .slp file doesn’t exist) creates the slope file
IntSim -- the program that actually does the calculations to convert from geographic to radar coordinates
Aik_resample – interpolates SIM_raw.hgt to fill in holes caused by projecting DEM to radar coordinates (especially on steep slopes) or because DEM and interferogram had different pixel spacings
Length.pl -- which updates the rscfile with the new length of the file Synth_offset.pl -- Finds the offset field between the synthetic interferogram and satellite images because the orbital information does not align them perfectly
Uses: (date1)-(date2)_rlks.cor SIM_rlks.hgt Makes: ampmag_gross.in ampmag_gross.off ampmag_gross_cull.off ampmag_gross.out ampmag.in ampmag.off ampmag.out ampmag_cull.off cull.out
Calls: offset.pl -- which actually calculates the offsets once all files are set
fitoff – selects good points from the ampmag file and puts them in the *cull* file
[EndItAt= done_sim_off] [DoItFrom= done_sim_off]
Synth2radar.pl -- Transforms the synthetic interferogram in radar coordinates to precisely align with the interferogram using the offsets calculated from
syth_offset.pl
uses:SIM*.hgt; cull.out
makes:radar_*.hgt; (date1)-(date2)_*rlks.aff
calls:rect.pl – morphs an rmg or cpx file given affine transformation
parameters
find_affine.pl – extracts the affine matrix and translation vector from cull.out
diffnsim.pl – looks down the interferogram, removes the orbital effects, and subtracts the simulated topography
uses: (date1)-(date2).in; radar_*rlks.hgt makes: (date1)-(date2)-sim-*rlks.int;
radar_*rlks.unw; diffnsim*.int.in
calls: diffnsim -- computes the differential interferogram (in this instance,
removing the orbital and topographic effects) [EndItAt= done_sim_removal]
[DoItFrom= done_sim_removal]
radarseismodel.pl – optional script (if $model is not “null”): builds the radar files corresponding to seismic deformation models: calls geo2radar.pl; disp2phas.pl; Aik_resample; rect.pl; add_rmg.pl; rmg2mag_phs; mag_phs2rmg
removeModel.pl – optional script (if $model is not “null”) removes the model from the interferogram
rect_cut.pl – optional script (if $man_cut is not “null”) rectifies a BYTE file given an affine transformation
[DoItFrom= begin_filt]
int2filtmaskunwrap.pl -- is a script that calls other scripts to go from the steps of filtering, masking, and unwrapping the interferogram
filter.pl – filters an interferogram using the power spectrum filtering method (Goldstein and Werner, 1988) (psfilt is the default) or the adaptive filtering
Uses: (date1)-(date2).int makes:filt_(date1)-(date2).int
calls: psfilt – program that actually does the power spectrum filtering
adapt_filt.pl – program that does the adaptive filtering
rmg2mag_phs; cpx2mag_phs; mag_phs2cpx; mag_phs2rmg -- These are utility programs written in C used throughout the perl scripts to separate rmg or cpx files into their magnitude and phase components and then combine them together again.
[EndItAt= done_filt] [DoItFrom= done_filt]
make_mask.pl – computes a mask from an interferogram to remove low
correlation areas. Used in branch-cut unwrapping or to mask out areas in snaphu unwrapping
uses:
makes:phase_var*rlks.msk
calls: phase_slope -- removes a local phase slope
phase_mask -- makes a mask based on local slope-corrected phase variance to remove noisy areas apples $sigma_thresh
add_phs
rmg2mag_phs; cpx2mag_phs; mag_phs2cpx; mag_phs2rmg -- These are utility programs written in C used throughout the perl scripts to separate rmg or cpx files into their magnitude and phase components and then combine them together again.
[EndItAt= make_mask] [DoItFrom= make_mask]
new_cut.pl -- First step of branch-cut unwrapping algorithm -- computes the residues and builds trees to connect them
Uses: filt_(date1)-(date2)-sim_rlks.int; filt_(date1)-(date2)-sim_rlks.int.rsc Makes: filt_(date1)-(date2)-sim_rlks_cut.flg;
filt_(date1)-(date2)-sim_rlks_cut.flg.rsc
calls: residues -- calculates phase residuals from filtered and
slope-removed interferograms
trees -- draws phase cuts to connect the residues (this program can take days or longer to run if the file is large and/or noisy – if so, consider running on a looked down version).
unwrap.pl -- unwraps an interferogram using the branch-cut algorithm using a .glf file. Leaves gaps in the unwrapped interferogram where correlation is below the threshold.
Uses: filt_(date1)-(date2)-sim_rlks.int;
filt_(date1)-(date2)-sim_rlks_cut.flg; Phase_var*rlks.msk
Makes: filt_(date1)-(date2)-sim_rlks_cTHRES.flg; THRES is the
UnwrappedThreshold set in the .proc file
filt_(date1)-(date2)-sim_ORBIT_*rlks_cTHRES.unw; this is the filtered, unwrapped, flattened interferogram with the topography removed. Any voids are from regions that did not unwrap. If the whole file is blank see the troubleshooting
section.
Calls: rmg2mag_phs; mag_phs2rmg -- These are utility programs written
in C used throughout the perl scripts to separate rmg or cpx files into their magnitude and phase components and then combine them together again. corr_flag -- combines the unwrapping mask with the phase cut trees applying the coherence threshold $UnwrappedThreshold
grass -- unwraps the filtered interferogram phase between the tress of phase cutes, using the mask and starting at the $unw_seedx, $unw_seedy location which is defined as zero.
[EndItAt= unwrapped] [DoItFrom= unwrapped]
add_rmg.pl – only used with “old” unwrapper to combine phase_var*.msk with filt_(date1)-(date2)-sim_rlks_cTHRES.unw to create low_cor*.msk with zeros in areas that didn’t unwrap
add_rmg.pl – only used with “old” unwrapper to combine low_cor*.msk with radar_*.hgt to create basest*rlks.msk to use in baseline re-estimation
add_rmg.pl – only used with “old” unwrapper and when $model is not equal to “null” to combine baseest*rlks.msk with seismic model to create basest*rlks.msk to use in baseline re-estimation
add_rmg.pl -- This is a utility program used throughout the scripts that works on two files of the same size and will add or subtract rmg them together or you can not
add/subtract but mask certain values. You have the choice of outputting zero at a given pixel when one of the images has a zero in it or both do. In this location in the script, the program combines the unwrapped file with the file that includes the topographic and orbital effects to make what is effectively an unwrapped “original” interferogram with no orbital or topographic effects removed
uses: filt_(date1)-(date2)-sim_rlks_cTHRES.unw; radar*rlks.unw makes: total_(date1)-(date2)_*rlks.unw and the identical file
add_rmg.pl – only used when $model is not equal to “null” to remove seismic model from
total_(date1)-(date2)_*rlks.unw to make new pha4baseest.unw
phase2base.pl – used to re-estimate the baselines from the unwrapped interferogram and the DEM in corrected radar coordinates.
uses:pha4baseest.unw; radar*rlks.hgt; baseest*rlks.msk; phase2base.in makes:phase2base.out
calls:baseestq – fortran program that actually calculates quadratic baseline [EndItAt= redo_base]
[DoItFrom= redo_base]
diffnsim.pl – looks down the interferogram, removes the orbital effects using the re-estimated baselines, and subtracts the simulated topography
uses: (date1)-(date2).in; radar_*rlks.hgt makes: (date1)-(date2)-sim_SIM*rlks.int;
radar_SIM*rlks.unw; diffnsim*SIM*.int.in
calls: diffnsim -- computes the differential interferogram (in this instance,
removing the topographic effects and orbital effects from the re-estimated baselines)
[EndItAt= sim_removal_bsim] [DoItFrom= sim_removal_bsim]
add_rmg.pl -- This is a utility program used throughout the scripts that works on two files of the same size and will add or subtract rmg them together or you can not
add/subtract but mask certain values. You have the choice of outputting zero at a given pixel when one of the images has a zero in it or both do. In this location in the script, the program removes the re-estimated orbital and topographic effects from the “original” unwrapped interferogram
uses:; total_(date1)-(date2)_*rlks.unw; radar_SIM*rlks.unw makes: filt_(date1)-(date2)-sim_SIM_*rlks_cTHRES.unw
add_rmg.pl – only used when $model is not equal to “null” to remove seismic model from
total_(date1)-(date2)_*rlks.unw to make new filt_(date1)-(date2)-sim_SIM_*rlks_cTHRES.unw
[DoItFrom= unwrapped_bsim]
radar2geo.pl – transforms the interferogram in radar coordinates into the geographic coordinates of the DEM
make_geomap.pl – creates the look-up table to convert from radar to geographic coordinates
uses: *.dem; SIM.orrm
makes (date1)-(date2)_*rlks.trans; *.dem.slp (if it doesn’t already exit)
IntSim.in; IntSim.out
calls:IntSim -- creates the look-up file (date1)-(date2)_*rlks.trans
length.pl
geocode.pl – uses the look-up file ((date1)-(date2)_*rlks.trans ) to convert interferogram to geocoded file
uses: ((date1)-(date2)_*rlks.trans;
filt_(date1)-(date2)-sim_ORBIT_*rlks_cTHRES.unw
makes:rect_lookup.in; geo_(date1)_(date2).unw
calls:rect_lookup – actually does the transformation from radar to
geographic coordinates length.pl
geomodel.pl – only used if $model is not “null” creates the geocoded interferograms minus the seismic deformation models
Appendix 4: User defined options for ROI_PAC processing The int*.proc file (see Appendix 4) must contain the following:
•SarDir1 = name of image directory 1 (master scene) with raw data •SarDir2 = name of image directory 2 (slave scene) with raw data
•IntDir = name of interferogram directory (e.g., int_031101_060916), better to include full path
The following commands are all options within the int*.proc file (with the default values):
$DEM= path to the location of the .dem and .dem.rsc file If no DEM is available you can use the process_Npass.pl method (this hasn’t been used for many years, so it is probably out of date)
$concurrent_roi (no) when =yes runs roi on both scenes at the same time. Only use if enough CPU’s and RAM are available
$GeoDir ("GEO") Location to put the geocoded interferograms
$FilterStrength (0.75) coefficient in the power spectrum filtering algorithm of psfilt and filter.pl – bigger numbers= more filtering. The complex FFT is taken of a
two-dimensional square of pixels, then raised to the power of this coefficient, and finally an inverse FFT is taken. This removes high frequency noise, but beware: sharp
discontinuities (like faults) have power at all frequencies and this process smooths them out.
$UnwrappedThreshold (0.1) correlation threshold for unwrapping with the branch-cut algorithm of unwrap.pl Smaller values mean that more of the image will be unwrapped, potentially including more errors
$OrbitType ("ODR") The type of orbital parameters to use in processing to remove the orbital effects.
$BaselineType ($OrbitType) I’m not sure why you would change this from the default value.
$Rlooks_sim (4) Specifies the number of looks to take of the full resolution interferogram before removing the topographic effects from the DEM (a.k.a. the simulated interferogram). The default value of 4 comes from the ERS full resolution pixel resolution (~20 m) and the SRTM global resolution (~90 m). That is, int(90/20) = 4. For other DEM’s or satellite pixel resolutions, other values should be used.
$Rlooks_int ($Rlooks_sim)
$Rlooks_unw ($Rlooks_sim) Number of looks to take of the full resolution interferogram before doing the unwrapping.
$Rlooks_sml (16)
$pixel_ratio (5) The ratio between the range_ground_pixel_size and the
azimuth_pixel_size to be used when creating the interferogram so that the pixels are square. These values were taken from ERS where the azimuth pixel size is ~4 m/pixel and the range ground pixel size is ~20 m/pixel (slant range pixel size divided by
sine(incidence angle)), so we want to compress 5 azimuth pixels together (20/4 = 5). For other satellites, you should change this number or your interferogram will look “stretched out.”
$Alooks_sml ($Rlooks_sml*$pixel_ratio) $usergivendop1 (0)
$usergivendop2 (0)
$unw_seedx (-9999) starting point in x direction for branch-cut unwrapping algorithm (default is image center) See next section of this Appendix for more details.
$unw_seedy (-9999) starting point in y direction for branch-cut unwrapping algorithm (default is image center) See next section of this Appendix for more details.
$x_start (0.01) Mean offset between slc’s in the range (x) direction $y_start (0.01) Mean offset between slc’s in the azimuth (y) direction $Threshold_mag (5.0e-5) Used in creating .msk file
$Threshold_ph_grd (0) Used in creating .msk file
$sigma_thresh (1.0) Used in creating .msk file (1.4 is a good value – larger sigma_thresh mean that fewer values will be masked out)
$smooth_width (5) Used in creating .msk file $slope_width (5) Used in creating .msk file $mapping ("dem_based")
$cleanup ("no") Deletes slc and *gross* files when reamp.pl is complete
$CO_MODEL ("NULL") Use a co-seismic deformation model in the geographic coordinates of the DEM
$INTER_MODEL ("NULL") Use an inter-seismic deformation model in the geographic coordinates of the DEM
$Filt_method ("psfilt") type of algorithm to use in filter.pl, alternative is adapt_filt $unw_method ("old") type of unwrapping algorithm to use – default is branch-cut, but other options are icu and snaphu. See next section of this Appendix for more information. $flattening ("topo") specifying “topo” means that the baselines should be re-estimated after the unwrapping step before geocoding (the first flattening in raw2ampintcor.pl will use the orbit files as always). Specifying “orbit” means that the baselines will not be re-estimated
$do_sim ("yes") If set to “no” the make_sim.pl step will be skipped. You might choose “no” if you are processing multiple interferograms in a time series that will be
co-registered so that you can use the same DEM converted to radar coordinates for all of them.
$do_mod ("yes") #could be: no ,GP 11.10.04 $unw_mod ("yes") #could be: no ,GP 11.10.04 $MAN_CUT ("NULL") #filename of man_cut (SIM)
$BaselineOrder ("QUAD") When calculating how the baseline changes as a function of line number in the interferogram include quadratic terms (alternative is only linear terms: “LIN”)
$MPI_PARA ("no")
$NUM_PROC (1) Number of processors to use $ROMIO ("no")
$ref_height (0.) Sets the reference elevation
The optional roi.proc or (DATE).proc files have the following options that can be set (see roi_prep.pl script for details on what each variable means):
$before_z_ext (int($az_frac * $synth_apert_samps + 0.5)) Use this command to crop the top of the image by a certain number of pixels (a negative number removes or crops pixels while a positive number adds pixels or zero-pads the image)
$after_z_ext (int($az_frac * $synth_apert_samps + 0.5)) Use this command to crop the bottom of the image by a certain number of pixels (a negative number removes or crops pixels while a positive number adds pixels or zero-pads the image)
$near_rng_ext (int($chirp_samps * $rg_frac )) Use this command to crop the image in the near range (small x or left side of the image) direction by a certain number of pixels