Diffin3dis written in the Fortran programming language, which is used widely for mathemat- ical and scientific work. It is well suited for this type of work for a number of reasons. Fortran has a large library of standard mathematical functions such ascos(x),exp(x), etc. It has a range of data types (integers, decimals, complex numbers and boolean values) and can operate with a precision of at most 8 or 16 decimal places. Fortran has a well established standard, which makes the software extremely portable between different platforms.
Diffin3dexploits the Message Passing Interface (MPI) provided by Fortran. MPI is a library specification for message-passing, proposed as a standard by a broadly based committee of ven- dors, implementors, and users. MPI commands allow messages to be passed between processors and, therefore, enables the code to be run in parallel across any number of processors. The simu- lations described in detail in this thesis typically use 16 or 32 processors. Spreading the work load over several processors running in parallel can drastically reduce the run time of a simulation. It should be noted that the total number of computational hours for the simulation will not reduce since this is the sum of each processor’s runtime.
The optimal number of processors for an individual simulation to be run over will be determined by several factors:
• the number of gridpoints in the computational domain,
• the time available for the code to run,
• any limit imposed on the size of individual files when data is output and stored individually for each processor.
Increasing the number of processors is not always advantageous since the inclusion of each new processor adds communication time, which is usually slower than the same calculation carried out
Figure 3.13: The computational grid is divided amongst the allocated processors, such that the cells in the same vertical strip are housed in the same processor. In this example, the grid has been split over 4 processors.
on a single processor.
WhenDiffin3dis run, the MPI commands divide the computational domain amongst the num- ber of processors requested by the user. Division of the grid occurs only in the horizontal direc- tions, thus data points in the same vertical column will be housed on the same processor, as shown in Figure3.13.
During the finite difference operations in the horizontal directions, it is necessary for the proces- sors to communicate with each other when the neighbouring points required lie across processor boundaries. Additionally, the use of periodic boundaries requires information to be transferred at grid boundaries. By using the virtual topologies provided with MPI, communication between the processors can be written in a simpler, minimal style within the code. A virtually topology is a mechanism for naming the processors being used such that each processor knows which other processors it is connected to.
Diffin3duses the “cartesian virtual topology”, which is suitable for grid-like topologies. As illustrated by Figure3.14, each processor is connected to its neighbours on a virtual grid system and knows exactly which other processors lie immediately north, south, east and west of it. The use of the topology means data can be passed during the finite difference calculations without the code having to figure out exactly which processor corresponds to which gridpoints prior to each calculation.
3.6.1 Copson
The parallel computer network upon which the simulations in this thesis were run is called “Cop- son”. It is housed in the Mathematics and Statistics Institute at St Andrews University and was
Figure 3.14: Cartesian topologies for grids with closed and periodic horizontal boundaries. The processors are indicated by the numbered red circles and the lines of communication by the black and blue lines between the processors. The example shown here consists of four processors and is rather a special example since all of the processors contain cells that correspond to the edge of the grid. For the closed boundaries grid, each processor on the edge of the grid only communicates information with two other processors. However, for a grid with periodic boundaries in both hori- zontal directions, each processor on the edge of the grid communicates with four other processors. In an example with a larger number of processors, there would be processors corresponding solely to the inner parts of the domain and, independent of the type of boundary, these would always communicate with four other processors.
installed in January 2003. The network contains 86 nodes with each node comprising of 2 proces- sors. Each processor has 2 GB of RAM and, thus, the network as a whole has 172 GB.
The network is named after Professor Edward Copson (1908 – 1980). His links with the University of St Andrews began in 1930 when he was given lectureship in the Mathematics department. He left in 1934 but returned again in 1950 when he was appointed as professor to the Regius Chair of Mathematics. Copson’s postings at St Andrews continued to increase in prestige, from Dean of the Faculty of Science between 1950 and 1953 to first Master of the United College in 1954. After his official retirement in 1969, Copson continued to teach, research and write books. The network was bought on behalf of the Mathematics department and the University of St An- drews Solar Group. Funding came from the Scientific Research Investment Fund (SRIF) and the Particle Physics and Astronomy Research Council (PPARC).
Copson is available for use by those currently involved in research who are members either of the School of Mathematics and Statistics at the University of St Andrews or the UK MHD consortium.