Galatea Programming System

The Galatea Neural Network Programming System is based on the same principles as those outlined in the Pygmalion system. However, it is more sophisticated, and as a design philosophy, a parallel distributed processing approach has been adopted. It encompasses the following modules:

• The User Interface is a sophisticated tool consisting of a set of independent programs. It has been built using the Motif graphics application builder. Motif uses the X Windows graphics protocol with a set of customised library functions. The user interface has three modules; a Graphic Monitor, an Execution Monitor, and a Debugging Monitor. The Graphic Monitor is responsible for opening various display windows for the input and output patterns, system error graphs, bar charts etc. The Execution Monitor is used to initialise, interrupt, save or load the execution. The Debugging Monitor allows line by line tracing of the intermediate level code during execution. The three programs are independent, and they communicate with the system through the central Scheduler.

• The High-Level Language ^ is an Object Oriented language. It is a further development on N, from the Pygmalion programming environment. Again, it is supported by a High-Level Language Library containing the code for the most popular neural models. Users can use the HLL library, or write their applications in

N which is similar to C++, or use the Systems Application Builder (SAB), which allows the development of applications using graphic tools.

The Intermediate-Level Language VML is a vector-matrix based language. Vectors are considered as one-dimensional matrices. The instruction set for VML

contains matrix and scalar arithmetic operations, data transfer and control commands, and file I/O statements.

The M apper is responsible for the efficient partitioning and distribution of the VML

neural network representation. It schedules the operation, and downloads the code to VMs generating appropriate data exchange instructions.

The Scheduler is a run-time process handling user requests and communications between the VMs and the host. It runs like a Unix deamon, passively waiting for requests from users or the modules of the system.

HL Library Virtual Machine Virtual Machine Virtual Machine Compiler Systems Architecture Builder Mapper USER High Level Language

Figure 3.5. Galatea Programming System

The code generation process (Figure 3.5) for the Galatea NNFS is a complex process. Users defining applications in the high-level language V, prompt the N to VML

compiler to generate vectorised VML code. The compilation process also generates a Symbol Table that contains a correspondence between N and VML concepts. Then, the Mapper processes the VML representation, and partitions it to exploit the parallel execution environment. The Mapper generates parallel VML code with appropriate data transfer instructions. This code is downloaded to the VMs, the GPNC is prepared for execution and awaits the start signal from the user. Users can initialise execution using the Execution Monitor, and the Scheduler issues the start signal to the VMs so that the parallel execution starts.

Neural Network Representation and Parallelism in Galatea

The Galatea project has adopted a two-stage representation scheme, involving an object-oriented, high-level language and a matrix-based, intermediate-level language. The high-level language N meets the user requirements of design simplicity, flexibility and modularity. The intermediate-level language VML aims to exploit vector-based general-purpose hardware boards efficiently.

VML consists of a set of scalar and matrix operations with a C-like syntax. It is an interpreted language containing; control statements, I/O operations and arithmetic instructions. Tlie VML interpreter which has been developed at UCL, is used to execute

VML programs on SUN workstations. The interpreter creates an executable image of the code, which can be either executed or used to generate Low-Level VML instructions.

LLVML will be executed on high-performance matrix operator VMs, currently being manufactured at Siemens and Philips.

Two levels of parallelism are possible on the Galatea GPNC. On the higher level, neural network applications can be distributed across a number of VMs. Further parallelism can be exploited within each VM by mapping matrix based operations onto processor array based Execution Units. This thesis work involves the high level partitioning and mapping of matrix based neural network representations onto a number of VMs.

3.4. Summary

This chapter provided a review of the Pygmalion programming environment and the Galatea Neurocomputing project with a special emphasis on the Galatea GPNC and its programming system.

The Pygmalion programming environment has all the features common to general programming environments; an object-oriented, high-level language, an intermediate- level representation, an algorithm library and a set of compilers for a number of target hardwares. Pygmalion principally targeted neuron-based hardware architectures suitable for silicon compilation and similar to the UCL Generic Neuron presented in chapter 2. Pygmalion partly achieved its aims of increasing interest in neural computing in Europe, providing the end-user with a set of algorithms and an experimental tool. However, it has not become a standard for neural network software environments nor has it provided a high performance execution.

The Galatea GPNC marks a change in direction from a neuron-based to a vector- based philosophy. It comprises high performance Virtual Machines that exploit vector operator Execution Units. The vector-based trend is also reflected in the intermediate- level language, VML. Although Pygmalion and Galatea share the same high-level language V, their intermediate-level languages are remarkably different. This is further investigated in chapter 5, in the analysis of neural network representations. The development of the GPNC is still in progress, and the Galatea GPNC is the test domain for the mapping strategies developed in this thesis. The mapping efforts for the Galatea GPNC are presented in chapter 7.

Chapter 4

Analysis of Neural Networks

In this chapter, three neural network models, the Hopfield, the Self- Organising Map and the Backpropagation are analysed in three case studies with appropriate real-world applications. The analysis focuses on the computational requirements and possible structural mappings for these models. A comparison o f the three models is presented, and the potential benefits o f modular, multiple-neural-network architectures are

discussed.

4.1. Introduction

Neural networks models appear in a variety of topologies with a number of training and recall procedures. Chapter 2 presented a survey, which covered a number of models in terms of their main characteristics and popular applications. In this chapter, using three case studies, three neural network models are analysed. In these case studies, the Hopfield nets, the Self-Organising Map and the Backpropagation model are chosen as they together contain properties common to most neural network models. Each case study presents the model, the application and the results, and a computational analysis is followed by structural partitioning and mapping examples. Finally a comparison of the three models is presented, and the strengths of the multiple-network models are discussed.