Machine Data Structures

It was clear that each processing element acts as a store of information that altered during query evaluation, and in a similar fashion the controller has to maintain a data store if it were to perform its allocation of busses and load balancing tasks. The basic data structures to represent the parallel machine were therefore defined as:

a) a controller

b) an array of processing elements.

Fig.7.2 shows these data structures and indicates the data relationships between them. It indicates how the separation between the machine structures and the interpreter is bridged by the two global queues, ready_to_run_queue and ready_to_allocate_queue. This will be discussed in detail below.

The controller data structure was subdivided to hold data on:

a) the amount of work each processing element was performing throughout the execution time; this was represented by a count of the number of processes awaiting execution in each processing element at any given time, b) the timings of processes on each processing element; data was held on the finish time of the latest processes to run on an individual processing element, ie it represented the time at which a processing element became available to execute a new process,

c) data on busses in use throughout the execution time.

These internal arrays involved the storage and updating of timings during each run. The decision to represent work loading as a simple count of processes waiting execution was made as a first approximation: the suggested measure of work load involved looking at the "size" of each process waiting for execution (see Chapter 6.5.4).The load balancing methods are discussed in Chapter 9 in light of the results obtained from the simulation.

Each member of the array of processing elements held data on: a) processes awaiting execution in that processing element,

b) processes that had been spawned within the processing element and were awaiting allocation and transfer to remote processing elements,

c) the maximum usage value reached as execution proceeds for the input memory corresponding to each bus,

Machine System

PE No.2 PENoji

PENo.l •a •a •a Controller Timing of Processes Bus Usage Ready_to_ allocate_ queue Ready_to_ run_queue

Bus for results return Results Process to execute

Parallel Rewrite Interpreter Spawned Processes for allocation

Interpreter System

Represents movement of data structures ■f**' Represents information flow

Fig. 7.2 - Machine Data Structures

d) the time that the maximum input memory usage was first reached.

The first two data items, ie the process queues, consisted of a linked list of process structures which were defined in the format shown in Fig.7.3.

Proc_no Bus_no Time Nsxt Proc_desc

1

Fig. 7.3 - Process Represent.

r \

ition

r

Proc_no represents a unique global identifier, Bus_no refers to the bus identifier on which the process was transmitted to the processing element and Proc_desc is a pointer indicating the list which defines the expression to be evaluated (see Chapter 5.4.3). Time means creation (spawn) time in the case of processes awaiting allocation, and time of reception at the designated processing element in the case of processes waiting to execute. In the case of a process that was transmitted as soon as it was spawned (ie no delay due to bus contention) the Time field would be incremented by the message passing time when it is moved from the parent processing element to its designated processing element. Obviously if there was a delay in obtaining a bus this delay would have to be added to the new Time value.

It can be seen from this description that processes awaiting execution now reside within an individual processing element structure rather than on the global queue of the abstract interpreter. In fact, to ease processing, the global queue was maintained and augmented with a similar queue representing processes awaiting allocation. These global queues now held modified structures known as "process records" and "allocation records" respectively; these did not include any of the process description or bus details but merely acted as a central list of where to find the appropriate process and its time value for scheduling purposes. Process records were defined as shown in Fig.7.4.

Procjno PE_no Time N<;xt

Fig. 7.4 - Process Record Representatic r

in

Allocation records were slightly different in format because they each represented a group of processes that had resulted from a spawning operation in the parent process. They were defined in Fig.7.5.

No. ofProcs PE_no Time Ni;xt Pckt_size

Fig. 7.5 - Allocation Record Repri

r

esentation

Instead of using one allocation record per process this combined record showed the number of processes spawned in the first field. This data plus the processing element identifier and the time of spawning allowed the group of processes being held on the allocation queue within the processing element to be identified. The fifth field held data on the size of the data packet which represented the group of processes and would be transmitted to the receiving processing elements (see Chapter 7.4.3.3).

The two global queues holding them were designated "ready_to_run_queue" and "ready_to_allocate_queue". Unlike the data structures representing the controller, the processing elements and processes, these global queues would have no direct counterpart in the real system.