How to Combine these Features

The architectural features extracted from MIMD and SIMD arrays that it is intended to combine are:

•MIMD Control

•Single Level Communications Strategy •SIMD Control

•SIMD Communication

The next stage in the development of this architecture was to find a means to combine these features into a single device. It was envisaged that the bulk of the processing, and communications, would be carried out under SIMD control, with MIMD control being associated with the scheduling of virtual neurons onto each processor.

5.3.1 Combining MIMD and SIMD Control

The combination of both MIMD and SIMD control strategies presents several problems:

•Where to store the two different control strategies •How to select between the control strategies

•How to represent each virtual neuron

•How to implement the communications scheme •How to synchronise between processors

•How to do all the above with the minimum of complexity

The processes involved in solving these problems, and the options that were available, are discussed in the sections below. The solutions

Kicnara raimer rna inesis

eventually used are then presented as an overall architecture, bringing all the ideas presented together.

5.3.1.1 Storage of MIMD and SIMD Control Strategies

The solution proposed for the storage of MIMD and SIMD control strategies involves the use of two different program stores. This is necessary since the hardware requirements for each of these memory blocks are different.

The SIMD program has to be broadcast to the array, adding extra complexity to the SIMD program store. This is due to the additional sequencing and synchronisation that needs to be performed during the broadcasting of the SIMD program.

The requirements for the MIMD program store are simpler since each processor controls its own program flow, allowing conventional memory to be used. It was decided to hold the MIMD program local to each processor, an alternative method would have involved a central MIMD store with bus arbitration and memory protection schemes. However the use of this central store was rejected due to bus bandwidth problems.

5.3.1.2 Control Switching and Virtual Neuron Definition

Once the storage requirements of the two control strategies were defined, it was possible to consider how to control the switching

between the control types, and how to define each virtual neuron. These

two problems are related, and are considered together.

The representation of each virtual neuron involves loading the various

pointers, the definition of the weights, computation of the sigma

function, and the writing of the neuron's output. It was decided to store the weights together with the MIMD instructions, minimising the number of memory blocks, and pointers required.

The technique of combining data and instructions is not a new idea in

neural processors - for example it is seen in the Phonetic

Richard Palmer Phd Thesis

processor uses fast on-chip memory, which allows an immediate addressing mode to be used without any reduction in speed. This requires each weight to be defined as an immediate operand in the multiply instruction:

MPYK weight ^Multiply Immediate

Such a technique enables both data and program instructions to share the same memory without the use of any additional control logic. However it does represent a considerable overhead in the memory requirement; this being due to each weight having to be defined as an opcode followed by a data value.

First Proposed Solution

The method first considered for this architecture used a similar scheme to this immediate operand addressing mode. However in this architecture there is no need to define the multiply opcode for each weight since

this operation is performed under SIMD control. All that would be

required in this case would be some means to distinguish between the following:

Use this byte as a Weights Value Use this Byte as a MIMD Instruction

To distinguish between these two, a single flag bit is required, which would be added to each byte in the MIMD program store. From this flag bit each processor could decide whether to execute a byte as a MIMD instruction, or whether to switch to SIMD control, and use the byte as a Weights Value.

This method is shown in Diagram 5.1(a). Here the flag bit from the memory is used to select which instruction to execute. If the flag bit is high SIMD control is used, and if it is low the byte is used as a MIMD instruction.

This scheme allows the weights and MIMD instruction to co-exist in memory with a fairly low overhead (9 bits required to hold each byte of

Richard Palmer Phd Thesis M I M D S T O R E S I M D 0 I N S T R 0 I N S T R 1 W E I G H T 1 W E I G H T 1 W E I G H T 0 I NS T R 0 I N S T R F l a g Bi t M I M D S T OR E S I M D I NSTR L D - C O U N T W E I G H T W E I G H I W E I G H T I NSTR I NSTR S t a t u s f r o m D a t a p a t h M I M D S T O R E S I M D I N S T R 1 START W G H T S _I M W E I G H T W E I G H 1 ---- U W E I G H T 0 E N D W G H T S Q X 1 NST R |— C S I M D O F F S I M D ON

a) E lag Bi t S el ec t i o n b) Wei ght s Bl ock Counter c) Actual Sol ut i on U6ed