1. Explain how a dataflow machine avoids the “von Neumann bottleneck.”
A dataflow machine, unlike one based on a von Neumann architecture, does not rely on the use of sequential algorithms to guide the processing of data.
Processing is data-driven rather than instruction-driven; it is not inherently sequential, but instead allows the hardware to exploit any parallelism inherent in the task. Since there is no need to fetch “instructions” separately from data, the von Neumann bottleneck described in Chapter 1 does not come into play.
2. Draw a dataflow graph and an activity template for the following programming construct: if (x >= 0) { z = (x + y) * 4; } else { z = (y - x) * 4; }
The dataflow graph will be similar to Figure 7.2 except for the details of the operations. Likewise, the activity template will be similar to Figure 7.3.
3. If you had a scientific application that involved a large number of matrix manipulations, would you rather run it on a dataflow computer or a SIMD computer? Explain.
It would probably be better to run such an application on a SIMD computer, since that type of system is designed to optimize performance on array processing (matrices are handled as two-dimensional arrays). Dataflow computers do
reasonably well with “unstructured” parallelism but are not particularly good at exploiting array-type parallelism.
widely adopted?
There are several legitimate reasons why dataflow computers have not reached the mainstream. One is their reliance on specialized programming languages to express the constructs represented in a dataflow graph or activity template. Machines that do not use standard programming languages tend to have a higher software development cost. Also, dataflow machines do not perform particularly well (at least, not well enough to justify their cost) on many common applications. Unless the parallelism inherent to the task is a good match for the parallelism of the machine hardware, performance gains will be modest at best. Finally, dataflow machines are not easy or cheap to build and do not take much advantage of the locality of reference that is essential to the function of hierarchical memory systems.
5. Give an example of how dataflow techniques have influenced and/or been used in conventional computer design.
Dataflow techniques were a part of the control strategy developed by Robert Tomasulo for the IBM 360/91 computer in the 1960s. While the machine was programmed like a traditional von Neumann computer, internally its hardware execution units were scheduled using a dataflow approach: an operation was sent to a functional unit once its operands were available. Some modern, superscalar microprocessors still use Tomasulo’s method (or variations on it) and thus bear the influence of dataflow computing.
6. Are superthreaded and hyper-threaded processors the same thing? If not, how do they differ?
Superthreaded and hyper-threaded processors are close cousins, but not identical. Superscalar machines that use superthreading (also called time-slice multithreading) can issue multiple instructions belonging to one process (or thread) during a given clock cycle; during a different clock cycle, instructions belonging to another process can be issued. Effectively, use of the CPU is time-multiplexed on a cycle by cycle basis. Hyper-threading (or simultaneous multithreading) takes this concept one step further: during a given clock cycle, instructions from more than one process may be issued in order to make the maximum possible use of CPU resources.
7. Would you classify an artificial neural network as an SISD, SIMD, MISD, or MIMD system, or something else? Make a case to support your choice.
The best answer is probably “something else” – ANNs represent a unique class of architectures with their own special characteristics. Artificial neural networks really do not fit the description of any of the systems described in Flynn’s taxonomy of computer systems. If one had to try to pigeonhole them into one of his four classifications, they could be said to at least resemble, in some ways, MIMD or even MISD machines. (ANNs are clearly not SISD or SIMD architectures because they lack a single instruction stream.)
8. Explain how the processing elements and interconnections in an artificial neural network relate to the structure of the human nervous system.
The many, simple processing elements in an artificial neural network
correspond to the many neurons in the human body. Each processing element, like each real neuron, accepts several inputs and computes the weighted sum of those
inputs. This sum is applied to an activation function that simulates the action potential threshold of a biological neuron. The activation function determines whether a given processing element will send an output to the input of another processing element (simulated neuron).
9. How is a supervised artificial neural network programmed to carry out a particular task? What is the difference between a supervised vs. unsupervised ANN?
An artificial neural network is not so much programmed, but trained iteratively to perform a given task. A supervised ANN receives its training via the user’s repeated applications of inputs, producing outputs that are compared to the corresponding desired outputs; the neuron weights are adjusted after each pass until the network “learns” to produce good output for the full range of inputs. Unsupervised ANNs are used in situation for which feedback is unavailable (no “known good” output data exists). Instead, they use “competitive learning” techniques to learn on their own without intervention by a human trainer.
10. Why are ANNs well suited to applications such as robotic control? Give an example of an application for which you do not think an ANN would be a good choice.
ANNs are a good choice for robotic control because complex motions are difficult to program algorithmically using traditional programming languages. Since it is possible to generate examples of the desired functionality and since most ANNs operate on the principle of “training” a system to produce outputs
corresponding to given examples, they are a natural “fit”. After all, the biological neural networks of human beings and animals can be trained to produce desired output, so why shouldn’t artificial neural networks exhibit similar strengths? On
the other hand, applications such as computational fluid dynamics, etc. that require a great deal of numeric computations (“number crunching”) would probably
perform a lot better on a conventional supercomputer than on an ANN.
11. What is different about logical variables in a fuzzy system as compared to a conventional computer system?
In a conventional computer system, logical variables are binary in nature. That is to say, they are either true or false; on or off; 1 or 0; 100% or 0%. In a fuzzy system, logical values can take on a continuum of truth values between the limits of 0 and 1, inclusive.
12. Both ANNs and fuzzy logic systems attempt to mimic the way human beings make decisions. What is the main difference between the two approaches?
The main difference is that artificial neural networks attempt to mimic the actual structure of the human brain by simulating the functionality of neurons and the connections that exist between them. Fuzzy logic systems attempt to model the uncertain, imprecise methods people use to make decisions based on (often
incomplete) available information, but their structure is not based on any biological model.
13. What is a fuzzy subset and how does the idea of a membership function relate to it? Propose a simple membership function rich() that deals with the concept of a fuzzy subset of wealthy people.
A fuzzy subset is a portion of the universe of discourse (the set of all things under consideration in formulating a given problem), whose membership is not defined precisely. A membership function expresses the perceived likelihood that a
given member of the universe belongs to a particular fuzzy subset. In other words, it produces a truth value (in the range of 0 to 1, inclusive) that indicates an object’s degree of membership in the fuzzy subset. One possible definition of a membership function rich() would be as follows:
rich (x) = {0, if income (x) < $100K;
(income (x) - $100K) / $900K, if $100K ≤ income (x) ≤ $1M; 1, if income (x) > $1M.}
14. Can the Boolean, or crisp, logic operations AND, OR, and NOT be defined in regard to fuzzy logic? If so, explain how; if not, explain why not.
Yes, the Boolean functions AND, OR, and NOT correspond to fuzzy
operations. NOT is generally defined such that truth (not x) is equal to 1.0 – truth (x). Various definitions have been used for the AND and OR functions; the most common are truth (x AND y) = min (truth (x), truth (y)) and truth (x OR y) = max (truth (x), truth (y)). Note that if the variables x and y are restricted to only the discrete values 0 and 1 (as in binary logic systems), these definitions are consistent with Boolean algebra properties.
15. Explain, in the context of a fuzzy expert system, what rules are and how they are used.
Rules are statements that reflect the knowledge of a human expert about how a given system works. They are typically expressed in terms of if-then relationships between fuzzy output subsets and linguistic variables derived from the inputs. The rules are used to make inferences about the system based on the fuzzified input data (that is, the results after the membership functions are applied to the “raw” input data). The outputs of the various rules that make up the system’s rule base are combined and used to create a single fuzzy subset for each output variable. These
outputs can then be defuzzified to produce “crisp” outputs if they are needed.
16. For what type(s) of physical system is fuzzy control particularly well suited?
Fuzzy control is a good choice for controlling systems that are nonlinear, complex, and/or have poorly specified characteristics, thus making them a poor match for conventional analog or digital control systems using algorithms that depend on having a well-defined, linear model of the process to be controlled.
17. What is Moore’s Law and how has it related to advances in computing over the last 40 years? Is Moore’s Law expected to remain true forever or lose its validity in the future? Explain your answer and discuss the implications for the design of future high-
performance computer systems.
Moore’s Law says that the continually shrinking sizes of semiconductor devices will result in an exponential growth (doubling on approximately a yearly basis) in the number of transistors that can feasibly be integrated on a single chip. This has resulted in a doubling of computational power approximately every 18-24 months (since, apparently, computational power is not a linear function of the number of transistors).
Given the known laws of physics, it is not possible that Moore’s Law will continue to hold true indefinitely. The problem is that transistors used as switching elements in computers cannot keep shrinking once they get to the size of individual atoms or small groups of atoms. Devices that small will no longer work under the binary logic principles of Boolean algebra, and the performance of traditional computer architectures will reach a hard limit. (This has been estimated to occur within the next 10-20 years.) At that point, further increases in performance will
only be achievable if some new approach, for example quantum computing, is adopted.
18. How does a quantum computer fundamentally differ from all the other computer architectures discussed in this book? What allows a quantum computer to achieve the effect of a massively parallel computation using a single piece of hardware?
A quantum computer differs from all traditional computer architectures in that it does not operate on the principles of Boolean algebra, where computations are done sequentially (or in parallel, by replicating hardware) on binary digits (bits) or groups of bits. In conventional machines, each bit can only be 0 or 1 at any given time. Quantum computers instead operate on quantum bits (qubits). Qubits can take on not only the distinct values 0 or 1, but also – by the principle of quantum superposition – they can take on states that can be 0 and 1 at the same time. By adding more qubits, a quantum computer becomes exponentially more powerful. While a 16-bit binary register can take on only one of its 65,536 possible states at a time, a 16-qubit quantum register can be in all 65,536 states at once in coherent superposition. This allows the effect of a massively parallel computation to be achieved using only one piece of hardware.
19. What are some of the problems scientists must solve in order to make supercomputers based on the principles of quantum mechanics practical?
Researchers working on quantum computers have encountered a number of problems that have so far made it impractical to construct machines that can
compete with supercomputers based on conventional architectures. First of all, it is difficult to build a quantum computer, since there is a need to separate one or a
small number of atoms from others and keep them in a steady state in order to use them for computation. Another significant problem is decoherence, a phenomenon that can introduce errors in computations due to interactions of the computer hardware with the surrounding environment. Finally, assuming one has performed a quantum computation, it is difficult to observe the result without collapsing the coherent superposition of states and destroying one’s work. Research into solving these problems is ongoing.
20. What application(s) are expected to be a good match for the unique capabilities of quantum computers? Explain.
If large-scale quantum computers can be practically constructed, they will probably be used to solve extremely numerically intensive problems that have proven to be intractable with even the fastest conventional systems. (They won’t be used for word processing, sending e-mail, or surfing the Net!) One area that seems to be a good potential match for the capabilities of quantum computers is
cryptography – the making and breaking of highly secure codes that protect sensitive information from being intercepted by unauthorized parties.
21. Fill in the blanks below with the most appropriate term or concept discussed in this chapter:
Dataflow machine - a type of computer architecture in which execution depends on the
availability of operands and execution units rather than a sequential-instruction program model
Node (actor) - an element in a dataflow graph that represents an operation to be
Tokens - these are used to represent data values (operands and results) in algorithms for a
dataflow architecture
IBM 360/91 - this (outwardly) von Neumann machine made use of dataflow techniques
for internal scheduling of operations
Hyper-threading - a machine using this technique can issue instructions from more than
one thread of execution during the same clock cycle
Artificial Neural Network (ANN) - a type of computer architecture with a structure
based on that of the human nervous system
Neurons - the fundamental units that make up a biological neural network
Dendrites - these are fibers that act as “input devices” for neurons in human beings Convergence - when an artificial neural network achieves this, it is “trained” and ready
to be put into operating mode
Single-Layer Perceptron (SLP) - the earliest and simplest type of artificial neural
network
Unsupervised neural network - a type of artificial neural network that does not require
user intervention for training
Fuzzy logic architecture - a type of computer architecture in which logical values are
not restricted to purely “true” or “false” (1 or 0)
Linguistic variable - a type of variable that expresses a “fuzzy” concept; for example,
“slightly dirty” or “very fast”
Universe of discourse - the set of all objects under consideration in the design of a fuzzy
system
object has in a fuzzy subset
Fuzzification - the first step performed in doing “fuzzy computations” for an expert
system, control system, etc.
Defuzzification - this is necessary if a fuzzy result must be converted to a crisp output Quantum computer - a type of computer architecture in which the same physical
hardware can be used to simultaneously compute many results as though it were parallel hardware; its operation is not based on Boolean algebra, but on the physics of subatomic particles
Moore’s Law - a prophetic observation of the fact that conventional computers would
tend to grow exponentially more powerful over time as integrated circuit features got smaller and smaller
Quantum bit (qubit) - the basic unit of information in a quantum computer
Quantum interference - this phenomenon results from the superposition of multiple
possible quantum states
Quantum entanglement - a state in which an atom’s properties are identically assumed
by another atom, but with opposite spin
Decoherence - the tendency for interactions with the surrounding environment to disturb
the state of qubits, possibly resulting in computational errors
Thirty - a quantum computer with this many qubits has been estimated to have 10
TFLOPS of computational power
Cryptography - so far, this appears to be the most likely application for supercomputers