6·4·14·1 Milieu
A specialist area for load distribution algorithms is the implementation of distributed operating systems. (The interested reader is referred to the excellent reference by Goscinski: [Goscinski 1991].) Many distributed operating systems also offer pointers for a future thread migration implementation as well as for the current thread placement implementation. Surprisingly, none of the following mention the use of prioritised schedulers.
6·4·14·2 Wisdom
Murray’s doctoral thesis [Murray 1990] presented the design and implementation for a new distributed operating system: Wisdom.
Wisdom utilises process placement where the recipient’s selection was based on a number of evaluated criteria. Murray’s base load metric is the number of processes present on a processing element at an instant but averages over time are also used. When determining whether a particular processing element is suitable to receive a process three thresholds (watermarks) are trialed:
• does the candidate recipient processing element have strictly fewer processes than the source?
• does the candidate recipient processing element have one less process than the source?
• does the candidate recipient processing element have the same number or fewer processes than the source?
In addition, different location policies are tried. Firstly, a round-robin check of each processing element’s eligibility is used. Secondly the usual determination of the optimal processing element is used.
Murray concluded that optimal recipient selection should be used as the location policy, that average load figures should be ignored in favour of instant load figures, and that the transfer of a process should be to a processing element with at least one fewer process than the source processing element. His final load distribution algorithm utilised bidding and was sender-initiated with a migration-limiting policy that restricted the number of movements of a process to one.
Interestingly, Murray mentions the possibility of local maxima occurring through load distribution. This situation has also (independently) been identified in earlier
incarnations of spark percolation [Dermoudy 1994]. Murray dismissed the possibility of occurrence as unlikely; Section 9·4·8 of this thesis discusses the possibility and its
Chapter Six: Related Work
6·4·14·3 Sprite
Sprite is another distributed operating system that supports process migration [Douglis and Ousterhout 1991]. A load-average daemon runs on all processing elements and is responsible for detecting the idleness of the processing element on which it resides. (The definition of “idle” is complex and the interested reader is referred to [Douglis and Ousterhout 1991].)
When the processing element becomes idle the load-average daemon communicates this fact to a central migration server. The processing element has now become a candidate receiver for load distribution and if the central migration server is aware of homeless (or excess) processes then one is sent to the processing element. When a machine moves from the idle state because of a local user, all foreign processes (those that have been migrated to the processing element) are evicted. These evicted processes are moved to an alternative idle processing element.
6·4·14·4 Utopia
Utopia [Zhou, Zheng, Wang, and Delisle 1993] is a system for the distributed execution of programs on large (thousands of processing elements) heterogeneous computing systems. The load placement algorithm is centralised within a cluster of processing elements with the load placement ‘controllers’ of each cluster interacting with other cluster controllers in a decentralised manner. The authors claim that all aspects of the scheduling algorithms are scalable.
6·4·14·5 Amoeba
In 1996, Steketee, Socko, and Kiepuszewski [Steketee, Socko, and Kiepuszewski 1996] presented extensions to the distributed operating system Amoeba to facilitate process migration. In 1999 [Steketee 1999] experiments were undertaken on load distribution algorithms in conjunction with process migration to determine the effectiveness of process migration in comparison with process placement. The workload utilised is artificial; three bidding load distribution schemes are used: random selection for process placement, selection of the lowest loaded processing element for process placement, and random selection for process migration. Although random selection was seen to perform poorly, when coupled with process migration it performed better than process placement to the lowest loaded processing element.
6·4·14·6 Condor
Condor is a software package for executing long-running threads on the processing elements of a network of work stations that would otherwise be idle. Hou and Shin [Hou and Shin 1997] replace the centralised load distribution system with a
decentralised load distribution system that is a neighbourhood based, sender-initiated bidding algorithm, but each processing element contains a preferred list of processing elements. Processing elements on this list should be consulted when a recipient for a process is to be chosen. The first processing element found in the list with load sufficient to receive the process will be the load transfer partner.
The transfer mechanism quasi-migrates processes. A shadow process remains on the originating processing element and the migrated process communicates with the shadow whenever system calls, et cetera are required.
6·4·15 Others
Many other ‘specialist’ load distribution algorithms have been developed. Some examples include: [Krueger and Finkel 1984] which presents the Above-Average algorithm for a system connected via a broadcast network; [Tantawi and Towsley 1985] which is a general, static load distribution algorithm; [Hollander and Silberman 1988] which proposes a mechanism for heterogeneous load migration; [Mirchandaney, Towsley, and Stankovic 1989] which examines the effects of delays upon load sharing and in particular load metrics; [Artsy and Finkel 1989 and 1995] which contains a discussion of the implementation of process migration in the distributed operating system Charlotte; [Lin and Raghavendra 1992] which proposes the LBC policy for centralised load distribution; [Kremien and Kramer 1992] which includes a method for the qualitative and quantitative analysis of load sharing algorithms; [Setia, Squillante, and Tripathi 1993] which discusses load distribution in a time-sharing environment; [Xu 1993] which presents a real-time load distribution algorithm; [Goswami, Devarakonda, and Iyer 1993] which contains a dynamic load sharing algorithm that bases load distribution decisions upon heuristic prediction; [Le Sergent and Berthomieu 1994] which presents a load distribution algorithm that copes with high fluctuations in system load; [Carriero Freeman, Gelernter, and Kaminsky 1995] which contains a load
distribution system that runs on a continually varying sized pool of processing elements; [Pande, Agrawal, and Mauney 1995] which through compile-time analysis determines a
Chapter Six: Related Work schedule for the execution of functional programs; and [Huang and Oudshoorn 1999] which contains a discussion of the advantages of pre-emptive scheduling in the context of load distribution.
Table 6·1: Summary of related machine models
Work
Implementation?
Parallel?
Prioritised Scheduling?
Speculative Evaluation? Annotations?
Distributed Memory?
Load Sharing?
Receiver-Initiated? Thread Placement?
G ✓ ✗ – – – – – – – <ν,G> ✓ ✓ ✗ ✓ ✓ ✗ – – – ALICE ✓ ✓ ✗ ✗ ✗ ✗ – – – ZAPP ✓ ✓ ✗ ✗ ✗ ✓ ✓ ✓ ✓ FSRM ✓ ✓ ✗ ✗ ✓ ✗ – – – TIM ✓ ✗ – – – – – – – MaRS ✓ ✓ ✓ ✓ ✓ – – – – Alfalfa ✓ ✓ ✗ ✗ ✗ ✓ ✓ ✓ ✓ Buckwheat ✓ ✓ ✗ ✗ ✗ ✗ – – – COBWEB ✓ ✓ ✗ ✗ ✗ ✓ ✓ ? ✗ GAML ✓ ✓ ✗ ✗ ✓ ✗ – – – ABC ✓ ✗ – – – – – – – PABC ✓ ✓ ✓ ✓ ✓ ✓ – – – HDG ✓ ✓ ✗ ✗ ✗ ✓ ✓ ✓ ✓ Spineless G ✗ ✓ ✗ ✗ – – – – – Spineless-Tagless G ✓ ✓ ✗ – – – – – – GRIP ✓ ✓ ✗ ✓ ✓ ✗ – – – Partridge v2 ✗ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ν-STG ✗ ✓ ✗ ✓ ✓ ✗ – – – Mattson ✓ ✓ ✓ ✓ ✓ ✗ – – – WYBERT ✓ ✓ ✗ ✗ ✓ ✗ – – – mSTG ✗ ✓ ✓ ✓ ✓ – – – – DREAM ✓ ✓ ✗ ✓ ✓ ✓ – – ✓ GranSim ✗ ✓ ? ✓ ✓ ✗ – – – GUM ✓ ✓ ✗ ✓ ✓ ✓ ✓ ✓ ✓ HERE ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
6·5 Summary
Each piece of work mentioned in this chapter has something to contribute to managing the effective runtime management of parallelism in a functional programming context. For the purposes of summarising these implementations, algorithms, designs, et cetera, Tables 6·1 and 6·2 indicate the components of each in order for a comparison with the implementation described in this thesis.
Table 6·2: Summary of load distribution algorithms
Work
Implemented? Decentralised?
Receiver-Initiated?
Adaptive?
Minimal Impact?
Simple Load Metric?
Load Sharing?
Thread Placement?
Ni, Xu, and Gendreau (1985) ✗ ✓ ✓ ✗ ✓ ✓ ✓ ✓
Hudak and Goldberg (1984) ✓ ✓ ✓ ✗ ✗ ✗ ✓ ✓
Alfalfa ✓ ✓ ✓ ✗ ✗ ✓ ✓ ✓
Lin and Keller (1992) ✓ ✓ ✓ ✓ ✗ ? ✓ ✗
Barak and Shiloh (1985) ✓ ✓ ? ✓ ✗ ✓ ✓ ✗
Krueger (1988) ✗ ✓ ✓ ✗ ✗ ✓ ✓ ✓
ZAPP ✓ ✓ ✓ ? ✗ ? ✓ ?
HDG ✓ ✓ ✓ ? ✗ ✓ ✓ ✓
The PAM ✓ ✓ ✓ ✗ ✗ ✓ ✓ ✗
Partridge v2 ✗ ✓ ✓ ✗ ✗ ✗ ✓ ✗
Suen and Wong (1992) ✗ ✓ ✗ ✗ ✓ ? ✓ ?
Aharoni, et al. (1992) ✓ ✓ ✗ ✗ ✗ ✗ ✓ ✓ Hammond, et al. (1994) ✓ ✓ ✓ ✗ ✗ ✓ ✓ ✓ Wisdom ✓ ✓ ✗ ✗ ✗ ✓ ✓ ✓ Sprite ✓ ✗ ✗ ✗ ✗ ✓ ✓ ✗ Utopia ✓ ✗ ✗ ✗ ✗ ✗ ✓ ✓ Amoeba ✗ ✓ ✓ ✗ ✗ ✓ ✓ ✗ Condor ✓ ✓ ✗ ✗ ✓ ✓ ✓ ✗ GUM ✓ ✓ ✓ ✗ ✗ ✓ ✓ ✓ HERE ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Chapter Seven: Effective Management of Speculative Evaluation