Scheduling and Load Balancing in
Parallel and Distributed Systems
Behrooz A. Shirazi
Ali R. Hurson
Krishna M.Kayi
IEEE Computer Society Press
Los Alamitos, California
Washington • Brussels • Tokyo
Contents
Preface ix Acknowledgments xi CHAPTER 1: INTRODUCTION TO SCHEDULING AND LOAD BALANCING 1 A Taxonomy of Scheduling in General-Purpose Distributed Computing Systems 7
T.L. Casavant and J.G. Kuhl
(IEEE Transactions on Software Engineering, February 1988, pages 141-154)
Multiprocessor Scheduling with the Aid of Network Flow Algorithms 21
H.S. Stone
(IEEE Transactions on Software Engineering, January 1977, pages 85-93)
Load Distributing for Locally Distributed Systems 30
N.G. Shivaratri, P. Krueger, and M. Singhal
(Computer, December 1992, pages 33^44) -^
CHAPTER 2: STATIC SCHEDULING 43 Analysis and Evaluation of Heuristic Methods for Static Task Scheduling 50
B. Shirazi, M. Wang, and G. Pathak
(Journal of Parallel and Distributed Computing, Volume 10, 1990, pages 222-232)
Compile-time Partitioning and Scheduling of Parallel Programs 61
V. Sarkar and J. Hennessy
(Proceedings of the SIGPLAN '88 Symposium on Compiler Construction, 1986, pages 17-26)
Guided Self-Scheduling: A Practical Scheduling Scheme for Parallel Supercomputers 71
CD. Polychronopoulos and D.J. Kuck
(IEEE Transactions on Computers, December 1987, pages 1425-1439)
Multiprocessor Scheduling with Communication Delays 86
B. Veltman, B.J. Lageweg, and J.K. Lenstra
(Parallel Computing, Volume 16, 1990, pages 173-182)
Heuristic Algorithms for Task Assignment in Distributed Systems 96 V.M. Lo
(IEEE Transactions on Computers, November 1988, pages 1384-1397)
Allocating Modules to Processors in a Distributed System 110
D. Fernandez-Baca
(IEEE Transactions on Software Engineering, November 1989, pages 1427-1436)
Compile-Time Scheduling and Assignment of Data-Flow Program Graphs
with Data-Dependent Iteration 120 S. Ha and E.A. Lee
(IEEE Transactions on Computers, November 1991, pages 1225-1238)
A Program Allocation Scheme for Data Flow Computers 134 A.R. Hurson et al.
CHAPTER 3: TASK GRANULARITY AND PARTITIONING 143
On the Granularity and Clustering of Directed Acyclic Task Graphs 148
A. Gerasoulis and T. Yang
(IEEE Transactions on Parallel and Distributed Systems, June, 1993, pages 686-701)
The Effects of Problem Partitioning, Allocation, and Granularity on the Performance of
Multiple-Processor Systems 164
Z. Cvetanovic
(IEEE Transactions on Computers, April 1987, pages 421^432)
Grain Size Determination for Parallel Processing 176
B. Kruatrachue and T. Lewis
(IEEE Software, January 1988, pages 23-32)
Lazy Task Creation: A Technique for Increasing the Granularity of Parallel Programs 186
E. Mohr, D.A. Kranz, and R. H. Halstead, Jr.
(IEEE Transactions on Parallel and Distributed Systems, July 1991, pages 264-280)
CHAPTER 4: SCHEDULING TOOLS 203
PARSA: A Parallel Program Software Development Tool 207
B. Shirazi et al.
(Proceedings 1994 Symposium on Assessment of Quality Software Development Tools,
1994, pages 9£-l 11)
The TOPSYS Architecture 223
T. Bemmerl
(Proceedings ofCONPAR '90 VAPPIV: LNCS, Volume 457, 1990)
Parallax: A Tool for Parallel Program Scheduling 235
T. Lewis and H. El-Rewini
(IEEE Parallel & Distributed Technology, May 1993, pages 62-72)
Parafrase-2: An Environment for Parallelizing, Partitioning, Synchronizing, and Scheduling
Programs on Multiprocessors 246
C D . Polychronopoulos et al.
(Proceedings 1989 International Conference on Parallel Processing,
Volume II, 1989, pages II-39-II-48.)
Mapping Function-Parallel Programs with the Prep.-P Automatic Mapping Preprocessed 256
F. Berman and B. Stramm
(Technical Report Number CS 94-397, December 1994, pages 1-28)
OREGAMI: Tools for Mapping Parallel Computations to Parallel Architectures 284 V.M. Lo et al.
(International Journal of Parallel Programming, 1991, pages 237-270)
CHAPTER 5: LOAD BALANCING 309
Load Sharing in Distributed Systems 314
Y.-T. Wang and R.J.T. Morris
(IEEE Transactions on Computers, March 1985, pages 204—217)
The Probability of Load Balancing Success in a Homogeneous Network 328
CG. Rommel
(IEEE Transactions on Software Engineering, September 1991, pages 922-933
Adaptive Load Sharing in Homogeneous Distributed Systems 340
D.L. Eager, E.D. Lazowska, and J. Zahorjan
(IEEE Transactions on Software Engineering, May 1986, pages 662-675)
Imbedding Gradient Estimators in Load Balancing Algorithms 354
S. Pulidas, D. Towsley, and J.A. Stankovic.
(Proceedings IEEE 8th International Conference on Distributed Computing Systems,
1988, pages 482-490)
The Diverse Objectives of Distributed Scheduling Policies 363
P. Krueger and M. Livny
(Proceedings IEEE 7th International Conference on Distributed Computing Systems, 1987, pages 242-249)
Effects of Response and Stability on Scheduling in Distributed Computing Systems 371 T.L. Casavant and J.G. Kuhl
(IEEE Transactions on Software Engineering, November 1988, pages 1578-1588)
Condor-A Hunter of Idle Workstations 382
M. J. Litzkow, M. Livny, and M.W. Mutka
(Proceedings IEEE 8th International Conference on Distributed Computing Systems,
1988, pages 104-111)
GAMMON: A Load Balancing Strategy for Local Computer Systems with
Multiaccess Networks 390
K.M. Baumgartner and B.W. Wah
(IEEE Transactions on Computers, August 1989, pages 1098-1109)
The Stealth Distributed Scheduler 402
P. Krueger and R. Chawla
(Proceedings IEEE 11th International Conference on Distributed Computing Systems, 1991, pages 336-343)
CHAPTER 6: M E C H A N I S M S FOR PROCESS MIGRATION 411 Design Issues of Process Migration Facilities in Distributed Systems 414
M.R. Eskicioglu
(IEEE Technical Committee on Operating Systems Newsletter, Winter 1989, pages 3-13)
Heterogeneous Process Migration by Recompilation 425
M.M. Theimer and B. Hayes
(Proceedings IEEE 11th International Conference on Distributed Computing Systems, 1991, pages 18-25)
Attacking the Process Migration Bottleneck 433
E.R. Zayas
(Proceedings of the 1 lth ACM Symposium on Operating Systems Principles, 1987, pages 13-24)
Designing a Process Migration Facility 445
Y. Artsy and R. Finkel
(Computer, September 1989, pages 47-56)
Process Migration in DEMOS/MP 455
M.L. Powell and B.P. Miller
(Proceedings of the 9th ACM Symposium on Operating Systems Principles, 1983, pages 110-119)
Preemptable Remote Execution Facilities for the V-System 465
M.M. Theimer, K.A. Lantz, and D.R. Cheriton
(Proceedings of the 10th ACM Symposium on Operating Systems Principles,
1985, pages 2-12)
CHAPTER 7: LOAD INDICES 477
Predictability of Process Resource Usage: A Measurement-Based Study on UNIX 479
M.V. Devarakonda an R.K. Iyer
(IEEE Transactions on Software Engineering, December 1989, pages 1579-1586)
An Empirical Investigation of Load Indices for Load Balancing Applications 487
D. Ferrari and S. Zhou
(Proceedings Performance '87, The 12th Annual International Symposium on Computer Performance Modeling, Measurement and Evaluation, 1987, pages 515-528)
The Influence .of Different Workload Descriptions on a Heuristic Load Balancing Scheme 497
T. Kunz
(IEEE Transactions on Software Engineering, July 1991, pages 725-730)
About the Authors 503
