When can we be sure that the computation has finished?
Static load balancing
Typically termination is easy to detect has layout of application is fixed and well controlled.
Centralized dynamic load balancing
Also easy for master to recognize termination: task queue is empty
every slave is idle, having requested another task, and without generating any new task
Alternatively, if a slave indicates that a solution has been found, master can terminate all slaves.
Decentralized dynamic load balancing Not so trivial...
Termination Detection
When can we be sure that the computation has finished?
Static load balancing
Typically termination is easy to detect has layout of application is fixed and well controlled.
Centralized dynamic load balancing
Also easy for master to recognize termination: task queue is empty
every slave is idle, having requested another task, and without generating any new task
Alternatively, if a slave indicates that a solution has been found, master can terminate all slaves.
Decentralized dynamic load balancing Not so trivial...
Termination Detection
When can we be sure that the computation has finished?
Static load balancing
Typically termination is easy to detect has layout of application is fixed and well controlled.
Centralized dynamic load balancing
Also easy for master to recognize termination: task queue is empty
every slave is idle, having requested another task, and without generating any new task
Alternatively, if a slave indicates that a solution has been found, master can terminate all slaves.
Decentralized dynamic load balancing Not so trivial...
Termination Detection
When can we be sure that the computation has finished?
Static load balancing
Typically termination is easy to detect has layout of application is fixed and well controlled.
Centralized dynamic load balancing
Also easy for master to recognize termination: task queue is empty
every slave is idle, having requested another task, and without generating any new task
Alternatively, if a slave indicates that a solution has been found, master can terminate all slaves.
Decentralized dynamic load balancing Not so trivial...
Termination Detection
When can we be sure that the computation has finished?
Static load balancing
Typically termination is easy to detect has layout of application is fixed and well controlled.
Centralized dynamic load balancing
Also easy for master to recognize termination:
task queue is empty
every slave is idle, having requested another task, and without generating any new task
Alternatively, if a slave indicates that a solution has been found, master can terminate all slaves.
Decentralized dynamic load balancing Not so trivial...
Termination Detection
When can we be sure that the computation has finished?
Static load balancing
Typically termination is easy to detect has layout of application is fixed and well controlled.
Centralized dynamic load balancing
Also easy for master to recognize termination:
task queue is empty
every slave is idle, having requested another task, and without generating any new task
Alternatively, if a slave indicates that a solution has been found, master can terminate all slaves.
Decentralized dynamic load balancing Not so trivial...
Termination Detection
When can we be sure that the computation has finished?
Static load balancing
Typically termination is easy to detect has layout of application is fixed and well controlled.
Centralized dynamic load balancing
Also easy for master to recognize termination:
task queue is empty
every slave is idle, having requested another task, and without generating any new task
Alternatively, if a slave indicates that a solution has been found, master can terminate all slaves.
Decentralized dynamic load balancing
Not so trivial...
Termination Detection
When can we be sure that the computation has finished?
Static load balancing
Typically termination is easy to detect has layout of application is fixed and well controlled.
Centralized dynamic load balancing
Also easy for master to recognize termination:
task queue is empty
every slave is idle, having requested another task, and without generating any new task
Alternatively, if a slave indicates that a solution has been found, master can terminate all slaves.
Termination Detection
In general, termination at time t requires the following conditions:
1 local termination conditions exist for all processes at time t
2 there are no messages in transit at time t
Second condition is what makes this problem difficult.
Wait long enough before assuming program has finished?
Termination Detection
In general, termination at time t requires the following conditions:
1 local termination conditions exist for all processes at time t
2 there are no messages in transit at time t
Second condition is what makes this problem difficult.
Wait long enough before assuming program has finished?
Termination Detection
Using Acknowledgment Messages
each process has two states: active andinactive.
processes start inactive
they become active when they receive first task sender of tasks becomesparentof process
process may receive many other tasks while active (computation itself need not be a tree)
every time a process receives a task sends an acknowledgment, with the exception of the parent
acknowledgment to parent is only sent when process is ready to become inactive
process is ready to become inactive when
local termination conditions exist (all tasks finished) it has transmitted all acknowledgments for tasks it received it has received all acknowledgments for tasks it has sent out When first process becomes idle, computation can terminate.
Termination Detection
Using Acknowledgment Messages
each process has two states: active andinactive.
processes start inactive
they become active when they receive first task sender of tasks becomesparentof process
process may receive many other tasks while active (computation itself need not be a tree)
every time a process receives a task sends an acknowledgment, with the exception of the parent
acknowledgment to parent is only sent when process is ready to become inactive
process is ready to become inactive when
local termination conditions exist (all tasks finished) it has transmitted all acknowledgments for tasks it received it has received all acknowledgments for tasks it has sent out When first process becomes idle, computation can terminate.
Termination Detection
Using Acknowledgment Messages
each process has two states: active andinactive.
processes start inactive
they become active when they receive first task sender of tasks becomesparentof process
process may receive many other tasks while active (computation itself need not be a tree)
every time a process receives a task sends an acknowledgment, with the exception of the parent
acknowledgment to parent is only sent when process is ready to become inactive
process is ready to become inactive when
local termination conditions exist (all tasks finished) it has transmitted all acknowledgments for tasks it received it has received all acknowledgments for tasks it has sent out When first process becomes idle, computation can terminate.
Termination Detection
Using Acknowledgment Messages
each process has two states: active andinactive.
processes start inactive
they become active when they receive first task sender of tasks becomesparentof process
process may receive many other tasks while active (computation itself need not be a tree)
every time a process receives a task sends an acknowledgment, with the exception of the parent
acknowledgment to parent is only sent when process is ready to become inactive
process is ready to become inactive when
local termination conditions exist (all tasks finished) it has transmitted all acknowledgments for tasks it received it has received all acknowledgments for tasks it has sent out When first process becomes idle, computation can terminate.
Termination Detection
Using Acknowledgment Messages
each process has two states: active andinactive.
processes start inactive
they become active when they receive first task sender of tasks becomesparentof process
process may receive many other tasks while active (computation itself need not be a tree)
every time a process receives a task sends an acknowledgment, with the exception of the parent
acknowledgment to parent is only sent when process is ready to become inactive
process is ready to become inactive when
local termination conditions exist (all tasks finished) it has transmitted all acknowledgments for tasks it received it has received all acknowledgments for tasks it has sent out When first process becomes idle, computation can terminate.
Termination Detection
Using Acknowledgment Messages
each process has two states: active andinactive.
processes start inactive
they become active when they receive first task sender of tasks becomesparentof process
process may receive many other tasks while active (computation itself need not be a tree)
every time a process receives a task sends an acknowledgment, with the exception of the parent
acknowledgment to parent is only sent when process is ready to become inactive
process is ready to become inactive when
local termination conditions exist (all tasks finished) it has transmitted all acknowledgments for tasks it received it has received all acknowledgments for tasks it has sent out When first process becomes idle, computation can terminate.
Termination Detection
Using Acknowledgment Messages
each process has two states: active andinactive.
processes start inactive
they become active when they receive first task sender of tasks becomesparentof process
process may receive many other tasks while active (computation itself need not be a tree)
every time a process receives a task sends an acknowledgment, with the exception of the parent
acknowledgment to parent is only sent when process is ready to become inactive
process is ready to become inactive when
local termination conditions exist (all tasks finished) it has transmitted all acknowledgments for tasks it received it has received all acknowledgments for tasks it has sent out When first process becomes idle, computation can terminate.
Termination Detection
Using Acknowledgment Messages
each process has two states: active andinactive.
processes start inactive
they become active when they receive first task sender of tasks becomesparentof process
process may receive many other tasks while active (computation itself need not be a tree)
every time a process receives a task sends an acknowledgment, with the exception of the parent
acknowledgment to parent is only sent when process is ready to become inactive
process is ready to become inactive when
local termination conditions exist (all tasks finished)
it has transmitted all acknowledgments for tasks it received it has received all acknowledgments for tasks it has sent out When first process becomes idle, computation can terminate.
Termination Detection
Using Acknowledgment Messages
each process has two states: active andinactive.
processes start inactive
they become active when they receive first task sender of tasks becomesparentof process
process may receive many other tasks while active (computation itself need not be a tree)
every time a process receives a task sends an acknowledgment, with the exception of the parent
acknowledgment to parent is only sent when process is ready to become inactive
process is ready to become inactive when
local termination conditions exist (all tasks finished) it has transmitted all acknowledgments for tasks it received
it has received all acknowledgments for tasks it has sent out When first process becomes idle, computation can terminate.
Termination Detection
Using Acknowledgment Messages
each process has two states: active andinactive.
processes start inactive
they become active when they receive first task sender of tasks becomesparentof process
process may receive many other tasks while active (computation itself need not be a tree)
every time a process receives a task sends an acknowledgment, with the exception of the parent
acknowledgment to parent is only sent when process is ready to become inactive
process is ready to become inactive when
local termination conditions exist (all tasks finished) it has transmitted all acknowledgments for tasks it received
Termination Detection
Using Acknowledgment Messages
Inactive
Active Process
Parent
First Task Final
Ack
Ack Task
Other processes
Termination Detection
Ring Termination Algorithm
processes become white when they terminate
when P0 becomes white, it sends a white token to P1
the token is passed to the next process in the ring when the process finishes, and:
if the color of the process is black, the token becomes black otherwise, the token keeps the same color
a process Pi becomes black if it sends a message to process Pj and j < i
a black process becomes white when it passes the token when P0 receives a black token, it passes a white token when P0 receives a white token, computation can terminate
Termination Detection
Ring Termination Algorithm
processes become white when they terminate
when P0 becomes white, it sends a white token to P1
the token is passed to the next process in the ring when the process finishes, and:
if the color of the process is black, the token becomes black otherwise, the token keeps the same color
a process Pi becomes black if it sends a message to process Pj and j < i
a black process becomes white when it passes the token when P0 receives a black token, it passes a white token when P0 receives a white token, computation can terminate
Termination Detection
Ring Termination Algorithm
processes become white when they terminate
when P0 becomes white, it sends a white token to P1
the token is passed to the next process in the ring when the process finishes, and:
if the color of the process is black, the token becomes black otherwise, the token keeps the same color
a process Pi becomes black if it sends a message to process Pj and j < i
a black process becomes white when it passes the token when P0 receives a black token, it passes a white token when P0 receives a white token, computation can terminate
Termination Detection
Ring Termination Algorithm
processes become white when they terminate
when P0 becomes white, it sends a white token to P1
the token is passed to the next process in the ring when the process finishes, and:
if the color of the process is black, the token becomes black otherwise, the token keeps the same color
a process Pi becomes black if it sends a message to process Pj and j < i
a black process becomes white when it passes the token when P0 receives a black token, it passes a white token when P0 receives a white token, computation can terminate
Termination Detection
Ring Termination Algorithm
processes become white when they terminate
when P0 becomes white, it sends a white token to P1
the token is passed to the next process in the ring when the process finishes, and:
if the color of the process is black, the token becomes black otherwise, the token keeps the same color
a process Pi becomes black if it sends a message to process Pj and j < i
a black process becomes white when it passes the token
when P0 receives a black token, it passes a white token when P0 receives a white token, computation can terminate
Termination Detection
Ring Termination Algorithm
processes become white when they terminate
when P0 becomes white, it sends a white token to P1
the token is passed to the next process in the ring when the process finishes, and:
if the color of the process is black, the token becomes black otherwise, the token keeps the same color
a process Pi becomes black if it sends a message to process Pj and j < i
a black process becomes white when it passes the token when P0 receives a black token, it passes a white token
when P0 receives a white token, computation can terminate
Termination Detection
Ring Termination Algorithm
processes become white when they terminate
when P0 becomes white, it sends a white token to P1
the token is passed to the next process in the ring when the process finishes, and:
if the color of the process is black, the token becomes black otherwise, the token keeps the same color
a process Pi becomes black if it sends a message to process Pj and j < i
a black process becomes white when it passes the token when P0 receives a black token, it passes a white token
Review
Load Balancing static dynamic
centralized decentralized
Termination Detection