Chapter 3 A Delay Composition Theorem for Real-Time Pipelines
3.3 Delay Composition for Pipelined Systems
We first prove the preemptive version of the delay composition theorem in Section 3.3.1. For the sake of brevity, we only show how this proof can be extended to the non-preemptive version in Section 3.3.2.
3.3.1
Proof for the Preemptive Case
The delay composition theorem can be proved by induction on task priority. We first prove the theorem for a two-job scenario (Lemma 1). We then prove the induction step, where we assume that the delay composition theorem is true for k − 1 jobs, k ≥ 3, add a kthjob with highest priority, and prove that the
delay composition theorem still holds.
Lemma 1. When J1 and J2 are the only two jobs in the system, and J2 has a higher priority than J1, the
delay experienced by J1 is at most
Q = 2 X i=1 Ceqi+ N −1 X j=1
maxi=1,2(Ci,j), (3.3)
where:
Ceqi = Ci,max1+ Ci,max2, if A1< Ai
= Ci,max1, if A1≥ Ai
Proof. We shall prove the lemma by considering two cases; J2 arrived before (or together with) J1, and J2
arrived after J1(special cases where one task arrives after the other exits the system can be trivially shown
to satisfy the lemma as well).
Case 1: J2 arrived before or together with J1 to the system
Since J2is the highest-priority job in the system, it executes uninterrupted on all stages, completing each
stage k exactly after a time equal to C2,1+ ... + C2,k. Job J1 executes after J2 on the first stage. When J1
finishes some stage, it moves to the next, where it may encounter J2(again) and must wait for it to finish.
If J2 had already cleared that stage, J1 can execute there immediately. Let stage L be the last stage where
J1 had to wait for J2. In this case, as shown in Figure 3.3-a, J1completes the pipeline with a delay at most
equal to:
Delay(J1) ≤ C2,1+ ... + C2,L+ C1,L+ ... + C1,N (3.4)
Note that, C2,1+ ... + C2,Ltakes us to the completion time of J2 on stage L (where J1 last waited for J2).
C1,L+ ... + C1,N is the additional time taken by J1 to execute on L and the remaining stages. The delay
expression in Inequality (3.4) has N + 1 terms, each representing a per-stage job computation time. There is exactly one per-stage computation in this expression from each stage, except stage L that contributes two.
J2 J1 J2 J1 J1 J2 Stage 1 Stage L Stage N J1 J2 S t a g e L + 1 C +C +...+C Last stage where J1 waits for J2 J1 J2 Stage 2 2,1 2,2 2,LC +C +...+ C1,L 1 , L + 1 1,N
(a) J2 arrives before or together with J1
J1 J2 J2 J1 J1 J2 Stage 1 Stage L Stage N J1 J2 J2 J1 Stage 2 C +C +...+ C1,L 1 , L + 1 1,N (b) J2 arrives after J1 J2 J1 Stage j J1 J2 J1 C +C +...+C1,1 1,2 1,j C +C +...+C2,j 2 , j + 1 2,L
Figure 3.3: Figure showing the delay for the two cases of Lemma 1.
To compute a delay bound, let us replace one per-stage computation time at each of the first N − 1 stages by maxi=1,2(Ci,j) for that stage. Inequality 3.4 can now be re-written as:
Delay(J1) ≤ N −1
X
j=1
maxi=1,2(Ci,j) + C2,L+ C1,N (3.5)
Since the last two terms are at most Ceq2 = C2,max1 and Ceq1 = C1,max1 respectively, the expression in the
lemma is a valid upper bound.
Case 2: J2 arrived after J1 to the system
Let J2 preempt J1 on some stage j. Up to stage j − 1, the delay of J1 on each stage is simply its own
execution time. At stage j, J2 preempts J1 after the latter has executed for some time C1,j∗ < C1,j. As in
the case above, J1 executes after J2 on subsequent stages. Let L be the last stage where J1 waits for J2.
The delay of J1 is thus C1,1 + ... + C1,j∗ + C2,j+ ... + C2,L+ C1,L+ ... + C1,N, as shown in Figure 3.3-b.
Following the same substitution as above, we can show that:
Delay(J1) ≤ N −1 X j=1 max i=1,2(Ci,j) + C2,j+ C2,L+ C1,N (3.6)
Since C2,j+ C2,L≤ Ceq2 and C1,N ≤ Ceq1, the expression in the lemma is a valid upper bound in this case
as well. This completes the proof of the lemma.
We now prove the pipeline delay composition theorem by induction on job priority.
Preemptive Pipeline Delay Composition Theorem. Assuming a preemptive scheduling policy with the same priorities across all stages for each job, the end-to-end delay of a job J1 of lowest priority in an
N-stage pipeline with n − 1 higher priority jobs is at most
Delay(J1) ≤ n X i=1 Ceqi+ N −1 X j=1 n max i=1(Ci,j)
where Ceqi is as defined in Lemma 1.
Proof. Without loss of generality, we assume that a job Jihas a higher priority than a job Jk, if i > k, i, k ≤ n.
That is, Jn has the highest priority, and J1 has the least priority.
The basis step is the case when there are only two jobs in the system, J1and J2. The delay composition
theorem for two jobs is precisely Lemma 1.
Assume that the result is true for n = k − 1 jobs, k ≥ 3. That is,
Delayk−1(J1) ≤ k−1 X i=1 Ceqi+ N −1 X j=1 max i≤k−1(Ci,j) (3.7)
We need to show the result when a kth job J
k, with highest priority, is added. Let Lk be a pipelined
system with k jobs, with arbitrary arrival times for each of the jobs. Let Lk−1 be the system without job
Jk. The outline of the proof is similar to the proof of Lemma 1. We consider two cases, Jk arrived before
(or together with) J1 to the system, and Jk arrived after J1to the system.
Case 1: Jk arrived before or together with J1 to system Lk.
Notice that, if there exists an idle time between the execution of Jk and J1 on some stage j, the delay of
J1 on stage j is independent of the execution time of Jk (and other jobs that execute before the idle time)
on stage j. Therefore, beyond the last stage j, where there is no idle time between the execution of Jk and
J1, Jk will not influence the delay of J1 (jobs that Jk preempts on a stage will also execute before the idle
period on that stage). After Jk completes execution on stage j, the delay of J1 in system Lk is at most its
worst case delay in system Lk−1 starting from stage j. As we make no assumption on the arrival pattern
of higher priority jobs, the delay composition theorem provides the worst case delay for any possible arrival pattern of jobs. Although, adding job Jk does perturb the schedule, the worst case delay due to jobs J2
through Jk−1as per the delay composition theorem accounts for any arrival pattern of J2through Jk−1. We
can therefore apply induction assumption starting from stage j. Hence, the delay of J1 can be expressed as
the delay up to the time Jk completes execution on stage j (Fk,j), added to the worst case delay of J1 in
system Lk−1starting from stage j (as shown in Equation 3.8). This is shown in Figure 3.4.
Delayk(J1) = F1,N− A1,1
= (F1,N − Fk,j) + (Fk,j− A1,1) (3.8)
As Jk arrived before J1 to the system, the duration between the arrival of J1 to the system (A1,1) and
(Fk,j−Ak,1) (Inequality 3.9). Jkis the highest priority job in the system, and does not wait to execute on any
of the stages. The time for Jk to complete execution up to stage j is (Pj−1t=1Ck,t+ Ck,j). In addition to this,
from induction assumption, the delay of J1 from stages j through N isPk−1i=1 Ceqi +
PN −1
t=j maxi≤k−1(Ci,t)
(Inequality 3.10). Thus,
Jk J1
Stage j+1
Jk J_l1 J_l2 J1
Stage j Last stage where there is no idle time between Jk and J1 J_l2 J_l3 J_l3 J_l1 System L_{k-1} starting from stage j Delay of J1 Jk Stage 1 Jk arrives J1 arrives C +C +...+C k,1 k,2 k,j i = 1 k-1 Ce q i + t = j N-1 Ci,t i max( )
Figure 3.4: Figure showing the delay of J1 for the
case when Jk arrived before J1.
Jk J1 Stage j Jk J_l1 J_l2 J1 Stage j+1 Jk preempts J1 J_l2 J_l1 J1 is delayed by at most C(k,j) due to Jk up to stage j
From stage j+1, system similar to case 1; Jk contributes at most
one stage execution time to the job-additive component of J1’s delay
Jk
Stage j-1 J1
J1 Jk
Stage N
Figure 3.5: Figure showing the case when Jkarrived
after J1 and preempts J1 at stage j.
Delayk(J1) ≤ (F1,N− Fk,j) + (Fk,j− A1,1) ≤ (F1,N− Fk,j) + (Fk,j− Ak,1), as Ak,1< A1,1 (3.9) ≤ ( j−1 X t=1 Ck,t) + Ck,j+ k−1 X i=1 Ceqi+ N −1 X t=j max i≤k−1(Ci,t) (3.10) ≤ k X i=1 Ceqi+ N −1 X t=1 max i≤k(Ci,t) (3.11)
which proves the delay composition theorem. Case 2: Jk arrived after J1 to the system.
Until the time Jk preempts J1, the delay of J1 is independent of Jk. Let Jk preempt J1 at stage j.
Beyond stage j, Jk arrives at each stage before J1. Therefore, the pipeline beyond stage j can be thought
of as one having N − j stages, and Jk arriving before J1. We can then apply the result from case 1.
The fact that Jk preempted some job at stage j (it is possible that Jk preempted some job, which in
turn had preempted J1), implies that there was a job executing when Jk arrived at stage j. Further, there
is no idle time between the executions of Jk and J1. Let Jl1, Jl2, . . ., Jls, be the jobs that execute between
Jk and J1 on stage j (Figure 3.5). Jl1 is delayed by Jk up to stage j by at most Ck,j. Similarly, irrespective
of previous stages, each of Jl2, Jl3, . . ., Jls, and J1are delayed by an amount Ck,j due to Jk up to stage j.
Beyond stage j, as mentioned earlier the system is identical to case 1 (as Jk arrived before J1 to stage
j + 1). From the result of case 1, the additional delay that Jkcauses J1is one maximum stage execution time
for j + 1 ≤ t ≤ N − 1 (from Inequality 3.10). Figure 3.5 shows this scenario. We showed that the delay due to Jk up to stage j is at most Ck,j. Therefore, the total job-additive delay to J1 due to Jk is at most the
sum of the two maximum stage execution times of Jk, that is Ck,max1+ Ck,max2= Ceqk.
This proves the induction step. Using this together with Lemma 1, the theorem is proved.
3.3.2
Proof for the Non-Preemptive Case
We first prove the delay composition theorem in the presence of higher priority jobs alone. As this is similar to the preemptive case, for the sake of brevity, we only outline this proof in Lemma 2. We then show how the presence of lower priority jobs can be accounted for, and show that the delay due to resource blocking in only proportional to the number of stages and is independent of the number of lower priority jobs. Lemma 2. Assuming a non-preemptive scheduling policy with the same priorities across all stages for each job, the end-to-end delay of a job J1of lowest priority in a pipeline with n − 1 higher priority jobs is at most
Delay(J1) ≤ n X i=1 Ci,max+ X t≤N −1 n max i=1(Ci,t)
Proof. The proof of this lemma is very similar to the proof of the preemptive case (Section 3.3.1). However, there is one main difference which needs to be carried forth throughout the proof. As motivated in Section 3.1, while two maximum stage execution times of higher priority jobs are considered under preemptive scheduling, a higher priority job can overtake J1at most once, and can hence delay J1 by at most one maximum stage
execution time under non-preemptive scheduling. The stage-additive component is the sum of one maximum stage execution time of any job accrued over all stages.
We now proceed to characterize the delay due to lower priority jobs and prove the theorem in its entirety. Non-preemptive Pipeline Delay Composition Theorem. Assuming a non-preemptive scheduling pol- icy with the same priorities across all stages for each job, the end-to-end delay of a job J1 in an N -stage
pipeline can be composed from the execution parameters of other jobs that delay it (denoted by set S) as follows: Delay(J1) ≤ X Ji∈ ¯S Ci,max+ X j≤N −1 max Ji∈ ¯S (Ci,j) + X j≤N max Ji∈S ¯ (Ci,j) (3.12)
Proof. Under preemptive scheduling lower priority jobs cause no delay to higher priority jobs. However, under non-preemptive scheduling, a higher priority job may block on a resource while a lower priority job is accessing it. In the worst case, a higher priority job may be delayed by at most one lower priority job at every stage in the pipeline. Figure 3.6 illustrates such a scenario.
J1 Ji J1 Stage 1 Stage 2 Stage 3 Ji arrives at stage 1 before J1 Ji
Ji starts executing on stages 2 and 3 just before J1 arrives
Execution of lower priority jobs
J1 Ji
Figure 3.6: Figure illustrating how J1 can be delayed by one lower priority job at each stage of the pipeline.
In the worst case, the lower priority job Ji would start executing at stage j, just before J1arrives at the
stage, causing J1to wait for one complete stage execution time of Ji. Figure 3.6 illustrates such a scenario,
where lower priority jobs delay the execution of Ji until just prior to the arrival of J1 to stages 2 and 3.
Note that, J1 may be delayed by a different lower priority job at each stage of the pipeline. Thus, J1 is
delayed by at most one maximum stage execution time of any lower priority job at each stage which is the third term in the delay bound as per the delay composition theorem. This delay is in addition to J1’s own
computation times on each of the N stages, which is accounted as one maximum stage execution time on each of the first N − 1 stages (the second term in the delay expression), and one maximum stage execution time of J1 (part of the first term).
In the proofs of Lemma 2, we assumed a worst case arrival pattern of higher priority jobs that cause a worst case delay to job J1. This worst case arrival pattern of each higher priority job is independent of other
jobs in the system, and is therefore applicable in the presence of lower priority jobs too. A detailed proof is omitted in the interest of brevity. This completes the proof sketch of the delay composition theorem for the non-preemptive case.