Priority Inversions and Progress Mechanisms

Apriority inversionoccurs whenever lower-priority work is scheduled instead of ready higher-priority work. Some sources of priority inversions are forced upon us by real-world constraints. For instance, device interrupts can be a source of priority inversions—we examine this at length later in Section 2.2.3. Resource sharing can also lead to such inversions.

Example 2.3. Figure 2.6(a) depicts a classic priority-inversion scenario. Here, three jobs are scheduled on a uniprocessor system. A low-priority jobJLis released at time 0. At time 5, jobJLobtains a lock on a shared resource. A time 8, a high-priority jobJHis released, preemptingJL. JobJH requires the resource held by job JLat time 15, soJHblocks and is suspended from the processor; jobJLresumes execution. At time 18, a medium-priority jobJM is released and preempts jobJLbecauseJMhas a higher priority. JobJMcontinues to

Release Complete Block Unblock Resource Held Priority Inversion Elevated Priority J M J L J_H time 0 5 10 15 20 25 30 35 40 45 50

(a)JH experiences an unbounded priority inversion.

J_M J_L J_H J H J_H time 0 5 10 15 20 25 30 35 40 45 50

(b)JLinheritsa priority fromJHat time 15.

J_M J L J_H J_VH time 0 5 10 15 20 25 30 35 40 45 50 MAX MAX

(c) Priority ofJLisboostedto a maximum priority at time 15.

Figure 2.6: Resource sharing can lead to priority inversions. Progress mechanisms, such as priority inheritance (inset (b)) and priority boosting (inset (c)), can shorten priority inversion duration.

execute beyond the depicted schedule. JobJH is ready to be scheduled at time 15, but it cannot since it must wait to obtain the resource held by jobJL. Thus, jobJHsuffers from a priority inversion, starting at time 15.

♦

We say that the priority inversion suffered by jobJHin Figure 2.6(a) isunboundedbecause the length of the inversion depends upon the execution time of jobJM, which may be arbitrarily large. Priority inversion durations must be bounded. We accomplish this by using “progress mechanisms” that expedite the scheduling of lower-priority resource-holding jobs. That is, we intentionally increase the scheduling priority of a resource-holder from abase priority(i.e., its default priority) to a highereffective priority. Naturally, the resultant priority inversion bounds are a function of resource critical section lengths, as opposed to arbitrary processor demand (as was the case with jobJMin Figure 2.6(a)).

We discuss three general methods employed by real-time locking protocols to bound the duration of priority inversions due to resource sharing: priority inheritance, priority boosting, and priority donation.

Priority Inheritance. Underpriority inheritance(Rajkumar, 1991; Shaet al., 1990), the effective priority of a jobJi holding resource`kis set to the maximum ofJi’s base priority and the effective priority of all jobs

blocked (or that may block, depending upon the locking protocol) on`k. That is, jobJiinherits the priority of the highest-priority job that is waiting for`k. JobJi’s effective priority remains elevated until`kis released. Example 2.4. Figure 2.6(b) depicts an example of priority inheritance. As before, jobJH requires the resource held by job JL at time 15, so JH blocks and is suspended from the processor; jobJL resumes execution. However, jobJLinherits the priority of jobJH, soJLis scheduled with an effective priority ofJH. When jobJMis released at time 18, it lacks the sufficient priority to preemptJL. JobJLremains scheduled. JobJLrelinquishes the share resource to jobJH at time 20. With the needed resource obtained, jobJH is

immediately scheduled. ♦

Priority inheritance is often viewed as the temporary transference of the priority from a high priority job to a low priority job. However, it is better to conceive of priority inheritance as a transference, occurring in the opposite direction, ofwork. That is, we may view priority inheritance as the transference of low priority work,i.e., a critical section of a low priority job, to a higher-priority job. Conceptually, we may think of a job as obtaining any requested resourceimmediately—the job is never blocked. However, such a job may be required tonotionallyexecute the critical sections of lower priority jobs on their behalf.6 This view is embodied by a strengthened form of priority inheritance calledbandwidth inheritance(BWI) (Lamastra et al., 2001; Nogueira and Pinho, 2008). Under BWI, a resource holding job that inherits a priority also inherits the execution timebudgetof the job associated with the inherited priority. Thus, the execution time of the critical section is charged against the budget of the blocked high-priority job, not the lower-priority resource holding job. Tasks can be provisioned with enough budget to cover any budget lost due to BWI. However, budgets may be exhausted if critical sections take longer to execute than expected (or provisioned). We may take additional measures to isolate the temporal effects of such a fault. If the budget of a task with an unsatisfied resource request is exhausted, then weabortthe task’s request,refreshthe budget of the task (possibly decreasing the task’s priority), andreissuethe aborted request (Brandenburg, 2012, 2014a).

Priority Boosting. Underpriority boosting(Brandenburg and Anderson, 2013; Lakshmananet al., 2009; Rajkumar, 1990, 1991; Rajkumaret al., 1988), a jobJi’s effective priority is set to the highest scheduling priority when access to a shared resource is contended, or when access is granted, depending upon the locking protocol. JobJi’s effective priority remains elevated until`kis released.

Release Complete Block Unblock Resource Held Priority Inversion Elevated Priority J_H J_L CPU 1 CPU 0 time 0 5 10 15 20 25 30 35 40 45 50 JJ_MM J_M

(a) Priority inheritance ineffective across CPU partitions.

J_H J_L CPU 1 CPU 0 time 0 5 10 15 20 25 30 35 40 45 50 MAX J_M MAX

(b) Priority boosting effective across CPU partitions.

J_H J_L CPU 1 CPU 0 time 0 5 10 15 20 25 30 35 40 45 50 JJ_HH J_M

(c) Priority donation ensures resource-holding jobs are always scheduled.

Figure 2.7: Stronger progress mechanisms may be needed in multiprocessor systems.

Example 2.5. Figure 2.6(c) illustrates an example of priority boosting. This figure depicts the same scenario we have studied before, except that we have added an additional very high priority job, JV H, in order to illustrate an important difference between priority inheritance and priority boosting. As before, job JH requires the resource held by jobJLat time 15, soJHblocks and is suspended from the processor; jobJL resumes execution, but when jobJMis released at time 18, it lacks the sufficient priority to preemptJL—JL remains scheduled. At time 19, the very high priority jobJV His released. However, since jobJLhas a boosted priority,JV H cannot preemptJL, either. JobJLrelinquishes the share resource to jobJHat time 20, and job JV H is immediately scheduled. JobJV H completes at time 25. With the needed resource obtained, jobJHis

scheduled next. ♦

Observe that in Figure 2.6(c)JV H suffers from a priority inversion, even though it does not require the shared resource. This is a drawback to priority boosting: anyjob may suffer a priority inversion due to the priority boosting of another job. This can be detrimental to schedulability.

Priority boosting is implicitly used by most spin-based locking protocols, as these protocols commonly disable interrupts (disabling preemption) while spinning and within a critical section—this behavior is essential to avoiding deadlock in a spin-based locking protocol. Disabling preemption essentially gives the resource holding job a maximum priority. Priority boosting is also used in suspension-based locking

protocols. Particularly, inmultiprocessorlocking protocols where resources are sharedamongpartitions or clusters of processors. In general, priorityinheritanceis an ineffective progress mechanism across partitions and clusters. This because a priority that guarantees that a job is scheduled within its own local partition or cluster (i.e., the priority of a blocked job) maynotbe sufficient to guarantee that the resource holder is scheduled within a remote partition or cluster.

Example 2.6. The ineffectiveness of priority inheritance for resources shared across partitions or clusters is illustrated in Figure 2.7(a). Here, jobJM is partitioned to CPU0, while jobsJH andJLare partitioned to CPU1. JobJLinherits a priority fromJMat time 15. However, since the priority ofJH is greater thanJM,JL is not scheduled. JM experiences an unbounded priority inversion, even though it has sufficient priority to be scheduled on CPU0.

Priority boosting is effective in the same scenario, as illustrated by Figure 2.7(b). Here, the priority inversions that effect jobsJH andJMare bounded by the critical section length ofJL. ♦ Priority Donation. Priority donationis a recently developed progress mechanism for multiprocessor locking protocols (Brandenburg and Anderson, 2013). Priority donation is similar to priority inheritance in that a job may adopt a priority from another job. Priority donation is also similar to priority boosting in that resource holding jobs are always scheduled if they are ready. Adonoris a job that donates its priority to adoneejob. The effective priority of a donee job is set to that of its donor. The donation relationship between donor and donee is established upon job release of the donor.

We define two sets of pending jobs to help describe priority donation. LetJ denote the set of all pending jobs; these may be ready or suspended. Let the subsetJtop-c∈_J denote the set of pending jobs with the top-c priorities (recall thatcdenotes the processor cluster size). Upon the release of a jobJd, ifJd∈Jtop-c, and the arrival ofJdcauses a jobJito be moved fromJtop-cto the setJ\Jtop-c, thenJd maybecome a prioritydonor under the following conditions:

1. IfJiis blocked waiting for a resource or it is a resource holder, thenJd donates its priority toJi;Ji becomes thedoneeofJd.

2. IfJiis a donor to a jobJj, thenJddonates its priority toJj, ending the donor relationship betweenJi andJj, and replacing it with a relationship betweenJd andJj.

The effective priority of a donee job remains elevated until it releases any shared resources, terminating any donor relationship it may have. A priority donor may not be scheduled until its donor relationship is terminated.7

Example 2.7. Figure 2.7(c) illustrates an example of priority donation. Here, jobJLobtains a shared resource at time 5. The higher-priority jobJH is released at time 8. Since jobJLholds a resource, jobJH donates its priority toJL, andJLcontinues to execute. JobJLterminates the donation relationship at time 13, when it

releases the shared resource. JobJHis immediately scheduled. ♦

In general, any job may required to become a priority donor upon its release. As a result, any job can experience a priority inversion due to donation, since donors always suffer from a priority inversion. That is, every donor is inJtop-c(i.e., every donorshouldbe scheduled), and no donor is ever scheduled. Priority donation and priority boosting are similar in that any job can suffer a priority inversion due to shared resources used by other jobs. However, under priority donation, this priority inversion occurs at most once per job. Whereas, under priority boosting, a job may experiencemultiplepriority inversions.

Priority donation is used by the “Clusteredk-exclusionO(m)Locking Protocol” (CK-OMLP), developed by Brandenburg and Anderson (2013). Wardet al. (2012) adapted the ideas behind priority donation for the “Replica-Request Donation Global Locking Protocol” (R2DGLP). However, theR2DGLPlimits donors to the set of jobs that actually share a resource. The protocol also defers the establishment of donation relationships to the moment a shared resource is requested, instead of at job release. Both theCK-OMLPandR2DGLPare a foundational element to GPUSync, so we discuss them at length in Section 2.1.7.