Integrated Scheduling Contexts

For implementation we choose the less complex variant of embedding scheduling contexts in another kernel object. This approach is less flexible, however, sufficient to implement the mechanism of scheduling contexts. Generally, the number of scheduling contexts used by a thread is dynamic and thus not known before starting the thread. The number might change during runtime so that more scheduling contexts need to be allocated, or unused ones can be freed. Although a scheduling context is not a first-class object, it is created by the kernel and this creation requires kernel memory. The allocation of that memory must be

bound to a task-specific quota, so that the creator of scheduling contexts can only allocate so many scheduling contexts that the limit is not exceeded.

Looking at the internals of the kernel, several kernel objects also store a reference to the quota of the object memory which they have been created from, for the purpose of returning the memory to the quota on their destruction. The availability of the quota is of benefit here as we also need the quota for allocating scheduling contexts.

As scheduling contexts are used with threads and the L4::Thread object also stores a reference to its quota object, it is natural to add allocation (sc_add) and release (sc_del) functionality for scheduling contexts to the thread object. For identifying a scheduling context an ID must be provided. The ID is an integer type, must be uniquely chosen by the creator of scheduling contexts and its scope is local to the thread.

After being allocated, the scheduling context is not yet usable because no scheduling parameters have been set yet. Configuration of a scheduling context is accomplished through an L4::Scheduler as the scheduler is the entity that manages the resource CPU. For that task, the scheduler provides a sc_cfg function, that takes an L4::Thread, the corresponding scheduling context ID and the scheduling parameters that shall be set for that scheduling context.

After configured, the scheduling context can be used. It can be activated explicitly via the thread’s sc_set function, which puts the given scheduling context in use, or it is used when an interrupt triggers. The L4::Irq object is extended with an attach_sc function that behaves similar to the L4::Irq::attach function but also includes the ID of the scheduling context.

A scheduling context can only be destroyed when the scheduling context is not in use, not bound to any interrupt and when it has been unregistered at the corresponding scheduler.

For that purpose the scheduler offers the sc_release function. Besides unregistering the scheduling context at the scheduler this operation also gives the scheduler the possibility to recalculate its CPU allocations, for example, for load balancing.

After being unregistered, the scheduling context can be freed with the L4::Thread::sc_del function. The kernel memory used for the scheduling context is freed and returned to the quota.

The following listings 4.1, 4.2 and 4.3 give an overview on the additional interfaces required for implementing scheduling contexts. Each class is derived from the original class to which the functionality is added. Due to the object-oriented design of Fiasco.OC and L4Re such enhancements can be added without modifying original files or functionality.

1 namespace L4 2 {

3 class Thread_SC : public Thread

4 {

5 public:

6 l4_msgtag_t sc_set(unsigned id, l4_utcb_t *utcb = l4_utcb());

7 l4_msgtag_t sc_add(unsigned id, l4_utcb_t *utcb = l4_utcb());

8 l4_msgtag_t sc_del(unsigned id, l4_utcb_t *utcb = l4_utcb());

9 };

10 }

Listing 4.1: Enhanced interface for L4::Thread

1 namespace L4 2 {

3 class Scheduler_SC : public Scheduler

4 {

5 public:

6 l4_msgtag_t sc_cfg(L4::Cap<L4::Thread> thread,

7 unsigned sc_id,

8 l4_sched_param_t *sp,

9 l4_utcb_t *utcb = l4_utcb());

10 l4_msgtag_t sc_release(L4::Cap<L4::Thread> thread,

11 unsigned sc_id, l4_utcb_t *utcb = l4_utcb());

12 };

13 }

Listing 4.2: Enhanced interface for L4::Scheduler

1 namespace L4 2 {

3 class Irq_SC : Irq

4 {

5 public:

6 l4_msgtag_t attach_sc(l4_umword_t label,

7 L4::Cap<L4::Thread> const thread,

8 unsigned sc_id,

9 l4_utcb_t *utcb = l4_utcb());

10 };

11 }

Listing 4.3: Enhanced interface for L4::Irq

Usage Example

The following shall give a brief overview on a typical usage of threads or vCPUs with multiple scheduling contexts. The scenario consists of multiple virtual guest systems that have been stand-alone systems and that have been consolidated to run on a single host system. The operating system is para-virtualized, that is the operating system kernel is running as a native application on the host system and allows it to issue system calls as any other host application. The task of this operating system with its applications is to control a sorting machine that separates items that do not meet a particular weight. The items come by on a conveyor belt where a photoelectric beam triggers an interrupt when an item is about to reach the scale. The control program will be woken up by the interrupt of the beam and query the scale device for the weight of the unit on the belt. Depending on the measured weight the control program will trigger a pushing device that will redirect the item to a bucket and move back to its original position. Besides the control program, this system also offers a maintenance facility where operators can monitor the system status and query statistics information on the weights of the items and thus of the failure rate of the previous production steps.

The described scenario requires real-time execution of the control program because the belt is running at a constant speed and requires timely reaction of the pushing device to sort out any non-confirming items. The speed of the belt cannot be influenced without also influencing other task handling items on the belt, possibly the whole factory. Missing a timely reaction of the pushing device will let through non-confirming items for further processing, which can possibly hinder or even stop the whole production process. On the other hand, the maintenance and statistics tasks of the system do not require real-time execution and can be run whenever the system has otherwise unused processing capacity available.

The host system is required to setup the guest operating system and one part of this setup phase is to create an additional scheduling context for the single vCPU the guest is using.

The first task is to create the scheduling context. As seen in Listing 4.1 of the extended L4::Thread interface, we require the thread object, thus adding the scheduling context is done after creating the vCPU for the guest. The variable vcpu is of type L4::Thread_SC and stores the capability to the vCPU to be used. To identify the scheduling context we choose the ID 1:

1 L4::Cap<L4::Thread_SC> vcpu;

2

3 // Create / retrieve vCPU capability and store in vcpu variable 4

5 // Add scheduling context and check for errors 6 L4Re::chksys(vcpu->sc_add(1));

Listing 4.4: Creating a scheduling context

After successful execution of the sc_add() call the vCPU has one additional scheduling context available. Note that this call is only successful when the resource quota associated with the vCPU has enough resources available to create the scheduling context.

At this point the scheduling context is not yet usable because it has not been configured with proper scheduling parameters. Setting up scheduling parameters is done using the scheduling interface provided by the program environment. Prior analysis of the control program on the target platform concluded that a 10% share of the CPU with a period of 20ms is sufficient to run it. Therefore we choose the scheduling parameters of the scheduling context as seen the following listing 4.5. For the scenario the priority specified must be the highest for the guest system, that is the highest priority as seen by the local view of the guest. The scheduler proxies are then responsible for setting an appropriate host priority, considering a global view.

1 L4::Cap<L4::Scheduler> sched = L4Re::Env::env()->scheduler();

2 l4_sched_param_sc_t sched_params;

3

4 sched_params.prio = 5;

5 sched_params.budget = 2000;

6 sched_params.period = 20000;

7 L4Re::chksys(sched->sc_cfg(vcpu, 1, &sched_params));

Listing 4.5: Configuration of scheduling parameters of the scheduling context.

As a final configuration step, we need to bind the scheduling context to the device interrupt

of the photoelectric beam. For that we are required to supply the ID of the scheduling when attaching the IRQ to the vCPU.

1 L4Re::chksys(irq->attach_sc(label, vcpu, 1));

Listing 4.6: Binding the scheduling context to a device interrupt.

When all those described steps have been completed successfully the scheduling context can be used. Using it requires a single call on the vCPU to switch between the two available scheduling contexts:

1 // Switch to high priority scheduling context 2 L4Re::chksys(vcpu->sc_set(1));

3

4 // Switch to best-effort (default) scheduling context 5 L4Re::chksys(vcpu->sc_set(0));

Listing 4.7: Selecting a scheduling context as the active one.

The sc_set function must be invoked whenever the guest operating system kernel switches to the control program or away from it, so that the host priority of the virtual machine can be adapted accordingly.