Category I: Deeply-Embedded Real-Time Operating Systems

The primary concern in deeply embedded RTOSs, which are in some cases deployed in billions of devices, is to minimize resource usage, and in particular their memory footprint. Consequently, such RTOSs offer only a minimum of abstractions. Support for processes and address-space separation (i.e., MMU support) are commonly absent (or deactivated), so that all tasks and the RTOS kernel itself run in a single address space with unrestricted access to all devices (i.e., in kernel mode). The lack of access mediation avoids overheads and thus lowers resource requirements, but prevents effective fault containment. This category corresponds in spirit to the POSIX profiles PSE51 and PSE52.

Not all RTOSs in this category support scheduling, but those that do typically support multi- threading,FPscheduling, and mutex semaphores with optional priority inheritance (i.e., thePIP discussed in Section 2.4.3). Further, since tasks execute in kernel mode, supporting theNCPis trivially possible by disabling interrupts. Multiprocessor support is rare in this category, but does exist, although often only in the form of message passing. That is, each processor is managed inde- pendently as a uniprocessor and communicates only by exchanging messages with remote processors. Scheduling is inherently partitioned in this design, and direct access to global resource is often not supported.

Proprietary RTOSs for deeply embedded systems with advertised multiprocessor support include Quadros Systems’RTXC/mp(message passing only), Express Logic’sThreadX (FP scheduling with only 32 distinct priorities), Mentor Graphics’Nucleus RTOS(advertised as requiring as little as 13 KB of memory, and only 50 KB when multiprocessor support is included), and ENEA’s OSE (message passing only). Besides these proprietary, closed-source systems, there are also open- source RTOSs in this category: the TOPPERS consortium provides several multiprocessor-capable reference implementations of the ITRON standard (Monden, 1987; Takada and Sakamura, 1991,

1995), which is in widespread use among Asian car and consumer electronics manufacturers, and the OAR corporation distributes itsRTEMSsystem under a liberal open-source license.

RTEMS. We consider RTEMS, theReal-Time Executive for Multiprocessor Systems, as the repre- sentative RTOS of this category.25 We chose RTEMS because it is one of the oldest RTOSs to support

multiprocessors that is still in use (RTEMS has been in development and use since 1988) and since it is particularly attractive for academic research due to its open-source nature. The RTEMS version discussed in the following is release 4.10, which is the latest stable release at the time of writing. Notably, RTEMS supports over 15 processor architectures, among them Intel’s x86 architecture and several radiation-hardened processor designs.

RTEMS does not support MMUs; instead, the “kernel” and the user applications, which are just procedures that are executed in separate threads, are compiled into one binary image. In a multiprocessor environment, such an image is required for each processor. This implies that the assignment of tasks occurs already at design time or, at the latest, at compile time. Task migrations are not supported by the OS. However, task migrations could be emulated by the user by compiling a task’s code into the images for multiple processors and then simultaneously “terminating” the task on one processor and starting it on another. However, since RTEMS does not inherently support task migrations, these are technically two (or more) tasks to the OS and the transfer of runtime state from one processor to another must be programmed manually by the developer. Global scheduling is not feasible in such an environment.

Uniprocessor support. All OS resources are statically allocated. This requires the number of tasks and locks to be statically defined at compile time. RTEMS supportsP-FPscheduling with 256 distinct priorities and has explicit runtime support for scheduling (e.g., tasks can declare a period and detect deadline misses), but currently does not supportEDF. For local resources, RTEMS supports two queuing policies (jobs may wait in FIFO or in priority order) and two priority management policies: the well-known PIP and a second protocol that the RTEMS documentation calls the “priority ceiling protocol,” which, however, is not thePCPas defined in Section 2.4.3. Rather, under RTEM’s priority ceiling protocol, a job’s priority is elevated to the priority ceiling immediately

25_{Curiously, the acronym “RTEMS” stood first for “Real-Time Executive for Missile Systems” and then “Real-Time} Executive for Military Systems” in prior versions.

when it acquires the resource instead of relying on priority inheritance as in thePCP. This protocol, sometimes also referred to as the “basic ceiling protocol,” is in fact equivalent to theSRP. By raising the job’s priority to the priority ceiling, it is ensured that newly-released jobs of equal or lower priority are not scheduled until the resource is released. This has exactly the same effect as raising the system ceiling and considering a newly-released job to be ineligible to execute until its priority exceeds the system ceiling—theSRP’s schedulability rule. RTEMS thus supports theSRP, even though not under that name. However, the priority ceiling for each lock must be configured manually, which allows for human error.

RTEMS includes compatibility layers for both the ITRON (Monden, 1987; Takada and Sakamura, 1991, 1995) and POSIX (IEEE, 1993, 2003, 2008b) interface definitions. While not fully compliant, RTEMS closely resembles the POSIX profile PSE51.

Multiprocessor support. As mentioned above, each processor requires an individual binary RTEMS image (e.g., the unique processor index is hard-coded in the image). Further, at runtime, each processor maintains only its own local state. In contrast, in asingle-image multiprocessor OSsuch as UNIX, Linux, or Windows, all processors are initialized from a single binary image, processor indices are assigned dynamically, and all processors share the (mostly) global runtime state. RTEMS thus resembles more a distributed system than a traditional shared-memory OS, which enables it to support both distributed- and shared-memory platforms. Processors communicate with each other by exchanging message packets as over a network link. This packet-based communication is medium-agnostic (e.g., it could be implemented over an actual message bus or via shared memory) and must in fact be provided by the application developer—RTEMS itself does not include drivers to interface with processor interconnects.

RTEMS allows semaphores (and other OS objects) to be marked as being a global object. The identifiers of global objects are broadcasted to all processors so that remote tasks can issue requests. Due to the distributed nature of RTEMS, requests for remote resources follow the RPC model: a request packet is sent to the processor to which the resource is local and the requesting job suspends. On each processor, there isonelocal agent, called themultiprocessing server, that handles requests fromallremote tasks. This local agent executes at the highest priority (i.e., at priority zero). When a processor receives a packet, the local agent is resumed by the (developer-provided) processor-

interconnect driver and acquires the local semaphore on behalf of the remote task (subject to either thePIPor the SRP, depending on the configuration of the semaphore). Once it has locked the semaphore, the local agent sends a reply packet to the remote task and suspends again to wait for the next request. Superficially, this protocol resembles theDPCP, but it does in fact deviate in significant ways.

• The critical section itself is not executed by the local agent, which merely carries out lock and unlock requests.

• There is only one local agent and the priority of the currently requesting remote task is not reflected by the local agent’s priority.

• If there are multiple requests arriving at the same processor, then there is no guarantee in which order they will be processed. This cannot be “fixed” by the developer-provided processor- interconnect driver since the lock-request packet does not relay the priority of the requesting task (theDPCPassumes priority order).

• The remote task isnotpriority boosted while it executes its critical section, nor is it subject to priority inheritance, nor is the (implicit) system ceiling modified.

TheDPCPanalysis does thus not apply and unbounded priority inversion is in fact possible due to the lack of priority boosting (recall Figure 2.37).

Since RTEMS effectively supports theSRP, and because tasks execute in kernel mode and thus can disable interrupts, theMSRPcan be implemented in a straightforward way by the application developer. A close approximation of the DPCP could similarly be implemented on top of the provided primitives. However, neither protocol is part of the standard RTEMS distribution. In fact, to the best of our knowledge, neither theMPCP nor theDPCP (as defined and analyzed in the real-time literature) are supported by any of the RTOSs in this category.