XtratuM hypervisor redesign for LEON4 multicore processor

XtratuM hypervisor redesign for

LEON4 multicore processor

E.Carrascosa, M.Masmano, P.Balbastre and A.Crespo Universidad Politécnica de Valencia, Spain

Outline

European aero-space sector

● Quite conservative sector

● However, systems have more and more powerful

processors and larger memories

– The same workload that required hundred of processors

now just requires a few ones

● So, why do not use less hardware to run the same

software

– Isolation between execution components must be

guaranteed

– Interest in adopting a TSP based-architecture principle

Time & Space Partitioning (TSP) (I)

TSP based-architecture

● Defines a set of isolation execution environments ● Guarantees temporal and spatial isolation of the

components of the system

● Enables to concentrate modules with low criticality

level jointly with high criticality ones in the same computer

Time & Space Partitioning (TSP) (II)

● Software (several technological solutions)

– Hypervisor

– Partitioning kernel – Micro-kernel

● Standard

– ARINC-653 (avionics) defines the interface for building

XtratuM hypervisor (I)

● Bare metal, open source hypervisor designed to

meet safety critical real-time requirements

● Uses paravirtualization techniques

● Supported architectures: x86, LEON2/3/4

(SPARCv8), ARM Cortex R4f

● Strong temporal isolation: fixed cyclic scheduler ● Strong spatial isolation: partitions are executed in

XtratuM hypervisor (II)

● Robust communication mechanisms (ARINC

sampling/queuing ports)

● Robust error management via health monitor ● Devices can be directly managed by partitions.

Shared devices can be organised in a IOServer

● All resources are allocated via Configuration Table

(XML)

Hypercalls example

void XM_sparc_set_psr(xm_u32_t psr); void XM_sparc_flush_cache(void);

void XM_sparc_flush_regwin(void); void XM_rett(void);

xm_s32_t XM_create_queuing_port(char * portName, xm_u32_t maxNoMsgs, xm_u32_t maxMsgSize, xm_u32_t direction);

xm_s32_t XM_hm_status(xm_HmStatus_t *hmStatusPtr);

xm_s32_t XM_trace_event(xm_u32_t bitmask, xmTraceEvent_t *event);

High-level hypercalls

XML configuration file

<SystemDescription version="1.0.0"> [...]

<ProcessorTable>

<Processor id="0" frequency="80Mhz"> <CyclicPlanTable>

<Plan id="0" majorFrame="2000ms">

<Slot id="0" start="0ms" duration="1000ms" partitionId="0"/> </Plan> </CyclicPlanTable> </Processor> </ProcessorTable> [...] <PartitionTable>

<Partition id="0" name="Partition1" flags="system" console="Uart"> <PhysicalMemoryAreas>

<Area start="0x40180000" size="256KB" mappedAt="0x40000000"/> </PhysicalMemoryAreas>

</Partition>

</PartitionTable>

XtratuM Roadmap

CNES: Centre National d'Études Spatiales ESA: European Space Agency

XtratuM-SMP

● Project “System impact of distributed multicore

systems”

– Participants: Astrium-SAT, UPV – Funded by ESA

● Aim:

– Support SMP – Support IOMMU

– Test the NGMP processor

LEON4-NGMP processor

● Fault-tolerant synthesizable VHDL

– Quad-Core 32-bit LEON4 SPARCv8 – MMU

– Two shared FPUs

– 4x4KB instruction/data L1 cache – 256KB L2 cache

– System frequency: 50MHz

SMP model extensions (I)

● Booting:

– CPU0 sets up the system

– CPU1-3 set up their local info – Scheduler synchronisation

● Devices

– Global clock → one clock for the whole system – Local timer per CPU

SMP model extensions (II)

● Inclusion of the VCPU concept

– Number restricted by configuration

– Mimics underlying CPU behaviour: All halted but VCPU0,

which is in charge of starting up the others

– VCPUs are local to partitions

– New hypercalls to manage VCPUs: XM_get_vcpuid(),

XM_reset_vcpu(), XM_halt_vcpu()

– Behaviour of some hypercalls (XM_halt_partition, …)

SMP model extensions (III)

SMP model extensions (IV)

Local cyclic scheduler

● Each CPU has its own plan

● VCPUs are referenced in the plan

<Slot start=”0” duration=”200ms” partitionId=”0” vcpuId=”0”/>

● Same MAF for all the plans

● Synchronisation of the plans at the start of each hyperperiod ● A VCPU cannot be allocated to more than one CPU

CPU1 CPU0 1, 1 0, 0 1, 1 1, 0 0, 1 MAF 1, 1 0, 0 1, 1 1, 0 0, 1 MAF

SMP model extensions (V)

Local fixed priority scheduler

● It can be used instead of the cyclic scheduler

● Each VCPU is configured with a priority, a VCPU can

be preempted by a higher priority one

● Included to experiment with alternative scheduling

policies

SMP model extensions (VI)

IOMMU support

● ESA requirement: Implement IOMMU support

– Provides I/O isolation at bus level (DMA or other master

devices)

● IOMMU tables are statically configured as a part of

the XML configuration file

Evaluation (I)

● Aim: Measure the impact of the hypervisor layer ● Benchmarks: Dhrystone and CoreMark

● Assumptions:

– One partition, one CPU

Evaluation (II)

● Aim: Measure the impact of the hypervisor layer and

the context switch

● Benchmarks: Dhrystone and CoreMark ● Assumptions:

– One partition, one CPU, different slot durations – Overhead due to hypercalls + context switch

Evaluation (III)

● Aim: Measure the impact of the hypervisor layer, the

context switch and several CPUs

● Benchmarks: Dhrystone and CoreMark ● Assumptions:

– Several partitions, several CPUs

– Overhead due to hypercalls + context switch + CPU

Conclusions

● XtratuM has been adapted to support SMP architectures

– Partitions also implement SMP: VCPU – New hypercalls required

● IOMMU is supported: Paritions can use DMA ● Board evaluation shows:

– Little overhead caused by XtratuM internals (CS,

hypercalls)

– Almost parallel access to the L2 content (128-bit bus