Science Highlights
Embedded Systems’ Virtualization: The Next
Challenge?
Alexandra Aguiar, Fabiano Hessel
Faculty of Informatics – PUCRS
Traditionally, virtualization has been adopted by enterprise industry aiming to make
better use of general purpose multi-core processors and its use in embedded systems (ES)
seemed to be both a distant and unnecessary reality. However, with the rise of each more
powerful multiprocessed ESs, virtualization brings an opportunity to use simultaneously
several operating systems (OS) besides offering more secure systems and even an easier
way to reuse legacy software. Although ESs have increasingly bigger computational
power, they are still far more restricted than general purpose computers, especially in
terms of area, memory and power consumption. Therefore, is it possible to use
virtualization - a technique that typically demands robust systems - in powerful yet
restricted current embedded systems? In this paper we show why the answer should be
yes.
Laboratory:
GSE
Publication:
To appear in IEEE International Symposium on Rapid System Prototyping, 2010
Funding:
FAPESP, CNPq, CAPES,
Ministério da Ciência e Tencologia
Embedded Systems’ Virtualization: The Next Challenge?
Alexandra Aguiar, Fabiano Hessel
Faculty of Informatics – PUCRS – Av. Ipiranga 6681, Porto Alegre, Brazil
[email protected], [email protected]
Abstract
Traditionally, virtualization has been adopted by enter-prise industry aiming to make better use of general purpose multi-core processors and its use in embedded systems (ES) seemed to be both a distant and unnecessary reality. How-ever, with the rise of each more powerful multiprocessed ESs, virtualization brings an opportunity to use simultane-ously several operating systems (OS) besides offering more secure systems and even an easier way to reuse legacy soft-ware. Although ESs have increasingly bigger computa-tional power, they are still far more restricted than gen-eral purpose computers, especially in terms of area, mem-ory and power consumption. Therefore, is it possible to use virtualization - a technique that typically demands robust systems - in powerful yet restricted current embedded sys-tems? In this paper we show why the answer should be yes.
1. Introduction
Virtualization of computing systems consists on creating a logical group of resources that seems to be the physical resources of a given computing environment [9]. It has been widely adopted in the enterprise market, especially to make better use of multi-core processors, along with several other advantages:
• allows several operating systems to run on a single ma-chine;
• provides isolation from one virtual machine to another, improving security;
• increases system flexibility;
• enhances workload management, and;
• brings hardware independence.
On the other hand, virtualization can be considered as a high consuming technique, since it usually requires a lot
of disk space and RAM use, besides adding a non-trivial management layer: the hypervisor. The hypervisor, also known as Virtual Machine Monitor (VMM), must allow that instructions ran by the virtual machine (or virtual board) can be executed correctly on the real hardware.
In enterprise servers, virtualization allows a single phys-ical server to perform as multiple logphys-ical servers and pro-vides the host of multiple instances of different Operat-ing Systems (OS), like Windows, Linux and others. Of-ten, these systems use multi-core processors from Intel and AMD, which is a very strong trend, as most vendors have presented projects for beyond four cores to the near future.
This trend of using multiprocessed platforms is also present in Embedded Systems (ES), where Multiprocessors System-on-Chip (MPSoC) are being increasingly adopted and have become each more a viable choice to implement embedded systems [5]. Thus, these new multicore devices force a true change in the way embedded developers are de-signing their systems, since there are several key differences in the way that ES developers face the use of multicore pro-cessors and virtualization techniques [7].
While virtualization provides advantages as the capabil-ity of running multiple instances of operating systems on a single processor (mono- or multi-core), embedded systems are far different from enterprise systems [14]. This means that in order to take the advantages offered by virtualiza-tion, much effort must be spent in understanding how to better adapt them to embedded special needs: they are usu-ally constrained in terms of power consumption, amount of memory, timing restrictions and area.
Therefore, the use of previously known solutions opti-mized for enterprise Windows and Linux systems are not well-suited. In spite of it, virtualization techniques must be adapted to fit into embedded systems. But then, a question arises: is it worth?
In this paper, we discuss the most important concepts involving virtualization and exploit the possibilities of using this technique in the embedded systems world. To do so, we detail the most suitable ways of employing virtualization in ESs aiming to achieve certain goals. Also, we present the main expected drawbacks and possible ways of avoiding
them.
The paper continues as it follows. The next section shows the virtualization background to be adopted through-out this paper. Section 3 discusses the advantages of using virtualization in embedded systems followed by Section 4, that shows the drawbacks of it. In Section 5 we discuss how virtualization can be applied in ES and finally, Section 6 concludes the paper.
2. Basic concepts
In this section, we present basic concepts about both vir-tualization and embedded systems. We highlight the con-cepts needed to better understand the remainder of the pa-per.
2.1
Embedded Systems
In the past few years, embedded systems used to be relatively simple devices with severe hardware constraints, like memory use, processing power and battery life. Their functionality was also mostly determined by hardware, with software consisting largely of device drivers, a task sched-uler and some control logic, resulting in a low complexity of the software. Besides, these systems used to be closed, which means that during its lifetime no change in the soft-ware in required [1].
This scenario, however, is under change. Increasingly, modern non-critical embedded systems are bringing char-acteristics typical of general-purpose systems. The main change is their growing functionality which dramatically affects and increases the complexity of their software. It is also very common to run general purpose applications on embedded systems as well as to use applications written by developers that have little or no acknowledge at all about the embedded systems constraints [16].
In this case, more and more high level application ori-ented OSs with common APIs are required. Moreover, de-vice owners also want to load their own applications on their systems by using common and rather open APIs. It is also important to point that the more open and common develop APIs get, the more security it is against attacks that were present, previously, only in general purpose systems.
Nevertheless, some of traditional differences between general purpose and embedded systems still last [6]. Even on high-end multimedia entertainment-driven embedded systems, some real-time constraints remain. The energy consumption is always a matter of concern, since these de-vices are supposed to operate through several hours (up to days) without any battery recharge. This impacts on the processor frequency allowed to be used: usually lower fre-quency rates are mandatory in order to accomplish energy consumption goals. Another common restriction regards
the memory use, since modern embedded devices are de-sired to be cost effective and memory, besides being a high energy consuming device, is frequently a cost factor is-sue [4].
Finally it is possible to say that embedded systems are becoming more and more part of everyday life, be-ing increasbe-ingly used in mission and life-critical scenar-ios [15]. Although these systems are far more restricted that so-called modernembedded systems, virtualization could bring advantages, by increasing their safety, reliability and security.
2.2
Virtualization
As stated before, virtualization allows a single computer to host multiple virtual boards (or virtual machines), each isolated from one another, with the possibility of running different operating systems. The main advantage is that, if a virtual board fails, the other ones are kept safe at a reason-able cost [2].
In enterprise IT, although it causes a single point of fail-ure, since many servers can be placed at a unique hardware machine, it is considered as a safer approach because most of service interrupts are not caused by hardware failures, instead, the main problem usually is the use of a software which is not reliable besides being big enough so mainte-nance is harder. This piece of software usually is the oper-ating system. Therefore, virtualization uses a single piece of software, that operates in kernel mode: the hypervisor, which is at least two orders of magnitude smaller than gen-eral purpose OSs and less likely to have failures [12].
According to [9], there are two main approaches to im-plement the virtualization technique, by using hypervisor either type 1 or type 2.
In hypervisor type 1, also known ashardware level virtu-alization, the hypervisor itself can be considered as an op-erating system, since it is the only piece of software that works in kernel mode, like depicted in the left side of Fig-ure 1. Its main task is to manage multiple copies of the real hardware - the virtual boards - just like an OS manages multitask.
On the other hand, in type 2 hypervisors, also known as
operating system level virtualization, depicted in the right side of Figure 1, the hypervisor itself can be compared to another user application that simply “interprets” the guest machine ISS.
In both cases, the virtual board must behave the exact same way as the real hardware. More specifically, it should be possible to start and restart them as real computers, as well as install different OSs. Thus, the creation of this sce-nario is up to the hypervisor.
The hypervisor (independently from its type) and the real hardware are responsible for dealing with instructions
Figure 1. Hypervisors Type 1 and Type 2
ing from the virtual board. It is important to highlight that since a virtual board “copies” the real hardware, it also has separate execution modes: kernel and user mode.
In order to derive the virtualization requirements, classic studies of Popek and Goldberg [8] introduce a classification of instructions of an ISA (Instruction Set Architecture) into three different groups:
1. privileged instructions: those that trap when used in user mode and do not trap if used in kernel mode;
2. control sensitive instructions: those that attempt to change the configuration of resources in the system, and;
3. behavior sensitive instructions: those whose behav-ior or result depends on the configuration of resources (the content of the relocation register or the processor’s mode).
Therefore, those researches first declared that in order to virtualize a given machine, sensitive (control and behavior) instructions must be a subset of privileged instructions. This is not a reality in many processors, as Intel’s x86 family and the common solution in this case, is to adopt processor’s hardware support. To name, Intel’s support is named as VT (Virtualization technology) and AMD’s named as SVM (Se-cure Virtual Machine). Hardware’s support probably is not an option for embedded systems, since the desirable solu-tion must deal with existing hardware, especially to accel-erate time-to-market.
Considering the options to virtualize systems: either OS or hardware level, although at hardware level we may need some support from the processor, at OS level the virtual boards share both the hardware and the host’s operating sys-tem. Since one of the most promising advantages of using virtualization is to allow several operating systems in a sin-gle hardware, this type of virtualization will no longer be considered for our analysis.
Thereby, let’s detail virtualization concerns when done at hardware level without hardware support (like VT or SVM). Initially, since the hypervisor is at charge of trans-lating instructions whenever the virtual board attempts to execute a privileged instruction (I/O request, memory write etc), which causes a trap into the hypervisor. This is known as pure virtualization and is often a very expensive way of dealing with virtual machines [13].
Another other option at hardware level is known as im-pure virtualization and requires that sensitive instructions (those that require a trap into the hypervisor) are removed from the code executing in the virtual machine. This can be done either at compile time, by a technique called pre-virtualization, or bybinary code rewriting, where the ex-ecutable code is scanned in order to replace such instruc-tions. The main issue in this approach is that both of them can cause performance loss.
Alternatively, para-virtualizationcan be used as it re-places sensitive instructions of the original code by explicit hypervisor calls (hypercalls). In reality, the guest OS is act-ing like a normal user application runnact-ing on a regular OS. Instead of it, is the guest OS running on the hypervisor.
When para-virtualization is adopted, the hypervisor must define an interface composed by system calls allowed to be used by the guest OS. Still, it is possible to remove all sen-sitive instructions of the guest OS, forcing it to only use hy-percalls which makes the hypervisor to become more like a microkernel. Besides working on hardware that is unsuit-able for pure virtualization, it can also bring performance boost.
The difference between pure virtualization and para-virtualization is depicted in Figure 2. In part A of the fig-ure, pure virtualization is showed. In this case, whenever the guest OS calls a sensitive instruction, a trap is caused to the hypervisor, which emulates the instruction behavior and return the proper results. In part B, para-virtualization is showed. The guest OS has been modified in order to make hypercalls instead of containing sensitive instructions. In this case, the trap is similar to the one that occurs in non virtualized systems, whenever a user application makes a system call on its OS.
Figure 2. Hypervisor control of pure virtual-ization (part A) and para-virtualvirtual-ization (part B)
Therefore, by using the basic concepts presented here re-garding to embedded systems and virtualization, we discuss the advantages, disadvantages and how (and whether) virtu-alization should be used in embedded systems.
3. Why should I
want
to use virtualization in
ESs?
In this paper, we consider ESs as being multiprocessed non-critical yet timing constrained systems, and in this sec-tion we highlight possible virtualizasec-tion use cases consider-ing such systems.
The first use case for virtualization on embedded sys-tems consists of allowing several operating syssys-tems to be executed concurrently. Here, the use of different operating systems can be specially indicated in two different situa-tions:
1. when legacy software must co-exist with current and incompatible applications, and;
2. when it is desired to part real-time software from user interface applications, by using different OSs.
When applying virtualization with this goal, the hyper-visor must have total control of the hardware besides cre-ating different virtual boards, one per OS. As depicted in Figure 3, this approach can be used in both single and multi-core architectures. It can strongly increase soft-ware development quality, since it allows the designer to choose among several OSs that one which is the most suitable for its application or even the one with the best cost/performance ratio. Moreover, the time required to de-velop an application can be reduced, since in the case that an ES has the virtual support for a given OS, any former appli-cation can be reused, without the onus of rewriting it [11].
Figure 3. Hypervisor for separation with sev-eral OSs
Furthermore, this approach offers advantages to achieve a unified software architecture that can be executed on mul-tiple hardware platforms. In this case, a current issue in embedded systems - software portability - could be widely affected and developers would be able to faster satisfy the increasingly restricted time-to-market.
Besides that, security is another important issue to be managed by using virtualization, since it provides an
ronment that protects and encapsulates embedded operat-ing systems and other software components. Initially, the idea of using an application specific operating system apart from the RTOS is again encouraged as user attacks will only be able to cause damages at the user OS, thus keeping the RTOS and specific system components safe.
This approach is depicted in Figure 4, where the sce-nario containing separate OSs and an ongoing user attack is shown. Nevertheless, in order to actually guarantee that vir-tualization will improve security, the underlying hypervisor has to be significantly more secure that the guest OS. The most suitable way of achieving it, is by keeping the hyper-visor as small as possible.
Figure 4. User attack blocked by virtual boards’ isolation
Consequently, the use of previously known hypervisors, as those employed in general purpose computing, will prob-ably not be possible. In order to keep a small hypervisor, specific constraints from embedded systems must be taken into account, what reflects in its whole development cycle.
In multi-core architectures, there are different ways of utilizing the many processors of the system. A very com-mon way of doing it so, is to run a single OS onto the pro-cessors, thus creating a symmetric multiprocessing (SMP) configuration. This approach brings the main advantage of making load balancing across the processors straightfor-ward. However, it does not allow the use of different OSs in the same MPSoC, which we discussed previously to be an attractive option, besides consisting of a single failure point and - whenever the system crashes - all the cores must be restarted.
In this case, we can adopt an asymmetric multiprocess-ing (AMP) configuration, where each processor has its own
separate OS, responsible for scheduling its own tasks. AMP is a configuration that takes advantages of virtualization since it provides arbitration of resources use between the virtual boards, avoiding the user OS to cause unexpected behavior on the RTOS [3]. If no virtualization is used, the only way to achieve such separation is by doing it manually, which is a complicated and more likely to error manner. The hypervisor can map every virtual board on each core of the multi-core processor or even map a single OS onto multiple cores, creating an SMP subset of cores [10].
When working with SMP configuration, workload bal-ancing is also affected, since information regarding sys-tem’s tasks are kept in OS’s tables. By migrating the mem-ory footprint, everything related to a given set of tasks is also migrated, thus helping in this complicated matter.
To embedded enterprise world, the reduction of costs brought by virtualization along with the possibilities of eas-ier updates are also promising advantages that should en-courage its use.
Summing up, we have the following main advantages by using embedded virtualization:
• the use of different OSs;
• increased security and reliability of the system, and;
• different configuration alternatives in a multi-core en-vironment.
4. Why should I
avoid
to use virtualization in
ESs?
Whereas embedded virtualization can bring indeed a great number of advantages, it is important to clearly under-stand at what cost they can be achieved. Some limitations are already present when using virtualization in general pur-pose computing, while others arise from its use in severely constrained environments, as embedded systems.
The first issue to be dealt with, when bringing virtualiza-tion to embedded systems, is how the hypervisor schedules system tasks. This occurs due to the fact that each virtual board is scheduled by the hypervisor as black boxes, while the guest OS is responsible for scheduling the tasks within a given board. This behavior should be strongly avoided in embedded systems.
Consider the case where a given multi-core presents asymmetric multiprocessing behavior, by having two dif-ferent OSs: a user OS and an RTOS. In this case, each OS is treated like a separate virtual board and, in embed-ded systems, is clear that the RTOS should be prioritized over the user OS, as well as real-time tasks that eventually are executed on the user OS (like media applications) must also gain different priorities. This priority based scheduling goes against the principles of virtual machines, where all
virtual boards are supposed to share the underlying hard-ware at equal proportions.
Another challenge to ES virtualization comes from the inherit ES heterogeneity. While in general purpose com-puting, where Intel x86 architecture is widely used, in ES the architecture range is wide enough to go from DSPs, to ARM cores, passing through MIPS and PowerPC architec-tures. This is a struggle for the hypervisor implementation, which must support the processor architecture of the ES.
Still, the excessive and absolute isolation brought by vir-tual machines - while increasing security and reliability lev-els - can also mean that the various embedded subsystems will struggle in other to achieve its strong cooperation, a behavior that is highly present in ESs.
5. When and how could I use virtualization in
ESs?
After considering the previously explained concepts, we briefly discuss some scenarios where embedded virtualiza-tion seems to be more suitable to be adopted.
InScenario 1, we aim to reduce the number of proces-sors in a system by consolidating them onto virtual boards in a single processor (single or multicore). Figure 5 shows a configuration where this is possible. Even in case the pro-cessor offers hardware support to virtualization (like VT or SVM), it is desirable to adopt para-virtualization, which al-lows the hypervisor to be independent from hardware vir-tual support besides bringing performance boost.
Figure 5. Scenario 1: processor number re-duction
In Scenario 2, we can increase the reliability of AMP systems by guaranteeing resource (memory, devices) sep-aration and the ability to restart virtual boards, as de-picted in Figure 6. Note that we can use it also when an AMP subset of processors is present in the MPSoC.
Figure 6. Scenario 2: improved reliability
InScenario 3, it is possible to migrate existing systems into a virtual board and to add more functionality in new virtual boards, providing the opportunity for reuse and in-novation. This is showed in Figure 7.
Figure 7. Scenario 3: migration between vir-tual boards
InScenario 4, the combination of real-time, legacy and general purpose operating systems in the same device is achieved thanks to virtualization, as Figure 8 shows.
Figure 8. Scenario 4: legacy software use along with new user applications
All this scenarios are suitable enough to be used in non-critical embedded systems, as multimedia mobile devices. Specially, as consumer demand continues to grow, stricter time-to-market tends to be more present and virtualization can enable their achievement.
6. Conclusions
Virtualization has been usually adopted by enterprise IT to better enjoy the multi-core processors computing power. Meanwhile, embedded systems (ES) used to be extremely restricted and single purpose systems. This scenario is un-der change. ESs are increasingly becoming part of people’s lives and their multiple functionalities lead to an non-linear growth of software complexity. In this context, many so-lutions are being studied, like virtualization. The great ap-peals of embedded virtualization, mainly, are: (i) to allow several OSs (RTOS and user OSs) to run in the same proces-sor; (ii) reduce cost of manufacturing, since it can increase processor utilization; (iii) improve security and reliability, and; (iv) helps decrease ES software development complex-ity.
The next challenge, thereby, is to adapt the hypervisor approaches with microkernel implementations, in order to enable the development of light virtualization techniques, suitable enough for modern and future embedded systems.
References
[1] H. Hansson, M. Nolin, and T. Nolte. Real-time in embedded systems. In R. Zurawski, editor,Embedded Systems Hand-book, chapter 2. CRC press, 2005.
[2] G. Heiser. The role of virtualization in embedded systems. InIIES ’08: Proceedings of the 1st workshop on Isolation and integration in embedded systems, pages 11–16, New York, NY, USA, 2008. ACM.
[3] M. Hermeling. The key differences between enterprise and embedded virtualisation. Web, Available at http://ece-news.stc-d.de /custom/enews220709.htm. Accessed at 10 feb., 2009.
[4] M. Hohmuth, M. Peter, H. H¨artig, and J. S. Shapiro. Reduc-ing tcb size by usReduc-ing untrusted components: small kernels versus virtual-machine monitors. InEW11: Proceedings of
the 11th workshop on ACM SIGOPS European workshop,
page 22, New York, NY, USA, 2004. ACM.
[5] A. Jerraya, H. Tenhunen, and W. Wolf. Multiprocessor systems-on-chips.Computer, 38(Issue 7):36– 40, July 2005. [6] L. Lavagno and C. Passerone. Design of embedded systems. In R. Zurawski, editor,Embedded Systems Handbook, chap-ter 3. CRC press, 2005.
[7] G. Martin. Overview of the mpsoc design challenge. In
DAC ’06: Proceedings of the 43rd annual conference on Design automation, pages 274–279, New York, NY, USA, 2006. ACM Press.
[8] G. J. Popek and R. P. Goldberg. Formal requirements for virtualizable third generation architectures.Commun. ACM, 17(7):412–421, 1974.
[9] M. Rosenblum. The reincarnation of virtual machines.
Queue, 2(5):34–40, 2004.
[10] J. Stoess, C. Lang, and F. Bellosa. Energy management for hypervisor-based virtual machines. InATC’07: 2007 USENIX Annual Technical Conference on Proceedings of the USENIX Annual Technical Conference, pages 1–14, Berkeley, CA, USA, 2007. USENIX Association.
[11] S. Subar. Virtualisation to enable next billion devices. Web, Available at http://www.embeddeddesignindia.co.in/ ART 8800576093 2800003 TA 7cb7532e.HTM. Accessed at 10 feb., 2009.
[12] A. S. Tanenbaum.Modern Operating Systems. Prentice Hall Press, Upper Saddle River, NJ, USA, 2007.
[13] C. A. Waldspurger. Memory resource management in vmware esx server. SIGOPS Oper. Syst. Rev., 36(SI):181– 194, 2002.
[14] W. Wolf. Computers as components: principles of embed-ded computing system design. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2001.
[15] W. Wolf. A decade of hardware/software codesign. Com-puter, 36(4):38–43, April 2003.
[16] Y. Zorian and E. Marinissen. System chip test - how will it impact your design. InDAC’2000 - Design Automation Conference, Las Vegas, EUA, Jun 2000. ACM Press.
