An Evolvable Operating System for Wireless Sensor Networks

ABSTRACT

In the recent years, rapid development of Wireless Sensor Network (WSN) technology promises a revolution in sensing applications which are deeply embedded in the physical world. Each node in the network has to operate with a very limited memory, energy, and computation capacity and is very prone to failures. Besides, it has to provide a lot of functionalities as the applications for WSN become more and more sophisticated.

Dynamic reconfiguration and self-adaptation are also vital for WSN to operate in a dynamic environment. Runtime technologies which allow software to evolve as system requirements and environment changes are critical to develop and deploy applications for such networks. As the result, more and more challenges are posed to the developers. One of those is to design an efficient operating system for sensor nodes. With low power consumption, small code and data size, evolvability as design criteria, we develop an evolvable operating system (EOS) for WSN applications. The EOS provides memory space efficient thread management, collaborative thread communication model and network stack. It also supports power management of microcontroller and radio transceiver, and network wide time synchronization function. Above all, we propose our concept of evolvability and an approach for evolvable

M.S.

20042115

Thu Thuy Do

An Evolvable Operating System for Wireless Sensor Networks

School of Engineering. 2006. 61p.

Major Advisor: Daeyoung Kim.

Text in English

software reconfiguration for WSN. Our approach provides a general framework for on-the-fly upgrading software ranging from operating system to application modules. Upgrading procedures can be implemented transparently to users with minimal system interruption.

Our system was implemented on our ANTS-H2 outdoor hardware platform and deployed in Halla Mountain, Jeju Island (Haroobang project).

We also describe our system’s performance evaluation in terms of time and memory

usage.

C ONTENTS

Contents ...iii

List of Figures... v

List of Tables ... vii

List of Abbreviations ...viii

1 Introduction... - 1 -

2 Related works ... - 3 -

2.1 Wireless Sensor Network Operating Systems ... - 3 -

2.1.1 Mantis Operating System - University of Colorado... - 3 -

2.1.2 TinyOS ... - 4 -

2.1.3 SOS - UCLA ... - 7 -

2.1.4 Contiki ... - 8 -

2.2 Dynamic reconfiguration consideration in the field of OS ... - 8 -

2.3 Embedded Operating Systems...- 11 -

3 Requirements for WSN OS... - 12 -

4 EOS Hardware ... - 14 -

5 EOS System architecture ... - 17 -

5.1 Task Management... - 18 -

5.1.1 Related hardware ... - 19 -

5.1.2 Preemption round robin scheduler ... - 22 -

5.1.3 Hybrid Task scheduler ... - 26 -

5.1.4 Thread Synchronization ... - 29 -

5.2 Message Handling ... - 32 -

5.2.1 Inter-Thread Communication (ITC)... - 33 -

5.2.2 Remote-Thread Communication (RTC)... - 36 -

5.3 Network Stack ... - 37 -

5.4 Power Management ... - 37 -

5.5 Time Synchronization... - 38 -

5.6 Light-weight memory management... - 39 -

6 Evolvability – A dynamic reconfiguration framework for WSN software... - 41 -

6.1 Evolvability concept ... - 41 -

6.2 Evolvable reconfiguration framework constraints... - 41 -

6.3 Function reference problem... - 42 -

6.4 Dynamic reconfiguration framework ... - 43 -

6.4.1 Configuration Manager ... - 44 -

6.4.2 Module design... - 45 -

7 EOS applications ... - 53 -

8 EOS Implementation and Evaluation... - 55 -

8.1 Development environment ... - 55 -

8.2 Performance Evaluation ... - 58 -

9 Conclusion ... - 61 -

10 Reference... - 62 -

L IST OF F IGURES

Figure 1. Mantis OS architecture [1][2] ... - 3 -

Figure 2. Photograph and schematic for representative network sensor platform [3] ... - 5 -

Figure 3. A simple network sensor application with Tiny OS[3] ... - 6 -

Figure 4. SOS architecture[4]... - 7 -

Figure 5. Contiki – Partition into core and loaded program[5] ... - 8 -

Figure 6. An example exokernel-based system[9] ... - 9 -

Figure 7. SPIN architecture[10] ... - 10 -

Figure 8. ANTS-H2 outdoor Hardware... - 14 -

Figure 9. Evolvable OS architecture ... - 17 -

Figure 10. Customised EOS for Haroobang Project... - 18 -

Figure 11. Execution context... - 20 -

Figure 12. Program memory map [16] ... - 21 -

Figure 13. Data Memory Configuration [16] ... - 22 -

Figure 14. Stack allocation in the memory map ... - 23 -

Figure 15. Ready Queue architecture ... - 24 -

Figure 16. Thread Transition Diagram ... - 25 -

Figure 17. State transition diagram ... - 28 -

Figure 18. Comparison of stack usage ... - 29 -

Figure 19. Message Handling Engine ... - 33 -

Figure 20. Message Queue Structure... - 34 -

Figure 21. Relationship between tasks and a message queue... - 34 -

Figure 22. Communication Models... - 37 -

Figure 23. Exchanging timestamps ... - 39 -

Figure 24. Function reference problem ... - 43 -

Figure 25. Evolvable Reconfiguration Framework ... - 44 -

Figure 26. Kernel module interaction... - 46 -

Figure 27. Application module interaction... - 47 -

Figure 28. Upgrading procedure ... - 48 -

Figure 29. Ready Queue structure ... - 50 -

Figure 30. PSC model ... - 51 -

Figure 31. Publish Message ... - 52 -

Figure 32. Subscribe Message... - 52 -

Figure 33. EOS applications... - 53 -

Figure 34. Interactive configuration for EOS... - 56 -

Figure 35. EOS Source tree... - 57 -

Figure 36. Program a sensor node by PonyProg2000 software... - 58 -

L IST OF T ABLES

Table 1. ATmega 128L features [16] ... - 15 -

Table 2. CC1020 Features [17]... - 16 -

Table 3. Memory Configuration [16]... - 21 -

Table 4. Timers ... - 22 -

Table 5. Thread Scheduler APIs ... - 26 -

Table 6. Mutex APIs ... - 31 -

Table 7. Semaphore APIs ... - 32 -

Table 8. ITC APIs ... - 36 -

Table 9. ATMEGA128L Six Sleep modes [16] ... - 38 -

Table 10. Light-weight memory management APIs ... - 40 -

Table 11. Code and data size breakdown for our system ... - 59 -

Table 12. Delay in EOS ... - 60 -

L IST OF A BBREVIATIONS

AM Application Module

ANTS An evolvable Network of Tiny Sensors API Application Programming Interface BCC Base station Centered Communication

CGC Collaboration/Group Communication

CM Configuration Manager

EOS Evolvable Operating System

ERF Evolvable Reconfiguration Framework

ES Exportable Subroutine

HAL Hardware Abstraction Layer

IS Internal Subroutine

ITC Inter Thread Communication

KM Kernel Module

MCB Module Control Block

MHE Message Handling Engine

MOS Mantis Operating System

MSDM Message Send/Dispatch Engine

PC Program Counter

PSC Publish/Subscribe Communication

SOS Sensor Operating System

SREG Status Register

TCB Thread Control Block

TMB Thread Main Block

WSN Wireless Sensor Network

1 I NTRODUCTION

Wireless Sensor Networks (WSN), as a key enabling technology for ubiquitous computing have a wide range of applications, which are deeply embedded in the physical world. Since sensor nodes that comprise sensor networks are highly resource constrained, there are many research challenges that must be overcome, such as a light-weight network stack, security mechanism, localization, operating system, etc. WSN can include several billions of heterogeneous sensor network connected world wide. Sensor nodes, of which, the behaviors (born, live and die) resemble living creatures, are often densely deployed in large numbers in those networks. These characteristics pose a new requirement of a deeply dynamic future for WSN technology: The sensor networks should be ubiquitous, heterogeneous and interoperable. Sensor network software should be fault tolerant, self-configurable to flexibly adapt to any change of environment.

In this thesis, we propose our design of EOS (Evolvable Operating System). It provides a new architecture of a flexible and efficient operating system, which satisfies the requirements and constraints for the WSN. The key features of EOS are a flexible multithreaded engine which is portable, scalable and evolvable, easy to use with standard programming language and a sufficient Application Programming Interface (API) set for building different application scenarios. Above all, we introduce the concept of evolvability applied for WSN software, with which, both the applications and the operating system can be easily configurable and upgradeable.

As the applications for the WSN becomes more and more complex which require great support from the operating system, the multithreaded engine was chosen for designing EOS.

The noticeable design point in this architecture is that, EOS can provide an efficient

combination of two classes of threads: the micro-threads which operate in a shared stack and

the generic-threads which have their own stacks. With simple applications, EOS scheduler can

be configured as a preemption-round robin scheduler. Besides, with complex applications,

EOS scheduler with the combination of two kinds of threads can meet the memory constraint

in a sensor node. Meanwhile, it still has a very efficient interleaving thread mechanism.

Advances in hardware technology offer multiple platforms for different sensing applications. EOS with a flexible Hardware Abstraction Layer (HAL) is capable of porting to different platforms with no interference with the development of its applications.

EOS is scalable because of its high modularity. Module organization in the system is flexible enough to enable the “Plug and Play” feature. Different requirements for the system can be easily satisfied by upgrading functional modules in the OS or application modules.

EOS’s evolvability is one of the emerged features of this operating system. It ensures the sensor node to meet these conditions: Self-configurable - The sensor node can adapt to the change in the network topology, fault-tolerant and reliable - The sensor node as well as the whole network is not affected by another node’s failure, transparent - The change of internal node does not affect users, upgradeable - The sensor node is easy to upgrade with inheritance features.

Another feature by which EOS is hopefully applied to a wide range of sensor network applications is the ease to use. EOS applications can be easily developed with the familiar C programming language and a sufficient OS API set. Applications can be developed with standard ANSI C and compiled with avr-gcc. No specialized programming language and compiler are needed. The choice of C programming language also benefits from the reuse of a large numbers of open-source projects, thus it reduces the period to develop the projects and utilizes success from the vast open source community.

The rest of this thesis is organized as follow. We introduce several related works in section 2. Section 3 will discuss several requirements to design an operating system for WSN.

We briefly describe the initial hardware platform for EOS in section 4. EOS architecture is

described in section 5. We present the concept of evolvability and the dynamic reconfiguration

framework for WSN applications in section 6. Section 7 will explain how to write a simple

EOS application. EOS development environment and our study’s implementation and

evaluation are described in section 8. The conclusion and future work are presented in section

9 of this thesis.

2 R ELATED WORKS

In recent years, research efforts on developing operating systems, especially, operating systems for WSN have had a lot of achievements. They offer EOS a lot of opportunities as well as challenges to become an outstanding operating system for wireless sensor networks.

In this section, we will briefly describe some noticeable operating systems for WSN. As evolvability is the most important feature of our operating system, we will describe the related works involved with that concept. The extension technologies in some Extensible Operating System will be discussed. Some other related works, which will be considered, are several famous embedded operating systems.

2.1 Wireless Sensor Network Operating Systems 2.1.1 Mantis Operating System - University of Colorado

EOS shares the same idea to be a multi-threaded operating system with Mantis Operating System (MOS) [[1], [2]]. MOS has been developed by the MANTIS Group at CU Boulder, which is an open source, multi-threaded operating system (MOS) written in C for wireless sensor networking platforms.

Figure 1. Mantis OS architecture [1][2]

Mantis (MOS) provides an architecture of layered multithreaded operating system, which includes multithreading, pre-emptive scheduling with time slicing, layered network stack which is, implemented as one or more user-level threads. MOS provides a flexible architecture in the form of cross-platform support and testing across PCs, PDAs and multiple micro sensor platforms (MICA2, MICAz, and TELOS motes). It also offers an ability of remote management of in-situ sensors via dynamic reprogramming and remote login. The architecture of Mantis Operating System is described in Figure 1.

MOS kernel resembles classical, UNIX-style schedulers with a subset of POSIX threads.

Threads are priority-based and scheduled with round-robin semantics within a priority-level.

Whenever a thread is created, a stack size ranging from 64 bytes to 512 bytes is allocated for that thread. The maximum number of threads in Mantis is 12. As the result, the RAM size needed to allocate for all the stacks in the system is considerable in such a resource constrained system like a sensor node. EOS kernel is more efficient in that sense. The co- existence of micro-threads and generic threads in the system provides the means for efficient use of memory as well as flexible application development.

2.1.2 TinyOS

Tiny OS [3] is also a well-known operating system for the sensor network. It was initially developed by the U.C. Berkeley EECS Department. Unlike EOS, Tiny OS is an event- based operating system. The very lean multithreading architecture composes of a tiny scheduler and a graph of components. TinyOS's component library includes network protocols, distributed services, sensor drivers, and data acquisition tools. Those components can be used as-is or be further refined for custom applications. Figure 2 presents a photograph and schematic for representative network sensor platform. Figure 3 illustrates a sample configuration of components to provide a simple network sensor application.

High modularity, small code and data size are among recognized advantages of this operating system. However, tasks can not preempt each others and must run to completion.

Due to the atomic feature of tasks in TinyOS, there are no mechanism of communication

among tasks and no synchronization between tasks. In addition, nesC – the programming

language of Tiny OS, the dialect of C – is not easy to use for novice programmers.

Figure 2. Photograph and schematic for representative network sensor platform [3]

Figure 3. A simple network sensor application with Tiny OS[3]

Wiring mechanism of this language seems to be very efficient when the application is simple with a small number of involved components. But applications for the sensor network tend to be more and more sophisticated so that they can be a “real” bridge connecting the physical and the computational world. That means the wiring mechanism will become very cumbersome and thus, difficult to use.

EOS achieves succeeds of high modularity and satisfies the requirement of memory constraint. Our operating system has a mechanism of Inter-thread communication with effective synchronization method to enable resource sharing among threads in the system.

Besides, a comprehensive set of OS APIs written in C programming language is an advantage

of EOS. No special compiler and programming language, ease to use and a great deal of open-

source code are the good reasons for sensor network application developers to choose EOS as

an operating system for their target applications.

2.1.3 SOS - UCLA

SOS [4] is an operating system for mote-class wireless sensor networks developed by the Networked and Embedded Systems Lab (NESL) at UCLA. SOS consists of a common kernel and dynamic loadable modules. The architecture of SOS is described in Figure 4.

Modules in this OS have well-defined and generalized points of entry and exist.

Modules interact with each others through functions which were registered in kernel jump table or messages which are delivered by the message scheduler.

Figure 4. SOS architecture[4]

SOS uses publish-subscribe scheme which is similar to MOAP for distributing modules to deployed sensor nodes. This OS has advantages over the preceding OS as it can update modules during runtime which does not require system to reboot. Moreover, a module is not necessary to be a thread. It is just a unit of code which fulfills one function in the system.

However, SOS can not upgrade its static kernel on-the-fly. As the result, SOS still has some

inflexibility in sense of extensibility. In EOS, we provide the ability of upgrading both the OS

kernel module and application modules. EOS’s thread scheduler can also be upgraded online

to meet requirements of different application scenarios.

2.1.4 Contiki

Contiki [5] is also another light weight operating system which supports for dynamic loading and replacement of individual programs and services. Conkiti architecture has two parts: a core and loaded programs as shown in Figure 5.

Figure 5. Contiki – Partition into core and loaded program[5]

The core consists of Contiki kernel, the program loader, language runtime and communication service. This core can not be modified after deployment. Application program can be implemented in one process. Service functionality can be implemented by more than one process. Although Contiki supports the runtime replacement of services, the main drawback of this operating system is that it limits the extension unit to only process. That means it has no way to replace a chunk of code which can fulfill a function in the system but is not a process.

In our dynamic reconfiguration framework, we suggest that both the operating system and applications can be upgraded. We also consider the case that the module to be upgraded can be not only a thread but also a group of subroutines, or even it is a thread scheduler.

2.2 Dynamic reconfiguration consideration in the field of OS

Dynamic reconfiguration in Extensible Operating Systems has gained much focus since

many decades ago. Some excellent representations of this group are Micro kernel [6], [7], [8],

Exo kernel [9] and SPIN [10].

The basic idea of Micro kernel is to build a system with a kernel that supports a small number of abstractions such as threads, address space and inter-process communication and a set of user-level servers which provide common services such as file system, processes and virtual memory. With that way, the system is divided into smaller and more manageable pieces which are called modules. Those modules can be replaced if necessary. Examples of micro kernel architectures are Mach [11], L3/L4 [6], [7]] and Arena [12] .

Exo kernel provides application-level of physical resources. Virtual memory, inter- process communication and other system abstractions are implemented at application level by untrusted software.

Figure 6. An example exokernel-based system[9]

The kernel, which is called Exokernel, is written as small as possible, only exporting the essential low level resource control primitives. On top of this runs the library OS, which are application level programs that manage the resources exported from this kernel.

Application developers can choose these libraries or build their own. A prototype Exokernel

system based on secure bindings, visible revocation and abort protocols was implemented. It

includes an Exokernel named Aegis and a trusted library Operating system which is called

ExOS. An example of Exokernel-based system is shown in Figure 6, which consists of a thin

Exokemel veneer that exports resources to library operating systems through secure bindings.

Each library operating system implements its own system objects and policies. Applications link against standard libraries (e.g., WWW, POSIX, and TCP libraries for Web applications) or against specialized libraries (e.g., a distributed shared memory library for parallel applications).

SPIN is another extensible operating system which provides facilities for applications to modify basic operating system services such as the virtual memory manager or scheduling, in order to optimize for specific tasks. SPIN architecture is presented in Figure 7 as follow.

Figure 7. SPIN architecture[10]

The kernel and the extensions run in the same address space to lower communication costs. Modula-3, a type-safe language, which can protect the kernel from the extensions, is used to develop this system. Several extensions were built, such as a UNIX operating system server, a client/server video system, and communication components. They can prove that SPIN provides an extension infrastructure, with a core set of extensible services, which can allow application to change the operating system’s interface and implementation.

Although the preceding Operating Systems have gained considerable succeed in term of Extensibility, these extension technologies can not be applied for WSN Operating Systems.

Considering the requirement of evolvability for WSN OS, we develop our Evolvable

Reconfiguration Framework (ERF) with EOS as its main part. Our contribution is to provide a

thread-based system with high modularity and evolvability. The evolvable unit in our framework is not limited to a thread or process. It can be a module which handles a complete function in the system. The details in our design will be discussed in the next sections.

2.3 Embedded Operating Systems

Other related work should be considered are some major embedded operating systems

(for example: WinCE, Embedded Linux, VxWork, QNX, PalmOS, eCos, uC/OS-II, etc). All of

them have the multithreaded architecture which supports many functionalities and

programming APIs. A lot of effort is paid on building an operating system that has a small

code size, flexible and extensible. Unfortunately, these systems are just suitable for the world

of embedded PCs such as PDAs, cell phones or set-top-boxes. They do not come close to

satisfy the strict requirements of the operating system for the sensor network.

3 R EQUIREMENTS FOR WSN OS

As the nature of the operating system for the WSN is quite different in comparison with

“regular” embedded operating systems, we identify here some characteristics and requirements which directly impact on the design of the system’s architecture:

§ Concurrency: A sensor node needs to do a lot of job simultaneously. It senses data, manipulates and forwards the results to other nodes in the network. Due to the limitation of memory size, a large buffer is not recommended. So, high currency intensive operations which process information on-the-fly are among important requirements for WSN OS.

High concurrency intensive operations could be achieved by improving the context switch speed among tasks in the system.

§ Small code size: This requirement is necessary for the operating system of the sensor network as the given memory is small. For example: TinyOS operates in a very memory constrained platform with 8KB of Flash as the program memory, and 512 bytes of SRAM as the data memory. Our system also has to operate with 128 KB of Flash and 4KB of RAM. Small code size is one of the strategies to use the small memory of the sensor node efficiently.

§ Efficient Resource Utilization: The resources in the system can be energy, memory, radio, processor computation capability, etc. These resources are very limited, thus, and efficient use is needed to ensure the overall system performance.

§ High Modularity: This requirement ensures the flexibility of the system. High modularity allows the system to be easily configured.

§ Scalability and Easy to upgrade: The sensor software will tend to be application specific rather than general purpose. As the operating environment of wireless sensor networks can change very often, scalability and easy to upgrade allow the operating system as well as the application easily meet the new requirement of these changes.

§ Low Power Consumption: This requirement is obligatory as the sensor node is battery

operated. The capacity and the performance of the system are controlled by energy

constraints. Maintenance is expensive. Thus, prolonging the life time of a sensor node is

an important issue for developers. To achieve this requirement, some approaches can be

considered such as: choosing a power-aware scheduling algorithm or an adaptive control strategy for power consumption mode in the node.

§ Robust Operations: Since the sensor nodes are often densely deployed and their placements are sometimes in hostile environments. Changes in network topology and node failures are inevitable. Robustness enables individual node to self-test, self-configuration, and fault-tolerant.

§ Collaborative and Distributed Processing: As the applications for the sensor network are increasingly complex, the computation jobs become impossible for an individual node to complete. This feature implies that each node can locally carry out a part of the job and then, aggregate the data with the others.

§ Friendly Programming Interface: This feature makes the system easy to use, especially

for novice programmers, thus, it can win a large community of developers and users.

4 EOS H ARDWARE

EOS is implemented on our ANTS-H2 outdoor hardware platform with 8 bit ATMEGA128LCPU, 4KB RAM, 128 KB flash memory on chip, and CC1020 transceiver, various sensors such as light, accelerometer, temperature , humidity, and air pressure as shown in Figure 8.

Figure 8. ANTS-H2 outdoor Hardware.

The features of ATMega128L- 8 bit AVR Microcontroller [16] and CC1020 [17] are presented in Table 1 and Table 2 as follows.

ATmega128L features

High-performance, Low-power AVR 8-bit Microcontroller Advanced RISC Architecture

- 133 Powerful Instructions - Most Single Clock Cycle Execution

- 32 x 8 General Purpose Working Registers + Peripheral Control Resister - Up to 16 MIPS Throughput at 16MHz

- Fully Static Operation - On-chip 2-cycle Multiplier

Non-volatile Program and Data Memories

- 128k Bytes of In-System Self-Programmable Flash - Optional Boot Code Section with Independent Lock Bits - 4K Bytes EEPROM

- 4K Bytes Internal SRAM

- Programming Lock for Software Security

- Up to 64K Bytes Optional External Memory Space

- SPI Interface for In-System Programming

JTAG (IEEE std. 1149.1 Compliant) Interface

- Boundary-scan Capabilities According to the JTAG Standard - Extensive On-chip Debug Support

- Programming of Flash, EEPROM, Fuses, and Lock Bits through the JTAGS Interface Peripheral Features

- On-chip Analog Comparator

- Programmable Watchdog Timer with Separate On-chip Oscillator - Master/Slave SPI Serial Interface

- Two 8-bit Timer/Counters with Separate Prescalar, Compare

- Two Expanded 16-bit Timer/Counters with Separate Prescaler, Compare and Capture mode - Real Time Counter with Separate Oscillator

- Six PWM Channels with Programmable Resolution from 1 to 16 Bits - Dual Programmable Serial USARTs

- 8-channel, 10-bit ADC

- Byte-oriented Two-wire Serial Interface - Four PWM Channels

- Dual Programmable Serial USARTs Special Microcontroller Features

- Power-on Reset and Programmable Brown-out Detection - Internal Calibrated RC Oscillator

- External and Internal Interrupt Sources

- Six Sleep Modes: Idle, ADC Noise Reduction, Power-save, Power-down, Standby, and Extended Standby

- Software selectable Clock Frequency

- ATmega103 Compatibility Mode Selected by a Fuse - Global Pull-up Disable

I/O and Packages

- 53 Programmable I/O Lines - 64-lead TQFP, and 64-pad MLF Operating Voltages

- 2.7 – 5.5 V for ATmega128L Speed Grades

- 0-8 MHz for ATmega128L

Table 1. ATmega 128L features [16]

CC1020 Features

• True single chip UHF RF transceiver

• Frequency range 424 MHz - 470 MHz and 848 MHz - 940 MHz

• High sensitivity (up to –119 dBm for a 12.5 kHz channel)

• Programmable output power

• Low current consumption (RX: 17 mA)

• Low supply voltage (2.3 V to 3.6 V)

• No external RF switch / IF filter needed

• Low-IF receiver

• Very few external components required

• Small size (QFN 32 package)

• Digital RSSI and carrier sense indicator

• Single port antenna connection

• Data rate up to 153.6 kBaud

• ASK, FSK and GFSK data modulation

• Integrated bit synchronizer

• Image rejection mixer

• Programmable frequency and AFC make crystal temperature drift compensation possible without TCXO

• Suitable for frequency hopping systems

• Complies with EN 300 220 and FCC CFR47 part 15

• Development kit available

• Easy-to-use software for generating the C1020 configuration data

Table 2. CC1020 Features [17]

5 EOS S YSTEM ARCHITECTURE

We present the architecture of EOS in Figure 9. It consists of six main components:

Figure 9. Evolvable OS architecture

§ Hardware: EOS runs on various sensor hardware platforms, mostly with strictly limited resources (e.g.: an 8-bit μC with 128KB Flash, 4 KB RAM).

§ Hardware Abstraction Layer: It abstracts different types of hardware, eventually providing portability of EOS.

§ Kernel: As the heart of our operating system, it consists of eight functional blocks:

Configuration Manager, Resource Management, Power Management, Message Handling, Network Stack, Task Management, Time Management, and Synchronization.

§ Device driver: It provides device drivers for all the devices in sensor nodes such as ADC, serial port, eeprom, etc.

§ Debug/monitoring interface: It provides a command set with which users can debug and

monitor the status of a sensor node.

§ OS API: It provides EOS API to various EOS applications.

As the first step to deploy EOS on the real field, we also customized this architecture to provide the basic functional blocks for Haroobang project as in Figure 10 as follows:

Figure 10. Customised EOS for Haroobang Project

We will describe in details the design of EOS architecture in the following sections.

5.1 Task Management

Task management is one of the most important functional blocks in EOS architecture.

We define task (or thread) as a basic execution unit in the system. Two terms “task” and

“thread” are used in this thesis interchangeably.

As the resource, especially the memory in a sensor node is very limited, so, the main design point for the task management functional block is to provide a flexible multi-threaded mechanism with efficient RAM usage for stacks. In a conventional multi-tasking operating system, a number of tasks are allocated separate stacks for each task. This stack is used to store local variables, return addresses, etc. and the task’s context when context switching occurs.

When an interrupt is serviced, this stack can be used to store the interrupt handler.

Unfortunately, this allocation method can use a large a mount of memory, which is very

limited in a sensor node, when we have a great number of tasks running concurrently in the

system.

In this thesis, we propose two scheduling algorithms. With a simple application with a small number of tasks, application developers can choose the preemption round robin scheduling algorithm. This is a very well known scheduling algorithm which was implemented in many conventional multi-tasking operating systems. We show that this scheduling algorithm still works well in such a limited memory system like a sensor node. For complex applications with a great number of tasks, we improve the preemption round robin scheduling algorithm to solve the problem of memory limitation in the Hybrid Task Management model. Application developers can choose either of the two Task scheduling algorithms based on the requirement of the application scenarios.

5.1.1 Related hardware

We consider three main related hardware parts for our task management model. They are the execution context, memory configurations and timers.

§ Execution context: On the AVR microcontroller [16], the context consists of:

• 32 General Purpose Registers: The GCC compiler assumes register R1 is set to zero. Each General Purpose Registers is of 8 bit length.

• Status Register (SREG): The value of the status register affects instruction execution, and must be preserved across context switches. It contains information about the result of the most recently executed arithmetic instruction. This information can be used for altering program flow in order to perform conditional operations. SREG is of 8 bit length.

• Program Counter (PC): Upon resumption, a task must continue execution from the instruction that was about to be executed immediately prior to its suspension.

The PC is 16 bit wide.

• The Two Stack Pointer Registers: The Stack Pointer Register always points to the top of the Stack. The AVR Stack Pointer is implemented as two 8-bit registers in the I/O space.

The execution context is described in Figure 11 as follows. The total size of execution

context is 37 Bytes. The Stack is mainly used for storing temporary data, for storing local

variables and for storing return addresses after interrupts and subroutine calls. So, the minimum stack size is 64 Bytes.

Figure 11. Execution context

§ AVR ATmega128 Memories

The AVR architecture has two main memory spaces: the Data Memory and the Program

memory space. The ATmega128 contains 128Kbytes On-chip In-System Reprogrammable

Flash memory for program storage. Since all AVR instructions are 16 or 32 bits wide, the Flash

is organized as 64K x 16. For software security, the Flash memory is divided into two sections,

Boot Program section and Application Program sections. The program memory map is

presented in Figure 12.

Figure 12. Program memory map [16]

For SRAM data memory, the ATmega128 supports two different configurations as listed in Table 3.

Configuration Internal SRAM Data Memory External SRAM Data Memory

Normal mode 4096 Up to 64K

ATmega103 Compatibility mode

4000 Up to 64K

Table 3. Memory Configuration [16]

In normal mode, the first 4352 Data Memory locations address both the Register file, the I/O Memory, Extended I/O Memory, and the internal data SRAM. The first 32 locations address the Register file, the next 64 location the standard I/O memory, then 160 locations of Extended I/O memory, and the next 4096 locations address the internal data SRAM.

In ATmega103 compatibility mode, the first 4096 Data Memory locations address the

Register file, the I/O Memory and the internal data SRAM. The first 32 locations address the

Register file, the next 64 location the standard I/O memory, and the next 4000 locations

address the internal data SRAM. The Data Memory Map is described in Figure 13. For our

implementation, we use Normal mode.

Figure 13. Data Memory Configuration [16]

§ Timers:

The Atmel ATmega128 architecture provides 4 timers, which are described in Table 4 as follows.

Timer Description Usage

Timer 0 § Asynchronous

§ 8 bits

RTC: for PM (using external clock)

Timer 1 § Synchronous

§ 16 bits For timeslice (MAX: ~ 8s)

§ 1024 Prescaler (1tick = 128 usec)

§ Suggested default timeslice: 128msec

Timer 2 § Synchronous

§ 8 bits Other purpose (MAX: 32 msec)

Timer 3 § Synchronous

§ 16 bits

For thread sleeping (MAX: ~8s)

§ 1024 Prescaler Table 4. Timers

5.1.2 Preemption round robin scheduler

5.1.2.1 Thread Specification

We define maximum number of threads in the system is 10. The total number of

priorities in the system is 4, ranging from PRIORITY_IDLE (lowest priority) to

PRIORITY_HDL (highest priority). The definition of priorities is as follows.

#define NUM_PRIORITIES 4

#define PRIORITY_IDLE (NUM_PRIORITIES - 1)

#define PRIORITY_NORMAL (NUM_PRIORITIES - 2)

#define PRIORITY_HIGH (NUM_PRIORITIES - 3)

#define PRIORITY_HDL 0

The stack sizes are defined with 3 levels ranging from minimum of 64 Bytes to maximum of 256 Bytes. The definition of stack sizes is as follows.

#define IDLE_STACK_SIZE 64 //size of stack for init/idle thread

#define START_STACK_SIZE 64 //size of stack for start thread

#define STACK_MIN 64 // minimum stack size

#define STACK_SMALL 128

#define STACK_LARGE 256 // maximum stack size

5.1.2.2 Thread Control Block

When a thread is created, a data structure called Thread Control Block (TCB), which stores the specification for that thread is allocated in the memory. We define TCB as follows.

typedef struct _pthread_tcb {

uint8_t *sp; // Stack pointer

uint8_t *stack; // Pointer to stack memory for de-allocating uint16_t stackSize; // Size of stack

void *(*func)(void *); // Function pointer to thread's start function uint8_t state; // Current thread state

uint8_t priority; // priority

uint16_t sleep_time; // thread sleep time uint8_t id; // thread id

void *owner; // owner of this tcb } pthread_tcb_t;

The stack grows upside down toward the lower address. We define two pointers to keep

the information of stack address. ^*sp points to the top of stack and ^*stack points to the

starting of stack memory for de-allocating. We describe the stack allocation in the memory

map and the positions of ^*sp and ^*stack in Figure 14 as follows.

Note that, we also have to store the sleep time of each thread in its TCB because all the threads in the system share timer 3 for sleeping. We explain the method to share timer 3 for sleeping time as in the following example with 3 threads A, B and C.

5.1.2.3 Ready Queue architecture

The Ready Queue architecture is presented in Figure 15 as follows. We define Ready Queue as an array of linked lists. Each linked list stores the TCBs which have the same priority.

With this data structure of Ready Queue, to dispatch a new thread, scheduler just needs to pick the TCB from the head of the linked list which has the highest priority among the non-empty ones.

Figure 15. Ready Queue architecture

5.1.2.4 Preemption round robin scheduling algorithm

A thread can have the following states: BEGIN, READY, RUNNING, BLOCKED, SLEEP and END. The thread transition diagram is described in Figure 16.

Figure 16. Thread Transition Diagram The scheduling algorithm is presented as follows.

BEGIN While (1) {

Take the Highest Priority Task from the READY Queue;

If Idle Task then { Go to SLEEP state;

Continue;

}

If time_slice expires then Continue;

If Task is suspended then Continue;

If Task is put to sleep then {

Return Task to READY Queue after time_slice expires;

Continue;

} } END.

5.1.2.5 Thread Scheduler APIs

API prototypes Description

void _init_scheduler (void); Initialize scheduler, called by apps

void start_scheduler(void); Start the scheduler, called in the main()of apps uint8_t pthread_create(void

*(*function_start)(void *), void

*arg, uint16_t stack_size, uint8_t priority);

Create new thread and insert its TCB to the ready list.

void pthread_exit(void); Terminate the calling thread void pthread_suspend(void); Suspend the current thread void pthread_continue(pthread_tcb_t

*thread);

Resume the suspended thread specified by its TCB

void pthread_continue2(uint8_t thread_id);

Resume the suspended thread specified by its thread id

void pthread_sleep(uint16_t sleeptime);

Put the calling thread to sleep

void pthread_wake_up(void); Wake up a sleeping thread. This thread is picked up from the head of sleeping queue.

int pthread_cancel(int thread_id); Cancel a thread which has the thread id equal to thread_id.

pthread_tcb_t *pthread_current(); Get the TCB of current thread (used in Thread Sync.)

Table 5. Thread Scheduler APIs

5.1.3 Hybrid Task scheduler

The preemption round robin scheduling algorithm is a very well known algorithm. We showed that not only in the big operating systems but also in WSN OS, this scheduling algorithm can be used. However, if it is not a small number of tasks concurrently used in the system, the limited resource of a sensor node can be an obstacle.

We improve the preemption round robin scheduling algorithm to a new one which is named Hybrid Task Management. All the tasks can be classified into two categories: the generic threads with their own separate stacks and the micro threads sharing a single stack within the group. The approaches of using a single stack in the system and the fixed priority scheduler with the Preemption Threshold mechanism have been used. To control the levels of preemption, we use the concept of Non-Preemption Group [13], which is a collection of tasks that are not permitted to preempt each other in the same group. The idea of using a single stack in the system and the fixed priority scheduler with the Preemption Threshold mechanism has been proposed, both from the research community and as commercial off-the-shelf tools [13].

Our work extends this idea to combine two classes of tasks and applied it in the sensor

operating system.

The micro-threads should meet the following requirements for an efficient Single Stack operation:

- Each task executes to completion and can possibly be preempted by a higher priority task before returning to the lower priority task

- The number of tasks can be preempt each other is limited.

The first requirement ensures that all the tasks operate in the same stack properly. The second requirement can control the memory usage by stack allocation.

5.1.3.1 Task priority and task state

To control the level number of preemption, we use the concept of Non-Preemption Group [13]. This is a collection of tasks that are not permitted to preempt each other. Since this concept is applied for the micro-threads, each thread can have two kinds of priorities. Base (Ready) priority is the priority that is used to enqueue a thread in the ready queue. Dispatch priority is the priority which is assigned to a thread when it becomes a running thread. This is the highest priority in the Non-Preemption Group that the thread belongs to. This priority ensures that the thread will not be preempted by the other threads that share the same Non- Preemption Group with it. It is also the key to control the stack usage level in the system.

These definitions have been used in SSX kernel and achieved success in reducing memory usage in the system. But, if tasks runs interpolately and tasks are non-completion, this approach will be failure. We inherited these concepts and extended this so that both the micro- thread and the generic thread can be scheduled in the system. The failure of interpolated and non-completion tasks can be overcome.

Since each generic thread has its own stack, so, each thread’s priority can be considered as a preemption level. As the result, for the generic thread, we have only the concept of Base priority.

5.1.3.2 State Transition diagram

While being scheduled, each task in the system can have the following states:

- READY: task is in the ready queue - RUNNING: task is running

- STACKED: task is preempted, task’s context is stored in the stack

We also define two other dummy states in the scheduler:

- BEGIN: the task is created - COMPLETE: task completes

The state transition diagram is shown in Figure 17.

Figure 17. State transition diagram

With this algorithm, the total stack usage for the system can be calculated as the follows:

Total stack usage = Max stack size for each preemption level + (Number of preemption level * size of context) + interrupt stack usage

The reduction of stack size by using two groups of threads will be illustrated by the

following example. Assume that we have two systems S1, S2 (EOS) with each system’s stack

usage as follow. In Figure 18, we compare the stack size of two systems. The number of

threads in both systems increases from 1 to 8. The stack size allocated for 1 thread is the same

in two systems, which ranges from 64 Bytes to 128 Bytes. S1 has one type of threads which

have their own separate stack. EOS uses two types of threads with the concept of Non-

preemption Group as described before. We apply the concept of Non-preemption Group when

the system has more than 3 threads. Our system is still as flexible as any other systems in

which multi-threads can run alternately. Furthermore, the stack usage is more efficient which

is shown in Figure 18.

We compare the stack usage of the two systems in the following figure.

0 100 200 300 400 500 600 700 800

1 2 3 4 5 6 7 8

EOS S1

# of threads

Figure 18. Comparison of stack usage

5.1.4 Thread Synchronization

When the system has multiple threads run concurrently, these threads may access and

manipulate shared data. Race condition can be occurred and the final value of shared data

depends upon which process finishes last. To prevent race conditions, concurrent threads must

be synchronized. In addition, threads sometimes should have to wait for a resource to become

available. At that time, they need synchronization. We provide two methods for thread synchronization: mutex (binary semaphore) and (counting) semaphore. Although mutex and semaphore have some different features, they share the same functions and requirements. They are to provide a synchronization method for concurrent threads to prevent race condition in the system and to wait for a resource to become available. The data structure and APIs for thread synchronization are presented in the following sections.

5.1.4.1 Mutex

Mutex module provides APIs so that an application developer can use when his thread needs to enter a critical section. Mutex is a kind of binary semaphore. This module also provides wait function for threads to wait for a resource. A thread which wants to wait on a mutex also has two statuses: LOCKED and UNLOCKED. These statuses are defined in an enum variable as follows:

/**

* Thread lock status */

enum Lock { LOCKED, UNLOCKED };

A mutex data structure has three elements:

• lock : Lock status of this mutex

• owner: The pointer to the owner thread of this mutex

• wt_q: List of threads waiting on this mutex We define the Mutex data structure as follows:

typedef struct _MUTEX {

enum Lock lock; /* Lock status of this mutex */

pthread_tcb_t *owner; /* Pointer to the owner of this mutex */

list_t wt_q; /* List of threads waiting on this mutex */

}MUTEX;

The basic algorithms for mutex lock and mutex unlock are described as follows:

Mutex lock

1. Disable interrupts

2. If (lock == LOCKED) then

- add the current thread to the waiting queue of this mutex - enable interrupts

Else

- the current thread becomes the owner of this mutex - lock = LOCKED

- enable interrupts

Mutex unlock 1. Disable interrupts

2. If the current thread is the owner of this mutex then - enable interrupts

- return

3. Make the waiting thread ready and enable interrupts 4. If no thread is waiting then

- unlock the mutex - clear the owner - enable interrupts

The Mutex APIs are summarized in the following table:

Mutex API Prototypes Description

void mutex_init(MUTEX *mt); Init a mutex data structure

void mutex_lock(MUTEX *mt); Block the thread if the mutex is already locked void mutex_unlock(MUTEX *mt); Unlock a mutex so that another thread can enter

the critical section int8_t mutex_try_lock(MUTEX *mt); Take the lock if its free

Table 6. Mutex APIs

5.1.4.2 Semaphore

Semaphore module provides APIs so that an application developer can use when his thread needs to enter a critical section.The number of threads to wait on this semaphore depends on its initial value when it is initialized. This module also provides wait (sem_wait) function and test function (sem_try_wait) to test on a semaphore. For more details about semaphore APIs, refer to the table bellows.

A semaphore data structure can have the following elements:

• sem_val: the current semaphore count. This is the maximum number of threads which can wait on this semaphore.

• wt_q: list of threads waiting on the semaphore

typedef struct _SEM {

int8_t sem_val; // Current semaphore count

list_t wt_q; // List of threads waiting on the semaphore } SEM;

The basic algorithms for sem_wait, sem_try_wait and sem_post are described as

follows:

Sem_wait

1. Disable interrupts

2. Check if no resource is available, the current thread has to wait in the queue 3. Otherwise, enable interrupts

Sem_try_wait

1. Disable interrupts

2. Test the semaphore: If (sem_val > 0) then decrease sem_val 3. Enable interrupts

4. Otherwise, enable interrupts and return

Sem_post

1. Disable interrupts

2. Post a unit and wake up the next waiting thread 3. Enable interrupts

4. If no waiting thread, then enable interrupts and return.

The Semaphore APIs are summarized in Table 7 as follows.

Semaphore API prototypes Description

void sem_init(SEM *s, int sem_val); Init semaphore with an initial value void sem_post(SEM *s); Post semaphore s

void sem_wait(SEM *s); Wait on a semaphore

uint8_t sem_try_wait(SEM *s); Test the semaphore and decrease if possible, otherwise return

Table 7. Semaphore APIs

5.2 Message Handling

The general architecture of Message Handling Engine (MHE) with the details in

message queue structure is shown in Figure 2. The Message Send/Dispatch Module (MSDM)

receives requests from user threads. If the request is for inter-thread communication, the

message will be forwarded to the ITC message queue; otherwise, the data is packed into a

packet through the network stack and then, sent to the network destined to the target.

Figure 19. Message Handling Engine

5.2.1 Inter-Thread Communication (ITC)

The basic idea of the Inter-Thread Communication in EOS is to use a message queue to store data exchanged among tasks. The message queue is implemented as a circular buffer. The way a message is inserted to the queue and then, retrieved, depends on the adopted policy.

Parameters specifying a message queue are defined in the Message Queue Control Block.

Message pointer is always extracted from the entry pointed by itc_msgq_busy. Message pointer is always deposited in the queue from the entry pointed by itc_msgq_free. The pointer itc_msgq_start and itc_msgq_end are simply markers used to establish the beginning and the end of the array so that itc_msgq_busy and itc_msgq_free can wrap around to implement this circular motion. The structure of a Message queue is shown in Figure 20 We have the data structure for message queue control block defined as follows:

typedef struct _itc_msgq_cb_ {

BYTE *itc_msgq_start; /* Pointer to start of queue data */

BYTE *itc_msgq_end; /* Pointer to end of queue data */

BYTE *itc_msgq_busy;

/* Pointer to first busy element in the queue ---> Retrieve data from this element */

BYTE *itc_msgq_free;

/* Pointer to first free element in the queue ---> Insert data from this element */

uint8_t itc_msgq_size; /* Maximum number of entries */

uint8_t itc_msgq_entries; /* Current number of entries */

list_t itc_msgq_pendq;

/* List of threads pending data in this queue */

} itc_msgq_cb;

Figure 20. Message Queue Structure

A message queue needs to be created before it can be used. The content of a message queue is several pointers to some messages. What the pointer points to is application specific.

A message queue can only contain one or a number of pointers (the number of pointers is decided when message queue is created) or a NULL pointer (message queue is empty). Figure 21 shows the diagram to illustrate the relationship between tasks and a message queue.

Figure 21. Relationship between tasks and a message queue

The basic algorithms to send, receive and pend a message are described as follows.

Sending a message

1. Validate message queue id

2. If the message queue is FULL then return 3. Else

Disable interrupts

Assign the current itc_msgq_free to the address of the message Increase itc_msgq _free and itc_msgq _entries

If itc_msgq_free== itc_msgq_end then itc_msgq_ free = itc_msgq_ start See if any tasks pending on the queue à make them ready

Enable interrupts 4. Return

Receiving a message

1. Validate message queue id 2. Disable interrupts

3. If there is a message in message queue then a. If P2P == TRUE then

i. Extract it

ii. Increase itc_msgq_busy iii. Decrease itc_msgq_entries

iv. If itc_msgq_free== itc_msgq_end then itc_msgq free = itc_msgq_ start v. Clear the message in the queue

b. Else Extract the message

c. Enable interrupts and return a pointer to that message 4. Else Enable interrupts and return a NULL pointer.

Pending a message

1. Validate message queue id 2. Disable interrupts

3. If there is a message in the message queue then a. Extract the message

b. Increase itc_msgq_busy c. Decrease itc_msgq_entries d. Clear data

e. Enable interrupts f. Return

4. Else, suspend the current task to wait for a message and enable interrupts 5. Return a NULL pointer

The APIs for Inter-Thread Communication are summarized in Table 8 as follows.

API prototypes Description void _init_itc(void); Initialize message queue module Void *itc_msgq_accept(uint8_t itc_msgq_id,

uint8_t len, BOOL P2P,uint8_t *err );

Accept message from queue

uint8_t itc_msgq_create(BYTE *start, uint8_t size );

Create a message queue

uint8_t itc_msgq_query( uint8_t itc_msgq_id);

Query the status of message queue

uint8_t itc_msgq_delete( uint8_t itc_msgq_id, uint8_t opt );

Delete a message queue

uint8_t itc_msgq_post( uint8_t itc_msgq_id, uint8_t len, void *msg);

Post a message to a message queue

void *itc_msgq_pend( uint8_t

itc_msgq_id, uint8_t len, uint8_t *err);

Pend on a message in message queue

Table 8. ITC APIs

5.2.2 Remote-Thread Communication (RTC)

Communication in sensor network is different from those of wired and wireless traditional networks. Besides the requirements of robustness, energy efficiency, etc. as mentioned in section 3, the design of communication model for the sensor networks should has the following features:

§ Data centric: A sensor node does not have a global id. Applications in the sensor network are interested in the data sensed or acquired by a sensor node rather than the sensor node itself. They pose questions like “Which sensor nodes have the sensor over 20 C?” rather than asking questions like “Where is the sensor 14?”

§ Application specific: In traditional network (example: the Internet), many applications are deployed in the same network infrastructure. But in the sensor networks, it is uncontroversial that each sensor node performs a specific job or caches a specific type of data that contributes to the global application of the network.

§ Collaborative communications: Sensor nodes need collaborative communications to compensate for the limited capacity and reliability of each sensor.

So, what are the best programming abstraction and the communication model suitable for the WSN? For EOS, we adopt three communication models:

§ Base station Centered Communication (BCC): Base station (BS) can flood

message(s) to all the sensor nodes or send message(s) to an arbitrary node in the

network. Also, sensor nodes can send message(s) to BS.

§ Publish / Subscribe Communication (PSC): BS or an arbitrary node can be either a publisher or a subscriber of specific message.

§ Collaboration / Group Communication (CGC): A group of sensor nodes can collaborate using this communication model.

5.3 Network Stack

The network stack is implemented as a pluggable module, which can be updated or changed easily. It supports three kinds of communication models as discussed. Currently, a tree-based topology on top of IEEE 802.15.4 PHY/MAC is used for routing. For BCC, the base station can broadcast the message to all nodes. To send the message to an arbitrary node in the network, the base station sends the message with the list of intermediate nodes on the way to the destination. From sensor node backward to the base station, each node just uses the parent node to reach the base station. For PSC, nodes send the advertisement and subscription message to the base station. The base station then forwards the published data to appropriate subscribers. For CGC, communication between nodes is performed through the base station.

First, the sender sends the message to the base station. The base station then, forwards the message to the corresponding receivers.

BS broadcasts to all nodes BS broadcasts to an arbitrary node

Node to node communication for

PSC and CGC

Figure 22. Communication Models

5.4 Power Management

To increase the lifetime of a sensor node, the power management module of EOS

monitors the behaviors of threads and other parts of the kernel, and determines whether

on a sensor node stop to run, power management module is activated to transfer the sensor node to sleep state. As soon as the power management module is invoked, it gathers the wake up sources which each thread wait to wake from sleep state such as timer expiration, or network packet arrival. Under performing such procedure, the power management module selects one sleep mode which meets conditions that include wake up sources at that time and consume lowest power among several provided sleep modes. In the end, the sensor node transfers into selected sleep mode. And it stays on that mode until one of expected interrupts is raised. The Six Sleep modes to be used in ATMEGA128L are present in Table 9 as follows.

Table 9. ATMEGA128L Six Sleep modes [16]

5.5 Time Synchronization

The time synchronization module has been implemented using “Sender-Receiver”

method. The synchronization procedure is initiated by the base station. And we let the RF

module timestamp just before passing the packet to the RF chip for the sake of the more

precise time stamping. After base station initiates a synchronization procedure, the children in

the next level exchange two time-stamped packets with the base station. Then the child can

calculate the own offset with exchanged packets, correct the own local clock. In the meantime

the children of the next level can overhear its own parent’s radio activity. Thus after detecting

the parent’s synchronization activity, the children can start to exchange time-stamped packets

with the own parent. Whenever any node requests to synchronize with its own parent, it should