We are using Raspberry Pi 3 Model B as the IoT gateway connected to dth 1125tempera-
ture and humidity sensor in our implementation. The fog layer is constructed using three docker swarms with three Raspberry Pi 3 Model B in each swarm. All Raspberry Pi devices are build using raspbian26 operating system. We are using ubuntu 16.04 LTS as
our edge layer device whereas IBM cloud functions is used as the cloud layer. For the database specific to our demonstration, we are using MongoDB 3.6 in high-availability. Our serverless function is collecting temperature and humidity values from the dth 11 sensor and writing the data to the database. In a realistic scenario, the function is writing data in milliseconds to the database. However, we successfully tested function offloading mechanism by changing the defined threshold values. We use python 3.5 to implement the design explained in Section 4.2.3. A few of the supporting functions are explained in the rest of this section. These functions are essential for the implementation of the proposed architecture.
• threshold_values() defines the threshold value for system resources. We use python psutil27 package to determine the current CPU, memory and power utilisation at IoT, fog and edge devices. Our threshold value for memory and CPU consumption is 80% each and 30% for remaining battery.
• multicast_sender() sends multicast requests over a defined multicast IP and port. • whitelister() function takes a Function bucket in form python dictionary as with an
input and removes a function name if condition satisfies. 25https://learn.adafruit.com/dht/overview
26https://www.raspberrypi.org/downloads/raspbian/ 27https://pypi.org/project/psutil/3.3.0/
• serverless_executor() function is responsible for triggering the serverless function at each layer. It can be achieved by accessing an OpenFaaS function using the URL. This function also takes care of metrics evaluation, function offloading from one layer to another.
Our approach gives end-users flexibility of writing custom functions suitable to their requirements.
5
Evaluation
This chapter evaluates and analyses the proposed solution based on different evaluation criteria. In this chapter, We describe the evaluation criteria and their applicability to our solution.
5.1
Methodology
The architecture introduced in Section 4.1 is evaluated based on seven desired character- istics. We defined these characteristics considering several aspects of an ideal solution such as secure communication, work with limited system resources and heterogeneous devices. Additionally, we also evaluate function execution offloading by comparing the execution timeout in different conditions. The characteristics considered in this work are as detailed next:
5.1.1 Evaluation criteria
• Ease of deployment: The deployment of the proposed architecture should be smooth and easy. An ideal deployment minimizes manual installation efforts by automating the related procedure.
• High availability: The solution is expected to deploy across the number of IoT devices and multiple cloud computing layers such as fog, edge and cloud. It is extremely critical for the desired solution to guarantee essential services with minimum downtime.
• Fault tolerance: Fault tolerance is one of the key properties considering the vulnerable nature of IoT devices. IoT devices are more susceptible to component failure. The solution should trigger immediate recovery from any failure.
• Device Heterogeneity: The solution should effectively abstract the heterogeneity of IoT devices. Developers often like to use multiple programming languages depends on the scenario. Our solution should provide function runtime heterogeneity by allowing the development in desired programing language to developers.
• Scalability: We can not predict the rate of function execution in advance which leads to the requirement of a highly scalable solution that can scale up and down instantly based on the requirement.
• Security and Privacy: Security and privacy are arguably one the most critical require- ment of any solution. An ideal architecture should extend the secure function execution environment to developers along with secure internal communication among internal components.
5.2
Analysis
This section presents the detailed analysis of the proposed solution from criteria defined in Section 5.1.1:
Ease of deployment
OpenFaaS is an easy platform to deploy in IoT networks. The only pre-requite for OpenFaaS is Docker. It is feasible to deploy Docker on many devices such as Raspberry Pi and Arduino. A framework such as resin.io28make it simple to deploy docker across the heterogeneous IoT networks [30]. Any device with Docker and Internet connection can be used to deploy the OpenFaaS framework [27]. The platform has specific installation instruction for ARM-based devices. The whole OpenFaaS installation at Raspberry Pi 3 takes approximately 55MB on the device’s memory.
High availability
Devices at the fog and edge layers are divided into more than one Docker swarms. Docker swarm based deployment provides two levels of high availability. The first level of high-availability comes from the multiple nodes in the Docker swarms. More than one number of swarms are responsible for the second level of high- availability. For instance, if one node is not responding within a swarm, the manager delegates the task to another node. Similarly, if one of the swarm is not responding, the offloading mechanism will forward the task to another available swarm. Figure 13 shows the high availability of the proposed architecture. In addition to the above mentioned high-availability scenarios, our architecture has the capability of offloading the function execution to the next layer of the current layer is not available. Figure 14 shows the high available multi-layered architecture. High availability at the IoT-gateway entirely depends upon the presence of the redundant gateway. Cloud providers implement the cloud layer’s high-availability. Fault tolerance
The proposed architecture is susceptible to failure like any other system. However, Docker swarm-based deployment makes sure to initiate new container upon failure to maintain the desired state. Docker recommends using an odd number of swarm managers to handle the failure of one or more manager nodes. An odd number of swarm manager nodes allows a Docker swarm to remain functional in case of the failure. Moreover, fault tolerance can be increased by distributing swarm manager nodes to different IoT devices. In such scenario, Failure of an IoT device does not fail multiple manager nodes, i.e. swarm failure.
Figure 13. High availability within the layer
Device Heterogeneity
The architecture supports heterogenous IoT devices that have different characteristics such as processor architecture, resources, memory and physical location. However, Dis- tributed computing framework can be deployed for heterogeneous devices using Docker containers [31]. It is also feasible to create a Docker swarm over multiple IoT devices to deploy OpenFaaS. The proposed solution uses OpenFaaS as the underlying serverless platform that supports many runtime environments such as Python, Javascript, Nodejs and Docker. Furthermore, the solution supports both synchronous and asynchronous functions.
Scalability
The proposed architecture uses the auto-scaling feature implemented in OpenFaaS. The openfaas architecture includes function auto-scaling by scaling up or down the function deployment on-demand. OpenFaaS accomplishes the auto-scaling using Prometheus matrices. penFaaS has a defined value of min and max replicas. By default, the minimum possible number of replicas is 1 whereas, one can launch maximum 20 replicas while scaling up. It is also possible to disable auto-scaling by setting same value for min and max replicas or setting com.openfaas.scale.factor=0. The scaling factor defines the number of replicas initiated upon the alarm is generated and is set to 20% by default. We have explained auto-scaling in detail in Section 3.3.2.
Figure 14. High availability across layers
Security and Privacy
OpenFaaS allows us to implement basic authentication (username/password authentica- tion) while deploying functions. Furthermore, a certificate-based HTTP authentication can also be enabled for the functions [32]. However, the security mechanism also de- pends upon the capability of the IoT device. The proposed solution also incorporates the built-in security features of the Docker implementation.
5.2.1 Offloading Analysis
from pymongo import MongoClient
from datetime import datetime
import json
def handle (req):
client = MongoClient (" mongodb ://10.10.10.10:27017 ")
db = client . test
json_req = json . loads (req)
db_entry = {" Humidity ": json_req [" Humidity "],
" Temperature_farenheit ": json_req [" Temperature_farenheit "],
" Temperature_celsius ": json_req [" Temperature_celsius "],
" Latitude ": json_req [" Latitude "],
" Longitude ": json_req [" Longitude "],
" Time ": datetime .now ()} db. sensordata . insert ( db_entry )
return {" statusCode ": "200"}
Listing 17. Mongowriter OpenFaaS function
We evaluated our proposed solution by analysing the function execution offloading at the different layers. The exec_timeout for each layer are presented in the Table 3. We used the setup described in section 4.3 that consisted of Raspberry Pi 3 as part of Docker swarm at IoT-Gateway and edge layer. We used dth 11, Ubuntu 16.04 LTS machine and IBM cloud functions for sensor, fog and cloud layers respectively. Listing 17 consists of a synchronous OpenFaaS function that takes values from the sensor and writes them to the MongoDB database. This function was deployed across all layers. We performed 10 iterations for each of the cases mentioned in Table 5 and 6 to calculate the average value and the standard deviation.
Case Executed at 1st execution(time taken in seconds) 2nd execution(time taken in seconds) IoT-Gateway timeout Edge Layer 22.1764 2.1617
IoT-Gateway and Edge timeout Fog Layer 58.3487 1.8188 IoT-Gateway, Edge and Fog Timeout Cloud Layer 99.8052 1.8126
Table 4. Function execution time in case of threshold timeout
Table 5 shows the function execution time in case of threshold timeout at different layers. The function takes higher time for the first execution as it runs for whole exec_timeoutat each layer. Our solution also creates a bucket entry at each layer that allows the function to skip exec_timeout and results in low function execution time values during second execution. we have a low value of standard deviation in each case as he time taken during each iteration does not differ significantly. Figure 15 demonstrates the difference in the time taken for the first and second execution in each layer.
Case Executed at First execution(standard deviation in seconds) Second execution(standard deviation in seconds) IoT-Gateway timeout Edge Layer 0.0449 0.0469
IoT-Gateway and Edge timeout Fog Layer 0.2801 0.0755 IoT-Gateway, Edge and Fog Timeout Cloud Layer 0.1179 0.1341
Figure 15. Comparison of time taken during first and second execution on timeout
Table 6 contains the time taken for the function execution and standard deviation when lower layers are unavailable. Our solution checks the availability of the devices at each layer before offloading function execution to the layer above. As previously discussed, the primary reason for comparatively higher values of execution time is due to the time taken in the second execution in case of timeout. Figure 16 shows the time taken at different layers upon function execution offloading. Our proposed solution demonstrate the successful deployment of a serverless platform for the Internet of Things.
Case Executed at Execution time (in seconds) Standard deviation (in seconds) All layers are functional IoT-Gateway 4.5269 0.0575
IoT gateway is unavailable Edge layer 2.1860 0.0417 IoT-Gateway and Edge are unavailable Fog layer 12.3600 0.2232 IoT-Gateway, Edge and Fog are unavailable Cloud layer 7.3726 0.2833
6
Conclusion
This work demonstrated an implementation of serverless computing in an IoT network. We considered the resource constrained nature and heterogeneity of IoT devices in our solution. Devices with different attributes such as processing capability, memory, battery, sensors were set up in a cluster using a Docker-based orchestration mechanism called Docker swarm. Our work also inherits some beneficial properties of Docker swarm such as fault-tolerance and high availability.
In particular, we constructed a multi-layered architecture for our solution with layers such as IoT-Gateway, fog, edge and cloud layer. Available open-source serverless platforms such as Apache OpenWhisk, OpenFaaS, Kubeless and Fission were surveyed. Upon careful evaluation, we selected OpenFaaS at IoT-Gateway, fog, edge layers due to its ease of deployment on arm architecture-based devices and flexibility. OpenFaaS was also preferred over Kubeless as latter can only be deployed using kubernetes. The architecture used IBM cloud functions as cloud layer due to its cost-effectiveness. IBM cloud functions is an IBM proprietary version of Apache OpenWhisk.
An algorithm was designed for the function execution offloading among different layers of our architecture. The maximum function execution time was defined at each layer depending upon the resources availability. The offloading decision was made based on the availability of active nodes in the Docker swarm. Functions were written in python and deployed at each layer by using the OpenFaaS command line interface.
The solution later evaluated on various factors such as ease of deployment, high availability, fault-tolerance, device and function heterogeneity, security and privacy. We also compare the function execution time at various layers of proposed solution. We showed that it is possible to successfully deploy a serverless platform on IoT devices and use it to perform various tasks with the help of serverless functions. The concepts presented in this work can be expanded, and further research can be conducted. In this context, some interesting directions for future work are the following:
1. Defining the trigger alarms for function execution offloading specific to our ar- chitecture. At the moment, the architecture uses features built into to offload a function from one layer to another.
2. An implementation using kubernetes as this solution heavily relies on the Docker swarm.
3. Implementing a logging server along with monitoring functionality for the pro- posed architecture. This solution relies on the OpenFaaS functionalities that do not take offloading mechanism into account.
4. An authentication mechanism for function deployment and execution separated from OpenFaaS authentication.
5. Implementation of secure offloading of function execution from one layer to another.
References
[1] Janet Ellen Abbate. From arpanet to internet: A history of arpa -sponsored
computer networks, 1966–1988, 1994. https://repository.upenn.edu/
dissertations/AAI9503730/. Accessed 31 July 2018.
[2] A. L. Russell. The internet that wasn’t. IEEE Spectrum, 50(8):39–43, August 2013.
[3] Cisco. Cisco visual networking index, 2017. https://www.
cisco.com/c/en/us/solutions/collateral/service-provider/
visual-networking-index-vni/complete-white-paper-c11-481360.html. Accessed 31 July 2018.
[4] Michael Armbrust, Armando Fox, Rean Griffith, Anthony D. Joseph, Randy Katz, Andy Konwinski, Gunho Lee, David Patterson, Ariel Rabkin, Ion Stoica, and Matei Zaharia. A view of cloud computing. Commun. ACM, 53(4):50–58, April 2010. [5] Yury Izrailevsky. Completing the netflix cloud migration, 2016. Completing
the Netflix Cloud Migration :https://media.netflix.com/en/company-blog/
completing-the-netflix-cloud-migration. Accessed 31 July 2018.
[6] Cisco. Internet of things, 2016. Internet of Things - CISCO :
https://www.cisco.com/c/dam/en/us/products/collateral/se/
internet-of-things/at-a-glance-c45-731471.pdf. Accessed 24 June 2018.
[7] Jayavardhana Gubbi, Rajkumar Buyya, Slaven Marusic, and Marimuthu Palaniswami. Internet of things (iot): A vision, architectural elements, and fu- ture directions. Future Generation Computer Systems, 29(7):1645 – 1660, 2013. [8] S. Deshmukh and R. Shah. Computation offloading frameworks in mobile cloud
computing : a survey. In 2016 IEEE International Conference on Current Trends in Advanced Computing (ICCTAC), pages 1–5, March 2016.
[9] S. Singh and N. Singh. Internet of things (iot): Security challenges, business opportunities amp; reference architecture for e-commerce. In 2015 International Conference on Green Computing and Internet of Things (ICGCIoT), pages 1577– 1581, Oct 2015.
[10] M. H. Miraz, M. Ali, P. S. Excell, and R. Picking. A review on internet of things (iot), internet of everything (ioe) and internet of nano things (iont). In 2015 Internet Technologies and Applications (ITA), pages 219–224, Sept 2015.
[11] University of Maryland website. Don’t we all need arms, 2016. WWW
page of the cs.umd.edu:https://www.cs.umd.edu/~meesh/cmsc411/website/
proj01/arm/. Accessed 04 Aug 2018.
[12] A. B. S., H. M.J., J. P. Martin, S. Cherian, and Y. Sastri. System performance evaluation of para virtualization, container virtualization, and full virtualization using xen, openvz, and xenserver. In 2014 Fourth International Conference on Advances in Computing and Communications, pages 247–250, Aug 2014.
[13] Docker.com. What is docker. WWW page of the www.docker.com: https:
//www.docker.com/what-docker.
[14] A. Tosatto, P. Ruiu, and A. Attanasio. Container-based orchestration in cloud: State of the art and challenges. In 2015 Ninth International Conference on Complex, Intelligent, and Software Intensive Systems, pages 70–75, July 2015.
[15] Docker.com. Manage swarm security with public key infrastructure (pki).
WWW page of the www.Docker.com: https://docs.docker.com/engine/
swarm/how-swarm-mode-works/pki/. Accessed 11 May 2018.
[16] Docker.com. Administer and maintain a swarm of docker engines. WWW page
of the www.Docker.com: https://docs.docker.com/engine/swarm/admin_
guide/. Accessed 11 May 2018.
[17] Docker.com. Raft consensus in swarm mode. WWW page of the www.Docker.com: https://docs.docker.com/engine/swarm/raft/. Accessed 11 May 2018.
[18] Docker.com. Manage swarm service networks. WWW page of
the www.Docker.com: https://docs.docker.com/v17.09/engine/swarm/
networking/. Accessed 11 May 2018.
[19] Mike Roberts. Serverless architectures, 2016. WWW page of the martinfowler.com: https://martinfowler.com/articles/serverless.html. Accessed 14 May 2018.
[20] T. Lynn, P. Rosati, A. Lejeune, and V. Emeakaroha. A preliminary review of enterprise serverless cloud computing (function-as-a-service) platforms. In 2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom), pages 162–169, Dec 2017.
[21] Openwhisk. Git repository of Apache OpenWhisk project:https://github.com/ apache/incubator-openwhisk. Accessed 11 May 2018.
[22] Openwhisk cli. Git repository of Apache OpenWhisk CLI:https://github.com/ apache/incubator-openwhisk-cli. Accessed 21 May 2018.
[23] Michael Mendenhall and Buell Duncan. Bluemix is now ibm cloud, 2017. WWW
page of the IBM cloud blogs:https://www.ibm.com/blogs/bluemix/2017/10/
bluemix-is-now-ibm-cloud/. Accessed 11 May 2018.
[24] Openfaas. Git repository of OpenFaaS: https://github.com/openfaas/faas.
Accessed 11 May 2018.
[25] Deployment guide for docker swarm. Git repository of OpenFaaS:https://docs. openfaas.com/deployment/docker-swarm/. Accessed 11 May 2018.
[26] Alex Ellis. Alert rules, 2017. Git repository of OpenFaaS: https://github. com/openfaas/faas/blob/master/prometheus/alert.rules.yml. Accessed 22 June 2018.
[27] Docker comes to raspberry pi. WWW Raspberrypi blog: https://www.
raspberrypi.org/blog/docker-comes-to-raspberry-pi/. Accessed 14 May 2018.
[28] Kubernetes on (vanilla) raspbian lite. Git repository of alexellis: https://gist. github.com/alexellis/fdbc90de7691a1b9edb545c17da2d975. Accessed 14 May 2018.
[29] Alex Ellis. Your serverless raspberry pi cluster with docker,
2017. WWW Alex Ellis’ Blog: https://blog.alexellis.io/
your-serverless-raspberry-pi-cluster/. Accessed 14 May 2018.
[30] Y. Gao, H. Wang, and X. Huang. Applying docker swarm cluster into software defined internet of things. In 2016 8th International Conference on Information Technology in Medicine and Education (ITME), pages 445–449, Dec 2016.
[31] Daniel José Bruzual Balzan. Distributed computing framework based on soft- ware containers for heterogeneous embedded devices. Master’s thesis, Depart- ment of Computer Science and Engineering, Aalto University School of Science
and Technology, Espoo, Finland, 2017. https://aaltodoc.aalto.fi/handle/
123456789/28568?show=full.
[32] Alex Ellis. Lock-down openfaas for the public internet, 2017. WWW Alex Ellis’
Blog:https://blog.alexellis.io/lock-down-openfaas/. Accessed 29 May
Appendix
II. Licence
Non-exclusive licence to reproduce thesis and make thesis public
I, Manoj Kumar,
1. herewith grant the University of Tartu a free permit (non-exclusive licence) to: 1.1 reproduce, for the purpose of preservation and making available to the public,
including for addition to the DSpace digital archives until expiry of the term of validity of the copyright, and
1.2 make available to the public via the web environment of the University of Tartu, including via the DSpace digital archives until expiry of the term of validity of the copyright,
of my thesis Serverless computing for the Internet of Things supervised by Mario Di Francesco and Satish Narayana Srirama 2. I am aware of the fact that the author retains these rights.
3. I certify that granting the non-exclusive licence does not infringe the intellectual