Security architecture for Fog-To-Cloud continuum system

(1)

Security architecture for

Fog-To-Cloud continuum system

Sarang Kahvazadeh

ADVERTIMENT La consulta d’aquesta tesi queda condicionada a l’acceptació de les següents condicions d'ús: La difusió d’aquesta tesi per mitjà del r e p o s i t o r i i n s t i t u c i o n a l UPCommons (http://upcommons.upc.edu/tesis) i el repositori cooperatiu TDX ( h t t p : / / w w w . t d x . c a t / ) ha estat autoritzada pels titulars dels drets de propietat intel·lectual

únicament per a usos privats emmarcats en activitats d’investigació i docència. No s’autoritza la seva reproducció amb finalitats de lucre ni la seva difusió i posada a disposició des d’un lloc aliè al servei UPCommons o TDX. No s’autoritza la presentació del seu contingut en una finestra o marc aliè a UPCommons (framing). Aquesta reserva de drets afecta tant al resum de presentació de la tesi com als seus continguts. En la utilització o cita de parts de la tesi és obligat indicar el nom de la persona autora.

ADVERTENCIA La consulta de esta tesis queda condicionada a la aceptación de las siguientes condiciones de uso: La difusión de esta tesis por medio del repositorio institucional UPCommons (http://upcommons.upc.edu/tesis) y el repositorio cooperativo TDR (http://www.tdx.cat/?locale- attribute=es) ha sido autorizada por los titulares de los derechos de propiedad intelectual

únicamente para usos privados enmarcados en actividades de investigación y docencia. No se autoriza su reproducción con finalidades de lucro ni su difusión y puesta a disposición desde un sitio ajeno al servicio UPCommons No se autoriza la presentación de su contenido en una ventana o marco ajeno a UPCommons (framing). Esta reserva de derechos afecta tanto al resumen de presentación de la tesis como a sus contenidos. En la utilización o cita de partes de la tesis es obligado indicar el nombre de la persona autora.

WARNING On having consulted this thesis you’re accepting the following use conditions: Spreading this thesis by the i n s t i t u t i o n a l r e p o s i t o r y UPCommons (http://upcommons.upc.edu/tesis) and the cooperative repository TDX (http://www.tdx.cat/?locale- attribute=en) has been authorized by the titular of the intellectual property rights only for private uses placed in investigation and teaching activities. Reproduction with lucrative aims is not authorized neither its spreading nor availability from a site foreign to the UPCommons service. Introducing its content in a window or frame foreign to the UPCommons service is not authorized (framing). These rights affect to the presentation summary of the thesis as well as to its contents. In the using or citation of parts of the thesis it’s obliged to indicate the name of the author.

(2)

Security architecture for Fog-To-Cloud

continuum system

Universitat Politècnica de Catalunya

Departament d' Arquitectura de Computadors

Thesis presented in fulfilment of the requirements for the degree of Doctor for the Universitat Politècnica de Catalunya Research Group: CRAAX

PhD Student: Sarang Kahvazadeh Advisor: Xavier Masip-Bruin Co-Advisor: Eva Marín-Tordera July, 2019

(3)

i | P a g e

Acknowledgements

Firstly, I want to thank my partner, Lidia, for being always by my side on moments of happiness and difficulties, always manifesting love and patience. I want also to extend my acknowledgement to all the other members of my family, who were not physically present but have always supported me on my decision of studying abroad, especially my parents, Reza and Mahbobeh, my brother and my sister Behrang and Sara.

In addition, I thank all the friends I met in Vilanova i la geltru, for the relaxing moments and for being like a family to me during the last four years.

I thank my advisor Xavi and co-advisor Eva for the valuable tutorship and high availability for both meeting and reviewing my papers. This is further extended to my colleagues, who I have worked in collaboration since my first year in CRAAX, contributing for the publication of several ideas. I must also thank all CRAAX students. The good working environment has enabled me to go through this process as smoothly as possible.

Barcelona, July 2019

(4)

2 | P a g e

Abstract

Nowadays, by increasing the number of connected devices to Internet rapidly, cloud computing cannot handle the real-time processing. Therefore, fog computing was emerged for providing data processing, filtering, aggregating, storing, network, and computing closer to the users. Fog computing provides real-time processing with lower latency than cloud. However, fog computing did not come to compete with cloud, it comes to complete the cloud. Therefore, a hierarchical Fog-to-Cloud (F2C) continuum system was introduced. The F2C system brings the collaboration between distributed fogs and centralized cloud. In F2C systems, one of the main challenges is security. Traditional cloud as security provider is not suitable for the F2C system due to be a single-point-of-failure; and even the increasing number of devices at the edge of the network brings scalability issues. Furthermore, traditional cloud security cannot be applied to the fog devices due to their lower computational power than cloud. On the other hand, considering fog nodes as security providers for the edge of the network bri ngs Quality of Service (QoS) issues due to huge fog device’s computational power consumption by security algorithms. There are some security solutions for fog computing but they are not considering the hierarchical fog to cloud characteristics that can cause a no-secure collaboration between fog and cloud. In this thesis, the security considerations, attacks, challenges, requirements, and existing solutions are deeply analyzed and reviewed. And finally, a decoupled security architecture is proposed to provide the demanded security in hierarchical and distributed fashion with less impact on the QoS.

(5)

3 | P a g e

List of Figures:

Figure 1. F2C continuum system ... 21

Figure 2. Fog-cloud layer security ... 24

Figure 3. Device-fog layer security ... 25

Figure 4. Secure mobility ... 26

Figure 5. Most potential attacks in F2C system ... 33

Figure 6. Distributed Security Architecture ... 81

Figure 7. Distributed security architecture in F2C system ... 82

Figure 8. Number: Fog-to-Cloud communication ... 84

Figure 9. Number: Fog-to-Fog communication ... 84

Figure 10. Security Architecture in CIs ... 87

Figure 11. Security Architecture in CIs ... 87

Figure 12. Security Architecture in Smart City ... 88

Figure 13. Authentication workflow in mf2 ... 90

Figure 14. Authentication workflow in CAUs as authenticator ... 92

Figure 15. Cloud key management workflow ... 95

Figure 16. Distributed key management and authentication (DKMA) ... 96

Figure 17. Cloud key management and authentication ... 97

Figure 18. Proposed DKMA ... 97

Figure 19. Proposed Architecture ... 98

Figure 20. a) Storing edge device’ resource information workflow ... 101

Figure 21. b) Accessing edge device’s information in same fog area ... 101

Figure 22. c) Accessing edge device’s resource information from different fog areas... 102

Figure 23. Algorithm of securely storing resource information ... 103

Figure 24. Algorithm of secure retrieve of resource information ... 103

Figure 25. Embedded security architecture (ECF) ... 104

Figure 26. Decoupled transversal security architecture (DCF) ... 104

Figure 27. The ECF workflow ... 106

Figure 28. The DCF workflow ... 108

Figure 29. Smart city test-bed ... 110

Figure 30. Key distribution and authentication delay comparison... 112

Figure 31. Network Time Delay ... 112

Figure 32. Network overhead comparison (Kbytes) ... 113

Figure 33. Penetration test over authentication and TLS communication ... 115

Figure 34. Data Storing: Traditional Cloud vs CAU-based Distributed Database ... 117

Figure 35. Data Retrieving: Traditional Cloud vs CAU-based Distributed Database ... 117

Figure 36. Security topologies: a) Non secure F2C (nF2C); b) Embedded CAUs F2C (ECF); c) Decoupled CAUs F2C (DCF). ... 118

Figure 37. Service allocation time delay: (a) Service A time delay; (b) Service B time delay; (c) Service C time delay ... 120

Figure 38. Service blocking: (a) service A; (b) service B; (c) service C ... 121

(9)

7 | P a g e

(10)

8 | P a g e

List of Tables:

Table 1. Security attacks by architectural layer ... 31

Table 2. Security requirements in different layers ... 35

Table 3. Most potential security requirements in F2C ... 45

Table 4. Security challenges in F2C ... 50

Table 5. Cloud security solutions ... 64

Table 6. Fog security solutions ... 69

Table 7. IoT security solutions ... 78

Table 8. Security Architecture Advantages ... 85

Table 9. ECDSA Algorithm sign description ... 93

(11)

9 | P a g e

List of Acronyms:

F2C Fog-to-Cloud IoT Internet of Things QoS Quality of Service

NIST National Institute of Standards and Technology IaaS Infrastructure as a Service

PaaS Platform as a Service SaaS Software as a Service API Application Interface

VM Virtual Machine

DoS Denial of Service

DDoS Distributed Denial of Service

OS Operating System

SQL Structured Query Language RFID Radio Frequency Identification XML eXtensible Markup Language

CIA Confidentiality, Integrity, and Availability

ID Identification

KGC Key Generator Center CA Certificate Authority CPU Central Processing Unit TLS Transport Layer Security

RSA Ron Rivest, Adi Shamir and Leonard Adlema IP Internet Protocol

IPv4 Internet Protocol version 4 IPv6 Internet Protocol version 6 ECC Elliptic Curve Cryptography

ECDSA Elliptic Curve Digital Signature Authentication ECDH Elliptic Curve Diffie-Hellman

PKI Public Key Infrastructure PKG Private Key Generator MD5 Message Digest algorithm USB Universal Serial Bus SDI Security Device Issuer e-ID Electronic Identification

IMEI International Mobile Equipment Identification CP-ABE Ciphertext Policy Attribute Based Encryption ABS Attributed Based Signature

HABE Hierarchical Attribute Based Encryption KP-ABE Key Policy attribute Based Encryption EACF Extensible Access Control Framework

SD Software Defined

SDN Software Defined Networking

SEC Security

DTLS Datagram Transport Layer Security SHA Secure Hash Algorithm

(12)

10 | P a g e

HTTPS Hyper Text Transfer Protocol Secure NED Network Edge Device

CoAP Constrained Application Protocol HIP Host Identity Protocol

TTP Trusted Third Party

MAC Message Authentication Code

ACK Acknowledgment

CH Cluster-Head

RA Registration Authority

DNS Domain Name System

CAU Control-Area-Unit CI Critical Infrastructure CSR Certificate Signature Request

DKMA Distributed Key Management and Authentication SSL Secure Sockets Layer

AES Advanced Encryption Standard ECF Embedded CAUs in Fog nodes DCF Decoupled CAUs from Fog nodes

RP Raspberry Pi MS millisecond S Second V4 Version 4 GB Giga Byte MB Mega Byte KB Kilo Byte TB Tera Byte

RAM Random Access Memory TCP Transmission Control Protocol PC Personal Computer

SD Secure Digital GHz Gigahertz

(13)

11 | P a g e

Chapter.1 Introduction

1.1 Fog-To-Cloud system

By growing devices connected to the internet rapidly such as mobiles, tablets, sensors, actuators, etc, cloud computing [1], [2] was emerged to provide huge computational power, network and storage for processing, filtering, aggregating and storage to the huge amount of produced data by devices. However, cloud computing is located far from distributed devices, and then it cannot provide low latency and real-time processing for all distributed devices. Therefore, fog computing [3] is merged into the system for bringing cloud characteristics closers to the users and devices. Fog computing provides distributed computing, network, and storage closer to the users in a virtualized or non-virtualized environment. Fog computing facilities data filtering, processing, aggregating, and storage closer to the users. Fog devices physically can be considered as set-top-boxes, access points, routers, switches, base stations, smart phones, tablets, and etc. It is worth to mention that fog computing was not introduced to compete with cloud, fog computing is completing cloud. Therefore, a hierarchical Fog-To-Cloud (F2C) [4] computing system was introduced to bring fog and cloud into the one framework. In this hierarchical architecture, distributed fog devices can be clustered into the different layers for providing computing, network and storage. In the F2C architecture, fog devices provide initial data processing, filtering, and storage for devices and then aggregated data can be sent to the cloud for more processing and for storing.

The F2C system has hierarchical characteristics such as device might send services to the nearby fog device, if corresponding fog device has enough capacity then it provides service execution, otherwise services might be allocated in upper layers (fog devices or cloud) for service execution. The F2C system has many challenges ongoing such as fog devices discovery, resource categorization, resource allocation, service execution, security and etc. In this thesis, the security in F2C is analyzed deeply and it is proposed a security architecture for handling security in the F2C system with a hierarchical and distributed nature.

1.2 Problem statement

The traditional cloud as a centralized and distanced component provides security for the hierarchical F2C system, but it can bring issues such as be a single-point-of-failure, scalability, and quality of services issues such as time delay, etc. On the other hand, existing cloud security solutions cannot be applied into the fog devices due to their lower computational power than cloud. Even, the existing fog security solutions without considering the whole hierarchical F2C system can bring challenges and issues such as not secure coordination between the layers.

In the bottom layer of F2C, there are distributed low computational power devices that are called internet of things (IoT) [5]. IoT devices are not capable of handling their security, they rely on

(14)

12 | P a g e

other components in upper layers for security provisioning. Some existing solutions are using cloud as security provider for IoT devices, although, scalability issues might arise due to the growing number of devices and even, because cloud as centralized entity can be considered as a single-point-of-failure. On the other hand, other existing solutions are using fog devices as IoT security providers. In this case and although security provisioning uses fog device’s computational power, it might cause quality of service (QoS) degradation. Therefore, designing a security architecture for handling distributed devices in a hierarchical F2C system is a though challenge.

1.3 Thesis motivation and objective

In this thesis, the theoretical objectives are:

1- Security requirements and challenges are analyzed deeply in all layers of F2C. 2- According to the requirements, security considerations for F2C are illustrated. 3- Security attacks in all F2C layers are described and analyzed.

4- Existing security solutions for all layers in the F2C continuum are reviewed and analyzed. The Technical Objectives are:

1- A novel security architecture is designed to be adopted into the F2C system.

2- Authentication is implemented in security architecture in distributed fashion (distributed authenticators) to provide authentication with less impact on QoS.

3- Key management is implemented in security architecture in distributed fashion (distributed key managers) to handle F2C system key management securely with less impact on QoS. 4- Access control and secure distributed data storage are implemented in security architecture

for providing distributed secure data storage closer to the users.

5- Finally, the security architecture is decoupled and puts transversal to the F2C system for providing security functionalities with less impact on QoS

1.4 Thesis structure

In this section, the structure for the rest of the thesis is presented in details.

Chapter 2: Presents all layers in the combined F2C.

 Section 2.1describes cloud layer in F2C scenario and the cloud features and weaknesses points.

 Section 2.2 presents fog layer in F2C and fog features and disadvantages.  Section 2.3 describes IoT layer in F2C system.

 Section 2.4presents the whole combined layers in F2C system.

Chapter 3: Presents the most basic and potential security consideration for F2C system.

Chapter 4: Analyses most possible attacks in all different layers of F2C.  Section 4.1 illustrates the most possible attacks in cloud layer of F2C.

(15)

13 | P a g e

 Section 4.2 presents the most possible attacks in fog layers of F2C.  Section 4.3 presents the most possible attacks in IoT layer of F2C.

 Section 4.4 illustrates the most potential attacks in combined all layers of F2C.

Chapter 5: Illustrates most potential security requirements in all layers of F2C.

 Section 5.1 describes most potential security requirements in cloud layer of F2C.  Section 5.2 presents most potential security requirements in fog layers of F2C.  Section 5.3 shows most potential security requirements in IoT layer of F2C.

 Section 5.4 describes most potential security requirements in combined all layers of F2C.

Chapter 6: Describes security challenges and directions for the F2C system.

Chapter 7: Reviews and analyzes possible security solutions in all different layers of F2C.  Section 7.1 reviews and analyses solutions in cloud layer of F2C.

 Section 7.2 reviews and analyses solutions in fog layers of F2C.  Section 7.3 reviews and analyses solutions in IoT layer of F2C.

Chapter 8: Proposes a security architecture for handling authentication, key management, and access control in hierarchical F2C system.

 Section 8.1 describes the security architecture proposal.

 Section 8.2 illustrates the proposed security architecture benefits in critical infrastructures.

 Section 8.3 presents the authentication process in the security architecture.  Section 8.4 describes key management in the security architecture.

 Section 8.5 presents access control and secure distributed storage in the security architecture.

 Section 8.6 provides analysis between decoupled security architecture from F2C components versus embedded security architecture in fog devices.

(16)

14 | P a g e

Chapter.2 Fog-to-Cloud scenario

The fog-to-cloud (F2C) continuum system has different layers such as cloud, fog and edge (IoT devices). For describing the F2C system in a sophisticated way, all t he layers will be discussed and analyzed in details and finally putting all layers together to define the hierarchical F2C system.

2.1 Cloud computing

The cloud computing [1], [2], [6], [7] concept was established first by the national institute of standards and technology (NIST) [8]: Cloud computing is a model to make able ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources such as networks, storage, applications and services that can be provided and released with minimal management effort or service provider interaction. Cloud computing facilitates service provisioning to the external users by using internet technologies. The cloud computing can be considered as a collection of interconnected and virtualized computers that unified as a one computing resources based on service level agreement between services providers and consumers. Cloud computing provides access to virtualized resources such as computers, networks and storage. The cloud computing is conceptually centralized and able to handle huge volume of data processing, filtering, aggregating, storage, and network facilities for huge amount of consumer in the scalable way. In a nutshell, cloud computing provides broadband network, data center, virtualization, web technology, multitenancy, and facilitates service execution and delivery. The cloud computing provides three type of services such as:

 Infrastructure as a Service (IaaS): It is the base of the cloud which provides processing, storage, network resources, and other computational functionalities to the consumer. All the IaaS’s users can deploy arbitrary applications, any type of software, operating systems, and services that able to scale up and down dynamically and rapidly. The users and costumers have the control over operating systems, storage, deployed applications, and some part of system administration parts. IaaS is responsible for providing cloud’ virtualized environment for users and customers.

 Platform as a Service (PaaS): It provides customers and users an environment and solution to develop, test, and deploy their applications. In this case, users have no control and power over the network, servers, operating system and storage provided by cloud infrastructure. Although, the users have control over their deployed applications. In a nutshell, PaaS facilitates platform management and maintenance for the users that wish to develop, deploy, and test their applications.

 Software as a Service (SaaS): In the SaaS model, users are able to use applications running on a cloud infrastructure. The applications providers make them accessible to the users and customers through an interface. For example, the applications are available through a web browser for providing cloud services. The users and costumers have no control over

(17)

15 | P a g e

infrastructure such as network, system, servers, operating systems, storage, and even no control over the platform. This SaaS makes users and customers free from the need of installing software locally and consuming their resources.

The cloud computing can be accessible and deployable through four types of model such as public, private, community and hybrid. These type of cloud accesses can be deployed into the organizations according to their policies and management. In the following, the four type are described in details:

 Public cloud: In this type, all the resources such as computing, storage, network, virtualization, and applications are provided by third party service providers and all resources are accessible for public users over the internet. It is less expensive rather than others types for users, but it is less secure.

 Private cloud: In this type, cloud environment and infrastructure are operated merely for an organization. Private one can only be used by one organization, one company or one of its customers. In this type of cloud, data resources and applications are only accessible to specific and permitted clients. Compare to public one, private cloud is more expensive but more secure.

 Community cloud: The organizations with the same or similar requirements such as security policies and compliance consideration share this type of cloud (Community). Community cloud facilitates organizations with the same or similar requirements to share computing resources, data storage and other capacities for integrating their companies. The community cloud can be managed by organizations member or a third party. This type is less expensive for integrating companies with similar requirements rather that each one of them have private cloud.

 Hybrid cloud: It is the combination and integration of two or more above of the models (public, private, and community). All the cloud environment and infrastructure are managed and handled by mix of external and internal cloud members. Sensitive data are stored internally and takes pack up externally in the public cloud in case of system failures. In the case of insensitive data, all are public.

The most potential cloud computing characteristics, features and advantages are [1], [9], [2], [6], [7] :

a. On-demand self-services: In cloud computing, services and computing resources can be accessible and provided automatically to the users without human-service provider interaction.

b. Scalability and elasticity: Cloud resources can be scaled up and down flexible due to business dynamic, organization’s and user’s needs. Users can access current and historical data anytime easily and fast. Organization and users can easily scale up and down the cloud resources according to their policies and needs.

c. Resource pooling: cloud service providers and administration provide and supply their pool of resources for users and clients. For example, a cloud server as a standalone server might host many users and clients in its virtual environment provided by cloud administrative. d. Fast deployment: All the cloud’ users and client can get application running quickly due to

(18)

16 | P a g e

e. Accessing resources easily: The cloud services can be accessible anywhere and anytime by multiple users. Users even can easily access and make configuration on the services host by cloud.

f. Huge amount of storage: The cloud can provide almost unlimited data processing, filtering and storage with low cost. Cloud is pay-as-you-go online computing system that means user pay only for the amount of storage that they use.

g. Cost: Cloud service provider or third party can provide computing infrastructure for organization instead of they purchase all their computing infrastructures.

h. Data recovery: All the historical data from organizations or even simple user will be stored in the cloud huge data center, therefore, it can prevent any disaster in recovery.

i. Application updates: In cloud infrastructure, any software provided by the cloud will be updated to the last version automatically.

j. Facilitating collaboration and integration: In the organization, any staff according to their restriction can access, edit and share data among others. Even, by using community cloud, some organizations can integrated by their interest and policies.

k. Multitenancy: In the cloud, a centralized data center can host multiple services by multiple providers. This characteristic facilitates management and interactions between different service providers.

Cloud computing provides and facilitates computing capabilities for the organizations and users, although, this computing system has disadvantages as will mention in the below [1], [9], [2], [6], [7] :

 Centralized computing: Cloud computing conceptually is centralized. This might cause single point of failure. If the cloud being down, failed, or it is attacked; it might not work properly for the organizations. Or even if the cloud as a conceptually centralized gets an attack, it might cause a fundamental disaster because all data stored in the cloud.

 Distance: The cloud computing conceptually is far from users. This distance might cause some security attacks such as man-in-the-middle and etc. even in some case might effect on the Quality of service.

 Security: The cloud nature makes data stored accessible on the internet anywhere. This might cause that unauthorized users to get access to this sensitive data. Even though cloud has huge computational power to provide security over the data and the network, however the conceptually centralized (single point of failure) and the distance makes cloud security on the risks.

 Interoperability and portability: The lack of appropriate application interface (APIs) between clouds with different service provides, makes the data movement and applications between clouds difficult and in some case impossible. Nowadays, there are not any interoperability and adequate APIs between clouds with different service providers.

 Connectivity issues: Sometimes, cloud ‘services cannot be done due to low internet connection or due to which is called downtime. Cloud services are based on internet and sometimes connectivity might be low for getting services to be executed.

 Availability and Reliability: Cloud services must be available and reliable all the times and always. Most of the time, this feature is in a danger due to failures in connectivity, possible

(19)

17 | P a g e

attacks, conceptually distance cloud data center. In the case of any failure, cloud service provider must take adequate actions.

 Inefficient use of network bandwidth: The cloud computing is capable of doing huge volume of computation and processing for big data, although, sending all data from edge of the network to the cloud causes network traffic increasing significantly. For sending all generated data at the edge of the network to the cloud, a huge volume of network bandwidth is needed.

 Latency: The conceptually distanced cloud makes impossible real-time processing for some application such as gaming, smart homes and traffics with higher latency.

Most of the disadvantages in cloud computing can be overcome by utilizing fog computing closer to the users. In the next sub-section, fog computing will be analyzed and discussed in details.

2.2 Fog computing

The fog computing concept was introduced by cisco [3] in 2012, fog computing concept is a decentralized computing system at the edge of the network (closer to the users). The fog computing provides distributed computing, network, and storage closer to the users in a virtualized or non-virtualized environment. The fog layer can be considered as a middleware between the edge devices, users and cloud. Fog can be considered as an extension of cloud at the edge of the network that facilitates computing, network and storage in a distributed fashion and closer to the users. Fog computing facilities data filtering, processing, aggregating, and storage closer to the users. Fog devices physically can be considered as set-top-boxes, access points, routers, switches, base stations, smart phones, tablets, end devices, and etc.

Most of the mentioned cloud computing challenges were mentioned in the previous section can be overcome by fog computing features and advantages. Fog computing advantages, characteristics and features are as below [10], [11], [12], [13], [14]:

 Geographically distributed (Geo-distributed): Fog devices which provide distributed computing can be routers, cell towers, road-side units, base stations, and etc. These distributed devices can provide and facilitate geo-distributed communication, network, storage, and computing.

 Mobility: Geo-distributed nature of fog computing facilitates mobility. Some edge devices such as phones, tablets, cars with on-board units, etc. are on the move. Fog devices have inter communication, therefore, they can handle and facilitate data handover for devices on the move.

 Reduce load on the cloud: In the traditional cloud concept, all the produced data by devices must be transferred to cloud servers for aggregating, filtering, processing, and storage. Fog computing can be considered as a middleware between end devices and cloud. Fog facilitates data aggregation, filtering, processing, and storage closer to the users and then the processed, aggregated, and filtered data can be sent to the cloud for more investigation and storage. Therefore, the cloud over the cloud servers can be minimized by fog devices.

(20)

18 | P a g e

 Data computation offloading: In the traditional cloud, for uploading and sending data such as video-stream, huge bandwidth is required. The fog computing can facilitate data computation by compressing the data before sending it to the cloud.

 Low-latency: The fog computing makes service execution closer to the users at the edge of the network, therefore, compare to cloud (conceptually distanced from users) provides services with lower latency and time delay.

 Real-time processing: The fog locations at the edge of the network where is closer to the users and its distributed nature, makes possible the real-time decision making and processing. Therefore, all the users-fogs interactions occur in real-time.

 Heterogeneity: Fog computing is considered as a middleware between users and cloud in a distributed nature. All fog nodes must have inter-communication to provide resource orchestration, exchange of information, perform load balancing and finally provide heterogeneity into the system.

 Location-awareness: Fog computing is located at the edge of the network in a distributed nature, therefore, it supports mobility and can track users location in real-time.

 Scalability: Fog computing can provide distributed computing, network, and storage for the large-scale edge devices and bring scalability into the system.

 Interoperability: Fog computing is capable of working under different fog service providers. Therefore, all fog service providers have inter-communication to bring interoperability into the system.

 Cost and lower expenses: The operation done by fog computing is lower in terms of cost and expenses compare to the cloud because the process in fog is done at the edge of the network and it saves the needed network bandwidth.

 Data security and privacy: Fog computing provides computation, network, and storage closer to the users, therefore, users can have more control than cloud over their own data in terms of security and privacy.

Although, fog computing has some unsolved challenges and disadvantages such as [15], [16], [17], [18], [19]:

 Security issues: The fog computing characteristics such as its distributed nature and the fact of being closer to the users makes it more vulnerable than cloud. Even, the traditional existing security mechanism for cloud cannot be applied to the fog system due to its distributed nature and because it has lower-computational power than cloud.

 Control and management over resources: The fog nodes must provide the required resources for computation, network, storage, latency and bandwidth at the edge of the network and also ensuring the expected QoS. In some case, the inherent mobility associated to fog causes that those metrics dynamically change and even the fog environment most of the time is virtualized which causes additional latency therefore, controlling and managing resources at the dog computing is one of the main challenges and issues.

 Energy management: The distributed characteristics of fog makes energy consumption higher than centralized cloud, therefore, one of the issue is optimizing energy at the edge of the network.

(21)

19 | P a g e

 Virtualization choose: One of the main characteristics in fog nodes is the use of a virtualized environment, then choosing the best virtualized environment (hypervisor or container) is one the challenges for designing the desirable fog computing system.

 Providing low-latency: Low-latency is one the fundamental advantages of fog computing. Although, the desirable low-latency for users in some case such mobility, providing desirable resources for tasks execution, handling distributed data aggregation and processing are challengeable.

 Lower-resource fog devices: All the user devices and network management devices with computational power can be considered as fog devices. Although, these devices have lower computational power than cloud, therefore, efficient job allocation and chosen desirable policies are needed.

 Power limitation: All the devices such as smart phones, tablets, and laptops can act as fog devices, although, these devices are power-limited. Tasks which are running in these devices may not be completed due to devices can be power off because of power-limitation.

 Connectivity: Most of the fog devices at the edge of the network uses wireless technology for connectivity. Although, edge devices such as Internet of things (IoT) devices, sensors, actuators, and etc. might lose their connectivity to fog devices by increasing number of connected devices to the fog. Therefore, an estimation for fog devices connectivity capabilities must be done.

 Scalability: the huge number of edge devices generates huge amount of data at the edge of the network. The produced data needs to be processed, filtered and aggregated then it requires huge amount of resources (processing capabilities and storage). Therefore, fog devices at the edge of the network must be selected carefully for handling huge amount of generated data.

 Distributed architecture: The distributed fog nature makes it vulnerable in case of redundancy. Therefore, the framework must be designed for reducing the redundancy.

 Device’s heterogeneity: At the edge of the network, several devices are naturally heterogeneous. Therefore, the fog computing environment must consider device’s heterogeneity while developing fog applications.

As mentioned above, there are some of the issues and challenges in fog computing which can be overcome by utilizing the hierarchical F2C system. In the next section, the IoT and edge devices layer will be described. Then, finally putting all layers together, the F2C continuum system will be described and analyzed.

2.3 IoT and edge devices

The Internet of things (IoT) [5] , [20] concept is referring to make devices such as sensors, actuators, and etc. connected without human interaction. The IoT makes all devices and humans connected to each other. One of the main motivations of the IoT is to make interoperable communication and connections between devices at the edge of the network such as smart devices, sensors, PCs, cars, and etc. However, most of the IoT devices and edge devices have not enough

(22)

20 | P a g e

computational power, battery power, memory network, and storage. Therefore, fog computing (fog devices with more computational power at the edge of the network) and cloud (conceptually distanced centralized huge datacenters) can provide data processing, filtering, aggregation and storage for IoT and edge devices. There are many challenges in IoT such as real-time communication and processing, security and privacy, low-computational devices, the distributed nature of IoT, etc. which can be overcome by using collaborated Fog-to-Cloud (F2C) continuum system.

In the next section, the hierarchical F2C system considering all mentioned layers above such as cloud, fog and IoT will be discussed, analyzed and illustrated.

2.4 Fog-to-Cloud (F2C) continuum system

Putting all the layers mentioned above together and according to the fact that fog computing comes to complete cloud computing rather than compete, the Fog-to-Cloud (F2C) Computing has been proposed in [4] intended to enable a coordinated management of resources available at both fog and cloud. By means of a hierarchical layered-architecture, network controllers are deployed in a distributed fashion enabling a multi-layer resource allocation as well as the distributed and parallel service execution in fog resources, cloud, or both. Therefore, service demands are mapped into fog or cloud resources according to their suitability and availability to meet the expected QoS requirements. The integrated and combined F2C system can provide higher performance, higher energy efficiency, real-time processing, faster responding, scalability and localization for IoT and edge devices due to their distributed, hierarchical and combined characteristics.

The F2C is a hierarchical multi-layered architecture conceived to cover a broad area with plenty of computing devices. The hierarchical distributed nature of this architecture allows combining the advantages of both computing paradigms, i.e., proximity at the edge of the network by fog and high performance at the cloud, while the coordinated management of the whole system allows the feasibility of providing optimal resource allocation that meets the expected services QoS requirements. The F2C ecosystem is shown in Figure 1. The whole area of coverage is organized in fog areas, which include the set of resources (nodes) located inside that area. The exact scope of an area is a topic of current research, and affects the scalability of the system. One fog node at each area is selected to become the manager of the area for handling other fog devices and edge devices (IoT devices). The fog node as the manager is a node with certain features, such as enough computing and networking capabilities to manage its area, and good network access. The responsibilities of such fog node are managing the devices inside the area as well as coordinating with higher level layers. In this figure, the fog nodes are connected, and managed, by the Cloud layer, thus crafting the hierarchical architecture. Obviously, the Cloud layer has enough capacity to perform a higher level management of the fog nodes set.

(23)

21 | P a g e

Figure 1. F2C continuum system

Additionally, in a large scenario with millions of IoT and edge devices and spanning several squared kilometers, such architecture could increase the number of layers in order to facilitate an efficient coordination between nearby areas and, thus, becoming a multi-layered architecture. The multi-layered hierarchy guarantees the scalability of the system, as well as an efficient service management.

In this scenario, and the set of resources (nodes) organized in layers and areas, users share their resources to the F2C system, but also become F2C clients requesting the execution of services or applications. To take advantage of the execution of services in this combination of the different computing paradigms, fog, edge and high performance at cloud, it is necessary a system controlling and managing the execution of services. The outlined characteristics in the execution of a service is mentioned below:

 Launching the service: The service can be requested to the system in any node belonging to it.

 Hierarchical search of resources: If the service is requested to a specific node:

When the node is a normal node, if it has enough resources to execute the service, it will be executed in this node; otherwise the request will be forwarded to its fog node as the manager (higher layer).

If the node is a fog node as the manager, it will also check if it has enough resources, but in this cases considering the resources of all the nodes belonging to the area it is controlling. Again, if in the area there are enough resources the service will be executed in the nodes of the area; otherwise the service will be forwarded to the higher layer, in the case of Figure.1 to the cloud layer, but with more hierarchical layers to the corresponding upper layer.

(24)

22 | P a g e

 Mapping of services and resources: The previous description about the hierarchical search of resources will be based on the smartness to map services into fog or cloud resources according to their capabilities, availability, expected QoS requirements, etc.  Distributed and parallel execution: The F2C system must allow the distributed execution

of services. Services can be monolithic applications or services divided into subservices or tasks. When a service allows its division into tasks, the F2C system must perform the best division into tasks and also assign the tasks to the more suitable resources.

Moreover, this distributed execution may be also parallel in some services. Taking

advantage again of the large number of nodes, different tasks of a service can be executed in different nodes. The F2C will have a runtime controller controlling the synchronized execution of tasks.

Other main aspects of the F2C easing the distributed execution of services in this ecosystem are:  Resource discovery: Nodes can be on the move in the city, such as mobile phones. It

must exist a mechanism to mutual discover between fog node managers and normal nodes.

 Identification: Nodes participating in the system must be uniquely identified.

 Sharing model: Users sharing their devices in the system should indicate the amount of resources they want to participate (memory, storage, etc.)

 Handover: As it is mentioned, nodes can be on the move, belonging to an area, and disappearing of this area after a time. There should be a handover mechanism to reallocate tasks being executed in this on move devices.

The F2C system is bringing management coordination between fog and cloud. However, there are many challenges and issues to provide F2C system. One of the main challenges in F2C scenario is bringing security into the system. The F2C system has a hierarchical and distributed nature with the combination of the cloud huge data centers, fog devices with enough computational power and low-computational power, and finally edge and IoT devices with limited computational power. The mentioned F2C system characteristics make it so vulnerable to the attacks. There are many existing security solutions for cloud which cannot be applied in fog and IoT due to the lower computational power; and even existing fog security solutions without considering the whole combined F2C system cannot cover the security requirements of the F2C system. The motivation here is to analyze the security considerations, attacks, requirements and challenges in the existing solutions, and finally to propose a novel distributed and decoupled security architecture for F2C systems for handling authentication, key management, access control, and encryption/decryption with the desired QoS. In the next chapter, the most potential security considerations for F2C will be discussed and analyzed.

(25)

(26)

24 | P a g e

Chapter.3 Fog-to-Cloud basic security consideration

In this chapter, we analyze the basic security view in the hierarchical F2C scenario to illustrate the basic security considerations that must be applied into the F2C continuum system.

In the F2C system, there are distributed fog nodes as distributed managers to provide computation, network, and storage for other fog devices, edge and IoT devices closer to the users for their corresponding areas. The distributed fog nodes and cloud in the combined F2C system must communicate securely to avoid any passive and active attacks such as man-in-the-middle, masquerade, etc. Fog nodes need to connect to the cloud (not the fake or malicious), and in parallel, F2C cloud must connect to the trustable fog nodes (not the fake or malicious). The fog nodes are distributed in a distributed way to provide management to the nodes in its area, moreover this can be exploited to bring security in a distributed way to the F2C system. To bring security in all the fog and cloud layers some steps are needed such as it is illustrated in Figure 2.

Each fog node must be securely discovered and then perform mutual authentication with cloud by providing credentials and identities to provide system and data integrity and confidentiality. After performing authentication, cloud provides keys for fog nodes. Fog nodes can use these keys to encrypt and decrypt exchange information with cloud, for preventing attackers to eavesdropping, modifying, or deleting exchange information between cloud and fogs. The network technologies between cloud and fog nodes such as wired, wireless, etc. must be secured to avoid any passive and active attacks. It means cloud and fog nodes must exchange information in secure channels (all blue lines in the Figure 2). And the final step is that cloud acts as access control and provides access to F2C cloud and data centers for the distributed fog nodes to processing, aggregating, filtering, and storing information according to their attributes and preventing any unauthorized access.

(27)

25 | P a g e

In parallel, after fog nodes are properly installed and authenticate to the F2C cloud, they get authorization to provide computation, network, storage, and shareable computational environment to IoT devices at the edge of the network in a distributed way. In this case (Figure 3), when a device arrives to the fog area, first must be discovered by fog nodes securely. Then, fog nodes with devices must be mutually authenticated by providing credential and identities to bring data and system integrity and confidentiality at the edge of the network. After authentication, fog nodes can generate and distribute keys to devices to be used for encryption and decryption. All exchange information between devices (such as IoT devices) and fog nodes must be encrypted to prevent attackers from eavesdropping, modify, or delete the information. All information exchange must occur in secure channels for different network technologies (blue lines in Figure 3). At the end, fog nodes can act as access control for devices to prevent any unauthorized access to the F2C system at the edge of the network.

Figure 3. Device-fog layer security

At the edge (in a fog area) there are different type of devices such as fog devices, edge devices and IoT devices, managed by fog nodes as managers. These devices might be on the move. Therefore, secure inter-communication between fog nodes is a must to provide secure handover. For example in Figure 4, a device (car) that was connected to the fog area 1 to a fog node, it leaves and heads to the fog area 2. When the car arrives to the fog area 2, it must be authenticated and start executing services securely. This is possible thanks to the secure handover provided by secure fog node inter-communication to prevent any unauthorized or attacker eavesdrop and modified exchange information.

All the involved components and their communications in the F2C system can get attacked. Therefore, the next chapter deeply analysis all possible attacks in different F2C layers.

(28)

26 | P a g e

(29)

(30)

28 | P a g e

Chapter.4 Fog-to-Cloud attacks

In the last chapter, the basic security consideration was described for F2C system. This chapter provides all possible attacks on each layer in F2C and finally illustrates the most potential and important attacks to overcome in a hierarchical F2C continuum system (Table 1).

4.1 Cloud attacks

The attacks on the cloud layer in F2C system are described in the following articles, [9], [21], [22], [23], [24], [25], [26], [27]and the type of attacks are:

Backdoor channel attacks: attackers take remote access to the compromised system. Attackers take control over victim’s resource by using backdoor channel, then the attacker can use victim’s resource to launch a zombie attack, even they can disclose private victim’s information.

Malware injection: Hackers can inject malware application, services, or virtual machines into the cloud system or datacentres to interrupt the whole system.

Virtualization attack: There are two types of virtualization attacks including VM escape and rootkit in hypervisor.

In VM escape, the attacker runs a program in a VM and breaks the isolation layer in order to run with hypervisor’s root privileges instead with the VM privileges, which allows the attacker to interact with the hypervisor. Therefore, attacker gets access to the host OS and other VMs running on the physical machine.

Rootkit in hypervisor: VM-based rootkit initiates a hypervisor including the existing host OS to a VM. In reality host OS does not exist; however, the new initiated guest OS assumes that it is running as the host OS with the corresponding control over resources. Hypervisor produces a channel to execute unauthorized code into the system which attacker get control over running VM on the host machine and activities manipulation on the system.

Denial of service (DoS): Attackers can affect the availability of the cloud and prevent legitimate users to access to cloud by jamming or flooding requests to the server.

Man in the middle attack: If a secure channel between cloud and users is broken, attackers are able to access data exchange.

Metadata spoofing: An attacker can modify web service’s description languages where descriptions of services are stored.

Malicious insider: Person who is an employee in the cloud organization can use their privileges to disclose private information.

(31)

29 | P a g e

Phishing attack: Attackers can manipulate the web link and redirect users to a fake one to get user’s private information. An attacker may use cloud services to host a phishing attack site to hijack accounts and services of other users in cloud.

SQL injection: Attackers can inject malicious data into the SQL and get the private information or interrupt the whole SQL.

Sniffer attack: The attacker tries to read the content of a network packet, or to derive partial information (e.g. number of letters in a password).

Zombie attack (DoS/DDoS): Through the Internet, an attacker tries to flood the victim by sending requests from innocent hosts (normal host not the fake one) in the network. There are 2 types of Zombie attacks; the first is when an attacker floods a large number of requests via a zombie (innocent host) to affect availability of cloud services. The second case is when a huge number of requests overloads cloud to be exhausted which can cause DoS and DDoS attack. DDoS is a type of DoS attack where multiple compromised systems are used to target a single system causing a DoS attack.

Spoofing attack: This occurs when an attacker impersonates to be a legitimate cloud user with the intention of stealing sensitive information or launching the attack to the whole cloud system. The cloud characteristics are high computation, storage, and network, therefore we have to be able to provide also high security. There are many cloud security solutions in the market that can be applied, however there are so many challenges still unsolved that will be discussed in chapter 6.

4.2 Fog attacks

The fog computing inherits cloud characteristics to provide computing, network and storage closer to the users. Therefore, there are some mutual attacks in both layers. Some important attacks in fog are [28], [29], [30], [31], [32], [33]:

Man in the middle: If a secure channel between fog nodes, users, and servers is broken an attacker will be able to access data exchange.

Virtualization attack: Fog inherits the virtualization attacks from cloud due to their similar characteristics. This attack is already described in the cloud section in details.

DoS/DDoS attack: Attackers can affect the availability of fogs and prevent legitimate users from accessing to fog servers. These attacks are described at the cloud part.

Malware injection: Hackers can inject malware data, services, or virtual machines into the fog system to interrupt the whole system.

Gateway attack: Gateways are acting as bridge between fog and cloud. An attacker can get control over gateways to disclose fog information, interrupt the fog system, use that gateway to launch zombie attacks, etc.

(32)

30 | P a g e

Spoofing attack: This occurs when an attacker impersonates to be a legitimate fog device, user, or server to steal data or launch attack to the fog system.

Due to its mobility nature, one of the main security issues in fog is secure mobility and a secure handover. However, many challenges are yet unsolved in the fog security area, and unfortunately, most of the cloud security solutions cannot be applied to the fog scenarios due to their low computation and storage capabilities and high mobility. All the security challenges in fog will be discussed in the next chapter 6.

4.3 Edge attacks

Some vital attacks in this layer are [5], [34], [35], [36], [37], [38], [39]:

Hardware attack: It’s a malicious modification of an integrated circuit. An attacker can access to

data and software running on the integrated circuit.

Cryptanalysis attacks and side channel attacks: In this type of attacks, attacker by analysing the cryptography that is used in edge device can obtain cipher text or plain text and at the end can get the encryption key used in the algorithm.

Denial of service attack: There are 3 types of attacks: 1. Battery draining: edge nodes have a small battery with limited energy capacity. An attacker may disable battery and cause node failure. 2. Sleep deprivation: an attacker sends a set of legitimate requests to the power-battery limited energy capacity edge node to interrupt the device. 3. Outage attack: it happens when an edge node stops performing normal operations. It causes devices stop functioning.

Physical attacks/ tampering: The attacker with a physical access may get valuable information, tamper with the circuit, modify programming and change the operating system because edge devices are in the physical environment where physical access may be possible.

Node replication attack: The attacker replicates node identification and enters a new malicious node to the system. It affects network performance.

Camouflage attack: Attackers hide an authorized edge node or insert a counterfeit edge node to catch, modify or redirect packets.

Corrupted/malicious node: Attackers take access to the network by corrupting a legitimate edge node or by injecting a malicious node to the system to access to other nodes.

Tracking: A fixed radio-frequency identification (RFID) tag has a unique identifier that can be read by nearby unauthorized readers, therefore attackers use a large number of RFID readers of this unique identifier to access into the system and get authorization.

Inventorying: An attacker can obtain a manufacturer code and product code and other valuable information that are attached to the RFID tags to use for other attacks such as impersonating attacks.

(33)

31 | P a g e

Counterfeiting: Attackers manipulate tags by modifying their identity.

Eavesdropping: Attackers intercept, read, and save message for future analysis to launch more attacks.

Due to the edge devices’ characteristics, cloud solutions or even fog security solutions cannot be always applied to them, and new solutions should be designed. The security requirements will be discussed in the next section 5.

Cloud security attacks Fog security attacks Edge security Attacks

Backdoor channel attacks Man in the middle Hardware attack

Virtualization attack Virtualization attack Cryptanalysis attacks and side channel attacks

Denial of service (DoS) DoS/Ddos attack DoS/DDoS attack Malware injection Malware injection Physical attack/tampering Metadata spoofing Spoofing attack Node replication attack Malicious insider Gateway attack Camouflage attack Phishing attack Most of the cloud attack can be

happen in Layer 1 (Layer 1 inherits security challenges from cloud)

Corrupted/malicious node

SQL injection Tracking node

Sniffer attack Inventorying attack

Zombie attack (DoS/DDoS) Tag cloning

Man in the middle Counterfeiting

Spoofing attack Eavesdropping

(34)

32 | P a g e

4.4 Most potential attacks in F2C system

In the previous sub-sections, all the most possible attacks in the different layers of F2C were described. Now putting altogether, there are many potential security vulnerabilities in F2C-like systems, paving the way for attackers to launch attacks in different layers in the system. In this section, we identify most potential attacks to be faced by F2C-like systems as illustrated in Figure 5, all grouped into three categories, as follows.

Man-in-the-middle attack: Attackers can take the network control between devices at different levels (IoT devices, fog nodes, fog nodes and cloud) to either eavesdrop communication, modify information or even to inject malicious information and code into the system. For example, attackers can obtain the identity of a F2C component and then impersonate it to be an eligible component. Due to the obtained identity, the attacker can impersonate a fog node (malicious fog node) thus getting devices and users information and locations. Also, an attacker can impersonate users and devices to take information or gain access to services it is not authorized to. In upper layers, i.e. fog-cloud communication, an attacker can impersonate fog node or even cloud to launch a man-in-the-middle attack. In all these cases, attackers can launch the attack in passive (eavesdropping without changing information) and active (information modification, manipulation and malicious injection) ways. This type of attack effects the integrity and confidentiality of any F2C-like system (see Figure 5.A).

Denial of service and Distributed denial of service (DoS and DDoS): In this case, attackers either launch multiple service requests to the fog node or perform a jamming wireless communication between fog node-devices to deplete the fog node resources and consequently making it down. An attacker can use legitimate devices, such as IoT devices, fog devices or fog nodes to launch DoS and DDoS using their identities. DoS and DDoS attacks can also occur in upper layers such as fog node-cloud. As a consequence, attackers successfully prevent legitimate users and devices from accessing services provided by a fog node or even by cloud (see Figure 5 .B). In short, this attack severely affects the availability of the F2C system.

Database attacks: In a F2C system, databases may meet a hierarchical architecture, keeping for example one centralized at cloud and some other locally distributed at fog layers. If an attacker can access to these databases, it can modify, manipulate and even leak the data, what may have a high impact on the total system performance. Database attacks may be internal –coming from F2C service providers–, or external –legible and illegible users. This attack intensely effects the F2C integrity and confidentiality (see Figure 5.C).

Thus, proposing a solution for F2C undoubtedly requires a strong background on possible security attacks in each layer and security aspects in the cloud, fog and IoT devices. In the next section, security requirements in each layer of F2C system and security challenges will be analyzed and illustrated.

(35)

33 | P a g e

(36)

(37)

35 | P a g e

Chapter.5 Fog-To-Cloud security requirements

F2C as described in previous section includes 3 layer (cloud, fog node (Leader), and Edge devices). We analyses each layer separately to discover most potential F2C security requirements. Table 2 illustrates security requirements in different layers and finally combined F2C system, and in the next subsections 5.1, 5.3 and 5.4 we detail all these security requirements.

Table 2. Security requirements in different layers

5.1 Cloud security requirements

The cloud security requirements are considered and analyzed according to ( [25], [40], [41], [22], [42], [21], [43], [24]). From these references, we infer the main security requirements in cloud:

1. Secure storage: All data stored at cloud must be encrypted and shared only with authorized users.

2. User and device authentication and authorization: All devices and users such as fog nodes (layer 1 and layer 2), IoT devices must be authenticated to access to the cloud, to prevent

(38)

36 | P a g e

undesired information disclosure to unauthorized users. An open challenge here is to design a distributed authentication mechanism for this hierarchical F2C system.

3. Key management: A key management mechanism to handle key distribution to fog nodes (layer 1 and layer2), fog users, and IoT devices is mandatory to encrypt messages and thus provide secure communication. One of the main challenges is how to distribute and manage keys in a hierarchical and distributed F2C system. A distributed key management system is needed for hierarchical F2C system.

4. Identity management: Fog nodes, cloud services, servers, clouds and all entities must have a unique identity to be recognizable by the system and parties. Identity must not disclose user private information.

5. Policy management: well-structure policies for security provisioning must be defined by F2C cloud for fog layers.

6. Logging protection mechanisms: A secure password-based or other type of logging strategy needed to protect user private information. F2C provider must provide a proper secure way to the users and fog layers for accessing F2C cloud to avoid somebody steal or eavesdrop on user’s activities, transactions, and credential.

7. Access control: A well-secure access must be defined for users to prevent hackers and attackers to access to the infrastructure. In hierarchical F2C system, a distributed access controls might be needed.

8. Trust: F2C cloud service providers must be trustable enough for users and fog layers to store their data in the infrastructure.

9. Data protection: All data processing, aggregation, storing must be encrypted and protected from unauthorized users. Data must transfer in encrypted way to the F2C users to do not disclose to unauthorized users.

10.End-to-end encryption: F2C cloud must provide secure end to end communication for protecting data against leakage or breach. F2C cloud must provide end to end communication to their lower layer (fogs in layer 1 and 2).

11.Application programming interface security: Software application communication can be defined by a set of protocols and standards through Internet. Cloud APIs provide all the infrastructure, platform and software service levels communication: i) Platform as a Service API provides access to the service; ii) Software as a Service provides the software application API connection with cloud, and; iii) Infrastructure as a Service provides access and management to resources such as network and VMs.

12.Web application security: Some critical applications, such as banking must have a high secure web quality to avoid attackers to gather any user information.

13.Federation of security among multi clouds: When multiple clouds are federated or some services from different clouds are needed, their security requirements must be federated 14.Heterogeneity: When different service providers deliver a huge amount of services using

different technologies, the heterogeneity problem arises, such as no security compatibility at software and hardware levels.

15.Integrity: This refers to data and system integrity. Information can only be changed in an authorized manner. Integrity provides accurate and reliable information between cloud components.