Top PDF Cross Layer Routing in Cognitive Radio Network Using Deep Reinforcement Learning

Cross Layer Routing in Cognitive Radio Network Using Deep Reinforcement Learning

Cross Layer Routing in Cognitive Radio Network Using Deep Reinforcement Learning

routers. Fig. 3.1 shows four secondary users transmitting over such system. Routing path to the destination node through each router is compared by the secondary users and the packets are transmitted to the router with highest throughput. If the queue of the selected router is not full, packets arriving are placed in the queue, else the packets get dropped. These queues are modeled as M/M/1/K queues. They are finite queue with size of K packets. The arrival of packets is modeled as a Poisson process with arrival rate λ. The service times for packets follow an exponential distribution with mean µ. When the arrival rate is greater than the service rate (λ >µ), then the utilization of M/M/1/K queues is large ρ = λ µ >1. This indicates that the system is unstable resulting into increase in the queue lengths to the congestion point. When the queue length reaches its size K all packets that arrive will be dropped until the queue length decreases. This results in packet loss. In this work we consider that the traffic carried over the secondary links is video transmissions for Internet Protocol Television (IPTV) services. The data at network layer is measured in packets, where one packet size is 7*188 bytes (10528 bits).
Show more

67 Read more

Maximizing Throughput using Adaptive Routing Based on Reinforcement Learning

Maximizing Throughput using Adaptive Routing Based on Reinforcement Learning

Network lifetime is extremely critical for most applications, and its main limiting factor is the energy consumption of the nodes, it requires being self-powering. Although it is frequently assumed that the transmit power related with packet transmission accounts for the lion ’ s share of sensing, power consumption, signal processing and even hardware operation in standby manner consume a consistent amount of power as well [10], [11]. In some applications, additional power is needed for macro-scale actuation. Many researchers recommend that energy consumption could be reduced by considering the presented interdependencies between individual layers in the network protocol stack. Channel access protocols and Routing, for instance, could greatly advantage from an information exchange with the physical layer. At the physical layer, benefits can be obtained with dynamic modulation scaling and lower radio duty cycles (varying the constellation size to minimize energy expenditure [12]). Using low-power mode for the processor or disabling the radio is generally beneficial, even though periodically turning a subsystem on and off may be more expensive than always keeping it on. Techniques aimed at reducing the idle mode leakage current in CMOS-based processors are also noteworthy [13]. MAC (Medium
Show more

8 Read more

PHY-layer Security in Cognitive Radio Networks through Learning Deep Generative Models: an AI-based approach

PHY-layer Security in Cognitive Radio Networks through Learning Deep Generative Models: an AI-based approach

the AC-GAN seems to provide better performance than C-GAN and VAE models. From another point of view, when GAN-based models are compared to VAE, it can be noticed that: in the first case, since the generator is trained to learn a mapping between a random noise vector, z in Fig. 5.7, and the generated data (by learning hidden, complex structure in the real data x), then G is able to capture the dynamics in the real data. In the second case, a VAE model returns the posterior probability that an observation belongs to a specific cluster by learning the latent vector, z in Fig. 5.8. In this way, observations x from different clusters will correspond to different z vectors and the dynamics of x is captured according to the way and the time instants the vector z changes. In effect, learning from dynamic data as in the first case should provide better performance as confirmed by the results. Alternatively, an advantage of the VAE, with respect to GAN, is the possibility to exploit the encoder’s output latent variables (µ and σ) that represent probabilistic distributions. Indeed, such variables can be clustered to learn temporal dependencies among them and draw a probabilis- tic graphical representation; for example, by using a Self Organizing Maps (SOM) method [4, 5] or a Growing Neural Gas (GNG) network [2]. The latent variables can also be used to reduce the complexity due to high dimensionality data in wideband RF spectrum.
Show more

170 Read more

A Review of Cross-layer Design in Dynamic Spectrum Access for Cognitive Radio Networks

A Review of Cross-layer Design in Dynamic Spectrum Access for Cognitive Radio Networks

A cross-layer framework is done by combining adaptive modulation and coding ( AMC ) with truncated automatic repeat request ( ARQ ) in an SU to achieve high spectral efficiency by maintaining target packet loss probability ( Yuli Yang et al., 2012 ) . This minimizes packet er- ror rate ( PER ) at each AMC transmission mode subjected to satisfying spectrum-sharing limi- tations. In ( Leila Musavian and Tho Le-Ngoc, 2012 ) , the spectrum sensing in physical layer is integrated with packet scheduling at MAC layer to quantitatively identify the tradeoff between the aggregate traffic throughput and the packet transmission delay in non-saturation network. In ( Yi Peng et al., 2009 ) , dynamic channel al- location is achieved by optimizing joint power control and link scheduling in OFDMA-based cognitive radio networks. A cross-layer antenna selection algorithm is used in ( A. Ghosh and W. Hamouda, 2012 ) to achieve high transmis- sion efficiency and beamforming is employed to cancel interference between cognitive users and primary users. The complexity of the algorithm increases as the number of transmits antenna increases. In ( Amiotosh Ghosh and Walaa Hamouda, 2011 ) , cognitive nodes access the spectrum by using spectrum overlay approach and each node is equipped with Multiple-Input Multiple-Output ( MIMO ) system to improve the spectrum utilization. The cross-layer an- tenna selection is used to improve the transmis- sion efficiency and to reduce data rate variance among cognitive nodes and learning based al- gorithm is also used. The complexity of the an- tenna selection algorithm increases as the num- ber of channel increases.
Show more

9 Read more

CMCS: a cross layer mobility aware MAC protocol for cognitive radio sensor networks

CMCS: a cross layer mobility aware MAC protocol for cognitive radio sensor networks

This paper proposed a novel cross-layer mobility-aware MAC protocol for CRSN, which is robust against the activities of PUs, as well as node mobility in networks. This was realized by integrating the spectrum sensing at the PHY layer with the packet scheduling at MAC layer. The proposed spectrum-aware clustering scheme was designed in such a way that it ensures that there is stability in the formation of clusters in order to avoid frequent reclustering. A greater number of common chan- nels results in clusters that are more robust against the mobility of both SUs and spectrum because of the PUs’ activity. To handle the mobility in the network, the CMCS uses an adaptive data period to handle the upcoming new nodes in the clusters. The simulation results show that the proposed protocol can achieve around five common chan- nels per cluster in lower-density networks, and around three common channels per cluster in higher-density net- works. However, it overtakes the other candidate MAC protocols by more than 60 %, where the number of SUs increases in the network. Moreover, CMCS outperforms KoN-MAC, CogMesh, and cluster-based MAC protocols in terms of the packet delivery ratio, energy consump- tion, and delay by up to 5, 30, and 25 %, respectively. This work focused on the design of an efficient MAC proto- col for channel assignment. However, by considering the cross-layer approach, an appropriate routing protocol can also be integrated with the MAC protocol, and this will be done as future work.
Show more

15 Read more

Improving Speed and Energy Efficiency of Cognitive Radios using Cross layer Multichannel Routing and Compression

Improving Speed and Energy Efficiency of Cognitive Radios using Cross layer Multichannel Routing and Compression

[2] In this paper, “a survey on mac strategies for cognitive radio networks”, which is published by a.d. domenico, e.c. strinati and m.d. benedetto in the year of 2012, we have discussed various methods and techniques used so far in the design and development of mac protocols for manets. we also looked into a few protocols developed for wsns that can be deployed in a manet environment with minor adjustments. some of the techniques proposed call for interaction between different layers of the protocol stack such as, a mac solution that works in conjunction with routing. the traditional layered architecture for network communication is rigid and thus limits the ability of nodes to select better routes. we believe that a mac solution that interacts with the physical layer and network layer (routing) would provide better results compared to a strict layered approach. we also looked into antenna technologies used in manets especially the directional or the beam forming antennas. in communication environments where a single radio interface is using a single channel, only one device can transmit whereas the rest of the nodes in its transmission range either receive the data being transmitted or waits for the transmission to end before they can transmit their own data. these enhanced antenna based mac solutions can achieve better throughput performance by opportunistic transmission without affecting other transmissions in their neighborhood. specialized antennas based mac solutions also fall under the paradigm of cross-layer design because beam forming antennas needs instruction from the mac layer before directing their transmission at particular node or group of nodes.
Show more

6 Read more

Enhanced Network Performance in Cognitive Radio Networks using Reinforcement Learning

Enhanced Network Performance in Cognitive Radio Networks using Reinforcement Learning

SMART is for overcoming the challenges of multihop routing in CRNs through cluster-based routing and RL. Clustering aims to form clusters that fulfill the requirements on the number of common channels in a cluster and allow nodes to forward routing control messages efficiently without the need for broadcasting on all the available channels. RL aims to find a route that increases the usage of white spaces for maximizing SUs’ network performance. SMART also provides extension to clustering through cluster merging and splitting. SMART adjusts cluster size as time goes by so that a cluster fulfills the requirement on cluster size for improving scalability, as well as stability. SMART estimates the OFF-state probability of a channel at the next time instant and uses this estimation to rank and select the operating channels in clustering and routes in routing.
Show more

5 Read more

Sentence Simplification with Deep Reinforcement Learning

Sentence Simplification with Deep Reinforcement Learning

One caveat with using SARI as a reward is the fact that it relies on the availability of multiple references which are rare for sentence simplifica- tion. Xu et al. (2016) provide eight references for 2,350 sentences, but these are primarily for system tuning and evaluation rather than training. The majority of existing simplification datasets (see Section 5 for details) have a single reference for each source sentence. Moreover, they are unavoid- ably noisy as they are mostly constructed automat- ically, e.g., by aligning sentences from the ordi- nary and simple English Wikipedias. When rely- ing solely on a single reference, SARI will try to reward accidental n-grams that should never have occurred in it. To countenance the effect of noise, we apply S ARI (X, Y , Y ˆ ) in the expected direc-
Show more

11 Read more

Paraphrase Generation with Deep Reinforcement Learning

Paraphrase Generation with Deep Reinforcement Learning

Neural paraphrase generation recently draws at- tention in different application scenarios. The task is often formalized as a sequence-to-sequence (Seq2Seq) learning problem. Prakash et al. (2016) employ a stacked residual LSTM network in the Seq2Seq model to enlarge the model capacity. Cao et al. (2017) utilize an additional vocabu- lary to restrict word candidates during generation. Gupta et al. (2018) use a variational auto-encoder framework to generate more diverse paraphrases. Ma et al. (2018) utilize an attention layer instead of a linear mapping in the decoder to pick up word candidates. Iyyer et al. (2018) harness syntac- tic information for controllable paraphrase gen- eration. Zhang and Lapata (2017) tackle a simi- lar task of sentence simplification withe Seq2Seq model coupled with deep reinforcement learning, in which the reward function is manually defined for the task. Similar to these works, we also pre- train the paraphrase generator within the Seq2Seq framework. The main difference lies in that we use another trainable neural network, referred to as evaluator, to guide the training of the generator through reinforcement learning.
Show more

14 Read more

Self reflective deep reinforcement learning

Self reflective deep reinforcement learning

The number of episodes is envisaged (as was evident in the simulation in [14, 15 and 16]) to show a pattern of convergence towards minimal number of steps if the robot where left to run for a very long time. This was not needed in this realistic robotics scenario since the agent reached a good policy in 30 episodes only. However, due to time and physical constraints, this was difficult to do and a powerful and fast model was developed to reach a suitable strategy. The experiments were conducted by starting always form roughly the same position. The variation is due to the different learning stages as well as due to the continuum of possible states at any location and due to inherited inaccuracy of actions taken (due to the mechanics of the used robot). Our results show that out of 35(30 training + 5 testing) times, the agent reached the goal location in all of them but 7 with not a desired orientation; the goal was not directly inside the visual field of the robot. Hence, it can be concluded that the success rate is 35/35 ≈ 100% while goal orientation recognition is 80%.
Show more

7 Read more

Deep Reinforcement Learning for Dialogue Generation

Deep Reinforcement Learning for Dialogue Generation

To achieve these goals, we draw on the insights of reinforcement learning, which have been widely ap- plied in MDP and POMDP dialogue systems (see Re- lated Work section for details). We introduce a neu- ral reinforcement learning (RL) generation method, which can optimize long-term rewards designed by system developers. Our model uses the encoder- decoder architecture as its backbone, and simulates conversation between two virtual agents to explore the space of possible actions while learning to maxi- mize expected reward. We define simple heuristic ap- proximations to rewards that characterize good con- versations: good conversations are forward-looking (Allwood et al., 1992) or interactive (a turn suggests a following turn), informative, and coherent. The pa- rameters of an encoder-decoder RNN define a policy over an infinite action space consisting of all possible
Show more

11 Read more

Deep Reinforcement Learning for Swarm Systems

Deep Reinforcement Learning for Swarm Systems

While a typical experiment with 20 agents in our setup takes between four and six hours of training on a machine with ten cores (sampling trajectories in parallel), a forward pass through the trained neural network to compute the instantaneous control signal takes only about 1 ms, which enables an execution in real time. Furthermore, all control strategies learned through our framework are decentralized, which allows an arbitrary system size scaling in a real swarm network, where the required computations are naturally distributed over all agents. When learning new policies, the memory requirements scale O(N (N − 1)) with the number of agents (assuming global observability) since we need to store the local views of all agents. However, decentralized execution after the policy is learned scales linearly in N per agent. An incremental online computation of the mean can be chosen if memory restrictions exist (Finch, 2009).
Show more

31 Read more

A Reinforcement Learning Network based Novel Adaptive Routing Algorithm for Wireless Ad-Hoc Network

A Reinforcement Learning Network based Novel Adaptive Routing Algorithm for Wireless Ad-Hoc Network

Ad hoc network is a collection of mobile nodes which dynamically form a temporary network without any infrastructure or centralize entity. There are number of routing protocol exists in ad hoc network and this protocols have been compared. These protocols are like DSR (Dynamic Source Routing), AODV (Ad-hoc On Demand Distance Vector Routing Algorithm), and TORA (Temporally Ordered Routing Algorithm) like more. AODV is Reactive routing protocol. We modify the existing AODV protocol. All Routing Protocol have different Strategies of routing like End to End delay, Packet delivery ratio, Traffic overhead and Power Consumptions. Routing Deals with route discovery between source and destination. Aim of Dissertation is to improve route error tolerance mechanism of AODV. In our propose scheme the transmission starts from closest neighbor node if the link fail in middle of the transmission. That shows very important reductions in delay and it improves the packet delivery ratio. It also reduces the routing overhead by reducing frequency of route discovery process.
Show more

8 Read more

Mathematical Reinforcement to the Minibatch of Deep Learning

Mathematical Reinforcement to the Minibatch of Deep Learning

Problem I What is the meaning of the assumption in the total flow of data ? Last in this section let us comment on the minibatch of Deep Learning. When input data A is huge the calculation of time evolution of the weights of synaptic connections will give a heavy load to the computer. In order to alleviate it the minibatch is practical and very useful, see Figure 5.

14 Read more

Methods for Efficient Deep Reinforcement Learning

Methods for Efficient Deep Reinforcement Learning

Reinforcement learning (RL) is a broad family of algorithms for training autonomous agents to collect rewards in sequential decision-making tasks. Shortly after deep neu- ral networks (DNNs) advanced, they were incorporated into RL algorithms as high- dimensional function approximators. Recently, “deep” RL algorithms have been used for many applications that were once only approachable by humans, e.g., expert-level per- formance at the game of Go and dexterous control of a high degree-of-freedom robotic hand. However, standard deep RL approaches are computationally, and often financially, expensive. High cost limits RL’s real-world application, and it will slow research progress. In this dissertation, we introduce methods for developing efficient DNN-based RL agents. Our approaches for increasing efficiency draw upon recent developments for the optimization of DNN inference. Specifically, we present quantization, parameter pruning, parameter sharing, and model distillation algorithms that reduce the computational cost of DNN-based policy execution. We also introduce a new algorithm for the automatic design of DNNs which attain high performance while meeting specific resource constraints like latency and power. Intuition, which is backed by empirical results, states that a naive reduction in DNN model capacity should lead to a reduction in model performance. However, our results prove that by taking a principled approach, it is often possible to maintain high agent performance while simultaneously lowering the computational expense of decision-making.
Show more

147 Read more

Reinforcement learning based dynamic band and channel selection in cognitive radio ad hoc networks

Reinforcement learning based dynamic band and channel selection in cognitive radio ad hoc networks

In this paper, we propose a band group and channel se- lection method considering the consecutive channel op- eration time, data transmission rate, channel utilization efficiency, and cost of the band group change for a cog- nitive radio ad-hoc network composed of CH and MNs. The proposed method uses the Q-learning in order to operate in a channel environment that varies dynamic- ally according to the geographical region, time zone, band group, channel, and primary user ’ s activity. As the core of the Q-learning operation, a Q-table and reward function consisting of an action and state are designed to consider various parameters related to the channel se- lected by the CR ad-hoc system. In particular, the reward for channel utilization is designed to select the appropri- ate band and channel so that the frequency resources are not wasted and a CR ad-hoc system can coexist with other CR systems with fair resource utilization efficiency. The simulation results represent how the proposed sys- tem selects an adaptive band and channel for the re- quired data rate and also explain the principle of operation through the change of action and state in Q-table. It also can be confirmed that the system oper- ates according to the intended purpose through the weight change, and the channel is selected adaptively when the required transmission rate is changed. These simulations clearly demonstrate these advantages of the proposed method.
Show more

25 Read more

An Intelligent Routing Model using Multi-Radio Diversity for Cognitive Radio Ad-Hoc Networks

An Intelligent Routing Model using Multi-Radio Diversity for Cognitive Radio Ad-Hoc Networks

With increase demand of wireless operated electronic gadgets, there is tremendous research taken place in the fields like wireless communication, signal processing, VLSI. So, lots of wireless communication technologies have been deployed in personal area network to wide area network to fulfil users need. This has increased the use of radio spectrum. Thus radio spectrum is one of the most heavily used resources. The recent radio spectrum measurements show that the fixed spectrum allocation policy is not suitable for current wireless system [1]. Moreover, most of the licensed bands assigned for licensed users are under-utilized, many portions of the radio spectrum are not utilized for a significant period of time or in particular areas, while unlicensed bands used to operate by various well-known wireless technologies, such as Wi-Fi, cordless phones, Bluetooth, NFC (Near Field Communication), and so on, are always crowded, approved by Federal Communications Commission (FCC)’s experiment results [2]. Cognitive radio (CR) [3][4][5] is a important technology for future wireless communications and to solve the problems of limited availability of spectrum and spectrum underutilization as well as to address the increasing congestion in the unlicensed bands by enabling unlicensed users to opportunistically access the vacant portions of the spectrum bands, referred to as Spectrum Opportunities (SOP) [5], which is always statistically underutilized by licensed users (also known as primary users: PUs). CR networks are envisioned to provide high bandwidth to mobile users via heterogeneous wireless architectures and dynamic spectrum access techniques. This goal can be realized only through dynamic and efficient spectrum management techniques. CR networks, however, impose unique challenges due to the high fluctuation in the available spectrum, as well as the diverse quality of service (QoS) requirements of various applications.
Show more

6 Read more

SECURITY IN COGNITIVE RADIO NETWORK

SECURITY IN COGNITIVE RADIO NETWORK

Finally, we combine the information from both physical layer and upper layer to integrate the cross-layer mechanism. Numerical results have demonstrated that the cross-layer detection scheme can efficiently detect the injection attack. In this thesis, security in cognitive radio is designed. Here, LABVIEW, VISUAL BASIC is used for graphical representation and signal processing. The results are viewed in PC. From the PC the signal is converted to serial form by means of UART. Microcontroller is used along with the wireless transceiver which in turn secure the information by this it is been used for rescue purpose and other application.
Show more

7 Read more

JOINT CHANNEL AND POWER ALLOCATION FOR COGNITIVE RADIO SYSTEMS WITH PHYSICAL LAYER NETWORK CODING

JOINT CHANNEL AND POWER ALLOCATION FOR COGNITIVE RADIO SYSTEMS WITH PHYSICAL LAYER NETWORK CODING

In [6], channel assignment in cellular communications is addressed to maximize the frequency spectrum utilization and to minimize the frequency interference effect. However, in cognitive radio secondary users can access the spectrum bands that are not used by primary users [1]. Spectrum sensing detects the availability of spectrum bands. Spectrum bands available at the secondary users may not be the same in most of the cases [7]. Secondary users located at different locations can have different sensing results. If no common band is available between the two cognitive users, then the communication is established between them using relay discussed in [8]. The power allocation issues in CR systems attract a lot of attention because performance of the CR system is improved by properly allocating the power [9]. In [10], power is allocated separately for source node and relay node for a cooperative relay in cognitive radio networks, when multiple spectrum bands are available at secondary users. However, power and channel allocation is only on the single cast instead of the multi cast transmission model. In [11], joint relay selection and power allocation scheme is addressed to maximize the capacity in single cast system. In [12], iterative algorithm is developed to allocate the power for the source node and relay node jointly in physical layer network coding, however, the system is not considered for Cognitive Radio network and there is no primary interference limit constraints in the optimization problem.
Show more

13 Read more

Routing in Cognitive Radio Ad-Hoc Networks

Routing in Cognitive Radio Ad-Hoc Networks

The radio spectrum is divided into two parts: licensed spectrum band and unlicensed spectrum band. Licensed spec- trum bands consists of Ultra High Frequency (UHF)/ Very High Frequency (VHF) TV frequency bands and many other licensed frequencies while unlicensed spectrum bands consists of ISM (Industrial, Scientific and Medical) and U-NII (Un- licensed National Information Infrastructure) [1]. Previously, fixed licensed policy was used to allocate spectrum bands to different users. Only licensed users could access the licensed spectrum. Studies of Federal Communications Commission (FCC) [2] showed that only 15-85% of licensed spectrum band was utilised, hence large part of this spectrum band was idle in different time and space. On the other hand, the unlicensed spectrum band became congested with the applications of different fields like WLANs, mesh area network, body area networks and sensor networks; which lead to the problem of inefficient spectrum usage. Hence, Cognitive Radio was discovered in 1999 to mitigate the problem of this inefficient spectrum usage. Devices having Cognitive Radio capabilities are called Cognitive Radio devices and the network that they form together is called Cognitive Radio Networks (CRNs). Cognitive radio is based on Dynamic Spectrum Access [2], through which unlicensed users can use the licensed spectrum band without causing harmful interference to the licensed users. Unlicensed users are referred as Secondary Users (SUs) and licensed users are referred as primary users (PUs). The idea is based on the fact that SUs can access the available licensed spectrum band but have to immediately change the spectrum band once an allocated PU need to access the spectrum band (PU still accesses the spectrum in fixed license policy). With the help of Software Defined Radios (SDRs) [1],
Show more

7 Read more

Show all 10000 documents...