IEEE 802.11s Wireless Mesh Networks for Last-mile
Internet Access: An Open-source Real-world Indoor
Testbed Implementation
Technical Report# CSSE10-02
Riduan M Abid, Taha Benbrahim, Saâd Biaz
Computer Science and Software Engineering Department Auburn University
Shelby Center for Engineering Technology, Suite 3101 Auburn University, AL, 36849-5347, USA
{abidmoh, benbrah, biazsaa}@auburn.edu
ABSTRACT
Due to their easy-to-deploy and self-healing features, WMNs (Wireless Mesh Networks) are emerging as a new promising tech-nology with a rich set of applications. While the standardization of this new technology is still in progress, its main traits are already set in the IEEE 802.11s standard: WMN architecture and MAC Routing.
WMNs are attracting considerable research in Academia (and industry as well). However, the lack of open-source testbeds is re-stricting such a research by limiting it to simulation tools. This paper presents an open-source implementation of a real-world in-door IEEE 802.11s WMN testbed: the implementation is transpar-ent and easy-to-deploy as it uses off-the-shelf commodity hardware and software. The source code and deployment instructions are made available online. The current testbed source code can serve as a blueprint for the WMNs research community to deploy their own testbeds.
By delving into the testbed implementation subtleties, the paper is also shading further light into the still-on-process IEEE 802.11s standard. Major encountered implementation problems (e.g., Clients Association, Internetworking and addressing multiple gateways) are highlighted and addressed. To ascertain the functionality of the testbed, both UDP and TCP trafc are supported and operational. The testbed uses the default IEEE 802.11s HWMP (Hybrid Wire-less Mesh Protocol) routing protocol along with the default IEEE 802.11s Airtime routing metric.
1. INTRODUCTION
Due to their easy-to-deploy and self-healing features, WMNs [9] are emerging as a promising technology. WMNs are easy-to-deploy as setting a WMN involves minimal wiring and conguration over-head: Settling down the WMN nodes and powering On is all what is required to operate a WMN. On the other hand, WMNs are self-healing thanks to the redundancy of wireless links whereby a failure
of a wireless link incites the network to choose an alternative op-erational link, thus healing the link failure and continuously main-taining the network operational.
WMNs may serve a rich set of applications, e.g., wireless com-munity networks, wireless enterprise networks, transportation sys-tems, home networking and last-mile wireless Internet access.
Providing last-mile wireless Internet access is one of the most promising application as WMNs tremendously reduce the cost and conguration overhead when compared with current solutions. e.g. Wi-Fi (802.11) WLANs. The proliferation of Wi-Fi LANs played a tremendous role in the promising success of WMNs. Even though very successful, 802.11 WLANs still suffer from the cost and over-head of setting the wired backbone that connects the different 802.11 APs (Access Points).
In 802.11 WLANs, covering a cell requires the deployment of an AP which must be wired to the Internet through a backbone net-work, e.g., a 802.3 LAN. When another adjacent cell is to be con-nected, another AP has to be settled and wired to the Internet.This way, the settlement of further APs becomes more and more costly when the cells to cover are quite far from the wired backbone net-work. Furthermore, in case of dense regions, a parallel dense de-ployment of APs is needed to meet customers needs, thus resulting in more wiring and settlement overhead. Even though APs are rel-atively cheap, wiring the APs and nding the appropriate canals for the wires remains expensive. With WMNs, no wiring is re-quired except for the AP which will serve as a gateway towards the Internet (Even the gateway can be wirelessly connected to the Internet). All other APs serve as routers and route data on behalf of each other towards/from the gateway AP connected to the wired network. Therefore, covering a cell reduces to adding an AP with-out wiring it to the backbone network.
Easing wireless coverage facilitates increasing wireless cover-age: A fact which is of tremendous importance to industry since more coverage involves more user connections and hence more re-turn to industry. In this context, IETF (Internet Engineering Task Force) is actively working to nalize the IEEE 802.11s standard which will pave the way towards a successful worldwide industri-alization of this promising technology.
In 2005, IETF set a mesh networking TG to standardize IEEE 802.11s. The work is still in progress and the last IEEE 802.11s TG meeting was held in January, 2010 [24]. Even though the stan-dard is not nal yet, its main traits have are set, e.g. IEEE 802.11s Architecture and Routing. The IEEE 802.11s TG did set HWMP (Hybrid Wireless Mesh Protocol) [11] as a default routing
proto-col for 802.11s compliant devices. HWMP uses MAC addresses for routing. Besides, Airtime [10] has been set as a default rout-ing metric for HWMP. These two amendments aim to maintain a minimum compatibility between the IEEE 802.11s devices to be manufactured by different companies.
Albeit a new technology, several WMNs solutions are already commercialized (e.g., by Strix Systems , Cisco, Nortel, Tropos, BelAir). Besides, valuable research real-world deployments are in place too, e.g., The MIT Roofnet [12] and the Microsoft Mesh Net-working project [22]. However, these deployments are not compli-ant to the new IEEE 802.11s standard and are proprietary.
WMNs are attracting considerable research efforts in academia and industry. However, the absence of real-world open-source testbeds is restricting the proliferation of such a research by forcing re-searchers to use simulation tools, e.g., ns-2 [6] and Qualnet [18]. Eventhough simulators are relatively good at approximating real-world scenarios, they are still quite far from a good approxima-tion of wireless indoor environments, which is the case with IEEE 802.11s technology: In wireless indoor environments, RF propaga-tion is very crucial in determining the interferences level, this later has been proved to be of great impact on multi-hop wireless mesh networks [13,15].
Indeed, simulators can not catch the complex aspects of RF prop-agation such as multi-path fading, modeling mobile obstacles, mod-eling external interferences, etc. In fact, when dealing with RF propagation, most simulation tools remain simplistic: for instance, ns-2, which is the most widely used tool in academia, uses a sim-plied version of a physical model (the capture threshold model) that accounts only for one interferer at a time [4], which is not the case in real-world interferences where interference can be caused by different interferers. However, simulators are still very common in research as they remain the only research experimentation al-ternative when the deployment costs of real-world testbed are not affordable. Besides, and depending on the kind of application, sim-ulators can be a good alternative as they are close to reality, e.g., in simulating TCP, IP, 802.3, etc. However, when it comes to wireless technology and especially when it is in indoor environments, sim-ulators are just far from accurately presenting the real world.
In this context, we are presenting an open-source real-world IEEE 802.11s WMN testbed implementation that can be easily deployed as it uses off-the-shelf commodity hardware and software. The linux-based testbed code is open-source and written in C language. The code is made available online at http://www.eng.auburn.edu/ abid-moh/ [7]. Thus, the academia WMN research community can eas-ily migrate to the use of real-world testbeds, instead of simulators, by deploying the proposed testbed or by using it as a blueprint for deploying their own testbed. Besides serving the purpose of allow-ing the WMN researches to easily build their own WMN testbed, the testbed can serve as a building block for a future widely-used open-source WMN.
By delving into the implementation subtleties of the testbed, the paper shades further light into the new IEEE 802.11s standard. Major encountered implementation issues such as clients associ-ation, Internetworking and multiple gateways are highlighted and addressed. Both TCP and UDP trafc are supported in order to as-certain the functionality of the testbed: Real-world web browsing sessions from Wi-Fi enabled stations, using the testbed as a wire-less backhaul, are operational. The current testbed uses 15 nodes and spans two separate locations that are connected through the In-ternet.The testbed uses HWMP as a routing protocol along with the IEEE 802.11s default Airtime routing metric. However, It is very important to mention that this work is not meant to evaluate the performance of the new IEEE 802.11s standard but to present an
easy-to-deploy and open-source real-world in-door IEEE 802.11s WMN testbed that can serve as a seed for other WMN testbeds, hence enriching the WMN research in academia.
The primary contributions of this work are:
• Present an easy-to-deploy and open-source real-world IEEE 802.11s WMN testbed implementation.
• Encourage Academic researchers on IEEE 802.11s WMNs to use real-world testbeds.
• Present a building block for a future academic widely-used open-source WMN testbed.
• Highlight and solving major implementation issues, e.g.,: Clients association problem
Internetworking WMNs Supporting multiple gateways
• Highlight the main traits of the new IEEE 802.11s standard. • Highlight and implement the new IEEE 802.11s HWMP
rout-ing protocol and the IEEE 802.11s Airtime routrout-ing metric. The rest of the paper is organized as follows: Section 2 overviews the main specications of the IEEE 802.11standard. Section 3 highlights the details of the HWMP routing protocol and the Air-time routing metric. The implementation details and the encoun-tered problems are respectively covered in Sections 4 and 5. In Section 6, we outline the necessary steps to deploy the testbed. In Section 7, we present the experimental settings of the testbed, the topologies and the experimental results. Finally, Section 8 con-cludes and presents future work.
2. IEEE 802.11S OVERVIEW
The IEEE 802.11s standard started initially as a study group in 2003, then it became a Task Group in July, 2004. The rst draft was accepted in March, 2006 and the last TG meeting was held in January, 2010 [24]. The standard is concerned with ve main areas: 1. Architecture, 2. Routing in MAC layer, 3. MAC enhancements, 4. Internetworking and 5. Security. This work pertains to areas 1, 2 and 4. Areas 1 and 2 are introduced in next paragraphs, while Area 4 is deferred to Section 5.
2.1 802.11s Architecture
IEEE 802.11s denes three types of stations:
1. MPs (Mesh Points): MPs are wireless stations that perform only routing.
2. MAPs (Mesh Access Points): MAPs are MPs with addi-tional access point capabilities. Besides performing routing, MAPs aggregate trafc from/towards legacy 802.11 stations. A MAP can be thought of as a legacy access point which performs also routing.
3. MPPs (Mesh Portal Points): MPPs are MPs that serve as gateways to other types of networks such as 802.3 networks. MPPs aggregate trafc from/towards non-mesh networks, e.g., Internet.
Figure 1: WMN Architecture
To illustrate the functionality of every mesh node type, Figure 1 depicts a scenario where an end user, e.g., a Wi-Fi enabled station, is browsing the Internet. The end-user, at station sta1, is connected to a legacy 802.11 network through a mesh access point (MAP). The MAP is forwarding/routing user data towards/from the WMN and has mesh point (MP2) as a next hop. Mesh point (MP2) has mesh point (MP4) as its next hop and MP7 afterwards. This later forwards data towards the Mesh Portal Point (MPP1) which serves as a gateway towards the Internet where the web server is. The role of the routing protocol is to determine the best sequence of next hops for a data frame to get to its nal destination. In 802.11s, routing is performed using MAC addresses and uses Airtime [10] as a routing metric.
2.2 Routing at MAC Layer
Multi-hop routing is a key element in 802.11s. 802.11s performs routing using MAC addresses and uses either four or six MAC ad-dresses depending on the nature of the trafc. When both the source and destination are mesh points in the WMN, only four addresses are used. When at least the source or the destination is outside the WMN, i.e., a non-mesh point, e.g., a 802.11s legacy station, six addresses are used. 802.11s frame header bear an AE (Address Ex-tension) mode bit that discriminates between the two modes. In the case where WMNs are used for last-mile Internet access, the AE bit is set to 1 as both the destination and the source are non-mesh points.
Adding two extra MAC addresses, to the ordinary four 802.11 addresses, is meant to track the MAC addresses of the non-mesh source and non-mesh destination as they would be lost once the frame enters/quits the mesh network. The six MAC addresses are:
1. Destination MAC Address: It corresponds to the next hop MAC address.
2. Source MAC Address: MAC address of the source.
3. Destination Proxy Address: The MAC address of the MPP/MAP from where the frame leaves the mesh network.
4. Source Proxy Address: The MAC address of the MPP/MAP from where the frame enters to the mesh network.
5. Final Destination Address: The MAC address of the nal non-mesh destination.
6. Originator Address: The MAC address of the non-mesh sta-tion that originated the frame.
IEEE 80.2.11s set HWMP [11] as a mandatory routing proto-col for all 802.11s compliant devices. Furthermore, Airtime [10] was set as a mandatory routing metric too. This is in order to en-sure compatibility between different 802.11s devices manufactur-ers. HWMP and Airtime are detailed in next sections.
3. MAC ROUTING IN 802.11S
HWMP is a hybrid protocol as it is reactive and a proactive. The two behaviors can be used separately or at the same time depending on the application.
3.1 Reactive Routing in HWMP
RM-AODV (Radio-Metric Ad hoc On Demand Distance Vec-tor) [14] is the reactive protocol in HWMP. RM-AODV is an adap-tation of the AODV [19] protocol that uses Airtime [10] as link quality metric. Reactive routing protocols initiate route discovery requests only when needed, e.g., route failure.
In RM-AODV, when a node needs a path to a certain destina-tion, it broadcasts a PREQ (Path Request) message that contains the MAC address of the destination. Every PREQ message bears a sequence number that uniquely distinguishes it from other PREQ messages. When an intermediate MP receives a PREQ message, it rst checks the freshness of the received message by comparing its sequence number with the last locally stored sequence number: A PREQ message is processed only when it has a greater sequence number or when it has the same sequence number but exhibiting a better route. The sequence numbers are used to keep the network loop-free.
After receiving a fresher PREQ message, intermediate nodes up-date their reverse path towards the source, upup-date the routing met-ric by including the weight of the last hop then re-broadcast the updated PREQ message. Subsequent MPs proceed the same way until the PREQ reaches the destination.
When the destination MP receives the PREQ message, it checks its freshness the same way as other intermediate MPs did, then it updates its reverse path towards the source. The destination then formulates a RREP (Route Reply) message which is after-wards uni-casted toafter-wards the source. Intermediate MPs receiving the RREP message update their forward path towards the destina-tion, update the routing metric then forward the updated RREP to-wards the source.
In case of node failure, the intermediate MPs detecting the failure send a RERR (Route Error) message towards the source to inform it about the breakage of the link. The source can then re-initiate a new route discovery using a new PREQ message.
3.2 Proactive Routing in HWMP
HMWP has two proactive modes [11]: Proactive PREQ and Proac-tive RANN. The two protocols are variations of tree based rout-ing [20, 21]
3.2.1 Proactive PREQ Mode
In the proactive PREQ mode, every root MP periodically broad-casts PREQ messages bearing unique sequence numbers. The pro-tocol is similar to reactive HWMP, presented in Section 3.1, in how intermediate nodes react and process PREQ messages. The only difference is that in reactive HWMP, PERQ messages are sent in a reactive way, i.e. when needed, whereas in proactive PREQ, PREQ messages are sent proactively, i.e., in advance and in a periodic ba-sis.
3.2.2 Proactive RANN Mode
Like in proactive PREQ mode, MPPs periodically broadcast RANN (Route Announcement) messages with unique sequence numbers. However, RANN messages differ from PREQ messages in that they are not meant for requesting paths, they are instead meant only to communicate routing metrics in the network.
Upon the reception of a RANN, the intermediate MPs re-broadcast the RANN message only if it corresponds to a newer RANN mes-sage or to a RANN mesmes-sage with same sequence number but with better routing metric. Furthermore, if an intermediate MP nds the new RANN advising a better route to the root, it then sends a uni-cast PREQ message which sets the forward path to the root MP. Upon the reception of the PREQ message, the root MP replies with a unicast PREP (Path Reply) which sets the reverse path to the root MP.
3.2.3 Which protocol to use in case of WMNs for
last-mile Internet access?
As illustrated in previous sections, HWMP advises three proto-cols in total. The question is which protocol to use and can they be used together?
In fact, the use of these different protocols is for the purpose of coping with the wide range of applications that WMNs may sup-port. Hence, the protocol to use depends on the kind of applica-tion. For instance, in case of wireless enterprise networks, where enterprise stations are mostly communicating between each other, the reactive HWMP is the appropriate protocol to use, and this is due basically to the random pattern of trafc in the network which involves high dynamism in the network in terms of link quality. Furthermore, if the enterprise network is to be connected to the In-ternet, which is usually the case, then the reactive AODV must be combined with proactive PREQ or proactive RANN, since a good part of the trafc will be always directed towards/from the gate-ways connecting the enterprise to Internet.
If the WMN is solely used for last-mile Internet access, the very specicity of the trafc which is always directed from/towards the MPPs and which shapes a tree topology advocates the use of a tree-based routing protocol where tree roots, i.e., Internet gateways, would disseminate broadcast messages for root selection. Both proactive PREQ and proactive RANN are tree-based routing varia-tions. The remaining point is which protocol to use? i.e., proactive PREQ or proactive RANN?
The proactive RANN and proactive PREQ protocols share the same principle of periodical broadcast of root advertisement mes-sages. However, and to our understanding, the proactive PREQ can be viewed as a lesser overhead version of the proactive RANN since this later broadcasts RANN messages in addition to PREQ and PREP messages. Proactive RANN would be more appropriate for networks exhibiting a dense trafc and/or more dynamism in trafc pattern. We would advise, for instance, the usage of proac-tive RANN instead of proacproac-tive PREQ in the case where the WMN is witnessing a dense trafc or when the WMN has many gateways to Internet. In case of a unique gateway or even a couple of gate-ways, proactive PREQ is quite sufcient and exhibits less overhead in terms of minimal control frames.
To conclude, there is a trade-off deal between the adoption of proactive PREQ or proactive RANN. The rst one exhibits less overhead while the other one has more overhead but allows more adaptation to network dynamism: The kind of the application is to decide.
3.3 802.11s Radio-Aware Airtime Metric
802.11s set Airtime as a default routing metric for all 802.11s compliant devices. Airtime metric is a radio-aware metric which is meant to measure the amount of consumed channel resources when transmitting a frame over a particular wireless link [10]. Airtime is computed as follows: Airtime = Oca+ Op+Bt r 1 1 − ef r (1)
where Oca, Opand Btare constants quantifying respectively the
Channel Access Overhead, the Protocol Overhead and the number of Bits in a probe frame. r is the transmission rate (in Mbps) for a frame of size Btand frame error rate ef r. IEEE 802.11s did not
set a specic way to measure ef r, it is left as an implementation
issue [8].
Unlike ETX (Expected Transmission Count) [12] which accounts solely for frame error rate, Airtime accounts for both frame error rate and link bandwidth as well, which is the case with ETT (Ex-pected Transmission Time) [22] as well. However, Airtime does further account for channel and protocol overhead. The next sec-tions highlight ETX and ETT main traits.
3.3.1 ETX (Expected Transmission Count)
De Couto et. al proposed ETX [12]. ETX accounts solely for frame loss rate and does so by measuring the expected number of transmissions needed to successfully transmit a frame. To com-pute ETX, every node periodically broadcasts a constant number of probe frames N over a certain constant time period T. The receiver counts the number of received frames and computes the forward delivery ratio as follows:
df =Rf
N (2)
where Rf is the number of received frames in the forward
direc-tion.
When the receiver broadcasts its N probe frames over the same period T, it does piggyback in its probe frames the computed df
values for all its neighbors. Upon receiving a probe frame, every node counts the received probe frames in order to compute the re-verse delivery ratio dr- in a similar way to computing df, equation
(2) - and does at the same time extract the corresponding embedded df value that was computed at the sender. This way every node has
both the forward and reverse delivery ratios. These later are used to compute ETX as follows:
ET X = 1
df × dr (3)
However, ETX suffers from the major shortcoming of not account-ing for link bandwidth. To cope with this problem ETT has been proposed.
3.3.2 ETT (Expected Transmission Time)
R. Draves et al. proposed ETT [22]. ETT is meant to cope with the major shortcoming of ETX in not accounting for link band-width. In [22] authors are aliasing ETT as Bandwidth-adjusted ETX. ETT accounts for both frame loss rate and bandwidth as well. ETT measures the needed time for a frame to be successfully transmitted. ETT is computed as follows:
ET T = ET X × t (4)
where t is the average time for a single frame to be transmitted regardless of the transmission being it successful or not.
t = S
B where S is the size of the probe frame and B is the link bandwidth. To compute t, R. Draves et al. are using the well-known technique of packet-pairs [16].
4. IMPLEMENTATION
4.1 System Design
As mentioned earlier, IEEE 802.11s performs routing at the MAC layer. We tried an implementation that can be deployed using off-the-shelf 802.11 commodity hardware, i.e., it does not require any change to the 802.11 driver. In fact, an optimal implementation would require a thorough change of the MAC driver as 802.11s uses six MAC addresses instead of the four ones which are used in 802.11. Furthermore, the code should be implemented at the Ker-nel level in order to minimize the context switchings involved with user processes, especially when accessing network data. Still, for our implementation to approximate as much as possible an opti-mal MAC protocol implementation, we deployed two sopti-mall kernel modules that drop all trafc going to/from the TCP-IP stack. These kernel modules are placed at the PRE-ROUTING and LOCAL-OUT hooks of Netlter [5], see Figure 3. Besides, by using Datalink Raw Sockets, the user-space generated 802.11s frames bypasses the TCP-IP stack. This way, the TCP-IP stack becomes transparent to our user-space implementation, hence ideally approximating a ker-nel implementation.
The system is made of three modules, see Figure 2, that commu-nicate between each other using IPC (Inter Process Communica-tion) shared memories: 1. Routing, 2. Data forwarding and 3. Link quality measurements. Only the last module (link quality measure-ment) is identical in all WMN nodes, i.e., MAPs, MPs and MPPs. For the two other modules, their implementation depends on the type of the node. Next sections detail implementations for each module and highlights how the implementations differ according to the type of the mesh point.
4.2 Data Forwarding Module
The Data Forwarding module implementation depends on the type of the mesh node.
4.2.1 Data Forwarding in MPPs
MPPs have two network interfaces: an 802.3 interface and an 802.11 ad-hoc interface, a fact which results in processing two different types of frames.
• Upon reception of an 802.3 frame, the MPP-Data-Forwarding module extracts the destination IP address from the 802.3 frame then fetches the corresponding MAC address and the associated MAP MAC address from the MPP Association Ta-ble (Association taTa-bles will be further highlighted in in Sec-tion 5 when addressing implementaSec-tion problems). Once the destination MAP MAC address is retrieved, the correspond-ing next hop MAC address is retrieved too from the MPP proxying table and the appropriate 802.11s header frame is built. This later contains basically the six MAC addresses that will be used to get the frame, through the 802.11s ad-hoc network, to its nal destination. The six MAC addresses are built as follows:
1. Destination MAC address: corresponds to the next-hop MAC address retrieved from the MPP proxying table 2. Source MAC address: The MAC address of the sender,
i.e., the MPP MAC address in this case
3. Destination Proxy MAC address: The MAC address of the corresponding MAP.
4. Source Proxy MAC address: The MAC address of the MPP
5. Final Destination MAC address: The destination MAC address which is retrieved from the local association table.
6. Originator MAC address: The source MAC address in the received 802.3 frame
The MPP then strips off the 802.3 frame header and re-places it with the 802.11s frame header while maintaining intact the payload of the 802.3 frame. The resulting 802.11s frame is then forwarded using the MPP 802.11 interface. • Upon reception of an 802.11s frame in its ad-hoc interface,
the MPP reads the 802.11s header frame, gets the originator MAC address and its IP address and creates an entry for it in the MPP association table. The entry contains the origi-nator IP address, the origiorigi-nator MAC address and the asso-ciated MAP MAC address. Afterwards, the MPP strips off the 802.11s frame header and replaces it with a 802.3 frame header containing as a source address the local MPP MAC address.
4.2.2 Data Forwarding in MPs
In contrast to MPPs, Data Forwarding in MPs is quite straightfor-ward as it deals only with one type of frames, i.e., 802.11s frames. However, the MP still receives two types of frames that differ in terms of their nal destination which can be either in an 802.3 net-work (e.g., frame destined to Internet) or an 802.11 netnet-work (frame destined to a Wi-Fi station). In other words, the two frames can differ in terms of whether their nal destination, within the WMN, is an MPP or a MAP?
The proxy type, i.e., a MPP or an MPP, is made transparent in our implementation by having the same type of entry in the MP forwarding table for all proxies.
Upon reception of an 802.11s frame, the MP extracts the destina-tion proxy address, i.e., the MPP or MAP MAC address, consults its forwarding table and retrieves the corresponding entry which contains the next hop MAC address. The 802.11s frame is then updated by changing only two of the six MAC addresses in the 802.11s frame header: The remaining four ones are kept intact. The two updated MAC addresses are the destination MAC address and the source MAC address which become respectively the retrieved next hop MAC address and the MP MAC address.
4.2.3 Data Forwarding in MAPs
MAPs have two network interfaces, a managed mode (Access point) 802.11 interface and an ad-hoc mode 802.11 interface, which re-sults in processing two different types of frames.
• Upon reception on an 802.11 frame, from its Access Point interface, the MAP rst reads the source MAC address, and formulates the six MAC addresses in the 802.11s header frame as follows:
1. Destination MAC address: It corresponds to the next hop MAC address, towards the best MPP, which is re-trieved from the MAP Proxying-Table.
2. Source MAC address: the MAC address of the local MAP.
Figure 2: System Design
Figure 3: 802.11s MAC Routing and the OSI Model
3. Destination Proxy Address: the MAC address of the MPP to which to forward the frame. It is retrieved from the MAP proxying table
4. Source Proxy Address: the MAC address of Local MAP 5. Final Destination: Unknown at this stage. A NULL
address is put in.
6. Originator: the source MAC address in the received 802.11 frame.
The MAP then strips off the 802.11 header, replaces it with the just shaped 802.11s header and forwards the frame through its ad-hoc 802.11 interface.
• On the other side, upon reception of a 802.11s frame in its ad-hoc interface, the 802.11s header frame is stripped off and replaced by a broadcast 802.11 header frame. Thereafter, the nal 802.11 legacy destination will look up its IP address in the broadcasted 802.11 frame and thus process the frame.
4.3 Routing Module
The routing module implementation depends on the type of the mesh node.
4.3.1 Routing in MPPs
In MPPs, the routing module is made of two sub-modules:
1. MPP-PREQ-Broadcast Module: periodically broadcasts PREQ messages which contain basically the MAC address of the local MPP, an incremental sequence number and a routing metric eld.
2. The MPP-RREP-Processing Module: processes RREP mes-sages forwarded by intermediate MPs and originated from different MAPs. Upon reception of a RREP message, the MPP gets the MAC address of the sender MAP, looks up its entry it the MPP proxying table and updates the corre-sponding routing metric and the correcorre-sponding next hop. The RREP messages do set the forward path towards MAPs.
4.3.2 Routing in MPs
In MPs, the MP-PREQ-Processing module processes two types of messages: PREQ and RREP messages.
• Upon reception of a PREQ message, the module checks rst the freshness of the message in order to process it as only fresher messages must be considered. The module extracts rst the MAC address of the sender MPP, from the PREQ message, then uses this address to look up its correspond-ing entry in the MP forwardcorrespond-ing table. The MP forward-ing table entries store, among other values, the last fresh sequence numbers of the concerned MPPs. Once the last fresh sequence number is retrieved, the PREQ message is discarded in case it corresponds to a non-fresh message (i.e., of a smaller sequence number). If the PREQ message is a fresher one then the routing module updates its reverse path entry towards the concerned MPP. Afterwards, the PREQ message is updated by adjusting the routing metric value: The module retrieves the corresponding routing metric value, e.g. ETX or Airtime, for the link from where it received the PREQ message and adds it to the current PREQ routing met-ric then rebroadcast the message. If the PREQ message is of equal freshness as the current sequence number, the PREQ message is then processed only and only if it corresponds to a better route. If it is the case, the PREQ is processed as if it was a fresh PREQ message.
• Upon reception of a RREP message, the MP gets the MAC address of the sending MAP, looks up its entry in the for-warding table, and then updates the corresponding forward path to the MAP. To forward the RREP message, the MP looks up the destination MPP MAC address and retrieves its entry from the forwarding table. The retrieved entry contains the MAC address of the next hop in the route towards the concerned MPP. Afterwards, the routing metric in the RREP message is updated and then forwarded to the next hop.
4.3.3 Routing in MAPs
Upon reception of a PREQ message, the MAP-PREQ-Processing module retrieves the MAC address of the MPP which issued the PREQ message, retrieves the entry, from the proxying table, that corresponds to the sender MPP and then checks the freshness of the current PREQ message against the already stored freshness. If the PREQ message corresponds to a fresher route or corresponds to an equal freshness route with better routing metric, the corresponding entry in the MAP proxying table is updated and the routing metric value is updated too the same way as with MPs.
However, the processed PREQ message can correspond to only a local better route, i.e., the best route leading to a specic MPP.
802.11a 802.11b/g Oca 75 µs 335 µs
Op 110 µs 364 µs
Table 1: Airtime metric constants
Hence, there may be better routes to other MPPs. Selecting the MPP that has the best routing metric would fail in case the entry was updated a long time ago or in case the MPP is off (This lem will be further highlighted in Section 5). To address this prob-lem, we timestamp all MPP routing entries in the proxying table. The timestamps correspond to the time at which the MPP routing entry was last updated. This way we select the MPP that has the better route as well as a fresher route by comparing its timestamp to the current time. We set a time threshold which is twice the MPP broadcast periods. We choose a twice period to account for the case when a PREQ broadcast has been accidentally lost. The best MPP is marked in the proxying table.
Once the proxying table is updated, a RREP message is formed and sent back to the selected MPP. The RREP message acknowl-edges the receipt of the MPP PREQ message and it plays also the role of an association message telling the MPP to associate the MAP with it. The RREP message bears mainly the MAC address of the MAP as well as the corresponding routing metric.
4.4 Airtime Measurement Module
As presented in Section 3.3, Airtime is computed as follows: Airtime = Oca+ Op+Bt r 1 1 − ef r (5)
Based on Equation 5, airtime depends on two basic variables: The frame error rate ef rand the bit rate r. The other constants are
given in Table 1.
To compute the frame error rate, we used the techniques high-lighted in ETX [12] in order to compute the reverse and the forward delivery ratios. The ETX module implementation is highlighted in next section.
Once the forward and reverse delivery ratios dfand drare
avail-able, the frame error rate is approximated as follows: ef r= (1 − df) × (1 − dd)
To compute the bit rates we used the "/Proc" system les: The rate adaptation algorithm is SampleRate [3].
4.4.1 ETX Module
To compute ETX, we need both the forward and the reverse delivery ratios. Our implementation of the ETX module consists on two sub-modules: The Broadcast module and the ETX-Collect module.
• The ETX-Broadcast module periodically broadcasts probe frames embedding the number of probe frames received in the reverse path from neighboring MPs during a certain time period T . The ETX-Broadcast gets these values from the ETX-Collect module.
• The ETX-Collect module is the module that computes the ETX values towards every neighbor. To get the reverse de-livery ratios, the module counts the probe frames broadcasted by every neighboring MP during a certain period of time T. Once computed, the delivery ratios are sent to the ETX-Broadcast module which embeds them in the broadcasted
probe frames. While reading the received probe frames, ev-ery ETX-module extracts his corresponding forward delivev-ery ratios that were embedded by the ETX-Broadcast modules of the neighboring MPs.
5. IMPLEMENTATION PROBLEMS
During implementation, we faced three basic problems: 1. clients association, 2. internetworking and 3. addressing multiple gate-ways. The next sections highlight these problems and suggest so-lutions.
5.1 The Clients Association Problem
The clients association problem arises when an 802.11s frame leaves the WMN, i.e., passes through the MPP gateway. In such a scenario, the MAC address of the source 802.11 legacy station as well as the MAC address of the MAP who originated the frame are lost forever since the IEEE 802.11s frame headers would be stripped off at the level of the gateway. The problem is that when the MPP will receive back a frame as a response to the already sent frame, the MPP cannot remember which MAP sent the frame. As a consequence, the MPP cannot forward the frame to the right des-tination as it does not know the MAC address of the corresponding MAP.
To solve this problem, we propose two alternatives: 1. Forcing every MAP to periodically broadcast the IP and the MAC addresses of its associated legacy 802.11 stations, 2. Having the MPP extract the needed information from forwarded frames and store them in a table.
Both alternatives would meet the goal of having the MPP remem-ber the MAP from which it received the concerned frame. How-ever, we opted for the second alternative as the rst one would in-duce more overhead in the network because of the broadcast nature of the approach. Besides, the rst approach becomes delicate in the case where the legacy 802.11 stations are mobile. In this later case, a MAP may broadcast that a station is associated with it while the station did already moved to another MAP coverage area. In other words, the rst approach does not handle hand-offs.
In the adopted approach, the MPP extracts the following data: Source IP address, source MAC address and the MAC address of the associated MAP. The extracted data is stored in a table. To cope with the look up time of the table which is of 0(n), we implemented the association table as a hash table with a chained list as a colli-sion resolution strategy. We simplied the hash function to allow for further alleviation in terms of computational speed:
Hash(IPaddr) = IPaddr[3] % 256
This way, when the MPP receives an incoming packet, the corre-sponding MAC addresses of both the 802.11 legacy station and the associating MAP can be easily retrieved, from the association ta-ble, to be used in formulating the appropriate 802.11s MAC header. This later will be used to forward the received packet till the in-tended MAP. The MAP, in its turn, will deliver the packet to the right destination using the MAC address of the 802.11 legacy sta-tion which was retrieved from the associasta-tion table too.
5.2 Interconnecting WMNs
The previously addressed problem, i.e., clients association, is meant basically to store/retrieve the MAP where a legacy 802.11 station resides. When trying to Interconnect WMNs, another prob-lem arises and it is due this time to the irrelevance of the IP ad-dresses of the legacy 802.11 stations outside the MWN, i.e., in external networks. This irrelevance stems from the fact that the
802.11 legacy stations would have IP addresses that are unknown to external networks as they are not directly connected to these net-works, and hence not contributing in the external networks routing protocols. To solve this problem we implemented a NAT-like (Net-work Address Translation) mechanism at the level of the WMN Gateways (i.e., MPPs). This later have two interfaces, a WMN in-terface and an 802.3 inin-terface.
Accordingly, when a frame is about to leave the WMN towards another network, e.g., Internet, the MPP gateway reads the IP ad-dress of the originator, i.e., the 802.11 legacy station, and replaces it by its own IP address. The packet is then sent. However, since there will be other frames originating from different WMN clients, replacing only the IP address is not sufcient as there should be a way for a unique mapping back to the original IP address. To solve this problem, we implemented another hash table, that has as a hash key the combination between the source port, the destination port and the destination IP address. This way, when a connection is created, e.g., TCP or UDP, an entry in the hash table is created con-stituting of the following quadruplet (Destination IP, Original IP, Destination Port, Source Port). When a response packet is received back, the gateway maps to the right entry in the hash table using the destination IP, the source port and the destination port. This way the original IP is retrieved. The original IP is then used to map into the association table in order to get its corresponding MAC address as well as the MAC address of the MAP where the original legacy station resides.
Even if the presented NATing framework seems logic, this ap-proach did not work at rst: We realized that the gateway (MPP), when receiving a response packet, especially in TCP, it does reply to the packet and establishes a TCP connection for itself as well. Hence resulting in sending duplicate packets (one from the MPP and one from the legacy station) as a response to a single TCP con-nection establishment. The problem was easily solved when we de-ployed the Netlter [5] modules, presented in Section 4.1, as they drop all IP packets destined to the gateway. However, the received packets, can still be forwarded to the user since the framework uses Raw Datalink sockets. These later read frames at the MAC layer, i.e., before the Netlter module drops the packets.
5.3 Addressing Multiple Gateways (MPPs)
When processing HWMP PREQ (Path REQuest) messages, a MAP has to process only fresher PREQ messages, i.e., the ones that bear a larger or equal sequence number than the last fresher route. However, when a fresher PREQ message is processed and the corresponding routing entry is updated, a problem arises when the wireless mesh network has multiple MPPs as there will be dif-ferent sets of PREQ sequence numbers. Each set pertains to a single MPP: a PREQ message is uniquely broadcasted by one MPP.
Accordingly, when selecting a fresh sequence number, this later would correspond only to a better route toward a specic MPP, i.e., the MPP which originated the concerned PREQ message. The sit-uation becomes quite critical because of the likeliness of having alternate routes to other MPPs with a better routing metric. Be-sides, since PREQ messages bear sequence numbers that cannot be synchronized as they are sent from different MPPs, comparing the freshness of sequence numbers issued from different MPPs is nonsense. Furthermore, if we just select the MPP that has the best route, this may not work since the selected route may shortly be-come an old route and hence invalid.
To cope with this problem, we timestamp every routing entry with the time when the entry was created. This way, we select the MPP that has the best route as well as a still-valid route. To deter-mine the validityof a route we heuristically set a threshold which
depends on the MPP broadcast period. The experimental value of the threshold was presented in Section 4.3.3.
6. TESTBED DEPLOYMENT INSTRUCTIONS
All the les referred to in this section are available online [7] and use the same names.
6.1 Deploying the Netlter Modules
Two Netlter kernel modules must placed at the PRE-ROUTING and LOCAL-OUT hooks of Netlter as explained in Section 4.1. To hook the two modules, do "make" and "insmod" the following two kernel les: "preModule.c" and "localOutModule.c".
6.2 Deploying Mesh Access Points (MAPs)
• Deploying the Data Forwarding module: Compile and run the "MAP_ Data_Forwarding.c" le.
• Deploying the Routing module: The routing module depends on the underlying "Link Quality Measurement" module as the two modules do exchange link quality values through the IPC (Inter process communication) mechanism. Hence, de-pending on whether the WMN is using ETX or Airtime as a routing metric, you need to compile and run one of the fol-lowing les:"MAP_PREQ _Processing_ETX.c" or "MAP_PREQ_ Processing_Airtime.c".
• Deploying the Link Quality Measurement module: The de-ployment of this module is the same for all of the three types of mesh node, i.e., MAPs, MPPs and MPs. Hence, it will be separately covered in section 6.5
6.3 Deploying Mesh Portal Points (MPPs)
• Deploying the Data Forwarding module: Compile and run the "MPP_ Data_Forwarding.c" le.
• Deploying the Routing module: Compile and run the follow-ing two les: "MPP_PREQ_Broadcast.c" and "MPP_RREP _Processing.c".
• Deploying the Link Quality Measurement module: See Sec-tion 6.5
6.4 Deploying Mesh Points (MPs)
• Deploying the Data Forwarding module: Compile and run the "MP_ Data_Forwarding.c" le.
• Deploying the Routing module: As with MAPs, you need to compile one of the following two les depending on whether the WMN is using ETX or Airtime as a routing metric: "MP_ PREQ_Processing_ETX.c" or "MP_PREQ_Processing_Airtime.c". • Deploying the Link Quality Measurement module: See
Sec-tion 6.5
6.5 Link Quality Measurement
This module is the same for all WMN nodes and it must be de-ployed in all WMN nodes. The deployment of this module depends on whether the WMN is using ETX or Airtime as a routing metric: • Deploying ETX: Compile and run the following two les:
Figure 4: WMN Testbed, Topology 1 Figure 5: WMN Testbed, Topology 2 • Deploying Airtime: Compile and run the "Airtime.c" le.
However, you need to compile and run also the "ETX_Broadcast.c" and the "ETX_Collect.c" les as the Airtime module uses ETX for error frame computation. See Section XXX
7. EXPERIMENTS
It is very important to remind that the purpose of this work is not to evaluate the performance of IEEE 802.11s WMNs. Instead, it is presenting a real-world IEEE 802.11s WMN testbed implementa-tion. However, we do go through some basic experiments for the purpose of implementation validation. In the following paragraphs, we describe the topological and experimental settings of the testbed and we compare the performance of IEEE 802.11s with Airtime as a routing metric against ETX.
7.1 Topologies
We used two different topologies for running experiments: 1. In the rst topology, see Figure 4, we deployed a wireless
mesh network composed of 11 mesh nodes. This later is lo-cated in the same building and is connected to the Internet. 2. In the second topology, we connected two WMNs, see
Fig-ure 5, through Internet. The rst WMN is composed of 7 nodes, and the other WMN is composed of 8 nodes. The two WMNs are located in different buildings.
Both topologies were deployed at the second oor of the Shelby center at Auburn university.
Besides the WMN nodes, we deployed an 802.11 legacy station (which serves as a client and is connected to the WMN through a MAP) and a 802.3 station (which serves as a server). Using Iperf [2], we created UDP and TCP connections between the client and the server. In topology 1, the server resides on the Internet while in topology 2 the server lies on a WMN.
7.2 Settings
In both topologies, the mesh nodes network interface cards (NICs) are operating in the 802.11g channel 1 while the 802.11 clients NICs are operating in the 802.11g channel 11. The selection of different channels for client stations and mesh nodes is for the pur-pose of minimizing interferences between the client and the mesh nodes.
The mesh nodes NICs run the open-source Madwi driver [4]:
For a list of NICs that are compatible with Madwi, please refer to [1]. The HWMP PREQ messages were periodically sent every 4 seconds and the ETX probe frames are 1024 Bytes long and are periodically sent every 1 second [12].
During experiments, the TCP and UDP sessions were repeatedly run over periods of 20 seconds and averaged to plot the network throughput in function of client bandwidth.
7.3 Results
First, during experimentation, we noticed a "ping-pong" effect in the behavior of the Airtime metric. This behavior is basically due to Airtime implicitly accounting for the load by means of account-ing for the bit rate: The bit rate is dynamic as Madwi uses the default SampleRate rate adaptation algorithm [3] which adapts the transmission bit rate according to the observed link quality. Thus, when Airtime advises a less-loaded path, this later will soon be-come loaded, a fact that incites Airtime to favor another less-loaded path which will become soon loaded too, thus resulting in a "ping-pong" effect. Hence, we strongly advise for a more stable routing metric.
Besides, we do clearly notice that, in both topologies, Airtime and ETX have a quite similar performance for both TCP and UDP, see Figures [6, 7, 8, 9] : We depicted the graphs throughput in a scale of [0, 5] Mbps in order to relatively match the large scale of bandwidth which is [0, 10] Mbps, and this in order to give also an idea about the network efciency:
Ef f iciency = T hroughput/Bandwidth
In fact, we cannot denitively state that Airtime and ETX have quite similar performance simply because the topologies we de-ployed have a quite limited set of alternative paths: In other words, Airtime and ETX are likely to pick the same hops as there are not many alternatives. Deploying larger topologies, with a larger set of alternative paths, is crucial to ascertain the performances of Air-time and ETX with regard to each other.
The actual performance is quite poor when comparing the net-work throughput to client bandwidth, i.e., very low netnet-work ef-ciency. We explain this poor performance by the following two main reasons:
• The testbed is using only a single channel: Performance degra-dation is a very known fact in single-channel single-radio multi-hop wireless networks [15], e.g., WMNs. This is mainly due to the fact that neighboring nodes, which are in the car-rier sense range of each other, cannot transmit
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 1 2 3 4 5 6 7 8 9 10 UDP Throughput (Mbps) Bandwidth (Mbps) UDP Throughput: Topology 1
ETX Airtime
Figure 6: Topology 1 - UDP Throughput
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 10 20 30 40 50 60 70 80 90 100 TCP Throughput (Mbps) Bandwidth (Mbps) TCP Throughput: Topology 1 ETX Airtime
Figure 7: Topology 1 - TCP Throughput
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 1 2 3 4 5 6 7 8 9 10 UDP Throughput (Mbps) Bandwidth (Mbps) UDP Throughput: Topology 2
ETX Airtime
Figure 8: Topology 2 - UDP Throughput
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 10 20 30 40 50 60 70 80 90 100 TCP Throughput (Mbps) Bandwidth (Mbps) TCP Throughput: Topology 2 ETX Airtime
Figure 9: Topology 2 - TCP Throughput ously. This results in having only one active hop, in a route,
at a time while the two other adjacent hops (always in the same route) cannot be active, a fact which noticeably affects network throughput. Furthermore, the nodes that are in the interferences range of each other would noticeably affect the network throughput as they would cause additional further bit errors. In this context, the multi-hop wireless researchers have been extensively investigating the use of multi-channels and multi-radios [17, 21, 23] in order to cope for the lim-itations of using single-radios. Actually, we are intending to have this open-source testbed support the use of multi-channels and multi-radios in our future work.
• In the deployed topologies, the WMN nodes have been de-ployed in a quite dense region (See Figures 4, 5), a fact that renders links quite close to each others. In fact, this was lim-ited by the available space. Ideally, the WMN nodes should be geometrically scattered and hence relatively far from each other in order to minimize the Interferences as well as to fa-vor simultaneous transmissions between links.
Still, we need to remind that evaluating the performance of IEEE 802.11s is out of the scope of this work.
8. CONCLUSION AND FUTURE WORK
We presented a real-world IEEE 802.11s WMN testbed imple-mentation which is an open-source and an easy-to-deploy one. We highlighted most important implementation problems and suggested suitable solutions. The implementation is an easy-to-deploy one as it does not involve any 802.11 MAC driver alteration, thus it can be deployed using using off-the-shelf 802.11 wireless cards. The implementation was made open-source and transparent by making the full code available online.
The implemented testbed uses 11 nodes in a rst topology and 15 nodes in a second topology where two WMNs, located in
sep-arate locations, are interconnected through Internet. The testbed is operational and supports real-world trafc, e.g., supporting web browsing sessions initiated from legacy 802.11 clients, that are con-nected to the Internet through the WMN.
We have a strong belief that the presented implementation will be of great input to the WMN research community, especially in academia, as it easily provides a way for deploying a testbed and hence being able to perform more concise and real-world experi-mental research.
The experiments we lead with HWMP, and with Airtime as a routing metric, showed a clear "ping-pong" behavior. On the other hand, ETX and Airtime exhibited quite similar performances.
As a future work, and after getting feedback from interested WMN research community about the testbed, we aim to contin-uously maintain the current web site about the testbed and work for further versions with further enhancements, thus paving the way towards a widely used open-source WMN testbed. We are, also, intending to add the support for the multi-channel multi-radio tech-nology and implement a more stable routing metric: A more stable routing metric might involve "looking" for a more stable link qual-ity metric, e.g., Interferences.
9. REFERENCES
[1] Hardware supported by madwi.
http://madwi-project.org/wiki/Compatibility. [2] The iperf project. http://dast.nlanr.net/Projects/Iperf/. [3] Madwi bit-rate selection algorithms.
http://madwi-project.org/wiki/UserDocs/RateControl. [4] The madwi project. http://madwi-project.org/wiki. [5] The netler project. http://www.netlter.org/. [6] The ns-2 manual. http://www.isi.edu/nsnam/ns. [7] A pre-ieee 802.11s wirless mesh network testbed.
http://www.eng.auburn.edu/∼abidmoh/. [8] Doc ieee 802.11-07/0631r0, May 2007.
[9] W. Wang Akyildiz, X. Wang. Wireless mesh networks: a survey. Elsevier Sience, 2005.
[10] M. Bahr. Proposed routing for ieee 802.11s wlan mesh networks. WICON, 2006.
[11] M. Barh. Update on the hybrid wireless mesh protocol of ieee 802.11s. IEEE Conference on Mobile Adhoc and Sensor Systems, 2007.
[12] J. Bicket D. De Couto, D. Aguayo and R. Morris.
High-throughput path metric for multi-hop wireless routing. MOBICOM, 2003.
[13] P. Gupta and P. R. Kumar. The capacity of wireless networks. IEEE Transactions on Information Theory, 2000.
[14] Y. Kengo H. Aoki, T. Shinji and Y. Akira. Ieee 802.11s wireless mesh network technology. IEEE NTT DoCoMo Technical Journal, 2006.
[15] V. Padmanabhan L. Qiu K. Jain, J. Padhye. Impact of interference on multi-hop wireless network performance. MOBICM, 2003.
[16] S. Keshav. A control-theoretic approach to ow control. SIGCOMM, 1991.
[17] Y. Liu and E. Knightly. Opportunistic fair scheduling over multiple wireless channels. INFOCOM, 2003.
[18] Scalable Networks. Qualnet.
http://www.scalable-networks.com/products/qualnet/. [19] C. E. Perkins and E. M. Royer. Ad-hoc on-demand distance
vector routing. WMCSA, 1999.
[20] M. Lewis R. Orgier, F. Templin. Topology dissemination based on reverse-path forwarding (tbrpf). RFC 3684. IETF, 2004.
[21] A. Raniwala and T. Chiueh. Architecture and algorithms for an 802.11-based multi-channel wireless mesh network. INFOCOM, 2005.
[22] J. Padhye R.Draves and B. Zill. Routing in multi-radio, multi-hop wireless mesh networks. MOBICOM, 2004. [23] J. So and N. Vaidya. Multi-channel mac for ad hoc networks
: Handling multi-channel hidden terminals using a single transiever. MOBIHOC, 2004.
[24] IEEE TGs. Status of project ieee 802.11s.
http://www.ieee802.org/11/Reports/tgs_update.htm, January 2010.
