3.4 Simulation results
3.4.2 Performance evaluation
The evaluation scenario (Figure 3.12) is composed by a set of clients (C1, C2 and C3), each supporting a single network architecture stack (NDN, IP and PURSUIT respectively), and two NDN content providers (P1 and P2) supporting only NDN procedures. The content provider P1 is connected to the FIXP via Router 2 which supports only the NDN stack, while the remaining entities are connected to the FIXP via Router 1 which supports the IP, NDN and PURSUIT stacks. The FIXP is coupled with a simplified set of FIC functionalities to support URI generation and storage.
This scenario was implemented in a simulation environment using ns-32, enhanced with NDN (ndnSIM 2.0 [7, 74]) and PURSUIT (Blackadder3) network stacks. The simulation environment runs on a virtual machine (with two 3.33GHz CPU cores and 2GB of RAM) hosted in an OpenStack Platform.
Using the proposed framework, besides the NDN client, both IP[HTTP] and PURSUIT clients are able to acquire the web page (i.e., the HTML and PNG files with size 119 and 6132 bytes respectively) deployed in the NDN producers (i.e., P1
ns-3 - https://www.nsnam.org/
P1 NDN-only Pursuit Topology Manager & Rendezvous Router 2 FIXP 10 Gbps / 10ms 100 Mbps / 1ms C2 IP-only C1 NDN-only C3 PURSUIT-only P2 NDN-only Router 1
Figure 3.12: Evaluation Scenario
and P2 ), in a transparent way to both the end-user devices and network entities on each network architecture.
In Table 3.3, the signaling size on each network architecture (for a single link) resulting from the interoperability process is presented, with the impact on the total exchanged information when fetching the web page from P1 and P2 being shown in Figure 3.13. The time required to fetch the web page from each network architecture when content is deployed in P1 or P2 is presented in Figure 3.14. To better evaluate the impact of the proposed framework in enabling the interoperability between different network architectures, the results while fetching the web page from a NDN consumer are also presented for comparison purposes (i.e., in a scenario where no interoperability mechanisms were needed). A scenario (referred to as “IP reference”) where all entities are IP-enabled (including both providers P1 and P2 ) was also evaluated, so it could provide reference values of the current Internet approach. Finally, Figure 3.15 details the delay introduced by the FIXP due to message conversion.
Table 3.3: Signaling size regarding each network architecture (for a single link)
Total Size w/ content (bytes)
Signaling size wo/
content (bytes) Exchanged messages
C1 NDN NDN: 7078 NDN: 827 NDN: 4 C2 IP[HTTP] IP: 7345 NDN: 7078 IP: 1093 NDN: 827 IP: 19 NDN: 4 C3 PURSUIT PURSUIT: 9355 NDN: 7078 PURSUIT: 2929 NDN: 827 PURSUIT: 10 NDN: 4 IP
Reference IP: 7345 IP: 1093 IP: 19
In terms of the amount of signaling exchanged in the foreign network architecture, fetching the web page from the IP[HTTP] and PURSUIT clients, when compared with a native NDN approach, required an higher amount of exchanged messages.
3.4. Simulation results 69 0 10000 20000 30000 40000 50000
NDN-NDNIP[HTTP]-NDNPURSUIT-NDNIP reference NDN-NDNIP[HTTP]-NDNPURSUIT-NDNIP reference
Total Exchanged Information (bytes)
(a) From P1 (b) From P2 Cx - Router 1 Router 1 - FIXP Router 1 - Rendezvous/TM Router 1 - P2 FIXP - Router 2 Router 2 - P1
Figure 3.13: Total exchanged information
This changed from 4 messages in NDN to 10 and 19 messages in PURSUIT and IP[HTTP], respectively. If in the IP[HTTP] approach such increase was mainly due to the TCP control signaling as well as the configured IP MTU (due to which 5 TCP Data messages were required to transfer the whole PNG file), in the PURSUIT approach the communication with Rendezvous Node required to (un)subscribe each resource was responsible for the increase of exchanged messages. This was also reflected on the total size of the exchanged messages, representing an increase of 3.8% and 32.2% for the IP[HTTP] and PURSUIT approaches, respectively. Considering just the overhead introduced by the messages, the protocols in the IP and PURSUIT architectures exchanged, respectively, more 166 and 2102 bytes. In the original network architecture, the same NDN messages were exchanged independently of the requester and, therefore, the number of messages and their size remained unchanged.
If contents are retrieved from provider P1 (Figure 3.14a), IP[HTTP] and PURSUIT clients required more time to fetch the web page when compared with a native NDN approach, representing an increase of 99% (roughly double) and 17% respectively. The main reason for such behavior is the delay introduced by the internal mechanisms of both the IP[HTTP] and PURSUIT architectures. In the IP[HTTP] use case, if this increase was due to the TCP control signaling and the configured IP MTU (due to which 5 TCP Data messages were required to fetch the PNG file), in the PURSUIT use case it was due to the communication with the Rendezvous Node. Even so, all the previous approaches had lower fetching times in the evaluation scenario when compared
0 50 100 150 200 250
NDN-NDNIP[HTTP]-NDNPURSUIT-NDNIP reference NDN-NDNIP[HTTP]-NDNPURSUIT-NDNIP reference
Fetch time (ms)
(a) From P1 (b) From P2 HTML file
Figure 3.14: Fetching time
to the IP reference. This behavior is due to the delay introduced by the TCP control signaling raised because of the higher RTT between the TCP endpoints in the IP reference approach. Regarding the total exchanged information (Figure 3.13a) the higher amount of exchanged information to fetch the web page from IP[HTTP] and PURSUIT clients, when compared with a native NDN approach, is mainly due to the signaling overhead introduced by both HTTP (over TCP/IP) and PURSUIT protocols. When considering a nearer provider P2 and comparing with a native NDN approach, the gap in terms of fetching time (Figure 3.14b) and total exchanged information (Figure 3.13b) while acquiring the web page from IP[HTTP] and PURSUIT clients is even higher. This is due to the anchoring-related issues (i.e., longer and/or sub-optimal forwarding paths) introduced the proposed framework. While the messages in the NDN-only approach only go through Router1 between the client and the provider P2 (i.e., C1↔Router1↔P2 ), in IP[HTTP] and PURSUIT approaches messages need to be forwarded to the FIXP in order to be converted to compatible messages in the destination network architecture (i.e., C2/C3↔Router1↔FIXP↔Router1↔P2 ).
The delay introduced by the FIXP while converting messages from one architecture to the other (Figure 3.15) is approximately 1.3% for IP[HTTP] and 3% for PURSUIT of the total communication time. Converting IP[HTTP] or PURSUIT request messages to be compatible with the NDN architecture (i.e., to NDN Interest messages) is less time-consuming than the reverse process (i.e., from an NDN Data messages), mainly due to integrity check mechanisms. This is clearer in the IP[HTTP] use case, because
3.5. Proof-of-concept prototype results 71 0 0.2 0.4 0.6 0.8 1 1.2
FIXP∆1 FIXP∆2 FIXP∆3 FIXP∆4
Processing time (ms)
Handle received packet FIXP Processing Create destination packet
Figure 3.15: Detailed FIXP delay. (FIXP∆1: Conversion from IP[HTTP] Request to NDN Interest; FIXP∆2: Conversion from NDN Data to IP[HTTP] Response; FIXP∆3: Conversion from PURSUIT Start Publish to NDN Interest; FIXP∆4: Conversion from NDN Data to PURSUIT Publish Data)
the conversion from IP[HTTP] to a NDN Interest message is straight forward (i.e., integrity of received IP[HTTP] messages is not verified), while the other way around the integrity of the contents contained in the NDN Data message are verified before converting the message to an IP[HTTP] response. This gap is smaller in the PURSUIT case because, in contrast with the previous case, the integrity of the PURSUIT message is also verified before being converted to a NDN Interest message.
Proof-of-concept prototype results
In this section, the implementation results for the proof-of-concept prototype of the proposed framework are presented, focusing on three use cases. This evaluation aims not only to demonstrate the feasibility of the proposed framework in supporting interoperability between different network architectures, but also to provide an assessment on the performance of such operation, serving as baseline for improvements in future research. This evaluation is mainly focused on performing a functionality validation in typical use case scenarios, highlighting the impact introduced by the FIXP and its conversion procedures between network architectures.