Web-based Automatic Network Discovery/Map
Systems
Chakchai So-In, Chinnakorn Netphakdee, Kasidit Wijitsopon,
Chavalit Panichayanubal and Pusadee Seresangtakul
Department of Computer Science, Faculty of Science, Khon Kaen University
Maung, Khon Kaen, Thailand, 40002
Abstract—This paper introduces a new automatic network discovery/map system via Web architecture, the so-called Web-based Automatic Network discovery/Map Systems (WANMS). The system functions as a plug-in for a well-known network management system, Cacti. Enriched features, especially the automatic networking discovery and map module, have been added in order to enhance the efficiency of Cacti embedded with a weather map plug-in. With user defined-pattern functions, the discovery process recognizes a proper type of networking devices. The discovery process generates XML-based information so that a graph visualization using jQuery via HTML5 can generate a simplified network map using a force-directed layout technique. WANMS is easy-to-use and less complicated, and so lessening the discovery time. In particular, for performance comparison with OpenMMS and DNMA, WANMS outperforms others in several perspectives, i.e., faster convergence, better coverage, and more details of types of networking devices. The system is presently used at the department of Information Technology, Provincial Police Region 4, Thailand.
Keywords—Network Management System; NMS; WANMS; Web-based Automatic Network discovery/Map Systems; Network Discovery; Network Map; SNMP; MIB; FCAPS
I. INTRODUCTION
†
Nowadays, almost all organizations (either academic or industry) are not only linked within their own departments but also joined into a bigger network, and resulting into the Internet. A huge number of networking devices, e.g., personal computer, laptop, mobile phone, sensor, server, switch, router, are included in this complex network. Several issues concerning heterogeneous networks and devices in term of networking control and management have been brought up such as Fault, Configuration, Accounting, Performance, and Security (FCAPS) in order to aid the network operating properly in an efficient way.
FCAPS is the standard defined by ISO [1] so that any Network Management Systems (NMS) can follow the requirements, e.g., to record statistical information for future network prediction, to specify multi-privileges for different users, to display a networking map of mixed devices for monitoring purposes, and to embed security protection systems against intruders.
†This work was supported in part by the grant (CSKKU12/2554#8)
from the Department of Computer Science, Khon Kaen University.
NMS is a combination system of hardware and software functioning as monitoring and administering tools for heterogeneous networks. Simple Network Management Protocol (SNMP) [2] is commonly used within NMS in that Management Information Base (MIB) [3] is acquired for each individual networking device as statistical information, e.g., packet sent-received and networking interface descriptions. Also, the protocol allows administrators to configure some features (read/write modes) in each particular networking device.
There have been several NMSs in both commercial (HP Openview, CiscoWork, Tivoli Netview, Solarwinds, etc.) and non-profitable or open-source (Nagios, Zenoss, OpenNMS, Cacti, etc.) [4-5]; each of which NNS provides different features, functions, and supports.
Cacti Traffic Grapher (Cacti) [6] is one of the open-source web-based network management systems providing enriched features such as embedded graph templates, easy-to-install and easy-to-use menus, and only required basic software components (mysql, php, rrdtool, and optional net-snmp). Additionally, Cacti supports adding-on plug-ins using as a tool to enhance the overall system performance for individual usage as an increase of complexity trade-off (versus OpenNMS, Nagios or Zenoss as examples of more complex NMSs). For example, a weather map plug-in including MRTG [7] allows the administrator to manually draw an individual map providing the network traffic usage.
However, Cacti with the weather map plug-in does not allow users and/or administrators to monitor and manage networking devices efficiently, such as port-based monitoring, active link status, link notification (either via e-mail or SMS), SNMP read/write-enable configuration, and web-based network configuration. Most importantly, a combination of various plug-ins cannot hold the necessity features of NMS, especially the automatic networking discovery/map.
As a result, in this paper, we enhanced Cacti performance by including a weather map plug-in embedded with necessity features as stated above. One of the features is the automatic networking device discovery plug-in in which the discovery information will be drawn as a networking map via XML-based information using an Arbor library, a graph visualization library using web
workers and jQuery (HTML5) [8], the so-called Web-based Automatic Network discovery/Map Systems (WANMS). WANMS adapts the use of ipAddrTable MIB information [2], and so the discovery time is reduced (versus ipRouteTable). The system also allows user-defined description patterns in order to identify the type of networking devices. In addition, with the plug-in concept, the operation time is bounded not to compute all unrelated monitoring and management functionalities.
This paper is organized as in the followings. In Section II, we briefly surveyed recent research/proposals regarding the techniques for network management systems (NMSs), and discussed about several issues, especially the automatic discovery and map functions. Then, in Section III, an overview of our system, WANMS, will be discussed. Section IV provides the detailed description for our automatic network discovery/map systems. After that we discussed the system performance comparatively in Section V. Finally, the conclusions are drawn in Section VI.
II. RELATED WORK
Due to the advance of networking technology, the communication among networking devices tends to be more complex, and so the networking management system (NMS) becomes necessity for administrators to smooth out the operation and to lead heterogeneous devices to function efficiently.
For years, several NMSs have been proposed in both open-source and commercial software/hardware [4-5]. In [9], J. Hughes showed that some of NMSs are more frequent to be downloaded. Nagios [10] is the top-most download; and the latter are MRTG-based tools [11], Zenoss [12], OpenMMS [13], and Hyperic [14], respectively. The reasons for users/administrators to adopt these tools include networking coverage accuracy, detailed statistical information, and complexity and easy-to-use/install criteria.
In [15], the comparative function of Nagios, Zenoss, OpenMMS, and Cacti (using MRTG) [6] was evaluated in terms of monitoring display, device discovery, plug-in usage, and report. Each has pros/cons, and especially for cons, Nagios has no details on display map unlike Cacti, and also has no support on SNMP built-in features for auto-discovery. One of its disadvantages is that it has no support on database such that to query pre-dated statistical information may not be feasible, in which NetHAM [16], the forked Nagios, offered in terms of XML-based file format using AdobeFlex to display.
Regarding an auto-map function, OpenMMS outperforms Zenoss in terms of accuracy with the complexity trade-off. Cacti itself does not provide the auto-map function although it provides the simplicity and easy-to-use/install function. Notice that a weather map module [7] offers the user-defined manual map built into Cacti; however, it still lacks the auto-map function.
Considering research-based network management proposals, W. Singhabut et al. [17] proposed the
algorithm, Dynamic Network Map (DNMA), to discover networking devices to avoid the networking loop within the local network. However, there is no support on Web configuration and a lack of database-support as Nagios.
W. Kersri [18] proposed the prototype of NMS using SNMP and ping/port checking functions to monitor both managed and unmanaged devices. P. Pitaksateanskul [19] proposed the module to monitor the performance of routers using SNMP resulting in the statistical display via web browser. P. Jirapanichakul and S. Huntakul [20] proposed the prototype for WLAN management via Web service. In addition, C. Maharak [21] applied the web-based network manager to utilize SNMP and ICMP to monitor the network. However, none of these approaches offers the automatic map function resulting in a complete networking map displayed via Web. In addition, they provide no details on an automatic discovery process.
Consider NMS requirements. Although X. Wang et al. [22] proposed the NMS framework according to FCAPS, there is no implementation and/or analytical analysis.
From various techniques discussed above, notice that there are advantages, but some useful techniques aren’t considered probably depending upon the user requirement and technology inadequacy. As a result, we have proposed a new system to collaborate necessity features, especially automatic discovery/map processes, to make a simplified network management system, the so-called WANMS.
III. SYSTEM ARCHITECTURE
Web-based Automatic Network discovery/Map Systems (WANMS) is a plug-in module embedded into Cacti in order to complete necessity functions for Network Management System (NMS) requirements. Notice that especially for auto-discovery/map purpose, this sub-module can be run individually excluding Cacti.
Fig. 1 shows Cacti architecture including a weather-map plug-in. Cacti consists of three main modules: Data Source, User Management, and Graph and Template. The data sources function as statistic collection. RRDTool is used to generate graphs based on statistical information. The user management is used for administrative purpose.
WANMS (Fig. 2) enhances the traditional Cacti with a weather-map plug-in with many features as follows:
1) WANMS Discovery Module: this function primarily performs the networking discovery process using ICMP and SNMP. There are two sub-discovery processes. The first one is for local discovery within the weather-map (in each sub-network); WANMS offers the local discovery feature to the networking devices to identify the type of those. Another one is to generate the entire map automatically. For auto-map purpose, the information after the discovery process is based on XML-format. This will be described again in details in Section IV.
2) WANMS Notification Module: WANMS provides the user-defined configuration to send notifications/alerts in case the networking devices do not function properly. There are two alert methods: either by email or by SMS services using Google SMS and Google Calendar API.
Cronjob Data Sources MySQL Database Script Command Cacti RRD Tool Graph & Templating User Management Weather Map Plugin
Fig. 1. Cacti Interaction Component
SMS Service WANMS GUI WANMS WANMS Discovery Module WANMS WMH Module WANMS Notification Module Google API XML Data File admin Network Devices Servers email Service WANMS Cacti Module
Fig. 2. WANMS Architecture
Fig. 3. Port-based Status
3) WANMS GUI: this module is used to display all information via Web architecture such as networking device descriptions. This module also includes the display of networking maps using XML-based information.
4) WANMS Weather Map Helper (WMH) Module: this module provides accessory features to a weather-map plug-in.
- Read/Write Function: instead of read-only for networking device statistics, WANMS offers a write-enabled feature to update the allowable information such as networking interface description, message of the day, and device name.
- Port-based Display Status Function: users can verify the status of each port for a particular networking device, i.e., router and switch, indicated by different colors – green when active and no color when inactive (Fig. 3).
- Device Display Status Function: users can identify the status of networking devices whether they are in stable states or not (PC and Server).
5) WANMS Cacti Module: this module performs the interaction functionality with the traditional Cacti including collecting the statistical information into MySQL database.
Start Create an XML tree
Add host detail to an XML tree End Unreachable Save an XML tree to a file End of Scan Network Address Scan network address Host Addresses Has next host
address
Send ICMP ping
Is Alive Alive
Not Alive
Fig. 4. Flowchart: IP Discovery Process
Notice that PHP scripts are mainly used to control the interaction among modules. In addition, Crontab is optionally enabled to perform regular updates for up-to-date networking information.
IV. AUTOMATIC DISCOVERY/MAP PROCESS
In this section, we focus on the automatic discovery/map process.
Fig. 4 shows the flowchart for IP discovery process, especially for local discovery (with a weather-map plug-in) in that a whole sub-network will be scanned with ICMP ping features. If the device responds an echo-reply or unreachable message (in case some devices do not allow ping to get through), this IP address will be stored into XML database tree.
Once feasible networking devices are available from XML tree information, Fig. 5 shows steps to identify the type of networking devices such as Router, Switch, Access Point, or Server. In this process, SNMP walk [3] is primarily used to acquire all related information.
Notice that this process is performed in recursive manner, especially for finding out completely connected interfaces to other networks using an ipAddrTable field.
In addition, to properly specify the type of networking devices, sysDescr and sysName are used as identifiers in which users can manually pre-define the pattern in the configuration file; for example, if sysDescr is “Linux” and sysName is “Access Point”, this device tends to be an access point.
Start Read an XML file A SNMP Service available Yes sysDescr is “Linux DD-WRT” sysDescr is “Linux” and sysName is “Access
Point”
sysDescr is “Switch” or “Cisco”
sysDescr is “Router” and sysName is
“Router” False False An XML file of network scan result sysName , sysDescr False True False True Is “WAP” Send an SNMP Walk to host True True Host Addresses Has next host True sysName,
sysDescr Check host type
True NMAP Scan for OS type
Read host information from SNMP Access Point True Sockopen to SSH, HTTP, RDP ports False If any of these ports is open Check host type No
Access Point Switch Router PC Server 2 False 1 End 2 1 1 False 1
Fig. 5. Flowchart: Local Discovery Process (with weather map plug-in)
<?xml version=“1.0”?> Í XML Declaration <map> Í Root Element <time>92.056804895401</time> Í Executed Time
<total-devices>41</total-devices>
Í Total Discovered Devices Í Node IP and Name
<device IP=“122.154.136.29” depth=“2” type=“Server” name=“ns1.police4.go.th”>
<interfaces> Í Interfaces Element
<interface IP=“122.154.136.29”/> Í Network Interface <interface IP=“127.0.0.1”/> <interface IP=“192.168.100.1”/> </interfaces> </device> ... Í Next Node </map>
Fig. 7. Example: XML-based information for Arbor library
For automatic discovery purpose for the entire network, Fig. 6 shows all steps resulting in proper XML-based format information so that a networking map can be created by a graph visualization library using web workers and jQuery (Arbor library). An example of XML-based file format is shown in Fig. 7.
From Fig. 6, the automatic discovery process initially starts from IP address of router gateway of the WANMS server using “ifconfig” command. Then, a SNMP walk process is applied to figure out more information about connected networks including active IP addresses. After that similar to Fig. 5, the justification among the types of networking devices is performed. Finally, all these information are stored in XML-based file format so that Arbor library is applied in order to generate the entire networking map. S t a r t F in d C u r re n t D e v ic e IP if c o n f ig D e v ic e IP S e n d S N M P w a lk t o r e t rie v e d e v iv e t y p e D e v ic e IP Is S e r v e r , S w it c h , R o u t e r S e n d S N M P w a lk t o g e t d e v ic e n a m e D e v ic e IP D e v ic e n a m e A d d D e v ic e in f o rm a t io n t o X M L t r e e D e v ic e IP D e v ic e t y p e D e v ic e n a m e D e v ic e t y p e E x t r a c t in t e r fa c e s d a ta D e v ic e IP A d d in te r fa c e in f o rm a t io n t o X M L t r e e A ll In te r fa c e s IP In t e r fa c e IP F in d c o n n e c t e d d e v ic e s A ll c o n n e c t e d d e v ic e s IP In t e r fa c e IP H a s n e x t d e v ic e S e n d S N M P w a lk t o g e t d e v ic e in f o r m a t io n Is S e r v e r , S w it c h , R o u t e r A d d d e v ic e in f o rm a t io n t o X M L t r e e D e v ic e in f o rm a t io n D e v ic e IP D e v ic e in f o r m a t io n Y e s N o Y e s Y e s N o H a s n e x t in t e r fa c e Y e s N o C r e a te a n X M L t r e e 1 1 N o 1 E N D S a v e X M L t re e t o a file
Fig. 6. Flowchart: WANMS Automatic Discovery
V. PERFORMANCE EVALUATION
In this section, we performed the evaluation process so as to justify our system performance. We only focus on discovery/auto-map processes. An automatic discovery speed, map-drawing display, and discovery coverage are our main metrics for comparison purpose.
Notice that although there are several proposals regarding to the network management systems (NMSs), some of them have limitations, and so incomparable to WANMS. For example, with Nagios [10], there is no built-in feature for an automatic discovery process.
Fig. 8. WANMS: Automatic Map
For the forked Nagios, Netham [16], there is no detailed algorithm for the automatic discovery process. The network mapping process for Zenoss is manually performed, and the display map does not entirely represent the actual devices [12]. As a result, OpenMMS [13] and DNMA [17] are chosen as our evaluation candidates. Table I shows a comparative features among those three.
For our testbed, the network management server ran over Debian 6.0.2.1-i386 (Linux 2.6.32-5.686): CPU Intel(R) Celeron(R) 2.80 GHz (256 KB Cache), 512 MB DDR-SDAM (400 MHz), Integrated Realtek RTL8100C 10/100 BaseTX, and 80 GB Disk-ATA133.
For OpenNMS, as a standard configuration, net-snmp, postgresql, java sdk: Tomcat (java servlet engine), rrdtool, and curl are included; and for DNMA, since there are no details of implementation tools, we followed its automatic discovery algorithm, and then applied into our existing configuration. For example, instead of Java, we used PHP and MySQL. Notice that DNMA does not support Web-based display and lacking of automatic global map generation features.
In addition, all devices for our testbed were used as a close system within the department of Information Technology, Provincial Police Region 4, Thailand.
A. Experimental Setup
We evaluated three main scenarios to test the system performance, especially the discovery/map process as follows.
1) To show the system performance in details of auto-map processes, OpenNMS was chosen to perform the auto-discovery, and then the auto-map process was run.
2) To show the performance of discovery speed and coverage, we ran our system comparing to OpenNMS and DNMA with default configuration. There are two cases: the first one is limited by 15 second evaluation period so as to figure out the number of discovered networking devices. Second, we set up 20 (2+4+3+8+3) and 30 (2+4+3+17+4) networking devices scenarios consisting of Router (Firewall+L3 switch), Switch,
Server, PC, and Access Point, respectively, and then we captured the completion time for the entire auto-discovery process.
3) To show the optimized results, we tuned up the configuration of OpenNMS such as limiting the number of retries to 1 as well as timeout to 100 ms (these parameters are optimized from over multiple trials) and ignoring the service discovery process.
Notice that due to the limitation and availability of our testbed within the department, for the second and third scenarios including the number of testing devices, we ran over three experiments; the average (mean) and standard deviation were computed.
B. Experimental Results
The first scenario shows two maps from WANMS (Fig. 8) and OpenMMS (Fig. 9). In these figures, our system can display detailed networking devices, especially including the wireless access point properly (3 devices). The reason is because our auto-discovery process includes the pre-configured pattern to evaluate the specific type of networking devices.
The second scenario shows the average and standard deviation of auto-discovery speed (Table II and III). With the time limitation (15 seconds), Table II shows that our system can discover up to 7 devices in average with less than 0.1 as a standard deviation. OpenNMS cannot find any devices (the first device found is at the 32nd second).
For DNMA, a number of found devices are limited to 4; the DNMA’s discovery process cannot get through the firewall (tudtu.aaa.bbb.ccc); the process is terminated at time around 6.65 second. Additionally, Table III shows that in both 20 and 30 devices’ scenarios, OpenNMS spent discovery time over 3 times as much as that of WANMS for the discovery process.
In the third scenario, the optimized configuration for OpenNMS, i.e., limiting the number of retries, timeout, and services, the discovery process reduced from 15 to around 11 minutes. The main reason may be because of the complexity of the implementation that offers several unrelated functionalities of OpenNMS.
Fig. 9. OpenNMS: Automatic Map TABLEI
FEATURE COMPARISON:WANMS,OPENNMS, AND DNMA
Feature WANMS OpenNMS DNMA
Language PHP Java Java
Database MySQL PostgreSQL N/A
Web-based Yes Yes N/A
Network Map Algorithm
Force-direct
Layout Yes N/A Discovery
Result Format XML XML XML
Device
Configuration Yes N/A N/A
Alert Email/SMS N/A N/A
Wireless Support
Yes
(Defined-Pattern) N/A N/A
TABLEII
NUMBER OF FOUND NETWORKING DEVICES (WITHIN 15 SECONDS)
Networking Device
WANMS OpenNMS DNMA
Router 2 0 1 Switch 0 0 0 Server 3 0 2 PC 2 0 1 Access Point 0 0 0 #Total 7 0 4 TABLEIII
DISCOVERY TIME VS.NUMBER OF DEVICES (20 AND 30)
Device WANMS #20 OpenNMS #20 WANMS #30 OpenNMS #30 Router 2 2 2 2 Switch 4 4 4 4 Server 3 3 3 3 PC 8 8 17 17 Access Point 3 3 4 4 #Total 20 20 30 30 Discovery Time (min.) 4.5 15 8.2 24
VI. CONCLUSION AND FUTURE WORK
A new automatic network discovery/map system via Web architecture has been proposed in this paper, the so-called Web-based Automatic Network discovery/Map Systems (WANMS), as an easy-to-use/install plug-in for Cacti, one of the network monitoring open-source software, as well as a weather-map plug-in module.
WANMS embeds the networking discovery process which includes the user-defined matching rules in order to specify the networking devices properly. In addition, after the discovery process, an XML-based output is to feed into a graph visualization library using web workers and jQuery so as to automatically redraw the entire networking map with a force-directed layout algorithm.
The comparative results show that the system outperforms other techniques, i.e., OpenNMS and DNMA, in both networking discovery speed and device coverage criteria.
Notice that in this paper, we experimentally performed the evaluation process over the controlled testbed within the department of Information Technology, Provincial Police Region 4, Thailand, due to the limitation and availability of the authorized administrative domain. However, similarly, other testbeds can be evaluated as well for further study, especially for scalability purpose.
Although the system can be easily integrated with Cacti, similarly to a weather-map plug-in, our future work is to embed/integrate this auto-map function into the weather-map plug-in directly so as to provide detailed statistical information. Thus, the administrators can perform similar functions directly into this module rather than using this as the preliminary maps.
REFERENCES
[1] ISO/IEC 10040, “Information technology - Open Systems Interconnection - Systems management overview” 1998. Available at http://www.iso.org/
[2] D. Mauro and K. Schmidt, “Essential SNMP,” O'Reilly Media, 464 pp., 2005.
[3] S. Southier, Ed., “Management Information Base for the Internet Protocol (IP),” RFC 4293, 2006.
[4] P. Moceri, “SNMP and Beyond: A Survey of Network Performance Monitoring Tools,” 2006. Available at
http://www.cs.wustl.edu/~jain/cse567-06/ftp/ net_traffic_monitors2.pdf
[5] C. So-In, “A Survey of Network Traffic Monitoring and Analysis Tools,” 2006. Available at
http://www.cs.wustl.edu/~jain/cse567-06/ftp/ net_traffic_monitors3.pdf
[6] Cacti. Available at http://www.cacti.net/ [7] Weather Map plug-in (Cacti). Available at
http://docs.cacti.net/userplugin:weathermap
[8] Arbor, “A graph visualization library using web workers and jQuery”. Available at http://arborjs.org/
[9] J. Hughes, “Open-source network management download comparison,” 2009. Available at http://www.openxtra.co.uk [10] Nagios. Available at http://www.nagios.org
[11] MRTG. Available at http://oss.oetiker.ch/mrtg/ [12] Zenoss. Available at http://www.zenoss.com [13] OpenNMS. Available at http://www.opennms.org
[14] Hyperic. Available at http://sourceforge.net/projects/hyperic-hq/ [15] K. Meesublark, C. Aitsariyapat, and K. Sripanichakulchai,
“Comparison of Network Management Software (OpenSource),” 2007. Available at http://inms.in.th/inmsweb/
[16] T. Kongpool, C. Aitsariyapat, and K. Meesublark, “NetHam (Network Health Analysis & Monitoring,” In Proc. of ECTI-Conf.
on Application Research and Development, 2009.
[17] W. Singhabut, C. Tangtongjan, and S. Kumanee, “Dynamic Network Map Algorithms,” In Proc. of National Conf. on
Computer Information Technologies, 2010.
[18] W. Kerdsri, “A Prototype of Completed Network Management System Using SNMP and Ping/Port Checking for Monitoring of Managed and Unmanaged Devices,” In Proc. of Int. Joint Conf.
on Computer Science and Software Engineering, 2008.
[19] P. Pitaksatieankul, “Program for Checking Router Performance Under SNMP Environment,” Thesis, King Mongkut’s University of Technology Thonburi, 2007.
Available at http://202.28.199.3/tdc/
[20] P. Jiyapanichakul, “Prototype Development of Wireless LAN Management System,” Dhurakijpundit University, 2009. Available at http://www.dpu.ac.th
[21] C. Maharak, “EMBEDDED WEB-BASED NETWORK MANAGER SYSTEM,” Thesis, Chulalongkorn University, 2004. Available at http://www.thaithesis.org
[22] X. Wang, L. Wang, B. Yu, and G. Dong, “Studies on Network Management System framework of Campus Network,” In Proc. of
Informatics in Control, Automation and Robotics, pp. 285-289,
