Copyright © 2004 Information Systems Audit and Control Association. All rights reserved. www.isaca.org.
As long as it has been possible to determine the common factors within any repetitive task, there has been a tendency to build tools to aid and supplement the human efforts. It is no different in the security auditing world, where there has been a surfeit of tools for security auditors.
This article discusses some of the factors behind the increasing use of tools in the security auditing community. It also looks at why traditional review methods might not be feasible in an increasing number of situations. It specifies those areas of the security audit that benefit most from automation, and the tools that are available for that domain. The domains covered here include vulnerability assessment, port scanners, host-based auditing, password cracking, forensics, log analysis, network auditing, web application testing, software security auditing and process auditing.
Various open source security and control testing tools are available to the auditor. The tools discussed are restricted to open source and are freely available over the Internet for download. Finally, a list of precautions and caveats that the auditor must keep in mind when using tools are provided. The most important of these is that no tool or set of tools can completely replace a skilled auditor. They can only make his/her task easier and increase his/her efficiency. The output from the tools requires a skilled auditor to weed out the false positives and inaccuracies.
Within this article, open source software is defined as having source code available to anyone who wishes to use it. This is a broader definition than that which is usually given. It does not differentiate between software that allow users to modify and reuse the original source code, and those that restrict such modifications, but still keep the source code open for viewing.
The use of tools for security auditing is prevalent and driven mainly by the inherent complexity within information systems. This complexity makes it almost impossible for any auditor to have the ability to completely test all the possible systems he might come across. For instance, when scanning a corporate network for vulnerabilities, it is almost impossible for the auditor to do this without the use of vulnerability assessment (VA) tools. Most VA tools have the capability to test for thousands of possible vulnerabilities comprehensively and quickly. Then there are some tasks that are just not possible to do manually, or if done manually, the results are far from satisfactory. These tasks include password cracking and log analysis. These are such daunting tasks that they can be easily
overlooked in the absence of the right set of tools. Most log files are so voluminous that a cursory scan will most likely miss some critical events. And, imagine having to check whether users have used a dictionary word as their password— in a thousand-user organization.
The traditional view of an auditor is someone with a checklist whose task is to determine compliance with a given set of measures. Although this may be quite true, in the case of a security audit, a large number of the checks need to be carried out with the help of tools for the reasons mentioned above—comprehensiveness, accuracy and speed. For instance, when auditing a UNIX server, the auditor’s checklist might contain various checks such as for open ports using Nmap, common vulnerabilities using Nessus, weak passwords using Crack and host-based vulnerabilities using COPS, etc.
This section covers the major classes of security auditing tools and provides a brief description of some of the popular tools within each category. The download locations for the tools are listed in the references section.
Vulnerability Assessment Tools
A vulnerability assessment exercise aims to determine systems that have been misconfigured or have not been fully patched. These tests may be as simple as finding out the version of the software, e.g., the Apache web server, and checking against a vulnerability database to determine whether it is the latest version or not. If not, the auditor must then determine the vulnerabilities to which this version of the software might be susceptible. Imagine having to do this in the data center of an ISP for 200 servers running five or six applications each. Then there are additional tests to determine network misconfigurations, such as an FTP server, which allows anonymous users to log in and gives them permission to modify/upload/delete files within the FTP directory. Very early on, in the nascent days of computer security, experts quickly realized the need for an automated tool to scan systems for known vulnerabilities and common misconfigurations. A number of tools were written and since then many more have been developed for this purpose. But the ones that have stood the test of time and have gained popularity are those that are regularly updated, because the number of security bugs discovered every week could simply overwhelm any small or medium-sized team that develops and maintains vulnerability assessment tools.
Nessus (figure 1) is one of the most popular and effective tools for assessing vulnerabilities in a network. It works in a
Open Source Tools for Security
and Control Assessment
client-server architecture, wherein the server stores the database of vulnerabilities to be tested, accepts connections from clients and completes the actual scanning. The client authenticates to the server and then chooses the various tests from those available. It also selects the targets to be scanned and other scanning features. A Windows version of the client is also available.
One of the most significant factors for the success of Nessus, besides it being open source, is its architecture. Running it in client-server mode allows an auditor to use a laptop with limited capabilities with the Nessus client loaded onto it, and a powerful system as the main Nessus server. Multiple auditors can audit multiple networks as long as the Nessus server is accessible to them. The second feature of its architecture is that Nessus is updated using plug-ins. These plug-ins can be written by anyone who can program either in C or in NASL (Nessus Attack Scripting Language)—a language specifically designed for programmers to write security tests easily and quickly for Nessus. As a result, anyone can contribute plug-ins, ensuring that Nessus is as current and up-to-date as possible.
The Security Administrator’s Tool for Analyzing Networks (SATAN) was one of the earliest vulnerability assessment tools, and was developed by security expert Wietse Venema to audit the susceptibility of various UNIX operating systems to known vulnerabilities. It has not been updated very frequently and is not as preferred as Nessus. A new tool called SARA (Security Auditor’s Research Assistant) is an updated and
much-enhanced VA tool based on the SATAN model. It contains many additional checks, which are updated regularly, and features for web-based control. It is maintained and updated by Advanced Research Corporation.
Every application that provides a service to clients uses a port number. Most such services use well-defined port numbers. For instance, the HTTP service, which is used by web sites, runs on port 80, SMTP for e-mail runs on port 25, and so forth. By determining the services running on various ports, the auditor can determine the many avenues of attack that an attacker might use against the system. Port scanning is the act of trying to connect to various ports on a given system and determining which ones respond to the request. One of the most widely used tools for this purpose is Nmap. In fact, Nessus itself uses Nmap to determine which ports it will scan for vulnerabilities.
In addition to being a port scanner, one of the most remarkable features of Nmap is its ability to accurately guess the operating system of the target. It does this by using a technique called TCP/IP fingerprinting by which it detects variances in each vendor’s implementation of the TCP/IP stack. Additionally, Nmap provides a variety of scanning techniques that determine the mode and speed of the port scan. These techniques are used to reduce the chances of a port scan being caught by port scan detecting software or intrusion detection systems. Both UNIX and Windows versions of Nmap are available (see figure 2).
Network Auditing Tools
Network auditing consists of testing the security of the network elements within the system, such as firewalls and routers, as well as testing network parameters of the operating systems of the servers. Two things need to be tested: the general configuration of the network component and specifically any access control lists that have been applied. Some of the most widely used network auditing tools follow.
Figure 1—Nessus Client Screen: Configuring Plug-ins to Use in the Scan
Hping2 is a network tool able to send customized TCP/IP packets and display target replies just as the ping program does with ICMP replies. Hping2 allows the user to construct fragmented TCP/IP packets and packets with arbitrary body and size. Among other things, hping2 allows the auditor to test firewall rules by sending various customized packets, carry out port scanning, test the network performance and try to transfer packets by circumventing the firewall rules.
Another tool to test firewall rule sets is Firewalk by Mike Schiffman and David Goldsmith. It is an active reconnaissance network security tool that attempts to determine what layer 4 protocols a given IP forwarding device will pass. What this means is that Firewalk will try to determine which TCP/UDP protocols are blocked and which are disallowed at the gateway. This filtering might be due to a firewall or a router.
Windows Network Testing
Windows systems communicate with each other using a protocol called NetBIOS. Unfortunately, this protocol is highly insecure. If the security policy is not properly configured, any unauthenticated user can obtain critical information including shares, usernames, hostname, groups, etc. One of the tools that gathers this information is Enum from Bindview, which enumerates all of this information from a Windows system. It can also then be used to launch a dictionary password attack to try and connect to shared folders on the system. Another tool that does this is NAT (NetBIOS Auditing Tool), which also tries connecting to shared folders using a list of user-supplied usernames and passwords (see figure 3).
Host-based Auditing Tools
Host-based auditing consists of scanning a system locally, and determining unpatched software and common
misconfigurations. A host-based auditing tool is theoretically
more comprehensive than a network-based vulnerability assessment tool simply because it has greater access to system information than a network-based tool. For instance, most host-based tools audit for weak permissions on critical system files/directories, weak passwords, etc., in addition to unpatched systems and insecure configurations.
The Computer Oracle and Password System (COPS) is a host-based security testing tool that checks for common misconfigurations, integrity of password files, insecure permissions on critical files, anonymous FTP security, weak passwords and more. It is written by Venema’s colleague Dan Farmer. COPS has not been updated in a while and may be outdated.
Texas A&M University (USA) wrote a tool similar to COPS, called Tiger. This tool was developed in 1992 primarily in response to an attack on the university’s network by hackers coming in from the Internet. This tool has also not been updated since 1994 and may be outdated. However, the same organization—Advanced Research Corporation—that maintains an updated version of SATAN, also updates and maintains Tiger. This updated version is called Tiger Analytical Research Assistant (TARA).
Sometimes it is necessary to determine the strength of user-selected passwords. This is true when the organization has a password policy and the auditor must determine if users are adhering to the policy. On both UNIX and Windows systems, the passwords are stored by encrypting them with a one-way hashing algorithm. The only way to determine a password from its hashed value is to pick a possible password, hash it and then compare the hashes. Doing this manually is out of the question. Most password-cracking tools take entire dictionaries, hash each entry and compare the results against the list found in the password file. The hashes that match represent guessed passwords. Advanced password guessing involves trying out permutations of numbers, letters and special characters in a brute force attack.
One of the most popular password-guessing tools ever written is Crack by Alec Muffet. It works on almost all UNIX platforms and helps detect weak passwords in the /etc/shadow file using both a dictionary attack and a customizable brute force attack. Its outstanding feature is the sheer speed with which it can manage to crack passwords.
Other password-cracking tools include John the Ripper, which is also in the same league as Crack, but is more general in purpose and has been adapted to guess passwords of NIS servers, Lotus Domino and even Windows NT. Another generic password-cracking tool is Lepton’s Crack, which works with various algorithms, such as MD4, MD5, NT hashing algorithm, Lotus Domino HTTP algorithm and SHA-1. It comes along with a plug-in system and is easily customizable for other algorithms using both dictionary and brute force methods.
Figure 3—Enum in Action: Gathering Shares, Users and Groups from a Windows System
Computer forensics deals with the investigation of a computer system that may have been compromised. The investigation is done by taking the affected system offline and is aimed at determining the attacker and his/her methodology, either for research or legal purposes. This is a vast field by itself with numerous articles and extensive research having being done on it. This section lists some of the common tools available to auditors should they be faced with such a situation. Even during a regular audit, it is essential for auditors to study the audit trail and determine if the system may have been compromised.
The Coroner’s Toolkit
The Coroner’s Toolkit (TCT) is one of the most widely used tools for this purpose and is developed and maintained by security experts Farmer and Venema. TCT is a collection of tools designed to assist in a forensic examination of a computer. It is primarily designed for UNIX systems, but it can also do small amounts of data collection and analysis from non-UNIX disks/media. It is strongly recommended to download and try the various tools within the tool kit to determine their functionality and features, and not wait until an emergency actually happens. Lsof
Another very useful tool for forensics is lsof (LiSt of Open Files). Lsof gathers a list of open files on the system. It also shows which processes are using what ports, pipes, streams, etc. This aids the auditor in determining what resources are being used up by the system. It requires experience and in-depth knowledge of the system to determine any suspicious entries in the lsof output. Also, the output can be quite voluminous.
Log Analysis Tools
One of the most important tasks for an auditor should be analyzing the log files to determine system malfunctions or security violations. This may be for a forensics exercise or even for a normal security audit. However, on any normal system the log files can be voluminous and auditing each entry is impossible. Tools to analyze log files exist on UNIX and Windows systems.
There are a number of utilities available to parse and analyze the entries within the UNIX syslog utility. One such utility, called log analysis, goes through several different kinds of logs (currently syslog, wtmp and sulog) over some period (defaults to yesterday), comparing each entry against a list of Perl regular expressions. If there is a match, a data-extracting rule is applied and the appropriate information is recorded under the appropriate category. Other utilities include LogSentry and Logwatch.
For Windows, most utilities are freely available, but are not open source. The authors of this article have written a Windows-based, open-source generic log analysis tool called Log Analyzer. Log Analyzer is a command-line utility capable of analyzing a large variety of log files, including those from UNIX, Windows, IIS, Apache and PIX, depending upon the pattern file supplied to it.
Software Auditing Tools
When an auditor is called upon to carry out a source code audit, he/she can quickly search out common programming errors using automated tools. This is especially true when the application is complex and reading each line of code is not feasible. One of the most popular tools in this category is Rough Auditing Tool for Security (RATS). Some of the authors of RATS were previously involved in a similar project, which developed the ITS4, at Cigital. Another tool used for this purpose is Flawfinder, which mainly checks C/C++ code and was written by David Wheeler, who is also the author of Secure Programming for Linux and UNIX.
Web Application Testing Tools
In the case of a web server, after checking the patch levels and configuration of the operating system, the auditor must turn his/her attention to the application level. This involves testing the organization’s Internet and intranet sites for common web application vulnerabilities. As with network-based VA tools, the list of checks to be made for web applications is extensive, and the auditor must resort to using a web-scanning tool.
Whisker is a Perl script written by Rain Forest Puppy (RFP). It was one of the first tools written for testing common web application vulnerabilities. Lately, RFP has suspended further development on the tool. However, the main library used by Whisker is available to other programmers and is being very well utilized in another tool called Nikto. Most web servers come with default help files, documentation, sample scripts, etc. This tool includes more than 2,000 checks for default files, server-side scripts and other vulnerabilities for a large array of web servers.
Auditing of Processes
Auditing of the IS processes within an organization usually takes place with the help of checklists, questionnaires, interviews, and actual observation of the systems and controls that have been put in place. Tools for this task must store various checklists and questionnaires, and allow the auditor to modify them. Additionally, such a tool must store the results from an audit into a database and allow for comparisons with earlier audits to determine increasing or decreasing levels of compliance.
The Automated Security Self-evaluation Tool is a security evaluation tool from the US National Institute of Standards and Technology (NIST). ASSET may be used to gather data and generate reports related to the status of the self-assessment. The intent of this tool is to provide a centralized place for the collection of data used to assess a system. ASSET contains the specific control objectives and suggested techniques for measuring the security of a system or group of interconnected systems as described in NIST SP 800-26. The control objectives and techniques are taken from long-standing requirements found in statutes, policies and guidance on security. ASSET consists of two host-based applications:
ASSET-System and ASSET-Manager. ASSET-System facilitates the gathering of individual system data. It provides a limited reporting capability and allows the user to determine the completeness of an individual system assessment in progress. ASSET-Manager aggregates individual system assessments created by ASSET-System. It assists managers in developing an organizationwide perspective on the state of IT system security.
When using open source tools, the auditor must be careful and take certain precautions to ensure the integrity of his/her own systems and those of the auditee. Even though open source by its very definition is available for inspection, it is not usually feasible for auditors to study the code of all the tools they use and determine any potential threats. The preferred method is to download the MD5 (Message Digest 5) hash of the tool along with its source code. An MD5 hash of any file is like a signature for that file. Even if there is only a single byte changed within the original file, the MD5 hash changes significantly. Also, it is not computationally feasible to determine the original file from a given MD5 hash. This means that if it is required to determine the integrity of a file downloaded from the Internet or elsewhere, then the hash value of that file must be compared with the hash value stated on the vendor’s web site. This can be done using any of the numerous MD5 hash programs to create a local hash of the source code and compare it with the downloaded MD5 hash. If the two match, the source code is the same and can be compiled and run.
Another precaution to take is to download the tool from the official web site, or the official mirrors listed on the main web site. The auditor must never download the tools from any other web sites. This article lists the URLs for the official sites of these tools. More often than not, site of the tool’s author is the authoritative site for any given tool.
Furthermore, the output of any tool must not be taken at face value. This is true for all security testing tools, not just open source ones. A tool is only as good as the auditor using it. The reports produced by most tools contain false
positives/vulnerabilities, which do not really exist, but are marked as such by the tool. It is up to the auditor to weed out these false positives, and treat the output of the reports as a part of the more comprehensive manual audit. Also, the tools may not accurately describe the risk levels associated with the discovered
vulnerabilities or the countermeasures. The auditor must check these carefully and ensure that the risk level associated with each vulnerability is accurate and the countermeasures are factually and technically correct. Some vulnerabilities may not assume significant proportions in the overall picture, whereas certain others that are marked as low or medium risk by the tool may, in conjunction with other vulnerabilities, allow an attacker to completely compromise the system. It is for the auditor to ascertain the risk level of each vulnerability as it fits in with the overall security posture of the organization.
The auditor must also correctly choose the features and options from the tool to get comprehensive and accurate results. If possible, he/she must try to use one or more tools for the same purpose and compare the results from different tools to arrive at an accurate picture.
Timing is also critical when running tools against a system. Some tools may consume vital system resources and the auditor may end up causing a denial-of-service attack, thus affecting business functionality. Extreme precautions must be taken to ensure that the use of security tools does not in any way affect the normal functioning of the systems. If there is any doubt, the tools must be used during off-peak or nonbusiness hours. This is especially true of vulnerability assessment tools, where dangerous checks/plug-ins must be disabled for critical systems.
Also, the concept that open source software is more secure than proprietary software is a myth. However, open source software has a greater possibility of being more secure. This possibility may or may not always translate into reality depending upon a number of factors. These factors include the popularity of the software, the inclination of other
programmers to verify the source code of the software and the quality of auditing to which the source code is subjected. The auditor must not make any assumptions on the inherent security of the software based on whether its source code is open or closed. Another risk that exists and has materialized in the past is when the primary site of the open source software is compromised, and malicious code is inserted into the source code itself. Numerous users then download this Trojaned software, and soon enough it is mirrored all around the Internet.
In essence, the number of tools available for auditing the security and controls of any system is vast. The auditor must begin with the list of tools enumerated in this article, and slowly work toward building his/her own tool kit for carrying out a complete security audit. Great care must be taken about the source from which the tools are downloaded. Even though it may not be possible to verify the source code, it is important to verify the MD5 checksum and, if possible, the PGP
signature of the downloaded files. The auditor must also carefully analyze the outputs from the tools and watch out for false positives. Also, it is advisable to use a combination of tools for the same purpose. Scanning the same target with two vulnerability assessment tools always gives a clearer picture. Some tools might utilize extensive system resources and could cause a denial-of-service attack. It is usually preferable to run tools during off-peak hours and choose the options correctly to prevent an inadvertent load on the target system.
The increasing use of tools make the auditor more efficient and reports more comprehensive. Manual efforts aided by the tools yield much more in-depth analyses of the situation than either approach by itself. By becoming familiar with the tools, their usage, features and options, auditors will be able to derive maximum benefit from their use. And, at the end of the day, they must keep in mind that no tool can replace a skilled auditor, and the final responsibility rests with them.
The download locations for the tools mentioned in this article are: Nessus, www.nessus.org
SATAN, www.porcupine.org/satan/ SARA, www-arc.com/sara/ SAINT, www.saintcorporation.com Nmap, www.insecure.org
TIGER, www.net.tamu.edu/network/tools/tiger.html TARA, www-arc.com/tara/
John the Ripper, www.openwall.com/john/ Lepton’s Crack, http://usuarios.lycos.es/reinob/ The Coroner’s Toolkit, www.fish.com/tct/
Lsof, ftp://ftp.cerias.purdue.edu/pub/tools/unix/sysutils/lsof/ Log_analysis, http://linux.umbc.edu/~mabzug1/log_
Log Analyzer, www.nii.co.in/research/tools.html Hping2, www.hping.org/
Enum, http://razor.bindview.com/tools/desc/enum_readme.html Nat, www.tux.org/pub/security/secnet/tools/nat10/
RATS, www.securesoftware.com/download_form_rats.htm ITS4, www.cigital.com/its4/
Nikto, www.cirt.net/code/nikto.shtml ASSET, http://csrc.nist.gov/asset/
Top 75 security tools, http://insecure.org/tools.html Details on analysis of logs, www.counterpane.com/log-analysis.html
Mann, Scott; Ellen L. Mitchell; “Linux System Security: The Administrator’s Guide to Open Source Security Tools”
is the chief technology officer at Network Intelligence India Pvt. Ltd.. He founded Network Intelligence India in 2001 to provide IT security consultancy in India. He has authored The UNIX Auditor’s Practical Handbook—A Step-by-step Guide to Auditing UNIX Systems.
Information Systems Control Journal, formerly the IS Audit & Control Journal, is published by the Information Systems Audit and Control Association, Inc.. Membership in the association, a voluntary organization of persons interested in information systems (IS) auditing, control and security, entitles one to receive an annual subscription to the Information Systems Control Journal.
Opinions expressed in the Information Systems Control Journalrepresent the views of the authors and advertisers. They may differ from policies and official statements of the Information Systems Audit
and Control Association and/or the IT Governance Institute® and their committees, and from opinions endorsed by authors' employers, or the editors of this Journal. Information Systems Control Journal does not attest to the originality of authors' content.
© Copyright 2004 by Information Systems Audit and Control Association Inc., formerly the EDP Auditors Association. All rights reserved. ISCATMInformation Systems Control AssociationTM
Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, Mass. 01970, to photocopy articles owned by the Information Systems Audit and Control Association Inc., for a flat fee of US $2.50 per article plus 25¢ per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited.