• No results found

Effective network security audit trail management

N/A
N/A
Protected

Academic year: 2021

Share "Effective network security audit trail management"

Copied!
66
0
0

Loading.... (view fulltext now)

Full text

(1)

CALIFORNIA STATE UNIVERSITY, NORTHRIDGE

EFFECTIVE NETWORK SECURITY AUDIT TRAIL MANAGEMENT A graduate project submitted in partial fulfillment of the requirements

For the degree of Master of Science in Accountancy By

Daniel Kenneth O’Kelley

(2)

ii

The graduate project of Daniel Kenneth O’Kelley is approved:

_____________________________ __________

Dr. David Miller Date

_____________________________ __________

Dr. Christopher Jones Date

_____________________________ __________

Dr. Richard Ye, Chair Date

(3)

iii Table of Contents Signature Page ... ii Abstract ... vi   Introduction ... 1   Objectives ... 2  

Literature Review of Prior Research ... 4  

Network Audit Trails ... 4  

Audit Trail Integrity ... 4  

Intrusion Detection Systems ... 5  

System Design and Theory ... 5  

IDS Management ... 6  

Intrusion Detection System vs. Intrusion Prevention System ... 7  

Research Program and Methodology ... 8  

Audit Trail Management Theoretical Model ... 8  

Table 1. ... 9  

Reasoning ... 10  

Adverse Events ... 12  

Formalized NIST Model ... 12  

Expanded Audit Trail Theoretical Model ... 14  

Intrusion Detection System Testing ... 15  

Proposed Variables and Conjectures ... 16  

Survey Instrument ... 21  

(4)

iv

Survey ... 23  

Findings: Preliminary Research Trial ... 26  

Summary of Responses ... 26  

Response Analysis ... 31  

Raw Response Data ... 31  

Observations ... 35  

Survey Revision Recommendations ... 37  

Recommendations: Further Research Plan Outline ... 39  

Research Hypotheses ... 39  

Distribute Modified Survey ... 39  

Increase Response Rate ... 40  

Analyze Survey Responses ... 40  

Limitations of This Study ... 41  

Recommendations: IDS Management ... 42  

Ensure Analysts are Familiar with Network Environment ... 42  

Establish a Training Program for Analysts ... 42  

Encourage Knowledge Sharing ... 43  

Consult with IDS/IPS Vendors and Consultants when Installing New Systems ... 43  

Contribution to Theory and Practice ... 43  

References ... 45  

Appendix A Employment Size of Employer ... 48  

Appendix B Data Leak Incidents ... 49  

(5)

v

Part A: Original Survey ... 50   Part B: Modified Survey ... 52   Appendix D Pilot Survey Results ... 56  

(6)

vi Abstract

EFFECTIVE NETWORK SECURITY AUDIT TRAIL MANAGEMENT

By Daniel O’Kelley

Master of Science in Accountancy

Modern organizations face an increasingly difficult task of protecting their information system networks from intruders. Network protection must also be balanced against accessibility and convenience for authorized users. Network monitoring tools are often employed to identify potential security problems. These tools generate considerable amounts of data relating to network traffic and security logs. Technologies such as Intrusion Detection Systems (IDS) or Intrusion Prevention Systems (IPS) are employed to assist information systems and network security personnel in identifying and stopping

(7)

vii

network intrusions. However, while utilities such as IDS and IPS are helpful in collecting and sorting through the volume of data that is generated from internal logging, human management of this information is important in effectively monitoring network traffic and detecting incidents. The process by which network security audit trails are managed varies considerably among organizations, and there does not exist currently a set of well-researched best practices for IS network security managers to follow.

This thesis analyzed the current industry practices and management policies used in audit trail management, and provided an overview of management practices and policies believed to be effective for network protection. The research formulated several conjectures regarding factors that may lead to faster malware detection times, and described a research plan and a survey instrument to test these conjectures. The effectiveness of the audit trail was measured by elapsed time from an adverse event occurring to its discovery in the audit trail, with shorter elapsed times indicating greater effectiveness. Given the differing natures of adverse events that could be discovered, the scope of the research plan was limited to the discovery of malware on computing devices in a company's network.

The pilot study uncovered certain issues that future researchers may encounter, such as difficulty in finding qualified respondents and unclear or ambiguous survey questions. Suggestions for improving the survey’s questions and response count are provided, based on the original issues the study encountered. Although the pilot study did not provide much new data to base recommendations on, this thesis used data from existing literature to provide general recommendations, which included ensuring analysts are familiar with

(8)

viii

the network environment via training programs and organizational knowledge sharing, and consulting with IDS/IPS vendors and specialists during the system installation.

(9)

1 Introduction

According to a recent study, the average cost of a data breach is over $7 million, or $214 per compromised record (Ponemon Institute, LLC, 2011). Effective audit trail

management could potentially improve the effectiveness of information system security controls and reduce the occurrence of breaches. Organizations of significant size face the increasingly difficult task of protecting their information system networks from intruders or malicious actors. Meanwhile, network protection must also be balanced against accessibility and convenience for authorized users.

In order to achieve this balance of security and usability, organizations generate considerable amounts of data relating to network traffic and security logs. This information is generated automatically, and technologies such as Intrusion Detection Systems (IDS) or Intrusion Prevention Systems (IPS) are employed to assist information systems and network security personnel in identifying and stopping network intrusions. However, while organizations rely heavily on utilities such as IDS and IPS to sort through the volume of data generated from internal logging, human management of this information can also be an important factor in effectively monitoring network traffic. The process by which network security audit trails are managed varies considerably among organizations, and there does not exist currently a set of well-researched best practices for IS network security managers to follow. Organizations invest heavily in information systems. Keeping these systems secure against unauthorized access and breaches is critical to maintaining business operations and protecting proprietary and confidential information.

(10)

2

This thesis analyzed the current industry practices and management policies used in audit trail management and provided an overview of management practices and policies

believed to be effective for network protection. The research formulated several conjectures regarding factors that may lead to faster malware detection times, and described a research plan and survey instrument to test these conjectures.

The Verizon RISK team conducted a study into data breaches for 2011. The team

discovered that external attackers committed the vast majority of breaches, and that 69% of breaches utilized malware to perpetrate the attacks (Verizon, 2012). The audit trail's purpose is to provide a resource for detection, though not necessarily prevention. This study focused on the detection of malware that had infected a networked device. The research measured the amount of time malware goes undetected as an indicator of effectiveness, such that shorter detection times indicated greater effectiveness. Objectives

In designing and implementing my research plan, I wished to identify management practices that might lead to quicker discoveries of adverse events, specifically those practices that relate to the implementation of IDSs. The first step was to identify what the key success factors of audit trail management were. These “success factors” were

specifically management policies and organizational conditions that allowed for quick detection of malware infections on networked computers.

In order to determine the key success factors of an organization's audit trail management, I proposed a survey of managers with access to policy data and breach information. This survey would ask respondents questions specifically related to their company's policies

(11)

3

over practices within the network and information security group. In addition, public information regarding information security breaches would be referenced to provide a more complete picture of security effectiveness in the organizations surveyed. Ultimately, the survey results would be analyzed to identify any correlation between information system policies and data security effectiveness. Effectiveness would be determined by measuring the time between an adverse event occurring and its discovery in the audit trail. The analysis of the survey responses was planned to provide insight into the effectiveness of audit trail management practices.

(12)

4

Literature Review of Prior Research

Prior research has examined the importance of the audit trail and information system management in preventing or reducing the incidence of data breaches. This section

reviews the current significant literature regarding methods of audit trail management and effectiveness.

Network Audit Trails

When referring to computer networks, the audit trail is considered “…a series of records of computer events, about an operating system, an application, or user activities”

(Gopalakrishna, 2000). Gopalakrishna identified four significant functions of the audit trail: individual accountability, reconstructing events, problem monitoring, and intrusion detection. Gopalakrishna also noted several challenges and areas of focus with the current state of audit trail tools and analysis:

• Content and format standards • Audit analysis

• Audit compression • Audit tamperproofing

Audit Trail Integrity

When considering the use of audit trails to detect or record malicious activity, it is important that the audit trail is appropriately tamper proofed, as a malicious user could attempt to mask their activity or misdirect auditors. While a malicious user with full system/network privileges could use direct system IO to author false logs, there are methods available for detecting audit trail tampering. To detect tampering, Cohen

(13)

5

recommended log and system behavior analysis to identify absence of expected behavior, introduction of unexpected behavior, and audit trail inconsistency (Cohen, 1995).

Cohen identified specific techniques a malicious user might employ to tamper with the audit trail. The first technique was full deletion of audit trail logs (or logs specific to the intruder’s access). This technique was most indicative of an intrusion, but might help the intruder to mask their purpose or identity. The second technique was a more focused variant of the first. The intruder would identify and modify select audit trails to remove indications of the intruder’s presence. The final technique was to specifically avoid audited events, so that any activity was simply not recorded in the audit trail (Cohen, 1995).

Intrusion Detection Systems

System Design and Theory

Computer systems in an organization generate a significant amount of log data (the audit trail). Helman and Liepins (1993) defined an audit trail as a collection of "transactions," which typically contain a base set of information regarding which action(s) was/were performed. They described the statistical model of analyzing the transactional data in the audit trail to create a misuse probability level. The statistical likelihood of misuse was the foundation for the application of monitoring systems to assist in the detection of misuse. Patel, Qassim, and Wills (2010) described the concept of intrusion detection and

prevention systems (IDS/IPS) and their current state. There were two primary ways in which an IDS/IPS identified potentially malicious traffic. The first method was rule-based, which required that any activity that violates a policy in place was flagged or

(14)

6

prevented when it happens. The second method was anomaly-based. The IDS/IPS recognized traffic that was abnormal or out of pattern (e.g., network access from a new device or location).

IDS Management

The literature on audit trail management with intrusion detection systems is still relatively sparse. Most of the prior research creates the framework for the audit trail to exist and studies the learning necessary for the trail to be effectively used to identify security incidents.

There is some debate over the application of intrusion detection systems. While there is currently an industry push towards completely automated systems, a study noted that in practice human involvement could be equally important (Goodall, Lutters, & Komlodi, 2009). This study of IDS/IPS managers helped to clarify the role that these systems play in the organization's information systems security strategy. According to study

respondents, an IDS should provide "awareness and control" for the security personnel, and display events for a human operator to respond to. Human management was

important for network and log management, as an IDS might trigger thousands of alerts every day, with a 99% false positive rate (Julisch, 2003). The Goodall study is one of a few that identify the human component of information system security management. However, it focused primarily on the roles of the intrusion detection analyst and how learning and training apply to make the analysts more effective.

(15)

7

Intrusion Detection System vs. Intrusion Prevention System

Throughout this thesis, the terms intrusion detection system (IDS) and intrusion prevention system (IPS) are used fairly interchangeably (sometimes combined to IDS/IPS). However, there is an important distinction between each system type, and some literature recommends using IDS, IPS, and host-based IDS (HIDS) in tandem to maximize the effectiveness of these network security systems (Holland, 2004).

Intrusion detection systems are often installed on the network perimeter either directly before or after the firewall (or in some cases multiple IDS “sandwich” the firewall to test firewall effectiveness) (Holland, 2004). The IDS primarily serves to “observe and report” network traffic that appears to be malicious. Host-based IDS is a special

application of IDS typically confined to a single device (such as a server), as opposed to a purpose-built standalone device.

Intrusion prevention systems might be considered an intrusion detection system

combined with a firewall, as they merge the packet-dropping function of a firewall with the deep packet inspection of an IDS (Holland, 2004). An IPS is empowered to drop packets or disable network connections automatically when it determines that malicious activities are present.

(16)

8

Research Program and Methodology

There is much that is unknown regarding effective network security audit trail and intrusion detection system management. This thesis established a model of current practices in place in organizations today. This research attempted to formalize and expand upon a model provided by The National Institute of Standards and Technology (NIST) for audit trail management.

Audit Trail Management Theoretical Model

The foundations for the theoretical model of audit trail management comes from

recommendations by NIST (An Introduction to Computer Security: The NIST Handbook, 1995) regarding certain factors that are believed to have a significant impact on the effectiveness of audit trail management. The NIST report listed the record of “audit events,” review of audit trails, reviewer’s relationship to the audit trail data, and tools for audit trail analysis as important audit trail considerations. Using these factors, a

theoretical model was constructed with the four considerations as the independent variables and some measure(s) of effectiveness as the dependent variables(s). This research sets the time (in hours) from event to discovery as this measure, with shorter times indicating greater effectiveness.

The nature of the adverse event can vary significantly, which could lead to mixed or unreliable results. This research plan attempted to mitigate this by limiting the scope to malware present on a device in the network.

(17)

9

The following is a list of components identified by NIST for effective audit trail

management. These components provide the foundation of a theoretical model for audit trail management.

Table 1.

Components of Audit Trail Management

Term Definition

Log Extensiveness The quantity and quality of what is recorded in the system logs. This extends to which systems and applications are logged, and the amount of detail contained within the logs.

Log Review The frequency and depth (level of detail) of

audit trail review.

Reviewer’s Role The relationship that the reviewer has to the system or application being audited. The NIST identified application/data owners, system owners, and security managers as potential reviewers.

Audit Trail Tools Techniques and tools used to assist in the management of audit trail data. The NIST report listed log reduction, trend and variance analysis, and attack signature detection tools.

(18)

10 Reasoning

Log Extensiveness

The extent of data recorded is a fundamental component of the audit trail system. When more data is recorded, the potential to identify or prevent adverse events is greater. However, if too much unnecessary data is logged, the “signal” of valuable data to a reviewer may become lost in the “noise” of useless data. If too much or too little data is collected, the analyst may not be able to identify security incidents. However, audit trail tools may have the effect of raising the upper limit of what can be recorded, as the “noise” can be filtered depending on the level of detail required for the log review. Log Review

Log review refers to the review frequency and review depth of audit trail data. Review frequency will theoretically provide for quicker detection of an adverse event. A daily review would limit the amount of time an adverse event can go undetected, and allow for the organization to more quickly enact or correct controls to prevent another event from occurring. However, a frequent review of audit trail data does not guarantee that adverse events are detected. The review “depth” or level of detail is important to ensure that adverse events are not missed. In theory, a more thorough review will identify more adverse events than a less thorough one.

Reviewer’s Role

Audit trail reviewers should be familiar with the system they are reviewing. The theory is that someone who knows the system well will be able to identify suspicious events while recognizing harmless events. The reviewer could potentially be the data or application

(19)

11

owner, the system owner, or the security manager. In some cases, too much familiarity may be detrimental to the audit trail review, as “fresh” eyes might scrutinize things that a familiar reviewer would overlook.

Audit Trail Tools

Because of the volume of data that an audit trail will produce in a large organization, the effective use of audit trail tools is required to augment the reviewer’s ability. NIST identified log reduction, trend and variance analysis, and attack signature detection tools as useful devices for enhancing audit trail effectiveness. Log reduction tools filter log information that is known or believed to be non-suspicious (such as nightly backups or standard operations), so that only information that can help identify adverse events remains. Log reduction is primarily a detection tool, as it is used to assist a human reviewer, who cannot actively prevent adverse events from occurring.

Trend and variance analysis tools are advanced audit trail management tools that look at the traffic as it occurs and attempt to identify out-of-pattern activity. These tools may require some “training” before they can be optimally effective, as they must learn the normal usage patterns. When an out-of-pattern activity occurs, these tools can either flag it as important for manual review at a later time, or actively prevent the activity. This property makes trend and variance analysis tools useful for both detection and prevention of adverse events.

The final audit trail tool listed by NIST was attack signature detection. This is the most aggressive of the tools mentioned. Attack signature detection tools actively monitor logs generated by the audit trail and compare them to known attack patterns. If the audit trail

(20)

12

matches an attack pattern, a warning is generated and countermeasures are potentially activated.

Adverse Events

The dependent variable in my research model is interpreted as the detection of some adverse event. An adverse event in this context is considered a breach or security incident related to the organization's information systems. An effective audit trail management process should be able to rapidly identify such events and notify the appropriate personnel of the event. This research uses time to discovery as a quantifiable metric to assess audit trail effectiveness.

A specific type of adverse event is necessary to derive comparable results from the study participants. Given the high incidence of malware in breaches of all kinds (Verizon, 2012) and the ability of the intrusion detection system to identify malicious traffic caused by such malware (Meyer, 2008), this research focused on the presence of malware as the adverse event necessary for the dependent variable. The research asked the respondents to provide detection times for their three most recent breaches, and used the mean time among the three events as the dependent variable for the respondent.

Formalized NIST Model

The formal audit trail management theoretical model developed by the NIST can be defined as:

T(AEd) = f(detail, review(frequency, depth), role, tools(reduction, analysis, signatures)) Where T(AEd) is the time until an adverse event is detected, detail is the log

(21)

13

“review” and “tools” are self-describing with the aid of the enumerated dimensions in parenthesis.

The following diagram illustrates the relationship between the components of the audit trail management model.

(22)

14 Expanded Audit Trail Theoretical Model

In addition to the variables presented by the NIST for a theoretical model of audit trail effectiveness, researchers and industry publications have identified other variables that may be relevant to maintaining an effective intrusion detection system (Goodall, Lutters, & Komlodi, 2009); (Innella, McMillan, & Trout, 2002). This research suggested that using IDS effectively relies on proper initial installation, configuration, and tuning, and proper operational procedures such as established incident response policies, IDS update frequency, and adequate staffing. Based on the research presented in these publications, I have expanded the theoretical model. The formal expanded theoretical model for my research plan was as follows:

T(AEd) = f(detail, review(frequency, depth), role(analyst familiarity, training), tools(reduction, analysis, signatures), installation(configuration, tuning), operations(incident response, staffing, updates))

This model added to the NIST model by including installation and operations to the function, as well as expanding the role variable.

While the expanded theoretical model was now more comprehensive, it was infeasible to test each component within the scope of a single study. The emphasis of this study was on the human management of IDS/IPS (i.e., installation, operations, role), so for the sake of making the study scope more manageable, technical components (detail, tools) were not measured. For the purposes of this research, I extracted a subsetted model from the expanded theoretical model and proposed tests for each variable in my modified theoretical model. The subsetted audit trail theoretical model follows:

(23)

15

In this model T(Md) (Time to Malware detection) is a function of installation process, operational procedures, and the analyst’s role and familiarity with the network

environment. This research model relies on a more specific dependent variable to test, compared with the more generic T(AEd) (Time to Adverse Event detection). This model can be conceptualized with the following diagram:

Figure 2: Subsetted Theoretical Model Intrusion Detection System Testing

My research focused on various aspects of Intrusion Detection System (IDS) policies and procedures to see what relationship these policies had to the rapid detection of malware on the system. Formally, the IDS policies served as the independent variables that were tested in my research.

In selecting specific variables to test, I attempted to align them with the variable categories from the research model above.

(24)

16 Proposed Variables and Conjectures

Installation, Integration, and Deployment Process Conjecture:

1. Intrusion detection systems installed with the assistance of the IDS vendor will lead to faster malware detection on the network.

The proper installation of an intrusion detection system may have a significant impact on the ability of the organization to properly monitor network traffic for signs of malware. In a report by Symantec, researchers identified several methods large organizations may employ to assist in the proper installation and integration of the IDS (Innella, McMillan, & Trout, 2002). These methods included using the system defaults, consulting user manuals and online documentation, and employing outside consultants or vendor technicians to oversee the installation. The research tested whether there was any measurable difference in effectiveness between an in-house installation and a vendor or consultant-managed installation.

IDS Operations

The Symantec report also mentioned challenges with the integration and deployment of an intrusion detection system (Innella, McMillan, & Trout, 2002). These challenges included:

• Establishing incident response guidelines • Staffing

• Configuration • Training

(25)

17

• Updating signatures

These represent a considerable set of variables that can be tested. This thesis research selected a subset of operational components, identified below. The research specifically selected components that relate to human management processes, and that could be feasibly measured within the scope of the survey.

Establishing Incident Response Guidelines Conjecture:

2. Organizations with a formal incident response process will discover malware infections faster than organizations without a formal incident response policy.

A standard policy might help to expedite the process of investigating and resolving alerts that the intrusion detection system created. I theorized that organizations with a well-defined process would discover and address malware incidents faster than organizations without such a process. This research asked respondents if there was a formal incident response policy in place. If there was a response policy in place, the research also asked respondents to detail the procedure in order to identify possible themes in effective response policies.

Staffing Conjecture:

(26)

18

4. More IDS analyst-hours will lead to faster malware detection times.

The Symantec report stated that it was important to have staff dedicated to IDS

management, log inspection, and analysis (Innella, McMillan, & Trout, 2002). The thesis research measured the estimated man-hours dedicated to management of the intrusion detection system, as well as the number of employees dedicated primarily to monitoring and administering the intrusion detection system.

Configuration

Proper configuration of the IDS is a recurring theme within the effective IDS

management literature. Network environments are complex and the literature suggests that specialized tuning may be helpful to optimize the IDS for the unique environment. The nature of possible tuning measures makes it difficult to gauge the appropriateness of the tuning for the environment over a survey. This dimension of IDS operations may be a topic for future research.

Training

Symantec listed training in configuration, incident response, log analysis, and IDS maintenance as important factors for staffing an IDS department (Innella, McMillan, & Trout, 2002). While the level of training received by employees may be significant, I believe that there were too many aspects of training to adequately explore for a single research plan. This dimension of IDS operations may be a good topic for future research.

(27)

19 Updating signatures

Conjecture:

5. Intrusion detection systems that are updated frequently will lead to faster malware detection than systems that are updated less

frequently.

Because an IDS looks both for patterns of malicious behavior, as well as established signatures of known malicious activity, the frequency of signature updates to the IDS is likely to have a considerable impact on the effectiveness of the IDS in detecting malware within the network. This research collected data on how often the signature and heuristics engines of the IDS are kept updated, including the time from the last update, and whether the organization adhered to a specified update schedule by the IDS vendor.

Analyst’s Familiarity with the Network Environment Conjecture:

6. Analysts who are more familiar with the organization’s network will identify malware-infected devices faster than analysts with less familiarity.

During interviews conducted by Goodall et al., several participants stressed the

importance of “learning the environment” as a means of effectively managing the alerts caused by the IDS (Goodall, Lutters, & Komlodi, 2009). This aligns with the ‘reviewer’s role’ segment of the NIST model presented above.

(28)

20

It would be difficult to accurately gauge an analyst’s familiarity with the network environment without implementing a comprehensive test. Instead, I proposed to approximate familiarity to the environment by measuring the tenure of the average analyst. The research measured the length of time that the analyst had worked in an IS role at the current location, and how long the respondent has held his or her current position. The research also asked the responding analysts to rate their perceived familiarity with the network they were responsible for monitoring.

The reasoning for conjecture #6 was the observation that the longer analysts have been employed at an organization, the more familiar with it they become. With increased familiarity, anomalies should be easier to spot, leading to decreased response times. Summary of Proposed Variables to Study

• Independent variables

o Installation process

§ In-house, vendor assisted, or third-party assisted o Operational procedures

§ Incident response policy in place (yes or no)

§ Number of weekly man-hours dedicated to IDS operations

§ Number of employees primarily dedicated to IDS analysis

§ IDS signature updates (yes or no)

• Frequency of updates • Days since last update

o Analyst’s familiarity with the environment

(29)

21

§ Number of years in current position

§ Self-evaluated familiarity (1 to 10 scale)

• Dependent variable

o Average time-to-detection for the three most recent malware incidents Survey Instrument

I utilized a survey to collect data regarding IDS management practices and malware detection time. The survey collected data to measure variables outlined in Summary of Proposed Variables above. Most questions yielded interval or ratio-type data. The research plan called for use of the Pearson correlation test for this data, as well as the point-biserial and Kendall correlation tests for the data that could not be evaluated via the Pearson correlation. Insufficient data were collected in the pilot survey to apply the statistical tests.

Survey administration can be accomplished through a number of channels. I elected that the survey be administered via a web application, with responses automatically recorded in a spreadsheet. The survey instrument questions are listed in the Survey section below, and the full survey, data types correlation tests, and notes are available in Appendix C. Respondents

Survey research follows the basic principles of statistical sampling. In order for the results to be meaningful, the sample should be representative of the population as a whole. For this research, there were several specific qualifications that permitted inclusion in the population in question. If these qualifications were met, then the entity that met them was considered a valid participant in the population.

(30)

22

The primary consideration for inclusion in the survey population was that the respondents be from an organization of significant size. Organizations smaller than a certain threshold may not have properties that scale consistently with larger organizations, therefore, results from these organizations would not be representative of the average organization. The question remains of where the threshold between large and small organizations should be. According to U.S. Census data, there are roughly an equal number of

employees working for organizations larger than 500 employees as there are employees in organizations with less than 500 employees (US Census Bureau, 2007). See Appendix A for Census data and charts.

The threshold of organizations larger than 500 employees represents roughly the midway point for organizations by the absolute number of people they employ; however, firms of at least 500 employees account for more than half of employee payroll and firm revenue for U.S. firms with at least one employee. In the interest of protecting respondent

confidentiality, I did not ask for respondents to provide the number of employees in their organization. However, I did approximate this by asking for the number of personal computing devices in the organization. No threshold was set for the survey, but each respondent’s organization was near to or greater than 500 devices.

Because this study’s objective was to better understand the inner workings of audit trail management, the survey targeted individuals who could provide unique and valuable insight into their firm’s information system management. The ideal respondent would be an active member in the organization’s information systems department, and have some definite interaction with network monitoring or security.

(31)

23

The respondent’s tenure with the firm or position and responsibilities would be recorded, but these are not necessary selectors for respondent inclusion. Insights from a wide variety of employees by responsibilities and years of experience in the information systems department were valuable for the purposes of this study.

The difficulty of acquiring enough acceptable respondents was largely unknown. I suspected that potential respondents would be hesitant to volunteer much information, given the proprietary and confidential nature of information security practices. In order to encourage participation, it was emphasized to potential respondents that their individual responses would be held confidential, and published data would be anonymized.

As a preliminary research trial, I solicited responses from professional organizations such as ISACA (Information Systems Audit and Control Association), and ISSA (Information Systems Security Association). These organizations have over 100,000 members, many of whom qualify as target respondents for the survey (ISACA, 2013) (Information Systems Security Association, 2012).

Survey

1. What is your current title or position at your organization?

2. Briefly describe your responsibilities as they relate to Information Security

3. How many years have you been employed with your organization in an IT-related role? Please enter a number, rounded up to the nearest year.

4. How many years have you been in your CURRENT position with your organization? Please enter a number, rounded up to the nearest year.

(32)

24

5. On a scale of 1 to 10, how familiar are you with your network environment and topology? 1 is not at all familiar, and 10 is completely familiar

Not familiar 1 2 3 4 5 6 7 8 9 10 Completely familiar

6. During the installation of your organization’s intrusion detection system, were external consultants utilized?

• Yes, the IDS vendor assisted with the installation

• Yes, a third-party consultant assisted with the installation • No, the IDS was installed using only in-house resources • Other:

7. When your organization’s intrusion detection system identifies a potential security incident, is there a formal incident response procedure in place?

• Yes

• No

8. Briefly outline your organization’s incident response policy

9. How many man-hours per week do you estimate are dedicated to managing and administering the intrusion detection system? Please enter a number, rounded up to the nearest hour.

10. How many individuals are employed whose job duties are primarily to analyze and respond to alerts generated by the intrusion detection system? Please enter a number

(33)

25

11. Does your organization’s intrusion detection system receive regular signature updates?

• Yes

• No

12. How frequent are the signature updates for the intrusion detection system? (e.g. monthly, weekly, annually)

13. How many days has it been since the most recent update to the IDS signatures? Please enter a number rounded up to the nearest day.

14. How many personal computers (laptops and desktops) are in your organization's network? Please enter a numeric estimate

15. How many unique devices were infected with a virus or malware in the past 30 days?

16. Please recall the three most recent malware infection incidents. For each

incident, how long did it take before the incident was responded to and the infected device(s) quarantined?

Please enter a number rounded up to the nearest minute.

Incident 1 Detection time in minutes Incident 2 Detection time in minutes Incident 3 Detection time in minutes

(34)

26

Findings: Preliminary Research Trial

The survey instrument above was distributed to a small sample in order to gather preliminary data and establish a potential direction for further research. Responses were gathered through an online survey distributed to members of ISACA and professional contacts. There were four qualified respondents from this survey. Due to the limited sample size, no statistically significant inferences can be made from the survey data. However, the response data provided some basic insight into procedures in practice today and hint at their effectiveness. This may be of value to future researchers.

Summary of Responses

Included below is a summary of responses that were received during the preliminary research program. Responses are presented in such a way as to anonymize the source of the response.

What is your current title or position at your organization?

All position titles listed indicated responsibility for network and/or information security at the respondent’s respective organizations. Specifically, the term “security” was present in every title.

Briefly describe your responsibilities as they relate to Information Security.

Each respondent had some responsibility to ensure the security and uptime of the network within their organization. Roughly half of the respondents were responsible for

(35)

27

How many years have you been employed with your organization in an IT-related role?

The mean response for this question was 6.25 years. The shortest amount of time was 1 year, while the maximum time was 12 years.

How many years have you been in your CURRENT position with your organization?

As expected, time at the respondent’s current position was less than total time within the organization on average. The mean response for years at the respondent’s current position was 2.75 years. This is 44% of the total time the respondent has been with the

organization. The shortest time was 1 year. The longest time was 5 years.

On a scale of 1 to 10, how familiar are you with your network environment and topology?

All respondents indicated fairly high familiarity with their network environment. The mean response to this question was 8.25. The lowest response was 7 out of 10 (with 10 being very familiar and 1 being not at all familiar). There is a moderate correlation between familiarity and years at the organization (r2 = 0.56), as well as familiarity and years at the respondent’s current position (r2 = 0.55).

During the installation of your organization’s intrusion detection system, were external consultants utilized?

Most respondents utilized external assistance when installing their IDS/IPS. Only one respondent indicated the IDS was installed using in-house resources. The IDS vendor

(36)

28

assisted with one installation, while the other two were installed with the assistance of third-party consultants.

When your organization’s intrusion detection system identifies a potential security incident, is there a formal incident response procedure in place?

All respondents indicated there was a formal incident response policy in place at their organization.

Briefly outline your organization’s incident response policy

All respondents indicated a well-defined incident response policy that included involving multiple groups as appropriate, documenting the incident for performance tracking, and incident classification.

How many man-hours per week do you estimate are dedicated to managing and administering the intrusion detection system?

There was drastic difference in responses to this question. Two respondents indicated 1-3 hours per week, while the other two indicated 45-50 hours. It is unclear whether the differences in responses are due to organizational differences or different interpretations of the question (i.e. what defines “managing and administering the intrusion detection system” may be interpreted differently by each respondent).

The original intent of the previous question was to capture the total man-hours dedicated to monitoring the organization’s IDS/IPS, regardless of the number of individuals monitoring the network. This question may need to be revised by future researchers in order to ensure respondents correctly interpret and respond to the question. I suggest

(37)

29

rephrasing the question as follows: “How many man-hours per week are dedicated to analyzing and responding to alerts generated by the intrusion detection system?” How many individuals are employed whose job duties are primarily to analyze and respond to alerts generated by the intrusion detection system?

Interestingly, the respondent who indicated only 1 man-hour of IDS/IPS monitoring reported that there were three people whose primary job duties were to analyze and respond to alerts generated by the IDS/IPS. It is possible this respondent transposed their responses to these two questions.

The other respondents indicated 1-3 employees who were primarily responsible for monitoring and responding to IDS/IPS alerts.

Does your organization’s intrusion detection system receive regular signature updates?

All respondents indicated that they did receive regular signature updates for their IDS/IPS.

How frequent are the signature updates for the intrusion detection system?

Intrusion detection signatures were updated as often as daily to weekly for most respondents. One respondent indicated that the update schedule was non-specific, and that updates were received “continually.”

(38)

30

The respondents indicated that their IDS signatures were updated most recently within one week.

How many personal computers (laptops and desktops) are in your organization's network?

Two respondents had organizations of 400-500 devices, and the other two have organizations with over 10,000 devices.

How many unique devices were infected with a virus or malware in the past 30 days?

This is the first of two questions containing the dependent variables. Three respondents had zero or one device infection. The respondent from the organization with the most devices had 40 malware infections within the past 30 days.

Please recall the three most recent malware infection incidents. For each incident, how long did it take before the incident was responded to and the infected device(s) quarantined?

Most respondents reported that infected devices were quarantined within two hours. One incident took roughly two days to quarantine, and the fastest response time was within five minutes.

See Appendix D for raw response data with potentially identifying information removed to protect respondent anonymity.

(39)

31 Response Analysis

The primary aim of the research presented here was to provide future researchers with a framework for identifying management practices and policies related to IDS/IPS

management that have an impact on malware detection times. As such, this preliminary study evaluated survey response data and demonstrated basic qualitative analysis related to malware detection times.

Raw Response Data

The following table represents the response data from the four respondents. Classification responses (current title and job description) have been removed to protect the

respondents’ identities. How many years have you been employed with your organization in an IT related role? 9 12 1 3

How many years have you been in your CURRENT position with your organization? 4 5 1 1 On a scale of 1 to 10, how familiar are you with your network environment and topology?

(40)

32 During the installation of your organization’s intrusion detection system, were external consultants utilized? No, the IDS was installed using only in-house resources

Yes, the IDS vendor assisted with the installation

Yes, a third-party consultant assisted with the installation Yes, a third-party consultant assisted with the installation When your organization’s intrusion detection system identifies a potential security incident, is there a formal incident response procedure in place?

(41)

33 Briefly outline your organization’s incident response policy Pre-defined process in terms of detection, reporting, follow-ups and closure.

- Mandatory that all the employees to report any IT security incident - On receiving the incident report InfoSec manager must immediately investigate the incident - A report on the incident is written highlighting cause of the incident, impact to the organization, mitigation measure/corrective actions, whether action plan is urgent or not - Include the incident in the incident database for performance tracking Identify, gather, analyze such incident, involves management and IR team which consists of security engineers. If confirmed, forensics and further detailed analysis will be performed and actions taken to rectify. Initial Incident Responder follows existing Incident Handling policy to determine severity of incident (low, medium, high) and involve appropriate parties.

How many man-hours per week do you estimate are dedicated to managing and administering the intrusion detection system? 1 45 3 50 How many individuals are employed whose job duties are primarily to analyze and respond to alerts generated by the intrusion detection system? 3 3 1 2

(42)

34 Does your organization’s intrusion detection system receive regular signature updates?

Yes Yes Yes Yes

How frequent are the signature updates for the intrusion detection system?

depends daily bi-weekly weekly

How many days has it been since the most recent update to the IDS signatures?

always

updated Just last week 2 5

How many personal computers (laptops and desktops) are in your organization's network? 10000 plus 500 400 30000 How many unique devices were infected with a virus or malware in the past 30 days? 0 1 1 40

(43)

35 In the past

Please recall the three most recent malware infection

incidents. For each incident, how long did it take before the incident was responded to and the infected device(s) quarantined? Incident 1 30 2 days 5 120 Incident 2 30 15 120 Incident 3 30 10 120 Mean incident response time 30 2 days (2880 minutes) 10 120 Observations

Most respondents reported an incident detection time of less than two hours, but there was an outlier response of “2 days” that drastically skewed any quantitative analysis. Of course due to the limited number of responses no meaningful quantitative analysis could be performed. Another interesting component of the responses was that two respondents entered identical times for all three incident response times. This led me to believe the respondents may have guessed the detection times based on recent memory.

For most survey questions, respondents were given response instructions (e.g. “enter time in minutes”) but were permitted to enter any text they wished in the response field. This led to some responses that were more difficult to quantify for analysis (e.g. “2 days”, “10000 plus”, and “jlast week”). Free form responses can be rounded to comparable units

(44)

36

(“2 days” to “2880 minutes”), but numerical precision is lost, as “2 days” may mean either something as precise as “exactly 2880 minutes from infection to detection” or as ambiguous as “we were infected sometime on Wednesday and isolated the machine by end of day Friday” to the respondent.

The final question In the past Please recall the three most recent malware infection incidents. For each incident, how long did it take before the incident was responded to and the infected device(s) quarantined? uses potentially ambiguous terms, which could yield inaccurate or incomparable responses. For the purpose of this research, the intended value to measure is the time to malware detection by the network security or information security team.

Two questions from the survey returned inconsistent responses even within respondents. The survey questions How many man-hours per week do you estimate are dedicated to managing and administering the intrusion detection system? and How many individuals are employed whose job duties are primarily to analyze and respond to alerts generated by the intrusion detection system? were expected to correlate to each other. The original intent of the previous question was to capture the total man-hours dedicated to

monitoring the organization’s IDS/IPS, regardless of the number of individuals

monitoring the network. The respondents may have misinterpreted this question, as it is unlikely that three employees spent a combined single hour on IDS management during a week (seen in column one). The question and responses are copied below.

(45)

37 How many

man-hours per week do you estimate are dedicated to managing and administering the intrusion detection system? 1 45 3 50 How many individuals are employed whose job duties are primarily to analyze and respond to alerts generated by the intrusion detection system? 3 3 1 2

Survey Revision Recommendations

Based on the observations above, I recommend that the survey questions and response format be revised slightly, so that future researchers can avoid the issues discovered with the pilot survey.

1. Limit response formats

For questions that require comparable units of analysis, I recommend permitting only numerical values. This should limit the amount of rounding and guessing the researcher must do in order to yield comparable data. If necessary, encourage the respondent to round or estimate to the best of their ability in the case a precise number is not available.

(46)

38

Some questions appear to be misunderstood by the respondents, while others yielded responses that were not optimal for qualitative analysis. I propose the following wording changes to the survey questions:

Original survey question Revised survey question How many man-hours per week do you

estimate are dedicated to managing and administering the intrusion detection system?

How many man-hours per week do you estimate are dedicated to analyzing and responding to alerts generated by the intrusion detection system?

How frequent are the signature updates for the intrusion detection system?

How many days are there typically between signature updates for the intrusion

detection system? (Limit responses to numerical values only)

Alternatively, keep original question and limit responses to specific date ranges (daily, weekly, every two weeks, monthly) In the past Please recall the three most

recent malware infection incidents. For each incident, how long did it take before the incident was responded to and the infected device(s) quarantined?

In the past please recall the three most recent malware infection incidents. For each incident, how long did it take before your network or information security team detected the incident?

(47)

39

Recommendations: Further Research Plan Outline

The following is a research plan outline that may be followed in order to further the preliminary research on the subsetted audit trail model provided in this thesis. The plan presented here utilizes lessons learned from the preliminary research discussed earlier. Research Hypotheses

I recommend the following hypotheses be tested:

1. Analysts who are more familiar with the organization’s network will identify malware-infected devices faster than analysts with less familiarity.

2. Intrusion detection systems installed with the assistance of the IDS vendor will lead to faster malware detection on the network.

3. Organizations with a formal incident response process will discover malware infections faster than organizations without a formal incident response policy. 4. More IDS analysts will lead to faster malware detection times.

5. More IDS analyst-hours will lead to faster malware detection times.

6. Intrusion detection systems that are updated frequently will lead to faster malware detection than systems that are updated less frequently.

A survey research methodology should be used to test these research hypotheses. Distribute Modified Survey

Provided in appendix C is a modified survey instrument to collect data from network security managers. I suggest preparing this survey for distribution through an online tool in order to make gathering and compiling responses easier. It is important to have a large pool of qualified potential respondents to distribute the survey to. For the pilot survey,

(48)

40

qualified respondents were difficult to find. Links to the survey were published on websites of organizations for IT professionals, but the links were not in prominent areas of the sites.

Increase Response Rate

In order for future research to be effective, the survey response rate must be increased. For future research, I recommend that the survey links be distributed via a more direct approach. Methods such as direct email from a list of qualified respondents could yield many more responses, provided the researcher can gain access to such a distribution list. It may be valuable to partner with a security-focused organization or a group that

represents (and has access to) network security professionals, so that the group’s members may be specifically targeted for survey distribution.

The modified survey addresses some concerns from the pilot survey. Specifically, ambiguity was removed where possible so that respondents answer the questions as intended.

Analyze Survey Responses

Using the survey response data, consult the correlation tests table in appendix C in order to determine which statistical test to apply to the survey results. Given the variety of independent variables used to test against detection times, a multiple regression test may reveal more than individual correlation tests.

(49)

41

Limitations of This Study

The extent of this research has certain limitations. First, many factors from the theoretical model were excluded from the scope of the study. The subsetted theoretical model omits factors that might have a significant impact on malware detection times. Factors such as analyst training or which specific IDS tools are used may play an important role in quickly detecting malware, but these factors were not considered in this study. By limiting the number of factors studied, the sample size necessary for statistical analysis was reduced. Additionally, certain omitted variables were not conductive to measurement via a survey. Second, responses to the preliminary survey were limited, so there may be issues that become apparent when analyzing survey data from a larger response pool. This limitation largely stems from the limited access to qualified respondents. A larger pool of qualified responses would provide more statistical confidence in the survey responses, but identifying and collecting more respondents would require more time than allotted for this study. Third, this study limits adverse event consideration to malware infections. Limiting the scope of adverse events to malware infections makes quantitative analysis of the theoretical model simpler, but does so at the risk of ignoring the impact of other adverse events, such as internal data theft or destruction, fraud, or malicious

(50)

42

Recommendations: IDS Management

The pilot survey response rate was generally insufficient for deriving statistically significant conclusions about IDS management. Some questions had inconsistent

response data, the dependent variable had an outlier that skewed data and made effective analysis difficult, and the small sample size was not sufficient to establish confidence levels in the correlations provided. Many of the correlation results are counter-intuitive, so caution must be exercised when evaluating responses, as the inferential analysis of response data yielded results that were likely a product of random chance rather than robust statistical testing.

There is still value in the literature review and analysis that went in to preparing the survey instrument; this thesis uses the literature review to provide some preliminary recommendations.

Ensure Analysts are Familiar with Network Environment

Based on research interviews conducted by Goodall et al., IDS managers strongly agreed that the analyst’s familiarity with the environment contributed to their effectiveness (Goodall, Lutters, & Komlodi, 2009). Senior management should work to retain longstanding analysts within the company.

Establish a Training Program for Analysts

A strong training program for junior analysts would ensure that internal knowledge regarding the environment is passed to all analysts. Ensuring that senior talent is available to monitor the organization’s network is not always possible. A training program should help ensure all analysts meet a minimum level of network familiarity.

(51)

43 Encourage Knowledge Sharing

To ensure knowledge and experience is not being inadvertently hoarded, foster a culture of knowledge sharing. This should ensure that compartmentalized knowledge about the organization’s network becomes distributed to all IDS analysts and network/information security personnel.

Consult with IDS/IPS Vendors and Consultants when Installing New Systems IDS vendors and external consultants can ensure that a new IDS/IPS is set up properly and trained to monitor traffic patterns (Innella, McMillan, & Trout, 2002). Organizations that are installing a new IDS/IPS should work closely with the vendor and/or external consultants. If a system is already in place, the organization may choose to have the vendor or a consultant review the system’s configuration to ensure the IDS/IPS is installed correctly and utilizing all available internal tools.

Contribution to Theory and Practice

Given the limited response rate from the pilot study, it is unlikely that a distinct

contribution to network security theory can be extracted from quantitative analysis of the survey results. Qualitative analysis does yield some interesting observations, though their effect on theory contribution is dubious, as they do little to explain relationships between the factors from the audit trail theoretical model.

Evaluating the survey responses from a qualitative standpoint suggested the following: network security managers are generally confident in their familiarity with the network environment, IDS/IPS installation often utilizes vendors or other 3rd parties, IDS/IPS are updated frequently (often weekly or sooner), robust incident response policies are

(52)

44

common in both medium and large organizations, IDS/IPS teams are small (1-3 people) relative to the number of end-user devices in the organization (400-30,000 devices in the case of this study), and finally there was little observed difference in network security management practices from 400 to 30,000 end-user devices.

(53)

45 References

Cohen, D. F. (1995). A Note on Detecting Tampering with Audit Trails. Retrieved July 6, 2014, from Fred Cohen & Associates: http://www.all.net/books/audit/audmod.html Forrester Consulting. (2010, March). The Value Of Corporate Secrets. Retrieved October 5, 2012, from National Security Institute:

http://www.nsi.org/pdf/reports/The%20Value%20of%20Corporate%20Secrets.pdf Goodall, J. R., Lutters, W. G., & Komlodi, A. (2009). Developing expertise for network intrusion detection. Information Technology & People , 92-108.

Gopalakrishna, R. (2000, April). Audit Trails. Retrieved July 7, 2014, from http://homes.cerias.purdue.edu/~rgk/at.html

Helman, P., & Liepins, G. (1993). Statistical foundations of audit trail analysis for the detection of computer misuse. IEEE Transactions on Software Engineering , 886-901. Holland, T. (2004, February 23). Understanding IPS and IDS: Using IPS and IDS together for Defense in Depth. Retrieved July 9, 2014, from SANS Institute:

http://www.sans.org/reading-room/whitepapers/detection/understanding-ips-and-ids-using-ips-and-ids-together-for-defense-in-depth-1381

Information Systems Security Association. (2012). About ISSA. Retrieved May 8, 2013, from Information Systems Security Association: http://www.issa.org/?page=AboutISSA Innella, P., McMillan, O., & Trout, D. (2002, April 4). Managing Intrusion Detection Systems in Large Organizations. Retrieved February 4, 2013, from Symantec Connect:

(54)

46

http://www.symantec.com/connect/articles/managing-intrusion-detection-systems-large-organizations-part-one

ISACA. (2013). 2013 ISACA Fact Sheet. Retrieved May 8, 2013, from ISACA: http://www.isaca.org/About-ISACA/Press-room/Pages/ISACA-Fact-Sheet.aspx

Jamieson, R., & Low, G. (1990). Local area network operations: A security, control and audit perspective. Journal of Information Technology , 63-72.

Julisch, K. (2003). Clustering intrusion detection alarms to support root cause analysis. ACM Transactions on Information and System Security , 6 (4), 443-71.

Meyer, R. (2008). Challenges of IDS in the Enterprise. SANS Institute.

National Institute of Standards and Technology. (1995). An Introduction to Computer Security: The NIST Handbook. In Special Publication 800-12 (pp. 213-223).

Patel, A., Qassim, Q., & Wills, C. (2010). A survey of intrusion detection and prevention systems. Information Management & Computer Security , 277-290.

Ponemon Institute, LLC. (2011). 2010 Annual Study: U.S. Cost of a Data Breach. Traverse City: Ponemon Institute.

US Census Bureau. (2007). Statistics about Business Size (including Small Business) from the U.S. Census Bureau. Suitland, MD: Author.

Verizon. (2012 April). Thought Leadership - Verizon Enterprise Solutions. Retrieved 2013 2-January from Verizon Business:

(55)

47

http://www.verizonbusiness.com/resources/reports/rp_data-breach-investigations-report-2012_en_xg.pdf

(56)

48

Appendix A Employment Size of Employer

Figure 1: Census data for firm size and payroll (US Census Bureau, 2007)

(57)

49

Appendix B Data Leak Incidents

(58)

50

Appendix C Survey Instrument Survey questions, data types, and correlation tests Part A: Original Survey

1. What is your current title or position at your organization?

2. Briefly describe your responsibilities as they relate to Information Security

3. How many years have you been employed with your organization in an IT-related role? Please enter a number, rounded up to the nearest year.

4. How many years have you been in your CURRENT position with your organization? Please enter a number, rounded up to the nearest year.

5. On a scale of 1 to 10, how familiar are you with your network environment and topology? 1 is not at all familiar, and 10 is completely familiar

Not familiar 1 2 3 4 5 6 7 8 9 10 Completely familiar

6. During the installation of your organization’s intrusion detection system, were external consultants utilized?

• Yes, the IDS vendor assisted with the installation

• Yes, a third-party consultant assisted with the installation • No, the IDS was installed using only in-house resources • Other:

(59)

51

7. When your organization’s intrusion detection system identifies a potential security incident, is there a formal incident response procedure in place?

• Yes

• No

8. Briefly outline your organization’s incident response policy

9. How many man-hours per week do you estimate are dedicated to managing and administering the intrusion detection system? Please enter a number, rounded up to the nearest hour.

10. How many individuals are employed whose job duties are primarily to analyze and respond to alerts generated by the intrusion detection system? Please enter a number

11. Does your organization’s intrusion detection system receive regular signature updates?

• Yes

• No

12. How frequent are the signature updates for the intrusion detection system? (e.g. monthly, weekly, annually)

13. How many days has it been since the most recent update to the IDS signatures? Please enter a number rounded up to the nearest day.

14. How many personal computers (laptops and desktops) are in your organization's network? Please enter a numeric estimate

(60)

52

15. How many unique devices were infected with a virus or malware in the past 30 days?

16. Please recall the three most recent malware infection incidents. For each

incident, how long did it take before the incident was responded to and the infected device(s) quarantined?

Please enter a number rounded up to the nearest minute.

Incident 1 Detection time in minutes Incident 2 Detection time in minutes Incident 3 Detection time in minutes

Part B: Modified Survey

1. What is your current title or position at your organization?

2. Briefly describe your responsibilities as they relate to Information Security

3. How many years have you been employed with your organization in an IT-related role? Please enter a number, rounded up to the nearest year.

4. How many years have you been in your CURRENT position with your organization? Please enter a number, rounded up to the nearest year.

5. On a scale of 1 to 10, how familiar are you with your network environment and topology? 1 is not at all familiar, and 10 is completely familiar

(61)

53

6. During the installation of your organization’s intrusion detection system, were external consultants utilized?

• Yes, the IDS vendor assisted with the installation

• Yes, a third-party consultant assisted with the installation • No, the IDS was installed using only in-house resources • Other:

7. When your organization’s intrusion detection system identifies a

References

Related documents

The Outreach Committee would like the Youth Faith in Action families to take turns filling the bags on Sunday after church and dropping them off on Monday morning in the lobby

14 When black, Latina, and white women like Sandy and June organized wedding ceremonies, they “imagine[d] a world ordered by love, by a radical embrace of difference.”

Even if the rule in toto did not increase total hours worked, key provisions like the daily driving limit and the 34-hour restart provision may have increased work hours,

Run the supply and fuel return lines to and from an alternate fuel source as described in the Chassis Fuel System Isolation Test?. Refer to section "Chassis Fuel System

In April 2015, the Central Research Laboratory, Hitachi Research Laboratory, Yokohama Research Laboratory, Design Division, and the overseas research centers were realigned

Recovery Stress Questionnaire for Athletes (RESTQ-Sport, Kallus & Kellmann, 2016; Kellmann & Kallus, 2001), a measure of perceived stress and recovery, is commonly

Having collected together the literary records of the Greek myths, the text-based disciplines, above all Classical philology, study them as testimony for the development of

By paying close attention to barriers to patient access and how their physicians’ time is being used, high-quality medical practices are able to ensure that more of that time is