FINAL REPORT
Incident Cost Analysis and Modeling Project
I-CAMP II
0 0 0 0 0 0 0 0
Project Staff Virginia Rezmierski, Ph.D.
Project Director
Director, CIO's Office of Policy Development and Education The University of Michigan
Adriana Carroll, M.P.P.
Project Coordinator and Research Associate Gerald Ford School of Public Policy
The University of Michigan Jamie Hine, B.A.
Research Associate
Gerald Ford School of Public Policy The University of Michigan
We are thankful for and acknowledge here the valuable part-time assistance of Todd Lee, M.P.P. and Jason Weller, M.P.P., during data gathering and project design phases of this project.
Project Advisory Board Robert Charette George Cubberly Chief Executive Office Assistant Risk Manager
Risk Management Consultant Office of Associate V. P. for Finance ITABHI Corporation Department of Risk Management
The University of Michigan Kathy Kimball, M. S. Eugene Spafford, PhD
Security Director Professor of Computer Sciences Computer Information Systems Director, CERIAS Project Pennsylvania State University Laboratory
Purdue University
Dennis Steinauer Larry Stephens, AIC, EPCU, ARM Computer Security Division Director of Risk Management
National Institute for Standards and Department of Risk Management
Acknowledgments
First and foremost, we are thankful to the USENIX Association for their vision and interest in understanding the impact, type, frequency, and cost of IT related incidents in college and university settings. Without their interest, support, and the project funding this work could not have been accomplished.
USENIX is the Advanced Computing Systems Association. Since 1975 the USENIX Association has brought together the community of engineers, system administrators, scientists, and technicians working on the cutting edge of the computing world. The USENIX Association and its members are dedicated to: problem-solving with a practical bias, fostering innovation and research that works, communicating rapidly the results of both research and innovation, and providing a neutral forum for the exercise of critical thought and the airing of technical issues. USENIX supports its members' professional and technical development through a variety of on-going activities: Annual technical and system administration conferences, a highly regarded tutorial program, SAGE, a special technical group for system administrators, student programs, and awards programs. Special thanks also go to the members of our Project Advisory Board for I-CAMP II. We thank them for their attendance at I-CAMP II Board Meetings in light of their demanding travel and work responsibilities. Most importantly we thank them for their critical thinking and input to the project. They were not hesitant to make suggestions, to criticize ideas or procedures, or to make themselves available to us when questions arose. Their guidance kept the research team moving forward.
We also thank each of the study's participating schools and their personnel who gave of their time and expertise. Instead of cautiously hiding incident data and refusing to openly discuss problems in data collection, these people were sincerely and professionally interested in trying to understand and improve IT-related incident handling on campuses. Special thanks go to Robert Bartlett, Andrea Basing, Mark Bruhn, David Brumley, Katrina Cook, Jacqueline Craig, Jane Drews, Bob Foertsch, Helen Green, Clair Goldsmith, Steve Griese, Stephen Hansen, Susan Levy Haskell, Margie Hodges Shaw, Kathy Kimball, Jim Knox, Doug Nelson, Rodney Peterson, Chris Pruess, Steve Romig, Roger Safian, Jeffrey Savoy, Sara Staebler, Kevin Unrue, Elaine Ward, and Ed Zawacki.
Finally, special gratitude goes to the staff of the Office of Policy Development and Education at The University of Michigan for their continued input and support to this project. Thanks go especially to our editor, Kathleen Young for her help with the final report, to David Nesom and Jon Leonard for technical advice, and to our Office Assistant Joyce Ruppert for her support and scheduling of Advisory Board meetings.
The original I-CAMP report was funded by the Chief Information Officers of the Committee on Institutional Cooperation (CIC). That report, describes and analyzes thirty technology-related incidents occurring on university campuses and provides discussion of factors that seem to affect both cost and occurrence of the incidents. It is available through the CIC representative by sending e-mail to [email protected].
TABLE OF CONTENTS TABLE OF CONTENTS 4 EXECUTIVE SUMMARY 7 PREFACE 8 INTRODUCTION 9 The Problem 9
Personnel Skills and Knowledge 9 Unfavorable Trends 9 Time and Skill Required 9 Management Implications 10
I-CAMP STUDY OVERVIEW 10
PURPOSE OF THE I-CAMP II STUDY 11
I-CAMP II METHODOLOGY 11
FIGURE I - ICAMP II PROJECT OVERVIEW 13
EXPANDING STUDY PARTICIPATION 14
PART I -COST ANALYSIS SECTION 15
Purpose 15 Expanding the Sample of Incidents 15
Table I - INCIDENT CATEGORIES TO BE COST ANALYZED 16 Providing Comparison Data 16 Procedure for Incident Identification 16 Procedure for Incident Cost Analysis 16
Assumptions 17
I-CAMP II Methodology for Calculating User-Side Costs 19 Refining and Increasing the Robustness of the Cost Analysis Model 19 Methodologies for calculating User-Side Costs 20 Table II - CALCULATIONS FOR WILLINGNESS TO PAY OF ONE HOUR OF
STUDY 22
FIGURE 2 - WILLINGNESS TO PAY AND OPPORTUNITY COST 22 Method of calculation 23
Table III - Summary of Results from the Incident Cost Analysis 24 Recommendations Regarding Cost Analyzing IT-Incidents in Academic Environments25
Questionnaire Template 25 Calculation Template 25
Machine cost 25
Reports 25
Part II - FREQUENCY SECTION 26
Database Analysis and Categorizations 26 Designing a Methodology and Gathering Frequency Data 26 Availability of Databases 26
Aggregated Data 26
Problems in Data Collection 27 Summarization of Interview Results 27
FIGURE 3 -DATABASE CREATOR ROLE AND PURPOSE 29
Status of School Categorizations 30 Fairness and Justice in Incident Handling 30 Similarities in Incident Categories 30 Factors Appear Again-Lack of Knowledge/Information, Lack of Resources 31 Frequencies from Database Schools 31 Frequencies from Expert Estimates 32 Who Were the Experts? 32 Methodology for Expert Estimates 32 TABLE IV -A, B, C - EXPERT ESTIMATES - MAIL BOMBS 34 TABLE V - A, B, C - EXPERT ESTIMATES - SYSTEM PROBES 37 TABLE VI - A, B, C - EXPERTS ESTIMATES - WAREZ SITES 40 Results of Expert Estimates 43 Towards a Comprehensive Categorization Scheme 43 Literature Review 43 I-CAMP II Recommended Categorization Scheme 43 FIGURE 4 - INTERFACE OF USERS, DATA AND OPERATING SYSTEMS IN
THE ACADEMIC ENVIRONMENT 45
FIGURE 5 - I-CAMP II CATEGORIZATION SCHEME 47
FINAL RECOMMENDATIONS 51 APPENDIXES
A: QUESTIONNAIRE TEMPLATE
B: CONVENTIONS FOR COST VARIABLES C: I-CAMP II INCIDENTS
D: QUESTION TEMPLATE
E: CATEGORIES AND INCIDENT TYPES USED BY DATABASE SCHOOLS F: INCIDENT FREQUENCY COUNT
EXECUTIVE SUMMARY
Information technology related incidents are occurring on college and university campuses. Some threaten the reliability or integrity of systems or data, operations, reputation, or resources and constitute a risk. To manage these risks senior managers need data to measure potential costs. The I-CAMP II study was designed to provide incident data.
The study, funded by the USENIX Association, had two major objectives. The first was to refine the costing model for calculating user costs for IT incidents. I-CAMP I showed that it was easy to calculate worker costs in IT-incidents. The number of hours - needed to resolve the incident was multiplied by the employee's hourly wage. When users are disrupted in using networks or other IT-resources, on which they are increasingly dependent, real costs also exist. This study improved the model for calculating user-side costs.
The study confirmed the usefulness of a common template for gathering data on IT-related incidents and for guiding the cost analysis process. It expanded the number and the geographical representation of participating schools in the study to eighteen.
It expanded the collection of cost-analyzed incidents by 15 including incidents of compromised access, harmful code, denial-of-service, hacker attacks, and copyright violations. For the 15 incidents, 90 employees were involved, 506 hours were devoted to resolution, and $59,000 in cost, was incurred.
The assumption that, though their frequency might be high, the costs of resolving these selected types of incidents would be low, was generally confirmed. The average cost for incidents of access compromise was $1,800, for harmful code was $980, for denial of service was $22,350, for hacker attacks was $2,100, and for copyright violations was $340.
The second major objective of the I-CAMP II study was to investigate the availability of incident frequency data and the incident categorization schemes. The study found that only 38% of the participating schools had incident databases. After analysis, the team concluded that colleges and universities are not currently equipped to understand the types of IT-related incidents that are occurring on their campuses. They are not currently able to identify the number or type of incidents or assess the level of organizational impact caused by IT-incidents because of the lack of robust database tools and insufficient staffing.
The study team analyzed the currently existing diverse incident categorization schemes. The category schemes reflected the specific roles of their creators, the individual institutional cultures, and organizational needs. Therefore, the frequency counts recorded in these disparate systems could not be statistically compared and aggregated across schools.
The team gathered expert estimates of the annual number of occurrences of mail bombs, system probes, and Warez sites. It concluded that expert estimates of incidents logged and handled annually were very similar to actual frequency counts for those same type incidents when compared to data from school databases. It concluded that school size did not appear to affect the level of expert estimates for any of the three types of incidents. They found that, in general, experts as a group, believe they are identifying and handling only about 28% of mailbombs occurring campus-wide-an average of 15 incidents per year, 12% of system probes-an average of 565 per year, and 28% of Warez sites-an average of 15 per year.
The study report provides nine specific recommendations for future study and practice. It provides a model for a comprehensive categorization system encompassing both operating system vulnerabilities and interpersonal and policy violations.
PREFACE
This final project report has two major parts; Part I covers Cost Analysis and Part II covers Frequency Analysis. In Part I we describe the problem of information technology incidents on campuses, and provide highlights from the initial Incident and Cost Analysis and Modeling Project (I-CAMP I) study. In the section of Part I, entitled Cost Analysis, we provide the new methodology for calculating user-side costs, analysis of fifteen new and specifically selected IT-related incidents and a guideline for cost analyzing IT-incidents in the academic environment. In Part II of the report we describes the investigation of incident frequencies, the state of existing incident databases, the results of expert frequency estimates, and provide a categorization scheme to assist administrators. The report concludes with a set of recommended best practices. The appendices of this report also provide valuable information. They include, among other information, the actual descriptions and analyses of incidents, the categories currently used by seven of the schools that have incident databases, and an explanation of the cost conventions used in this study.
INTRODUCTION
Information technology related incidents are occurring on college and university campuses. Incidents that threaten the reliability or integrity of systems or data, can constitute risks for the organization, risks to its operations, its reputation, its resources, and effect the trust that members of its community have in the organization. To manage these risks senior managers need to understand them, have sufficient data to measure their potential costs, and make informed management decisions. The I-CAMP 11 study is designed to help provide data regarding such information technology related incidents.
The Problem
The implementation and rapid evolution of information technology (IT) resources at colleges and universities have increased the number of security and risk management issues for institutions of higher education. Physical and electronic security processes, common to the mainframe environment, are often not suitable in the more distributed computing environment that exists today.
Personnel Skills and Knowledge
Individuals who handle these distributed services as system administrators have differing levels of sophistication regarding the technology, laws, and ethics governing data security. To guarantee a viable computing environment, colleges and universities are becoming aware that they must provide education and training for the system administrators who manage the environment. They must also ensure increased community awareness of key features of data protection and security, hardware maintenance, software compatibility, disaster planning and recovery, and basic security standards for configuring personal computing devices. Finally, they must take steps to ensure that administrators and senior management understand the legal, fiscal, and ethical implications of physical theft, infrastructure failure, and employee incompetence or inexperience and receive sufficient on-going data to manage responsibly the associated risks within these technology dependent and networked environments.
Unfavorable Trends
Recent news regarding denial-of-service attacks aimed at Internet sites and unauthorized access and modification of data at major government and commercial sites has raised awareness of potential security threats. Additional information from a recently completed Computer Security Institute survey showed that "90% of respondents (primarily large corporations and government agencies) had detected computer security breaches; 75% reported a variety of serious security breaches such as theft of proprietary information, financial fraud, system penetration from outsiders, denial of service attacks and so on; and 74% acknowledged financial losses due to computer breaches." ' This information is just beginning to cause the administrators within most organizations, profit or non-profit, to devote sufficient fiscal and human resources to security.
Time and Skill Required
Increased dependency on networks and technology brings with it a growing demand for the availability and reliability of information technology systems. Administrators realize that both time and skill are needed to keep systems and networks operating. "Availability" and "Reliability,"' concepts that have long been recognized as key components of security by security professionals, auditors, and risk managers are now being recognized by students, staff, and faculty as requiring resource commitment, and the time and attention of system administrators.
Time and skill are required to address known vulnerabilities in operating system and in various applications-to patch known operating system holes. Time and skill are also required to maintain a current and appropriate level of security knowledge. In the fast-changing technology environment, system administrators can not rely on the information they obtained even one year ago to do their current jobs with sufficient competency. The time of systems administrators is also required to provide security education to users and to set standards for ethical, legal, and appropriate use of resources.
Time and skill are needed to detect those vulnerabilities that are not readily obvious, to thoroughly understand who is accessing systems and data and whether they have authority to do so. This requires that system administrators gather systems log data, analyze existing patterns, and monitor access and use of resources. These important activities contribute valuable data to address the concerns of auditors, risk managers, and security professionals-data regarding the processes of identification, authentication, and authorization. Without robustness in these three features of security-"Identification", "Authentication", and "'Authorization"- and without knowledge of who is accessing and using networks and systems, no resources or data can be secure. Management Implications
Risk managers within organizations are finding it difficult to learn about the capabilities of technology at a pace fast enough to understand the implications of new and emerging applications. Risk managers, accustomed to thinking in terms of risks against which the organization can insure, find themselves behind innovation in the area of information technology. IT incidents involving risk, liability, and even significant financial loss are difficult to comprehend unless they are related to physical losses such as fire, flood, or theft-the more commonly known risks for management. Only recently have the large insurance companies begun to recognize that risks and losses within university information technology and networking areas may also require protection and insurance.
Like system administrators, risk managers find it difficult to convince senior management of the need for more attention to the management of IT risks and of the importance of systems security. Often it takes a major incident before sufficient attention is paid to security. The absence of data, the press of conflicting demands for fiscal resources, and an environment of rapid technological change can combine to create a climate in which administrators prefer not to hear about-more problems. As a result, a tendency can develop to underestimate the frequency with which incidents occur or to consider individual incident costs as insignificant when compared with the institution's overall expenses.
Such an approach, however, leaves an organization open to the possibility of serious financial liability. A single incident may cost only $2,000. If, however, that type of incident is repeated 60 times per month, then the costs to the organization are significantly increased to $120,000 per month or $1,440,000 per year-an amount that would be considered far from insignificant when compared with any institution's overall expenses. Determining the multiplier for specific types of information technology incidents requires data. We need to know about the nature of information technology incidents, about their different types, about the costs associated with each of the different types, and ultimately about the frequency of occurrence on each campus. Do these data exist?
I-CAMP STUDY OVERVIEW
In 1997, the first "Incident Cost Analysis and Modeling Project," I-CAMP, was funded by the Chief Information Officers of the CIC (Committee for Institutional Cooperation/Big 10) Universities. The objective of the study was to design a cost analysis model for IT-related incidents and to gather and analyze a sample of such incidents.
University of Michigan staff and graduate students developed the model. They gained access to information about such incidents through partnership with security professionals at each of the 13 CIC campuses. They also described the incidents and provided detailed cost analysis information about 30 such incidents.
No particular incident type was sought for this study. Rather, it was the goal to establish a mechanism for gathering information about any IT-related incidents occurring on the campuses and to collect cost data for the analysis. The study began identifying factors that appear to influence the occurrence of IT incidents and those that appear to affect the cost of incidents once they occur. For purposes of the first study, and extended to the present study (I-CAMP II), "incident" was defined as:
"Any event that takes place through, on, or constituting information technology resources requiring a staff member or administrator to investigate and/or take action to reestablish, maintain, or protect the resources, services, or data of the community or of its individual members."
In summary, the first I-CAMP study examined 30 IT-related incidents and researchers found that:
• 210 employees were involved in incident investigation/resolution,
• 9,078 employee hours were devoted to incident investigation/ resolution
• 270,805 computer /network users were affected by the incidents, and,
• Calculated costs for the 30 incidents exceeded $1,000,000.
Although a model had been developed and beginning data regarding the cost of IT-related incidents was made available through the I-CAMP I study, it was important to refine the cost analysis model, analyze additional incidents to ensure the usefulness of the model, and begin to collect data regarding incident frequencies to allow managers to evaluate organizational risks and costs. A second study was undertaken. This report, parts I and II, describe the work of this second study, ICAMP II.
PURPOSE OF THE I-CAMP II STUDY
The purpose of the study is threefold: First the study provides guidelines to cost analyze IT-incidents in the academic environment. Through the use of a template, IT personnel are able to identify true costs and follow a guide in analyzing them. This template is used to guide an interview process for the gathering of cost information in each incident.
Second, the study analyzes the status of the databases of the participating institutions and their categorization schemes for classifying incidents. It also begins the examination of the frequencies of occurrence for specific types of incidents in three different periods of time (periods of high, medium, and low academic activity.) Finally, the study provides a categorization scheme as a guide to encourage more incident data gathering and to encourage consistency in the classification process.
I-CAMP II METHODOLOGY
The I-CAMP II study, and this report, have two major parts as represented in Figure 1. (In this figure the circles rep resent data gathering efforts and the rectangles represent output from project activities.) Part I, the Cost
Analysis section, provides information about the gathering of 15 IT-incidents from the total participant pool-incidents of service interruption and copyright violation. The specific objectives for this section of the project were to increase the robustness of the cost-analysis model, to enhance the collection of IT-incidents, and to confirm the variables and factors that affect the cost and resolution of incidents.
Part II, the Frequency Analysis section, provides information from a subset of the total participant pool-those schools with incident databases. In this section of the project, data were gathered in three time periods, April, July, and October. To meet the objectives of: understanding database conditions and categorization schemes, and gaining knowledge of the frequencies for particular types of incidents, the project team completed in-depth interviews with participants and gathered expert estimates of incident occurrences from each of the campuses. As a result of this section, the study then developed and proposed a new categorization scheme for IT-related incidents within academic environments.
EXPANDING STUDY PARTICIPATION
In the initial I-CAMP study, existing expertise and cooperation from each of the participating campuses facilitated the study. These partners consisted primarily of the security and policy officers /professionals of the CIC schools. One of the problems that has existed in the past, as individuals have tried to understand the occurrence of IT-related incidents in profit and non-profit organizations, has been an unwillingness on the part of key personnel to share incident information. Their fear is that such incidents may reflect poorly either on the personnel within the organization or on the organizations themselves. In for-profit organizations, such incidents could have direct effects on a customer's trust and therefore on the profitability of the company. In all organizations where prestige and reputation is important, loss of trust can negatively impact the organization's success. Therefore, incident data historically has been difficult to obtain and study. Through the CIC partnerships, this was not the case for the I-CAMP study.
For I-CAMP II, we sought to retain the same partners as in the initial study. However, we determined that expanding the pool would be beneficial. This would provide more representative data by providing input from schools with different populations and experiences than the CIC schools. It would potentially increase the dissemination of results and increase the overall investment in the process of incident data analysis. We decided to include large West Coast, East Coast, and central states universities-universities that have had a history of information technology development and use.
The same CIC universities as in the initial I-CAMP study were encouraged, and agreed, to participate. These included:
• Indiana University,
• Michigan State University,
• Northwestern University,
• The Ohio State University,
• The Pennsylvania State University,
• Purdue University,
• The University of Chicago,
• University of Illinois at Chicago,
• University of Illinois at Urbana-Champaign,
• The University of Iowa
• The University of Michigan-Ann Arbor,
• University of Minnesota,
• The University of Wisconsin-Madison. Participation was expanded to include:
• Stanford University,
• The University of California, Berkeley,
• Cornell University,
• The University of Maryland, and,
PART I -COST ANALYSIS SECTION Purpose
The I-CAMP II study was designed to provide system administrators and others with additional information about which IT-related risks are of highest priority to address, about factors relating to their occurrence and costs, and about how best to manage certain risks. The USENIX Association funded the study.
The I-CAMP II study first sought to improve the methodology for cost-analyzing IT-related incidents. We wanted to learn more about particular types of incidents that were not included in the first study. We also wanted to create a classification scheme for incidents-a scheme that would help system administrators and others understand how to manage these incidents more effectively. Finally, we wanted to see if it was possible to gather frequency data regarding incident occurrences on campuses.
Expanding the Sample of Incidents
The first goal for I-CAMP II, noted above, was to enhance the existing model. The best way to determine which factors affect the cost and occurrence of incidents was to expand the sample of same-type incidents. The study team examined the original I-CAMP incidents and discussed the need for more data with IT security personnel from each of the participating schools. We determined that if IT-related incidents were put into two categories-those that were the result of purposeful acts and those that were the result of unwitting acts or accidents--it was the first category about which security professionals were most concerned. Systems security personnel indicated that they needed more data regarding the costs of service interrupts and copyright violations. They believed that while these incidents may be small in cost, these incidents are occurring with high and growing frequency on campuses. The aggregate costs of these types of incidents may be significant. Project staff continued also to be interested in data and identity theft. However, little information is currently available regarding such incidents. While it appears that a combination of data stewards, such as university registrars or directors of personnel, and law enforcement/campus safety personnel, are in the best position to learn about incidents of this type, it also appears that incidents of this type are only starting to be recognized on college and university campuses. Because of the paucity of information, I-CAMP II did not include this type of incident in the study.
The I-CAMP II study gathered and cost analyzed data regarding Purposeful/ Malicious behaviors of two types: 1) Service Interruptions- specifically, compromised access, insertion of harmful code, and denial of service, and 2) Copyright Violations- specifically, distribution of MP3 and Warez distribution of illegal software. The study goal was to augment the first sample of incidents (N=30) by adding the analysis of a small sample of these specific type incidents (N=15). (15 of these were analyzed and appear in Appendix C of this report). The incident categories collected in I-CAMP II appear in Table I.
Providing Comparison Data
The current study sought to "gather more same-type incidents to facilitate analysis and comparisons and refine the cost analysis model." It was determined that within the scope and time allowed for this study it would not be possible to gather a large enough number of same-type incidents to provide a statistically significant sample (40-50) for comparisons. The project Advisory Board recommended that instead, we narrow the study--collect and analyze three of each type of the targeted incidents if possible. Three same-type incidents would allow the project team to begin comparing costs and actions that were necessary to resolve the incidents. Two or more of each of the specific incident types shown in Table I were collected.
Procedure for Incident Identification
The system administrators of the participating schools identified incidents. As in the initial study, a specific process was used to gain access to campus information. Authorization came directly from campus Chief Information Officers (CIO's) who identified the key staff members from whom data should initially be sought and who relayed those names to the study team. Participating personnel in each of the 18 universities were then asked to identify incidents of service interrupt and copyright violations. Specifically, they were asked to inform the I-CAMP II team of access compromises, insertion of harmful code such as NetBus, Back Orifice, and others, denial of service attacks such as mail bombs, ping attacks, smurf attacks and others, and incidents involving distribution of MP3 and Warez. (See Glossary for definition of terms.)
Procedure for Incident Cost Analysis
Once an incident was identified, data gathering was accomplished in person through a visit to the campus, or by telephone call using a questionnaire template (see Appendix A). Often an incident required follow-up activities
to clarify particular aspects of the event, gather a piece of needed data, or ask a question when some aspect of the event was omitted. Additional details were exchanged using electronic mail or file transfer. (The standard template for cost analyzing IT-incidents appears in Appendix A.)
Assumptions
This section of the report details the assumptions and methods used in gathering data and the manner in which cost variables were treated. This information should be used as a guide for understanding the subsequent incident analyses. (Refer to Appendix B for a detailed description of the conventions used in calculating costs.)
Assumption I - Truthful Information
We assumed that the information that we received from the people directly involved in an incident was truthful to the best of their knowledge. Other than an occasional log of employee actions, we depended primarily on the person's best recollection of events. We attempted to gather data as close to the incident's occurrence as possible to minimize data loss due to memory lapses. While some measure of error exists when recalling past events, we have no grounds for disputing the information conveyed. If an incident was too old to gather valid data, it was not included in the study. All incident data collected were subjected to a final review by the provider of the data prior to inclusion in this report.
Assumption 2 - Appropriate Data Suppliers
We assumed that the individuals identified to provide data about incidents were appropriate and valuable for the purposes of this study. Within each participating school, others who had some involvement with a particular incident were identified to us for purposes of more complete data gathering. We recognized, however, that as a result of their association with the information technology organizations of these colleges and universities, they would identify incidents of one type more often than might individuals in non-technology departments.
Assumption 3 - User-Side Costs
Regarding the costs on the user side of the equation, we assumed that:
a) The tuition fee is the basis for the calculation using this methodology. This fee includes all the academic resources that the university offers to the student: libraries, professors, rooms, places to study, networked services, computer rooms, restrooms, etc. It is too difficult to separate which part of the overall fee corresponds to the particular service offered.
b) For each credit hour, the student incurs three "study hours." The total number of study hours includes class time plus required preparation. For example, a 3- course entails 3 hours in the classroom plus 9 hours of additional study and hence, a total of 12 hours of study per week. Note: This calculation, three times the number of in-class hours for each credit is an accepted standard device for estimating preparation time in most universities.
c) The fee/ student cost used for calculations depends on the number of in-state and out-of-state students in a particular university. From these numbers we can calculate an average student cost. To illustrate this ideal, we took the weighted student cost for a semester for one of the participating schools and we calculated, $17,802.00. d) It is "virtually impossible to speak to every affected user to determine her or his real loss in dollars. We expect that when the network manager is learning about the resolution of an incident, she or he can ask the users involved questions about how much time the user lost as a result of the IT-incident. Alternatively, the network engineer can estimate the traffic of users connected to the computer network at a specific period of time. These estimations can help to identify (either by increasing or decreasing) the number of users affected when there is
an incident that affects the entire community (e.g., a probe that results in a massive denial-of-service for members of the campus.)
We do not estimate any loss of hours due to an IT-incident. The following methodology (see "I-CAMP II Methodology for Calculating User-Side Costs") for the user side is based on the concept of the willingness to pay and the individual opportunity cost. Both methodologies will be used in instances where the user cannot perform an alternative activity; e.g., if the paper that the student was writing was stored on a hard drive that is compromised and cannot be obtained to be turned in on time.
When quantifying an incident, we do so from a university's perspective. However, we made the assumption that the term "university" implies the entire community of students, faculty, and staff. (Some feel that if the cost of an incident is not a direct cost to a university department it is not a cost to the university. This is not the position taken here.) Thus, costs borne by students, for example, from the inability to complete work as a result of a server crash, are considered a real cost to the university and estimated for reporting even though the university may not directly pay out resources. Generally, any quantifiable cost borne by any member of the community as a result of an incident, if we are able to estimate it reasonably, is included in our calculations. Otherwise, the real costs and their implications are described qualitatively in the incident report.
There were those who, when reading the first I-CAMP study results, commented that costs associated with system administrators who repair computers or networks as a result of unauthorized intrusions, for example, should not be included because they were already assumed as part of their salaries and their expected work. We might assume that if a system administrator spent all of his/her time managing IT-related incidents, there would be a real cost to the institution. The system administrator would be totally unavailable to perform the other duties associated with their position such as configuring machines, trouble-shooting network and system problems, supporting users, and so on. Likewise, we might assume that if no one spent time and effort dealing with the IT-related incidents that are occurring, there would be a real cost to the institution through unmanaged risk, and potential liability. Depending on the frequency and impact of the incident and between total and no response lies the real cost to the institution. For purpose of this study, we assumed that employee effort to detect and manage IT-related incidents should be considered a real cost to the institution, and therefore be included in the analysis.
In I-CAMP I, user costs for faculty and staff were calculated on an hourly basis from their stated salaries. In that study, student costs, where the number of students affected by an incident was known, were calculated according to an average part-time wage commensurate with undergraduate employment wages and graduate employment wages. It was assumed that students would have been working if they had not been interrupted by the downtime caused by the incident.
Assumption 4 - Limiting Incident Scope
For the purposes of this study, we concluded our cost analysis when the network, system, LAN, PC, or other environment was returned to its pre-incident condition. The decision of when to close the quantification of an incident is debatable and, in some sense, arbitrary. Often it involves a judgment in terms of natural closure-that is, a judgment of when the incident really ended. In general, we attempted to capture the essence of an incident without carrying it out too far. Therefore, if a security audit or review was stimulated by the event and was performed after the problem had been resolved or the hole closed, it was not included in the cost analysis. For the purposes of this study, we did not consider these additional events and their concomitant costs to be directly related to the cost analysis of the incident.
Assumption 5 - Common Costs Excluded
There are specific variables common to all incidents that we did not attempt to quantify unless they presented themselves as inordinately large in proportion to and specifically related to the overall incident. Generally, these variables did not provide any clearer sense of the situation, but would have required a significant commitment to data gathering. Included in this category are office supply costs (such as paper and pens), telephone bills, and costs of secretarial support to the individuals involved.
Assumption 6 - Study Team Cost Excluded
We did not include the time spent by the researchers of this project as part of the overall costs to a university. Under normal circumstances, an incident would not include an investigation by a separate party; thus, we did not want to skew the results of an incident analysis by including the commitment of the project team.
I-CAMP II Methodology for Calculating User-Side Costs
Refining and Increasing the Robustness of the Cost Analysis Model
We felt that it was important to focus a fair amount of attention to refining this part of the cost analysis model because students and their time, as well as faculty and staff, are the users of these information technology systems. Students are the customers of colleges and universities. Even though their losses in productivity do not reflect directly on the budget expenditures of the organization, such losses reflect on and affect perceptions of the value of the educational experience at a college or university, and the satisfaction of the students -therefore its reputation.
In calculating the costs of an IT-related incident, there are actual costs related to actions that need to be taken to manage the incident and return the environment to its original state. These actions may include incident investigation, patching identified vulnerabilities, repairing or replacing systems or applications, managing publicity, supporting and informing users, and so on. There are also costs related to the effects of the incident on users.
The original I-CAMP model underscored the difficulty in estimating the costs to users when an IT-related incident occurs. The authors of the first report specifically identified three areas of difficulty:
"First, it is virtually impossible to speak to every affected user to determine his or her real loss in dollars. Second, we cannot say for certain what the user's time is worth. Wage rates are traditional measures of a person's time, but it is difficult to put a number on, for instance, a student who is not employed. Third, opportunity costs are always involved. If a person cannot retrieve needed information from the network, he or she may be able to do some alternative activity that provides some utility. Measuring the difference between the real loss and gain in utility from an alternative activity is a difficult task at best.,, 2
A revised model for calculating the student user-side costs was developed for the ICAMP II project. This model is based on -the theory of willingness to pay as an approach to the concept of an allied market. It provides what we consider to be an improved methodology for calculating the user cost of IT-related incidents. Although the cost of the user side is not a direct cost for the university, it will have an important relevance when it is included as a shadow price.
The I-CAMP II team learned that there is no single methodology for cost analyzing the user side of an incident. Cost varies according to the type of incident. The two incident types we gathered for I-CAMP II (service interruptions and copyright violations) have two potentially different cost approximations: a) the marginal cost
to access the network, and b) the willingness to pay for one hour of study. These different cost
approximations-this methodology for examining costs-is based on the work of economist Edward Gramlich's concept of allied markets. He states:
"... Many times in a benefit-cost study there will be changes in quantities for which there is no market...' Further, "...the simplest answer is to -try to find some allied market where the price or quantity change can be used to infer valuations for the missing market". "
Methodologies for calculating User-Side Costs
Marginal cost to access the network
If an IT-incident denies access to a university's network, the user could pay for a connection to another service provider. In such cases, our proposed methodology uses the marginal average cost of the connection to another server times the hours the user spent connected to it (or, when appropriate, the fixed cost). The marginal average cost of the connection to another server can be obtained from Internet Service Providers. It is the simple average cost of one hour of connection time.
Willingness to pay for one hour of study
If the user loses worktime because he or she cannot access the hard drive of the computer that has been compromised or shut down, the cost analysis methodology depends on the type of user. If the user is a professor or a staff member, the analysis technique will be the individual opportunity cost, approximated by using the hourly wage rate. If, on the other hand, the user is a student, the methodology is the willingness to pay for one hour of study.
Based on the concept of allied markets, the methodology of willingness to pay can be used to derive missing values that previously were roughly estimated. The willingness to pay is a good alternative methodology. As shown in Figure 2, we derive marginal income from the consumer demand curve. It is interpreted as the willingness to pay where consumers, at quantity Q, are willing to pay just P for the last unit of product, but no more. Thus, the price that consumers are willing to pay exactly equals the marginal utility for the good they receive. Therefore, at the economically efficient allocation point, marginal income equals marginal cost (the willingness to pay is equal to the opportunity cost). But when there is an imbalance, markets suffer from irrationality problems. Such discontinuity makes the willingness to pay different from the opportunity cost. If this occurs, we must ask how to approach the imperfect market-from the demand side (the willingness to pay) or from the supply side (the opportunity cost).
The willingness to pay is a direct measurement obtained through an estimation of demand. That is, the amount of money the consumer is willing to pay for one additional unit of good. Conversely, the opportunity cost is an indirect measure addressed by the offer. In other words, the market states a price for the last unit of good, without differentiating if the price is correctly allocated.
To illustrate this concept more clearly, let's assume that in the unskilled labor market, students are offered a job. The unskilled labor worker is defined as the market where people having a high school education or less are offered a job. Students fall under this category because employers do not differentiate between a student and an unskilled worker until the students have received their college degrees.' When a student is willing to work while he/she attends school the opportunity cost is an hourly salary rate of $6.00. As can be seen on the left side of Figure II, the unskilled labor market represents the opportunity cost.
Conversely, as shown on the right side of Figure 2, we assume a "study market" denoted by a fixed supply curve representing the university's student cost (or tuition fee)" and a demand curve for study (without differentiating quality or quantity). We also assume that students are rational agents that will search for a university that best matches their preferences, depending on costs of tuition fee and living, quality of the degree they are pursuing and services offered by the university. Under such assumptions, the willingness to pay for one hour of study is equal to $15.00, the cost of a marginal hour of study (based on an average weighted student cost of $10,000.)
What is "one hour of study"? To study in an academic environment requires interaction with the computer. Thus, "one hour of study" would likely involve computers and networks. Using e-mail for communication between the school and the student, or between students, has become the norm. Students use computers to write essays and articles, work with data, work in the laboratories, and so on. The computer network is an essential part of the academic experience.
Method of calculation
As we showed early in the assumptions, on average, students take 4 classes per term with 3 credits for each class. Each class entails 12 hours of study per week (3 hours in class plus 9 additional hours of study). Therefore, in a month, a student is required to study 48 hours per week, or the equivalent of 192 hours per month. On average a semester lasts 3.5 months. Taking the student cost of X dollars and dividing it by the total hours per month times the duration of the semester, we obtained the student's willingness to pay for one hour of study, or the marginal hour of study equal to $15.00.
In summary, when the user is a student' and he/she is involved in an IT-related incident that causes a loss of time, the cost for the user's time should be $15.00 per hour, the weighted average of in- and out-of-state student cost plus fees. If the user is a . professor, the cost should be calculated at his/her hourly wage rate.
In reality, each university has different student costs. When applying this methodology, each university will have to calculate its own weighted average. For example, for the calculations of the user cost in the IT-incidents we cost analyzed, use the respective tuition fee (out-of-state vs. in-state) and student enrollment figures from each respective university. We applied this methodology in the incidents entitled, "Experts Lying", "Jumping Hacker", "'Possessed Mouse", and "Post Fourth of July", since these incidents involved specific user costs. (Table III provides a summary of results.)
To augment the incident sample, a total of 15 incidents of the two types-service interruptions and copyright violations- was gathered and cost analyzed. Within the service interruption category, the sample contains three examples of compromised accounts, three of hacker attacks, three of harmful code insertion, and three denial-of-service attacks. Within the copyright violation category, the sample contains three MP3 incidents.
Table III - Summary of Results from the Incident Cost Analysis
Examples of five types of selected incidents were collected, described, and cost analyzed. In these 15 incidents, we found the following:
• 90 employees were involved in incident investigation and resolution.
• 506 employee hours were devoted to incident investigation and/or resolution.
• The estimated number of computer and network users who were affected by the incidents was not available.
• Calculated costs for the 15 incidents totaled $59,250.
• The average calculated cost for the (2) compromises of access incidents was $1,800.
• The average calculated cost for the (3) harmful code incidents was $980.
• The average calculated cost for the (2) denial-of-service incidents was $22,350.
• The average calculated cost for the (3) hacker attacks incidents was $2,100.
• The average calculated cost for the (5) copyright violations incidents was $340.
At first review, the cost figures for these incidents appear so small as to be entirely insignificant. However, it is important to remember two key points:
• We purposefully solicited these types of incidents because security and policy experts in the
participating schools perceived that the frequency of occurrence of these types of incidents was high or rising, therefore, the overall costs to the organization may indeed be significant.
• We have used the most conservative figures for calculating costs in all cases. For these types of incidents it is extremely difficult to understand user costs because knowing the number of users that were actually affected in a denial-of-service attack, for instance, is impossible. The average costs are small and they provide only the beginning insights into the overall costs of these incidents for a campus.
In a future study we would suggest obtaining information about the number of users who were
connected to systems at the time of an incident and using that number to begin estimating user impact figures.
Recommendations Regarding Cost Analyzing IT-Incidents in Academic Environments Questionnaire Template
Once an IT-incident is reported to staff member in charge of handling IT-incidents, the staff member should try to gather as much information as possible about the incident. The template shown in Appendix A provides a useful guideline to direct this process of information gathering.
With this template the staff member can easily identify the relevant costs that affect the incidents: number of machines involved, number of users affected, and staff hours devoted to handling the incident among others. Calculation Template
Depending on the type of user affected in the incident, the staff member, following the cost data template in Appendix A, can decide which calculation is appropriate for the user costs. E.g., If the user is a faculty or staff member, the model suggests the hourly wage rate be used as the "opportunity cost". If the user is a student, the model suggests a weighted average of in- and out-state student costs as the "willingness to pay". In addition, the staff members must remember to calculate their own hourly wage rate times the number of hours spent handling the incident.
Machine cost
The staff member must add to the staff and user costs, the cost of any hardware or software that is damaged or lost during the incident thereby calculating the total cost of the incident.
Reports
Finally, it is recommended that the staff member keep a brief description of the response to and resolution of the incident.
Part II - FREQUENCY SECTION Database Analysis and Categorizations
The purpose of Part II of the study is first to understand the database condition and the categorization schemes of the participating schools and to begin to calculate the frequency of occurrence for particular types of incidents.
To address the above objectives, I-CAMP II designed a methodology to gather frequency data from the schools that had databases. During the process, the team found that the participating schools had difficulty delivering the database information in response to the study request. Therefore, the team proceeded to interview the participants to learn more about their database conditions. The results are discussed and reviewed in this section of the report. In addition, the team gathered additional information from experts at each of the participating schools in an attempt to gain estimates of incident frequencies.
Finally, I-CAMP II reviewed literature on incident categorization and together with the category schemes from the participating schools, the team developed a scheme that IT-personnel can use as a guide in categorizing incidents on their individual campuses.
Designing a Methodology and Gathering Frequency Data
The I-CAMP II team anticipated that Part II of the study would be relatively straightforward. The plan was to identify participating schools that maintained incident databases, arrange a method for them to transport the full database of incidents (stripped of individual identifying information) to the team three times during the project period (April, July, and October), and then analyze the frequency results. Anticipating that there would be some variance in the way the data were collected, we attempted to standardize the data we received by asking each school to provide the following information for each incident in their database:
• Name and type of the IT-related incident;
• Number of people involved in the incident resolution;
• Number of machines compromised; and,
• Dates the incident investigation /resolution began and ended. Availability of Databases
We began by interviewing the contact persons at each of the 18 participating schools to identify those who maintained incident databases. Several important conclusions were drawn from these interviews. To our surprise, only 38% (7 of 18 participating schools) maintained any form of incident database.
Representatives from a number of the schools expressed a desire to have a sophisticated incident database but indicated that they had neither the time nor resources to build and maintain it. Two of the schools had begun building such a database, however, they had so few incidents recorded at the time of this study that the data would not prove useful. There was a genuine interest in participating and a support for the effort of analyzing incident data. Everyone we interviewed indicated that they felt such data, if collected, would be valuable to their school.
Aggregated Data
Individuals from many of the schools indicated that the distributed nature of their computing environments made it difficult, if not impossible, to aggregate incidents from across their campuses. Most of the
representatives felt that there were many incidents going on that were unnoticed; some, if identified, were minimally managed or ignored altogether. Some were investigated and managed within individual departments, but not reported to a central database.
Within the seven schools that have incident databases, each made an effort to aggregate campus incident data into their database, but with varying degrees of success. Some schools relied on their departments to self-report the incidents to a database manager. Some systematically and routinely collected incident data from the two or three key areas on campus that they knew had such data. Two representatives indicated that they had established, with the support of campus executive officers, a campus-wide procedure for reporting incidents to the central database and felt confident that the database represented a comprehensive picture of the incidents on campus. They, however, were less sure about regional and medical campus reporting. For 100% (7 of 7 of the respondents), the comprehensiveness of the database information, its inclusiveness, was a problem. This inability to aggregate incidents and to see the full picture of IT-related incident activity on the campus, for all respondents, made it impossible to construct a true picture of the frequency of different types of incidents on any one campus.
The I-CAMP team initiated a data-gathering procedure with the seven schools that maintained incident databases.' For each school it was necessary to design a process for transporting the data to the study team. Data collection was scheduled for April, July',, and October of the project period. We expected that these dates would generally provide representation from an end-of-semester period, a low-enrollment/activity period, and an early-in-semester period.
Problems in Data Collection
As attempts were made to collect the database information in the first two periods, April and July, it appeared that our requests for the data were causing considerable difficulty for the participating schools. April data were not received until well into the summer; July data, well into the fall. We began to realize, due to the genuinely expressed willingness to participate on the part of the school representatives, yet the significant delay in delivering data, that what we were asking of them must be more difficult than seemed reasonable to us.
To determine why the process was so difficult, in-depth interviews were performed with each of the database school representatives. (The interview template to ensure comparable information from representatives appears in Appendix D.)
Summarization of Interview Results
We. asked what types of difficulties respondents were having meeting our request for data. Four basic problems were reported.
Too few or changing personnel
For 43% (3 of 7 respondents), new personnel or changes in personnel resulted in confusions or discontinuities in work processes and therefore, in fulfilling the requests of the I-CAMP II study. 57% (4 of 7 respondents) reported that they did not have enough staff members to maintain their logs or input data to the databases in a timely fashion and therefore, fulfilling the I-CAMP II study request was an additional burden on an already taxed staff.
It is important to note that "'lack of continuity in staffing and responsibilities" was one of the factors identified in the initial I-CAMP study that was felt to contribute to the increased costs of incidents when they occurred.
The authors wrote: "Incidents affected by this cost factor were characterized by high turnover and lack of clarity and continuity in passing responsibilities for system management functions from one employee to another, resulting in lost documentation and missed or poorly executed procedures."
It is interesting that this same factor caused problems with the collection, management, and maintenance of data about incidents in the I-CAMP II study. Limited or fast-changing human resources appear to be detracting from the ease with which we manage and understand incident data, and most certainly from the efficiency of managing the incidents themselves.
Confusion caused by the I-CAMP II request
For 14% (1 of 7 school representatives), the I-CAMP II staff were unclear in their request for data, causing confusion. For another, confusion resulted from the ICAMP II request being more comprehensive than what they had readily available in their database. The study team did not know that what they were requesting was simply not available in most of the databases retained by the schools or would require special calculations in order to report. The entire area of categorization of incidents, the diversity of the present category systems at the participating schools, and the need for a more common and mutually useful classifying, counting, and reporting incidents will be discussed later in this report.
Problems inputting data to the databases
Many of the respondents had difficulty due to the nature of their data input processes. Forty-three percent (3 of 7) reported that they manually enter data in the database or have to manually classify the data from e-mail messages or flat files. Twenty-nine percent (2 of 7) reported that they had to aggregate data manually from two or three individual sources on their campuses where incidents were recorded and that the different reporting mechanisms did not merge the information easily. For some, different people input the data and do so at different levels of specificity, perhaps also differing in their judgment as to how to classify a particular incident. For 71% (5 of 7 respondents), the greatest difficulty was that because of limited resources, the databases were not kept up to date and therefore, data had to be entered prior to their being able to respond to the I-CAMP II request.
Limited functionality of the log
Since logs contained information about student activities for many of the incidents, all of the school respondents were sensitive to privacy restrictions on the data. Access in nearly all cases was limited to three part- or full-time employees and was password protected to control access to only those authorized for investigation and handling of the incidents. In three of the seven schools, student-employees were involved in the investigation and resolution of incidents. The databases were not designed, however, to facilitate the sorting of the information by different fields, and therefore to automatically eliminate the personal identifier information from the aggregated records. Therefore, to respond to the I-CAMP II request, most of the respondents, to protect the confidentiality of the records, had to strip out the personally identifiable information manually before sending the information to I-CAMP II.
One -hundred percent (7 of 7 of the respondents) had some difficulty responding to the I-CAMP II request because their databases either were not up to date, could not sort by incident type, or did not have information on the other variables asked for by the study. For many of the respondents, the database tool they used did not allow an automatic classification of incidents as they were entered or a sorting function; they functioned more
like spreadsheets than interactive databases. They were designed to be useful as recording tools not as reporting tools.
One hundred percent (7 of 7) of the respondents indicated that they wanted an interactive logging and sorting tool that generate periodic reports for CIOs, senior management and others. None provided such reports on a regular basis at the present time, though some, with great effort, were pulling data from the logs and reporting occasionally. Respondents want the database tool to be able to aggregate data on an incident from multiple sources such as machine logs, trouble ticket, and police reports, and provide automatic incident entry and classification. Trend analysis and incident tracking were the two greatest needs.
Design and purpose of the existing database schemata
The fact that so few schools in the sample had a working database of incident information was surprising. However, our suspicions that many schools were struggling without the needed tool and with too few resources and personnel time to maintain this information source were confirmed during our interviews. The importance of this information resource was also confirmed during our interviews. Our next task then was to examine what existed, the purpose for which the category systems had been devised, and the roles of the people who had designed the databases in our participating schools. We wanted to understand the rationale behind the various categorization schemes. The existing database function as management tools rather than search or aggregating tools because they were designed to do just that-to capture information in one place and ensure that someone was assigned to handle each incident. Figure 3 provides information about role and background of the database creators and purpose of the tools.
In summary, the I-CAMP II requests for "regular data transfers of incident data from each of the participating schools" were very difficult to fulfill. The requests were difficult because of limited personnel, limited functions on the existing databases, difficulty maintaining the timeliness of the data, and non-automatic (or no)
aggregation of incident data from multiple sources on the campuses. Status of School Categorizations
Analysis of data confirmed that there was a wide and varied set of categorization schemes used among the database schools. (Appendix E provides a listing of the different category and incident types recorded by the database schools.)
Fairness and justice in Incident Handling
When an incident occurs and a suspect is identified, colleges and universities today too often judge cases and impose penalties on an almost ad hoc basis. Record keeping and human memories vary widely. Each incident stands nearly alone. Little can be done to reconcile actions with previous judgments or to create a written record for future judgments.
The quality of incident-justice may suffer accordingly. Arbitrary and capricious decisions, even naked exercises of power, become possible in such an environment. Fairness and consistency are more likely to occur by accident than by design. Any sense of legitimacy becomes elusive at best. Guidance for the future fares no better. If judgments are not collected and published, they can serve neither as deterrent cautionary tales nor as guidelines to permissible behavior. If decision-makers have no obligation to publish reasons for their judgments, then no tested body of principles can emerge, leaving prudent information technology users no choice but to limit their behaviors excessively so as to avoid being charged with a violation.
Carey Heckman writes:
"Evolution of just and efficient information technology regulation depends on creation of a system for constantly accumulating an accessible body of incident judgments. Oliver Wendell Holmes observed that "the life of the law has not been logic: it has been experience." Holmes rejected conceiving law as the product of abstract deduction from on high. He wrote about law as an organic product reflecting a community's evolving common practice in light of its evolving common values. The nature and pace of change in the information technology context makes particularly essential a process that produces an adaptive, constantly improving regulatory regime."
Similarities in Incident Categories
Each of the category systems used by the participating database schools served a functional role for that school. While representatives indicated that they would like to have the capability of reporting from their database data, the tools were not designed to allow sorting by incident type, or to provide aggregate reports from all sources on campus by incident type, or by date, or by whatever other variable might be sought.
Inspection of the incident categories across schools showed many commonalties in the titles that they gave to particular types of incidents. For example, 75% identified harassment as one of the categories in classifying incidents; 71% identified copyright violations; 71% categorized intrusions; 75% identified Spam as one of the categories in classifying incidents; and 71% identified various forms of abuse and misuse of systems as an
incident type. Though these commonalties in titles exist, we cannot be certain how the categories were understood and how reliably they were used by each of the participants during their classification of incidents Most striking, beyond the commonalties in the category systems, is the fact that they so clearly reflect the area of specialization of the incident database builder. Those builders, whose primary responsibilities lie in the area of system security, over time, have constructed incident databases that are much more robust in classifying intrusion-type incidents. Those builders with responsibility for handling misbehaviors or abuses of systems by students have constructed incident databases that classify abuse and policy violation-type incidents. But how consistently are these database categories being used in any of the schools that have them? While this question cannot be answered in this, study, we recommend a rater reliability measure in future work of this type.
Factors Appear Again-Lack of Knowledge/Information, Lack of Resources
The initial I-CAMP study team identified several f actors that they believed contributed to increases in costs when incidents occurred. Among the factors identified was lack of knowledge. The authors wrote:
"This factor affects costs when the personnel do not know how to handle, investigate, or manage an incident. Such incidents involve lack of direction, planning, and procedures."
Without an archive or database of incidents- an institutional "memory"- of how previous incidents of a particular type were handled, what was learned, what worked, what did not, what policies and laws were relevant, what procedures to follow, a "lack of knowledge" exists. Those who are trying to handle the incidents on a day-by-day basis are left without guidance for consistent, expeditious, and fair management of the incidents. Again, the factor of "lack of knowledge"' appears to be contributing to inefficiencies and, perhaps then to incident management costs.
The initial I-CAMP study team also identified lack of resources as a factor contributing to cost. They wrote: "The lack of human, physical, or fiscal resources needed to resolve an incident can contribute to both the occurrence and the cost of an incident, but they are distinct effects. For example, if an incident needs 20 employees to resolve it efficiently, yet the department can afford to pay only five employees, then the incident will take longer to resolve and the costs will rise."
Many of the schools in the current sample appeared not to have sufficient resources, human or fiscal, to develop and maintain an incident database or to manage incidents in the manner they felt was needed. Again, the factor of "lack of resources" appears to be contributing to inefficiencies and ultimately, the cost of data gathering, organization, and incident management.
Several other factors were thought to contribute to incident costs in the initial I-CAMP study. It was thought that these factors might be eliminated through the existence of a well-designed, maintained, and used database-a database that provided needed information about past related or similar incidents and their handling. But such databases do not yet exist in practice. They require a comprehensive, well designed, database may require the contributions of both those in roles that concern themselves with security features of operating systems and networks and those who focus on policy violations and interpersonal conflicts played out in the electronic environment.
Frequencies from Database Schools
We found that the category systems used by the different participating database schools varied significantly and that, while there were commonalties in some of the category titles that were used, we had no assurance that what one school called an intrusion, for instance, was the same as what another labeled such an incident.
