Characterizing violations in computer and information security systems

Characterizing violations in computer and information security systems

S. Kraemer

, P. Carayon

, J.F. Clem

a

Department of Industrial and Systems Engineering, Center for Quality and Productivity Improvement,University of Wisconsin-Madison, 610 Walnut Street 575 WARF, Madison, WI 53726, USA

b

Sandia National Laboratories, PO Box 5800, MS 0671, Albuquerque, NM 87185, USA

Abstract

This paper describes the various types of deliberate violations and associated mechanisms committed by computer and information security (CIS) network administrators and end users. As the basis for this analysis, we used Reason, Parker, and Free’s [1] classification for deliberate violations: routine, optimizing, and situational. Interviews with 14 red team members from Sandia National Laboratories’ Information Design Assurance Red Team™ program were conducted to characterize types of CIS violations and related work system factors. The interviews yielded a preliminary classification scheme. Both end users and network administrators perform routine violations, such as creating “weak” passwords. The red team members did not comment on optimizing or situational violations. Further, the red team members commented on a number of factors that contribute to performing CIS violations, such as high workload or lack of CIS training. The implications of this preliminary research points toward some early recommendations to support CIS end users and network administrators, however, more research is needed to create a comprehensive taxonomy of CIS violations.

Keywords: computer, information, security, violations, red team

1. Introduction

Computer and information security (CIS) may be typified by technology-centric systems. CIS systems may become vulnerable for a number of reasons, some of which may be a result of actions of the designers and users of the computer systems. CIS vulnerabilities and security breaches are often remedied by implementing a new technical control mechanism (e.g., stronger firewalls or encryption methods), often without consideration of how the end users, network administrators, and CIS managers will interact with the CIS system. The role of the ‘legitimate users’ are of critical importance because their actions are directly involved with performance of CIS, in fact, legitimate user behavior may be considered the ‘weakest link’ of the CIS systems [2, 3]. Often their role is not given as much consideration because the behaviors of legitimate users are often overshadowed by malicious insiders

or outside attackers [4, 5]. For these reasons, the legitimate CIS users are often forgotten in CIS design, implementation, or maintenance. In the paper, we explore the role of non-malicious

violations committed by CIS end users and network

administrators, their related work system factors, and the potential detrimental effects on CIS performance.

2. Background

2.1. Violations in safety

The cognitive and social sciences have developed various taxonomies for the classification of human error in safety [6, 7]. Human error is the failure of planned actions to achieve their intended consequences [9]. A closely related concept is

violations, or deliberate (though not necessarily

believed to maintain safe or secure operations [10]. Violations can only be described in their social context, where behavior is governed by operating procedures, codes of practice, rules, regulations, etc… [10]. Further other elements in the work environment may propagate the occurrence of violations, such as time pressure or a large number of tasks to perform.

Reason, Parker, and Free offer three major categories of deliberate violations: routine, optimizing, and situational [1]. Routine violations are “corner-cutting” or short cuts, optimizing violations reflect actions unrelated to the functionality of the task (e.g. job of high speed driving), and situational violations are seen as essential to get the job done in a particular work organization. Reason and colleagues’ [1] categorization of deliberate violations (i.e. routine, optimizing, and situational) is the basis for the work presented in this paper.

2.1 Violations in computer and information security

Besnard and Arief [4] have described users’ “trade-offs” to commit violations from a cognitive approach. They propose that user decisions regarding security in the workplace can be interpreted in terms of an intuitive cost-benefit trade-off. For example, logins and passwords require the user to recall and use. Given the mandated complexity of passwords and the number of passwords to recall, users may use storage features (i.e. cookies) or write them down, given their judgment of the importance of the data protected.

Stanton and colleagues [5] further explored this area with an empirical analysis of end user security behaviors. They created a two-factor, six category taxonomy of intentionality (i.e. malicious, neutral, beneficial) and technical expertise (i.e. high, low) by interviewing 110 IT professionals, managers, and regular employees. In this taxonomy, violations exist on the high/neutral or low/neutral dimensions, where behaviors have no clear intentions to do harm to the organization’s IT or resources. An example of a violation from someone with a high level of expertise and neutral intentions would be a network administrator configuring a wireless access gateway that inadvertently allows wireless access to the company’s network by people passing in cars. An example of violation from someone with a low level of expertise and neutral intentions would be an end user choosing a weak password that is easy to remember, such as their name. Stanton and colleagues [5] also conducted a follow-up survey study of 1167 end users in the financial, manufacturing, health, military, government, and

telecommunications sectors on password-related behaviors (e.g., frequency of changing the password, sharing passwords with others) as well as training and organizational awareness (e.g., organization provides training programs to promote CIS awareness). They found significant correlations between good password-related behaviors and training and awareness.

Although these expert opinions and studies observe the role of violations by CIS end users, network administrators, and managers, they have not fully taken into account how the work system or work environment contributes to the behavior or decisions. For example, network administrators may have to work with and satisfy many user groups. Kraemer and Carayon [11] studied how various work system elements propagate human error and violations and consequently, CIS vulnerabilities. They interviewed 10 network administrators and 10 CIS managers to obtain descriptions of the types of violations committed by end users and network administrators. For example, network administrators intentionally reconfigure firewalls in ways that may introduce vulnerabilities, so that outsiders who are collaborating on projects can have access to their networks. They may do this because of the lack of set procedures or rules that have been agreed upon by the user groups, or, they may be so overworked and pressed for time that allowing holes in the firewalls is the quickest and easiest way to complete their tasks. Further, network administrators tended to view errors created by end users as more intentional than unintentional (i.e. end users commit more violations than unintentional error), while errors created by network administrators as more unintentional than intentional (i.e. network administrators commit more unintentional errors than violations). Lastly, organizational factors, such as communication, security culture, policy, and organizational structure, were the most frequently cited work system factors associated with CIS.

3. Methods

The purpose of this paper is to describe the types of CIS violations that are associated with Reason and colleagues’ [1] classification for deliberate violations: routine, optimizing, and situational, as well their related mechanisms. Interviews with the members of the Sandia National Laboratories’ Information Design Assurance Red Team (IDART™) program were performed in order to capture information on CIS violations. The IDART™ program is composed of CIS experts hired by organizations to perform adversary-based analysis

of CIS, including simulation of hackers, breaching their CIS systems, and providing feedback on the weaknesses of their CIS systems [12]. The red team looks for opportunities to combine system, organizational, and architectural vulnerabilities in order to execute a successful attack.

The IDART™ program is a qualified source of information on the state of CIS because they perform multiple types of critical assessments for a variety of organizations. Their customers include those from the private sector, ranging from banking and finance, information technology, manufacturing and e-commerce, as well as the public sector, including the US Departments of Defense, Energy, Interior, Homeland Security, and State. Thus, their views on CIS systems are diverse and comprehensive, given the number and variety of assessments performed.

Fourteen semi-structured interviews with red team members were conducted at Sandia National Laboratories in Albuquerque, New Mexico. The red team members were asked to describe the human and organizational factors that are associated with CIS (see Appendix A for interview guide and probes). In their responses, they described the CIS violations committed by end users and network administrators and a number of factors that are associated with CIS violations.

Each interview was one hour in length. Twelve interviews were audio-recorded and two interviews were not audio-recorded. The content of the two interviews not audio-recorded was captured with hand-written notes. Both the audio-recorded interviews and the written hand notes were transcribed electronically. The transcribed notes and interviews were analyzed by coding the themes of interviews using the qualitative software package, QSR NVivo©. The analysis yielded a tree structure consisting of nodes, each representing a defined category of CIS violations (i.e. routine, optimizing, or situational) or type of factor that is associated with CIS violations. When coded, a node held references to passages of text from the interview data. Passages were coded at the lowest level node and were coded only once. Comments are quantified and aggregated in each category and sub-category and are reported in the following section.

4. Results

Results of the interviews are reported on the types of violations committed by CIS end users and

network administrators and the work system factors that are associated with violations. Tables summarizing the frequencies of interview data are presented for each major violation category and associated work system factors.

4.1 Violations committed by end users and network administrators

Red team members commented on the violations committed by CIS end users and network administrators (see Table 1). The red team members reported only one type of violation: routine (23 comments). Routine violations for both CIS end users and network administrators are defined as “corner-cutting” or short-cuts taken in order to accomplish their work tasks, whether those tasks are CIS-related or not.

4.1.1 End user routine violations

Red team members reported on routine violations by end users (9 of 23 total comments on violations). End users perform violations in relation to their passwords (3 comments) not only writing down their passwords (1 comment), but they also choose ‘weak’ passwords (i.e. passwords that are easily guessable and are not alphanumeric, which is a standard practice for strong passwords).

A routine end user violation is downloading unauthorized software from the Internet (2 comments). One red team member described it: “If [corporate] security requirements get in the way of their job, they’re going to find ways around that. And we’ve seen that over and over and over again. [For example…] in some organizations everybody is a local administrator for a machine. Why? So they can install software.” End users may also give sensitive information to non-employees (2 comments), especially if they think they are being helpful. For example, a secretary may expose a user name and password to someone if they are receiving help on a difficult software program

As another example, end users may not keep their security ID or token with themselves; they may leave it in a more vulnerable place, such as unattended with their personal belongings (1 comment). They may also send unencrypted files, just because encrypting is time-consuming (1 comment).

Table 1.

Quantification of comments related to routine violations

4.1.2 Network administrator violations

Network administrators also perform a number of routine violations (14 of 23 total comments). Network administrators may create weak passwords (1 comment) or they may create a single, root-level password that will key a number of machines (5 comments). One red team member described this: “Most of the time they [network administrators] do take shortcuts. If you have to take care of and manage 30 servers or 30 network machines and your policy says you can’t have the same password on each one, then that means you’ve got 30 different passwords. So, if you don’t write them down, you’ve locked yourself out and the chances of not being able to respond to a problem are 100%.

”

Network administrators may also create a back door to their network (5 comments). One red team member commented: “They [network administrators] typically don’t follow all of the rules, especially with regards to firewalls and [network] perimeters. We went to one place and they had an Internet service [link]…which bypassed all the corporate firewalls. They hooked it up to important networks through back doors. [And] just by virtue of them hooking it up into the network there were ways to get to and from the Internet without going through their corporate firewalls.”

In addition, network administrators may also choose not to follow CIS protocol entirely (1 comment), not perform backups (1 comment), and not adequately apply patches (1 comment).

4.2 Work system factors associated with routine CIS violations

Red team members reported a total of 44 comments on work system factors that contribute to routine violations of CIS end users and network administrators (see Table 2).

4.2.1 Factors associated with end users’ violations

A number of work system factors are associated with end users’ violations (15 of 44 total comments). End users may commit violations because they lack adequate CIS training or education (2 comments), which would afford them with a fuller understanding of purpose of CIS. End users may circumvent the CIS requirements that they do not fully understand in order to complete their work (1 comment). End users may be supplied with incorrect technology, given their work (1 comment). For example, if their computer is very slow, end users may skip scanning files for viruses.

End users may also have a number of beliefs that affect their propensity to commit violations (11 comments). They may believe that CIS does not affect them and are then not responsible for adhering to CIS principles (1 comment), they may have misplaced trust in others in that they believe that others always carry benevolent intentions (2 comments), they may feel that they do not have a stake in the organization and therefore do not care if the organization is secure or not (1 comment), and they may also have desire to help others, even if it means compromising CIS (3 comments). Red team members also reported that end users tend to view their work tasks as more important than CIS (4 comments). One red team member commented: “They may have a belief that their job requirements or their task requirements are of a higher importance to the organization than to follow the [CIS] policy 100% of the time, so maybe they even follow the policy part of the time. Maybe it’s just an exception that they choose not to follow that policy.”

4.2.2 Factors associated with network administrators’ violations

Policy may be associated with network administrators’ violations (2 comments). One red team member hypothesized that network administrators may just be more apt to violate policies than the average user (1 comment), and this may be because the CIS do not support

Table 2.

Quantification of comments on work system factors

network administrators’ work (1 comment). For example, a strict policy on hardened firewalls may conflict with the non-CIS related duties the network administrator has to perform. They may find their own ways to get around the firewalls and establish their own, yet insecure, Internet connection.

The work system may affect how network administrators beliefs or views of CIS. Network administrators may feel like they are exempt from following the rules because they know more about CIS than average users (1 comment). A red team member commented on this: “…you’ll have the administrators who get on the users; well, we can’t do this because this is a bad thing, but that same administrator is also the one that has shortcuts into every other machine because they understand security so somehow it [CIS policy and procedures] doesn’t apply to them anymore.” Further, network administrators may feel “invisible” (2 comments) to the organization (i.e. the organization does not recognize their good work; they only receive recognition when things go wrong). For this reason, they may feel some undue pressure from the organization (4 comments) and consequently, feel that they need to perform their jobs perfectly (2 comments). One red team member commented in regard to network administrators’ situation: “We’ve been in places where we’ve gotten to know the administrators even over…15, 20 minutes… and you can tell that they do feel invisible. They feel picked on. Because if something doesn’t work… the power goes out…it’s really not in their control, but they get blamed for it.”

Finally, network administrators’ tasks may also be related to their behavior. One part of this problem is that there is an overall lack of prioritization in their CIS-related tasks (14 comments). Network administrators may view their regular work tasks as more important than CIS-related tasks, making them more apt to disregard them (1 comment). They may ignore what they view as low priority CIS components (1 comment). They may also disregard system updates, normally viewed by CIS professionals as a more important task (2 comments).

Additionally, network administrators typically are extremely overburdened with too many tasks to

perform (10 comments). One red team member commented: “The more overloaded you [network administrators] are, the more shortcuts you’re tempted to take, because you just flat out don’t have the time do to everything the way it should be done.” Other tasks related to network administrator violations are: shortcuts on CIS tasks they find boring, such as performing backups (1 comment), difficulties in multi-tasking (1 comment), and understaffing related to many tasks to perform (2 comments).

5. Discussion

This paper described violations in the CIS context. Fourteen red team members from Sandia National Laboratories’ IDART™ program provided information on the types of CIS violations and related work system factors. Overall, the analysis yielded 67 comments: 23 comments on the types of routine violations and 44 comments on the work system factors related to routine violations. There was an emphasis on network administrators’ violations; 9 comments on end users’ routine violations and 14 comments on network administrators’ routine violations. Red team members did not comment on optimizing nor necessary violations. One reason for the emphasis on network administrators may be that they have more control and a deeper level of access to the IT systems, so their violations are potentially more devastating than end user violations (e.g., one ‘weak’ password that controls multiple machines). From the adversarial viewpoint of the red team members, they target areas of the CIS system that interact with individuals having greater access and influence on the CIS system. Further, compared with Kraemer and Carayon’s [11] study of CIS system defenders (i.e. network administrators and CIS managers) views of human error and violations related to CIS, network administrators tend to emphasize end user violations rather than their own violations. This disparity is that network administrators may not acknowledge the extent or severity of their own actions. This may be related to how network

administrators view themselves versus others who interact with their networks. Finally, it is important to note that the red team uses a critical viewpoint or adversarial perspective. CIS system defenders such as network administrators tend not to have the knowledge and training in critical assessments that red teams do.

There were 15 comments on work system factors affecting end users’ behaviors, while there were 29 comments on work system factors affecting network administrators’ behaviors. Red team members tend to emphasize end users’ views of their CIS-unrelated work as superior to end users’ views of the need to observe CIS principles. Network administrators’ task overloading (10 comments) was emphasized as contributing to the tendency to take short-cuts to complete their work. This may be an area for job redesign efforts, where tasks are prioritized and clearly defined. There also must be provision of sufficient resources, including technical staff and support, to complete the critical CIS work.

There are several limitations for this study. First, this research was of one red team program. Other types of red team programs could be accessed in order to capture their views of CIS violations and work systems factors. Secondly, the taxonomies of violations and associated work system factors are not comprehensive or exhaustive. For example, the red team only commented on one type of violation: routine. Further, there may be other people who commit violations, such as CIS managers. The red team was not queried to specifically address violations and associated work system factors, they were asked to describe the human and organizational factors associated with CIS in general. Future research in this area should include directly asking red team members about all types of deliberate violations and related work system factors. This study could serve as a preliminary framework to be expanded upon. Future research could also include testing some of these concepts for predictability of behaviors, or other outcomes, such as specific CIS vulnerabilities.

6. Conclusion

This study produced descriptive information on CIS violations committed by network administrators and end users. Examining the tandem issues of violations and the context, the CIS work system, is important in order to make redesign as specific and relevant as possible.

Acknowledgements

Funding for this research was provided by the United States Department of Defense on “Modeling and Simulation for Critical Infrastructure Protection” (#DAAD19-01-1-0502, PI: Professor Stephen Robinson, University of Wisconsin-Madison).

Appendix A. Interview guide

Global question: What are the human and organizational factors that adversely affect computer and information security?

Probes: (1) Do you have an example(s) and stories? (2) What are the organizational factors associated with CIS (e.g., security culture, security policies)? (3) What are the individual factors associated with poor CIS (e.g., no training in CIS, low acceptance CIS methods)? (4) What are the

technological factors associated with CIS (e.g., not user-friendly encryption methods, poor interface between users and CIS systems)? (5) What are the

tasks associated with CIS (e.g., overwhelmed by

managing patches, updating software)? (6) What are the environmental factors associated with poor CIS (e.g., noisy workplace environments)?

References

[1] Reason J, Parker D, and Free R, Bending the Rules: The Varieties, Origins and Management of Safety Violations. Leiden: University of Leiden, 1994.

[2] Sasse MA, Brostoff S, and Weirich D. Transforming the 'weakest link'-a human/computer interaction approach to usable and effective security. BT Technology Journal, 19(3) (2001) 122-131.

[3] Schneier B. Secrets and lies: Digital Security in a Networked World. New York: John Wiley & Sons, Inc., 2000.

[4] Besnard D and Arief B. Computer security impaired by legitimate users. Com & Sec. 23 (2004) 253-264.

[5] Stanton JM, Stam KR, Mastrangelo P and Jeffery J. Analysis of end user security behaviors. Com & Sec. 24 (2005)124-133.

[8] Rasmussen J. Human errors: A taxonomy for describing human malfunction in industrial installations. J of Occ Acc. 4 (1982) 311-333.

[9] Reason J. Human Error. New York, Cambridge University Press, 1990.

[10] Reason J, Manstead A, Stradling S, Baxter J, and Campbell K. Errors and violations on the roads: A real distinction? Ergo, 33-10/11 (1990) 1315-1332.

[11] Kraemer, S and Carayon P. A human factors model of human error and violations in computer and information security. App Ergo, (forthcoming).

[12] Wood BJ and Duggan R. Red teaming of advanced information assurance concepts, in DISCEX2000 DARPA

Information Survivability Conference: Hilton Head, South