Using a Grounded Theory Approach. (Under the direction of Dr. Laurie Williams).
Evolving adaptable security assessment techniques is important since the landscape of security threats is constantly changing and expanding. We adapt the notion of a software design pattern as proposed by Gamma et al. to the domain of black box security testing. A software security test pattern is a description of a generalized test case that could be used to reveal a recurring vulnerability type, that is described such that the test case can be instantiated a million times over, without ever doing it the same way twice. Using grounded theory to develop patterns allows researchers to reuse observations and reasoning made while developing patterns to evolve patterns or develop new patterns. The goal of this research is to help software testers adapt to the evolving demands of security testing by proposing and
evaluating an empirical process for developing software security test patterns using a
grounded theory approach.
In this dissertation, we produce 11 software security test patterns by using our empirical process on the CWE/SANS Top 25 Most Dangerous Software Errors [24], a list of generic vulnerability descriptions that CWE/SANS prioritized and ranked based on input from security experts. We first use the generic vulnerability descriptions from the Top 25 to produce 24 specific test templates that reveal these vulnerabilities. We then use 16 vulnerabilities that CWE lists as being “On the Cusp” of the Top 25 to produce two additional specific test cases. Using a grounded theory based approach, we then abstracted these 26 specific test templates into 11 general test templates by grouping specific test templates together based on similarities in their procedure and expected results templates. We then add an example of applying the general test template to a natural language artifact to each test template. We also add a list of keywords to each general test template that, when found in a natural language artifact, suggest the need for the application of the general test template to a system under test. The general test template plus the example natural language artifact plus the list of keywords make up a software security test pattern. After their development, software testers can apply these 11 patterns to a software engineering natural language artifact to produce a black box security test plan.
organization, and the sparse distribution of vulnerabilities in a software system, we find that providing a technique to discover commonly occurring vulnerabilities to software testers with no expertise in security is a valuable contribution.
To demonstrate that we can use patterns produced using our methodology to develop a black box test plan using requirements specifications from more than one source, we apply our security test patterns using the requirements specification from a commercial, proprietary system to form a test plan of 125 tests. To demonstrate that we can use the tests created for this commercial, proprietary system to reveal vulnerabilities, we executed these tests on the system. We find that 11 of our 125 (9.0%) of our text cases revealed security vulnerabilities in a commercial, proprietary software product. The fact that the we used a software security test pattern catalog to develop a test plan in two different domains indicates that the software security test pattern catalog we produced is not specific to a certain domain or system.
than tests created using our catalog because of the effort involved in using our catalog. However, tests developed using our pattern catalog can help software testers identify vulnerabilities introduced by design decisions even if those testers have no expertise in security. We concluded that this investigation provides more evidence that there is “no silver bullet” in software security testing, and that no single software security assessment technique can find every vulnerability in a system.
To investigate whether software testers who are not experts in security can use patterns developed using our methodology to create software security test plans that resemble test plans that experts would develop, we conducted a user study of 47 “novice” (with respect to security) students at North Carolina State University. These novices used six requirements statements from the previously mentioned public requirements specification for EHR systems to each produce individual security test plans. Separately, we created a panel of experts in software security from students who are studying software security in their research at North Carolina State. We required this panel to form a consensus about a software security test plan for EHR systems using the same six requirements that the novices used. We found that software testers who are not security experts create software security test plans that resemble test plans developed by experts.
In addition to these findings, this dissertation provides the following contributions:
• We provide a software security test pattern catalog of 11 patterns based on a public list of frequently occurring software security vulnerabilities.
• We provide a tool that implements the process of applying security test patterns to natural language artifacts.
by
Benjamin Hatfield Smith
A dissertation submitted to the Graduate Faculty of North Carolina State University
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy
Computer Science
Raleigh, North Carolina 2012
APPROVED BY:
_______________________________ ______________________________
Laurie Willliams, PhD Mladen Vouk, PhD
Committee Chair
________________________________ ________________________________
Annie Antón, PhD Jacquie Halladay, MD, MPH
DEDICATION
BIOGRAPHY
Ben Smith was born in Lexington, KY, USA in January of 1985. He enjoys swimming, hiking, biking, geocaching, playing piano, coffee, deep philosophical conversations, and writing code. After taking some programming courses in high school, Ben decided to attend NC State for his bachelor’s degree in Computer Science. Ben also received his Master’s and his PhD at State. Ben is a member of the NC State Computer Science Realsearch group, led by Laurie Williams.
In completing the PhD, Ben has learned volumes about Software Engineering, but he has also learned how to manage data. Ben has offered his expertise in data collection and analysis to medical researchers. He participated in a collaborative research project that analyzed the prevalence of sleep apnea in truck drivers. Ben also participated in a project that examined the sleep patterns of undergraduate students in a sleeping and wellness course at a major public university.
ACKNOWLEDGMENTS My thanks, appreciation, and love goes out to…
• Aaron, for validating my obscure ideas on public policy and government, for taking any side of the debate so we could continue to talk, for helping me deal with the treacherous issues related to finding a career, and for giving me a hard time and keeping me honest.
• Andy, for checking that I did the statistics correctly, for chiding reviewers with me when I got rejected, for not being afraid of wasting time talking science with me, for talking about the meaning of life with me, for teaching me that research is more fun when you do it with other people, and for making me laugh.
• Pete, for introducing me to new things, for giving me as hard a time as I give him, for acting as my comedy sidekick, for dragging me out of the house when I did not want to go, for being a friend in good times and bad, and for being just as much of a geek as I am.
• Josh, for participating in some of the longest conversations of my life, for teaching me the skill of innuendo, for introducing me to the higher forms of music theory, and for showing me that it is important to be phenomenal at something whether it is music or academia or both.
• Shelly, for always being there for me when I was feeling low, for telling me it was going to be okay, for just hanging out with me when that is what I wanted to do, for going on hikes with me, for always listening to everything I had to say, for
appreciating me for who I am, and for being the best girlfriend I have ever had.
• Annette, for being my “s-mom”, for teaching me to only wash red clothes with other red clothes, for getting me an excellent piano teacher, for taking care of my dad, for being there for the family when we needed it, and for showing me the difference between right and wrong.
• Mom, for showing me the world and teaching me how it works, for being
straightforward and honest about anything I wanted to talk about, for teaching me to enjoy the woods, for teaching me how to love a dog (or a cat, if necessary), for reminding me that this is only one phase of my life, and for being the coolest mom I know.
• Laurie, for funding me, for reviewing my papers (and presentations, and posters, and web pages), for teaching me what it means to write clearly, for listening to me
complain, for reminding me not to be so negative, for never letting me lose hope, for convincing me to try again, and for believing in me. I could not have done this without her.
TABLE OF CONTENTS
LIST OF TABLES... xi
LIST OF FIGURES ... xii
1 Introduction... 1
2 Background and Related Work... 7
2.1 Terminology... 7
2.2 Design Patterns ... 9
2.2.1 Background... 10
2.2.2 Security Test Patterns and Design Patterns ... 11
2.2.3 Pattern Writing... 12
2.3 Other Software Patterns ... 15
2.3.1 Security (Design) Patterns ... 16
2.3.2 Patterns for Automated Unit Testing ... 19
2.4 Grounded Theory ... 21
2.4.1 Background... 21
2.4.2 Related Work ... 24
2.5 Software Security, Requirements and Testing... 27
2.5.1 Security Testing and Functional Black Box Testing ... 27
2.5.2 Software Security Assurance Techniques ... 34
2.5.3 Using STRIDE to Model Security Threats ... 35
2.5.4 Security Requirements... 36
3 Formative Studies ... 44
3.1 Improving the Security Criteria for EHR Systems ... 44
3.1.2 Methodology... 47
3.1.3 Results... 51
3.1.4 Summary of Findings ... 54
3.1.5 Limitations ... 55
3.2 Challenges for Protecting the Privacy of Health Information ... 55
3.2.1 Motivation... 56
3.2.2 Methodology... 56
3.2.3 Successful Exploits: Implementation Bugs ... 61
3.2.4 Successful Exploits: Design Flaws ... 70
3.2.5 Recommendation ... 72
4 An Empirical Process for Developing and Instantiating Software Security Test Patterns.. 76
4.1 Form and Structure of a Software Security Test Pattern ... 76
4.1.1 Pattern Form and Structure ... 76
4.1.2 Example Pattern: Upload a Malicious File (UF) ... 77
4.2 A Process for Developing a Security Test Pattern Catalog Using Grounded Theory ... 78
4.2.1 Phase One: Developing Specific Test Templates ... 81
4.2.2 Phase Two: Developing Keywords ... 83
4.2.3 Phase Three: Abstracting Specific Test Templates into General Test Templates... 85
4.2.4 Phase Four: Identifying an Appropriate Example Requirement and Instantiating an Example Test Case ... 86
4.2.5 Incorporating the Advice of Expert Pattern Writers... 86
4.2.6 Limitations ... 87
4.3 Using Software Security Test Patterns ... 89
4.3.2 Instantiating the Test Pattern ... 90
4.3.3 Automation: STPI ... 91
5 Applying the Empirical Process to Develop a Security Test Pattern Catalog ... 96
5.1 Applicability of the CWE/SANS Top 25+ Test Pattern Catalog... 97
5.1.1 Development of the 2011 CWE/SANS Top 25+... 98
5.1.2 Methodology... 100
5.1.3 Analysis of Results ... 102
5.2 Developing a Security Test Pattern Catalog for the CWE/SANS Top 25+ and CCHIT... 105
5.2.1 Phase One: Developing Specific Test Templates for the CWE/SANS Top 25+ ... 106
5.2.2 Phase Two: Developing Keywords Using the CCHIT Ambulatory Certification Criteria ... 109
5.2.3 Phase Three: Abstracting Specific Test Templates into General Test Templates... 112
5.2.4 Phase Four: Identifying an Example Requirement ... 114
5.3 Application Neutrality ... 115
5.3.1 Evaluating Application Neutrality ... 116
5.3.2 Application Neutrality Results ... 116
5.4 Limitations ... 118
6 Using the Security Test Pattern Catalog to Create a Security Test Plan ... 120
6.1 Study 1: Three Electronic Health Record Systems... 120
6.1.1 Developing the Test Plan... 121
6.1.2 Test Subjects ... 121
6.1.3 Test Results... 123
6.1.4 Limitations ... 126
6.2 Study 2: Commercial Product... 140
6.2.1 Developing Keywords from the PropApp Requirements Specification... 141
6.2.2 Commonality Between Keywords for Two Studies ... 142
6.2.3 Developing the Test Plan... 144
6.2.4 Test Subject: PropApp ... 144
6.2.5 Test Results... 145
6.2.6 Limitations ... 146
7 Evaluating Novice Usage of Security Test Patterns ... 148
7.1 Study Overview ... 148
7.2 User Study Methodology ... 149
7.2.1 Research Instrument: STPI Prime ... 150
7.2.2 Establishing Expert Consensus... 150
7.2.3 User Subject Groups ... 151
7.2.4 Requirements Set ... 152
7.3 Study Results ... 153
7.3.1 Level of Agreement ... 153
7.3.2 Novice Times... 154
7.3.3 User Survey ... 158
7.4 Summary of Findings... 158
7.5 Limitations ... 159
8 Conclusions... 161
8.1 Summary ... 161
8.2 Contributions... 163
REFERENCES ... 166
APPENDICIES... 176
Appendix A. Notes for Developing Test Cases... 177
Appendix B. Notes for Developing Keywords... 227
Appendix C: Exploratory Security Test Pattern Catalog... 351
Appendix D: Initial Security Test Pattern Catalog ... 357
Appendix E. CCHIT Black Box Security Test Plan Summary ... 379
LIST OF TABLES
Table 1. Characteristics of OpenEMR... 48
Table 2. Static Analysis Summary of OpenEMR ... 51
Table 3. Rational AppScan Summary for OpenEMR... 53
Table 4. Locations of Cross-Site Scripting Exploits in Target Applications... 65
Table 5. CWE Issues in CVE Found in Top 25+... 103
Table 6. Information about Study Subjects ... 123
Table 7. Test Results for Three EHRs and Overall ... 128
Table 8. Priorities of the Issues Discovered Using Fortify... 132
Table 9. Vulnerabilities not Discovered by Fortify ... 133
Table 10. Priorities of the Issues Discovered Using Rational AppScan... 135
Table 11. Vulnerabilities not Discovered by Fortify ... 136
Table 12. Common Issues between our Test Plan and Other Techniques... 140
Table 13. Commonality Between Keywords for Two Studies ... 143
Table 14. Results for PropApp ... 147
Table 15. Response Rate and Sample Sizes ... 152
Table 16. User Study Requirements Set ... 152
Table 17. Average Times in Min:Sec for Requirement Completion... 157
LIST OF FIGURES
Figure 1. SQL Injection Example True Positive (PHP)... 49
Figure 2. SQL Injection Example False Positive (PHP)... 49
Figure 3. Detailed Diagram of Network Setup ... 58
Figure 4. An HTTP Request to localhost. POST parameter bolded ... 59
Figure 5. Code Excerpt from SQL Injection Attack, MD5 Password in Bold ... 62
Figure 6. Upload a Malicious File (UF) Pattern ... 78
Figure 7. Contributions of Each Phase of Our Empirical Methodology ... 80
Figure 8. Methodology for Developing Specific Test Procedure Templates ... 82
Figure 9. Methodology for Developing Keywords... 85
Figure 10. Using STPI to Parse Requirements ... 94
Figure 11. Using STPI to Instantiate Security Test Patterns ... 95
Figure 12. Percent CVEs Covered by the Top 25+ per Year ... 105
Figure 13. CWE-829 Extended Description ... 107
Figure 14. CWE-829 Example Attack... 109
Figure 15. The Resultant Specific Test Procedure Template ... 111
1 Introduction
Evolving adaptable security assessment techniques is important since the landscape of security threats is constantly changing and expanding. According to a 2010 report that was based on the interviews from 2,800 Information Technology professionals worldwide, the gap between hacker threats and suitable security defenses is widening, and the types and numbers of threats are changing faster than ever before [52]. In 2010, Jim Gosler, a fellow at the Sandia National Laboratory who works on countering attacks on U.S. networks, claimed that there are approximately 1,000 people in the country with the skills needed for cyber defense. Gosler went on to say that 20 to 30 times that many are needed [53]. Additionally, the Chief Executive Officer (CEO) of the Mykonos Software security firm indicated that today's graduates in software engineering are unprepared to enter the workforce because they lack a solid understanding of how to make their applications secure [60]. Particularly due to this shortage of security expertise [32], the development community needs a vehicle to capture and disseminate knowledge about how to assess whether software systems have adequate defenses against malicious users.
twice. Just as design patterns capture design knowledge into a reusable medium [34], software security test patterns capture security testing knowledge into a reusable medium.
Researchers compose patterns by observing recurring problems with either 1) informal observation over many years in a domain; or 2) formal observation based upon a structured analysis of artifacts in the field [4]. Researchers have proposed new pattern catalogs since the Gamma et al. catalog [13], [20], and patterns have reached many domains, such as telecommunications [1], information integrity [21], mobile applications [48], and patterns about patterns themselves [70]. Meszaros and Doble [70] describe a set of metapatterns; that is, patterns that describe the process of pattern writing itself. Additionally Vlissides, one of the Gang of Four authors, describes the seven successful habits of successful pattern writers [100–102]. Vlissides, Meszaors and Doble indicate that pattern writers should present their patterns in a universally understood form, that patterns in a domain should conform to a specific template, that pattern writers should create their patterns thoughtfully, and that pattern writing is an on-going process with continuous iterations that improve the patterns. Overall, many researchers do not describe the stepwise process they have used to develop patterns that they contribute.
Using grounded theory to develop patterns allows researchers to reuse observations and reasoning made while developing patterns to evolve patterns or develop new patterns. As attackers’ processes grow more sophisticated and complex [52], security assessment methodologies must grow and mature in tandem. The goal of this research is to help software testers adapt to the evolving demands of security testing by proposing and
evaluating an empirical process for developing software security test patterns using a
grounded theory approach.
make up a software security test pattern. After their development, software testers can apply these 11 patterns to a software engineering natural language artifact to produce a black box security test plan. We present the pattern catalog on our security test patterns wiki, which can be found at http://securitytestpatterns.org.
To demonstrate that we can use patterns produced using our methodology to develop a black box security test plan, we applied the 11 patterns using a public requirements specification for electronic health records (EHR) systems to produce a test plan consisting of 117 tests. To demonstrate that we can use a black box test plan we created using our patterns to reveal vulnerabilities, we then executed these 117 tests on three open source EHR systems. We find that 65 out of 351 (18.5%) of our test executions found a specific security exploit in the three EHR systems. Given each vulnerability’s potential damage to the software organization [94], and the sparse distribution of vulnerabilities in a software system [84], [106], we find that providing a technique that leads to the discovery of commonly occurring vulnerabilities by software testers with no expertise in security is a valuable contribution.
test pattern catalog to develop a test plan in two different domains indicates that the software security test pattern catalog we produced is not specific to a certain domain or system.
To investigate whether patterns developed using our methodology reveal vulnerabilities that other software security assessment techniques do not, we compared the vulnerabilities we discovered (test failures) in one EHR system to the vulnerabilities discovered by two proprietary software security assessment tools. We compared our results to IBM’s Rational AppScan, the market leader in penetration testing tools [75]. We also compared our results to Fortify 360, a market leader in static analysis for security [76], [107]. We found that the patterns developed using our methodology reveal different types of vulnerabilities than those revealed by automated penetration testing and static analysis. The tools only identify code-level security problems with software systems, whereas tests created using our catalog reveal vulnerabilities introduced by design decisions as well as code-level problems. Our results indicate that these tools might be more suited for detecting code-level security problems than tests created using our catalog. However, tests developed using our pattern catalog can help software testers identify vulnerabilities introduced by design decisions even if those testers have no expertise in security. This investigation provides more evidence that there is “no silver bullet” in software security testing, and that no single software security assessment technique can find every vulnerability in a system [11], [67].
statements from the previously mentioned specification for EHR systems to each produce individual security test plans. Separately, we created a panel of experts in software security from students who are studying software security in their research at North Carolina State. We required this panel to form a consensus about a software security test plan for EHR systems using the same six requirements that the novices used. We found that software testers who are not security experts will each make similar decisions to the experts when developing a black box test plan using security test patterns.
2 Background and Related Work
This chapter describes the background and related work for this dissertation. First, Section 2.1 presents the terminology specific to this dissertation. Next, Section 2.2 provides the background and related work on design patterns. Then, Section 2.3 presents related work on other types of software patterns. Next, Section 2.4 provides background on grounded theory. Finally, Section 2.5 explains the relationship between software security testing, requirements, and security assurance techniques.
2.1 Terminology
The following is a brief list of terminology specific to this dissertation:
• Exploratory Security Test Pattern Catalog: The pattern catalog we developed first, based on the CWE/SANS Top 25+ and based upon the work presented in Chapter 3. We present this catalog in Appendix C.
• Initial Security Test Pattern Catalog: The pattern catalog we developed based on detailed traceability and grounded theory analysis of specific test cases written to reveal the CWE/SANS Top 25+ vulnerabilities and examples derived from the a set of publicly available requirements. In this dissertation, when we do not specify exploratory or initial, we are referring to the initial security test pattern catalog. This catalog is presented in Appendix D.
• The Notes: The set of web pages on the security test patterns wiki that document the analysis for each step of the pattern development process (see Chapter 5).
origins and their consequences. The CWE dictionary identifies vulnerability types with CWE-n where n is the vulnerability type’s CWE unique identifier (provided by Mitre, Incorporated).
• Fault-based Testing: A software testing strategy in which the objective is to design and develop test case inputs that reveal pre-specified faults, as opposed to testing strategies whose goal is providing evidence that the program executes correctly under normal conditions [72], [73], [79].
• Design Flaws: High-level security problems associated with the architecture of the software as defined by McGraw [66]. According to McGraw, design flaws are security issues with a software system where the software system is implemented to specification, but the specification provides a lack of a desired level of security. An example of a design flaw would be the lack of sufficient auditing in the requirements specification to ensure that the users of a system are accountable for their actions. According to McGraw, security faults in the design flaw category generally occur with the same frequency as the implementation bugs in any particular software project [66].
• Test Template: A component of a security test pattern that includes the procedure template and the expected results template. To instantiate a test template into a security test, a tester inserts an action and object key phrases from a requirements statement into the template. We mined two different types of test templates:
o Specific Test Templates: the test templates that represent the grounded theory codes, which we obtained by examining each of the CWE/SANS Top 25+. In this dissertation, we refer to the 26 specific test templates as (T-n), where n is the unique identifier for that test.
o General Test Templates: those test templates created by abstracting specific
test templates into grounded theory categories.
• Penetration Testing: A technique for black box security testing in which the software tester assumes the role of an attacker and attempts to intentionally misuse the system to reveal vulnerabilities [9].
2.2 Design Patterns
2.2.1 Background
Design patterns were originally conceived by Alexander [4] in the field of building architecture, and tailored to software engineering by Gamma, et al. (the so called “Gang of Four”) in their book Design Patterns: Elements of Reusable Object Oriented Software [34]. Vlissides, one of the Gang of Four authors, later modified and updated the original patterns in Design Patterns with his book Pattern Hatching [101]. Alexander later introduced the notion of design pattern languages [3], which were tailored to software engineering by Coplien [19]. A pattern language is a collection of patterns that build on each other to generate a software system [3]. A pattern catalog is different than a pattern language in that a catalog is not necessarily complete or sufficient to develop or test an entire system. Therefore, a pattern catalog is a collection of patterns related to the same domain and exhibiting the same form.
According to Alexander [3], design patterns consist of four essential elements. However, the other pattern writers have deviated from these four essential elements. In many instances, pattern writers provide more or less elements, or rename the elements depending on the domain and intended use of the pattern catalog. According to Alexander, the four essential elements are:
1. The Pattern Name, which describes the design problem and its solution. The name provides a vocabulary for discussing design at a higher level of abstraction.
3. The Solution, which describes the elements that make up the design and the relationship between these elements. The solution does not describe a particular concrete implementation, because the pattern is like a template that be applied in many different situations. Instead, the pattern provides an abstract description of a design problem and how a general arrangement of elements solves the problem. 4. The Consequences, which are the results and trade-offs of using the pattern.
Software designers must constantly evaluate the trade-offs between using two alternate designs and listing the consequences of a pattern allows designers to understand the cost and benefits of applying a specific pattern.
The patterns provided by Gamma et al. [34] additionally consist of 1) the Sample Code, which provides code fragments that demonstrate how a developer could implement the pattern in a certain language, and 2) the Known Uses, which are examples of where designers have used the pattern in real-world systems.
2.2.2 Security Test Patterns and Design Patterns
when found in natural language artifacts, indicate when a tester should apply the test template to a system. We consider these keywords to be analogous to the problem statement of a design pattern because they help a software test know that the pattern is applicable. Finally, software security test patterns contain a concrete example of the test template applied to an example natural language artifact. We consider the concrete example of the test template analogous to the known uses and sample code sections of the Gamma et al. patterns because they illustrate how a tester might instantiate the pattern to produce a black box security test [34].
2.2.3 Pattern Writing
Vlissides included a chapter in Pattern Hatching entitled “Seven Habits of Successful Pattern Designers” [101], which he first articulated in a position paper [100]. The Seven Habits are also available on the web [102]. In Pattern Hatching, Vlissides records the habits that he and the other “Gang of Four” authors adopted while authoring the Design Patterns book [34]. In this section, we give a brief description of each of the Seven Habits.
1. Taking Time to Reflect. This habit explains that pattern writers should first write down their thoughts and experiences in solving problems before amalgamating the solutions to those problems into a pattern. In some ways, this process can be thought of as similar to the grounded theory process described in Section 2.4, only not as rigorous. Vlissides explicitly states that pattern creators should record their experiences incrementally.
catalogs can have different forms. The habit goes on to say that every pattern within a specific catalog should consistently follow the same template.
3. Being Concrete Early. In this habit, Vlissides asserts that people understand concepts better when concepts are presented in concrete terms first and then more abstract terms. Thus, Vlissides purports, patterns should always include an applied example from the real world.
4. Keeping Patterns Distinct and Complementary. Vlissides shows in this habit that people have a tendency, when developing a new pattern, to forget about the patterns that already exist in the catalog. Vlissides illustrates that by continuously evaluating whether two patterns are truly different, a pattern writer can develop patterns that are distinctive. This notion is similar to the idea of constant comparative method from grounded theory (see Section 2.4).
5. Presenting Effectively. Vlissides argues in this habit that the page layout,
typography, graphics and printer quality all have an effect on how easily others can use and understand a pattern catalog. Vlissides also points out that effectively communicating a pattern involves a writing style that includes clarity and an ease of reading.
7. Collecting and Incorporating Feedback. This habit reveals that the test of a pattern is when someone other than the pattern’s author uses the pattern. Vlissides
encourages pattern writers to disseminate their patterns as widely as possible.
Meszaros and Doble authored a pattern language for pattern writing [70], which captures the best practices that pattern authors follow while developing software design patterns. Meszaros and Doble present these best practices for writing patterns in the form of patterns.
The pattern language created by Meszaros and Doble consists of five categories, which we briefly describe in the following list:
1. Context-Setting Patterns. This category illustrates two patterns. One for the concept of a pattern and another for the concept of pattern language, both of which are defined and detailed in Section 2.2.1.
2. Pattern Structure Patterns. The five patterns in this category describe mandatory and optional components of a pattern template. Unlike the recommendation from Vlissides, Meszaros and Doble indicate that all patterns (of any type) must contain the same elements: pattern name, problem, solution, context, and forces. Meszaros and Doble also explain that some elements may be included additionally to help
understand the pattern: resulting context, related patterns, examples, code samples, rationale, aliases, and acknowledgements.
the pattern should contain a noun, and that the pattern should involve a meaningful metaphor that makes sense to the reader.
4. Patterns for Making Patterns Understandable. The five patterns in this category detail how Meszaros and Doble recommend pattern writers can allow pattern readers to comprehend their patterns. With these patterns, Meszaros and Doble recommend that patterns should have a clear audience, should have a terminology tailored to that audience, should have notations that are understandable, and should have code samples.
5. Pattern Language Structuring Patterns. With the six patterns in this section, Meszaros and Doble indicate that a pattern language should contain a summary of problems and solutions should have headings that convey structure, should contain a running example, and should contain a glossary.
In this dissertation, we use many of the patterns introduced by Meszaros and Doble in our empirical process for developing software security test patterns (see Section 4.2).
2.3 Other Software Patterns
information integrity [21], mobile applications [48], and patterns about patterns themselves [70]. This section reviews two other pattern domains that other researchers have proposed: security patterns and automated testing patterns. First, Section 2.3.1 reviews work on security patterns, which are more focused on designing security than testing for security. Section 2.3.2 discusses automated test patterns, which are used to correctly design the code for automated tests.
2.3.1 Security (Design) Patterns
Yoder and Barcalow were the first to introduce application security design patterns (a.k.a. “security patterns”) [105], and since then, many more security patterns have been introduced Similar to the patterns introduced in Gamma, et al. [34], security patterns have the following components: Context, Problem, Forces, Solution, and Consequences. Software security patterns provide expert guidance about architecture and design for a secure system designer. Our security test patterns deviate from these components because of the focus on testing rather than encapsulating design concepts. Heyman et al. provide an analysis of the security patterns landscape [46], but we provide a few examples in this section. In this section, we provide some examples of security design patterns and discuss two evaluations of security test patterns’ abilities to uphold security principles.
Finally, Kienzle et al. provide a catalog of over 30 patterns grouped into two main categories: Structural and Procedural [54]. As an example of one security pattern catalog, we discuss those patterns introduced in the Open Group Security Pattern Technical Guide [17], by Blakley and Heath. Blakley and Heath provide 15 security patterns, which are divided into three categories: architectural-level patterns, design-level patterns, and implementation-level patterns [17]. One example of architectural-level patterns introduced by Blakley and Heath is the Distrustful Decomposition pattern. In this pattern, a system that performs more than one high-level function or implements functions that require differing levels of permissions divides each of its functions into smaller, distinct programs or components that share limited information through an inter-process communication mechanism. One example of the design-level patterns is the Secure Factory pattern [17], which is based on the Abstract Factory pattern introduced by Gamma et al. [34]. The Abstract Factory pattern provides an
viewing the log must then have the appropriate authorization to use a Log Reader, which can appropriately decrypt or interpret the secured logs produced by the logger.
Security Pattern Examples. Schumaker introduces a catalog of three security patterns for application firewalls [81]. One firewall pattern Schumaker introduces is called Packet Filter, which indicates that network firewalls should monitor and control network traffic at the lower levels of the network stack so that users do not have to change their way of using applications and services provided on the network. Saridakis introduces three security patterns related to fault containment; that is, preventing an error from contaminating other parts of a system besides where the error occurred [80]. One pattern Saridakis introduces is called Input Guard, which indicates that developers should place a mechanism at every access point of a component in the system that checks the validity of the input. The developer provides each component with a specification of allowable input, and input is only provided to the component if the input matches the specification. Kodituwakku et al. introduce six patterns related to implementing role-based access control policies in object-oriented systems [55]. One pattern Kodituwakku et al. introduce is called Single Session, which addresses the issue of maintaining a single session for each user that conforms to the appropriate security checks. The solution contained within Single Session is to create a Session object that activates the user’s specific role and disables all other active roles. This solution ensures that the user can only be acting in one role at any given time.
some of the security patterns from the Open Group Security Pattern Technical Guide [17], and one group that did not use any patterns. Based on the results of an automated tool and prior experiences, Halkidis et al. determined the likelihood of certain attacks for each of these systems and then analyzed the ability of each security pattern to prevent exploits using these attacks. Halkidis et al. determined that the system that used the security patterns was significantly more resistant to attack than the system that did not.
Halkidis et al. also evaluated the same patterns by Blakley and Heath [17] based on their ability to uphold ten security principles introduced by Saltzer and Schroeder [78]. The ten security principles include notions such as securing the weakest link, practicing defense in depth, and the principle of least privilege [78]. Halkidis et al. found that although there was no single pattern that could be used to uphold all ten principles, designers could achieve all ten principles with a proper combination of the patterns. These researchers carefully thought through how each pattern would or would not uphold each principle and summarized the results. Additionally, Halkidis et al. found that most patterns uphold between one and three principles, and no pattern upholds more than six principles.
2.3.2 Patterns for Automated Unit Testing
all of the units related to a certain portion of the system or a certain piece of functionality, the tester need only execute the TestSuite object, rather than having to execute all tests or pick and choose amongst hundreds of unit tests to find the relevant tests. Meszaros also explains that the tester can make the TestSuite object’s task more straightforward by having each test case inherit from the same super class, so that a TestSuite need only look for any classes that have the super class as a parent and execute them.
Meszaros also introduces the Database Sandbox pattern, which addresses the problem that occurs when multiple testers are executing tests that deal with the same database management system at the same time. In this situation, automated tests can fail or enter a mutual exclusion lock because two separate testers happen to run tests simultaneously that require access to the same table or record in the database. If one tester’s execution requires a value to be “6” and the other tester’s execution increments that record, then the first tester’s execution will fail, but not because of a problem with the system. The Database Sandbox pattern entails either creating a dedicated database that runs locally on the user’s machine, or by simulating a database instance.
a TestSuite object when the tester is using a large number of unit tests, or that testers should emulate a database rather than sharing one (as described above).
2.4 Grounded Theory
This section provides information on grounded theory. Using traditional scientific method, researchers develop hypothesis based on existing theories and logic, and use experimental data to confirm or refute hypotheses [35]. In grounded theory, researchers develop concepts and theories (in our case, test patterns) from the data and observations (in our case, security test cases) [38], [64]. Section 2.4.1 provides the background for grounded theory. Then, Section 2.4.2 provides related papers that demonstrate the successful application of grounded theory techniques.
2.4.1 Background
In grounded theory, first proposed by Glaser and Strauss [38], theory is developed from data. Grounded theory has several elemental representations of data that researchers use to derive explanations of phenomena. Codes are identifying anchors that allow key points of the data to be gathered for later analysis. Concepts are pieces of the data that a researcher has grouped together by identifying similarities in their codes. Categories are broader collections of concepts that a researcher can group together, again by identifying similarities in the concepts. Similarities between concepts are at a higher level and more broad and general that similarities between codes. A researcher then uses a set of categories to establish the theory that is grounded in the data. Grounded theory also makes heavy use of the constant comparative method, where concepts and codes that a researcher is developing are
be restructured. Additionally, grounded theory researchers continuously reevaluate preconceived notions or existing theories from the scientific community based on the theories and categories that are emerging in their current qualitative analysis [38].
Extending the example provided by Seaman [83], if the field notes or observed data for a given study subject might say “John wrote 15 test cases a week, but Mary’s test cases were higher quality than John’s.” A researcher conducting grounded theory might write two codes for this data that state “john-numTestCases=15” and “mary-test-quality > john-test-quality”. If there are many occurrences of these codes, the researcher might then start grouping instances where one tester’s tests were of a higher quality than another tester’s into a concept. Separately, by looking at the codes established for number of test cases, the researcher might establish another concept that indicates a cut-off point for “high number of tests” at 10 test cases. Then, looking at the relationship between these two concepts the researcher might develop a category that indicates that testers who write a high number of test cases frequently produce test cases of a lower quality. Using this category with many others, the researcher may then develop a complete grounded theory about the relationship between numbers of test cases and the quality of those test cases. This example demonstrates a simple case in a domain (software testing) where researchers have already developed a set of theories, but grounded theory is typically used in situations where the relationships and even the substance of what is being studied is frequently unknown to the researcher before coding begins.
process of creating categories is “largely creative” [83]. The creative aspect of categorizing codes does not imply, as Seaman indicates, that coding is purely subjective, as every trend or hypothesis the researcher creates must be “grounded” in the data.
2.4.2 Related Work
Antón and Earp [6] introduced a taxonomy of privacy protection goals and techniques to help customers evaluate the privacy policies of e-commerce websites and search for potential weaknesses and limitations. The taxonomy can also help web site developers ensure that their systems comply with the stated policies and that these policies will correspond to what customers are interested in with respect to their online privacy protection. Antón and Earp employed a specific grounded theory technique, called content analysis, in which the researcher classifies observed data into a set of content categories where entries in a given category have the same meaning [56]. Antón and Earp used content analysis to analyze the privacy goals inherent in 25 separate e-commerce privacy policies from eight different industries to produce 11 categories for goals. This first step acted as the formative study, meaning that the theory and the techniques were developed during this step. The effort of categorizing these privacy policies into goals and categories resulted in a taxonomy for privacy goals as well as a process for categorizing privacy goals. Further, the resultant taxonomy from this formative work contains privacy protection goals, which express the desired protection of consumer privacy rights; and privacy vulnerabilities, which represent statements of fact or existing behavior that can be characterized as a violation of privacy for that website.
Using the results from their formative study, Antón and Earp were able to conclude several important points about privacy policies in software engineering.
The aspects of this study that engaged the principles of grounded theory were that Antón and Earp did not superimpose their own preconceived notions of what the goals of the privacy policies were, nor did Antón and Earp include their own preconceived notions of which of the 11 categories those goals could fall into. As with most grounded theory studies, the formative result of the work was then used in a summative study that simultaneously validates and extends the results [6].
Karhu et al. [51] used grounded theory to analyze interviews conducted in a study to explore the factors that affect the use of software testing automation in different types of development organizations. Specifically, Karhu et al. investigated 30 organizational units across five different software development organizations whose businesses ranged from telecommunication to electronics manufacturing. Karhu et al. targeted those organizational units within each organization who subjectively ranked their own software as being higher than average criticality on a scale of 1 to 5 with 1 meaning a failure is “irritating,” and 5 meaning that a failure could cause loss of life. The resultant sample was five organizational units from five separate organizations that all rated their software as having higher than average criticality with respect to the risk of failure.
the data, including: essential stakeholders, phenomena (such as testing knowledge, testing automation, and reuse), and problems. Karhu et al. then examined the codes produced from the data to examine trends and identified seven categories emergent from the data: (1) Type of Tested Products; (2) Size and Complexity of Tested Products; (3) Type of Testing Automation; (4) Advantages and Disadvantages of Testing Automation; (5) Development of Testing Automation; (6) Reuse; and (7) Testing Knowledge.
Using these categories, Karhu et al. were able to elicit six observations based on the qualitative data that Karhu et al. collected in their interviews. For example, Karhu et al. state their first observation as “Automated software testing may reduce costs and improve quality because of more testing in less time, but it causes new costs in, for example, implementation, maintenance, and training” [51].
grounded theory research, a formative study is considered a substantive contribution since the point of grounded theory is to generate new theories and ideas [38].
In the same way that Antón and Earp and Karhu et al. used grounded theory to develop the theories and concepts in their domains, we use grounded theory to develop security test patterns. Whereas Antón and Earp perform content analysis on privacy policies, and Karhu et al. perform a qualitative analysis of survey data taken from companies in the field, we present a methodology for performing a qualitative analysis of test cases produced to reveal descriptions of vulnerabilities to produce a security test pattern catalog.
2.5 Software Security, Requirements and Testing
This section reviews the relationship between software security assurance techniques, software security testing, and requirements-based testing methodologies. First, Section 2.5.1 provides an overview of functional black box testing and security testing techniques. Then, Section 2.5.2 provides an overview of other software security assurance techniques and development methodologies. Next, Section 2.5.3 describes STRIDE, a specific technique for modeling security threats. Section 2.5.4 gives an overview of security requirements development.
2.5.1 Security Testing and Functional Black Box Testing
files) [84]. The same study revealed that only 258 vulnerabilities were reported for the Red Hat Enterprise Linux Kernel across approximately 3 million lines of code (1.4% vulnerable files).
Even though vulnerabilities are sparsely distributed, each vulnerability can cause serious economic damages to the organization. A study by Telang and Wattal demonstrates that a software vendor loses approximately 0.6% value in stock price when that vendor reports a vulnerability [94]. In his light, software testers could benefit from a test creation technique that specifically targets software vulnerabilities.
Functional Black Box Testing. Black box testing, also known as behavioral testing, focuses on deriving a set of input conditions that will fully exercise all functional requirements for a system [77]. Black box testing attempts to find errors in the following categories: incorrect or missing functions, interface errors, errors in data structures or external database accesses, behavior or performance errors, and initialization or termination errors [77].
Graph-based Testing. In graph-based testing, the tester develops a graph that shows the relationship between objects in a system. In terms of graph-based testing, an object can be a component, a module, a data structure or any other number of entities. After the tester establishes this graph, the tester creates a series of tests that assert that the objects in the system in fact have the relationship to one another expressed by the graph [14]. For example, in a graphical user interface (GUI) for a word processor program, a software tester may establish a graph that consists of a “File” menu option called “New”, a document window, and the document text. In the graph, the file menu would relate to the document window in the sense that selecting the “New” option would generate a new document window. The document window would be related to the document text in that the document window contains the document text. The tester can then use several different methods for deriving test cases from this graph [15], such as data flow and transaction flow testing described below.
in the intended way, but security vulnerabilities emerge from the unintended consequences from a design or implementation decision that are likely not to be found on the graph. Although a tester could possibly reveal vulnerabilities this way, we contend that graph-based testing would constitute an inefficient method of analysis for finding security problems in a software system.
One type of graph-based testing is data flow modeling. In data flow testing, the objects on the graph are elements of data, and the relationships are the transformations that occur on these objects as the data passes through the system. For example, an “average” object would have a relationship to the sum of each of the members of the average and a relationship to an object containing the total number of elements in the average. The test cases produced for this example would be to assert that the sum object in fact represents the sum of its members, that the total number of elements is correct, and that the average is computed correctly.
Equivalence Partitioning. In equivalence partitioning, the tester divides the input domain of a program into classes of data from which test cases can be derived. Equivalence partitioning creates test cases that reveal classes of errors, thereby reducing the number of test cases that a tester must create [77]. Each equivalence class is a representation of a set of valid or invalid states for the input conditions of a system. According to Beizer, if a set of objects can be linked by relationships that are “symmetric, transitive, and reflexive, an equivalence class is present” [15]. For example, if an input condition specifies the range “0 through 10” for an integer input, then three equivalence classes are created. One invalid equivalence class is numbers less than zero, one valid equivalence class is the range specified, and another invalid equivalence class is numbers greater than 10. A tester would create three separate test cases, each with an input in one of these equivalence classes.
Boundary Value Analysis. In boundary value analysis, the tester selects inputs in a way similar to that in equivalence partitioning. Rather than selecting any element in the equivalence class, boundary value analysis entails selecting those values that border the edges of equivalence classes. Following the example from above, the tester would select input on the boundary, just above the boundary and just below the boundary of each equivalence class. For the boundary at zero, the tester would choose the inputs 0, -1 and 1. For the boundary at 10, the tester would choose 10, 9, and 11.
tester would not conceive of by looking at the range of acceptable input. The same applies to equivalence classes.
Random Testing. In random testing, the inputs used to develop test cases are explicitly random [44]. That is, the method for selecting test inputs in random testing is explicitly not systematic. Random testing is useful because testers can generate test cases using random number generation techniques, and using those variables to drive the execution of the system [44]. Performing testing in this fashion means that the process of selecting test case input requires little effort. However, random testing is less effective at exposing software defects than systematic methods that are designed specifically to reveal defects [44]. Specifically, the challenge in conducting random testing is determining the correct behavior for the system for each set of random inputs. In the usual case, the comparison with correct behavior is difficult because the requirements are imprecise. In the worst case, the required behavior is so complicated that determining the correct output for a set of random inputs creates a new computational problem.
of vulnerabilities implies that random testing will consume more resources to reveal the vulnerabilities than would testing techniques that specifically target those vulnerabilities.
Fuzz Testing. In fuzz testing, a tester chooses inputs that are exceptional, invalid, or unexpected and sends them to the system at random in an automated or semi-automated process [93]. Fuzz testing can be viewed as a specific type of random testing, where the input domain is a set of malicious inputs [8]. Security testers often use fuzz testing to reveal vulnerabilities in a software system, but this process is time consuming and expensive. Security testers also often use fuzz testing to automate the process of penetration testing (see Section 2.5.2), but penetration testing’s effectiveness is dependent on the skill and expertise of its users [31].
unintended ways, such as turning the message sending functionality into a spamming mechanism, sending malicious links to users within the system, or impersonating a different user. This unintended functionality is not often found in the requirements document unless the team has performed an explicit analysis of security requirements (see Section 2.5.4). Black box security testing's role is to provide a security evaluation of the product in its environment. Black box testing techniques like penetration testing can uncover vulnerabilities that are dependent on environmental specifics that other forms of testing cannot [89], however penetration testing is often dependent on the skill of its users to be effective (see Section 2.5.2).
2.5.2 Software Security Assurance Techniques
advice on how to conduct black box security testing, just that it should be done. For example, security experts use their extensive knowledge and experience to attempt attacks on an application in a black box but exploratory and opportunistic way in a process known as penetration testing [9]. Conducting penetration testing on a new or developing system is important since the security threat landscape is constantly evolving and attackers are always developing new methods and techniques of revealing weaknesses in software systems. However, the success of security assurance techniques like penetration testing that occur late in the product's lifecycle vary based on the skill, knowledge, and experience of testers [9].
A recent survey of information security and software development consultants indicates that although 81% of respondents were aware of formal secure development methodologies, only 30% indicated that they had adopted some methodology. The survey also indicates that the top two reasons for not adopting secure software development methodologies are that they are "too time consuming" or require "too many resources" [36].
2.5.3 Using STRIDE to Model Security Threats
STRIDE helps the development team to focus on applicable threat-element pairs. Knowing that a data store is threatened by tampering informs the development team that design decisions should be made to help protect the data store from tampering attacks by performing input validation and other preventions or mitigations.
Development teams use STRIDE when incorporating security principles into the design of the system. However, STRIDE does not provide any methodological advice for the systematic testing of a system’s ability to defend itself against attackers. Making secure design decisions early in the software development process ensures that a development team avoid having to redesign the entire system to address major security concerns. However, as McGraw indicates [99], approximately half of all security problems are implementation bugs as opposed to design flaws (see Section 2.1). That is, half of all security problems occur from problems with the implementation of a given design in the code, rather than an early design decision that STRIDE would have helped avoid. We introduce the notion of testing software systems using security test patterns to help development teams detect both high-level design flaws and code-high-level implementation bugs. STRIDE helps security experts determine the components or aspects of the design of a system that are most likely to be vulnerable, but security test patterns provide software testers with no expertise in security with stepwise instructions for emulating an attacker’s behavior to evaluate whether a system will successfully prevent an exploit from succeeding.
2.5.4 Security Requirements
example: "The system shall send an email message to the administrator containing the new user name, password and the time and date of creation when a new account is created.” Security requirements are often non-functional, meaning they specify criteria that are used the judge the operation of a system, as opposed to functional requirements that define specific functions or behaviors of the system [29].
The CIA Properties. As Lamsweerde explains, a software goal is “an objective the system under consideration should achieve” [57]. Lamsweerde explains that goals are important in the requirements elicitation process because:
• goals provide a precise criterion for the sufficient completeness of a requirements specification,
• goals provide a precise criterion of requirements pertinence, • goals provide the rationale for requirements,
• goals drive the identification of requirements to support them, through goal-based requirements analysis [5], and
• since software requirements are volatile [104], goals provide a higher level entity that software engineers can analyze that is more stable and less likely to change [7]. The ISO defines three important concepts with respect to information security, known as the “CIA Properties”: confidentiality, integrity, and availability [49]. The CIA properties can be considered security goals for any software system. The properties are described as follows:
• Integrity: The system shall not allow the information to be tainted. This does not guarantee the accuracy of the information, but guarantees that the same information that a user puts in will be the information that a user gets out.
• Availability: The system shall provide the information when the user requires the information.
Developing Security Requirements. Lamsweerde provides a technique for developing and analyzing security requirements [58] that is based on anti-goal requirements analysis. According to Lamsweerde, an anti-goal is the goal of any attacker of the system under evaluation. Lamsweerde augments the CIA properties with three additional security goals: Privacy, Authentication, and Non-repudiation for a total of six fundamental security goals. Lamsweerde explains that by negating these security goals, we can generate anti-goals that will capture the attackers’ motivation. Lamsweerde then goes on to say that we can decompose these anti-goals by asking questions such as “Why would an attacker want to achieve this anti-goal?”
results from this anti-goal decomposition: that the attacker can rapidly attempt various account numbers against the same personal identification number. This anti-requirement can then be refined into a security requirement that the system will allow a maximum number of guesses with the same personal identification number or same account number. Likewise, the analyst can convert the attacker’s anti-goal into a new security goal that entails that the system should avoid the thief’s ability to know or guess the personal identification number or account number.
Our method for empirically developing security test patterns is different from the construction of anti-goals because our method entails constructing a security test plan rather than a set of attackers’ goals. We base our security test patterns on recurrent vulnerabilities that are not specific to any particular system, rather than a detailed analysis of the goals for one system and the potential security consequences of the goals for that system as Lamsweerde did [58]. However, a requirements analyst who is conducting anti-goal analysis could use the anti-goal statements developed during the analysis as a natural language artifact. The analyst could then use our methodology to develop security test patterns that would reveal vulnerabilities that have not yet been reported.
Haley et al. [41] provide a framework for representation and analysis of security requirements. Haley et al. indicate that adequate security requirements must satisfy three criteria:
1. definition: one must know what the security requirements are,
3. satisfaction: one must be able to determine whether the security requirements will satisfy the security goals of the system.
Haley et al. define security requirements as constraints on the functionality of the system, where these constraints operationalize one or more security goals. Haley et al. use a similar set of security goals as Lamsweerde [58], again based on the CIA properties, and add another “A” for accountability. The framework that Haley et al. introduce consists of four phases:
1. identifying functional requirements, 2. identifying security goals,
3. identifying security requirements, and 4. constructing satisfaction arguments.
Haley et al. leave the first stage of identifying functional requirements open, allowing the requirements analyst to use any of the established functional requirements elicitation techniques in the literature.
principle shall apply. In this scenario, the analyst would then determine that the system must “achieve separation of duties when managing audit trails.”
Haley et al. [41] explain that the third phase involves writing constraints on the functional requirements that ensure that the system upholds the security goals established in the second phase. Following the above example, if the system must “achieve separation of duties when managing audit trails,” then the analyst would constrain the functional requirements related to managing audit trails to ensure that these requirements maintain the separation of duties. For example, a functional requirement that states “the system shall provide the ability to view audit trails in a tabular format, sorted from latest event to earliest event,” might be constrained to “the system shall only provide the ability to view an manage audit trails to system administrators.”
Chen et al. introduce the System QUAlity Requirements Engineering (SQUARE) methodology for eliciting and prioritizing security requirements [18], [68]. According to Chen et al., SQUARE consists of nine steps:
1. Establish consistent and agreed-upon definitions, 2. Identify safety and security goals,
3. Decide which elicitation techniques to use,
4. Develop artifacts, such as misuse case, templates, or forms, that support the elicitation techniques,
5. Elicit safety and security requirements,
6. Categories the requirements and determine whether they are functional requirements or some other kind of constraints,
7. Perform a risk assessment,
8. Prioritize the requirements using some well-known requirements prioritization technique, and
performing a risk assessment, and discussing the resultant requirements specification with the stakeholders at the end of the elicitation process. SQUARE makes use of other techniques that software requirements researchers have already established, including prioritization techniques, elicitation techniques, and inspection techniques. However, SQUARE provides an overarching methodology for developing and managing security requirements that allows analysts to subject these requirements to the same level of rigor and intensive inspection that they do with functional requirements.
