Detecting Inconsistency in Functional Software Requirements

(1)

Industrial and Manufacturing Systems Engineering

Conference Proceedings and Posters Industrial and Manufacturing Systems Engineering

2005

Detecting Inconsistency in Functional Software Requirements

Gboyega Sanni

Iowa State University

John K. Jackman

Iowa State University, [email protected]

Follow this and additional works at: https://lib.dr.iastate.edu/imse_conf Part of the Operational Research Commons

This Conference Proceeding is brought to you for free and open access by the Industrial and Manufacturing Systems Engineering at Iowa State University Digital Repository. It has been accepted for inclusion in Industrial and Manufacturing Systems Engineering Conference Proceedings and Posters by an authorized administrator of Iowa State University Digital Repository. For more information, please contact[email protected].

Recommended Citation

Sanni, Gboyega and Jackman, John K., "Detecting Inconsistency in Functional Software Requirements" (2005). Industrial and Manufacturing Systems Engineering Conference Proceedings and Posters. 169.

https://lib.dr.iastate.edu/imse_conf/169

(2)

Detecting Inconsistency in Functional Software Requirements

Abstract

Success in software development depends on the availability of complete, consistent, and unambiguous functional software requirements. Inconsistencies in software requirements can propagate problems throughout the development cycle. We introduce the concept of a quantitative measure for detecting inconsistencies, namely, Potential Structural Inconsistency (PSI). This measure is derived from a structural model for a given set of requirements. We show how this measure can be determined using a case study with known inconsistencies.

Keywords

inconsistency, detection, software and requirements

Disciplines

Operational Research

Comments

This proceeding is published as Sanni, Gboyega, and John K. Jackman. "Detecting Inconsistency in Functional Software Requirements." In Proceedings of the 2005 IIE Annual Conference and Exposition. May 14-18, 2005, Atlanta, Georgia. Posted with permission.

(3)

Detecting Inconsistency in Functional Software Requirements

Gboyega Sanni and John Jackman

Industrial and Manufacturing Systems Engineering, Iowa State University, Ames, Iowa, US.

Abstract

Success in software development depends on the availability of complete, consistent, and unambiguous functional software requirements. Inconsistencies in software requirements can propagate problems throughout the development cycle. We introduce the concept of a quantitative measure for detecting inconsistencies, namely, Potential Structural Inconsistency (PSI). This measure is derived from a structural model for a given set of requirements. We show how this measure can be determined using a case study with known inconsistencies.

Keywords: inconsistency, detection, software and requirements

1 INTRODUCTION

Software is an essential asset for information intensive industries such as, finance, aviation, manufacturing, and retail. Government, military, corporate and customer stakeholders invest significant resources in software systems in order to increase profit, automate business processes, and improve competitiveness. Before a software system is developed, stakeholders define a set of functional software requirements. This set of requirements describes the functions that a customer needs in order to achieve their objectives. The process of defining functional requirements can lead to inconsistencies between requirements in the set. An inconsistency can be viewed as two or more requirements that disagree with each other. Early detection and removal of inconsistencies can help reduce costs in downstream activities of the Software Development Life Cycle (SDLC). For example, the European Space Agency’s Ariane 5 rocket exploded [9] during launch, resulting in a loss more than $7 billion. The explosion was attributed to an attempt in the software to convert a floating-point number into a 16-bit integer without checking for overflow. Though it has not been proven that inconsistencies in functional requirements caused this incident, it is certainly plausible that inconsistent functional requirements can lead to inconsistencies in software modules.

One approach for detecting inconsistencies is to model software requirements as goals and use goal monitors to detect inconsistencies [14]. A limitation with this approach is that software requirements must be specified as goals, whereas most requirements are defined in natural language. Hunter [10] proposed a novel approach of detecting inconsistencies using formal logic. Proof theorems were used to determine whether requirements are logically inconsistent. Zowghi et al [20] translated software requirements defined in natural language into propositional logic.

The propositional logic is then validated with a theorem prover.

Our premise is that there are different types of inconsistencies and that their detection is associated with the type of representation used for the software requirements though overlaps may exist. If this is true, then our expectation is to find logical inconsistencies using Hunter or Zowghi’s methods. We propose a method that examines the structural consistency of a set of requirements. In our method, functional software requirements are transformed into a structural representation, which is then analyzed for any structural inconsistencies. We introduce the concept of potential, partial or relative inconsistency in this context. In the next section, we show how the requirements can be modeled from a structural perspective. This is followed by a description of the methods used to examine consistency in the model. Finally, we conclude with a well-known example from the literature and discuss the implications of our method in detecting inconsistencies.

2 MODEL FORMULATION

The term, inconsistent, has been defined as “If a reason, idea, reason, idea, opinion, etc. is inconsistent, different parts of it do not agree, or it does not agree with something else” [1]. Suppose that software requirements described with natural language are represented as a set of statements (written in English) that describe the functions of the software. Zowghi [20] has shown that it is possible to transform a set of statements in English into a corresponding set of logical statements. Logical inconsistencies are detected using an automated theorem prover ([4]; [11]). Using a similar approach, we transform a set of English statement into a set of corresponding structural models. A structural inconsistency occurs if different elements of the structural representation do not agree with each other. A disagreement is defined as the existence of dissimilarity between two or more structural elements. Potential

(4)

structural inconsistency (PSI) is specified as the degree to which the elements of two or more structures disagree, hence the use of the term “potential.”

Some overlap in elements must exist before inconsistency can be detected in a set of requirements ([16]; [17]; [12]).

Overlap is a measure of the similarity between two structures. For two identical structures, we can assign a value of 1 for similarity, since there is complete overlap from wholly shared or identical elements. When the structures are completely different, the similarity is 0 since there is no overlap. If we alter elements in one of two identical structures, then we introduce a potential structural inconsistency (PSI). We need a measure of structural similarity before a PSI can be detected. The degree of inconsistency indicates relative, partial or potential structural inconsistencies.

3 METHODOLOGY

3.1 TRANSFORMATION OF FUNCTIONAL SOFTWARE REQUIREMENTS INTO STRUCTURAL MODELS

We use an Entity Relationship Model (ERM) as the basis of our structural model. Previous attempts to transform text in natural language into ERMs used parsers to identify the Parts Of Speech (POS) and then mapped the POS to elements in the final model ([2]; [18]). One of the difficulties is that parsers tend to perform poorly when identifying the POS. Brill [3] showed that a rule-based tagger could be used to obtain the POS tags of words in statements with the context (i.e., word sequence) still preserved.

Using a similar approach as Brill, we can generate a structural model given a sequence of POS in a set of requirement statements. The ERM described by Chen [5] consists of cardinalities, entities, attributes and relationships. Chen [6] proposed that each member of the ERM could be mapped directly to English sentences.

Though Chen’s recommendations are simplistic, they can serve as a starting point for generating the model.

Validation of the model is necessary to show that the original requirements have been accurately represented.

3.2 SET THEORY REPRESENTATION OF ENTITY RELATIONSHIP MODEL

Let E = {e1, e2, e3, …, en} be the set of n entities in a model. Entity i has an attribute set, Ai={ai1, ai2, ai3, …, aim}, describing the entity. The relationships between entities are defined by a set of Boolean variables, R = {ri1, ri2, ri3,

…, rnn}, indicating whether a relationship exists. The cardinality of the relationships is a corresponding set of tuples, C = {ci1, ci2, ci3, …, cnn}, that show the number of instances that are involved of each entity type and cij =(νi, ν_j), where, ν_i is the number of instances for entity i. An ERM, Γ, is defined as a three-tuple given by

Γ = (E, R, C). (1)

3.3 DETECTION OF POTENTIAL STRUCTURAL INCONSISTENCY IN STRUCTURAL MODELS Based on the definitions of inconsistencies related to disagreement and similarity, the detection of PSI is indicated in part by the measure of similarity between two ERMs. Two measures of similarity were considered based on Tversky’s measure of similarity [19] and Edit Distances [13]. We used Tversky’s measure due to its simple set- based representation. Rodriquez [15] used Tversky’s measure of similarity in finding semantic similarity among entity classes. The measure uses a similarity function for determining similar cardinalities, entities, attributes and relationships at different levels. Tversky’s measure of similarity is used as a metric in the determination of PSI in functional software requirements.

Let X and Y be defined as two ERMs according to (1). We can perform set operations on elements of X and Y to determine which elements the models have in common and which ones are different. Let

Φ

K

(

^X

Υ

^Y

)

be all elements of type K (e.g., entities) and

Φ

K

(

^X

Ι

^Y

)

be the set of elements of type K common to both models. We can define those elements in X but not in Y as

Φ

K

(

^X

−

^Y

) = Φ

K

(

^X

Υ

^Y

) − Φ

K

( )

^Y . Similarly for Y we obtain

(

^Y ^X

)

K

(

^X ^Y

)

K

( )

^X

K

− = Φ − Φ

Φ Υ

. Using the elements from sets E, R, and A, we can obtain composite cardinality functions based on equal weights for the three sets giving us

(

^X ^Y

) (

^X ^Y

) (

^X ^Y

) (

^X ^Y

)

F

Ι = Φ

_E

Ι + Φ

_R

Ι + Φ

_A

Ι

,

(5)

(

X Y

) (

X Y

) (

X Y

) (

X Y

)

F

− = Φ

_E

− + Φ

_R

− + Φ

_A

−

, and

(

Y X

) (

Y X

) (

Y X

) (

Y X

)

F

− = Φ

_E

− + Φ

_R

− + Φ

_A

−

.

We can then define the similarity of X and Y as the difference between the number of common elements and those elements that are not in common, giving us

(

X Y

)

F

(

X Y

)

F

(

X Y

)

F

(

Y X

)

S

, = γ

₁

Ι − γ

₂

− − γ

₃

−

where, γ1, γ2 and γ3 ≥ 0, are assignable weights assumed to be γ1=γ2=γ3 =1. The weights are used to assign importance to each function depending on the context of similarity. To normalize ^S

(

^X^,^Y

)

between 0 and 1, it is necessary to determine the upper and lower bounds of ^S

(

^X^,^Y

)

. The upper bound corresponds to ^min

(

^{X ,}^Y

)

which would be the largest number of elements that X and Y could have in common. Likewise, the lower bound corresponds to −

(

^X +^Y

)

representing the condition when all elements are different. The normalized similarity (i.e., PSI) is given by

( ) ( ( ) )

( ) ( ( ) ) ( )

(

X Y

) (

X Y

)

Y X Y X S Y

X Y

X

Y X Y

X Y S

X + +

+

= + +

−

− +

−

= −

Ψ min ,

) , ( ,

min ) ,

, ( .

4 CASE STUDY

4.1 OVERVIEW

The London Ambulance Service (LAS) [7] has been used as a case study in requirements engineering. When the LAS system was deployed on October 26, 1992, many problems occurred. For example, a subsystem (Automatic Vehicle Locating System (AVLS)) failed to track the location and status of some of the dispatch units [8]. The system allocated multiple vehicles to the same incidents.

A record series of fatal incidents lead to the termination of the LAS dispatch system. One fatal incident was due to the late arrival of an ambulance. Fatalities were attributed to the LAS and the heavy volume of calls and messages that besieged the system. The report shows that some areas were not fully defined in the Software Requirements Specifications (SRS). The requirements used by Zowghi [20] are shown in Table 1:

Table 1: Sample Requirements

Requirements ID 4.1.1 Statements

IRC.1 A medical emergency is either an illness or an accident

IRC.2 When an operator receives a phone call concerning a medical emergency, the operator should dispatch the nearest available ambulance

IRC.3 When an operator receives a phone call concerning a non-medical emergency, the operator should not dispatch an ambulance and he should transfer the phone call to another service.

OM.1 When an operator receives a phone call, the operator should dispatch the nearest available ambulance.

OM.2 When an operator receives a phone call, if an ambulance is not the nearest available, then the operator should not dispatch that ambulance.

(6)

4.2 ANALYSIS OF THE CASE STUDY

Using our set notation, we can define Γ for IRC.2 as shown in Tables 2, 3, and 4.

Table 2: IRC.2 Relationship Matrix

operator call emergency Ambulance

operator receives should dispatch

call is received by concerning emergency is the subject of

ambulance is dispatched by

Table 3: IRC.2 Cardinality Matrix

operator call ^emergency ^ambulance

operator M M

call ^{M M}

emergency M

ambulance M

Table 4: IRC.2 Attribute Matrix Entity

operator call emergency ambulance Attributes 1. phone 1. medical 1. nearest

2. available

After performing the same operation for OM.1, we can find Ψ

(

^{X ,}^Y

)

for the pair IRC.2 and OM.1 as follows.

(

^X ^Y

)

^{^operator^,^call^,^ambulance^}

E =

Φ Ι

(

X Y

)

{receives,shoulddispatch}

R =

Φ Ι

(

^X ^Y

)

^{^phone^,^nearest^,^available^}

E =

Φ Ι

(

X,Y

)

=3+2+3=8 F

(

^X ^{− Y}

)

⁼³

F

(

Y− X

)

=0

F

(

^X^,^Y

)

⁼⁸⁻³⁻⁰⁼⁵

S

(

,

)

8

min X Y = Y = 19 8 11+ =

= + Y X

( ) . 89

19 8

19 , 5 =

+

= + Ψ

X Y

PSI was determined for each pairing of requirements as shown in Table 5.

(7)

Table 5: Potential Structural Inconsistency

Pair

Ψ(X, Y)

IRC.2|OM.1 0.89

OM.1|OM.2 0.8

IRC.1|OM.1 0

IRC.1|OM.2 0

The magnitude of PSI suggests the order in which the requirements should be examined and may also imply importance. The IRC.2 and OM.1 pair yields the highest PSI value, suggesting that this pair of requirements should be reexamined. On closer examination of the two requirements, OM.1 could be interpreted as an ambulance being dispatched whenever a phone call is received (e.g., wrong number or information request). Another interesting observation is that those pairs with IRC.1 consistently have the lowest values. Therefore, this implies that IRC.1 is the requirement with the least PSI.

4.3 CONCLUSION

The potential for structural inconsistency in functional software requirements presents new opportunities in requirements engineering, inconsistency detection, conceptual modeling, and metrics in reasoning about PSI. It confirms that numerous forms of representation can be used to reason about different types of inconsistency. Our initial results indicate that PSI could be used to prioritize requirements. Closely related to the concept of PSI is the possible reexamination or revision of requirement statements. PSI can provide the basis for a more sophisticated requirement analysis model that can be used in software engineering practice.

References

[1] Cambridge Dictionary, http://dictionary.cambridge.org, 01/02/05

[2] Ambriola V. and Gervasi V., (1997), “An Environment for Cooperative Construction of Natural-Language Requirements Bases”. Proc. 8th International Conference on Software Engineering Environments, Los Alamitos, April 1997, pp 124-130.

[3] Brill E., (1995), “Transformation-Based Error-Driven Learning and Natural Language Processing: A Case Study in Part of Speech Tagging”, Computational Linguistics, Vol. 21(4), pp. 543-565.

[4] Chechik, M., and J. Gannon, (2001), "Automated Analysis of Consistency Between Requirements and Design"

IEEE Transactions on Software Engineering, (27)(7), pp. 651-672.

[5] Chen P.P., (1976), “The Entity-Relationship Model: Toward a Unified View of Data”. ACM Trans. On Database Systems, Vol. 1(1), pp. 9-36.

[6] Chen P.P, (1983), “English Sentence Structures and Entity-Relationship Diagrams”. Information Sciences, Vol.

29, pp. 127-149.

[7] Flowers, Stephen (1996), "Software Failure: Management Failure", Chichester: John Wiley and Sons.

[8] Finkelstein A. and Dowell J., (1996), “A Comedy Of Errors: The London Ambulance Service Case Study,” Proc.

8^th International Workshop on Software Specification & Design, pp. 2-4.

[9] Gleick, J. (1996) “A bug and a crash,” The New York Times Magazine, December 1, 1996.

[10] Hunter A. and Nuseibeh B., (1995), "Managing Inconsistent Specifications: Reasoning, Analysis and Action", Department of Computing Technical Report, DoC 95/15, Imperial College, London, UK, October 1995.

[11] Mattolini, R. and Nesi, P., (2001), “An Interval Logic For Real-Time System Specification”, Software Engineering, IEEE Transactions, Vol. (27)(3), pp. 208-227

(8)

[12] Nuseibeh B. A., Easterbrook S. M. and Russo A., (2001), “Making Inconsistency Respectable in Software Development,” Journal of Systems and Software, Vol. 58(2), pp. 171-180.

[13] Ristad, E., and Yianilos, P., (1998), “Learning String Edit Distance,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20(5), pp. 522–532.

[14] Robinson W. N. and Pawlowski S.D., (1999), “Managing Requirements Inconsistency with Development Goal Monitors,” IEEE Transactions on Software Engineering, Vo. 25(6), pp. 816 – 835.

[15] Rodriguez, A., Egenhofer, M., (2003),“Determining Semantic Similarity among Entity Classes from Different Ontologies”, IEEE Transactions on Knowledge and Data Engineering, Vol. 15 No. 2., pp. 442-456.

[16] Spanoudakis G. and Finkelstein A., (1997), “Reconciling Requirements: A Method for Managing Interference, Inconsistency and Conflict,” Annals of Software Engineering, Vol. 3, pp. 433-457.

[17] Spanoudakis G., Finkelstein A. and Till D. (1999) “Overlaps in Requirements Engineering,” Automated Software Engineering Journal, Vol. 6(2), pp. 171-198.

[18] Tjoa A.M., Berger L., (1993), “Transformation of Requirement Specification Expressed in Natural Language into an EER Model,” Proc.12th International Conference on ER Approach, Springer-Verlag, Berlin, pp. 206-217.

[19] Tversky A., (1977), “Features Of Similarity,” Psychological Reviews, Vol. 84 (4), pp. 327-352.

[20] Zowghi D., Gervasi V. and McRae A., (2001), “Using Default Reasoning to Discover Inconsistencies in Natural Language Requirements,” Proc. of the 8th Asia-Pacific Software Engineering Conference, December 2001, pp. 133-140.

(9)

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.