A Case-Based Approach for Reuse in Software Design
4.5 Verification and Evaluation Module
The verification and evaluation module comprises two distinct phases: verifying a class diagram, and evaluating it. The next subsections describe each one of these functionalities.
4.5.1 Verification
The main idea of verification in REBUILDER is to identify errors in class diagrams applying four types of knowledge sources: design cases, WordNet, verification cases, and the designer.
CHAPTER 4. CBR Engine
Designer requests a verification action
REBUILDER requests a verification action REBUILDER generates
a new diagram
Package Verification
Attribute Verification
Relation Verification Method Verification Sub-Package
Verification
Figure 4.40: The verification process in abstract.
Not only REBUILDER generated diagrams can be verified, but also designer-created diagrams (see figure 4.40 for an abstract description of the verification pro-cess). The designer has only to select the diagram to be checked and activate the verification command to perform it. The designer can also select an option in the settings menu for automatic verification of each solution generated by REBUILDER.
Verification is performed on five diagram components: packages, object names, rela-tions, attributes and methods. Verifying a diagram component consists on determin-ing it’s validity (true or false), which is to say, if the object is correct or if it is invalid or incoherent within the diagram. In case of being correct it stays in the diagram, otherwise the object can be deleted from the diagram.
An important implementation aspect is that verification cases (which are cases that represent success or failure verification situations) are stored locally in the de-signer client. Thus, each dede-signer has her/his library of verification cases, which personalizes the system. This is important, since each software designer has a dif-ferent way of modelling the problem being addressed. Starting from this assumption we decided that the case base of verification cases would be local to each client (or designer). This assumption can be in the origin of some limitations when dealing with development teams. Notwithstanding, we still defend our point of view by say-ing that, from our experience, most of the times team development at design level is performed by one or two individuals, which then discuss their designs with the remaining of the team. Also, only one or two project members are responsible for using the CASE tool in modelling the system. Nevertheless, there are at least two ways of addressing this problem. Centralizing all the verification cases in one case base accessible to all members of the team, which would imply the maintenance of
such a case base, or impose project-wide agreements, which are always difficult to make. So, both solutions have pros and cons. In the next subsections we describe the different verification phases.
Package Verification
The package verification checks a class diagram starting with the diagram relations, then checking each object’s attributes and methods, finally checking the sub-packages, recursively calling the package verification functionality.
Name Verification
Name checking applies only to packages, classes and interfaces, and is performed when REBUILDER needs to assign a synset to an object. REBUILDER reasoning mechanisms depend on the correct assignment of synsets to objects, which makes this verification very important. REBUILDER makes a morphological and compo-sitional analysis of the object’s name, trying to match it to WordNet words. The disambiguation starts by extracting from WordNet the synsets correspondent to the object’s name. This requires the system to parse the object’s name, which most of the times is a composition of words. REBUILDER uses specific heuristics to choose which word to use. For instance, only words corresponding to nouns are selected, because usually objects correspond to entities. A morphological analysis must also be performed, extracting the regular noun from the word. After this, a word or a composition of words has been identified and will be searched in WordNet. The result from this search is a set of synsets. From this set of synsets, REBUILDER uses the disambiguation algorithm to select one synset, which is supposed to be the correct one. Once it finds a match it can easily get the list of possible synsets for the given name. Now, two things can happen: the designer selects the correct synset, or RE-BUILDER uses a word sense disambiguation method to do it (it is up to the designer to decide).
CHAPTER 4. CBR Engine
Relation Verification
Relation checking is based on WordNet, design cases, and relation verification cases, which are cases describing successful and failure situations in checking a relation validity. These cases are described by:
• Type of Relation {association, generalization, realization, dependency}.
• Multiplicity {1-1, 1-N, N-N}.
• Source Object Name.
• Source Object Synset.
• Destination Object Name.
• Destination Object Synset.
• Outcome {Success or Failure}.
Two relation verification cases c1 and c2 match if:
RelationT ype(c1) = RelationT ype(c2) ∧ M ultiplicity(c1) = M ultiplicity(c2) ∧
SourceObject(c1) ≡ SourceObject(c2) ∧ DestObject(c1) ≡ DestObject(c2) (4.39)
where RelationType(c), Multiplicity(c), SourceObject(c) and DestObject(c) are, re-spectively, relation type, multiplicity, source object, and destination object of verifi-cation case c. Two objects, o1 and o2 are said to be equivalent if:
o1≡ o2⇔£
Synset(o1) 6= ∅ ∧ Synset(o1) 6= ∅ ∧ Synset(o1) = Synset(o1)¤ h£ ∨
Synset(o1) = ∅ ∨ Synset(o2) = ∅¤
∧ N ame(o1) = N ame(o2) i
(4.40)
where Synset(o) gives the synset of object o and Name(o) yields the name of object o.
All the knowledge sources are used for validating the relation being inspected.
The relation verification algorithm is detailed in figure 4.41.
The algorithm starts by searching a verification case that matches the relation being verified. If a successful verification case is found, then the relation is considered valid and exits. If a case is found but it has a failure outcome, then the relation is considered invalid and it is removed from the diagram. Otherwise the algorithm
1. Search for an equivalent verification case in the library.
2. IF found and outcome is Success THEN 3. Consider the relation valid and exit.
4. IF found and outcome is Failure THEN
5. Consider the relation invalid and delete it from the diagram, and exit.
6. IF not found THEN 7. Continue.
8. Search for an equivalent relation in WordNet.
9. IF found THEN
10. Consider the relation valid, add a new successful verification case and exit.
11. ELSE 12. Continue.
13. Search for an equivalent relation in the design cases.
14. IF found THEN
15. Consider the relation valid, add a new successful verification case and exit.
16. ELSE 17. Continue.
18. Ask the designer the relation validity.
19. IF designer considers relation valid THEN
20. Consider the relation valid, add a new successful verification case and exit.
21. Consider the relation invalid, add a new failure verification case and exit.
Figure 4.41: The relation verification algorithm.
searches other knowledge sources. Next the algorithm searches for a similar relation in WordNet, if it founds one, then the relation is considered valid and exits. Otherwise, the design cases are searched for similar relations that could validate the relation being verified. If the relation’s validity could not be determined then the algorithm asks the designer about the relation’s validity.
A WordNet equivalent relation is a relation between two synsets in which one of them is the source object synset and the other is the destination object synset.
A design case comprises an equivalent relation if it has two objects connected by a similar relation to the one being investigated, and with equivalent source objects and destination objects. Retrieval of verification cases is based on two steps: first on the relation type, and then by source object name. If there is more than one equivalent case, the outcome that is common to more cases is chosen as the correct outcome. In case of a draw, the system retrieves the newest case.
Attribute Verification
Attribute checking is based on WordNet, design cases, and attribute verification cases, which are cases describing successful and failure situations in checking an attribute validity. These cases have the following description:
• Object Name.
CHAPTER 4. CBR Engine
• Object Synset.
• Attribute Name.
• Outcome {Success or Failure}.
Two attribute verification cases vc1 and vc2 match if:
• The Object’s names and synsets are the same, and attributes’ names are also the same, or
• one of the synsets is empty and their objects’ names and attributes’ names are the same.
As in relation verification, all the knowledge sources are used for validating the at-tribute being inspected. The order of search and the algorithm that is used are the same as in relation verification (adapted to the attribute situation). A WordNet equivalent attribute is represented by a substance-of, member-of or part-of relation between the synset of the object being inspected and every possible synset of the attribute’s name. A design case comprises a similar attribute, if there is an object with the same synset and name comprising an attribute with the same name as the attribute being inspected.
Retrieval of attribute verification cases is based on two steps. First the algorithm searches the verification cases based on object name, searching verification cases with the same object name. Then from these cases the algorithm selects only those with the same object synset. If there are more than one case in this situation, then the algorithm assesses the attribute as valid if there are more successful than failure cases, or invalid otherwise. In case of equal number of cases, then the algorithm does not decide about the attribute’s validity and asks the designer. The way cases are indexed in the verification case library is depicted in figure 4.42. As can be seen, verification cases are indexed by name and by synset, which enables a verification case to be retrieved by the object (or objects) involved in the verification process.
Method Verification
Method verification is similar to the attribute verification with the exception that WordNet is not used as a knowledge source, being replaced by an heuristic rule. The
VerificationCase1 Name1
Name2
NameN
…
VerificationCase2
VerificationCaseM
Synset1 Synset2
SynsetK
…
Figure 4.42: The indexing of verification cases (attribute and method).
heuristic used is: if the method name has a word that is an attribute name or a neighbor class name, then the method is considered valid. Method verification cases describe successful and failure situations in checking a method validity. These cases have the following description:
• Object Name.
• Object Synset.
• Method Name.
• Outcome {Success or Failure}.
Two method verification cases c1 and c2 match if:
• The objects’ names and synsets, and methods’ names are the same,or
• one of the synsets is empty and their names and methods’ names are the same.
As in relation verification, all the knowledge sources are used for validating the method being inspected. The order of search and the algorithm used is the same as in relation verification (adapted to the method situation and with WordNet replaced by the heuristic that was referred before). A design case comprises a similar method, if there is an object with the same synset and name comprising a method with the same name as the method being inspected. Retrieval of method verification cases is performed in the same way as in attribute verification cases.
CHAPTER 4. CBR Engine
Figure 4.43: The UML class diagram used as problem for the verification example.
4.5.2 A Verification Example
This subsection illustrates the verification with an example used before. Suppose that the designer uses the diagram presented in figure 4.43 as a problem for the design composition module, resulting in a new diagram. Part of this diagram is presented in figure 4.44, which shows some inconsistencies. For instance: the generalization between Teacher and Timetable, or the method addStudent in Timetable, or the attribute studentID in class Lecture.
The verification process starts by checking the package containing the new dia-gram. For illustration purposes we will describe the process by checking only the diagram elements considered invalid, the ones mentioned before. When the verifica-tion process reaches the generalizaverifica-tion between Timetable and Teacher, the system first searches in the verification cases for an equivalent case, suppose that it does not exist. Then it searches in WordNet for an is-a relation between the Timetable synset and Teacher synset, suppose also that it does not exist. Then searches the design cases that do not have any similar relation, and finally asks the designer for a validation on this relation, which s/he answers as invalid. Then the system adds a new relation verification case comprising: [Generalization, 1-1, Teacher, 1087564767, Timetable, 105441050, Failure], and deletes the relation. The next time the system finds an identical relation it will be considered invalid and will be deleted from the design. Now suppose the system is checking the studentID attribute of class Lecture.
The attribute verification process will search the case library of attribute verification
7Synsets are identified by nine digit numbers.
Figure 4.44: Part of the class diagram resulting from design composition.
cases and it finds an equivalent case: [MathLecture, 106035418, studentID, Failure].
The system considers the attribute invalid and deletes it from the diagram. Finally the system checks method addStudent from class Timetable and the method checking heuristic does not applies - the design cases have no similar method - then it asks the designer, which considers the method invalid. The method verification algorithm will delete the method and add a new method verification case: [Timetable, 105441050, addStudent, Failure].
4.5.3 Evaluation
The evaluation phase provides an assessment of the design characteristics based on software engineering metrics. It provides the designer with some properties of the design diagram. They are divided in two categories: software metrics for object oriented designs, and design statistics. The software metrics used are:
• Class/Interface/Package complexity (class complexity is defined as the sum of the number of methods and attributes; interface complexity is the number of at-tributes; package complexity is the sum of complexity of the objects it comprises ,sub-packages, classes and interfaces).
• Number of children of a Class/Interface.
• Depth of inheritance tree of a Class.
• Coupling between Classes/Interfaces: number of non-inheritance related couples of an object with other objects.
• Average of Class/Interface number of children by package.
• Average of Class/Interface depth of the inheritance tree by package.
CHAPTER 4. CBR Engine
Design statistics are (by package):
• Number of sub packages.
• Number of relations.
• Number of classes.
• Number of interfaces.
• Average number of classes by package.
• Average number of interfaces by package.
• Average number of relations by package.
• Average number of attributes by class.
• Average number of methods by class.
• Average number of methods by interface.