Integrating model-checking architectural analysis
and validation in a real software life-cycle
Daniele Compare1, Paola Inverardi2, Patrizio Pelliccione2, and Alessandra Sebastiani1
1 Lab. NMS C2, Marconi Mobile, L’Aquila, Italy, {daniele.compare,alessandra.sebastiani}@marconi.com
2 University of L’Aquila, Computer Science Department
Via Vetoio 1, 67010 L’Aquila, Italy
{inverard,pellicci}@di.univaq.it
Abstract. In this paper we describe the use of a model-checking based tool, Charmy, in an industrial software development environment. The goal of the project is to model and analyze the software architecture of a software system currently under development. We have defined and formally checked its overall architecture. By zooming into relevant sub-systems we have been able to identify a set of uncorrect behaviors. We have used an iterative process, where both the architectural sub-systems models and the properties to be checked were defined, checked and re-vised several times.
The results of the experience allow an evaluation of the effort to use the analysis framework in the industrial software development environment and an assessment of the efficacy and the role of the architectural analysis in the software development process. The main relevance of this expe-rience is in the effort to smoothly integrate the use of model-checking techniques in a standard software life-cycle, in particular concerning the discovery and definition of architectural properties.
1
Introduction
In the last years there has been a growing interest for the architectural level de-scription of software systems. New tools and architectural languages have been proposed to specify the Software Architecture (SA) of large scale systems. SA description represents the first, in the development life-cycle, complete system description. It provides at the high abstraction level of components and con-nectors both a description of the static structure of the system and a model of its dynamic behavior. The aim of SA descriptions is twofold: on one side they force the designer to separate architectural concerns from other design ones. On the other, they allow for analysis and validation of architectural choices, both behavioral and quantitative in order to obtain better software quality in an increasingly shorter time-to-market development scenario.
Despite the high level of abstraction very often SA can still be too complex to be managed. A way to tackle system complexity consists of representing the
system through several view points [18, 16, 9]; as a direct consequence, different models are used to represent the different views. The use of views allows to han-dle different aspects of the system separately and to manage the description of real systems. Several slightly different views (logical, module, deployment and so on) are defined in different approaches [16, 9, 4]; different notations, gener-ally graphical, have been introduced for their representation. All the approaches agree that dynamic aspectsare orthogonal to all the views and they allow for analysis and validation of architectural choices. Several works use these dynamic descriptions for testing [5, 21], analysis and model checking [3, 7, 8, 15]. Many specification languages [14, 1, 3, 17] have been introduced to specify and model
system behaviorat the architectural level but there is not a standardized way to represent SA dynamics. In the current industrial practicestate-based machines
and scenarios are the most common used tools to model behavioral aspects: state diagrams describe components behavior while scenarios (message sequence charts or sequence diagrams) identify how they interact. Although very expres-sive, this approach has two drawbacks with respect to analysis and validation.
The first one deals with themodel incompletenessof the system specification: notably in industrial contexts, these models are not completely specified. Thus we can in general rely only on a finite set of scenarios that complement (sub-) systems state models.
The second one is a problem of view consistency: the use of several views facilitates the growing of inconsistencies between views that represent different aspects of the same system [19, 20]. Indeed, state diagrams and scenarios pro-vide alternative views of the system: these views are not independent and can erroneously specify contradictory or inconsistent behaviors.
The approach proposed in Charmy (CHecking Architectural Model consis-tencY) [12, 13, 11] has been thought to deal with these problems. Charmy allows the use of the most common notations employed in the current industrial prac-tice to model behavioral aspects: i.e., state and sequence diagrams (scenarios) used to describe a system software architecture through two different views. Charmy, starting from these (incomplete) dynamic views, synthesizes, through a suitable translation into Promela language (the specification language of the SPIN model checker), an actual SA complete model that can then be executed and verified in the SPIN model checker [24]. The resulting model can then be validated with respect to a set of selected scenarios which are suitably translated in Linear Temporal Logic (LTL) formulae. The validated model can be used as a first prototype to control and eliminate problems of the software architecture model such as deadlock, correctness of properties, starvation, etc.
In industrial contexts like the one participating in this experience, (Unified modelling Language) UML [2] is commonly used for the system architectural description. The model checker SPIN, is a widely distributed software package that supports the formal verification of concurrent systems allowing to analyze their logical consistency by on-the-fly checks, i.e. without the need of construct-ing a global state graph, thus reducconstruct-ing the complexity of the check. It is the core engine of Charmy and it is not directly accessible by a Charmy user.
In this paper we describe the use of Charmy in an industrial software develop-ment environdevelop-ment. The goal of the project is to model and analyze an industrial software system currently under development. We have defined and formally checked its overall architecture. By zooming into relevant sub-systems we have been able to identify a set of uncorrect behaviors. We have used an iterative process, where both the architectural sub-systems models and the properties to be checked were defined, checked and revised several times.
The results of the experience allow an evaluation of the effort to use the analysis framework in the Marconi software development environment and an assessment of the efficacy and the role of the architectural analysis in the software development process. The main relevance of this experience is in the effort to smoothly integrate the use of model-checking techniques in a standard software life-cycle, in particular concerning the architectural properties discovery and definition.
In the literature much work has been directed to improve the model gen-eration phase, little attention has been put in the properties to be proved. In general these are assumed to exist as part of the problem specification. Research efforts in system properties and in the corresponding (temporal) formula mainly concern the ability to informally describe a property and then translate it in a formal formula [23, 10].
Little or no attention is devoted to the problem of propertieselicitation, that is the ability to determine from the software artifacts at hand which are the relevant properties to check for. We can certainly assume that at the require-ments level, explicit stated behavioral requirerequire-ments, like “my system has always to perform twob actions whenever it gets in input anaaction” exist. These are “global” system requirements, that is they characterize the input-output behav-ior of the system. Once a software architecture is specified, it is still possible to check if it guarantees the global functional requirements, but if this is not the case we need to be able to refine the global property into sub-properties that focus on relevant portions of the system software architecture.
In Section 2 we summarize the framework and the way it works. Sections 3, 4, 5 describe the framework applied on the case study. Section 3 describes the system specification of the case study and its main properties. Section 4 shows the modelling in the framework defining the components and its interaction. Subsection 4.1 is a modelling example while 4.2 describe two sub-systems rep-resenting most critical parts of the system. Section 5 shows the analysis process applied to the NICE case study, showing the detected problems and the pro-posed solutions. Section 6 summarizes our experience and presents conclusions and future work.
2
The Framework
The framework we used in the project tries to integrate several tools in order to correctly drive the software architect during the system modelling design phase. The framework involves the UML notation, and the Charmy tool.
UML is the design support used in the industrial context we operate. It is used for all software development phases, in particular as Architectural Description Language (ADL).
Charmy is a tool that helps the software architect to draw architectural state diagrams and scenarios and automatically translates these models in Promela code and in LTL formulae expressing suitable system behavioral properties. Charmy performs the model verification using the model checker SPIN. Details about Charmy’s related algorithms are given in [12].
In the following we summarize how we used the proposed framework starting from the system specification described in UML (use cases, activity diagrams, etc.) and requirement documents in natural language:
1. identify the system components and describe their architectural behavioral aspects through state diagrams;
2. define the significant dynamic interactions among components through se-quence diagrams;
3. use the two above set of views to produce a SA model in Promela code. The sequence diagrams are also separately translated in LTL formulae and used to test the correctness of the SA Promela model;
4. prove the global behavioral properties, stated in the requirements, on this first model of the system;
5. identify critical sub-systems and their relevant properties in order to guar-antee the global properties;
6. refine the sub-systems architecture and iterate the process;
The most critical part of this approach is the selection of the LTL formulae representing the properties that must be satisfied by the system. In a simplistic approach we can think that the set of the LTL formulae can be extracted from the system specification. This is only true for the global behavioral properties. The actual formulae that we prove at the architectural level must be more refined in order to point out critical architectural interactions. There is no automatic way to do this refinement step, which is connected with the identification of the most critical sub-systems that need a deeper modelling and verification. In the following we describe the process followed in the project, trying to highlight the motivations that allowed us to focus on significant sub-systems and to identify relevant properties.
Our approach is based on the construction of an executable prototype of the system software architecture in order to understand its behavior and its features. It is clear that we are not able to detect all potential erroneous behaviors in one verification step. For this reason we iterate the process; at each iteration we focus on a system part refining its specification and identifying other potentially critical sub-parts. To some extent, our approach is similar to extracting test cases. In eliciting test cases a tester focusses on the critical parts of the system in order to verify his intuitions. In the same way we elicit the properties that represent potential flows and check them on the system model by using model checking techniques.
3
The NICE case study
The Naval Integrated Communication Environment (NICE) operates in a naval communication environment. The purpose of the system is to fulfill the following main functionalities:
– provide voice, data and video communication modes;
– enable system aided message preparation, processing, storage, retrieval dis-tribution and purging;
– implement radio frequency transmission and reception, variable power con-trol and modulation;
– provision of remote control and monitoring including detection of equipment failures in the transmission/reception radio chain and the management sys-tem elements;
– provision of a data distribution service;
– implement communications security techniques to the required level of eval-uation, accreditation and certification.
C1 WS CTS PROXY AGENT NICE MS CTS C2 PROXY AGENT VUU Radio, Transmitting Antenna EQUIPMENT
Fig. 1.NICE static software architecture
The NICE Management Sub-system (NICE MS) is the more critical com-ponent. It controls both internal and external communications, supporting the following functionalities:
– fault and damage management; – system configuration;
– security management; – traffic accounting;
– performance management.
The NICE MS complexity and heterogeneity (different hardware and op-erating systems and relative applications) together with its real time context needs the definition of a precise software architecture to express its coordina-tion structure. The system involves several operacoordina-tional consoles that manage the heterogeneous system equipment including the ATM based Communication Transfer System(CTS) through blind Proxy computers. For this reason the high level design is based on a manager-agent architecture that is summarized in Figure 1, where the Workstation (WS) component represents the management entity while the Proxy Agent and the CTS Proxy Agent component represent the interface to control the managed equipment and the CTS respectively. Sub-systems are connected by connectors (C1 and C2) that allow the communication between them.
4
Modelling NICE in the framework
The first step in modelling the NICE system is to define its static SA, as shown in Figure 1. The second step is to define the state diagrams for the components and the scenarios defining components interaction. This second step is part of the usual Marconi software development process, therefore the effort to provide input for Charmy has been minimal. In the following subsection we present some state diagrams and scenarios related to the modelled system.
4.1 An excerpt of the system behaviors
In this Subsection we present only a small example of the modelled system, just to give an idea of the modelling technique.
A1 A2 !fail1VUU_CTS !Recovery1VUU_CTS (a) B1 B2 B4 B3 ?fail1VUU_CTS !fail2VUU_CTS !Recovery2VUU_CTS (b) C1 ?fail2VUU_CTS C2 ?Recovery2VUU_CTS (c) ?Recovery1VUU_CTS
Fig. 2.Components state diagrams
Interested readers can see [22] for a full treatment of the system. Starting from the NICE global model we consider the situation that occurs when the
CTS Proxy Agent detects a CTS or VUU (Voice User Unit) failure together with the relative consequent actions. Note that a VUU is an ISDN voice terminal equipment to access the ship’s operational internal and external communications, including the ship’s telephone system.
In Figures 2.a, 2.b and 2.c are shown the CTS, the CTS Proxy Agent and the WS state diagrams, respectively. To help diagrams comprehension, note that the symbol ? denotes a message reception and the symbol ! denotes a message sending.
In Figure 3 it is represented how these components interact each other through a sequence diagram. Since at present Charmy does not directly interface with UML, architectural state charts and sequence diagrams had to be directly provided. However, this was not a big effort due to the closeness among the UML and the Charmy notations. The state diagrams in Figure 2 and the scenario in Figure 3, obtained with the Charmy editors, are then automatically translated into Promela code and LTL formulae to perform the third step of the process described in Section2. For further details on the complete system verification the interested reader can refer to [22].
WS CTS CTS fail1VUU_CTS fail2VUU_CTS Recovery2VUU_CTS Recovery1VUU_CTS Proxy Agent
Fig. 3.Components interaction sequence diagram
Because of the large system context (26 components state diagram, 85 ex-changed messages), in the following we present the analysis of only two sub-systems. These sub-systems have been identified as highly critical, and therefore eligible for deeper modelling and analysis because of the high risk of mismatch and interoperability issues due to the different nature of their interacting com-ponents (hardware, software, different operating systems).
4.2 NICE Management Sub-system
The NICE Management Sub-system, in Figure 1 surrounded by dashed line, is composed of three interacting components as described at the end of Section 3.
Due to the nature of the system communications, it was decided to separately model and analyze the communication between WS and CTS Proxy Agent and the communication between WS and Proxy Agent. The size of the two sub-systems is limited to 3 components state diagram and 26 messages for the first one and 3 components state diagram and 22 messages for the second one. As we will see later on, the analysis of these models allowed the identification of several potential system communication problems. In the following we briefly describe the two sub-systems and their interactions in order to better understand the achieved analysis results. WS-CTS Proxy Agent communication is based on the SNMPv2 protocol [6] where WS is the manager entity while CTS Proxy Agent is the managed agent one. Messages exchanged are the following:
– Set PDU: the manager asks the agent to change a specified variable of agent MIB;
– Get PDU: the manager asks the agent to report value of a specified variable of agent MIB;
– Notify PDU: the agent acknowledges a received Set/Get PDU to the man-ager;
– Trap PDU: the agent warns the manager about its own MIB changing. WS-Proxy Agent communication is based on a proprietary protocol over LAN. The Proxy Agent is directly connected to the managed equipment and simply converts messages coming from WS in a format that the equipment is able to understand and vice-versa. In doing such activity, it also monitors the equipment state through a timer-based polling. In order to avoid useless network overload, it forwards to the WS only the variations of the equipment state.
5
The analysis process
Let us shortly recall how the analysis process described in Section 2 is carried on. First we analyze the overall NICE system in order to understand its main dynamic features. The analysis results, that we not report here, show the ex-istence of potential dangerous behaviors. Therefore we identify and analyze in detail two sub-systems. These two sub-systems realize the communications into the NICE MS as defined in Subsection 4.2. In particular the first one focus on the possible loss of messages during the communications between the WS and Proxy Agent (Subsection 5.1). The other one allows monitoring the communication be-tween the WS and the CTS Proxy Agent (Subsection 5.2). The interest on the latter model is driven by the fact that the interacting components are hetero-geneous in terms of operating system platforms and are developed by different project partners. Note that the two sub-systems models are still the product of an abstraction process. Some components were leaved out, focusing only on interesting ones and refining the communication among them. For further de-tails the interested reader can see [22]. In the following subsections we explain by means of sequence diagrams the results obtained by using the model checker SPIN. We decide to translate SPIN traces in sequence diagrams to make the analysis output comprehensible to system designers.
5.1 Results for WS-Proxy Agent communication
In this sub-model several anomalous scenarios were verified using a Pentium III 833 MHz personal computer with 256 MB of ram and Windows NT 4.0. The scenario expressed by the sequence diagram of Figure 4.a represents the result analysis for the LTL formula in Figure 5. It outlines how the loss of a message (“Data msg2”) does not allow Proxy Agent to update WS about the state of the managed equipment until a new state change applies on the Equipment itself. In Figure 4 the [State X] labels on each lifeline component represent the local Equipment state view.
Referring to the scenario Figure 4.a, an state inconsistency occurs between WS and the managed Equipment. After the reception of the “set2” message the Equipment changes its state to [STATE B]; after it applies the request, changing its state to [STATE C] while WS still knows [STATE B].
It does not warn the WS because the state is not changed. Equipment Proxy WS set1 set2 ack2 Time to apply the set request [State A] ack1 Polling [State C] Data msg1 Data msg2 It is lost Polling ack . . . . [State C] [State A] [State C] [State B] [State C] [State A] [State B] [State B] [State B] (a) Equipment Proxy WS set1 set2 ack2 Time to apply the set request It is lost Polling Data msg1 Data msg2 It is lost Timeout Not responding1 [State A] [State B] [State C] Polling ack . . . [State A] [State C] [State C] [State C] [State A] [State B] [State B] (b)
Fig. 4.WS P.A. anomalous scenarios
In the sequence diagrams the lost messages are highlighted with a note. In Figure 5 we present a high level description of the LTL formula which describes the scenario in Figure 4.a; we are omitting details about the actual formula syn-tax because a complete description requires the knowledge of the data structures in the Promela code.
The check of the LTL formula shows after a few minutes of processing the existence of a path that fakes the formula. The highlighted problem is the loss
set1 happens in the time t_set1
set2 happens in the time t_set2=t_set1+1 ack2 happens in the time t_ack2=t_set_2+1 ack1 happens in the time t_ack1=t_ack2+1 polling happens in the time t_polling=t_ack1+1 datamsg1 happens in the time t_datamsg1=t_polling+1 datamsg2 happens in the time datamsg2=t_datamsg1+1 polling happens in the time t_polling= t_datamsg2+1 ack happens in the time t_ack=t_polling+1
Fig. 5.LTL formula: an example
of “datamsg2” message and it suggests to introduce a timer which allows the Proxy Agent to resend periodically the equipment state.
The sequence diagram in Figure 4.b shows the system behavior caused by the expiration of the Proxy Agent’s timer due to the loss of the equipment “ack2” message. This implies a “not responding1” message sent from the Proxy Agent to WS which signals an equipment malfunctioning. Note that such timer is set by the Proxy Agent every time that it sends a message to the equipment. The loss of the message sent by the Proxy Agent and containing the new equipment state induces inconsistency. In fact WS continues to believe that the managed Equip-ment is in a not responding state ([STATE B]), even if the managed EquipEquip-ment is correctly working. This situation persists until a new equipment state change occurs. The LTL formula resulting from this scenario was also validated thus indicating a system behavior that has to be differently managed.
In an industrial context it is unfeasible to write by hand the LTL formulae, which are complex ones even in the above abstract version. On the contrary, the sequence diagrams are more intuitive and highly familiar.
Charmy offers to the user an interface to specify scenarios and a set of options on how these scenarios can be interpreted as behavioral properties. For example, among these options, a user can say that a scenario has to be verified for all possible system behaviors, or that it represents just one possible behavior. Other options allow for more sophisticated verifications.
5.2 Results for WS-CTS Proxy Agent communication
In this sub-model the anomalous scenario of Figure 6 was verified using the same hardware configuration described in the previous subsection. Note that the communication between the two entities WS and CTS Proxy Agent is based on a retry mechanism for sending at least N times (e.g. N=3) the same message when no acknowledge or “Msg1 corrupt” is received. Only after the failure of all the N retries the communication can not occur and a “Nack Msg1” message is sent to the NICE MS Manager. In the above scenario after N attempts, WS signals a failed sending since an acknowledge for the sent message is generated
NICE MS Manager WS CTS Msg1 request Msg1 timer expired Msg1 Msg1 corrupt Retry = N Msg1 Msg1 Nack Msg1 Ack timer expired timer expired Proxy Agent
Fig. 6.WS CTS P.A. anomalous scenarios
later by the CTS Proxy Agent. The analysis of this communication protocol allowed us to identify a mismatch: WS believes its request was not accepted while CTS Proxy Agent accepted it. This showed the critical importance of a right timer setting choice. The resulting LTL formula was validated after less than ten minutes of processing.
6
Conclusions and Future Work
In this paper we presented our experience in integrating Charmy in the Marconi software development environment. Only few results about the NICE MS system are reported in this paper: more details on the complete system model can be found in [22]. The goal of the experiment was twofold:
1. to evaluate the effort to use the analysis framework proposed; 2. to evaluate the efficacy and the role of the analysis itself.
Since state and sequence diagrams are commonly used in industry software development practice, it is easy and quick to use the framework, given that Charmy’s inputs subsumes UML diagrams. In order to follow this process no
particular skills is required since the framework feedback is easily comprehen-sible for a software architect. The time required for the verification is strongly dependent on the size of the model. Considering the abstraction used in the case study, the verification time was always acceptable. As far as the efficacy and the role of the proposed analysis is concerned, it must be noticed that the frame-work actually helps the software architect in reasoning about the system. The framework easies the definition of a right architecture and provides an executable prototype to perform analysis. As we discussed in the Introduction, it is not easy to define the properties to be investigated on a given model. In this experience, as one might expect, we saw that the process of properties elicitation and sub-systems identification proceeds incrementally and it is driven by the confidence and system knowledge that the software architect acquires along the modelling process. Properties were always described as set of sequence diagrams, character-izing either wanted or un-wanted behaviors and then automatically translated in LTL formula. In the same way, output traces were offered to the software architect always as sequence diagrams. This allowed a rather straightforward integration with the standard software development process and allowed early identification of critical sub-systems behaviors whose design and implementation should be carefully carried out.
Summarizing the study carried out clearly indicated future work directions: 1. the CHARMY tool should be integrated with existing case environment
based on UML to eliminate input costs;
2. scenarios are a good way to express properties at the architectural level. A comprehensive set of interpretation options has to be provided and their translation to LTL formula has to be completely hidden from the user.
Acknowledgments
The authors would like to acknowledge the Italian M.I.U.R. national project SAHARA that partly supported this work.
References
1. Darwin, an Architectural Description Language. In Web site: <
http://www-dse.doc.ic.ac.uk/research/darwin/ darwin.html>.
2. Rational Corporation. UML Resource Center. UML documentation, version1.3. On-line at:<http://www. rational.com/uml/index.jtmpl>.
3. R. Allen and D. Garlan. A Formal Basis for Architectural Connection. In ACM Transactions on Software Engineering and Methodology, pages 6(3):213–249, July 1997.
4. L. Bass, P. Clements, and R. Kazman. Software Architecture in Practice. SEI Series, Addison-Wesley 1998.
5. A. Bertolino, P. Inverardi, and H. Muccini. An Explorative Journey from Ar-chitectural Tests Definition downto Code Tests Execution. InACM Proceedings, ICSE2001, June 2001.
6. R. community. RFC On-line at: <http://community.roxen.com/developers/
idocs/rfc/rfc1901.html>.
7. D. Compare, P. Inverardi, and A. Wolf. Uncovering Architectural Mismatch in Dynamic Behavior. In Science of Computer Programming, pages 33(2):101–131, February 1999.
8. D. Giannakopoulou, J. Kramer, and S. Cheung. Analysing the behaviour of dis-tributed systems using tracta. In Automated Software Engineering, special issue on Automated Analysis of Software, pages 6(1):7–35, January 1999.
9. C. Hofmeister, R. Nord, and D. Soni. Applied Software Architecture. Addison Wesley, 1999.
10. J. G. Holzmann. The logic of bugs. In Proc. Foundations of Software Engineering (SIGSOFT 2002/FSE-10), 2002.
11. P. Inverardi, F. Mancinelli, H. Muccini, and P. Pelliccione. An Experience in Ar-chitectural Extensions: Active Objects in J2EE. InProc. FIDJI’2002 International Workshop on scientiFic engIneering of Distributed Java applIcations. To appear on LNCS, November 28-29, 2002.
12. P. Inverardi, H. Muccini, and P. Pelliccione. Automated Check of Architectural Models Consistency using SPIN. Inthe Automated Software Engineering Confer-ence Proceedings (ASE 2001). San Diego, California, November 2001.
13. P. Inverardi, H. Muccini, and P. Pelliccione. Checking Consistency Between Ar-chitectural Models Using SPIN. InProc. the First Int. Workshop From Software Requirements to Architectures (STRAW’01), year 2001.
14. P. Inverardi and A. Wolf. Formal Specifications and Analysis of Software Archi-tectures Using the Chemical Abstract Machine Model. InIEEE Transactions on Software Engineering, pages 21(4):100–114, April 1995.
15. N. Kaveh and W. Emmerich. Deadlock Detection in Distributed Object Systems. InProc. of the Joint 7th European Software Engineering Conference and 9th ACM SIGSOFT Int. Symposium on Foundations of Software Engineering, Vienna, Aus-tria. LNCS. Springer Verlag, 2001.
16. P. Kruchten. The 4+1 View Model of Architecture. pages pp.42–50. IEEE Software, 12(6) November 1995.
17. J. Magee and J. Kramer. Concurrency: State models & java programs. Wiley publisher, April 1999.
18. J. Magee, J. Kramer, and D. Giannakopoulou. Behaviour analysis of software architectures. In Proc. First Working IFIP Conference on Software Architecture (WICSA1), San Antonio, Texas, 1999.
19. C. Pons, R. Giandini, and G. Baum. Dependency Relations Between Models in the Unified Process. InProc. IWSSD 2000, November 2000.
20. G. Reggio, M. Cerioli, and E. Astesiano. Towards a Rigorous Semantics of UML Supporting its Multiview Approach. In Proc. FASE 2001, LNCS n. 2029. Berlin, Springer Verlag, 2001.
21. D. Richardson and A. Wolf. Software Testing at the Architectural Level. InSecond International Software Architecture Workshop (ISAW-2), October 1996.
22. A. Sebastiani. Specifica e verifica formale dell’architettura di gestione di un sistema integrato reale per le telecomunicazioni.Tesi di Laurea in Informatica, Universit´a di L’Aquila, Facolt´a di Scienze Matematiche Fisiche e Naturali, October 2002. 23. R. L. Smith, G. S. Avrunin, L. A. Clarke, and L. J. Osterweil. PROPEL: An
Ap-proach Supporting Property Elucidation. InProc. 24th International Conference on Software Engineering (ICSE2002), pages pp.11–21, May 19-25 2002.
24. SPIN. Home page on line at: <http://cm.bell-labs.com/cm/cs/what/spin/
