The construction of a formal model of a medical device and the use of formal verifica- tion technologies can help system designers to gain confidence that the device performs the required functions under all the stated conditions, thus enhancing patient safety.
In the present work we have described a framework that facilitates modeling and simulation of cyber-physical systems. Software functionalities are modeled in the PVS theorem proving system. The characteristics of the plant are modeled in Simulink. Simulation of the overall cyber-physical system is obtained through co-simulation of the two models. The integrated simulation facilitates validation of the models, lightweight formal analysis based on simulation and testing, and full formal analysis of the software based on assume-guarantee reasoning [33, 19] techniques.
The capabilities of the framework were demonstrated with an example based on a pacemaker-heart system. A timed automata specification of the pacemaker software is developed in PVS according to identified modeling patterns. The framework en- ables integrated simulation of the PVS pacemaker model with a Simulink heart model built on medical domain-specific knowledge. Integrated simulation allows software engineers to demonstrate the functionalities of the pacemaker software, and discuss hypotheses about its behavior for different physiological parameters of the patient. On the other hand, the correctness of the pacemaker design can be analyzed using PVS. This includes: well-formedness of the design (coverage of conditions, disjointness of conditions, type correctness), and analysis of design requirements.
An important aspect related to the generality of the framework is how easy is to extend the framework to support modeling and analysis tools other than PVS and Simulink. The question boils down to discussing the generality of the PVSio-web ar- chitecture used as a basis to build the co-simulation platform. The backbone of the ar- chitecture uses a standard WebSocket protocol for communication of simulation events and data. In fact, the PVSio environment of PVS is wrapped within a web-server re- sponsible for converting the native read-eval-print loop of PVSio into a tool-neutral event-based service. To do this conversion, the PVSio wrapper implements four main functions: start, which spawns a process running a PVSio instance; sendCommand, which translates simulation events encoded in JSON format into PVS expressions that can be evaluated in PVSio; processData, which translates the results returned by PV- Sio into tool-neutral JSON objects; and kill, which terminates the created process. If developers want to connect another tool providing an interactive read-eval-print loop to the PVSio-web infrastructure, they need only to re-implement these four interface functions of the wrapper.
Two important lessons were learned from this work, one related to the utility of co- simulation, and another related to challenges in the implementation of co-simulation engines.
• Co-simulation proved an effective technology for the analysis of extensive mod- els. The heart model considered in our case study was developed and validated by others in [18]. It included over 200 functional blocks. Translating such a large model in PVS or any another formal methods language would have been impossible in a reasonable amount of time with the available resources. Using co-simulation, we were able to use the heart model as-is, to validate the PVS model of the pacemaker and perform lightweight formal verification of system- level properties of the overall pacemaker-heart system.
• The co-simulation engine implemented in our framework uses ad-hoc APIs, and each co-simulated model needs to be instrumented with those APIs. This is not an ideal situation, as developers need to re-instrument the models if a different co-simulation engine is to be used. Moving towards a standard API such as that defined in the Functional Mock-up Interface (FMI) would bring clear benefits under this perspective. We are currently extending the PVSio-web co-simulation engine in this direction.
Being focused on co-simulation, we could dedicate only limited space to demon- strating how to carry out formal verification of safety requirements for the pacemaker model in PVS. Given that the heart model is not developed in PVS but in Simulink, an assume-guarantee reasoning is necessary to perform verification of system-level properties. An example such property is: “For any bradycardic episode, the pace-
maker ensures that cardiac cycles occur at the correct rate.” With assume-guarantee,
the verification effort allows developers to prove that the pacemaker model guarantees the desired behavior of the pacemaker-heart system under suitable assumptions on the heart model. Formalizing these assumptions will be the object of further work.
An interesting research strand that is worth exploring as future work relates to the use of co-simulation and formal verification technologies for the analysis of cyber-
threats and security-related properties. The specific case study considered in this paper is particularly relevant to such research strand, as pacemakers can be configured and re-programmed using low-range wireless technologies, and various pacemaker manu- facturers have discovered latent vulnerabilities in their pacemakers after the pacemaker was placed in the market. Device recalls have been carried out by manufacturers be- cause these vulnerabilities, if exploited, could lead to permanent impairment or life- threatening injury for the patients (see [62]).
Acknowledgment
We would like to thank Alexandru Mereacre (University of Oxford), who helped us with the Simulink model of the heart, and César Muñoz (NASA Langley Research Center), that provided support for technical aspects of the PVSio animation environ- ment. Piergiuseppe Mallozzi contributed to this paper with his Master’s thesis. We also thank the anonymous reviewers for their valuable comments. Paolo Masci is fi- nanced by the North Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, and through the European Regional Development Fund (ERDF) within Project “NORTE-01-0145-FEDER-000016”.
References
[1] Rajeev Alur and David L. Dill. A theory of timed automata. Theoretical Com-
puter Science, 126(2):183–235, 1994.
[2] Myla Archer. TAME: Using PVS strategies for special-purpose theorem proving.
Annals of Mathematics and Artificial Intelligence, 29(1):139–181, 2000.
[3] Ralph-Johan J. Back, Abo Akademi, and J. Von Wright. Refinement Calculus:
A Systematic Introduction. Springer-Verlag New York, Inc., Secaucus, NJ, USA,
1st edition, 1998.
[4] G. Behrmann, A. David, K.G. Larsen, J. Hakansson, P. Petterson, Wang Yi, and M. Hendriks. UPPAAL 4.0. In Third International Conference on Quantitative
Evaluation of Systems (QEST 2006), pages 125–126, Sept 2006.
[5] Gerd Behrmann, Alexandre David, and Kim G. Larsen. A Tutorial on UPPAAL. In M. Bernardo and F. Corradini, editors, International School on Formal Meth-
ods for the Design of Computer, Communication, and Software Systems, SFM- RT 2004. Revised Lectures, volume 3185 of Lecture Notes in Computer Science,
pages 200–237. Springer Verlag, 2004.
[6] Johan Bengtsson and Wang Yi. Timed automata: Semantics, algorithms and tools. In Lectures on concurrency and Petri nets, pages 87–124. Springer, 2004. [7] Cinzia Bernardeschi and Andrea Domenici. Verifying safety properties of a non-
linear control by interactive theorem proving with the Prototype Verification Sys- tem. Information Processing Letters, 116(6):409–415, 2016.
[8] Cinzia Bernardeschi, Andrea Domenici, and Paolo Masci. Integrated simulation of implantable cardiac pacemaker software and heart models. In CARDIOTECH-
NIX 2014, 2d International Congress on Cardiovascular Technology, pages 55–
59. SCITEPRESS, 2014.
[9] Dines Bjørner and Cliff B. Jones, editors. The Vienna Development Method: The
Meta-Language, London, UK, 1978. Springer-Verlag.
[10] T. Blochwitz, M. Otter, M. Arnold, C. Bausch, C. Clauß, H. Elmqvist, A. Jung- hanns, J. Mauß, M. Monteiro, T. Neidhold, D. Neumerkel, H. Olsson, J.-V. Peetz, and S. Wolf. The Functional Mockup Interface for Tool independent Exchange of Simulation Models. In Proc. of the 8th International Modelica Conference, pages 105–114. Linköping University Electronic Press, 2011.
[11] Torsten Blochwitz, Martin Otter, Johan Åkesson, Martin Arnold, Christoph Clauß, Hilding Elmqvist, Markus Friedrich, Andreas Junghanns, Jakob Mauß, Dietmar Neumerkel, Hans Olsson, and Antoine Viel. Functional Mockup Inter- face 2.0: The Standard for Tool independent Exchange of Simulation Models. In
Proceedings of the 9th International Modelica Conference, pages 173–184. The
Modelica Association, 2012.
[12] Pontus Boström. Formal Methods and Software Engineering: 13th International
Conference on Formal Engineering Methods, ICFEM 2011, Durham, UK, Octo- ber 26-28, 2011. Proceedings, chapter Contract-Based Verification of Simulink
Models, pages 291–306. Springer Berlin Heidelberg, Berlin, Heidelberg, 2011. [13] J. F. Broenink. Modelling, Simulation and Analysis with 20-Sim. Journal A,
38(3):22–25, September 1997.
[14] Victor Carreño and César Muñoz. Aircraft trajectory modeling and alerting algo- rithm verification. In Mark Aagaard and John Harrison, editors, Theorem Prov-
ing in Higher Order Logics, volume 1869 of Lecture Notes in Computer Science,
pages 90–105. Springer Berlin Heidelberg, 2000.
[15] Kalou Cabrera Castillos, Frédéric Dadeau, Jacques Julliand, Bilal Kanso, and Safouan Taha. A Compositional Automata-Based Semantics for Property Pat-
terns, pages 316–330. Springer Berlin Heidelberg, Berlin, Heidelberg, 2013.
[16] Chunqing Chen, Jin Song Dong, and Jun Sun. Formal Methods and Software
Engineering: 9th International Conference on Formal Engineering Methods, ICFEM 2007, Boca Raton, FL, USA, November 14-15, 2007. Proceedings, chap-
ter Machine-Assisted Proof Support for Validation Beyond Simulink, pages 96– 115. Springer Berlin Heidelberg, Berlin, Heidelberg, 2007.
[17] Taolue Chen, M. Diciolla, M. Kwiatkowska, and A. Mereacre. Quantitative ver- ification of implantable cardiac pacemakers. In 33rd IEEE Real-Time Systems
[18] Taolue Chen, Marco Diciolla, Marta Kwiatkowska, and Alexandru Mereacre. Quantitative verification of implantable cardiac pacemakers over hybrid heart models. Information and Computation, 236(0):87–101, 2014. Special Issue on Hybrid Systems and Biology.
[19] Alexandre David, Kim G Larsen, Axel Legay, Mikael H Møller, Ulrik Nyman, Anders P Ravn, Arne Skou, and Andrzej W ˛asowski. Compositional verification of real-time systems using ECDAR. International Journal on Software Tools for
Technology Transfer, 14(6):703–720, 2012.
[20] Alexandre David, Kim G Larsen, Axel Legay, Ulrik Nyman, and Andrzej Wa- sowski. Timed I/O automata: a complete specification theory for real-time sys- tems. In Proceedings of the 13th ACM international conference on Hybrid sys-
tems: computation and control, pages 91–100. ACM, 2010.
[21] Leonardo De Moura and Nikolaj Bjørner. Z3: An Efficient SMT Solver. In
Proceedings of the Theory and Practice of Software, 14th International Con- ference on Tools and Algorithms for the Construction and Analysis of Sys- tems, TACAS’08/ETAPS’08, pages 337–340, Berlin, Heidelberg, 2008. Springer-
Verlag.
[22] Rocco De Nicola and Frits Vaandrager. Semantics of Systems of Concurrent Pro-
cesses: LITP Spring School on Theoretical Computer Science La Roche Posay, France, April 23–27, 1990 Proceedings, chapter Action versus state based logics
for transition systems, pages 407–419. Springer, Berlin, Heidelberg, 1990. [23] Matthew B. Dwyer, George S. Avrunin, and James C. Corbett. Property specifica-
tion patterns for finite-state verification. In Proceedings of the Second Workshop
on Formal Methods in Software Practice, FMSP ’98, pages 7–15, New York, NY,
USA, 1998. ACM.
[24] Camille Fayollas, Célia Martinie, Philippe Palanque, Paolo Masci, Michael D. Harrison, José C. Campos, and Saulo R. e Silva. Evaluation of formal IDEs for human-machine interface design and analysis: the case of CIRCUS and PVSio- web. In 3rd Workshop on Formal Integrated Development Environment (F-IDE),
satellite workshop of Formal Methods 2016. Electronic Proceedings in Theoreti-
cal Computer Science (EPTCS), 2016.
[25] I. Fette and A. Melnikov. The WebSocket Protocol. RFC 6455, RFC Editor, December 2011. http://www.rfc-editor.org/rfc/rfc6455.txt.
[26] Colin J. Fidge, Ian J. Hayes, A. P. Martin, and Axel Wabenhorst. A set-theoretic model for real-time specification and reasoning. In Proceedings of the Mathemat-
ics of Program Construction, MPC ’98, pages 188–206, London, UK, UK, 1998.
Springer-Verlag.
[27] John Fitzgerald, Peter Gorm Larsen, Ken Pierce, Marcel Verhoef, and Sune Wolff.
Integrated Formal Methods: 8th International Conference, IFM 2010, Nancy, France, October 11-14, 2010. Proceedings, chapter Collaborative Modelling and
Co-simulation in the Development of Dependable Embedded Systems, pages 12– 26. Springer Berlin Heidelberg, Berlin, Heidelberg, 2010.
[28] John S. Fitzgerald, Peter Gorm Larsen, and Marcel Verhoef. Vienna Development
Method. John Wiley & Sons, Inc., 2007.
[29] Nathan Fulton, Stefan Mitsch, Jan-David Quesel, Marcus Völp, and André Platzer. KeYmaera X: An axiomatic tactical theorem prover for hybrid systems. In International Conference on Automated Deduction, pages 527–538. Springer, 2015.
[30] Cláudio Gomes, Casper Thule, David Broman, Peter Gorm Larsen, and Hans Vangheluwe. Co-simulation: State of the art. ACM Computing Surveys, 2017, to appear.
[31] G. Hamon and J. Rushby. An operational semantics for Stateflow. In Fundamen-
tal Approaches to Software Engineering (FASE), volume 2984 of Lecture Notes in Computer Science, pages 229–243. Springer Berlin Heidelberg, 2004.
[32] T. A. Henzinger. The theory of hybrid automata. In Proceedings of the 11th
Annual IEEE Symposium on Logic in Computer Science, LICS ’96, pages 278–
292, Washington, DC, USA, 1996. IEEE Computer Society.
[33] Thomas A. Henzinger, Marius Minea, and Vinayak S. Prabhu. Assume-guarantee reasoning for hierarchical hybrid systems. In Maria Domenica Di Benedetto and Alberto L. Sangiovanni-Vincentelli, editors, HSCC, volume 2034 of Lecture
Notes in Computer Science, pages 275–290. Springer, 2001.
[34] Thomas A. Henzinger, Xavier Nicollin, Joseph Sifakis, and Sergio Yovine. Sym- bolic model checking for real-time systems. Information and computation, 111(2):193–244, 1994.
[35] Leila Jalali, Sharad Mehrotra, and Nalini Venkatasubramanian. Simulation inte- gration: Using multidatabase systems concepts. Simulation, 90(11):1268–1289, November 2014.
[36] Zhihao Jiang, Miroslav Pajic, Allison Connolly, Sanjay Dixit, and Rahul Mang- haram. Real-time heart model for implantable cardiac device validation and ver- ification. In Real-Time Systems (ECRTS), 2010 22nd Euromicro Conference on, pages 239–248. IEEE, 2010.
[37] Zhihao Jiang, Miroslav Pajic, and Rahul Mangharam. Cyber–physical modeling of implantable cardiac medical devices. Proceedings of the IEEE, 100(1):122– 137, 2012.
[38] Zhihao Jiang, Miroslav Pajic, Salar Moarref, Rajeev Alur, and Rahul Mangharam. Modeling and verification of a dual chamber implantable pacemaker. In Cormac Flanagan and Barbara König, editors, Tools and Algorithms for the Construction
and Analysis of Systems, volume 7214 of Lecture Notes in Computer Science,
[39] Dean Karnopp and Ronald Rosenberg. Analysis and simulation of multiport sys-
tems; the bond graph approach to physical system dynamics. M.I.T. Press, Cam-
bridge, MA, USA, 1968.
[40] Marta Kwiatkowska, Gethin Norman, and David Parker. PRISM 4.0: Verification of probabilistic real-time systems. In Computer aided verification, pages 585– 591. Springer, 2011.
[41] Peter Gorm Larsen, Nick Battle, Miguel Ferreira, John Fitzgerald, Kenneth Laus- dahl, and Marcel Verhoef. The Overture Initiative Integrating Tools for VDM.
SIGSOFT Softw. Eng. Notes, 35(1):1–6, January 2010.
[42] Peter Gorm Larsen, Carl Gamble, Kenneth Pierce, Augusto Ribeiro, and Kenneth Lausdahl. Support for Co-modelling and Co-simulation: The Crescendo Tool, pages 97–114. Springer, 2014.
[43] E. A. Lee and D. G. Messerschmitt. Synchronous data flow. Proceedings of the
IEEE, 75(9):1235–1245, Sept 1987.
[44] E. A. Lee and A. Sangiovanni-Vincentelli. A framework for comparing models of computation. IEEE Transactions on Computer-Aided Design of Integrated
Circuits and Systems, 17(12):1217–1229, Dec 1998.
[45] K. Rustan M. Leino. This is Boogie 2. Technical Report MSR-TR-2008-194, Microsoft Research, June 2008.
[46] Nancy A. Lynch and Mark R. Tuttle. An introduction to input/output automata.
CWI Quarterly, 2:219–246, 1989.
[47] Paolo Masci, Anaheed Ayoub, Paul Curzon, Michael D Harrison, Insup Lee, and Harold Thimbleby. Verification of interactive software for medical devices: PCA infusion pumps and FDA regulation as an example. In Proceedings of the 5th
ACM SIGCHI symposium on Engineering interactive computing systems, pages
81–90. ACM, 2013.
[48] Paolo Masci, Piergiuseppe Mallozzi, FL De Angelis, G Di Marzo Serugendo, and P Curzon. Using PVSio-web and SAPERE for rapid prototyping of user interfaces in Integrated Clinical Environments. In in Verisure2015, Workshop on
Verification and Assurance, co-located with CAV, 2015.
[49] Paolo Masci, Patrick Oladimeji, Yi Zhang, Paul Jones, Paul Curzon, and Harold Thimbleby. PVSio-web 2.0: Joining PVS to HCI. In Daniel Kroening and S. Co- rina P˘as˘areanu, editors, Computer Aided Verification: 27th International Con-
ference, CAV 2015, Proceedings, Part I, pages 470–478. Springer International
Publishing, 2015. Tool available at http://www.pvsioweb.org.
[50] Paolo Masci, Yi Zhang, Paul Jones, Paul Curzon, and Harold Thimbleby. Formal verification of medical device user interfaces using PVS. In International Con-
ference on Fundamental Approaches to Software Engineering, pages 200–214.
[51] Paolo Masci, Yi Zhang, Paul Jones, Patrick Oladimeji, Enrico D’Urso, Cinzia Bernardeschi, Paul Curzon, and Harold Thimbleby. Combining PVSio with state- flow. In Proceedings of the 6th NASA Formal Methods Symposium (NFM2014), Berlin, Heidelberg, April-May 2014. Springer-Verlag.
[52] The MathWorks, Inc. Stateflow Reference.
[53] C. Muñoz. Rapid prototyping in PVS. Technical Report NIA 2003-03, NASA/CR-2003-212418, National Institute of Aerospace, Hampton, VA, USA, 2003.
[54] César Muñoz, Anthony Narkawicz, George Hagen, Jason Upchurch, Aaron Dutle, and María Consiglio. DAIDALUS: Detect and Avoid Alerting Logic for Unmanned Systems. In Proceedings of the 34th Digital Avionics Systems Confer-
ence (DASC 2015), Prague, Czech Republic, September 2015.
[55] S. H. Attarzadeh Niaki and I. Sander. Co-simulation of embedded systems in a heterogeneous MoC-based modeling framework. In 2011 6th IEEE International
Symposium on Industrial and Embedded Systems, pages 238–247, June 2011.
[56] Patrick Oladimeji, Paolo Masci, Paul Curzon, and Harold Thimbleby. PVSio- web: a tool for rapid prototyping device user interfaces in PVS. In FMIS2013,
5th International Workshop on Formal Methods for Interactive Systems, London, UK, June 24, 2013, 2013.
[57] S. Owre, S. Rajan, J. Rushby, N. Shankar, and M. Srivas. PVS: combining spec- ification, proof checking, and model checking. In R. Alur and T.A. Henzinger, editors, Computer-Aided Verification, CAV ’96, number 1102 in LNCS, pages 411–414. Springer-Verlag, 1996.
[58] André Platzer and Jan-David Quesel. KeYmaera: A hybrid theorem prover for hybrid systems (system description). In International Joint Conference on Auto-
mated Reasoning, pages 171–178. Springer, 2008.
[59] Ragunathan Raj Rajkumar, Insup Lee, Lui Sha, and John Stankovic. Cyber- physical systems: the next computing revolution. In Proceedings of the 47th
Design Automation Conference, pages 731–736. ACM, 2010.
[60] Robert Reicherdt and Sabine Glesner. Software Engineering and Formal Meth-
ods: 12th International Conference, SEFM 2014, Grenoble, France, Septem- ber 1-5, 2014. Proceedings, chapter Formal Verification of Discrete-Time MAT-
LAB/Simulink Models Using Boogie, pages 190–204. Springer International Publishing, Cham, 2014.
[61] Kristin Yvonne Rozier. Specification: The biggest bottleneck in formal methods and autonomy. In Verified Software. Theories, Tools, and Experiments - 8th In-
ternational Conference, VSTTE 2016, Toronto, ON, Canada, July 17-18, 2016, Revised Selected Papers, pages 8–26, 2016.
[62] Johannes Sametinger, Jerzy Rozenblit, Roman Lysecky, and Peter Ott. Security challenges for medical devices. Communications of the ACM, 58(4):74–82, 2015. [63] I. Sander and A. Jantsch. System modeling and transformational design refine- ment in ForSyDe. IEEE Transactions on Computer-Aided Design of Integrated
Circuits and Systems, 23(1):17–32, Jan 2004.
[64] SCADE suite®web site.
[65] Scicoslab web site.
[66] Lijun Shan, Susanne Graf, and Sophie Quinton. RTLib: A Library of Timed Au-
tomata for Modeling Real-Time Systems. PhD thesis, Grenoble 1 UGA-Université
Grenoble Alpe; INRIA Grenoble-Rhone-Alpes, 2016.
[67] Mary Sheeran and Gunnar Stålmarck. Formal Methods in Computer-Aided De-
sign: Second International Conference, FMCAD’ 98 Palo Alto, CA, USA, Novem- ber 4–6, 1998 Proceedings, chapter A Tutorial on Stålmarck’s Proof Procedure
for Propositional Logic, pages 82–99. Springer Berlin Heidelberg, Berlin, Hei- delberg, 1998.
[68] B. Silva, K. Richeson, B. Krogh, and A. Chutinan. Modeling and verifying hy- brid dynamic systems using CheckMate. In Proc. Conf. on Automation of Mixed
Processes: Hybrid Dynamic Systems, pages 323–328, 2000.
[69] Simulink®web site.
[70] Simulink Design Verifier®web site.
[71] Raymond Merrill Smullyan. First-order logic. Dover publications, New York, 1995.
[72] Stavros Tripakis, Christos Sofronis, Paul Caspi, and Adrian Curic. Translating Discrete-time Simulink to Lustre. ACM Trans. Embed. Comput. Syst., 4(4):779– 818, November 2005.
[73] Valentín Valero, Gregorio Díaz, and María-Emilia Cambronero. Timed automata modeling and verification for publish-subscribe structures using distributed re- sources. IEEE Transactions on Software Engineering, 43(1):76–99, 2017. [74] B. Wang and J. S. Baras. Hybridsim: A modeling and co-simulation toolchain for
cyber-physical systems. In Distributed Simulation and Real Time Applications
(DS-RT), 2013 IEEE/ACM 17th International Symposium on, pages 33–40, Oct
2013.
[75] Pei Ye, Emilia Entcheva, and Scott A. Smolka. Efficient modeling of excitable cells using hybrid automata. In Computational Methods in System Biology, pages 216–227, 2005.
[76] Franco Zambonelli, Andrea Omicini, Bernhard Anzengruber, Gabriella Castelli, Francesco L De Angelis, Giovanna Di Marzo Serugendo, Simon Dobson, Jose Luis Fernandez-Marquez, Alois Ferscha, Marco Mamei, et al. Develop- ing pervasive multi-agent systems with nature-inspired coordination. Pervasive
(skosimp∗)
(expand “en PVARPtau”)
(expand “en tau”)
(split)
(flatten)
(lemma “invar pvab”)
(inst
−1“st!1”)
(split)
(assert)
(flatten)
(lemma “invar pvarp”)
(inst
−1“st!1”)
(split)
(assert)
Figure 10: Proof tree of Lemma ap_en_pvarptau. The ⊢ symbols represent (sub)goals, each followed by a PVS rule.