Generated Lustre code is verified against SO annotations using SMT-based model checking tech- niques allowing to prove the validity of the generated observers. Verification of the C generated code is done by relying on the Frama-C toolset.
8.3 Gain wrt classical verification activities
The approach detailed here provides the ability to express verification properties on high level design of systems. Such properties are expressed directly on the model and are then translated as code annotations. Their verification can be done using the Frama-C toolset, SMT solvers and if needed proof assistants. 8.3.1 A complement to state of the art design verification
State of the art embedded systems design verification is done by relying first on simulation with tools like Simulink. As was previously detailed, formal verifications can be applied on the design using Simulink design verifier. Automatic test cases generation can also be done. Test cases can also be used on the system executable code in order to ensure the respect of the behavior previously assessed. Whereas this approach provides good insights on the correctness of the design, early verification of the design and allows detection of errors, it does not provides a formal verification of the properties as it is only test based and suffers from exhaustiveness.
In most safety-critical industries, Scade is used to express the system after its design and early verifica- tion using Simulink. At the Scade level, properties can be expressed using SO and as Scade relies on Lustre, formal verifications can be done using model checking with for example the Prover plugin⁴ also usable for the verification of systems specified in languages such as C, Simulink or UML.
These design verification approaches does provide formal verification regarding the system correctness at a design level. They allow also to generate tests that can be used on the manually written or automatically generated code but does not provides formal assurance of the properties verification on the code.
On the contrary, by relying on our approach, design level properties (both HLR and LLR) can be gath- ered on the code level and formally verified. Translation and verification on the code is made possible by the providing of our formally specified block library.
8.3.2 Current limitations and perspectives
The core of our verification approach is based on the ability to generate both LLR and HLR expressed at the design level as annotations to be verified at the code level. This generation relies on the formal specification of blocks provided by the BlockLibrary specification approach. Both LLR and HLR are embedded as code annotations, LLR as blocks of code contracts and HLR as predicates used in specific locations of the system computation. We have shown the ability for our annotation generation mechanism to provide properties verifiable on concrete examples.
As was previously stated, the annotation generation mechanism is currently not fully implemented. We presented in Chapter 7 a translation mechanism converting OCL constraints as Why expressions. On going work at CEA aim at providing the ability to rely on Why3 formalisation (theories and modules) in ACSL verification. Providing such a functionality in a tool like Frama-C could ease our translation work as it might then be possible to rely on the theories generated in Chapter 7 in that purpose.
The block specification and SO are providing the expected behavior of the generated code. As such they may be used as a source for the automatic generation of test cases. The tests will be used for the runtime verification of the system ensuring fault detections or to complete the formal verification of the generated code when the automatic proof fails. We will rely on these in the following for the automatic generation of certification and qualification data.
9
Certification/Qualification data generation
Software certification activities aims at increasing the confidence a user can have on the use of a software system. Such confidence is often achieved by providing data detailing: the system development activities and processes; the results of the software verification activities; or data produced by reliable (i.e. qualified) third party tools used in the development activities.
State of the art techniques for confidence assessment on a software is usually produced by relying on proof reading and intensive testing. By nature, these approaches, while being widely used, cannot be ex- haustive and thus cannot be fully trusted especially while developing safety-critical software.
In this chapter, we will show how the BlockLibrary DSML and related tools presented in this docu- ment could be integrated in the qualification process of an ACG. We will detail the possible uses we have identified for BlockLibrary instances in order to generate certification data and the (formal) methods that can be used along these data in order to gain certification credits.
9.1 BlockLibrary for qualification
BlockLibrary specification writing aims at providing detailed and formal specification for an ACG input language. In order for our approach to be valuable for an ACG qualification use, one should ensure that the formalism and underlying technologies used are likely to be accepted by certification authorities.
As detailed in this document, our BlockLibrary DSML has been designed using a model based de- velopment approach, its semantics has been defined by formalisation and by model transformation, and its correctness is ensured using formal methods. Each of these three elements, as the building blocks of our approach, must be considered in the context of certification.
The BlockLibrary DSML may provide confidence in the code generated by an ACG. One may use the BlockLibrary DSML as means to verify the generated code and thus may avoid some verification activities on the generator.
There is no explicit text in DO-330 stating whether it is possible to apply DO-178C technology related ’supplements’ in the qualification activities of a tool. Indeed, the ’supplements’ are meant to be applicable on DO-178C related qualification and DO-330 is meant to be domain agnostic. As this applicability is not explicit, it is recommended that their application should be justified through the elicitation of the DO-330 document guidance to be satisfied through the use of ’supplement’ documents [126]. It is important to put an emphasis on the fact that as a specialisation of the DO-178 document for tools qualification, ’supplement’ documents application seems likely to be doable.
In the following sections, we will detail how the BlockLibrary DSML definition, formalisation and verification applies in the context of the certification of an ACG according to DO-178C and DO-330 espe-
9.1. BLOCKLIBRARY FOR QUALIFICATION
cially by relying on DO-178C ’supplement’ documents DO-331 and DO-333. 9.1.1 DO-331: Model-based technology
We have shown previously that our BlockLibrary approach is appropriate for the specification and veri- fication of block libraries for dataflow languages and for the verification of the generated code with respect to the language semantics and the user requirements. We claim that our BlockLibrary approach has the required characteristics expressed in DO-331 (detailed in Section 2.1) in order for a model to be used for qualification activities:
“The model is completely described using an explicitly identified modeling notation”. The BlockLibrary metamodel is specified using Ecore that is a standard (defined after the MOF notation) and a well ac- cepted modeling notation (highly used in industrial applications). Building models from standard meta- models (i.e. Ecore) and additional standard constraint languages (i.e. OCL) is one of the strengths of the model-driven approach as it allows to formally specify a data structure (the model) from an already specified and accepted formalism.
“The modeling notation has a precise grammar (also called “syntax”) and meaning (also called “semantics”). The modeling notation may be graphical and/or textual.” From the BlockLibrary, a grammar has been de- fined in order to provide a syntax for the definition of BlockLibrary instances. In Chapter 6, we defined the content of the language and all its components. For each of them, we detailed their meaning and seman- tics. In this document, we have defined the overall BlockLibrary semantics definition as being twofold: a) the structural semantics composed of the definition of the BlockVariant and StructuralFeature elements and their composition through the BlockVariant elements. The former elements structure is defined and constrained using OCL constraints (Section 6.2). The latter elements have been specified in Sections 6.4.4 and 6.4.5. Formal verifications are applied to them as depicted in Section 6.6 and success- fully done as detailed in Section 7.7; b) the behavioral semantics of the BlockLibrary specification is given through the definition of the BlockMode elements containing functions definition as Hoare triples as depicted in Section 6.5, their formal verification is provided in Section 7.8. We thus have defined a precise syntax and a formal semantics to our BlockLibrary modeling notation.
“The model contains software requirements and/or software architecture definition.” It is the purpose of the BlockLibrary specification model to provide the specification for dataflow blocks through the elicitation of the structural variability and their corresponding semantics. These are related to software requirements as they are the source for automatic code generation of embedded systems design models. We detail in Section 9.2 the uses for BlockLibrary models as a source for automatic generation of requirement and software development artifacts.
“The model is of a form and type that are used to direct analysis or behavioral evaluation as supported by the software development process or the software verification process.” As we are working in the context of a tool qualification, reference to ’software’ in this definition must be replaced by ’tool’. A BlockLibrary instance contains specification for the input elements of an ACG and the expected behavioral informations of the generated code, it is thus likely to be used as an information source for both tool development and tool verification process. We will detail these in Section 9.2.
9.1.2 DO-333: Formal methods
According to the DO-333 document, the combination of formal modeling and formal analysis can be used to produce a formal method that might be used for the replacement of some of the traditionally conducted verification activities. As we presented previously, BlockLibrary model instances qualifies for being considered as a formal model.
The use of theorem provers have already been proven trustworthy and usable as a formal analysis mean. On the other side, works are in progress in order to bring such a confidence on the use of: SMT solvers by the formalisation of Alt-Ergo SMT solver for example; and on toolsets like Frama-C and Why3 in the U3CAT project¹. Early works on the use of formal methods for the verification of embedded systems