The results are promising in the sense that they show that a model comparison tool can be made for 20- sim that still achieves good performance, while being able to support most types of models. One should make the sidenote that the tool has not been tested much for very large models. This is something that might have improved the results chapter, since then more tests of industrial practice could be used to verify the workings of the tool. Furthermore, it was difficult to obtain decent results for the abstract concept of impact, since there is not an exact definition of what impact means to a user. The main
difficulty here lies in the fact that not one model is perfectly competent to model all behaviour that can be observed. A model has a problem context in which it is accurately modelling the physical behaviour. As long as this context is unknown to the tool, determining an impact score is difficult. For example, a user that wants to perform a hardware in the loop simulation is interested in other aspects of the model than someone performing a simulation to find the natural frequencies of a physical system.
Interestingly, due to the linear fit of data through the performance curves, it can be seen that for EMX models under about 250 kilobyte or 200 equations, quite a good performance can be achieved. First of all, the fact that a linear fit can be made is already a good result at itself. There is no quadratic increment in waiting time for the tool or dynamic memory usage when the size of the EMX model or the amount of equations increases. One should however note that adding 100 kilobytes of initial EMX model size only increments the time to wait by about 5 seconds and the process memory by a about 70 MB. The same results also apply to an increment of 100 equations.
Chapter 7
Conclusions and Recommendations
7.1
Conclusions
This master assignment showed several aspects of model management. First of all, an initial design of model management within an existing version control system was developed, which actually showed that the desired project structure as described in section 3.3 could be achieved within GIT. The subgoal about model version comparison (subgoal 2), that was defined in section 1.1, was succesfully expanded into a full-blown proof of concept that actually did exactly that: comparing model versions. Furthermore, concepts like traceability and provenance were used to find initial ideas on how to use the results of model management to improve the modelling workflow itself, by identifying traces from a model to its related external dependencies like requirements, tests and simulation results.
The new format for a 20-sim model, that was specifically designed for this model comparison tool, is also a format that is very useful for 20-sim. Not only is part of the old EMX format that was still plain text converted to an XML representation, also the format itself is more transparent. The format is more separated into functional components in 20-sim, and the combination of the XSD and the model file itself form the complete set of information that 20-sim needs to know to open, process and simulate the model. The XSD model remains the same for all models, and contains information about the default value of tags that are missing in the actual model file. A proper conversion was also made during this master assignment from the old to the new format.
The separate components of the merge tool each were designed in such a way that they were also useful as separate tool. The XSD-XML tree builder application for example also works on random other XSD files. This was already shown in section 5.1.2, in which the XSD of an image storage type called SVG was embedded within the tool without problems. This behaviour can however also be achieved on any random XSD and XML, given the fact that they do not use unsupported components in their XSD. Also, adding the context layer seemed to be a good approach on reproducing the behaviour of model transformations: it becomes possible to identify components of a 20-sim model, and say something about if they were added, removed or altered.
The impact analysis proved to be a new feature for model comparison tools, in the sense that there are no competitor model comparison tools that actually say anything about the impact of a difference on the model behaviour. Even though the ideas posed in the impact aspect are still initial ideas on how to approach this problem, the consequences of giving feedback to the user about impact and being able to trace outdated results are a major step towards easier model management. Furthermore, computing an impact score was based on a technique from artificial intelligence, which means that the user can even change the outcome of the impact score per difference type, based on his own preferences.
Finally, a graphical user interface was made that more concrete shows the workflow that a user takes: deciding which two models to compare, deciding if a merge of these two models should be performed, observing the differences, and deciding on further actions about what to do with the comparison results.