experiments with Jackrabbit and Hibernate. Performance measurements are taken with the framework configured to use both the JDI as well as instrument probes, and compared against the native execution performance of the two software products. Execution times of the monitored software are also analysed against the number of architecture violations.
PANDArch is critically assessed against its core principles. While the current implementation of the framework has satisfied a number of these principles, further enhancements and additional evaluation are needed to meet the expectations of some other principles. Some other areas of concern such as difficulties surrounding specifying architectures and implementation mappings are also discussed.
8.2
Future Work
The design and implementation PANDArch framework presented in this thesis can be expanded in a number of useful ways. One such extension is the support for different architectural views and their associated conformance rules. A limitation in the current framework implementation is that it can only use runtime structural views for checking architecture conformance. While this is mostly because of a similar limitation in Grasp, the framework itself needs to be extended with conformance rules related to other types of architectural views used commonly in practice, such as the physical view. Such a capability should be developed in conjunction with support for extracting architecture properties from an ADL that supports these views. A useful notation to support in this regard is UML, as industry surveys show significant use of UML for describing certain architectural views [147].
Incorporating functionality to process multiple architectural views requires validating the consistency among these views. Although automated consistency checking of architectural models using formal methods [166] is somewhat different to architecture conformance checking, it can be considered a necessary pre-processing step for ensuring there are no discrepancies among views given to PANDArch. For instance, a runtime view and a physical view of an architecture should convey the same component interactions, failing which the conformance checker could produce conflicting notifications during conformance monitoring. While existing approaches may be used for consistency verification, they should be made an inherent part of the framework so that the automation principle is not violated.
Besides providing the capacity to deal with multiple architectural views, PANDArch should also be capable of supporting varying configurations of software systems. Extending the framework to check conformance of distributed architectures is considered a priority in this regard, as a significant portion of contemporary software are considered to be distributed systems.
Another important area of future work is building a static conformance checking capability to supplement dynamic conformance monitoring available at present. Checking conformance by statically analysing the implementation is a fundamental aspect of the PANDArch approach as discussed in Chapter 3. It can be carried out during the development phase, even before the software is ready to be tested. Although some static architecture properties can be checked for conformance during program execution, certain others such as references to external libraries cannot be checked at runtime. These external references are often lost during the build process when the program is statically linked to those libraries. Therefore, the inclusion of static conformance checking will allow PANDArch to increase the coverage of detectable architecture violations. Static conformance checking may also contribute towards minimising the performance impact by limiting runtime conformance checking to only dynamic architecture properties. The approach taken for evaluating PANDArch can be further improved as part of future work. The strategy presented in this thesis for evaluating the framework with open source software involved extracting the implemented architecture of those systems followed by artificially inducing architecture violations. This was required due to the lack of accurate and up to date architectures of the chose open source products. While such an approach is effective in evaluating whether the framework performs as expected from a functional point of view, it does not represent its use case in practice. Therefore, in order to measure its effectiveness in detecting architecture violations PANDArch needs to be deployed within an on going software development project. The framework should use the existing software architecture to analyse the implementation for conformance violations while conformance checking is made an integral activity of the development process. A useful extension to the framework prior to carrying out this work is enabling static conformance checking as discussed in the previous paragraph.
An important area of further exploration is runtime conformance checking of applications that execute native code outwith a virtual machine. Applications written in languages such C or C++ fall under this category and are in widespread use. A fundamental premise in the PANDArch approach is that an executing application is able to provide some information about changing state in the form of events. However, applications that execute natively do not meet this premise because they are hosted by an operating system that does not track the internal state of the application to the extent done by a virtual machine. Therefore, an elaborate mechanism needs to be devised to extract runtime information from executing native code. Some compilers allow embedding runtime-type information, which can serve as a starting point, although retrieving that information during execution may require injecting probes into the compiled code. However, as PANDArch will remain pluggable, the negative consequences runtime-type information and code injection can be contained to the duration when conformance is monitored.