Our overall approach to adopting Experimental Software Engineering practices in the context of component-based development can be viewed as an instantiation of the pro- cess model that we will describe in detail in chapter 3.
While studying the problem of selecting, or creating, a quality model for CBD (see, for instance, [Goulão 02a, Goulão 02b]), we concluded that defining fine-grained qual- ity models aimed at specific niches of CBD is more feasible than aiming for a general CBD quality model. The diversity of issues that such a generic quality model involves would lead to a quality model too complex to be easily grasped by practitioners and therefore useless. This option for quality models aimed at specific niches is supported by the analysis of work of [Bertoa 02, Bertoa 06], where we observed how an evolu- tion from a generic quality model to a specific one facilitated the feasibility of their validation.
Although we do not propose new quality models in this dissertation, we present quality concerns that underpin the goals of each of the experimental works described throughout this dissertation. This is followed by a Goal-Question-Metric approach [Basili 94], to determine which metrics should be collected to assess components, as- semblies, or some process aspect.
Figure 1.1 outlines the metrics collection process. We use a domain ontology to express the basic concepts from which we want to extract relevant properties. Then, we populate the ontology with an instantiation that represents the target to be assessed. For instance, in the most frequent case in this dissertation, the ontology is a metamodel. Therefore, the instantiation is a graph where the nodes are meta-objects and the edges
are meta-links. We use OCL expressions which define the metrics to be collected to traverse the graph and compute the metrics. Heuristics definitions may also be defined in OCL, at this stage. Finally, we analyze the results of the OCL expressions - both metrics and heuristics.
Figure 1.1: Metrics collection process
The OCL provides the required formality without sacrificing understandability, since it was conceived with usability in mind for UML practitioners.
The approach itself is generic: we can use it for assessing products and processes. For products, we will use ODM for assessing both individual components and com- ponent assemblies in chapters 4, 5, and 7. These assessments are carried out using a variety of component models, including JavaBeans - chapter 4 -, UML 2.0 components, the CORBA Component Model (CCM) - chapter 5 -, and Eclipse plugins - chapter 7.
described in chapter 6.
The approach is also flexible: adding a new metric, requires defining a new OCL ex- pression that specifies how the metric should be computed. Heuristics may also be de- fined using the same technique, typically through the specification of OCL predicates that check for metrics values outside their expected range [Goulão 04a, Goulão 05c].
The approach is open in the sense that the metrics are defined using standard OCL clauses, and it only requires a common UML tool with OCL support, upon which we can load a model (the domain ontology) and populate it with the appropriate instances. We are currently using the USE tool 1 [Richters 01] for this purpose, but this kind of computational support is becoming increasingly available in several UML tools, as they become “OCL-enabled”. Several of those tools support the UML 2.0 component metamodel, either internally (on their data dictionary) or at least through their external interface (e.g. by providing an import/export feature using UML 2.0-compliant XMI). By basing our approach in a de facto standard such as UML 2.0, combined with OCL, we enable a smooth integration of our proposals into current and future IDEs that support UML 2.0 and OCL.
For other component metamodels, we will still have to go on developing instances generators, or, in alternative, to use UML profiles, such as the one for EJB, so that we can specify and collect the metrics on top of the UML 2.0 metamodel and the used UML profile.
The evaluation of our proposals is carried out in four different ways:
• The work described in chapters 4 through 7 included setting up an assessment in- frastructure and exercising it with different underlying ontologies and metrics for different purposes. It covers covering different aspects of component interfaces and interaction mechanisms, at the product level. It also covers different parts of the software process, namely code inspections and software maintenance and evolution. By doing so, we are able to assess the flexibility of our approach. • The peer review of our work in the context of international scientific forums pro-
vided us with an external assessment of the soundness of our proposals. The vast majority of the contents of chapters 2 through 6, as well as appendix B has been published in peer reviewed journals, conferences or workshops.
• The primary source of validation of our proposals consists on the experimental work in chapter 6 [Goulão 06], and the observational studies described in chapter 7. A case study for the validation of the proposed experimental approach is also presented, in chapter 3.
• We also contribute to the external validation of proposals by other authors, as in the cross-validation case study presented in chapter 4 [Goulão 04a, Goulão 05c].