3.1
Introduction
A key motivation for undertaking the research documented in this thesis is a need to improve industrial software engineering practice. In order to address this need, it is necessary to adopt an approach to research that responds to industry problems and facilitates the orderly evaluation, adoption and continuous improvement of research outputs.
In this chapter, I propose a novel conceptual model which can be used to describe, discuss and compare a broad range of research approaches. I validate this model by using it to discuss prior contributions in the area of software engineering research approaches. I then use this information to develop a two-phase approach
to Aspect-Oriented Thinking research. The first phase comprises the theoretical
work and proof-of-concept presented in this thesis. The second phase constitutes future research including industrial evaluation, adoption and continuous improve- ment of theAspect-Oriented Thinkingapproach.
3.2
A conceptual model of software engineering re-
search
The subject of this thesis was initially one of software engineering. The research approach adopted was therefore derived from ideas presented in the software engineering research literature.
In order to compare and evaluate prior contributions regarding software engineering research approaches and to describe new ones, a conceptual model of research is proposed. The model is loosely based on the structure (not the content) of ISO/IEC 12207 Software Lifecycle Processes [International Organization for
Standardization, 1997] and ISO/IEC 15288Systems Enginering - System life cycle
processes[International Organization for Standardization, 2002] and is applicable
in any research domain.
The model, represented as the UML class diagram shown in Figure 3.1, considers software engineering research approaches in terms of three orthogonal concerns: Research method, activities and life-cycles.
3.2.1 Research methods
Research methods describe the philosophical basis of a research approach. No concern is given here to the specific activities involved or the precise order in which
3.2 A conceptual model of software engineering research
Research Approach
Research Method Research Life−Cycle
Research Activity are ordered to form a 0..* 1..1 R2 1..1 1..1 follows a
describes the process of
is implemented by
comprises 1..* 0..*
R3 R1
provides the basis of
Figure 3.1: Class diagram depicting the proposed conceptual model of software engineering research
they are conducted.
During the 1990’s a series of papers were published dealing with software engineering research. One of the early and often cited papers by Adrion [1993] presents the following software engineering research methods:
• ‘Scientific Method: observe the world, propose a model or theory of behaviour, measure and analyse, validate hypothesis of the model or theory, and, if possible, repeat.’
• ‘Engineering Method: observe existing solutions, propose better solutions, build or develop, measure and analyse, and repeat until no further improve- ments are possible.’
• ‘Empirical Method: propose a model, develop statistical or other methods, apply to case studies, measure and analyse, validate the model, and repeat.’
• ‘Analytical Method: propose a formal theory or set of axioms, develop a theory, derive results, and, if possible, compare with empirical observations.’
These methods and their descriptions are adoptedas-isfromAdrion[1993] as part of the proposed conceptual model of software engineering research.
3.2.2 Research activities
Research activities are sets of tasks that may be carried out to implement part of a research approach. The purpose here is to identify and describe a generic set of activities that can later be arranged in various ways to form a specific approach to software engineering research. No concern is given to whether all activities are used in a particular approach, the sequence in which they are conducted or how they are conducted.
Glass [1995], when considering each of the research methods described by
Adrion[1993], concluded that they all, in general, involve the following phases.
• ‘The Informational Phase: gathering or aggregating information via reflection, literature survey, people/organisational survey, or poll (e.g. Delphi approaches).’
• ‘The Propositional Phase: proposing and/or formulating a hypothesis, method or algorithm, model, theory, or solution.’
• ‘The Analytical Phase: analysing and exploring a proposition, leading to a demonstration and/or formulation of a principle or theory.’
• ‘The Evaluative Phase: evaluating a proposition or analytic finding by means of experimentation (controlled) or observation (uncontrolled, such as a case study or protocol analysis), perhaps leading to a substantiated model, principle, or theory.’
Because, as Glass [1995] points out, not all of these phases are used in each research method and because some research approaches comprise iterative and/or repetitive applications of the phases, the proposed model recasts these phases as
activities that should be considered when forming a research approach tailored to
meet the needs of a specific research program.
In addition to activities derived fromGlass[1995], the proposed model includes the following activities:
• Practice. Used to describe industrial practice that is an integral part of a research program.
3.2 A conceptual model of software engineering research
• Development. Used to describe the development and maintenance of software and other tools that support a research program or use of research outputs.
The selection and sequencing of research activities to form a specific research approach is the subject of research Life-Cycles described below.
3.2.3 Research life-cycles
Research life-cycles identify and specify the order in which research activities need to be executed to conduct a research project using a particular research method.
3.2.3.1 Idealistic life-cycles
Each of the research methods described in Section 3.2.1 can be conducted using an idealistic lifecycle comprising all of the activities described in Section 3.2.2 ordered as depicted in the UML activity diagram shown in Figure 3.2. Some iteration may be present between the Evaluationactivity and previous activities as depicted in Figure 3.2. required] [more information <<information>> <<proposition>> <<analysis>> <<evaluation>> [further analysis required] [done] [new proposals required]
Idealistic Lifecycle
Figure 3.2: Activity diagram depicting the idealistic life-cycle common to all of the research methods described in Section 3.2.1.
In practice, Potts [1993] and Fenton et al. [1994] found that most software engineering research programs follow life-cycles that are modifications of the idealistic one depicted in Figure 3.2. In the following sections a number of such life-cycles, derived from the literature, are described using the conceptual model presented in Section 3.2.
3.2.3.2 The ‘Analytical Advocacy Research’ life-cycle
Fenton et al.[1994] show that many software engineering research findings can be
categorised as ‘Analytical Advocacy research’ in which authors develop and analyse the benefits of a new approach and then, based on this analysis, advocate its use in industrial settings. is identified A new concept <<practice>> <<analysis>> <<proposition>> technology in industry Advocate use of
No Evaluation other than via Analysis
Research Practice
Figure 3.3: Activity diagram depicting the ‘Analytical Advocacy Research’ life-cycle
[Fenton et al., 1994].
The life-cycle of such an approach can be formed from the activities described in Section 3.2.2 and represented using the UML activity diagram shown in Figure 3.3. Stereotypes are used to indicate the type of each activity involved. The two partitions depicted (‘Industry’ and ‘Research’) are used to indicate the domain in which each research activity is conducted.
Because this approach is conducted in isolation from industrial practice, relevance of the research and its likely uptake by industry cannot be easily understood. As such, the ‘Analytical Advocacy research’ approach is very risky and likely to generate research outputs of very little or no industrial value.
3.2 A conceptual model of software engineering research
3.2.3.3 The ‘Research-then-Transfer’ life-cycle
Potts[1993] describes the dominant approach to software engineering research as
‘Research-then-transfer’. Despite its apparent popularity, this life-cycle is little more than a sophisticated version of the often ineffective ‘Analytical Advocacy Research’ approach. That is, research proceeds separately from any industrial application of its outputs.
The ‘Research-then-Transfer’ approach depicted in Figure 3.4, starts with motivation to use an idea or specific technology to solve a perceived industrial problem. The research then proceeds with little or no involvement with industry. At some point, when the research is considered ‘ready for transfer’, it is introduced to industry. Because industrial practice is likely to have evolved during the period of research, the original problem may have been solved or become irrelevant. New problems may have emerged and technological, social, organisational and commercial changes may have occurred. The result is that research conducted with such an approach may be inapplicable by the time it is put to use.
<<practice>>
<<evaluation>>
<<analysis>> <<proposition>> Problems are based on indirect and anecdotal knowledge
<<practice>> Research Practice to transfer] [more research required] [research ready
Figure 3.4: Activity diagram depicting the ‘Research-then-Transfer’ life-cycle
[Potts, 1993].
Interestingly, this scenario is similar to those software development ap- 41
proaches which begin with the elicitation and specification of stakeholder re- quirements and are completed by applying a sequence of activities to develop the software with no further input from stakeholders. At the end of the process, the software is delivered to bewildered users who now require software different to what was originally specified. A common solution to this problem in software development is to introduce a more incremental approach in which there is frequent interaction between the software developers and stakeholders. Potts
[1993] proposes a similar approach to software engineering research. He refers to this improved approach as ‘Industry-as-Laboratory’.
3.2.3.4 The ‘Industry-as-Laboratory’ life-cycle
It is important that software engineering problems be identified from an industrial basis rather than an academic perception.
Potts[1993]
The UML activity diagram shown in Figure 3.5 depicts the ‘Industry-as- Laboratory’ approach described by Potts[1993]. Using this approach, researchers identify a problem through close involvement with industry and with an open mind regarding potential solutions.
Ideas, technology and approaches emerge from interactions between the research and practice domains. Results are always evaluated in an industrial setting. An important consequence of this interaction is that industrial projects not only demonstrate the effectiveness of research results, but also increase the understanding of characteristics that certain approaches and technology exhibit in specific industrial contexts. Such interaction may also lead to the discovery of real-world problems that could not have been imagined in an isolated research environment.
3.2.3.5 Research life-cycles and the OODA loop
Important insights related to research life-cycles can be drawn from concepts that underpin the use of Boyd’sObserve-Orient-Decide-Act (OODA)decision loop [Boyd, 1986] in competitive military and business environments. The idea is that decision makers who cycle through their OODA loop more rapidly than others, will have a competitive advantage. The reason for this is that slower decision makers tend to make decisions based on observations and orientation that are invalid by the time decision are implemented. As a result, their actions are often inappropriate and costly.
3.2 A conceptual model of software engineering research
Transfer technology for evaluation in industry
Problems are identified through close involvement with industry <<practice>> <<information>> <<proposition>> <<analysis>> <<evaluation>> [research complete] [more research required] Practice Research
Figure 3.5: Activity diagram depicting the ‘Industry-as-Laboratory’ life-cyclePotts
[1993].
The same idea can be applied to research. Research programs are likely to be more successful if they are able to react quickly to evolving industrial problems. Those that do not, like those that follow the ‘Research-then-Transfer’ approach or slowly cycle through the ‘Industry-as-Laboratory’ approach, are more likely to be unsuccessful. They may perceive changes in industrial practice as chaotic and will often research insignificant, old or non-existent problems that are mis-aligned with industrial needs. Even worse, they might involve industry in work that is of little value.
The significance of this insight is that adoption of the ‘industry-as-laboratory’ life-cycle might not be enough to ensure the success of research programs which aim to benefit industry. It will also be necessary to react quickly to changes in industrial practice.
3.3
The Aspect-Oriented Thinking research approach
Within this section I use the conceptual model introduced in Section 3.2 to describe the research approach that has been adopted to develop, demonstrate, evaluate and continuously improveAspect-Oriented Thinking.3.3.1 Research method
As stated in Section 1.2, the aim of this thesis is to develop, model, demonstrate and evaluate an aspect-oriented interdisciplinary approach to engineering that can be used to build and operate systems required to understand and improve Problem
Situations of all kinds.
To achieve this aim, theEngineering Methodof research has been adopted. The research has been conducted by observing the use of current approaches to system development and operation (Chapter 2), and then proposing and evaluating an improved approach in the form ofAspect-Oriented Thinking(Chapters 4 to 6).
3.3.2 Research life-cycle
Potts [1993] identifies a number of issues that need to be dealt with before
‘wholesale’ adoption of the ‘Industry-as-Laboratory’ approach. Of significance is his view that the approach can lead to an over-emphasis on the development of short-term solutions at the expense of more general solutions, and that it leads to evolutionary change rather than revolutionary change.
While Aspect-Oriented Thinking cannot (yet) be classified as a revolutionary approach, it is one that may have broad applicability to Problem Situations of all kinds. In order to ensure that this general applicability is fully explored, the two phase research life-cycle depicted in Figure 3.6 has been adopted.
3.3.2.1 Phase One - Aspect-Oriented Thinking development
Phase Oneof the research has been completed and documented within this thesis.
The objective of Phase One was to satisfy the first part of the research aim to develop and demonstrate an aspect-oriented interdisciplinary approach to engineering (Section 1.2). It comprised the following activities depicted in the upper horizontal partition of the activity diagram shown in Figure 3.6.
3.3 The Aspect-Oriented Thinking research approach
<<analysis>> Assess the potential value
of further research <<proposition>> Improvements to AO−Thinking <<development>> AO−Thinking tool development <<analysis>> Proof of Concept <<proposition>> Aspect−Oriented Thinking <<analysis>> Determine readiness for
industrial evaluation <<practice>> Current Practices [Motivation to research requirements problems] <<information>> Background
Transfer technology for use/evaluation in industry [suitable for use] [unsuitable for use] <<evaluation>> Evaluation of AO−Thinking <<practice>> Application of AO−Thinking <<information>> industrial practice Update knowledge of
[more research required] [research complete] [Conjecture]
evaluate]
[more research required] [ready to
Phase Two (subject of future work)
Phase One (subject of this Thesis)
Research Practice
Chapters 4 and 5
Chapter 6 Chapter 2
Figure 3.6: Activity Diagram depicting the Aspect-Oriented Thinking research approach.
• Information - Background. This activity established the context within which Aspect-Oriented Thinking was developed. This included the develop- ment of Capability Dynamics, an approach that can be used to understand
Problem Situations, andAspect-Oriented Systems Engineering, a preliminary
aspect-oriented approach to engineering.
The results of this activity have been reported in Chapter 2.
• Proposition - Aspect-Oriented Thinking.This activity proposed the idea
of Aspect-Oriented Thinking as an interdisciplinary approach to improving
Problem Situations (Section 2.8). The proposed approach has been fully
described in the form of aconceptual model (Chapter 4) and aprocess model
(Chapter 5).
• Analysis - Proof-of-Concept. Tichy [1998] says ‘that demonstrations can provide a proof-of-concept or incentives to study a question further’. The purpose of this activity was to demonstrate the potential value of Aspect-
Oriented Thinking as an approach to engineering any kind of man-made
system. This has been achieved by developing and documenting a complete software system using theAspect-Oriented Thinkingapproach.
The results of this activity have been documented in Chapter 6 and Appen- dices B to G, and will be used to justify further development and evaluation of the approach duringPhase Twoof the project.
It could be argued that this initial research phase shows signs of the deficient ‘Research-then-Transfer’ lifecycle [Potts, 1993]. However, the research built upon existing research results (X
T UML and System Dynamics) that have been developed, evaluated and improved through close collaboration with industry over many years. The research has generalised the concepts that underpinX
TUML to produce an approach that may help solve very long standing problems associated with the improvement of dynamically complexProblem Situations.
In order to ensure that Aspect-Oriented Thinking remained industrially relevent during its development, and to improve the likelihood of industry based evaluation, ideas that emerged during the development of Aspect-Oriented Think- ingwere discussed with industry based practitioners on a regular basis.
This approach was necessary because continuous close collaboration with industry would require a significant methodological shift for most companies and their employees. This would have been a very expensive and risky proposition for most potential partners. It was felt that Aspect-Oriented Thinkingneeded to be complete, well documented and of demonstrated potential value before industry could be expected to risk an investment in effective evaluation of the approach.
3.3 The Aspect-Oriented Thinking research approach
3.3.2.2 Phase Two - Ongoing development and evaluation
The objective of Phase Two is to satisfy the second part of the research aim to
evaluatethe approach developed during Phase One. In addition, Phase Twoaims
to further developAspect-Oriented Thinkingin collaboration with industry and as part of the interdisciplinary research community. It is an iterative process, based on the ‘Industry-as-Laboratory’ research life-cycle described in Section 3.2.3.4, which will continue until it is determined that Aspect-Oriented Thinking is of no further value to industry or it has been refined to a point where additional research would be ineffective.
Phase Twowill comprise the following activities depicted in the lower horizon-
tal partition of the activity diagram shown in Figure 3.6.
• Analysis - Assess the potential value of further research. The purpose of this activity is to determine if further research is likely to be of value. If so, the subsequent activities of Phase Two will be undertaken. If not, the research program will end.
• Proposition - Improvements to Aspect-Oriented Thinking. In this activity, improvements to the Aspect-Oriented Thinking approach will be proposed. These improvements will be modelled and described as refinements