Conceptual level (GOAL): A goal is defined for an object, for a variety of reasons, with respect to various models of quality, from various points of view, relative to a particular environment.
Operational level (QUESTION): A set of questions is used to charac- terize the way the assessment/achievement of a specific goal is going to be performed based on some characterizing model. Questions try to characterize the object of measurement (product, process, resource) with respect to a selected quality issue and to determine its quality from the selected viewpoint.
Quantitative level (METRIC): A set of data is associated with every question in order to answer it in a quantitative way. The data can be [objective or subjective]1.
This approach lends itself very well to creating questionnaires. Here one defines the goal, which leads to the research questions, which again leads to questions one can use on a questionnaire.
2.3
Research Paradigms
In the book “Experimentation in Software Engineering: An Introduction” [31] the authors Wohlin et al. present the two main research paradigms in empirical research; qualitative and quantitative research.
Qualitative: A qualitative research is performed on the object of interest in it’s natural setting. This is done by gathering descriptions of the object of interest from different sources and then generalizing the descriptions into a conclusion or hypothesis.
Quantitative: A quantitative research is a research in which one tries to identify a relationship between to objects or group of objects. This research is usually performed in a controlled or semi-controlled set- ting in which the researcher can modify certain values or settings and measure the effect on the object of interest. The relationship can be precise, e.g., a correlation factor, or it can be a more general ordering, e.g., method A is better than method B.
2.4
Empirical Strategies
Wohlin et al. [31] identify three main approaches, or strategies, for collecting data for a research; surveys, case studies, and experiments.
1
10 CHAPTER 2. RESEARCH METHOD
2.4.1 Survey
A survey is an activity in which one tries to make a snapshot of the current situation. It asks a subset of a population a set of questions regarding their perception and understanding of the object of interest. These answers are then analyzed, generalizing the information relating to the object, enabling assertions or conclusions to be drawn. Surveys are often used in market research and other polls, as they are suitable to perform exploratory research, e.g., by charting new opinions or conceptions.
Wohlin et al. [31] identify three main objectives for conducting a survey:
Descriptive surveys are used to enable assertions regarding a population without considering the reasons for this assertion
Explanatory surveys are used to explain certain phenomenon in a pop- ulation
Exploratory surveys are often used as a pre-study for another research, allowing the researcher to verify that no key point is missed
2.4.2 Case Study
A case study is an activity in which one tries to measure a single object or phenomenon at a particular time. A strong point of the case study is that it is performed in a semi-controlled setting. In this setting, the researchers are able to control at least parts of the situation, but still allow the participants to act as if everything is normal.
For instance, a case study is a good way to compare a new method or principle with a baseline. If a large company is considering changing it’s software development methodology, it may be a good idea to allow one or a few teams to try it out first. If the teams and projects are chosen carefully, making sure that they are comparable to other teams and projects in the organization, the researcher has a good starting point. The team can then use the new methods, and the researcher can record the necessary metrics to compare this new method to the old one.
There are two major limitations to case studies; they cannot be repli- cated, and they have limited validity. Since a case study is a measure taken at a specific time in a specific situation, there is no way to reconstruct this exact situation at a later time. Because of this, it is difficult to generalize the results of such a case study, since there is no viable, scientific way to validate the conclusions. This again limits the validity of the results. Often the results of a case study can only be used by the group or organization which performed the research, and then again, only for a relatively short period after it was performed.
2.5. LITERATURE REVIEW 11
2.4.3 Experiments
An experiment is a research activity in which the researcher controls every aspect of the situation in which the test subjects are placed. Since the researcher then can vary every single aspect of the situation randomly, it is well suited to confirming theories or conventional knowledge.
As an example, a researcher wishes to test the calculation speed of a cer- tain mathematical operation in different programming languages. She then has control over which developer writes the program, which programming language she uses, which compiler is used, which computer/architecture is used, etc. By varying these variables, the researcher can identify the rela- tionships among them and draw a conclusions.
A great strength of an experiment is that the results are valid far beyond the organization which performed it. Anyone can redo the experiment and check it’s validity, and thus it allows for general theories or concepts to be proven.
2.5
Literature Review
As a part of this research, literature review will be used to answers parts of the research questions. A literature review is a process wherein one goes through existing literature on the subject. This is done to give the researcher a thorough understanding of the subject matter, as well as helping to identify gaps in the current knowledge where further research is appropriate [5].