Soundness of the empirical research

In the following the validity and reliability are discussed in the context of qualitative research. In addition, means for achieving validity and reliability in this research are presented. In table 3, the used terms are presented together with their counterparts from positivistic research approach and as well as the means for achieving them.

Qualitative as well as quantitative research is evaluated by using the terms reliability and validity. Reliability is traditionally defined as the extent to which a research procedure yields the same result, whenever it is carried out. Validity, in turn, is the extent to which research gives correct answers. These concepts defined as above cannot be used, as such, in qualitative research. Reliability and validity are based on the assumptions that link positivist philosophy and positivist approaches into methodology. The most important of these assumptions is the assumption of unchanging universe.

The assumption of the constantly changing world which is linked to the qualitative research seems to rule out any systematic research since it implies that we cannot assume any stable properties in the social world (Silverman 1993; p. 146). Reliability as such is therefore not applicable to the qualitative research. The questions to ask in qualitative research are: how can we be reasonable sure that the findings would be replicated if the study was conducted with the same participants, in the same context, and how can we be sure that the findings are reflective of the subjects and the inquiry itself, rather than the product of biases and prejudices on the part of the researcher (Marshall and Rossman 1989).

Table 3. Means for achieving data quality. Bolded text are used in this study. (See Lincoln and Cuba 1985; Wallendorf and Belk 1989).

Term in Term here Means for achieving

positivistic research

Reliability Dependability Observation over time Explanation of change Documentation of procedures

Internal validity Credibility Respondent validation Triangulation

Prolonged engagement Regular on-site team interaction

External validity Transferability Documentation of procedures

Seeking limiting exception

Emergent design (used to some extent)

Objectivity Confirmability Triangulation Reflective journals Auditing

Respondent validation

Integrity Prolonged engagement

Triangulation

Safeguarding informant identity

Kirk and Miller (1986, p. 72) suggest that for reliability to be calculated in qualitative research, it is incumbent on the scientific investigator to

document his or her procedure. This includes interview guides, the actual interviewing, documentation and analysis (ibid.). The interview guide used in this study was commented by Tuula Mittilä and my supervisors and it was also pre-tested. Also, some minor changes were done to the interview guide after the first interview. As mentioned, all the interviews were tape recorded and transcribed, but the other sources of data, for example, the

discussions with the managers from the Avionics division were not recorded. Notes were made during these discussions and they were transcribed immediately after the conversations. The procedure of data analysis is fully described in chapter 5.4.

Lincoln and Cuba (1985, p. 290) have replaced the term reliability with the term dependability, i.e. the extent to which interpretation was constructed in a way that it avoids instability other than the inherent instability of a social phenomenon. Following the lines of Lincoln and Cuba (1985) Wallendorf and Belk (1989, p. 77) argue that it is important to ascertain the extent to which explanation is enduring and the extent to which it derives from peculiar convergence of a particular time and place. In order to achieve dependability observation over time and explanation of change are needed (ibid.). The observation over time and explanation of change would have in this study (as in any other study) demanded a longditunial research method, which is not used here. Instead, the change processes attempted to be captured by asking questions about the changes in the evaluation process, changes in the relationship, changes in perceived satisfaction etc. By using a cross-sectional research method, this is all that can be done to ensure the dependability of the study. It is, however, possible after this study is completed to go back and ask the same questions from the interviewees and that by this method reach dependability.

The other aspect of dependability is the relatedness of explanation to a certain time and place. To make a differentiation between general explanations and time and space specific explanations clear, three types of frameworks (preliminary, modified and refined) are presented in this study. In building the preliminary framework, which is based on the theory, different theoretical perspectives related to the phenomenon under study were studied in order to form as a complete picture about the phenomenon as possible. The time and space specific explanations are used in making conclusions on the basis of the empirical research, and they are taken into the modified framework (built on the basis of theoretical framework and empirical research). Only if differences between theoretical (preliminary) and modified framework were regarded as not being due to the characteristics of the special empirical reality, were they included to the refined framework.

Validity is usually defined as the extent to which an account accurately represents the social phenomena to which it refers (see Hammersley 1990, p. 57). Internal validity refers to what extent we can infer that a causal relationship exists between two (or more) variables. In qualitative study, internal validity is according to Marshall and Rossman (1989) derived from the embeddedness of the data in the setting it studies. The empirical setting of this study is presented in chapter 5.2. and the theoretical framework in chapter 4.2..

In post-positivist inquiry Lincoln and Cuba (1985) suggest the term

the adequate and believable representations of the constructs of reality studied (ibid.). The means for achieving credibility in the following is mainly related to the empirical research, and here it is related to the theoretical research. In order to build adequate and believable representations of the reality, in theoretical analysis, different theoretical aspects were used in building the theoretical framework. Thus, the reality or phenomenon was looked at many different perspectives, and these different perspectives allowed the multifaceted analysis on a theoretical level. The many different perspectives also caused problems in analysis. The different perspectives use different terms and concepts when discussing about the same phenomenon. In order to form believable and adequate concepts, from this type of situation, the perspective of phenomenon, instead of the perspective of a certain theoretical tradition, has to be taken.

An other issue related to the credibility, in theoretical research, is the match between theoretical constructs and empirical data. As mentioned earlier (see chapter 5.4.) this was not fully reached in this study. This can be argued, by the role of the case study, in this study and by the extensiveness of the preliminary framework. The case in this study has the role of an instrumental case study (Stake 1994, p. 243). Thus, the aim is not making generalizations on the basis of the case study, but in facilitating the conceptual analysis. Also, as the preliminary framework is quite comprehensive, it was already assumed that all the concepts of the preliminary framework could not be covered by one empirical case.

In this study, four means were used to improve credibility: respondent validation, triangulation across sources, prolonged engagement and regular team interaction (see Lincoln and Cuba 1985). These methods are discussed in the following.

Respondent validation was in this study used as a means for assessing the internal validity of the study. The final report of empirical analysis was given for comments to the manager of the Avionics division and to the manager of the aircraft-electronic office of the depot. The tentative results were presented to two of the managers of the Avionics division in September 1997. In December 1997 the results also were presented to approx. 20 representatives of the Avionics division. These discussions provide me as a researcher a strong belief that the empirical findings and interpretations I have made were valid.

Triangulation across sources is often used in ethnographic fieldwork, but it can be applied also to other kind of qualitative research. Triangulation across sources requires that the evidence for interpretation is developed from interaction with several informants, particularly several types of informants (see Wallendorf and Belk 1989, p. 72). In this study, the interviewees were from several organizational levels (see figure 9) and represented different types of informants. In this research, the main source of information were the interviews. The unofficial discussions with the managers of the Avionics division also served as an important source of

data. Different types of written material: history of the Avionics division, annual reports, and reports from the media, were used as secondary sources of data. In answering the research questions the main source of data were the interviewees, which were supported by the discussions. The secondary data and discussions were mainly used in order to form a complete picture about the case relationship.

The buyer The seller Managing director Division manager Manager avionics maintenance/ Quality manager Supervisor (responsible for prising) Group leader Top management/ generals Depot management Superior Electronic office Type office Commercial office Wing

Figure 9: The different organizational levels of interaction in the relationship. The levels from which the interviewees were from are bolded.

Prolonged engagement is needed for developing an understanding of the phenomenon and its context. In this kind of research, spending time and observing the interaction in relationship was impossible. Thus, in this research prolonged engagement can be supported by maintaining contact with the persons involved in the research, and by getting to know the business, relationship and partners involved before hand. The maintaining contact with the interviewees was done both before and after the actual interviews, and it facilitated in building a trusting atmosphere. Information concerning the partners, the relationship and the business was gathered from the secondary data sources and from the discussions that took place before the interviews.

Regular on-site team interaction is also a method which enchanted the credibility in empirical research. Regular on-site team interaction is usually linked to the ethnographic field research, but it can also be applied in other

kinds of research. In ethnographic field research regular on-site team interaction means interaction between researchers in the field when gathering data. In this study it was used as numerous discussions with Tuula Mittilä, who also did research in the seller company, before the interviews, right after the interviews, and during coding and analyzing the data. These discussions clearly gave ”the second opinion” for many problems that occurred during the research process. For example, the discussions helped in forming the interview guide, in making the interviews and in analyzing the data. From the point of view of credibility these discussions with another researcher closely related to this study gave the valuable comments which helped the researcher to see the problem or matter from another point of view.

Lincoln and Cuba (1985) have replaced the term external validity by

transferability, i.e. to the extent to which working hypotheses can also be employed in another context, based on the assessment of similarity between the two contexts. This means that the results of the study should also be employed in other (similar) context, but not in all possible contexts (ibid.). Lincoln and Cuba (1985) suggest an easy answer to the demands of transferability is that if other researchers are concerned with the applicability of findings, in another context, they should be able to do research using similar methods in another time and place and then compare results(see also Wallendorf and Belk 1989, p. 76).

The uniqueness of this case was one of the reasons for selecting the particular the case (see chapter 5.2.). However, the uniqueness of this case also affects the transferability of the findings into other contexts. In order to avoid this transferability problem modified and refined frameworks are built.

Lincoln and Cuba (1985) also see transferability as a purpose to understand the phenomena across a variety of circumstances. Phenomena according to Wallendorf and Belk (1989, p. 76) can be a type behavior or variety of behaviors, i.e. the behavior phenomenon could occur through a variety of behaviors. In this study, the studied phenomenon, relationship quality evaluation, can occur through variety of behaviors. The procedures of establishing a transferability of this kind are, for example, seeking limiting exceptions and emergent design. In this study, in analyzing of the empirical data, the data was given an opportunity to speak and the data also modified the theory. Thus, it can be argued that method of data analysis that reminds emergent design was used (cf. emergent design by Wallendorf and Belk 1989, p. 76). The data was also handled in a way that the exceptions were taken into account in the analysis, but seeking limiting exceptions was not used as preliminary data analysis method.

Lincoln and Cuba (1985) also added confirmability as a mean for judging the soundness of qualitative research. Confirmability refers to the ability to trace a researcher’s construction of an interpretation by following the data and other records kept (Lincoln and Cuba 1985, p. 290). Because

post-positivist research recognizes that there does not exist absolute objectivity, the term neutrality is inappropriate to be used in qualitative research. The post-positivist research accepts the notion, that at best, the researcher can become conscious of, and hopefully also reduce his/her biases, semantic accents and ethnocentrism (see Wallendorf and Belk 1989, p. 78). The methods that can be used are triangulation, reflective journals and auditing (ibid.). Triangulation was used in this study, but reflective journals and auditing were not used. Also, respondent validation, which was used and was discussed earlier can be regarded as being a source confirmability.

Triangulation here primarily means collecting data by multiple members of a research team and later comparing the data. In this study, this was done by separately collecting the data, but comparing it in discussions afterwards with Tuula Mittilä. It has to be noted, however, that the data we collected was only partly comparable (i.e. the interview guides only partly covered the same issues). In spite of this, the comparison and the discussion served at least for me, as a source of confirmation of findings and interpretation.

Integrity is also a mean for assessing the trustworthiness of qualitative research developed by Wallendorf and Belk (1989, p. 80). Lack of integrity can arise when the informants fear or dislike the researcher, or they try to attract the researcher (ibid.). Integrity is defined as the extent to which the interpretation is unimpaired by lies, aversions, misinformation or misrepresentation by informants (Wallendorf and Belk 1989, p. 70). Prolonged engagement in order to construct trust and triangulation are the means for facilitating integrity. These means were discussed earlier. Here safeguarding informant identity is discussed.

In order to safeguard informant identity the interviewees were informed that their names would not be published, and that the single statements cannot be traced to a certain individual. Also, the admission for recording the interviews was asked for the interviewees. The purpose of the study and the use of the data collected were also explained to each interviewee. The interviewees were free to ask any questions concerning the research. It was also pointed out that the interviewees should answer according to their own perceptions and feelings and not to think what is the right or wrong answer. The aim of the interviews like the discussions succeeded in most cases. One of the interviewees was, however, not obviously feeling very comfortable with the research and he was afraid to give wrong answers. This was quite natural, because the interviewees were bound to secrecy concerning certain matters of the relationship. But the other interviewees did not seem to have fear of breaking this bound. Concerning that certain interview, a trusting atmosphere was tried to be built by asking easy and general questions first and this was at least partly successful. However, in analyzing the data, these problems in interviewing process were taken into account.

6. RESULTS OF THE EMPIRICAL