Reliability and validity of the study

The plausible outcomes from qualitative research are highly dependant on how the researcher can prove the reliability and validity of the study. This includes not only the final research results but also the process of the research and the theory generalisation from the data.

The aim of the empirical parts together with final conclusion is not only to show what the analytical framework looks like and to give a description

of social settings but also to provide a way for controllability and empirical testing of these descriptions (Peräkylä 1997:201). The latter may be achieved by a clear and consist application of reliability and validity.

Reliability

The aim of reliability is to show the ‘degree of consistency’ where the researcher obtains the same results on different occasions or where different researchers measure the same research setting and receive the same results (Hammersley 1992:67). The reliability of the study – or credibility – can be achieved by following standardized methods, for example by ‘cross-checking and cross- validating sources during the fieldwork’ (Patton 2002:544-545, Silverman 2001).

In this study the attempt has been to obtain reliability by studying the same research question or phenomena not only by interviews but also by field observations, such as participation in activities organised by the immigrant associations and ETNO. Nevertheless, high reliability is not a rule for high validity (Hellevik 1977:140).

Validity

Validity of the data refers to proving the trustworthiness of the data. Attention is also paid to how validity of the theory is constructed, amongst others. By comparing the categories from data with the research setting and the interviews it is possible to find the ‘so called’ theoretical validity (Svensson 1996:214). This, in turn, is connected with construct validity. Here, the aim is to figure out the ‘true statements’ of the data by constructing useful concepts and by creating categories that can give validity to the study. An additional way to achieve validity is by analytic induction, in other words by exploring small amounts of collected data where the hypotheses are tested and reformulated (Silverman 2001: 240).

In this study most useful way to find the validity of the data has been by comparing statements of interviewee’s earlier and later interviews, i.e. to find out if there are gaps in the interviewee’s story and if so, make control questions in the final, open-ended interviews. The interviews are compared with a broader setting of data: a) memos done from events where I had taken

part, such as meetings, parties and seminars, b) other existing data of the association in question.

In addition, I have gone through different kinds of documents which I received from the immigrants. I tried find out their authenticity by paying attention to the form and context of the documents and to explore if there were some deviating aspects from its original purpose. According to Platt it is also important to focus on issues in the document that fit “too well” to the purpose (Platt 1999). For example during the analysis of the meeting protocols of one immigrant association it appeared that some of them looked very “simply” made both in their context and lay-out38_{. To find out the authenticity of the} protocol the context of the documents were compared with the issues they referred to (such as cooperation projects and seminars organised by the association). A further picture of the meaning and how they were perceived was gained by asking the Chair the issues discussed in the meetings39_. Usingtriangulation

Further ways to improve the validity of the study can be achieved by

triangulation. This allows the researcher to find possible gaps in her data

and to construct a theoretical framework. The term triangulation has both a practical and metaphoric origin. The term is taken ‘from broadcasting; radio triangulation means determining the point of origin a radio broadcast by using directional antennae set up at the two ends of a known baseline’ (Oakley 2000:67).

Michael Patton has identified four kinds of triangulation that provide reliability and validation for research: ‘1. Methods of triangulation: checking

38 the meeting protocols included basically only place, date, name of participants, opening of the meeting, decisions done and topic discussed. the latter two issues were only written with two sentences.

39 normally, the chair started to talk about the current happenings without asking. in this case it appeared that the chair had not written any protocols in english and had done it mainly because of the researcher. this was certainly not the purpose! on the other hand during the collection of the material the chair started to write more extensive protocols, in some cases with a help of a finnish nGo assistant. another aspect that might have caused unreliability was the lack of competence in the finnish language. furthermore, there are cultural differences between management of associational activities. finnish associational life is known for an organised way that may sound strange to cultures differing from the finnish one.

out the consistency of findings generated by different data collection methods, 2. Triangulation of sources: Checking out the consistency of different data sources within the same method, 3. Theory/perspective triangulation: Using multiple perspectives or theories to interpret the data, 4. Analyst triangulation: Using multiple analysts to review findings.’ (Patton 2002: 556).

In this study the focus was mainly on the all but the last type types of triangulation: the consistency and validity of the research was controlled by comparing the way material was collected. This consists of literature (including both scientific and non-scientific literature), interviews and participation observations. Another kind of triangulation was based on the comparison of the collected data. The data consisted of different types of sources like face-to- face and telephone interviews, booklets, seminar reports and homepages done by the members of the associations, as well as of research notes and memos written during the research.

4.5 Summary

This chapter has outlined the methods and data of this study. During the data collection, semi-structured (telephone and e-mail questionnaires), structured and open-ended interviews were conducted. Additional data collected consisted of documents and participant observations. Participation in activities organised by immigrant associations, and on some occasions by civil servants, provided a good opportunity for me to get a better insight.

The collection of data has been done from a more general to a more specific form. Therefore, a general illustration of the data has been obtained by a charting of immigrant associations in Finland. This was followed by studies of: the cooperation forms between local authorities and immigrant associations, activities of immigrant women’s associations and of the function and change of The Advisory Board for Ethnic Relations − ETNO.

The data consists of some overlapping information due to the diverse research settings. For example, the criteria for data collection for Research Setting 1, was very different from the others. In other words, it was somewhat problematic to quote telephone interviews in comparison to e.g. face to face interviews conducted with members from immigrant women’s association.

Nevertheless, both interview techniques provided lots of new and valuable information. The number at the end of quotations refers to the code of the interviewee.

During the course of data analysis, notice has been given to the reliability and validity of study. Reliability of the data has been achieved by following standardized methods, for example by cross-checking and cross-validating data, while exploration of suitable categories has helped to construct validity. A more profound illustration of the analytical process will be described in the Chapters 5, 6 and 7.

Finally, I would like to stress that I tried to take into consideration ethical issues during the data collection. During the research process data and personal information was carefully protected. During the conduct of the interviews particular attention was paid to the interviewees. The interviews were only recorded with the permission of the interviewees and their names have remained anonymous in this thesis. Information from civil servants has been collected by sending a letter asking for permission to conduct an interview. In the e-mails the objective and level of confidentiality was explained. Permission for data collection was granted in all cases. The names of the immigrant associations have been used with the permission of the interviewees.

PART III

5 The Organisational Scenery