Research Data Collection Process

The present research used a multiple case studies research method. Yin (1994) points out multiple case studies will help researchers prevent from the possibility of misrepresentation and also ensuring validity and reliability of data collection. There are three main methods in data collection process that have been used; face to face interview (interview), focus group interview (focus group) and survey questionnaire. All the data collection processes are done almost simultaneously. For the interview and focus group purposes, researchers have been guide by an interview guide and focus group question guide (Taylor and Bodgan, 1984) which was an output from GQM process. In the same time, survey questionnaires have

Q1 Q2 Q3 Q4 Q5 Data collection Method1 Data collection Method2 Data collection Method3 D E F I N I T I O N

been mailed to identify respondents. The overall data collection process is shown in figure 4.5

Figure 4.5: Research Concurrent Triangulation Design- Data Collection

Data Collection (Qualitative) Data Analysis (Quantitative) Data Collection (Quantitative) Interpretation based on quantitative and quantitative results Data Analysis (Qualitative)

In order to help researchers identified suitable companies as research respondents, researchers have browsed Enterprise Ireland website (www.enterprise-ireland.ie). The list and information of the software companies in this website were been used as a main reference in searching and identifying suitable research respondents. Beside that several related internet website such as linked-in (www.linkedin.com), internet search engines such as Yahoo (www.yahoo.com), Google (www.google.com) and through several IT experts who were contacted to support the identification process.

From the list of the companies being complied and suggested, only companies which have less than 25 employees (ie. compatible with the definition of a VSE) and having software development activities as their main business operation have been selected and evaluated. This process allowed the researchers to list several potential Irish Software VSE companies as our research respondents. In order to be more focused on the research issues, researchers gave higher priorities to companies which have less than 10 employees and this group of companies is then labelled as Micro-VSE. This process was then followed by the companies that have more than 10 employees but less than 25 employees and this group of companies is then labelled as Larger-VSE. The similar division process also has been made by Laporte et al., in developing a new international standard for VSEs (Laporte et al., 2008). All suitable and potential companies had been emailed or phoned to ask their willingness to participate in this research study. From the feedbacks, only 4

companies are agreed to involve in face to face and/or focus group interviews and 10 companies are willing to fill in the questionnaires.

In overall, the data collection process took around 8 months period from August 2008 to April 2009 to complete, which include identifying suitable companies’ process, contacting and confirming potential respondents’ process, conducting individual and focus group interviews process and distributing and receiving questionnaires process. Meanwhile, the data analysis processes which involved the qualitative and quantitative data have been done in parallel with the data collection process. All the received data were consolidated and analysed according to the specific methods as discussed in part 4.7.3. In general, the data analysis process has taken around 8 months (February 2009 - October 2009). Appendix A shows the list of all potential and involved companies for the present study.

•

Interview

The face to face interview approach was used in this study in order to discuss topics in depth, to get respondents’ candid discussion on the topic and to be able to get the depth of information of the study situation for the research context. According to Kvale (2007), interview reveals both direct and indirect data. Direct data are responses that subjects provide to direct questions, they are spoken responses. Indirect data are the less obvious or hidden information conveyed by gestures, body language, or a lack of eye contact. In an interview, interviewers can follow-up on half-answered questions; they can probe for deeper responses. He added that an interview data can be used to develop objective questions or closed-ended questions for a questionnaire. Flick (2001), stated that there are three types of interview formats that can be followed; structured, unstructured and semi-structured interview. A structured (focused) interview means that the questions are developed a head of time with some opportunity to ask pre-planned, open-ended, probing questions. This way, there are few variations, and the questions are asked in a specific order. As with observers, the interviewer is practiced on how to ask the questions, and how to probe the subject for depth. An unstructured interview can be used to explore alternative opinions, attitudes, or beliefs. It can help to identify new types of information and define areas of importance that might not have been

thought of ahead of time. A semi-structured interview includes the open-ended and specific question. It has been designed to gather not only the information foreseen, but also unexpected type of information (Li, 2006). In this study a semi structured interview approach was used in order to gather information from the target respondents.

For this data collection phase, the researchers have interviewed 11 respondents from 4 identified Irish software companies which fall under VSEs categories. This process was divided into 2 types of interviews namely personal interviews and focus group interviews. Table 4.2 show the list of the companies involved and type of interviewed method that have been applied.

Table 4.2: List of involved companies

No Company Interviewee No of Staff(s)

Involved Personal Interview

Focus Group 1. Company B CEO 1 X Company B Development Team 3 X 2. Company C Software Development Manager 1 X Company C Development Team 2 X 3. Company AE Joint-CEO 1 X Company AE Joint-CEO 1 X

4 Company AF Chief Architect 1 X

Company AF Senior Software

Developer

1 X

The interview processes have been done in different places and time. The respondents have been divided into 2 categories; the managers and the development team. For the managers researchers have applied a face to face interview method and for the software development team, researchers have adopted the focus group interview method. All interviewed session were happened approximately 40-90 minutes and has been recorded with the respondents’ permission. The details of the

focus group approach are explained in the next bullet point. Furthermore, in order to guide the interviewer to gather a data during an interview session, an interview guide (Appendix B) has been developed (Taylor and Bodgan, 1984). The interview guide consists of close and open ended question, and some related notes about the direction in which to drive the interview under difference circumstances. Lastly, in order to ensure the interview questions do not divert from the research objective, the interview questions were prepared following the GQM (Lethbridge et al., 2005) principle technique as been explained in section 4.7.1

•

Focus Group Interview

Powell and Single (1996) have defined focus group as “a group of individuals selected by researchers to discuss and comment on, from personal experience, the topic that has been a subject of the research”. The focus group interview approached was used in this study because team members develop the software and the existence team interactions helped to release inhibitions amongst the team members. In addition this method could activate forgotten details of experiences and also generate data better through wide range of responses. Focus group interviews were also chosen because it was the most appropriate method to study attitudes and experiences; to explore how opinion were constructed (Kitzinger, 1995) and to understand behaviours, values and feelings, (Patton, 2002). According to Powell and Single (1996), the advantage of focus group is the ability to help the researcher in identifying quickly a full range of perspectives held by respondents. They added that focus groups expand the details that might have been left out in an in-depth interview. Kruger and Casey (2000) supported that focus group technique is a proper way to understand and explore how people think and feel about the issues. They also added that focus groups also elicit data that allows a better understanding of the difference between groups of people. Furthermore, in order to gather data and guide interviewer in the focus group interview, a focus group interview guide as in Appendix C have been developed (Morgan et al., 2008). The focus group interview guide started with a broad and open question that more toward the participant perspectives before narrow the question into the research interest. The entire focus group interview questions have been gathers from GQM process.

•

Survey Questionnaire

The third data collection method for this study is survey questionnaire. Survey questionnaires were used in this study because it could facilitate the data collection and analyse the substantial data quickly. Moreover this method could help this study avoiding bias problems when interpreting qualitative data (Pfledger and Kitchenham, 2001). There are several advantages and disadvantages about applying survey questionnaires. The main advantages of using survey questionnaire are cost and time effective and also could easily collect a substantial data for the research study. While lower percentage of return rate of the filled questionnaires is the main disadvantage of this method (Lethbridge et al., 2005). Appendix D shows the survey questionnaire sample for this research.

The questions in the survey questionnaire were divided into 5 main parts: the background information, the questions on people issues, questions focussed on the software development process, questions focussed on the software development knowledge and questions focussed on the software development team issues. The survey questionnaires have been sent to substantial number of VSE companies by email and post mail in order to get a good response rate. Table 4.3 shows the list of the companies that have been involved in this exercise.

Table 4.3: List of companies – survey questionnaire

No Company Survey

Respondent Survey Questionnaire

1. Company D Owner/COO X

2 Company E Managing Director X

3. Company F Director X

4 Company G CEO X

5. Company H Development Manager X

6 Company AC Practice Director X

7 Company AD Managing Director X

8 Company AG CEO/CTO X

9 Company AG Senior Software Developer X