Questionnaire design process
Step 5: Dependability and Trustworthiness of the questionnaire
4.9 Data Analysis Techniques
Both qualitative and quantitative data were collected during the fieldwork at PAAET using the following data collection methods: interviews and questionnaires. A significant amount of qualitative data was collected from the interviews and very few from the open-ended questions of the questionnaire. Therefore, such data was analysed manually due to the language of the transcripts that were in Arabic and the researcher had to translate the findings into English.
As mentioned before in this research, the mixed methods were used that include semi- structured face-to-face interviews and questionnaires. After gathering the data from the participants in PAAET, data analysis is an important step of the research as it helps the researcher to examine the collected information and prepare the conclusions based on them.
A significant amount of qualitative data was collected from the interviews and the open- ended questions of the questionnaire. In qualitative data, according to Saunders et al., (2007), Bryman (2004) and Robson (2002), there is no fixed rule or standardised approach, no clear and acceptable set of agreements for analysis of qualitative data. Furthermore, Saunders et al. (2007) explained that qualitative analysis usually involves the development of data categories, determining units of the researcher’s original data to suitable categories, identifying relationships within and between categories of data, and developing schemes to create well-grounded conclusions. On the other hand, According to Oates (2006, p.267) qualitative data analysis involves extracting from the research data the verbal, visual or aural themes and patterns that the researcher thinks are important to the research topic. Collis and Hussey (2003) clarified that many authors have tried to identify what they concern as the main elements of an analysis of qualitative data and stated that general analytical procedures can be use with any methodology; the researcher followed all of the steps outlined below:
1. Convert all rough field notes into the form of written record. 2. Confirm that any materials collected are properly referenced. 3. Read the data many time in order to become familiar with it.
4. Start coding the data as early as possible, coding each concept or theme, as the coding allows for the effective storage, retrieval and organisation of data.
5. Start grouping the codes into smaller categories according to patterns or themes that emerge.
6. Write summaries of the findings at various stages.
7. Use the summaries to construct generalisations that confront existing theories or be used to construct a new theory.
8. Continue the process until satisfied that data collected are sufficiently robust to stand the analysis of existing theories or the construct of a new theory.
According to Collis and Hussey (2003) the value of the analysis of qualitative data is dependent upon the quality of the researcher’s interpretation, and the final step in analysing qualitative data is to evaluate the analysis reached. Different measures can be used to evaluate an interpretive research and consequently, evaluate the quality of the
analysis. Lincoln and Guba (1985, cited in Collis and Hussey, 2003) suggest that four criteria can be used:
Credibility: demonstrates that the research was conducted in such a manner that the
subject of the enquiry was correctly identified and described. Credibility can be improved by the researcher involving him/herself in the study for a prolonged period of time, by persistent observation of the subject under study to obtain depth of understanding, by triangulation by using different sources of evidence, and by peer debriefing by colleagues on a continuous basis. Among those techniques, the credibility of this study was enhanced by triangulation of data collections by different sources of evidence such as interviews and questionnaires. Peer and colleagues’ reviews enhanced the credibility; the researcher and other researchers from different schools within the University of Salford held regular meetings to discuss their research and related methodology.
Transferability: this is concerned with whether the findings can be generalised to another
situation.
Dependability: this illustrates that the research process is systematic, rigorous, and well
documented.
Conformability: this should be used as a criterion where the study has described the
research process fully and it is possible to assess whether the findings flow from the data. According to Yin (2009) there are five analytic techniques used for case study analysis, these being: Pattern Matching, Explanation Building, Time-Series Analysis, Logic Models, and Cross-Case Synthesis.
• Pattern Matching: pattern matching logic is used to compare an empirically-based
pattern with a predicted one. If the case matches the predicted patterns then the case supports the theory in the same way as successful experiments support a theory. If the pattern coincides, the results can help to strengthen the internal validity of a case (Yin 2009).
• Explanation-building: explanation- building is a special type of pattern matching. The goal of this technique is to analyse the case study data by building explanations about the case. Yin (2009) suggested that in explanation-building processes, the findings are compared to any statement or proposition created.
• Time-Series: the time-series technique is a special and more rigorous case of
process tracing, Yin (2009) argued that if the events over time have been traced in detail and with precision, time-series analysis technique may be possible.
• Logic Model: the logic model deliberately stipulates a chain of events over an
extended period of time. The events are in a repeated cause-effect-cause-effect pattern, whereby a dependent variable (event) at an earlier phase becomes the independent variable for the next phase. This process can help define the sequence of programmatic actions that will accomplish the goals (Yin 2009).
• Cross-Case Synthesis: cross-case synthesis is a technique especially relevant to a
research consisting of at least two cases. According to Yin (2009) this technique treats each individual case study separately.
Based on the above description and discussion of different techniques used for qualitative data analysis, the researcher adopted the explanation-building technique as a data analysis method since during the data collection process, unpredicted patterns emerged and needed to be tackled.
Moreover, the questionnaire results are presented using; percentages distribution tables of the comments by the respondents. Analysis of the data was undertaken using the descriptive method; this will allow the participants’ perceptions to be identified. Excel 2011 software has been used (pivot tables) to analyse the data collected from the questionnaire.
As a result of the analysis procedure, the findings from the data analysis were engaged with the other sources that were used during the data collection such as literature and according to the research methodology. Therefore, to reduce the possibility of errors different approaches and techniques were used for investigation.
4.10 Summary
This chapter introduced the research philosophy, strategy and design of the chosen methods. It was well described that this research was a single exploratory case study, as the research explores factors that influence the readiness of OA policy implementation in PAAET that cannot be generalised to other organisations or institutions. Furthermore, a mixture of methods, using both qualitative and quantitative approaches, were adopted to answer the research questions and to fulfil the aim and objectives of this research. Interviews and questionnaires were carefully selected as methods and were designed to avoid improve trustworthiness in data collected.
In the following chapter, the research findings (related to the readiness of OA policy implementation) from the case study Kuwaiti public higher education institute (PAAET) will be highlighted and presented in the relation of the Kuwaiti public higher education environment context.