Validity and reliability are important aspects for the trustworthiness in both qualitative and quantitative research (Cohen et al., 2011). As Merriam (2009) demonstrates, ‘all research is concerned with producing valid and reliable knowledge in an ethical manner’ (p.209) and these two aspects should be taken into consideration in ‘the way in which the data are collected, analysed, and interpreted’ (p.210).
Validity refers to ‘the integrity of the conclusions that are generated from a piece of research’ (Bryman, 2012, p.47). There are many kinds of validity. However,
116 both qualitative and quantitative research emphasise internal and external validity. Internal validity is concerned with the extent to which the data can explain certain events, social practices or phenomenon and how the events can be presented by the data, which sometimes is discussed using the concept of accuracy (Cohen et al., 2011; Leech, 2006). The aim of external validity is to achieve the transferability of findings (Cohen et al., 2011) so that the results of certain research can be generalised and applied to the wider research contexts. It is suggested that in quantitative research rigid sampling strategies, the use of measurement for certain questions and the use of statistical analysis should be included to minimise invalidity and maximise validity (Bryman, 2012; Cohen et al., 2011). Issues of validity in qualitative research are often questioned by positivists (Shenton, 2004) as such research is considered to be lacking in a set of rules for designing instruments and data analysis. However, many researchers argue that validity should be considered and addressed in many forms such as ‘honesty, depth, richness and scope of the data collected, the participants approached, the extent of triangulation and the disinterestedness or objectivity of the research’ (Shenton, 2004)). In addition, credibility and transferability were regarded as internal and external aspects of validity to enhance trustworthiness for qualitative research (Guba, 1981; Lincoln & Guba, 1985).
Reliability is concerned with the consistency and repeatability of the measures of a concept (Bryman, 2012; Joppe, 2000) which is usually addressed in quantitative research. Stability, equivalence and internal consistency are considered as three main forms of reliability (Carmines & Ziller, 1979 cited in Cohen et al., 2011). Measurements can be stable over time and over a similar sample if the instrument reliable. Through equivalent form and inter-rater reliability, similar results and agreements between researchers can be achieved. Conducting tests with two halves of the instrument is required for internal consistency to understand whether it is reliable (Cohen et al., 2011). Based on the understanding of three forms of reliability, it can be seen that ‘reliability’ is a
117 concept of testing and evaluating of information elicitation (Golafshani, 2003). In this sense, what matters is the evaluating quality of research (Stenbacka, 2001) based on its research purposes and research paradigm (Healy & Perry, 2000) and the procedure to achieve its purposes (Patton, 2002). As qualitative research is concerned with the understanding of individual practices in social contexts, it is important to know that ‘published descriptions are static and frozen’ (Florio-Ruane, 1991) in the ‘ethnographic present’ (Linclon & Guba, 1985). Therefore, the concept of dependability should be considered in qualitative research, which closely corresponds to the notion of reliability in quantitative research. Dependability can be achieved by providing details of research design, data gathering and by reflective appraisal of the project in specific research contexts (Shenton, 2004).
Even though this study adopted both qualitative methods, focus groups and semi-structured interviews, and quantitative methods, questionnaires, the research did not aim to achieve generalisations and predictions with respect to adolescents’ perceptions of reading in both UK and China. The same procedure and instruments were applied in the administration of the methods in both UK and China in order to obtain a clear description of perceptions based on participant literacy practices in different social and cultural contexts.
As perceptions of reading in the digital age between the UK and Chinese adolescents had been under-researched, general ideas of what students were thinking about reading with different formats in real life should be gathered at an exploratory stage. Therefore, focus groups were adopted as the first stage of the data collection.
Regarding focus groups, the researcher managed to approach target participants with school support. Questions were well prepared through discussion with the research supervisor and school teachers to ensure that students could fully
118 understand what questions meant without confusion. Meanwhile, focus groups were piloted carefully. Feedback from focus groups regarding the researcher’s questioning style, structure of questions and usage of words was analysed. Participants in focus groups were encouraged to talk freely about their experience of reading and literacy practices in order to gather as much related data as possible to enrich understanding of adolescents’ perceptions and to help to set the research constructs and research items for the questionnaire.
Questionnaires were used as a research instrument in order to work as a convenient and efficient way for the exploration of adolescent perceptions of digital reading because of the difficulties in the real research which were discussed previously. Validity and reliability of questionnaires can to some extent be achieved at the beginning because the research items were based upon results of focus groups. In addition, even though the sampling strategy for questionnaires was non-probability sampling, due to practical difficulties, the researcher tried not to recruit respondents from only one class or one school. A pilot study was carried out to make sure each research item could be interpreted easily and understood with similar interpretations among students in terms of words and expressions, which was described in detail in the previous section. Changes were made based on students’ feedback and discussion with colleges, teachers and the research supervisor.
Semi-structured interviews were conducted individually so as to have a deep look at the underpinnings of students’ perceptions, demonstrated through the results of questionnaires. Research questions were discussed with both the research supervisor and school teachers before pilot studies. Sequencing and structuring of questions, words and distance from students were improved after piloting with some students. I talked to interviewees and teachers to gain basic ideas about reading related practices before the start of interviews. Adequate engagement seemed to be achieved through informal talks so that students
119 tended to talk more openly (Merriam, 2009).
How researchers deal with data can to some extent affect validity and reliability. In qualitative research, the same data can have different interpretations. It is critical to interpret data within the context, so called ‘ethnographic present’, because social practices are embedded in highly contextual settings (Merriam, 2009). Therefore, I analysed focus groups and individual data within the contexts of technology integration, digital text availability, the current curriculum, education system and so on. In this study, validity and reliability were achieved through careful data collection procedures, including design, collection, and analysis with the support of rich and thick data.