Survey

Rationale and uses of questionnaires

Questionnaires can be employed in quantitative research to generate structured and numeric data (Cohen et al., 2012; Nelson & Cowles, 2015). However, some researchers use questionnaires to obtain qualitative data for exploration in terms of individuals’ beliefs, perspectives, attitudes and perceptions (Harris & Brown, 2010). It seems that questionnaires can be used in both qualitative and quantitative research to meet research needs and purposes. As discussed earlier, gaining knowledge of the patterns of what the students thought about digital reading in new textual landscapes through data in breadth would provide insight into the nature of literacy and reading literacy in different social and cultural contexts. Employing questionnaires could be a good way to provide evidence of patterns of perceptions and practices of reading in the digital age among UK and

100 Chinese adolescents. Building upon the findings of the focus groups and upon the existing literature, questionnaires, therefore, were chosen to explore patterns of adolescents’ perceptions of reading in breadth.

As has been argued, questionnaires are often used because rich data can be gathered relatively quickly at low cost without the presence of the researcher (Bryman, 2012; Cohen et al., 2012; Newby, 2010). There are several ways to distribute questionnaires, such as by using postal, emails or web links, and they allow participants to finish a questionnaire at their convenience, which may require no presence from the researcher. However, Cohen et al. (2011) argue that the absence of the researcher may lead to a low response rate. They suggest that a possible way to ‘ensure a good response rate’ is to have ‘the presence of the researcher’. They argue that the presence of the researcher or the questionnaire designer can ‘enable any queries or uncertainties to be addressed immediately’ (p.404). However, the presence of researchers who are normally strangers to participants may cause ‘a sense of compulsion’ (ibid.) to respondents. Such a challenge was considered in this study before I started collecting questionnaire data. During the collection in the UK schools, I showed no presence because, on the one hand, the teachers suggested that it would be easier to arrange students to complete questionnaires if teachers could take charge of the administration. On the other hand, I was told that some students preferred online questionnaires so that they could complete them after school. In China, I appeared with the main teacher of the class to wait for students’ inquiries and completion.

In addition to challenges relating to the response rate, some other concerns, such as question types and the use of language are widely discussed (Bryman, 2012). Bryman (2012) argues that it is hard to ‘probe respondents to elaborate an answer’ in a questionnaire. In other words, in-depth data may not be easy to collect even from open-ended questions. Meanwhile, it is possible that individuals may not be willing to participate if they need to spend a long time

101 writing answers. Considering these challenges, the questionnaire in this study was designed mainly with multiple choices questions and Likert scale type questions. Regarding the use of language, the students’ literacy rate was considered. 12-16 years-old students were invited to complete the questionnaires, and thus the words used needed to be understood by both older and younger students. School teachers in the UK and China were consulted by sending them questionnaire drafts with an inquiry about the use of language before piloting questionnaires. How the questionnaire was designed will be discussed later.

Designing and conducting the questionnaire Designing questionnaires

As explained in the section on research strategy, the questionnaires in this study were adopted to explore patterns in adolescents’ perceptions of reading and their practices of reading through various media. The findings from the focus groups and the knowledge of the existing literature laid the groundwork in terms of what questions to ask in the questionnaires. Based on what the literature review (e.g., McKenna et al., 2012; Melnink et al., 2009) and the findings from the focus groups suggested, preferences for certain text formats, feelings of reading certain texts formats and beliefs about online reading comprehension were mentioned. Meanwhile, as adolescents’ perceptions of digital reading were explored in the light of literacy practices, investigations of reading activities of both printed and digital texts were also included.

As research constructs for the questionnaire in this study had been clearly established through the focus groups and the literature review, a number of multiple choice questions and Likert scale questions were used as these types of questions would be straightforward for participants (Brace, 2013). Multiple choice questions can ‘enable respondents to select the response that most closely represents their views’ (Cohen et al. 2012, p.384). Meanwhile, the teachers in

102 both countries said that they could spare no more than 20 minutes for the questionnaire completion. Given this practical issue, having multiple choice and Likert scale type questions could be straightforward and easy to finish within the limited period of time. The questionnaire had six sections. Two sections covered questions of facts regarding adolescents’ access to digital texts and their reading practices of print and digital reading. Three other sections dealt with their views about reading through different media, such as preferences and strategies for online reading comprehension. The last part covered participants’ basic information: gender, age and year group. At the end of the questionnaire, a question regarding whether the participant would like to participate in the follow-up individual interview was also added. A very brief introduction of the purpose of this study was given at the beginning on the questionnaire, together with a guarantee of confidentiality.

The first part was designed to explore adolescents’ access to digital reading both in and outside school. As suggested by the existing literature, adolescents are exposed to the digital environment of reading. Some findings of focus groups also suggested this. The exploration of adolescents’ access to digital texts would be useful to understand their reading environment, which may influence how they perceive digital reading. Multiple choice questions were designed to refer to both in and out-of-school settings, including accessing digital texts through schools’ digital devices, personal devices and others’ devices. One simple open question, termed ‘Other’, was included in case participants wanted to provide answers that were not covered in the multiple-choice options.

Followed by the section on access, questions about reading activities were asked in the second part. Based on the existing literature, reading in the digital age is known to involve far more than reading from paper (e.g., Lankshear & Knobel, 2011). Meanwhile, the students in the focus groups had claimed to use several reading activities with multiple modes. Therefore, understanding their reading

103 activities would help to gain an understanding of what reading in the digital age means for adolescents. Multiple choices questions were designed which covered reading that transferred from a printed format into a digital one. Reading that happens in social networking, texting or using a search engine were included. 10 reading activities in school and 12 activities outside school were listed based on the focus groups and the literature review. One open-ended question at the end of each setting was given so that they could write any other frequent reading activities that they had in daily life.

The three remaining parts were designed as Likert scale questions with a five point scale to explore adolescents’ preferences for text formats, the nature of a reader when reading print and digitally, and their beliefs about skills for online reading. Rating scales (Conradi et al., 2013; McKenna et al., 2010) are frequently used to test students’ attitudes toward reading. Likert scales do not simply look for ‘yes/ no’ answers. The degree of opinion could be achieved by using answer choices ranging from one extreme to another (Allen & Seaman, 2007). However there have been ongoing debates over using a middle range choice on Likert scales or not (e.g., Garland, 1991; Moors, 2011). It is argued that omitting a mid-point may cause respondents to veer more towards positive answers (Worcester & Burns, 1975). Garland conducted a study focusing on the effects of having no mid-point in Likert scale questions, and found the opposite results to Worcester and Burns. It may be that having a middle choice on a Likert scale is content specific (Garland, 1991). In this study, middle range choices were included because this research focused on the students’ personal opinions of reading both digital and printed texts rather than simply making choices from provided answers. For example, the mid-point in the question of ‘I prefer digital texts when reading for enjoyment’ may suggest either negative tendency or tendency of using both printed and digital texts for enjoyment reading. The evidence of mid-points used in the questionnaires would provide evidence for elaboration in the follow-up interviews.

104 As suggested by the literature and by the focus groups, adolescents seem to read certain text formats depending on their reading purpose. Therefore, questions about preferences for differences were designed in the third part of the questionnaire. Meanwhile, it has been argued (e.g., Ito et al., 2009) that many adolescents have a range of interactions with digital texts, which suggests the changed nature of the reader in the digital age. The findings from the focus groups also suggested similar issues. In addition, it was found that many students in the focus groups claimed that they learnt better with certain text formats. Hence, the fourth section included questions concerning adolescents’ notions of and behaviours in various textual engagements. Regarding online reading comprehension, strategies and skills for reading online have been recognised as being different to those involved in print reading (Leu et al., 2011), which was also suggested by the focus groups in this study. Questions in the fifth part were concerned with reading skills and strategies for online reading comprehension, including information location, evaluation, and synthesis. A detailed description of how the questionnaire was designed based on the findings from the focus groups and the existing literature will be discussed later in the Findings chapter.

Web questionnaires (See Appendix 3) were also designed because one UK school asked if their students could complete it online. The structure and question order were the same as the paper-based questionnaire. The questionnaire was designed in English (See Appendix 4) and translated into Chinese (See Appendix 5). Questionnaires in two languages were piloted to assure meanings were well expressed with appropriate uses of words.

Pilot study

The issues of participants’ interpretation of questions, use of words, measurement and range of response choices were considered in piloting the questionnaire. Pilot studies were conducted with both UK and Chinese students using both the web questionnaire and the paper-based questionnaire. It

105 appeared to make no difference whether respondents completed the paper or online version.

Three UK students (aged 13-15) were invited to complete ‘mock’ questionnaires. One student could not find a suitable time, therefore a copy of the questionnaire through a web link was sent to him. The student’s feedback, including the use of language, and the total time spent on the questionnaire was sent to me through email. Ideas concerning some questions were given according to what he understood about these. He thought that the questionnaire was well designed and could be finished within 15 minutes. The other two students completed the questionnaire in the researcher’s presence. Their reactions to questions were recorded. After completion, several questions were asked based on their recorded reactions. One student hesitated in one question for almost 30 seconds at the beginning. He explained that he was not sure whether it was necessary to put examples after the questions. Another student who was 14 years old said that some younger students may not be able to understand the word ‘Neutral’ because it sounded too academic.

Two Chinese students were asked to take part in the pilot study online because of the distance. Copies of the questionnaire were sent through QQ (a popular multifunction online chatting tool). Feedback was sent to me by texting on QQ. One student said that some words were not easy to understand. Some sentences were too long which made them a bit tired of reading.

Based on all feedback, some changes were made. Examples were not introduced in the questionnaire. The word ‘Neutral’ was changed to ‘Not sure’. Some sentences were shortened. The final questionnaire was then designed so that it could be finished within 15 minutes.

106 Data collection (questionnaires)

Paper-based copies of the final questionnaire and a link to the web questionnaire were sent to UK teachers. Teachers were expected not to tell potential participants about the content detail of the questionnaire beforehand. As I would not be present in the UK schools, some expectations about questionnaire completion and some explanations of potential inquiries were given to the teachers. According to talks with the UK teachers after collection, they walked around the classroom and supervised students to ensure full completion. They also helped to check whether students chose the middle range answer for every question without carefully reading the question. Therefore, all paper-based questionnaires distributed with the help of the UK teachers were returned with very high response rates. Apart from distributing paper-based questionnaires, web questionnaires were also distributed. The web link was sent to several schools. The teacher from one school responded that students could complete questionnaires in an ICT class by using computers in the school. One organisation helped to send the link to students as well. 168 questionnaires in total were returned through web questionnaires. After checking all web questionnaires one by one carefully in terms of time spent and answers, all these questionnaires can be considered as valid data. See Table 3.3 regarding data collection of questionnaires in the UK schools.

107 Table 3.3: Data collection of questionnaire in the UK with return rate of 99.3%

I travelled to China and showed up on site during the questionnaire distribution. Copies of the questionnaires were sent to head teachers. A copy of the questionnaire was sent to the teachers of sampled students before my arrival. I was invited to meet with the main teachers and the head teacher of each Year group (Grade 7 to 10) every time before distributing the questionnaire to sampled students. Some rules in terms of contact with students and time were discussed at the meeting. Three days after this meeting when all things had been arranged by the main teachers, questionnaires were distributed to 8 classes in two schools (4 classes in each school). The researcher presented with the teacher in case of any inquires. Teachers asked students to finish the questionnaire carefully with no question left blank. With the teachers’ help, the return rate was very high (See Table 3.4). Some students did not put the information of their Age or Gender. These questionnaires were not used due to the incompleteness of the questionnaires.

108 Table 3.4: Data collection of questionnaires in China with return rate of 93%

3.5.3 Individual interview