Questionnaire surveys

Questionnaire surveys are a widely used type of research instrument that allows the collection of information in the form of structured data. The principal advantages of questionnaire surveys are that they can be administered without the presence of the researcher, and potentially to a large cohort; they are relatively straightforward to analyze; and a wide variety of types of data can be obtained through them, depending on their particular design (Frazer & Lawley, 2001; Wilson & McLean, 1994). In addition, they are particularly useful when investigating issues of prevalence, and to examine large-scale patterns in education (Desimone

& Le Floch, 2004).

On the other hand, the main disadvantages are that: they can be time consuming in their design, preparation, and refinement; they can lead to data overload; they do not provide the depth of understanding that interviews and observations do; and their administration always represents an intrusion to participants’ lives (Desimone & Le Floch, 2004; Parlett &

Hamilton, 1972). This last factor may lead to untrustworthy accounts from participants, or simply determine the response rate. Another critical issue with questionnaire surveys is the construct validity of the research instrument. The quality of data is strongly dependent on the instrument design, in particular the issue regarding how similar are the researcher’s constructs to those of participants, and how meaningful the survey instruments then are to participants (Cohen et al., 2007; Wilson & McLean, 1994). Section 4.6 of this chapter provides further insights regarding data validity and trustworthiness.

There are different types of questionnaire surveys: structured, semi-structured, and unstructured. There are also many types of questionnaire questions: open questions, closed questions, dichotomous questions, multiple choice questions, open-ended questions, and rating scales

questions among others (Cohen et al., 2007; Frazer & Lawley, 2001). The selection and use of a particular type of question, or of a particular set of types of questions, will directly depend on the purpose and objectives of the questionnaire survey, and on the research questions being addressed through the instrument. Other key considerations during the design of questionnaire surveys, apart from validity-related issues, are to avoid pitfalls in question writing, being particularly mindful when asking sensitive questions, and considering attractive layouts and organizational structure in an effort to obtain a good response rate (Cohen et al., 2007; Desimone

& Le Floch, 2004).

In the present study, different questionnaire surveys were used throughout the data collection phases. The selection of this research instrument was based on the advantages provided by questionnaire surveys to describe demographically the participants of this study; identify patterns of recurrence and prevalence within participants’ backgrounds (e.g. level of knowledge of the target sustainability issue); and that questionnaire surveys are relatively easy to administer, which in community-based educational study greatly contributes to the logistics of the data collection.

More detail on the particularities of the different questionnaire survey instruments is discussed in the coming sections. All research instruments used in this research, including questionnaire surveys, can be found in the appendix.

4.4.2 Interviews

An interview is a purposeful conversation between the researcher and the respondent, where the purposes can include, among others, to obtain

‘here-and-now’ constructs from individuals, and the ‘projections’,

‘reconstructions’ and/or ‘verifications’ of such constructed entities (Lincoln

& Guba, 1985). In other words, the purpose of interviews is to enter another person’s perspective (Patton, 2002). As mentioned above, one advantage of interviews over questionnaire surveys is that they can provide “depth, detail, and meaning at a very personal level of

experience”, contributing to the achievement of a deep understanding of personal constructs of respondents (Patton, 2002, p. 17).

Another advantage of interviews is that they allow the respondent to move back and forward in time. This is particularly so with unstructured interviews that can access the unique, idiosyncratic, and holistic viewpoint of the interviewee; in contrast, structured interviews seek information more in terms of the interviewer’s framework (Lincoln & Guba, 1985). In both cases, nonetheless, the interview represents a flexible research tool for the researcher, as she/he can control it while still allowing for spontaneity, and respondents can be pressed for complete answers and/or about responses regarding complex and deep issues (Cohen et al., 2007)

An interview represents a relationship between the participants, i.e. both the interviewer and the interviewee are seen as participants, therefore the quality of the data obtained through an interview directly depends on the quality of such relationship (Patton, 2002). Related to this, Oakley (1981) raised some criticisms of the prescriptive nature of this method, arguing that the traditional approach of interviewing collected data from essentially passive informants from a set of pre-determined questions. She also stressed that the role of the interviewer ought to be of a debater, not of a recorder retrieving information from a knowledge-producing object, as in positivistic traditions. Instead, to enhance the relationship existing between the participants of the interview, the relationship between the interviewer and the interviewee needs to be non-hierarchical, with the interviewer prepared to engage with his/her own personal identity in the relationship (Bishop, 1997, p. 32; Oakley, 1981).

As with questionnaire surveys, there are different types of interviews, each one being more appropriate to different purposes and perspectives, and each one with its strengths and weaknesses. Cohen, Manion and Morrison (2007), echoing Patton (1980), recognize four main types of interviews in educational research. The characteristics, strengths, and weaknesses of these types of interviews are presented in Table 4.1.

Table 4.1. Characteristics of the four principal types of interviewing methods found in educational research (source: Patton, 1980).

Type of

interview Characteristics Strengths Weaknesses

Informal

The type of interview depends research purposes and objectives of the interviews. A combination of different types of interviews is encouraged in order to enhance the collection process of data (Patton, 2002). In the context of the present research, interviews were carried out throughout the data collection phases, both in relation to the design and development of an EfS website, and for the investigation of the use of the website by community members in Chile. All four types of interviews described in Table 4.1 were adopted and used during this research, depending on the characteristics and objectives of each data collection phase. More detail on the use of interviews in the context of the present research is described later in this chapter, as well as in following chapters.

4.4.3 Observations

Observational methods of inquiry represent a powerful research tool for naturalistic researchers. The key advantage of observations is the depth of understanding that can be achieved while being in the natural field (Lincoln & Guba, 1985; Cohen et al., 2007). Observations as a research tool become potent when used for fairly long periods of time, as the researcher may be present at different yet related events that inform the research (Parlett & Hamilton, 1972). Observations allow the description of the setting from the researcher’s perspective, providing first-hand experience with the dynamics of the setting. Moreover, the observer may catch and record events that can escape participants’ perceptions by being immersed in a context-dependent routine, and which may be missing in data collected through questionnaires and/or interviews (Patton, 2002). In addition, in a similar way with interviews, observations are flexible and can vary in scope at different stages of a research study (Lincoln & Guba, 1985), allowing the observer to record and organize data on the go (Parlett & Hamilton, 1972).

As with questionnaire surveys and interviews, there are different approaches to carry out observations during educational research.

Observations can be highly structured, semi-structured, or unstructured, depending on the degree and predisposition to ‘look for’ particular aspects,

events and/or situations in the research setting (Cohen et al., 2007;

Patton, 2002). Highly structured observations will know what to look for, and will have the different categories of observations defined in advance.

Whereas unstructured observations will be “far less clear on what it is looking for”, hence deciding what constitutes relevant evidence for the research after the observations were carried out (Cohen et al., 2007, p.

397). In addition, there are degrees of researcher participation in observations. For example, the ‘complete participant’, where the researcher takes an insider role within the group being studied, or the contrasting ‘non-participant’ case, where the researcher is not considered being part of the group under study. In the present study, this second situation characterized the observations, as the researcher was not a member of the target community.

In another aspect, as indicated by Patton (2002), “getting close to the people in a setting through firsthand experience permits the inquirer to draw on personal knowledge during the formal interpretation stage of analysis” (p. 264), recognizing that reflection and introspection are important aspects of field research. Cohen et al. (2007) further highlight that, as observations are continuous in time, to determine what counts as evidence may become cloudy, as ultimately this depends on different factors related to the observer, and to the observed setting (Cohen et al., 2007). This validity issue is where the main criticisms of observations are grounded, as different people will see different things in the same research setting, biased by their own individual backgrounds or worldview. Similarly, questions about the reliability of observations have been raised from positivistic perspectives, in terms of how repeatable data can be.

However, the specificity to the context and depth of understanding is of course where the very essence of naturalistic research lies, and one of the most powerful aspects of observations (Lincoln & Guba, 1985; Patton, 2002). Hence issues of validity and reliability of observations are much the same as with any other qualitative method, for which different procedures can be followed to enhance these, e.g. triangulation of data. For details on validity and reliability aspects see Section 4.6.3.

In the context of the present research, observations of the target community were critical during the early stages of the intervention. These helped achievement of depth of understanding in relation to different aspects of the study, such as the socio-cultural, as well as the ICT and EfS characteristics and needs of the target community. Observations in the target community were carried out at different levels and scope, and within different research frames, depending on the particular study phase of the intervention. These included the spectrum from unstructured observations aimed at understanding the socio-ecological sustainability issues present in the target community, and more structured types of observational research assessing the use of the EfS website by participants during the late stages of the research.