Online Questionnaire Design

Questionnaires are a data collection technique that consists of a series of structured and appropriate written or verbal questions (Webb, 1992), the answers to which, translating to provide the researcher with information about the overall research question (Brace, 2008). Brace (2008) also stipulates that; a questionnaire must uplift, motivate and encourage the respondents to become involved in the questionnaire, to co-operate and to complete the questionnaire, and the error of response should then be minimised.

Effective online questionnaire design encompasses: simplicity; cultural independence;

completeness; relevance and neutrality (Swoboda, Muhlberger, Weikunat and Schneeweiss, 1997), in an attempt to eliminate sampling, measurement and

non-response errors (Lumsden and Morgan, 2005). Online questionnaire design also incorporates unique questionnaire features such as pop-up instructions and error messages (Lumsden and Morgan, 2005) and required completion of certain questions (Evans and Mathur, 2005) in order to eradicate non-responses significant to the accuracy of the data. The online software, incorporating a simple structure with required completion of all questions would eliminate non-response. An effective online questionnaire design process is depicted by Lumsden and Morgan (2005) in Figure 4.9.

Figure 4.9 Online Questionnaire Design Process

Source: Lumsden and Morgan (2005)

4.7.3.1 The Question Content

Questions included within an academic research questionnaire must attribute information relevant to the research objectives (Webb, 1992) and are therefore most effectively developed after the formulation of the research objectives (Dillon, Madden and Firtle, 1994). The content of the online questionnaire is most effective when it:

applies to respondent; is technically accurate; asks one question at a time; and specifies the response task (Dillman et al. 2009). Although the primary information of the function of the question content is to make sure that the demands of the research objective for the particular data are met (Brace, 2008), the question’s language and phrasing must be at the appropriate level to aid respondent understanding (Webb,

Define the research question

Divide the research question

into sub-categories

Determine and profile the target

audience

Design and implement the content

Pilot the questionnaire

Administer the questionnaire

1992). The question content of online questionnaires is required to respect the intelligence and time restraints of respondents (Van Selm and Jankowski, 2006) in order to minimise questionnaire abandonment.

4.7.3.2 The Question Phrasing

Despite of any intricacies and sophistications of the academic content of the questions, the question phrasing of questionnaires needs to be understandable and answerable by the respondents (Dillman et al. 1994). The questions need to be unambiguous and the following criteria met to ensure that they: test for alternative question meanings (Webb, 1992); use simple and familiar words; use specific and concrete words to specify the concepts clearly; and use as few words as possible, ‘yes’ means yes and

‘no’ means no (Dillman et al. 2009). Brace (2008) also specifies a limit of only twenty words per question for online questionnaires.

4.7.3.3 Types of Response Format

There are two primary types of question response formats: open-ended where the respondent can choose words freely to answer the question; and closed-ended where the respondents are asked to provide options from a selection of answers (Crask, Fox and Stout, 1995). Scalar questions are also a utilised response format and provide respondents with a list of answers from which to choose (Dillman et al. 2009). Online questionnaires in particular allow for a diversity of question response formats:

dichotomous; multiple-choice; scales; multimedia format; both single-response and multiple-response; and open-ended (Evans and Mathur, 2005).

4.7.3.4 The Question Sequence

A typical question sequence is organised like a conversation (Dillman et al. 2009), it is systematic, and begins with general questions to capture the interest of the respondents leading into the more specific and difficult questions (Webb, 1992). If questions are as simple as possible and follow a logical flow, error and respondent frustration are minimised (Crask et al. 1995). Questions that are related need to be clearly grouped with questions at the beginning of the questionnaire salient to all respondents (Dillman et al. 2009). Online questionnaires allow skip patterns within the question sequence to reduce complexity for the respondent and reduce response errors (Schonlau, Fricker and Elliot, 2001; Lumsden and Morgan, 2005). The question

sequence of online questionnaires can also be organised in such a way that further questions cannot be viewed until the questions available have been answered (Evans and Mathur, 2005). This type of question sequence control can increase respondent frustration levels, as the questionnaire can seem endless (Schonlau et al. 2001), unless a progress indicator is included within the online questionnaire design (Evans and Mathur, 2005).

4.7.3.5 Questionnaire Layout

Simple issues such as format, spacing and position of questions are important to the effectiveness and results of a questionnaire (Webb, 1992; Malhotra, 2007) and the accuracy of the data collected (Brace, 2008). Simple format decisions such like using radio buttons can affect fundamental response issues such as decreasing response error (Couper, Traugott and Lamias, 2001). Effective questionnaires are consistently spaced and response options are aligned vertically in column or horizontally in a row (Dillman et al. 2009). Plain online questionnaires are noted to result in a better response rate, completeness and completion time (Dillman, Tortora, Conradt and Bowker, 1998; Manfreda Lozar, Batagelj and Vehovar, 2002). The length of computer screens of the online questionnaire (Sheehan and McMillan, 1999), and use of multiple item screens also have an effect on completion times and response error (Couper et al. 2001).

4.7.3.6 Questionnaire Pre-Testing

Pre-tests are conducted on questionnaires in order to identify potential problems and mistakes within the questionnaire before it is too late to rectify them (Webb, 1992;

Dillon et al. 1994). Pre-test methods include: requesting feedback from people with experience of questionnaires; conducting cognitive interviews of the complete questionnaire to identify wording, question order, visual design and navigation problems; and conducting a small pilot study to evaluate interconnections among questions, the questionnaire, and the implementation process (Dillman et al. 2009).

4.7.3.7 Measurement and Measurement Scale

In the context of a questionnaire methodology, measurement refers to the allocation of numbers to the characteristics or objects being measured (Webb, 1992; Dillon et al.

the real world. There are four levels of measurement: nominal scales which allocate numbers to objects to indicate their belongingness to certain categories; ordinal scales which rank the characteristics in order of importance; interval scales which number the characteristics in order and distance with no definite zero point; and ratio scales which have all the properties of nominal, ordinal and interval scales, but they also have an absolute zero point (Webb, 1992; Dillon el al. 1994; Malhotra and Birks, 2000). Nominal scales are also defined as a set of answer categories (Dillman et al.

2009). All of the levels of measurement focus on three important components:

measurement is a process, measurement translates characteristics into numbers, and measurement is governed by formal rules (Webb, 1992).

4.7.3.8 Likert Scale

Likert developed and published this uni-dimentional rating scales in 1932 (Fowler, 1984), in which attitude dimensions are scored or rated on a pointed scale representing strength of agreement or disagreement (Brace, 2008). The scores or ratings for each attitude dimension of each respondent are subsequently summed to provide an overall attitudinal score for each dimension and individual (Fowler, 1984).

Factor analysis can utilise the data collected in likert scale format by grouping the attitude dimensions that have similar response patterns to determine factors that impact the research question (Tabachnick and Fidell, 2007). For online questionnaires, scaled response formats are particularly effective as respondents are noted to be less likely to respond using either extreme ends of the scales (McDonald and Adam, 2003).

Likert scales were utilised in the research methods of the reviewed plus size online shopping motivations literature; some research measured with a five point scale (e.g.

Rintamaki et al. 2006), but mostly with a seven point scale (e.g Nguyen et al. 2007).

There is limited agreement as to be optimum number of points on an online questionnaire scale (Brace, 2008). It is deemed that too many would make the distinction between options meaningless (Dillman et al. 2009) however, indicating that the average of a seven point scale is deemed most effective. A forced seven point Likert scale will hence be adopted in this research, including a ‘no opinion’ category to improve the accuracy of the data (Malhotra, 2007) by ensuring that the respondent answered positively or negatively if they had an opinion, resulting in less response error when a respondent genuinely ‘doesn’t know’ (Brace, 2008).