Design of questionnaire 56!

Consideration of research objectives

The aim of this questionnaire was to address research objective 2, namely:

O2 – To develop and evaluate an innovative Design for Environment tool with improved

performance.

The questionnaire results are the method of evaluating the success of the new tool. Therefore, it was necessary to define two sub-objectives that would determine the success of the outcome. The sub-objective were defined as follows:

1. To assess the users experiences with the tool regarding the performance of individual aspects.

2. To assess the performance of the guidance components and content relevance. The outcome from this questionnaire will determine the successful completion of O2.

Determination of survey administration

According to Bryman and Bell (2007) the self-administered questionnaire and the structured interview are in many ways very similar methods of research, the main difference being that with the structured interview method there is a trained interviewer present to ask the questions. However, with the self-administered questionnaire the respondent is responsible for reading, understanding and answering the questions. There are many disadvantages to the use of a self- administered questionnaire, including issues such as the respondent can not be prompted or probed further regarding answer choice, there is a greater risk of missing data and low response rates causing bias, and issues with asking to many non-salient or open-ended questions or inducing respondent fatigue (Bryman, Bell 2007). Despite these disadvantages there are some prominent advantages which make the self-administered questionnaire more attractive than structured interviews. The self-administered questionnaires are cheaper and quicker to administer. This is especially appropriate for this research given the nationwide location of potential respondents and the limitation of resources available.

The self-administered questionnaire was hosted online for quicker administration and completion at the respondent’s convenience, eliminating possible ‘interviewer effects’. Online hosting at SurveyMonkey.com ensured the greatest availability with the aim to maximise response rates, allowing respondents to access the questionnaire via a URL link embedded in correspondence emails. The added bonus of this method meant that the completed questionnaires were managed via the online host enabling instant access to respondent completed questionnaires, reducing time spent gathering feedback.

who have not received training for the task and often lack motivation to do the job well, are solely responsible for understanding and completing the questionnaire. These have been summarised as following:

1. A self-administering questionnaire should be self-explanatory;

2. A self-administered questionnaire should mainly be closed-answers, i.e. checking a box, clicking on a response, or circling a number;

3. The question forms in a self-administered questionnaire should be few in number; 4. A questionnaire should be laid out in a way that seems clear and uncluttered;

5. Provide redundant information to respondents by having written and visual cues that convey the same message about how to proceed (Fowler 2009).

These principles were adopted throughout the design and development of the questionnaire. Care was taken to ensure the self-explanatory nature of the questionnaire with minimal written instructions. All questions led to closed answers, with only a few open ended ‘text-boxes’ where it was thought unpredicted answers would occur. The questionnaire consisted mainly of questions with attitude scale answers, i.e. either Yes/No or five-point Likert scale. The sequence and layout was considered to reduce clutter. Clear ‘next’ and ‘previous’ buttons were used for simple navigation.

Wording

The wording of the questions and answers were kept simplistic in order to avoid ambiguity, and to reduce completion time. Questions are formulated to ask respondents for information that can be readily accessed. Attention was taken to ensure that each question “was as clear and precise as possible so that all respondents interpret and all understand the same thing”(Synodinos 2003). The questions were simply phrased to ensure the respondents fully understood and gave relevant answers. Care was taken so that all questions were “as concise as possible to convey the intended meaning and respondents should be able to answer them with relatively minimal

meaningful to the respondents where necessary, i.e. common units and metrics. This was especially important due to the overseas origin of many postgraduate students.

Sequence

A flow chart was used to assess and optimise the questioning sequence during the initial generation of the questions and through content and intention refinement. As recommended by Synodinos (2003) “similar questions should be grouped together and the within topic order should be from the general to the specific”. The questions were subsequently organised into four parts, namely:

1. Previous tools – the use of environmental design tools before this study; 2. Proposed tool – the use and performance of the new tool;

3. Strengths and weaknesses – the strengths and weaknesses of the new tool; 4. Improvements – potential areas for tool improvement.

Response choice

When designing the questions, the type of choice given for the answers of each question is integral. In order to avoid ambiguity and ensure the respondent finds an appropriate degree of agreement with an answer it was necessary to use a five-point subjective continuum scale, from very positive to very negative. The respondent is then “asked to consider the labels, consider their own feelings or opinions, and place themselves in a proper category”(Fowler 2009). There are two main issues with this type of response as explained by Fowler. Firstly, the respondents will differ in their opinion of what the labels mean, and secondly, an ordinal scale measurement like this is relative, resulting in different results depending on the scale presented (Fowler 2009). As a five-point scale was common to every subjective continuum scale used the concern of varying results due to scale was not an issue. As previously mentioned, simple

misunderstanding the labels used. An example of a potential question and associated answer labels used is given as follows:

How easy did you find this tool to use? 1 = Very hard 2 = Hard 3 = Neutral 4 = Easy 5 = Very easy

Pretesting

Pretesting was conducted by three members of staff at the University of Liverpool and pilot testing was conducted by a small group of four post-graduate students. The pretesting helped to check that the questionnaire was easy and relatively quick to complete, and it ensured that the concepts under consideration were clearly explained.