Survey Procedure

5.3.1 Focus Group

A preliminary questionnaire, cover letter and reminders were prepared after a review of relevant literature (Dillman, 1978; Heberlein & Baumgartner, 1978; Seller, Stoll & Chavas, 1985; Sutherland & Walsh, 1985; Mitchell & Carson, 1989; Stevens et al., 1991; Arrow et al., 1993; Amirnejad et al., 2006; Wattage & Mardle, 2007). These were then given to a focus group consisting of 6 staff members at the New Zealand Centre for Ecological Economics (NZCEE) for their expert evaluation. Each member was asked to do the following:

a. Read the cover letter and complete the questionnaire as would a normal respondent and record the time they took to complete the survey.

b. Go through the cover letter and questionnaire again in detail, this time analyzing the structure and content of the two documents and the reminders to see if they adequately address the needs of the study and ensure they are straight-forward enough for an average New Zealander to handle without any serious difficulty.

c. Comment and suggest any changes that need to be made to the documents before a pre-test is carried out.

Comments from the focus group indicated that the questionnaire adequately addressed the research question and that it was logically structured. The language was viewed as clear enough for an average person to handle and the time required to complete the survey was reasonable at between 15 and 20 minutes. A copy of the survey questionnaire was also sent to HBRC for their comments especially on how well the study captures the important aspects of the current restoration programme. Their response was that overall they were satisfied with the survey questionnaire but advised against including income as some respondents may object to disclosing their incomes.

5.3.2 Pre-test

A sub-sample of 159 was randomly drawn from the main sample of 1,117 households, originally drawn from the Hawke’s Bay Region, to test the survey instrument. Details of the sampling technique will be provided later. The main objectives of the pre-test questionnaire were to provide feedback on the content and structure of the instrument, provide an insight into the distribution of WTP values, and provide an indication of the extent of problems associated with value formulation. In this regard the pre-test questionnaire differed from the draft final survey in three ways: (1) it contained a question requesting respondents to comment on the structure and composition of the questionnaire; (2) an open ended question was used to elicit WTP values; and (3) a question requesting respondents to comment on the WTP amount they stated was posed.

Because of time constraints it was decided from the onset that there would be no follow-ups to respondents on the pre-test survey. To ensure a reasonable number of responses are obtained, a fairly large pre-test sample was employed. Kirkland (1988) and Omwenga (1995) used pre-test sample sizes of 40 and 75 respectively. Some

pertinent results of the pre-test are presented in Tables 5.2 and 5.3.

Table 5.2 Pre-test Survey Responses Number of questionnaires mailed out Number of questionnaires returned undelivered Number of questionnaires assumed delivered Number of responses No-Response 159 16 143 25 (17.48%)* 118 (82.52%)*

* Calculated as a percentage of the number of questionnaires assumed delivered.

The pre-test survey results may be affected by self selection bias if non-respondents differ from respondents significantly in terms of socio-economic, demographic, behavioural, and attitudinal attributes. Results from a study by Wellman et al. (1980) suggest that there is no important difference between early respondents and the reluctant respondents who are reined in through follow-ups to increase the response rate.

Table 5.3 Summary of responses to the questionnaire structure and content

Issue Agree (%) Disagree (%) No-opinion (%)

Some of the questions were hard to understand.

14.3 71.4 14.3

Some parts of the questionnaire were hard to follow.

20.0 70.0 10.0

The questionnaire was too long. 25.0 75.0 0.0

Respondents’ comments were reviewed before the final survey questionnaire was produced. At least 70% of the respondents disagreed with the statements listed in Table 5.3 with about a quarter of the respondents indicating that the questionnaire was too long. The comments on the length of the questionnaire did not affect the length of the final draft as the 10 page questionnaire complies with the requirements of the Total Design Method (TDM) of Dillman (1978).

Question 25 (see Appendix 2.2b) concerns value formulation. Mitchell and Carson (1989), Bishop and Heberlein (1979), and Arrow et al. (1993) contend that asking respondents to formulate monetary values in response to an open-ended CV question presents them with an extremely difficult task. Results in Table 5.4 seem to support this view as only 42.11% of the respondents indicated that they were quite certain with their

valuation. It is interesting to note that the majority of respondents who stated that they were quite certain about their valuation reported zero WTP values. The results are pleasing as they indicate that those who responded took the survey seriously.

Table 5.4 Responses to the WTP value formulation

Response Percentage

Quite certain

There is no way we can be certain about the amount

42.11% 15.79%

Just a guess 36.84%

Not sure what to think 0%

Other 5.26%

5.3.3 Main Survey

The Pekapeka Swamp contingent valuation survey was administered to a sample of 958 households in the Hawke’s Bay Region, from November 2008 through to January 2009 using a mail survey questionnaire as discussed earlier. The cover letters were personalized with the name and address of the respondent; the salutation was “Dear Householder”; and the letters were individually signed. The first mail out was on November 7, 2008 and consisted of a cover letter, survey questionnaire and an addressed postage-paid return envelop. Two reminders were mailed out at approximately three week intervals. The first follow-up was a reminder/thank-you post card to all non-respondents encouraging them to respond. The second reminder consisted of a replacement questionnaire with a more emphatic cover letter and an addressed postage-paid return envelop.

The follow-up procedure adopted in this study differs slightly from the TDM of Dillman (1978) by using only two follow-ups instead of three, and different timings of the mailings. Our objective in taking this approach was to achieve an acceptable response rate within a given budget and time frame. The three stage follow-up as recommended by Dillman is to increase the response rate by reigning in the reluctant respondents. Apart from time and budget constraints, we considered the findings of Wellman et al. (1980), cited earlier, in adopting the two stage follow-up. Bennett,

Morrison and Blamey (1998) used a two stage follow-up in their study using a reminder card and a complete re-mail of the questionnaire to stimulate responses.

Concern was expressed at the timing of the last mail out as it was only six days prior to Christmas. The alternative was to wait until the third week of January 2009 which could have prolonged the length of the study. Advice from Dillman (1978, p. 180) is to “avoid mailing close to holidays and the entire month of December” because of increased mail during this period and increased likelihood that a lot of people will be holidaying away from their homes.

The cut-off date was January 23, 2009, exactly 11 weeks after the first mail-out. Non- response checks were not made as a two-stage follow-up was adopted as explained earlier. Responses were categorized as indicated in Table 5.5.

Table 5.5 Response categories for the Pekapeka Swamp Survey, 2008

Response details Response category (code)

Questionnaire returned undelivered 0

First time response (Early) 1

Response after first reminder (Middle) 2

Response after second reminder (Late) 3

No response 4

All the information capture by the survey instrument was carefully coded21 (see Appendix 2.3) before being captured electronically on an excel spread sheet.