Instruments

The following subsections discuss the instruments developed for this study. They include the pre-test and post-test, tutorial tasks, guided exploration tasks and post-study questionnaire. Between the pre- test and post-test, a tutorial and a guided exploration task were conducted. After the pre-test and post-test, a post-study questionnaire was used to gather information on participants’ impressions and experiences with the ElleVis visualisation tool (see Figure 6.1).

Pre-test and post-test

The organisation of the pre-experimental one-group pre-test and post-test design is shown in Table 6.5.

Table 6.5 Organisation of the one-group pre-test and post-test design

Steps Procedure

(on all participants) Aim

Step 1 Pre-test To measure the degree of the personal and shared understanding before the intervention, using a set of questions.

Step 2 Intervention To expose the test population to the ElleVis visualisation tool. Step 3 Post-test To measure the degree of change in the personal and shared

understanding using the same participants by administering a set of questions identical to that used in the pre-test.

The pre- and post-tests were structured in four sections with twenty-two options, and they were identical in every respect. (See Appendix C.8 for the exact wording.)

Section 1: This assesses participants’ understanding of which lake values will be affected by the current (2009) Environment Canterbury (ECAN) opening regime in managing Te Waihora/Lake Ellesmere:

• Between 1 April and 31 July, open the lake at 1130mm AMSL

• Between 1 August and 31 March, open the lake at 1050mm AMSL

Section 1 is designed to capture participants’ understanding of how the current ECANprotocols affect lake values.

Section 2: This assesses participants’ understanding of which lake values will be affected when Te Waihora/Lake Ellesmere is managed at a level of 1300mm AMSL all year round. (At 1300mm AMSL, the lake is considered very high (Taylor & Hughey, 2009).)

Section 3: This assesses participants’ understanding of which stakeholders’ values benefit from the following opening regime:

• From 18 April, open lake at 600mm AMSL

• From 24 July, open lake at 1200mm AMSL

• From 20 September, open lake at 500mm AMSL

(Below 600mm AMSL and above 1000mm AMSL, Te Waihora/Lake Ellesmere is considered to be ‘low’ and ‘high’ respectively (Taylor & Hughey, 2009).)

Section 3 is designed to capture participants’ understanding of how lake values are affected when the scenario is a low lake level at the start and later parts of the year and a high lake level at mid-year, to reflect extreme lake level patterns within a single year.

Section 4: This assesses participants’ understanding of which opening regime in Table 6.6 – Scenario A, B or C – is most likely to benefit Banded Dotterels (a type of bird for which Te Waihora/Lake Ellesmere is considered a habitat stronghold).

Section 4 is designed to capture participants’ understanding of how well they know which opening regime is deemed “appropriate” for this type of bird.

Participants’ responses to the above questions before and after usage of the ElleVis visualisation tool were designed to facilitate a measure of changes in personal and shared understanding of the impact of opening regimes on lake values at Te Waihora/Lake Ellesmere. These scenarios have previously been used in lake opening protocol meetings regarding future directions for Te Waihora/Lake Ellesmere.

Table 6.6 Opening regimes for Te Waihora/Lake Ellesmere used in Section 4 of the pre- test and post-test

Scenario A Scenario B Scenario C

From 1 April, Open lake at: 1130mm AMSL From 1 Aug, Open lake at: 1050mm AMSL From 18 April, Open Lake at: 1300mm AMSL From 24 July, Open Lake at: 1300mm AMSL From 20 Sept, Open Lake at: 1300mm AMSL

From 18 April, Open Lake at: 600mm AMSL From 24 July, Open Lake at: 1200mm AMSL From 20 Sept, Open Lake at: 500mm AMSL

Tutorial and Guided-exploration

Participants undertook the tutorial and guided-exploration after they completed the pre-test but before they undertook the post-test (see Figure 6.1, dotted box).

The tutorial is intended to provide a familiarisation with the detailed features of the ElleVis tool and the test procedures. The design of the tutorial enables participants to: (a) conduct the guided- exploration tasks (see next paragraph); and (b) quickly locate the various features of ElleVis. The tutorial involves participants going through a series of tasks to give them an overview of using the tool. The purpose of the guided exploration is to ensure participants use the ElleVis tool to explore the effects of the various opening regimes on lake values at Te Waihora/Lake Ellesmere. Pictorial representations of lake values – for example, species of birds and fish – that are of significance to stakeholders, are provided on the guided-exploration task sheet. The purpose of having a pictorial representation is for the participants to be informed of stakeholder interests that they may be less familiar with and which become less or more significant under different lake opening regimes.

The guided-exploration tasks required participants to run the ElleVis program under various opening scenarios that are provided to them and in a variety of rainfall conditions (wet, normal and dry years), and to observe the changes to the visualisation in order to answer questions such as the following:

“Which of the AMSLs below is favourable to Pied Stilts (a type of bird found in the lake)?

Having completed the tutorial and the guided-exploration, participants then completed the post-test, described above.

Post-study questionnaire

This questionnaire was administered at the very end of the testing procedure. The purpose of the post- study (distinct from the post-test study above) questionnaire was to obtain participants’ impressions of the visualisation tool (ElleVis). Participants were encouraged to answer three questions:

Question 1. Do you think that the interactive visualisation tool has enhanced your understanding of the lake management issues? (Participants are asked to rate themselves on a 3-point scale: “A lot”; A little”; or “Not at all”.)

Question 2. Do you think that the interactive visualisation prototype presents an accurate picture of the lake behaviour?

Question 3. Please give any general comments that you have about this prototype. (For all questions, space was provided for respondents to offer additional comments)