Survey development process

The development of the stated choice survey consisted of:

1. An initial reference group meeting.

2. First survey instrument and choice experiment design.

3. First pilot survey.

4. Firstrevision of the survey and stated choice experiment design.

5. Development of the survey website.

6. Second pilot survey.

7. Second revision of the survey and stated choice experiment design.

8. Implementation of the main survey.

9. Follow-up contact to maximise response.

10. Data processing.

The survey process occurred during the period October 2009 to December 2010.

Reference group

A reference group meeting comprising seven New Zealand car owners residing in the Wellington region was held to discuss car purchases, EVs, and the issues facing car transport. Recruitment was by sending out invitations to a number of

Wellington based email lists. The group comprised four males and three females aged from 35 to the mid-60s. Of the attendees, six had bachelor level degrees and one was a PhD student. Five attendees identified themselves as having a strong

99

interest in environmental issues. Two of the male attendees expressed less concern about the environment and indicated that they were attending because they were interested in new vehicle technologies.

The meeting used an open discussion format, but was structured to gain information on the participants’:

 last car purchase and the factors they considered when making that purchase

 knowledge of the characteristics of LPVs currently in the market (i.e. prices, fuel running costs, range, and performance)

 expectations of the future development of petrol and diesel LPVs

 knowledge and expectations of EV technology

 concerns about EV technology

 views about the risks from the continued use of petrol and diesel fuelled vehicles

 ranking of the factors identified earlier in the meeting that influence vehicle choice.

The participants also gave feedback on the draft of supporting material to be used in the first pilot survey

First pilot survey

This survey comprised a personally supervised pen and paper stated choice experiment, carried out with a pilot sample of 11 people, six of whom had

previously attended the reference group. The experiment design was a fractional factorial design consisting of a random subset of the full factorial design. The design of this experiment consisted of 22 parameters, excluding constants, all with three attribute levels. The smallest viable design found with attribute level balance contained 30 choice tasks³². The order of the alternatives in the choice tasks was randomised to limit the possibility of order effects (Dillman et al., 2009b, p 128).

32 In stated choice experiments the number of choice sets in the experimental design must be equal to or greater than the degree of freedom. The degree of freedom is the number of attributes in the

100

The ChoiceMetrics NGENE software was used to design all the choice experiments in this study. Due to the small size of the sample, the experiment design was not blocked and the respondents were required to complete all 30 choice tasks.

The supervisor’s role during the pilot survey was to elicit feedback from the respondents about the survey, and supporting material, and encourage them to complete all the choice tasks.

The respondents were also asked to complete a feedback form, which collected information about the survey and the attributes they took into account when making their decisions. This information was then used in the development of the second pilot survey.

Second pilot survey

The second pilot survey provided an opportunity to test the website to be used in the main survey. For this survey, the choice experiment was created using the efficient design method (Bliemer et al., 2011).

Feedback from the first pilot survey indicated that the attributes of “top speed for general purpose electric vehicles (GEVs)”, and “EV recharging time” were not considered to be as important as other attributes. To simplify the choice experiment, these attributes were excluded from the design of this survey.

The prior values for the parameters used in the efficient design process were from the best performing multinomial logit (MNL) model estimated using the data from the first pilot survey. The D-error criterion was used to determine the best

experiment design as this measure is considered the best when designing choice experiments intended to collect data for discrete choice models that will be used for estimating market shares (Kessels et al., 2006).

The best design was found to have 42 choice tasks and these were blocked into six blocks of seven choice tasks.

experiment, excluding the constants, plus one. To achieve attribute level balance the number of choice sets is some multiple of the number of attribute levels in the design (Bliemer et al., 2011).

101

The intention from early in the design process of the VFM was to use a MMNL model for the vehicle choice sub-model (see section 3.4). Using a MMNL model as a design template will ensure the most efficient design (see section 3.3.3). However, this is often impracticable due to computational difficulty. An alternative approach is to use the less computationally intensive MNL model as the design template even though the intention is to use the data collected to estimate a MMNL model.

Bliemer and Rose (2010) have shown that this substitution does not result in a significant loss of efficiency in the experimental design.

The second pilot survey was open to anyone interested in participating. An email invitation requesting participation, and feedback on the design of the website and survey instrument was widely distributed. The respondents were encouraged to pass the invitation on to anyone else who might be interested in participating.

The second pilot resulted in 55 completed survey responses and a number of suggestions for improvement of the website design.

Main survey

The candidates for the main survey were randomly selected from the 2007 New Zealand electoral roll. The survey was conducted over the period 1 June 2010 to 13 October 2010, and 8,000 names and addresses were initially selected from the electoral roll. The names and addresses were then matched to telephone numbers using the White Pages directory website. This process resulted in a pool of 3,262 candidates.

The main survey design had the same specification as the second pilot survey, 42 choice tasks divided into six blocks of seven choice tasks. An example of one of the 42 choice tasks used in the main stated choice survey is provided in Appendix 1.

The “PHEV electric driving range” attribute was dropped from the main survey after the results from both pilot surveys indicated that if the electric driving range was increased PHEVs would be less likely to be chosen.

One reason for obtaining unexpected signs in RPL models is the use of an incorrect distribution form for the random parameter. This explanation could be discounted

102

as the same result occurred in all the MNL and error components (ECMNL) models that were estimated.

A more likely reason is that the attribute was being excluded from the cognitive process (Hensher, 2006b). This behaviour seems possible as the first and second pilot surveys contained two driving range attributes for PHEVs: (1) the total driving range, fixed at 500 km; and (2) electric only driving range, which varied between 20 and 100 km. The unexpected sign for the electric only driving range attribute suggests that the respondents were ignoring it, possibly because they did not understand its meaning or because they were focused on the total driving range and fuel running cost attributes.

Feedback from the pilot surveys indicated that some respondents found some of the attribute level combinations unrealistic. Unrealistic, or infeasible, attribute level combinations are a feature of stated choice experiments and often these cannot be completely eliminated from the design. However, to reduce this concern,

constraints were imposed on the experiment design that eliminated the occurrence of some of the more unrealistic combinations. Imposing constraints on the design of a choice experiment will result in a decrease in the efficiency of the design (Bliemer et al., 2011). In this case, the loss of design efficiency was considered to be an acceptable trade-off as it increased the realism of the choice tasks and the credibility of the respondents’ answers.

The constraints imposed on the design ensured that the choice tasks did not contain a situation where the battery replacement cost for a new EV was greater than the purchase price and, for general purpose EVs, the highest level purchase price ($160,000) always occurred in conjunction with the highest battery

replacement price ($130,000) and greatest driving range (500 km).

When estimating the most efficient design, two different MNL specifications were used as templates. The first model specification was based on the best performing MNL model from the second pilot. This model comprised a mixture of alternative specific and generic parameters. The second model specification comprised only

103

generic parameters. The choice experiment design for the main survey was then estimated using the averaging approach suggested by Rose et al. (2009b).

The efficient experiment design process indicated that with a minimum sample size of 250 respondents it would be possible to develop a discrete choice model with statistically significant parameter estimates for all the attributes included in the stated choice experiment.