Hypothetical bias

Because respondents to surveys are not making an economic commitment to their responses by paying money, their responses may be subject to bias and may provide researchers with hypothetical values rather than actual WTP. This is referred to as the hypothetical bias in SP research.

The exact impact of hypothetical bias is uncertain. The 1993 NOAA panel report offered the rule-of-thumb that CVM values should be divided by two to yield true WTP (List & Shogren, 1998). This has been challenged by research finding that CVM values can be within about 10% of RP values (Hanley et al., no date) or three times RP results (List & Shogren, 1998). A

meta-analysis of hypothetical bias comparing several studies that used both RP and SP methods found that the relationship between actual WTP and stated WTP was complex (Murphy, Allen, Stevens, & Weatherhead, 2005). The median bias, expressed as the ratio between the hypothetical value and the actual value, was 1.35. Researchers found that the distribution was very skewed, with a few observation exhibiting severe hypothetical bias. In other research examining RP data with several SP techniques, most of the SP methods were found to yield similar preference structures to the RP data (Cameron et al., 2002). To complicate the issue, research indicates that the factors used to calibrate the two types of values depend on the person and product (Fox, Shogren, Hayes, & Kliebenstein, 1998) as well as the source of RP data (Shogren, Fox, Hayes, & Roosen, 1999). Furthermore, it is not clear that the bias is systematic or even the result of deliberate misrepresentation (Polome, 2003). Although Murphy, et al. (2005) suggested that hypothetical bias has been insufficiently theorised, Blamey (1998b) has made a start on a theory of hypothetical bias by describing and quantifying sources of hypothetical bias. He also found that the impact of hypothetical bias was a priori unknown.

Some researchers have suggested that values for non-GMF are inflated by hypothetical bias (Chern et al., 2002). Because consumers do not have to commit money to their survey responses, they are free to indicate that they would double or treble their food spending in order to have non-GMF. Lusk (2003) examined the impact of hypothetical bias on

expressions of WTP using a double-bounded dichotomous choice CVM question and a technique called ‘cheap talk’. With this method, researchers inform respondents about the problem of hypothetical bias in an attempt to reduce or remove it. Lusk found that cheap talk reduced by about 40% the premium on a vitamin-enhanced GMF product called ‘golden rice’. The result is that, while hypothetical bias may be influencing respondents’ valuations, the

direction of the impact is a priori uncertain: it is not clear whether it is GM or non-GM products whose value is being inflated by hypothetical bias.

Conjoint analysis, a type of CM research used in the marketing literature, was one of several SP techniques that were all found to elicit similar preference structures (Cameron et al., 2002). Thus, to the extent that respondents are giving hypothetical values in response to SP questions, conjoint analysis is no differently affected by this bias than any other SP technique, nor is there any reason to suspect a priori that it would be (Bateman et al., 2002).

While hypothetical bias will always be an issue for survey-based research, because it is by its very nature not a market, it is possible to reduce its impact. Bateman, et al. (2002) provide a detailed discussion of hypothetical bias and the allied issue of validity. They suggest that a well-designed survey will create scenarios or options that would appear realistic to

respondents; the survey must have content or face validity. An additional consideration is that the payment mechanism must also be realistic, so that the respondent would find the way of paying for the good plausible. Thus, while it is possible to test for hypothetical bias only by comparing the results of a survey to results from an actual market, the validity of SP research can be assessed without external measurement.

4.3.7 Validity

Validity of SP research is a multi-faceted concept. That a piece of research is ‘valid’ can mean one or more of the following (Bateman et al., 2002; Morrison et al., 1996):

• The results conform to prior expectations (expectations-based validity).

• The relationships between measures within a survey conform to relationships seen elsewhere (construct validity).

• The research produces accurate predictions (predictive validity).

• The content of the survey is accurate (content validity).

Because SP research is based on neoclassical economic theory, it is easy to identify prior expectations and then determine whether results conform to them. Economics research is nearly always assessing the expectations-based validity of research, and to some extent its construct validity: if the signs and magnitudes of estimated parameter are not as expected, then they must be explained.

SP research often generates estimates of WTP, which allow results from different surveys to be compared with each other to assess convergent validity. Convergent validity has been tested, and the results are mixed. Under some circumstances, some elicitations methods arrive at similar values, while in other circumstance the results may diverge (Adamowicz, Boxall et al., 1998; Cameron et al., 2002; List & Shogren, 1998).

To the extent that SP research is concerned with predicting demand for future products, there is scope for comparing the predictions generated by such research against future market data. However, predictive accuracy can only be determined after such market data become

available. Research assessing predictive validity in other contexts has often but not always found good correspondence between predicted and actual choices (Louviere, 1988; Morrison et al., 1996).

Finally, the constructed nature of a survey-based research and the use of questionnaires mean that both content validity and construct validity can be assessed by examining the design of a survey instrument.

4.3.8 Yea-saying

Respondents may respond positively to researchers’ suggestions in order to be pleasant. Thus, they may agree with researchers regardless of their true WTP, creating a bias in survey

results. DC CVM questions have been found to lead to higher values than payment card approaches, and a likely culprit is yea-saying (Hanley et al., no date). Regardless of one’s true WTP, one must respond ‘yes’ to some payment level in order to register a positive WTP on the survey (Blamey, 1998b). A payment card approach allows respondents to indicate positive response at lower payment levels, whereas they need to agree to whatever payment level is randomly generated in a DC survey, regardless of how high it is. DC CVM questions also generate higher values than open-ended questions (Amigues et al., 2002; Bateman & Jones, 2003), possibly for the same reason.

Yea-saying may be less of a problem for choice-based SP methods than CVM techniques (Bateman et al., 2002). The valuation task is not to say ‘yes’ or ‘no’ to a bid value, but instead to select one option from many. One may be equally pleasant by choosing any of the offered alternatives.

4.3.9 Information bias

Providing respondents information in the course of SP research can influence the results of research (Spash et al., 2000). However, it is not clear to what extent this represents a bias, or, in particular, an improper influence on respondents (Spash et al., 2000). It is important, for example, to provide respondents with enough information that they can accurately and comfortably respond to the valuation task (Bateman et al., 2002). The point at which information provision becomes information bias is unclear.

The impact of providing information to individuals has been explored largely in the context of experimental economics using RP methods, but the findings may have relevance to SP

research. In a series of auction experiments, researchers at Iowa State University examined how the provision of different types of information affected bids on GM food and cigarettes (Huffman et al., 2003b; Huffman et al., 2001; Lusk et al., 2003; Rousu et al., 2003; Tegene et al., 2003). They found that negative information made people less willing to pay for GMF and that positive information made them more willing to pay. They also found that ‘neutral’ information reduced the sizes of both the positive and negative bids, and suggested that ‘third party’ information is welfare enhancing.

The main drawback to this research is that the tenor of the information is co-determined with the reaction of the auction participants. ‘Negative’ and ‘positive’ are qualities that are difficult to define except by the influence that information has on people’s WTP. Particularly difficult is the notion of ‘neutral’ information: if information can only be considered neutral when it has little impact on WTP, then research into the impact of neutral information on WTP is begging the question. To further complicate the issue, other research using auction experiments found that ‘it is possible that providing biased information contrary to that previously believed may have further entrenched prior-held beliefs’ (VanWechel,

Wachenheim, Schuck, & Lambert, 2003). That is, telling people things they do not agree with may push them to hold their ideas more strongly. The information issue might therefore be one of concordance: respondents’ reactions to the information provided may depend on whether they are pre-disposed to believing it.

4.3.10 Framing effects

The way in which an issue is framed or presented to individuals can affect survey responses (McFadden, 2001b). This has been extensively studied in the context of risk assessments, and it has been demonstrated that the way in which risks are presented to respondents affects the judgements they make about those risks (Kahneman et al., 1990). Some researchers maintain,

however, that framing effects may not affect WTP estimates significantly (Hanley et al., no date), at least in well-designed CM research (Louviere, 1988).

Framing effects have been discussed in the context of GMF. Most GMF research, particularly that using CVM methods, has focussed solely on the issue of genetic modification.

Respondents may therefore have been sensitised to the GM issue and accorded it more weight in their survey response than it may have in their purchases. The ‘food futures’ research in the UK and Australia (Burton et al., 2001; S. James & Burton, 2003) attempted to place the issue of GM in the wider framework of the food system. Genetic modification was presented as one of a number of food-related issues, along with agrochemical use and the distance that food travelled from field to plate. This research did not compare different frames of reference, but their research had broadly similar results to more narrowly framed GMF surveys. This research highlighted one of the benefits of choice-based SP research over CVM, that the former tends to highlight the tradeoffs that consumers potentially face (Bateman et al., 2002). An important caveat for work on framing effects is that it may be difficult to design plausible scenarios when respondents want to find fault with them (Blamey, 1998a); for these

respondents there may not be a ‘right’ or ‘accurate’ way to frame an issue, regardless of the content validity of the survey instrument.

4.3.11 Summary: SP methods

This discussion of SP research has pointed to a number of known issues. Response data may be affected by non-response, protest responses, and lexicographic responses. The research needs to consider the ways in which data will be collected and analysed. The survey instrument itself may lead to hypothetical bias, yea-saying, validity concerns, information bias, and framing effects. Nevertheless, as discussed earlier, research on demand for GMF needs to consider using SP methods simply because real market data are unavailable.

A number of SP methods are available, and they may have different strengths with regard to exploring the issues with GMF identified above. One issue was the separability of preferences over the GM attribute from preferences over other attributes of food. A second issue was the apparently lexicographic choices that some people make concerning GMF. This issue then led to the problem of aggregating individuals’ choices into market-level indifference curves. As Bateman, et al. (2002) point out, the main choice regarding SP methods is between CVM and CM. CVM is better for determining the total value of a good, such as the total value of a program of environmental remediation. CM, by contrast, is better for finding the values of the attributes of goods. Choice experiments, in particular, have been found to provide ‘a richer description of the attribute trade-offs that individuals are willing to make’ (Adamowicz, Boxall et al., 1998). The issues that have been identified with regard to GMF centre on the values that people place on food attributes, especially on the single attribute ‘GM’. This is true for both the issue of separability and the issue of continuity. Thus, some type of CM method may be best for considering these research questions.

Importantly, not all CM techniques are consistent with neoclassical economic theory (Bateman et al., 2002; Louviere, 2001). Neoclassical theory, as described above, posits that consumers choose the alternative that provides the greatest utility. Contingent rating is not consistent with utility theory because it is not a choice-based process. Respondents do not directly compare the alternatives to each other, but instead give rating to each option individually. Thus, it is not a choice-based process (Bateman et al., 2002; Louviere, 2001). Paired comparisons and contingent ranking both are problematic with regard to utility theory, unless they always contain a status quo option against which the alternatives can be

compared. Otherwise, the evaluations made by respondents is not anchored, but merely a relative evaluation of two hypothetical alternatives (Bateman et al., 2002). Contingent ranking additionally suffers from the concern that the scale that the respondent uses for making

rankings is essentially unknown, so that responses to different question by the same respondent and responses from different respondents are not necessarily comparable (Bateman et al., 2002; Louviere, 1988, 2001).

One CM technique that is consistent with neoclassical theory is choice experiments (CE) (Bateman et al., 2002; Louviere, 2001). The valuation task for respondents to a CE survey is to choose the single best alternative from a set of options. This is exactly the type of decision theorised in neoclassical consumer theory. The chosen alternative must theoretically yield the greatest utility for the respondent. This utility can then be decomposed into the contribution that each attribute makes, using Lancaster’s theory, and the effects of the latent term, using RUM theory. Thus, CE has a firm basis in neoclassical economic theory (Louviere, 2001). A CE survey appears to be an appropriate method for considering the research issues

identified above. It is well-grounded in neoclassical economic theory and provides a method for determining the effect of the specific product attribute ‘GM’ on consumer behaviour. In particular, a CE survey offers the potential for distinguishing protest responses from

indifference and from lexicographic preferences regarding GM technology, allowing closer examination of the continuity issue. Furthermore, the attribute-based nature of CE surveys may make it possible to examine preference separability. To consider these issues further, and in particular to examine how they have been addressed in prior research, the next section examines choice experiments more closely.

4.4 Choice experiments

This section is a review of the CE literature. It covers the links between CE and neoclassical theory, a description of the survey method, and approaches to modelling data, including recent developments.

Before the 1960s, consumer theory relied on the ‘representative agent’ (McFadden, 2001b). This approach modelled demand with a single agent who represented the preferences of all consumers. Theoretical developments and increases in computing power led to disaggregated approaches that modelled individual choices (McFadden, 2001b). One of the earliest

examples of this type of work was research on the choice of transport mode for the San Francisco Bay Area Rapid Transit (BART) system in the 1970s (McFadden, Train, & Tye, 1978). The success of this model relative to forecasts using earlier gravity models led to the further use of such choice models in transport analysis (McFadden, 2001a).

This approach for modelling observed choices amongst discrete alternative was developed further in the 1980s for use in analysing data from SP research (Bateman et al., 2002; Hanley et al., 2001). The result was choice experiments, which are also called attribute based stated choice methods (Adamowicz & Boxall, 2001), stated preference discrete choice modelling (Gerard, Shanahan, & Louviere, 2003), or simply choice modelling.

In a CE survey, a respondent is asked a series of choice questions. Each question presents several alternatives, including one which represents the status quo. For each question, the respondent is asked to choose one option from each set. Choice experiments are constructed to resemble the choice situation described in neoclassical consumer theory, so the elements of a choice experiment are similar to the theoretical situation.