Issues measuring satisfaction

A major concern of researchers studying the validity of self-reported measures of happiness is accuracy of the subjective measure of well-being (Diener et al., 1993, Diener et al., 2002). This concern applies to post-move satisfaction as well, but is commonly omitted (Kettlewell, 2010). The first concern is whether subjective responses mirror objective measures of satisfaction. Self-reported measures of satisfaction have been found to closely reflect expert analysis and the objective measurement of factors such as the frequency with which individuals smile (Diener et al., 1993, Pavot et al., 1991, Sandvik et al., 1993).

The one-off, reflective nature of the DMM survey introduces potential inaccuracies, such as cognitive dissonance, where “people always try to avoid dissonance between their attitudes and behaviour” (Lundholm and Malmberg, 2006: p.

39). Festinger (1957) theorised that respondents are prone to answer in a manner that aligns their response with their behaviour. In the case of post-move satisfaction, cognitive dissonance may cause movers to be more likely to report a positive improvement in satisfaction as it would be aligned with their decision to move and affirm their decision-making. For example, movers who fail to meet their anticipated outcomes may rationalise their actual outcomes by retrospectively lowering their

anticipated outcomes. By retrospectively reducing their anticipated outcomes to match their realised outcomes, they reduce the dissonance that they experience.

Kettlewell (2010) notes that “the fact that the response is reflective creates difficulties due to psychological factors that prevent peoples‟ ability to accurately assess their previous state of well-being (Stone et al., 1999)” (Kettlewell, 2010: p. 190).

Further, he cites Easterlin‟s (2001) work, which “demonstrates how evolving material aspirations may distort peoples‟ view of their wellbeing in the past” (Kettlewell, 2010:

p. 190). If movers are unable to accurately determine their previous level of satisfaction then the reported change in domain satisfaction may therefore be inaccurate. I suggest that the overall level of satisfaction that movers have with the outcomes of their move avoids the possible upward bias which direct comparison of present with past satisfaction levels might bring.

In the DMM survey, movers were asked to report their overall level of satisfaction with the outcomes of the move after they were asked about their domain satisfaction. Because they were asked in the five preceding questions about their change in satisfaction in specific domains, it is possible that this may have had an influence on how some movers evaluated their overall level of satisfaction with the outcomes of their move. Some movers may have treated „satisfied‟ and „dissatisfied‟ as

„better‟ and „worse‟ respectively.

As mentioned earlier, post-move satisfaction is measured using a five point Likert type scale. The scale ranges from “very dissatisfied” through a neutral response to “very satisfied” for overall post-move satisfaction and from “much worse off” and through a neutral response to “much better off” for each satisfaction domain. The use of Likert-type scales is a common psychometric method for measuring satisfaction and has been used widely for measuring residential satisfaction and almost exclusively by those studying post-move satisfaction. Likert scales are favoured by some for being a cognitively light method of self-reporting (Knowles and Nathan, 1997).

Despite the extensive use of Likert scales to measure satisfaction and post-move satisfaction, they are subject to possible central tendency bias, acquiescence bias and cross-cultural biases. Central tendency bias is the tendency of respondents to avoid using the outermost categories, effectively narrowing the band of responses. In the case of the 5 point Likert scale used by the survey, respondents would have a tendency to

shun the two most extreme responses in favour of the middle three, reducing the sensitivity of the scale.

Acquiescence bias is the tendency of respondents to favour positive answers, resulting in respondents reporting a higher level of satisfaction than they actually experience (Friborg et al., 2006, White and Mackay, 1973). Whether these criticisms outweigh the benefits of the cognitively light nature of Likert scales is a matter of intense debate. Knowles and Nathan (1997) found the bias created by cognitive demands is increased when the cognitive demand of a respondent is increased, supporting the use of such cognitively light methods. Strong acquiescence bias towards respondents favouring positive answers will result in higher post-move satisfaction than is evident in the population. A strong acquiescence bias may result in those who experience lower levels of post-move satisfaction reporting a higher level of satisfaction than they actually experience.

Dealing with movers from a range of cultural backgrounds may alter the distribution of biases across the population. Some groups of respondents may respond differently to those of other groups artificially influencing their reported satisfaction outcomes:

“There are potential cultural differences in moderacy response styles, in which people from one culture are more likely to answer toward the center of a scale than are people from another (Chen et al., 1995).

Translation errors can produce measures conveying different meanings across cultures (Brislin, 1970). Members of some cultures may have more familiarity with completing questionnaires than do others (Greenfield, 1997). It is plausible that members of some cultures may be more likely to disguise their responses behind a facade of modesty than are others (Heine et al., 2000)” (Heine et al., 2002: p. 904)

While I acknowledge the potential presence of these biases, the degree to which these biases occur in the case of this survey is unknown.

The survey‟s Likert scales run from 1 to 5. According to Cummins (2003a), there is a majority opinion within the study of life satisfaction that the number of points on a bi-polar Likert scale should exceed five and may in fact be much higher. Cummins discusses the optimum number points as follows:

“On the dimensions of scale sensitivity it seems clear that more choice points are better (Andrews, 1984; Andrews and McKennell, 1980; Chang, 1994). But as the number of response options rises,

measurement consistency decreases and the tendency for response-set increases, most particularly if the number of response-options exceeds people‟s ability to discriminate between them (Chang, 1994).” (Cummins, 2003a: p.228)

Therefore, while the DMM survey‟s use of a 5 point Likert scale is in keeping with other post-move satisfaction literature, it may lack the sensitivity that a 7 point or higher scale may reveal.

Within the study of post-move satisfaction, there is no standard number of points used in Likert scales. Data has been drawn by post-move satisfaction researchers from surveys that utilise scales varying between three to seven points. The American Housing Survey (AHS), of which Lu (2002) used the 1989 dataset and Barcus (2004) used the 1991 dataset, employed three points of variation; better, worse, or about the same for measuring satisfaction. The Thailand National Migration Survey, of which De Jong et al. (2002) used the 1992 survey, also used a three point scale of „better, the same, worse‟ to measure post-move satisfaction. At the other end of the scale, Lundholm and Malmberg (2006) used a dataset that employed a 7-point likert scale.

The British Household Panel Survey used by Nowok et al. (2011) also employed a 7-point Likert scale. Meanwhile, the German Socio-Economic Panel Study, used by Melzer (2011) and the Households, Income and Labour Dynamics in Australia survey used by Kettlewell (2010), asks respondents to place their well-being on a 1 - 10 point scale.

In summary, there are a number of potential biases that may affect the ability of the DMM survey to accurately measure the level of satisfaction that movers have with the outcomes of their moves. These include cognitive dissonance, central tendency bias, acquiescence bias, and cross cultural biases. In addition, the number of points on the Likert scale also justifies consideration.