Generally, there are three types of ways to study heuristics in judgment and decision making (Shah &
Oppenheimer, 2008); computer simulations to investigate the effort and expected accuracy of heuristics; what information is searched for and used over time to arrive at a decision; and how people's behavioural outcomes match patterns that are indicative of certain strategies. The latter we investigated in Chapter 2.
Shah & Oppenheimer (2008) state while “computer simulations are useful, they cannot prove whether people actually use a given heuristic” (p. 218) whereas process tracing and outcome analysis are strong empirical tests of effort-reduction. Process tracing observes how people search for information prior to making a judgment or decision, allowing determination of which types of decision processes are being used (Payne et al., 1993; Schulte-Mecklenbeck et al., 2010). It is especially useful for studying whether decision makers are examining fewer cues or alternatives (Shah & Oppenheimer, 2008), for instance, by examining the relative number of cues accessed for alternatives, or the average duration spent on them. Lau and Redlawsk (2006, p.233) inferred ‘heuristic’ use by participants in their experiments (e.g. party label, candidate images) by comparing the mean (or relative) amount of times the information items (or all items in a related category) for each proposed ‘political heuristic’ were accessed by participants; comparing them between studies mimicking USA primary and general election campaigns. This approach is problematic. Access of information items itself can be part of a heuristic strategy (i.e. seeking out highly valid cues, like party label), but doesn’t show if, or how, such information may subsequently result in heuristic processing (e.g. by eliminating alternatives).
The Dynamic Process Tracing Environment (DPTE) software used in our methodology allows for a degree of exploratory analysis into participants’ information processing during our experiments. The DPTE is designed to capture participants’ access of informational stimuli as they appear on screen during a simulated
119
election campaign (the ‘flow stage’); recording when an item is opened, for how long it remains open on a participant’s screen, and which alternative the informational item belongs to (and more; see Lau & Redlawsk, 2006). As such, we decided to investigate how participants’ amount of information search and the average duration on items searched (as proxy measures of information processing), differed with choice set size and party label’s presence, in line with research on heuristic processing and effort reduction.
Amount of Information Access (IA): Lau and Redlawsk (2006, p.109) explore the various ways one might
assess a voter’s depth of information search in a study, namely looking at the total amount of access (i.e. total number of informational items searched) for each candidate. The amount of information accessed overall will be influenced by the number of candidates in the choice set (i.e., there is more to access in expanded choice sets), therefore we need an additional measure that takes into account the differing number of items accessed.
Average Duration (AD): considering the general use of reaction times and duration data assess processing
patterns, including in static information boards (Jacoby, Kohn, & Speller, 1974; Jacoby, Speller, & Berning, 1974; Payne, 1980), it is somewhat surprising that Lau and Redlawsk (2001; 2006) do not include analysis on how party affiliation, number of candidates, or campaign effects, impact on the duration participants access items for. The duration spent on information searched is a useful way of assessing information processing, as one could expect that larger choice sets would result in less time spent on items or alternatives as participants attempt to search, and compare, information within/between more alternatives. We average duration across the number of items accessed to control for differences between participants who access differing amounts of information. Similarly, if party label’s presence does result in more heuristic processing, reducing the need for more information search and comparison, participants may spend a shorter average duration on items.
Equality of Search (ES): The standard indicator of search equality is the variance in the amount of
information gathered across alternatives; if the variance is low, then search is relatively equal; if it is high, then search across alternatives is unequal. However, this does not tell us in which direction (i.e. for which alternative) search is unequal (i.e. preferential for an alternative). Therefore, we construct our own measures of information search equality amongst alternatives in the choice set, using the amount of information accessed and average duration, and the chosen candidate as the reference point. We do this in two ways, as discussed below.
120
‘Suppressing’ Alternatives: First, we compare whether the equality of search between the chosen alternative (candidate) and the alternatives in the smaller choice set (e.g. 2 candidates) is the same as that in the expanded choice set (e.g. 3 candidates). This is in order to assess if unequal search (i.e. preferential search) patterns for the chosen alternative differ by choice set size. If the amount and average duration of search between the alternatives in both conditions (chosen +1 alternative, vs. chosen +2 alternatives) are not meaningfully different (i.e. equal), then it may be that participants are ‘suppressing’ the presence of additional alternatives by considering a smaller subset of alternatives in the larger choice sets. For example, an alternative pair comprised of the chosen alternative and whichever other alternative is next-most-accessed by the participant. Equality of search between choice sets would correspond to heuristic strategies that utilize pairwise comparisons (Russo
& Dosher, 1983), which reduce the number of alternatives that must be kept in working memory at once and demand on cognitive resources. We can also see if party label has any effect across choice sets. However an effect using this measure may correspond to participants suppressing additional alternatives, or focusing on a preferred pair, therefore we construct an additional metric to test for these differences.
‘Focussing [on]’ Alternatives: We test this idea more directly by ‘removing’ the additional ‘least-considered’ candidates from our calculating our suppressing measures in larger choice sets (e.g. comparing 2 the two most accessed alternatives in both the 2-/3-candidate conditions). This allows us to see if participants are ‘focussing’ on their chosen candidate relative to another candidate. In the smaller 2-candidate conditions, there is no choice but to focus on the chosen candidate and the remaining alternative (either Labour or Conservative), but in expanded choice sets the focussed pair of the chosen candidate and ‘next-most-considered’41 could be any of a number of alternatives (Labour, Conservative, Liberal, Green, or UKIP).
Therefore, we remove the alternative(s) that are the least considered when calculating our measures for the larger 3-/5- candidate conditions and re-calculate the score as if there were only that many alternatives in the choice set. This allows us to test if equality of information search between the chosen and next-preferred candidate differs based on choice set size, and if party label has any effect.
41 We use the word ‘considered’ as to not become confused with ‘preference’. We have no way of knowing if an alternative is accessed heavily indicates participants’ preference (though it seems likely), and we only assume preference for the chosen candidate as participants’ vote choice has made that explicit a posteriori.
121
The method of calculating of our ‘suppressing’ and ‘focussing’ preference measures are detailed in
‘Procedure’ (Section 3.3.5, this chapter).
Timebin: Considering that heuristics like EBA and other lexicographic heuristics initially include all
alternatives and reduce effort by gradually paring down the number of alternatives in the set that fall below a minimum threshold for a particular cue, it is logical to analyse our IA, AD, and ES measures over the duration of the experimental flow stage. We follow Lau and Redlawsk (2006) in dividing up the flow stage into three equal time-bins. Looking at the pattern of our ES measures over time will allow us to examine how a preference for the chosen candidate relative to the alternatives emerge over time, per condition.
Information Processing, Heuristics, & Correct Voting: Ultimately, we wish to investigate how participants’
information processing may relate to whether or not they vote correctly. In line with rational assumptions, it seems uncontroversial to state that the more information participants access should lead to more informed decisions, and thus be more likely to vote correctly. The alternative view is that ‘information overload’ arising from excess information results in lower quality decisions (Bargh & Thein, 1985, Kerstholt, 1992). In such a case, Gigerenzer and Gaissmaier (2011) argue that using less information (e.g. by utilizing heuristics) can actually increase performance and accuracy in decision-tasks under certain conditions. For example, Take-the-Best can predict more accurately than linear multiple regression models (Czerlinski et al.1999), provided cues are searched through in the order of their validity, and there is high cue redundancy and high variability in cue weights (Gigerenzer & Gaissmaier, 2011). Famously, Ortmann, Gigerenzer, Borges, and Goldstein (2008) reported in a study on financial portfolio success, that general members of the public utilizing the recognition heuristic outperformed (on average) managed funds, the Dow/Dax markets, and stock experts.
In political information processing, Lau and Redlawsk (2006) put forward a theoretical voter model of decision-making (‘Model 4’, p.8) that is ‘rational bounded’ and ‘intuitive’ in decision-making; utilizing heavily on heuristic strategies (i.e. satisficing) and ‘political heuristics’. They find evidence that cognitive limits are indeed a problem for voters and that non-compensatory strategies corresponding to those for Model 4 voters are positively related to correct voting in certain scenarios, often outperforming ‘rational’ models corresponding to effortful decision-making strategies (e.g. weighted additive; ‘Model 1’ voters). Both of these
122
bodies of research suggest that in certain cases, voters following heuristic strategies may lead to positive outcomes, improving correct voting rates.
Similarly, if participants are spending longer on items, rather than quickly skimming information, this should lead to better evaluations of the cues and alternatives overall, and thus better decisions on their utility (i.e.
more correct voting). Alternatively, average duration on items may not matter, as individual differences in reading and processing speed may lead to participants spending only as long as is needed.
One would expect that greater preference for the chosen candidate on our constructed measures should be related to rates of correct voting, in that, when participants show greater preference for a chosen candidate and they are the (in)correct choice, this should be related to (in)correct voting rates. Yet, greater amount of search for one specific candidate, and the longer spent on one candidate reduces evaluation of other candidates (as there is limited time); and may be negatively related to correct voting (if the participant is dwelling on the incorrect choice).