Structured Analogies for Forecasting
Kesten C. Green,* Department of Econometrics and Business Statistics, Monash University, VIC 3800, Australia
e-mail: [email protected]
Phone +61-3-990-55438
J. Scott Armstrong, The Wharton School, University of Pennsylvania Philadelphia, PA 19104 e-mail: [email protected] Phone 610-622-6480 Fax 215-898-2534 Version 106 September 10, 2004
* Corresponding author. Address correspondence to e-mail address or to Department of Econometrics and Business Statistics, PO Box 11E, Monash University, Victoria 3800, Australia.
Structured Analogies for Forecasting
When people forecast, they often use analogies but in an unstructured manner. We propose a structured judgmental procedure that involves asking experts to list as many analogies as they can, rate how similar the analogies are to the target situation, and match the outcomes of the analogies with possible outcomes of the target. An administrator would then derive a forecast from the experts’ information. We compared structured analogies with unaided judgments for predicting the decisions made in eight conflict situations. These were difficult forecasting problems; the 32% accuracy of the unaided experts was only slightly better than chance. In contrast, 46% of structured analogies forecasts were accurate. Among experts who were independently able to think of two or more analogies and who had direct experience with their closest analogy, 60% of forecasts were accurate. Collaboration did not improve accuracy.
It seems natural to use analogies when making decisions or forecasts as, by definition, they contain information about how people have behaved in similar situations in the past. For example, Breuning (2003) found that one-third of testimony at the Senate hearing on proposals for the first U.S. program for development aid was based on analogies. Khong (1992) concluded that most of the decisions made early in the Vietnam War were based on forecasts derived from analogies. Indeed, Kokinov (2003, p. 168) asserts “…we may explain human behavior by assuming that decisions are made by analogy with previous cases…” In the belief that such information is useful, MIT professor Lincoln P. Bloomfield has developed a historical database of post-World War II conflicts
(web.mit.edu/cascon) in order to help policy analysts and others identify appropriate analogies.
We agree that information about analogies should be useful for forecasting, but we suspect that without structure people rush to forecast. To the extent that they think about analogies, they will find one that supports their forecast and then stop their search. For example, when the U.S. Environmental Protection Agency approved a new oil refinery in Eastport, Maine, decision makers relied on the analogy of Milford Haven in the U.K (Stewart and Leschine, 1986). The EPA decision makers considered Milford Haven was the most comparable site, and looked no further, but Stewart and Leschine observed that Milford Haven had not been in operation long enough to provide evidence that it was safe. They were right. The supertanker Sea Empress ran aground near Milford Haven on 15 February, 1996, spilling 70,000 tonnes of crude oil (Canada Centre for Remote Sensing, 1996). Neustadt and May (1986) described how inappropriate selection and inadequate analysis of analogies led U.S. government decision makers to make poor forecasts of the decisions of other governments’ leaders. Drawing on their litany of poor decisions by political leaders, they described an elaborate structured approach to analyzing current and historical information that they suggested should lead to a more effective use of experts’ knowledge and hence to improved prediction.
Many areas of judgmental decision making and forecasting have shown that structured judgmental processes make more effective use of the information people possess. This occurs, for example, when people are asked explicitly to decompose a problem (MacGregor, 2001). More generally, Armstrong (1985, Chapter 6) summarizes evidence that structured methods of judgmental forecasting are more accurate than unstructured ones. A structured approach to
forecasting with analogies, then, might encourage experts to consider more information on analogies, and to process it in an effective way.
Kahneman and Lovallo (1993) report an anecdote that illustrates how inducing an expert to use analogies in a structured way can affect predictions. Kahneman had worked with a small team of academics to design a new judgmental decision making curriculum for Israeli high schools. He asked each team member to predict the number of months it would take them to prepare a proposal for the Ministry of Education. Predictions ranged from 18 to 30 months. Kahneman then turned to a member of the team who had considerable experience developing new curricula and asked him to think of analogous projects. After some consideration, the man stated that, among the many analogous situations he could recall, about 40% of the teams eventually gave up. Of those that completed the task, he said, none did so in less than seven years. Furthermore, he thought that the present team was probably below average in terms of resources and potential. In the event, the project took eight years to complete.
Procedure for forecasting with structured analogies
Because the literature provides no evidence on how to structure forecasting with analogies, we started with a simple procedure. If analogies are useful, it is because they are similar to a target. Imposing structure on experts’
assessments of similarity should encourage more complete processing of information and reduction of biases. We also wanted a procedure that would be easy for experts to use. At a minimum then, a structured approach to using analogies for forecasting requires experts to identify analogies and the outcomes they imply for the target, and to assess the analogies’ similarity to the target in a structured way. Our structured analogies procedure involves five steps, two of which involve experts analyzing analogies. The administrator (1) describes the target situation, and (2) selects experts; the experts each (3) identify and describe analogies, and (4) rate similarity; the administrator (5) derives forecasts.
(1) Describe the target situation
The administrator prepares an accurate, comprehensive, and brief description. To do so, the administrator should seek advice either from unbiased experts or from experts with opposing biases. When feasible, include a list of possible outcomes for the target situationto make coding easier.
(2) Select experts
The administrator recruits experts who are likely to know about situations that are similar to the target situation. The administrator should decide how many experts to recruit based on how much knowledge they have about analogous situations, the variability in responses among experts, and the importance of obtaining accurate forecasts. Drawing upon the research on the desirable number of forecasts to combine, we suggest enlisting the help of at least five experts (Armstrong, 2001).
(3) Identify and describe analogies
Ask the experts to describe as many analogies as they can without considering the extent of the similarity to the target situation.
(4) Rate similarity
Ask the experts to list similarities and differences between their analogies and the target situation, and then to rate the similarity of each analogy to the target. We suggest providing a scale against which the experts can rate the similarity of their analogies. Ask them to match their analogies’ outcomes with target outcomes.
(5) Derive forecasts
To promote logical consistency and replicability, the administrator should decide on the rules to derive a forecast from experts’ analogies. Many rules are reasonable to use. For example, one could select the analogy that the expert rated as most similar to the target and adopt the outcome implied by that analogy as the forecast.
Hypotheses
We examined predictive validity using conflicts. Prior research has shown that the method currently used formaking predictions for conflicts, unaided judgment, produces inaccurate forecasts (see, for example, Green 2002). We hypothesized that forecasts derived from experts’ structured analysis of analogies would be more accurate than forecasts by experts who used their unaided judgment.
Our structured analogies procedure is based on the assumption that while unaided experts can provide useful information, they are not good at processing complex information reliably. For that reason, we did not rely on the experts to make forecasts but instead used a rule. On the other hand, perhaps experts’ understanding of their own analogies might enable them to forecast more accurately than we could by using rules. To test this aspect of our procedure, we asked our experts to predict the decision made in the target situation after they had described and rated their analogies.
Does it help if experts collaborate and discuss analogies with others? Collaboration could help experts to produce more analogies and flesh out the details, or it could hinder them by suppressing their creativity and search. Both positions are reasonable, so we had no prior hypothesis on collaboration. We asked some experts to collaborate with others and all experts were asked to report the number of people they discussed the forecasting problem with.
Prior evidence
We searched for evidence on methods for forecasting with analogies. Schrodt (2002) searched for empirical evidence on the accuracy of forecasts for decisions in conflicts in the foreign policy arena. He found no evidence on the accuracy of forecasts based on analogies relative to that of forecasts based on any other method.
In a marketing study, McIntyre, Achabal and Miller (1993) tested a procedure called case-based reasoning, which is a way to structure analogies, for forecasting sales during sales promotions. When tested on two products, the forecasts were no more accurate than those of an expert buyer.
We conducted a further search for evidence by using the Social Sciences Citation Index (SSCI) for the period 1978 to August 24, 2004 using the terms “analogies” and “forecasting,” and then “analogies” and “prediction.” We searched the Internet on August 24, 2004 using Google™ and the terms “comparative”, “forecasting,” “prediction,” “accuracy,” and “analogies”. We conducted similar searches on JSTOR. In November 2001, we sent e-mail appeals to the 278 members of the International Institute of Forecasters list server and to the 579 members of the Judgment and Decision Making mailing list. We also contacted key researchers. The only relevant study we uncovered was Buehler, Griffin, and Ross’s (1994). The authors asked 123 participants to estimate how long it would take to complete a computer assignment. Their predictions, made using unaided judgment, were inaccurate as they were overly optimistic. Participants who had been asked to think of analogous situations were less biased, especially when they described how the analogies related to the assignment. As a consequence, proportionately twice as many of those who recalled analogies finished their assignments before their estimated completion times.
In sum, prior to the research we describe here, little evidence was available on the accuracy of forecasts based on the use of analogies relative to the accuracy of forecasts made using other methods. Furthermore, no prior evidence exists on the use of structured analogies.
Procedures used for the study
Preparing materials
We compiled descriptions of conflicts, including brief descriptions of the roles of the parties involved in the conflict. The conflict descriptions were accounts of real situations. We abstracted all but one (Personal Grievance) from mass media reports or experts’ accounts. The lead author developed the Personal Grievance from information collected in interviews and from exchanges of e-mail messages with the parties involved in the dispute. In the case of Nurses Dispute, he gathered information from published sources (Langdon, 2000a; 2000b; 2000c; Radio New Zealand, 2000a; 2000b; 2000c) and by interviewing representatives of the two disputant parties. When we considered it necessary, we disguised the conflicts that had already occurred to reduce the chance that our participants would know the outcomes. As a precaution, we asked our experts whether they recognized the situations. In eight cases, experts correctly identified a conflict, and their responses were eliminated.
In all, we used eight conflict situations in our research. We provided between three and six possible outcome options for each of them (Table 1). Our descriptions were short, running to no more than two pages. The full descriptions are provided at conflictforecasting.com.[For reviewers, descriptions are attached as Reviewer Appendix 1 and outcome options as Reviewer Appendix 2.] The materials, identity of the disguised conflicts, and descriptions of actual outcomes are available to researchers on request.
_____________________________________________________________________________ Table 1
Conflict Situations
Artists protest: Members of a rich nation’s artists’ union occupied a major gallery and demanded generous financial support from their government. What will be the final resolution of the artists’ sit-in? (6 options) Distribution channel: An appliance manufacturer proposed to a supermarket chain a novel arrangement for retailing its wares. Will the management of the supermarket chain agree to the plan? (3 options)
55% Pay plan: Professional sports players demanded a 55 percent share of gross revenues and threatened to go on strike if the owners didn’t concede. Will there be a strike and, if so, how long will it last? (4 options)
Nurses dispute: Angry nurses increased their pay demand and threatened more strike action after specialist nurses and junior doctors received big increases. What will the outcome of their negotiations be? (3 options)
Personal grievance: An employee demanded a meeting with a mediator when her job was downgraded after her
new manager re-evaluated it. What will the outcome of the meeting with the mediator be? (4 options)
Telco takeover: An acquisitive telecommunications provider, after rejecting a seller’s mobile business offer, made a hostile bid for the whole corporation. How will the standoff between the companies be resolved? (4 options)
Water dispute: Troops from neighboring nations moved to their common border, and the downstream nation
threatened to bomb the upstream nation’s new dam. Will the upstream neighbor agree to release additional water and, if not, how will the downstream nation’s government respond? (3 options)
Zenith investment: Under political pressure, a large manufacturer evaluated an investment in expensive new technology. How many new manufacturing plants will it decide to commission? (3 options)
Selecting experts
To select experts, we sent e-mail messages to ten public list servers, two organizations’ e-mail lists, the faculty of a university political science department, and a convenience sample of 15 experts. We chose lists that were likely to include high proportions of experts on conflicts or on judgmental forecasting. We took additional steps to ensure people were suitably qualified for these tasks. In our appeals, which were personalized when possible, the lead author wrote “I am writing to you because you are an expert…” and “I am engaged in a research project on the accuracy of different methods for predicting the outcomes of conflicts…” (Appendix A). We sent only descriptions of conflicts that were likely to be relevant to the particular recipients. For example, we did not send a situation dealing with a proposed new marketing channel to experts in employment relationship disputes. Most importantly, we counted on people to recognize when they had expertise on a topic.
We sent as many as three reminders. Details of the lists and participation are provided at conflictforecasting.com. [For the purpose of review, the details are attached as Reviewer Appendix 3.]
Using the methods
In our e-mail appeal, we gave experts instructions on how to participate (Appendix A). For structured-analogies participants, our one-page questionnaires asked the experts to (1) describe each analogous situation; (2) describe their source of knowledge about it; (3) list similarities and differences compared to the target conflict; and (4) provide an overall similarity rating (where 0 = no similarity… 5 = similar…10 = high similarity). Finally, we asked the experts to select (from a list of possible outcomes that we prepared for each target conflict) the outcome closest to the outcome of their analogy. To illustrate, a completed structured-analogies treatment questionnaire for one of the conflicts, Telco Takeover is provided as Appendix B.
Questionnaires for unaided-judgment participants first asked them to select the outcome they thought would occur. We gave them the same lists of possible outcomes that we gave to the structured-analogies participants.
We varied the order in which we attached the conflict documents to our e-mail appeals. To test our hypotheses, with our appeals we sought responses for each of the following treatments:
1. unaided judgment (no instructions on how to forecast) without collaboration, 2. unaided judgment with collaboration,
3. structured analogies without collaboration, 4. structured analogies with collaboration.
For our first appeal, we sent equal numbers of each treatment to members of the International Association of Conflict Management mailing list. The structured-analogy and collaboration treatments were more onerous for participants than unaided judgment, so we obtained relatively few responses for those treatments. As a consequence, in most of our subsequent appeals we sought responses for structured analogies with collaboration. Finally, we sought responses for combinations of conflict and treatment for which we needed more forecasts. Because we were seeking participants for their expertise, rather than as part of a representative sample of some larger group, random assignment to treatments was unnecessary. The form of collaboration was at the discretion of the participants.
Coding responses
We obtained two groups of unaided-judgment forecasts from experts. One was from the unaided-judgment treatment (62 forecasts), and the other from experts who were asked to use structured analogies but could think of no analogies (44 forecasts).We analyzed results separately for each group and the forecasts were similar; the latter group’s being somewhat more accurate. We combined the two groups under the title “unaided judgment” for our analyses,
reasoning that neither of these groups used structured analyses and that our action favored unaided judgment relative to the structured analogies method.
For each conflict, we derived a structured-analogies forecast from each expert’s analogy information, where the information was available. It is trivial to derive a forecast from analogies information when an expert provides a single analogy. On the other hand, many mechanical schemes could be used to derive a forecast when an expert provides information on more than one analogy. To obtain a forecast, we selected the target conflict outcome implied by the analogy given the highest similarity rating by the expert. Our reasoning was that predictive validity
should increase with relative similarity. Where there was a tie, we selected the outcome that had the most support from the expert’s analysis of analogies. (Details on the rules for determining support are provided at
conflictforecasting.com). [For the purpose of review, details of the rules are attached as Reviewer Appendix 4.] Given our uncertainties about the best procedure, we subsequently analyzed other mechanical schemes.
We asked a convenience sample of five people who knew the actual outcomes of the conflicts to rate the outcome options we provided to the research participants. The raters were told that an option that matched the actual outcome of a conflict should be given a rating of 10. Forecasts were counted as accurate if the outcome option chosen by our rule was the option that had been given the highest median rating by our raters. Outcome options were unconditional statements of decisions and did not specify timing, for example, “Expander’s takeover succeeded at, or close to, their August 14 offer price of $43-per-share.”
Results
As Tetlock (1999) demonstrated, it is difficult for experts to forecast decisions made in conflicts situations. He found that forecasts by 20 experts of the outcomes of foreign-policy conflicts were no more accurate than could be expected from chance. Our results were similar. Our 66 unaided experts were correct for 32% of predictions in an unweighted average across the eight conflicts (Table 2).
As hypothesized, forecasts from structured analogies were more accurate. They were more accurate for seven of the eight conflicts. Averaging the accuracy figures across the conflicts, structured-analogies forecasts were 46% accurate (P = 0.04, one-tailed permutation test for paired replicates; Siegel and Castellan, 1988). We used the permutation test for paired replicates to compare the differences in the percentage of correct forecasts between the two methods for each conflict (e.g., for Artists Protest, the difference between structured analogies and unaided judgment was 17%). Viewed another way, structured analogies reduced the average forecast error by 21% (where forecast error is the percentage of forecasts that were wrong)1.
1 We calculate average error reduction figures as {(100 – AC) – (100 – AX)} / (100 – AC) * 100, where AC is the unweighted average percentage accuracy across conflicts of the comparison forecasts (or chance) and AX is the corresponding figure for the forecasts of interest.
Table 2
Accuracy of structured-analogies and unaided-judgment forecasts by experts Percent correct forecasts a (number of forecasts)
Chance Unaided judgment Structured analogies Telco Takeover 25 0 (8) 8 (12) Artists Protest 17 10 (20) 27 (11) 55% Pay Plan 25 18 (11) 57 (14) Personal Grievance 25 31 (13) 36 (14) Zenith Investment 33 36 (14) 38 (8) Distribution Channel 33 38 (17) 50 (12) Water Dispute 33 50 (8) 92 (12) Nurses Dispute 33 73 (15) 57 (14) Averages (unweighted) 28 32 (106) 46 (97)
a Bold figures denote the most accurate forecasts for each conflict, and overall.
Value of experts’ experience
We tested whether structured-analogies forecasts were more accurate when they came from experts with more experience than when from those with less. We used two measures: (1) we asked our experts how many years experience they had as “a conflict management specialist,” and (2) we asked them to rate their experience (on a scale from 0 to 10)with situations similar to the target conflict.
Structured-analogies forecasts from experts with five or more years experience as conflict management specialists were less accurate (average across conflicts) with 21% error reduction compared to chance, than those with less experience (26% error reduction). Furthermore, where experts gave high ratings to their experience with similar
conflicts their forecasts were less accurate (16% error reduction) than where they gave themselves lower ratings (31%).
Effect of number of analogies
We found that forecasts based on data from experts who could think of two or more (plural) analogies were more accurate than those based on data from experts who recalled a single analogy for six of the eight conflicts. Accuracy averaged 38% for forecasts derived from single-analogy data, but 56% for those derived from plural-analogy data (P
= 0.02, one-tailed permutation test for paired replicates).
All else being equal, conflicts with more outcome options are more difficult to forecast than those with fewer options. To control for this, we examined the reduction in error versus chance. Forecasts based on recall of a single analogy reduced error by an average of 15% compared to chance, while forecasts derived from plural analogies reduced error by 39% (Table 3). The difference in error between single-analogy forecasts and plural-analogy forecasts is P = 0.02 using the one-tailed permutation test for paired replicates. The error was reduced by 42% versus chance by accepting data only from experts who described three or more analogies. Thus the usefulness of an individual expert was related to the number of analogies he described.
Table 3
Accuracy of forecasts by number of analogies Percent error reduction versus chance a (number of forecasts)
None b One only Two or more
Telco Takeover -33 (8) -33 (5) -14 (7) 55% Pay Plan -33 (2) 26 (9) 73 (5) Distribution Channel -19 (5) 0 (6) 50 (6) Artists Protest -3 (7) -3 (7) 40 (4) Personal Grievance 20 (5) 0 (8) 33 (6) Water Dispute 25 (8) 100 (4) 81 (8) Zenith Investment 25 (6) -12 (4) 25 (4) Nurses Dispute 100 (3) 40 (10) 25 (4)
Average error reduction (unweighted)
10 (44) 15 (53) 39 (44)
Average % correct (unweighted)
34 38 56
a Bold figures denote the most accurate forecasts for each conflict, and overall. b Forecasts from experts we asked to use the structured analogies method, who were
unable to think of analogies. We classified these forecasts as unaided judgment forecasts in all our other analyses.
Effect of experts’ familiarity with their analogies
We expected that the information experts provided would be more useful the more closely involved they had been in the analogous situations they identified, because they would be likely to know more about the situations. For example, someone who was an adult during the Vietnam War is likely to know more about that situation than someone born since, and someone who fought in the war is likely to know more again. To examine this, we identified forecasts that had been based on analogies from either experts’ own experiences (45) or that of close others (5 forecasts based on the experiences of, for example, a wife or brother-in-law). In an unweighted averaged across the eight conflicts, these direct-experience forecasts were more accurate (49%) than the 45 forecasts based on analogies from third-party accounts (37%); P = 0.07, one-tailed permutation test for paired replicates. Viewed another way, the forecasts based on analogies from experiences close to experts reduced the average error across conflicts by 31% (compared to chance) while forecasts that were based on indirect experience provided only 13% error reduction.
Familiarity and plural analogies
The ideal situation when forecasting with structured analogies is to find experts who can think of many analogies with which they have had direct experience. When our experts were able to think of two or more analogies and they had direct experience with the analogy that was most similar to the target, structured analogies forecasts were 60% accurate (23 forecasts). In other cases, 72 forecasts were 39% accurate (P = 0.04, one-tailed permutation test for paired replicates).
Mechanical schemes to derive forecasts
We wondered whether experts who had used the structured analogies process then provided forecasts that were more accurate than unaided experts. They did. Their predictions were on average 42% accurate (94 forecasts) compared to 32% for unaided-judgment forecasts (P = 0.06, one-tailed permutation test for paired replicates). As we anticipated, however, a structured mechanical process was more effective for deriving forecasts from the experts’ analogies information than experts’ own judgments. As we have seen, structured-analogies forecasts were 46% accurate. Why the difference when experts derived their own forecasts? Analogies are only useful if they are used. In 22 cases,
experts made forecasts that were inconsistent with the outcomes of their own analogies; of these, 25% were
accurate. When the mechanical rule was used to derive forecasts from these experts’ analogies, 45% were accurate.
When experts thought of more that one analogy, our mechanical scheme did not use all of the analogical information to make predictions. We tested four alternative approaches in order to determine whether we would improve
accuracy further if we derived combined forecasts from all of the 210 analogies with similarity ratings and implied decisions. For example, if an expert provided information on three analogies, for the purpose of testing our four combining alternatives we effectively derived three forecasts instead of the one we would have derived using the structured analogies method.
For our first alternative, we used the outcome implied by the most analogies, and obtained an average accuracy of 40% across all conflicts, compared to 46% for the approach we had adopted. For the second, instead of assuming that the analogies were all of equal value as we did for the first alternative, for each conflict we chose the option with the highest aggregate similarity rating as the forecast (39% accurate). For the third alternative, we reallocated each expert’s total number of analogies for a conflict to outcome options in proportion to each option’s share of the expert’s similarity ratings (40%). For the fourth alternative, we calculated each expert’s average similarity rating for each option; we then allocated all of each expert’s analogies to each option in proportion to the average similarity rating for the decision as a fraction of the sum of the expert’s average similarity ratings (39%). In sum, all of these alternatives provided forecasts that were less accurate than those derived by applying the mechanical scheme that we had specified prior to the testing the accuracy of structured analogies.
Effect of collaboration
While we had no directional hypothesis about collaboration, we analyzed the data to see whether collaboration among experts was useful. When experts using structured analogies collaborated with others, their median working time was 45 minutes compared to 30 minutes for those who worked alone. (We do not know how much time the collaborators spent on the task, nor do we know the nature of their collaboration.) As it happened, those who collaborated claimed to have had much more experience with conflict-management (median of 14 years versus 5 years) and experience with similar conflicts (a median self-rating of 4.0 out of 10, versus 2.8). Despite the greater
investment of resources by more knowledgeable experts, collaboration produced no gain in accuracy: Forecasts from solo experts were on average 44% accurate across conflicts (75 forecasts), compared to 42% for forecasts by collaborating experts (22 forecasts).
Given our findings, we saw no need to distinguish between solo and collaborative forecasts in our analysis. In view of the time savings, we recommend that structured analogies be done by individuals.
Limitations
The structured analogies method is useful only in cases in which experts can think of analogies. This limitation can be overcome in many situations by identifying people with relevant expertise. While this may be difficult to know in advance, one can gauge people’s expertise from their responses – that is, did they provide analogies, if so, how many, and did they have direct experience?
Using structured analogies is more costly than using unaided judgment. However, relative to the costs of making bad decisions in many conflict situations, such as selecting strategies to achieve peace in the Middle East or to deal with threatening behavior by the North Korean government, the costs are negligible.
Further research
Research on additional situations would help to better assess the improvements that might be expected, and the conditions under which structured analogies is most effective. Our conclusions are based on a sample of only eight situations.
This is the first published study on the use of structured analogies. More research needs to be done to develop the operational procedures for the method. For example, what is the best way to frame the issues for the experts so that they provide more and better analogies? Would a more structured approach to rating analogies’ similarity to a target help administrators derive more accurate forecasts? To what extent might improvements in accuracy be obtained, in the case of well-documented analogies, by checking the facts of the situation and correcting any errors in experts’ matching of analogy outcomes with potential target outcomes.
It seems plausible that the Delphi technique could be used to improve assessments of analogies’ similarity to a target, potentially increasing accuracy further at a low cost. Rowe and Wright (2001) provide evidence on the value of Delphi, and software for implementing of Delphi is provided at forecastingprinciples.com. Experts’ confidence ratings may be useful for weighting structured-analogies forecasts in a combination (Arkes, 2001).
We have examined conflict situations because of their importance and the difficulty of obtaining useful forecasts. Structured analogies might also improve forecasting for situations other than conflicts. We expect that it would be most useful where situations are complex and where there are plural analogies.
Research is needed on how to encourage adoption of structured analogies. Currently, people use unaided judgment, a method that is little better than chance, to decide whether to go to war, get a divorce, make a hostile takeover bid, go on strike, or mount a competitive pricing campaign. Better forecasts would aid decision making in such
situations.
Conclusions
It is difficult to forecast decisions made in conflict situations. On average, unaided experts were correct for only 32% of their predictions.This was little better than chance at 28%.
For our structured analogies method,the two key criteria for identifying an expert were the number of analogies generated, and the presence of direct knowledge about those analogies. When experts produced two or more analogies from experience, forecasts from structured analogies were correct for 60% of the predictions. Given the importance of forecasts in conflict situations and in other arenas, such improvement could have considerable benefits.
Appendix A
E-mail message appeal and instructions: Structured analogies / collaboration treatment
Subject: Using analogies to predict the outcomes of conflicts
Dear Dr _____________
I am writing to you because you are an expert on _________. I am engaged on a research project on the accuracy of different methods for predicting decisions made in conflicts. At this stage, I’m investigating the formal use of “analogies” for forecasting. That is, forecasting on the basis of the outcomes of similar conflicts that are known to the forecaster.
What I would like you to do is to read the attached descriptions of some real (but disguised) conflict situations and to predict the outcome of each conflict. If you can’t read the attachments, please let me know and I’ll send the material in your preferred format if I’m able.
Each attached file contains a conflict description and a short questionnaire. Please follow these steps for each conflict:
1/ Read the description and
2/ try to think of several analogous situations and 3/ about how similar your analogies are to the conflict. 4/ Fill-in the questionnaire (electronically if you can)
a) describe your analogies b) rate your analogies
c) make your prediction (either pick an outcome or assign probabilities) d) record the total time you spent on all tasks
e) return the questionnaire.
One of the objectives of this research is to assess the effect of collaboration on forecast accuracy. You have been allocated to the collaboration treatment, so please do discuss these forecasting problems with colleagues. Do not, however, discuss them with other people who have received this material as I want independent responses from participants.
Although I intend to acknowledge the help of all of the people who assist with this research, my report will not associate any prediction with any individual.
Your prompt response is very important to the successful completion of my project. Please help me to prove the sceptics wrong about the level of cooperation I get!
Best regards,
Appendix B
Telco Takeover Bid 1) (A) In the table below, please briefly describe
(i) your analogies,
(ii) their source (e.g. your own experience, media reports, history, literature, etc.), and (iii) the main similarities and differences between your analogies and this situation. (B) Rate analogies out of 10 (0 = no similarity… 5 = similar… 10 = high similarity). (C) Enter the responses from question 2 (below) closest to the outcomes of your analogies.
(A)
(i) description, (ii) source, (iii) similarities & differences
(B) Rate
(C) Q2 a. Bank takeover Personal Issue same, industry different 8 C
b. Govt Agency merger Personal Takeover same, government, but ordered takeover
4 D
c. Facility Merger Personal/family Combine similar operations 3 B
d. e.
2) How was the standoff between Localville and Expander resolved? (check one 9, or %)
a. Expander’s takeover bid failed completely [__]
b. Expander purchased Localville’s mobile operation only [__]
c. Expander’s takeover succeeded at, or close to, their August 14 offer price of $43-per-share [X_] d. Expander’s takeover succeeded at a substantial premium over the August 14 offer price [__]
3) If you have not given a prediction, please state your reasons:
4) Roughly, how long did you spend on this task?
{include the time you spent reading the description and instructions} [_1__] hours 5) How likely is it that taking more time would change your forecast?
{ 0 = almost no chance (1/100) … 10 = practically certain (99/100) } [_0_] 0-10 6) Do you recognise the actual conflict described in this file? Yes [__] No [X__]
If so, please identify it: [_________________________________________________]
7) How many people did you discuss this forecasting problem with? [_2___] people
8) Roughly, how many years experience do you have as a conflict management specialist? [20+] years
9) Please rate your experience (out of 10) with conflicts similar to this one [6____] 0-10
When you have completed this questionnaire, please return either this document as an email attachment to…
References
Arkes, H. R. (2001). Overconfidence in judgmental forecasting, in J. S. Armstrong (ed.) Principles of Forecasting. Boston: Kluwer Academic Publishers.
Armstrong, J. S. (2001). Combining forecasts, in J. S. Armstrong (ed.) Principles of Forecasting. Boston: Kluwer Academic Publishers.
Armstrong, J. S. (1985). Long-Range Forecasting. New York: John Wiley. Full text at hops.wharton.upenn.edu/forecast
Breuning, M. (2003). The role of analogies and abstract reasoning in decision-making: Evidence from the debate over Truman’s proposal for development assistance. International Studies Quarterly, 47, 229-245.
Buehler, R., Griffin, D., Ross, M. (1994). Exploring the ‘planning fallacy’: Why people underestimate their task completion times. Journal of Personality and Social Psychology, 67, 366-381.
Canada Centre for Remote Sensing (1996). ‘Sea Empress’ oil spill monitoring. RADARSAT Image, Milford Haven, Wales, United Kingdom, February 22, 1996. Retrieved July 1, 2003, from
http://www.ccrs.nrcan.gc.ca/ccrs/data/satsens/radarsat/images/uk/ruk01_e.html.
Green, K. C. (2002). Forecasting decisions in conflict situations: A comparison of game theory, role-playing, and unaided judgement. International Journal of Forecasting, 18, 321-344. Full text at forecastingprinciples.com. Kahneman, D., & Lovallo, D. (1993). Timid choices and bold forecasts: A cognitive perspective on risk taking.
Management Science, 39, 17-31.
Khong, Y. F. (1992). Analogies at War: Korea, Munich, Dien Bien Phu, and the Vietnam Decisions of 1965. Princeton, NJ: Princeton University Press.
Kokinov, B. (2003). Analogy in decision-making, social interaction, and emergent rationality. Behavioral and Brain Sciences, 26, 167-168. Full text at http://www.nbu.bg/cogs/personal/kokinov/bbskokinov.pdf
Langdon, C. (2000a). Nurses vote today on strike. The Dominion, Edition 2, 20 September, 3. Langdon, C. (2000b). Nurses support call for strike. The Dominion, Edition 2, 21 September, 3. Langdon, C. (2000c). Nurses’ pay boosted, strike off. The Dominion, Edition 2, 6 December, 3.
MacGregor, D. G, (2001), Decomposition for judgmental forecasting and estimation. In J. S. Armstrong (ed.)
Principles of Forecasting. Boston: Kluwer Academic Publishers.
McIntyre, S. H., Achabal, D. D., & Miller, C. M. (1993). Applying case-based reasoning to forecasting retail sales.
Journal of Retailing, 69, 372-398.
Neustadt, R. E., & May, E. R. (1986). Thinking in Time: The Uses of History for Decision Makers. New York: Free Press.
Radio New Zealand Limited (2000a, September 20). Brenda Wilson (Chief Executive, New Zealand Nurses Organisation) interviewed by Geoff Robinson. Morning Report, Transcript: Newztel News Agency Ltd.
Radio New Zealand Limited (2000b, September 20). Rae Lamb (Health Correspondent, Radio New Zealand) interviewed by Mary Wilson with exerted material from interviews with Annette King (Minister of Health), Susan Rolls (Emergency Nurse at Wellington Hospital), and Russell Taylor (Wellington Nurses Union Organiser). Checkpoint, Transcript: Newztel News Agency Ltd.
Radio New Zealand Limited (2000c, September 22). Margot Mains (Chief Executive Officer, Capital Coast Health) interviewed by Geoff Robinson. Morning Report, Transcript: Newztel News Agency Ltd.
Rowe, G. & Wright, G. (2001). Expert opinions in Forecasting: The role of the Delphi technique,in J. S. Armstrong (ed.) Principles of Forecasting. Boston: Kluwer Academic Publishers.
Schrodt, P. A. (2002). Forecasts and contingencies: from methodology to policy. Paper presented at the American Political Science Association meetings, Boston, 29 August – 1 September. Retrieved May 7, 2004, from http://www.ukans.edu/~keds/papers.dir/Schrodt.APSA02.pdf.
Siegel, S. & Castellan, N. J. Jr. (1988). Non-parametric Statistics for the Behavioral Sciences, 2nd ed. Singapore: McGraw-Hill.
Speke, C., & Reuter, S. (2003). German military historians predict Anglo-American defeat in Iraq. Online Journal, March 29. Available from http://morris.wharton.upenn.edu/forecast/Conflicts/PDF%20files/
Iraqi%20conflict%20forecast.pdf.
Stewart, T. R., & Leschine, T. M. (1986). Judgment and analysis in oil spill risk assessment. Risk Analysis, 6, 305-315.
Tetlock, P. E. (1999). Theory driven reasoning about possible pasts and probable futures: Are we prisoners of our perceptions? American Journal of Political Science, 43, 335-366.
Acknowledgements
We thank the experts who participated in the research reported here. They included Roderic Alley, Barry Anderson, Don Baker, Corrine Bendersky, Constant Beugre, Doug Bond, Michelle Brackin, JosJ Ram\n Cancelo, Nihan Cini, David Cohen, Ike Damayanti, Serghei Dascalu, Nikolay Dentchev, Ulas Doga Eralp, Miguel Dorado, Erkan Erdil, Jason Flello, Paul Gaskin, Andrew Gawith, Kristian Skrede Gleditsch, Joshua Goldstein, David Grimmond, George Haines, Claudia Hale, Ragnar Ingibergsson, Patrick James, Michael Kanner, John Keltner, Daniel Kennedy, Susan Kennedy, Oliver Koll, Rita Koryan, Talha Köse, Tony Lewis, Zsuzsanna Lonti, Dina Beach Lynch, David Matz, Bill McLauchlan, Kevin Mole, Ben Mollov, Robert Myrtle, W. Bruce Newman, Randall Newnham, Konstantinos Nikolopoulos, Glenn Palmer, Dean G. Pruitt, Perry Sadorsky, Greg Saltzman, Amardeep Sandhu, Marlies Scott-Wenzel, Deborah Shmueli, Mike Smith, Marta Somogyvári, Harris Sondak, Dana Tait, Scott Takacs, Dimitrios Thomakos, William Thompson, Ailsa Turrell, Bryan Wadsworth, James Wall, Daniel Williams, Christine Wright, Becky Zaino. We also thank Lisa Bolton, Nikolay Dentchev, Don Esslemont, Stanley Feder, Paul Goodwin, Clare Harries, Oliver Koll, and Tom Yokum for providing pre-submission peer review. Editorial assistance was provided by Mary Haight and Marian Lee.
Note to reviewers: The following materials will be placed on www.conflictforecasting.com as part of full-disclosure to aid replication and extensions. Our paper is self-contained; there will be no need for other readers to access these materials.
Reviewer Appendix 1
ARTISTS’ REPRIEVE
The Country
Histavia is a relatively small country which is also one of the most densely populated nations in the world. It has some energy resources (mostly gas) but very few other natural resources. Farming, fishing and trading have always been important parts of this country’s economy. More recently the country has become quite involved in manufacturing with over 40% of the people employed in various industries. The government is renowned for its bureaucracy and for the extravagant systems it puts together. For example, in order to help with the population problem and overcrowding, the government subsidised emigration.
The government is a constitutional monarchy. Executive power lies in a cabinet selected by the majority in Parliament. Histavia’s parliament has two houses. The 1st House contains appointed representatives from each province’s legislative body. The 2nd House is elected by direct vote and develops all legislation. A bill passed by the 2nd House is sent to the 1st House for a vote.
Artistic Problems
Visual art has always been a very important part of Histavian culture. Three hundred years ago, Histavia’s seaports and rivers made it a major trading hub for the surrounding nations. Histavians became relatively wealthy and wanted to chronicle the times. They wanted pictures of themselves, their families, homes, and the countryside. Demand was so great that an artist could specialise in whatever subject she wanted to paint. Out of this environment came some of the world’s greatest artists. The Histavian people thus developed a special regard for art and artists.
This prosperity didn’t last. The private market for art has been very weak for a long time. There are currently sufficient jobs available for artists but there isn’t enough demand for many artists to make a living selling their works. The government feels a compelling need to do something about this problem. Since the private market isn’t there, the government feels obliged to protect the artists’ financial freedom and thereby to preserve art for the sake of art. Partially to reward artists for their contribution to Histavia’s culture and partially because the main artists’ union has made a big fuss over the continuing conditions for art, the government has developed a simple programme to keep artists from having to drive taxi cabs. An artist will sign up on the programme for up to a year. During that time the government will buy the artist’s wares. Once the artist’s time period is up, the artist must sell his works on his own or find another occupation. During the government purchase period, if the artist begins to sell her works in the private market, the government will end its assistance early.
Counterculture
The programme was accepted and seemed to work well for about twenty years. In the late 1960’s the counterculture boomed worldwide. Artists in Histavia were particularly upset with the continuing conditions for artists. They felt anyone who said they were an artist was one, and that the programme’s length wasn’t sufficient time for an artist to get established given the continuing weak demand for art. To press their point, members of the main artists’ union occupied the most valuable room of Histavia’s primary art museum. They refused to leave until entrance requirements were relaxed and artists were allowed to remain in the programme indefinitely.
Additional Information
• The conflict occurs in the late-1960’s
• This is a developed, industrialised country (i.e.; not Third World)
• Not all artists are members of the main union, but union leaders feel they speak for and represent all of Histavia’s artists. They also believe they can gain public support for their cause fairly easily.
Current entrance requirements to the programme involve acceptance by the government on the basis of some criteria.
Roles: Histavian Bureaucrats
You are two Histavian bureaucrats responsible for administering the government’s programme for purchasing art. You report directly to the executive cabinet, which represents the majority in both Houses and the view of the Queen. The cabinet has given you negotiating power to resolve the artists’ sit-in and feels confident it can win support for whatever agreement you reach with the artists. The government and the people are very concerned for the well being of art and artists. The majority party, as always, is concerned with negative public opinion affecting the outcome of the next election. Additionally, the room the artists are holding contains some of the country’s most treasured art works. It’s your job to negotiate and resolve the issue.
All facts and impressions that are available are given. Extrapolate as necessary.
Roles: Artists’ Union Leaders
You are two leaders of the main artists’ union and have taken it upon yourselves to represent not only your fellow union members in the room, but all Histavian artists. You know one of the most important things to Histavian people is their art. Still the private demand for art isn’t sufficient to support all the artists Histavia’s rich culture has inspired. You’re meeting with two government agents responsible for administering the current programme. They are influential people who have been sent to resolve the sit-in and have the backing of the majority party in power. Right now you occupy the room in the art museum which contains the country’s most treasured art works. You want exit and entrance requirements to the programme relaxed before you’ll leave. You feel strongly you represent the viewpoint of all artists and it will be easy to gain public support for your cause.
DISTRIBUTION PLAN Background
The year is 1961. The Ace Company has been in business over seventy years and has become a major producer of home appliances. The home appliance industry has had a terrible start this decade. Sales have been weak, inventories are high, dealers are demoralised, and mass merchandisers and foreign competitors have entered the market slashing margins. While the recession seems to have bottomed out by mid-1961, Ace’s operating deficit is approaching $6 million for the year. The company, however, feels the fall introduction of the colour TV set might recover some the loss.
Ace’s problems seem to be short term. Some existing new products in the development cycle are draining funds and the year’s poor sales has created a cash flow crisis. Now that consumer purchases are picking up, funds aren’t available for heavy promotion and prices are still soft due to foreign competition and excess inventories. One component of the marketing mix can be attacked – distribution.
Appliance Discount Plan
Competition between supermarkets for customers has always been heavy. Discount houses are opening up and every store seems to be giving trading stamps or running some kind of promotion. Ace feels it has come up with an excellent new distribution plan that will make everyone involved happy and return much needed appliance sales. It’s called the Cash Register Tape Plan (CRTP). An Ace dealer will link up with an area supermarket. The supermarket will be given an exclusive contract in return for floor space to display some major appliances. The dealer will supply a sales person to explain the appliances and show pictures of items not on display. A shopper accepted by Ace Financing receives a 12-36 month, no down payment instalment plan and has the item immediately delivered by the dealer. Each month the purchasers bring in their cash register tapes from that supermarket. Five and a half percent of the tape total is taken off the monthly instalment (up to 75% of the payment). The payment of the monthly discount is split between Ace and the supermarket based on a sliding scale. If a shopper purchases less than $50, Ace pays the full 5½%. If she purchases more than $120, then Ace pays 2½% and the supermarket 3%.
The CRTP is designed to benefit all those involved. Naturally Ace and their dealers expect to enjoy increased sales. The shopper will get a reduced price on a major appliance by altering purchasing habits. The supermarket should be able to reduce split market shopping, increase purchases by regular customers of items often purchased elsewhere, obtain new customers, and generally build traffic. There are, however, costs to the plan. Ace receives a lower price, the dealer has a salesperson at a remote location, the customer won’t be able to “shop around” for the lowest prices and the supermarket has to give up floor space and pay part of the discount.
Selection Process
Ace wants to deal with regional chains. This strategy will require getting agreement from only one source (per regional market) before they’re able to start up the CRTP in the metropolitan areas they’ve selected. Expansion to additional cities will also be easily accommodated. Ace will select the dealers who will participate based on their proximity to the individual stores selected to participate from the supermarket chains. Both the dealer and the supermarket must be approved by Ace Financing before they can join the CRTP. Dealers not near a selected supermarket or in a region not selected for the plan won’t participate. If dealerships overlap a supermarket’s territory they will sell in the stores on alternate days.
Additional Information
• Ace carries a full line of major household appliances and is a well-known national manufacturer.
• Small appliances (like toasters) have been sold through supermarkets as promotion items
• Assume Ace can present some attractive return figures for the supermarket based on the given sliding scale discount payment procedure
• No actual test of the CRTP has been conducted Roles: ACE Executives
You are the ACE marketing vice-president and the marketing director of the appliance division. You are preparing to meet with the management of BIG VALUE supermarkets (a mid-West grocery store chain). They have been approached about the CRTP, received an outline of the plan, and have agreed to meet with you. In addition, you’ve sent a market analysis your division pulled together indicating how much a chain like BIG VALUE should profit by participating in the CRTP.
It’s your job to sell the CRTP! You see it as a potential saviour for ACE’s current problems and a unique distribution tool. You are particularly worried about other stores, dealers, and/or manufacturers copying the CRTP. You want to move as quickly as possible once the programme starts. Obviously the continued sales slump and cash flow problems will negatively reflect on your careers.
All facts and impressions that are available are given. Extrapolate as necessary. Roles: BIG VALUE Vice-Presidents
You are the two top vice-presidents of BIG VALUE supermarkets, a mid-West, regional grocery store chain. You have agreed to meet with the marketing v-p and marketing director of ACE’s appliance division after having received information on ACE’s CRTP and a market analysis (from ACE) which projects favourable returns for a store chain like yours. You have also heard about ACE’s current problems.
It’s your job to discuss the pros and cons of the CRTP for your chain and be prepared to react to the ACE personnel. Ultimately you must decide whether or not to accept the CRTP and in what form. BIG VALUE’s top management is aware of the plan and will accept your decision. Positive or negative results of your decision will obviously impact on your career.
THE 55% PLAN
The collective bargaining agreement between the National Football League (NFL) Management Council and the NFL Players’ Association (NFLPA) expires on July 15, 1982. The payers’ number one demand is for a fixed percentage of the football clubs’ gross revenue. Ed Garvey, the executive director of the NFLPA since 1971, says that 55% of the gross revenue is the players’ “bottom line” demand. Jack Dolan, executive director of the Management Council since 1980, rejects the union’s demand for a percentage of the gross revenue regardless of what the percentage is.
NFLPA
The players’ last contract ended in 1974. The players struck for six weeks during the ’74 season but returned to work without a contract to begin playing regular season games. The major issue then was free agency. A free agent is a player who doesn’t sign a new contract with his team when the current one expires. Once the contract ends, any team can bid for his services. The owners refused to agree to a free agent system so the NFLPA took the case to federal court and won most of their demands. This decree, however, didn’t come until 1976. The final contract was signed in 1977, three years after the last contract expired.
Unrestricted free agency in most sports has caused players’ salaries to increase substantially, however this has not happened in football. Of the five hundred players who became free agents since 1977, only six have changed teams. The average salary in the NFL was $78,000 in 1980 compared with $108,000 in hockey, $143,000 in baseball, and $186,000 in basketball. Garvey insists the poor performance of the free agent system in football occurs because the NFL clubs practice “corporate socialism”. All clubs share equally in TV revenues and playoff monies. This ensures even a team with a terrible record will be financially sound. There is, therefore, economic incentive to replace more expensive veterans with less expensive rookies. Others feel Garvey, at least in part, is to blame for the slow movement of free agents. In the ‘77 contract, the union agreed to compensation procedures for a team losing a free agent. Many feel the nature and magnitude of the compensation restricts free agency to the point where no club is willing to sign a good free agent. They point to the case when one of the best running backs became a free agent and not one club made him an offer.
The 55% plan is designed to increase all players’ salaries and provide incentive bonuses for excellent playing. Some of the higher paid players, however, may not benefit very much under the proposed method for distributing the wages. This plan will also eliminate the individual contracts negotiated by the teams with each player and, in effect, make the players’ agents obsolete. As for the players’ unity and their resolve to attain the 55% plan, Garvey feels virtually every player is ready to strike and there is no comparison between the situations in 1974 and 1982.
All players now belong to the NFLPA. Communication with the dispersed membership is vastly improved⎯players and their wives have been thoroughly briefed on the 55% plan, and there will be a union meeting in mid-March to iron out all the details of the wage distribution. The AFL-CIO has been asked to provide assistance from the entire labour community and a battle cry is being sounded in a 20 minute film featuring popular sports and entertainment personalities.
NFL Management Council
While Dolan is the new kid on the block, he’s no rookie. As a senior vice-president of industrial relations for nine years with National Airlines, he negotiated contracts with eight unions and went through four strikes. Dolan consults with six owners who make up the council’s executive committee. All, except one, are described as “hard liners”.
To the owners, a fixed percentage is equated with control and the owners “are not going to let the union run their business”. They deny Garvey’s contention they have no incentive to win because of the leagues financial structure. Besides being successful businesspeople who want their clubs to do the best they can, the team’s won-lost record effects the price owners can charge for tickets, the demand for luxury boxes, and the amount of money lost from concessions because of no shows. From the union’s own estimates, the LA Rams had $7.7 million in profits for 1980 while the Denver Broncos had $2.5 million.
The council feels Garvey will have trouble gaining public sentiment and maintaining rank and file support for such an unorthodox plan. In addition, their pre-contract strategy appears to be to build a lack of confidence in Garvey, so it will be difficult for him to maintain players’ support for a strike. People either love Garvey or hate him. His shoot from the hip, wisecracking style often leaves a trail of damning quotes. The NFL Management Council uses a monthly newsletter to “their employees” and the sports press to present Garvey’s quotes and actions in the worst light. Dolan is intent on using these weaknesses to stop any attempt to have a fixed percentage of gross revenues as the collective bargaining agreement’s wage clause. Estimates of the percentage of gross revenues currently going to the players are between 25% and 45%. Naturally the NFLPA claims the figure is between 25-30% and the NFL Management Council places it near 45%.
Additional Information
• Current compensation losing a free agent is by way of draft choices, i.e. if the player’s salary is greater than $x, the team signing the free agent loses two 1st round draft choices. A draft choice is a team’s right to select a player from a list of all available college players. Teams take turns to pick, with the weakest team choosing first and so on.
• The percentage of gross revenues will go into one pool, which the NFLPA will distribute based on a base salary plus incentive bonuses.
Roles: Management & Owner Representatives
You are the executive director and the head owner representative of the NFL Management Council. You are about to meet with the NFLPA to decide the 55% issue. You both hold very strongly to the statements attributed to the owners and view the NFLPA comments as “hogwash” and propaganda. Prepare and defend your position for the upcoming negotiation.
All facts and impressions that are available are given. Extrapolate as necessary. Roles: Player Representatives
You are the executive director and the head player representative of the NFLPA. You are about to meet with the
Management Council to negotiate and decide the 55% issues. You both hold very strongly to the statements attributed to the NFLPA and view the owners’ comments as “hogwash” and propaganda. Prepare and defend your position.
A Pay Dispute Between City Wide Health and Nursing Staff
City Wide Health Ltd is a government-owned Hospital and Health Service.
The Government funds CWH primarily by purchasing a set level of health services, for example so many hip operations. The level of funding is based on what the Health Funding Authority has calculated are reasonable prices to pay for the services the Government wishes to purchase.
It is the responsibility of the CWH Chief Executive, Hillary Strong, to manage the company’s affairs so that it can deliver the services it has contracted to provide while remaining financially viable. CWH’s 2000/2001 contract with the HFA is worth $258 million, an increase of nearly 10% over the previous year’s contract; this reflects both some price increases and some increase in services provided. There are more than 3,000 staff, and payroll costs are more than $140 million per year – over half of the value of the HFA contract. CWH employs nearly 1300 full time equivalent nurses; their pay costs roughly $55 million per year. The collective employment contract many of the nurses are party to has expired and the nurses and management are currently involved in a dispute over their pay.
The National Nurses Organisation (NNO) represents more than half of the nurses employed by CWH. The nurses are angry. They consider that they have borne the brunt of budget constraints over the years and that their dedication to their profession has been exploited. Perhaps as a consequence of this, the turnover of nurses is high – as many as one-third of nurses leave CWH during the course of a year, often taking up highly paid nursing jobs overseas.
Some specialist nurses are more difficult to replace than others. Earlier this year, Intensive Care nurses obtained a 7% pay increase from CWH. Junior doctors also obtained a large pay increase in separate negotiations. With these precedents, nurses instructed NNO to retract the bid they made in February for a 5% pay increase and instead seek 7% on all salaries and allowances for the 12 months from 1 October. In addition, the NNO are seeking substantial increases in penal rate multipliers, meal, night and weekend allowances, study and long-service leave provisions, and reimbursement of professional association costs.
Hillary Strong and the CWH negotiation team, led by Mark Jensen, have now offered the nurses a 5% pay rise with no new contract for 2 years and have stated that this is most the company can afford. At a stop-work meeting nurses rejected the 5% offer and voted almost unanimously in support of holding a 16 hour strike on Sunday, October the 1st
. Nurses at the meeting criticised NNO negotiators for being too willing to compromise. The nurses gave NNO negotiators Jack Naylor, Fergus Black, and the rest of the team clear instructions “not to come back” without an agreement for at least a 7% increase.
The two sides are at odds in their interpretation of CWH’s ability to afford a 7% increase. CWH has been struggling to reduce debt levels by reducing costs, increasing efficiency, and only providing services it is funded to provide. The Government requires CWH to break-even, and the CWH negotiators claim that even a 5% pay increase would result in a deficit if cost savings cannot be found elsewhere. The NNO, on the other hand, maintains that CWH is effectively receiving a bonus from the HFA when they are paid for additional procedures. This is because providing an additional procedure of a type CWH is already providing does not cost as much as the standard price per procedure the HFA pays. Moreover, NNO claims that a 2.5% “creep” in salaries that CWH has budgeted for will not occur due to the current composition of the nursing workforce and the high attrition rate. Creep occurs when staff receive increases in salary that are related to their length of service. The NNO position is that the “bonus” revenue that will accrue from payments for additional procedures and the absence of creep are sufficient to meet the $1.1m difference between CWH’s 5% offer and the nurses’ 7% claim.
Both parties have a lot at stake. Hillary Strong’s performance is assessed, in part, on her financial management of CWH, while the nurses are prepared to go on strike in support of their strongly held belief that they deserve a 7% pay increase. Meanwhile, the Minister of Health has been sending mixed signals to the two parties about what might be an appropriate settlement and has also stated that the Government will not become involved in the dispute.
Assume it is now Monday the 2nd
of October. The Sunday strike is over and the two parties are about to recommence negotiations for a settlement of the dispute. If in impasse in negotiations is reached Stephen Worley, an independent mediator from the Government’s new Mediation Service, will be appointed. Stephen will briefly interview each party separately and will then endeavour to mediate an agreement. The dispute will be resolved, but after how long and with what terms?
Hillary Strong, Chief Executive of City Wide Health
Hillary Strong trained and worked as a nurse, but has an MBA and has for some years held senior management positions. Most recently, before taking up the appointment at CWH, she was the Chief Executive of another health provider. It was her success in that position that made her a strong candidate for the position she currently holds.
Her role is not easy, and probably rather thankless. Government expects her to run CWH as a business in many ways but, in reality, it is neither fish nor fowl. She is faced with political pressures and with the fact that many people, including many nurses, do not believe that hospitals really are, or should be, businesses. In this ambiguous environment, she is obliged to keep within her budget while ensuring that the hospitals continue to run as smoothly as possible.
Hillary has a good working relationship with Mark Jensen, her chief negotiator. She has used him and his company for advice with labour relations and human resource matters for many years.
Mark Jensen, City Wide Health Negotiator
Mark Jensen is a seasoned industrial negotiator and human resources expert with his own successful business. He has a lot of experience in the health sector. Mark has a direct and pragmatic style and, like Hillary, has quite a lot of sympathy for the nurses. He is, however, employed by CWH and his role is to help Hillary Strong by negotiating an agreement with the nurses’ representatives that is consistent with achieving a healthy financial position for CWH without undermining CWH’s role as a health services provider.
Jack Naylor, New Zealand Nurses Organisation Organiser
Jack Naylor is a seasoned industrial negotiator and union organiser. Before taking up his present job, he worked for a major public sector employees union. Jack’s current role is to represent the interests and wishes of the nurses employed by CWH who are members of the NNO, the nurses’ union. This includes organising the nurses to take effective protest and strike action. Jack and his team negotiate on their behalf in order to get the best possible deal for them. Because he is dealing with the CWH managers and negotiators a lot, he understands their point of view, perhaps better than most members, and has developed a good working relationship with them. At the end of the day, however, he can only make recommendations to the nurses. It is they who will decide whether to accept or reject a pay offer and whether or not they will go on strike or take other action. Feelings are running high in this dispute and there is every sign that the nurses are determined to take further action if necessary.
Jack has an open style and believes in the cause of his members. He is being assisted in the current round of negotiations by Fergus Black. Fergus is a senior nurse, and a real asset to the negotiating team.
Fergus Black, Nurse and Negotiator
Fergus is an experienced nurse who has relatively recently returned home after a number of years abroad. He was very well paid working as a nurse overseas. The contrast with the poor situation of nurses locally is marked.
As well as working as a nurse for City Wide Health, Fergus is currently helping the NNO, as a negotiator in the pay dispute. He represents the interests and wishes of the nurses employed by CWH who are members of the union.
Stephen Worley, independent mediator, Mediation Service
Stephen Worley may be asked to act as mediator in the dispute. His role as mediator would be to:
• control the process
• ensure that each party states their case
• help the parties to identify issues and options for settlement • look for a solution that will satisfy the needs & interests of both parties • record the agreement in writing
The Mediation Service web-site includes the following information on mediation:
“Mediators will help the parties decide on the process that is most likely to resolve problems as quickly and fairly as possible.” and
“If the employer and employee cannot reach agreement in mediation, they can agree, in writing, to the mediator making a final and binding decision. The mediator will explain to the parties that once he or she makes a decision, that decision cannot be challenged. The mediator’s decision is enforceable…”
