Testing Part A: Situating the analysis

Part A of the framework for anticipating and interrogating bias was tested with the three Australian leaders of the biophysical, sociocultural, and DSS components during May

199728. Note that Part A is intended to situate the analysis within a particular decision­ making environment, see Section 4.7.

Participants’ perspectives as to the purpose of the DSS were closely convergent and therefore provide insight into the dominant framing of the DSS held at the time that the discursive interrogation of biases commenced. In describing the purpose of the DSS, the participants all spoke of: assisting decision-makers to clarify options; what-if simulation to allow users to view the impacts of different options; generating better understanding of the consequences of different options; and providing trade-off analysis between conflicting options. These perspectives closely matched the description within the research proposal which framed the DSS as a computer-based tool integrating different models with a GIS database to allow systems modelling of different management options.

Although originally intended merely to situate subsequent discussions, Part A illustrated how consideration of the framework questions may highlight individual or group

ignorance and thereby facilitate learning. For example, although one participant was familiar with the broad policymaking environment (“fragmented decision-making by different departments; conflicting, top-down impositions of policy/practices on villagers; business... having relative freedom in laissez-faire political context”), the participants were generally unsure about the nature of the specific policymaking processes that the DSS was intended to support, including whether there were any existing processes by which local communities could give input into decisions. As the DSS leader remarked:

“This is an important question because we are talking about decision support: raises an important gap in our project planning. We want the DSS to be more than an end-point. We need to be able to integrate it into the decision-making processes... Its important if we want the technology to be more than just a computer”.

The topic of potential users of the DSS also revealed uncertainty. In their individual questionnaires, the DSS leader and sociocultural leader had each tentatively suggested that all stakeholders could be users, while the project leader had nominated technical personnel and some policy advisers from government agencies. During the group dialogue, the DSS leader, revisiting the earlier debate about policymaking processes, argued the need to “look at exactly how the stakeholders are actually going to use it” if the DSS was to be incorporated into policymaking processes. Taking up the challenge, the project leader proposed that different groups could use different packages in the DSS, in which case “any groups, except the local landholders because of their literacy level, could use the technology”. The DSS leader countered, “But the output of the technology can be presented in other forms like maps for the local people; we are not entirely dependent on a software interface - we will need people to be the human

28 Note that funding o f the Australian component o f the IWRAM project by ACIAR had not been approved yet, but was expected.

interface too”. Forestalling resolution of this issue, the sociocultural leader closed this discussion by suggesting that a decision could be made about this matter at some later date.

Group participants were also unsure about how the DSS was supposed to relate to the participatory framework, the other principal output of the project. In both their individual questionnaires and the group dialogue, neither the project leader or DSS leader felt able to comment. In her questionnaire, the sociocultural leader suggested that “I’d prefer it to be subsidiary, adjunct to the participatory stakeholder process, not to dominate it”. She reinforced this position during the group dialogue, and raised a possible tension between the DSS technology and a participatory methodology:

“I would prefer the DSS to be subsidiary to the participatory framework; that is one problem with LUPIS where people are brought to participate around technology rather than a true participatory approach. At most DSS should be a parallel process to the participatory framework - the DSS should illuminate but not substitute for these processes”.

When asked what they thought were the major advantages of developing a DSS for this application, the participants all focussed on the users. The sociocultural leader

suggested the DSS would help the stakeholders clarify their options, the DSS leader felt that the DSS offered the “ability to present complex science and analysis in a format more readily understood in real-world terms”, and the project leader emphasised potential efficiency gains for users.

When asked about the major disadvantages, the participants again focussed on the users, highlighting issues relating to both biased access and embedded bias. In terms of the former, the DSS leader raised the “danger of the system being in the hands of

technocrats” (political biases), and the project leader suggested that “the technicality may prejudice the use of it by certain stakeholders” (literacy biases). Revisiting the

DSS-versus-participation tension, the sociocultural leader identified the risk that people would be “put off the whole participatory process because it’s seen as technocratic”. Lighting on a research-versus-use tension, the DSS leader also noted a danger that “we develop an academically interesting system but fail to deliver an operational system”. In terms of embedded bias, the sociocultural leader identified a “risk of inequity in what it includes because it is so easy to lean to the measurable” (absences of knowledge), while the DSS and project leaders both highlighted the problem of identifying and communicating uncertainties (absences/distortions in knowledge). The project leader recognised that due to uncertainty, the output of the technology could be misleading, and the DSS leader argued that:

“we are talking about coupling together many types of models, data - all these errors propagate, when we try and integrate those the challenge is to quantify that uncertainty to stakeholders, who already have enough to deal with in terms of multiple options. But at least we can quantify the confidence of output of the DSS”.

However, the project leader feared that it would not be possible to quantify confidence bounds, in which case communication would be important. With the other participants agreeing that communication to stakeholders of uncertainties in terms of both

quantitative and qualitative information would be vital, the group discussion closed. With the benefit of hindsight, this initial discussion foreshadowed many important issues which were to reemerge continually over the course of my research process.

Indeed, the story o f shifting constructions about the DSS may equally be reframed as a story o f conflict and shifting constructions about how the DSS ought to relate to the current and future policymaking environment, how the DSS ought to relate to the participatory framework, who would be likely or able to use the DSS, and the

implications o f different users’ identities and characteristics for the form and content o f the DSS. Intertwined with these issues are the tensions between developing a DSS for research and a DSS which will be used in practice, the dilemma o f who should be responsible for considering communication o f the DSS output to whom (and when), and the tension between the DSS-as-technology and participatory ethics and methodology. Shortly after the Australian dialogue based on Part A o f the analytical framework had taken place, a summary o f the Australian responses was sent to the Thai hydrologist who had assumed practical responsibility for co-ordinating the Thai team. In an e-mail communication to me, he raised several issues regarding the summary, including apparent ambiguity about the scope o f the project. Highlighting a potential bias o f the Australian perspective, he also noted the absence o f population pressures from the Australian list o f potential changes in the decision problem over time:

“A colleague from RFD29 and I recently discussed this issue. Both of us agree that the key to problem changes is the population pressure. With limited land and water resource [sic], social and economic structures will change over time trying to accommodate more people. In that process people will develop different perceptions, values (or whatsoever [sic] appropriate terms) for their resources. The good news is that population growth is very quantitative and highly predictable. The bad news is that how people react to new constraint is unpredictable... Well, I guess that [Australian] group members grew up in countries where there are very few people and vast unoccupied land”.