Testing Part B: Embedded bias

Part B o f the framework was tested during July 1997. The group discussion (which included the participation o f the recently appointed project manager) explored how the design and development o f the DSS might be improved in the light o f the collated responses. In particular, participants were asked to reflect on whether any o f the identified problems or biases might put at risk any o f the criteria for success or advantages o f the DSS, or conversely which might increase the likelihood o f any potential disadvantages o f the DSS, which had been identified during Part A.

Framing the DSS fo r newcomers

To open the discussion, I asked the project leader to b rief the new project manager about the DSS. In his briefing, the project leader concentrated on outlining potential software options that could be customised and integrated to form the DSS, therein indicating his own technical interest and focus:

“The type of technology we want is not really available - it will need quite a bit of customising and enhancing. Several options - WHAT-IF and RAISON from Canada - doubts about RAISON because of expense, ownership problems. We’ve been looking at LUPIS - all quite keen on that - allowing us to get stakeholders to input their preferences with respect

to spatial land use options... But LUPIS is a bit separate. The main thing is what package we can come up with for displaying our spatial data and linking it up with the modelling. Nothing does that particularly well... So, its looking like we might develop our own system, cobbling together various tools”.

Note in the preceding quote how LUPIS, the software to assist conflict management in land use planning, is framed as distinct from the core DSS package which is framed as a technological system which integrates data and modelling.

Following the briefing, the project manager raised the issue of who would be likely to use the DSS. The sociocultural leader responded that she and the project leader

“probably view it quite differently - the LUPIS type route - which I’m most attracted to because it’s more equitable among users - versus the high powered end of things in what you can represent in a system and I’d be worried about having two systems - one that’s more open to stakeholders and one that’s more high-powered and techy”. Hence, within the context of DSS construed as software, divergent framings were identified,

contrasting an equitable, user-friendly approach with a more closed but, technically, more sophisticated system. This suggests recognition that literacy-based biases in access could be potentially problematic in the IWRAM project.

Who should be involved in developing ihe DSS?

Shifting to focus debate more explicitly on the framework questions, I asked the group to consider the extent and type of involvement of different stakeholders and researchers in the development of the DSS. In their questionnaires, the participants had each expressed a hope that all project researchers and stakeholders would be involved in development from early in the process, however the questions of when and how different people would be involved remained fairly open. In the group dialogue, the project leader suggested those questions could be answered at an impending project workshop. However, the sociocultural and DSS leaders cautioned that questions of when and how stakeholders or users should be involved in development required further discussion about the purpose and nature of the DSS. The sociocultural leader again raised concerns about a conflict between the participatory framework and the DSS, fearful that the DSS might discourage participation in and support for the project by not only highland villagers but also the Thai social science research team:

“DSS - either people haven’t heard o f it at all, or it carries different connotations. So on the one hand w e’re saying we want to get your aspirations, get to know you, help you, and on the other hand w e’re trying to build this thing that you construe as a very technical, foreign sort o f thing. So you are potentially engaging and blowing cooperation at the same time. I think that’s the same for the village reps - surrogates30 - and

the research team... Social scientists on the whole don’t know much about DSS and are pretty wary o f it so those sort o f project team members are likely to be wary o f the very thing they are being asked to collect data for... I can’t see any way around that except to break the principle o f consult early... and then you’ve got the risk o f a breach o f trust at that point”.

30 The term “surrogates” refers to spokespeople for highland villagers, such as an NGO representative who

Reinforcing his earlier distinction between LUPIS and the technical modelling

component o f the DSS, the project leader offered a different way o f viewing the DSS as a resolution to the apparent conflict:

“I want to separate out the DSS into two components - a technical modelling system which is one of the inputs into the decision-making process so it’s as objective as scientists and economists can make it and includes the options of the social scientists in terms of what we want to simulate... I see LUPIS, however it’s managed in a stakeholder setting, as taking the outputs of the modelling component along with all the other preferences that stakeholders have and weighing them up in some way - then I don’t have your problem of a big sophisticated tool. I see this big sophisticated tool as something to be seen as very technical and from the start pigeonholed as such”.

Note that objectivity is explicitly identified as a goal o f DSS development, and that development o f the technical modelling system is framed as primarily a task for (biophysical) scientists and economists, with the social scientists’ roles limited to provision o f information regarding simulation options. In the light o f the project leader’s earlier description o f LUPIS as separate from the main DSS, this quote reinforces his earlier framing o f the DSS as a sophisticated and technical tool. The quote also illustrates how, through dialogue, an individual may confront an alternate framing o f decision support (a participatory approach which responds to stakeholders’ land use planning needs), and attempt to reconcile it with their own primary framing. The other participants supported the project leader’s perspective; the sociocultural leader remarking that it made her feel much more comfortable about the relationship between the DSS and the participatory process. Note that this marks the point where the DSS came to be accepted by the group as involving both a technical modelling

component and a stakeholder process component, and that at this stage the latter was technology-driven through its focus on LUPIS31.

Meanwhile, the DSS leader had argued that they still needed to pin down exactly which decision-making processes the DSS was supposed to be supporting, in particular, whether the Thais intended to put in place an innovative stakeholder based decision­ making process to facilitate the flow and use o f the information stemming from the DSS, or whether the DSS was destined to be cornered by the top-down government to support the status quo interests. Thus, a distinction was introduced between a DSS to support current interests versus one aimed at future innovations. Note also the implicit recognition o f the politics surrounding the use o f technology for highland environmental decision-making.

Embedded biases

The next topic o f discussion concerned embedded biases. In their questionnaires, the participants had identified several potential forms o f embedded bias such as

uncertainties in climate surface inputs, biophysical assumptions about discharge and

31 The sociocultural leader later commented that although at this stage the stakeholder process was technology-driven, she did not see it as being the sole locus o f participation. Instead, she hoped participation could be around issues, although she was having difficulties in seeing how participation could be integrated into the project without being tool-driven.

water quality concentrations, economic assumptions about the calculation o f off-site costs, and assumptions about the availability o f data to drive the models. During the discussion, a question I posed about how consideration o f what would be left out o f the DSS might improve the project proved most useful in prompting participants to reflect on means o f managing embedded biases. The DSS leader suggested that although modellers tended not to document what they were leaving out o f a system:

“We’re talking about information leading to decisions affecting people’s livelihoods... and especially if we’re using some optimisation procedures, you need to explain to people as part of explaining to people what the limitations are and the limits which can be put on interpreting the results”. Note that in the preceding quote, the practical intent o f the DSS, and consequent potential ramifications on those affected by use o f the DSS, is recognised. Within this context, the quote suggests an ethical imperative on the part o f the researchers to communicate embedded biases.

Both the project manager and sociocultural leader suggested potential embedded biases in terms o f absence o f knowledge. The project manager noted that the DSS would only be robust within the boundaries o f the other three project components and therefore it would be difficult to capture gross externalities like the recent collapse o f the Thai baht32. The sociocultural leader noted that the content o f the DSS would be shaped not only by scoping, or selective inclusion o f those factors deemed by the researchers most important and relevant, but also by technical feasibility in that factors not amenable to computational treatment would have to be discarded on those grounds. To manage for embedded bias, the project leader suggested that a document was needed which outlined the limitations o f the DSS and how they might influence the model results.

Flexibility o f the DSS

In terms o f the potential flexibility o f the DSS to respond to changes in the decision­ making environment, the project leader argued that there would be no problems in upgrading the technical part o f the DSS; that models could be changed or recalibrated as new data became available. When asked about the flexibility o f the DSS to respond to changes in people’s visions and options, he remarked that catchment management groups were used for this task in Australia, but queried the practicality o f a similar process in Thailand. This led the sociocultural leader to recognise a tension between developing the DSS for their research purposes and developing a DSS for use:

“...its one thing to look at where we are in three years time at the end of the project - everything neat for that point in time. In order to be capable of upgrades and to keep the tool relevant, you actually need a monitoring process to be able to be collecting the data to feed in. It’s one thing if you’ve got time series hydrological data being collected anyway, its quite another if you have to fund and organise separate data collection exercises if you had to update visions. So the long term future is a real issue we didn’t address in our proposal”.

One o f her suggestions to manage for flexibility was to consider carefully at the beginning o f the project the types o f social science data that should be built into the DSS: “I know the temptation will be, having field staff available, to generate a very rich

database that will be very dependent on more labour to keep updated. The other extreme is to be much more targeted and to say that realistically the DSS is going to need 8 types o f data to continue - they are the ones that need to be collected”. Thus, a trade-off between ideal process and practicality was introduced with embedded bias implications in terms o f absence o f knowledge.

The project manager’s perspective on managing flexibility had been that “People who will be interested in upgrading the system will be the people who use it most - comes back to the original question o f who the core group o f stakeholders is who see

themselves using the technology afterwards”. This prompted the DSS leader to reiterate his earlier sentiments that more knowledge was needed o f the decision-making

environment because “how those stakeholders interact, use information and make decisions makes a huge difference to the sort o f system we construct such as how the system is upgraded and who maintains it and flexibility”. He suggested that a DSS advisory group made up o f key highland stakeholders, akin to the Australian catchment management groups mentioned earlier, needed to be put in place in parallel with the conduct o f the research project.

Biased access

The final topic o f discussion concerned biased access to the information stemming from the DSS. In their questionnaires, each participant had suggested that the output o f the DSS could be portrayed in different ways for different users. In the group discussion, I asked the sociocultural leader to try to identify some o f the characteristics o f a particular user which might influence the way in which the output would be presented to them. She raised literacy as one dimension: “there are some cultures who can’t recognise a photograph as a 2D representation o f a 3D thing. That’s only been researched in Africa, but we need to be wary that what we put on the screen is going to be what people see”. To manage this potential bias, she suggested trusted surrogates, such as NGOs,

academics, or educated village youth, may need to be included in the suite o f DSS communication media.

A second dimension raised by the sociocultural leader related to embedded bias: “There’s the question of the way they see the world. I’m very interested in building that in, so instead of the public servants and ourselves thinking there is only one way of viewing the world and going at it in a Western way thinking that when you draw the land units onto a GIS there’s only one way of seeing them... [One of the Thai social researchers] does lovely research on ethnographic land classification - how they classify land into different parcels. So steps like that would make it more user-friendly in being able to use their language and terminology - approaching their mindsets and classifications as we build it up.”

This implies that the sociocultural leader had identified the risk o f the nonreflective imposition o f a Western approach to resource management, manifested in the physical DSS through the demarcation o f particular spatial units, thereby undermining alternate constructions. She had also pointed out a potential means o f managing this bias through incorporating within the DSS different modes o f classification and description more consistent with alternate constructions.

As the discussion turned to communication of assumptions and uncertainties, the DSS leader introduced a value judgement confronting the researchers in regards to access:

“Who has to have access to what level? Its the output o f the system the stakeholders are interested in. I don’t know if w e’re giving stakeholders access to underneath the surface o f the system, where you are turning the buttons and pushing the levers. I think that will have to remain the domain o f the technocrats.... Its unresolved what extent stakeholders need to or want to actually turn levers - or will they be happy having technical intermediaries who do that and just dealing with the output which is visualised in a way which they can understand?”

In the preceding quote, note how the DSS leader initially responds to his own question by asserting that stakeholders (which are framed as distinct from a technical bureaucrat) are only interested in “the output”, implying that stakeholders would not be interested in the mechanics or reasoning underlying the output. As he continues reflecting on the issue, he becomes less sure, and shifts from focussing on the researcher’s judgement (“if

we ’re giving”) to the stakeholders’ needs and preferences.

Injecting a pragmatic voice, and emphasising a tension between ideal participation and project manageability, the project manager suggested that the issue could be displaced onto one of the Thai government departments:

“Get the DLD to be the agency to say from the beginning ‘we will take on the internal set-up, internal availability and track how its used’. And other people can also access it through the DLD. If we start off accepting all stakeholders have the same rights o f access from the beginning, won’t it go on for endless subcycles o f different needs and demands?”

The sociocultural leader suggested that it was a technical inevitability that some information would be hidden from certain users, and that the fundamental issue was therefore one of trust: “will all the stakeholders trust the guardians and managers of the box? If people have confidence in what the managers are doing and what the output is saying, that’s fine. If they distrust whoever’s managing it and think its being used to put one over them...”. The DSS leader then suggested that biased access could be managed through the proposed DSS advisory group:

“Which is why establishing some group - a stakeholder advisory committee or steering committee which would have some ongoing involvement - that would build trust in the system so that at the end when you come to a high level interaction they would have some confidence, some understanding, some opportunity to look at what was underneath that surface layer.”

Thus, a proposal to create a DSS advisory group initially arose as a possible body to