Analytical framework: Anticipating and interrogating bias

The preceding review has established a theoretical basis to underpin the reorientation of DSS development as collective, collaborative learning via discursive interrogation of biases. However, with the exception of Anghen and Jelassi (1994), whose

approach was inadequate for exploring developers’ biases, the literature has not yielded analytical advice. Procedures are required which assist developers or users committed to or potentially affected by a particular DSS application to explore potential and existing biases associated with the DSS. To this end, Table 4.1

proposes a generic analytical framework, based on the taxonomy of bias presented in Chapter 3, which is intended to structure an interrogation of potential and existing biases associated with an environmental DSS.

The analytical framework is divided into three sections. Part A is a background

section which situates the analysis within a specific decision-making environment. Question A1 queries the motivations for development of the DSS, in order to explore interests and commitments. By asking participants to detail the decision problem the DSS is being developed to support, A2 prompts broad problem-framing. A3

emphasises the evolving, dynamic nature of the decision-making environment, and is designed to inform subsequent reflection on embedded biases introduced through time. A4 explores the stakeholders who have an interest in the decision problem for which the DSS is being developed. A5 examines the policymaking processes which are deemed relevant to the decision problem, with a view to exploring how the DSS may alter stakeholders access to the policy process. A6 and A7 identify who the developers consider potential users’ of the DSS, and the benefits they are expected to gain through use of the DSS, pointing to assumptions about the purpose of the DSS. A8 directs attention to the types of information which the developers consider necessary to be incorporated within the DSS, signalling problem-framing

assumptions. To delineate the perceived functional bounds of the DSS, if any, A9 examines whether the DSS will be introduced in conjunction with other types of policy mechanisms, such as non-computer-based operational policy frameworks. A10 inquires of the criteria for success of the DSS, prompting identification of further assumptions about the purpose of the DSS.

Part B is an interrogation of embedded bias. B1 deals with who the DSS developers are, and their varying roles in the development process. Beyond identifying the developers, B1 is intended to prompt reflection on the partiality of participation, and the different degrees of participation possible. Those stakeholders who have an interest in the decision problem, but who will not be involved in development of the DSS, are explicitly identified in B2. Thus, perspectives are located which will not be directly involved in development, pointing to potential embedded biases. B3 continues this line of reflection, exploring how these non­ participating stakeholders’ views, as well as those of participating stakeholders, are intended to be incorporated within the DSS. In particular, this may inform reflection on absences and distortions of representation. Question B4 examines problem framing in greater detail, and is intended to prompt developers to consider assumptions underpinning their problem framing and alternate perspectives. By exploring the capacity of the DSS to be updated or reconstructed, B5 focuses on the potential for management of biases such as absences or distortions in knowledge which may be introduced by the evolving decision-making environment. Pointing to developers’ commitments to a particular system, system components or the logics underpinning the system, B6 identifies the hardware, software, models and datasets which are to be incorporated within the DSS. To catalyse reflection on assumptions made about relationships and interactions within the decision-making environment, B6 also examines the integration of components. To signal potential assumptions regarding relevance, and potential absences or distortions in knowledge, B7 focuses on those systems or components which may have been rejected by developers for inclusion in the DSS. B8 inquires about the uncertainties and assumptions which the developers are aware of in relation to the either the system components or underlying processes, suggesting potential biases due to distortions in knowledge. To explore potential absences or distortions in knowledge introduced as a result of the interface, in particular those due to the user-friendliness dilemma, B9 and B 10 query the mode of presentation of the DSS output and associated uncertainties and assumptions.

Part C is an interrogation of biased access. To inform potential geographic biases in access, Cl queries the geographical structure of the DSS, for example, whether the system is intended to be centrally located or networked. C2 then compares the geographical structure of the DSS with the locational constraints of potential users. To inform potential financial biases in access, C3 queries the costs of

developing/purchasing, establishing and maintaining the DSS, and who is likely to bear the costs. C4 then compares the financial costs of the DSS with the financial resources available to different potential users. To inform potential biases due to literacy, C5 and C6 explore the conventional, computer and scientific literacy which are anticipated to be necessary either to use the DSS, or to interrogate and analyse the potential for embedded biases. C7 then compares the literacy demands of the DSS with the literacy bounds of potential users. C8 aims to catalyse reflection on the interaction of the DSS with policymaking processes and institutions within the

decision-making environment, including whether the DSS is likely to supplement or supplant policymaking processes, or whether the DSS may serve to mediate either the empowerment or the disempowerment of an institution or stakeholder group. Given the existing policymaking processes, and the potential geographic, financial, literacy and political biases in access, C9 and C 10 explore whether the use of the DSS is likely to improve or worsen the access of any of the stakeholders in environmental decision-making.

For any application, it is likely that certain questions proposed may be deemed irrelevant or inadequate. Thus, to ensure relevance and practical feasibility, the generic framework (including the wording of questions and the methods of application) should be modified to suit each specific case. The framework is not intended as a prescriptive methodology, but rather as a flexible yet detailed guide to stimulate critical reflection on and discussion of biases. If used early, the framework may facilitate the anticipation of biases, thus allowing the management of potential biases before they emerge as problems. It should be emphasised that the framework is intended as an iterative heuristic aid, with developers and users revisiting questions periodically throughout the development and use process. If reapplied throughout the design, development and implementation of the DSS, the reflective responses may provide a (partial) history of the rationale underlying design, development or implementation decisions, thus facilitating a more transparent process. This may prove useful where a participatory approach is limited, allowing scrutiny after the fact of the underlying logic and assumptions. It may also prove useful to guard against loss of critical history if key participants in development decisions leave the DSS development community.

4.9 Conclusions

This chapter has outlined the conceptual underpinnings of a new approach to DSS development, which emphasises a precautionary treatment of development biases. The anticipation and management of biases is cast as an iterative process of critical reflection on development practice, the DSS, and the decision-making environment. Through this process, individual and collaborative learning emerge, as the DSS, DSS development practice, and the identities of DSS development participants are

reproduced and transformed. To catalyse and guide reflexive interrogation of biases, a heuristic analytical framework has been proposed. In order to ground and further develop both the conceptual and analytical frameworks, in subsequent chapters, the frameworks will be used to guide reflexive DSS development in the IWRAM project, which is developing a DSS to assist integrated environmental management in the highlands of Northern Thailand. In Chapter 5, the frameworks will be used in conjunction with reviews of the highland environmental history and the political culture of highland decision-making to foreshadow potential biases which may arise if DSS is introduced into the highland decision-making environment. Chapter 6 presents a narrative of how I used the proposed frameworks to catalyse researchers participating in the IWRAM project to interrogate potential biases and negotiate convergent framings of decision support.

Table 4-1 Framework for anticipating bias in DSS

A: Background to the application context