Summary of implications

The case empirics and analysis raise various points related to development impact evaluation processes, data/knowledge construction, and the transparency of power and practice in evaluation models and results. Three implications are important.

The first implication is whether the CHAT analysis of impact contradictions and temporally diffused power dynamics in funder‐NGO partnerships is unique to the case, or if the analysis resonates with broader development 2.0 processes. For example, does ICT2.0 deliver progress that is increasingly responsive to poor people's demands, as Heeks (2008: 33) states? And are data‐intensive technologies “on balance” both relevant and beneficial to developing nations (Walsham & Sahay, 2006: 7)? Answering such questions implicates the specific data/knowledge supply and demand chains in development 2.0-related initiatives, between diverse kinds of organisations (e.g. investors, governments, IT vendors, researchers and consultants), and diverse kinds of data/knowledge intensive development processes (e.g. planning, policy development, e‐development, data analytics/big data, and new innovations, such as blockchain for development 27_{. The take-away problem from the Rural India analysis} is: how damaging are the unequal power dynamics generated by data/knowledge intensity? This implication points beyond the need for broad approaches and analysis (Brigham & Hayes, 2013: 127) to understand these shifts and success demonstrations. The case elevates: a need for further thinking on power and practice in data/knowledge intensive processes; a need to make the diverse organisations and processes involved more visible; and the diffused relations that generate power inequalities and push evaluation actors to ignore practice. CHAT offers conceptual tools for understanding these relations, via activity systems, contradictions, and temporal chains. However, CHAT has not always interrogated diffused networks of agents and activities beyond a small number of localised actors.

The second implication is whether and how approaches to social practice and power such as CHAT might lead to ideas, tools or methods that can be pragmatic, palatable, and useful for adoption and adaptation by development agents. In the case, the partners were open to advice on improving their annual evaluations. CHAT delivered an analysis of activity systems and temporal chains that illustrated distributed data, knowledge, and power dynamics. However, the partners found it difficult to reflect or revisit qualitative evaluation. The managers did not acknowledge problems regarding unequal power relations in their professional work, bifurcating data, packaging or pitching impact knowledge. Business needs, expert terminology, statistical metrics, evaluation concepts (e.g. before/after profiles), and digital tools supported

and were elements prioritised by the managers for their evaluation machine. Reflection on power and practice was challenging and risky for them.

If sector demands, TIEK, and the DIKW legacy combine to normalise and silence power and practice, then how can more sensitive approaches be re‐embedded into impact evaluation and development 2.0 more broadly? In development literature, Gardner & Lewis (2015: 180–181) implore anthropologists and aid agencies to collectively engage and critique together. Guijt (2015: 207) asks development practitioners to acknowledge the messy politics of their work or find “space and time for more appropriate protocols and methods.” In reviewing practice theories, Nicolini (2012: 240–241) concludes that testing practice‐based toolkits in fieldwork is required. Engeström, Virkkunen, Helle, Pihlaja, & Poikela (1996) and others (e.g. Virkkunen & Newnham, 2013: 24–25) facilitate CHAT change laboratories, bringing stakeholders together to explore potential changes. These are some ways forward.

However, in this case, as in global development generally, stakeholders are geographically dispersed and inhabit different positions across data/knowledge supply chains. Participatory forms of ICT4D, KM4D, and evaluation emphasise local communities and local knowledges as a participatory response to knowing more about the local context. Yet being sensitive to power/data/knowledge relations means acknowledging wider activity chains across development 2.0, not just targeting locales of data capture, but also locales of governing, design, editing, packaging and pitching, and the points of submergence or elevation. Thus, the case has no clear antidote to the diffused generation of power inequalities in evaluation, but it does suggest ideas, methods and tools that can be adopted or adapted for use across multiple sites and professional concerns in and around evaluation. Such a multisite response can amplify the problem of silent power and practice more tangibly across more agencies than a single-site intervention to boost scientific or participatory results at the aid-target beneficiary locale alone. The third case implication concerns the prescriptive knowledge and expert models targeting development and evaluation professionals, ICT4D and KM4D researchers, and those undertaking development 2.0 data/knowledge intensive work. This implication relates to how the data/knowledge intensive discourse and network is built, constituted, made mobile and made unequal. This involves impact evaluation models and prescriptions, data/knowledge management methods, e-development information systems, and all the sector demands for data/knowledge products which satisfy targets, populate information systems, mitigate risks, and communicate success etc. These prescriptions, products, information systems and

networks have temporally sequenced activities that generate power relations, submerge voices, and elevate marketing narratives. They dismiss clamour, generate cells, spreadsheets and profiles, and bolster the confidence to know what is really happening at target sites and in target communities in faraway places. IS and KM models do not track power or practice in these relations, nor the submerging and elevating of interests and experiences. However, approaches to social practice, such as CHAT, can and do elevate such diffused issues of practice and power. This adds to arguments that suggest researchers and practitioners need not ignore power in mundane data/knowledge work, managerial controls (e.g. Bernardi & De Chiara, 2011: 37– 38), or local knowledge generation (Walsham & Sahay, 2006: 11; Thompson, 2002). However, the danger in this case and more broadly in development 2.0 is that data/knowledge intensive processes such as impact evaluation may be captured by vested interests, bureaucratic or neoliberal market forces (Picciotto, 2015: 152). This is especially noticeable where mundane power dynamics operate expertly and silently.

This third and most problematic implication, begs the question of how to respond in ways more comprehensive than occasional reflections or doubts that can be parked and ignored by staff, professionals or managers, as evidenced in the case. Blackler (2011: 732–733) advised CHAT researchers to take heed of Hardy & Clegg's (1996) observation that power is best theorised as itself constituting “the medium of collective action”. Practitioners and researchers may benefit from Blackler's (2011: 733) advice on how they influence clients or participants in research or evaluation endeavours. Building on Blackler’s advice, this case leads to a further focus on the following four considerations when promoting sensitivity to power and practice in networked evaluation data/knowledge construction.

1. the mundane, diffused power relations in data/knowledge activity chains;

2. the submerging of doubt, uncertainty, or participation, potentially viewed as old, unclear, out‐of‐scope, unmarketable, inarticulate, illegible, or peripheral;

3. the elevating of success, expertise, technical methods, and scientific rigour, considered as professional, certain, modern, unproblematic, and virtuous; and

4. how knowledge and “raw data” always have histories, etymologies and provenance through which their demand, supply, and power dynamics can be articulated and critiqued.

Blackler’s advice and the list above are calls to acknowledge how we ourselves and our professional data/knowledge processes produce power relations and hide micro-activities and messy practice from our legitimate work spaces. The NGO director’s “sales pitch” comments are crucial in this. The managers in this case did not know how to reflect on or entertain an alternative way of evaluating that could acknowledge and respond to power inequalities generated by their own professional work.

The next chapter features an NGO which lacked expertise in doing impact evaluation, in order to see if they too elide issues of power and practice. The later discussion chapter then responds to the power/data/knowledge dilemmas raised by the two cases.