Activity systems and two contradictory impacts

Research Question 1 asked the following: how is development NGO impact data and knowledge constructed in practice? The concerns of impact evaluation data and knowledge construction in practice appear wider than those contained in TIEK or DIKW-inspired decision support. Practice involves more than methods and results, and more than inputs and outputs for presentation to decision-makers. In the case, data was only a part of the many foundations for making impact knowledge: sector demands framed required data, much data was discarded in households, expert knowledge was packaged in with data, and organisational strategy also moulded final knowledge products.

At Rural India, impact construction required motivation (e.g. sector demands, or Vijay's need to account for money spent); methods (e.g. surveys, collection of discrete data, YES/NO question techniques); assumptions about measuring, bifurcating data and pitching impact knowledge; tools and technologies (e.g. spreadsheets, survey software, tablets, desktop PCs); hiring and training of staff and volunteers; collaboration between Imagine and Rural India; authority lines; and organisational strategies. These were not incidentals or contingencies hanging off a more real evaluation model. Their assemblage constituted the practices and power inherent in a cycle of evaluation activities. Therefore, a first response from a CHAT perspective is that evaluations are constituted by many activities evident in practice. These activities, not a model or a prescription (technical or moral), manifest the evaluation process. A second CHAT response to the case findings is that there is not one, but two forms of impact involved. These two forms conflict, generating systemic contradictions. The first kind of impact is not represented in documents, but is part of farmer interactions and livelihoods in the case. It is difficult to capture, and thus unclear to the evaluators. It could at times be considered as “clamour” or “not articulate”. It is orally shared, uncertain, undocumented, and not data‐ centric. It is part of stakeholder exchanges between those relatively close to a site of supposed change, who may be directly involved with or have their livelihoods affected by local changes or interventions. A good term for this kind of impact, and in contrast with the one that follows,

is “humble impact” – personally shared, materially experienced, locally circulated, and often uncertain, partial, locally situated and negotiated by stakeholders (Blackler, 1995). This is Impact-1.

The second kind of impact in this case is impact as designed, captured, recorded, stored, analysed, packaged and pitched – impact as predominantly digital representations, or in its complete state, as “knowledge products” (Mosse, 2004a: 77) to be exchanged. These are produced instrumentally, for circulation within aid chains, markets, and bureaucracies, in response to sector demands (Wallace et al, 2006; Mosse, 2004a; Quarles van Ufford, 1988). They are produced for professionals trained in development or evaluation discourses, and require expert terminologies of different kinds. They align with development and evaluation prescriptions, and increasingly with marketing and digital strategies. In development 2.0, such digitised impact representations are made mobile, rationalised and transported globally between local context and expert contexts. They are global representations (Avgerou, 2002: 77) used for exchange with other organisations, decision-makers, and groups in formal aid markets and bureaucracies. Mobile impacts are strategically constructed and mobilised for exchange, and as in this case, often exchanged for funding revenue (Hayes & Westrup, 2014: 28). They can also attract legitimacy and status from other actors in the market and bureaucracies, promoting views of the producer as a reputable and dependable partner. This is “Impact-2”.

Figure 5.8 below shows the activity system and contradictions evident between humble impact or Impact-1, and impact marketing or Impact-2. The diagram shows how tools (e.g. data cells, spreadsheets), rules (e.g. for designing questions, writing reports), and divisions of labour (e.g. philanthropy and NGO as designers, analysts, writers, marketers, or farmer’s as sources of raw data), limit the object of knowing impact, and how they facilitate expert construction of impact marketing narratives as outcomes or products to satisfy sector demands. The activity system bifurcates impact data/knowledge, rendering humble impact (clamour, doubt, farmer views, etc) as illegible to the impact machine, and thus it and discarding it in the field. These silences are not shown, because this activity system, at Rural India and Imagine, had already become an instrumental machine for producing outcome representations for sales pitches.

In CHAT, these two impacts form a primary contradiction, based on the exchange value of representational impact to meet sector demands and secure funds or legitimacy. This contradicted the use value of humble impact in farmers' daily lives; value found in sharing,

complaining, finding solutions, building local relations - “clamour” from Chandan’s perspective. Secondary contradictions between activity elements (roles and object; labour and object; tools and object) are included in Figure 5.8.

Figure 5.8: Activity system and contradictions in the Rural India evaluation process

Again, tools (e.g. data cells, survey methods), rules and norms (data management, removing clamour), and divisions of labour (NGO and philanthropy evaluation control, farmer as raw data supply alone) all contradicted the potential object of knowing farmer-defined impact or impact one. The elements limited knowing to impact two, impact for marketing. Farmer voices, contexts, uncertainties, and participation were professionally and mundanely taken out of outcomes. Humble impact was submerged or masked (Engeström, 2008: 36–42).

It is noteworthy that such tensions were not considered a problem for the Rural India or Imagine management, because marketing impact had become an expert‐normalised process, a problem of professional efficiencies, technical aptitude, and effective results. Therefore, in response to Research Question 1 on how impact data and knowledge are constructed, the CHAT analysis suggests impact evaluation data/knowledge is constructed in an evaluation machine assembled

over years of practice between partners, during which time partners such as Rural India and Imagine can become expert at harvesting data and constructing knowledge products to support management and marketing activities. The machine and construction process are not intended to understand, amplify or mobilise humble impact from farmer lifeworlds.