Project implementation performance

Although the relationship between organisational outputs and sustainability is one of the key concerns in the SCOPE framework, the concept of “output” lacks a precise definition (Brown, 1996). Although several authors (Gustafson, 1994; Finsterbusch, 1990; Ingle et al., 1990) who applied the SCOPE framework have used the term “implementation performance” under the broader concept of

“programme success” in establishing a relationship between organisational outputs and sustainability, the term performance was not defined. Compared to this, the IAD literature (Ostrom et al., 2002) used the concept of “outcome”. Furthermore, in the descriptive literature, concepts like project effectiveness, project benefits, project impacts, and so on, are used.

Several empirical studies (Bossart, 1990; White et al., 2005) have shown that the sustainability of donor-supported projects was related to their ability to produce convincing evidence of impacts to the stakeholders concerned. These observations suggest the importance of an organisation’s pragmatic exchange relationship with its stakeholders, as suggested in the SCOPE literature. For example, in a study of the donor-supported health projects in Central America and Africa Bossert (1990) found that poorly sustained projects were characterised by an inability to achieve the targeted objectives and goals. In addition, in such projects the goals were vague, which made it difficult to demonstrate achievement of the objectives. Similarly, an empirical study on the sustainability of Integrated Coastal Management (ICM) projects in the Philippines and Indonesia revealed a relationship between visible improvements in the local ecosystem and sustainability of the ICM project activities beyond project life (White et al., 2005).

In some other instances (Bossert, 1990; Ingle et al., 1990), however, evidence contradictory to the above, was observed. This evidence suggests that projects may sometimes be sustained even without real evidence of positive impact. In such cases, the “reputation” of the projects’ performance was found to be important. The study on the sustainability of donor-supported health projects in Central America and Africa carried out by Bossert (1990: 1019) showed that it was the projects’ “reputation for being successful” that positively affected their sustainability despite the fact that there was no objective means of determining their effectiveness. The reputation of the projects influenced the decisions of the health officials and beneficiaries concerned to support the projects regardless of whether or not the reputation was backed by ‘scientifically designed evaluation studies’.

According to AusAID (2000), the ability of a donor-supported project or programme to generate financial gains for the stakeholders is important for its sustainability. The agency argues that if a programme or project does not deliver clear and

equitable financial or economic benefits, which are apparent to the stakeholders, it is most unlikely to be sustained after donor funding finishes (AusAID, 2000). Similarly, the study by White et al., (2005) found that perceived increase in economic gains and improved economic returns for stakeholders and their income generation were important for sustaining ICM project activities in the Philippines and Indonesia.

In addition to financial gain, an organisation’s economic efficiency is also argued to be important for its sustainability. In other words, the benefit must be equal to or more than the perceived cost that stakeholders have to invest in terms of time, money and labour (AusAID, 2000). Therefore, AusAID suggests that benefits are not conducive for sustainability if the net benefit arising from a project or programme is negative or very small when all the costs are considered (AusAID, 2000:5). The study by Steckler and Goodman (1989) of 10 health promotion programmes revealed that when benefits associated with the programmes outweighed the costs, individuals became predisposed to supporting the programme’s continuance. These observations also point to the importance of pragmatic legitimacy of an organisation for its sustainability, as articulated in the SCOPE literature.

At the same time, empirical evidence produced in the development literature (Steckler & Goodman, 1989; Ingle et al., 1990; White et al., 2005), showed that organisational performance in generating visible impacts was a necessary, but not a sufficient, condition of sustainability. For example, White et al., (2005) found that although measurable gains were important for the sustainability of the Integrated Coastal Management (ICM) projects in Indonesia and the Philippines, they were not sufficient conditions for sustainability.

The literature in extension and rural development also produced evidence that an organisation’s ability to demonstrate impacts is important for its sustainability (Gustafson, 1994; Anderson & Feder, 2003; Anderson et al., 2006). For example, Gustafson (1994) found that programme success in generating visible impacts was important in building demand and prestige, which helped achieve greater sustainability of the T&V projects. However, demonstration of impacts has always been a challenge in the case of agricultural extension projects, a problem that is considered to be generic (Feder et al., 1999). This is because of the nature of

goods and services that extension organisations deal with, which in most cases are characterised by attributes of “public goods” (Anderson & Feder, 2003). Traditionally, mainly under the T&V system, extension organisations have been associated with the transfer of information and advice to farmers in order to increase agricultural production (Anderson & Feder, 2003). The impacts or values of these services are, however, very difficult to prove or quantify (Feder et al., 1999).

In descriptive case studies on recent donor-supported extension projects embodying the principles of privatization, GO-NGO collaboration and demand-led extension, it has also been recognised that demonstration of benefits to the stakeholders was important for the sustainability of these new extension approaches (Nahdy, 2004; Lightfoot, 2004; Sulaiman & Hall, 2004). At the same time, it is observed that there has been little improvement in resolving this issue because of the nature of the current reforms. For example, some of the rationales underpinning demand-led extension reforms include enhancing farmer participation, empowerment and capacity building. Pursuing these goals has made it more difficult to prove extension’s contribution (Fleischer, 2004). This is because participatory extension impacts are mainly relating to increased human resource capacity, which is not usually manifested over a short period (Fleischer, 2004). Based on experiences with two participatory extension projects in Egypt, Fleischer (2004) has recognised that participatory extension poses new challenges for methodologically sound assessment, as there is no longer a primary focus on the diffusion of externally supplied technologies, but on the creation of human and social capital, which is “difficult to measure and value” (Fleischer, 2004: 47).

Considering the shortcomings of extension organisations in producing visible impacts, some authors (Sulaiman & Hall 2004) recommend that extension should involve a wide range of activities beyond the single goal of information transfer and involve some provision of private goods. In order to support their arguments, Sulaiman and Hall (2004) have reported the cases of two privatised and partnership-based extension projects in India, which were sustained and expanded beyond the period of donor involvement over a short period. One of these projects offered extension services to clients in a wide range of areas such as development of self-help groups and master farmers, group marketing, a unique credit package, funding and facilitating technology development, and fruit and seed processing.

Reportedly, due to this wide focus, the project resulted in an increase in crop yields in the cultivated areas, improved marketing and credit facilities for farmers, and reduced production costs – all of which contributed to an increase in farmers’ income. The other project improved the access of farmers to quality inputs such as farm machinery, agricultural credit, advisory and field supervision services including buy-back, and better prices. Interviews carried out by the authors showed that the farmers were satisfied with the services offered by the project, although no quantitative evidence of impact was reported.

The importance of net benefits or economic efficiency, based on cost-benefit comparison, is also reported by extension and rural development authors (Lightfoot, 2004, Datta, 2005). Evidence for this came from the experience of a demand-driven extension reform initiative in East Africa (Lightfoot’s 2004) and from the integrated rural development projects supported by Concern Bangladesh (an NGO) that attempted to develop community-based organisations of the poor rural people in Bangladesh. The benefits of playing leadership roles in the groups was perceived by some group leaders as insufficient compared to the amount of time they needed to invest to ensure the smooth operation of the groups (Datta, 2005:5). Owing to this, the group leaders demanded ‘incentive payments’ (compensation) and when the general group members refused to provide the payments, some leaders left the groups resulting in a collapse of the community-based organisations (CBOs) (Datta, 2005).

The study by Datta (2005) also showed that fairness of the group leaders involved with the community development projects in Bangladesh was important for the sustainability of the groups. The author reported that, despite a strong perception among the group members that the community organisations were beneficial, the members of the groups that did not survive were not interested in reviving their activities. The reason was that the members lost trust in their leaders owing to the latter’s malpractices such as the mismanagement of group funds, and favouritism (Datta, 2005).