Developing the interview guide

I developed the interview guide for the South African case using the Australian case study as a template. This was because I was addressing the same research questions in the two cases and I wanted to be able to compare and contrast the results, at least qualitatively. Prior to interviews I discussed the questions with the leader of the NFEPA project and several changes and additions were made to clarify questions and take into account the different context. One of the key issues identified was the potential difficulty interviewees might have recalling events from at least five years prior to the interview. To help overcome this we included some retrieval cues (Singleton & Straits, 2010) e.g. mentioning specific workshops and where they were held (see Appendix C).

4.3.2 Conducting the interviews

Human Ethics research approval for the interviews (protocol 2016/077) was granted by the Chair of the Science and Medical DERC, Australian National University on 15/04/2016.

The NFEPA project leader provided assistance in the recruitment of interviewees by making initial contact with a subset of 28 of the over 200 people who had participated in the project. These 28 were selected by the project leader based on: their continued employment in the field; a substantial degree of engagement in the project (in governance, management or knowledge co-production); maximizing geographic representation across provinces; and to try and achieve a good mix of people from both the water and environment sectors. In the end there were fewer people contacted from the water sector (Table 10) because fewer were still in the sector as a result of high staff turnover (project leader pers comm.). Some provinces also had minimal engagement with the project through lack of capacity, which was a key reason why participants from only five out of nine provinces were interviewed. This targeted approach to recruitment was adopted because a high proportion of the original participants were no longer in the sector or working for the same institution.

96

Table 10: Breakdown of NFEPA interviewees by jurisdiction, sector and mode of interview. Sector Jurisdiction (# provinces) Number of interviewees Face-to- face phone Public/water National 4 4 0 Provincial (1) 1 1 0 Public/Environment National 2 2 0 Provincial (4) 6 3 3 Private/Consultants (2) 3 1 2 Project team - 2 1 1 total 18 12 6

This process yielded eighteen interviews, two of which were members of the project team. Contacting only a selected cohort of participants rather than all means I likely missed some perspectives that I was able to capture in AdaptNRM (e.g. from those with a peripheral involvement) and may have been more likely to capture positive rather than negative perspectives (although criticisms were certainly raised by some interviewees). However, as my purpose is to develop a picture of the values associated with engagement rather than testing for quantitative differences between cases this should not invalidate my results.

Recording and transcription of interviews follows the method described in Chapter 3. More face-to-face interviews occurred than in my first case because the project leader judged that as an outsider I needed to offer face-to-face interviews to encourage participation. As discussed in relation to AdaptNRM (section 3.3.2), the non-threatening nature of the questions and having only one interviewer may ameliorate differences between interview modes. Phone interviews for NFEPA were 14% shorter on average than face-to-face. Even if this means less information was gathered from phone interviewees (possible, but not certain given face-to-face likely included more small talk), this does not invalidate results given I was not quantitatively testing for differences between cases, provinces, etc. It is difficult to ascertain what effect, if any, being considered an “outsider” had on either recruitment or responses. It possibly had a negative impact on recruitment (no obligation to volunteer) but not on candor—I didn’t get the impression people were avoiding making negative remarks about the project.

97 4.3.3 Analysing interviews

The same approach was used to analyse interview transcripts as with the AdaptNRM case (see: 3.3.3 Analysing interviews) including the process of coding and validation of coding. The length of interviews (non project team) was 48 min on average and ranged from 35 minutes to just over 1 hour. The coding frame was modified from the CSIRO case (see Appendix D) both prior to and after the process of validation. After transcribing interviews it became clear that while the overall themes were similar to those coming from AdaptNRM, there were different emphases which meant differences in categories. For example, comments about outcomes were more detailed for NFEPA and so split into personal, institutional and societal outcomes (Appendix D). In contrast, NFEPA interviewees, didn’t talk about the timing of project deliverables or lack of time to engage (although the latter was mentioned occasionally).

Perusal of the coding frame also gives some indication about the impact of conducting interviews 5 years after project completion. While there was a nearly identical number of categories from the two cases, NFEPA interviewees made richer observations about outcomes compared with AdaptNRM interviewees who had more to say about the mechanics of the AdaptNRM project and broader program. This may indicate that some detail of specific events was forgotten by NFEPA interviewees who nevertheless could reflect on broader impacts of the project with the benefit of time since completion.

4.4 Results

In this section I will analyse the results from the eighteen interviews. I start with insights that came from the interviews about the political and social context in which the project played out as the impression from interviewees was these had a critical bearing on the outcomes of the project. As with the AdaptNRM case, I then present results as they pertain to the key elements of the evaluation framework presented in Chapter 2.