Quality of knowledge

The quality of knowledge engaged with was one of the broad values identified in my draft evaluation framework for decision-maker engagement. As discussed in Chapter 2, the relevance (or salience), credibility and legitimacy of science (Cash et al., 2003) as well as its accessibility (Walsh et al., 2014) have been identified as of importance to decision-makers in

64

the science-policy gap discourse. The regional NRM decision-maker perspective on these issues is discussed below.

Quality of knowledge: relevant

The relevance of scientific knowledge can be judged by decision-makers as the degree to which it relates to the matter at hand (Heink et al., 2015). The qualifier to “the matter at hand” implies a timeliness component that matches how interviewees described seeking out science—not as a generic undirected task but when a problem arose that needed addressing (particularly in a new domain) or when a new process or project commenced. In AdaptNRM, the funders provided this imperative for NRM groups by funding a parallel process to revise and deliver updated regional NRM plans. However, deficiencies in program design meant that AdaptNRM outputs were not delivered in time to be incorporated in the regional plans in several jurisdictions. This was one of the most common criticisms of the project with seventeen out of twenty-five interviewees saying timing of knowledge delivery was a problem:

NRM01 “The only thing that I would say was a bad thing was the timing of it. Like, it would have been awesome to have all this stuff probably about five months beforehand.”

NRM19 “if we had of had it 18 months ago it potentially could have really been really useful in helping us guide our process.”

NRM03 “my only criticism of them is the timing was rubbish you know?”

In terms of engagement this meant that the time-consuming processes of consultation and deliberation were less attractive to those who knew they could not use the knowledge for its intended purpose. Regional NRM groups also tend to work from project-based funding. The funding for the planning process included salary while AdaptNRM did not pay for participants’ time. This meant that when the planning money finished some did not have the capacity to engage with AdaptNRM, as described by one interviewee:

NRM13 “we didn’t have the funding or capacity to keep supporting it or having a dedicated person working on it”

Timeliness of delivery therefore had a key impact on the relevance of knowledge delivered through AdaptNRM. For time-poor and project-funded regional NRM decision-makers, knowledge not delivered in a timely manner became irrelevant, at least in the short term.

65

Lemos and Morehouse (2005) have posited that in addition to relevance, the usefulness of knowledge or how its form and scale fits with user needs is a key variable determining the application of science. The scale of the information under discussion was an important component of decision-makers’ perception of its usefulness:

NRM04 “Its gotta be at the right scale I think and useful in answering the questions that we’re after because scientific information is of no use if it’s not answering the specific questions that you are raising in the first place.”

Concerns were expressed that information at a national level—whether it be from AdaptNRM or other projects in the program—did not provide the level of detail required for regional managers. Whether or not it was possible to develop information at a regional scale, the perception that information was too coarse to address regional priorities was certainly an issue for some.

Quality of knowledge: accessible

Science can be inaccessible to decision-makers if they can’t retrieve it. For example, five interviewees said that lack of institutional access to peer-reviewed journals was a barrier to their use of science in their institutional role. Accessibility may also relate to awareness or discovery of science that is otherwise accessible. In this way AdaptNRM increased accessibility simply by centralising relevant information and links to relevant information on the project website. Another mechanism for increasing access to science that was valued by some interviewees was the direct access to scientists enabled through AdaptNRM:

NRM07 “Probably the major benefit to this project I think is the interaction with scientists”.

The second level of knowledge accessibility relates to situations where information can be retrieved but decision-makers can’t determine its meaning or how they may apply it. I have considered this a function of communication and so it is discussed below under “Improving access to knowledge”.

Quality of knowledge: credible

For some interviewees the credibility of science was unquestioned or, in other words, science was trusted. Trust seemed to be linked to the source of science, both written and from individual experts. The peer-reviewed literature was often seen as the most credible written form as the process by which the knowledge was created (via a rigorous scientific process

66

including hypothesis-testing, experimentation, peer review, etc) was judged legitimate. For example the following are typical views on the nature of science broadly and its value:

NRM05 “So to me science means that its come out of one of our recognised tertiary institutions. It therefore has credibility. It’s generally been peer reviewed…”

NRM15 “real science I guess is you know peer reviewed, and based on scientific methodology”

Individual experts were judged on their own or their institution’s professional reputation, qualifications, relationship with the decision-maker (knowing them personally) or where they were based geographically. Locally based experts tended to be more trusted which may be linked to decision-makers knowing the expert.