Stéphane Michel Bellon1, Ulla Sonne Bertelsen23839
Key words: ERA-Net, organic food & farming, research, transnational, wordcloud, added value.
Abstract
An original and comprehensive approach was used in the ERA-Net CORE Organic (CO) to assess the added values of 14 transnational funded research projects. Such added values were identified by project leaders, and analysed with three complementary methods (frequency and comprehensive analyses, confrontation with call topics). The results show that these views from inside the projects enrich both classical scientific merits and impact assessment. They also serve capacity building in CO as a whole, and can improve cooperation in organic research and stakeholders communities.
Acknowledgments
The CORE Organic II ERA-Net was funded by the European Union's FP7programme, under the grant agreement number 249667.
Introduction
With the internationalization of organic food & farming (OF&F), cooperation among countries is increasingly necessary. At EU level, the European Commission (EC) provides a scheme for collaborative research. Since 2002 more than 100 ERA-NETs have been established to contribute to the strategy of a European Research Area (ERA). As other ERA-Nets, CORE Organic (CO) joins a number of European countries (n=11 to 21 between 2003 and 2016) in a collaboration supported by the EC. The main activity is to raise national funds and launch one or several calls, based on shared research and development priorities (Bellon et al., 2011). This paper addresses the benefits of the transnational collaboration, analysing the added values of transnational projects supported by CO. It contributes to the Organic 3.0 perspective in terms of R&D and creation of “a culture of innovation”, namely in evaluating our achievements and assessing our impacts in terms of answering to the needs of the organic sector.
Material and methods
An excerpt of 14 project final reports from the latest FP7 COII (2010-2013; see http://www.coreorganic2.org/) was used to get feedbacks from project leaders and participants. It consisted in the answers to the following specific question: “describe the main advantages of the transnational research cooperation compared to a national research project approach in regard to the subject of the project …” (max 1 page). Due to the limited size of the corpus, a manual treatment was feasible, extracting key words and sentences, and organizing them. Frequency of answers was also considered (using the “wordcloud” application, while standardizing terms such as Europe and EU), although emphasis was laid upon new ideas or suggestions, with a view to identify a wide set of added values. Research seminars organized in 2011, 2013 and 2016 enabled us to complete or discuss these added values. In addition, a literature survey enabled identifying how the issue is being addressed by the EC and in specific journals, based on an initial query [TS: “international collaboration" AND (assessment OR evaluation OR impact OR added value) AND Europe].
138
INRA, Ecodevelopment Unit, 228 route de l'Aérodrome, CS 40509, Domaine Saint Paul - Site Agroparc, 84914 AVIGNON Cedex 9, France. Email : [email protected]
139
Scientific Track “Innovative Research for Organic Agriculture 3.0” 19th Organic World Congress, New Delhi, India, November 9-11, 2017 Organized by ISOFAR, NCOF and TIPI
Results
When considering all the content of the texts from the 14 project reports, the word frequency of occurrence can be distributed in three main categories, as follows:
Table 1: Major terms used in COII project final reports
Very high (>20) High (10-20) Medium (6-10)
project(s) country(ies) researcher(s) different data knowledge partners across new transnational(ly) cooperation experiments analysis results value production possible among added established common Whereas the first column (3 most used terms) is contextual and descriptive, the second and third ones are more informative about added value in knowledge generation. This is enabled by differences among national situations and partners associated in the various projects. Importance is given to identifying common protocols, mostly based on experiments, and to cooperating in analyses. Additional words used with a lower frequency (62 words with 3-5 occurrences, data not shown) confirm this importance, with terms such as collaboration, interactions, shared, joint, network, etc. Other terms with the lowest frequency (78 words with less than 3 occurrences) are related with projects outcomes, using active verbs (e.g. gained, integrated, involved, obtain, perform) and nouns (e.g. relevant, synergies, together).
A comprehensive analysis based on full texts on “added values” complements the previous section. The answers given by coordinators in project reports vary. Their length ranges from a couple of sentences to a whole page. Their content can be organised in 3 main categories: (i) project approach and knowledge generation, (ii) methodological dimensions, (iii) outcomes and prospects. Excerpts of the answers included in the final reports were organized according to these categories and anonymised, using a code for each project (between brackets in this section).
(i) “The project enabled assembling scattered knowledge and disseminate synthesis through various channels”, “A critical mass in various research disciplines was reached, in a transdisciplinary network”, “The project enabled working at various levels of organisation” (TO); “The transnational cooperation is essential to this project, not only in respect of the input of case studies that constitute a unique basis for the analysis and insight but also by contribution with different analytical perspectives and contextual insight” (HG). As a whole, complementarities among teams and valuation of differences among countries are stressed in project reports. The network created within consortia enable a critical mass of expertise on a common topic. Differences among national situations are turned into complementarities for interdisciplinary and multi-level approaches. In the research process, both the contribution of stakeholders and the learning potential for students rarely appear as transnational added-values.
(ii) “Field experiments of the same type in the different countries have not always showed the same results. This can be explained by differences in soil and climate situations among regions. Therefore it was important to achieve experiments from more countries to be able to understand which basic mechanisms are involved in the interactions we try to manipulate” (SM).” A common laboratory protocol for sampling and analyses was established to ensure comparability of results between partners. The optimal sampling scheme was agreed” (SO). Special attention is usually given to elaborating knowledge synthesis and defining common methodologies during the initial phase of the projects. A wider range of variation can also be explored, due to the diversity of national
Rahmann et al.(2017) Proceedings of the Scientific Track
“Innovative Research for Organic Agriculture 3.0”,
Organic World Congress 2017 in New Delhi, India, November 9-11, 2017
680
trajectories and situations. This provides a more complete experimental setting or data base, and possibly more robust conclusions and a wider pallet of solutions to solve the issues addressed. (iii)“The common discussions and the experiences shared are promoting the build-up of a common expertise about living mulch as potential tool to manage organic cropping systems will contribute to spread within the European organic stakeholder communities a consistent knowledge on targeted crops” (IV). “Strong synergies have been observed between partners, which is evident from several joint publications. In addition, the networks and collaborations established during the project period have been and will be continued and further expanded in new projects” (AF). Apart from intrinsic merit of scientific results, based on peer review (sometimes after the end of the project reporting period), this section of reports often consider the project trajectory i.e. how activities undertaken in the consortium can continue. Capacity building within consortia and extension potential at EU level were also mentioned.
In order to identify possible specificities among the answers, we differentiated them according to the initial topics of the call. We only considered the first call, to have enough data. It included three thematic research areas:
(1) Designing robust and productive cropping systems at various levels (6 projects) (2) Robust and competitive production systems for monogastrics (3 projects) (3) Ensuring quality and safety of organic food along the whole chain (2 projects).
In overall pictures provided by “wordcloud” for each topic, terms with higher frequency (Table 1) are present for all three topics. Since methods used differ among topics, “experiments” dominate in topic 1, whereas “data” are more important in topics 2 and 3. Specific terms related to the general orientation of each topic are not dominant (e.g. crops for topic 1; monogastrics for topic 2; food for topic 3). Interestingly, other key-words of the call topics do not appear explicitly in the texts. This may be due to the fact that project leaders consider that this dimension is addressed when presenting results in final reports, and not in the section related with added-values.
Discussion
In the reviewed scientific literature, the term “added-value” of research projects is scarce, except at national level (e.g. in Norway and Finland). Based on our query, terms such as research impact and outputs prevail in academic journals. One issue sometimes addressed is the balance between scientific outputs and broader societal impacts of the projects, and how this balance can be assessed (Holbrook and Frodeman, 2011). In this ERA-Net, we are trying to increase the added values by 1) including sector experts focussing on the needs of the transnational sector in the first step of evaluation of the proposals, 2) having explicit procedures to facilitate the selection of proposals within the consortium and 3) supporting dissemination of results also in countries not taking part in the CO project.
As a whole, financial evaluation remains a more critical assessment criterion for scientists. Funding and its rules differ among countries, and funding bodies have an important role in this accountability at all the steps of a project, from its selection to the evaluation of reports. Since many projects refer to previous and future projects, it would also be interesting to consider larger time spans (over 3 years projects) and identify pathways of projects (Gaunand et al., 2015). Other
authors also reviewed and proposed methodologies for evaluating the relative success of collaborative research projects. For instance, Bartlett (2016) suggests integrating two dimensions: achievements of the intended research activities, and evidence of the apparent impacts. Each dimension includes four criteria, involving the evaluators assessing project reports to assign scores to each criterion. Such proposals could be considered by funding bodies to evaluate project reports. However, economic and social or policy outcomes (2 of the impact criteria suggested by Bartlett,
Scientific Track “Innovative Research for Organic Agriculture 3.0” 19th Organic World Congress, New Delhi, India, November 9-11, 2017 Organized by ISOFAR, NCOF and TIPI
2016) can’t be assessed with the current reporting framework in CO, and possibly within the duration of a project.
Interestingly, a report from European Commission explicitly refers to “added-values” (Vullings et al., 2014). The authors identify 5 criteria for added value: networking; facilitating European
excellence and capacity building; coordinating a critical mass; fostering mutual learning and harmonisation; avoiding redundancies and improving efficiency. The first criterion is not easily accessible, because selection criteria in designing a consortium are rarely explicit. Whereas a minimum number of 3 participating countries was suggested in this call, one project involved 42 partners from 20 countries. The other 4 suggested criteria are globally fulfilled, based on the overall results presented in the previous section, although the relative weight given to each of them differs among projects. Our results will also be used in the newly started CO project (CO Cofund, 2016- 2021), where attention will be given to project management at all steps, from the evaluation of submitted proposals to the enhancement of the innovation potential of selected proposals.
Suggestions for future challenges of transnational organic research
• Opening to other countries, within and outside Europe, would enhance our experience and stimulate exchanges with similar initiatives,
• Create a dynamic mapping of collaborations in projects, combining topicality (why and how a topic makes sense or not within a given country?) and scientometrics (which institutions collaborate? on which topics?),
• Amplify the role of social sciences in projects and their evaluation, as suggested during a CO
Research Seminar held in October 2016.
Conclusions
Two main conclusions can be drawn from this experience. First, it stresses the importance of addressing added-values in the reporting, and making them integral part of impact assessment of selected research projects. Secondly, it shows a community and capacity building in the pathway and achievements of this ERA-Net, which can be considered as an opened learning system.
References
Bartlett AG, 2016. Evaluating relative success of donor-funded collaborative research projects. Research Evaluation, 1-16.
Bellon S, Alföldi T, Anil S, van der Meulen S, 2011. Identification of research priorities in transnational EU projects in Organic Food & Farming. Ifoam-Isofar Korean Organic World Congress [kowc], September 2011: 342-345.
Dax T, 2014. Shaping rural development research in Europe: acknowledging the interrelationships between agriculture, regional and ecological development. An enhanced research strategy supported by the ERA-NET RURAGRI. Studies in Agricultural Economics 116 : 59-66.
Gaunand A, Hocdéc,A, Lemariéa S, Matta M, TurckheimdE.de, 2015. How does public agricultural research impact society? A characterization of various patterns. Research Policy 44, 849–861.
Holbrook J.B., Frodeman R., 2011. Peer review and the ex-ante assessment of societal impacts. Research Evaluation 20 (3): 239-246.
Vullings W. et al., 2014. European Added Value of EU Science, Technology and Innovation actions and EU- Member State Partnership in international cooperation. EC, DG for R& I.
Rahmann et al.(2017) Proceedings of the Scientific Track
“Innovative Research for Organic Agriculture 3.0”,
Organic World Congress 2017 in New Delhi, India, November 9-11, 2017
682