We chose not to use all the evaluation questions set out earlier as a way to structure this report as it would make it complex and difficult to read. For the sake of order, here we briefly set out to answer those questions.
B.1 Formative questions
B.1.1 Create and provide strategic intelligence on research and innovation to stakeholders in the National Research and Innovation System (NRIS), including itself
RCN maintains knowledge bases about its own funding activities and beneficiaries, monitoring and indicator reports and contributes to the national availability of a level of information abut the research and innovation system that is of a high standard by international comparison. It is closely linked to the European Union programmes and other sources of intelligence about international research and innovation activity and policy such as TAFTIE. Its advice to government is used in the development of research White Papers, whose priorities are in turn reflected in RCN’s own budget proposals and strategies. Its ‘meeting places’ provide it with needed information but offer participants limited influence over RCN activities. Its evaluation strategy is, however, not well developed though evaluation results are used in process and design improvement. Evaluation and performance indicators do not focus sufficiently on impacts.
B.1.2 Operate effective organisation structures and processes within a national division of labour among government authorities
RCNs budget dialogues, the use of multi-ministry programmes and support to ministry strategy processes makes RCN both responsive to sector ministry demands and gives it influence over them. Internal structures work effectively, even if they are complex in order to cope with the high coordination load, and RCN is itself able to reorganise to good effect, as in 2010. The institutional boundary is clear and rational within the division of labour with Innovation Norway and SIVA. Research funding process have been improved to reflect good international practice.
B.1.3 Implement and add value to national research and innovation priorities and policies
RCN has successfully focused funding on national priorities, to the apparent satisfaction of the ministries. Cross-sector programmes and strategies and RCN structural interventions add value to sector ministry priorities. However, impacts of RCN funding are insufficiently explored.
B.1.4 Play a developmental role in the NRIS, supporting the needs of the various component communities and institutions
RCN funded Norwegian research is of higher quality than that funded from other sources, based on bibliometric indicators. The performance-based research funding system for the institutes appears also to have increased their output of scientific papers. RCN has funded a growing volume of research, suggesting that capacity is being built. There is an implicit division of labour between the performance-based core funding systems (which drive quality and output within research performers’ existing fields of activity) and RCN thematic funding, which additionally steers the direction of research. The distinction with ‘free’ research projects is not clearly articulated. However, the higher performance of RCN-funded researchers suggests RCN may still have a more significant effect on quality (though that appearance could usefully be explored in more detail). RCN instruments address both systemic and market failures.
B.1.5 Embed Norway in the changing international Research and Innovation System
Participation in the Framework Programme, certain other multilateral programmes and bilateral agreements all embed Norwegian researchers in the international science system. The high average quality of joint publications with foreign authors suggests that Norwegian researchers are able to work with successful people abroad. Inward and outward mobility remain at low levels, however. The internationalisation strategy needs clearer goals.
B.2 Summative goals
B.2.1 Goal 1: Increase quality, capacity and relevance in Norwegian research nationwide
1.1 Strengthened research in nationally prioritised areas. RCN spending in the priority areas has increased. There is at least one RCN centre (SFF, SFI or FME) associated with each of the national priorities. The priorities do not map easily onto the discipline categories used in the bibliometric study for this evaluation. In most fields, impact factors for RCN-funded researchers are higher than for non- funded ones. However, in the following disciplines relevant to the priorities the reverse is true: basic medical sciences; biomedical sciences; Computer sciences; electrical engineering and telecommunications. In economics and business, RCN- funded researchers do better but their impact is low. Much more detailed analysis would be needed to understand whether these apparent mismatches are of importance. Research performers say RCN funding has strengthened their strategies.
1.2 Strengthened breadth of long term, fundamental research and concentration of
resources on the best research. Structural instruments (centres, infrastructure),
increased project sizes and a modest increase in FRIPRO have supported this goal. The SFFs attract further resources, so there is some evidence of concentration. At the aggregate level, the impact of Norwegian research is not rising.
1.3 Strengthened breadth of research-based innovation and concentration of
resources on the best research and innovation milieux. RCN spending has
increased its focus on innovation and industry-linkage instruments. Centres of excellence and large programmes exist in priority areas. The SFIs attract further resources, so there is some evidence of concentration. Some innovation-relevant disciplines have poor impact performance (see 1.1, above).
1.4 More research in business, both in terms of breadth and excellence. CIS data show declining Norwegian innovation performance. BERD has risen some 15% in real terms between 2001 and 2010. While Skattefunn began in 2002, BERD stayed flat at around $1.8 bn72 between 2001 and 2005 then rose sharply to a new plateau
of about ¢2 bn from 2007 to 2010.
1.5 Strengthened research to serve the knowledge needs of the sectors and the
administration. The amount and proportion of ministries research money spent
through RCN has been rising. RCN maintains programmes that the ministries fund. Ministry feedback about RCN performance is positive. Centres have been created in areas where ministry and national priorities overlap and that are clearly relevant to KD, NHD, OED, FKD, LD, HOD, BFD and UD.
1.6 Increased returns from international research cooperation. RCN has supported
national participation in the Framework Programme. Norway has been involved in 56 ERA-NETs, ESFRI and is very present in ERA instruments, including all 10 JPIs. It also participates in multi-lateral arrangements such as CERN and NordForsk. The rate of return from the Framework Programmes is comparatively
low but this is mostly due to the contribution being high because of Norway’s high GDP.
B.2.2 Goal 2: Good use of resources and appropriate division of labour, cooperation and structure in the research system
2.1 A good relationship between the national research effort and international cooperation. A growing proportion of RCN projects have foreign participants.
There are high levels of international co-publication among RCN-funded researchers, often with N America and countries outside as well as inside the EU. Researchers value international links. RCN has an internationalisation strategy of mainstreaming; current programmes are developing specific strategies. On RCN’s analysis (Annual Report, 2011) there is good correspondence between FP7 and national priorities.
2.2 Value added through efficient use of research funds. Funding processes have been
improved, standardised and codified to a greater extent than before and resemble good international practice. Larger project sizes reduce administration costs. Programmes covering multiple sector interests have increased in scale and number. Some measures are in place to promote gender equality but the proportion of projects led by women remains low (20%).
2.3 Dynamic, efficient and effective cooperation and division of labour in the research system. Centre programmes have tended to increase effective research
group sizes and influence strategy. Research performers say this is leading to increased thematic specialisation. RCN funds research infrastructure and channels core funds to the institutes in ways intended to improve cooperation and division of labour. Large programmes are intended to focus effort.
2.4 Assuming strategic responsibility for the research institutes. RCN monitors and
reports on the institutes individually and by sector. It operates the performance- based core funding system and the strategic projects intended for capacity building in the sector. It has largely stopped evaluating individual institutes and does not play a role in developing the strategy of individual institutes or groups of institutes. There is a slow process of restructuring in the sector, in which RCN plays little role. The institute sector is well regarded and continues to grow.
B.2.3 Goal 3: Research results are used by business, society and administration throughout the country
3.1 Good and appropriate communication of research to society and good learning arenas among companies, institutes and the higher education sector, the health sector and the administration. The primary channel for communicating research
results is scientific publications but RCN also maintains a considerable communications activity both centrally and decentralised in the programmes. There are dedicated activities aimed at children of various ages. RCN’s web site is a major communications channel, where the main audience is researchers. The small role of impact evaluation means RCN under-exploits opportunities to demonstrate the value of research and its own role.
3.2 Increased commercialisation of research results. RCN maintains indicators of
actual and potential take-up of results from the research it funds. However, these say little about socio-economic impact. The FORNY evaluation suggests that activity’s direct economic impacts have been modest but the growth of technology transfer companies associated with the research performing sector means it has probably contributed less directly to a growing commercialisation activity.
3.3 A strengthened knowledge base in RCN’s research policy and advisory work.
RCN plays an important role in assembling and publishing a rich body of strategic intelligence and advice to government about research and innovation policy. It has recently launched a research programme in the area, after a gap of some years. Evaluation is being strengthened but still needs further to be integrated into RCN processes.