• No results found

Promoting Excellence through National Research Assessment Exercises


Academic year: 2021

Share "Promoting Excellence through National Research Assessment Exercises"


Loading.... (view fulltext now)

Full text


Promoting Excellence through National

Research Assessment Exercises

Research Planning: Embedding Analytics in a New Performance

Challenge Process at the University of Warwick, UK

Dr Giles Carden,



University of Warwick – A Recent History

Research Assessment – UK background and landscape

Leadership, Vision 2015 and Research Performance

Building the Challenge Process in a Changing Environment

Implementation of Analytics to benchmark and measure performance

Future Analytics’ Developments


University of Warwick – Historical Context

The University of Warwick opens, taking its first cohort of 450 undergraduates when it received its Royal Charter in 1965.


Established as one of the UK top ten

research intensive universities covering over 400 hectares hosting over 23,000 students and 5000 staff drawn from 120 countries across the globe.




The University establishes its Medical School and Warwick Ventures Ltd

which has helped launch 60 companies since its inception.

Warwick ranks 7th overall in the

Research Assessment Exercise



The £92m National Automotive

Innovation Campus 2013 is given the green light, jointly funded by the UK Government, Jaguar Land Rover and TATA Motors.


University of Warwick – Partnerships and Alliances

National Automotive Innovation Campus (NAIC)

Due for

completion in


£100m joint enterprise between


• TATA European Technical Centre • WMG/University of Warwick • HM Government (HEFCE)


University of Warwick – Partnerships and Alliances

Warwick – Monash alliance launched to collaborate on research, education and enhancing the experience of its students.


Strategic partnership with the Liverpool School of Tropical Medicine (LSTM)

announced to support joined up long –term research, focusing on improving quality of health and social care and influencing policy and practice.


University of Warwick is selected as CUSP’s sole European partner for the New York based initiative involving NYU, Carnegie Mellon, CUNY, Indian Institute of Technology in Bombay, University of Toronto and a number of industrial partners including IBM, Microsoft, Cisco and Siemens.


Warwick signs memoranda of understanding with Sun Yat-Sen University

strengthening its relationships with Chinese institutions and providing new study opportunities for students.


University of Warwick Strategy – Vision 2015

July 2006

Professor Nigel Thrift appointed Vice-Chancellor of University with an ambitious drive to improve

research performance launching the overarching strategy Vision 2015 in 2007.

Vision 2015 Goal 1:

To make Warwick an undisputed world leader in research and scholarship.

Our commitment to be demonstrably a centre of world class research and innovation across all of our academic disciplines remains as strong as it always has been. This means that we must maintain our focus on the quality and impact of research and ensure that we are embedding our quality expectations in our internal processes, in particular those to support and develop existing staff and in the appointment processes when we seek to recruit new talent.


UK Research Assessment – Historical Context


Thatcher government instigates the first formal

assessment of university research, conducted by the

University Grant Committee, forerunner to the current

Funding Councils.

RAE transforms in to REF and the ‘impact’ of research

beyond academia in the wider society, economy or culture

is introduced for the first time. (2008-2013)

56,089 staff submitted from 155 Higher Education

Institutions across the UK

Subsequent research assessments (RAE) are held in

1992, 1996, 2001 and 2008.


A UoA’s overarching environment narrative and a number of metrics

• Doctoral degrees


• Research income • Research


Are assessed to establish the ‘vitality and sustainability’ of the UoA.

Outputs (65%)

Impact (20%)



Individual work is assessed for its

‘originality, significance and rigour’

• Journal Articles, conference contributions • Books/Book Chapters

• Physical artefacts

• Exhibitions and Performances • Digital Artefacts

List is not exhaustive, refer to main panel criteria for detail.

Process of expert review by panellists.

However, some UoA sub-panels will be using bibliometrics such as journal impact factors and citation counts to assist in determining quality.

A UoA’s overarching impact narrative and case studies are assessed for their

‘reach and

significance’ of impacts on society, culture, the economy, commerce etc.


UK Higher Education Funding Environment

HEFCE Funding Breakdown


HEFCE – Higher Education Funding Council of England


Establishment of Research Assessment Planning Group

March 2009

Establishment of annual Research Assessment Planning Group (RAPG) process and preceded by investment in business analytics capability in 2006.

Two Pro Vice-Chancellors (PVC) cover the University’s research portfolio across 36

departments in the Arts & Social Sciences and Science, Engineering and Medicine respectively.

Group’s Constitution

Pro Vice-Chancellor (Research) - Chair Chair of the Faculty Board

Head of Department

Departmental Director of Research

Director of Strategic Planning & Analytics



Establishment of Analytics Capability

Means of Evidence-Based

Performance Challenge



Research output quality, citation performance, bibliometrics, publication strategy

Research income generation

Doctoral registrations, supervisions and awards

Strategies for the support and development of research impact

Key Foci


Publication Collaboration Analysis


Publication Collaboration Analysis

SciVal Harvard

Boston Tufts


Individual Performance


Citations – Performance

• Key driver in the University’s performance in the THE World University Rankings. It

demonstrates the impact and influence of our research and our ability to spread new ideas and knowledge.

• This chart normalises citation data by showing the number of

citations per paper. 0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00

1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

Ci ta tio ns p er P ap er

Russell Group Interquartile Range Russell Group Median Warwick

0 2 4 6 8 10 12 14

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

N um be r o f A rt ic le s Pu bl is he d in Na tu re & Sc ie nc e

Highly Cited Journals – Nature & Science

• Nature and Science are two of the

most highly cited scientific journals in the World.

• This chart shows the total number of articles the University has published in these journals each year since 1999.


Heat map – Income generation

Smith, John, Professor Brown, Tony

Professor Yates, George


Research Income – RAE 2008 vs REF 2014

291 480 0 100 200 300 400 500 600 £m

Total Research Income Submitted (not accounting for inflation)

RAE (2001-2007) REF (2008-2013)

0 50 100 150 200 Re se ar ch In co me (£m)

2001/02 - 2011/12

Russell Group Interquartile Range Russell Group Median Warwick

0 20 40 60 80 100 120 140 160 Va lu e of A w ar ds (£m)

2003/04 – 2012/13

• The University was able to submit significantly more research income to REF 2014 than RAE 2008. • The University had a very successful year in 12/13

winning over £139m in research awards, a rise of 86% on the previous year. Whilst awards do not translate immediately in to research income, this is a strong


What did we achieve ?

Improved academic staff accountability and performance

Growth in research income

Growth in research student numbers

Publications in higher impact journals, books with high reputation


Used analytics in remuneration reviews, one tool in the kit

Cultural change


Emerging Developments in Analytics

© CrispMe









(web and social media)


A form of technology that uncovers relationships and patterns in large datasets

Forward-looking, using past events to anticipate the future

Can be used to predict behaviours and events

Can identify risks and opportunities

Uses statistics, machine learning (AI) & modelling

Three principal models:


– past performance informs likelihood of future behaviour


– quantifies relationships in data to classify prospects in specific

groups and predict a single customer behaviour


- describes the relationship between all the elements of a decision,

known data (including results of predictive models), the decision, and the

forecast results of the decision


Staff retention analysis

Staff performance analysis

Applications to internet logs and better understanding HEIs web presence

Predicting highly cited papers and researchers

Analysing library usage to inform resourcing strategies


Not everything that counts can be

counted, and not everything that can be

counted counts.


If you torture the data enough, nature will

always confess.

Ronald H. Coase, Noble Prize Winning



Thank you for your










Related documents

The $VarInfo directive allows variable information (e.g., the guess value, limits, units, and display format) to be entered from the Equations window, rather than from the Variable

This prospective cohort study was thus conducted among 512 pregnant women attending antenatal clinic in the Cape Coast Teaching Hospital, Ghana, between January, 2011

Furthermore, these frameworks are concerned only with signalling in the network side, without taking any cross-layer design in the mobile host (MH) into account. Thus,

The Current Readiness Cross-Function Team is responsible for attaining prescribed readiness levels with the people, equipment, and training necessary to generate units

In 2001, the sector covered the National Pensions Fund, the National Savings Fund, 22 active insurance companies and 1007 occupational pension funds created by statutory

(See Figure 2.) Finance executives at midsize firms do recognize that maintaining financial discipline carries a degree of risk—in particular, the risk of underinvestment at a

Unlike the FDIC, the Federal Savings and Loan Insurance Corporation ("FSLIC") was never subject to the ratable distribution requirements of the NBA [National Bank

Once a consumer has told the lender they want to move forward with the loan and assuming no events occur to trigger a Revised Loan Estimate, the consumer next receives the