The next section of the chapter considers in some depth the most effective performance metrics to be used in this work to analyse and conclude on the financial performance of USOs.
In the context of economic development through knowledge transfer in a science and
innovation knowledge-based economy, which includes technology transfer from universities, it is widely acknowledged that there needs to be consensus about the metrics used to measure the performance of technology transfer from universities. Measurement of both quality and quantity of transfer (Library House, 2008) are necessary to enable effective analysis to be undertaken. USOs form an important part of this technology transfer process, and their financial performance at a company level needs to be considered alongside the wider performance of the parent university in this field to give a complete picture. This is necessary both from a policy perspective as well as enabling effective assessments to be made of existing studies.
Various metrics in the area of measuring university technology transfer have been used to date.
In the US, the AUTM (Association of University Technology Managers) measures annually the revenues obtained by universities from their intellectual property, although other studies have attempted to use a wider range of measures both financial and non-financial to give a fuller picture. In the UK, one of the most significant studies to date in this field is that of Library House
43 (2008), which was commissioned by UNICO (a technology transfer association) to build a
coherent framework of metrics that could be used to both measure absolute performance and to benchmark performance between universities on an international basis, with USOs forming a part of this exercise. The commission exercise recognised the lack of an existing study on performance metrics and sought to address this gap, and other studies in the wider literature have noted the difficulty in obtaining a rigorous measure of performance e.g. Siegel and Wright (2015).
3.1.1 Views of university stakeholders on performance metrics
In the study undertaken by Library House (2008), within the wider context of measuring UK university technology transfer, potential financial performance metrics were initially generated through discussion with three key groups of university stakeholders: research funders, senior university management and the business community, reflecting the policy focus of the study. In the specific case of USOs, the following metrics were identified:
Stakeholder Measures of quantity Measures of quality Research funders External investment raised
Revenues generated Market value at flotation
n/a
Business community Number of USOs formed External investment raised Revenues generated Flotation/exit value
Survival rate/viability Growth rate
Customer feedback Senior university
management
Revenues generated External investment raised
Survival rate/viability Table 3.1 Stakeholder views on performance metrics - Library House (2008)
There is a significant degree of overlap between the metrics suggested despite the different backgrounds and perspectives of the stakeholders, which gives confidence that the metrics proposed are fit for purpose. Interestingly, it should be noted that the business community did not propose USOs at all as a mechanism of knowledge transfer for a university, although the reasons for this were not investigated further, but this finding may cast doubt at least on the perceived significance of USOs amongst policy makers and governments as a means for commercialising university knowledge.
In order to aid international comparison, the survey also undertook a similar exercise in identifying performance metrics by contacting senior technology transfer personnel at a number of US universities, on the basis that the US is generally seen as the world leader in technology transfer from universities. In terms of USOs, the following metrics were proposed from a US perspective (the UK findings across all stakeholders from above are shown again to aid comparison):
Measures of quantity Measures of quality US universities Number of USOs formed Investor satisfaction
44 Survival rate
Amount of external investment raised Quality of investors Number of USOs that are geographically close to the university
Survival rate Amount of external investment raised
UK universities Number of USOs formed External investment raised Table 3.2 Measures suggested by TTO personnel - Library House (2008)
In general, the proposed metrics from the two countries are very similar. One interesting quantity metric proposed by the US stakeholders only is the number of USOs geographically close to the university, which may be a consequence of the different ways in which USOs obtain significant parts of their funding: by national research councils in the UK i.e. not region-specific, but by local state governments in the US. The US stakeholders also proposed the metric of amount of external investment raised as both a measure of quantity and quality, given that external investors offer a form of ‘peer review’ in assessing and then investing in USOs, reflecting their quality, a theme echoed in parts of the UK literature (Lambert, 2003).
3.1.2 Wider USO performance metrics - literature review
Apart from the Library House report, there is a small but growing academic literature assessing the financial performance of USOs at a company level. Rasmussen et al. (2012) undertook an extensive literature review of the academic work in this area on entrepreneurial firms including USOs. One of their research questions was ‘what are the strengths and weaknesses of the different methodologies and indicators used to measure the impacts of SBEFs (science-based entrepreneurial firms)?’ They produced the following table which includes only those studies identified which utilised financial performance measures:
Author(s) Which measures/indicators
of impact or performance is used?
Key indicators
Chrisman et al., 1995 Impact measured as venture creation and employment growth
Economic impact (technology transfer) Wallmark, 1997 Measure impact in terms of
employment
Employment Shane and Stuart, 2002 Analyses three dimensions of
performance: the ability to attract venture capital, experience IPOs and failing, Also considers the time it takes to achieve each of these outcomes.
Resource acquisition, financial, survival
Nerkar and Shane, 2003 Performance measured as firm survival (acquired firms
Firm survival
45 included). Authors test
several factors that influence survival and success of a new start-up firm.
Perez and Sanchez, 2003 Impact measured as number of employees. Explores technology transfer by looking at networking activities.
Technology transfer
Ensley and Hmieleski, 2005 Performance measured as net cash flow and revenue
growth. University-based start-ups are found to be significantly lower performing in terms of net cash flow and revenue growth than
independent new ventures
Financial firm performance
Garnsey and Heffernan, 2005 Impact measured as survival, employment and turnover.
Discusses impact from clustering at regional level.
Survival, employment, financial
Leitch and Harrison, 2005 Discussing impact measures such as survival rate, turnover, employment and second order spin-offs
Regional development
Lindelof and Lofsten, 2005 Performance measured a sales growth and profitability.
There is no evidence that USOs exhibit slower growth (sales) than CSOs – they are also equally profitable.
Financial
Moray and Clarysse, 2005 Performance measured as financial measures and
Lawton Smith and Ho, 2006 Impact measured as employment, turnover,
Walter et al., 2006 Performance measured as sales growth, sales per employee, profit attainment, perceived customer
relationship quality, realised
Financial
46 competitive advantages and
long-term survival
Buenstorf, 2007 Time period of presence in the laser industry is taken as a measure of firm performance
Firm survival
Clarysse et al., 2007 Start-up capital raised within 18 months of start-up.
Average capital increase (total capital divided by firm age)
Valentin et al., 2007 Performance measured as financial performance, number of patents
Financial, innovation
Zahra et al., 2007 Performance measured as productivity, profitability
Zhang, 2009 Performance measured as
firm survival
Survival Bonardo et al., 2010 Performance measured as
probability of being acquired after IPO
Venture acquisition
Colombo et al., 2010 Performance measured as growth in number of employees
The contribution of university research to the growth of academic start-ups (number of employees)
Employment growth
Harrison and Leitch, 2010 Impact measured as employment and turnover.
Impact on the
entrepreneurial system is discussed.
Employment, financial
Vincett, 2010 Impact is measured as the present value of past and future sales. Discusses the contribution to Gross Domestic Product.
Financial
Munari and Toschi, 2011 Performance measured as firm’s ability to attract VC funding. Venture capitalists do not have a bias against
Attract VC funding
47 investment in academic
spin-offs.
Rasmussen et al., 2011 Performance defined as reaching the credibility threshold (adding new team members beyond the original inventor(s) and early stage investment from a private sector investor).
Process of venture start-up
Table 3.3 Performance metrics taken from Rasmussen et al. (2012)
In reviewing the relevant literature, Rasmussen et al. (2012) note that studying firm
performance during the early start-up process is challenging because it is difficult to identify samples of firms at an early stage of development, and USOs often have long development paths before they grow and become profitable, making it difficult to effectively use growth-based performance measures. Measures such as gaining external equity investment and additional team members are thus useful in studying the early stages of a firm as it seeks to overcome the credibility threshold. As a consequence, Rasmussen et al. (2012) note that to investigate the long term effect of start-up conditions there is a need for longitudinal datasets over long periods of time. This observation applies equally to financial performance studies.
They further note that creating datasets by using historical data is labour intensive and restricted to issues covered by historical documentation, which is the driving force behind the current work. One example of a long-term study in the literature is that of Buenstorf (2007) who examined the evolution of the German laser industry using historical data from various sources over forty years. Rasmussen et al. (2012) consider, however, that the recent increase in studies on USOs will allow more follow-up studies on existing datasets. It should also be noted that there are a number of common performance measures between the studies of Rasmussen et al.
(2012) and Library House (2008), which gives a valuable insight for the current work as it seeks to identify the most effective performance metrics to answer the overarching research
question.