Library analytics
Understanding impact and value
Graham Stone
Information Resources Manager
This work is licensed under a
Creative Commons Attribution 3.0
Unported License
…to improve existing
services
…to gain insights into
user behaviour
…to measure the
impact of the library
To support the hypothesis that…
Library Impact Data Project 1
Original data requirements• For each student who graduated in a given year, the following data was required:
– Final grade achieved
– Number of books borrowed
– Number of times e-resources were accessed
Library Impact Data Project
Phase I– Showed a statistical significance between:
• Final grade achieved • Number of books
borrowed
• Number of times e-resources were accessed
– Across all 8 partners
Not
a c
aus
e a
nd
effe
ct r
elat
Library Impact Data Project
Phase I looked at over 33,000 students across 8 universities
Library Impact Data Project 2
Additional data• Demographics
• Discipline
• Retention
• On/off campus use
• Breadth and depth of
e-resource usage
• UCAS points (entry data)
Library usage
Library usage
Retention• Looking at one year of data for every student
• Using a cumulative measure of usage for the first two terms of the 2010-11 academic year
• Only looking at people who dropped out in term three • All the students included in this study were at the
Other factors
Number of e-resources accessed
• Both borrowing books and logging onto electronic
resources does not guarantee the item has been read, understood and referenced
• Heavy usage does not equate to high information seeking or academic skills
Adding value
Initial results• Rank entry points and final grade as percentage
• Does the difference correlate with measures of usage?
• WARNING! This needs further testing!
• Methods are untried
• Missing data
Going forward
@Huddersfield• Identifying retention issues and our impact on lowering them as part of a University dashboard
Engagement
Workload
Performance
Going forward
@Huddersfield• Two spin off projects
– Lemon Tree
– Roving Librarian
• Look at specific subjects in order to work towards:
– A best practice toolkit for information skills sessions
– Further understanding by holding focus groups with target areas
• Create an action plan to engage with academic colleagues • Showing value for money and the impact of the service on
Library Analytics Survey
We asked:
How important will analytics be to academic libraries now and in the future, and what is the potential for a service in
this area?
How important will analytics be to
academic libraries
• Significant appetite for analytics services among this sample
– 96% were interested in the automated provision of analytics demonstrating the relationship between student attainment and library usage
• Strong willingness to share a broad range of data
Key strategic drivers
1. Enhancing the student experience
2. Demonstrating value for money
JiscLAMP
Library Analytics and Metrics Project
• Looking at the benefits of scale
• To develop a prototype shared library analytics service for UK academic libraries
– Envisioned as a data dashboard
– Enabling libraries to capitalise on the many types of data they capture in day-to-day activities
JiscLAMP
A brief word on ethics
• Should we be holding and analyzing this kind of data
– Data protection issues – ‘Big brother’
– All students pay the same fees – shouldn’t they be treated the same?
• But what if we didn’t do this
– What would the reaction be if it was found that we had this data but didn’t act on it?
The epic user stories
Consulting with the community
• connect the library with the university mission
• contribute to the institutional analytics effort • demonstrate value added to users
• ensure value from major investments • develop investment business cases
• impact student measures of satisfaction, such as NSS
• address measures of equality and diversity of opportunity
• inform / justify library policy and decisions as evidence led
• engage stakeholders in productive dialogue
• identify basket of measures covering all key areas
• inform librarian professional development
Job stories
JiscLAMP
JiscLAMP
JiscLAMP
JiscLAMP
What did we achieve?
• LAMP project outputs
– We managed to clean up and process the data from all of the partners – We created a prototype – our analytics engine
– We performed a benchmarking exercise
JiscLAMP
What can we do with the data?
• We can demonstrate usage by cohorts: Department
Degree name Course
Course ‘type’?
Gender/Ethnicity/Nationality/Disability/Age Level of attainment
Attendance mode (full time/part time) UCAS points
JiscLAMP
JiscLAMP
JiscLAMP
Where do we go from here?
• LAMP Phase 2
– We have funding for Phase 2 – We started testing the ‘ugly’
prototype yesterday!
The initial prototype
Prioritizing user stories• Merge data from multiple systems
– Library, student registry, IT services
• Contribute to the institutional analytics mission
– Avoid data and reporting silos, e.g., spreadsheets and reports
• Compelling visualisations
• Map e-resource usage to actual users • Key usage indicators by discipline
• Examine events by specific user groupings
Other issues to address
Future prototype functionality?• ‘High value’ users stories
– Access to raw data
– Correlate NSS scores, enquiries and collection strength – Correlate reading lists with actual usage
• Wider issues
– How do library analytics fit in with the SCONUL return – Triangulate usage with cost and license terms (JUSP/KB+) – Understand the patterns of e-resource use (JUSP/Raptor)
– Inform decisions about relegation/relocation/weeding of stock (Copac Collection Management)
JiscLAMP
Phase II• Workshop with SCONUL (London 7 May 2014)
– engaging the wider library community, specifically library directors
• Key contacts/relationships for next phase
– HESA (NSS)
– Shibboleth/Athens
– SCONUL (performance group)
• Business case ideas
Thank you!
http://jisclamp.mimas.ac.uk
This work is licensed under a
Creative Commons Attribution 3.0
Unported License
Graham Stone
