The University Library System (ULS)7 is used by students, faculty, and staff of the University of Pittsburgh and, through col- laboration with organizations and institutions worldwide, the global research community. ULS is focused on becoming even more centered on users’ needs.
ULS has been challenged over the past decade by massive changes in the relationship between users and information. These changes have stimulated deliberation about the system’s ability to address internal and external user needs by creating new initiatives for better com- munication; organizational agility; and, most importantly, assessment. Consequently, ULS has undergone a radical transformation, especially in its approach to assessment.
35
university of Pittsburgh self-study rePort 2012
Planning, Assessment, and Links to Institutional Goals
Formerly, ULS assessment focused on counting quantities, such as the number of patrons who entered the library and the number of books lent, in addition to statistics on data- base and electronic journal usage. This was a traditional form of assessment upon which many libraries have relied. ULS has come to realize, however, that assessment must qualify what these quantities mean to ULS. The evidence of the evo- lution of this approach can be traced through the ULS planning and budgeting report outcomes, which are on file from 2000 to 2011 (available on the DVD and in the document room).
Annual departmental goals will include measurable goals for each step in a project cycle. These new assessment standards represent a major shift from previous ULS planning, which tended to be more insular and at times in conflict with the larger goals of the University. The many departments of ULS had many different objectives that needed to be streamlined into an organization-wide culture of assessment that would allow the organization to create goals to complement the University’s mission. The ULS planning and budgeting report for fiscal year 2011 provides a clear matrix demonstrating the relationship of these.
Given the broad mission of ULS, the first step in creating a culture of assessment was the retention of a librarian who would be responsible for assessment. This assessment librarian now acts as a consultant for all assessment practices as the ULS-wide assessment plan is being formed. This plan also includes assessment requirements for the departmental goals of each area of ULS. To strengthen the inclusivity of this new assessment culture, the assessment librarian and associate directors met with every department head and any staff members who wished to be included in the meetings. During this time, goals were reassessed for the effect they had on the ULS mission and how assessment could be further retooled. These meetings also informed the creation of the new long-term ULS plan.
Examples of Assessment Used to Improve Infrastructure
Several new and continuing projects dem- onstrate the ongoing commitment to assessment. For example, in addition to library holdings, ULS’s assessment practices naturally include library services. In the past two years, the analysis of transactions at various libraries on campus showed that the circulation (use) of the Graduate School of Public and International Affairs
(GSPIA) collection decreased 64 percent from 2004 to 2010, and the use of electronic resources increased by 67 percent from 2004 to 2009. As a result, the GSPIA Economics Library was moved to Hillman Library. This consolidation has resulted in new, much-needed space for aca- demic purposes and in more efficient service to the University community. During the same time period, ULS also was implementing a greater focus toward online or virtual reference services. Through the use of instant messaging software, any user could contact a librarian from any loca- tion with a computer or cellular device. Yet, ULS needed to ensure that users would continue to receive the same level of quality offered at the face-to-face reference desk. A methodology for collecting user transactions was implemented, and those data allowed for a new set of best practices for the virtual reference transactions. This assessment will become part of the ULS ongoing plan so that users enjoy a consistent level of service in all reference transactions.
Another assessment project is related to information literacy, one of the learning objectives of the University (see Assessment of Student Learning Outcomes section in the Using Assessment to Improve the Student Experience chapter for a more comprehensive discussion of this subject). To create an assess- ment methodology, ULS began by using the Standard Assessment of Information Literacy Skills project, which was developed at Kent State University. ULS measured the information literacy aptitudes of incoming freshmen for several years; these data were then compared to a national standard and divided up into multiple categories. The first seniors who were tested as
freshmen also have been reanalyzed for compari- son. Initial results have been promising and have led to a retooling of information literacy instruc- tion sessions for ULS. However, this project can- not ensure that the same students were directly tested, leaving a clear deviation in the results. ULS is currently analyzing additional assessment possibilities for information literacy, including the development of an immersion program that can be integrated into all academic departments. Improving and Defining a Sustainable
Assessment Practice
ULS assessment efforts have become much more active in the past few years, as the recent long-term plan attests. The ULS director elaborates:
Whether we are analyzing library catalog searches, gate traffic, or statistics of digital downloads or simply making sure that users have a comfortable place to sit and read, ULS makes assessment a priority. We have taken key initiatives and focused them into measurable objectives that are based on our principles of developing innovative, user- centric services; the ability to adapt to the fast-paced technological changes facing the future of academic libraries with our orga- nizational agility; and, most importantly, we are learning that our qualitative analysis, rather than quantitative, has encouraged dynamic change and the ability to keep up with technology, rather than stagnation. This dynamism is essential to the future success of ULS.
There is an explicit commitment to assess- ment at every level of the ULS organization as well as the recognition that mining assessment data is critical to the future relevance and ability of ULS to execute its mission. The ULS opera- tion demonstrates a high level of sophistication in planning and is poised to continue in an organized, systematized, and sustained effort that generates useful results.