Metric Development Form
Part 1. About You
Please fill out the following table: Date: Division Unit: Prepared by:Part 2. Background
The purpose of this form is to help each Division manager organize their thoughts about what metrics best tell the story of their unit, and how their unit supports the T&C mission.T&C Mission
To provide and support technology and communication that enhances the educational experience and University operations.Root Question To Answer:
The question we are trying to answer with these metrics is: “How effective are we at enhancing the educational experience and University operations for our customers?” Let’s deconstruct this statement and look it in a bit more detail: 1. What is “effective?” For our purposes, it means “how well are we able to accomplish a purpose or produce an expected result.” Effectiveness depends on such factors as responsiveness, speed, availability, accuracy, predictability, and openness. It also includes looking at how the systems & services we offer are used, and how satisfied our customers are with them. 2. “educational experience” encompasses all activities related to teaching, learning, research and student life. First and foremost, think about the complete life‐cycle of a student in their interaction with the University, and how we support the University’s interaction with them, from the moment they become interested in CI, to the moment they graduate or become an alum. 3. “University operations” encompasses activities which involve the day‐to‐day function of University business and organization. University operations may or may not directly support the educational experience, and most oftenPart 3. Visualization Exercise
In the space below on this page, draw a picture or pictures (using photos, clipart, text/phrases, etc), which answers the question: “what does it mean for my division unit to be effective?”Part 4. Determining measures for your unit
Based on the qualities you visualized in Part 1 of the exercise, in this section, you will define measures that help answer this question: “How effective is my division unit at enhancing the educational experience and University operations for our customers?” Please complete the following tables for at least 3 (but no more than 7) measures that would help you answer this question.Measure #1
Measure Description Write a 1 sentence description of the measure. See the Appendix for examples of measures. How does this measure help tell the story of your division unit? Write 1‐2 sentences which describe how this measure will help your unit improve delivery speed, accuracy, availability, usage, or customer satisfaction. How does this measure help tell the story of the Division? Write 1‐2 sentences which describe how this measure will help the Division enhance the education experience or University operations. If a measure supports a specific item from the strategic plan, please list it here. How will the data be gathered? Type your answer here. Examples: observation, automated tools, customer survey, focus groups, in‐person interviews Who will gather & store the data? Enter the name of person(s) responsible for obtaining and storing the data How often (at what interval) will the data be gathered? Specify daily, weekly, monthly, annually, or another specific interval. Data gathering interval must be less than 1 year. Where & how will the data be stored? Specify location (e.g., network shared drive) and format (Excel, log files, etc) for data. How often (at what interval) will the data be analyzed & reported? Specify daily, weekly, monthly, annually, or another specific interval. Data analysis and reporting interval must be less than 1 year. Who will analyze & report the data? Enter the name of person(s) responsible for analyzing and reporting the data. What is the expected minimum “normal” range for results for this measure? Enter the number or percentage expected as a minimum indicator of effectiveness for the measure (example: 15 users, 50% usage, 10 minutes, etc). Enter “Unknown” if no regular range is known/can be known. What is the expected maximum “normal” range for results for this measure? Enter the number or percentage expected as a minimum indicator of effectiveness for the measure (example: 200 users, 90% usage, 2Measure #2
Measure Description Write a 1 sentence description of the measure. See the Appendix for examples of measures. How does this measure help tell the story of your division unit? Write 1‐2 sentences which describe how this measure will help your unit improve delivery speed, accuracy, availability, usage, or customer satisfaction. How does this measure help tell the story of the Division? Write 1‐2 sentences which describe how this measure will help the Division enhance the education experience or University operations. If a measure supports a specific item from the strategic plan, please list it here. How will the data be gathered? Type your answer here. Examples: observation, automated tools, customer survey, focus groups, in‐person interviews Who will gather & store the data? Enter the name of person(s) responsible for obtaining and storing the data How often (at what interval) will the data be gathered? Specify daily, weekly, monthly, annually, or another specific interval. Data gathering interval must be less than 1 year. Where & how will the data be stored? Specify location (e.g., network shared drive) and format (Excel, log files, etc) for data. How often (at what interval) will the data be analyzed & reported? Specify daily, weekly, monthly, annually, or another specific interval. Data analysis and reporting interval must be less than 1 year. Who will analyze & report the data? Enter the name of person(s) responsible for analyzing and reporting the data. What is the expected minimum “normal” range for results for this measure? Enter the number or percentage expected as a minimum indicator of effectiveness for the measure (example: 15 users, 50% usage, 10 minutes, etc). Enter “Unknown” if no regular range is known/can be known. What is the expected maximum “normal” range for results for this measure? Enter the number or percentage expected as a minimum indicator of effectiveness for the measure (example: 200 users, 90% usage, 2 hours, etc). Enter “Unknown” if no regular range is known/can be known. How should this measure should NOT be used? If applicable, describe in brief narrative how the measure or corresponding data should not be used. Enter N/A if not applicable. Describe any known (or desired) benchmarks that could be used to enable comparative analysis List any benchmarks that can be used to help gauge progress. Example: CSU ATI annual campus evaluation for web accessibility; Educause Core Data Survey/Core Data Service. See Appendix C for more information.Measure #3
Measure Description Write a 1 sentence description of the measure. See the Appendix for examples of measures. How does this measure help tell the story of your division unit? Write 1‐2 sentences which describe how this measure will help your unit improve delivery speed, accuracy, availability, usage, or customer satisfaction. How does this measure help tell the story of the Division? Write 1‐2 sentences which describe how this measure will help the Division enhance the education experience or University operations. If a measure supports a specific item from the strategic plan, please list it here. How will the data be gathered? Type your answer here. Examples: observation, automated tools, customer survey, focus groups, in‐person interviews Who will gather & store the data? Enter the name of person(s) responsible for obtaining and storing the data How often (at what interval) will the data be gathered? Specify daily, weekly, monthly, annually, or another specific interval. Data gathering interval must be less than 1 year. Where & how will the data be stored? Specify location (e.g., network shared drive) and format (Excel, log files, etc) for data. How often (at what interval) will the data be analyzed & reported? Specify daily, weekly, monthly, annually, or another specific interval. Data analysis and reporting interval must be less than 1 year. Who will analyze & report the data? Enter the name of person(s) responsible for analyzing and reporting the data. What is the expected minimum “normal” range for results for this measure? Enter the number or percentage expected as a minimum indicator of effectiveness for the measure (example: 15 users, 50% usage, 10 minutes, etc). Enter “Unknown” if no regular range is known/can be known. What is the expected maximum “normal” range for results for this measure? Enter the number or percentage expected as a minimum indicator of effectiveness for the measure (example: 200 users, 90% usage, 2 hours, etc). Enter “Unknown” if no regular range is known/can be known. How should this measure should NOT be used? If applicable, describe in brief narrative how the measure or corresponding data should not be used. Enter N/A if not applicable. Describe any known (or desired) benchmarks that could be used to enable comparative analysis List any benchmarks that can be used to help gauge progress. Example: CSU ATI annual campus evaluation for web accessibility; Educause Core Data Survey/Core Data Service. See Appendix C for more information.
Appendix
Appendix A. Examples of Measures
These examples are organized by measure categories, and designed to help division managers get ideas for measures which respond to our root question. Delivery Measure Examples (How well are we delivering service to our customers? Are our services available when the customer wants them? How long does it take to deliver our services to our customers? Do we deliver what we say, or are there errors involved?) % of XYZ system/service downtime % of XYZ system/service uptime # of hours of XYZ system/service during maintenance “blackout” periods # of emergency change requests incidents key “blackout” periods % of urgent/emergency change requests # of campus customizations of PeopleSoft % of chartered projects completed on time % of chartered projects completed on budget % of successfully completed RFCs % of RFCs that have caused major issues % of repeat calls to the Help Desk for the same incident # of Help Desk calls missed % of tickets resolved in 24 hours or less % of tickets resolved in 1 hour or less % of tickets resolved in 5 days or less % of services provided through external (off‐ campus) IT % of backups restored in under 30 minutes % of press release requests which become press releases Usage Measures Examples (Are our services being used?) % of active projects for external (non‐T&C customers) % of Help Desk tickets resolved using knowledge base % of CI web site visits using mobile devices # of PeopleSoft security requests # of Help Desk tickets (from ABC unit/program, for customer XYZ, etc) # of logins to XYZ system/service # of myCI users during registration periods # of T&C web site visits # of password reset assistance requests at the Help Desk Top 10 most visited web pages % of available network storage space in use # of backup requests # of courses in CI Learn # of active users in XYZ system # of project requests via project request form # of press releases requests # of workshop/training participants # of downloads of regular publications # of new web sites created # of brochures published # of hours of video footage captioned Customer Satisfaction Measures (Are our customers satisfied with the services and products we provide?) # of strongly agrees and agrees vs. other to customer satisfaction question “The staff in unit XYZ is knowledgeable” # of strongly agrees and agrees vs. other to customer satisfaction question “XYZ system/service has the tools I need to do my job” # of strongly agrees and agrees vs. other to customer satisfaction question “XYZ system/service is easy to use” # of strongly agrees and agrees vs. other to customer satisfaction question “XYZ system/service is fast” # of strongly agrees and agrees vs. other to customer satisfaction question “Projects I’ve requested are completed by T&C staff on time” # of strongly agrees and agrees vs. other to customer satisfaction question “I am satisfied with how the CI web site works on a mobile device” # of strongly agrees and agrees vs. other to customer satisfaction question “I am happy with XYZ publication” # of strongly agrees and agrees vs. other to customer satisfaction question “I am happy with the result of with DEF project” More examples of measures can be found at: http://itbenchmark.files.wordpress.com/2010/06/sample_metrics_for_itil_processes.pdfAppendix B. Possible Areas to Analyze
Planning, Policy & Assessment ‐ this plan, business practices, external requirements, strategic planning System Design Networking and Telecommunication Coding and Programming Project and Portfolio Management Security and Safety Resource Management – hardware and software inventories Support (& customer confidence) Desktop PC Management Mobile device management Instructional Technology Classroom Technology IT Operations Human Resorce Management – retention, satisfaction Training: inward and outward facing Procurement Budget Communication (technology‐specific) Communication & Marketing (unit)
