29
EVALUATION FRAMEWORK FOR SERVICE CATALOG MATURITY IN
INFORMATION TECHNOLOGY ORGANIZATIONS
Carlos Moreno Martínez
Information Systems Department, Universidad Europea de Madrid – Spain Email: 20839394@live.uem.es
Abstract
Today, Information Technology services are a fundamental part of the success of enterprises and allow the development of the Information Society. Every IT service provider should deliver services aligned with the changing needs of their
customers to an agreed level of quality.
The IT Service Catalog is a list of these services and represents the value proposition of the IT Service provider. Many Service providers have based their Catalog on industry and academic best practices such as ITIL V3©. However these best practices do not include a specific maturity assessment model, do not advise about how to adapt the Service Catalog
to organizational and business changes or how to make the best use of innovations in technology.
This article proposes an evaluation framework (ECAT) to measure the maturity level of an IT Service Catalog, which should help service providers understand their value proposition and how it fits with the current situation and business needs of their customers. It also allows a periodic alignment review of the Service Catalog with the continual changes in
the organization and business strategy.
Keywords IT Service Catalog, IT maturity, Assessment framework, Business and IT alignment
1. INTRODUCTION
The services supplied by every IT Service provider, either internal or external, are a fundamental part of the Business Operations. They support the evolution of the organization and, in some cases, even with the development and general growth of society and nations [1][2].
For this reason IT Service Management best practices (ITSM) highlights the need to manage the service lifecycle and the importance of having a Catalog that describes the relationship between the consumer and the supplier of these services. Multiple organizations have adopted these best practices, and have documented their provision of services in an IT Service Catalog.
To ensure the organization gets the best out of these services and to add true value, these services must be integrated with the business processes they support and have the ability to adapt to changes in the organization [3]. Therefore the mutual relationship between the IT Service Provider and the consumers of these services must be described from the point of view of adaptation. Figure 1 represents this interdependence, and reflects how
changes in the organization, in their products and services, or changes in the business model, affects or is affected by changes in IT, services or technology innovations1.
Business change
Organization Products & Services
Market Business model
IT Change
Provider type Products & Services
Technology Innovation
Figure 1: Relationship between business changes and IT changes
The dynamics of this relationship requires a continuous revision of the IT Service Catalog, to ensure it adapts to these changes, and reflects the needs and vision of the consumer as well as of the supplier of the services. New
1 Examples of recent Technology innovations are Cloud services and Big Data
30 innovations in technology arise and it becomes necessary to have a standard framework to review the Service Catalog. This article proposes an evaluation framework to analyze the maturity level of an IT Service Catalog and the operational management required to maintain it up to date and valid. This model could be applied to any IT Service provider organization that has defined a Service Catalog based on IT Service Management best practice such as the Information Technology Infrastructure Library (ITIL©).
The proposed evaluation framework:
Allows a Service Provider to understand the current position of their Service Catalog and its alignment to the business’s objectives. This is the primary goal of the proposed model.
Serves as a reference point for the continuous improvement of the Service Catalog and IT Services
Helps gather feedback after any IT Service management improvement plans.
The proposed model is based on concepts of ITSM and extends into the best practices of Information technology management.
2. STANDARDS, MODELS AND
RELATED FRAMEWORKS
The proposed framework (ECAT) shares an approach used in various standards and best practices that are widely adopted and recognized in business, industry and academic environments:
Control for Objects Information and related Technology (CobiT®)
Information Technology Infrastructure Library (ITIL®) [2]
CMMI® for Services, Version 1.2
ISO/IEC 20000: Information Technology— Service Management©
This section summarizes relevant aspects of those models in relationship to Service offerings.
2.1. Control Objects for Information and related Technology (CobiT®)
The framework for the Control for Objects Information and related Technology and securing of IT Government (Cobit®) establishes best practices in five areas [4]:
Strategic alignment is aimed at assuring the
relationship between business and IT plans, defining, maintaining and validating the IT value proposition and aligning the business operations with IT operations.
Value delivery is about executing the value
proposition throughout the delivery cycle, ensuring that IT delivers the promised benefits against the strategy, concentrating on optimizing costs and proving the intrinsic value of IT.
Resource management is about the optimal
investment in, and the proper management of, critical IT resources: applications, information, infrastructure and people. Key issues relate to the optimization of knowledge and infrastructure.
Risk management requires risk awareness by
senior corporate officers, a clear understanding of the enterprise’s appetite for risk, understanding of compliance requirements, transparency about the significant risks to the enterprise and embedding of risk management responsibilities into the organization.
Performance measurement tracks and monitors
strategy implementation, project completion, resource usage, process performance and service delivery, using, for example, balanced scorecards that translate strategy into action to achieve goals measurable beyond conventional accounting.
2.2. Information Technology Infrastructure Library (ITIL®)
In its current version 3, ITIL ([5], [6], [7], [8], [9]), is the de facto standard for IT Service Management, covering processes, organization and technology. It is arranged around five Management areas, aligned with the lifecycle of a service:
Service Strategy provides recommendations and
defines concepts to translate IT Services into strategic business assets. Some of the concepts defined include business relationship management roles, value of a service from a
31 utility and warranty perspective, the IT portfolio and Service Catalog, as well as provider categorization and market segmentation in relation to service levels and business activities.
Service Design defines recommendations for the
design of new or enhanced services, along with policies and best practices for service implementation, to satisfy quality and customer expectations, as well as cost efficiency.
Service Transition establishes guidelines and
recommendations for the development of capacities that allow the release to production of services, assuring service design requirements, with risk control
Service Operation defines best practices for the
delivery and support of the services, assuring the value provided to the client by the service, utilizing the defined strategic targets.
Continual Service Improvement proposes
recommendations and techniques for optimizing Service Management through a cycle of continuous improvement that considers all stages of the service lifecycle.
2.3. CMMI® for Services, Version 1.2
CMMI defines improvement practices for service delivery related processes. It is based on the Capability Maturity Model Integration for Software development as defined by the Software Engineering Institute. A fundamental part of the framework proposed by CMMI is the maturity and qualification levels that allow the evaluation of the maturity of an organization against a standard, in this case IT Service Management. The model associates process areas with categories and levels of maturity. Specific levels of maturity are reached if the area of processes is in place and fulfills all the objectives required for this level. The CMMI model proposes processes that are structured around a project phase and a delivery phase [10]:
Project phase
Capacity and Availability Management (CAM)
Project Monitoring and Control (PMC) Project Planning (PP)
Requirements Management (REQM) Service Continuity (SCON)
Service Delivery phase
Service Delivery (SD)
Incident Resolution and Prevention (IRP) Service System Development (SSD) Service System Transition (SST) Strategic Service Management (STSM)
In addition to these process areas there are other areas that are common to other models of the CMMI space, such as the definition of organizational processes and organizational training.
2.4. ISO/IEC 20000: Information Technology— Service Management©
This international standard promotes the adoption of a process framework for IT Service management according to business needs and service consumer [11]. The standard defines the minimum requirements a service provider should provide to ensure acceptable quality for its customers. The provider organization can be certified under this standard, stating that all or parts of its services are managed through the levels proposed by the standard. The standard aims to improve the processes through a standard continuous improvement methodology that adopts the Plan, Conduct, Check, Act (PDCA) approach. The process framework proposed by the standard is organized into the following groups:
Relationship processes, including Business relationship management and supplier management
Service Delivery processes, including Capacity Management, Continuity Management, Security Management, Service Level Management, Budgeting and Accounting for IT Services and Service Reporting
The Control Processes group includes Change Management and Configuration Management
Release Processes includes Release Management
Resolution processes includes Incident Management and Problem Management
32
3. THE IT SERVICE CATALOG
MATURITY EVALUATION
FRAMEWORK
As discussed in the previous section on standards, models and related frameworks (hereinafter referred to as
schemas for simplicity), all of them present the process as
a key part of their approaches. The ECAT model also considers the IT management processes a fundamental aspect for assessing the quality of the Service Catalog, but it also includes other areas that should be measured before arriving at the overall maturity of an IT Service Catalog.
These additional areas include communication, the consumer’s knowledge of the Service Catalog, the relationship of the Catalog to the business processes and the ongoing Catalog management.
The above schemas classify the IT management processes into categories which cover the entire service life cycle. The ECAT assessment model shares the common categories used in each schema but broadens the scope by adding additional management areas for the assessment. Table 1 shows the relationship of the schemas and the areas covered by ECAT.
Schemas Scope of ECAT
Cobit ITIL V3
CMMi-SVC
ISO20000
Integration with the organization and the business mission
Integration with business processes, products and services
Communications strategy and publication of service offering
Flexibility and adaptation to change
Provider's operational efficiency
Regulatory compliance
Table 1: Scope of ECAT and relationship to other frameworks
The areas listed in Table 1 are described below:
a) Integration with the organization and the
business mission. This area provides the analysis
of the effectiveness of the IT Service Catalog as a relationship management tool between the business (service consumer organization) and the IT department (supplier), and to what extent the Catalog reflects the mission and business model. Another area to cover is the effectiveness of the Catalog as a tool to position the provider as the best alternative as technology supplier and strategic partner for the business, allowing among other things, to show the value of the service offerings.
b) Integration with business processes, products and services. This area allows the analysis of the
scope and accuracy of the Catalog with regards to business processes, and the relationship of IT Services to these processes. For example it should allow the measurement of which business processes are covered by the Service Catalog, or
if the prioritization assigned to IT Services corresponds to the prioritization understood by the business. It should also measure the suitability and integration of the Catalog with the products and services of the organization.
c) Communications strategy and publication of the service offering. This area allows the
measurement of the focus and effectiveness of the Catalog, the distribution methods and the degree of awareness among the consumer organization of the Catalog contents.
d) Flexibility and adaptation to change. This area
assesses the ability of the Catalog to reflect changes in the business and IT organization. This covers the amount of integration between the Service Catalog and the strategic and operational demand processes as well as processes for technology and infrastructure change. Furthermore, this section should also measure the effectiveness of the processes in place for adapting the Catalog to business changes.
33 e) Provider's operational efficiency. This area is
responsible for assessing the level of Catalog integration with the organization and the management frameworks of the IT supplier. It also assesses the Catalog’s integration with the IT management systems including Finance. f) Regulatory compliance. This area assesses the
compliance of the Catalog with corporate policies and regulations, as well as with external regulations.
3.1. The evaluation framework
The ECAT model structure, components and relationships are depicted in Figure 2.
Analysis & recomendations Plans Indicators Metrics and thresholds Evaluation areas Evaluation domains Maturity level Business goals &
Corporate strategy & IT Strategy Service management frameworks Evaluation data ECAT
Figure 2: IT Service Catalog Maturity evaluation framework (ECAT)
The ECAT model (central area of Figure 2) assesses information at various levels to evaluate the Service Catalog maturity. The highest level, the domain level,
provides assessment guidelines for each proposed IT Management area, in line with the principles and scope of the ECAT model. The first step therefore is to identify the management domains to be evaluated. This selection must be consistent with the current business goals and corporate strategy which are in place, and the strategy of the service provider organization (left block in Figure 2).
Once decided which domains to evaluate, the model uses lower levels to add more detail for the evaluation. These levels are Evaluation areas, Indicators, and Metrics and
Thresholds. Each level is based and dependent on the
previous level.
The Evaluation areas are used to structure and organize the domains. They facilitate the definition of improvement plans after the assessment has been carried out. The
Indicators establish the specific objectives to be evaluated
within each area. Indicators are the basis for further analysis and assessment.
Finally the Metrics and thresholds define the data required for the maturity level measurement, which allows the quantitative calculation of the maturity.
The input to the model is data needed to evaluate the concepts measured by the metrics. The output is the information to calculate the maturity levels of the IT Service Catalog.
Plans, analysis and recommendations are beyond the scope of the ECAT model (right area of Figure 2).
3.2. Evaluation areas and indicators
The following table shows the summary of the evaluation areas and indicators proposed by ECAT for each domain (Table 2). This represents a comprehensive overview of ECAT’s evaluation framework.
34
Domain Area Indicators
1. Integration with the organization and the business mission
Scope of the service offering and perception of the value of the Catalog
Business units (BU) covered by the Catalog
Level of understanding and opinion about the Catalog by the Board of Directors and Executive level management
Existence of an agreed definition of “IT Service” and a glossary in the Catalog
Existence and quality of a Strategic Service Map2
Level of usage of the glossary by IT customers and users
Existence of IT Services not based on Information Systems (such as professional services, etc.)
Relationships of the Service Catalog with the Business mission and the current Business model
Existence and quality of a Strategic Service Map
Existence and quality of a Service Catalog Brochure
Existence of an IT provider mission statement aligned with the corporate mission statement
Existence of a detailed customer-centric service description (external vision of the Catalog)
Integration with the business organization and
relationship with customers and users
Inclusion of terms & conditions for IT Service users in the Service Catalog
Inclusion of IT Service provider’s responsibilities in the Service Catalog
Identification of Services owners and Service managers for each service defined in the Catalog
2. Integration with business processes, products and services
Level of integration with business processes
Business processes covered by the Service Catalog
Does the Catalog reflect the IT Service importance agreed with service owners and in line with the importance of the related business processes?
Level of integration with business products and services
Scope of Products and Services included in the Catalog
Direct relationship of IT Services with business customers 3. Communications strategy
and publication of the service offering
Communication and publication of the Catalog
Existence of an official Web publication of the Service Catalog (Intranet, Extranet, Internet)
Existence of a corporate method to spread news related to the IT Service offering (newsletters, distribution lists, etc.)
Existence and quality of the Service Catalog Brochure and leaflets.
Understanding of the Catalog by the service consumer community
Level of involvement of the service consumer community in the review of the Service definitions and the Catalog
Results of customer surveys i. Global value of the Catalog
ii. Value of the accessibility to the Catalog contents iii. Quality of communication related to IT Service offering iv. Level of usage of the Catalog by users and external related entities 4. Flexibility and adaptation
to change
Relationship to business changes
Level of usage of the Catalog in the corporate Strategic and Operational demand management process
Level of usage of the Catalog in Project Portfolio management and Technology Investment analysis
Existence of corporate policies and procedures to manage new services and changed services and incorporate them into the Catalog
Ratio of changes in Business organization vs. changes in Service Catalog
Catalog adaptation and agility to changes (internal,
Level of usage of the Catalog in Business technology change management process
2 The Strategic Service Map presents the relationship between the Business organization, the Business processes and the IT Services which supports and enables these processes. It represents the global overview of the Service Offering provided by the IT Organization.
35
Domain Area Indicators
external) Usage and acceptance of the Catalog by business relationship management roles within IT (Customer Managers, Service Managers, etc.)
Efficiency of the Service Catalog Management process
Existence of Service Catalog Management roles and responsibilities
Level of Catalog quality measurement 5. Provider's operational
efficiency
Integration with the organization and with management frameworks of the service provider
Existence of a service description including requirements and information intended for the service provider (internal service description)
Relationship of the Catalog contents with Operational Management Systems (Event management, Incident and Problem management, inventory management systems, etc.)
Integration with Quality management systems and policies
Existence of relationship with corporate Quality Management models and systems (Service Level Management process, EFQM, etc.)
Usage of the Catalog as approved source for audit process
Relationship with IT performance measurement
Relationship with the technology asset lifecycle
Existence of the relationship and usage of the Catalog with service development frameworks and methodologies
Relationship with IT Asset management process
Relationship with IT providers
Level of integration of the Catalog with IT provider Management systems 6. Regulatory compliance Integration with external
regulations
Integration with corporate policies and regulations
References in the Catalog of corporate regulations and policies
Relationship with policies and regulations of the IT Service provider
Usage of the Catalog as source of information in relation to corporate policies
Table 2: Evaluation areas and indicators
4. PROPOSED EVALUATION
METHODOLOGY
The proposed evaluation Framework has been designed to be used in conjunction with an evaluation methodology based on a continual management cycle and periodic review of the maturity level.
In general terms the evaluation methodology should include the following recommended activities:
4.1. Define the goals for the evaluation and selection of the evaluation domains
The ECAT model has been designed to use all the proposed evaluation domains, although it is possible to limit the maturity assessment to specific management domains.
The selected domains are aligned with the corporate and IT strategic priorities. For example, if an IT organization wants to focus on their relationship with the business, domains 1 and 2 could be assessed. Domain 4 should be included if a software quality improvement program is in place. If the focus is placed on IT management best
practices for Service Operation, domain 5 could be used to evaluate the adequacy of the Catalog for these initiatives.
The decision about which domain to assess should be taken by the CIO and IT Executives with the support of the Service Catalog Manager. Also the CIO should appoint an Evaluation Manager who should be in charge of leading and coordinating the complete evaluation of the maturity of the Catalog and to present the final results to the stakeholders.
4.2. Set up the evaluation plan and the evaluation resources
This step is about defining the needed scheduling for the assessment, as well as scheduling the required tasks of the evaluation team. Other tasks are to identify the people involved from the IT provider and the business. This step should be carried out by the Evaluation Manager.
36 Using the indicators from ECAT as guidelines, the data should be obtained using standard evaluation techniques, for example, interviews with stakeholders and a review of existing documentation.
4.4. Service Catalog maturity calculation
The resulting evaluation data should be formulated according to a qualitative method, for example from level 1 (minimum) to 5 (maximum). If the metric is qualitative then a conversion algorithm should be used to convert the value to a numeric format. The following example contains a simple table to convert qualitative metrics into numerical format.
Qualitative metric value Evaluation level
Measured aspect Nonexistent and not planned to build in the Service Catalog
Example: The following aspect is not in place and not planned to be included in the Service Catalog structure:
“Existence of a service description including requirements and information intended for the service provider (internal service description)”
1
Measured aspect Nonexistent but planned
3
Measured aspect exists in the current Service Catalog
5
The final results obtained from each metric should be weighted with the rest of the evaluation area to get the final evaluation for the corresponding domain.
4.5. Definition and execution of improvement plans base on the evaluation result
Once the maturity level has been determined for each assessed domain, improvement plans should be documented to improve the quality of the Service Catalog. The evaluation domains with lower levels of maturity could be used to prioritize the required improvement tasks.
The improvement plan should have an owner responsible for the coordination and monitoring of the plan and it should involve all affected groups within the service provider and service consumer organizations.
5. CONCLUSIONS AND NEXT STEPS
The proposed maturity evaluation framework for Service Catalogs can be a valuable tool for CIO, IT Executives and Service Managers in assessing the Catalog maturity. It could also be used as a framework where no Service Catalog exists and the design and development of the service offering is to be carried out. In addition, the ECAT model is proposed as a reference and guide related to general IT Service Management topics, including people, processes and technology.The proposed model could be evolved in relation to the metrics and their thresholds for each evaluation domain. Each indicator could be further developed defining specific metrics.
Another interesting model evolution area is the definition of specific industry measurement frameworks to allow a more precise vision of the efficiency and correctness of the Service Catalog in relationship to industry and business sectors.
6. REFERENCES
[1] World Economic Forum, “The Global Information
Technology Report 2009–2010”, ISBN-13: 978-92-95044-81-4 [2] R. Kaplan and D. Norton, "Alignment: Using the Balanced Scorecard to Create Corporate Synergies", Harvard Business School Press, 2006
[3] C. Moreno, "The IT Service Catalog as foundation for IT Governance", IV International Congress: "Evolving from IT Service Management to IT Governance", itSMF Spain, 2009
[4] IT Governance Institute, “Cobit 4.1”, ISBN 1-933284-72-2
[5] OGC, ITIL® V3 Service Strategy, OGC, 2007, ISBN
978-0-11- 331045-6
[6] OGC, ITIL® V3 Service Design, OGC, 2007, ISBN 978-0-11-331047-0
[7] OGC, ITIL® V3 Service Transition, OGC, 2007,
ISBN 978-0-11-331048-7
[8] OGC, ITIL® V3 Service Operation, OGC, 2007,
37
[9] OGC, ITIL® V3 Continual Service Improvement,
OGC, 2007, ISBN 978-0-11- 331049-4
[10] CMMI® for Services, Version 1.2, February 2009, Software Engineering Institute, Carnegie Mellon University
[11] International Organization for Standardization,
ISO/IEC 20000-1 Information technology – Service
management – Part 1: Specification
[12] Center for Information system Research MIT Sloan, Working Paper No. 4237-02, “MIT Sloan School – Don’t Just Lead, Govern - Implementing Effective IT Governance” [13] Sallé, M., Rosenthal, S., “Formulating and Implementing an HP IT program strategy using CobiT and HP ITSM”, IEEE Hawaii International Conference on System Sciences 2005 [14] Kris B., Curtis, D., “The Fundamental Starter Elements for IT Service Portfolio and IT Service Catalog”, Gartner Research ID Number: G00201445, July 2010
[15] Johnson, Alice M., Lederer, Albert L., “CEO/CIO mutual understanding, strategic alignment, and the contribution of IS to the organization”, Journal of Information & Management 47 (2010) 138–149
[16] McNaughton B., Ray O., Lewis L., “Designing an evaluation framework for IT service management”, Journal of Information & Management 47 (2010) 219–225
[17] Sánchez, José I., “¿Pueden las tecnologías de la información mejorar la productividad?”, Universia Business Review, Primer trimestre 2004
[18] Abeck S., Link S., Mayerl C., Mehl O., Vogel T., “A system supported method to design IT services”, Cooperation & Management, Universität Karlsruhe (TH)
[19] González Z., Calvo M., Gil A., “Fundamentos científicos para el análisis de las complementariedades entre el binomio SI/TI y la calidad de servicio”, DOCUMENTO DE TRABAJO 2002-02, Facultad de Ciencias Económicas y Empresariales Universidad de La Laguna
[20] Niessink F., Vliet H., “Towards Mature IT Services”, Software Process – Improvement and Practice, volume 4, issue 2, June 1998, pp. 55-71
