delivering integrated IT management solutions

(1)

agility

made possible

™

Stephen Tyler

CA Catalyst Product Management

delivering

integrated IT

management

solutions

(2)

Challenge

IT management products are rarely used in isolation. The deployment of integrated and automated solutions to problems that span functional domains increasingly requires sophisticated integration of the data and capabilities of multiple products, often from different vendors. To enable integration, most products expose programmatic interfaces using technologies such as web services. However, there is huge variability in the ways in which products represent the data objects that are exposed and consumed through these programmatic interfaces, as well as in the technology and nature of the interfaces themselves. These challenges make it difficult for enterprise IT organizations to rapidly deploy integrated solutions with an attractive Total Cost of Ownership (TCO).

Opportunity

The increasing need for IT organizations to deliver integrated solutions more reliably and more quickly creates an opportunity and an imperative both to implement best practices and to apply technology to simplify solution engineering and maintenance. CA Catalyst is the CA Technologies response to this opportunity. CA Catalyst makes IT management products from CA Technologies and other vendors easier to integrate by exposing their data and capabilities through a consistent programming model and a common data model. It also provides a set of services to standardize the implementation of various forms of integration-related functionality from the trivial (e.g. product X provides a data feed to product Y), to the complex (e.g. service model driven IT management powered by automation and governed by policy).

Benefits

The benefits of engineering–integrated solutions leveraging CA Catalyst include:

• More reliable solutions due to standardization in the way integrations are delivered, reducing the

risk of solution integrity being undermined by patches or upgrades to underlying products, and increasing the ability to rapidly triage and determine root cause when problems arise.

• Rapid time-to-value due to the delivery of integration logic in the form of automated process content

and integration policy, leveraging common process and policy libraries, rather than custom code. • Attractive total cost of ownership due to more efficient use of services to construct integrations as

part of initial deployments, a lower likelihood of needing further services engagements to maintain the integration over time, and a lower likelihood of significant support issues.

(4)

Section 1: Challenge

Delivering integrated IT management solutions

IT management products are rarely used in isolation. The deployment of integrated and automated solutions to problems, that span functional domains, increasingly requires sophisticated integration of many products from many vendors.

To enable this integration, most products expose programmatic interfaces using technologies such as web services. However, there is wide variability in the ways in which products represent the data objects that are exposed and consumed through these programmatic interfaces, as well as in the technology and nature of the interfaces themselves. With regard to how products represent data objects that they manage and expose through their programmatic interfaces, there are many relevant standards including MIBs, SID, CIM, CMDBF, OVF etc. that apply to specific domains within IT

management. There is not however currently a universal standard that can be used as a coherent schema and semantics for the entire IT management domain. Even where standards do exist, support for them is anything but universal across IT management products.

This creates a situation where most products that need to be integrated are not easily integrable. They have APIs, but they don’t have normalization of schema and semantics, and this lack of normalization creates significant integration challenges.

Standards support is somewhat better at the transport protocol layers where web services protocols are fairly well defined and supported. Even here, many products do not support the more recent standards and some older but important products may only support highly proprietary or command line style interfaces.

The bottom line, to engineer a successful integration between two (let alone many) products requires deep and specific knowledge of how each product:

• represents and exposes its data and capabilities • is programmatically accessed and driven

Tools such as process automation engines bring a degree of automation and standardization to solution engineering, but the inherent lack of standardization in the data semantics and programming models of the APIs exposed across the target products leads to relatively time consuming and costly work to initially engineer integrated solutions. A follow on and significant challenge is the work required to maintain these integrations over time, as changes to the underlying products are delivered by vendors through release patches and version upgrades. Further, delivery of integrated solutions today typically requires the creation of custom coding or product customization as opposed to configuration of common integration services.

Consequently, engineering and delivering reliable, supportable, integrated solutions that provide rapid time-to-value (TTV) and attractive TCO are extremely challenging for enterprise architects and industry solution providers alike.

(5)

Section 2: Opportunity

Standardizing solution engineering

The increasing need for IT organizations to deliver integrated solutions more reliably and quickly creates an opportunity and an imperative to apply best practices and technology to simplify solution engineering and maintenance.

To engineer and deliver integrated solutions that are truly quick to build yet reliable and supportable, there needs to be standardization in the way that:

• Data and capabilities of each product are represented and exposed in terms of schema, semantics, transport protocols and programming model in general

• Integrated solutions are engineered and maintained CA Catalyst enables this standardization.

CA Catalyst

CA Catalyst is a set of technology components to standardize the mechanism by which: • CA Technologies makes its products easily integrable

• CA Technologies and its partners make 3rd-party products more easily integrable

• CA Technologies and its partners construct solutions involving integration of CA Technologies and 3rd-party products

unified Service Model

Foundational to CA Catalyst is a common, extensible data schema called the Unified Service Model (USM). The USM Schema, introduced by CA Technologies several years ago defines types (e.g.

ComputerSystem, Router, Person, ProvisionedSoftware, etc.) and the attributes that define instances of each type. USM also defines operations that can be performed on type instances (e.g. “escalate” is an operation that can be performed on an instance of the type “incident”) and other types of data and metadata that are relevant to the purposes of solution engineering.

The USM schema is the common language of CA Catalyst-based integration. One of the defining characteristics of a CA Catalyst-enabled product is that it offers a programmatic interface that exposes its data and capabilities normalized to a USM representation, making the product itself, in effect, an instantiation of the USM schema.

CA Catalyst also provides a number of integration services that consume the USM instances exposed by CA Catalyst-enabled products and act on them to implement various data-level integration use cases. The USM schema draws heavily on the prevailing standards that exist for modeling data in more specific domains (e.g. CMDBF, OVF, SID, CIM, etc.) and provides a consistent and comprehensive schema spanning those domains. By basing the USM design heavily on existing standards, USM inherits much of the best practice thinking that has gone into those standards, as well as the property of requiring very little processing to transform data between any of these domain specific standard representations and a USM representation.

(6)

The USM schema is also inherently extensible. This allows it to be extended to enable exchange and integration of data that is not part of the core USM schema definitions, such as line of business data. The extension of USM for such cases is done through CA Catalyst configuration.

The USM schema is openly published by CA Technologies and is freely available on our CA Catalyst Community Site.

Integration scenarios supported by CA Catalyst services

Some integration use cases are common and others are quite specific to individual customers. They are many and varied, ranging from the relatively trivial to the incredibly complex. To provide a brief overview, it is useful to consider a series of generalized use cases or as we call them, integration scenarios. These will also serve as a frame of reference for discussing some of the more important CA Catalyst services. CA Catalyst is focused on addressing the following nine generalized integration scenarios. The first two deal purely with making products more easily integratable; the remainder deal with constructing integrations.

Scenario 0—Product Interoperability through WS-Management

WS-Management is a SOAP-based protocol for exchanging management information between IT management systems, and is defined as a standard by the DMTF. A basic requirement for an IT management product to be easily integrable is full compliance with WS-Management and related WS standards. This ensures a basic level of interoperability with many WS-Management compliant clients such as process automation tools or integration middleware including, but not limited to, the CA Catalyst integration services.

Figure A

USM Schema: HTML documentation example

(7)

One example of this scenario is enabling IT management products from CA Technologies and others that do not currently fully support WS-Management, to be easily integrated using Oracle Fusion middleware. CA Catalyst addresses this scenario by providing a common transport binding layer that exposes each CA Catalyst-enabled product over WS-Management. CA Catalyst enabling a product involves writing a CA Catalyst connector that exposes the product’s data and capabilities in a USM representation using a Java-based API. This connector is installed inside a CA Catalyst Container. The container provides the transport binding from the Java API of the connector, exposing it on the wire via WS-Management.

Scenario 1—Access to Product Data and Capabilities via REST

Representational State Transfer (REST) is a type of software architecture for remotely accessing and updating resources managed by some component. It imposes certain constraints. Product APIs (resource managers) conforming to these constraints are referred to as RESTful. A popular and typical use of RESTful interfaces is to surface data and capabilities directly to a web browser through portal gadgets or mashup-style UI components. Unlike WS-Management, REST is implicitly synchronous and stateless, which makes it inappropriate for supporting many integration scenarios, but its simplicity makes it ideal for others. As with WS-Management, support for REST is a critical requirement for making products easily integrateable. One example of the use of RESTful services is to expose the capabilities of CA Service Desk Manager in a portal gadget, mashup or mobile device UI component.

REST differs significantly from WS-Management, but CA Catalyst addresses the need for products to support REST-based interoperability in exactly the same way as it supports WS-Management, (i.e. the CA Catalyst container provides a common implementation of the required transport bindings to expose the APIs of CA Catalyst-enabled products over REST).

Figure B

Scenario 1: access to product data and capabilities via REST

CA Service Desk Manager

CA Clarity™ Project and Portfolio Management

Line of Business Apps

Access to data and capabilities via REST

Mashups/ Portal Gadgets

(8)

The prevous diagram illustrates Scenario 1. The diagram would be the same for Scenario 0 with the access to data being via WS-Management instead of REST and the end usage being another system or middleware (e.g. Oracle Fusion), rather than a portal or mashup container. While the diagram does not make it explicit, REST and WS-Management can both surface data from the integrated products and call operations on them. For example via a REST interface, a portlet could be surfaced that exposes “My Open Approvals” along with the ability to take certain actions on them, such as “Approve”, “Reject”, “View More Info”. The first two of these would directly invoke the related action on the relevant application via REST and the latter might access another URI to view the additional information. This is therefore a powerful way to surface task-specific data and capabilities for enabling solution use cases without necessarily requiring access to the full UIs for the underlying products. This is also very useful for enabling a solution UI to be surfaced on mobile devices.

The value of CA Catalyst in these first two scenarios is to standardize the data model and the implementation of the REST and WS-Management interfaces across all CA Technologies products and CA Catalyst-enabled 3rd-party products.

Scenario 2—One way data feed between one product and another

In this scenario, the requirement is to take data from one product and use it to create or update data in another product. The integration is one way and does not update the source system in any way. This can be achieved with an Extract-Transform-Load (ETL) approach in many cases, but the difficult aspect is the Transform step. Real world examples of this relatively simple scenario are often dogged with spurious duplicate entries for data objects being created in the target system.

Examples of this scenario include feeding data from a discovery tool (e.g. CA Configuration Automation) to a CMDB or feeding alerts from domain specific monitoring tools (e.g. CA Insight™ Database

Performance Monitor for Distributed Databases) to a higher level monitoring tool (e.g. CA Spectrum®). CA Catalyst makes this simpler to achieve, in two ways. First, once the two products are CA Catalyst enabled, they expose their APIs in the common schema and semantics defined by USM which reduces the amount of “transform” logic required in the integration itself. CA Catalyst also provides a policy-configurable correlation service to detect when a data object from the source product already exists in the target product (therefore requiring update rather than create). The logic for the integration then becomes expressed as correlation policy executed by the CA Catalyst correlation service (i.e. configuration) rather than custom scripting (i.e. custom code).

This does not mean the need for ETL tools goes away entirely, though in many cases it may. CA Catalyst considerably simplifies the task and offloads much of the processing such that the need for complex logic in the ETL tool is reduced substantially.

Scenario 3—unified view of data from multiple products

This scenario requires the construction of a Unified View of the data exposed by several underlying products, where unification applies when:

• The same object (e.g. a router) is surfaced by multiple underlying systems. It appears only once in the Unified View, with the rules for determining the “sameness” of entities across systems being defined

(9)

• Underlying systems differ in their representation of the state of a correlated object. In this case, reconciliation policy governs how the Unified View represents the object.

CA Catalyst provides correlation and reconciliation services, both configurable by policy, to construct and expose a Unified View. The Unified View is actually itself exposed through the CA Catalyst API with the same USM schema and programming model exhibited by CA Catalyst-enabled products. In this way, the Unified View becomes an instance of the USM schema accessible in exactly the same way as CA Catalyst-enabled products.

CA Service Operations Insight (CA SOI), formerly known as CA Spectrum Service Assurance, is one such product that implements this scenario. It presents a real-time operational service-oriented dashboard— federating, correlating and reconciling data feeds from a wide range of CA Technologies and 3rd-party service modeling and infrastructure monitoring tools. CA Spectrum Service Assurance is built on a foundation of CA Catalyst integration services.

Figure C

Scenario 3: CA SOI embeds and uses CA Catalyst to integrate data feeds from multiple monitoring tools to construct a Unified View of the service model. It then surfaces this through its UI.

In this illustration, alerts from CA Configuration Automation, CA Introscope®, CA Spectrum and CA Service Desk Manager relating to this service and its component CIs are shown in a single pane, made possible by the capabilities of CA Catalyst to do CI correlation and reconciliation across products.

Scenario 4—Synchronization of products based on the unified view and synchronization policy

This scenario extends Scenario 3 to also update data in the underlying products based on the “accepted truth” view of the service model represented by the Unified View. What exactly gets updated where and when is defined by synchronization policy.

An example of this scenario is synchronization of a CMDB and infrastructure monitoring/discovery products such as CA Spectrum, or other scenarios where multiple products maintain their own representation of certain data elements (e.g. service model CIs and relationships) that needs to be kept consistent.

(10)

Scenario 5—Process Automation via a unified view / Federated Configuration Management System (CMS)

This scenario requires the ability of a Unified View exposed by CA Catalyst integration services to be updatable (i.e. to not only surface data but to allow operations to be invoked on that data and to be directed to the appropriate underlying system or systems for execution, via appropriate federation of operations). A process automation tool can of course be configured to directly connect to an instance of say, CA Service Desk Manager or it could instead code against the CA Catalyst Unified View. There will be situations where the former is the right approach, especially if the complexity of the solution is quite low and the overhead of constructing a Unified View is not warranted. However, in more complex situations, there is an advantage to both accessing data and invoking operations via the Unified View, since it abstracts the process author from needing to be aware of the specifics of the API’s of the underlying systems. In the event that at some later stage one of the underlying systems is significantly upgraded or even replaced with a different product entirely, the solution logic in the form of the process automation content would not need to change. The change would be in CA Catalyst

configuration to tell the Unified View that, say, “Incident Management” now lives somewhere else. In fact, USM defines “profiles” to enable precisely this sort of abstraction. For instance, the USM Incident Management Profile defines the set of types and operations that must be implemented for a connector to claim it supports “Incident Management”. Process authors can then code against the types, metrics and operations in the USM Incident Management Profile, and not concern themselves with what system is providing that functionality. Switching to a new system would simply require an update to the configuration of CA Catalyst and of course a connector for the new system that implements the relevant profile. This is quite a powerful concept with regards to insulating the integrity of solution/ integration logic from changes to the underlying integrated products, whether those changes are patches, version upgrades or complete replacement with an entirely different product.

Many integration scenarios, including service automation, service portfolio management, application release management, private cloud management and many others fall into this category, where a combination of data integration to compose and expose an updatable model, and automated processes to drive it, are required.

Further scenarios

Beyond the scope of this paper are 3 additional scenarios that are driving the evolution of CA Catalyst. These pertain to enabling the:

• Publishing, handling and aggregation of performance and other metrics

• Integration of multi-tenant cloud-hosted management capabilities with systems at multiple premises which may or may not be tenant-aware (this requires tenant-aware integration between premises, across networks)

• Creation of a true Service Knowledge Management System (SKMS), providing everything required to construct a federated Configuration Management System (CMS). A CMS is a critical subset of the larger concept of an SKMS per ITIL® V3. Creating an SKMS requires additional knowledge processing functionality such as inferencing capability to reason over the Unified View’s data and deduce

(11)

data and capabilities in portals and other UI frameworks, and so forth. This is really about enabling next generation model-based IT management, where the model not only integrates data and

capability from underlying systems, but also encapsulates the knowledge and best practices of the IT organization, codified in the form of policy and automation.

Key CA Catalyst services

As discussed above, CA Catalyst provides a series of integration services that can be configured through policy to implement integration logic. The framework is fundamentally extensible to allow additional services to be developed by CA Technologies or partners to extend the CA Catalyst capabilities over time, to address specific use cases.

The following key services are however particularly important due to their relevance to a large number of integration scenarios and are worth a little further description here:

• Correlation processes objects from various data sources to determine when they represent the same real world object (e.g. a router). In some cases, correlation rules can be as simple as a string match of a single property such as DNS name, but in other cases, correlation rules can be relatively complex. The ability to do configurable correlation can be used in a very lightweight way to avoid erroneous data duplication issues during simple transfer of data from one system to another (e.g. discovery tools feeding a CMDB). It is also a pre-requisite service for constructing a Unified View in order to ensure that the Unified View only represents each real world object once.

• Reconciliation processes representations of correlated objects coming from multiple data sources to determine what should be accepted as the definition of “truth” in the event that the data sources hold conflicting information with regards to one or more of the object’s attributes. This is configurable through policy. • Synchronization processes objects across multiple source systems based on the reconciled version of

their state by updating the relevant underlying system(s) to match the reconciled state of the object, as well as propagating newly created objects across multiple synchronized products.

• Federated Query across multiple CA Catalyst-enabled products to submit a query in all relevant places, and return a coordinated, reconciled and correlated result set.

CA Catalyst architecture and deployment

The target scenarios for CA Catalyst are extremely varied and demand a flexible modular architecture to enable the deployment of only those services that are required to achieve the integration use cases for a specific deployment.

To achieve this flexibility, the architecture utilizes a CA Catalyst module that provides the basic CA Catalyst infrastructure components. This on its own is very lightweight. Everything else (connectors, integration services etc.) are optional components that can be deployed as and when needed inside one or more instances of the CA Catalyst Container. Hence, the configuration of CA Catalyst as required by Scenarios 0 and 1 is very simple and lightweight, while the deployments for the later scenarios require increasingly complex deployments of CA Catalyst functionality. In the simpler cases the CA Catalyst configuration could even run entirely inside CA Catalyst software modules embedded in CA Technologies product instances, requiring no additional hardware configuration. For more complex cases, standalone modules on separate machines (virtual or otherwise) may be required.

(12)

CA Process Automation

As an example, CA Process Automation is a CA Catalyst-enabled technology providing orchestration, integration and automation of work instructions for IT operational processes, enabling companies to define and standardize best practices and improve operational efficiency.

It is available as a standalone product for doing Run Book Automation. It is also embedded as a workflow engine in several CA Technologies products. It supports CA Catalyst in two main ways: • It is CA Catalyst-enabled (i.e., it exposes its own data and capabilities through CA Catalyst, such as

the ability to initiate a process). Note: the USM schema describes processes as well as CIs and many other types of data related to IT management.

• It is a CA Catalyst client (i.e. it can connect to 3rd-party products via the CA Catalyst infrastructure and connectors). It can also connect to and drive a CA Catalyst Unified View.

Figure D

CA Process Automation can access the data and invoke the capabilities exposed by IT

management products or CA Catalyst Unified Views, via CA Catalyst infrastructure.

The screenshot above shows a process flow driving functionality in CA Configuration Automation via CA Catalyst.

(13)

Section 3: Benefits

Reliable, supportable, rapidly-deployed,

cost-effective solutions

The benefits of buying an integrated solution from CA Technologies or one of its partners, based on CA Catalyst integration technology, include:

• More reliable solutions due to standardization in the way integrations are delivered, reducing the

risk of solution integrity being undermined by patches or upgrades to underlying products, and increasing the ability to rapidly triage and determine root cause as and when problems arise.

• Rapid time-to-value due to the delivery of integration logic in the form of automated process content

and integration policy, leveraging common process and policy libraries, rather than custom code. • Attractive total cost of ownership due to more efficient use of services to construct integrations as

part of initial deployments, a lower likelihood of needing further services engagements to maintain the integration over time, and a lower likelihood of significant support issues.

Policy-governed, automation-driven, model-based IT management

CA Catalyst provides the ability to integrate many sources of information across the entire set of IT management domains. It does this by constructing a Unified View (an instance of the Unified Service Model representing the single source of truth for the IT infrastructure), as defined by correlation and reconciliation policy. This model can then be driven by and/or drive automated IT management processes that encapsulate the organization’s knowledge and best practice, as an integral part of the model. This opens the door to a new paradigm of process- and service- oriented IT management, driven by the integrated model, rather than traditional siloed functional management enabled by domain specific tools. This will lead to the benefit of much more direct alignment between the way IT is managed and the business objectives it supports, through the services it provides.

(14)

Section 4:

Conclusions

CA Catalyst is part of our integration strategy to address the challenges our customers experience today when deploying solutions requiring multiple products.

It is about enabling CA Technologies and its partners to deliver more reliable, and supportable solutions, with faster time to value and lower total cost of ownership for its clients.

The Integration Strategy from CA Technologies is not just about doing things we do today better, it is about creating the capability to do new things that open new frontiers in IT management. Through CA Catalyst and CA Process Automation, we enable pervasive integration across the IT management domains, not just within small pockets of capability such as Service Management. This enables a comprehensive operational service model to be constructed based on data from Infrastructure Monitoring, Application Monitoring, Workload Scheduling, Process Automation, Service Management, Project Portfolio Management, Asset Portfolio Management, Mainframe Management, and all other aspects of IT management. It opens the door to real model-based management, based on a model defined by policy and driven by automation. This constitutes the realization of a federated Configuration Management System or CMS per ITIL V3. This CMS in turn provides a platform on which an enterprise can construct automated solutions that, in the aggregate, constitute a Service Knowledge Management System or SKMS, again referencing ITIL V3. Finally, it is important to reiterate that CA Catalyst is not just for CA Technologies products, solutions need to work across a combination of CA Technologies and 3rd party products. CA Technologies is investing in building connectors for many of these, working actively with the ISV community and our partners generally to build out their support for CA Catalyst. Many important line of business applications are developed in house by our customers, CA Catalyst provides SDKs to enable connectors to be built to integrate these applications and provides the ability to extend our USM schema to enable line of business data to be shared via CA Catalyst.

(15)

Section 5:

References

CA Catalyst Community Site for USM Schema and related technical information:

https://communities.ca.com/web/ca-usm-catalyst-global-user-community/welcome

Basic Information regarding REST:

http://en.wikipedia.org/wiki/Representational_State_Transfer

Basic Information regarding WS-Management:

http://www.dmtf.org/standards/wsman

ITIL V3 Site. For definitions of CMS and SKMS concepts:

http://www.itil-officialsite.com/home/home.asp

Information regarding CA Spectrum Service Assurance, a product built on CA Catalyst integration technology:

http://www.ca.com/products/detail/CA-Spectrum-Service-Assurance.aspx

Information regarding CA Process Automation, a CA Catalyst-enabled product:

http://www.ca.com/products/detail/CA-IT-Process-Automation-Manager.aspx

Section 6:

About the author

Stephen Tyler is the Product Manager for the CA Catalyst program at CA Technologies. He has over 20 years experience, much of it as a software developer, architect and technical leader across a wide range of technology platforms and industry verticals. Since joining CA Technologies in 2006, Stephen ran R&D for 2 products in the Service Management portfolio prior to taking a strategy role in Product Management working on product, business unit and corporate strategy initiatives. He is ITIL V3 certified.

(16)

Copyright © 2011 CA. All rights reserved. All trademarks, trade names, service marks and logos referenced herein belong to their respective companies. ITIL® is a Registered Trademark and a Registered Community Trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. This document is for your informational purposes only. CA assumes no responsibility for the accuracy or completeness of the information. To the extent permitted by applicable law, CA