Technical Whit
e P
aper
webMethods Insight for webMethods
Integration Server
Gain Real-Time Visibility and Minimize Disruptions
Ann Marie Bond
Systems Engineer, Software AG
COnTEnTS
introduction
4
Meeting the challenges of transaction ManageMent
5
webMethods insight: how it works
7
webMethods insight: technical details
12
webMethods insight: architecture advantages
14
INTRODUCTION
it organizations that use webMethods integration server have access to the most complete integration infrastructure in the market. its ability to tie together existing applications and partners, and to enable developers to efficiently develop new applications, provides busi-nesses with an agility that’s hard to match. now, webMethods integration server users can take even more control over today’s heterogeneous it environment with webMethods insighttM.
webMethods insight is a business transaction management tool that provides real-time visibility into every business transaction across every siloed system and application that touches the webMethods integration servertM. with webMethods insight you can quickly pinpoint business and technical exceptions and disruptions, identify root causes, and fix problems.
webMethods insight not only reduces the time and costs of troubleshooting, it also short-circuits the blame game that typically takes place when problems occur. and it enables it to be an even better partner to the business by maintaining the highest availability of intercon-nected customer-facing systems and applications.
in this white paper you will learn:
how webMethods insight addresses the key challenges of running an integration
infra-•
structure
the capabilities and benefits webMethods insight provides for integration teams and
•
business application owners
how the webMethods insight server and the webMethods integration server work
•
together to provide unparalleled visibility across a heterogeneous environment
how the architecture of webMethods insight provides superior performance, security, and
•
visibility
fast to deploy, with lower overhead and lower total cost of ownership, webMethods insight is lightweight and non-invasive. it provides a proven, best-of-breed solution for monitoring sla management and for troubleshooting services and business transactions. together, webMethods integration server and webMethods insight set the standard for the combined power of integration and end-to-end business transaction management.
With webMethods Insight you
can quickly pinpoint business
and technical exceptions and
disruptions, identify root
causes, and fix problems.
Together, webMethods
Integra-tion Server and webMethods
Insight set the standard for the
combined power of integration
and end-to-end business
transaction management.
MEETING THE CHALLENGES OF TRANSACTION MANAGEMENT
as an integration team member or business application owner, you’re expected to have more transparency and accountability than ever for your distributed and heterogeneous environ-ments. webMethods integration server has provided the application integration capabilities, high-speed messaging, support for b2b partner integration, legacy applications and event-driven technologies needed to tie together the many disparate components of these environ-ments. you know you can rely on webMethods integration server to optimally link your siloed applications, orchestrate services and expose data for internal and external consumers (figure 1). now, with webMethods insight, you can take the performance of your webMethods integration server to the next level.
by providing real-time visibility into every business transaction across every siloed system and application that touches the webMethods integration server, webMethods insight helps you address the key challenges of integration management:
Uptime
•
: Mission-critical applications must not only be up, but also functioning properly all the time. a webMethods integration server application can be dependent on packaged applications, databases, even applications outside the firewall. when problems arise it is costly, inefficient, and time-consuming to identify the problem’s source and correlate pieces of transactional information. research shows it can take up to 20 people about six hours each to perform root-cause analysis, fix the problem and test the resolution.1 but with the deep visibility into service transactions and automatic transaction correlation provided by webMethods insight, you can quickly react to, identify, and resolve the root cause of problems, and thereby:> reduce the time/costs of problem-solving by 85% > improve problem avoidance by 25%
> increase your team’s productivity Performance
•
: developers and operators need to be able to understand what’s going on both at design-time and at run-time. with webMethods insight, you know how transac-tions flow through webMethods integration server and where the bottlenecks are that might affect your consumers. you know how many calls you’re making or how much data is being passed. you can aggregate statistics across an end-to-end transaction or view time-bounded “snapshots” with average response times.1 source: forrester consulting: the total economic impacttM of Progress actional Management for interconnected
applications, february, 2009.
webMethods Integration Server
Custom & Packaged Applications Databases B2B Trading Partners Software As a Service (SaaS) Mainframe & Legacy Systems C t & B2BTrading S ft A M i f D t b
Figure 1. webMethods Integration server links your siloed applications, orchestrates services and exposes data for inernal and external consumers.
Research shows it can take up
to 20 people about six hours
each to perform root-cause
analysis, fix the problem and
test the resolution. But with the
deep visibility into service
transactions and automatic
transaction correlation provided
by webMethods Insight, you
can quickly react to, identify,
and resolve the root cause of
problems.
With webMethods Insight, you
can take the performance of
your webMethods Integration
Server to the next level:
Implement error handling
•
framework without coding
Isolate performance issues
•
non-invasively
Find problems before
•
encountering them
Create run-time
•
dependency tree
Generate alerts
•
Manage SLAs
•
Capture payload and
•
Governance and Compliance
•
: governing your ecosystem and monitoring compliance is difficult without the right level of visibility. with webMethods insight, you can see who consumes the services offered by your webMethods integration server and how. you can identify unexpected or “rogue” consumers of your esb services and ensure that applica-tions are operating in compliance with organizational processes and policies.Chargeback
•
: organizations commit over 80% of their it budget to run, debug, and maintain their systems, applications and services. as organizations grow service-oriented architecture (soa) adoption or complex distributed infrastructures, the ability to implement utilization chargebacks becomes even more challenging. webMethods insight collects and aggregates transaction and service usage to make chargebacks easy, accurate and defensible.Rapid Development
•
: webMethods insight enables you to reduce errors and maximize productivity during development. for example, you could avoid common mistakes like promoting a webMethods integration server package to production that’s still calling a development service or database. you can also analyze the performance of services while still in development, when problems can be addressed easily.Planning
•
: webMethods insight helps you understand the current requirements of the webMethods integration server from an it standpoint and extrapolate future requirements. historical performance metrics enable you to analyze the effectiveness of your server infrastructure and recommend changes that will improve performance and business results.Key Benefits
for integration teams and application owners, the benefits that webMethods insight offers and the capabilities that it puts in their hands are unsurpassed:
Provides Deep Visibility
gain insight into transaction and service performance
•
automatically discover calling patterns and dependencies
•
discover and manage service transactions
•
Minimizes Disruptions
actively manage transactions and services with detailed real-time metrics
•
implement alerts or alarms to prevent performance problems before you reach sla limits
•
isolate and diagnose the root cause of problems more quickly
•
Ensures Compliance
track and enforce compliance with governance policies
•
detect rogue services and bring them into the governance process
•
ensure design-time and run-time policies are not avoided or evaded
•
with webMethods insight there are no more lengthy troubleshooting sessions or manual looking at logs and laboriously trying to correlate transactions. Just as importantly,
webMethods insight puts an end to finger-pointing, which typically finds the integration team at the center of an unproductive blame game. webMethods insight changes the conversation instantly, transforming the people who have been unfairly regarded as the problem into the solution.
webMethods Insight helps you
meet the challenges of
transac-tion management:
Increase uptime Improve performance Establish governance Manage chargebacks Develop rapidly Effective planningWEBMETHODS INSIGHT: HOW IT WORKS
the webMethods insight network overview in figure 2, a screen shot from webMethods insight, shows a complete picture of an organization’s it infrastructure. webMethods insight will dynamically create updates. when new systems are deployed webMethods insight detects and displays them, making obsolete the huge network diagrams hanging on walls in it shops. in this example, two webMethods integration server instances are at the center of a typical implementation, including packaged applications, an application server, a database, and more. the webMethods integration server instances are orchestrating services across saP, layer 7, oracle, .net, and a custom portal application. the other servers have been auto-detected as consumers or providers of services on these managed nodes.
Figure 2. webMethods Insight network overview: In this example, two webMethods Integration server instances are at the center of a typical implementation, including packaged applications, an application server, a database, and more.
Two primary elements of webMethods Insight
webMethods Insight Server
the server is installed on a standalone system and is highly scalable. the server runs standalone or in any number of application platforms. it requires a database and supports many popular databases. figure 3 shows how policies are distributed from the server to the agents and statistics and alerts reported back.
webMethods Insight Agent
the agent is installed on a platform, turning it into an instrumented platform. the term “instrumented” means that the platform can be managed by the server. webMethods insight is lightweight and non-invasive, so neither installation nor operation of the agent requires making changes to the application.
Instrumented Platform Components
an agent is made up of two types of components: at least one interceptor and an analyzer. an interceptor2 is a lightweight component that plugs into a particular protocol stack or applica-tion platform to gather informaapplica-tion about calls being passed by the system. interceptors are specific to the application that they instrument. webMethods integration server and a number of other protocol stacks and platforms are currently supported. there is one interceptor for each application that is being tracked, or managed. an analyzer processes events by reading them from the in-memory buffer, called the “flight data recorder”.
webMethods insight agents plug into the container, leveraging the normal container parsing of XMl. because the agents are running in the container, the applications within do not need to be modified. what’s more, the container is session/thread aware and can use the container’s session context to correlate across the session as it goes from input on one protocol to one or more output calls on one or more protocols. webMethods insight provides support for multiple protocols. all this adds up to much better performance, auto-discovery of services, automatic correlation across transactions, and no impact to applications.
Figure 3. webMethods Insight server-Agent Communication: Policies are distributed from the server to the agent and statistics and alerts are reported back.
An interceptor is a lightweight
component that plugs into a
particular protocol stack or
application platform to gather
information about calls being
passed by the system.
Interceptors are specific to
the application that they
instrument. Integration Server
and a number of other protocol
stacks and platforms are
currently supported.
webMethods Insight is
light-weight and non-invasive, so
neither installation nor
opera-tion of the agent requires
making changes to the
application.
SLA Management and Root Cause Isolation
service level agreements (sla) and transaction level agreements (tla) in webMethods insight can be defined to detect a range of issues from pure outages to non-responding transactions to performance slowdowns to business exceptions. when sla violations are reported, users can review the related transaction in a visual way by tracing through each step to isolate the root cause. webMethods insight collects and automatically correlates data across all webMethods integration server instances and 3rd-party platforms, extending out to consumers or providers “one hop” away from managed nodes. webMethods insight provides automatic correlation that requires no changes to payload – it uses a real-time decentralized process. during this process, it can capture a payload transaction snapshot for root-cause analysis.
the webMethods insight agent provides this information by buffering in-flight transactions in memory as the transactions flow through the enterprise. if there is a policy/sla violation, the server gathers transaction history from all the involved agents for the specific problem transaction. collecting the full transaction history only on-demand greatly reduces the amount of information the agent has to share with the server. however, the sliding window of buffered data maintained by the agent ensures that a complete set of transaction statistics will be available when needed for an end-to-end analysis of problem transactions. operations teams can then drill into reported violations and evaluate both the content and the context of a transaction to take action on problems and minimize disruptions to consumers.
Figure 4. webMethods Insight Root Cause Analysis: This screen shows an alert generated by an sLA.
Dashboards
all the metrics collected by webMethods insight can be aggregated, filtered and organized to create custom dashboards to provide management and accountability. by default, transac-tional metrics such as soaP faults, service response time and data transmitted are collected. administrators can also request pipeline information from webMethods integration server calls or web services and combine them with transactional metrics to build chart dimensions that reflect mission or business transaction criteria. operators, administrators, project manag-ers and architects may each have different dashboards focused on their requirements, as shown in figure 5 below.
Figure 5. webMethods Insight server Dashboard: operators, administrators, project managers and architects may each have different dashboards focused on their requirements.
Deep And Wide Visibility
webMethods insight’s powerful capabilities provide unparalleled visibility into webMethods integration server and beyond, including:
Internal Visibility
•
: webMethods insight traces application transactions from consumer to provider across all webMethods integration server protocols, eliminating costly manual searches and logging on multiple machines. even complex implementations can be traced as they execute and follow application logic. and because webMethods insight collects performance metrics for all service invocations, it’s easy to determine how long a service takes at run-time or how much data it’s transmitting.Cross-Platform Visibility
•
: automatic correlation of transactions across many common application platforms enables end-to-end monitoring and management of transactions that reach beyond webMethods integration server. with a visual representation of a transaction linked to performance metrics, debugging multi-level, cross-platform transac-tions is much easier.Automatically Discovered Services
•
: webMethods insight runs in the webMethodsintegration server container, not in the application, so it automatically detects whatever services are running.
“One-hop” Visibility
•
: webMethods insight tracks both service consumers and outbound requests. so when webMethods insight is installed on a single webMethods integration server it will discover all inbound and outbound service requests and expand your visibility without requiring additional agents.Reconnecting Flows:
•
webMethods insight provides support for auto-correlation across an un-instrumental node which is sandwiched between two instrumental nodes.Data Capture
•
: automatic capture of performance data and pipeline data fromwebMethods integration server provides a powerful debugging and analysis tool. you can: drill into pipeline content and service context for quick root cause analysis.
•
use historical performance data to aggregate metrics across transactions, servers,
•
time and many other dimensions.
Pull out pieces of message data from the pipeline and aggregate them to derive
•
mission-oriented metrics. Service-Level Agreements (SLAs)
•
: webMethods insight has the ability to detect bothcontent issues via pipeline data capture from webMethods integration server services and technical exceptions via infrastructure metrics. slas can be set on technical exceptions and mission-oriented exceptions.
WEBMETHODS INSIGHT: TECHNICAL DETAILS
The Role of Interceptor Plug-in for Monitoring
webMethods insight is the only service and transaction monitoring product designed specifically for webMethods integration server. the interceptor for webMethods integration server is the critical monitoring component of webMethods insight’s architecture and has been designed to work seamlessly with webMethods integration server. the interceptor for
webMethods integration server provides unique visibility into flow services, with automatic discovery and dynamic handling of flow changes at run-time. by instrumenting the webMethods integration server container, webMethods insight requires no changes to applications. options with the installation include deciding which protocols, services, and packages to show or hide to fine-tune the network and service path views.
Visibility
the webMethods integration server interceptor has visibility into servlet traffic hosted in the tomcat package, which hosts JsP and dsP pages. it also tracks Jdbc and JMs traffic going in/out of the webMethods integration server. and it records all httP client traffic going out of webMethods integration server.
the interceptor shows service traffic between: Java services • XML services
•
Internal services • XSLT services•
Flat file services • JMS trigger services•
MIME/SMIME services • Flow services•
the interceptor also correlates inbound requests through soaP via the web services handler (/ws/...) and the built-in soaP handler (/soap and /soap/rpc). for outbound requests, the interceptor collects the service, operation, steps and metrics for services using the httP, soaP, ftP, sMtP and remote invoke protocols.
Instrumentation
the interceptor implements a style of instrumentation called “staged classloading” whereby certain interceptors (Jdbc, servlet, axis) are only deployed in the webMethods integration server package classloader, while others (such as the base and JMs interceptors) are deployed in the server’s main classloader. this is done in order to avoid intercepting purely internal calls.
Protocols
webMethods integration server protocols that are instrumented by the interceptor include soaP, httP, eJb, Jdbc, rMi, axis and JMs. the interceptor supports webMethods integration server versions 6.5, 7.1.1 and 7.1.2.
Fine-Tuning the Display
the webMethods integration server transaction Path (figure 6) shows a typical detailed path view. Path displays can be filtered and adjusted to focus on key transactions in several different ways:
webMethods integration server packages can be omitted from reporting.
•
specific services can be omitted from reporting.
•
flow steps are not reported by default. however, they can be enabled and displayed for
•
more detailed analysis.
system-wide visibility of Jdbc adapter database transactions can be enabled.
•
Installation
webMethods insight is installed by instrumenting the webMethods integration server container and is non-invasive. it requires no customization or application changes.
Figure 6. Flow service visibility: This screen shows the path taken by the transaction as it calls one flow-service after another. The Interceptor is what makes this happen.
WEBMETHODS INSIGHT: ARCHITECTURE ADVANTAGES
webMethods insight is built from the ground up for enterprise-level deployments and is designed to manage mission-critical distributed applications like those handled by webMethods integration server.
Unique Visibility into Integrations at Multiple Protocol Levels
integration services do not depend only on httP(s) and soaP traffic but incorporate a multitude of protocols such as JMs, rMi, Jdbc, and with webMethods integration server, XMl-based proprietary protocols as well. regardless of the protocols used, webMethods insight can provide a complete end-to-end view of a webMethods integration server network – something no other management solution can do. what’s more, this network can be extended to many other applications and protocols, seamlessly tracking applications across platforms.
Designed and Architected for Performance, Security and Visibility
Automatic service discovery: webMethods insight can automatically discover service producers, service consumers, and the dependencies between them. this auto-discovered interconnection can also be viewed graphically. Message flows between nodes are identified down to the operation level. in addition, there is no need to explicitly configure the services in advance for them to be discovered or managed. simply installing the webMethods insight agent enables operators to deploy policy that takes effect on the first service call to the node. High scalability: webMethods insight’s architecture uniquely blends distributed policy evaluation with centralized command and control, enabling very high scalability. the webMethods insight management server is designed to be able to handle at least 1,000 managed nodes and 50,000 service interdependencies. also, the server is never a single point of failure. Policies are evaluated at the agents, so a server failure does not affect policy evaluation.
High performance with minimum impact: with less than 50 microseconds of latency added to each message, webMethods insight has the highest performance of any solution on the market. this excellent performance enables webMethods insight to be run all the time in production. in addition, because the server is never “in the flow of messages,” the server or network will never detract from management performance. on-site tests have shown that it is possible to stay under 2% of cPu utilization and to achieve message throughput of more than 3.5 million messages per hour on webMethods insight-managed nodes – even though this is where policy processing occurs.
Rapid Deployment with no Application Changes
using webMethods insight to manage an integration ecosystem does not require any application changes. webMethods insight can be installed into an existing application environment without modifications to the application. additionally, there is no need to deploy webMethods insight agents everywhere to take advantage of the product; even a single instrumented (managed) node can bring value because of the automatic discovery of consumers and services “one hop” away from each instrumented node.
KEY ADVANTAGES
Rapid Deployment
easy to install and manage using
•
webMethods insight server console deploying agents requires no impact
•
on your applications and services
Smart Monitoring
dynamic auto-correlation without
•
modifying the soa environment agents give you “one-hop” visibility,
•
uncovering underlying service dependencies
Minimal Overhead
Minimal overhead even on
high-•
volume transaction environments “snapshot” problem transactions
•
ABOUT THE AUTHOR:
Ann Marie Bond is a systems Engineer with the soA Practice at software Ag. she has been working with
webMethods Insight since the inception of the product to provide technical expertise for sales. Prior to working in the sales organization, Ms. Bond led the Integration Competency Center at webMethods, using webMethods’ own software to solve business problems. Ms. Bond is a certified Zapthink soA Architect and has a PMP. she also worked in software development for 14 years on projects ranging in size from very small to over 400 developers. Ms. Bond has a B.s. in Computer science from george Mason University.
CONCLUSION: BENEFITTING THE ENTIRE COMMUNITY OF USERS
webMethods insight provides extraordinary visibility, management, and control in a very efficient way to a broad spectrum of users:
Business Managers: they benefit from the ability to extract payload or pipeline information as it passes through the webMethods integration server. this expands the management capabilities of the platform to analysis at a business transaction level. while it’s certainly possible to install network sniffers and collect data from traffic passing through a network, correlating and modeling that data in business context for business users is nearly impossible. Architects: implementing webMethods insight allows architects to transform from the point-to-point, or just-a-bunch-of-services structures that are often found in webMethods integration server implementations toward a true soa. using webMethods insight, architects get the visibility into services, complete transactions and run-time dependencies. this gives them a greater awareness into what is really happening in their enterprise and enables them to optimize their plans for maturing soa strategically. what’s more, information from web-Methods insight can be transferred directly into centrasite activesoa or another registry/ repository, empowering architects to define policies and lifecycles that would return the benefits of a disciplined, governed service-oriented architecture.
Operations and IT Managers: it responsiveness is critical to their ability to support the business side of the house. webMethods insight gives the it operations team the ability to detect problems quickly and instantly and to accurately isolate the root cause, and prevent some problems before they happen. additionally, webMethods insight is an invaluable tool for proper capacity planning with soa, since capacity requirements can vary for each service consumer and provider. operations and security Managers will also appreciate the quick identification of “rogue” services.
Developers: webMethods insight enables developers to gain deeper knowledge into how their services or application behave and execute in the context of a business transaction along with all interconnected dependencies. Performance benchmarking can now be extended to the transaction level with performance statistics shown for each invocation or hop. the entire community of users will benefit from the increased uptime, faster problem resolution, and more stable deployments made possible by webMethods insight. using webMethods insight, your organization can proactively manage every single important transaction.
Trying out webMethods Insight is easy. Schedule a free ½ day on-site workshop or pilot it in your environment. Deployment is fast and non-invasive. You’ll surely gain visibility into something new. Find out more at www.softwareag.com/insightworkshop.
T O F I N D T H E S O F T W A R E A G O F F I C E N E A R E S T Y O U ,
P L E A S E V I S I T
W W W . S O F T W A R E A G . C O M
Take the next step to get there – faster. ABoUT soFTWARE Ag
software ag’s 4,000 global customers use our software to improve business processes and drive an agile it infrastructure. our customers’ goals are to reduce costs and increase flex-ibility and efficiency. we help them do this by optimizing and governing their operations and aligning it with the business goals.
our leading business infrastructure software portfolio is used for data and system integra-tion and modernizaintegra-tion. it fosters new levels of it agility through soa and allows the rapid creation of new business processes with bPM. our 40-year history of success ensures our customers have a reliable platform for driving future business results – faster.
software ag – get there faster
Copyright © 2009 software Ag, Darmstadt, germany and/or software Ag UsA, Inc., Reston, vA, United states of America, and/or their suppliers. All rights reserved. sA g _Insigh t_I s_T ech_WP_ o ct09
