IN THIS eGUIDE
2
“ITIL-Lite” for APM
A simple, effective way to proactively manage performance
5
Gartner: BI,
Analytics Software
Spending Jumps 13.4%
The results reflect the BI market’s continued strength throughout the world’s eco-nomic downturn, as customers looked to such software to find efficiencies and gain competi-tive advantage
6
Survey Says: Bad
Alignments Hamper
App Management
NetForecast finds a significant gap between what gets mea-sured and what’s important
8
Study Shows Need
for Infrastructure
Plus Application
Performance Views
Infrastructure and Application Performance Monitoring: you shouldn’t have one without the other
9
‘The More the
Merrier’ Doesn’t Apply
to Management Tools
Two or three performance man-agement tools are all you need
10
Application
Perfor-mance Management
Resources
Additional tools, tips and documentation
Application Performance Management (APM) solutions are essential instruments in helping enterprise IT departments
keep their business-critical applications running smoothly. By measuring and monitoring the performance of application
components in real-time, these solutions equip IT teams with the insights they need to plan capacity, tune performance,
and troubleshoot problems. In this eGuide, Network World, with sister publication CIO, offer news on the latest trends, best
practices, and straightforward advice on these solutions and the challenges of managing application performance.
“ITIL-LITE” FOR APM
Peter Sevcik and Rebecca Wetzel, Network World
If you work in a big organization replete with financial and human resources, ITIL may be your best bet for structuring your application performance management. But if your organization is like most, and you don’t have the where-withal and time to implement full-blown ITIL, we suggest a much more practical and less onerous approach to managing application performance. At NetForecast we call it “ITIL-Lite” for APM. This approach consists of four APM best practices, which when well executed get demon-strable results. These are to understand, measure, and communicate about application performance--and to link application performance to the business.
UNDERSTAND performance by learning about your
ap-plications and their requirements, application users and their requirements, and your infrastructure environment.
The most basic understanding comes from watching us-ers do their jobs--or have fun if that is what you deliver--interacting with an application. It is also valuable to talk to users about the experience. We know many technolo-gists who play key roles supplying application services, yet never talk to users! Understanding users and how an application works is a must.
MEASURE any technical parameter that influences
ap-plication performance or user satisfaction. You can never have too much data. Yes, you may not use it all--but you will have data to show changes. Performance management is about incremental improvements. Measure small improve-ments early, even if they appear modest. This gives you data to justify bigger changes for larger gains. The bottom line is you must measure, measure, and then measure some more.
COMMUNICATE your findings. This means showing
peo-ple your measurements and explaining the performance gains--or losses. Write your reports so non-geeks can un-derstand them. You must communicate technical data dif-ferently to non-technical audiences than to your tech-savvy peers. We know many skilled application performance managers who never show their reports to anyone. Like gnomes, they hoard data for their own use--and only retain it to cover their backsides should a decision backfire.
LINK performance to business needs. Performance
re-ports shared outside of IT should be grounded in “what matters” to the business. Is management eager to see availability and/or utilization numbers? Not! What execu-tives want is information like the amount of revenue made or lost, number of orders processed, patients treated, users positively or adversely affected, etc. The business metric you choose must foremost be important to your business, and it must have a business goal. Linking
tures how well the application’s performance supported the business goal.
The goal of APM best practices is to improve application performance--and for the best results these best practices cannot stand alone. Each must be part of a continuous improvement process that ensures that your application performance supports your business needs.
The process begins by understanding your user and ap-plication needs. Then you must measure data that reflects your understanding. The data has little value unless you report it to the right people within your organization. Final-ly, the reports need to serve as input to link performance to key business needs. At that point, your IT and business groups engage in dialogue that improves your IT group’s understanding of what is important and how to measure against the new objectives.
Dialogue is vital because the business group injects contextual understanding about what really matters for applications users, and that understanding enables you to measure the right things and set thresholds that will help you optimize performance where it counts most.
Here’s an example of how these best practices work in the real world. Picture this. You spend a morning
look-ing over the shoulder of web-based order entry system user we’ll call Shirley as she enters customer order infor-mation, and you watch as around 11 AM Shirley starts to fidget and finally takes a coffee break when her data input task completion times (which you are timing with a stop watch) exceed 10 seconds. After she takes her break, you head back to your cube and over the course of the afternoon and subsequent days you point your user response time tool to track order entry application us-ers and correlate response time with active usus-ers, time of day, and general network traffic. The measurements show that during peak network usage times order entry task completion times routinely exceed 10 seconds. You document your findings and send them to your manager and to Shirley’s boss, noting that based on your obser-vations Shirley’s productivity is being hampered by poor application performance--and recommending that steps be taken to improve order entry system response times during times of network stress.
You ask Shirley’s boss how many orders Shirley should reasonably be able to enter per hour so you can trans-late that information into the application response times needed to meet that business goal using the data
gath-ered which shows users are most productive (active) when response time is below 4 seconds. You tell your boss that to meet that business goal, he’ll have to open the depart-mental purse strings to install a QoS solution to prioritize the web-based order entry application, and you convince him to fund a QoS pilot.
You complete the pilot, measure the resulting order entry task times, and create a report showing that the pilot QoS deployment consistently reduces task response times during peak hours to four seconds. Your boss asks you to present your findings to the budget committee, which agrees that the business benefits justify deploying the production QoS solution. Shirley and her boss are de-lighted, and treat you to lunch to thank you.
During lunch they ask if you could reduce the task re-sponse times to two seconds. You respond by showing management a chart that compares performance for web-based order entry activity in Shirley’s group with another group performing the same work but closer to the data center. The closer group gets two-second response times, but is not more productive. You recommend that improving order entry productivity will take a combination of better re-sponse time and a simpler entry process with fewer steps.
Improving response time to two seconds by itself will be expensive now that QoS has done all it can, and you will need to upgrade WAN bandwidth. You are then invited to work with the order entry application group to show them that improving productivity is now in their court.
Your tools helped solve the issue and educate the com-pany, but you had to know how to apply the tools, where
to look for answers, and communicating your insights to company. This is the difference best practices bring to your job and your success in your job.
—Peter Sevcik is president of NetForecast and is a leading authority on Internet traffic, performance and technology. Peter has contributed to the design of more
than 100 networks, including the Internet, and holds the patent on application response-time prediction. Rebecca Wetzel is an associate of NetForecast and a veteran of the data networking industry. She works with network product vendors and service providers to develop and implement successful product and mar-keting strategies.
Dialogue is vital because the business group injects contextual understanding about what
really matters for applications users, and that understanding enables you to measure the
right things and set thresholds that will help you optimize performance where it counts most.
WHAT REALLY
MATTERS
The results reflect the BI market’s continued strength throughout the world’s
economic downturn, as customers looked to such software to find efficiencies
and gain competitive advantage
Global spending on BI (business intelligence), analytics and performance management applications jumped 13.4 percent in 2010 to $10.5 billion, according to figures released recently by analyst firm Gartner (IT). The results reflect the BI market’s continued strength throughout the world’s economic downturn, as customers looked to such software to find efficiencies and gain competitive advantage.
Although enterprise software spending dropped 2.5 percent in 2009, during the height of the global recession, BI sales grew “in the low single digits,” Gartner analyst Dan Sommer said via e-mail. While customer demand played a role,
ag-gressive marketing efforts of BI vendors were also influential, according to Gartner.
BI remains dominated by a handful of major players. SAP, IBM, Oracle and Microsoft alone control 59 percent of the mar-ket for BI and performance management software, according to Gartner, while SAS Institute holds the top spot for analytics. While not able to provide specific percentages, Sommer es-timated that “a fairly big portion” of new BI revenue came from existing users adding licenses, versus brand-new projects.
“But overall we did see a shift from ‘maintenance’ to ‘license’ and from ‘build’ to ‘buy,’” he added. “In 2009, most
ary spending in IT budgets was frozen. As a result, vendors came up with increasingly creative strategies to squeeze main-tenance revenue out of their customers.”
Last year, improved economic conditions and some major product releases from IBM, Oracle and others “drove renewed license selling efforts and released pent-up de-mand,” Sommer said.
BI revenues could only grow further in coming years, as vendors pursue strategies meant to get the software into more users’ hands.
This week, Microstrategy followed other companies in an-nouncing a product that enables “self-service” BI, wherein busi-ness users can explore and analyze data on their own, without much help from IT staffers.
BI vendors such as SAS are also moving quickly to add sup-port for mobile devices like Apple’s iPad and Android tablets.
Forrester Research recently predicted that the rise of tablets will significantly increase the prevalence of mobile BI since their larger screens are more suitable for the types of visualizations and navigation styles that make BI software most useful. —Chris Kanaracus covers enterprise software and general tech-nology breaking news for The IDG News Service, a CIO affiliate.
Chris Kanaracus • IDG News Service
GARTNER: BI, ANALYTICS SOFTWARE
SPENDING JUMPS 13.4%
SURVEY SAYS: BAD ALIGNMENTS HAMPER
APP MANAGEMENT
NetForecast find a significant gap between what gets measured
and what’s important
A recently completed NetForecast/Network World survey of application performance management (APM) practices has uncovered serious gaps between the performance at-tributes IT managers cite as important to measure, and those they actually do measure.
These gaps point to a misalignment between needs and practice, and should be corrected to ensure good performance outcomes. In particular, end-user page re-sponse time and server query rere-sponse time showed enormous gaps between their perceived importance and the frequency with which they are measured.
In the survey, 364 participating IT managers were asked to rate the importance of 12 application performance at-tributes, and to tell us whether or not they measure each of the attributes.
We considered needs and practices to be aligned if an attribute is cited as important and is measured, or if it is cited as unimportant and is not measured.
Needs and practice are misaligned when an attribute is described as important and is not measured, or if it is described as unimportant and is measured.
A performance attribute that is important yet not
mea-sured suffers from a measurement gap. And an attribute that is unimportant yet measured exhibits what we dub measurement squander.
In aggregate, we found that 60% of the response com-binations were misaligned, and only 40% were aligned.
Of those that are misaligned, 37% show a measure-ment gap - in other words important attributes were not measured. And 23% of that 60% were squandering their measurement efforts because they were measuring things they considered unimportant.
The top five application performance attributes respon-dents most often describe as important to measure are net-work availability, server availability, end-user page response time, server query response time, and bandwidth utilization. Of these five attributes, a serious measurement gap of more than 50% exists for end-user page response time
and server query response time -- compared to modest gaps of under 30% for network availability, server avail-ability and bandwidth utilization.
We surmise that end-user page and server query response times fare poorly compared to the other important attributes because they are harder to measure, and there are fewer and less established tools available to measure them.
Measuring response times takes synchronization, which is not easy to pull off. You need the virtual equiva-lent of a stop watch to determine start and stop times, and you need automated report-generating capabilities. But response times are very important because they
mea-sure what matters most to end users and are where the rubber meets the road. Everyone talks about how impor-tant it is to ensure a good end-user experience, but more often than not response times go unmeasured, making it impossible to know how well you are doing on that front.
In contrast, network availability, server availability and bandwidth utilization are well-established, easy-to-gather metrics, reported by tools ensconced in corporate net-work operations centers.
Although less prevalent than measurement gaps, what we call measurement squander also bears addressing. In particular, server errors, network protocol errors, and
net-work latency receive more measurement attention than their general importance would dictate.
We are not advocating dropping these performance metrics. A metric’s value can be increased if its results are properly linked to business results.
To correct misalignment due to measurement gaps, IT managers need to invest in appropriate measurement ca-pabilities. And to correct misalignments due to measure-ment squander, IT managers need to either improve the value of what is measured, or cease investing resources in measuring unimportant attributes.
It’s as simple as that.•
The top five application performance attributes respondents most often describe
as important to measure are network availability, server availability, end-user
page response time, server query response time, and bandwidth utilization.
WHAT REALLY
MATTERS II
STUDY SHOWS NEED FOR INFRASTRUCTURE PLUS
APPLICATION PERFORMANCE VIEWS
Infrastructure and Application Performance Monitoring: you shouldn’t
have one without the other
Many enterprises zero in on watching either infrastructure or application performance. Based on results from Net-Forecast’s latest application performance management benchmarking survey, we can unequivocally say that those with a “stereoscopic” view of infrastructure as well as application performance achieve dramatically better results than those with a “monoscopic” view of just one or the other. So if you are focusing on only one view, we suggest you get with the program and expand to a more holistic, dual view of performance.
Those enterprises in our recent APM benchmarking sur-vey with access to both infrastructure and application per-formance views resolved perper-formance problems
dramati-cally faster than their single-view-enabled counterparts. The typical time needed to resolve a critical application incident for those with a single view was a whopping 11 hours, while those with a dual view improved by 40 per-cent to seven hours.
The percent of performance problems discovered by IT rather than end users also increased for those with a dual rather than single view of performance. Those watching both infrastructure and application performance discov-ered performance problems proactively 47 percent of the time rather than 37 percent of the time for those enterpris-es with a single view--a 29 percent improvement. Not bad! Nearly all of the single-view enterprises rely solely on
infrastructure tools for their performance information. For most single-view enterprises, expanding to a dual view means adding application or end-user performance infor-mation to infrastructure inforinfor-mation. This more comprehen-sive approach brings to light 54 percent more performance incidents. Those with a single view detected 28 critical ap-plication incidents per month on average, compared to 43 for those with a dual view. Clearly performance incidents discovered by application-centric monitoring tools remain undetected by enterprises relying only on infrastructure-centric performance monitoring tools. This means that 35% of performance incidents go undetected!
If you rely on a monoscopic view, this should keep you awake at night and should kick start you to lobby your boss to expand your performance management tool arse-nal to include application as well as infrastructure views.
Let us know what you think of our findings. •
‘THE MORE THE MERRIER’ DOESN’T APPLY TO
MANAGEMENT TOOLS
You may be yearning to buy management tools from yet another vendor, but NetForecast’s recent survey of performance management best practices and their results shows that using more management tools is not necessarily better. Most high-performing enterprises use two or at most three vendors, while low-performing enterprises use one or two. The takeaway is that most enterprises need at least two and no more than three vendors to achieve good application performance re-sults. And if you have stellar best practices, you can get by with one vendor.
The finding that two vendors are required for good APM results dovetails with findings cited in our recent
posting, that to ensure optimal performance you need two tools--one for monitoring infrastructure performance and another for monitoring application performance. The resulting stereoscopic view ensures that you are able to solve more problems faster.
Our previous blog on the topic shows that enterprises with a benchmark score above six really do deliver better performance for their users--75% better perfor-mance. Even though enterprises in the high group got there primarily through best practices, it is also impor-tant to factor in the management vendors they use to support those best practices.
There exists an association between the number of
management vendors used and benchmark results. Scores improve with the number of vendors deployed up to three, and among the very highest performing enter-prises the vendor count dips precipitously. We postulate that the highest-performing enterprises get good results with fewer vendors because they have very effective APM processes (i.e., best practices) in place.
Based on our survey results and experience, we conclude that if you want to be a high-performing enterprise, you need the stereoscopic view provided by two performance management tools--one providing an infrastructure-centric view, and the other an application-centric view, and you need best practices. The number of vendors required to get the stereoscopic view is evolving as vendors develop new and more comprehen-sive tools. •
Peter Sevcik and Rebecca Wetzel • Network World
CASE STUDY
ExtraHop Helps Alaska Airlines
Ensure That 25 Business-Critical
Applications Do Not Fail
Alaska Airlines designates over 25 applications as business critical, managing everything from aircraft weights and balances to ecommerce applications that drive billions of dollars in rev-enue. Alaska Airlines uses the ExtraHop system to passively observe its application infrastruc-ture, continuously and without interfering with the applications.
Download now
CASE STUDY
Concur Improves On-Demand
Services with the ExtraHop
System
Concur is a leading provider of integrated travel and expense management solu-tions. After deploying the ExtraHop system, Concur identified the root cause for hard-to-diagnose intermittent slow performance, gained early warning on issues that could have caused larger problems, and improved its triaging and troubleshooting processes. Download now
WEBINAR
Best Practices for Achieving
Application Delivery Assurance
in the Enterprise
In this 16-minute webinar, Raja Mukerji, Presi-dent and Co-Founder of ExtraHop Networks, explains how the ExtraHop Application Delivery Assurance system equips IT managers to regain control over application performance. The Extra-Hop system passively analyzes all transactions on the wire in real time to provide assurance for critical application infrastructure components. Watch now
WEBINAR
APM, NPM and ANPM,
Oh My!
EMA Managing Research Director Jim Frey and ExtraHop CEO and co-founder Jesse Rothstein discuss the convergence of application and network performance management. In this webinar, you’ll learn about the options for performance management, including why not all approaches are created equal. Watch now
