Gartner
Entire contents © 2002 Gartner, Inc. All rights reserved. Reproduction of this publication in any form without prior written permission is
Markets, M-18-6400
B. Eisenfeld, F. Karamouzis
Research Note 14 November 2002
Americas CRM ESPs: 2003 Magic Quadrant Criteria
Gartner has developed high-level evaluation criteria for the 2003 Americas customer relationship management Magic Quadrant for external service providers. These criteria are used to assess ESPs' vision and execution capabilities.
Many enterprises look to external service providers (ESPs) to help them navigate the complexities of customer relationship management (CRM) initiatives. If the enterprise selects the right ESP for its specific situation — and this is a critical "if" — the CRM service provider can supply a number of important benefits.
A CRM service provider brings CRM program- and project- experienced personnel who can help reduce risks. Enterprises not only get the benefits of that experience, they gain an understanding of best practices, using proven methodologies to sort through the complexities, while gaining supplemental resources for staffing their projects.
To assist enterprises with evaluation and selection, Gartner has developed a framework called the "Magic Quadrant." Here, we define the evaluation criteria that we use to assess service providers in the CRM ESP market.
The Magic Quadrant Framework
The Magic Quadrant is a graphical framework (see Figure 1) used to place a set of vendors from a specific technology or service market into a strategic matrix. Gartner analysts use a combination of objective and subjective criteria to evaluate individual vendors along two dimensions: Completeness of Vision and Ability to Execute.
Core Topics
Customer Relationship Management:
Business Strategies, Technologies and Applications for Marketing; Business Strategies, Technologies and Applications for Sales; Business Strategies,
Technologies and Apps. for Customer Service and Support; Creating Business Value for CRM
Key Issue
What is CRM, how will it evolve, and what drivers are emerging to force its adoption?
Figure 1
The Magic Quadrant Framework
Challengers Leaders
Niche Players Visionaries
Completeness of Vision Ability
to Execute
Source: Gartner Research
Completeness of Vision (X-Axis): This dimension analyzes the breadth of an ESP's vision and its depth of knowledge of the markets and customers it serves. The detailed criteria focus on the service provider's vision relative to current market trends, client demands, and the difference between its stated vision and Gartner's vision for CRM service providers. The analysis is focused on how the ESP positions itself to address future industry scenarios through investments in research and development, alliances and partnerships, and skill building. In addition, the ESP's demonstrated and recognized thought leadership to guide the enhancement and evolution of the market is assessed.
Ability to Execute (Y-Axis): This dimension evaluates an ESP's ability to execute its vision. The detailed criteria focus on the management team, its tenure and skills, as well as the organization enterprisewide and CRM practice financial stability.
Other key factors include the vendor's sales channels, quality of research and development, mix of clients and industry verticals, service reputation and track record in delivering solutions to market in a timely fashion. A key component of execution is client feedback.
When the two axes are taken together, they provide the
framework for evaluation. A service provider's positioning in the
Magic Quadrant is based on an analysis of the multiple
evaluation points considered along each of the axes. It is
possible that a provider that is strong in some areas, but weak in
one particular area, will not be designated a leader. However,
that area may not be important to a particular enterprise evaluating the market. Enterprises should consider how a provider ranks and weigh its position in the Magic Quadrant, as well as its location and size, before eliminating a vendor from consideration.
Magic Quadrants do not always include all of the vendors in a given sector. Vendors evaluated are chosen based on their current market impact, potential market impact and how often Gartner analysts receive inquiries about them. In addition, they must meet the minimum inclusion criteria for placement on the 2003 CRM ESP Magic Quadrant in the Americas (see "Americas CRM ESPs: Understanding 2003 MQ Positions").
The Completeness of Vision (X-Axis) Criteria
The Magic Quadrant process involves primary research with direct client feedback to analysts throughout the year, as well as references supplied by each service provider. This information is supplemented by the ESP's representation of its organization.
The analysis involves weighting all sources of information, with a heavy emphasis on direct feedback from clients. As a result, many of the individual categories defined below have a "client reference" criterion factored into the analysis and evaluation.
Gartner considers client feedback to be one of the most-critical measures of the success of a CRM ESP, and this criterion continues to influence positioning.
The service providers are evaluated across a number of high- level criteria for each axis. We define the Completeness of Vision criteria as follows:
CRM Practice Viability: The services market continues to be one of the fragmented segments of IT, with no single player commanding more than a single-digit market share. There are thousands of ESPs in the United States alone, and hundreds claim some type of CRM service offering. In addition, in 2002 we have seen large-scale service providers — for example Arthur Andersen, with more than $2 billion in service revenue and $8.9 billion in overall revenue — go out of business, as well as others that have been purchased (such as PwC Consulting). All of these movements continue to reinforce the importance of ensuring that your service provider not only has a solid foundation and high probability of long-term viability at the enterprise level, but also has a solid foundation and long-term viability within the CRM practice itself.
A service provider structure that presents stable financial results,
coupled with year-over-year stability and the CRM practice
leader's commitment to invest and expand competencies, is
critical to the long-term success of the practice. Without this support and commitment, the practice becomes just another service offering chasing the market. ESPs must ensure practice viability through investments in R&D, methodologies, services offerings, sales and marketing. In addition, the management team must be able to articulate a compelling vision and execution strategy, while delivering a satisfied, committed, expanding referenceable client base, as well as complementary practices to support CRM. CRM practice viability is driven in part by the leadership's ability to see ahead of the curve and the technology hype cycle and adjust accordingly to chart the course, rather than lead by just reading a road map.
CRM Practice Strategy and Vision: Gartner analysts consider the CRM service provider's point of view on the evolution of service offerings and its corporate commitment to align services with current and future market needs. We examine which target markets the provider serves and how coherent and well- articulated its strategy and vision are. Is there a coherent and articulated plan? We evaluate providers' positions in the market as thought leaders and innovators, as well as their ability to lead client enterprises by supporting and extending their businesses so that they can become leaders in their respective industries or market segments.
Services and Methodologies: What services are offered by the provider's internal capabilities and what is available through partnerships, alliances and joint ventures? How are other related services offered and delivered? We examine the tools, methodologies and prepackaged accelerators providers can bring to an engagement to deliver service in the different domains of CRM. Do the methodologies for delivering CRM services contain embedded features that enable achievement of bottom-line business results, and are methodologies an integral part of a service's capabilities? What types of innovative and visionary offerings are developed? How are they developing services for emerging areas within CRM, such as partner relationship management (PRM) and marketing resource management (MRM). And what unique tools, services and methodologies do they develop to enhance industry-specific capabilities?
Engagement Customization: How well does a provider tailor
and adapt standard practices for particular clients? Does the
practice have the capability to reuse what it has learned from
prior project engagements and leverage this into new
engagements to speed delivery and time to market? Do the
consultants tailor engagements to do things in new, innovative
and creative ways to achieve success for their clients? Are there
knowledge management systems that encourage practitioners to
contribute to and use experiential learning? Pricing and pricing strategy, plus terms of engagement, all contribute to a provider's overall vision and its ability to tailor engagements to particular client needs, as opposed to just delivering "cookie cutter"
solutions.
The Ability to Execute (Y-Axis) Criteria
The service providers are evaluated across a number of high- level criteria for each axis. We define the Ability to Execute criteria as follows:
Enterprise and CRM Practice Viability: The composition of a CRM practice (including its relative size, revenue, the leadership of the management team and the team's position in the enterprise's structure) is key for execution, now and in the future.
We also examine financial information, such as total enterprise revenue, percentage of total revenue attributable to CRM, percentage of total CRM revenue by geography and percentage of CRM revenue by industry. Does the provider continue to gain new clients year over year or is it continuing to serve only its established base? Is it continuing to win business across all the domains of CRM (sales, marketing and service) or does it really specialize in a particular domain or industry? We evaluate the CRM service providers' plans to achieve their vision and a balanced portfolio of CRM engagements through ongoing investments in this service delivery area, which are key to viability, as is their sales engagement model, including their ability to market and sell CRM services.
Partnerships, Alliances and Joint Ventures: Because the services required to implement a complete vision for CRM and the resulting strategy are very broad, many enterprises team with each other to provide breadth and depth of services beyond the
"four walls" of the enterprise. The relationships CRM service providers embark on with other service or product providers can add value, but partnerships for partnership sake are all too common. What are the criteria by which partnerships and alliances are determined? The strength of partnerships is more important than just the number of them.
Partnering to bring innovation to market is key to furthering the ability to execute, as well as overall vision. Have these enterprises aligned alliances beyond the current market to extend capabilities into what the market may be in the future?
How are these partnerships managed? Has money changed
hands that serves to take partnerships beyond just joint
marketing arrangements? What resources are dedicated to the
various partnerships and joint ventures and what, if any,
certifications for these resources are promised and delivered?
And to what degree do these alliances affect an enterprise's ability to recommend and implement other solutions objectively?
People and Skills: The investments CRM service providers make in their people — including practices to recruit, train and retain qualified staff — are important in terms of execution and delivery. When CRM expertise is needed on a multidisciplinary engagement and stand-alone CRM projects are undertaken, can the providers deliver their resourcing practices? Are there enough of the right people with the right skills available to deliver across all client engagements?
CRM Experience: This aspect of execution involves demonstrated experience in CRM across implementation stages, disciplines, industry verticals and geography in relation to the target market evaluated. Does the provider segment and target the market based on its skill set to deliver quality for clients? Are knowledge transfer and quality assurance tasks performed regularly, or only when a problem occurs or when the engagement is about to end? What are the real numbers and types of industries in which expertise can be delivered, and what do clients say about the overall experience of working with the provider. Does the provider have experience across sales, marketing, and service and support? Can it provide advice across the customer life cycle in a multichannel environment?
Does it have the experience in implementing technologies that are found in the CRM "technology sweet spot" and integration to multiple front- and back-office functions and applications?
Geographic and Vertical Coverage: What are the capabilities of resources and the provider's experience in working in different locations? What is the provider's penetration of industries, and is it able to leverage vertical expertise within the sector and into other sectors? Although the criteria are evaluated for delivery primarily in the United States, Canada and Latin America, what are the provider's capabilities to provide full-service capabilities beyond this region and support clients with global operations?
What are the limitations of these providers in terms of geographical reach?
Customer Experience: Most importantly, what do the clients
say? Are the delivery capability claims and practices for the
spectrum of services offered — from strategy to project and
program management, contracting flexibility, knowledge transfer,
as well as quality control and quality assurance — meeting or
exceeding client expectations? Can these providers work within a
client's organization to deliver and influence the organizational
changes that are key to success in CRM? Have these vendors
been instrumental in ensuring that the business benefits (that is,
decreased costs and increased revenue) have been achieved through proven measurement processes?
Bottom Line: The evaluation criteria presented here will be used to position service providers in the Magic Quadrant. The relevant service providers assessed will be evaluated based on demonstrated results for their customers and may be a viable option for any particular enterprise's specific CRM needs. This evaluation process should be used to contribute to an enterprise's overall decision framework regarding its CRM service providers.
Acronym Key
CRM Customer relationship management
ESP External service provider
MQ Magic Quadrant
MRM Marketing resource management PRM Partner relationship
management
