Nemertes Research PilotHouse Awards
IP Telephony
The Nemertes Research annual PilotHouse Awards provide insight on the performance of technology vendors, according to feedback from IT decision-‐makers who use their products or services.
Q3
12
08
Fall
T
ABLE OFC
ONTENTS
IP Telephony ... 3 Award Definition ... 3 Overview ... 3 Market Classification ... 3 Ratings ... 5 Ratings Categories ... 5 Technology ... 5 Customer Service ... 5 Value ... 5 Results Summary ... 6 Analysis ... 7
PilotHouse Market Leader Winner ... 10
Cisco ... 10
Technology ... 11
Customer Service ... 12
Value ... 12
PilotHouse Market Challenger Winner ... 13
Mitel ... 13
Technology ... 14
Customer Service ... 14
Value ... 15
Pilothouse Finalists: Market Leaders ... 16
Avaya ... 17
PilotHouse Finalists: Market Challengers ... 19
Siemens Enterprise Communications ... 20
ShoreTel ... 22
Alcatel-‐Lucent ... 24
Conclusion ... 25
Methodology ... 26
Sample Frame ... 26
Planned Sample Size ... 26
Survey Sub-‐Groups/Stratification ... 27
Awards ... 28
Timing ... 28
Incentives to Participate & Time Commitment ... 29
Future Plans ... 29
IP
T
ELEPHONY
By Irwin Lazar
Vice President and Service Director, Nemertes Research
Award Definition
The Nemertes PilotHouse award for IP telephony recognizes vendors of IP call-‐control servers and handsets/softphones. They also typically offer voice/unified messaging, auto-‐attendant, gateways, and audio conferencing bridges. IT professionals who use these services rated their providers on technology, value, and customer service.
Overview
The goal of the PilotHouse awards is to provide analysis of vendor and service-‐ provider performance from the perspective of the IT professionals who actually use the technology. Many research firms offer market rankings, but Nemertes' research and analysis is unique. It is 100% based on the views and experiences of actual users. The research is wholly independent and not sponsored, and neither Nemertes nor the vendors or service providers being rated have any influence over ratings performance. The opinions are those of business users of the technology or service. By combining direct interviews of IT leaders with surveys of screened IT users, Nemertes is able to provide unique insights into why IT professionals rate vendors the way they do.
For this award, Nemertes gathered ratings on IP telephony providers with a range of offerings. (More detail on the program, and demographics of participating IT professionals is available in the methodology at the end of this report.)
Market
Classification
We segmented providers into two categories: Market Leaders and Market Challengers, and offered awards within each category. To determine the categorizations, Nemertes’ analysts evaluated IP Telephony market presence
(looking at revenue, device shipments, and number of customers) based on our own research and publicly available data. Analysts also examined natural breakpoints in the data, and segmented the Market Leaders as those who collectively accounted for the vast majority of each market, and Market Challengers who accounted for a smaller percentage of the overall market.
IP Telephony
Market Leaders
Market Challengers
Avaya, Cisco
Alcatel-Lucent, Mitel, Siemens Enterprise Communications,
ShoreTel,
Table 1: Vendor Classification, IP Telephony, 2012
Nemertes classifies the IP telephony market as “consolidated.” In a consolidated market, no single vendor controls more than 50% of the market, and no two
vendors combined control more than 75% of the market. Avaya and Cisco combined control is somewhere between 50% and 60% of the U.S. market according to
analysts and Nemertes PilotHouse data. About 20 vendors share the remaining U.S. market. None of the other vendors control a large enough share of the U.S. market presence as Avaya or Cisco. The telephony market does shift globally. For example, in Europe, Siemens and Alcatel-‐Lucent would be Market Leaders.
This year’s ratings include most of the leading vendors in the North American IP telephony market, however some vendors with small market share such as Aastra, Digium (Asterisk), NEC, Toshiba, and Zultys, did not receive enough ratings for award consideration. We did not include Microsoft in this year’s results largely because most organizations deploying Microsoft Lync as a telephony solution tell us they are doing so in trial or limited production, rather than as an enterprise-‐wide solution.
This award only covers the on-‐premises IP telephony providers. Hosted IP
telephony services, including vendors such as AT&T and Verizon, as well as hosted offerings from those who also offer on-‐premise platforms, are not included in this report.
Ratings
We asked IT professionals to rate IP telephony providers using a 5-‐point scale, where 5 is excellent, 4 is good, 3 is fair, 2 is poor, and 1 is unacceptable. Nemertes then used these raw scores to compute average scores for each category.
The maximum possible score is a 5.0 (and although some vendors did receive perfect scores from individual IT participants, none received a perfect score when all ratings were averaged).
Ratings
Categori
es
The participants rated their technology providers in three areas:
⇒ Technology ⇒ Customer Service ⇒ Value
From these three ratings, Nemertes calculated an overall score (weighting each of the three categories equally).
Technology
Technology ratings gauge how customers view the sophistication, features, and implementation of the IP telephony service they’re rating. Additionally, this score reflects how much of a leader a vendor is in the IP telephony industry, from the perspective of the customer.
Customer Service
Customer-‐service ratings cover how providers perform in areas such as technical support, responsiveness to deployment problems and concerns, sales support, and general customer care. Additionally, technology users considered the willingness and ability of the vendors to answer questions effectively and promptly.
Value
Value ratings aren’t based solely on cost; rather, we ask participants to rate what they get in technology and support compared to what they pay for it. In other words, are they getting their “bang for the buck?”
Result
s
Summary
Figure 1: IP Telephony All Vendors, Overall
⇒ Among Market Leaders, Cisco wins the PilotHouse Award.
• Cisco’s overall score is 4.13.
⇒ Among Market Challengers, Mitel wins the PilotHouse Award.
• Mitel’s overall score is 4.10.
⇒ A total of six providers received enough responses to be included in this year’s analysis.
• Two providers are Market Leaders; four are Market Challengers. • Market Leaders’ overall average score is 4.00.
• Market Challengers’ overall average score is 3.99 4.13 4.10 4.05 4.02 3.86 3.79 3.00 3.10 3.20 3.30 3.40 3.50 3.60 3.70 3.80 3.90 4.00 4.10 4.20 Cisco Mite l Siem ens Shor eTel Avay a Alca tel-L ucen t
IP Telephony: All Vendors, Overall Scores
A
NALYSIS
Table 2: Vendor Scores, IP Telephony, 2012
⇒ This year’s IP telephony ratings show a comeback of sorts for Market Leaders, with Cisco achieving the top overall score (4.13 versus 3.86 in 2011). Overall scores were down from 2011, from an average of 4.12 in 2011 to 3.99 in 2012. Customer service took the biggest hit, dropping from an overall average of 4.17 in 2011 to 3.85 in 2010. Why? Participants often cited channel difficulties, perceived cuts in support resources from vendors trying to reduce operating costs, and increased interoperability challenges as companies shift from voice-‐ focused solution to broader unified communications portfolios.
⇒ Last year, Challengers dominated, with all four challengers outscoring Market Leaders Cisco and Avaya. This year Cisco outscores everyone while Avaya manages to pass Alcatel-‐Lucent.
⇒ Among the top three Market Challengers (Mitel, Siemens Enterprise
Communications, and ShoreTel) the gap in overall scores is relatively small (a total of .08), while the gap between Market Leaders Cisco and Avaya widens from a tie in 2011 to .27 in 2012. This change reflects not only Cisco’s success in improving its own scores, especially customer service, but Avaya’s overall decline in customer service and value from 2011.
⇒ Challengers overall largely maintain their edge in value, with all but Alcatel-‐ Lucent edging out both Cisco and Avaya in this category. As in years past,
\
Winners
Overall TechnologyCustomer Service Value Cisco 4.13 4.15 4.20 4.04 Mitel 4.10 4.10 3.90 4.30
Other Finalists
Siemens 4.05 4.07 4.00 4.07 ShoreTel 4.02 4.27 3.55 4.23 Avaya 3.86 4.11 3.68 3.79 Alcatel-Lucent 3.79 3.77 3.77 3.85Nemertes 2012 PilotHouse Awards
IP Telephony
Market Leaders
Market Challengers
Cisco, Avaya
Mitel, Siemens Enterprise Communcations, ShoreTel, Alcatel-Lucent
customers often lean toward Challengers because they perceive they are getting more for what they are spending.
⇒ Overall, Market Leaders achieve a score of 4.00, and Challengers have a 3.99.
• Reason: This year’s scores are reflective of the split between Avaya and Cisco, coupled with the decline of Market Challengers. In 2011, both had an overall score of 3.96. Meanwhile, Market Challengers decline considerably from 4.20 in 2011 to a 3.99 this year. Cisco gains slightly more than Avaya lost, but Market Challengers are down across the board. These scores reflect a
tightening market, where 2011 was the “Year of the Market Challengers,” the Challengers are facing increasing difficulty crossing the chasm to larger, more complex customers that often require considerable integration with legacy equipment and other unified communications platforms. Meanwhile Cisco slightly offsets Avaya’s decline, leading to a near tie among overall scores for Leaders and Challengers.
⇒ In technology, Market Leaders achieve a score of 4.13, while Challengers trail with a 4.05.
• Reason: Alcatel-‐Lucent brings down the rest of the Market Challenger pack, which led Leaders in this category in 2011. Alcatel-‐Lucent’s 3.77 score, down from 4.18 in 2011 brings the entire Market Challenger group down. Without Alcatel-‐Lucent, the remaining three Challengers led Avaya, while two of them trail Cisco. In past years, we have noted that IT leaders set a higher bar for Market Leaders–they expect Market Leaders to drive technology innovation, whereas they often look to Challengers for better value and customer service.
⇒ In customer service, Market Leaders pull in a score of 3.94, and Challengers receive a 3.81.
• Reason: Although Market Leaders actually fell .02 from 2011, Challengers fell even more (from 4.27 to 3.81). All the Challengers saw drops in their
customer service ratings. This represents a significant red flag for
Challengers who often compete on their ability to provide more personal service, and tight integration with their channel partners.
⇒ In value, Market Leaders garner a score of 3.92, and Challengers earn a 4.11.
• Reason: Here’s the one area where Challengers soundly out-‐perform Market Leaders, but here too the gap narrows with Challengers declining .11 while Leaders improve .08 from 2011. Market Challengers still demonstrate a great deal of value, but their edge is declining.
⇒ For a few vendors, success and/or failure in specific areas makes a big difference in their overall score:
• Mitel outperforms all vendors in value. Mitel’s move to focus on
differentiating itself through virtualized offerings is winning hearts and minds of its customers.
“We had existing Mitel gear, it was a cheap and easy upgrade for us, cheaper than Avaya or ShoreTel” says the director of IT for a midsize professional-‐services firm.
• ShoreTel, the top-‐scoring winner of PilotHouse IP telephony award every year Nemertes has collected ratings, has the highest overall technology and value scores, but the lowest customer-‐service rating of any vendor. This is a huge shift from years past, when ShoreTel’s customer-‐service ratings were among the highest in the entire PilotHouse program. Most of the concerns we heard weren’t specific to ShoreTel, but rather to its channel partners.
“The VAR doesn’t have a deep bench. When there’s an issue, I have to escalate to ShoreTel, which takes time. It takes a while to get to the right person,” says the manager of telecom for a midsize healthcare company. ⇒ Challengers perform best this year in value, while Market Leaders top
Challengers in customer service and technology. Leaders continue to leverage their perceived ability to drive technology innovation, while Challengers continue to largely compete on price and bundled capabilities.
P
ILOT
H
OUSE
M
ARKET
L
EADER
W
INNER
Figure 3: PilotHouse Market Leader Winner: Cisco, IP Telephony, 2012
Cisco
⇒ Summary:
• Cisco receives a 4.13 overall score, a nice improvement from its 3.96 score in 2011. Cisco continues to score well in technology and customer service, while value is its lowest score. Last year, Cisco tied with Avaya as the overall winner for Market Leaders. Making huge improvements this year, Cisco wins hands down. Cisco continues to base its success on driving technology
change and providing superior technical support through its channels and its direct partners. But its 4.04 value score shows that although it may not be the cheapest solution, its customers like what they get for their money. Cisco gets high marks, as well, from those who integrate its IP telephony
equipment into Cisco network infrastructure.
“What we like most is the integration of features and integration with rest of network infrastructure, which are primarily Cisco components,” says the telecom manager of a midsize manufacturing company.
⇒ Then and Now:
• Cisco improves across the board from 2011, increasing technology from a 4.03 to 4.15, customer service from 4.00 to 4.20, and value from 3.85 to 4.04. Combined, Cisco’s overall increase is .17 from 2011. Cisco continues to execute well given its renewed focus on collaboration as a result of recent corporate restructuring. Cisco historically has been viewed by its customers as an expensive solution, but one that is stable, feature rich, and comes with strong technical support. The fact that its value score improved by .19 indicates customers are seeing not only greater cost competitiveness, but increased business benefit from their Cisco purchases.
“Since we're an all-‐Cisco environment, we're using their networking gear, their VOIP, their television stuff, digital signage, and because we have a strong relationship with them, we're able to things with a smaller staff than we otherwise could do. That's been an advantage,” says the director of IT for a large manufacturing company.
⇒ Future Direction:
• Cisco faces increasing competition from both other IP telephony vendors, as well as Microsoft, and improving hosted offerings. Cisco customers continue to complain about expensive support and equipment costs, concerns Cisco is trying to address via broader channel support and virtualization-‐ready offerings. Still, as always, Cisco must fend off a growing cadre of competitors trying to be better, faster, and cheaper.
“We have had tremendous support. A solid, consistent account manager who understands his role, shepherds and looks after us, but it is so expensive. The software model is costly, but it does seems as if they are getting more competitive,” says the director of telephony services for a large educational institution.
Technology
⇒ Last year Cisco’s technology score was the lowest of all vendors. This year, it’s the second highest, trailing only ShoreTel. Cisco’s efforts to differentiate itself through a broad set of collaboration offerings targeting desktop, mobile, and virtual environments is resonating with its customers, and is a key driver for Cisco’s .12 improvement.
• “They are a leader. We went through their jabber presentation, it's state of the art,” says the telecom director for a global pharmaceutical.
• “The technology is really good. Basically we have all the features we could want, we can program the phones on the fly, Unity voicemail system is great, and so is the presence and the IP communicator works well,” says the
technology director for a midsize professional-‐services firm.
Customer Service
⇒ Cisco’s customer-‐service score of 4.20 is its strongest rating area, leading all other IP telephony vendors. This represents a strong turnaround from last year when Cisco scored near the bottom. Cisco’s focus on its core offerings, and in further developing the capabilities of its channel partners are, creating benefits in the eyes of its customers.
• “We’re dealing with both a VAR and Cisco. We’re getting special treatment thanks to a strong and long history with them. We’ve developed personal relationships with executives at Cisco, we can call a senior VP on the
weekend because of a switch being down,” says the director of telecom for a global professional-‐services company.
Value
⇒ Value has historically been Cisco’s Achilles heel, and this year is no different as not only is value Cisco’s lowest ratings area, but also where it trails everyone but Avaya and Alcatel-‐Lucent. IT professionals continue to express concern over the cost of Cisco solutions including licensing, maintenance, and support. Still, Cisco improved significantly in value, going from 3.85 in 2011 to 4.04 in 2012. For Cisco to continue to show improvement, and more effectively compete with other IP telephony providers, it must better address customer concerns about total cost of ownership.
• “Licensing strategies are terribly confusing, and maintenance is very
expensive for the amount of time we leverage it, but you have to have it as a protection,” says the director of telephony for a global pharmaceutical. • “We're getting what we need, but their advanced services team rates are
P
ILOT
H
OUSE
M
ARKET
C
HALLENGER
W
INNER
Figure 2: PilotHouse Market Challenger Winner: Mitel, IP Telephony 2012
Mitel
⇒ Summary:
• “Value” is what leads Mitel to its first Market Challenger PilotHouse award for IP telephony. Its value score of 4.30 is tops among all Market Challengers, while its customer-‐service and technology Scores were just high enough to edge out Siemens Enterprise Networks and ShoreTel. Mitel’s focus on leveraging virtualization through its partnership with VMware is paying dividends, though this is a temporary gap as its competitors quickly develop virtualized offerings of their own. Nonetheless, Mitel hopes to leverage the VMware channel to continue to bring value to its customers. Still, Mitel faces stiff competition. It beats Siemens by only .05 and ShoreTel by .08. If
ShoreTel successfully addresses customer-‐service concerns, it could move back into the top spot in 2013. Mitel, too, has its weakest showing in customer service, where it trailed both Siemens and Cisco, but edged out ShoreTel, Avaya, and Alcatel-‐Lucent. Mitel’s ratings also came largely from
small companies. Moving forward, Mitel must broaden adoption among midsize and large businesses to sustain growth.
“Mitel guaranteed our price for five years, we’re now planning to deploy them to our larger offices,” says the manager of telecommunications for a small retail chain.
⇒ Then and Now:
• Mitel failed to gain enough ratings for award consideration in 2011, but its 2012 showing represents remarkable turnaround from 2010, improving in technology (3.39 to 4.10), customer service (3.44 to 3.90), and value (3.44 to 4.30) and overall from 3.43 to 4.10. Mitel’s shift in focus toward
virtualization, along with explaining the value proposition for the architecture, are the biggest reason for the increased scores.
“We’re a VMware shop, and we run VMware in our stores, so their alignment with VMware makes them a natural choice for our telephony platform,” says the director of IT for a small retail company.
⇒ Future Direction:
• Mitel must work hard to stay ahead of an increasingly competitive market, especially as companies transition their voice platforms to unified
communications, Microsoft gains traction, and new hosted vendors enter the mix. Still, focusing on virtualization and value are a winning combination this year.
“Their solution is low cost and it works,” says the managing director for a small hospitality company.
Technology
⇒ Mitel’s technology score of 4.10—though still impressive—is behind both Market Leaders Avaya and Cisco, as well as Market Challenger ShoreTel. Mitel must do more to leverage its integration with VMware and develop its cloud-‐ based offerings to improve its technology ratings. It also should focus on mobility integration.
• “It could be worse, could be way better; they almost discourage keeping current once you make an investment,” says the IT manager for a small healthcare company.
Customer Service
⇒ Mitel’s customer-‐service score of 3.90 trails that of Cisco and Siemens, but is ahead of all other vendors, Market Leader and Market Challenger alike. Small organizations seemed to struggle the most with Mitel customer service, while larger companies generally rated it higher. Here represents the greatest room for improvement, especially as it builds up the capability of its channels to support its virtualized solutions.
• “We haven't had any real problems, they provide good support,” says the IT manager for large retail company.
• “They've never delivered service to me where I said wow, that was great,” says the manager of IT for a small healthcare company.
Value
⇒ Value is where Mitel truly shines, with a 4.30 score, tops among all IP telephony vendors, but only slightly ahead of ShoreTel. Mitel’s score is driven by its low cost, with its customers largely citing Mitel’s overall prices as coming in lower than those from competing vendors.
• “The solution was cheap, cheaper than Shoretel for us, and easy upgrade from older phones,” says the voice manager for a small retail organization.
P
ILOTHOUSE
F
INALISTS
:
M
ARKET
L
EADERS
Figure 5: Market Leader: Avaya, IP Telephony, 2012
Avaya
⇒ After two years of improving scores, Avaya declines in 2012, going from a 3.96 rating in 2011 to this year’s 3.86. Avaya’s biggest area of decline was customer service, where it fell from 3.92 to 3.68. Technology held steady at 4.11, while value slightly dropped from 3.84 to 3.79. Avaya’s customer service fall-‐off is a red flag for the company that is working hard to overcome concerns over financial viability and debt as it readies itself for a potential IPO in the near future. Several Avaya customers noted concerns over the ability of partners to support Avaya, which is a concern given Avaya’s increasing reliance on the channel. Avaya’s 3.68 score was the lowest customer-‐service rating for all companies except ShoreTel. Avaya, like Cisco, is also facing increasing competition from Microsoft as it pushes its Lync platform as an enterprise telephony alternative
⇒ The average overall score of all Market Leaders is 4.00; Avaya’s is 3.86.
• “Avaya’s feature set is exceptional, but their future is very unclear, and we don’t want to pay more for features,” says the CIO of a midsize educational institution.
• “They are developing a strong roadmap, but there are some areas of
weakness, including migration with Nortel,” says the director of telecom for a large financial-‐services firm.
⇒ The average customer-‐service score of Market Leaders is 3.94; Avaya’s is 3.68.
• “Customer service is fine, but when issues are escalated, we run into issues with channel customer service. Partners don't have enough knowledge to provide good customer service. We might renew our contract with just Avaya later this year instead of the VAR,” says the IP telephony engineer for a small financial-‐services firm.
⇒ The average value score of all Market Leaders is 3.92; Avaya’s is 3.79.
• “They want to push everything through the VARs, and either they aren't training the VARs or they just don’t have the technical expertise,” says the director of telecom for a midsize energy company.
P
ILOT
H
OUSE
F
INALISTS
:
M
ARKET
C
HALLENGERS
Figure 6: PilotHouse Market Challengers: IP Telephony, 2012
4.10 4.10 3.90 4.30 4.02 4.27 3.55 4.23 3.79 3.77 3.77 3.85 4.05 4.07 4.00 4.07 3.30 3.40 3.50 3.60 3.70 3.80 3.90 4.00 4.10 4.20 4.30 4.40 4.50
Overall Technology Customer Service Value IP Telephony: Market Challengers
Figure 7: Market Challenger: Siemens Enterprise Communications, IP Telephony, 2012
Siemens Enterprise Communications
⇒ Siemens continues to hover near the top of the Market Challenger category. Its 4.05 overall rating puts it second to Mitel, and third-‐highest overall. Siemens falls a bit from 2011 in technology (4.10 vs. 4.07) and value (4.20 vs. 4.07), while its customer-‐service rating declined more significantly from 4.30 to 4.00. Like Avaya, Siemens faces its own set of challenges in trying to gain market share in the North American market, develop channel partners, and embrace cloud-‐based offerings. Though its customer-‐service rating is still higher than the overall rating for Market Challengers, for Siemens to win it must address customer-‐ service issues that drag down its overall rating.
⇒ The average overall score of all Market Challengers is 3.99; Siemens’ is 4.05.
• “They have a great product; they need to fix their support. Customer service/support is where the issue lies with Siemens for me,” says the director of telecom for a midsize healthcare provider.
⇒ The average technology score of all Market Challengers is 4.05; Siemens’ is 4.07.
• “We especially like Siemens’ mobility capabilities, it helps our employees stay connected wherever we go,” says the CIO for a midsize energy company. ⇒ The average customer-‐service score of Challengers is 3.81; Siemens’ is 4.00.
• “Overall, we’re very happy, Siemens provides great service and support,” says the IT manager for a midsize manufacturing company.
⇒ The average value score of all Market Challengers is 4.11; Siemens’ is 4.07.
• “It’s a well-‐priced solution compared to others on the market,” says the IT manager for a small retail company.
Figure 8: Market Challenger: ShoreTel, IP Telephony, 2012
Shore
Tel
⇒ After eight years of holding the top spot, not just for Market Challengers, but among all providers, ShoreTel finishes third among Market Challengers with an overall rating of 4.02, beating out only Avaya and Alcatel-‐Lucent. As we expected in last year’s analysis, ShoreTel’s 12.0 release alone would improve its
technology scores, and that it did. ShoreTel increases its rating from 4.18 in 2011 to a 4.27 this year—the best of all IP telephony vendors. It’s efforts to expand its feature portfolio by embracing virtualization for its management platform, while also acquiring Agito to provide UC mobility, are helping its customers see it as a technology leader. ShoreTel declines in value, though its value score is better than everyone’s except Mitel’s. It’s customer service where ShoreTel took its biggest hit, dropping from a 4.18 in 2011 to a 3.55 in 2012, the lowest of all IP telephony vendors. Nemertes issued this warning in last year’s PilotHouse report: “The vendor is pushing uphill with solid growth on its way to reach the $1 billion annual revenue mark by focusing on larger businesses and expanding globally. Making that transition is often when many companies lose their focus and their roots. ShoreTel has some other Challengers trailing not too far behind, and all of them score better in customer service—ShoreTel’s hallmark area.”
Indeed, ShoreTel’s customer-‐service did not improve, and ShoreTel this year was plagued by problems with its channel partners and their inability to deliver the level of support ShoreTel customers have come to demand. ShoreTel’s
international expansion and drive up-‐market into larger installations are all putting pressure on its ability to execute to the level it has maintained in years past.
⇒ The average overall score of all Market Challengers is 3.99; ShoreTel’s is 4.02.
• “ShoreTel has a very easy-‐to-‐manage solution. It doesn't require a dedicated person to handle on a daily basis. We get a lot of value for our money,” says the manager for a midsize financial-‐services firm.
⇒ The average technology score of all Market Challengers is 4.05; ShoreTel’s is 4.27.
• “They are extremely innovative in the way the solution is put together,” says the director of telecom for a midsize professional-‐services company.
⇒ The average customer-‐service score of all Market Challengers is 3.81; ShoreTel’s is 3.55.
• “Customer service isn’t with ShoreTel, it is with the VAR. Maybe ShoreTel isn’t as responsive as they need to be to the VAR, so there’s a lot of blame game going on. The VAR needs a better trouble ticket, tracking process for managing incidents,” says the director of IT for a small professional-‐services company.
• “The execution is the problem, their channel partner is terrible, but there’s no other choice, we can't go direct to ShoreTel. The partner is trying to improve, but has struggled to deal with complex problems, though it is responsive to quick break/fix,” says the architect for a midsize financial-‐ services company.
⇒ The average value score of all Market Challengers is 4.11; ShoreTel’s is 4.23.
• “Our ShoreTel solution has been a good investment for us,” says the IT manager for a small professional-‐services company.
Figure 9: Market Challenger: Alcatel-‐Lucent, IP Telephony, 2012
Alcatel
-‐Lucent
⇒ Alcatel-‐Lucent sees a large fall from 2011, going from an overall rating of 4.14 in 2011 to a 3.79 this year. Its biggest fall-‐offs come in technology (4.18 vs. 3.77) and customer service (4.36 vs. 3.77), while it held steady in value (3.86 vs. 3.85). Alcatel-‐Lucent continues to struggle with an uncertain future amid reports that the company is trying to sell off the enterprise division after divesting itself of its Genesys contact center business in 2011. Alcatel-‐Lucent trails overall Market Challenger scores in all categories, making it tough for it to make any headway in an extremely competitive market.
⇒ The average overall score of all Market Challengers is 3.99; Alcatel-‐Lucent’s is 3.79.
• “Overall the solution is innovative and up-‐to date,” says the CIO for a small hospitality company.
⇒ The average technology score of all Market Challengers is 4.05; Alcatel-‐Lucent’s is 3.77.
⇒ The average customer-‐service score of all Market Challengers is 3.81; Alcatel-‐ Lucent’s is 3.77.
⇒ The average value score of all Market Challengers is 4.11; Alcatel-‐Lucent’s is 3.85.
C
ONCLUSION
⇒ The IP telephony market is in a state of flux. New entrants such as Microsoft, coupled with expanding hosted service offerings, threaten to chip away at long-‐ time players in the on-‐premise market. Meanwhile, telephony vendors expand their platforms to embrace unified communications, virtualization, and mobility. Success in this market requires a broad set of options including mobile, virtual, and unified communications solutions, and for some organizations, IP contact center integration. The vendors that scored the highest in this year’s ratings reflect this ability to deliver a broad solution set, providing options for both on-‐ premise and in-‐the-‐cloud applications across a variety of end-‐points.
⇒ Vendor Selection: Based on the outcome of the PilotHouse program, here is Nemertes’ guidance (with vendors listed in priority order):
• Evaluate a minimum of five providers. Cisco, Mitel, Siemens Enterprise
Communications, and ShoreTel are solid options.
• If technology is your key concern, consider ShoreTel, Cisco, Avaya, and
Mitel.
• If customer-‐service is your key decision criteria, consider Cisco, Siemens
Enterprise Communications, and Mitel.
• If value is your key goal, consider Mitel, ShoreTel, Siemens Enterprise
Communications, and Cisco.
⇒ Differentiation of Leaders and Challengers: Overall there’s not much
differentiation between Leaders and Challengers. Market Leader Cisco was at or near the top in all areas, while fellow Market Leader Avaya finished third in technology, but much lower in customer service and value. This indicates the need to cast a wide net when evaluating potential solutions.
⇒ Advantages of Adding During Technology Refresh: Most companies investing in IP telephony will do so as part of a broader unified communications rollout. Evaluate IP telephony providers that can best integrate with your overall UC— and perhaps contact-‐center—architecture.
⇒ Improvement Outlook: Unified communications, virtualization, mobility, and cloud-‐based offerings will reshape IP telephony over the coming years. Evaluate vendors not only on where they are today, but where they will be tomorrow and their ability to offer innovative solutions to meet changing business
requirements.
M
ETHODOLOGY
The population includes individuals primarily from U.S. companies (based in the United States, but many of which are global multinationals) who are responsible for selecting or influencing the selection of suppliers of data-‐center and
communications products and services.
Sample Frame
In selecting the sampling frame, Nemertes has asked individuals in the following populations to rate their providers:
Nemertes Research IT executive database, limited to individuals who meet the criteria for the representative population. Individuals from this list represent primarily U.S. companies, but also include companies based elsewhere that have presence in North America. The database includes individuals who have participated in, or who have expressed interest in participating in our research, or with whom Nemertes’ analysts have established a business relationship.
U.S. business subscriber lists, including individuals who have opted to participate in surveys and who have been pre-‐screened to determine responsibility for selecting or influencing relevant products and services. Individuals participated in this project using two methods:
Benchmark interviews. This is a smaller percentage of the respondents, but they provide more detail on the reasons for their ratings. In addition to providing quantitative ratings, Nemertes’ analysts asked numerous detailed qualitative questions to gauge why they rated their service providers the way they did, as well as gathering other information about their usage of communications services.
Electronic survey. This is the largest percentage of the respondents. Those who meet the sample frame randomly received invitations to participate in the survey.
Benchmark participants spent one to three hours on the phone or in person with a Nemertes analyst discussing, among other areas, issues relating to their use of products and services. The electronic survey participants answered a subset of the benchmark questions that focus on rating the providers, stack-‐ranking important criteria, providing financial data, open-‐ended comments, and demographics.
Planned Sample Size
According to U.S. Census Bureau figures, there are 2,208,596 companies with five or more employees. Our goal was to receive responses from a minimum of 1,000 individuals, which would give us a 95% confidence level and 3% margin of error—if every individual rated every vendor in every technology area rated. We received
substantial ratings for each technology category (several hundred per category), but each vendor in each area did not receive a rating from every research participant. About 4,100 individuals accessed the survey or participated in a benchmark interview. Of those, about 1,500 meet Nemertes’ stringent standards to be
considered “valid.” Our survey tool automatically exited individuals employed by IT vendors and providers. Analysts reviewed all other ratings (survey and benchmark) line by line, and categorized as “invalid” those who demonstrated inconsistencies or inaccuracies in their responses as part of Nemertes’ complex qualification
methodology.
We achieved validity across the survey and interviews by ensuring the questions we asked were the same, and that the interview group and survey group represent discrete samples of the same population. Nemertes achieves survey and interview consistency through the use of pre-‐scripted interview forms and peer review of interview protocols. Analysts also relied upon their own knowledge of the technology areas, natural breakpoints in the data, and interview notes from the survey participants to further validate ratings.
Survey Sub-‐Groups/Stratification
Nemertes’ analysts researched which providers offer products and services in each category and created lists from which participants identified their primary service providers. Participants also were able to select “other,” and identify a service provider they use that may not be included on the explicit list provided.
The challenge is that some providers (Market Leaders) have thousands or tens of thousands of business customers and significant market share, while others (Market Challengers) have a few hundred or few thousand customers and smaller market share. We realized some providers would garner a relatively large number of ratings, based on the number of customers they have, while others would have a relatively small number of ratings.
Therefore, we created the two distinct categories for the awards, Market Leaders and Market Challengers, and compared providers within each category. Nemertes placed providers within each category based on its own research and publicly available data. Analysts also examined natural breakpoints in the data. Market Leaders typically have >10% of market share, based on these analyses. Market Challengers typically have smaller market shares. In some categories, there were not enough ratings to issue an award in the Market Challenger category, or the market is so new that all vendors are considered Challengers. In these cases, Nemertes issues an award only in the appropriate category.
Nemertes reserves the right to address acquisitions occurring during the
benchmark and survey period on a case-‐by-‐case basis. Unless otherwise noted, an acquisition merging two companies in the same award category must be complete before the start of the survey and benchmark interview period to be counted as one company in the ratings.
Awards
Nemertes is issuing awards in the following categories:
Nemertes PilotHouse Awards, 2012
Award Category
Market Leaders
Market Challengers
Advanced Communications Services
MPLS Services
Carrier Ethernet Services
Internet Access Services
Wireless and Mobility
Wireless LANs
Mobile Device Management No award
Application Delivery
Application Delivery Optimization
Voice Communications
IP Telephony
Managed IP Telephony
Data-Center Technologies
Servers for Virtualization
Unified Communications Unified Communications IP Contact Centers Video Conferencing Security Data-Center Firewalls
Timing
The Web-‐based survey was conducted between March and May 2012. The benchmark research was conducted between January and April 2012.
Incentives to Participate & Time Commitment
Participants of the electronic survey received a small incentive for participating in the survey. Participants from Nemertes’ database receive the findings and are invited to participate in a results-‐overview Webcast, in exchange for their time. The Web-‐based survey takes about 15 minutes to complete; the benchmark requires one to three hours of participants’ time.
Future Plans
Nemertes plans to conduct its PilotHouse Awards program annually, though it retains the right to cancel or revise the program at any time.
About Nemertes Research:
Nemertes Research is a research-‐advisory and strategic-‐consulting firm that specializes in analyzing and quantifying the business value of emerging technologies. You can learn more about Nemertes Research at our Website:
http://www.nemertes.com