Performance
Support Tools:
A Case Study
How Aetna Improved the Quality of its
Call Center and Claims Operations
Karen O’Leonard
February 2005
Table of Contents
Performance Support Tools: A Case Study ... 3
The Business Problem ... 3
The Solution... 3
Results... 4
Lessons Learned... 6
Future Evolution of Program... 7
About Us... 8
About This Research ... 8
Table of Figures
Figure 1: Screen Shot of the epiLearn Performance Support Tool ... 4Figure 2: Classroom Training Required Before and After Performance Support Tools.... 5 Figure 3: Relationship Between Usage of Performance Support Tools & Late Payments . 6
Performance Support Tools:
A Case Study
The Business Problem
Aetna has 6,000 customer service and support employees located in call centers and claim operations around the country. The company’s National Customer Operations (NCO) Learning and Performance group develops all of the web-based training materials for these employees, and is known for its cutting-edge learning program development and design expertise. The team has won numerous awards, including the Silver Award at the October 2004 Training conference for its “Dr. Bones” online medical terminology program.
Three years ago, the NCO Learning and Performance team was organized to facilitate knowledge transfer. Aetna’s call and claims systems had been evolving rapidly from “green screen” mainframe terminals to GUI-based systems, and more recently to web-based applications. Aetna’s training programs struggled to keep pace with the rapid change of the systems. Traditionally, training was delivered through instructor-led courses. Since employee performance is based largely on production (i.e. the number of claims processed per hour), taking an employee away from work to attend a training class was very costly. Aetna’s learning team began searching for alternative methods to provide effective training, while minimizing the impact on employee work schedules.
The Solution
The 80-20 Rule
In Aetna’s traditional approach, new employees attended a 13-week training program, where they were expected to learn and understand every aspect of Aetna’s call and claims handling processes and procedures. The volumes of training books and reference materials filled four feet of shelving in a room and, as one Aetna learning professional quipped, “…amounted to more reading than required by a first-year law student.” This training approach was just not feasible or economical.
Aetna’s learning team conducted research by monitoring call center employees’ tasks over time. They found that call center employees used roughly 20% of the training material 80% of the time. In other words, they perform a small subset of tasks on a daily basis, and a large number of tasks on an infrequent basis. A more sensible training approach, Aetna concluded, would be to focus on the 20% of the job that is critical to an employee’s every day job. These tasks and processes should be mastered by every call center employee.
Aetna
refocused its
training on the
20% of tasks
that are critical
to an
employee’s
every day job.
For the other
80% of tasks,
the company
provided
performance
support tools.
For the other 80% of the job, mastery is less important. Employees are unlikely to retain knowledge on tasks performed infrequently, so the traditional training on these tasks was ineffective. Instead, providing support tools so that employees can access necessary information to complete these tasks when needed would be a more economical and effective approach.
Performance Support Tools
With this new strategy, Aetna began developing performance support tools for its call center applications. These tools are either process-driven or informational in nature. Informational tools will display a definition or description as the user moves a mouse over a term, or they may provide short cuts or links to other resources. Process-driven tools are typically more complex. They may walk an employee through a task, checking off each item as it is completed. More sophisticated tools notify the employee that a step has been skipped so he or she can go back and correct the error.
Aetna’s learning team built many of these tools internally using HTML and Javascript. In addition, the company licensed Epiance’s epiLearn performance support system as one of its primary process-driven tools.
Figure 1 shows a screen shot of Epiance’s epiLearn tool for an HR application. Although not specific to call center operations, the screen shot provides a flavor for how the tool is used.
Figure 1: Screen Shot of the epiLearn Performance Support Tool
In this example, the HR professional needs to update an employee’s personal data record. epiLearn brings up a list of steps need to complete the task. As the employee completes each step successfully, the step is checked off the list. If a step is skipped or performed incorrectly, the tool notifies the user, who then can go back and complete the task correctly.
Developing
performance
support tools
requires a
different set of
skills than
traditional
content
development.
Aetna’s call center employees use epiLearn to complete similar tasks, such as updating a customer’s address or personal information, changing the vehicle listed on the insurance record, and opening or closing a claim. In each case, the tool works in much the same way as described above—providing the steps needed to complete the task, checking off each step as it is completed, and notifying the employee if an error has been made.
Results
In 2003, Aetna developed and deployed 5 performance support tools; by the end of 2004, the company had over 40 support tools in operation. These tools have enabled Aetna to reduce the 13-week training course for new hires to 11 weeks—saving each call center employee 2 weeks of time away from work. “Our training approach is not about making employees experts on all tasks,” said Dave Ulm, Project Manager of Performance Support. “We train the employees to know the basics really well. For everything else, there are performance support tools.”
Number of Weeks in Classroom Training:
Before and After Performance Support Tools
1 3 5 7 9 11 13 15
Instructor-led training (ILT) only ILT + performance support tools
# w e e ks of class ro om tr aining
Newly hired call center employees now spend two weeks less in classroom training.
Figure 2: Classroom Training Required Before and After Performance Support Tools
After rolling out a new performance support tool in March 2004, Aetna tracked the usage of the tool and quality of claims processed for a 3-month period. Aetna tracked tool usage through cookies placed on employees’ desktops after they registered the tool. The study showed a clear correlation between the use of the performance support tool and the quality of the claims processing:
• 100% of the employees who used the performance support tool processed their claims without error.
• In every case where the processed claim contained an error, the call center employee handling the claim had not used the tool.
In a second study, Aetna gathered data on usage of performance support tools and late payments. Again, the results showed a direct correlation between the percentage of employees who used the tool and the number of late payments. The greater the percentage of employees using the tool, the lower the number of late payments.
Relationship Between Tool Usage and Late Payments
% of employees using tool No. of late payments
As the percentage of employees using the performance tool increased, the number of late claims payments decreased.
Figure 3: Relationship Between Usage of Performance Support Tools & Late Payments
Inaccurate and untimely claims processing have huge cost implications for insurers. Reworking claims to correct errors carries significant costs in personnel time. Inaccurate and late payment of claims adversely impact customer satisfaction. And, it is difficult to recoup money in the case of overpayment. The performance support tools have proven to be extremely effective in improving quality and reducing costs.
Lessons Learned
“Performance support tools are really a different animal from other training development,” says Ulm. “They look, feel, and act differently.” For most of their online training programs, Aetna uses the “3 e™” approach: educate; engage; entertain. But for performance support tools, the last two (engage and entertain) don’t make sense. The tools must be focused and provide the necessary information quickly, then get out of the way. There is no need or desire for engagement or entertainment.
Ulm recommends that people building performance support tools must look at what the employee will use, and determine how to make it fit with what they do. This requires more user interface expertise than traditional content development. One rule-of-thumb Aetna uses, for example, is to make the performance support tools accessible from the left side of the screen, which conforms to typical website navigation toolbars. And, the information must be no more than three clicks away or it will take too long for the learner to access.
Future Evolution of Program
The success of the performance support tools has also brought about drawbacks. In 2005, the company plans to have over 100 tools in use. The sheer number of support tools has become unmanageable and has added too much complexity for the learner. So, the company has licensed Epiance’s Intelligent Dialogue system to manage the learning tools and simplify the process. Epiance’s tool is context sensitive to where the employee is in the application, and opens tools only appropriate to that section.
In addition, Aetna’s eCoach technology, based on Epiance’s “cue cards,” will be the primary help tool for call center employees. The eCoach technology goes one step further than the current performance support tools in use. For each process, eCoach brings up a list of tasks required, and, as the user completes each step, it is checked off the list. This way, call center employees immediately get the support they need, and the confidence of knowing that their tasks have been completed successfully. The company plans to officially launch a new call center system with the eCoach technology in April 2005.
About Us
Bersin & Associates is the only research and consulting firm solely focused on What Works™ e-learning research. With more than 20 years of experience in e-learning, training, and enterprise technology, Bersin & Associates provides a wide range of services including market research, best-practices, vendor and product analysis, corporate workshops, corporate implementation plans, and sales and marketing programs. Some of Bersin & Associates innovations include a complete methodology for learning management system (LMS) selection and application usage, an end-to-end architecture and solution for training analytics, and one of the industry's largest research studies on blended learning implementations.
Bersin & Associates offers the industry's first e-learning research subscription, "The E-Learning Research Center," (http://www.elearningresearch.com), which offers up-to-date research, product selection guides, white papers, and access to other experts online. This service enables corporate training managers, vendors, and consultants to make faster, better decisions about e-learning strategy, programs, and technology. Bersin & Associates can be reached at http://www.bersin.com or at (510) 654-8500.
About This Research
Copyright © 2005 Bersin & Associates. All rights reserved. What Works™ and related names are registered trademarks of Bersin & Associates. No materials from this study can be duplicated, copied, republished, or re-used without written permission from Bersin & Associates.
