Executive Summary
From regulatory changes to demographic trends, healthcare systems face unprecedented pressure to deliver higher-quality care in a more coordinated, cost-effective, and sustainable fashion. These changes are causing both the volumes of medical data to grow and the number of simultaneous user requests against the healthcare database to soar.Now, a new generation of technologies from Intel, InterSystems, and VMware offers significant advances in scalable, high-throughput solutions for Epic’s electronic medical record (EMR) systems. To explore the scalability impact of these new technologies, Epic and InterSystems engineers conducted a series of performance tests with the four-socket Intel® Xeon® processor E7-8890 v3, InterSystems Caché® 2015.1, Epic® EMR software, and a beta version of VMware ESXi™ 6.0.
Running workloads that reflect realistic healthcare environments, they achieved 7 million database accesses per second (measured as global references or GREFs in the Caché environment) on a single system while maintaining responsiveness for users. This is a 60 percent increase over the 4.2 million GREFs the test team recorded using Epic software with Caché® 2013.1 and VMware ESXi™ 5.5 on the Intel® Xeon® processor E7-4890 v2. It is also more than triple the 1.9 million GREFs achieved on the Intel® Xeon® processor E7-4860 with Epic software, Caché®
InterSystems and VMware Increase Database
Scalability for Epic EMR Workload by 60 Percent
with Intel® Xeon® Processor E7 v3 Family
Delivering reliable, affordable performance for healthcare, InterSystems Caché® 2015.1, VMware
vSphere® ESXi™ 6.0, and the Intel® Xeon® processor E7-8890 v3 achieve 7 million database accesses
per second with an Epic® EMR workload compared to 4.2 million on previous-generation technologies.
These results highlight the increased performance and scalability of the Intel® Xeon® processor E7 v3 family. They illustrate the synergies that can result when technology innovators collaborate to optimize their solutions and address customer requirements. Building on this next-generation foundation, even very large healthcare systems can implement their Epic and InterSystems software on the Intel Xeon processor E7 v3 family with confidence. By taking advantage of the platform’s headroom and scalability, they can accommodate larger patient volumes and a higher volume of user requests against the database on a single, scale-up platform.
If an organization exceeds the vertical scalability of the Intel Xeon processor E7 v3 family, it can use InterSystems Caché’s Enterprise Cache Protocol (ECP)™ to scale horizontally. Tests with Caché 2015.1 and ECP technology on the Intel Xeon processor E7-4890 v2 achieved over three times more end-user database accesses per second than Caché 2013.1 on the virtualized platform based on the Intel Xeon processor E5-2680.3 The result of this
combination of technologies is steadily expanding scalability that keeps pace with the rising demands on the healthcare enterprise while creating a more reliable, cost-effective and agile data center.
“Moving from the Intel®
Xeon® processor E7-4890
v2 with Caché® 2013.1 and
vSphere® ESXi™ 5.5 to the
Intel® Xeon® processor
E7-8890 v3 with Caché® 2015.1
and vSphere® ESXi™ 6.0, we
saw the scalability of a single
operational database server
increase by 60 percent. With
these gains, we expect our
customers to scale further with
a smaller data center footprint
and lower TCO.”
– Seth Hain Senior Performance Engineer Epic
Driving the Need for Scale-Up
Performance in Healthcare
Healthcare systems around the world are transforming to address a common challenge: How do we provide affordable, high-quality care as our population grows, ages, and increasingly develops lifestyle-related chronic conditions?The efforts to meet this challenge are triggering a cascade of changes. Healthcare systems are expanding and consolidating to achieve economies of scale. They’re focusing on team-oriented services and collaborating with community- and home-based health organizations to ensure a positive patient experience across the continuum of care. They’re coping with a shifting set of regulatory and reporting requirements, a more complex healthcare ecosystem, intense cost pressures, and higher patient loads. These changes are having a dramatic impact on EMR systems and healthcare databases, which must maintain responsiveness while handling more concurrent users, more diverse workflows, and a greater range of application functionality accessing the database. As a result, database scalability is more crucial than ever to patient care, provider productivity, and enterprise success.
Now, a new generation of Intel’s most robust server processors provides exceptional scalability, performance, and headroom to help healthcare organizations meet the challenges. By cost-effectively scaling their databases with the Intel Xeon processor E7 v3 family, hospital systems can position themselves to not only deliver
outstanding care for their patients, but also thrive in the fast-changing new world of healthcare.
Benchmarking to Help Ensure a
Robust Experience for Users
Epic maintains a rigorous evaluation process that includes scalability and performance tests. Epic uses these results, along with a variety of additional factors, to develop comprehensive sizing guidelines and configuration recommendations aimed at ensuring reliable, productive experiences for users of its EMR software.With the introduction of a new generation of hardware and software elements, engineers from Epic and InterSystems ran a series of tests using the Intel Xeon processor E7-8890 v3, InterSystems Caché 2015.1, Epic EMR software, and a beta version of VMware ESXi 6.0. (See sidebar, A New Generation of Key Technologies.) The test engineers compared their results to those achieved with previous-generation technologies, including the Intel Xeon processor E7-4890 v2 and the Intel Xeon processor E7-4860. Tests with the newest technologies were performed in January 2015 at Epic’s headquarters in Verona, Wisconsin. Intel engineers provided configuration assistance.
Table of Contents
Executive Summary . . . 1
Driving the Need for Scale-Up Performance in Healthcare . . . 2
Benchmarking to Help Ensure a Robust Experience for Users . . . 2
System Configurations . . . 3
Workloads . . . 4
Methodology . . . 4
Results . . . 4
Moving Forward with Confidence on a Sustainable Foundation . . . 5
System Configurations
Table 1 summarizes the test platform configurations and provides the platform-level database throughput (in global references) achieved by three generations of Intel® processors. The test team focused on practical, real-world relevance rather than aiming for artificial peak performance. Platforms
were configured to Epic’s best practices for Caché configurations, and in many cases used default settings and standard system elements such as DIMMs.
All systems were configured to ensure that elements such as memory, network, and storage capacity would not artificially limit scalability. The BIOS was configured to run in high-performance mode.
A New Generation of Key Technologies
When Intel, InterSystems, and VMware made major advances in their flagship products, Epic wanted to understand what this might mean for organizations using its EMR software. Epic engineers benchmarked their software using the three new technologies: • Intel® Xeon® Processor E7 v3 Family. Intel’s
newest server processor line, the Intel Xeon processor E7 v3 family is designed for the most demanding, business-critical computing workloads. Now available with up to 72 cores, it improves scalability through higher performance, up to 12 TB of memory, and a range of I/O enhancements. Meeting the highest enterprise requirements for reliability and uptime, it includes more than 40 reliability, availability, and serviceability (RAS) features. The resulting balanced of high performance and throughput allows for highly scalable platforms.4 An open,
industry-standard ecosystem helps reduce costs and accelerate innovation by offering a broad choice of applications, tools, and peripherals.
• InterSystems Caché® 2015.1. InterSystems
Caché is the industry’s leading SQL/ NoSQL data platform designed to deliver high performance and massive scalability. Identified as a Leader in the Gartner Magic Quadrant5 for operational database
management systems, Caché combines advanced data management, integration, and analytics. For Caché 2015.1, InterSystems engineers worked with Intel to take maximum advantage of the Intel® multi-core architecture and instruction set. InterSystems’ code optimizations enhance performance for non-uniform memory access (NUMA), memory management, thread management, and other areas. This enables Caché 2015.1 to optimize its use of CPU resources and increase performance and scalability as the number of cores and threads increases.
• VMware vSphere® ESXi™ 6.0. VMware
solutions use the capabilities of Intel® Virtualization Technology (Intel® VT)4 to
enhance performance and availability on Intel® processor-based platforms. VMware vSphere 6.0 provides more than 650 new features and innovations, enabling organizations to create a highly available, resilient, on-demand infrastructure that can help reduce data center costs and increase both system and application uptime. Higher configuration maximums in vSphere 6.0 allow for a one-to-one mapping of physical cores to vCPUs with the Intel Xeon processor E7 v3 family. This empowers organizations to more efficiently utilize the physical hardware and to virtualize highly
TECHNoloGy PRoCESSoR E7 V3 INTEl® XEoN® FAMIly INTEl® XEoN® PRoCESSoR E7 V2 FAMIly INTEl® XEoN® PRoCESSoR E7 FAMIly
Family Launched May 2015 February 2014 April 2011
Processor Intel® Xeon® processor E7-8890 v3 Intel® Xeon® processor E7-4890 v2 Intel® Xeon® processor E7-4860 Platform-Level
Sizing 7.0 million GREFs 4.2 million GREFs 1.9 million GREFs
Application Epic Epic Epic
Database Caché® 2015.1 (pre-release) Caché® 2013.1 Caché® 2011.1 Virtualization vSphere® 6.0 (pre-release) vSphere® 5.5 vSphere® 5.0 Number of vCPUs
Configured in the
Virtual Machine 72 60 32
Operating System Red Hat Enterprise Linux* 7 Red Hat Enterprise Linux* 6 Red Hat Enterprise Linux* 6 Clock Frequency 2.5 GHz 2.8 GHz 2.4 GHz
Cores per Socket 18 15 10
Total Cores 72 60 40
Total Memory 512 GB 512 GB 512 GB
Memory Speed 2133 MHz 1600 MHz 1066 MHz
Network Speed 10GbE 10GbE 10GbE
Hyperthreading Enabled Enabled Enabled Intel®
Virtualization
Technology Enabled Enabled Enabled
NUMA Enabled Enabled Enabled
Workloads
Reflecting the focus on practical results, Epic has designed the workloads and data sets used in the tests to reflect realistic, meaningful work scenarios of clinicians at large healthcare enterprises. Epic increases the validity and usefulness of the test results by working with a customer site to obtain a copy of its database. (Potential personal health information such as patient identifiers is scrambled, and relevant privacy protections are in place to protect data confidentiality.) For the benchmarks described in this paper, Epic collaborated with a large healthcare organization to obtain a sanitized copy of its multi-terabyte database.
Methodology
The testing used actual elements of the Epic application suite and their associated database code—the same code base used by Epic users, the same suite and scope of enterprise application code, and the same network protocols. InterSystems Caché is configured for symmetric multiprocessing (SMP) operation in a virtualized environment with one virtual machine (VM) per system. Client-side user loads are generated by Epic simulation tools. These user loads simulate the second-to-second activities of a broad range of EMR users, including physicians, nurses, pharmacists, and others. The loads also model health systems’ scale-up activities realistically, reflecting the addition of more users and application functionality.
Epic and InterSystems engineers set out to validate the level of throughput the systems could achieve (in
GREFs) while maintaining acceptable responsiveness. They stressed the database by incrementally increasing the volume of connected users and thus the volume of database accesses.
As the loads increased, the test team measured system metrics and application response times at each load level. Load levels were considered valid only if the application response times were deemed fast and system metrics remained within a specified range. Results
• The Intel Xeon processor E7-8890 v3, InterSystems Caché 2015.1, Epic, and a beta version of VMware ESXi 6.0 delivered sustained performance of 7 million GREFs on a single system while maintaining excellent response times (see Figure 1).
• This was 60 percent more than the 4.2 million GREFs sustained with the Intel Xeon processor E7-4890 v2, Caché 2013.1, Epic software, and VMware ESXi 5.5.
• The new solution delivered more than a threefold increase over the 1.9 million GREFs achieved on the Intel Xeon processor E7-4860 with Epic software, Caché 2011.1, and VMware ESXi 5.0.
Application Epic Epic Epic
Database Caché 2011.1 Caché 2013.1 Caché 2015.1 (pre-release) Virtualization vSphere 5.0 vSphere 5.5 vSphere 6.0
(pre-release) Maximum vCPUs 32 60 72 1 2 3 4 5 6 7 8 1.9M GREFs (database accesses/sec) 4.2M GREFs (database accesses/sec) 7.0M GREFs (database accesses/sec)
E7-4860
E7-4890 v2
E7-8890 v3
Figure 1. More Than Triple the Platform-Level Sizing Capacity Over Three Generations1,2
Highlighting the significant advances in scalability InterSystems achieved with Caché 2015.1, global references rose from 4.2 million with Caché 2013.1 on the Intel Xeon processor E7-4890 v2 and ESXi 5.5, to 4.9 million with Caché 2015.1 on the same system.
The test team found that VMware showed minimal consumption of resources as the system scaled (see Figure 2). Even using the beta version of VMware ESXi 6.0, the hypervisor was extremely frugal in its use of CPU resources. This efficiency, along with the increase in the number of vCPUs and the size of the VMs, enables organizations to improve scalability and availability while leaving nearly all virtual CPU resources for the VM. Figure 2 depicts raw performance, and shows scaling beyond the 7 million GREFs shown in Figure 1, where more practical constraints were imposed.
Tests such as those described in this paper play an important role in the sizing advice Epic provides for its customers. However, many other factors are important. Epic’s detailed sizing advice also encompasses factors such as variance across different types of workload patterns, maintaining
Figure 2. Minimal Hypervisor Impact on CPU Resources (Guest versus Host CPU Utilization)
Compute Utilization - CPU Used%
Database Accesses Per Second 0 1,770,000 Host_Util Guest_Util 3,430,000 5,190,000 7,040,000 8,080,000 8,670,000 25 50 75 100
real-time replicated environments, and other conditions. To achieve optimal performance for your Epic deployment, consult your Epic representative.
Moving Forward with Confidence
on a Sustainable Foundation
Today’s healthcare environment presents significant challenges and opportunities. With rich digital data and intelligent applications, healthcare organizations can improve collaboration and deliver personalized, team-oriented care. With increasingly sophisticated analytics, theseorganizations can turn clinical data into actionable insight, improving treatment plans and, ultimately, patient outcomes. Empowered with insights, they can improve quality and costs, reducing waste and enhancing the experience for patients and providers alike. They can capitalize on rapid advances in areas ranging from wearables to genomics, envisioning whole new ways of delivering care.
The Intel Xeon processor E7 v3 family gives healthcare organizations the sustainable foundation they need to address their challenges and make the most of the opportunities. By choosing
Intel’s newest and most powerful server processors with VMware ESXi 6.0 for their InterSystems Caché data platform, Epic customers gain a powerful, affordable, and highly available platform for scalable, data-intensive computing.
With the outstanding performance and scalability these technologies provide, even very large medical systems can confidently choose Intel® architecture-based platforms for their critical EMR software and healthcare databases. These systems can support more users cost-effectively and provide answers more rapidly, enabling organizations to potentially reduce per-patient costs, deliver a better patient and clinician experience—and prepare to thrive as leaders in a data-enabled, value-based healthcare environment.
“The new Intel® Xeon®
processor E7 v3 family is
a huge win for healthcare
organizations that want to
deliver breakthroughs in
patient care while controlling
costs. With generation
scalability of 60 percent,
organizations implementing
Epic with Caché® 2015.1
get much more power to
implement their connected
health strategy—to link data,
applications, and processes,
and make smarter decisions
and provide the best possible
care for their patients.”
– Robert Nagle Vice President, Data Platforms InterSystems Corporation
1 Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured
using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to
www.intel.com/performance
2 Intel does not control or audit the design or implementation of third party benchmark data or Web sites referenced in this document. Intel encourages all of its customers to visit the referenced
Web sites or others where similar performance benchmark data are reported and confirm whether the referenced benchmark data are accurate and reflect performance of systems available for purchase.
3 For more on these results, see Data Scalability with InterSystems Caché and Intel® Processors, 2015.
http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/xeon-e7-intersystems-cache-paper.pdf
4 Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation.
Performance varies depending on system configuration. Check with your system manufacturer or retailer or learn more at [intel.com].
5 InterSystems Recognized As a Leader in Gartner Magic Quadrant for Operational DBMS, October 16, 2014.
http://www.intersystems.com/our-products/cache/intersystems-recognized-leader-gartner-magic-quadrant-operational-dbms/
Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.
Copyright© 2015 Intel Corporation. All rights reserved.
Intel, the Intel logo, and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. InterSystems, Caché, and ECP are trademarks or registered trademarks of InterSystems Corporation. Epic is a registered trademark of Epic Systems Corporation.
VMware, vSphere, and ESXi are trademarks or registered trademarks of VMware, Inc. in the United States and/or other jurisdictions. *Other names and brands may be the property of their respective owners.
Printed in USA 0415/SA/MESH/PDF Please Recycle 332398-001US
