Technical Case Study
By Robert Syphax, Chief of
Storage Operations for the California Natural Resources Agency and the Department of Water Resources
California is the world’s eighth-largest economy, and our
IT challenges exceed those of many national governments.
That was particularly true during the budget crisis that began
in 2008. In the face of severe budget cutbacks, state agencies
needed to radically change the way they operated to keep
meeting their missions to serve citizens.
The State CIO, Carlos Ramos, developed a vision to improve IT efficiency by adopting shared services. Instead of each department purchasing and managing its own IT infrastructure, all departments in an agency would share a common IT infrastructure. The idea was to slash IT capital and operational expense—and increase service levels at the same time.
State of California Sets a New Standard
for IT with Shared Services
How the California Natural Resources Agency harnessed the power of cloud computing
to reduce IT capital costs by 42% and speed up service delivery by 70%
2 The “Before” Picture: Storage Islands, Unpredictable Costs,
Increased Demands
The vision struck a chord with Tim Garza, CIO for the California Natural Resources Agency (CNRA). The agency consists of 29 departments, boards, and commissions with a common mission to restore, protect, and manage the state’s natural, historical, and cultural resources for current and future generations. Departments range in size from environmental consortiums with a few dozen employees to CalFire, with 7,200 employees across California. At the time, each department in the California Natural Resources Agency maintained its own siloed IT infrastructure. Duplicate equipment and management efforts were costly. Some departments had excess capacity while others were strained. The time needed to fulfill service requests varied widely. And departments couldn’t share information to collaborate on the mission. The Natural Resources Agency’s Newest Resource:
Private Cloud Computing
Today, all departments in the California Natural Resources Agency consume shared services from a private cloud built on various industry technologies and NetApp® storage technology. In total, the private cloud stores 8 petabytes—a combination of virtual machines and data.
The shared-services model has transformed the economics of IT services for the agency. We’ve increased storage capacity by 300% while shrinking our storage physical footprint by 30%. At the same time, we’ve reduced overall IT capital expense across the agency by 42% and operational expense by 35%. The CNRA private cloud has also helped agency departments become more agile. They no longer have long wait times for computer resources required for critical projects to restore, protect, and manage the state’s natural resources. Instead, they receive capacity on demand. In addition, departments have been able to avoid costly IT budget spikes associated with equipment refresh costs every four years.
Nondisruptive Operations Support 24/7 Government
Enabling nondisruptive operations is one of the most compelling advantages of NetApp clustered Data ONTAP for the California Natural Resources Agency. Storage availability is critical because we don’t have physical servers. Almost 98% of our applications are virtual servers that live on storage, and agency personnel count on these applications to plan responses to fires and floods, for example.
With nondisruptive operations, we don’t need to schedule downtime windows to add or maintain storage nodes. Instead, we can move data and virtual machines to any other array so that our customers can continue working without interrup-tion. We’ve increased availability from the 99% we provided before we used NetApp storage to 99.99% today. As we complete our migration to NetApp clustered Data ONTAP, we expect to reach our goal of 99.999% availability. NetApp nondisruptive operations also help to keep the workforce fully produc-tive, because personnel don’t have to sit idle while we’re doing upgrades and refreshes. That’s especially important in today’s business climate in which staff resources are lean.
More Efficient Storage, More Efficient Government
• 300% more storage capacity • 70% faster storage provisioning • 42% less overall IT capital expense
across the agency
• 35% less overall IT operational expense across the agency
• 30% smaller storage physical footprint • 50% increase in cooling efficiency
and 40% increase in power efficiency • 8 petabytes of data
Lower Costs Through Infrastructure Consolidation and Storage Efficiency NetApp storage played a big part in helping us shrink the overall agency infrastructure footprint from 22,000 to 6,500 square feet. As a result, cooling efficiency increased by 50% and power efficiency by 40%. Freeing up space also gives us room for growth. That’s important for the Natural Resources Agency because we look at historical trends. Models of the effects of climate change on water levels, for example, might use 100 years of data. So the Department of Water Resources never throws away data, and storage increases 30% to 40% every year.
One way we reduced infrastructure floor space was by consolidating separate IT infrastructures in multiple locations to one shared infrastructure. The other was increasing storage efficiency—to the point that we store 300% more data in 30% less space. In combination, NetApp deduplication, compression, and thin provisioning technology increased storage efficiency by 40%. For example, deduplication eliminates the need to maintain hundreds of copies of a virtual machine or a separate copy of a virtual desktop for all employees. Instead, we store one golden image along with just the changes to the other copies. Thin provisioning, in turn, helps us to not purchase more storage than is actually used. If a department requests 100TB for a new application, we allocate it using thin provisioning. All 100TB are there if the customer needs them. But if the customer overestimated the need or later scales back application use, the capacity is available for other uses.
Capacity on Demand: Customers Receive Storage 70% Faster
Departments can now receive new storage capacity in as little as a few hours, compared to three to four weeks on our legacy storage platform. That’s because provisioning capacity from a shared pool with NetApp OnCommand System Manager takes just a few clicks. Our customers receive new storage capacity more quickly than they could provision it themselves—a major reason why they move to the CNRA internal private cloud.
Easy provisioning also benefits our IT team. Now that we’re not bogged down in day-to-day operations, we have more time for strategic initiatives to support the mission. The CNRA private cloud has helped to increase IT business value by allowing us to better align our IT resources and services to the needs of our business programs.
High Performance for Big Data Analytics and Virtual Desktops
Performance of the Department of Water Resources’ geographic information systems and data-intensive applications such as SAP® improved when we moved them to the CNRA private cloud platform. This is due to the use of the Data ONTAP operating system and its use of on-board PAM cards to assist in the execution of application requests. In addition, NetApp flash storage, deduplication, and compression features improved the storage efficiency and performance of scientific and engineering applications, resulting in the ability of scientists to perform critical computation and analysis more effectively and in reduced time frames. This helps them more accurately predict the effects of changes in water flow and water level on commercial fishing, for example.
Four Requirements for Storage in a Multi-Tenant Private Cloud
Only NetApp storage met all of our requirements:
Secure multi-tenancy: Our agency
works with sensitive data, so security is essential. For example, information about critical infrastructure like dams and aqueducts can’t get into the wrong hands. Using NetApp MultiStore®
technology, we’ve created a “virtual partition” for each department. The virtual partitions are securely isolated within the shared environment. Each department can manage its own datastore without making it accessible to anyone else. Secure multi-tenancy helped make the shared-services cloud attractive.
Nondisruptive operations: Taking
virtual machines offline to add, upgrade, or maintain storage arrays can interrupt disaster response or critical research. That’s unacceptable in our agency. With the NetApp clustered Data ONTAP®
operating system, we can add or main-tain arrays with zero interruption to our customers’ business.
One management interface: NetApp
V-Series storage virtualization controllers helped to minimize capital outlay because we could continue using existing third-party disk drives. We use one interface, NetApp OnCommand® System Manager,
to manage all storage arrays from NetApp and our legacy vendor, which we’re phasing out. Allowing our staff to focus on NetApp technology lets them build up their skills, helping to improve time to deliver and stability.
Storage efficiency: NetApp
deduplica-tion, compression, and thin provisioning technology increase storage efficiency. That’s important because storage space is at a premium for the agency, and storage takes up 70% of our rack space.
4
The virtual desktop infrastructure (VDI) is also hosted in the CNRA private cloud. The VDI experience improved even more because of NetApp virtual storage tiering and flash storage. These technologies improve the performance of virtual desktops and other applications with high input-output operations per second (IOPS). We’ve offloaded a large amount of read IOPS from disk.
Interdepartmental Collaboration
In addition to increasing IT efficiency, our shared-services model created a foundation for improved interdepartmental collaboration. Departments can grant each other permission to view their data in the shared environment. A scientist studying the effects of climate change on a lake in Central California, for example, can also view geospatial data from other departments pertaining to fire hazards, fishing conditions, nearby oil and gas wells, and more. A Look Inside Our Private Cloud
The agency’s shared-services cloud platform includes NetApp storage, HP BladeSystem c7000 servers, VMware® vSphere® and vCloud Director®, and Cisco® Nexus® switches. The CNRA team can quickly orchestrate the provisioning of virtual computer resources including storage utilizing a combination of VMware software and NetApp Data ONTAP. In the future, CNRA will offer customers the ability to self-provision storage and other computer resources. Cloud storage consists of two partner pairs of NetApp V6240 storage virtualiza-tion controllers. The controllers front-end legacy storage, which lets us continue using them. NetApp supports all common protocols, and right now we use Fibre Channel, Network File System (NFS), and Common Internet File System (CIFS) protocols.
We migrated from NetApp Data ONTAP in 7-Mode to NetApp clustered Data ONTAP in 2014. Today we have three high-availability storage arrays using NetApp clustered Data ONTAP. Moving the data from our legacy storage platforms to clustered Data ONTAP immediately freed up five racks.
Our VM and storage teams both use the VMware management console. A NetApp management plug-in lets them manage storage as well as virtual machines. Most storage vendors write to the same VMware API, but we’ve found that the NetApp solution works the best. This convenient way to monitor capacity and VM health helps us identify storage issues before they affect our customers. Using the VSC plug-in, VM administrators are able to clearly identify the relationship between their datastores and the underlying volumes presented by the NetApp storage system. This aids in troubleshooting problems and pro-viding the proper tier of storage to customer environments.
We take daily backups. Backup and recovery times are two to three times faster with NetApp than with our legacy storage platform. The reason is the tight compatibility between NetApp Data ONTAP in 7-Mode and Sympana IntelliSnap Recovery Manager. We’re in the process of implementing a disaster recovery data center. We’ll use a NetApp FAS6250 storage system running clustered Data ONTAP and SnapMirror® technology to replicate data daily.
Robert Syphax
Chief of Storage Operations The California Natural Resources Agency and the Department of Water Resources
Mr. Syphax’s organization provides services to the Department of Water Resources that include storage, backup, and recovery. The Department of Water Resources provides water delivery, power generation, and flood protection to the State of California, and also focuses on environmental concerns related to water quality.
NetApp Products
• NetApp FAS6290 and FAS6250 storage systems
• NetApp clustered Data ONTAP • NetApp Data ONTAP in 7-Mode • NetApp V6240 storage
virtualization controllers • NetApp MultiStore • NetApp deduplication,
compression, and thin provisioning
• NetApp Flash Cache™ intelligent caching • NetApp SnapMirror • NetApp OnCommand
System Manager • NetApp Virtual Storage
Console for VMware vSphere
Protocols • Fibre Channel • NFS • CIFS Third-Party Products • HP BladeSystem c7000 • VMware vSphere and
vCloud Director • Cisco Nexus switches • CommVault Sympana 10 and
IntelliSnap Recovery Manager Bottom Line: More Agile, Efficient Government
By providing more capacity and capabilities in a shared-services environment, NetApp storage helps the California Natural Resources Agency set new standards of excellence for government IT. Now departments can start new projects to support the mission without waiting weeks for servers and storage. Projected data growth of 30% to 40% doesn’t faze us because we can add storage without disrupting operations. We have the performance we need to harness big data to protect and manage California’s natural resources. And, importantly, we’ve achieved our goals at less cost to taxpayers.
About the California Natural Resources Agency
The California Natural Resources Agency consists of 29 departments whose mission is to restore, protect, and manage the state’s natural, historical, and cultural resources for current and future generations using creative approaches and solutions based on technology, science, and collaboration.
About NetApp
NetApp creates innovative storage and data management solutions that deliver outstanding cost efficiency and accelerate business breakthroughs. Discover our passion for helping companies around the world go further, faster at www.netapp.com.
