Reducing Risks and Optimizing the Data
Center with Veritas™ Operations
Manager
Who should read this paper
Who should read this paper
System/Storage/Server Administrators
APER:
A
S™ OPERA
TIONS MANA
GER
Content Executive overview . . . 1 Introduction . . . 1 Architecture . . . 1 Reducing risk . . . 2 Health Check . . . 3
Cluster Server fire drill and reporting . . . 4
Patch reporting . . . 4
License reporting and keyless licensing. . . 5
Improving operational scalability . . . 6
Secure multi-user access . . . 6
Application to spindle visibility. . . 6
Storage Foundation management . . . 6
Storage tiering . . . 7
Simplified template based provisioning . . . 7
Storage migration . . . 8
Simplified Dynamic multi-pathing management . . . 8
Automated Recovery Plans . . . 9
Cluster Server management . . . 9
ApplicationHA management . . . 10
Management of multitier applications with Virtual Business Services . . . 11
Maximizing storage utilization . . . 12
Storage utilization for applications . . . 13
Storage Foundation thin reclamation . . . 14
Storage reporting. . . 14
Executive overview
The data center today continues to be a place where the only constant is change. For two decades Veritas Storage Foundation™ from
Symantec has been simplifying storage management and helping improve application performance and storage utilization. For over a decade Veritas™ Cluster Server from Symantec has been improving application availability and management and is at the core of some of the world’s most critical applications. As the data center and customer needs have evolved, Storage and Availability solutions from Symantec have continued to improve to help solve our customers’ most daunting challenges. Over the years, we have seen many transformational technologies come into the marketplace; more recently we have witnessed wide adoption of server virtualization and thin provisioning. Symantec has been right there to support these technologies with Symantec™ ApplicationHA to enhance application availability for virtual environments, including VMware®, Kernel-based Virtual Machine (KVM), and UNIX® virtualization solutions as well as pioneering work toward efficient thin storage utilization and reclamation with our Storage Foundation offerings.
In addition to these capabilities, we have introduced a centralized platform called the Veritas™ Operations Manager from Symantec™ that simplifies reporting, monitoring, and management even further by providing details about the applications, servers, and storage surrounding the Storage and Availability solutions. With our multivendor support across applications, operating system platforms, server and array vendors, and virtual environments, our customers are empowered to choose the solution that is best for them as well as avoid vendor lock-in and reduce costs.
The purpose of this white paper is to illustrate the immense value that Operations Manager brings to current and new users of Storage Foundation, Cluster Server, Veritas™ Dynamic Multi-Pathing, and ApplicationHA. This includes capabilities that often span multiple teams within an organization from applications to server to storage administrators.
Finally, this white paper is intended to serve as a reference, not as an exhaustive discussion of all of the features and capabilities of Operations Manager or the other Symantec products it supports.
Introduction
In recent years, data centers have become increasingly large and complex and many IT organizations have fewer people with which to manage those growing environments. There is a constant struggle to maintain existing applications and services while at the same time consolidate, reduce costs, and deploy new solutions. The Storage and Availability products from Symantec have been at the center of making all of this possible. With the addition of Operations Manager, you can get make these solutions work even better!
• Reduce risk with improved visibility and monitoring across your applications, servers, and storage.
• Improve operational scalability with management of Storage Foundation, Cluster Server, Dynamic Multi-Pathing, and ApplicationHA. • Maximize storage utilization with detailed reports and reclamation tools.
Operations Manager is available on hosts with Symantec Storage and Availability products at no additional cost.
Architecture
Operations Manager discovers information about heterogeneous applications, servers, and storage in your environment by communicating directly with the storage arrays via the vendor tools, the server virtualization managers such as VMware vCenter™ via their Application Programming Interfaces (APIs), as well as having an agent deployed on the host alongside Storage Foundation, Cluster Server, Dynamic Multi-Pathing, or ApplicationHA products. The lightweight Operations Manager agent runs across Windows®, Linux®, and UNIX operating environments and is bundled with newer versions of Storage Foundation and Cluster Server. While the agent is bundled, it is not tightly
coupled with the Storage Foundation and Cluster Server deployment, and it is easily managed and upgraded centrally through the Operations Manager Web-based console. It discovers information through regular timed discovery cycles, as well as being event driven to detect various state and configurations changes relating to the Symantec Storage and Availability products. By collecting detailed information from these points, Operations Manager can give you a complete picture of your applications, servers, and storage and how they are related.
Figure 1 – Operations Manager High Level Architecture.
Reducing risk
TTroubleshoo
roubleshooting and monitoring the en
ting and monitoring the environment
vironment
The Operations Manager agent is constantly listening on the host for various configuration and state changes about the Storage Foundation, Cluster Server, Dynamic Multi-Pathing, and ApplicationHA configuration. This includes file system utilization reaching a defined user threshold, paths to storage becoming unavailable, logical volumes becoming degraded, as well as services controlled by Cluster Server faulting. When events like these occur on a host or cluster, the Operations Manager agent will update the Operations Manager Management Server giving the users of the Operations Manager Web console near-real-time updates across an environment consisting of thousands of hosts, all from a single pane of glass.
In addition to these events from the host, Operations Manager can also help reduce risk associated with oversubscription of thin storage. Operations Manager can also monitor the details of the thin pool configuration on the storage array. Storage administrators can configure thresholds for subscribed and consumed percentages at the thin pool level and get notifications through Operations Manager when those thresholds are breached. Those notifications also bubble up to the Storage Foundation configurations built on top of the thin storage, allowing server and applications owners to have additional awareness.
These notifications are collected centrally in the Operations Manager Management Server and can be viewed from the Operations Manager Web console (Figure 2). They can also be forwarded outside of Operations Manager by creating rules to send them to your Network
to receive notification via email and even execute scripts in response to various conditions as they occur. The graphical root cause analysis tool not only shows the possible causes of the fault but all the assets impacted by it.
Figure 2 - Monitoring Dashboard and Root Cause Analysis Tool.
Health Check
Health Check
While getting notifications when various changes occur is critical to running a smooth environment, getting notifications before critical issues arise can be even more powerful. Operations Manager includes a health check feature that can be configured to periodically scan the configuration to ensure that the supported Symantec Storage and Availability solutions are optimally configured. These include signatures that check to verify if the storage required for an application is visible to all nodes in a cluster, whether file systems require defragmentation, as well as ensuring that mirrored Operations Manager volumes have Dirty Region Logging (DRL) configured for optimal performance.
Figure 3 - Health Check Scan Overview.
In addition to signatures that analyze for availability, utilization, performance, and best practices, Operations Manager also has a simple API that lets you plug in your own custom signatures. This simple yet powerful feature can be used to check for a variety of conditions in the environment including simple checks for service states, file existence, or any other analysis that can be scripted.
Cluster Server fire drill and reporting
Operations Manager provides an interface to schedule and run Cluster Server High Availability/Disaster Recovery (HA/DR) fire drills to validate the ability of business-critical applications to start at a secondary site without actually needing to disrupt the application at the primary site. For more information specifically on the fire drill feature of Cluster Server, please review the details here:
http://go.symantec.com/vcs.Operations Manager will allow you to schedule and run fire drills across potentially hundreds of clusters reporting into a single Operations Manager Management Server. In addition to being able to view the details and output of these tests, the results are used to give the cluster administrator in Operations Manager a visual representation of the readiness of a service to effectively failover. This visualization provides the administrator with additional confidence that should a failover need to occur, it can be done successfully.
There are also a wide range of reports including several historical reports that look specifically at the Cluster Server configuration. These cover status reports at the cluster, service group, and resource levels as well as reports that look more deeply into uptime and failover history. Specifically, the Failover Summary report details the failovers that have occurred during the specified timeframe to detail not only when failovers occurred, but how they were triggered. Was it something that Cluster Server detected or was it initiated by an administrator? The Operations Manager report provides this level of detail so you can understand when failovers have occurred, the result, as well as who triggered them.
Figure 4 - Failover Summary Report.
Patch reporting
Operations Manager also collects details about the Storage Foundation and Cluster Server product versions and updates deployed across the environment. It lets administrators easily view information about the specific patches that have been deployed in the environment as well as get details about new patches that are available. The Operations Manager Management Server communicates with the Symantec™Operations Readiness Tools, to pull details down about the latest updates and fixes for Operations Manager, Storage Foundation, Cluster Server, Dynamic Multi-Pathing, and ApplicationHA. Operations Manager will correlate the information from Operations Readiness Tools with your
local configuration discovered within Operations Manager to show you specifically what patches are available and where they have not yet been deployed. Operations Manager also shows additional details about the patch, the products that are affected, the criticality of the patch, whether applying the patch will cause a service interruption or machine reboot, as well as links to get even more details about the patch and the patch itself.
Figure 5 - Patch Report Snippet.
License reporting and keyless licensing
Amongst other information about the configuration, Operations Manager collects details about the licenses for Storage Foundation, Cluster Server, and Dynamic Multi-Pathing. The reporting capability will provide comprehensive insight into the deployments across the environment including the product deployed, product version, platform, as well as help with tracking demo and expired keys. You can easily view what license keys have been deployed where, and get details on what entitled features are actively being used. Deployment policies can also be configured, which will allow you to be notified when the number of product deployments reaches a user defined amount.
Additionally, Operations Manager is a required component to enable the keyless licensing feature for these products. Keyless licensing is available on Storage Foundation/Cluster Server 5.1 and higher for Linux and UNIX and Storage Foundation/Cluster Server 6.0 and higher for Windows. This option can improve your experience with these products by making it simpler to deploy and manage your entitlements. You simply need to identify the host where you are installing Storage Foundation, Cluster Server, or Dynamic Multi-Pathing, enable the keyless option on that host, and then have that host report into a Operations Manager Management Server so the entitlement can be tracked. No more need to install and manage keys on every server!
Figure 6 - License Reporting Summary.
Improving operational scalability
Secure multi-user access
Secure multi-user access
Operations Manager integrates with Microsoft Active Directory®, LDAP(Lightweight Directory Access Protocol), as well as UNIX Authentication services such as NIS/NIS+ to facilitate user authentication. A group of users defined within those authentication modules can be associated with one or more Business Entities, which are logical groups of hosts or clusters defined within the Operations Manager configuration. Additionally, those users are assigned a level of access (administrator, operator, or guest) to the resources associated with those Business Entities. This flexible approach allows different users to potentially have different levels of access to different sets of resources within the Operations Manager environment. Business Entities can also be used to scope reports and serve as the basis for defining multitier applications with Virtual Business Services (VBS).
Application to spindle visibilit
Application to spindle visibilityy
Administrators often struggle to simply understand the relationships and dependencies between disparate storage and server subsystems and the applications associated with them. Without true visibility into the environment, it is much more difficult to make wise and informed decisions. Additionally, without a single point to observe these details, it can be very difficult to pull together all the pieces of information stored across many different tools, proprietary databases, and spreadsheets, to really understand the basic details required to initiate a required operation.
Operations Manager gathers information from all of the critical points in this ecosystem to be able to give administrators the power to understand the true makeup of their environment. The Operations Manager Agent understands the internal constructs of applications such as Oracle® Database and Microsoft SQL Server®, as well as what server and storage resources that those applications are using. From application internals, to server virtualization details, to array configuration, Operations Manager has the information you need to help you make informed decisions and make sure your environment runs optimally.
Storage Foundation management
Storage Foundation includes several integrated products including Veritas™ Volume Manager from Symantec, Veritas™ File System from Symantec on UNIX/Linux, and Dynamic Multi-Pathing which provide a diverse set of capabilities around multi-pathing, logical volume management, software RAID(Redundant Array of Independent Disks), snapshotting, auto-tiering, block and file level replication, thin
reclamation, and much more. The Operations Manager Web based console serves as the central point through which you can configure and manage these powerful features.Operations Manager allows the administrator of the environment to operate more efficiently by providing additional details about the surrounding configuration, giving a central point through which to monitor and manage the entire environment, as well as providing a higher level perspective to simplify a variety of management tasks. From traditional tasks around creating volumes and file systems to more complex operations around disabling paths associated with an array controller, Operations Manager has the visibility and the flexibility to simplify your day-to-day operations.
Storage tiering
In addition to the wealth of configuration details that come from the host discovery, Operations Manager also has the capability of
communicating directly with the arrays via the vendor tool APIs. Putting this together, Operations Manager has knowledge about a number of different attributes of the storage that is presented to the host. Details like the vendor and model of the array, whether the Logical Unit Number (LUN) is composed of solid-state or spinning disk, LUN RAID levels, the thin capabilities of the storage, whether the LUN is replicated or not are discovered by Operations Manager. These different criteria can be used to create custom tiers of storage. They are applied to the storage that is discovered in the Operations Manager environment, and then available to be used for reporting, provisioning, and for the array migration capabilities of Operations Manager that will be covered in more detail below.
Simplified template based provisioning
Creating storage configurations on the host can be a tedious and error prone exercise. Storage Foundation has been helping in this area for years by simplifying device identification with the enclosure based and Array Volume Identifiers (AVID) naming feature. Operations Manager takes this a step further by incorporating the use of the rule based tiering with the ability to define application-centric templates for simplified and less error prone storage provisioning. Storage architects can use the simple wizard driven interface to create a library of templates for the applications that their organization needs to provision. Server administrators can then easily provision this template on a host, specify any optional criteria, and have the confidence that the correct storage is used for that application configuration.
Storage migration
Storage Foundation has long been a great asset to administrators when assisting with the process of migrating and consolidating data on the Storage Area Network (SAN). Operations Manager extends this core functionality to give a broader perspective on the migration process. With its visibility into the arrays and applications running on the host, Operations Manager can provide the administrator with a more complete view to simplify storage migration tasks. It utilizes the SmartMove feature of Storage Foundation to only copy data that is in use in the file system when migrating. In addition to the performance benefits, this also helps when migrating from traditional storage to thinly provisioned storage by ensuring that your new configurations start out optimally thin. The storage migration feature will also allow you to transform the logical volumes during the migration process, so if you are moving mirroring or striping characteristics from the host into the array, or vice versa, you can easily make that adjustment seamlessly during the migration process. Also, like the template based provisioning feature, the criteria based storage tiers can be used to define where to migrate to, simplifying the task and reducing errors.
Simplified Dynamic multi-pathing management
Simplified Dynamic multi-pathing management
With the mix of physical and virtual environments in the data center, performing maintenance on array ports requires visibility across muti-vendor physical and virtual environments. This proves to be a significant challenge for administrators as they don’t have a single tool that provides this level of visbility. Operations Manager solves this problem by providing a uniform interface for enabling/disabling paths under Dynamic Multi-Pathing control. So, regardless of whether your array is serving LUNs to a physical server or a virtual server, you can not just enable/disable the paths but also see a simple impact analysis report that shows the applications, virtual machines, datastores physical servers, hosts, volumes and disks impacted by the operation.
Automated Recover
Automated Recovery Plans
y Plans
Co-ordinating the orderly start-up of multiple applications is an extremely time consuming and error prone process for administrators. This is true when bringing these applications up in a disaster recovery site as well as when repurposing spare capacity in the primary site to meet the every changing needs of production workloads. The process requires co-ordination between multiple teams and is mostly manual with many custom written scripts that are hard to maintain and track. The situation is further complicated by the fact that spare capacity in the disaster recovery site as well as in the primary site is often repurposed to run testing or other non-critical workloads which need to be brought down before bringing up the production workloads. Operations manager solves this problem by allowing administrators to record these steps in an automated recovery plan. Each step of the recovery plan can be an action (start/ stop or online/offline) on a Cluster Server/ ApplicationHA service group or a virtual business service. To provide flexibility in integrating the recovery plan with other scripts/tools, the plan allows execution of custom scripts. Multiple recovery plans can be executed in parallel to reduce the recovery time objective (RTO).
Figure 9 – Automated recovery plan for server repurpose
Cluster Server management
Cluster Server continues to be a highly rated solution for reducing both planned and unplanned downtime for even the most demanding application environments1. In addition to providing traditional local high availability, Cluster Server also helps facilitate disaster recovery over long distances as well. As these configurations grow in terms of applications and data centers, it becomes critically important to have a single pane of glass through which to monitor, manage, and report on the overall configuration. Operations Manager provides this single point and enhances the overall cluster management experience by providing access to the breadth of the Cluster Server functionality and advanced visualizations of the configuration (Figure 10). From performing simple failovers to creating more advanced cluster configurations, Operations Manager will help you with your cluster management needs.
Figure 10 - Resource Dependency View.
ApplicationHA management
Server virtualization has changed the way many application architects have come to think of high availability. We’ve seen an explosion in the adoption of VMware and other virtualization solutions. As customers have virtualized more and more of the data center, they’ve run into some obstacles in those environments when it comes to high availability of applications. Symantec has introduced the ApplicationHA solution to help virtualize the remaining applications that have significant availability requirements. ApplicationHA works with leading virtualization solutions including VMware, KVM, and UNIX environments, and builds on the Cluster Server technology, to provide application visibility and availability for virtual environments.
Operations Manager provides an interface through which administrators are able to monitor and manage the virtualized applications under ApplicationHA control across platforms and multivendor virtualization solutions. With Operations Manager, you can view your traditional clusters and ApplicationHA deployments side by side and manage them through a single easy to use interface.
Figure 11 - ApplicationHA Management.
Management of multitier applications with Virtual Business Services
Complex services that are made up of many individual applications running across different operating environments with some virtualized servers and some traditional physical servers are becoming commonplace in many organizations. This model helps companies achieve better resource utilization, improves flexibility, and can lower implementation costs. A typical example of this is a three-tier application with a Web server farm or cluster front end, an application server cluster in the middle, all backed by a database server cluster (Figure 12).
There are many challenges to bringing these pieces together so they can truly be understood and managed as a single entity. In addition to facilitating access rights and scoping reports, Business Entities also allow you to associate multiple applications to form a truly singular concept of a multi-tier application called a VBS.
VBS provides continuous high availability and reduces frequency and duration of service disruptions for multi-tier business applications running across operating system platforms and multivendor virtualization technologies. Operations Manager is the single central point through which you can configure, manage, and monitor each VBS. Operations Manager will help you understand the relationship between the applications that make up the VBS, as well as notify you of any issues with an underlying component or the VBS as a whole (Figure 13). You can also perform operations that were once potentially very complicated, like starting or stopping the entire multitier application in the correct order or coordinating a disaster recovery failover, all of which can be achieved through Operations Manager with just a few simple clicks.
Figure 13 - Monitor Virtual Business Services.
Maximizing storage utilization
Detailed device inf
Detailed device information
ormation
Visibility from multiple perspectives in the environment is critical for getting a clear picture of how your assets are being utilized. Operations Manager achieves this by communicating directly with the storage arrays via the vendor tools, the Server Virtualization Managers such as vCenter via their APIs, as well as having an agent deployed on the host alongside Storage Foundation, Cluster Server, Dynamic Multi-Pathing, or ApplicationHA solutions. By interfacing with the array directly, Operations Manager can gather detailed information about the physical storage assets, how they are logically configured, and then putting those details together from what is discovered from the server side can see how that storage is being consumed by the volume manager, file system, and applications.
This means being able to get additional information about the storage than what can be seen from the host perspective using traditional tools. This includes LUN RAID levels, subscribed and consumed sizes for associated thin pools, array port and adapter connectivity, array snapshot and replication details, and much more. As an administrator, having that additional detailed information can help you make more informed decisions by removing unnecessary guess work and making potentially error prone legacy processes irrelevant.
Figure 14 - Enclosure details view.
Storage
Storage utilization f
utilization for applications
or applications
In addition to collecting details from multiple perspectives in the environment, the Operations Manager Agent understands the internal constructs of applications such as Oracle Database and SQL Server. By understanding these configurations, administrators can get a truer sense of how the storage is being utilized. Putting this all together, Operations Manager has visibility into the physical storage in the array, how that is carved up logically and consumed by hosts, how the storage host side is pooled into disk groups, how the logical volumes and file systems format that storage, and how the applications such as Oracle and SQL Server ultimately consume the capacity.
Understanding this progression is key to being able to identify your true utilization and identify potential inefficiencies. Locating those areas for improvement is the first step, making configuration changes such as shrinking or growing file systems, migrating storage, or potentially consolidating or giving back storage can be performed right from these views using the Storage Foundation capabilities available in the Operations Manager Web console.
Storage
Storage FFoundation thin reclamation
oundation thin reclamation
Low storage utilization has been and continues to be a significant challenge for many organizations. Thin provisioning is one such technology that has promised to help drive up storage utilization through allocation-on-demand and reduce the burden associated with re-provisioning storage for growing applications. For block storage, there are several significant obstacles toward realizing this dream of simplified
provisioning and higher overall utilization. To properly implement thin storage, the storage not only needs to grow when additional resources are required, but also needs to give those resources back when they are no longer needed. To implement a truly elastic solution
requires visibility on the host to understand how the file system and applications are utilizing the storage.
Symantec has developed the thin-reclamation API in Storage Foundation along with leading storage vendors with the intelligence to efficiently communicate with the array to de-allocate storage that is no longer required and put it back in the free pool for general use. This allows the storage being consumed by hosts in the environment to stay true to what is actually required, rather than reflecting peak usage over time. Operations Manager provides additional visibility from the array perspective into the thin pools, across hosts and clusters, to allow you to schedule use of this capability for regular and transparent reclamations. It also will report the results of those reclamations, showing you how much storage it was able to get back. Using the combination of Storage Foundation and Operations Manager will help you realize the promise of thin provisioning, and keep your thin storage utilization optimal.
Figure 16 - Thin Reclamation Output.
Storage
Storage reporting
reporting
Operations Manager collects information about the storage configuration from what is visible to Storage Foundation as well as
communicating directly with the storage arrays via the vendor tools. This holistic view allows Operations Manager to understand the storage configuration more completely than other tools that only interface with the Storage Arrays and the SAN. The Operations Manager
Management Server contains several detailed views into the storage that are linked into the views of the virtualization servers, hosts, and applications. In addition to these views, Operations Manager contains a number of reports on the storage configuration including thin pool usage, array storage allocation, and underutilized LUNs to name a few. Operations Manager provides reports that show exactly how much space can be reclaimed at enclosure, thin pool and host level. This enables administrators to limit the scope of reclamation to a small set of objects where they can reclaim the most space.
Figure 17 - Thin Pool Usage Report (partial).
The Operations Manager infrastructure also serves as the underpinnings for Operations Manager Advanced, which can also discover
configuration details from hosts that don’t have Storage Foundation, Dynamic Multi-Pathing, Cluster Server, or ApplicationHA. This discovery can be done using the same Operations Manager agent or using agentless technology to provide visibility across your entire data center. This additional visibility will help you better understand the complete storage configuration potentially across multiple datacenters and the end-to-end utilization of those assets. It will also help you create chargeback or showback reports to assist with billing or promote responsible usage as well as identify ineffectively allocated storage and help you reclaim it.
For more information about Operations Manager Advanced, a white paper discussing its broad reporting capabilities can be found at
http://go.symantec.com/vomadvanced.
Summary
Managing the diverse and ever-changing data center continues to be a challenge even at the most advanced IT organizations. Storage and High Availability from Symantec solutions help to simplify and optimize these environments across server and storage vendors. These solutions can help you achieve higher performance as well as help lower operating costs. Bringing Operations Manager into the environment allows these solutions to work even better and further optimize your infrastructure and operations. The superior visibility and monitoring capabilities of Operations Manager help reduce risks and achieve higher service availability. Easy-to-use wizards and policy driven operations allow you to easily manage and maximize your Storage Foundation, Dynamic Multi-Pathing, Cluster Server, and ApplicationHA
implementations. Detailed reports on the storage configuration and advanced reclamation tools allow you to get more out of the resources you’ve already purchased. Even better, these differentiating capabilities provided in Operations Manager are available on hosts with licensed Storage and High Availability from Symantec solutions at no additional cost. Go tohttp://go.symantec.com/vomfor additional information as well as links to download so you can try it yourself.
About Symantec
Symantec protects the world’s information and is
the global leader in security, backup, and
availability solutions. Our innovative products and
services protect people and information in any
environment – from the smallest mobile device, to
the enterprise data center, to cloud-based systems.
Our industry-leading expertise in protecting data,
identities, and interactions gives our customers
confidence in a connected world. More information
is available at www.symantec.com or by connecting
with Symantec at go.symantec.com/socialmedia.
For specific country offices
and contact numbers, please
visit our website.
Symantec World Headquarters
350 Ellis St.
Mountain View, CA 94043 USA
+1 (650) 527 8000
1 (800) 721 3934
www.symantec.com
Symantec helps organizations secure and manage their information-driven world with storage management, email archiving, and backup and recovery solutions. Copyright © 2012 Symantec Corporation. All rights reserved. Symantec, the Symantec Logo, and the Checkmark Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. 8/2012 21254001-1