Stingray Services Controller User s Guide

(1)

Stingray™ Services Controller

User’s Guide

Version 2.0

December 2014

(2)

Riverbed Technology 680 Folsom Street San Francisco, CA 94107

Phone: 415-247-8800

Riverbed®, SteelApp™, SteelCentral™, SteelFusion™, SteelHead™, SteelScript™, SteelStore™, Steelhead®, Cloud Steelhead®, Virtual Steelhead®, Granite™, Interceptor®, Stingray™, Whitewater®, WWOS™, RiOS®, Think Fast®, AirPcap®, BlockStream™, FlyScript™, SkipWare®, TrafficScript®, TurboCap®, WinPcap®, Mazu®, OPNET®, and Cascade® are all trademarks or registered trademarks of Riverbed Technology, Inc. (Riverbed) in the United States and other countries. Riverbed and any Riverbed product or service name or logo used herein are trademarks of Riverbed. All other trademarks used herein belong to their respective owners. The trademarks and logos displayed herein cannot be used without the prior written consent of Riverbed or their respective owners.

Akamai® and the Akamai wave logo are registered trademarks of Akamai Technologies, Inc. SureRoute is a service mark of Akamai. Apple and Mac are registered trademarks of Apple, Incorporated in the United States and in other countries. Cisco is a registered trademark of Cisco Systems, Inc. and its affiliates in the United States and in other countries. EMC, Symmetrix, and SRDF are registered trademarks of EMC Corporation and its affiliates in the United States and in other countries. IBM, iSeries, and AS/400 are registered trademarks of IBM Corporation and its affiliates in the United States and in other countries. Juniper Networks and Junos are registered trademarks of Juniper Networks, Incorporated in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States and in other countries. Microsoft, Windows, Vista, Outlook, and Internet Explorer are trademarks or registered trademarks of Microsoft Corporation in the United States and in other countries. Oracle and JInitiator are trademarks or registered trademarks of Oracle Corporation in the United States and in other countries. UNIX is a registered trademark in the United States and in other countries, exclusively licensed through X/Open Company, Ltd. VMware, ESX, ESXi are trademarks or registered trademarks of VMware, Inc. in the United States and in other countries.

This product includes software developed by the University of California, Berkeley (and its contributors), EMC, and Comtech AHA Corporation. This product is derived from the RSA Data Security, Inc. MD5 Message-Digest Algorithm.

NetApp Manageability Software Development Kit (NM SDK), including any third-party software available for review with such SDK which can be found at http://communities.netapp.com/docs/DOC-1152, and are included in a NOTICES file included within the downloaded files.

For a list of open source software (including libraries) used in the development of this software along with associated copyright and license agreements, see the Riverbed Support site at https//support.riverbed.com.

This documentation is furnished “AS IS” and is subject to change without notice and should not be construed as a commitment by Riverbed. This documentation may not be copied, modified or distributed without the express authorization of Riverbed and may be used only in connection with Riverbed products and services. Use, duplication, reproduction, release, modification, disclosure or transfer of this documentation is restricted in accordance with the Federal Acquisition Regulations as applied to civilian agencies and the Defense Federal Acquisition Regulation Supplement as applied to military agencies. This documentation qualifies as “commercial computer software documentation” and any use by the government shall be governed solely by these terms. All other use is prohibited. Riverbed assumes no responsibility or liability for any errors or inaccuracies that may appear in this documentation.

(3)

Stingray Services Controller User’s Guide iii

Inventory Resources ...125 Resource Reference...126 action Resource ...126 add_on_pack_license_key Resource...127 add_on_sku Resource ...127 bandwidth_pack_license_key Resource...128 cluster Resource ...131 controller_license Resource...132 controller_license_key Resource...133 feature_pack Resource ...135 host Resource ...135 instance Resource ...137 license Resource ...146 manager Resource ...147 monitoring Resource ...148 service Resource...149 sku Resource...150 user Resource ...151 version Resource...152

Using the REST API to Check Status ...153

Understanding REST Request Errors...154

Chapter 8 - Installing the Services Controller Virtual Appliance ...155

Overview of the Services Controller VA ...156

Prerequisites ...156

Resource Requirements ...157

Required Configuration Information ...158

(7)

Stingray Services Controller User’s Guide vii

Contents Installing, Configuring, and Administering the Services Controller VA ...160

Creating the VM in vSphere...161

Running the Services Controller VA Setup Wizard ...162

Changing the Password for the Admin User ...175

Upgrading the Services Controller VA ...175

Downgrading the Services Controller VA...176

Upgrading Instance Host VAs ...176

Migrating Instance Host VAs...177

Chapter 9 - Configuring the Stingray Services Controller Virtual Appliance ...179

Using the Administration UI to Manage the Services Controller...179

Connecting to the Administration UI...180

Using the Administration UI ...181

Overview of Services Controller Tasks...183

Configuring Network Settings...184

Configuring General Network Settings ...184

Configuring Base Interfaces ...185

Configuring System Settings...187

Configuring Announcements ...187

Configuring Email Settings...188

Configuring Resources for the Services Controller ...188

Managing REST API User Credentials...189

Uploading the Traffic Manager Image ...190

Creating a Traffic Manager Version Resource ...190

Managing the Services Controller Licenses...192

Managing Bandwidth Pack Licenses...193

Managing Add-On Licenses ...193

Managing Flexible Licenses ...194

Managing Services Controller Certificates ...194

Managing Feature Packs...197

Managing the Services Controller Service ...199

Configuring Database Settings ...199

Importing and Exporting the Local Database ...200

Configuring Services Controller Mode Settings ...202

Viewing and Modifying Services Controller Settings ...203

Managing Service Controller Resources...205

Creating and Managing Instance Hosts ...205

Creating and Managing Traffic Manager Instances...209

Creating and Managing Clusters ...214

Creating and Managing LBaaS Services...215

Creating a Service with the LBaaS Wizard...215

Creating an LBaaS Service...221

Changing the Status of an LBaaS Service...225

(8)

Contents

Starting and Stopping an LBaaS Service ...226

Changing the Error State of an LBaaS Service Host ...226

Deleting an LBaaS Service...227

Viewing Reports and Diagnostics ...228

Instance Report ...228

Bandwidth Allocation Report...230

CPU Utilization Report...232

Throughput Utilization Report...233

Viewing Logs and Generating System Dumps ...234

Viewing Logs...234

Generating System Dumps ...235

Generating Metering Logs...236

Getting Help ...237

Using the CLI to Manage the Services Controller...238

Importing the SSL Certificate, Key, and Licenses ...238

Importing an Instance Host OVA into the Services Controller...240

Enabling Passwordless SSH Communication ...240

Creating a Feature Pack for Instances in the Services Controller ...241

Provisioning an Instance Host...241

Creating a Traffic Manager Instance Without a Container...243

Configuring a Traffic Manager Instance with a Container...245

Working with LBaaS Services ...248

ESXi vSphere Host Port Mapping...250

Exporting a database...250

Generating Metering Logs ...250

Chapter 10 - Configuring for High Availability...251

High Availability Configuration Prerequisites...251

Shared Access to the Database...251

High Availability Mode Settings ...252

FLA-based Load Balancing and Mode Settings API ...253

Prerequisites for an HA Licensing Environment ...253

Redeeming Licensing Tokens ...254

Suggested Mode Settings for Enterprise setups...255

Suggested Mode Settings for CSP setups...255

How Failures are Detected ...256

Recovering from a Failure ...256

Enabling GUI Access over HTTPS ...256

Making the SSC Database Highly Available...256

Chapter 11 - Troubleshooting...257

Resolving Configuration Errors When Starting the Services Controller VA ...257

Tracking the Progress of Actions ...258

(9)

Stingray Services Controller User’s Guide ix

Contents

Generating a Technical Support Report ...259 Identifying Instances from a Support Entitlement Key ...259 Performing Advanced Settings...260

(10)

Contents

(11)

Stingray Services Controller User’s Guide 1

Preface

Welcome to the Stingray Services Controller User’s Guide. Read this preface for an overview of the information provided in this guide. This preface includes the following sections:

“About This Guide” on page 1

“Documentation and Release Notes” on page 2 “Contacting Riverbed” on page 3

About This Guide

The Stingray Services Controller User’s Guide describes how to configure and manage the Riverbed Stingray Services Controller and the Riverbed Stingray Services Controller Virtual Appliance.

This guide includes information relevant to the following products: Riverbed Stingray Services Controller (Services Controller)

Riverbed Stingray Services Controller Virtual Appliance (Services Controller VA) Riverbed Stingray Traffic Manager (Traffic Manager)

Audience

This guide is written for system administrators familiar with administering and managing large-scale hosting environments. This guide assumes that you are familiar with Riverbed Stingray Traffic Manager.

(12)

Preface Documentation and Release Notes

Document Conventions

This guide uses the following standard set of typographical conventions.

Documentation and Release Notes

To obtain the most current version of all Riverbed documentation, go to the Riverbed Support site at https://support.riverbed.com.

If you need more information, see the Riverbed Knowledge Base for any known issues, how-to documents, system requirements, and common error messages. You can browse titles or search for keywords and strings. To access the Riverbed Knowledge Base, log in to the Riverbed Support site at

https://support.riverbed.com.

Each software release includes release notes. The release notes identify new features in the software as well as known and fixed problems. To obtain the most current version of the release notes, go to the Software and Documentation section of the Riverbed Support site at https://support.riverbed.com.

Examine the release notes before you begin the installation and configuration process.

Convention Meaning

italics Within text, new terms and emphasized words appear in italic typeface.

boldface Within text, CLI commands, CLI parameters, and REST API properties appear in bold typeface. Courier Code examples appear in Courier font:

amnesiac > enable

amnesiac # configure terminal

< > Values that you specify appear in angle brackets: interface <ip-address>

[ ] Optional keywords or variables appear in brackets: ntp peer <ip-address> [version <number>] { } Elements that are part of a required choice appear in braces: {<interface-name> | ascii <string> |

hex <string>}

| The pipe symbol represents a choice to select one keyword or variable to the left or right of the

symbol. The keyword or variable can be either optional or required: {delete <filename> | upload <filename>}

(13)

Contacting Riverbed Preface

Contacting Riverbed

This section describes how to contact departments within Riverbed.

Technical support - If you have problems installing, using, or replacing Riverbed products, contact Riverbed Support or your channel partner who provides support. To contact Riverbed Support, open a trouble ticket by calling 1-888-RVBD-TAC (1-888-782-3822) in the United States and Canada or

+1 415 247 7381 outside the United States. You can also go to https://support.riverbed.com.

Professional services - Riverbed has a staff of professionals who can help you with installation, provisioning, network redesign, project management, custom designs, consolidation project design, and custom coded solutions. To contact Riverbed Professional Services, email [email protected] or go to http://www.riverbed.com/services-training/Services-Training.html.

Documentation - The Riverbed Technical Publications team continually strives to improve the quality and usability of Riverbed documentation. Riverbed appreciates any suggestions you might have about its online documentation or printed materials. Send documentation comments to

(14)

(15)

CHAPTER 1

Understanding the Services

Controller

This chapter provides an overview of the Services Controller and describes the product features. It includes the following sections:

“Overview of the Services Controller” on page 5 “Traffic Manager Instance Hosts” on page 7 “Nonmanaged Mode for Instances” on page 8

“Network Isolation for Multiple Traffic Manager Instances” on page 8 “Usage Metering and Activity Metrics” on page 11

“Health and Performance Monitoring” on page 14 “Database-Only Updates” on page 16

“Using INSTALLROOT in This Guide” on page 16

Overview of the Services Controller

The Services Controller enables you to manage multiple instances of the Traffic Manager through a Representational State Transfer (REST) API. Alternatively, you can use the Services Controller Virtual Appliance (Services Controller VA) to manage multiple instances of the Traffic Manager.

For detailed information about the Services Controller VA, see Chapter 8, “Installing the Services Controller Virtual Appliance.”

The Services Controller stores information about deployed Traffic Manager instances, including the resources that it needs to manage in an inventory database. The Services Controller supports the MySQL database running against the default InnoDB back end.

The Services Controller can perform a range of life-cycle actions on Traffic Manager instances. The Services Controller can:

deploy an instance of the Traffic Manager with specified parameters. start and stop an instance of the Traffic Manager.

uninstall an instance of the Traffic Manager.

(16)

Understanding the Services Controller Overview of the Services Controller

These actions are triggered through the Services Controller REST API, through a REST API instance

resource. You issue a REST API request for the Services Controller server to update the inventory database, which in turn queues an action to implement the requested operation before responding to the request. To avoid time-out issues, the response is returned before the action completes. After the action completes, the inventory database is updated. There is no progress callback from the Services Controller; you must poll the Services Controller to check for the status of the action. For detailed information about REST API resources, see Chapter 7, “Using the REST API with the Services Controller.”

The Services Controller supports two licensing models:

Cloud Service Providers (CSP) with billing dependent on instance metering.

Enterprise licensing, which is prepaid and is used for managing Services Controller bandwidth allocations to internal customers.

For detailed information about Services Controller licenses, see Chapter 2, “Managing Services Controller Licenses.”

Riverbed recommends that you install the Services Controller in a high availability deployment (that is, with multiple Services Controllers) to ensure redundancy in case of a server failure. For detailed information about high availability deployments, see Chapter 10, “Configuring for High Availability.”

The following sections summarize Services Controller features and concepts that will help you install, configure, and deploy the system.

Functionality that is Specific to the Services Controller or the Services

Controller VA

The following functionality is only present in the Services Controller, and not the Services Controller VA: Instance Tagging. See Appendix , “Understanding the Tag Property.”“Understanding the Tag

Property” on page 49.

FLA checker. See “Checking the Health of an FLA License” on page 23. Metering log phone home. See “Creating Metering Logs” on page 13. Monitoring data retrieval. See “Retrieving Monitoring Data” on page 15.

The following functionality is only present in the Services Controller VA, and not the Services Controller: Can be used via a CLI or an Administration UI. See “Using the Administration UI to Manage the Services Controller” on page 179 and “Using the CLI to Manage the Services Controller” on page 238. Instance Host deployment. See “Creating and Managing Instance Hosts” on page 205.

Linux container management on Instance Host. See “To create a new instance host resource” on page 206.

Visualized reporting, including instance status, bandwidth allocation, instance CPU and throughput utilization. See “Viewing Reports and Diagnostics” on page 228.

(17)

Traffic Manager Instance Hosts Understanding the Services Controller

Traffic Manager Instance Hosts

The Services Controller enables you to use one or more external host machines to run instances of the Traffic Manager. These instances are referred to as Traffic Manager instance hosts or instance hosts.

Instance hosts are physical servers or virtual machines. The Services Controller does not directly provision these instance hosts. You must set up each instance host manually and provide the settings required to access them through the Services Controller REST API.

The Services Controller uses passwordless SSH to communicate with all instance hosts. You must configure the appropriate SSH login credentials and provide these settings to the Services Controller through the REST API.

For example, consider that the Services Controller is installed on a host called sschost and runs as a username of sscuser. If the Services Controller is expected to manage instances of the Traffic Manager on a separate instance host server called IHserver, you must enable passwordless SSH from sscuser@sschost to

root@IHserver. For detailed information about setting up passwordless SSH, see “To generate a new SSH key” on page 42.

You use the Services Controller REST API to create logical host resources for each instance host you want to deploy. The REST API does not immediately verify that the Services Controller can access these hosts. This is for the following reasons:

It enables a user to prepopulate the inventory database, even though the instance hosts might not be currently available.

In a scenario involving multiple Services Controller installations, it ensures high availability. For example, verifying that one instance of the software has the correct passwordless SSH credentials available does not automatically mean that the other instances are in the same state. You can make the Services Controller check instance host accessibility by using the status_check parameter when accessing a host resource. Riverbed recommends that you perform this check on all Services Controller installations for all available instance hosts.

To facilitate manageability, the Services Controller assumes that all Traffic Manager instances on one instance host are installed under a single root directory. This directory is referred to as the installation root directory, and a separate subdirectory is automatically created for each Traffic Manager instance when it is deployed.

For detailed information about configuring instances, see Chapter 4, “Configuring Traffic Manager Instances.”

Verifying the Instance Host UTF-8 Locale

You must make sure that instance hosts are set to the UTF-8 locale for correct behavior during deployment operations.

To verify the instance host UTF-8 locale

To check that the locale of an instance host is set correctly, run the locale command on that instance host. The output of this command should include the following line:

LANG=<language code>.UTF-8

If the locale is set correctly, the LANG variable ends with the .UTF-8 suffix (the language code might vary).

(18)

Understanding the Services Controller Nonmanaged Mode for Instances

Nonmanaged Mode for Instances

You might want to deploy Traffic Manager instances outside the Services Controller in nonmanaged mode. When the Services Controller accesses a nonmanaged instance, it is only able to manages licenses and provide metering capabilities for billing; it does not support Services Controller life-cycle operations on nonmanaged instances.

Reasons for deploying instances outside the Services Controller include:

Your Traffic Manager instance hosts are running a currently unsupported operating system, so the Services Controller cannot carry out deployment and other tasks.

Your Traffic Manager instances are deployed as virtual appliances, which the Services Controller cannot manage.

You can register nonmanaged instances with the Services Controller with a database-only update (see

“Database-Only Updates” on page 16) or a nonmanaged update. If you issue a database-only REST API PUT request (either when initially creating an instance record or when modifying one), you can set a number of properties that are otherwise managed directly by the Services Controller. These properties are:

rest_address admin_username admin_password snmp_address ui_address

Note: Setting any of these properties through a database-only REST API PUT request does not result in changes being passed to the Traffic Manager instance; only the Services Controller database is affected.

The Services Controller relies on the admin_username, admin_password, and rest_address properties to carry out REST proxy requests. The rest_address property must be unique and accurate to identify a Traffic Manager instance for licensing purposes.

The Services Controller uses the snmp_address property to meter a Traffic Manager instances. It must be set correctly for metering to occur.

Network Isolation for Multiple Traffic Manager Instances

A Traffic Manager instance uses TCP and UDP ports for internal and external communication. Each of these ports has its own default value that a Traffic Manager uses in a typical deployment: for example, the Administration UI defaults to listening on port 9090. However, multiple Traffic Manager instances running on the same instance host cannot share the same default port numbers.

(19)

Network Isolation for Multiple Traffic Manager Instances Understanding the Services Controller

The Traffic Manager instances can cohabit on an instance host using these methods:

Linux Containers - Run the Traffic Manager instances on the instance host inside Linux containers (LXC) to provide network isolation.

Port Offsets - Run the Traffic Manager instances directly on the instance host without using LXC, and specify a port offset. Port offset settings calculate a unique set of default port numbers that differs for each Traffic Manager instance.

Network Isolation Using LXC

You can use LXC to provide network isolation and a degree of resource isolation. The Services Controller expects prepared container configuration files in advance of instance deployment. You must put these files in the installation root directory of the instance host.

Riverbed recommends that you plan your container deployment strategy before installing the Services Controller to ensure adequate system resource availability. For more information about LXC, see the configuration and management instructions supplied with your Linux distribution.

Note: For CentOS, Riverbed recommends LXC v0.7.5.

To install LXC on an instance host for Ubuntu

As root user, at the system prompt, enter:

apt-get install lxc

Note: To install LXC for CentOS, you should consult the CentOS documentation.

Within the context of LXC, the Services Controller does not assume that you have created the containers. The Services Controller creates the requisite container based upon the container configuration you set up when it starts a Traffic Manager instance. The Services Controller destroys the container when it stops the instance.

These items must match for the Services Controller to correctly identify an instance for license validation: The container_name property of the instance resource

The container configuration filename (with a .conf extension)

The lxc.utsname parameter in the container configuration file (if you set up your containers to use virtual networking)

For example, a Traffic Manager instance deployed with a container_name property set to

stm1.example.com requires a container configuration file called stm1.example.com.conf, which is placed in the instance host installation root directory and contains this setting:

lxc.utsname=stm1.example.com

You must use a fully qualified domain name (FQDN) for the container name. Failure to set these values results in an unlicensed instance.

Traffic Manager instances running inside containers do not automatically raise a default network gateway, so your instances cannot contact hosts outside of the local network.

(20)

Understanding the Services Controller Network Isolation for Multiple Traffic Manager Instances

The Services Controller uses the container_configuration property in the instance resource to set the default gateway for that instance. This property value is a string representation of a JavaScript Object Notation (JSON) structure. Special JSON characters (such as double quotation marks) must be escaped correctly. Use the following JSON format to set the container_configuration property in a REST API PUT request of the instance resource:

"container_configuration" : "{\"gateway\":\"<gatewayIP>\"}"

For detailed information about configuring instances and their properties, see Chapter 4, “Configuring Traffic Manager Instances.”

In a typical installation, the Traffic Manager uses as many CPU cores as possible to achieve maximum performance. If you want to deploy a large number of Traffic Manager instances on one Services Controller instance host, you must associate each instance with specific CPU cores to ensure balanced resource allocation. In this way, each instance has a lower maximum performance, but you can run multiple instances on one instance host.

Instance CPU usage is controlled by the container configuration file and must contain the following setting:

lxc.cgroup.cpuset.cpus=0

If you are running Traffic Manager instances without using LXC, you must manually set CPU usage with the cpu_usage property of the instance resource.

Network Isolation Using Port Offsets

If you choose not to use LXCs for network isolation, you can use port offsets to create a unique set of management and control port numbers for each instance.

For port offsets, you must create each instance with a port_offset defined in the config_options property of the instance resource. You must provide each instance on a particular instance host with a unique

port_offset value.

Note: For ports that must be consistent across all Traffic Manager instances in a cluster, set the cluster_port_offset setting in the cluster resource. The port_offset option in the config_options for the instance must also be set. Any change to the config_options settings on a managed instance will cause a restart of the instance. Nonmanaged instances are not affected.

Specifying the port_offset setting automatically creates a new predefined base port number from which to apply the offset. This number differs from the standard Traffic Manager default port number to account for the potentially large number of instances on one instance host machine. If you apply the offset to the standard default port (for example, the REST API uses port 9070), there is a significant potential for port clashes.

When using port offsets, consider the following rules:

If you do not define port_offset for an instance, the Traffic Manager uses default port 9070 for the REST API.

If you do define port_offset for an instance, the base port for the REST API of that instance changes to a starting value of 50000.

(21)

Usage Metering and Activity Metrics Understanding the Services Controller

A similar calculation is also made for other internal and external ports on which the Traffic Manager listens. You can determine the ports in use by a particular instance by performing a REST API GET or PUT request for the instance resource. The REST API response includes several properties corresponding to the key externally facing services for that instance:

rest_address - The location of the configuration REST API for that instance. This property must be an FQDN.

ui_address - The location of the Administration UI for that instance, if enabled. This property must be an FQDN.

snmp_address - The location of the SNMP service for that instance, if enabled. This property must be an FQDN.

Each property contains a value in the form management_address:<port>, where <port> is the port number that has been set using these rules.

Note: The rest_address and ui_address ports are TCP, while the snmp_address port is UDP.

Usage Metering and Activity Metrics

The Services Controller automatically meters usage on a regular basis, and it optionally sends this information to Riverbed for billing purposes. By default, it records this information once per hour. If a Traffic Manager instance is active, the Services Controller polls it to obtain total throughput and peak activity metrics. The Services Controller creates a metrics log file with one line of metrics data for each Traffic Manager instance. Each line of metrics data records the name of the instance, the time elapsed since the resource was created, and the polled metrics. If an instance is not active, only the elapsed time is recorded.

If you want to generate usage or billing information, typically you process all metering log files and aggregate the results. You should use caution when aggregating data results for billing since metering records include failed deployments.

Note: Generating log files has a cumulative impact on disk space.

The Services Controller records the most recent metrics information for each instance in the inventory database. You can obtain this data using the REST API. The REST API does not supply bulk metrics data. The set of metrics is recorded across one line of the log file. The Services Controller writes the metering log file in comma-separated value (CSV) format.

The Services Controller appends an extra line to each metering log file containing an MD5 hash of the previous lines. Ignore this line in aggregating data for billing.

(22)

Understanding the Services Controller Usage Metering and Activity Metrics

The metering log file contains these fields.

Field Description

Timestamp The date and time, in UTC format, that the line was written. Instance ID The unique instance ID for the Traffic Manager instance. Owner Optionally, the owner of the Traffic Manager instance. Management IP The management IP address of the Traffic Manager instance.

Instance SKU The SKU assigned to the Traffic Manager instance (at the time of writing to the log). The SKU might vary between readings, and variations are not recorded in the metrics log file.

This property includes a hash of features applicable to the SKU. Ignore these features for billing purposes.

Instance Bandwidth The bandwidth (in Mbps) allocated to the Traffic Manager instance.

Feature Pack The feature pack assigned to the Traffic Manager instance (at the time of writing to the log).

Deploy Time The length of time (in days, hours and minutes) since the instance was deployed. Throughput The number of bytes sent by the Traffic Manager instance, as recorded in the SNMP

counter.

This number is cumulative and is reset whenever the Traffic Manager instance is restarted. It is not the throughput since the latest metering action.

To generate usage or billing information based on throughput, you should set your aggregating script to detect a drop in throughput and designate this as a restart. This property is applicable to active Traffic Manager instances only.

• For Idle or Inactive instances, it contains a value of 0 (zero) in the log. • For uncontactable instances, it contains a value of -1 in the log.

Peak Throughput The highest number of bytes sent by the Traffic Manager instance in any second of the previous hour.

This property is applicable to active Traffic Manager instances only. • For Idle or Inactive instances, it contains a value of 0 (zero) in the log. • For uncontactable instances, it contains a value of -1 in the log.

Peak Requests The highest number of requests received by the Traffic Manager instance in any second of the previous hour.

Peak SSL Requests The highest number of Secure Socket Layer (SSL) requests received by the Traffic Manager instance in any second of the previous hour.

Record Hash An MD5 or similar hash of the record from the Services Controller license file for tamper detection. Ignore this for billing purposes.

(23)

Usage Metering and Activity Metrics Understanding the Services Controller

If metrics are not collected for a period of time, peaks for the missing time are not recorded. If you reduce the metering interval, the peak values are still relative to the previous hour rather than the time since metrics were last collected.

Creating Metering Logs

By default, the Services Controller retains and archives metering log files in the /var/log/ssc/metering directory. You define the log output location when you first configure the Services Controller.

To allow the Services Controller to send metering logs to Riverbed, enable the phone home feature through the REST API. Perform a PUT request on the settings/phone_home resource, setting the

phone_home_enabled property to true. This property is set to false by default.

If the phone home feature is disabled, you must manually extract the metering log files and send them to Riverbed. Employing phone home on your system ensures that the process is entirely automatic, secure, and without staffing overhead.

The Services Controller employs a cron job, created during installation, to run a script that prepares a metering data archive file in this format:

<ssc_hostname><controller_license_name>.zip

The script sends this file to a secure SFTP server at Riverbed. The cron job runs once a month, at a randomly selected date and time. The Services Controller authenticates the destination SFTP server against a preconfigured locally stored host key. You can change this key by placing a new value in:

~/.ssh/known_hosts

If the host key is not found, or does not match the SFTP server, the phone home process stops. The Services Controller makes two attempts to send the log file. If the first attempt fails, the Services Controller waits a random period of up to 12 hours and then tries again. If the second attempt fails, the Services Controller sends an alert to the email notification list. Equally, if the file is successfully sent, the Services Controller sends an alert to the email notification list announcing the success.

Errors and warnings pertaining to the phone home mechanism are all logged to the file

metering_phone_home.log. This file provides a source of debugging information for problem resolution.

Note: You can also extract metering logs using the Services Controller Virtual Appliance. See “Generating System Dumps” on page 235.

Creating Metering Records Manually

The Services Controller includes a command line utility, prepare_metering_records, to manually prepare metering records for processing. You can use this script to create the metering archive file ready for transmission to Riverbed.

To create metering records

At the system prompt, enter:

prepare_metering_records [--help] [--force]

(24)

Understanding the Services Controller Health and Performance Monitoring

Creating Metering Records Using the Phone Home Script

You can run the metering_phone_home script to manually perform a phone home operation. This script is stored in the bin directory of the Services Controller installation directory. For example, in:

/opt/riverbed_ssc_1.4/bin

To run the metering phone home script

At the system prompt, enter:

metering_phone_home -v/--verbose -n/--noretry

-v/--verbose - Redirects logging to stdout instead of a file.

-n/--noretry - Prevents further attempts at sending.

Health and Performance Monitoring

Each Services Controller in your deployment monitors the health of all other peer Services Controllers and deployed Traffic Manager instances recorded in the inventory database. If a failure occurs, the Services Controller records a warning entry in the event log and sends an email notification to any system administrators declared in the database.

Note: You can configure the "From" e-mail address of alert e-mails. This address can be set in INSTALLROOT/conf/ email_config.txt, in the common section, as from_address. The symbol "$fqdn" will be replaced by the fully-qualified domain name of the SSC host. The other sections in this file should not normally be modified. For SSC installs on AWS it is likely that you will need to change this setting to be an address that is resolvable to the instance's public IP.

The Services Controller also monitors supported versions of deployed Traffic Manager instances for a number of key performance metrics. You can obtain these metrics through the REST API monitoring

resource.

A series of inter-controller REST API requests are used to identify the status of each Services Controller and Traffic Manager instance in your deployment. You must make sure that each Services Controller has network access to all other Services Controllers and instance hosts. You must also enable the REST service on your running Traffic Manager instances and record the REST access details in the inventory database.

Note: This process is performed automatically for Traffic Manager instances that are deployed by the Services Controller. However, you must enable and record REST API credentials for nonmanaged instances manually to allow monitoring to take place for these instances.

Your Services Controllers in an Active state are monitored.

Monitoring Settings

By default, all active Services Controllers share the task of monitoring. You can alter this behavior by modifying the Services Controller's monitoring mode settings in the REST API manager resource. These settings do not normally need to be modified from their default values. For details, see Chapter 7, “Using

(25)

Health and Performance Monitoring Understanding the Services Controller

You select whether each Services Controller individually monitors all other Services Controllers and Traffic Manager instances in the deployment, shares the responsibility of monitoring a proportion of Traffic Manager instances with other Services Controllers, or performs no monitoring actions at all.

You can view and modify various monitoring interval settings for the Services Controller in the REST API

monitoring resource. These settings do not normally need to be modified from their default values. For details, see Chapter 7, “Using the REST API with the Services Controller.”

You set distinct interval values for monitoring Services Controllers and Traffic Manager instances for each of the following two categories:

Monitoring Interval - The period of time, in seconds, that must elapse between health checks. The default value is 60. A setting of 0 forces the Services Controller to use a predefined short interval suitable for deployments that require very frequent monitoring.

Failure Identification Interval - The period of time, in seconds, that must elapse between continuous health check failures before your Services Controller determines that a service failure has occurred. This setting helps to prevent against transient network errors being incorrectly identified as service outages. The default value is 180.

You can also set the following failure identification settings:

Overdue Monitoring Warning Interval - The period of time, in seconds, that must elapse before any pending monitoring actions are considered overdue. This might occur during periods of unusually heavy load. If defined, a breach of this interval causes the Services Controller to issue an alert. The default value is 300.

Warning Email Interval - The period of time, in seconds, that must elapse before subsequent email alerts are sent. You can use this setting to avoid large numbers of emails being sent, one for each occasion a warning is triggered. A new email is sent only after this interval has passed. This email contains all monitoring events since the previous email was sent.

Retrieving Monitoring Data

You can retrieve stored monitoring state data from the Services Controller by using the REST API

monitoring resource. This resource is read-only and supports only the REST API GET request method. For details, see Chapter 7, “Using the REST API with the Services Controller.”

You can access the following child elements through the REST API monitoring resource:

/monitoring/manager - This element contains monitoring state data for all of your Services Controllers, whether or not they have failed.

/monitoring/host - This element contains monitoring state data for all of your service hosts, whether or not they have failed.

/monitoring/instance - This element contains monitoring state data and key performance metrics for your Traffic Manager instances, whether or not they have failed.

/monitoring/failures - This element contains a pair of arrays for failed Services Controllers and Traffic Manager instances. You can use this element to retrieve a list of currently failed devices without needing to check the status of each one individually.

(26)

Understanding the Services Controller Database-Only Updates

Database-Only Updates

The Services Controller uses the inventory database to store and maintain information about the state of each Traffic Manager instance it is aware of. This information includes the current status of each instance. For example, Inactive or Active.

However, the Services Controller does not actively monitor this state. If you start or stop a Traffic Manager instance outside of the Services Controller, it is unaware of this change of state.

You can use these techniques to resolve this monitoring issue:

Issue a GET REST API request for the Traffic Manager instance resource, including the URL parameter

status_check=true. The Services Controller actively checks the state of the Traffic Manager instance and updates the stored status accordingly.

Issue a PUT REST API request to modify the Traffic Manager instance resource and set the status

property to the known correct state with URL parameter deploy=false. The Services Controller updates the status of the Traffic Manager instance in the inventory database but does not attempt to start or stop the instance itself.

You can also use a database-only update if you need to update the recorded admin user password. This action is useful if the password has been set or changed on the Traffic Manager instance directly.

Using INSTALLROOT in This Guide

This guide uses the term INSTALLROOT to refer to the location of the Services Controller software installation directory. It is not an environment variable and is used in this guide for consistency only. Previous versions of the Services Controller used a deprecated environment variable, $SSCHOME, for this purpose. If this is still set in your environment, you must explicitly unset it prior to installing or upgrading the Services Controller. The default installation location for the Services Controller software package is:

(27)

CHAPTER 2

Managing Services Controller

Licenses

This chapter provides an overview and installation instructions for the Services Controller licenses. It includes the following sections:

“Overview of Services Controller Licenses” on page 17 “Services Controller Software Licenses” on page 18 “Traffic Manager FLA Licenses” on page 22

Overview of Services Controller Licenses

The Services Controller requires the following licenses, depending on your type of deployment and features:

Services Controller Software Licenses - The following Services Controller licenses differ in how they license Traffic Manager instances.

– Cloud Services Provider (CSP) - A CSP license allows the deployment and licensing of any number of Traffic Manager instances, and places no limits on the features or bandwidth they can use. Using a CSP license, the Services Controller implements a metering scheme to obtain throughput and other metrics from each Traffic Manager instance on a regular basis (typically, hourly) and records the data in central log files.

Service providers and hosting organizations use the metrics data to bill end users accordingly. Riverbed uses the same metrics data to charge the Services Controller customer.

– Enterprise - An Enterprise license enables prepayment for features and bandwidth. It also manages bandwidth and feature allocation to internal customers within licensed limits. An Enterprise license does not provide a billing option and must be used in conjunction with Bandwidth Pack licenses and Add-On licenses to enable deployment and licensing.

Bandwidth Pack - A Bandwidth Pack license is a secondary type of license for Enterprise customers. Each Bandwidth Pack license associates a specific amount of bandwidth (typically, 5 Gbps) with a Traffic Manager SKU. Each Bandwidth Pack is tied to a specific FLA.

Add-On - An Add-On license is a secondary type of license that provides a mechanism for adding feature specific licenses: for example, Federal Information Processing Standards (FIPS), Stingray Application Firewall (WAF), Stingray Aptimizer Web Accelerator, and Load Balancing as a Service (LBaaS). Each Add-On license is tied to a specific Enterprise License Key.

(28)

Managing Services Controller Licenses Services Controller Software Licenses

Traffic Manager Flexible Licensing Architecture (FLA) License - A Traffic Manager FLA license is intended for the Traffic Manager instances rather than Services Controller itself. With the FLA license you do not have to obtain licenses for individual Traffic Manager instances. Instead, the Services Controller applies a site-specific license to each instance and dynamically sets the feature set (SKU) and bandwidth desired for each instance. The FLA license requires a self-signed (or equivalent) certificate to be generated prior to use.

To retrieve Stingray and Services Controller product licenses

1. License tokens are automatically emailed to you when you order your product. If you have not received your tokens, contact [email protected].

2. Redeem license tokens at Riverbed Support at https://support.riverbed.com. To redeem tokens you must have a support site login and password. You can register for a new account at Riverbed Support.

3. Licenses are emailed to you as attachments.

Services Controller Software Licenses

This section describes the different licensing schemes for the Services Controller. For detailed information about installing Services Controller licenses, see “Installing the Services Controller Software License” on page 35.

The Services Controller supports the following licensing models:

Cloud Service Providers (CSP) with billing dependent on instance metering.

Enterprise licensing for managing Services Controller bandwidth allocation to internal customers. Before starting the first Services Controller in a cluster, you must place a valid license key file in the INSTALLROOT/licenses directory. New licenses in this location are automatically added to the database as they are discovered. The Services Controller license is automatically copied to the inventory database after communication is established. A newly installed Services Controller instance that is configured to use an existing inventory database containing valid license keys uses those keys to operate after it has started. Riverbed recommends that you install all subsequent Services Controller licenses via the REST API’s

controller_license_key resource. For details, see “controller_license_key Resource” on page 133. A Services Controller license can be free or it can be tied to an IP or MAC address:

Free - A free Services Controller license can be used on any host and is shared so that a single Services Controller license is used for multiple Services Controllers operating as a cluster and sharing one database.

Tied - A tied Services Controller license is associated with a single host by either the IP or MAC address. A tied license can be installed from any Services Controller in the same cluster as the Services Controller to be licensed. As a result, this Services Controller may not match the IP address and MAC address defined in the license.

– If the license is installed by the intended Services Controller, the license is written to the database and then verified. Once verified, the database is updated.

– If the license is installed by another Services Controller in the cluster, the license is written to the database, but it cannot be verified until the intended Services Controller is active. Once active, the Services Controller then verifies the license and updates the database.

(29)

Services Controller Software Licenses Managing Services Controller Licenses

If a Bandwidth Pack or Add-On license is associated with a tied Services Controller license, then that Services Controller license must be active to verify the license. Once active, the license is available to all Services Controllers in the cluster.

Bandwidth Pack Licenses

A Bandwidth Pack is a secondary type of license that is tied to a specific Enterprise Services Controller license.

Riverbed recommends that you add the Bandwidth Pack license to the shared database from a running SSC using the REST API’s bandwidth_pack_license_key resource. For details, see

“bandwidth_pack_license_key Resource” on page 128.

However, you can also place the Bandwidth Pack license in the INSTALLROOT/licenses directory and restart the SSC. New licenses are automatically added to the database as they are discovered.

Each Bandwidth Pack enables a specific amount of bandwidth (typically, 5 Gbps) to a Traffic Manager SKU. The Bandwidth Pack license allows you to deploy and license Traffic Manager instances with an aggregate bandwidth allowance equal to that of the Bandwidth Pack.

Each Bandwidth Pack is associated with one Services Controller license and cannot be used unless that Services Controller license has been loaded and found to be valid. A Bandwidth Pack only allows the deployment and licensing of Traffic Manager instances with one SKU. If you want to deploy Traffic Manager instances with different SKUs, then they require multiple Bandwidth Packs.

Services Controller licenses and Bandwidth Packs are perpetual or they can have start and end dates. Multiple Bandwidth Packs can license bandwidth for a SKU; their allowances are added to determine the total. Bandwidth Packs can be upgraded from one SKU to another. Where an existing Bandwidth Pack license is upgraded, a new license is issued with the same serial number as the existing one, but licensing a different SKU. Only one of a set of Bandwidth Pack licenses with a shared serial number is used at any one time.

Where licensed capacity is exceeded for a given SKU, all licensing requests for instances using that SKU are rejected. This behavior is also true of instances using an Add-On license SKU with insufficient licensed bandwidth.

If you are using a Bandwidth Pack license, the Services Controller does not allow you to exceed the licensed bandwidth with your deployed Traffic Manager instances. This behavior includes failed instances (with a status of Failed to deploy), but not instances scheduled to be deleted. Only instances that have been deleted are excluded; instances scheduled to be deleted still count toward deployed totals. Any instance whose status is not Deleted continues to consume licensing bandwidth. The same rules apply to the consumption for Add-On license SKUs bandwidth for instances using Add-On SKUs.

Installing the Bandwidth Pack License

To install the Bandwidth Pack license

1. Place the license on an accessible location in your infrastructure. For details about obtaining your license keys, see “To retrieve Stingray and Services Controller product licenses” on page 18.

2. Install and configure the Services Controller. For details, see Chapter 3, “Installing and Configuring the Services Controller Software.”

(30)

Managing Services Controller Licenses Services Controller Software Licenses

3. Riverbed recommends that you add the Bandwidth Pack license to the shared database from a running SSC using the REST API’s bandwidth_pack_license_key resource. For details, see

“bandwidth_pack_license_key Resource” on page 128.

Alternatively, copy a Services Controller license file containing the license key to the INSTALLROOT/ licenses directory of an SSC and restart the SSC. New licenses are automatically added to the database as they are discovered.

Upgrading Bandwidth Pack Licenses (Enterprise Licensing Model Only)

When your Services Controller is using the Enterprise Licensing model, you can upgrade a bandwidth pack to support a different STM SKU. This supports the replacement of existing purchased licensing with the same quantity of a more feature-rich STM SKU.

For example, a deployment of Traffic Manager instances is using the STM-300 STM SKU, and an upgrade to the STM-400 STM SKU is required.

For each existing bandwidth pack license key being upgraded, the Administrator will be provided with two new bandwidth pack licenses keys:

The first license will contain the same serial number as the existing bandwidth pack license key. Only one of these licenses can contribute licensed bandwidth in your Services Controller deployment at any time.

The second bandwidth pack license key will be a time-limited key which provides extra bandwidth used during the switchover.

This provides a workaround for the Services Controller’s protection against licensing compliance breaches.

Upgrading bandwidth pack licenses

1. Obtain the replacement (upgrade) license key and the supplementary temporary bandwidth pack license keys from Riverbed.

2. Install replacement and supplementary bandwidth pack license keys on the Services Controller. It may be necessary to set the upgrade bandwidth pack license key(s) to an Active status after

installation. Once complete, the controller_license_key resource's cluster_bandwidth property should show sufficient unused STM-400 bandwidth for the instances that are to be switched to use this STM SKU.

3. Create a feature_pack resource using the STM-400 STM SKU if one does not already exist.

4. Set the feature_pack property of each affected Traffic Manager instance resource to the desired STM-400

feature_pack resource (as created in step 3).

5. Remove the supplementary bandwidth pack license keys.

If the SSC does not allow removal of the supplementary license keys, it may indicate a licensing shortage. This situation may result in unlicensed Traffic Managers after these keys expire.

(31)

Services Controller Software Licenses Managing Services Controller Licenses

Add-On Licenses

An On license is a secondary type of license that is tied to a specific Enterprise License Key. Each Add-On license contributes license bandwidth for a single specific feature, known as an Add-Add-On SKU. These Add-On SKUs can be combined with base SKUs when the user creates a Feature Pack. For an instance set to use such a Feature Pack, the feature capabilities of the base SKU are augmented by those of the Add-On SKU.

Add-On SKUs can be used with CSP licensing model, and do not require the use of an Add-On license. The Services Controller supports the following Add-On licenses:

Federal Information Processing Standards (STM-B-ADD-FIPS, STM-CSP-U-ADD-FIPS) Stingray Application Firewall license (STM-B-ADD-WAF, STM-CSP-U-ADD-WAF)

Stingray Aptimizer Web Accelerator (STM-B-ADD-WEBACCEL, STM-CSP-U-ADD-WEBACCEL) Add-On licenses have unique serial numbers and do not support upgrades.

Installing an Add-On License

To retrieve an Add-On license, you are sent a token via email to redeem at Riverbed Support at https:// support.riverbed.com.

To install an Add-On license

1. Place the licenses on an accessible location in your infrastructure. For details about obtaining your license keys, see “To retrieve Stingray and Services Controller product licenses” on page 18.

2. Install and configure the Services Controller. For details, see Chapter 3, “Installing and Configuring the Services Controller Software.”

3. Riverbed recommends that you add the Add-On license to the shared database from a running SSC using the REST API’s add_on_pack_license_key resource. For details, see “add_on_pack_license_key Resource” on page 127.

Alternatively, copy a Services Controller license file containing the license key to the INSTALLROOT/ licenses directory of an SSC and restart the SSC. New licenses are automatically added to the database as they are discovered.

Licensing on Services Controller Clusters

A cluster of Services Controller instances that share a single inventory database also shares license restrictions:

If you make licensing changes through the REST API of one cluster member, the licensing changes are immediately accessible to the other cluster members.

If you put a license in INSTALLROOT/licenses directory of one cluster member, the license is added to the database when the Services Controller reboots. These changes are then accessible to the other cluster members.

Where each Services Controller in a cluster is licensed with a separate address-restricted key, each Services Controller must raise the required IP address or have the appropriate MAC address present to run. If you are using the Enterprise Licensing model, Bandwidth Packs might be tied to any of the Services Controller licenses, so it is important that all licenses are validated by their respective Services Controller instances.

(32)

Managing Services Controller Licenses Traffic Manager FLA Licenses

For detailed information about installing Services Controller licenses, see “Installing the Services Controller Software License” on page 35.

Traffic Manager FLA Licenses

A Traffic Manager FLA license is a license file intended for the Traffic Manager instances rather than Services Controller itself. The FLA license allows a Traffic Manager to contact one or more Services Controllers and to set Traffic Manager features (SKU) on a dynamic basis.

This information is required to generate a Traffic Manager FLA license:

A list of the fully qualified host names that is used for Services Controllers acting as license servers, along with port numbers.

The SSL server certificate that is used by all of the Services Controllers (different controllers are not permitted to use different certificates).

An FLA license attempts to contact each of the listed license servers and succeeds if they are contacted in turn, until it makes a successful connection or has attempted and failed to contact each one.

The SSL server certificate is verified by the FLA license. If an SSL server certificate does not match what is required by the FLA license, then that FLA license will not connect to the Services Controller license servers. If this failure occurs, you may need to generate a new FLA license or correct the key/certificate used by the Services Controller.

Generating a Self-Signed SSL Server Certificate

The Services Controller is commonly deployed using self-signed certificate/key pairs, using the self-signed server certificate in the FLA license.

To generate a self-signed SSL server certificate

1. At the Linux prompt, enter:

openssl req -x509 -nodes -newkey rsa:1024 -keyout key.pem -out cert.pem -days 365

Parameter Description

req Specifies an X509 certificate signing request management.

-x509 Specifies a self-signed certificate rather than a certificate request.

-nodes Specifies that the private key will not be encrypted (otherwise, the server needs a password to start).

-newkey rsa:1024 Generates a new certificate request and sets the key size. -keyout key.pem Sets the target for the new private key.

-out cert.pem Sets the target for the certificate.

-days 365 Specifies the duration of the certificate (default is 30 days). A longer period may be desirable as a fresh FLA license will need to be generated and then deployed to all STM instances when the certificate expires.