Cisco Application Services Engine Deployment Guide, Release 1.1.3

(1)

Cisco Application Services Engine Deployment Guide, Release 1.1.3

First Published: 2020-05-06 Last Modified: 2020-09-03

Americas Headquarters

Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883

(2)

(3)

C O N T E N T S

New and Changed Information 1

C H A P T E R 1

New and Changed Information 1

Deployment Overview and Requirements 3

C H A P T E R 2

Deployment Overview 3

Prerequisites and Guidelines 4

ACI Fabric Connectivity 6

Deploying as Physical Appliance 13

C H A P T E R 3

Deploying Cisco Application Services Engine as Physical Appliance 14

Deploying in VMware ESX 21

C H A P T E R 4

Deploying Cisco Application Services Engine in VMware ESX 22

Deploying in Linux KVM 29

C H A P T E R 5

Deploying Cisco Application Services Engine in Linux KVM 30

Deploying in Amazon Web Services 37

C H A P T E R 6

(4)

(5)

C H A P T E R

1

New and Changed Information

•New and Changed Information, on page 1

New and Changed Information

The following table provides an overview of the significant changes to the organization and features in this guide from the release in which the guide was first published to the current release. The table does not provide an exhaustive list of all changes made to the guide.

Table 1: Latest Updates

Where Documented New Feature or Update

Release

--First release of this document. 1.1.3d

(6)

(7)

C H A P T E R

2

Deployment Overview and Requirements

•Deployment Overview, on page 3 •Prerequisites and Guidelines, on page 4 •ACI Fabric Connectivity, on page 6

Deployment Overview

Cisco Application Services Engine provides a common platform for deploying Cisco Data Center applications. This release of Application Services Engine supports the Cisco Day-2 Operations apps, which provide real time analytics, visibility, and assurance for policy and infrastructure, and the Cisco ACI Multi-Site Orchestrator app, which provides a single pane of glass view into managing multiple Cisco ACI fabrics.

Cisco Data Center apps are resource intensive applications that rely on modern technology stacks. Cisco Application Services Engine can host containerized applications on a common platform.

Cisco Application Services Engine is deployed as a cluster of three service nodes. This clustering provides reliability and high-availability software framework.

Hardware vs Software Stack

Originally, the Application Services Engine was offered as a cluster of specialized Cisco UCS servers with the software framework pre-installed on it and referred to the combined package of the hardware and the software stack. With addition of other form factors, the Cisco ASE software stack has been decoupled from the hardware and the term can be used to refer to either the physical appliance nodes, the software stack alone when deploying in a virtual environment, or the combination of both.

This document describes how to deploy the Application Services Engine software stack.

Available Form Factors

Cisco Application Services Engine can be deployed using a number of different form factors. Keep in mind however, you must use the same form factor for all nodes, mixing different form factors within the same cluster is not supported.

• Cisco Application Services Engine physical appliance (.iso)

This form factor refers to software stack that can be deployed on the original physical appliance hardware that you purchased with the Application Services Engine software stack pre-installed on it.

(8)

The later sections in this document describe how to re-deploy the software stack on the existing physical appliance hardware. Deploying the original Cisco Application Services Engine physical appliance, including physical network connectivity is described inCisco Application Services Engine Hardware Installation Guide.

• VMware ESX (.ova)

• Amazon Web Services (.ami)

• Linux KVM (.qcow2)

If you have previously deployed an earlier release of Cisco Application Services Engine, stateful upgrade or migration of the cluster is not supported. You will need to deploy a brand new Release 1.1.3d cluster and reinstall all the applications.

Note

Application Services Engine and Cisco DCNM

Application Services Engine may be used in context of Cisco DCNM. In this case, DCNM is not an application running in the Application Services Engine software stack. Instead, the DCNM image (.iso) is installed

directly on the Application Services Engine physical servers in order to provide additional compute resources to the applications installed and running in Cisco DCNM thus enabling horizontal scaling of the DCNM platform. As this document deals with the Application Services Engine software stack deployments, seeCisco Application Services Engine Installation Guide For Cisco DCNMfor any DCNM-related information.

Supported Applications

For the full list of supported applications and the associated compatibility information, see theCisco Day-2 Operations Apps Support Matrix.

At this time, the virtual Cisco Application Services Engine form factors support the Cisco ACI Multi-Site Orchestrator application only. For other applications, you must deploy the Application Services Engine as a physical appliance.

Note

Prerequisites and Guidelines

Network Time Protocol (NTP)

The Application Services Engine nodes use NTP for clock synchronization, so you must have an NTP server configured in your environment.

Application Services Engine External Networks

When first configuring Application Services Engine, you will need to provide two IP addresses for the two Application Services Engine interfaces—one connected to the Data Network and the other to the Management Network.

(9)

• Application Services Engine node clustering • Application to application communication

• Application Services Engine nodes to Cisco APIC nodes communication

For example, the network traffic for Day-2 Operations applications such as NAE.

This network must have IP reachability to the ACI fabric's in-band IPs for Day-2 Operations apps and out-of-band IP for Multi-Site Orchestrator. The data network uses MTU of1500.

Cluster nodes must be able to communicate with each other on this network. • Management Network is used for:

• Accessing the Application Services Engine GUI

• Accessing the Application Services Engine CLI via SSH • DNS and NTP communication

• Application Services Engine firmware upload • Intersight device connector

The two interfaces can be in the same or different subnets. In addition, each network's interfaces across different nodes in the cluster can also be in different subnets.

Connectivity between the nodes is required on both networks with the round trip time (RTT) not exceeding 150ms.

Application Services Engine Internal Networks

Two additional internal networks are required for communication between the containers used by the Application Services Engine:

• Application overlay is used for applications internally within Application Services Engine Application overlay must be a/16network.

• Service overlay is used internally by the Application Services Engine. Service overlay must be a/16.

Communications between containers deployed in different Application Services Engine nodes is

VXLAN-encapsulated and uses the data interfaces IP addresses as source and destination. This means that the Application Overlay and Service Overlay addresses are never exposed outside the data network and any traffic on these subnets is routed internally and does not leave the cluster nodes. As such, when configuring these networks, ensure that they are unique and do not overlap with any existing networks or services you may need to access from the Application Services Engine cluster nodes

Note

Communication Ports

The following ports are required by the Application Services Engine cluster and its applications:

Deployment Overview and Requirements

(10)

• For traffic to and from the Application Services Engine cluster: • Port 53 for DNS

• Port 443 for HTTPS • Ports 22 and 1022 for SSH

• For traffic between the Application Services Engine nodes: • Ports 3379, 3380, 9969, 9979, 15223 for KMS • Port 19999 forconfd

• Ports 30000-30100 for Application Services Engine cluster services • Ports 30500-30600 for Kubernetes

• For traffic between the Application Services Engine cluster and Cisco ACI fabrics: • Ports 2022 and 884 for Network Insights Assurance

• Ports 5640-5656 for Network Insights Resources

ACI Fabric Connectivity

Previous releases of the Application Services Engine allowed for two modes when it came to cluster connectivity to the ACI fabrics: cluster nodes connected directly to leaf switches and configured by an application running in the APIC (fabric internal) and cluster nodes separated by a Layer 3 network (fabric external). The fabric internal mode has been deprecated in Release 1.1.3d and later and the following section illustrates how to connect the Application Services Engine cluster to an ACI fabric via an L3Out or an EPG.

Connecting via an L3Out

Connectivity depends on that type of applications deployed in the Application Services Engine:

• If you are deploying Day-2 Operations applications, you will use the data interface IP to communicate to the in-band management network in the management tenant of each fabric. Connectivity to all fabrics is established via an L3Out to the in-band management network in the in-band VRF.

• In addition, if you are deploying Multi-Site Orchestrator, you must also establish connectivity to the out-of-band (OOB) interface of each site's Cisco APIC cluster.

If you plan to connect the cluster across a Layer 3 network, keep the following in mind:

• You must configure an L3Out and the external EPG for Cisco Application Services Engine data network connectivity.

Configuring external connectivity in an ACI fabric is described inCisco APIC Layer 3 Networking Configuration Guide.

• If you specify a VLAN ID for your data interface during setup of the cluster, the host port must be configured astrunkallowing that VLAN.

(11)

However, in most common deployments, you can leave the VLAN ID empty and configure the host port inaccessmode.

The following two figures show two distinct network connectivity scenarios when connecting the Application Services Engine cluster to ACI fabrics via an L3Out. The primary purpose of each depends on the type of application you may be running in your Application Services Engine.

Figure 1: Connecting via L3Out, Day-2 Operations Applications Deployment Overview and Requirements

(12)

Figure 2: Connecting via L3Out, Multi-Site Orchestrator

Connecting via an EPG/BD

Like in the previous example, connectivity depends on that type of applications deployed in the Application Services Engine:

• If you are deploying Day-2 Operations applications, you will use the data interface IP to communicate to the in-band management network in the management tenant of each fabric. The data interface IP subnet connects to an EPG/BD in the fabric and must have a contract established to the local in-band EPG in

(13)

the management tenant. We recommend deploying the Application Services Engine in the management tenant and in-band VRF. Connectivity to other fabrics is established via an L3Out.

• In addition, if you are deploying Multi-Site Orchestrator, you must also establish connectivity to the out-of-band (OOB) interface of each site's Cisco APIC cluster.

If you plan to connect the cluster directly to the ACI leaf switches, keep the following in mind:

• If deploying in VMware ESX or Linux KVM, the host must be connected to the fabric via trunk port. • We recommend configuring the bridge domain (BD), subnet, and endpoint group (EPG) for Cisco

Application Services Engine connectivity in management tenant.

Because the Application Services Engine requires connectivity to the in-band EPG in the in-band VRF, creating the EPG in the management tenant means no route leaking is required.

• You must create a contract between the fabric's in-band management EPG and Cisco Application Services Engine EPG.

• If you specify a VLAN ID for your data network during setup of the cluster, the Application Services Engine interface and the port on the connected network device must be configured astrunk

However, in most cases we recommend not assigning a VLAN to the data network, in which case you must configure the ports inaccessmode.

• If several ACI fabrics are monitored with apps on the Services Engine cluster, L3Out with default route or specific route to other ACI fabric in-band EPG must be provisioned and a contract must be established between the the cluster EPG and the L3Out's external EPG.

The following two figures show two distinct network connectivity scenarios when connecting the Application Services Engine cluster to ACI fabrics using an EPG. The primary purpose of each depends on the type of application you may be running in your Application Services Engine.

Deployment Overview and Requirements

(14)

(15)

Figure 4: Connecting via an EPG/BD, Multi-Site Orchestrator Deployment Overview and Requirements

(16)

(17)

C H A P T E R

3

Deploying as Physical Appliance

•Prerequisites and Guidelines, on page 13

•Deploying Cisco Application Services Engine as Physical Appliance, on page 14

Prerequisites and Guidelines

Before you proceed with deploying the Nexus Dashboard cluster, you must:

• You must have reviewed and completed the general prerequisites described in theDeployment Overview, on page 3.

• Ensure you are using the correct hardware.

The physical appliance form factor is supported on the original Application Services Engine appliance hardware only. The following table lists the PIDs and specifications of the physical appliance servers:

Table 2: Supported Hardware

Hardware PID

• UCS C220 M5 Chassis

• 2x 10 core 2.2G Intel Xeon Silver CPU • 4x 25G Virtual Interface Card 1455 • 4x 2.4TB HDDs

• SSD, NVMe and M2 SATA • 256 GB of RAM

• 1050W power supply SE-NODE-G2

A cluster of 3xSE-NODE-G2appliances.

SE-CL-L3

The above hardware supports Nexus Dashboard software only. If any other operating system is installed, the node can no longer be used as a Nexus Dashboard node.

(18)

You must have at least a 3-node cluster. Up to four additional worker nodes can be added for horizontal scaling if required by the type and number of applications you will deploy.

Deploying Cisco Application Services Engine as Physical

Appliance

This section describes how to deploy Cisco Application Services Engine cluster on the physical Cisco ASE servers. You cannot deploy this image on any other hardware.

When you first receive the Application Services Engine physical hardware, it comes preloaded with the software image. If you simply want to configure the existing software, skip the first two steps in this section. Alternatively, you can choose to redeploy the software stack on the Application Services Engine hardware. For example if your existing hardware came with an earlier release image and you want to deploy the Release 1.1.3d instead without upgrading.

Note

Before you begin

• You have familiarized yourself with deployment options, recommendations, Application Services Engine connectivity described inDeployment Overview and Requirements, on page 3.

• Ensure that you meet the requirements and guidelines described inPrerequisites and Guidelines, on page 13.

• You must have the physical servers racked and connected as described inCisco Application Services Engine Hardware Installation Guide.

The ISO form factor is supported only for the original Cisco Application Services Engine hardware cluster.

Step 1 Download the Cisco Application Services Engine image. Skip this step if you are configuring the pre-installed image. a) Browse to the Software Download page.

https://software.cisco.com/download/home/286324815/type b) Click Application Services Engine Software.

c) From the left sidebar, choose the Application Services Engine version you want to download. d) Download the Cisco Application Services Engine image (case-dk9.<version>.iso).

Step 2 Deploy the ISO to every server in your cluster.

Skip this step if you are configuring the pre-installed image. Step 3 Configure the first node.

a) Connect to first node's console using CIMC management IP. You will be prompted to run the first-time setup utility:

(19)

[...]

[ OK ] Started atomix-boot-setup. Starting atomix-ready...

Starting Initial cloud-init job (pre-networking)... Starting cloud data source ...

Press any key to run first-boot setup on this console...

b) Enter the cluster name for the service node.

Cluster Name: ServiceEngine

c) Specify that this is the master node.

All nodes in the cluster will be set tomaster. Master Node? (Y/n): y

d) Specify that you are configuring the first node.

When configuring second and third node, you will be able to skip some steps by downloading configuration from the first node. Since this is the first node you are configuring, enter n.

Download Config From Peers? (Y/n): n

e) Enter the node name for the service node.

Node Name: ServiceNodel

f) Enter and confirm the password for therootuser.

This password will be used for the Application Services Engine'srescue-userlogin, as well as the initial password

for the GUI'sadminuser. Admin Password:

Reenter Admin Password:

g) Enter the data network information.

You will be prompted to enter the data network IP address, netmask, and gateway.

Optionally, you can also provide the VLAN ID for the network. For most deployments, you can leave this field blank.

Data Network:

IP Address/Mask: 192.168.6.172/24

Gateway: 192.168.6.1

Vlan ID (optional): 410

h) Enter the management network information.

Management Network:

IP Address/Mask: 192.168.9.172/24

Gateway: 192.168.9.1

i) Provide the list of other nodes.

Data network IP address and serial numbers of the other master nodes in the cluster.

Master List (Space Separated Data Network IP,Serialnumber List)

(Ex: 192.192.5.101,WZP22451Q1R 192.192.5.103,WZP22451Q5A): 192.168.6.173,WZP23340A7P 192.168.6.174,WZP23340A7Q

j) Provide the DNS details.

You will need to specify the DNS hostname or IP address as well as the search domain to be used by the Application Services Engine node.

Deploying as Physical Appliance

(20)

DNS:

Providers (Space Separated IP List): 192.168.10.10

Search Domains (Space Separated List): tme-lab.local

k) Provide the NTP servers.

NTP Servers (Space Separated IP List): 192.168.10.120

l) Provide the internal networks information.

You will need to provide the service and application subnet information.

The application overlay network defines the address space used by the application's services running in the Application Services Engine. The services network is an internal network used by the Application Services Engine and its processes. Both of these subnets must be/16.

Communications between containers deployed in different Application Services Engine nodes is

VXLAN-encapsulated and uses the data interfaces IP addresses as source and destination. This means that the Application Overlay and Service Overlay addresses are never exposed outside the data network and any traffic on these subnets is routed internally and does not leave the cluster nodes. As such, when configuring these networks, ensure that they are unique and do not overlap with any existing networks or services you may need to access from the Application Services Engine cluster nodes

Note

Service Subnet (not exposed externally) [100.80.0.0/16]: 100.80.0.0/16

App Subnet (not exposed externally) [172.17.0.1/16]: 172.17.0.1/16

Example:

Starting apic-sn setup utility

Setup utility for Application Services Engine with SerialNumber WZP23340A7X and running version 1.1.3d Use AD anytime to start over

Cluster Name: ServiceEngine Master Node? (Y/n): y

Download Config From Peers? (Y/n): n Node Name: ServiceNodel

Admin Password:

Reenter Admin Password: Data Network: IP Address/Mask: 192.168.6.172/24 Gateway: 192.168.6.1 Vlan ID (optional): 410 Management Network: IP Address/Mask: 192.168.9.172/24 Gateway: 192.168.9.1

(Ex: 192.192.5.101,WZP22451Q1R 192.192.5.103,WZP22451Q5A): 192.168.6.173,WZP23340A7P 192.168.6.174,WZP23340A7Q

DNS:

Providers (Space Separated IP List): 192.168.10.10 Search Domains (Space Separated List): tme-lab.local NTP Servers (Space Separated IP List): 192.168.10.120

Service Subnet (not exposed externally) [100.80.0.0/16]: 100.80.0.0/16 App Subnet (not exposed externally) [172.17.0.1/16]: 172.17.0.1/16

Step 4 Review the configuration

After you enter all the configuration, review it and confirm.

Please review the config: App Subnet: 172.17.0.1/16 Cluster Name: ServiceEngine Cluster Size: 3

(21)

Domain Name: dev-infra12.case.local Providers:

- 171.70.168.183 Search Domains: - atomix.local Download Config: false Data Network: Gateway: 192.168.6.1 IP Address/Mask: 192.168.6.172/24 Vlan ID: 410 Management Network: Gateway: 192.168.9.1 IP Address/Mask: 192.168.9.172/24 Master List: - ipAddress: 192.168.6.173 name: WZP23340A7P serialNumber: WZP23340A7P - ipAddress: 192.168.6.174 name: WZP23340A7Q serialNumber: WZP23340A7Q NTP Servers: - 192.168.10.120

Node Name: ServiceNodel Node Role: Master Node Type: Physical Password: <hidden>

Service Subnet: 100.80.0.0/16

Re-enter config?(y/N) n

Login with rescue-user & issue acidiag health to check cluster status

CentOS Linux 7 (Core)

Kernel 4.14.174stock-1 on an x86_64

ServiceNodel login:

Step 5 Configure the second node.

a) Connect to the node's console using CIMC management IP. You will be prompted to run the first-time setup utility:

[...]

d) Specify that you've already configured the first node.

When configuring second and third node, you can skip some steps by downloading configuration from the first node.

Download Config From Peers? (Y/n): y Deploying as Physical Appliance

(22)

Node Name: ServiceNode2

f) Enter and confirm the password for therescue-useruser.

We recommend configuring the same password for all nodes, however you can choose to provide different passwords for the second and third node. If you provide different passwords, the first node's password will be used as the initial password of theadminuser in the GUI.

Admin Password:

Optionally, you can also provide the VLAN ID for the network. For most deployments, you can skip the VLAN ID parameter.

Data Network:

IP Address/Mask: 192.168.6.173/24

Gateway: 192.168.6.1

Vlan ID (optional):

Management Network:

IP Address/Mask: 192.168.9.173/24

Gateway: 192.168.9.1

(Ex: 192.192.5.101,WZP22451Q1R 192.192.5.103,WZP22451Q5A): 192.168.6.172,WZP23340A7X 192.168.6.174,WZP23340A7Q

j) Review the configuration

After you enter all the configuration for second and third node, review it and confirm.

Please review the config

Cluster Name: ServiceEngine Cluster Size: 3 Data Network:

Gateway: 192.168.6.1

IP Address/Mask: 192.168.6.173/24 Vlan ID: 410

Download Config: true Management Network:

Gateway: 192.168.9.1

IP Address/Mask: 192.168.9.173/24 Master List:

ipAddress: 192.168.6.172 serialNumber: WZP23340A7X ipAddress: 192.168.6.174 serialNumber: WZP23340A7Q Node Name: ServiceNode2

Node Role: Master Node

Type: Physical Password: <hidden> Re-enter config?(y/N): n

Step 6 Repeat the previous step to configure the third node. Step 7 Verify that the cluster is healthy.

(23)

It may take up to 30 minutes for the cluster to form and all the services to start.

After all three nodes are ready, you can log in to any one node via SSH and run the following command to verify cluster health:

a) Verify that the cluster is up and running.

You can check the current status of cluster deployment by logging in to any of the nodes and running theacidiag healthcommand.

While the cluster is converging, you may see the following outputs:

$ acidiag health

k8s install is in-progress

k8s services not in desired state - [...]

k8s: Etcd cluster is not ready

When the cluster is up and running, the following output will be displayed:

All components are healthy

b) Log in to the Application Services Engine GUI.

After the cluster becomes available, you can access it by browsing to any one of your nodes' management IP addresses. The default password for theadminuser is the same as therescue-userpassword you chose for the first node of the

Application Services Engine cluster.

When you first log in, you will be prompted to change the password.

Deploying as Physical Appliance

(24)

(25)

C H A P T E R

4

Deploying in VMware ESX

•Deploying Cisco Application Services Engine in VMware ESX, on page 22

Prerequisites and Guidelines

You must have reviewed and completed the general prerequisites described in theDeployment Overview, on page 3.

When deploying in VMware ESX, ensure that the VMware Tools periodic time synchronization is disabled. To disable the VMware Tools period time synchronization:

1. Right-click the node's VM and select Edit Settings. 2. In the Edit Settings window, select the VM Options tab.

3. Expand the VMware Tools category and uncheck the Synchronize guest time with host option.

In addition, the following apply when deploying when deploying in VMware ESX.

Table 3: Deployment Requirements

Requirements Orchestrator Version • ESXi 5.5 or later • 16 vCPUs • 48 GB of RAM • 230 GB disk

• We recommend that each Application Services Engine node is deployed in a different ESX server.

• Additional or higher requirements may depend on the applications installed in the cluster. Consult individual applications' documentation for details

Release 1.1.3d*

(26)

Deploying Cisco Application Services Engine in VMware ESX

This section describes how to deploy Cisco Application Services Engine cluster using VMware vCenter.

Step 1 Download the Cisco Application Services Engine image. a) Browse to the Software Download page.

https://software.cisco.com/download/home/286324815/type b) Click Application Services Engine Software.

c) From the left sidebar, choose the Application Services Engine version you want to download.

d) Download the Cisco Application Services Engine image for VMware ESX (case-dk9.<version>.ova).

Step 2 Log in to your VMware vCenter.

You cannot deploy the OVA directly in the ESX host, you must deploy it using the vCenter. Step 3 Start the new VM deployment.

a) Right-click the ESX host where you want to deploy. b) Then select Deploy OVF Template...

(27)

Step 4 Select the OVA image.

a) Select Local file.

b) Click Choose Files and select the OVA file you downloaded. c) Click Next to continue.

Step 5 Provide a name and location for the VM. a) Provide the name for your virtual machine. b) Select the location for the virtual machine. c) Click Next to continue

Step 6 Select a compute resource.

a) Select the vCenter datacenter and the ESX host for the virtual machine. b) Click Next to continue

Step 7 Review the information and click Next to continue. Step 8 Select storage.

Deploying in VMware ESX

(28)

a) From the Select virtual disk format dropdown, select Thick Provision Lazy Zeroed. b) Select the datastore for the virtual machine.

We recommend a unique datastore for each node. c) Click Next to continue

Step 9 In the Select networks screen, accept default values and click Next to continue.

There are two networks, fabric0 is used for the data network and mgmt0 is used for the management network. Step 10 Provide the node and network information.

a) In the Node Name field, provide the hostname for node. For example,case-node1

b) In the Password fields, provide the password.

c) Check the Master Node checkbox.

(29)

d) Provide the Management Address and Subnet for the node. For example,192.168.10.11/24.

e) Provide the Management Gateway IP. For example,192.168.10.1.

f) Provide the Data Network Address and subnet. For example,172.10.10.11/24.

g) Provide the Data Network Gateway. For example,172.10.10.1.

h) (Optional) If the data traffic is on a VLAN, provide the Data Network Vlan.

For most deployments, you can leave this field blank. If you do want to provide a VLAN ID for your data network, you can enter it in this field, for example100.

Step 11 Provide Application Services Engine cluster information for this node. a) Provide the name for the Application Services Engine cluster.

This name must be the same for all nodes. For example,case-cluster.

b) In the Master List field, provide the IP addresses of the other 2 nodes you will configure for your cluster. Each IP address in the list must be separated by a space.

c) Provide a value for the dbgtoken field.

Since this is the first node you are deploying, provide any 11-character value for this field (for example,

abcdef12345). When you deploy the other two nodes, you will use this field to provide a token from the first node

to simplify configuration.

d) Leave the Download Config From Peers checkbox unchecked. You will use this option when configuring the other two nodes.

Step 12 Provide the common Application Services Engine cluster information. You must provide this information when configuring the first node. a) Provide the App Subnet.

The application overlay network defines the address space used by the application's services running in the Application Services Engine. This must be a/16subnet.

For example,1.1.0.0/16.

Communications between containers deployed in different Application Services Engine nodes is VXLAN-encapsulated and uses the data interfaces IP addresses as source and destination. This means that the Application Overlay and Service Overlay addresses are never exposed outside the data network and any traffic on these subnets is routed internally and does not leave the cluster nodes. As such, when configuring these networks, ensure that they are unique and do not overlap with any existing networks or services you may need to access from the Application Services Engine cluster nodes

Note

b) Provide the Services Subnet.

(30)

The services network is an internal network used by the Application Services Engine and its processes. This must be a/16subnet.

c) Provide the NTP Servers information. For example,10.197.145.2 10.197.146.2.

d) Provide the Name servers information. For example,10.197.145.3.

e) (Optional) Provide the Search Domains information. For example,company.com.

Step 13 Verify that all information is valid and click Next to continue.

After you complete the Customize template screen, a verification banner is shown at the top.

Step 14 In the Ready to complete screen, verify that all information is accurate and click Finish to begin deploying the first node.

Step 15 Wait for the VM deployment to complete, then start the VM. Step 16 Log in to the first node's console as therescue-user.

Use the password you specified in the OVF template when deploying the VM. Step 17 Retrieve thedbgtoken.

Run the following command:

$ acidiag dbgtoken

09GZ1PMB8CML

Make a note of this token, you will use it to deploy the other two nodes.

Keep in mind, the token expires and is refreshed every 30 minutes, so ensure to retrieve it when ready to deploy the second and third nodes.

(31)

The steps to deploy the second and third nodes are similar, with the exception that you can now use thedbgtokenfrom

the first node to skip some of the configuration.

a) Right-click the ESX host where you want to deploy and select Deploy OVF Template.... We recommend using a different ESX host for each node.

b) Select Local file, click Choose Files, and select the OVA file you downloaded. Then click Next to continue. c) Provide the name for your virtual machine and select the location for the virtual machine. Then click Next to

continue.

d) Select the vCenter datacenter and the ESX host for the virtual machine. Then click Next to continue. e) Review the information and click Next to continue.

f) From the Select virtual disk format dropdown, select Thick Provision Lazy Zeroed. Then select the datastore for the virtual machine and click Next to continue.

g) In the Select networks screen, accept default values and click Next to continue. h) In the Node Name field, provide the hostname for node.

i) In the Password fields, provide the password.

j) Provide the Management Network Address and subnet. k) Provide the Management Gateway IP.

l) Provide the Data Network Address and subnet. m) Provide the Data Network Gateway.

n) (Optional) If the data traffic is on a VLAN, provide the Data Network Vlan. o) Provide the name for the Application Services Engine cluster.

This name must be the same for all Application Services Engine nodes.

p) In the Master List field, provide the IP addresses of the other 2 nodes in your cluster. Each IP address in the list must be separated by a space.

q) Provide the dbgtoken you obtained from the first node.

The token expires and is refreshed every 30 minutes, ensure to obtain the latest valid token from the first node before continuing. For example,09GZ1PMB8CML.

r) Check the Download Config From Peers checkbox.

The second and third nodes will download common configuration parameters from the first node using thedbgtoken.

Note

s) Skip Cluster Configuration Optional fields and click Next to continue.

t) In the Ready to complete screen, verify that all information is accurate and click Finish to begin deploying the second node.

Step 19 Repeat the previous step to deploy the third node.

Step 20 Wait for the second and third node VMs deployment to complete, then start the VMs. Step 21 Verify that the cluster is healthy.

(32)

After the cluster becomes available, you can access it by browsing to any one of your nodes' management IP addresses. The default password for theadminuser is the same as therescue-userpassword you chose for the

first node of the Application Services Engine cluster.

(33)

C H A P T E R

5

Deploying in Linux KVM

•Deploying Cisco Application Services Engine in Linux KVM, on page 30

Prerequisites and Guidelines

(34)

Table 4: Deployment Requirements

Requirements Orchestrator Version

• Linux Kernel3.10.0-957.el7.x86_64or later

with KVMlibvirt-4.5.0-23.el7_7.1.x86_64

or later • 16 vCPUs • 48 GB of RAM • 800 GB disk

Each node requires a dedicated disk partition • The disk must have I/O latency of 20ms or less.

You can verify the I/O latency using the following command:

# fio --rw=write --ioengine=sync --fdatasync=1

--directory=test-data_with_se --size=22m --bs=2300 --name=mytest

And confirm that the99.00th=[<value>]in the fsync/fdatasync/sync_file_rangesection is

under 20ms.

• We recommend that each Application Services Engine node is deployed in a different KVM server.

• Additional or higher requirements may depend on the applications installed in the cluster. Consult individual applications' documentation for details

Release 1.1.3d*

*We do not recommend deploying earlier releases

Deploying Cisco Application Services Engine in Linux KVM

This section describes how to deploy Cisco Application Services Engine cluster in Linux KVM.

Step 1 Download the Cisco Application Services Engine image. a) Browse to the Software Download page.

(35)

b) Click Application Services Engine Software.

c) From the left sidebar, choose the Application Services Engine version you want to download.

d) Download the Cisco Application Services Engine image for Linux KVM (case-dk9.<version>.qcow2).

Step 2 Log in to your Linux KVM as therootuser.

Step 3 Copy the image to all KVM hosts.

Keep in mind, theqcow2image on each node must be on a unique disk partition.

Note

You can usewgetorscpto copy the image, for example:

# scp case-dk9.1.1.3d.qcow2 root@<kvm-host-ip>:/home/sn_base/qcow2

The following steps assume you copied the image into the/home/sn_base/qcow2directory.

Step 4 On each node, create a snapshot of the baseqcow2image.

You will create a snapshot of the image you downloaded and use the snapshots as the disk images for the nodes' VMs. a) Log in to the KVM host where you will host a node.

b) Create a folder for the node's snapshot.

The following steps assume you create the snapshot in the/home/mso-node1directory. # mkdir -p /home/mso-node1/

# cd /home/mso-node1

c) Create the snapshot.

# qemu-img create -f qcow2 -b /home/sn_base/qcow2/case-dk9.1.1.3d.qcow2 /home/mso-node1/disk0.qcow2

d) Repeat this step for every node in the cluster. Step 5 Deploy the VMs for all nodes.

a) Open the KVM console and click New Virtual Machine.

b) In the New VM screen, choose Import existing disk image option and click Forward. c) In the Provide existing storage path tab, choose thedisk0.qcow2file.

We recommend that each node's disk image is stored on its own disk partition. d) ChooseGenericfor the OS type and Version, then click Forward.

e) Choose 48GB memory and 16 CPUs, then click Forward.

f) Enter the Name of the virtual machine and check the Customize configuration before install option. Then click

Finish.

g) Select the NIC for the Virtual Network Interface and choose the device model as e1000. h) Leave the default Mac address.

i) Click Apply.

j) Click Begin Installation.

k) Repeat this step for the other nodes. Step 6 Configure the first node.

a) Connect to first node's console.

You will be prompted to run the first-time setup utility:

[...]

Starting Initial cloud-init job (pre-networking)...

Deploying in Linux KVM

(36)

Starting cloud data source ...

d) Specify that you are configuring the first node.

When configuring second and third node, you will be able to skip some steps by downloading configuration from the first node. Since this is the first node you are configuring, enter n.

Download Config From Peers? (Y/n): n

Node Name: ServiceNodel

f) Enter and confirm the password for therootuser.

for the GUI'sadminuser. Admin Password:

Optionally, you can also provide the VLAN ID for the network. For most deployments, you can leave this field blank.

Data Network:

IP Address/Mask: 192.168.6.172/24

Gateway: 192.168.6.1

Vlan ID (optional): 410

Management Network:

IP Address/Mask: 192.168.9.172/24

Gateway: 192.168.9.1

Unlike physical appliance deployments where you obtain the serial numbers from CIMC, for Linux KVM you can choose the strings to use. We recommend usingCiscoSN01,CiscoSN02,CiscoSN03for node1, node2, node3. Master List (Space Separated Data Network IP,Serialnumber List)

(Ex: 192.192.5.101,WZP22451Q1R 192.192.5.103,WZP22451Q5A): 192.168.6.173,CiscoSN02 192.168.6.174,CiscoSN03

j) Provide the DNS details.

You will need to specify the DNS hostname or IP address as well as the search domain to be used by the Application Services Engine node.

(37)

DNS:

Providers (Space Separated IP List): 192.168.10.10

Search Domains (Space Separated List): tme-lab.local

k) Provide the NTP servers.

NTP Servers (Space Separated IP List): 192.168.10.120

l) Provide the internal networks information.

You will need to provide the service and application subnet information.

The application overlay network defines the address space used by the application's services running in the Application Services Engine. The services network is an internal network used by the Application Services Engine and its processes. Both of these subnets must be/16.

Note

Service Subnet (not exposed externally) [100.80.0.0/16]: 100.80.0.0/16

App Subnet (not exposed externally) [172.17.0.1/16]: 172.17.0.1/16

Example:

Starting apic-sn setup utility

Setup utility for Application Services Engine with SerialNumber WZP23340A7X and running version 1.1.3d

Use AD anytime to start over Cluster Name: ServiceEngine Master Node? (Y/n): y

Download Config From Peers? (Y/n): n Node Name: ServiceNodel

Admin Password:

Reenter Admin Password: Data Network: IP Address/Mask: 192.168.6.172/24 Gateway: 192.168.6.1 Vlan ID (optional): 410 Management Network: IP Address/Mask: 192.168.9.172/24 Gateway: 192.168.9.1

(Ex: 192.192.5.101,WZP22451Q1R 192.192.5.103,WZP22451Q5A): 192.168.6.173,CiscoSN02 192.168.6.174,CiscoSN03

DNS:

Providers (Space Separated IP List): 192.168.10.10 Search Domains (Space Separated List): tme-lab.local NTP Servers (Space Separated IP List): 192.168.10.120

Service Subnet (not exposed externally) [100.80.0.0/16]: 100.80.0.0/16 App Subnet (not exposed externally) [172.17.0.1/16]: 172.17.0.1/16

Step 7 Review the configuration

After you enter all the configuration, review it and confirm.

Please review the config: App Subnet: 172.17.0.1/16 Cluster Name: ServiceEngine Cluster Size: 3

(38)

DNS:

Domain Name: dev-infra12.case.local Providers:

- 171.70.168.183 Search Domains: - atomix.local Download Config: false Data Network: Gateway: 192.168.6.1 IP Address/Mask: 192.168.6.172/24 Vlan ID: 410 Management Network: Gateway: 192.168.9.1 IP Address/Mask: 192.168.9.172/24 Master List: - ipAddress: 192.168.6.173 name: CiscoSN02 serialNumber: CiscoSN02 - ipAddress: 192.168.6.174 name: CiscoSN03 serialNumber: CiscoSN03 NTP Servers: - 192.168.10.120

Node Name: ServiceNodel Node Role: Master Node Type: Physical Password: <hidden>

Service Subnet: 100.80.0.0/16

Re-enter config?(y/N) n

Login with rescue-user & issue acidiag health to check cluster status

CentOS Linux 7 (Core)

Kernel 4.14.174stock-1 on an x86_64

ServiceNodel login:

Step 8 Configure the second node.

a) Connect to the second node's console.

You will be prompted to run the first-time setup utility:

[...]

d) Specify that you've already configured the first node.

When configuring second and third node, you can skip some steps by downloading configuration from the first node.

(39)

Download Config From Peers? (Y/n): y

Node Name: ServiceNode2

f) Enter and confirm the password for therescue-useruser.

Admin Password:

Optionally, you can also provide the VLAN ID for the network. For most deployments, you can skip the VLAN

ID parameter.

Data Network:

IP Address/Mask: 192.168.6.173/24

Gateway: 192.168.6.1

Vlan ID (optional):

Management Network:

IP Address/Mask: 192.168.9.173/24

Gateway: 192.168.9.1

Unlike physical appliance deployments where you obtain the serial numbers from CIMC, for Linux KVM you can choose the strings to use. We recommend usingCiscoSN01,CiscoSN02,CiscoSN03for node1, node2, node3. Master List (Space Separated Data Network IP,Serialnumber List)

(Ex: 192.192.5.101,WZP22451Q1R 192.192.5.103,WZP22451Q5A): 192.168.6.172,CiscoSN01 192.168.6.174,CiscoSN03

j) Review the configuration

After you enter all the configuration for second and third node, review it and confirm.

Please review the config

Cluster Name: ServiceEngine Cluster Size: 3 Data Network:

Gateway: 192.168.6.1

IP Address/Mask: 192.168.6.173/24 Vlan ID: 410

Download Config: true Management Network:

Gateway: 192.168.9.1

IP Address/Mask: 192.168.9.173/24 Master List:

ipAddress: 192.168.6.172 serialNumber: CiscoSN01 ipAddress: 192.168.6.174 serialNumber: CiscoSN03 Node Name: ServiceNode2

Node Role: Master Node

(40)

Type: Physical Password: <hidden> Re-enter config?(y/N): n

Step 9 Repeat the previous step to configure the third node. Step 10 Verify that the cluster is healthy.

(41)

C H A P T E R

6

Deploying in Amazon Web Services

•Deploying the Cisco Application Services Engine in AWS, on page 39

Prerequisites and Guidelines

In addition, the following apply when deploying in the Amazon Web Services (AWS): • You must have appropriate access privileges for your AWS account.

You must be able to launch multiple instances of Elastic Compute Cloud (m5.2xlarge) to host the

Application Services Engine cluster. • At least 6 AWS Elastic IP addresses.

A typical Application Services Engine deployment in AWS requires 6 AWS Elastic IP addresses as shown in the following figure:

Figure 5: Elastic IPs

• Detailed information about AWS configuration is outside the scope of this document, but in short, to create a VPC:

(42)

• In your AWS console, navigate to Computer > EC2.

• In the EC2 Dashboard, click Network & Security > Elastic IPs and note how many Elastic IPs are already being used.

• In the EC2 Dashboard, click Limits and note the maximum number of EC2-VPC Elastic IPs allowed.

Subtract the number of IPs already being used from the limit to get. Then if necessary, click Request

limit increase to request additional Elastic IPs.

• You must create a VPC (Virtual Private Cloud).

A VPC is an isolated portion of the AWS cloud for AWS objects, such as Amazon EC2 instances. Detailed information about AWS configuration is outside the scope of this document, but in short, to create a VPC:

• In your AWS console, navigate to Networking & Content Delivery Tools > VPC.

• In the VPC Dashboard, click Your VPCs and choose Create VPC. Then provide the Name Tag and IPv4 CIDR block.

The CIDR block is a range of IPv4 addresses for your VPC and must be in the/16to/28range.

• You must create an Internet Gateway and attach it to the VPC.

Internet Gateway is a virtual router that allows a VPC to connect to the Internet. Detailed information about AWS configuration is outside the scope of this document, but in short, to create an Internet Gateway:

• In the VPC Dashboard, click Internet Gateways and choose Create internet gateway. Then provide the Name Tag.

• In the Internet Gateways screen, select the Internet Gateway you created, then choose Actions >

Attach to VPC. Finally, from the Available VPCs dropdown, select the VPC you created and click Attach internet gateway.

• You must create a routes table.

Routes table is used for connecting the subnets within your VPC and Internet Gateway to your Application Services Engine cluster. Detailed information about AWS configuration is outside the scope of this document, but in short, to create a routes table:

• In the VPC Dashboard, click Route Tables, choose the Routes tab, and click Edit routes. • In the Edit routes screen, click Add route and create a0.0.0.0/0destination. From the Target

dropdown, selectInternet Gatewayand choose the gateway you created. Finally, click Save routes.

• You must also create a key pair.

A key pair consists of a private key and a public key, which are used as security credentials to verify your identity when connecting to an EC2 instance. To create a key pair:

• Navigate to All services > Compute > EC2.

• In the EC2 Dashboard, click Network & Security > Key Pairs. Then click Create Key Pairs. • Provide a name for your key pair, select the pem file format, and click Create key pair.

(43)

This will download the.pemprivate key file to your system. Move the file to a safe location, you

will need to use it the first time you log in to an EC2 instance's console.

By default only PEM-based login is enabled for each node. If you'd like to be able to SSH into the nodes using a password, you will need to explicitly enable password-based logins. You can do that by logging into each node separately using the PEM file the first time, then executing the following command:

# acidiag login prompt enable

Deploying the Cisco Application Services Engine in AWS

This section describes how to deploy Cisco Application Services Engine cluster in Amazon Web Services (AWS).

Step 1 Subscribe to Cisco Application Services Engine product in AWS Marketplace. a) Log into your AWS account and navigate to the AWS Management Console

The Management Console is available athttps://console.aws.amazon.com/. b) Navigate to Services > AWS Marketplace Subscriptions.

c) Click Manage Subscriptions. d) Click Discover products.

e) Search for Cisco Application Services Engine and click the result. f) In the product page, click Continue to Subscribe.

g) Click Accept Terms.

It may take a couple of minutes for the subscription to be processed. h) Finally click Continue to Configuration.

Step 2 Select software options and region.

Deploying in Amazon Web Services

(44)

a) From the Delivery Method dropdown, selectCisco Application Services Engine for Cloud.

b) From the Software Version dropdown, select the version you want to deploy. We recommend version 1.1.3d or later.

c) From the Region dropdown, select the regions where the template will be deployed. This must be the same region where you created your VPC.

d) Click the Continue to Launch.

The product page appears, which shows a summary of your configuration and enables you to launch the cloud formation template.

Step 3 From the Choose Action, selectLaunch CloudFormationand click Launch.

The Create stack page appears. Step 4 Create stack.

(45)

a) In the Prerequisite - Prepare template area, selectTemplate is ready.

b) In the Specify Template area, selectAmazon S3 URLfor the template source.

The template will be populated automatically. c) Click Next to continue.

The Specify stack details page appears.

Step 5 Specify stack details.

(46)

a) Provide the Stack name.

b) From the VPC identifier dropdown, select the VPC you created. For example,vpc-038f83026b6a48e98(10.176.176.0/24).

c) In the SE cluster Subnet block, provide the VPC subnet CIDR block.

Choose the subnet from the VPC CIDR that you defined. You can provide a smaller subnet or use the whole CIDR. For example,10.176.176.0/24.

d) From the Availability Zones dropdown, select one or more available zones.

We recommend you choose 3 availability zones. For regions that support only 2 availability zones, 2nd and 3rd nodes of the cluster will launch in the second availability zone.

e) From the Number of Availability Zones dropdown, select the number of zones you added in the previous substep. Ensure that the number matches the number of availability zones you selected in the previous substep.

(47)

a) Enable Data Interface EIP support.

This field enables external connectivity for the node. External connectivity is required for communication with Cisco ACI fabrics outside AWS.

b) From the Instance type, selectm5.2xlarge

c) Provide the Cluster name.

The cluster name must be the same across all nodes you deploy. d) Provide the Host name.

The host name must be unique for each node. e) Provide the NTP servers information. f) Provide the Name servers information.

g) (Optional) Provide the DNS search domains list. h) Provide the Application IP subnet.

The application overlay network defines the address space used by the application's services running in the Application Services Engine. This must be a/16subnet.

For example,10.101.0.0/16.

(48)

Note

i) Provide the Service IP subnet.

The services network is an internal network used by the Application Services Engine and its processes. This must be a/16subnet.

For example,10.102.0.0/16.

Finally, provide the login and access information.

a) In the Password fields, provide the password.

for the GUI'sadminuser.

b) From the SSH key pair dropdown, select the key pair you created.

c) In the Access control field, provide the external network allowed to access the cluster. For example,0.0.0.0/0to be able to access the cluster from anywhere.

d) Click Next to continue.

Step 6 In the Advanced options screen, simply click Next.

Step 7 In the Review screen, verify template configuration and click Create stack. Step 8 Wait for the instance deployment to complete, then start the instance.

You can view the status of the instance deployment in the CloudFormation page, for exampleCREATE_IN_PROGRESS.

You can click the refresh button in the top right corner of the page to update the status. When the status changes toCREATE_COMPLETE, you can proceed to the next step.

(49)

Step 9 Verify that the cluster is healthy.

After all three nodes' status isCREATE_COMPLETE, proceed with the following substeps to verify cluster health.

a) Verify that the AWS EC2 instances are up and running.

Navigate to Services > EC2. Then confirm that the Status Checks tab displays2/2checks.

b) Login in to one of the nodes.

You will need to use the private key.pemfile you downloaded when creating a key pair in the following command: $ ssh -i <pem-file-name>.pem rescue-user@<node-ip-address>

c) Verify that the cluster is up and running.

d) Log in to the Application Services Engine GUI.

(50)

When you first log in, you will be prompted to change the password.

Step 10 (Optional) Enable password-based login.

By default only PEM-based login is enabled for each node. If you'd like to be able to SSH into the nodes using a password, you will need to explicitly enable password-based logins. You can do that by logging into each node separately using the PEM file the first time, then executing the following command: