Introduction . . . 81 Migrating the System Catalog . . . 81 Export an Existing SYSCAT . . . 82 Migrating Vault Data . . . 83
Migrating the System Catalog and Vault Data
Introduction
This topic describes how a user can use PostgreSQL and SAS tools to migrate data from one SAS Data Agent instance to another. This can be used to move the deployment to a new system, or to promote a test system to production status, and so on.
There are two primary data stores that must be migrated:
• The system catalog database for SAS Data Agent. Otherwise, known as ‘SYSCAT’.
• The Vault that contains the data for your database credentials.
Migrating the System Catalog
The SAS Data Agent system catalog (SYSCAT) in SAS Viya 3.5 is stored in a Postgres database. In a single tenant deployment, this database is named
DA_SYSCAT_instance where the instance is the name of the SAS Data Agent if you have more than one instance on your system. For a single machine, the instance name is
“default”, so the database name is DA_SYSCAT_default. For multiple machines, default is appended with a machine name, such as
DA_SYSCAT_instance‑machine_name. CAUTION:
Stop the Data Agent server before proceeding.
Before migrating SYSCAT data from an existing Data Agent, it is essential to stop the Data Agent server process so that no further data is written to the data store. Here is an example command that stops the default data agent on the current node:
sudo systectl stop sas-viya-dagentsrv-default
Migrating the SYSCAT database can be done using the Postgres database dump tool, pg_dump. To use this tool, you will need the credentials for communicating with the PostgreSQL server, which are stored in the local Consul system.
The sections below describe the export and import steps.
Export an Existing SYSCAT
Here is an example script that you can run to create a new dump file of the existing database.
# Get the Consul token first
export CONSUL_HTTP_TOKEN=$(sudo cat \
/opt/sas/viya/config/etc/SASSecurityCertificateFramework/\
tokens/consul/default/client.token)
# Use Consul to get PG credentials KEY="config/application/postgres"
export PGUSER=$(/opt/sas/viya/home/bin/consul kv get $KEY/username) export PGPASSWORD=$(/opt/sas/viya/home/bin/consul kv get $KEY/password)
# Connect to Postgres and export SYSCAT
CMD="/opt/sas/viya/home/postgresql11/bin/pg_dump $ZIP -f $OUTFILE --create \ --clean --insert -h localhost $SCHEMA $DBNAME"
$CMD
This script generates an output file containing a representation of the SYSCAT database, which is written to the current working directory. This file must then be transported to the destination system (Secure Copy Protocol (SCP) over a network, for example) so that the file is available on the new system where the data is to be restored.
If you are migrating data to Cloud Data Exchange version 2.5 or later, you must edit the script to specify the correct schema name for your system when you back up the data.
You can verify the schema name by looking in the file, /opt/sas/viya/
config/etc/dagetsrv/default/da_entities.dtd and look for the line defining cfg.pg.syscat.schema. After creating the dump file, determine the schema name of the target CDE system to which the data is being migrated. Edit the dump file and update the schema name to match the correct schema name of the “target”
system.
Note: The dump file is a text file containing SQL statements.
Here is a sample script that is used to invoke the psql tool supplied by Postgres, using credentials fetched from the local Consul system.
# Get the Consul token
export CONSUL_HTTP_TOKEN=$(sudo cat /opt/sas/viya/config/etc/ \ SASSecurityCertificateFramework/tokens/consul/default/client.token)
# Use Consul to get PG credentials KEY="config/application/postgres"
export PGUSER=$(/opt/sas/viya/home/bin/consul kv get $KEY/username) export PGPASSWORD=$(/opt/sas/viya/home/bin/consul kv get $KEY/password)
# Connect to Postgres
/opt/sas/viya/home/postgresql11/bin/psql -d postgres -h localhost -U $PGUSER
Migrating Vault Data
Vault data contains encrypted information about database credentials that are stored locally. In a data agent configuration, it can be desirable for a remote user to access local data through SAS Data Agent, but they should never be allowed access to the credentials that are used to access data. That information is used only on the local system by SAS Data Agent.
When migrating to a new system, you must migrate Vault contents so that you avoid losing the Vault data associated with credentials managed in SAS Viya. This is implemented using a special command in the Data Agent CLI. This command is run only as the installation user for the SAS Data Agent deployment.
Here is an example of the command that performs the migration. Specify the destination endpoint, server, user name, and password of the deployment where you are migrating the Vault credentials.
Note: This example shows the “\” continuation character; this command is executed as a single shell command.
/opt/sas/viya/home/bin/sas-admin dagentsrv security credentials migrate \ --domain mydomain \
--destination-endpoint https://test-cloud.openstack.abc.com:443 \ --destination-server dagentsrv-shared-default-test-onprem \ --destination-username sasadmin \
--destination-password mypasswd
The domains are created on the destination system if they do not exist. The new domains are labeled as “Migrated domain.” The migration fails if a domain exists of the same name that is not a “shadow” domain.
The command reads the domain and credential information from the local SAS Viya instance, along with the contents of the local Vault associated with those credentials.
This information is then used to create the domain on the destination system if it does not exist. This information is also used to create the credential including the password that is stored in the destination Vault.
The migration of credentials does not affect any domains or credentials already in the destination system unless they match the name and type of the shadow domain being migrated.
Here are key points to consider when migrating domains:
• The destination deployment must be running, including the target SAS Data Agent receiving the secured credentials.
• If you do not specify the ‑‑destination‑password or
‑‑destination‑username values, you are prompted to enter those from your console window.
• If you specify a domain that is not a “shadow” domain type, the migration fails. If you do not specify a specific domain, then all “shadow” domains are migrated.
• You must run the migration as a deployment user. Otherwise, the migration fails.
• You must have the user name and password credentials of a SAS Data Agent Administrator for use on the destination system.
You do not need to migrate Vault credentials for a co-located instance of SAS Data Agent. Vault credentials are used only for password domain type “shadow” and are a mechanism to enable the remote Data Agent to store password credentials only on the remote system. Migration of a remote instance of SAS Data Agent involves migrating both the Vault and the SYSCAT data. By contrast, migration of a co-located instance of SAS Data Agent involves migrating only SYSCAT since domains created in the co-located instance should be created as type “password” which requires no additional migration steps.