Configuring USS

The following sections describe the USS configurations in Pivotal HD 1.0.1 and above.

Pivotal HD Manager sets up all these configurations and no user action is necessary.

Enabling USS on your Pivotal HD Cluster

To enable USS, the following properties are set in /etc/gphd/hadoop/conf/core-site.xml.

/etc/gphd/hadoop/conf/core-site.xml

<?xml version="1.0"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration>

<property>

<name>fs.uss.impl</name>

<description>The FileSystem for uss: uris.</description> </property>

</configuration>

Where:

Property Description

fs.uss.impl USS is implemented as a pseudo hadoop file system. USS URI's use the scheme uss. Hadoop compatible file systems set the property fs.<scheme>.impl to the class that implements the file system. The class that implements the USS pseudo file system is

.

com.greenplum.hd.uss.USSProtocolTranslator

6.4.1 USS Namenode Configuration

These are the server-side configurations for USS. The location of this configuration is /etc/gphd/uss/conf/uss-nn-site.xml.

/etc/gphd/uss/conf/uss-nn-site.xml

<?xml version="1.0"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration>

<property>

<name>uss.namenode.address</name>

<value>uss://USS_NAMENODE_HOST:16040</value> <description>Uri for USS Namenode.</description> </property>

<property>

<name>uss.catalog.type</name> <value>db</value>

<description>Type of catalog to use.</description> </property>

<property>

<name>uss.db.name</name>

<value>dbApplicationContext.xml</value>

<description>Settings for postgres catalog.</description> </property>

<property>

<name>uss.remote.proxy.rmi.name</name> <value>RemoteFilesystem</value>

<description>RMI Name for USS Remote Proxy</description> </property>

<property>

<name>uss.remote.proxy.timeout</name> <value>10000</value>

<description>Timeout for USS Remote Proxy</description> </property>

<property>

<name>uss.remote.proxy.port</name> <value>1099</value>

<property>

<name>uss.remote.proxy.launcher.script.path</name>

<value>/usr/lib/gphd/uss/libexec/uss-remote-proxy-launcher.sh</value>

<description>Path to the script to launch the USS Remote Proxy</description> </property>

<property>

<name>uss.remote.proxy.classpath.prefix</name>

<value>/etc/gphd/uss/conf:/usr/lib/gphd/uss/*</value>

<description>USS Classpath which pre-pended to the classpath of the remote proxy process as provided by users while registering filesystems.</description>

</property> </configuration>

Where:

Property Description

uss.namenode.address The fully-qualified URI of the USS Namenode. Specifies the hostname and port on which the USS Namenode listens for requests.

uss.catalog.type Type of catalog to use. Defaults to db. We do not recommend changing this property.

uss.db.name Settings for postgres catalog. Defaults to . We do not

dbApplicationContext.xml

recommend changing this property.

uss.remote.proxy.rmi.name

Name of the RMI endpoint that the Remote Proxy Server on the USS Gateway starts up with. We do not recommend changing this property.

uss.remote.proxy.timeout

Timeout for the USS Remote Proxy startup.

uss.remote.proxy.port

Port that the remote proxy listens to. We do not recommend changing this property.

uss.remote.proxy.launcher.script.path

Path to the script to launch the USS Remote Proxy. We do not recommend changing this property.

uss.remote.proxy.classpath.prefix

USS Classpath which is pre-pended to the classpath of the remote proxy process as provided by users while registering filesystems. We do not recommend changing this property.

These are the client-side configurations for USS.

uss-client-site.xml

<?xml version="1.0"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<property>

<name>fs.uss.name</name>

<value>uss://USS_NAMENODE_HOST:16040</value>

<description>Fully qualified URI of the USS Namenode</description> </property>

<property>

<name>uss.tmp.fs</name>

<value>hdfs://NAMENODE_HOST:9000</value>

<description>Filesystem that hosts the temporary directory for map-reduce jobs.</description>

</property>

</configuration>

Where:

Property Description

fs.uss.name This property specifies the hostname and port of the USS Namenode on the client side. By default, it is set to uss://localhost:16040/ . If the client is a non-USS

Namenode host, then this property can be used to specify the hostname and port of the USS Namenode on the client.

uss.tmp.fs This property indicates the location that USS uses to store temporary data. All

Hadoop-compatible file systems require a location to store temporary. Hadoop uses this location for creating the staging area for map-reduce jobs (.staging directory) as well as to store the .Trash directory. As USS is implemented as a pseudo Hadoop file system, it also requires a location for storing temporary data. However, as there is no storage/file system tied to USS, USS stores temporary data on the file system specified against this parameter. Users could choose to set this to a filesystem of their choice, as long as the file system is compatible with Hadoop (i.e. it has an HDFS implementation available). This file system should be available for USS to store temporary data. It could either be one of the underlying file systems, or a dedicated file system for storing temporary data.

6.4.3 USS Catalog Configuration

The USS Namenode as well as the USS CLI uses the configuration file

to read connection properties for the USS Catalog. If you wish

/etc/gphd/uss/conf/uss.properties

to modify the USS Catalog Database URI, please update the property uss.db.url from this file. The value for this property is of the form:

uss.properties - uss.db.url

uss.db.url=jdbc:postgresql://[USS_CATALOG_DB_HOST]:[USS_CATALOG_DB_PORT]/usscatalog

We recommend that users only make sure that the host and port of the postgres server designated to be the USS Catalog is correct. Do not modify any other property from this file.

See Security for information about USS Secure Configuration.