6 USS Documentation Main
6.2 Getting Started
USS (v0.5.0) is part of Pivotal HD 1.1. USS is distributed as a set of three RPM packages that are part of the tarball. You can download the PHDTools tarball from the
PHDTools-version EMC Download Center
(paid version) or the Pivotal HD product page (Community Edition). Architecturally, USS has the following four components:
6.2.1 USS Namenode (Server)
The USS Namenode is a stateless server-side component. It is implemented as a Hadoop RPC server, that is capable of interacting with a catalog system to delegate file system calls to appropriate underlying file systems. It is expected to run on a dedicated server. In this alpha version, this runs as a single server, with no HA and failover. It is designed to be the central place where all file system calls converge, before they are delegated to a multitude of underlying file systems. It abstracts datasets on underlying file systems as mount-points. It stores the mapping between mount-points and the directories that they point to in the USS catalog. The USS Namenode does not store any state, its state is contained within the USS Catalog.
6.2.2 USS Catalog
The USS Catalog is a metadata store for the USS Namenode. In this version, the USS catalog is contained in a postgres database. USS ships scripts that can help users manage the USS Catalog. The USS RPMs do not install the postgres database to be used as the USS catalog. It is expected that users have a postgres database ready. However, if users use Pivotal Command Center to deploy USS, it can deploy a postgres database as the USS Catalog.
6.2.3 USS Agent
The USS Agent is the FileSystem driver that is expected to be present on all Pivotal HD cluster nodes. This is the USS client-side library that accepts a FileSystem request (containing a mount-point), maps the request to the appropriate File System and path, by making an RPC request to the USS Namenode.
6.2.4 USS Client
The USS Client hosts allow administrators to update the USS Catalog using the USS CLI. Pivotal HD Manager installs this role on all Client Hosts in your cluster configuration. Users can also use the USS Namenode or the USS Catalog host as the client node.
6.2.5 USS Gateway
USS uses a ‘Gateway’ to enable access to protocol-incompatible filesystems. It is a host that acts as a bridge between a PHD cluster and a protocol-incompatible cluster by delegating the filesystem request to the target remote filesystem. This host is expected to have access to the wire-incompatible cluster that you wish to access using USS. The Pivotal HD Manager does not install this component, since it requires some information about the wire-incompatible cluster. More information about setting up this component can be found later in this document in the Setting up USS Gateway section.
USS Directory Contents
The contents of USS 0.5.0 within the PHDTools-version tarball are located here:
$ tree PHDTools-1.1.0.0-90/uss/ PHDTools-1.1.0.0-90/uss/ rpm uss-0.5.0-40.noarch.rpm uss-0.5.0-40.noarch.rpm.md5 uss_catalog-0.5.0-40.noarch.rpm uss_catalog-0.5.0-40.noarch.rpm.md5 uss_namenode-0.5.0-40.noarch.rpm uss_namenode-0.5.0-40.noarch.rpm.md5 1 directory, 6 files
RPM Package Name Description Dependencies Hosts to install
on uss-0.5.0-40.noarch.rpm This is the base
USS package. It installs the USS library and Pivotal HD hadoop package (version 2.0.2-alpha-gphd-2.0.1 and above)
All hosts using USS (agent, namenode, client and catalog hosts)
RPM Package Name Description Dependencies Hosts to install on
required configuration files.
uss_catalog-0.5.0-40.noarch.rpm This package installs the scripts and configuration files
required to interact with and administer the USS Catalog.
(version >= 0.5.0)
uss USS Client
hosts. Installed on the USS Namenode host by default as the uss_namenode RPM package has a dependency on it.
uss_namenode-0.5.0-40.noarch.rpm This is the USS Namenode package.
It installs the USS namenode service daemon and its runtime configuration. uss_catalog package (version >= 0.5.0). USS Namenode host
6.2.6 Package Contents
uss-0.5.0-40.noarch.rpm
Files/Directories Description/usr/lib/gphd/uss/uss-0.5.0.jar The USS library
/usr/lib/gphd/uss/uss-javadoc-0.5.0.jar The USS javadocs
and
/etc/gphd/uss/conf/commons-logging.properties /etc/gphd/uss/log4j.properties
USS log configuration files
/etc/gphd/uss/conf/uss-client-site.xml USS client side configuration file
Files/Directories Description
/etc/gphd/uss/conf/uss-catalog.xml USS Catalog configuration file
/etc/gphd/uss/conf/uss-env.sh USS Client side environment
setup script
/etc/gphd/uss/conf/uss.properties USS Catalog database
connection properties
/etc/gphd/uss/conf/uss-fs-metadata.xml Sample Filesystem
Registration configuration file for USS. This is a sample of the file that can be used to register a filesystem with USS using the USS CLI
/etc/default/uss USS runtime configuration
script. Exports environment variables that USS processes use.
/usr/lib/gphd/uss/libexec Contains USS utility scripts.
/usr/lib/gphd/uss/libexec/uss-remote-proxy-launcher.sh Script used to launch the USS Remote Proxy Server for accessing
protocol-incompatible Filesystems (like Hadoop 1.x based HDFS)
/usr/lib/gphd/uss/libexec/uss-utils Unix utilities for USS processes
/var/log/gphd/uss USS log directory. Currently
contains log files for the USS Remote Proxy Process
/var/run/gphd/uss Directory for storing runtime
files like pid files, lock files for USS processes
/usr/lib/gphd/uss/docs The USS documentation
directory. Contains README, Changelog, Notice and License files
Files/Directories Description
/usr/bin/uss This is the USS CLI to help admins/users to interact with the USS catalog.
/usr/lib/gphd/uss/migration Contains utilities for migrating the USS catalog database.
/usr/lib/gphd/uss/lib Contains the java libraries required to access the USS catalog.
uss_namenode-0.5.0-40.noarch.rpm
File Description
/usr/sbin/uss-namenode-daemon The USS Namenode daemon script.
/etc/rc.d/init.d/uss-namenode USS Namenode service script.
/var/log/gphd/uss-namenode Log directory for USS Namenode