Set up the HIVE_METASTORE

In document Pivotal HD Enterprise (Page 36-40)

Install Hive metastore using:

$ sudo yum install postgresql-jdbc

$ sudo rpm -ivh working_dir/hive/rpm/hive-<PHD_HIVE_VERSION>-nn.noarch.rpm

2. Open the /etc/gphd/hive/conf/hive-site.xml and change it to following: <configuration> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>hive</value> </property> <property> <name>hive.metastore.uris</name> <value>thrift://<CHANGE_TO HIVE_METASTORE_ADDRESS{_}>:9083</value> </property> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:postgresql://<CHANGE_TO_HIVE_METASTORE_ADDRESS>:10432/metastore</value> </property> <property> <name>hive.metastore.warehouse.dir</name> <value>/hive/gphd/warehouse</value> </property> <property> <name>hive.hwi.war.file</name> <value>/usr/lib/gphd/hive/lib/hive-hwi-0.9.1-gphd-2.0.1.0.war</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>org.postgresql.Driver</value> </property> <property> <name>datanucleus.autoCreateSchema</name> <value>false</value> </property> <property> <name>hive.metastore.local</name> <value>false</value> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>hive</value> </property> <property> <name>hive.metastore.execute.setugi</name> <value>true</value> </property> </configuration>

: Replace in above file.

3.

4.

5.

6.

7.

Create file /etc/gphd/hive/conf/hive-env.sh and add the following:

export HADOOP_HOME="/usr/lib/gphd/hadoop" export HADOOP_CONF_DIR="/etc/gphd/hadoop/conf"

export HADOOP_MAPRED_HOME="/usr/lib/gphd/hadoop-mapreduce" export HIVE_CONF_DIR="/etc/gphd/hive/conf"

Make it executable using:

chmod +x /etc/gphd/hive/conf/hive-env.sh

Edit file /etc/gphd/hadoop/conf/hadoop-env.sh and add the following before export : HADOOP_CLASSPATH export HIVELIB_HOME=$GPHD_HOME/hive/lib export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:\ $HIVELIB_HOME/hive-service-*.jar:\ $HIVELIB_HOME/libthrift-0.7.0.jar:\ $HIVELIB_HOME/hive-metastore-*.jar:\ $HIVELIB_HOME/libfb303-0.7.0.jar:\ $HIVELIB_HOME/hive-common-*.jar:\ $HIVELIB_HOME/hive-exec-*.jar:\ $HIVELIB_HOME/postgresql-jdbc.jar

Link postgresql jar:

$ ln -s /usr/share/java/postgresql-jdbc.jar /usr/lib/gphd/hive/lib/postgresql-jdbc.jar

Create the schema:

$ sudo -u postgres psql -U hive -d metastore -p 10432 -f

/usr/lib/gphd/hive-0.9.1_gphd_2_0_2_0/scripts/metastore/upgrade/postgres/hive-schema-0.9.0.postgres.sql

Start the hive-metastore:

$ service hive-metastore start

:Note: MySQL is no longer supported. Please migrate from MySQL to PostgreSQL.

1.7.5 Hive Client Setup

$ sudo rpm -ivh working_dir/hive/rpm/hive-<PHD_HIVE_VERSION>-nn.noarch.rpm

1.7.6 Hive Thrift Server Setup

[OPTIONAL]

Install the Hive core package and Hive thrift daemon package to provide Hive service through thrift.

$ sudo rpm -ivh working_dir/hive/rpm/hive-<PHD_HIVE_VERSION>-nn.noarch.rpm

$ sudo rpm -ivh working_dir/hive/rpm/hive-server-<PHD_HIVE_VERSION>-nn.noarch.rpm

1.7.7 Hive Server2 Setup

[OPTIONAL]

Install the Hive core package and Hive thrift daemon package to provide Hive service through thrift.

$ sudo rpm -ivh working_dir/hive/rpm/hive-<PHD_HIVE_VERSION>-nn.noarch.rpm

$ sudo rpm -ivh working_dir/hive/rpm/hive-server2-<PHD_HIVE_VERSION>-nn.noarch.rpm

1.7.8 Hive MetaStore Server Setup

[OPTIONAL]

Install the Hive core package and Hive Metastore daemon package to provide Hive metadata information through centralized Metastore service:

$ sudo rpm -ivh working_dir/hive/rpm/hive-<PHD_HIVE_VERSION>-nn.noarch.rpm

$ sudo rpm -ivh working_dir/hive/rpm/hive-metastore-<PHD_HIVE_VERSION>-nn.noarch.rpm

1.7.9 Hive Configuration

The configuration files for Hive are located here: /etc/gphd/hive/conf/

This is the default configuration for quick reference and modification. It is a symbolic link to /etc/gphd/hive-version/conf

You can make modifications to this configuration template or create your own. If you want to use a different configuration folder, adjust the symbolic link /etc/gphd/hive/conf to point to the folder you want to utilize at runtime.

1. 2.

3.

4.

5.

1.7.10 Hive Post-installation Configuration

Login to one of the cluster nodes as root. Create the hive.warehouse.dir

$ sudo -u hdfs hadoop fs -mkdir -p /hive/gphd/warehouse

Set permissions for the hive.warehouse.dir

$ sudo -u hdfs hadoop fs -chmod 775 /hive/gphd/warehouse

Set the ownership for the hive.warehouse.dir

$ sudo -u hdfs hadoop fs -chown hadoop:hadoop /hive/gphd/warehouse

Add hive user to hadoop group if not already present using

$ usermod -G hadoop hive

1.7.11 Hive Usage

In document Pivotal HD Enterprise (Page 36-40)