Chapter 5 Preparing to Build the System
6.5 Installing to a Collaboration Server
6.5.1 Installing the Collaboration Server Functionality
1.
Creating bdpp.conf2.
Installing the Collaboration Server Functionality3.
Applying Software Updates6.5.1.1 Creating bdpp.conf
To install the collaboration server functionality, configure in bdpp.conf the values that were designed in advance.
See
6.5.1.2 Installing the Collaboration Server Functionality
Follow the procedures below to install the collaboration server functionality.
1.
Set the "BDPP_CONF_PATH" environment variable.In the "BDPP_CONF_PATH" environment variable, set the directory where bdpp.conf is stored.
Example
If bdpp.conf is stored in /home/data/collaborate:
# export BDPP_CONF_PATH=/home/data/collaborate <Enter>
2.
Mount the DVD-ROM.Place the DVD-ROM in the DVD drive, then execute the command below to mount the DVD-ROM. If the DVD-ROM is automatically mounted by the automatic mount daemon (autofs), installer startup will fail because "noexec" is set in the mount options.
# mount /dev/hdc dvdRomMountPoint <Enter>
3.
Start the installer.Execute bdpp_inst to start the installer.
# dvdRomMountPoint/bdpp_inst <Enter>
4.
Select "4. Collaboration Server installation".================================================================================ Interstage Big Data Parallel Processing Server
V1.0.1
All Rights Reserved, Copyright(c) FUJITSU LIMITED 2012
================================================================================ << Menu >>
1. Master Server installation 2. Slave Server installation 3. Development Server installation 4. Collaboration Server installation
================================================================================ Please select number to start specified Server installation [?,q]
=> 4 <Enter>
Note
If installation fails, refer to the messages output during the installation operation and/or installation log file (/var/tmp/bdpp_install.log) to diagnose the failure. Then, remove the cause of the failure and perform installation again.
6.5.1.3 Applying Software Updates
Apply the updates for this product and the updates for software included with this product. Refer to the note in "Chapter 6 Installation" for details.
6.5.2 DFS Setup
The DFS setup sequence is as follows:
1.
Mount Point Creation and fstab Settings2.
Register the hadoop Group and mapred User3.
Mounting6.5.2.1 Mount Point Creation and fstab Settings
Create the mount point and add the DFS entry to /etc/fstab.
Creating the mount point
Create the mount point for mounting the disk partitions on the storage system used as the DFS. Make the mount point the same as that specified at the master server.
Example
Create the mount point "pdfs" under "/mnt".
# mkdir /mnt/pdfs <Enter>
fstab settings
At "/etc/fstab", define the mount points created above and the logical file system name.
The logical file system name is used to identify the DFS file system. Use the name defined when generating the configuration information of the file system on the master server.
Example
This example shows the mount point "/mnt/pdfs" and the logical file system name "pdfs1" defined at "/etc/fstab".
LABEL=/ / ext3 defaults 1 1 LABEL=/home /home ext3 defaults 1 2 LABEL=/boot /boot ext3 defaults 1 2 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 LABEL=SWAP-sda3 swap swap defaults 0 0 pdfs1 /mnt/pdfs pdfs _netdev 0 0
See
Refer to "6.2.2.1 Creating the Mount Point and Configuring fstab Settings" used when installing slave servers for information about the parameters specified in the fields of the entries added to /etc/fstab.
6.5.2.2 Registering the hadoop Group and mapred User
Register the hadoop group and mapred user.
The hadoop group GID and the mapred user UID must be the same values as the master server hadoop group GID and mapred user UID. Check the master server "/etc/passwd" file and register the same values.
Example
1.
Check the hadoop group amongst the groups registered at the master server.# cat /etc/group <Enter> ... omitted ...
hadoop:x:123:hbase hbase:x:1503: bdppgroup:x:1500:
2.
Check the mapred user amongst the users registered at the master server.# cat /etc/passwd <Enter> ... omitted ... bdppuser1:x:1500:1500::/home/bdppuser1:/bin/bash bdppuser2:x:1501:1500::/home/bdppuser2:/bin/bash mapred:x:202:123:Hadoop MapReduce:/tmp:/bin/bash hdfs:x:201:123:Hadoop HDFS:/tmp:/bin/bash hbase:x:203:1503::/tmp:/bin/bash
3.
Register the hadoop group.# groupadd -g 123 hadoop <Enter>
4.
Register the mapred user.# useradd -g hadoop -u 202 -c "Hadoop MapReduce" -d /tmp -s /bin/bash mapred <Enter>
Note
If the GID and UID need to be changed to different values because the GID and UID are already registered at the collaboration server, the hadoop group GID and the mapred user UID registered at the master server, slave servers and development server must all be changed to the same values.
6.5.2.3 Mounting
Distribute the DFS configuration information file generated on the master server to the collaboration server to mount the DFS file system.
1.
Copy the DFS configuration information file generated in "6.1.3.11 Generating the DFS File System Configuration Information" for the master server to /etc/pdfs.# scp -p root@master1:dfsConfigurationFileDirectory/client.conf.pdfs1 /etc/pdfs <Enter>
2.
Mount the DFS file system.Example
# mount pdfs1 <Enter>
Note
-
The DFS configuration information file must be distributed to the collaboration server before the DFS file system is mounted.-
Ensure that the DFS mount sequence is: master servers, slave servers, development servers and collaboration servers. If the slave servers, development servers or collaboration servers are mounted first, then mount will fail, because the master server (MDS) does not exist.See
Refer to the "mount.pdfs" under "Appendix A Command Reference" in the "Primesoft Distributed File System for Hadoop V1 User's Guide" for details of mounting.