UNIX Shell Scripting
22-24 Jul 2013
by
Topics
About UNIX
UNIX OS
Files, Process and Directory
Shell Anatomy
List Commands
Directory Commands
About UNIX
UNIX is a powerful, portable, multi-tasking and multi-user computer operating system
Unix is more flexible and can be installed on many different types of machines, including main-frame computers, supercomputers and micro-computers.
Unix is more stable and does not go down as often as Windows does, therefore requires less administration and maintenance.
Unix has greater built-in security and permissions
Software upgrade do not require Hardware upgrade/pre-requisites
Sun Solaris, HP-UX, GNU/Linux, and MacOS X
What is Unix
Unix Advantage
UNIX OS
UNIX operating system is made up of three parts; the kernel, the shell and the programs.
Kernel
The kernel of UNIX is the hub of the operating system: it allocates time and memory to programs and handles the files store and communications in response to system calls.
Shell
The shell acts as an interface between the user and the kernel
The shell is a command line interpreter(CLI)
Key Flavor's : Bourne Shell, Bash Shell and Korn Shell
Program
The commands are themselves programs
Multiple commands can be combined as a script
Unix Files, Process and Directory
Everything in UNIX is either a file or a process.
Process
A process is an executing program identified by a unique PID (process identifier).
File
A file is a collection of data. They are created by users using text editors, running compilers etc.
Files and processes
The file-system is arranged in a hierarchical structure, like an inverted tree. The top of the
hierarchy is traditionally called root (written as a slash / )
Anatomy and Commands
List File/Directory : To find out what is in your home directory, type % ls
% ls -all
Basic Commands:
Commands (cont.…)
Change Directory
% cd directory
Make New Directory
% mkdir directory
Remove Directory
%rmdir directory
Print Working (Show Current) Directory
% pwd
Directory Commands:
Seeing Permissions % ls -l filename Changing Permissions % chmod nnn filename Where n, a digit from 0 to 7Commands (cont.…)
Display File Contents
% cat filename File Copy % cp <source> <destination> Move (Rename) % mv <oldname> <newname> Remove (Delete) % rm <filename>
File Manipulation Commands:
Creating a Variable
% set var
Assigning a Value
% set var = value
Expressing a Value
% $var
Displaying a Value
% echo $var
Commands (cont.…)
Unix treats every device as a file
In effect, terminal and keyboard can be treated as files Keyboard is an input file
Terminal monitor is an output file Unix treats every device as a file
Every program has three default files – stdin, stdout, and stderr These files are respectively numbered as 0, 1, and 2
command 1>&2 > tmp.txt
I/O Commands:
Search Commands:
Search a string in a file
Commands (cont.…)
The flow of control within Shell scripts is done through constructs
If..Then..Elif..Else.. if <> then elfi <> then else fi Example #!/bin/sh if [ "$1" = "1" ] then
echo "Parameter value is 1" elif [ "$1" = "2" ]
then
echo "Parameter value is 2" elif [ "$1" = "3" ]
then
echo "Parameter value is 3" else
echo "Parameter value is $1" fi
Control Constructs:
Do..while.. while list do list done Example: #!/bin/sh count=$1 while [ $count -gt 0 ] do count=$(expr $count -1) doneCommands (cont.…)
For
for variable in word ... do list
done
Example:
#!/bin/sh
fruitlist="Apple Pear Tomato Peach Grape" for fruit in $fruitlist
do
if [ "$fruit" = "Tomato" ] || [ "$fruit" = "Peach" ] then
echo "I like ${fruit}es" else
echo "I like ${fruit}s" fi done
Control Constructs:
Case case word in pattern) list ;; ... esac Example: !#/bin/sh case $1 in1) echo 'First Choice';; 2) echo 'Second Choice';; *) echo 'Other Choice';; esac
Commands (cont.…)
The syntax of an SH function is defined as follows:
name() { commands } Example: printlog() { echo "$1" }
Note: The function has to be called to get executed
Functions:
Shell assignment #!/bin/sh Variable declaration var1=„Hello World‟; Control statements if [ "$1" = " Hello World " ] then echo “$1" fi Exit status exit 0Anatomy:
Topics
vi Editor
FTP
SSH (including SCP and SFTP)
Unix Mailer - mailx
Registering a Host Program
Reading a DB value by launching SQL Plus
Vi Editor
vi is a screen-oriented text editor originally created for the Unix operating
system
Inbound/Outbound File Transfer - FTP
File Transfer Protocol (FTP) is a network protocol used to copy a file from one computer
to another over the Internet or LAN
Basic Commands
Open
– connect to remote server FTP port
open <remote_machine(Hostname)/IP>
lcd
– Local current directory in the remote
server
pwd
– present working directory in the
local server
FTP (cont…)
put
put <filename>
mput <filename*.*>
get
get <filename>
mget <filename*.*>
Transfer Mode
ascii For text file like data files and Shell Scripts
bin For executable like .rdf, .fmx etc.
SSH
SSH, which is an acronym for Secure SHell, was designed and created to
provide the best security when accessing another computer remotely.
Not only does it encrypt the session, it also provides better authentication
facilities, as well as features like secure file transfer, X session forwarding,
port forwarding.
vs
SSH:
SSH (cont.…)
Secure Copy:
SCP is a means of securely transferring computer files between a local host and a
remote host or between two remote hosts. It is based on the Secure Shell (SSH)
protocol.
Copying file to host:
scp SourceFile user@host:directory/TargetFile
Copying file from host: scp user@host:directory/SourceFile TargetFile
SFTP:
SFTP is a network protocol that provides file access, file transfer, and file management
functionalities.
Key features :
1. Delete remote file
2. Execute a shell in the remote
Unix Mailer
mailx: An utility to send and receive email. SMTP/mail server setting has to be
pre-configured prior to mailx usage
Options:
1.
Mail body
2.
Attach files
3.
Add cc and bcc
Example1:
mailx –s “Test Mail from Unix System”
john@abc.com
Example2:
(echo "${MAIL_BODY}" ";uuencode ${FILE1};uuencode ${FILE2}") | mailx -s
“${MAIL_SUBJECT}" "${EMAIL}"
Unix Capabilities in Oracle Apps Environment
Load data file into table through SQL Loader
Connect to DB and get a value e.g. Alert Distribution Mail
Download/Upload file to remote servers for Inbound/Outbound interface
programs
Execute a DB Procedure/Function
Submit a Concurrent Program
Migrate AOL Objects and WF Objects through FNDLOAD and WF Load
Encrypt/Decrypt sensitive data files
Registering as Host Program
Step 1 : Create a shell script and save in .prog extension Step 2 : Copy the file to $CUSTOM_TOP/bin
Step 3 : Create as link as follows
ln –s $FND_TOP/bin/fndcpesr <filename without extension>
Why fndcpesr? - Otherwise we need to read the concurrent program parameters using cut commands
Step 4 : Create a executable with execution type as Host
Step 5 : Create a concurrent program and attach the executable and create parameters if required
System parameters:
$1 apps_username_pwd [e.g. apps/appspwd]
$2 userid
$3 username
$4 request_id User parameters:
From $5 to till defined
Registering as Host Program(cont.…)
User parameters: From $5 to till defined
Note: Use SHIFT to read vale after $9
Example: APPS_LOGIN=$1 USERID=$2 USERNAME=$3 REQUESTID=$4 USER_PARAM1=$5 USER_PARAM2=$6 USER_PARAM3=$7 USER_PARAM4=$8 USER_PARAM5=$9 SHIFT USER_PARAM6=$9 SHIFT USER_PARAM7=$9
Set Program completion status:
At the end of the script use exit command
For Completed - Success: exit 0
For Completed - Error: exit2
For Completed - Warning: ????????????
Launching SQL Plus
Connect to DB and read a value from the table:
Example:
# Get the Mail subject
MSG_SUBJECT=`sqlplus -s $APPSLOGIN<<!
set heading off
set verify off
set wrap off
set feedback off
set echo off
SELECT REPLACE(REPLACE(message_text,CHR(13),NULL),CHR(10),NULL)
FROM FND_NEW_MESSAGES
WHERE MESSAGE_NAME = „XXX_CST_REFRESH'; exit;
!`
Execute DB Procedure
sqlplus -s $DB_CONNECT << EOF > ${TEMPFILE0} SET ECHO OFF
SET VERIFY OFF SET LINESIZE 10000
SET SERVEROUTPUT ON FORMAT WRAP DECLARE
lv_file_archive_flag VARCHAR2(5); BEGIN
dbms_output.enable(100000);
xxx_fx_dly_exch_rate_pkg.submit_daily_req_set( p_user_id => '$APPL_USERID',
p_file_archive_flag => lv_file_archive_flag ); dbms_output.put_line (' File Archive Flag :'|| lv_file_archive_flag);
EXCEPTION when others then
dbms_output.put_line (' Exception raised while calling xxx_bloomberg_exch_rate_pkg.submit_daily_req_set, Error Message : '||SQLERRM); END; / EXIT; EOF
Execute DB Procedure:
Topics
Data load through SQL Loader
CONCSUB Utility
Password less login through SSH
File Encryption/Decryption
Load data through SQL Loader
Examine the data and build the Control file accordingly and load the data into
the table
Example:
cd $CUSTOM_TOP/data
for file in sample*.dat
do
sqlldr userid=$login control=$CUSTOM_TOP/bin/sv_test.ctl
data=$CUSTOM_TOP/data/$file
mv $CUSTOM_TOP/data/$file $CUSTOM_TOP/archive/$file
done
CONCSUB
Concurrent program can be submitted from Shell script through concsub utility
shipped by Oracle
Example:
cd $FND_PATH
v_concreq=`./CONCSUB $ORA_USER_PASS $APPLN_NAME $RESP_NAME1 $USER_NAME WAIT=Y CONCURRENT SQLAP "APXMCCDF3" $CARD_PROGRAM_ID ${1}`
SSH Password – less Login
SSH Password-less Login:
SSH provides password less login mechanism which is useful in process
automation and batch processing
Step 1:Generate SSH Keys [both public and private keys]
Step 2:Copy the public key to the remote host
Step 3: Try login
File Encryption / Decryption
Sensitive data requires file encryption/decryption to maintain security when it is
transferred through a computer network
Public Key Cryptography
File Cryptography
PGP/GPG
Pretty Good Privacy (PGP) / GNU Privacy Guard is a data encryption and decryption computer program that provides cryptographic privacy and authentication for data communication.
PGP is a CPU licensed software and GPG is GPL
PGP has got product support and GPG do not have support
PGP is recommended for enterprise level encryption/decryption
Encrypt Command:
/opt/pgp/bin/pgp encrypt ${FILE_NAME}.txt output ${FILE_NAME}.pgp overwrite remove --passphrase ${PGPPASSPHRASE} -r ${PGP_RECIPIENT}
Decrypt Command:
/opt/pgp/bin/pgp --decrypt --input ${FILE_NAME} --output ${DECRYPT_FILE_NAME}.dat --overwrite remove --passphrase ${PGPPASSPHRASE}
Host Program – Set Program status to Complete -
Warning
Host Program completion status can be set using exit command.
exit 1 In this case, program status will not set to Complete – Error
Work around:
sqlplus -s $p_usr_pwd <<-EOF set feedback off
declare
l_stat Boolean; l_session_id Number; begin
fnd_global.initialize(session_id => l_session_id, user_id => fnd_global.user_id, resp_id => fnd_global.resp_id , resp_appl_id => fnd_global.resp_appl_id, security_group_id => null
, site_id => null, login_id => null, conc_login_id => null
, prog_appl_id => null, conc_program_id => null, conc_request_id => $p_req_id, conc_priority_request => null); l_stat := fnd_concurrent.set_completion_status('WARNING','Completed in Warning. Review log for details.');
commit; end; / exit; EOF