Overview
The EBMX SFTP shell scripts provide a parameter driven workflow to place files on the EBMX servers
and queue the files for processing. SFTP is the protocol used to transmit and queue the data to EBMX.
SFTP requires that each environment (yours and EBMX) exchange Public Keys before any transmissions
can occur. In order to expedite the setup, please submit a Trouble Ticket requesting SFTP setup to the
TTTS group EBMX. [Please include server name/IP, application name, AMS#, and technical contact(s)].
The EBMX group will work with you during the setup and testing if you are unfamiliar with SFTP.
The shell scripts (Korn & Bourne shell) were developed using Ubuntu Linux and tested on AIX and
Solaris. The EBMX Group provides these scripts as-is, if you find there is an issue or have a question
about functionality, please submit a Trouble Ticket and route to EBMX.
The EBMX SFTP shell script is one of two options for applications to access the functions of EBMX.
EBMX also provides an interface based on IBM WebSphere MQ Series. These options should be
discussed with an EBMX analyst to determine the best choice when selecting an interface.
Features
•
Transfer file using SFTP
•
Accept parameters as arguments
•
Check for expected response
•
Full activity logging
• Return
code
setting
•
Korn shell version (ksh)
•
Bourne shell version (sh)
Operating Environment
The EBMX SFTP shell script is expected to run on UNIX variants such as Solaris, AIX, and Linux.
The operating system must be able to support either the Korn shell or the Bourne shell and have
a standard SFTP client installed.
Operation
The EBMX SFTP shell script has two versions:
Description
Shell Script Name
Korn Shell Version
sftp_put_rename.ksh
Bourne Shell Version
sftp_put_rename.sh
The EBMX SFTP shell script requires eight arguments:
Name Description
Value
User ID
The EBMX member ID of the
sender of the data.
Supplied by the EBMX
administrative team.
SFTP Host Name
The IP address or DNS name
of the EBMX system.
EBMX Production =
ebmxprod.appl.chrysler.com
EBMX Development =
ebmxdev.appl.chrysler.com
SFTP Port
SFTP port number.
50322
Local File Name
Name of the file to send
including path, if necessary.
Supplied by the user of the
script.
Sender ID
The EBMX member ID of the
sender of the data.
Supplied by the EBMX
administrative team.
Receiver ID
The EBMX member ID of the
receiver of the data.
Supplied by the EBMX
administrative team.
Document Type
The document type of the data
being sent.
Supplied by the EBMX
administrative team.
Log file name
The log file name for activity
logging including path, if
necessary.
Supplied by the user of the
script.
All arguments are mandatory.
The shell script exits with at return code of zero if the process was successful. The shell script exits with
at return code of 1 if the process fails. It is the responsibility of the user of the shell script to check the
return code for success or failure. If the process fails, interrogate the log file to determine the reason for
the failure. The EBMX administrative team can assist in the interpretation of the log file.
If the log file does not exist, the shell script will create a new one. If the log file does exist, the shell script
will append the results to the end of the existing log.
If the put and rename is successful, the log file will contain the character string “Success!”. If an error
occurs at any time during the workflow, the character string “Failure!” is placed in the log file. Therefore,
the EBMX team recommends that log files are kept for a minimum of seven days.
Examples
Bourne Shell
./sftp_put_rename.sh testdoc clmsd058b.cldc.chrysler.com 50322
test_file_1 TESTDOC LOOPTEST LOOPBACK mylog.log
Korn shell
./sftp_put_rename.ksh testdoc clmsd058b.cldc.chrysler.com 50322
test_file_1 TESTDOC LOOPTEST LOOPBACK mylog.log
Note: The above examples should be entered as one continuous command and not broken into
individual lines.
Example Log File
Each run of the EBMX SFTP shell script will produce the following log:
Success Example:
Running on: 2010-11-03:14:52:21 number of arguments received: 8
ARG[1] remoteuser: testdoc
ARG[2] remotehost: clmsd058b.cldc.chrysler.com ARG[3] remoteport: 50322
ARG[4] remotefile: test_file_1 ARG[5] ecxsender: TESTDOC ARG[6] ecxreceiver: LOOPTEST ARG[7] ecxdoctype: LOOPBACK ARG[8] remotelog: mylog.log
localfile for put is: test_file_1.20101103145221.30236
Connecting to clmsd058b.cldc.chrysler.com...
You are accessing FCA (Fiat Chrysler Automobiles) systems.
Connected to EBMX SFTP server.
Connection accepted.
Access to FCA's computer systems is controlled.
FCA authorizes use for its business purposes only.
Unauthorized access is prohibited due to risk of irreparable harm to FCA.
FCA management may monitor use to ensure compliance with its policies.
FCA may terminate access privileges, take disciplinary action
and/or institute civil or criminal proceedings to enforce this policy.
If any part of this policy is unacceptable to you, please disconnect now!!!
Thank you.
YOUR ACCESS IS BEING LOGGED & MONITORED!!!! sftp> dir
ERRORS RESULTS TESTDOC TMP sftp> cd /TMP
sftp> put test_file_1 test_file_1.20101103145221.30236
Uploading test_file_1 to /TMP/test_file_1.20101103145221.30236 sftp> rename test_file_1.20101103145221.30236
/TESTDOC/LOOPTEST/LOOPBACK/test_file_1.20101103145221.30236 sftp> quit
End of SFTP PUT and RENAME About to check exit status...
Failure Example: (Invalid file)
Running on: 2010-11-03:14:52:41 number of arguments received: 8 ARG[1] remoteuser: testdoc
ARG[2] remotehost: clmsd058b.cldc.chrysler.com ARG[3] remoteport: 50322
ARG[4] remotefile: xtest_file_1 ARG[5] ecxsender: TESTDOC ARG[6] ecxreceiver: LOOPTEST ARG[7] ecxdoctype: LOOPBACK ARG[8] remotelog: mylog.log
localfile for put is: xtest_file_1.20101103145241.24721 Connecting to clmsd058b.cldc.chrysler.com...
You are accessing FCA (Fiat Chrysler Automobiles) systems. Connected to EBMX SFTP server.
Connection accepted.
Access to FCA's computer systems is controlled. FCA authorizes use for its business purposes only.
Unauthorized access is prohibited due to risk of irreparable harm to FCA. FCA management may monitor use to ensure compliance with its policies. FCA may terminate access privileges, take disciplinary action
and/or institute civil or criminal proceedings to enforce this policy. If any part of this policy is unacceptable to you, please disconnect now!!! Thank you.
YOUR ACCESS IS BEING LOGGED & MONITORED!!!! sftp> dir
ERRORS RESULTS TESTDOC TMP sftp> cd /TMP
sftp> put xtest_file_1 xtest_file_1.20101103145241.24721 stat xtest_file_1: No such file or directory
sftp> rename xtest_file_1.20101103145241.24721
/TESTDOC/LOOPTEST/LOOPBACK/xtest_file_1.20101103145241.24721 Couldn't rename file "/TMP/xtest_file_1.20101103145241.24721" to
"/TESTDOC/LOOPTEST/LOOPBACK/xtest_file_1.20101103145241.24721": No such file or directory sftp> quit
End of SFTP PUT and RENAME About to check exit status... Failure!
Failure Example: (Invalid Public Key)
Running on: 2010-11-03:14:53:16 number of arguments received: 8 ARG[1] remoteuser: testdoc
ARG[2] remotehost: clmsd058b.cldc.chrysler.com ARG[3] remoteport: 50322
ARG[4] remotefile: xtest_file_1 ARG[5] ecxsender: TESTDOC ARG[6] ecxreceiver: LOOPTEST ARG[7] ecxdoctype: LOOPBACK ARG[8] remotelog: mylog.log
localfile for put is: xtest_file_1.20101103145316.20968 Connecting to clmsd058b.cldc.chrysler.com...
You are accessing FCA (Fiat Chrysler Automobiles) systems. Connected to EBMX SFTP server.
Connection accepted.
Access to FCA's computer systems is controlled. FCA authorizes use for its business purposes only.
Unauthorized access is prohibited due to risk of irreparable harm to FCA. FCA management may monitor use to ensure compliance with its policies. FCA may terminate access privileges, take disciplinary action
and/or institute civil or criminal proceedings to enforce this policy. If any part of this policy is unacceptable to you, please disconnect now!!! Thank you.
YOUR ACCESS IS BEING LOGGED & MONITORED!!!!
Permission denied (publickey).
Couldn't read packet: Connection reset by peer End of SFTP PUT and RENAME
About to check exit status... Failure!
Appendix A. Shell Scripts (Korn)
Copy and save to a file with the extenstion .ksh
#!/bin/ksh #
# This script will accept parameters/arguments and connect to the EBMX app # server via the OpenSourceSSH SFTP server and PUT a file. The reply # is examined for any errors - this determines success/failure of this # script. This script is intended for use with ksh.
#
# store the date for display/file naming purposes datetime=`date '+%Y%m%d%H%M%S'`
displaydatetime=`date '+%Y-%m-%d:%H:%M:%S'` sftp_response_log=" "
# store the number of arguments received and the arguments themselves numargsreceived=$# argumentsreceived=$* remoteuser=$1 remotehost=$2 remoteport=$3 remotefile=$4 ecxsender=$5 ecxreceiver=$6 ecxdoctype=$7 remotelog=$8
# construct the name of the local file to be PUT localfile="$remotefile.$datetime.$RANDOM"
# this routine echos the date, arguments and localfile echo_args () {
echo "Running on: $displaydatetime";
echo "number of arguments received: $numargsreceived"; echo;
# check to see if the number of arguments is correct if [ $numargsreceived -eq 8 ];
then echo "ARG[1] remoteuser: $remoteuser"; echo "ARG[2] remotehost: $remotehost"; echo "ARG[3] remoteport: $remoteport"; echo "ARG[4] remotefile: $remotefile"; echo "ARG[5] ecxsender: $ecxsender"; echo "ARG[6] ecxreceiver: $ecxreceiver"; echo "ARG[7] ecxdoctype: $ecxdoctype"; echo "ARG[8] remotelog: $remotelog"; echo;
elif [ $numargsreceived -gt 8 ];
then echo "Too many arguments received!!!"; echo "Received $numargsreceived arguments"; echo "Arguments received: $argumentsreceived";
echo "Arguments expected: <remoteuser> <remotehost> <remoteport> <remotefile> <ecxsender> <ecxreceiver> <ecxdoctype> [<remotelog>]";
Appendix A. Shell Scripts (Korn) (continued)
# this routine is responsible for the PUT of the file and RENAME # we also do a DIR and PWD for completeness
sftp_put_rename () {
# open sftp as a co-process (ksh only) so that we may write to/read from the # process. The reads are stored in a shell variable
sftp -oPort=$remoteport $remoteuser@$remotehost 2>&1 |& print -p dir
print -p cd /TMP
print -p put $remotefile $localfile print –p chmod 666 $localfile
print -p rename $localfile /$ecxsender/$ecxreceiver/$ecxdoctype/$localfile print -p quit
print -p :
# read all the responses at the end of the process while read -p sftp_response
do
# add a new line so it looks right - otherwise, it wraps on one line without # newlines xx="$sftp_response"; echo "$xx" >> $remotelog; sftp_response_log="$sftp_response_log$xx"; done : }
# this runs the echo_args routine - the eval is necessary for filename # expansion prior to redirection
eval echo_args 1>>$remotelog 2>&1
# run the sftp_put_rename routine - since we use a coprocess to store output # in a shell variable, we don't need to worry about redirection
sftp_put_rename
# grep the SFTP session log looking for ”Couldn|denied” - if we find one, it failed echo "End of SFTP PUT and RENAME" >>$remotelog 2>&1
echo "About to check exit status..." >>$remotelog 2>&1
if [ `echo $sftp_response_log | egrep -c '(Couldn|denied)'` -eq 0 ] then echo "Success!" >>$remotelog 2>&1;
exit 0;
else echo "Failure!" >>$remotelog 2>&1; exit 1;
Appendix B. Shell Scripts (Bourne Shell)
Copy and save to a file with the extenstion .sh
#!/bin/sh #
# This script will accept parameters/arguments and connect to the EBMX app # server via the OpenSourceSSH SFTP server and PUT a file. The reply # is examined for any errors - this determines success/failure of this # script. This script is intended for use with ksh.
#
# store the date for display/file naming purposes datetime=`date '+%Y%m%d%H%M%S'`
displaydatetime=`date '+%Y-%m-%d:%H:%M:%S'`
# store the number of arguments received and the arguments themselves numargsreceived=$# argumentsreceived=$* remoteuser=$1 remotehost=$2 remoteport=$3 remotefile=$4 ecxsender=$5 ecxreceiver=$6 ecxdoctype=$7 remotelog=$8 #echo $datetime;
# define temporary log...
tempremotelog="./SFTP_LOG.$datetime";
# construct the name of the local file to be PUT localfile="$remotefile.$datetime" if [ $# -gt 0 ] then numargsreceived=$#; else numargsreceived=0; fi argsreceived=$*;
# this routine echos the date, arguments and localfile echo_args () {
echo "Running on: $displaydatetime";
echo "number of arguments received: $numargsreceived"; # check to see if the number of arguments is correct if [ $numargsreceived -eq 8 ];
then echo "Start of FTP PUT and SUBMIT"; exitError=0;
elif [ $numargsreceived -eq 7 ];
# generate a remote log file when only 7 arguments are passed
Appendix B. Shell Scripts (Bourne Shell) (continued)
echo;
echo "ARG[1] remoteuser: $remoteuser"; echo "ARG[2] remotehost: $remotehost"; echo "ARG[3] remoteport: $remoteport"; echo "ARG[4] remotefile: $remotefile"; echo "ARG[5] ecxsender: $ecxsender"; echo "ARG[6] ecxreceiver: $ecxreceiver"; echo "ARG[7] ecxdoctype: $ecxdoctype"; echo "ARG[8] remotelog: $remotelog"; echo "Local File: $localfile"; echo "Temp Log: $tempremotelog"; echo;
}
# this routine is responsible for the PUT of the file and ensuing SUBMIT # we also do a DIR and PWD for completeness
sftp_put_rename () {
sftp -oPort=$remoteport $remoteuser@$remotehost << !EOF cd /TMP
put $remotefile $localfile chmod 666 $localfile
rename $localfile /$ecxsender/$ecxreceiver/$ecxdoctype/$localfile quit
!EOF
echo End of SFTP PUT and RENAME; :
}
# this runs the echo_args routine - the eval is necessary for filename # expansion prior to redirection
eval echo_args 1>>$tempremotelog 2>&1
# this runs the sftp_put_rename routine - the eval is necessary for filename # expansion prior to redirection
if [ $exitError -ne 1 ]; then
eval sftp_put_rename 1>>$tempremotelog 2>&1; else
echo "Incorrect number of Arguments, please see $tempremotelog"; exit;
fi
echo About to check exit status... >>$tempremotelog
# grep the remote log looking for a tracking ID - if we find one, it # worked
if [ $exitError -ne 1 ]; then
it_worked=`egrep -c '(Couldn|denied)' $tempremotelog`; else
it_worked=0; fi
# if we find one tracking ID, it worked. If we find more than one, the log # contains multiple sessions, and we can't tell for sure if the last one # worked
