Thissection showsyouhow todefinea WrappedstagecalledexhortwhichrunstheUNIX sort
commandinparallel.Thestagesortsdataintwofilesandoutputstheresultstoafile.Theincoming data hastwocolumns,ordernumberand code.Thesortcommandsortsthedataonthesecond field,code.
Youcanoptionallyspecifythatthesortisruninreverseorder.
Wrappingthesortcommandinthisway wouldbeusefulifyouhada situationwhereyouhadafixed sortoperationthatwaslikelytobeneededinseveraljobs.Havingitasaneasilyreusablestage would savehavingtoconfigurea built-insortstageeverytimeyouneededit.
Whenincludedinajoband run,thestagewilleffectively calltheSortcommandasfollows:
sort -r -o outfile -k 2 infile1 infile2
Thefollowingscreenshots showhowthisstageisdefinedinWebSphereDataStageusingtheStageType dialogbox:
1. Firstgeneraldetailsaresupplied intheGeneraltab.Theargument definingthesecondcolumnasthe keyisincludedinthecommandbecausethisdoesnotvary:
2. Thereverseorderargument (-r)areincludedaspropertiesbecauseitisoptionalandmayormaynot beincludedwhen thestageisincorporatedintoajob.
3. ThefactthatthesortcommandexpectstwofilesasinputisdefinedontheInput sub-tabonthe Interfacestabof theWrapper page.
4. Thefactthatthesortcommandoutputstoa fileisdefinedontheOutputsub-tabontheInterfaces taboftheWrapperpage.
Note: Whenyouusethestagein ajob,makesurethatyouusetabledefinitionscompatiblewiththe tablesdefinedintheinputandoutputsub-tabs.
5. Becauseall exitcodesotherthan0are treatedaserrors,andbecausetherearenospecialenvironment requirementsforthis command,youdo notneed toalteranything ontheEnvironment tabofthe Wrappedpage.All thatremainsistoclickGenerate tobuildthestage.
Chapter 6. Environment Variables
Therearemanyenvironmentvariablesthataffectthedesignand runningofparalleljobs inWebSphere DataStage.Commonlyusedonesare exposedintheWebSphereDataStageAdministratorclient,and can beset orunsetusingtheAdministrator(seeWebSphereDataStage AdministratorClientGuide). Thereare additionalenvironmentvariables,however.Thischapterdescribesalltheenvironmentvariablesthat applytoparalleljobs.Theycanbe setorunsetasyouwouldanyotherUNIXsystemvariables,oryou canaddthemtotheUserDefinedsection intheWebSphereDataStageAdministratorenvironment variabletree.
Theavailableenvironmentvariablesaregrouped accordingtofunction.They aresummarizedinthe followingtable.
Thefinalsectioninthischaptergivessomeguidancetosettingtheenvironmentvariables.
Category
EnvironmentVariable Buffering
APT_BUFFER_FREE_RUN
APT_BUFFER_MAXIMUM_MEMORY APT_BUFFER_MAXIMUM_TIMEOUT APT_BUFFER_DISK_WRITE_INCREMENT APT_BUFFERING_POLICY
APT_SHARED_MEMORY_BUFFERS BuildingCustom Stages
DS_OPERATOR_BUILDOP_DIR OSH_BUILDOP_CODE
OSH_BUILDOP_HEADER OSH_BUILDOP_OBJECT OSH_BUILDOP_XLC_BIN OSH_CBUILDOP_XLC_BIN Compiler
APT_COMPILER APT_COMPILEOPT APT_LINKER APT_LINKOPT DB2Support
APT_DB2INSTANCE_HOME APT_DB2READ_LOCK_TABLE APT_DBNAME
APT_RDBMS_COMMIT_ROWS DB2DBDFT
Debugging
APT_DEBUG_OPERATOR APT_DEBUG_MODULE_NAMES APT_DEBUG_PARTITION APT_DEBUG_SIGNALS APT_DEBUG_STEP APT_DEBUG_SUBPROC APT_EXECUTION_MODE APT_PM_DBX
APT_PM_GDB
APT_PM_SHOW_PIDS APT_PM_XLDB APT_PM_XTERM APT_SHOW_LIBLOAD DecimalSupport
APT_DECIMAL_INTERM_PRECISION APT_DECIMAL_INTERM_SCALE
APT_DECIMAL_INTERM_ROUND_MODE DiskI/O
APT_BUFFER_DISK_WRITE_INCREMENT APT_CONSISTENT_BUFFERIO_SIZE APT_EXPORT_FLUSH_COUNT
APT_IO_MAP/APT_IO_NOMAPandAPT_BUFFERIO_MAP/APT_BUFFERIO_NOMAP APT_PHYSICAL_DATASET_BLOCK_SIZE
GeneralJobAdministration APT_CHECKPOINT_DIR APT_CLOBBER_OUTPUT APT_CONFIG_FILE
APT_DISABLE_COMBINATION APT_EXECUTION_MODE APT_ORCHHOME APT_STARTUP_SCRIPT APT_NO_STARTUP_SCRIPT APT_STARTUP_STATUS APT_THIN_SCORE JobMonitoring
APT_MONITOR_SIZE APT_MONITOR_TIME APT_NO_JOBMON
APT_PERFORMANCE_DATA LookUpsupport
APT_LUTCREATE_NO_MMAP Miscellaneous
APT_COPY_TRANSFORM_OPERATOR
“APT_EBCDIC_VERSION”onpage65 APT_DATE_CENTURY_BREAK_YEAR
APT_IMPEXP_ALLOW_ZERO_LENGTH_FIXED_NULL APT_IMPORT_REJECT_STRING_FIELD_OVERRUNS APT_INSERT_COPY_BEFORE_MODIFY
APT_OLD_BOUNDED_LENGTH APT_OPERATOR_REGISTRY_PATH APT_PM_NO_SHARED_MEMORY APT_PM_NO_NAMED_PIPES APT_PM_SOFT_KILL_WAIT
APT_PM_STARTUP_CONCURRENCY APT_RECORD_COUNTS
APT_SAVE_SCORE
APT_SHOW_COMPONENT_CALLS APT_STACK_TRACE
APT_WRITE_DS_VERSION OSH_PRELOAD_LIBS Network
APT_IO_MAXIMUM_OUTSTANDING APT_IOMGR_CONNECT_ATTEMPTS APT_PM_CONDUCTOR_HOSTNAME APT_PM_NO_TCPIP
APT_PM_NODE_TIMEOUT APT_PM_SHOWRSH
APT_PM_USE_RSH_LOCALLY APT_RECVBUFSIZE
APT_RECVBUFSIZE
NLS APT_COLLATION_SEQUENCE APT_COLLATION_STRENGTH APT_ENGLISH_MESSAGES APT_IMPEXP_CHARSET APT_INPUT_CHARSET APT_OS_CHARSET
APT_OUTPUT_CHARSET APT_STRING_CHARSET OracleSupport
APT_ORACLE_LOAD_DELIMITED APT_ORACLE_LOAD_OPTIONS APT_ORACLE_NO_OPS
APT_ORACLE_PRESERVE_BLANKS
APT_ORA_IGNORE_CONFIG_FILE_PARALLELISM APT_ORA_WRITE_FILES
APT_ORAUPSERT_COMMIT_ROW_INTERVALAPT_ORAUPSERT_COMMIT_TIME_INTERVAL Partitioning
APT_NO_PART_INSERTION APT_PARTITION_COUNT APT_PARTITION_NUMBER ReadingandWritingFiles
APT_DELIMITED_READ_SIZE APT_FILE_IMPORT_BUFFER_SIZE APT_FILE_EXPORT_BUFFER_SIZE APT_IMPORT_PATTERN_USES_FILESET APT_MAX_DELIMITED_READ_SIZE
APT_PREVIOUS_FINAL_DELIMITER_COMPATIBLE APT_STRING_PADCHAR
Reporting
APT_DUMP_SCORE
APT_ERROR_CONFIGURATION APT_MSG_FILELINE
APT_PM_PLAYER_MEMORY APT_PM_PLAYER_TIMING APT_RECORD_COUNTS OSH_DUMP
OSH_ECHO OSH_EXPLAIN
OSH_PRINT_SCHEMAS SASSupport
APT_HASH_TO_SASHASH APT_NO_SASOUT_INSERT APT_NO_SAS_TRANSFORMS APT_SAS_ACCEPT_ERROR APT_SAS_CHARSET
APT_SAS_CHARSET_ABORT APT_SAS_COMMAND APT_SASINT_COMMAND APT_SAS_DEBUG
APT_SAS_DEBUG_IO APT_SAS_DEBUG_LEVEL APT_SAS_DEBUG_VERBOSE APT_SAS_NO_PSDS_USTRING APT_SAS_S_ARGUMENT
APT_SAS_SCHEMASOURCE_DUMP APT_SAS_SHOW_INFO
APT_SAS_TRUNCATION Sorting
APT_NO_SORT_INSERTION
APT_SORT_INSERTION_CHECK_ONLY TeradataSupport
APT_TERA_64K_BUFFERS APT_TERA_NO_ERR_CLEANUP APT_TERA_NO_PERM_CHECKS APT_TERA_NO_SQL_CONVERSION APT_TERA_SYNC_DATABASE APT_TERA_SYNC_USER TransportBlocks
APT_AUTO_TRANSPORT_BLOCK_SIZE APT_LATENCY_COEFFICIENT
APT_DEFAULT_TRANSPORT_BLOCK_SIZE
APT_MAX_TRANSPORT_BLOCK_SIZE/APT_MIN_TRANSPORT_BLOCK_SIZE
Buffering
Theseenvironmentvariable areallconcerned withthebufferingWebSphereDataStageperformsonstage linkstoavoiddeadlock situations.ThesesettingscanalsobemadeontheInputspageorOutputspage Advancedtaboftheparallelstageeditors.