Validating and tracing jobs
Exercise 6: Setting traces and annotations
Exercise Objectives
After completing this exercise, you will be able to:
• Use descriptions and annotations
• Setting traces on jobs
Business Example
You are sharing your jobs with other developers during the project, so you want to make sure that you identify the purpose of the job you created. You also want to ensure that the job is handling the movement of each row appropriately.
Task:
You add an annotation to the data flow with an explanation of the purpose of the job.
1. Add an annotation to the workspace of the job you have already created.
2. Execute the Alpha_Customers_Job after enabling the tracing of rows.
Solution 6: Setting traces and annotations
Task:
You add an annotation to the data flow with an explanation of the purpose of the job.
1. Add an annotation to the workspace of the job you have already created.
a) Open the workspace of the Alpha_Customers_Job by selecting the job.
b) From the Tool Palette, select the icon for an Annotation item and drag it in to the workspace beside the data flow. Then click the workspace to add the Annotation.
c) Type in an explanation of the purpose of the job, such as: “The purpose of this job is to move records from the Customer table from the Alpha datastore to a template table, Alpha_customers in the Delta staging datastore.”
d) Save all objects you have created by using the icon Save All.
2. Execute the Alpha_Customers_Job after enabling the tracing of rows.
a) Right click the Alpha_Customers_Job and select the option Execute.
b) In the Execution Properties dialog box, select the Trace tab and select the Trace rows option.
c) Select OK in the Execution Properties dialog box.
d) In the Trace log, you should see an entry for each row added to the log to indicate how it is being handled by the data flow.
Lesson Summary
You should now be able to:
• Use descriptions and annotations
• Setting traces on jobs
Lesson: Setting Traces and Adding Annotations
Lesson Overview
To document decisions and troubleshoot any issues that arise when executing your jobs, you can validate and add annotations to jobs, work flows, and data flows, set trace options, and debug your jobs. You can also set up audit rules to ensure the correct data is loaded to the target.
To document decisions and troubleshoot any issues that arise when executing your jobs, you can validate and add annotations to jobs, work flows, and data flows.
Lesson Objectives
After completing this lesson, you will be able to:
• Use descriptions and annotations
• Setting traces on jobs
Business Example
Your company has recognized how useful it can be to integrate people, information and business processes in a heterogeneous system landscape and would like to obtain this benefit. Practice has shown, though, that loading large datasets makes considerable demands on hardware and system performance. It is therefore necessary to examine if and how the data records can be loaded into SAP NetWeaver Business Warehouse with a delta process and to understand the modes of operation and the different variants of a delta loading process.
Using descriptions and annotations
Descriptions and annotations are a convenient way to add comments to objects and workspace diagrams.
Figure 46: Annotations and Descriptions
Using descriptions with objects
A description is associated with a particular object. When you import or export a repository object, you also import or export its description.
Designer determines when to show object descriptions based on a system-level setting and an object-level setting. Both settings must be activated to view the description for a particular object.
Note: The system-level setting is unique to your setup.
There are three requirements for displaying descriptions:
• A description has been entered into the properties of the object.
• The description is enabled on the properties of that object.
• The global View Enabled Object Descriptions option is enabled.
To show object descriptions at the system level
• From the View menu, select Enabled Descriptions.
Note: The Enabled Descriptions option is only available if it is a viable option.
To add a description to an object
1. In the project area or the workspace, right-click an object and select Properties from the menu.
The Properties dialog box displays.
2. In the Description text box, enter your comments.
3. Select OK.
If you are modifying the description of a reusable object, Data Services provides a warning message that all instances of the reusable object are affected by the change.
4. Select Yes.
The description for the object displays in the Local Object Library.
To display a description in the workspace
• In the workspace, right-click the object in the workspace and select Enable Object Description from the menu.
The description displays in the workspace under the object.
Using annotations to describe objects
An annotation is an object in the workspace that describes a flow, part of a flow, or a diagram. An annotation is associated with the object where it appears. When you import or export a job, work flow, or data flow that includes annotations, you also import or export associated annotations.
To add an annotation to the workspace
1. In the workspace, from the tool palette, select the Annotation icon and then select the workspace.
An annotation appears on the diagram.
2. Double-click the annotation.
3. Add text to the annotation.
4. Select the cursor outside of the annotation to commit the changes.
You can resize and move the annotation by clicking and dragging.
You cannot hide annotations that you have added to the workspace. However, you can move them out of the way or delete them.
Validating and tracing jobs
Validating jobs
It is a good idea to validate your jobs when you are ready for job execution to ensure there are no errors. You can also select and set specific trace properties, which allow you to use the various log files to help you read job execution status or troubleshoot job errors.
As a best practice, you want to validate your work as you build objects so that you are not confronted with too many warnings and errors at one time. You can validate your objects as you create a job or you can automatically validate all your jobs before executing them.
Figure 47: Validating Jobs
To validate jobs automatically before job execution 1. From the Tools menu, select Options.
The Options dialog box displays.
2. In the Category pane, expand the Designer branch and select General.
3. Select the Perform complete validation before job execution option.
4. Select OK.
To validate objects on demand
1. From the Validation menu, select Validate → Current View or All Objects in View.
The Output dialog box displays.
2. To navigate to the object where an error occurred, right-click the validation error message and select Go To Error from the menu.
Tracing jobs
Use trace properties to select the information that Data Services monitors and writes to the trace log file during a job. Data Services writes trace messages to the trace log associated with the current Job Server and writes error messages to the error log associated with the current Job Server.
Figure 48: Setting Traces in Job Execution Properties
These trace options are available.
Trace Description
Row Writes a message when a transform imports or exports a row.
Session
Writes a message when the job description is read from the repository, when the job is optimized, and when the job runs.
Work flow
Writes a message when the work flow description is read from the repository, when the work flow is optimized, when the work flow runs, and when the work flow ends.
Data flow Writes a message when the data flow starts and when the data flow successfully finishes or terminates due to error.
Transform Writes a message when a transform starts and completes or terminates.
Custom Transform Writes a message when a custom transform starts and completes successfully.
Custom Function Writes a message of all user invocations of the AE_LogMessage function from custom C code.
SQL Functions
Writes data retrieved before SQL functions:
• Every row retrieved by the named query before the SQL is submitted in the key_generation function.
• Every row retrieved by the named query before the SQL is submitted in the lookup function (but only if PRE_LOAD_CACHE is not specified).
• When mail is sent using the mail_to function.
SQL Transforms
Writes a message (using the Table Comparison
transform) about whether a row exists in the target table that corresponds to an input row from the source table.
SQL Readers
Writes the SQL query block that a script, query transform, or SQL function submits to the system and writes the SQL results.
SQL Loaders
Writes a message when the bulk loader starts, submits a warning message, or completes successfully or unsuccessfully.
Memory Source Writes a message for every row retrieved from the memory table.
Memory Target Writes a message for every row inserted into the memory table.
Optimized Data Flow For SAP BusinessObjects consulting and technical support use.
Tables Writes a message when a table is created or dropped.
Scripts and Script Functions
Writes a message when a script is called, a function is called by a script, and a script successfully completes.
Trace Parallel Writes messages describing how data in a data flow is
Access Server Communication
Writes messages exchanged between the Access Server and a service provider.
Stored Procedure Writes a message when a stored procedure starts and finishes, and includes key values.
Audit Data Writes a message that collects a statistic at an audit point and determines if an audit rule passes or fails.
To set trace options
1. From the project area, right-click the job name and do one of these actions::
• To set trace options for a single instance of the job, select Execute from the menu.
• To set trace options for every execution of the job, select Properties from the menu.
Save all files.
Depending on which option you selected, the Execution Properties dialog box or the Properties dialog box displays.
2. Select the Trace tab.
3. Under the name column, select a trace object name.
The Value dropdown list is enabled when you select a trace object name.
4. From the Value dropdown list, select Yes to turn the trace on.
5. Select OK.
Using log files
As a job executes, Data Services produces three log files. You can view these from the project area. The log files are, by default, also set to display automatically in the workspace when you execute a job.
You can select the Trace, Monitor, and Error icons to view the log files, which are created during job execution.
Examining trace logs
Use the trace logs to determine where an execution failed, whether the execution steps occur in the order you expect, and which parts of the execution are the most time consuming.
Figure 49: Using the Trace Log
Examining monitor logs
Use monitor logs to quantify the activities of the components of the job. It lists the time spent in a given component of a job and the number of data rows that streamed through the component.
Figure 50: Using the Monitor and Error Logs
Examining error logs
Use the error logs to determine how an execution failed. If the execution completed without error, the error log is blank.
Using the Monitor tab
The Monitor tab lists the trace logs of all current or most recent executions of a job.
The traffic-light icons in the Monitor tab indicate:
• Green light indicates that the job is running.
You can right-click and select Kill Job to stop a job that is still running.
• Red light indicates that the job has stopped.
You can right-click and select Properties to add a description for a specific trace log. This description is saved with the log which can be accessed later from the Log tab.
• Red cross indicates that the job encountered an error.
Using the Log tab
You can also select the Log tab to view a job’s log history.
To view log files from the project area 1. In the project area, select the Log tab.
2. Select the job for which you want to view the logs.
3. In the workspace, in the Filter dropdown list, select the type of log you want to view.
4. In the list of logs, double-click the log to view details.
5. To copy log content from an open log, select one or more lines and use the key commands [Ctrl+C].
Determining the success of the job
The best measure of the success of a job is the state of the target data. Always examine your data to make sure the data movement operation produced the results you expect.
Figure 51: View Data in Data Flow
Be sure that:
• Data is not converted to incompatible types or truncated.
• Data is not duplicated in the target.
• Data is not lost between updates of the target.
If a job fails to execute, check the Job server icon in the status bar to verify that the Job Service is running. Check that the port number in Designer matches the number specified in Server Manager. If necessary, you can use the Server Manager Resync button to reset the port number in the Local Object Library.
Exercise 7: Setting traces and annotations
Exercise Objectives
After completing this exercise, you will be able to:
• Use descriptions and annotations
• Setting traces on jobs
Business Example
You are sharing your jobs with other developers during the project, so you want to make sure that you identify the purpose of the job you created. You also want to ensure that the job is handling the movement of each row appropriately.
Task:
You add an annotation to the data flow with an explanation of the purpose of the job.
1. Add an annotation to the workspace of the job you have already created.
2. Execute the Alpha_Customers_Job after enabling the tracing of rows.
Solution 7: Setting traces and annotations
Task:
You add an annotation to the data flow with an explanation of the purpose of the job.
1. Add an annotation to the workspace of the job you have already created.
a) Open the workspace of the Alpha_Customers_Job by selecting the job.
b) From the Tool Palette, select the icon for an Annotation item and drag it in to the workspace beside the data flow. Then click the workspace to add the Annotation.
c) Type in an explanation of the purpose of the job, such as: “The purpose of this job is to move records from the Customer table from the Alpha datastore to a template table, Alpha_customers in the Delta staging datastore.”
d) Save all objects you have created by using the icon Save All.
2. Execute the Alpha_Customers_Job after enabling the tracing of rows.
a) Right click the Alpha_Customers_Job and select the option Execute.
b) In the Execution Properties dialog box, select the Trace tab and select the Trace rows option.
c) Select OK in the Execution Properties dialog box.
d) In the Trace log, you should see an entry for each row added to the log to indicate how it is being handled by the data flow.
Lesson Summary
You should now be able to:
• Use descriptions and annotations
• Setting traces on jobs