Advanced XML Seminar. Objective. Personal Introductions Jeff Rosler. Agenda. XML Application Architecture Core Documentum/XML Features:

(1)

1 1

Advanced XML Seminar

Jeff Rosler Flatirons Solutions DevCon 2004 San Ramon, CA 10-4-2004 2 2

Objective

Describe tools and methods to handle common

problems found in consulting projects

implementing XML Solutions with Documentum

3 3

Agenda

z

Personal Introductions

z

Presentation Overview

z

Problems/Solutions and Tools and Tricks

z

Questions

4 4

Personal Introductions – Jeff Rosler

z Flatirons Solutions

–Full lifecycle consulting and implementation experience

•Strategic and business process consulting

•Systems Architecture and Information Architecture

•Systems design, integration and implementation

•Support and evolution

–Content-based solutions expertise

•Collaboration, document management, WCM, and imaging

•Integrations with ERP, CRM and financial systems

•Experts in information analysis and process re-engineering

•Leaders in XML-based solutions

z 20 years of software engineering experience in consulting and product development

z Over 3 years Documentum experience specializing in XML Solutions

z 5 years Content Management experience

www.flatironssolutions.com [email protected] 5 5

Presentation Overview

z

XML Application Architecture

z

Debugging XML Applications

z

Migrating XML content to Documentum

z

Calling Java code from XML Applications

z

Efficient XML document authoring

z

Integrating DFC code in XSL Transforms

6 6

XML Application Architecture

z

XML Applications

–Rules for processing XML content in and out of Documentum – XML Application config files

z

XML Editor Integration

–Allow reuse of content

z

Searching

–XDQL

–Verity zone searches

z

Transforming XML Content

–XDQL/DFC integration Core Documentum/XML Features:

(2)

7 7

XML Application Architecture

Simple block diagram:

DFC DMCL Docbroker Docbase Server Database/ Filesystem Dctm Desktop WDK/ Webtop DFC DMCL Custom Code DFC DMCL 8 8

XML Application Architecture

z

Import, check in, check out, export

z

Implemented in DFC

–com.documentum.operations.* (IDfImportOperation, IDfCheckinOperation, IDfCheckoutOperation, IDfExportOperation, etc.)

z

DFC may be located on user’s desktop (in case of

Documentum Desktop)

z

Operations may be heavyweight operations if large

amounts of chunking are involved

XML Application Processing :

9 9

XML Application Architecture

z

DFC Disclaimer

–The following descriptions are very general bullet points on DFC operations and are mentioned to allow users to better understand operations for debugging purposes.

z

Import/Check in (DFC logic)

–Retrieve XML Application config file –Parse XML Application config file –May download Application Support files –Parse XML content file and perform dctm updates

•May include SAX parse (using Xerces) •Process based on config file rules

•Create content chunks, fix-up virtual document, set properties, etc.

Operation Logic:

10 10

XML Application Architecture

z

Export/Check out (DFC logic)

–Retrieve XML Application config file –Parse XML Application config file –Download content file

–Download support files –Fixup of content files

•May include adding DOCTYPE declaration •Including SAX parse of content files (using Xerces)

Operation Logic:

11 11

XML Application Architecture

z

DFC java docs/samples

–Default location on windows install - C:\Program Files\Documentum\Help\DFC\api\samples

z

Documentum Developers site

(

http://customernet.documentum.com/developer/index.h

tml

)

–Look under DFC and XML sections

z

Flatirons Solutions site

(

http://www.flatironssolutions.com/Developers.htm

)

Sample Operations Code:

12 12

Debugging XML Applications

z

Important Points from Architecture to remember

–XML Config file is parsed

–XML Content file is parsed

–DTD and files included in DTD are parsed

z

Most run time errors occur because of problems in the

above content

z

Use custom code or Flatirons Docbase Shell Tool to

debug operations

–Be careful using Desktop Client to test, it can hide the real error messages (i.e. Unknown error)

(3)

13 13

Debugging XML Applications

z Flatirons Solutions Docbase Shell Tool The following commands are accepted by the shell:

cd <full or relative path> - Change current Docbase directory ci <relative path> - Check-in specified object

co <relative path> - Check-out specified object cc <relative path> - Cancel check-out export <full or relative path> - Export specified object get <full or relative path> - Use getFile for file content h/help - Print shell commands

import <full file system path> - Import specified document into Docbase ls - List all files in the current directory

q/quit/exit - Exit the application shell trace dfc [level] <full file system path> - Toggle DFC tracing trace dmcl [level] <full file system path> - Toggle DMCL tracking Overview:

14 14

Debugging XML Applications

z

Incorrect element/attribute names

z

No defaults specified (or empty </default> tag)

z

Incorrect context for variables

z

Incorrect context for content rule

z

Can’t find java class file (for java customizations)

–Make sure that jar or class is in CLASSPATH

•If using desktop, remember to restart desktop after CLASSPATH changes so that JVM is restarted

XML Application Config File: Common Problems

15 15

Debugging XML Applications

z

Bad date format

–Remember to use <date_format> and make sure mask is correct (based on java SimpleDateFormat class)

z

Missing content with no default

–Cannot find content for a <variable> (incorrect <element_selection_pattern>

z

Missing content for context rule

–Have <context_rule><child_of> in

<element_selection_pattern> on content rule and there is no parent in content

z

Content not well formed

XML Content File: Common Problems

16 16

Debugging XML Applications

z

Invalid Content (i.e. carriage return after <?xml?> decl)

z

Missing files (i.e. entity files)

z

Incorrect references (i.e. includes)

DTD/Support Files: Common Problems

17 17

Debugging XML Applications

z

Check content well formedness

–XML Editor (i.e. Epic, XMetal, XMLSpy, etc.) –Use Xerces to parse content

• http://xml.apache.org/xerces2-j/

•Run java sax.Counter sample to parse and validate z

If errors are found, check content and then DTDs

z

If no errors are found, re-check config file against

content

DTD/Support Files: How to determine which is which

18 18

Debugging XML Applications

z

If all else fails, use the DFC trace log

z

DFC logs are huge, try and turn trace on just for the

operation to debug (Flatirons docbase shell tool is good

for this)

z

Search from the bottom up for Exception – see if there

are any other exceptions before the first exception that

might give more information (sometimes early errors

mask final error – “Unknown error”)

z

Based on methods called and output of methods, try

and determine where problem occurred (i.e. loading

config file, setting a variable, etc.)

(4)

19 19

Migrating XML content to Documentum

z

Situation

–Client engagement to move from unstructured authoring to structured authoring using XML and Documentum –DTDs and screen stylesheets were created early in the

engagement before Documentum was available and certified –Client wished to have authors start authoring content

immediately

z

Problem to solve

–Determine best way for authors to author content on filesystem

–Create methods to migrate the authored content into Documentum retaining the reuse of content

Overview:

20 20

Calling Java code from XML Applications

z

Basics of adding custom java code to XML Applications

z

Three custom applications from customer engagements

–Validate content during import/check in

–Setting object names on link rules

–Integrating XML into external database tables

Overview:

21 21

Calling Java code from XML Applications

z

XML Application

–Declare the java code in XML Application –Call java code in <expression>

z

Java Code

–Use static main() to create test scenarios –Use logging to debug problems

z

Installation

–Make sure that code is in CLASSPATH wherever DFC is installed

–After deploying code and updating CLASSPATH if using Desktop Client, will need to restart JVM (log off windows and back on or kill explorer)

The basics:

22 22

Calling Java code from XML Applications

z

Requirements

–Client wants additional metadata validation before XML content is imported or checked in

z

Solution

–Create custom java code in XML Application to run business rules for content validation

–If content not valid, stop the operation and report an error Validate Content during import/check in: Overview

23 23

Calling Java code from XML Applications

z

Declare the Java code via <java_classes>:

<java_classes>

<java_class prefix=“Validation" classname =“com.flatironssolutions.documentum.validation"/> </java_classes>

z

Create variables to gather data for validation

<variables> <variable> <name>title</name> <content_of_element> <element_selection_pattern> <element>title</element> </element_selection_pattern> </content_of_element> </variable>…

Validate Content during import/check in : XML Application

24 24

Calling Java code from XML Applications

z

Use <expression> to set value and

validate

<metadata> <dctmattr> <name>title</name> <template><expression>Validation:validateTitle('<var name="title"/>','<var name="language"/>')</expression></template> </dctmattr> </metadata>

(5)

25 25

Calling Java code from XML Applications

package com.flatironssolutions.documentum; import org.apache.log4j.Logger;

public class validation {

static Logger logger = Logger.getLogger(validation.class);

public static String validateTitle(String title, String language) throws Exception {

String returnValue = null;

logger.debug(" validation:validateTitle (" + title + "," + language + ")"); // perform logic and set returnValue

// if validate logic fails run

// throw new Exception(“Validate failed for title ‘“ + title + “’ and language ‘“ + language + “’. Please fix problem in file and re-run check in or import”); return returnValue;

} }

Validate Content during import/check in : Java Function

26 26

Calling Java code from XML Applications

z

Requirements

–Client wants linked graphics to have name based on file name from fileref and language

–<graphic href=“/graphics/foo.gif” language=“EN”/> •should be imported into the docbase as an object named

–“foo EN.gif”

z

Solution

–Create custom java code in XML Application to parse fileref and construct object name

Setting object names on link rules : Overview

27 27

Calling Java code from XML Applications

z Declare the Java code via <java_classes>:

<java_classes>

<java_class prefix="util" classname="XmlAppUtil"/>

</java_classes>

z Create variables to gather data for validation

<variables> <variable> <name>language</name> <attribute_value_of_element> <this_element/> <attr_name>language</attr_name> </attribute_value_of_element> <default>EN</default> </variable> <variable> <name>href</name> <attribute_value_of_element> <this_element/> <attr_name>href</attr_name> </attribute_value_of_element> <default>Unknown href.jpg</default> </variable>

Setting object names on link rules : XML Application

28 28

Calling Java code from XML Applications

z

Set the object name

<object_name><expression>util:getObjectNameFromHref

('<var name="DF_SESSION"/>','<var

name="href"/>','<var

name="language"/>')</expression></object_name>

z

Pass in href and language (and DF_SESSION?)

to getObjectNameFromHref method

Setting object names on link rules : XML Application

29 29

Calling Java code from XML Applications

public static String getObjectNameFromHref(String stringSession, String filePath, String language) throws Exception {

logger.debug("XMLAppUtil:getObjectNameFromHref(" + stringSession + "," + filePath + "," + language + ")");

try {

String normalizedFilePath = normalize(stringSession,filePath);

File file = new File(normalizedFilePath); String objectName = ""; String extension = ""; String fileName = file.getName();

int nFileNameEnd = fileName.lastIndexOf('.'); if (nFileNameEnd == -1)

{ nFileNameEnd = fileName.length(); } else

{ extension = fileName.substring(nFileNameEnd); } String fileNameWithoutExtension = fileName.substring(0,nFileNameEnd);

objectName = fileNameWithoutExtension + " " + language + extension;

return objectName; }

catch (Throwable te)

Setting object names on link rules : Java Function

30 30

Calling Java code from XML Applications

z

What is normlize() and why do we need it?

z

The fileref values fit into various types

–Fileref for new graphic (import scenario)

•<graphic href=“/graphics/foo.gif” language=“EN”/>

–Fileref for pasted graphic (Copy and paste reference) •<graphic href=“c:\documentum\export\foo.gif” language=“EN”/>

–Fileref for checked out or exported content (check out without descendants)

•<graphic

href=“dctm://dfcdev_solora/0500ae198000f8f9?DMS_OBJECT_SPEC= RELATION_ID” language=“EN”/>

(6)

31 31

Calling Java code from XML Applications

private static String normalize(String stringSession, String filePath) throws Exception {

logger.debug("XMLAppUtil:normalize(" + stringSession + "," + filePath + ")"); String newFilePath = filePath;

try {

IDfClientX clientx = new DfClientX(); IDfClient client = DfClient.getLocalClient(); IDfSession session = client.findSession(stringSession);

// check to see if filePath is a DRL

if (filePath != null && filePath.length() >= 5 && filePath.substring(0,5).compareToIgnoreCase("dctm:")== 0) {

IDfXMLUtils xmlutils = clientx.getXMLUtils(); xmlutils.setSession(session);

IDfId id = xmlutils.getObjectIdFromDRL(newFilePath);

IDfSysObject sysObj = (IDfSysObject)session.getObject(id);

// use the object name as a basis

newFilePath = fixupObjectNameSuffix(sysObj.getObjectName()); }

// else // if it isn't a DRL, let's see if it's listed in the Registry

32 32

Calling Java code from XML Applications

else // if it isn't a DRL, let's see if it's listed in the Registry {

IDfClientRegistry clientRegistry = clientx.getClientRegistry();

IDfViewedObject viewedObject = clientRegistry.getViewedObjectByPath(filePath); IDfViewedObject viewedObjectExportFullPath =

clientRegistry.getViewedObjectByPath(clientRegistry.getExportDirectory() + File.separator + filePath); IDfViewedObject viewedObjectCheckoutFullPath =

clientRegistry.getViewedObjectByPath(clientRegistry.getCheckoutDirectory() + File.separator + filePath); IDfCheckedOutObject checkedOutObject = clientRegistry.getCheckedOutObjectByPath(filePath); IDfCheckedOutObject checkedOutObjectExportFullPath =

clientRegistry.getCheckedOutObjectByPath(clientRegistry.getExportDirectory() + File.separator + filePath); IDfCheckedOutObject checkedOutObjectCheckoutFullPath =

clientRegistry.getCheckedOutObjectByPath(clientRegistry.getCheckoutDirectory() + File.separator + filePath); if (viewedObject != null) // was it exported - is it already a full path?

{

IDfId id = new DfId (viewedObject.getObjectId());

IDfSysObject sysObj = (IDfSysObject)session.getObject(id);

// use the object name as a basis

newFilePath = fixupObjectNameSuffix(sysObj.getObjectName()); }

else ... other conditions

33 33

Calling Java code from XML Applications

z

Test all of your use cases in your XML Applications to

find issues for all chunked content

–Import (with and without reuse) –Checkout (with and without descendants) –Checkin (with and without reuse) –Export

–Etc.

–All clients and custom code should be checked

z

Whitepaper with sample code and XML Application can

be downloaded from Flatirons Solutions Developer

Zone

-http://www.flatironssolutions.com/Downloads/xml_app_j

ava_ref_article.zip

Setting object names on link rules : Summary

34 34

Calling Java code from XML Applications

z

Requirements

–Client needs to export XML metadata to Oracle during an import or check in operation

–Multiple tables need to be updated based on content –Automated mechanism desired to insert XML values into

database rows

z

Solution

–Through java extensions of XML applications and linking and registering the target tables, extracted values can be passed and written

Integrating XML with Database Tables: Overview

35 35

Calling Java code from XML Applications

z

Declare the Java code via <java_classes>:

<java_classes>

<java_class prefix="StaticDataTableUtilities" classname ="dss.integ.dataObjects.StaticDataTableUtilities"/> </java_classes>

z

For the XML mapped to metadata, do not make an

object

<xml_content_rule export="inline"

editable_virtual_doc="false" make_object="false"> <element_selection_pattern>

<element>Admin</element> </element_selection_pattern> <variables>…

Integrating XML with Database Tables: The XML Application

36 36

Calling Java code from XML Applications

z

After the variables are mapped, pass them to your Java

code:

<metadata> <dctmattr> <name>title</name> <template> <expression> StaticDataTableUtilities:insertAdminElement ("<var name = "DF_SESSION"/>", "<var name=

"admin_org_name"/>", "<var name="admin_prog_name"/>", "<var name = "admin_prog_type"/>", "<var name= "DSSSchemaName"/>", "<var name="DocbaseSchemaName"/>", "<var name="CreateUserID"/>")

</expression> </template>…

z

Note the session passed from the XML application

(7)

37 37

Calling Java code from XML Applications

z

All arguments are strings

public static void insertAdminElement(String sSessionID,…

z

The query is built from the arguments as an IDfQuery

string and executed

IDfQuery oInsertQuery; oInsertQuery = new DfQuery(); oInsertQuery.setDQL(“INSERT INTO… …

IDfCollection col = oInsertQuery.execute(oSession, 4) col.close();

Integrating XML with Database Tables: Java function

38 38

Calling Java code from XML Applications

z

To pass the schemas as variable, the import needs

to be called from code

oImportOperation = (IDfImportOperation) clientX.getOperation("Import"); oImportOperation.setSession(oSession); oImportOperation.setExternalVariable("DocbaseSchemaName", sDocbaseSchemaName); oImportOperation.setExternalVariable("DSSSchemaName", sDSSSchemaName);

z

Specify the XML file and XML application and

execute

oImportNode = (IDfImportNode) oImportOperation.add(oXMLFile);

oImportNode.setXMLApplicationName(sXMLAppName); oImportNode.setKeepLocalFile(true);

if (oImportOperation.execute()) { // log results

Integrating XML with Database Tables: The Import Code (optional)

39 39

Calling Java code from XML Applications

z

Pre-defined External Variables that can be used

–DF_USER_NAME

–DF_HOME_CABINET –DF_DOCBASE –DF_SESSION_ID

z XML Application config file

<name>title</name> <template>

<expression>Myutils:useAllVars ("<var name = "DF_SESSION"/>", "<var name= “DF_USER_NAME"/>", "<var name=“DF_HOME_CABINET"/>", "<var name = “DF_DOCBASE"/>")

</expression> </template>…

Aside – External variables:

40 40

Calling Java code from XML Applications

z

Query other parameters from Documentum

–If based on a user, consider specifying alias on user

–Create custom configuration object

z

Efficient XML document authoring

The most pervasive complaint about authoring XML content in Document is the performance. When decisions are made to chunk many objects assembly and disassembly can take a long time – how can this be addressed?

z

Out of the box

–Chunking and authoring decisions –Webtop versus Desktop –Desktop with Citrix

z

Draft Objects (in editing environment)

z

Draft Objects in workflows

Overview:

42 42

Efficient XML document authoring

z

You should chunk

-–Core components that will be shared between documents –Major components that are redundant between documents –Components that form a logical unit of work for authors –When you want to allow authors to edit smaller pieces of the

overall document without locking other authors out

–When you want separate approval workflows for pieces of the document, or to be able to control access to chunks

z

You should not chunk –

–XML components at too low a level –When you don’t have a clear reason to do so

–Just because some of the information in a component is redundant

(8)

43 43

Efficient XML document authoring

z

Checking out content

–Check out at the appropriate virtual document level –When updating structure don’t download descendants

Out of the box: Chunking and authoring decisions

44 44

Efficient XML document authoring

Even using good chunking and authoring decisions many solutions may take too long for authors to checkout and check in content

z

Consider using an Application Server Architecture to

move assembly/disassembly to a server

–Webtop –Desktop with Citrix

z

For XML Editor integration consider

–Arbortext Epic 5.1 with WDK/XML adapter –Arbortext Contributor 5.1

–Blast Radius XMetal ActiveX control

Out of the box: XML Processing location

45 45

Efficient XML document authoring

If all else fails and you still need further performance improvements, consider these custom solutions

z

Draft objects are objects which contain preassembled

content

z

Draft objects are created by assembling and locking

content on server and copying to a new draft object

z

Authors edit/check in/check out as much as desired

z

When authoring is complete, draft object is

disassembled and individual chunks are checked in

Draft Objects: Overview

46 46

Efficient XML document authoring

Draft Objects (In Editing Environment): Overview

Author Documentum Server Current Data Flow:

Each requested object of Virtual Document, thousands of objects

Network Traffic

Author Documentum Server Future Data Flow:

Single File _{Queries and}

Assembly/ Disassembly of objects Single Request Multiple Requests Queries and Assembly/ Disassembly of objects 47 47

Efficient XML document authoring

z Server

–Documentum method to handle all Draft Object assembly/disassembly

–Server side methods can be pushed to more then one java method server to balance load

z Client

–Epic Documentum Adapter customizations –Additional menus and dialogs to work on draft objects –Call server code for all heavy lifting

Draft Objects (In Editing Environment): Architecture

48 48

Efficient XML document authoring

z Epic Editor Menu customizations

(9)

49 49

Efficient XML document authoring

z Browse and assemble content into a Draft Object

Draft Objects (In Editing Environment): User Interface

What happens if some content is locked?

50 50

Efficient XML document authoring

z Operate on draft objects (My Draft objects)

Draft Objects (In Editing Environment): User Interface

51 51

Efficient XML document authoring

z Testing of the solution showed an average 50% reduction in the amount of time it took to view/edit and check in content

–Exact numbers were greater and lesser depending on the number of objects in a virtual document

z Additional gains were seen based on the number of times draft objects were checked in and out (i.e. checking a draft object in and out was very fast since no XML processing was done) z Reduced network load (useful for remote authors) z Reduced need for upgrading underpowered authoring PCs z Even with other remoting mechanisms (Citrix, Web application),

still saw performance improvements

Draft Objects (In Editing Environment): Summary

52 52

Efficient XML document authoring

If authoring tasks are determined through a more formal process, consider using draft objects in workflows.

z Authoring workflow is started

z Method is launched to create draft object for author and attach to workflow

z Task is put into author’s inbox with attached draft object package z Author edits draft object and makes appropriate changes z Author completes task

z Workflow can continue and create renditions for review and pass/ fail content

z Workflow method runs to disassemble draft object and check changes back in to individual chunks

Draft Objects (In Workflows): Overview

53 53

Integrating DFC code in XSL Transforms

z

How to call DFC within an XSL Transform

z

How to call your own DFC classes within an XSL

Transform

–Allows you to implement more complex logic •Conditional / Fallback logic

•Execute side-effects •Error handling

–Potentially faster execution

Overview:

54 54

Integrating DFC code in XSL Transforms

z

Declare java in your xsl:stylesheet

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xalan-nodeset="http://xml.apache.org/xalan" xmlns:java="http://xml.apache.org/xslt/java" xmlns:xalan="http://xml.apache.org/xalan" exclude-result-prefixes="xalan" xmlns:jsp="jsp"> z

Establish a session

<xsl:param name="DMS_SESSION_ID" select="'default value'"/> <xsl:param name="DMS_INPUT_OBJECT_ID" select="'default

value'"/>

<xsl:variable name="clientX"

select="java:com.documentum.com.DfClientX.new()"/> <xsl:variable name="client"

select="java:getLocalClient($clientX)"/>

<xsl:variable name="session" select="java:findSession($client, $DMS_SESSION_ID)"/>

(10)

55 55

Integrating DFC code in XSL Transforms

z

Set object to XSL variable and make DFC calls

<xsl:variable name="persistentObject" select= "java:getObjectByQualification($session, $qualificationQuery)"/>

<!– Get Subject value --> <xsl:variable name=“sSubject"

select="java:getString($persistentObject, ‘subject')"/>

z

DFC method definitions:

public com.documentum.fc.client.IDfPersistentObject getObjectByQualification(String qualification) throws DfException

public String getString(String attrName) throws DfException

Call DFC within an XSL Transform:

56 56

Integrating DFC code in XSL Transforms

z

Set an xsl:variable to a new object instance

<xsl:variable name=“oXsltHelper"

select="java:com.company.common.xslt.Helper.new()"/>

z

Call Methods with oXsltHelper, the DFC Session, and

required arguments

<xsl:variable name="oSysObject"

select="java:getSysObjectFromObjectId($oXsltHelper,$session, $sObjectType,$sObjectId)"/>

z

Method Definition

public IDfSysObject getSysObjectFromObjectId(IDfSession oSession, String sObjectType, String sObjectId) throws DfException

Call your own DFC classes within an XSL Transform:

57 57

Advanced XML Seminar

z Remember where XML developer resources can be found

–Documentum Developers

http://customernet.documentum.com/developer/index.ht ml

–Flatirons Solutions Developer Zone

http://www.flatironssolutions.com/Developers.htm

–Documentum Support Site Groups (XML)

–Yahoogroups Documentum XML newsgroup

http://groups.yahoo.com(Documentum-xml) z

Questions?

Summary

58 58