Developing Ensemble Productions

(1)

Developing Ensemble

Productions

Version 2009.1

30 June 2009

(2)

This book was assembled and formatted in Adobe Page Description Format (PDF) using tools and information from the following sources: Sun Microsystems, RenderX, Inc., Adobe Systems, and the World Wide Web Consortium at www.w3c.org. The primary document development tools were special-purpose XML-processing applications built by InterSystems using Caché and Java.

and

Caché WEBLINK, Distributed Cache Protocol, M/SQL, M/NET, and M/PACT are registered trademarks of InterSystems Corporation.

, , and

InterSystems Jalapeño Technology, Enterprise Cache Protocol, ECP, and InterSystems Zen are trademarks of InterSystems Corporation.

All other brand or product names used herein are trademarks or registered trademarks of their respective companies or organizations.

This document contains trade secret and confidential information which is the property of InterSystems Corporation, One Memorial Drive, Cambridge, MA 02142, or its affiliates, and is furnished for the sole purpose of the operation and maintenance of the products of InterSystems Corporation. No part of this publication is to be used for any other purpose, and this publication is not to be reproduced, copied, disclosed, transmitted, stored in a retrieval system or translated into any human or computer language, in any form, by any means, in whole or in part, without the express prior written consent of InterSystems Corporation.

The copying, use and disposition of this document and the software programs described herein is prohibited except to the limited extent set forth in the standard software license agreement(s) of InterSystems Corporation covering such programs and related documentation. InterSystems Corporation makes no representations and warranties concerning such software programs other than those set forth in such standard software license agreement(s). In addition, the liability of InterSystems Corporation for any losses or damages relating to or arising out of the use of such software programs is limited in the manner set forth in such standard software license agreement(s). THE FOREGOING IS A GENERAL SUMMARY OF THE RESTRICTIONS AND LIMITATIONS IMPOSED BY INTERSYSTEMS CORPORATION ON THE USE OF, AND LIABILITY ARISING FROM, ITS COMPUTER SOFTWARE. FOR COMPLETE INFORMATION REFERENCE SHOULD BE MADE TO THE STANDARD SOFTWARE LICENSE AGREEMENT(S) OF INTERSYSTEMS CORPORATION, COPIES OF WHICH WILL BE MADE AVAILABLE UPON REQUEST.

InterSystems Corporation disclaims responsibility for errors which may appear in this document, and it reserves the right, in its sole discretion and without notice, to make substitutions and modifications in the products and practices described in this document.

For Support questions about any InterSystems products, contact:

InterSystems Worldwide Customer Support

+1 617 621-0700 Tel:

+1 617 374-9391 Fax:

[email protected] Email:

(3)

List of Figures

Overview of an Ensemble Production... 4

Example of Settings on the Configuration Page... 30

BPL <call> Element Showing <annotation> Text... 66

Ensemble Message Object with a Standard Message Body... 74

Typical Request and Response... 77

Request and Deferred Response... 78

How a Production Accepts Incoming Requests... 86

How a Production Relays Incoming Requests... 94

How a Production Orchestrates Incoming Requests... 108

Example of a BPL Diagram... 126

BPL Diagram with Join Selected, Related <if> Highlighted... 131

A New Shape Added to a BPL Diagram... 132

Connection Labels in a Switch Element... 133

Auto-Arranged Shapes in a BPL Diagram... 134

Example of a BPL Loop... 137

Example of a BPL Loop, Drilled Down... 138

Example of a BPL Sequence... 139

Example of a BPL Sequence, Drilled Down... 139

BPL Toolbar... 140

How a Production Fulfills Incoming Requests... 146

How a Production Relays Outgoing Requests... 159

The Data Transformation Utility Object... 167

DTL Visual Editor, Diagram and Code View... 171

DTL Visual Editor with Inspector Window... 173

DTL Visual Editor, Class Code View... 174

DTL Visual Editor, Generated Code View... 175

Connector from Source to Target... 176

Connector from Literal Value to Property in Target... 177

DTL Visual Editor <assign> Wizard... 177

Connector from Source to Parent Property in Target... 179

Connector from Source to Nested Property in Target... 179

Connector Between Two Identical Parent Properties... 180

DTL Visual Editor <if> Wizard... 181

Connector from Source Property to If Statement... 181

DTL Visual Editor <foreach> Wizard... 182

Connector from Source Property to Foreach Statement... 183

(9)

(10)

List of Tables

Production States... 11

Utility Functions for Business Rules and DTL Data Transformations... 44

Examples of Valid Expressions using Functions... 47

Time Stamp Symbols for Input and Output Filenames... 53

Time Stamp Symbols Different from POSIX, IEEE, and ISO Standards... 57

BPL Diagram Shapes... 128

BPL Diagram Icons... 129

Icons that Add Activities to a BPL Diagram... 140

Icons that Add Loops to a BPL Diagram... 141

Icons that Add Logic to a BPL Diagram... 141

Business Operation Properties... 153

Classes for Publish and Subscribe Messaging... 197

(11)

About This Book

This book explains how to plan, develop, and test enterprise integration solutions using Ensemble. InterSystems Ensemble shares many underlying core technologies with InterSystems Caché. This book refers you to books in both documentation sets.

This book contains the following chapters:

• “Production Concepts” outlines the architecture and life cycle of an Ensemble production. • “Creating a New Production” introduces Ensemble development tools and procedures. • “Ensemble Messages” explains the essential unit of exchange within an Ensemble production. • “Inbound Adapters” shows how a production receives and validates requests from external systems. • “Business Services” shows how a production directs incoming requests for fulfillment.

• “Business Processes” explains the essential core of activity within an Ensemble production. • “Business Operations” shows how a production represents an external system as a resource. • “Outbound Adapters” shows how a production constructs and sends requests to external systems. • “Production Data” explains data transformation, abstraction, and persistence.

• “Business Logic” includes rule-based decisions, dashboards, and human-computer workflow. • “Message Routing” shows how to develop a production as a fast, reliable message routing engine.

There is also a detailed Table of Contents. The following books provide related information:

• Introducing Ensemble provides an overview of the Ensemble product, its purpose and features. • Ensemble Release Notes describes the contents of this release of Ensemble.

• Ensemble Best Practices describes best practices for organizing and developing Ensemble produc-tions.

(12)

(13)

1

Production Concepts

An Ensemble production is a specialized package of software and documentation that solves a specific integration problem for an enterprise customer. The goal of any Ensemble development project is to deliver a production. This book explains how to generate the class code, XML, and custom software elements that comprise an Ensemble production.

The following figure illustrates some of the software elements that an Ensemble production can contain. Within an Ensemble development project, the developers’ job is to create some of these elements, configure all of them to work together, and leave documentation and tools to ensure that the production can be managed by the enterprise system administration team.

In addition to the elements that developers create, the Ensemble production infrastructure includes elements that support messaging, monitoring, persistence, management, tracking, recovery, caching, and storage. The diagram does not show these elements. It acknowledges them with arrows that connect the elements Inside Ensemble. The only requirement to take full advantage of these features is to use the Ensemble classes, tools, and wizards. The Ensemble development and management environments take care of these infrastructure details automatically.

(14)

Overview of an Ensemble Production

An Ensemble production typically processes incoming events in the following order, using the following elements. Alternative processing is possible, and some of these alternatives are indicated in the diagram, but this is the classic sequence:

1. An inbound adapter receives an incoming event, transforms it into an Ensemble message object, and passes it to its associated business service.

2. The business service formulates a follow-on request message, and passes this new message to a business process or business operation within Ensemble.

(15)

3. A business process that receives a request message executes a predefined set of activities, in sequence or in parallel. These activities may include sending follow-on requests to other business processes, to business rules, to data transformations, or to business operations. For example: • Business rules change the behavior of business processes at decision points.

• Routing rules direct messages to their destinations based on message type, message contents,

or where the message came from.

• Schema categories provide a means to validate and access message contents.

• Data transformations calculate and apply changes to message contents.

• Business operations map a request to functionality offered by an entity outside Ensemble.

Business services, business processes, and business operations are all called business hosts. 4. A business operation encapsulates the capabilities of a resource outside Ensemble, usually an

external software application. The business operation transforms properties of the Ensemble request message object into a format usable by the external application API. If the request requires human intervention, a workflow operation manages the interaction between person and computer. 5. An outbound adapter manages the details of communicating with a specific external system or

application from within Ensemble. It transmits the API call to the external entity.

6. The response from the external system or application can trigger a cascade of response messages back to the external entity that instigated the flow of events. Details depend on the design choices made by the production developers, as later chapters in this book explain.

The following sections further explain additional production concepts: • Requests and Responses

• External Interfaces

• Namespaces

• Host Classes

• Configuration Settings

• Startup

• Shutdown

• Production States

• Production Life Cycle

• Jobs

• Pool Size

(16)

• Scheduler

• Business Logic

1.1 Requests and Responses

The “Ensemble Messages” chapter explores message objects in detail. In the current chapter, it is important only to understand that:

• Every message is a request or a response. • Requests and responses are defined in pairs.

• A request may have one associated response, or no associated response.

• Any request may be sent synchronously (the caller waits for the response before continuing) or asynchronously (the caller does not wait). The business host that sends the request must indicate this choice and handle the result.

• Any request that is sent synchronously must, in fact, have a designated response type.

• The flow of control described at the beginning of this chapter could vary considerably each time a request is sent, depending on a variety of factors including:

- The type of request being sent.

- Whether or not each request involves waiting (or not waiting) for a response. - Whether or not expected responses are actually received.

- The values returned with any responses.

1.2 External Interfaces

A traditional way to begin work on an Ensemble solution is to list all of the externally and internally callable interfaces within the planned solution. For example, in the diagram at the beginning of this chapter, all the solid arrows in the gray-shaded area are considered to be Outside Ensemble. Some of these arrows indicate a call into the Ensemble production, and some indicate a call out from the pro-duction. These arrows comprise a list of interfaces that is called the processing model for this produc-tion.

Within the processing model for an Ensemble production, external applications that call into the pro-duction are known as client applications; these are shown at left in the diagram. External applications

(17)

whose operations are called by elements inside the production are known as external applications; these are shown at right.

Within the same production, the same application can serve as both client and external application. In this case, the distinction between the two roles depends on whether the application is initiating a request (client) or fulfilling a request (external). As with applications, calls from Ensemble out to external

databases are possible as well.

1.3 Namespaces

In InterSystems products, a namespace is a collection of data and programs in a virtual work space. InterSystems documentation provides a great deal of information about namespaces.

Important: The Ensemble installation procedure creates several namespaces for internal use by the Ensemble engine: %SYS, DOCBOOK, USER, SAMPLES, ENSLIB, ENSEMBLE, and ENSDEMO. All system-provided namespaces except ENSEMBLE and USER are overwritten upon reinstallation or upgrade. For this reason, InterSystems recommends that you always create new namespaces in which to work, rather than placing custom code in any of these system-provided namespaces, where it could be overwritten and lost.

This book frequently refers to something called an Ensemble namespace or an Ensemble-enabled

namespace. This is a namespace that has the Ensemble classes loaded into it. Of the system-provided

namespaces, only ENSLIB, ENSEMBLE, and ENSDEMO are Ensemble-enabled. Only ENSEMBLE is intended for your use; the other system-provided namespaces are reserved.

Once you have successfully installed Ensemble, any new namespace that you create is automatically Ensemble-enabled. You can create a new namespace by using the System Management Portal [Home] > [Configuration] > [Namespaces] > [New Namespace] page. For instructions, see the Configuring Data

section in the “Configuring Caché” chapter of the Caché System Administration Guide.

1.4 Host Classes

An Ensemble host class consists of executable source code. Class developers use the Studio tool, the BPL Editor, or a text editor to create, view, edit, and compile host classes. Every host class extends

Ens.Host, an Ensemble class that determines general characteristics for classes that operate as members of a production. Every host class is one of three types, all of which inherit from Ens.Host:

• Business service class — Instances of the Ens.BusinessService class accept requests from entities outside Ensemble and relay them to host classes inside Ensemble for processing and fulfillment.

(18)

A business service class corresponds to a single, well-defined action that an external entity needs to perform. This action enters an Ensemble production in the form of a request to the business service. The business service generates whatever activity is needed to satisfy this request. It invokes various lower-level host classes — business processes and business operations — and it controls the logic of these invocations. When all of its internal processing is complete, a business service formulates and returns a response to the outside entity’s original request. Some requests do not require a response; if this is the case, none is returned.

For details, see the “Business Services” chapter.

• Business process class — Instances of the Ens.BusinessProcess class accept requests from host classes inside Ensemble — business services or business processes — and either process the requests or relay them to other host classes inside Ensemble for processing. A business process class corresponds to a single, well-defined business task that needs to be performed. The business process generates whatever activity is needed to complete this task. It invokes various lower-level host classes — business processes or business operations — and it controls the logic of these invocations. When all of its internal processing is complete, a business process formulates and returns a response to the original request, if a response is needed. Some requests do not require a response; if this is the case, none is returned.

For details, see the “Business Processes” chapter.

• Business operation class — Instances of the Ens.BusinessOperation class accept requests from host classes inside Ensemble — business services or business processes — and either process the requests or relay them to entities outside Ensemble for processing. A business operation class consists of one or more methods, each of which corresponds to a single, well-defined external or internal operation that needs to be performed, such as retrieving a specific piece of data or making a specific type of calculation. When its internal processing is complete, a business process formu-lates and returns a response to the original request, if a response is needed. Some requests do not require a response; if this is the case, none is returned. By convention, a business operation is an extremely specific operation that contains very little logic and does what is requested of it without calling further operations or branching in any way. When the design of the production demands logic, this is contained in a higher-level business process.

For details, see the “Business Operations” chapter.

1.5 Configuration Settings

You must compile and load a host class before you can add the corresponding business host — business service, business process, or business operation — to the production. Once you add the business host to the production, you can adjust its configurable settings using the configuration display on the

(19)

For a description of the configurable settings that are built into the base classes for productions, business services, business processes, and business operations, see the lists and descriptions in the following sections of the “Configuration” chapter of Managing Ensemble Productions:

• Business Service Settings — includes settings for inbound adapters • Business Process Settings

• Business Operation Settings — includes settings for outbound adapters • Production Settings

A host class determines how a configuration item can be configured, by publishing its configurable properties in a class parameter called SETTINGS. Any property listed in the SETTINGS parameter for a host class automatically appears in the configuration display on the [Ensemble] > [Productions] page, whenever you select an item of that host class for configuration.

You can edit and save values on the [Ensemble] > [Productions] page without editing or recompiling the host class itself. Any changes you make are applied immediately, without requiring you to stop and restart a running production. The same is true of production-wide properties, which are defined in the production class and exposed for configuration by using the SETTINGS class parameter in the production class.

For programming instructions regarding the SETTINGS parameter, see the Configuration Settings

section in the “Creating a New Production” chapter.

1.6 Startup

This topic describes what happens when an Ensemble production starts up. The sequence is the same regardless of the mechanism used to start it. The Production Life Cycle section describes options for invoking this sequence.

When a production starts, the sequence of actions is as follows:

1. The production class is instantiated; its optional OnStart method executes.

2. The production instantiates each business operation and executes its optional OnProductionStart method.

3. The production instantiates each business process and executes its optional OnProductionStart method.

4. The production clears the business metric cache of any metric values left over from a previous run.

5. The production instantiates each business service and executes its optional OnProductionStart method.

(20)

6. The production processes any items already placed in queues. This includes messages with Normal priority (asynchronous messages) that were queued when the production stopped.

7. The production now accepts input from outside Ensemble.

1.7 Shutdown

This topic describes what happens when an Ensemble production shuts down. The sequence is the same regardless of the mechanism used to stop it. The Production Life Cycle section describes options for invoking this sequence.

When a production stops, the sequence of actions is as follows:

1. The production takes each business service offline and executes its optional OnProductionStop method. This action stops all requests from outside Ensemble.

2. All business hosts receive a signal to become Quiescent.

3. All queues go into a Quiescent state. This means that from this point forward, business hosts can only process queued messages with High priority (synchronous messages). Messages with Normal priority (asynchronous messages) remain on their respective queues.

4. The production finishes processing all synchronous messages to the best of its ability.

5. The production takes each business process offline and executes its optional OnProductionStop method.

6. The production takes each business operation offline and executes its optional OnProductionStop method.

7. The production goes offline. Ensemble executes the optional OnStop method in the production class.

1.8 Production States

During its life cycle, a production may enter any of the states listed in the following table. The corre-sponding status values are visible using the [Ensemble] home page or on the [Ensemble] > [Productions]

(21)

Production States Meaning

State

When a production has been started and is operating normally, it has a status of Running. This is an acceptable state.

Running

A production acquires a status of Stopped when, at the end of the shutdown sequence, all of its queues are free of synchronous messages. This is an acceptable state.

Stopped

A production acquires the Suspended status if, at the end of the shutdown sequence, some queues still contain synchronous messages, waiting for a response. Depending on how the production has been designed, this may or may not indicate a problem.

You may start a Suspended production. Starting the production again may permit the waiting messages to be processed. However, if you start the Suspended production, and more messages arrive while those in front of them in the queue cannot be resolved, queues may simply continue to fill. In that case you must investigate to discover why the incoming mes-sages never complete.

You may stop a Suspended production. InterSystems advises you do so only during initial development and not on a deployed, live installation. For instructions, see the Stopping a Suspended Production section in the “Creating a New Production” chapter.

Suspended

A production acquires a status of Troubled if Ensemble is stopped but the production did not shut down properly. This can happen if you restarted Ensemble or rebooted the machine without first stopping the production. In this case you need to issue a command to recover the production. For instructions, see the next section, Production Life Cycle.

Troubled

1.9 Production Life Cycle

This topic introduces several ways to start or stop an Ensemble production. It also explains how these techniques fit within the larger context of starting or stopping the Ensemble server instance where the production runs. At server startup, the major pieces start up in the following order:

1. The Ensemble server instance starts up.

The user startup routine runs automatically at this time, but InterSystems recommends you do not use the ^%ZSTART routine to control Ensemble production startup. The mechanisms that

(22)

Ensemble provides to control startup are much easier to use and more closely tied to the production itself. See Ensemble Auto-start Settings for details.

2. At any subsequent time, you may start an Ensemble production.

The Startup section, earlier in this chapter describes the full production startup sequence. Ensemble automatically runs the production class OnStart method and the OnProductionStart methods of the business host classes, if provided. These methods are the recommended locations for any special setup instructions that you want to provide for the production, or for individual business hosts, when Ensemble starts up. See OnStart and OnProductionStart for details.

1.9.1 Ensemble Auto-start Settings

You can use the Ensemble Management Portal page to configure Ensemble so that a specific production starts automatically at system startup, and stops automatically at system shutdown. The setting for this is called Auto-Start Production. InterSystems recommends that you use this setting to configure live, deployed productions to start when the server starts up:

1. From the [Ensemble] home page click Maintenance on the Ensemble menu to display the

[Ensemble] > [Maintenance] page.

2. In the Auto-Start Production list, choose the production name you want to automatically start when Ensemble starts.

3. To disable automatic production start, click the blank entry; this is the default.

For debugging situations in a development environment, you can override the automatic start by dis-abling the EnsembleAutoStart configuration setting:

1. From the InterSystems Systems Management Portal [Home] page, click Configuration under the

System Administration column.

2. Click Startup Settings under the Additional Settings column. 3. Click Edit next to the EnsembleAutoStart setting.

4. Clear the EnsembleAutoStart check box to disable the feature and click Save.

When you restart Ensemble with this setting disabled, the productions you have set to automatically start at system startup do not start.

1.9.2 Using the Portal to Start and Stop a Production

You can also use the Ensemble Management Portal to start or stop an Ensemble production manually, by selecting the production on the [Ensemble] > [Productions] page and clicking Start or Stop to the right of it.

(23)

While you are developing new production code, it sometimes happens that when you start the Ensemble Management Portal, Ensemble detects a discrepancy between the production as defined in the code, and the status of the running production. Examples would be if:

• A specific business host experienced an error and has died (the most likely cause).

• You changed a configuration parameter for a business host in a running production, such that the host now needs to be restarted.

• A production typically opens a Terminal window, but this window was closed by a user action while the production was still running.

When discrepancies such as this arise, the Update Production command becomes available on the

[Ensemble] home page. Click it, and Ensemble updates the production to resolve the discrepancy. Ensemble displays a confirmation message. Click OK to return to the [Ensemble] home page. For a deployed production, you can set up the production to update itself automatically. When you take this approach, Update Production never displays. To do this, add a production monitor service to the production. This is a business service with a Service Class of Ens.ProductionMonitorService. Con-figure the Call Interval setting to control how frequently, in seconds, the monitor service checks to see if there is a discrepancy between the production as defined in the code, and the status of the running production. Each time it finds such a discrepancy, the monitor service updates the production to resolve it.

If a production has a status of Troubled due to some unexpected program error, a Recover command appears on the Ensemble Management Portal [Ensemble] home page. Click Recover to shut down and clean up the troubled instance of the production so that you can run a new instance when you are ready.

1.9.3 OnStart and OnProductionStart

If you have code that must execute before a production starts up, but that requires the Ensemble framework to be running before it can execute, you must override the OnStart method in the production class. Place these code statements in OnStart so that they execute in the proper sequence: that is, after Ensemble has started, but before the production begins accepting requests. An OnStop method is also available to perform a set of tasks before the production finishes shutting down.

Each business host — business service, business process, or business operation — is a subclass of

Ens.Host. In any of these classes you may override the OnProductionStart method to provide code statements that you want Ensemble to execute on behalf of this host at production startup time. An

OnProductionStop method is also available.

Neither OnStart nor OnProductionStart are required, but they are available if you need them.

(24)

1.9.4 Ens.Director

It is possible to start or stop a production at the Terminal command line or from an ObjectScript routine. You can do this by invoking the following methods in the Ens.Director class. These methods work only if the current namespace is an Ensemble namespace:

StopProduction

Stop the currently running production in an Ensemble namespace:

Do ##class(Ens.Director).StopProduction()

StartProduction

Start the specified production in the Ensemble namespace, as long as no other production is running:

Do ##class(Ens.Director).StartProduction("myProduction")

RecoverProduction

Clean up a Troubled instance of a running production so that you can run a new instance in the Ensemble namespace:

Do ##class(Ens.Director).RecoverProduction()

There is no need to call GetProductionStatus to see if the production terminated abnormally prior to calling RecoverProduction. If the production is not Troubled, the method simply returns.

GetProductionStatus

This method returns the production status via two output parameters, both of which are passed by reference. The first parameter returns the production name, but only when the status is Running, Suspended, or Troubled. The second parameter returns the production state, which is a numeric value equivalent to one of the following constants:

• $$$eProductionStateRunning • $$$eProductionStateStopped • $$$eProductionStateSuspended • $$$eProductionStateTroubled

(25)

Set tSC=##class(Ens.Director).GetProductionStatus(.tProductionName,.tState) Quit:$$$ISERR(tSC)

If tState'=$$$eProductionStateRunning {

$$$LOGINFO($$$Text("No Production is running.")) Quit }

You can use the production state macros such as $$$eProductionStateRunning in code outside of the Ensemble classes, for example in a general class or routine. To do this, you must add the following statement to the class:

#include Ensemble

There is no need to do this inside Ensemble classes, such as in business hosts.

If you browse the online class reference documentation for Ens.Director, you can see that it offers many class methods, including many intended for use only by the Ensemble internal framework. InterSystems recommends that you use only the Ens.Director methods documented in this book, and only as docu-mented. For other recommended Ens.Director methods, see:

• Accessing Production Settings Programmatically in the “Creating a New Production” chapter • Invoking a Business Service Directly in the “Business Services” chapter

1.10 Jobs

Jobs are CPU processes that host the work done by an Ensemble production. This terminology is

intended to avoid confusion between CPU processes ( “jobs” ) and business processes ( “processes” ). From a low-level, system viewpoint, an Ensemble production consists almost entirely of jobs waiting to wake up to perform work. When a request arrives on the queue of a business host, the item claims a job from its pool of available jobs. The job then hosts the work that needs to be done. If no job is available in the pool of jobs for that business host, it must wait for a job to become available, so that it can do the work. The job returns to the pool after the work is done, or whenever the next task is to wait for something to happen. A job always returns to the same pool it came from.

1.11 Pool Size

The choice of Actor Pool Size for the production, and Pool Size for each business host, determines how many jobs are available to perform which types of work for the production. These numbers are an essential part of the production design and are unlikely to need adjustment once the production is deployed live. Larger numbers are not necessarily helpful; while the potential pool sizes actually range from 0–100, most pool sizes are best set to either 0 or 1, and there can be serious consequences when sizes are set to a number greater than 1. The following discussion provides details.

(26)

1.11.1 Private Pool Size for Business Hosts

Each business service, business process, or business operation can have its own, private pool of allocated jobs. You can configure the size of this pool using the settings in the configuration display on the

[Ensemble] > [Productions] page. The maximum setting for any one pool is 100 jobs. If, in testing your production, you find that you “need” more than 100 jobs per business host it probably indicates some other kind of problem, such as a bottleneck or deadlock that should be addressed. If it is a purely computational bottleneck then there is no point in having the pool size larger than the number of CPUs.

1.11.2 Actor Pool Size for Productions

Unlike other types of business host, with business processes you have the option of sharing jobs from a public pool; this pool is called the actor pool. You can configure the Actor Pool Size for the production as a whole, using the configuration display on the [Ensemble] > [Productions] page. Actors in the pro-duction-wide actor pool have no affiliation with or knowledge of a specific business process. Any business process that has a private Pool Size of 0 can use jobs from the public actor pool.

The choice as to whether or not a business process should use its own private pool or the public pool depends on the needs of the production. Any non-zero value for a private Pool Size ensures that the business process only uses jobs from its private pool. If you want any business process to use jobs from the actor pool, its private Pool Sizemust be 0. The default Ensemble configuration allows 1 job

in the private pool for each business host, and 2 jobs in the production-wide actor pool. This means that if you want business processes to share the actor pool it does not happen automatically; you must set their individual Pool Size settings to 0.

You must use a Pool Size of 0 for an adapterless business service. This is a business service that is invoked directly from outside Ensemble, rather than receiving its requests in the usual way, via an inbound adapter. An adapterless business service may be invoked via Caché Language Bindings, Caché Server Pages, SOAP, or a routine invoked from the operating system level. For details, see “Invoking a Business Service Directly” in the chapter “Business Services.”

1.11.3 First-In First-Out (FIFO) for Health Care

A private Pool Size of 1 ensures first-in, first-out (FIFO) processing. If all the business hosts in the production have only one job available, only one message can be processed at a time by each host. This gives each message from a given source only one possible path through the production, so each message is guaranteed to arrive at its configured destination in the same order in which it was sent. Without guaranteed FIFO, a message from a particular source could “skip over” other messages from the same source by using a faster, parallel job to arrive at its destination sooner.

FIFO is essential for health care applications. Suppose a patient enters a hospital and requires care. System A sends out an admit event, followed by a treatment order, but System B receives the order first. System B cannot process the order without an admit, so upon receiving the order, it produces an

(27)

error. This may delay patient care or require the information system to execute complex logic to associate admit with the order after the admit finally arrives at System B.

For this reason, InterSystems recommends that every business service, business process, and business operation in a message routing production for health care has its Pool Size setting configured to 1, with the Actor Pool Size for the production at 0.

1.11.4 Considerations and Trade-offs

In situations other than health care, private pools of a size 1 or greater can be useful for fast-running business processes in a production that also includes slow-running business processes. A fast-running business process can have a private pool to ensure that its requests never get stuck in the public actor queue behind accumulated requests for the slow-running business processes.

If every business process in your production has a private pool, then the Actor Pool Size for the produc-tion can be 0. On the other hand, if your producproduc-tion includes many business processes that use the public actor pool, you can raise the Actor Pool Size for the production from the default of 2 to prevent bottlenecks when many business processes are running. InterSystems recommends that, as a maximum, you set the Actor Pool Size equal to the number of CPUs in your Ensemble server machine. You could set the number higher, but at any one time there are only as many jobs available as there are CPUs.

1.12 Events

System events occur naturally throughout the life cycle of a production. They are generated and handled

internally by Ensemble. Among the ways that system events are used is putting background processes (jobs) to sleep and waking them up. System events are also involved in notification of the arrival of messages on a queue; an event tells the queue owner that a message has arrived, so that the owner knows to take the message from the queue.

You do not need a detailed knowledge of system events to develop an Ensemble production. System events are handled automatically by the Ensemble infrastructure. The truly important thing to remember is that system events are not the same as alerts, alarms, event log entries, or Ensemble messages.

The Ensemble event log does not provide any information about system events. The event log collects certain types of text message generated by host classes. A developer or system administrator can view these event log entries from the Ensemble Management Portal [Ensemble] > [Event Log] page. Viewing the event log is a way to “take the pulse” of a production by scanning the informational text messages that it produces while it runs. Event log entries are stored persistently in the Ensemble database and may be purged according to age, as they accumulate.

There are some automatic event log entries, such as the entries that acknowledge production startup or shutdown. These reflect normal occurrences in the life cycle of the production.

(28)

However, most event log entries are present in the log because the developer of an Ensemble class decided that certain information would be of interest to a system administrator in case of a problem while the production is running.

Important: For instructions, see the “Log, Alert, and Trace” section in the chapter “Creating a New Production” .

1.13 Scheduler

The default scheduling for business hosts is for them to run whenever the production is running. However, a finer control is possible. Not only can you enable and disable business hosts, but the con-figuration display on the [Ensemble] > [Productions] page also offers a Schedule field for configuring each business host. This is an optional command string that schedules the host to be stopped and started at specific times and on specific days of the week, month, or year. When it is time to start, if the host is enabled it will be started; when it is time to stop, if the host is running it will be stopped.

There are some limitations to scheduling abilities. For example, if a business host is started by the scheduler but cannot be stopped by the scheduler, this is because the host is in the middle of a syn-chronous call. The host must wait for a response to the call before it can be stopped.

For details, see the Scheduling Configuration Items section in the “What to Manage” chapter of

Managing Ensemble Productions.

1.14 Business Logic

Ensemble offers several features that enable non-technical users to assess and direct the flow of application logic in order to meet business goals. The chapter “Business Logic” describes them: • Business rules allow non-technical users to change the behavior of Ensemble business processes

at specific decision points. The logic of the rule can be changed instantly, using a simple forms-based interface in the Ensemble Management Portal. There is no need for programming or dia-gramming skills to change the rule, and there is no need to modify or compile production code for changes to take effect. See “Business Rules.”

• Ensemble supports business activity monitoring (BAM) by enabling you to develop browser-based “corporate dashboards” that display graphical, real-time information about the production. Each dashboard is associated with a specific production, displays values from that production, and can be viewed “live” from the Ensemble Management Portal whenever that production is running. See “Business Activity Monitoring.”

(29)

• Workflow makes it possible to incorporate human interaction into automated business processes.

Typical uses of workflow within the enterprise might include order entry, order fulfillment, contract approval, and help desk activities. A workflow management system automates the distribution of tasks among users. Distributing tasks automatically according to a predefined strategy makes task assignment more efficient and task execution more accountable. Ensemble has a built-in Workflow Engine that supports a variety of default and custom distribution strategies for tasks and users. See “Workflow.”

(30)

(31)

2

Creating a New Production

There are two ways to create a new Ensemble production.

• From the Ensemble Management Portal, using the Production Model

• From Ensemble Studio, using the New Production Wizard

Ensemble provides many resources and tools to help you complete the new production: • Configuration Settings

• Username and Password Credentials

• Log, Alert, and Trace

• Utility Functions

• Time Stamps in Filenames

• Testing and Debugging

• Documentation and Comments

• Ensemble Studio

2.1 Using the Production Model

You can construct a new production from the Ensemble Management Portal as follows:

1. Choose your working namespace from the selection box at the top right of the [Ensemble] home page.

(32)

3. Click Create New Production. The [Ensemble] > [Productions] > [Production Wizard] page displays. 4. Enter a Package Name, Production Name, and Description.

5. Choose one of the following:

• HL7 Messaging — If you are using HL7 with Ensemble, choose this option. You can add any generic Ensemble elements that you need as you build the HL7 production, but choosing this option provides a useful set of “starter elements” for HL7.

• Generic — Choose this option for all other types of production.

6. Click OK. The [Ensemble] > [Productions] > [Production Model] page displays.

The Production Model page offers the following menu options:

• Add Business Service — Click this link to start the Business Service Wizard. Choose: - HL7 Input

• Choose TCP, File, or FTP to determine the host class. Each class already exists and requires no programming. Simply choose one.

• Give the item a configuration Name. Do not use the characters ;:|,

• Use the Target Name field to identify the business process or business operation to which this business service will send the HL7 messages that it receives.

- Business Metric

• Choose a host class from the MetricClass drop-down list. If the class you need does not appear on this list, create the class in Studio as described in the chapter “Business Ser-vices,” then return to the [Ensemble] > [Productions] > [Production Model] page to choose it.

• Enter a text label in the Category field to sort and organize items within the production.

Category names are case-sensitive, and space characters are allowed. To place an item in multiple categories, list them in the Category field separated by commas (do not allow spaces around these commas).

• Comment is an optional text description.

• Call Interval determines how often the business metric will recalculate the values of its properties. The Call Interval is in seconds, starting from a minimum of 0.1 seconds. The default is 5 seconds.

(33)

• Choose a host class from the ServiceClass drop-down list. If the class you need does not appear on this list, create the class in Studio as described in the chapter “Business Ser-vices,” then return to the [Ensemble] > [Productions] > [Production Model] page to choose it.

Click OK to save your changes, Cancel to ignore them.

Most business services have a Call Interval and other operational settings whose values you must set from the configuration display on the [Ensemble] > [Productions] page. To navigate to the configuration display from the [Ensemble] > [Productions] > [Production Model] page, click Con-figure Production.

• Add Business Process — Click this link to start the Business Process Wizard. Choose:

- Business Process Component — Adds a business process that you have set up to work as a

reusable component as described in the chapter “Business Processes.”

• Choose a host class from the Component drop-down list. If the class you need does not appear on this list, create the class in Studio, then return to the [Ensemble] > [Productions] > [Production Model] page to choose it.

- HL7 Message Router — Adds a business process that routes HL7 message data. For details, see the Ensemble HL7 Version 2 Development Guide.

• Accept the default Router Class of EnsLib.HL7.MsgRouter.RoutingEngine. • Give the item a configuration Name. Do not use the characters ;:|,

• Use the Routing Rule Name field to identify the routing rule set to which this business process will send the messages that it receives.

(34)

- Other — Adds a normal business process.

• Choose a host class from the ProcessClass drop-down list. If the class you need does not appear on this list, create the class in Studio as described in the chapter “Business Processes,” then return to the [Ensemble] > [Productions] > [Production Model] page to choose it.

Click OK to save your changes, Cancel to ignore them.

Most business processes have an Inactivity Timeout, Alert On Error, and other settings that you must set from the configuration display on the [Ensemble] > [Productions] page. To navigate to the configuration display from the [Ensemble] > [Productions] > [Production Model] page, click

Configure Production.

• Add Business Operation — Click this link to start the Business Operation Wizard. Choose: - HL7 Output

• Choose TCP, File, or FTP to determine the host class. Each class already exists and requires no programming. Simply choose one.

- Other

• Choose a host class from the OperationClass drop-down list. If the class you need does not appear on this list, create the class in Studio as described in the chapter “Business Operations,” then return to the [Ensemble] > [Productions] > [Production Model] page to choose it.

(35)

Most business operations have a Retry Interval, Failure Timeout, and other operational settings whose values you must set from the configuration display on the [Ensemble] > [Productions] page. To navigate to the configuration display from the [Ensemble] > [Productions] > [Production Model]

page, click Configure Production.

• Remove Item — Delete the business service, business process, or business operation that is currently selected in the production model.

• Save Production — Save any changes made in the current editing session.

• Configure Production — Display the [Ensemble] > [Productions] page so that you can provide configuration details for items that you have added.

• Cancel — Revert to the previously saved version of the production model.

• Delete Production — Delete the production that is currently displayed on the [Ensemble] > [Pro-ductions] > [Production Model] page. A confirmation dialog displays. To permanently delete the production with no chance of recovery, click OK. The production and all of its configured settings are removed. The underlying classes for business hosts and data transformations remain.

Important: For details regarding the Ensemble Management Portal, including options for adding, configuring, and removing productions and their elements, see Managing Ensemble Productions.

2.2 Using the New Production Wizard

The Studio tool supports the development of Ensemble production software in a number of ways. This section and the “Ensemble Studio” section later in this chapter describe the Studio features that are unique to Ensemble. The book Using Caché Studio documents the Studio features that Ensemble and Caché products share.

You can create a new Ensemble production from inside the Studio tool as follows: 1. Start Studio and choose an Ensemble-enabled user namespace.

Important: InterSystems recommends that you do not place custom code or data in the system-provided namespaces ENSLIB or ENSDEMO where it will be deleted the next time you upgrade Ensemble. The ENSEMBLE namespace and any new namespace that you create to hold your work is preserved across Ensemble upgrades.

2. From the File menu select New.

3. The New dialog displays. Select the Production tab. 4. Select the Production icon and click OK.

(36)

5. Enter a package and class name for the production. Click Next.

Important: InterSystems recommends that you do not use the package names Demo, Ens,

EnsLib, EnsPortal, or CSPX. This causes your work to be deleted the next time you upgrade Ensemble.

6. To enable testing for the production, select the Enable Testing check box.

7. Enter and edit business hosts using the table. To add, reorder or delete business hosts, click the icons to the right of the table. Each time you click the Add icon, a dialog prompts you as follows: • Choose the Class Name from a drop-down list of valid host classes defined in your Ensemble

namespace.

• Enter a Name that will identify this business host in the Ensemble Management Portal. Do not use the characters ;:|,

• Determine a Pool Size for jobs associated with this business host. This value can be changed in the Ensemble Management Portal.

• Click OK to save the item definition or Cancel to ignore it.

8. Click Finish to save your new production definition or Cancel to discard it.

The result looks something like the following example.

Class User.NewProduction1 Extends Ens.Production [ ProcedureBlock ] {

XData ProductionDefinition {

<Production Name="User.NewProduction1" TestingEnabled="true"> <ActorPoolSize>2</ActorPoolSize> <Item Name="ReasonableBank" ClassName="Demo.Loan.BankUS" PoolSize="1" /> <Item Name="SlowBank" ClassName="Demo.Loan.BankManana" PoolSize="1" /> <Item Name="EfficientBank" ClassName="Demo.Loan.BankSoprano" PoolSize="2" /> <Item Name="FindLoanRate" ClassName="Demo.Loan.FindRateDecisionProcessBPL" PoolSize="3" /> <Item Name="TerminalSession" ClassName="Demo.Loan.FindRateTerminalService" PoolSize="1" /> <Item Name="EMailSession" ClassName="Demo.Loan.FindRateEMailService" PoolSize="1" /> <Item Name="TestingService" ClassName="EnsLib.Testing.Service" PoolSize="1" /> <Item Name="TestingProcess" ClassName="EnsLib.Testing.Process" PoolSize="1" /> <Item Name="BusinessProcessMonitor"

(37)

ClassName="Ens.MonitorService" PoolSize="1" />

<Item Name="DashboardMetrics"

ClassName="Demo.Loan.BankMetric" PoolSize="1" />

</Production> }

}

Using Studio, you can add methods to a production class definition, just before the start of the XData ProductionDefinition block. You can use Studio to examine the Demo.Loan.FindRateProduction class in the ENSDEMO namespace. This class provides examples of using methods in a production. The XData ProductionDefinition block holds the configuration information for the production. This consists of values for all the configurable settings that the host classes for the production provide. When you edit production settings using the Ensemble Management Portal [Ensemble] > [Productions]

page, you can apply your configuration changes immediately, even if the production is currently running. Any configuration changes that you apply while using the portal are visible in the production class XData ProductionDefinition block, the next time you open the production class definition in Studio. Although you can edit the XData ProductionDefinition block in Studio or a text editor, it is best to edit it from the Ensemble Management Portal. By doing so, you can avoid introducing syntax errors and take advantage of type validation by the portal.

However, there is one circumstance that requires you to edit the XData ProductionDefinition block; that is when you want to control the top-to-bottom order of the shapes displayed in each column of the configuration diagram on the [Ensemble] > [Productions] page: Business Services, Business Pro-cesses, and Business Operations.

Configuration items appear in production class in the order in which they were added to the production. The configuration diagram faithfully captures this order in each column, positioning shapes from top to bottom as you add them to the production. The only way to change this order within a column is to open the production class in Studio and edit the XData ProductionDefinition block, reorder <Item> elements, then recompile the production class. Any changes that you make in this way become visible in the configuration diagram the next time you refresh the display.

2.3 Configuration Settings

Production classes and business host classes can have configurable properties defined using a SETTINGS parameter in their respective class definitions. For an overview of configurable settings and why they are useful, see the “Configuration Settings” section in the previous chapter, “Production Concepts.” To provide new configurable settings for a production or for a business host, modify its class definition as follows:

(38)

1. Add a property for each configuration setting you wish to define. 2. Add a class parameter called SETTINGS to the class.

3. Set the value of SETTINGS to be a comma-delimited list of the names of the properties you have just defined. For example:

Property foo As %String; Property bar As %String; Parameter SETTINGS = "foo,bar";

The foo and bar settings now automatically appear in the configuration display on the [Ensemble] > [Productions] page whenever an item of that class is selected for configuration.

2.3.1 How Class Code Determines the Available Settings

The syntax for a SETTINGS parameter appears in this excerpt from the class Ens.InboundAdapter:

Class Ens.InboundAdapter

Extends Ens.Adapter [ ClassType = "", ProcedureBlock, System = 3 ] {

/// The minimum interval between invocations of the adapter by the Ensemble /// framework. For adapters that poll for external events, this is the polling /// interval. However, most polling adapters will process all inputs

/// immediately if several are detected at one time. This is also the interval /// at which each Business Service will check for shutdown requests.

Property CallInterval As %Numeric(MINVAL = 1) [ InitialExpression = 5 ]; Parameter SETTINGS = "CallInterval";

/// (...Additional lines removed from this example...) }

Any extension of a class with a SETTINGS parameter inherits the list of configurable properties from its parent class or classes. You can extend the list of configurable properties in the new class by pro-viding a SETTINGS parameter in the new class.

The following example is similar to the built-in class EnsLib.File.InboundAdapter. It extends

Ens.InboundAdapter by adding several configurable properties and a SETTINGS parameter:

Class EnsLib.File.InboundAdapter

Extends (Ens.InboundAdapter, Ens.Util.File) [ ClassType = "", ProcedureBlock, System = 3 ] {

/// Path to directory to look for files in

Property FilePath As %String(MAXLEN = 1000) [ Required ];

/// Wildcard filename to look for; may be a semicolon-separated list of /// wildcards.

Property FileSpec As %String(MAXLEN = 400) [ InitialExpression = "*" ]; /// Path to move a file to after processing is completed. If not given, the /// file will be deleted after processing is completed

(39)

/// Path to move a file to while it is being processed. If not given, the file /// will not move while in process. This setting is useful when the same /// filename is used for repeated file submissions.

Property WorkPath As %String(MAXLEN = 1000);

/// Use this Character Set to translate input from the file. For binary input, /// use "Binary".

Property Charset As %String [ InitialExpression = "Default" ];

/// Amount of time to wait to be able to take posession (open,move) a file /// we've found (in case it's still being written in place)

Property FileAccessTimeout As %Integer [ InitialExpression = 10 ];

/// Append a timestamp to Archive and Working filenames in order to prevent /// possible name collisions

Property AppendTimestamp As %Boolean;

Parameter SETTINGS = "FilePath,FileSpec,ArchivePath,WorkPath,Charset, FileAccessTimeout,AppendTimestamp";

/// (...Additional lines removed from this example...) }

2.3.2 How a User Configures Production Settings

When an adapter of the class EnsLib.File.InboundAdapter is selected in the configuration diagram on the [Ensemble] > [Productions] page, a combined list of the configurable properties from

Ens.InboundAdapter and EnsLib.File.InboundAdapter displays in the bottom half of the configuration display. Each property appears in the display because it is listed by the SETTINGS parameter in the

EnsLib.File.InboundAdapter class or its parent class Ens.InboundAdapter.

Note: The previous topic displays the class code from Ens.InboundAdapter and

EnsLib.File.InboundAdapter that causes these settings to appear in the configuration display on the [Ensemble] > [Productions] page.