SigmaLink
The Intranet Based Document Management
and Editorial System with full SGML and XML Support
Petra Leuser, Tibor Tscheke
STEP Electronic Publishing Solutions GmbH Overview
1. SigmaLink -- Transmission Belt of Electronic Publishing 2. Document-Management
2.1 SigmaLink -- the integrated document management system 2.2 Terms
3. Editorial System
3.1 SigmaLink -- the Intranet Based Editorial System 3.2 Terms
4. SGML: from Document to Information Management 4.1 SigmaLink with Complete SGML Support
4.2 SigmaLink -- the Sum of all Links 4.3 Terms
5. Workflow Management
5.1 SigmaLink offers Workflow from the Market Leader 6. Customizing & Rendering
6.1 SigmaLink Admin - the Interface to Administration and Configuration 6.2 Rendering
7. Integration / Interfaces 8. Migration
8.1 SigmaLink offers Migration Strategies 9. Architecture / System Components 9.1 System Components
1. SigmaLink - Transmission Belt of Electronic Publishing
In the course of the last 15 to 20 years, computers have gained acceptance, in the widest sense of the word, for the production, storage and maintenance of documents and publications. The goals and paths have frequently remained the old ones.
Software and hardware are used to produce paper. The eraser is called Backspace and the carbon paper is called Copy.
There are only isolated stable strategies to use the strengths of the new means of production consistently, only isolated concepts of keeping information consistent and as one copy instead of several copies. What has long since been accepted for databases, freedom from redundancy, standardization, single source, central access,
is hardly used for documents. Instead of normalizing and standardizing information, it is entrusted to more and more playful tools, which support authors, editors and office workers in everything but the production and maintenance of the information useful for the company.
Digital information is a thousand times easier to duplicate than Guthenberg's Bible. Digital information, generated with any concept, only crystallizes on paper. The digital era thus creates rubbish dumps, every day, easier than ever, more and more ever! How many terabytes of old data are created daily, in the word processing programs of this world which can produce nothing but paper? Information is lost in digital graveyards and in the mountains of paper constantly produced simultaneously in many places in the production process.
With the widespread popularity of the PC and networking of working places, this weakness is becoming more and more obvious, all the more so the closer the publication process itself is connected to the branch of business, whether technical documentation or a publishing product. Every system that claims to solve the problem will have to be measured according to all of the following criteria:
§ In what way does it support the production process? § Does the cross-company access to the information exist?
§ What starting basis does it create for the publication of multiple media? § Does it offer migration strategies?
Some software suppliers react to these problems by quickly creating new tools for document management and workflow management. Many of these products have detailed and good answers to individual aspects, but provide no or only inadequate answers to overall problems.
In 1992/1993 STEP implemented a system for a large publishing house
Bibliographisches Institut & F.A. Brockhaus AG -- for the generation, maintenance and management of the information substance of the company. The publishing house uses this editorial system with over 150 editors for almost all company publications, successfully and with notable effects on costs, speed and production quality. We were able to deepen and expand our experience through further large-scale projects with other publishing houses and industrial customers.
We productively used this experience as well as existing approaches on the market to develop an Intranet-based document management and editorial system with full SGML and XML support: SigmaLink.
SigmaLink addresses: Information-Management:
information objects, versions, users, roles, authorizations, data security, data storage and redundancy prevention.
Efficiency and Quality:
easy to learn, reliably prompted, quality-assured measures towards information objects, corporate identity, uniform structure, concentration on the most important matters by authors, editors and office workers.
Workflow-Management:
management of workflows, clarity, control, management of synchronous access. Migration:
handling of all data formats, special support of SGML, selective access to document parts, arrangements for various applications.
Orientation towards the Future:
single-source principle for all considerable applications, transformation into documents, CD-ROM applications and to the Internet, use of Intranet and Extranet.
SigmaLink offers the basis for the effective use of digital media in the publication process and at the same builds the bridge towards digital publication.
2. Document-Management
The core of a modern editorial system is a sophisticated document management system. Professional document management presupposes the use of database systems. Different digital data formats must be treated equally transparently and methodically for the user, whether during import or export, or in the management of the
documents, during their entire lifetime. On the one hand, version control and the connection to archiving systems contributes to an increase in quality -- the correct document for the current job cuts the error rate -- and on the other hand the effectiveness of navigation, or retrieval, has a measurable cost-cutting effect. Depending on its business purpose, the subject matter of its activities, its size etc., every company must organize different operational sequences. The performance of a document management system rises with its ability to being adaptable through the user to continously changing organizational preconditions.
2.1 SigmaLink -- the integrated document management system Data Formats:
With its document management functions, SigmaLink supports all data formats, from CCITT-IV to pdf, from mpeg to SGML.
Metadata:
All documents to be stored in the database (information objects) require a defined set of metadata. Different project or data format-dependent metadata structures can be defined independently. The metadata form the basis of the object retrieval by means of the relational database.
Import:
All digital documents can be imported from the desktop to the database by simple drag and drop, and are thus subjects to control by SigmaLink. For bulk-import there are interfaces that can be assigned their own processes. (A programmable transformer is optionally available for such operations.) These operations are of course checked for acceptability and appropriately booked in the database.
Export:
Freely defined export functions can be integrated into all workflow states or modifications. A programmable transformer is available for such operations as an option.
Version Management:
At each check-in, a new version of the document is generated. Old versions are retained to the extent defined for the relevant publishing project.
Archiving:
An interface is available for archiving. Archiving processes can be triggered depending on the version or the workflow states.
Navigation:
Navigation though the data basis is affected according to a hierarchy plan that can be defined by the user, is generated in the database and can be changed at any time. Since it practically reflects the organization of the work or projects, it can be used intuitively, that is, it is easy to learn, and contributes substantially to simplified system handling.
Retrieval:
The relational database used in SigmaLink is available for retrieval at object level (document level); all information defined as metadata -- e.g. version, person in charge, date, status, name, project, title etc. -- can be combined for any kind of retrieval. A full text retrieval engine is integrated that gives back also information objects as result.
Database Administration:
Database administration is implemented in HTML format. Commercially available programs such as Netscape or the Internet Explorer with their familiar interfaces can be used.
Configuration:
In each case, configuration can be changed by the user. The necessary database tables are dynamically generated and filled in as required.
User Administration:
SigmaLink supports a two-stage user concept. Each project is assigned roles. These roles have defined rights. Individual users may occupy different roles in different projects and thus have different rights.
2.2 Terms bulk-import
The import of several data using a defined batch process metadata
Information about the information objects CCITT-IV
Standardized format for fax data pdf
Proprietary page description language by Adobe mpeg
Popular video format SGML
ISO Standard 8879 -- Standard Generalized Markup Language roles
Function definitions, such as editor-in-chief, person in charge, production manager etc.
3. Editorial System
An editorial system can be measured by the extent to which it supports and
optimizes the collective editorial process of the publishing house, its editor as well as the authors and providers involved. It must provide satisfactory answers to problems of work organization, rapid access to contents, the current status of objects being processed and optimum availability.
Just as important as internal collaborative and co-ordinated working together within the publishing house is the organized and system-supported working process with external editors and authors, as well as a smooth data communication with the service providers, whether in the pre-press sector, for printing, CD-ROM publication or online.
Newer concepts, such as those used in the Internet, contribute towards
standardization. Editorial systems integrating these concepts in their core secure investments. Soon, no system will be able to survive if it does not open up Intranet, Extranet and Internet for the company.
3.1 SigmaLink -- the Intranet Based Editorial System Publishing project:
Each publishing project has its special characteristics, such as the number of authors, scope of the internal editorial activity, various work stages and processes etc. SigmaLink supports these special characteristics by allowing the publishing house itself to set up, for any publishing project, the metadata structures which best support the particular project.
Workflow:
A professional, fully integrated workflow management system makes it possible to set up and modify any sequences at any time. Self-defined automatic
processes are actuated at any stage of the editorial process and transform all information concerned as well as the metadata involved in accordance with any specification laid down.
Collaboration:
A precondition for regulated access control is productive cooperation between the editors involved. While eyeryone can search and navigate through the data basis, access to each individual information unit, with write authorization, is only possible by an authorized user. The relational database uses check-in- and check-out- mechanisms to assure unambiguous conditions and also provides information about who is processing which objects at what stage. All access rights are derived on the one hand from the division of roles as defined in the publishing projects and on the other hand from the users assigned to these roles.
Distributed work:
The actual "intelligence" of SigmaLink is provided by the SigmaLink Server while the user communicates with the server via the Client SigmaLink Workbench. By means of a unique mechanism, SigmaLink reduces the
dependence on the server to the greatest extent possible. Only central functions, such as search and navigation via the data basis, require an active link to the server, while time-consuming editing processes themselves take place locally in the SigmaLink Workbench. This reduces the load on the server, increases the number of possible clients and allows comfortable working even in off-line mode, in which there is no active connection to the server.
SigmaLink Workbench
The personal working environment of the editor is implemented by the SigmaLink Workbench. This allows navigation and retrieval in the data basis, using the freely defined order relations of the information objects, the metadata, which describe the properties of the information objects, as well as the full text -- the letter by means of the full text retrieval system. SigmaLink Workbench provides the mechanism for browsing and viewing, editing and annotating, as well as for linking information objects.
Information objects:
All of the basic functions of SigmaLink described here may be used for information objects of any data structure.
Intranet base:
Communication between the clients and the server is implemented with the help of the HTTP protocol. This isolates SigmaLink from lower-level
communication technologies and at the same time ensures functionality in the Intranet as well as when using the Internet, depending on authorization. External editors, authors and service providers:
The combination of the Intranet base with the properties described under "Distributed Work" ensure that with SigmaLink external editors, authors and service providers can work together ideally with the internal editor's office.
3.2 Terms check-in
Depositing information in a database check-out
Making information in a database inaccessible for write access browse
Viewing of information with comfortable navigation mechanisms view
Viewing of information
4. SGML: from Document to Information Management
SGML, ISO standard for more than ten years, has prevailed world-wide as the data modelling language for text. The majority of the important sectors of industry, from aviation to the automotive industry, from the semi-conductor to the computer and software sector, from news agencies to telecommunication, have decided in favor of SGML. The advantages such as standardized data keeping, free from dependence of changes in the system environment, the operating systems and software used, coupled with the modelling capacity of the information in accordance with corporate
requirements have been seen to be unbeatable and to project investment in the generation and maintenance or digital information.
Particularly in case where the medium or presentation-neutrality of information is of importance, whether it concerns the publication of such information on paper, CD-ROM or on-line, SGML is proving to be the only cost-effective alternative in question of the maintenance and preparation of information. This suits the need of the publishing house to acquire possession of their own data by pulling them away from the data graves of type composition.
New publication forms open new markets with different requirements. On-line and CD-ROM publications have to meet the needs of the user for excellent navigation and research facilities, and also presuppose a high availability of links within the body of information in order to achieve a level of use suitable for the media. The
preparation and maintenance of the fundamental information requires uninterrupted interplay between SGML, editorial system and document management.
4.1 SigmaLink with Complete SGML Support SGML-Editors:
SigmaLink has several SGML editors integrated. SGML-Viewer / Browser:
SigmaLink is supplied with a "general purpose" SGML viewer and browser. Parsing:
SGML information objects are parsed (structually examined) when they are imported into the system. Integrated links (references) are automatically posted in the database.
DTDs & FOSIs:
Split:
SigmaLink makes it possible at any time to split up SGML objects into smaller addressable units that can thus offer parallel editing processes. SGML structures such as ID and IDREF are accordingly resolved in the database to form
external references. Join:
Join operation make it possible to join split objects together again; external links, which are transformed in this process into internal links, are converted accordingly into ID-IDREF structures.
4.2 SigmaLink - the Sum of all Links
SigmaLink makes it possible to create, track and control the whole range of links. Since most types of links can only be implemented on the standardized basis of SGML or XML, the following "link" features, with the exception of object references, are restricted to SGML objects.
Object links:
links between objects, e.g. text with the appropriate video Including links:
e.g. links to an image to be located at a certain text position Links into objects:
e.g. a link to a table in another object Links out of objects:
links from random sites in one object to or into another object Links to database contents:
The contents of an SGML element or attribute are obtained from a certain database query, e.g. the number of inhabitants on a country in an almanac. Internal and external links:
since SGML only supports controlled links within an object, these are automatically externalized or internalized in split or join operations.
Wherever possible, the implementation of the link mechanism was made compatible to HyTime.
4.3 Terms SGML
ISO Standard 8879 -- Standard Generalized Markup Language DTD
Dokument-Typ-Definition; set of digital rules for a document class, e.g. dictionary article
FOSI
File Output Specification Instance; the appearance properties defined for a DTD, e.g. layout
XML
Extensible Markup Language; WWW standard in the making WWW
HyTime
ISO Standard 10744 -- for electronic linking of information, based on SGML
5. Workflow Management
Every company endeavors to make internal workflows as effective as possible. Workflows to ensure optimal organization in individual workgroups have usually developed historically. Thus the problem of the editors in publishing houses, with the aim of making a printed product, seems to have been ideally solved.
However, the question of whether all workflows within the editorial offices have in fact been ideally solved may rightly be asked. Clear definitions, for example which input processes have to be completed so that the data supplied by an author (as a manuscript or as a file) can be forwarded for further editorial processing, help in the organization of workflows. Normally, an editor-in-chief has to release all information before it is published, that is, he has to pass it on to the next production process. Important questions, such as the completeness of all of the information of a
publication and the effect of delays at any stage in the workflow, can usually only be answered instinctively or with a considerable amount of effort.
With the introduction of an information management system which gives the publishing house free access to its entire information base, a definition of the workflows according to company-wide as well as editorial-wide requirements is also necessary.
Complex workflows and their interconnections can be practically defined using the electronic means of workflow management. An editorial system must therefore include an efficient workflow engine which optimally supports the modelling and realizations of the different and, in some case, highly complex editorial and publishing processes.
5.1 SigmaLink offers Workflow from the Market Leader
The workflow engine of Staffware, currently the most powerful and most successful workflow system on the market, is integrated in SigmaLink. The Staffware workflow engine allows cross-company solutions, including the following features:
§ department-specific or cross-company procedures § graphic interface to define workflows
§ password protection
§ modifications to active workflow procedures § sophisticated form handling
§ centralized backup and restore functions § status reports
6. Customizing & Rendering
An editorial system can be used paricularly successfully in a publishing house if it can be adapted to the individual characteristics of the various editors' offices.
Different editorial tasks give rise to different demands that have to be met by an editorial system. Up to now, publishing houses have only had the choice between laborious, cost-intensive individual solutions, or the use of standard software, which can only be used more or less effectively in some sectors.
SigmaLink is an integrated program package that covers practically all requirements of editorial work in different contexts. A generally valid solution has been found in SigmaLink for the basic problems of intelligent document management in a heterogeneous software environment, editing and viewing documents as well as reorganization potentials through powerful workflow management. The actual concrete appearance can be defined using the SigmaLink Administration Client.
6.1 SigmaLink Admin - the Interface to Administration and Configuration
SigmaLink Admin is an integrated program package for system administration. With SigmaLink, the publishing house can set up new projects at any time and carry out all necessary definitions itself using the SigmaLink Admin client, for example:
§ free definition of metadata for the individual projects § introduction of new DTDs
§ adaption of SGML editors and SGML viewers to new DTDs § definition of new workflows
§ definition of new export and import paths
§ integration of non-SGML-based standard software 6.2 Rendering
The integrated SGML editor and SGML viewer provide the user with layout-supported information.
The tools used here are in a position to process the information graphically with the help of the elements defined in the DTD and to display them on the screen. The SGML-structured information -- as is familiar from conventional software tools (e.g. MS Word ...) -- can be presented to the user in this way. The layouts for the editor and viewer can be defined at any time by the system administrator using the SigmaLink Admin client. This is equipped with all of the necessary utilities in accordance with the SGML viewer and the selected SGML editors.
7. Integration / Interfaces
The design of the information flow in a company is ideal when systems are in a position to exchange information with each other despite different tasks and
objectives. This does not only apply to systems with obvious task spectrums, such as editorial systems and picture archives/databases but also to apparently completely separate task areas such as editorial and bookkeeping systems. For example, the procedure for setting up authors fees can be triggered off by a defined status of the information in the workflow of the editorial system.
The system architecture of SigmaLink is adapted to such links. SigmaLink has open interfaces as well as the integrated standard tools themselves. The linking up of other systems with SigmaLink can easily be effected by STEP.
8. Migration
In the introduction of an SGML-based editorial system to a publishing house, a clear and in some cases long-term strategy is essential. The publishing house must be able to forward the already available information to the document management system and the workflow as quickly as possible without having to convert the entire data base immediately to SGML.
Intelligent document management must therefore be applicable for any information unit -- for example MS Word files, images, video and audio clips, composition system files and much more. The relevant processing programs must be able to be integrated
easily flexibly into the editorial system. Not every publishing project is immediately suitable for conversation to SGML -- a few publishing products may not be
produced with SGML at all. Conversation to SGML, right down to the part-areas of the projects, must take place at exactly the point in time when it can best be integrated into the production process.
All projects and editorial offices, however, should profit from the increase in efficiency due to document management and workflow management.
8.1 SigmaLink offers Migration Strategies
SigmaLink offers ideal conversion options, which occur with the minimum of frictional losses.
Standard-Software:
Any standard software that exists on PCs can be integraded into the system by means of a simple menu. In this way, files can still be maintained and updated. When the edit function is called *.doc files are provided in Word, for example, and *.sgm files in the integrated SGML editor.
9. Architecture / System Components
The use of SigmaLink guarantees a working environment characterized in particular by stability and reliability, even in complex environments with many users. In the selection of system components, STEP relies on the latest technology, which, however, must have proved its worth in large-scale use.
The system architecture of SigmaLink follows similar principles as those that apply for SGML-structured information. With SigmaLink, STEP has created a system that reflects the state of the art, in that the latest technologies (Java, http connection ...) have been integrated. At the same time, the interfaces to the integrated standard products have been so clearly defined that STEP can guarantee a service life for SigmaLink which is unique in the software world. The system components used (such as database system, full-text retrieval engine, SGML editor, viewer ...) themselves are subject to a permanent process of development, in which the user of SigmaLink can definitely participate. New technologies in these sectors, which today are not yet ready for the market, can be integrated at any time as required.
SigmaLink users thus have the chance to use a system that can be kept
technologically up to date for years without having to adapt to a completely new working environment.
With the use of SigmaLink and SGML you can safeguard your investment in § your information
§ the knowhow of your staff § the system
9.1 System Components
SigmaLink is supplied as a client/server system with the following standard software components:
SigmaLink Server: relational database
structure capable full-text retrieval system workflow engine
SGML transformation and communication engine http-server
SigmaLink Workbench (Client): selectable SGML editor SGML viewer
http-client
SigmaLink Admin Workbench (Client):
DTD and FOSI compiler (according to the editor selected) Web browser (i.e. Netscape)
